OCapN Pre-standardization Group

September 2023

Agenda

If there are no objections, after the meeting I'll merge #79 minutes of the August meeting.

Minutes

New Implementation in Haskell

Jonathan Rees: There's an anouncement to be made by ICRainbow or Christine Webber

ICRainbow: Announcement that we have a grant from NLnet to implement another implementation in Haskell, hopefully there, about 2/3 there. We're pre handoffs, but 2-party protocol is working and pass test suite by Jessica

Jessica Tallon: really awesome news, assuming 3 party handoff is in the works given comments on issue tracker

Opportunity for objections for closed issues

Jonathan Rees: No objections, considering processed

Request to merge documents into repository

Jessica Tallon: I put this on agenda for folks to review with same idea as with CapTP document, not that you agree that's way that things are going but that we can merge it open issues against it. Eg locators document which used to be called URIs document... I know there's a lot of discussion about whether or not these should be URIs or URI-like things, how they should look, etc. I think there's a lot of discussion about that which probably shouldn't block the PR (though it's good that it's happening). The other is the netlayers spec. Any comments that should block before merging? I do see that Dan Connolly reviewed it, I know ICRainbow you've also looked at it

Kris Kowal: I also looked at it. I think we need to think more about protocol negotiation and identity across transports. I don't think that's addressed yet, I do think it would be useful and desirable, but not enough to block. I think it would be useful to move information about the transport protocol into... I think there would be an identity that could be transport-agnostic for purposes of identifying a capability, and that would involve moving the transport protocol into the hints information

Jessica Tallon: that also kind of feeds into having locators which specify multiple netlayers at the same time which is something Spritely has not implemented but is interested in. Spritely has ideas, I think we've mentioned some in the PR, but since we haven't implemented there may be many isses, we'll need to see. We're certainly keen to work on it with the group on how best to address that.

Kris Kowal: Feedback from Brian Warner is it would require entraining an encryption protocol above the message passing layer, which is maybe undesirable. But maybe we could use noise protocol above transport protocol, not entirely sure it's necessary, but it's feedback from Brian when discussing this. I mean to suggest concretely we think about having a node that is listening for websocket and tor for example (such that a web browser node could filter the connection hints for just websocket)

Dan Connolly: My strategy for Agoric is it's ok to publish these things as long as it's clear where to show there are issues. Didn't give thumbs up because I don't think Agoric will implement as-is but good to merge with further feedback process

Jonathan Rees: Mant to make sure any potential blockers raised have been reviewed

Dan Connolly: I believe so

Christine Webber: PROPOSED: Merge current state of netlayers and locators documents with understanding that further discussion and change to documents and expected behaviors is expected

RESOLVED: Merge current state of netlayers and locators documents with understanding that further discussion and change to documents and expected behaviors is expected

Dan Connolly: I appreciate that the docs include elaborate "subject to change" disclaimers

ICRainbow: long one on encryption: +1 for protocol "stacks" (i.e. noise over tls over tcp over tor), -1 for mandatory noise or something like that

PR: Address additional feedback to CapTP spec

Jonathan Rees: Any reasons not to merge https://github.com/ocapn/ocapn/pull/60

Richard Gibson: It looks good to me

Jessica Tallon: There are a few additional issues I had neglected to address in review, this addresses those, and I also forgot to document the op:start-session field to tag what is signing, fixed that here

Dan Connolly: Is this editorial or observable in test suite?

Jessica Tallon: It's all editorial except for the text regarding what was already in teh test suite (and the expected behavior)

Jonathan Rees: No objections, moving ahead

Spritely's response to Agoric's proposal for concrete syntax

Jessica Tallon: brief intro, we looked at two presentations by Agoric at last meeting and also a previous meeting, have also looked at smallcaps a few months ago. We have a few questions for Agoric, I think we're not fully understanding certain things, I thought it would be better to present a brief worldview from Spritely's perspective around concrete syntax, and to propose some concrete questions for Agoric to help us come to a better and more nuanced opinion of syntax proposed

Jessica Tallon: This slide is showing Spritely layers... kind of akin to the layer diagrams Agoric was presenting last meeting. We have split into low, middle, high level, with low level being Syrup, we've got the mid layer abstraction which is specifically all the OCapN types, eg the node locator, sturdyref locator, all the different operations and descriptors which are defined in the 3 documents we've been discussing, and finally the high level abstractions built on top of these core data types such as custom records, and that's up to the application to define as they see fit. Low end is Syrup, what's written on the wire

Jessica Tallon: Here's a brief explainer of Syrup, it's small and contains a few core data types, it's fairly generic, I know it's been reviewed by Agoric and sets and maps/dictionaries can be problematic and I know we've discussed that. But I thought it would be useful to mention and show the whole of the concrete syntax of Syrup in this one comment. But this is enough to implement it, and has been for many implementations.

Jessica Tallon: And here's syrup used in a captp context. The top is an abstract description, here you see op:deliver which has a desc:export with a reference to an exported description, arguments, etc. In Syrup it looks very similar to this captp record. Here it is in Syrup, this is the exact representation on the wire, displayed in ascii encoding. As said before I know there's discussion about whether or not maps and sets are necessary, and captp and rest of implementation doesn't use them, so while I think they're useful for applications, it's fine to drop them or make them copymaps and copysets using tagged datastructures using the data type. definitely something that can happen, so in terms of the Spritely concrete syntax, it should be considered more of the minimal syrup since I think it's fine to drop those two types specifically in the captp syntax

Jessica Tallon: This is more of a devil's proposal, not suggesting we actually adopt this, but here's SPKI canonical s-expressions, showing how minimal you can get with a minimal concrete syntax, it's just lists and binary data, and in this case they encode type data and value data with binary syntax. And I think quite a few people will probably be familiar with this

Mark Miller: do I need to understand this representation?

Jessica Tallon: it's not a serious proposal, a mental model

Christine Webber: (explains how canonical s-expressions works)

tysesika: here's a diagram of a specification where the core types are very strongly encoded, moving from csexps to json to syrup to things completely app-specific things. Syrup doesn't have type specific things to a set domain. One of the questions we have is, we seem to recall that smallcaps has references and promises in it, and obviously some of those...

Mark Miller: We have remoteables and promises, and a remoteable and a promise, both of those are capabilities, they are able to convey messages to emote objects. Remoteables have a stable and comparable identity. Promises do not, because it might not be known what they are a promise for, but in the data syntax, the body as we call it, what we distinguish in the smallcaps json as well as in the binary syntax, we'll just stick with smallcaps, we have a specific encoding for promise slot number 3 or remoteable slot number 7. and there's a separate from the body portion the conveyance of the capabilities themselves which are not necessarily data, depending on what kinds of transport context you're in, and when they're data they're often c-list indecies, so the data is portable and the slots are kind of the c-list for the message itself, and contains c-list indecies into c-lists for the context in which it appears

Jessica Tallon: I think that means there is a captp-specific type for references and promises

Mark Miller: remoteables and promises

Jessica Tallon: right, in the specific concrete syntax

Mark Miller: yes that's correct, it's just the marker saying remoteable or promise, with a marker into whatever the slot is.

Jessica Tallon: this was sort of our exercise into where is teh concrete syntax richness scale that we were mocking up internally, we also weren't sure how much pattern matching, guards, etc appear in there.

Mark Miller: what you labeled as your low, middle, and high level I think corresponds to... your low level is the pass... should I present?

Jonathan Rees: we only have about 10 minutes

Mark Miller: I will verbally answer this particular question and try to make it quick. We have a low-level layer with pass style and marshall packages, we have a layer corresponding to mid level which is our captp package which provides meaning into what these slot entries are, and a package corresponding to high level which is our patterns package. both captp and high level depend on low level but high level does not depend on middle level. they're both extensions... built on low level and compose smoothly. patterns are part of high level, built out of tags

Jessica Tallon: continuing, another quesiton we had is whether or not smallcaps is based on json is aspect of canonicalization, current goblins and current draft of captp uses canonicalization because several things are signed... we could impelment this without canonicalization, and could pass a blob of binary data signed and check signature and parse binary data, but you can't store that in a unsearelized form and later re-verify without storing that blob. it's also maybe an opinion but it's a shame to give a shame that has a very simple canonicalization system, especially if looking at syrup's simple canonicalization is nice and would be a shame to look that up, and matrix and ssb have struggled with this, and not sure if people have seen this with the complaints about impelmenting a matrix-compatible server. canonicalization does matter and how you at least in current goblins implementation and how to go about that if it's not provided can raise other problems so it's something we wanted to mention that's on our radar, not that it's a deal breaker, but something we find compelling

Jessica Tallon: so explaining why goblins uses syrup, what goals when designed, can be implemented less than an hour, consistent and understandable canonicalization, want to make sure that an adopter can implement quickly and easily, should have consistent and understandable canonicalization, and wanted it to have most of the same types as json but also to have a way to extend it, the record type in syrup or the tagged type, to allow for extensibility of the syntax without the suffering the experiences we had with json-ld, wanted it to be compact enough to be sent on the wire, and we wanted it to be a binary syntax, so entire format is, easy encoding of binary data or bytestring

Jessica Tallon: so again just a review of the layers, syrup at the bottom, layers...

Jessica Tallon: finally questions to agoric. First one is assuming the minimal syrup, the one without sets and maps, how does Agoric feel about that, what are the problems with it, for other suggestions what are the pros and cons when as a group (not as Spritely), why should one win over the other? We feel a bit like we're not sure how to implement this, and what we're looking for I think is similar to the slide with the comments that explained syrup's syntax, I think we want to implement agoric's approach so we can better understand. So I think it's fair to say we don't quite understand the proposal and feel a bit buried in the numerous layers and passable types. Can it be summed up in something like the minimal comment, can Agoric's stuff be described in something akin to the 3 layers, and how core are things like pattern matching and etc, is that in the application layer etc

Mark Miller: I can try my best to answer in the 5 minutes we have left

Kris Kowal: I'd like to give the first pass, want to convey I don't believe it's agoric's position that ocapn should involve json at all. don't think we object to ocapn being expressed in a syrup like format

Mark Miller: I do object

Kris Kowal: I guess we'll come back to that. It would be helpful if there's a tier that helps us with layers of abstractions, but not something I hold strongly. Advantages to hold json in the JS world. Higher level questions, Mark Miller and I both agree that the high level requirement we need to converge upon is where the messages over ocapn, regardless of whether the impelmentations in scheme js and etc must round trip, imposes certain constraints for higher level approaches. One point of confusion is there's the notion of a record, differentiating whether the record should correspond to a map or struct must be clear. Additional constraints imposed on sets and maps, JS has a very particular idea of insertion order presumably not preservable, begin to entrain concerns about sortability etc

Christine Webber: big thing we are asking for is we want a concrete syntax we can implement so we can understand

Mark Miller: I volunteer to specify smallcaps, and also "compact ordered" format, we need to specify both of those so they can be implemented independently in non-js platforms

Mark Miller: I think there are desire for both smallcaps and compact ordered format. eg C primitives, usable in other languages... textual in a way that's largely readable once you know the escape conventions, gives smallcaps a place I think will continue. So I disagree with kris that json doesn't have a place. the other one for the binary syntax, that's where we've gone beyond canonicalization. Canonicalization can be stated as "if they're semantically equal then the concrete encodings should be". Binary syntax also provides order preservation. Buried in layers as already covered in 3rd bullet, I think smallcaps can be stated as minimally so we haven't tried. I'm convinced compact can't, but it's also binary, but looking at binary stuff in a text editor is inherently confusing. I don't think the binary form needs to be easy to state. Agoric's stuff in 3 levels? We have exactly the layers corresponding to 3 layers with the distinction that layer 3 depends on level 1 and layer 2 depends on level 1 but level 3 is not dependent on level 1. Maps and sets are encoding into tag. That's it.

Jessica Tallon: thank you for volunteering to do that and giving explanations, I think that will be useful, coming back with better understanding and ability to review properly. I am keen for the binary syntax to be easy to implement. It doesn't have to be as viewable in a text editor, certainly you can have a representation that programmers look at with a printer for these types that's completely different and separate from on wire representation, Syrup happens to not have one. We are curious with how this compares, something I didn't stress in this presentation, while we do like syrup a lot, it addresses things we need solutions to, we are flexible to change it or coming up with something new or using something different entirely, but ideally we can coalesce on something that has all the properties both parties need. Keen if possible that we do speak the same concrete syntax. Obviously if not achievable we can cross that bridge.

Mark Miller: mmhmm, good!

Jonathan Rees: I think we can adjourn, any other business?