OCapN Pre-standardization Group

April 2023

Agenda

Minutes

jar: main thing today is going through documents

cwebber: how are we doing the queue?

jar: using queueing in the BBB chatroom, because it seems like most transparent and reliable

(more conversation about queuing)

cwebber: I think we should talk about captp but also serialization

markm: I advocate whether part of same set to talk about, that they should be part of two different layers

jar: yes I put something about data layer on the agenda

jar: but I'm not sure what we have something prepared on that

jar: cwebber if you think it can be productive I think we should talk about it

cwebber: I think yes,

Jessica can talk about it, we did the review isd and markm asked for

Jessica: for an overview of things, I did a PR of the CapTP spec, I am of course working on the work following after

Jessica: I'm not sure how much people have had a chance to look at the captp spec, I wanted to remind folks that the initial draft is based on spritely's implementation, that's not meant to favor spritely, but meant that we can file issues on where we think we don't have consensus and can do with working on them

Jessica: and obviously as we come to consensus the document can evolve over time

Zarutian (on bbb chat): drop resolve-desc from op:deliver and only relie on op:listen for that functionallity?

Jessica: I am reviewing and addressing Richard Gibson(?)'s and isd's feeback

Jessica: I propose review it and merge it into the repo and that it be easier to have PRs on parts on the spec

Jessica: eg if we're going to drop the bootstrap object, we can review that language

Jessica: and I think it'd be easier to work with than one big pull request where it's hard to see what we review

Jessica: if the group wants to go a different way that's possible

Jessica: there's a note at the top that says it's a draft spec, we could strengthen it to say that this really will probably change quite drastically, and people will probably not implement it

Jessica: the doc does mention syrup, the intention isn't to not go with that, the intention isn't to go against it, but the first draft is what spritely itself implements

Jessica: my hope is that we can converge on concrete syntax, my hope is tthat we can get more adoption by other implementers, where we can get more implementations with other concrete syntax

Jessica: it could be something else, it could be cbor or msgpack, my preference is we can discuss some kind of syntax

Jessica: other than this spec, I am working on test suite

Jessica: that's more or less the status of my sowrk and to introduce it

jar: isd is next

isd: I'm going to punt to the end of the queue

jar: markm, are you diving in?

markm: responding to what Jessica was just saying

markm: you've been very good verbally to apply context, this is written from spritely's perspective, as such it's a step towards consensus, but not yet something we can reflect something we can all agree on

markm: my concern is that it be more explicitly marked not just a preliminary draft, but rather a perspective from one of the participants, rather than something we can work on towards agreement

Jessica: yes. i can add that, make it more clear that this isn't something the group has converged on, perspective of one of the parties

Jessica: will add that to the PR

markm: regarding converging on a concrete syntax, thank you very much to look at smallcaps. i've been under the impression that it's hard to... let me ask the spritely people collectively, would you consider smallcaps directly and literally as the syntax

Jessica: I can speak to that. spritely is not tied to syrup, we can switch away from that, am fairly free in what we can change, we think its' reasonable and good, we have possibly two concerns

Jessica: both of them we could technically accept but we think it would be worse than what we currently have in syrup

Jessica: currently we can encode binary data in a very efficient way in a way that we wouldn't have to base64 encode it etc

Jessica: especially if you get store and forward sessions, if chained, you can get a balooning of data, so efficiently the efficiency of binary strings is potentially a concern

Jessica: currently we are using syrup canonicalization for the certificates we're signing, we could have that be a payload of data and sign it, that's certainly possible, to me it seems slightly less elegant than have the message and sign it, but that would be fine, we could do it, could adopt it. something we've been wondering is... if / how it would be possible agoric could change to anything, like msgpack or syrup, or cbor, or something else, something that might mitigate some of these

Jessica: eg the binary string type

Jessica: how flexible is it at agoric?

cwebber: I think we are interleaving, but I think this is one of the most critical topics of the group. I think this is an important question for how possible interrop is for the group, so I think this is important for the group.

cwebber: Ian a few meetings ago asked us to look at how compatible our technologies are(?) we've done the smallcaps. We've done that, and we think we're quite positive towards smallcaps. One of the things which kept coming up in internal conversations is how possible interop is. This is a rare opertunity, some of the things we've heard from Agoric folks is we already have stuff in production, we might no be able to change. You are hearing from us we can change, we can change the syntax. So a question to Agoric are you ossified and tied to your implementation, can you address that?

markm: it's expensive for us to change, it's not infeasible

markm: you can commit yourself to data where it's less expensive to live with co-existience

markm: the one we'd like to change to, if we were to change to a binary format. Is one that doesn't exist yet

markm: it's one I designed and implemented except a (?) that Richard Gibson addressed. It's what I refer to as encodePassable

markm: the textual representation in json, json does much of the up-front work of converting from textual to structured form, but the binary cost I'll ack as a huge problem. and there's no way to get binary cheaply in a purely textual format, I understand that

markm: the reason for binary to binary at least co-exist... the radical perspective of proposing a thing we haven't implemented is it seems to have a magical property without undo cost. order-preserving for certain semantics. semantics not at the AST but as we've architected the set of levels, it's a very nice ordering semantics that I'd like to show to the group at some point

cwebber: I think this is a very positive response, I'm happy to hear it. What you're hearing from Jessica and I is we had syrup because it was simple and we liked it okay, we're not saying it's ideal. What we're happy to hear is that you're saying smallcaps is not ideal. If we are perceeving this as an analysis and I think we should do the analysis and doing the right thing.

cwebber: I think it's worth while for this group to spend time in this design space, if we're saying yes we're going to do this right

markm: just temper your optimism a bit, the abstract syntax is something we're tied to

cwebber: having done the smallcaps review, I think we felt very optimistic about the abstract, especially given the tagged structure

Jessica: I had put myself on the queue to answer something else, but I will say that I'd be interested to hear more about your proposed solution, I find that very exciting. as I said in my comment, in terms of abstract types, I think we're fairly aligned in terms of what we need to get our captp implementations talking

Jessica: I did take a brief look pre-meeting. so I don't think the abstract syntax is a problem regarding the format

Jessica: I think you did mention a tag type similar to our records, we kind of have it, a pre-defined set of types used in our captp layer, so the rest are tagged as "user records", so I think that'd be okay too

isd: my sense of where we are on the data model stuff is that the next thing to do next, someone, I'd be willing to volunteer, is get a proposal for "this will be our abstract syntax", then this would give us something to pick at, critique, etc. we seem pretty aligned, I think we need to start specifying and nailing down the abstract syntax

Zarutian: this is a question to markm about smallcaps. how exactly wedded are you to the json that endo/smallcaps consumes? pure json, or might you say that there's eg a byte array buffer here

markm: the key thing about the json as I mentioned is that we get to use the fast native code on all platforms to do the basic json parse and stringify, and everything else we do from structure to structure transformation. that's a nice property, not just for us. but to answer your question, we know we need binary, we're committed to that, and one way to do that is an envelope..

markm: we already have an envelope containing both the json and the slots array

markm: the slots array can't have a universal encoding

markm: since we already have an envelope that has those two things in it

markm: what we have is an index into the slotsarray

markm: it would be some pain because we are using json on the envelope as well

markm: we do understand it's important

markm: in the envelope we can have room for pure binary, and for the part we call the body wich is json encoded smallcaps, what there would be is an index into the binary arrays in the envelope

Zarutian: better solution than what I was just proposing

jar: amazingly the queue is empty, interested in what Jessica is doing next

cwebber: I'd like to do it, I'd like to say some things about and the process we went through. Jessica was given not an easy task. The documentation that exists is scattered and is specific to an implementation. She worked very hard (more than I'd have done, if I had done it it's be very opinionaty) on avoding that. I would like to say that she did an increadible job of producing something neutral. It does have lispy kebab style syntax, which we could change. One thing she did do was stripped out implementation details like tables. All other explinations keep "this table"... and we had a conversation about if that was the right thing, but what I think is interesting about this is... Jessica is going to talk next about an implementation guide and stuff, which will talk about how you might implement it.

cwebber: The effort to go through and translate all the documentation to this spec.

Jessica: so what I'm working on next... obviously as my highest priority is address any feedback that's left on the captp spec that exists, there's still some that we need to get to, with the aim to get it merged, and as convos go on, I will try to keep on top of it

Jessica: most of my time is currently spent on the test suite, which will be in python

Jessica: I wanted to choose a language I was familiar with but which has a wider audience, I wanted something more people knew (not everyone knows scheme)... hope it was a decent choice. going to push it in a few days, still in the earliest stages, hoping by next meeting hoping to have something fairly decent and covering a lot of the spec

Jessica: so that's something I'll be doing, and as cwebber said, on top of that there are two more specs to write, the netlayers spec.... and there's some question about more whether I put in should go into netlayers, we talked internally, certainly open to it... but netlayers as I see it now is mostly opening a channel to the other session and that revolves around certain netlayers. first one is the tor onion netlayer, more is certainly welcome, but with more implementations, maybe people can contribute their own, but at least for now I can document the one spritely is using.

Jessica: and then there's the ocapn uri doc... I changed it to "references" because there was debate about "uris" but we can talk about it... and then there's the implementation guide which is going to be more opinionated about how one might implement this with tons of diagrams and etc (of course spec could have diagrams too)... will have import and export tables and gift tables and that is definitely coming in next few months' work

cwebber: First of all, I want to hear other people's response to what Jessica is working on and what's ahead. I think hopefully it'll be more obvious for why we structured the grant the way we did to get the ground running. Something I want to discuss is what licence we'll use. I would like to suggest Apache v2 for both the document and spec. We could use CC for the document but it's a perfectly fine licence for documents too

cwebber: Apache does interrop with GPL

markm: apache2 is our favorite license at agoric, was our favorite at when some of us worked at google... you've brought up an interesting point that is there reason that documentation can't be under code licenses?

jar: will there be contributor agreements?

cwebber: regarding contributer agreements, I think that's something we can talk about. I think it's worthwhile and simple enough to have a file of people who've signed off on an agreement that their contributions will be under Apache v2 and that they won't be agressive with their litigation(?)

cwebber: Regarding your opinion on why we have different licences. My impression from when I worked at CC is that it's partially due to historical reasons from where Richard stallman considered these very different things. This really bugged me at CC and something I kicked off before I left was the CC-BY and CC-BY-SA for GPL compatibility. It always troubled me that people made code things and people made non-code things and as someone who likes video games it always seemed strange to me that people treated these as very different things. Do we treat things differently that are functional vs cultural works

Jessica: I'm happy with apache v2, I'm happy for that to be the license for both the document and the code. I'm interested in feedback people have. I know Ian on the PR talked about organizational stuff. It's something I've been meaning to do, ask you for examples... I'll also remind folks that we might want to set a time for the next meeting before the hour is up

isd: I think I can give more detail on the PR

Jessica: also while I'm speaking, I'd like to encourage... if you think there is not consensus from reading the doc, and there isn't an issue about it, please file an issue

cwebber: I would like to hear the feelings from four people in specifically: Mark, Jessica, Ian and jar. It's worth doing a vibe check. It feels like a breakthrough moment in the group about being able to move forward

Jessica: I'm very positive, especially after this meeting. there's obviously been a lot of convo on the concrete syntax, I'm very invested in, I did a lot of work in the SocialWG and it didn't necessarily have interop and over the course of the group was difficult to work through. I'm interested in spec work in terms of getting implementations to talk with each other, getting new implementations, creating documentation and

Jessica: technologies that further this space. I'm very excited about that, and certainly excited to work with Mark and everyone at Agoric to work on this. I think we have a good shot of coming to consensus on this

markm: I agree with the general positivity. I think it's very interesting how the issues have... as we've broken things into layers that's been clarifying for me too, made my vision sharper. until recently I kept talking about 3 layers, all of which were about data, and the nice thing in Jessica's document is it's focusing on messaging in terms of capabilities, and that's really nice, we've got our data and slots arrays and things

markm: have to be translated from clist context to clist context. I think that's going to be very very challenging, we need to have blockchains be able to participate in the fabric, that's of course been central to what Agoric has done. IBC is the netlayer for talking to chains, and it's been very designed to meet these needs, and doing 3 party handoffs between combinations of chains and non-chains, we'll likely find it quite

markm: challenging, and it's necessary to bite this off. One thing I noticed about the document I read, it's more like old E, making assumptions that endpoints can keep secrets. weird thing about blockchain is that the endpoints can't keep secrets. One final note about data agreement is that we're really stuck on the abstract syntax, so things that are still minor issues are gating issues, there's two levels of tagging, distinction

markm: between record / string / symbol, and then there's the Tag, and we can't bridge to an encoding that takes some of the things problematic to you like null and just tosses it into the tag, because then it's not "the tag". if we did that it's an extension of the first level tagging, we'd still encode our ? as the top level tagging, because the tag has to be exaclty the tag

isd: also to echo the general positivity, I think... the kind of approach I am taking to this is kind of a one-issue-at-a-time thing. we'd been making good progress on the data model thing, and I think I'll try to comment, how does everyone like this particular form, because one of the things I found... it's a bit difficult I found to navigate the comments in the agoric source to understand the data model, so I might need to try to

isd: rephrase it in a way that makes sense to me, and think about what others might do to work into the tags. in addition to the discussion here I feel like we're getting close on that, and ask "how does everyone feel about this exact spec"

isd: from there I'd like to start pinning down the semantics of errors in particular

isd: that's something capnp has a lot of opinions about, and i might need to bother kenton about, but yeah, that's all I have to say

jar: I don't have much to add... I guess I admire the recent interchange of taking and kind of forcing the issue of taking what could be a disagreement and instead of talking around it and being nice to try and attack it head on and it seems like it worked really well and I'd like to see it repeated

Jessica: can I propose to markm that you provide info to issue #5 with the potential concrete syntax, the one you and Richard Gibson asked about

markm: richard gibson can we work on that before next meeting?

richard: I think that's possible, not too many obstacles

markm: I'll volunteer to next meeting talk about the magic ordering properties, they're really pretty cool, I think you all will be entranced by it

cwebber: can't wait!

Jessica: yes look forward!

Zarutian: looking forward to it!

jar: okay, glad for that specific action item, and I guess Jessica has plenty to do, and if anyone wants to volunteer...

Jessica: I can say I should be able to get test suite published, can you read and comment on spec

cwebber: and just to remind this is a starting point spec

jar: okay I think we're over and done! move to adjourn!

Decision: May meeting will be on the Tuesday 23rd of May 19:00 UTC