OCapN Pre-standardization Group
- Chair: Jonathan Rees
- Scribe: Christine Lemmer Webber
Jessica Tallon (Jessica)
Christine Webber (cwebber)
Mark Miller (markm)
Richard Gibson (richard)
Ian Denhardt (isd)
Jonathan Rees (jar)
- Discuss the draft CapTP spec
- Make some decisions on layer 0, or identify what needs to be done to resolve conflicts
jar: main thing today is going through documents
cwebber: how are we doing the queue?
jar: using queueing in the BBB chatroom, because it seems like most transparent and reliable
(more conversation about queuing)
cwebber: I think we should talk about captp but also serialization
markm: I advocate whether part of same set to talk about, that they should be part of two different layers
jar: yes I put something about data layer on the agenda
jar: but I'm not sure what we have something prepared on that
jar: cwebber if you think it can be productive I think we should talk about it
cwebber: I think yes,
Jessica can talk about it, we did the review isd and markm asked for
Jessica: for an overview of things, I did a PR of the CapTP spec, I am of course working on the work following after
Jessica: I'm not sure how much people have had a chance to look at the captp spec, I wanted to remind folks that the initial draft is based on spritely's implementation, that's not meant to favor spritely, but meant that we can file issues on where we think we don't have consensus and can do with working on them
Jessica: and obviously as we come to consensus the document can evolve over time
Zarutian (on bbb chat): drop resolve-desc from op:deliver and only relie on op:listen for that functionallity?
Jessica: I am reviewing and addressing Richard Gibson(?)'s and isd's feeback
Jessica: I propose review it and merge it into the repo and that it be easier to have PRs on parts on the spec
Jessica: eg if we're going to drop the bootstrap object, we can review that language
Jessica: and I think it'd be easier to work with than one big pull request where it's hard to see what we review
Jessica: if the group wants to go a different way that's possible
Jessica: there's a note at the top that says it's a draft spec, we could strengthen it to say that this really will probably change quite drastically, and people will probably not implement it
Jessica: the doc does mention syrup, the intention isn't to not go with that, the intention isn't to go against it, but the first draft is what spritely itself implements
Jessica: my hope is that we can converge on concrete syntax, my hope is tthat we can get more adoption by other implementers, where we can get more implementations with other concrete syntax
Jessica: it could be something else, it could be cbor or msgpack, my preference is we can discuss some kind of syntax
Jessica: other than this spec, I am working on test suite
Jessica: that's more or less the status of my sowrk and to introduce it
jar: isd is next
isd: I'm going to punt to the end of the queue
jar: markm, are you diving in?
markm: responding to what Jessica was just saying
markm: you've been very good verbally to apply context, this is written from spritely's perspective, as such it's a step towards consensus, but not yet something we can reflect something we can all agree on
markm: my concern is that it be more explicitly marked not just a preliminary draft, but rather a perspective from one of the participants, rather than something we can work on towards agreement
Jessica: yes. i can add that, make it more clear that this isn't something the group has converged on, perspective of one of the parties
Jessica: will add that to the PR
markm: regarding converging on a concrete syntax, thank you very much to look at smallcaps. i've been under the impression that it's hard to... let me ask the spritely people collectively, would you consider smallcaps directly and literally as the syntax
Jessica: I can speak to that. spritely is not tied to syrup, we can switch away from that, am fairly free in what we can change, we think its' reasonable and good, we have possibly two concerns
Jessica: both of them we could technically accept but we think it would be worse than what we currently have in syrup
Jessica: currently we can encode binary data in a very efficient way in a way that we wouldn't have to base64 encode it etc
Jessica: especially if you get store and forward sessions, if chained, you can get a balooning of data, so efficiently the efficiency of binary strings is potentially a concern
Jessica: currently we are using syrup canonicalization for the certificates we're signing, we could have that be a payload of data and sign it, that's certainly possible, to me it seems slightly less elegant than have the message and sign it, but that would be fine, we could do it, could adopt it. something we've been wondering is... if / how it would be possible agoric could change to anything, like msgpack or syrup, or cbor, or something else, something that might mitigate some of these
Jessica: eg the binary string type
Jessica: how flexible is it at agoric?
cwebber: I think we are interleaving, but I think this is one of the most critical topics of the group. I think this is an important question for how possible interrop is for the group, so I think this is important for the group.
cwebber: Ian a few meetings ago asked us to look at how compatible our technologies are(?) we've done the smallcaps. We've done that, and we think we're quite positive towards smallcaps. One of the things which kept coming up in internal conversations is how possible interop is. This is a rare opertunity, some of the things we've heard from Agoric folks is we already have stuff in production, we might no be able to change. You are hearing from us we can change, we can change the syntax. So a question to Agoric are you ossified and tied to your implementation, can you address that?
markm: it's expensive for us to change, it's not infeasible
markm: you can commit yourself to data where it's less expensive to live with co-existience
markm: the one we'd like to change to, if we were to change to a binary format. Is one that doesn't exist yet
markm: it's one I designed and implemented except a (?) that Richard Gibson addressed. It's what I refer to as encodePassable
markm: the textual representation in json, json does much of the up-front work of converting from textual to structured form, but the binary cost I'll ack as a huge problem. and there's no way to get binary cheaply in a purely textual format, I understand that
markm: the reason for binary to binary at least co-exist... the radical perspective of proposing a thing we haven't implemented is it seems to have a magical property without undo cost. order-preserving for certain semantics. semantics not at the AST but as we've architected the set of levels, it's a very nice ordering semantics that I'd like to show to the group at some point
cwebber: I think this is a very positive response, I'm happy to hear it. What you're hearing from Jessica and I is we had syrup because it was simple and we liked it okay, we're not saying it's ideal. What we're happy to hear is that you're saying smallcaps is not ideal. If we are perceeving this as an analysis and I think we should do the analysis and doing the right thing.
cwebber: I think it's worth while for this group to spend time in this design space, if we're saying yes we're going to do this right
markm: just temper your optimism a bit, the abstract syntax is something we're tied to
cwebber: having done the smallcaps review, I think we felt very optimistic about the abstract, especially given the tagged structure
Jessica: I had put myself on the queue to answer something else, but I will say that I'd be interested to hear more about your proposed solution, I find that very exciting. as I said in my comment, in terms of abstract types, I think we're fairly aligned in terms of what we need to get our captp implementations talking
Jessica: I did take a brief look pre-meeting. so I don't think the abstract syntax is a problem regarding the format
Jessica: I think you did mention a tag type similar to our records, we kind of have it, a pre-defined set of types used in our captp layer, so the rest are tagged as "user records", so I think that'd be okay too
isd: my sense of where we are on the data model stuff is that the next thing to do next, someone, I'd be willing to volunteer, is get a proposal for "this will be our abstract syntax", then this would give us something to pick at, critique, etc. we seem pretty aligned, I think we need to start specifying and nailing down the abstract syntax
Zarutian: this is a question to markm about smallcaps. how exactly wedded are you to the json that endo/smallcaps consumes? pure json, or might you say that there's eg a byte array buffer here
markm: the key thing about the json as I mentioned is that we get to use the fast native code on all platforms to do the basic json parse and stringify, and everything else we do from structure to structure transformation. that's a nice property, not just for us. but to answer your question, we know we need binary, we're committed to that, and one way to do that is an envelope..
markm: we already have an envelope containing both the json and the slots array
markm: the slots array can't have a universal encoding
markm: since we already have an envelope that has those two things in it
markm: what we have is an index into the slotsarray
markm: it would be some pain because we are using json on the envelope as well
markm: we do understand it's important
markm: in the envelope we can have room for pure binary, and for the part we call the body wich is json encoded smallcaps, what there would be is an index into the binary arrays in the envelope
Zarutian: better solution than what I was just proposing
jar: amazingly the queue is empty, interested in what Jessica is doing next
cwebber: I'd like to do it, I'd like to say some things about and the process we went through. Jessica was given not an easy task. The documentation that exists is scattered and is specific to an implementation. She worked very hard (more than I'd have done, if I had done it it's be very opinionaty) on avoding that. I would like to say that she did an increadible job of producing something neutral. It does have lispy kebab style syntax, which we could change. One thing she did do was stripped out implementation details like tables. All other explinations keep "this table"... and we had a conversation about if that was the right thing, but what I think is interesting about this is... Jessica is going to talk next about an implementation guide and stuff, which will talk about how you might implement it.
cwebber: The effort to go through and translate all the documentation to this spec.
Jessica: so what I'm working on next... obviously as my highest priority is address any feedback that's left on the captp spec that exists, there's still some that we need to get to, with the aim to get it merged, and as convos go on, I will try to keep on top of it
Jessica: most of my time is currently spent on the test suite, which will be in python
Jessica: I wanted to choose a language I was familiar with but which has a wider audience, I wanted something more people knew (not everyone knows scheme)... hope it was a decent choice. going to push it in a few days, still in the earliest stages, hoping by next meeting hoping to have something fairly decent and covering a lot of the spec
Jessica: so that's something I'll be doing, and as cwebber said, on top of that there are two more specs to write, the netlayers spec.... and there's some question about more whether I put in should go into netlayers, we talked internally, certainly open to it... but netlayers as I see it now is mostly opening a channel to the other session and that revolves around certain netlayers. first one is the tor onion netlayer, more is certainly welcome, but with more implementations, maybe people can contribute their own, but at least for now I can document the one spritely is using.
Jessica: and then there's the ocapn uri doc... I changed it to "references" because there was debate about "uris" but we can talk about it... and then there's the implementation guide which is going to be more opinionated about how one might implement this with tons of diagrams and etc (of course spec could have diagrams too)... will have import and export tables and gift tables and that is definitely coming in next few months' work
cwebber: First of all, I want to hear other people's response to what Jessica is working on and what's ahead. I think hopefully it'll be more obvious for why we structured the grant the way we did to get the ground running. Something I want to discuss is what licence we'll use. I would like to suggest Apache v2 for both the document and spec. We could use CC for the document but it's a perfectly fine licence for documents too
cwebber: Apache does interrop with GPL
markm: apache2 is our favorite license at agoric, was our favorite at when some of us worked at google... you've brought up an interesting point that is there reason that documentation can't be under code licenses?
jar: will there be contributor agreements?
cwebber: regarding contributer agreements, I think that's something we can talk about. I think it's worthwhile and simple enough to have a file of people who've signed off on an agreement that their contributions will be under Apache v2 and that they won't be agressive with their litigation(?)
cwebber: Regarding your opinion on why we have different licences. My impression from when I worked at CC is that it's partially due to historical reasons from where Richard stallman considered these very different things. This really bugged me at CC and something I kicked off before I left was the CC-BY and CC-BY-SA for GPL compatibility. It always troubled me that people made code things and people made non-code things and as someone who likes video games it always seemed strange to me that people treated these as very different things. Do we treat things differently that are functional vs cultural works
Jessica: I'm happy with apache v2, I'm happy for that to be the license for both the document and the code. I'm interested in feedback people have. I know Ian on the PR talked about organizational stuff. It's something I've been meaning to do, ask you for examples... I'll also remind folks that we might want to set a time for the next meeting before the hour is up
isd: I think I can give more detail on the PR
Jessica: also while I'm speaking, I'd like to encourage... if you think there is not consensus from reading the doc, and there isn't an issue about it, please file an issue
cwebber: I would like to hear the feelings from four people in specifically: Mark, Jessica, Ian and jar. It's worth doing a vibe check. It feels like a breakthrough moment in the group about being able to move forward
Jessica: I'm very positive, especially after this meeting. there's obviously been a lot of convo on the concrete syntax, I'm very invested in, I did a lot of work in the SocialWG and it didn't necessarily have interop and over the course of the group was difficult to work through. I'm interested in spec work in terms of getting implementations to talk with each other, getting new implementations, creating documentation and
Jessica: technologies that further this space. I'm very excited about that, and certainly excited to work with Mark and everyone at Agoric to work on this. I think we have a good shot of coming to consensus on this
markm: I agree with the general positivity. I think it's very interesting how the issues have... as we've broken things into layers that's been clarifying for me too, made my vision sharper. until recently I kept talking about 3 layers, all of which were about data, and the nice thing in Jessica's document is it's focusing on messaging in terms of capabilities, and that's really nice, we've got our data and slots arrays and things
markm: have to be translated from clist context to clist context. I think that's going to be very very challenging, we need to have blockchains be able to participate in the fabric, that's of course been central to what Agoric has done. IBC is the netlayer for talking to chains, and it's been very designed to meet these needs, and doing 3 party handoffs between combinations of chains and non-chains, we'll likely find it quite
markm: challenging, and it's necessary to bite this off. One thing I noticed about the document I read, it's more like old E, making assumptions that endpoints can keep secrets. weird thing about blockchain is that the endpoints can't keep secrets. One final note about data agreement is that we're really stuck on the abstract syntax, so things that are still minor issues are gating issues, there's two levels of tagging, distinction
markm: between record / string / symbol, and then there's the Tag, and we can't bridge to an encoding that takes some of the things problematic to you like null and just tosses it into the tag, because then it's not "the tag". if we did that it's an extension of the first level tagging, we'd still encode our ? as the top level tagging, because the tag has to be exaclty the tag
isd: also to echo the general positivity, I think... the kind of approach I am taking to this is kind of a one-issue-at-a-time thing. we'd been making good progress on the data model thing, and I think I'll try to comment, how does everyone like this particular form, because one of the things I found... it's a bit difficult I found to navigate the comments in the agoric source to understand the data model, so I might need to try to
isd: rephrase it in a way that makes sense to me, and think about what others might do to work into the tags. in addition to the discussion here I feel like we're getting close on that, and ask "how does everyone feel about this exact spec"
isd: from there I'd like to start pinning down the semantics of errors in particular
isd: that's something capnp has a lot of opinions about, and i might need to bother kenton about, but yeah, that's all I have to say
jar: I don't have much to add... I guess I admire the recent interchange of taking and kind of forcing the issue of taking what could be a disagreement and instead of talking around it and being nice to try and attack it head on and it seems like it worked really well and I'd like to see it repeated
Jessica: can I propose to markm that you provide info to issue #5 with the potential concrete syntax, the one you and Richard Gibson asked about
markm: richard gibson can we work on that before next meeting?
richard: I think that's possible, not too many obstacles
markm: I'll volunteer to next meeting talk about the magic ordering properties, they're really pretty cool, I think you all will be entranced by it
cwebber: can't wait!
Jessica: yes look forward!
Zarutian: looking forward to it!
jar: okay, glad for that specific action item, and I guess Jessica has plenty to do, and if anyone wants to volunteer...
Jessica: I can say I should be able to get test suite published, can you read and comment on spec
cwebber: and just to remind this is a starting point spec
jar: okay I think we're over and done! move to adjourn!