OCapN Pre-standardization Group
- Chair: Jessica Tallon
- Scribe: David Thompson
Alan Karp (alan)
Christine Lemmer Webber (cwebber)
David Thompson (dthompson)
Edward Platt (edplatt)
Ian Denhardt (isd)
Jessica Tallon (Jessica)
Jonathan Rees (jar286)
Kris Kowal (kriskowal)
Mark Miller (markm)
Mike Stay (pyrocto)
The presentation and discussion part of this meeting was recorded, you can find that:
- Approve meeting minutes from December 2022 meeting (2022-12-06)
- Spritely Goblins' handoff implementation presentation & discussion
cwebber: we're going to be recording this meeting because there are multiple people that cannot attend so they will watch after
cwebber: we are recording
TOPIC: Approval of last meeting's minutes
Jessica: let's take a vote on approving the previous meeting minutes https://github.com/ocapn/ocapn/blob/main/meeting-minutes/2022-12-06.md
APPROVED: meeting minutes for 2022-12-06
Jessica: meeting minutes approved.
Jessica: we're waiting on Mark Miller but Kris Kowal from Agoric is now here
cwebber: recording has been stopped and restarted to capture presentation
TOPIC: Presentation of Spritely Goblins' third party handoff implementation
Jessica: this is the worldview: alice on machine A, bob on machine B, and carol on machine C
Jessica: alice has a reference to bob and carol
Jessica: the goody bag in the diagram is a "gift table" where gifts can be deposited
Jessica: each side of a session has their own gift table
Jessica: connections in the diagram are import and export tables
Jessica: sessions have certain attributes: session id (pubkeys, sorted and hashed), key pair (pub/priv key), import/export table, and gift tble
Jessica: we want to do a "hand off". we are sending a reference to carol to bob.
Jessica: bob does not have a reference to carol so a handoff will occur
Jessica: a gifter is who is giving the handoff (machine a)
Jessica: the receiver is getting the reference (machine b)
Jessica: the exporter is the location of the gift (machine c)
Jessica: the first thing that happens when machine A tries to perfom the hand off to machine B is machine A sends a message: depositing a message in the A-to-C gift table.
Jessica: this is being sent to the bootstrap method on C using the 'deposit-gift method.
Jessica: something worth noting is that A sends it but C might not necessarily receive it
Jessica: this is done first because A needs the gift id
Jessica: the next step is sending a certficate from A to B
Jessica: certificates are represented by a handoff-give objet. it has a recipient key (B's public key in A->B session), exporter location (machine address to C), session (session id for A->C session), gifter side (machine A's pubkey in A->C session), and the gift id
cwebber: every session in ocapn sets up fresh keys from both sides, for clarity
Jessica: the handoff-give is wrapped in a sig-envelope with is signed by the A key of the A->C session. this is so C can verify that A actually made this certificate.
cwebber: "B-key-of-AtoB" is a key made by B specifically for that session
Jessica: B gets the certificate. the first thing B does is make a promise which will eventually (hopefully) resolve to a reference to carol on C
Jessica: Bob receives the 'say-hello message with a promise that will hopefully resolve but might break
Jessica: if B doesn't yet have a connection to C, it will create one
Jessica: B mades a handoff-receive object based on the certificate
Jessica: B sends that object machine C's bootstrap object
Jessica: handoff-receive has: receiving session (B->C session), receiving side (B's key in B->C session), handoff count (to prevent replay attacks), and signed give (the certificate from earlier)
Jessica: B is adding on extra information to the certificate so the handoff can happen
Jessica: the crucial thing to note is that the signature on the handoff-receive is B in the A->B session
Jessica: we're at the point where C has the handoff-receive. it pulls out the A->C session and gets the gift tables, import/export tables, and keys and starts looking at them.
Jessica: C checks the handoff-give signature. it knows how to do this because it has the signature of A in the A->C session.
Jessica: it can verify that the "A" is the same "A" that C knows.
Jessica: A helpfully included the pubkey for the A->B session
Jessica: C can grab that key and verify that B really is the B that A thinks B is and that B really did send this data
Jessica: C knows that this isn't a malicious message because they wouldn't have the key pair for the A->B session
Jessica: C can now look up the gift id in the A->C session
Jessica: if it doesn't exist, perhaps the A->C connection is slower than B->C.
Jessica: a promise is setup that resolves when the gift arrives
Jessica: C gets the gift id from the table and exports it to B
Jessica: B now has the reference, and it fullfills the promise to Bob
Jessica: Bob now has the reference to Carol
Ian: this seems similar to what is specced out in CapNProto
Ian: except for crypto bits
Ian: you mentioned the scenario where the message from B->C comes in before A->C
Ian: and it returns a promise
Ian: The way this works in CapNProto is that the message that B sends will eventually get a return
Ian: you can pipeline stuff on it but it won't return until it resolves
Ian: this sounds like a different
Jessica: you can promise pipeline on the reference that machine C gives to B
Jessica: when gift delivery comes in, queued messages on that promise will go through
cwebber: when the promise is fulfilled, and there are multiple promises...
Ian: that was the bit that was confusing
cwebber: there's a promise internally in C
Ian: okay I think it's exactly the same then
Ian: I've been thinking about different ways to connect 2 machines
Ian: maybe one machine will be a Tor hidden service and the other knows how to speak WebRTC
Ian: what do you think about tweaking the url to be a more structured form?
cwebber: it is parsed into more structured data
cwebber: first of all, the string representation was for simplicity
cwebber: there is an issue open about ocapn uri structure
cwebber: what you don't see here is the netlayer
cwebber: .foo in this example is the netlayer, such as .onion
cwebber: you could do .swebretc or something
cwebber: time pressure pushed up this conversation, so we're skipping over important ocapn uri details
Ian: one design thing to keep in mind: when we have a bridge setup, I want machines to discover their preference for, say, CapNProto if they both speak it.
Mark: I have a high level question: I do not know CapNProto in detail. I hear a lot of the same terminology as in E.
Mark: if you are familiar enough with CapTP for E, I'd love to hear what are the semantic differences between these handoffs and what E did.
Ian: I'm not familiar with all of the E terminology
Ian: CapNProto terminology is quite different
Ian: the names are quite different than the E names
Mark: I wanted to know if anyone is familiar enough with both to point out differences
cwebber: the differences between capnproto and this system are what we discussed in previous meetings
cwebber: I studied E's docs, capnproto's docs, and Agoric's docs.
cwebber: there are a lot of similarities
Mark: Agoric has only designed the point-to-point system so far
Mark: I'm hearing a lot about the 3 party system here
Mark: I'm wondering if anyone understands the semantics about the 3 party system
cwebber: correct if I'm wrong: 3 party handoffs were specced but not implemented
Ian: that's correct.
Ian: there is flexibility to tweak the design
Ian: I have a client that wants me to implement it for go capnproto
Ian: window of opportunity will close soon when that becomes top priority
Mark: the E captp did implement it. it's rather robust.
Mark: we should really compare this in some detail with what E did because it's a working handoff protocol.
cwebber: that would be a great thing to do.
cwebber: congratulations Jessica. great presentation! applauds
cwebber: Mark, what did you think?
Mark: I'm sorry I was late!
Mark: having missed the first 15 minutes, I could spot similarities, but I don't have confidence about spotting the differences right now.
cwebber: Jessica, can you give an accelerated recap of the diagram?
cwebber: for mark: the I/E things are import/export tables. the hand bags are gift tables.
Mark: question about gift tables I didn't fully catch
cwebber: for each pairwise session, each side generates a key.
cwebber: between A and C, "these are the hand-offs I'm giving"
cwebber: the certificates enforce and encode that
cwebber: gifter is A, receiver is B, exporter is C.
cwebber: A is depositing the gift on C. it's for B but A doesn't say that.
Mark: the table itself is inside C?
cwebber: that's correct
cwebber: A also sends a certificate to B. both operations happen simultaneously.
cwebber: recaps the handoff-give structure slide from earlier
Alan: does that have to be B's key in A->B?
cwebber: yes, this prevents having to renegotiate keys all the time
Alan: there's a correlation risk
cwebber: we're not considering a correlation attack in this scenario
cwebber: recaps signed handoff-give slide
cwebber: recaps the B->C certificate passing slide
cwebber: B is talking to C's bootstrap object
cwebber: recaps handoff-receive struct slide
cwebber: replay attacks are prevented as far as I can tell
cwebber: the messages can sit on the wire and promises can be pipelined
Mark: that makes sense. it seems to be equivalent.
Mark: the only secrets are the signing keys
cwebber: which are held privately
Jessica: B's key in the A->B session, A provided that.
Jessica: because C knows its valid, it can then verify B's signature that it gave for handoff-receive
Mark: what kind of message order are you trying to achieve?
cwebber: E order is provided.
cwebber: we don't advertise E order as being a thing
cwebber: I'm suspicious of it
cwebber: what we do promise is that between A->B and A->C there is ordering
Mark: you think E order is a property of the implementation that isn't implemented
Mark: so if it didn't follow E order you wouldn't have serious issues
Alan: Do you deal with lost messages?
cwebber: it's up to the netlayer to give the guarantee of sequential order
cwebber: ocapn was intentionally designed to handle different ways of achieving message order
cwebber: ocapn assumes that order has already been dealt with by the underlying netlayer
Alan: so promises could be broken at some point if no response occurs?
cwebber: I tried to reconcile two different needs: I imagined a realtime multiplayer game and a blockchain like Agoric, and supporting both. how would you support both?
cwebber: how would you deal with session severage and breakage?
cwebber: how to deal with irreconcilable messages? i think the session would need to be broken and re-established
cwebber: you can in theory have netlayers that are realtime that assume sessions break a lot and long-lived things like blockchains
cwebber: that's a conversation for a future time
Ian: the ability to semantically break a connection is important for avoid denial of service attacks
cwebber: in Mark's dissertation, he provides a good case for breaking sessions in the spreadsheet example
cwebber: example: there's a cell that is being updated but the connection has been lost
cwebber: Mark says it's perfectly reasonable to break the connection between the 2 parties and use a sturdyref to re-establish the connection
cwebber: that was very motivating for me for thinking about games
Mark: that's the E failure model which avoiding the big mistake I made
Mark: starting with the E failure model is good
Ian: Capnproto does this (more or less)
Alan: can you just break a promise and keep a connection?
cwebber: we don't do that
Ian: I have thought about designs for a next-gen captp with looser ordering requirements
Ian: it would be designing a new thing entirely
cwebber: would like to know what captp udp would look like
Mark: what about GC?
Ian: capnproto does reference counting
Ian: I'm open to collecting cycles but I want to see a design that does it without being too complex
Ian: Rust and C++ makes GC difficult
cwebber: our current stuff assumes acyclic
cwebber: I'm throwing this discussion back over to Jessica
Jessica: my plan is to create a github issue with a link to the recording and slides and the minutes will available in the next day or two
Jessica: I'll get a poll up for the next meeting date
Dan: could we come up with a time now?
cwebber: I'm stopping this recording
cwebber: thank you Jessica for the awesome presentation
cwebber: she only got trained on this 6 months ago and took a leadership position on captp internally and has done a great job!
cwebber: I'm happy to talk now about the next meeting
Dan: we can do one sooner
Decision: Next meeting will be 2023-02-23 8pm UTC to 9pm UTC
Jessica: could someone step up for scribing next time?
Jessica: it would be good to rotate