OCapN Pre-standardization Group

June 2023

#57 Agenda Review

markm: not as prepared as I'd like on ordering details #56

cwebber: Jessica Tallon could show test suite

Implementation plans

zenhack: perhaps folks who are about to implement something that might ossify should give a heads-up. nothing from capn-proto

markm: an encodePassable PR is in review. distinguishing -0 from 0 comes up in endo PR 1626

#12 #53 TLS+TCP netlayer, Third Party Handoffs (3PH)

tsyesika: The TCP netlayer that Spritely has implemented hasn't changed how handoffs work. For the application we have used it for so far, we've been careful to avoid handoffs as it currently lacks any NAT holepunching, but we do plan to implement something like STUN. We haven't changed the 3PH implementation.

cwebber: Ian's point that maybe there could be specicalized handoffs per netlayer is interesting. is it a good idea?

markm: 3PH is mutually suspicious... core operator somewhat surprising that Agoric has been able to postpone it this far -- ian: Cap'n Proto has likewise gone for a decade without it

markm: Part of the goal of OCapN is to have a generalized mutually suspicious capability fabric regardless of particular network encoding, so having generalized handoffs useful there

zenhack: I figured we'd have a protocol that might negotiate how to connect, eg a browser might say "I can connect over websockets and webrtc, but I can't connect over blockchain, I'm sorry". Ideally Agoric and Spritely converge on a shared protocol but capn'proto probably won't, but I'd like to make it where if there's an OCapN handoff between two cap'n Proto vats that they can choose to cut out OCapN, and vice versa. I think those are my clarifying goals

dave: describing the tls+tcp netlayer

kris: very interested in going down this rabbit hole, had a conversation about what non-blockchain protocol would be. We are considering a layering like “TCP + messaging + ed25519 encryption + capability transfer” without TLS, using an elliptic curve for signing and a different elliptic curve for transport opacity.

#6 Interrop Demo - right methods

dckc: Did I find the right API for the interop demo?

tsyesika: I only glanced at your comment before the meeting, but from my brief look I think you did. Goblin chat has a few things going on where there is a server <-> client relationship and a client <-> UI/client to attach multiple UIs to the same user, but I think you found the right place. I'll take a better look follow in the issue.

#8 Test suite demo

tsyesika: will briefly talk about test suite as-is. Have currently architected such that it could support many different netlayers. Currently only supports tor onion netlayer, but may likely add tls+tcp netlayer. In readme have defined certain actors that the implementation tested against must implement. They're small and concise so you can interact with them. Some of them have specific swissnums for the implementation must support. They are fairly small. For example, (shows Goblins implementation) as you can see it's fairly small, I'm trying to not put an undue burden on implementations, it's simple behavior to test against.

tsyesika: Because of Tor, the current demo is slow to run. As you can see it implements most but fails one because Goblins does not fully implement the spec I wrote, currently Goblins has a TODO for a section of it so the test suite.

cwebber: It was a longstanding TODO, so the test suite is correctly identifying it as a failure.

tsyesika: And the test suite has identified several problems Goblins previously had with its OCapN implementation so it has been helpful.

tsyesika: Here is how you run it, you run the test suite and give it an OCapN URI. Each test suite will have a fresh connection to that URI. Some tests will break the session by sending abort, etc.

dckc: the OCapN URI refers to the thing actually being tested?

tsyesika: Yes, this URI represents the actual thing being connected to.

tsyesika: In this case it's pulling the greeter object, a desc:export for us. This is just a helper to get the next import object, it increments an integer. There are no actual objects defined on our side. We send a message and we know what the reply should look like. This is just a placeholder, we know what it looks like on our side.

cwebber: It's interesting to look at the test suite because it's like an assembly language for CapTP, it's not like normally how CapTP abstracts over these things for you, it's kind of neat

zenhack: Yes Cap'N Proto has similar things in its test suite, it's neat to look at

tsyesika: Yes I have intentionally written it that way so that the test suite can be very predictable and explicit in its behavior and what it expects

tsyesika: Now comparing the code to the specification so you can see it side by side, on the left side of my monitor you can see the test suite code, on the right side here is the specification. Looking at op:deliver, you can see it takes a to-desc which can be a desc:export or desc:answer, and then a sequence of args. Then in the README, it says sending a hello message to the greeter. You can see how the test suite implements this and how it checks

markm: So if a message comes in and doesn't match those conditions it gets thrown away, is it an error condition or is it silently ignoring?

tsyesika: It's silently ignoring in that an implementation could be doing other things. I could design it in a different way and maybe I should, I'm certainly open to suggestions if people want to comment on the github issue tracker. But it looks for the right object and looks for that response, if it gets it it passes otherwise it fails.

dckc: Just a different question, Ian did you say you don't expect Cap'N Proto to interop?

zenhack: I think I'd like to get as close as possible with same message types and etc, and it would be convenient for me if Agoric and Spritely chose to use Cap'N Proto, but that doesn't seem like what folks are interested in doing. But if we can get enough semantic alignment so that it would be easier if Agoric & Spritely interoperated

dckc: If Cap'N Proto did this bridge, it could pass these tests, yes?

zenhack: Yes, the bridge would have to make that work but yes

dckc: Yes, that's what I'm thinking for interop with Agoric too

zenhack: My understanding is for the long term we are hoping for the Agoric and Spritely stuff to merge

MarkM: I agree with the way it was just put, we're shooting for that. That's one reason I keep bringing up EncodePassable. My hypothesis is that if EncodePassable has the same virtues as something like Syrup but in a binary protocol, but in addition has a rank order preserving protocol, that's a pretty magical property. If we could agree on encodepassable as a binary data type that would be fantastic

dckc: has that objective passed your radar

cwebber: I think there's two sides, one smaller thing is getting the captp in the room together. The bigger thing is not just 3 groups interop, but especially building the captp and friends for broad adoption on the internet, the way ecmascript and activitypub have gotten broad adoption.

markm: I agree with that completely, I think that that's why I've been pushing the encoding work

dckc: How does agreement on semantics without agreement on concrete syntax assist the goal of interoperability?

markm: I think it's a prerequisite for adoption more broadly. And I would like to see us agree on a concrete protocol.

dckc: ah... so if we define what bridges do, then we've contributed something.

markm: and that's why these seemingly minor details like do we have -0 is important, because the concrete work hinges on the abstract details as such

zenhack: From my perspective, the systems I'm working on are stuck. And so I think I'll have to bridge. So while the bridges maybe preclude, I want them to be shallow.

cwebber: I think in some ways, bridges are an implementation.

tsyesika: I think bridges in some ways are in implementation, that's true, but I don't think that's what we should aim for. And if we have things that are currently ossified, well bridges definitely serve a great purpose. But to me what motivates me is that I've seen how something looks like Cap'N Proto, whether it speaks Syrup or Spritely's things but the premise of CapTP, I think that's great, I want lots of implementations. The thing we're doing here isn't necessarily building a specification for us, that's great, but I hope we're defining a specification for all of the next implementations of CapTP that come after us. And they need to speak the same concrete syntax. It doesn't need to be the one Spritely implements

cwebber: And there's an exponential cost to implementing bridges if we only do bridges, rather than try to do a shared protocol

zenhack: Quadratic but yes

markm: I agree, we all want there to be a shared protocol and we've been doing work on something concrete. All of that's great. But I didn't hear any controversy about the ordering of priorities, that if we don't have a coherent story about semantic round tripping, then we're in a much worse situation than needing to do bridging. And I want to interject one more note which is that the way in which ordinary programmers interact, the world we're anticipating where captp is somehow broadly adopted, they don't write captp, they use objects that interact with objects, and the entire mechanism of captp is that at every endpoint there is a bridge or converter between language objects and the protocol. So it's already the case that in order to have a multi-language protcol there are converters at every endpoint. And so if there's more bridging going on, that's not necessarily that much more of an impediment.

tsyesika: I think you're right Mark, and I certainly don't diasagree with your assertion we should standardize on abstract syntax, that's important, I think bridges play a role, but when producing a test suite, I would prefer it if it has a concrete syntax. I think it makes sense for new implementations that haven't been written to speak the language.

zenhack: One thing, we were talking about the notion of 3rd party handoffs and having different lower netlayers pushed down... I'd like to ask people to review.. Cap'N Proto, it's not battle tested to do this, but I'd like to specify a way to do 3rd party handoffs that's agnostic to any netlayer details. I'd like to ask folks to read that, and that's something floated a few times. It would probably be useful as a starting point of what it might look like. It feels like that would be useful

https://github.com/ocapn/ocapn/issues/53#issuecomment-1572439783 https://github.com/capnproto/capnproto/blob/455824528cd01268d4f9491a0df961c9a3678852/c%2B%2B/src/capnp/rpc.capnp#L1411-L1512

cwebber: the handoffs we implemented are netlayer agnostic

dckc: both Agoric and Cap'n Proto have suggested otherwise

zenhack: disagreement because cryptography is mentioned

cwebber: It is the part where cryptography is mentioned, but the certificate structure is currently angnostic to the network protocol. Also I would like to ask MarkM if he thinks it can be agnostic?

markm: I think no, but many parts can be agnostic. and the non-agnostic parts is that if alice introduces bob to carol, A and B are connected somehow, and A and C are connected somehow, A has to instruct B to find C, and in order for the handoff table to work, A has to instruct C to expect it. What it is that constitutes the C that A meant. Between different transports and systems, there's I think not a univeral answer. In E that was called the vat ID, and that was a public key fingerprint. And a routing hints issue. And that's another challenge on universality. But just the identification, I think there's no universal way to identify B to C of A and C...

cwebber: If only there was a way to Universally Identify a Resource :) This is where OCapN URIs come in. But that should have more discussion

zenhack: Where I think this is analogous to HTTP, you have relatively few implementations of the protocol from a server perspective that you expect to expose to the general protocol and a large number of language implementations robust enough to talk to nginx. I expect a few applications that have robust implementations that are hardened against outside world and a large number of implementations that are, grab this robust daemon, talk to the unix domain socket, and then that daemon can handle it. And that's my objection, that having cryptography at all makes that more complicated.

tsyesika: one part of the discussion is that if it's on the same machine, you'd trust any process on the machine?

zenhack: Not really but finish your thought

tsyesika: If my understanding is correct, that's not necessarily the assumption we're working with. Even if using unix sockets you'd want to protect against a malicious other process.

zenhack: Not saying you process every process on the same machine, there's a well known daemon, you talk to this socket path, similar to some webapp sitting behind nginx. separated by users and this thing is handling web requests talking over nginx