OCapN Pre-standardization Group

May 2024

Minutes

Dave: I will scribe this time. I believe I scribed last time, too, so I would like someone else to scribe next time.

#93 Concrete Syntax

MarkM: (scribe note: I started scribing hastily and missed some of this) We have not been able to get to the work to show Compact Ordered as a viable alternative for the concrete syntax. We had a discussion internally and we agree that it's better to agree on a syntax and move forward then delay for an unbounded amount of time. We withdraw our objections to Syrup and we agree to go forward with Syrup as the concrete syntax. Kris Kowal will explain that we'll need some kind of negotation about future concrete syntax. We're agreeing to Syrup as the first concrete syntax.

Jessica: That's really good to hear about moving forward with a concrete syntax. One question I have: What does "first concrete syntax" mean?

MarkM: I'll let Kris represent himself.

Kris: is muted or has mic problems

MarkM: I think Kris mentioned something about protocol negotiation.

Kris: is unmuted A recurring observation is that even if we only have one concrete syntax, that syntax will evolve over time and there will be different protocol version. I am not proposing that we have a protocol at the base that has protocol negotation, advertisting which protocols we accept. I don't think that kind of complexity is necessary. I do think it's necessary for the ocapn locators to specify the version of the protocol so that we have some freedom to change. There are 2 reasons for this: Versioning and no single protocol satisfies all of our requirements and we will want to mix/match and experiment. Having the version in the locator gives us freedom to change. I don't think we should use "ocapn://" right now, but maybe in the future.

Jessica: I think identifying the version in the locator is a good idea. I think we're a long way from having a spec that isn't a draft spec. There will be a lot iterating that this group does and if we proceed to a standards body there will be even more iterations.

Cab: Hi! I am doing OCapN in Haskell with NLnet funding. I spoke with someone from Spritely on one of my PRs.

Jessica: The OCapN test suite PR?

Cab: Oh that's not what I meant, but thank you for merging that. I meant I was talking to Agoric about locators. A version is good, but a version is not the only thing you'd want to know about a host. You also want to negotiate acceptable protocols for the host. IIUC, there is a Guile implementation of OCapN that can past the test suite?

Jessica: Yes, the Guile version passes the test suite. I can add something to the README of guile-goblins to explain how to run it against the test suite.

Cab: That would be good as it would allow us to check against the logs. I've noticed in some places it uses strings instead of integers in the test suite, so I'm confused about how it actually works.

Jessica: So there could be mistakes in either the spec or the test suite. We can pick which one is wrong. I think the gift id in Goblins can be any value but I need to check the source code. It's possible that the spec is wrong.

Cab: I think the spec says that the gift id should be an incrementing integer. I was banging my head against the problem until I realized the Python test suite was using strings.

JAR: Can we move this to an issue and move back to discussing old business?

Cab: Sure.

#117 Transport protocol negotiation

Kris: I am offering to discuss transport negotiation.

JAR: Does anyone have a comment on this offer?

Jessica: You're referring to discussing negotation for netlayers?

Kris: Briefly, we discussed this at a previous meeting. I brought forth a presentation for how to do transport negotation and what I learned is that such a thing is planned. The idea is that the scheme of a locator currently says "ocapn://" and means Tor. The idea is that there would be a meta-netlayer that would encode multiple other netlayers. That in turn necessitates another layer of encryption where the public key for strattling multiple netlayers is different than the key for the contained netlayers. I have some work to do on this. For the time being, the Tor or TCP netlayers would be identified by different schemes and a multi-netlayer would come later.

Jessica: Clarifying point: The "ocapn://" scheme doesn't imply Tor. The current spec has "ocapn://" followed by an identifier followed by the netlayer name like ".onion".

JAR: I wanted to figure out where we stand on issue 93. This was introduced as a way to help organize the discussion of concrete syntax by articulating requirements to help make arguments less contentious. If requirements are no longer an issue then 93 is moot. If we have continuing need for requirements, then it isn't moot.

Dan C: One way to close the issue is to say the Syrup specs meets the requirements so far.

JAR: That would be great but I don't know if we've written the requrements.

Dan C: The March 2023 Syrup spec meets the requirements to my satisfaction.

JAR: We still need a document and editor right?

Dan C: No, the spec already exists.

JAR: Jessica, do we need to talk about the action items that you listed?

Jessica: The action items were to further the concrete syntax discussion, but it seems we've come to a concrete syntax that we can agree upon currently. So we don't need to discuss further unless the group disagrees.

Kris: To get progress on the requirements, what I think has changed, to summarize Mark, is that we are withdrawing our requirement that the concrete format be ordered or suitable for ordering at rest and we do accept the requirement that the format be able to carry bytes. Because of our change, it makes Syrup admissable for our purposes.

Jessica: It was discussed to have the version of the protocol as part of the locator. That currently doesn't exist, and I don't want to lose that. We should create an issue to move that forward.

JAR: Would you like to create the issue?

Jessica: I'm willing to create the issue, yes.

JAR: Sounds like you're on top of it. Sounds like we're coming to a decision to close issue 93 about requirements with a comment on the issue to the effect that...

Dan C: suggests using what the scribe wrote above as the text, or something to that effect

JAR: Do I have a volunteer to add the comment on the issue?

Dan C: I volunteer. p.s. done: https://github.com/ocapn/ocapn/issues/93#issuecomment-2111004643

Agoric withdraws the ordering proposed requirement and accepts that the concrete syntax should carry bytes.

Hence the March 2023 syrup spec of March 2023 97bfd6b meets requirements that the OCapN group has gathered to date.

JAR: Great! Jessica, are you the editor of the concrete syntax document?

Jessica: I have written the Syrup spec but anyone can contribute.

JAR: As far as you know, this resolution is okay?

Jessica: It's a welcome and appreciated resolution. I don't know if I conveyed that previously. A very good resolution as far as I'm concerned.

JAR: Okay, I suppose we can move on. The next item is about debugging. Jessica, you wanted to talk about that.

#55 Multiple value resolution for promises

Jessica: I do think that's important, but I could suggest something else... maybe put Dave on the spot. I propose slightly changing the order...

JAR: I'm not crazy about it, but go ahead... we have some time.

Jessica: I thought the concrete syntax would take the whole meeting. My proposition is that, I think it was last year, that we had a write-up about multi-value return. Everyone was largely unhappy about having multiple values returned from a message send.

David: It's been a while since we discussed this so I'm trying to page it back in. OCapN does not need to have any notion of returning multiple values, in Goblins we could implement this functionality by having a specific data structure which wraps multiple values.

Jessica: We could have something that isn't in the spec but Goblins uses to articulate multiple values.

David: Syrup has a way for us to make that unambiguous right?

Jessica: Yes.

Kris: I'm excited that there might be a way to get multi-values out of the wire protocol, but I'm curious about how it would affect conventions.

David: I wish Christine was here, I think I have opinions Christine doesn't agree with. I think it's a good question, we still have discussions to have on is calling an actor calling a method. Putting that aside, the way a lot of people can understand this, I come from a web developer with REST endpoints. Traditional web apps can define how to interact with them. Would haing a goblins convention inhibit interoop at all? ...

Kris: You're proposing that protocols around actors that Spritely/Agoric would return a tagged array of the values returned? An Agoric program would see a tagged array of values?

David: explains but can't talk and scribe at the same time so it's lost to time

Kris: I think the point may be moot in the protocols we discussed so far because generally we use single return. An alternative view of the universe is that the protobuf world says that all functions send and return a record since positional args/returns are not universally supported. We could adopt such a convention for programs that are to strattle Agoric/Spritely.

JAR: Jessica and Baldur have been on the queue for awhile.

Jessica: That's correct that in Goblins multi-value return would be a tagged list. Any other implementation would do as they please with the tagged array. I would say that for method invocation, I don't think that it affects this proposal because you wouldn't be doing op:pick on the values.

David: I agree.

Jessica: That feels like a separate issue.

Baldur: I might be misinterpreting op:pick but I thought it was like in E when promise pipelining a series of messages and you might want to pick a specific entry and use that as a target that you might want to handle in the same packet without a round-trip.

Jessica: That is how ocapn's op:pick was designed. If you know an actor returns multiple values and you wanted to promise pipeline on on value, you could do that. This would drop that feature. The reasoning is that it probably won't be used that much and having multi-value return is a contentious issue we could avoid but still get some value otherwise. We'd still retain programmer ergonomics with our proposal. It's a small feature loss.

Baldur: I think it would be a bigger feature loss for high latency applications.

Kris: Is variadic multi-return a thing on the Goblins side? That may be a problem for packing/unpacking single vs. multi returns. To answer Baldur's question, I think the E.get is similar to op:pick but distinct. It's for getting the indexed location of a list or the named entry of a promised struct. This is different from op:pick because op:pick gets the indexed entry of a multi-return. I don't remember what it corresponds to in the data model. I think we do want an op:get to pipeline and unpack an entry of a list or record.

JAR: I'm always concerned when we have discussion that isn't an issue on github. Is it already there?

Jessica: It's already there I think.

JAR: It's from awhile ago, though.

Jessica: Kris you asked if we do this in Goblins. We had a version of Goblins that had working multi-return but we didn't merge it because the group wasn't in agreement and it bitrotted. We still want to implement it for dev ergonomics and we'd like to revisit it.

JAR: Can I get a volunteer to make a connection between this meeting and that issue?

Jessica: I volunteer.

JAR: That would be great.

Dan C: Do you remember any words in the title?

Jessica: No I think it was general multi-value return feedback...

Dan C: Is it issue 55?

Jessica: That looks like it.

JAR: Kris has volunteered to make an issue.

Kris: I'm volunteering to create an issue for op:get to replace op:pick. If we remove multi-return from ocapn, that means that op:pick is no longer relevant but op:get will be needed. Agoric would want an operator to go with our existing E.get.

Dan C: My tiny brain would prefer that proposal in issue 55.

Kris: I'll look and make a call.

Errors & Debugging

JAR: Jessica, about debugging. I don't want to lose that thread but I don't think we have an issue for it.

Jessica: I think that's correct. I can volunteer to open an issue about debugging.

JAR: And with anything you can say in particular about it.

Jessica: Yup.

Next meeting admin

JAR: Can we get a volunteer for a scribe for next time, if possible?

Dan C: When is the next meeting?

Jessica: June 11th

Cab: Can we get an .ics file so the meetings are more discoverable?

JAR: I'd like to hold that question until we get a scribe.

Kris: Have we arrived at a resolution for multi-return?

JAR: I assumed it was still in discussion.

Dan C: Are you going to make a proposal to close the issue? Make it in text so we can all read it.

Kris: I will make a proposal on the issue to resolve next meeting.

Dan C: I volunteer to scribe next meething.

JAR: I was very curious about the capnproto office hours. Does someone have something 1-2 minutes long to say about it?

Kris: I was there. They asked what they can do to maybe forward the capnproto agenda or how to help with ocapn or if there are aligned interests. Kenton felt that capnproto would be a poor choice for a concrete syntax because of the level of abstraction since they do zero-copy on the wire by doing "arena allocator coordination" if I'm saying that right. Cloudflare is using capnproto to send v8's representation of structured clone between JS workers. Agoric is against using structured clone for ocapn. It would be great to have another office hours and have folks from Spritely able to attend. I think that captures everything. (scribe note: I think I missed one or two small points, though)

JAR: It's past time. Anything else before we close? Otherwise I'm going to adjourn.