OCapN Pre-standardization Group

October 2023

Minutes

Agenda

https://github.com/ocapn/ocapn/issues/83

Minutes

JAR: Any objections to list of closed issues?

JAR: no objections that I hear. Issues remain closed.

JAR: I took the initiative to take some time for myself. I'm confused of where we are on abstract vs. concrete syntax. There's been a lot of talk but it has been going on a long time and I want to drive it to some closure so we can work on interop issues.

JAR: This issue stepped up in April when Jessica requested review of smallcaps as an attempt to reconcile Agoric and Spritely.

JAR: The question is where were things then and where have things gone?

JAR: Ian gave an overview in April or May?

MarkM: It was in May

MarkM: The core data types issue is the main thread.

MarkM: We've resolved the -0 issue.

MarkM: There is the capability vs. promise + remotable. Not reflected in the issue, we have verbally agreed that considering remotable and promise to be 2 distinct types.

MarkM: There is the issue about strings: UTF-16 vs. UTF-8(??) as the basis. I think the consensus is: For unicode characters, we accept them as unicode characters. We avoid the UTF-16 encoding as the semantic encoding for unicode characters in the "other planes" (bigger than 16 bits), but we do accept unpaired surrogates. Things that come from the unicode character map that do not validly form unicode characters. We want to preserve those in round tripping as well.

JAR: I'd love to have a document that describes the abstract syntax that has consensus. It would be good have it as a consensus statement ready for the next meeting. It needs to go through at least one round of review before the next meeting. Do we have a volunteer? I don't want to do it, but I will if no one volunteers.

<someone asks for clarification, didn't catch who was speaking sorry>

JAR: I want a document and a github PR for it that gets reviewed. I want to record what consensus we have.

Christine: I think most of what Mark stated is in alignment with my understanding. I don't have a shared understanding that we agreed on the UTF-16 stuff, but we should go back and find what we've discussed. Regarding time allocation, I'd love it if someone else agreed to step up for this. If no one else can volunteer, we can see about Jessica and I taking care of it. Jessica is very close to having the implementation guide finished and I want her to finish so she can get paid.

JAR: Plus a lot of document drafts have come from the Spritely side so it would be great if another party wrote this one.

MarkM: One thing we don't have consensus on is "what are the contents of errors?" It's a messy black box that we have not tried to get agreement on yet.

JAR: I'll make a new issue for the document. The issue can be used as a guide for whoever makes this. This is mostly for listing what is resolved.

Kris: I wanted to say that I would like to volunteer for this. The work I did on the Rosetta Stone was to draw out where we are and what the open issues are, which are more than we think.

JAR: This is for the abstract syntax.

Kris: Yes, I will do that.

MarkM: What are the required equivalence classes for round tripping? That makes the abstract syntax distinct from any concrete syntax.

Christine: I think there are things that could round trip that we don't have full agreement on. Here is my loose proposal: Since Kris volunteered, I propose a meeting where Kris, Jessica, and I can meet and comb through the issue, gather what we have consensus on, and try to knock out the document in 1 hour. How do people feel about that?

<Kris and Jessica give +1 in chat>

<Mark asks to be included in the meeting and gives a +1>

JAR: I will try to scope this thing small enough that it's doable, assign it to Kris, and go from there.

JAR: Next on my list is requirements for serialization. I think I'm not going to push on that now. There is a question, and I think we may be close to consensus. Are we after a primary serialization or multiple serializations? I get the impression that we are looking for a primary one.

MarkM: I propose we agree on smallcaps as the primary one.

Christine: I am -1 provisionally, not permanently, on smallcaps. I think it's too early to agree on smallcaps as the concrete syntax. There are some things that still need to be discussed. I want to agree on a concrete syntax, and I want to consider smallcaps as part of that, but I don't want to rush. We are about to take a big step forward on agreeing about abstract semantics and I want to focus on that. Jessica and I don't feel like we can agree to it right now.

MarkM: I'm fine with not arriving at a conclusion to this at this meeting.

Jessica: I agree with Christine and would also -1, provisionally. Mostly because there is canonicalization concerns regarding how its implemented. Christine and I need to go look at that issue and come back with an informed opinion. I also would want the binary data type more concretely defined instead of a WIP. Depending on how that looks, that would influence my actual vote.

JAR: This dicussion goes beyond what I asked, which is: Do we want a common concrete syntax? We don't know what it is yet, but we're looking for it. If there's consensus on that, let me know.

<Christine +1, Kris +0, no objections>

JAR: How urgent are Jessica's questions?

Jessica: They're not amazingly urgent. The reason op:bootstrap is on there is because it really complicated the implementation guide, so we removed it with the assumption that the change would be merged. The test suite does not support op:pick and the implementation guide doesn't cover it because we haven't reached consensus. They could be bumped to next meeting if we don't have time.

JAR: Since we're on a roll with syntax, if it's okay I could move to Kris. Maybe we want to treat these issues according to our issue closing policy and handle them that way?

Christine: I think having the abstract syntax knocked down is great. I feel like what we really need are requirements that inform what our serialization format will be, which may be something we already have or something new. I want everyone to write down their requirements.

Kris: I indulged in an exercise to find where consensus stands with concrete syntax and tried to identify round tripping problems. I talked with many people and gathered data in a spreadsheet. The objective is to find a spreadsheet where there are no collisions in any column that would indicate a round tripping problem. Among the things I have learned is that Guile doesn't have a clear undefined value, like Python does. Null seems to exist purely so that all JSON values are expressible in smallcaps. One thing I learned from David Thompson is that there isn't a winning JSON serialization library for Guile which suggests that the same invariant could be reserved in Guile with a custom serializer/deserializer. When we last spoke of it we jokingly called it "zilch" that would be JSON's null as a Guile value.

Kris: I think it remains an open issue if we should have 1 or 2 bottom values. I believe consensus is that we will have 2 bottom values, though.

<Christine says in chat we reached that consensus in the past, Spritely is moving forward that way>

Kris: Having a single concrete syntax isn't as desirable as we may have thought. Each format has its pros and cons that might be useful in different situations. smallcaps can integrate with existing JSON tools, syrup is better for binary data.

Kris: I think there isn't a strong need for just one concrete syntax. Another thing I learned is that the OCapN doc today uses syrup shorthand notation, which I didn't realize at first. The record imply a symbol in the head position of a record, and there's an open question if symbol is a passable data type. Agoric needs to figure out if we will continue to support JS well-known symbols. The simple solution would be to agree to exclude well-known symbols. I think we shouldn't be codifying JS specific details in OCapN.

Kris: I also learned we won't have trouble round tripping objects since Guile will be using hash tables and not association lists. So there's no confusion between maps and lists. Empty hash table is different than empty list.

Kris: Spritely has user records and are variadic. Smallcaps can only have one value in a user record. We need to decide which approach to take. We are pretty entrenched for singly tagged records.

Kris: I have not dug deeply into the CapTP layer. I have more questions than answers so I won't focus on it in this meeting. I will say that op:pick seems to be something we are removing. Open question around round tripping multi-value return.

Kris: That's the high level overview of the "Rosetta Stone"

JAR: Can we get a version of this spreadsheet in the repo? It doesn't have to be checked in, could be in the wiki. Markdown format would be ideal but whatever you want to do.

Kris: Yes I'll do that.

JAR: Are floating point sent as decimal?

MarkM: In smallcaps, they are.

JAR: So how do you guarantee round tripping.

Kris: This pertains to the JS spec more than the JSON spec. <Kris is breaking up a bit so I missed the following sentence>

JAR: The round tripping has nothing to do with JS, it's an OCapN requirement.

MarkM: The OCapN requirement is that IEEE float 64 number round trip. Smallcaps preserves all the precision needed to satisfy the abstract requirements.

JAR: But it's non-obvious. If you look at the Scheme report you can see how to do it, but you have to read papers to figure it out. I want to raise it as an issue. The requirement is clear, the implementation is difficult. We can move on and I can file an issue for it. Next is Jessica.

Jessica: I have a question that I came across while implementing smallcaps. Goblins has desc:import-object and desc:import, and smallcaps has "$" to denote a remotable but I'm unclear how that works when one side is referring to an object that another side is exporting. Is that captured here?

Kris: We use positive and negative numbers for slot identifiers, near and far.

MarkM: The key property for the separation is that the smallcaps body, JSON encoded stuff that has a slot index, as sent is also the body as received. The body doesn't need translation as it goes over varies boundaries. You can think of the slot array as the clist for the body. In turn the slot themselves contain clist indices into some enclosing clist system. The content of the slots get translated as it goes from one context to another.

Christine: You can simply take the positive vs. negative and based on whether or not it has the negative sign you can consider it an import or export and translate it into the systems we have (in Goblins)

Kris: For a future meeting I'd like to open the can of worms about method names across the abstract syntax.

<agreement in chat to discuss in the future>

JAR: Any objection to adjourning?

Christine: No objections and I'm very happy with the way things are going.

MarkM agrees