OCapN Pre-standardization Group

May 2023

Recording

The presentation and discussion part of this meeting was recorded, you can find that:

Agenda

Minutes

Agenda review

markm: hoping to have this meeting focus on my and richard gibson's presentations on concrete syntax

jar: okay thank you. hope these first 3 items should go fairly quickly: regular meeting time (could take to GH if need be)...

TOPIC: Setting recurring meeting times

tsyesika: fourth tuesday conflicts with Christmas for those who celebrate, and midsummer. how about second tuesday

markm: second tuesday in july conflicts with tc39 meeting, many of us will attend. second tuesday in june and august looks good.

jar: we could handle tc39 meeting as one-off reschedule

markm: yes, one-offs are common in recurring. and usually tc39 does not happen so early

ian: I am fine tracking DST, or not

tsyesika: would rather not track DST

jar: we have some inertia towards DST

ian: I will propose not tracking DST, does anyone have a problem with that

markm: I will abstain

ian: we have one minor preference for not tracking dst

cwebber: as programers we should oppose timezones and daylight savings so we should oppose DST

Decision: 19:00 UTC on 2nd Tuesdays of the month.

TOPIC: Issue closing procedure

jar: dckc proposed structure for closing issues

dckc: we can do a test with closing one of the particular issues as a test anyone may (and should!) close an issue by making a comment that

Decision: jar: issue #50 and issue #46. hearing no objections to this, they remain closed. And that's it! And we have these two good examples of how we go about this. I'll figure out someplace to put it.

TOPIC: Test suite status (#8 and #37)

tsyesika: update on where work is: draft CapTP spec, got a lot of updates, I think I've addressed all the feedback to date, though I think Agoric has some review still pending

dckc: we've concluded our review

Decision: tsyesika: does anyone have objections to merging it? (no objections were heard)

dckc: I expect editorial issues to get addressed to our satisfaction in review of this PR. some technical details that I don't expect we'll resolve now; we made issues for those

markm: ok!

tsyesika: test suite is up on github, it is written in python, currently supports tor onion netlayers because that's what spritely does but it can be flexible for netlayers. has a bunch of tests. one thing is missing is third party handoffs, but planning on adding those in the next week. and missing op:pick. has status report, but it can be improved to show positive/negative passing status, I'm sure it can be improved if people have feedback. After that will do netlayers then implementation report. by next week hoping to have test suite updated

cwebber: congrats on getting captp test suite done, and good choice getting it done in python

zarutian silently applauds

dckc: is that a real captp implementation in python?

tsyesika: yes, just enough to run the test suite, but it would be a good starting point for implementing ocapn.

dckc: that's impressive, hadn't noticed. Has anyone else looked?

dthompson: I did look at it when porting code from racket to guile and it was useful for me and neat to look at

dckc: it would be nifty to get a few volunteers are not from spritely to take a look

ian: I can try to give a peek

tsyesika: if you do find issues anyone, please file them, will try to address them in a timely manner

markm: first of all, I'm very pleased with the progress and as you supsected, pleased the tests are written not in scheme. curious why python?

tsyesika: I can answer that, I'm just familiar with python, I've been a python dev for ten years

markm: ok!

tsyesika: could have been lua, scheme, python, but I chose the one with the widest use

markm: good choice

TOPIC: Ordering guarantees in concrete syntax presentation (#56)

(link to recording is at the top of the meeting minutes)

jar: just created issue for this, hope it helps with tracking

markm: consent check if everyone is ok with recording?

general expressions of support

markm: begins sharing screen This is mostly to provide big picture from Agoric's point of view how Agoric got to the stance it has with its layering of abstractions that touch on all the layers of abstractions we're talking about standardizing. agoric-centered for now, but aim for rest of group. focusing not on message ordering but ordering of data types layered in our system and how we make use of it

markm: much of this started with the following use case, this is idiosynchratic to what agoric is doing, useful for orientation so you can see how we got where we are. this slide represents an option for a coveredCall within ERTP. none of that will be explicit in the rest of the talk, but this is a concrete description of an individual option. When Fred in this scenario wants to express what kind of option Fred wants so that he can give what he wants, he might want to make a more generic option such that several different concrete options might satisfy it. This comes up also for assets in general. Rather than saying I want this specific concrete asset, you might say something more general. So in particular the highlighted aspects are too specific. So we've introduced a pattern language so he can say he doesn't care what particular invitation, instance, as long as the contract itself is an instantiation of this coveredCallINstantiation code. With regarding the underlying assets instead of saying he might want to buy exactly 3 moola, he can say anything that's greater or equal to 3. likewise the strikeprice doesn't have to be exactly simoleans, if it's less than that it's fine too. and the expirationDate, rather than saying exactly 100 on the fooTime clock, it can be within this range on the entry. the important thing about these patterns is that we want to be able to turn these patterns into queries against large data stores, so we can efficiently look up options that might match this particular query.

markm: so the elements of this will come back shortly. and one additional thing, instead of the greater than or equal to 3 moola, those are for fungible assets, but our system also has nonfungible and semifungible assets. and these invitations are nonfungible or semifungible assets. so the Underlying you may be asking for is maybe greater than or equal to the inviteDescSet. So any superset of this set would also be worthwhile as the underlying set purchased. these are also partially ordered (???)

markm: so the bottom later of our system is represented by the package with the epackage name endo/pass-style. and this is the taxonomy of passables, and this is the core of what we've been negotiating. and this is the only thing where we need a very strong notion of interoperability. we need these things to round trip robustly. how much interoperation we get with the higher layers, the more the better all being equal, but those are separate negotiations than the negotiations we are currently engaged in.

markm: the layer on top of that is endo/marshal, which provides a concrete serialization of those datatypes appropriate for transmission across systems. the ocapn data types which correspond to this passable taxonomy is what is reasonably language indepedent with mild biases towards javascript, with minor leakage like the two bottoms. likewise the marshalls should be fairly reasonable as a common choice. makeMarshal is paramegerized with slot2val and val2slot, so makeMarshal is reallly focused on the data component, and is parameterized according to the capability component. and then the captp layer makes use of marshal for all the captp layering for c-lists, etc. and this is where much of the other crucial thing that we need agreement on, which is interoperation at the level of capability messages, preserving capability security, this is where much progress you have made that we have not such as 3 party handoff comes up. let me do a correction, the critical things we need on interops are on pass-style and captp, marshal could be done by adapters if we need to, but in order to have a shared notion of capability security we need those of captp still. and I think nothing I a saying should be controvercial.

markm: now some of the containers we all want like sets and maps we have postponed to the patterns level because sets and maps require a notion of comparable equality. in order for a set to have equality, the elements of the set have to be different from each other from well defined notion of equality. by postponing to this pattern level we also postpone a strong sense of comparative equality to this level. at pass-style only ideas are sequences, taggables, etc which are all well defined for comparable equality of other data types. over here in patterns we have a comparative taxonomy done through the kindOf function. so kindOf and compareKeys... compareKeys is the partial order. we have this capital M which is the namespace for anything like M.any() or M.gte().

markm: about to get into representational tricks, by which I mean many systems in addition to the abstract semantics they chooose a representation which has other nice qualities. so for example IEEE if you flip the signed bit it's order preserving. so it's not necessary but it does some nice things, and likewise UTF-8 is order preserving for UTF-16. so given many degrees of freedom, without undue cost, you can make a choice that has a nice choice that has nice algebraic properties and then you can collapse in a pleasing matter. and then another pleasing aspect of the marshal layer is that it also defines compareRank, which is a rank ordering over all passable data types. it's a full order with ties. whenever anyone is talking about a sort order being stable, they're assuming the sort function is also a sort order with ties. without that a stable sort is not a meaningful concept. For example 0 and -0 might be considered to be tied in a full order. (reconsidering example.) A great example is our remotables, capability with identity. we can't order capabilities in a permissionless system, because as messages go through c-list systems, concrete order changes. abstractly order in a permissionless system reveals info that's nobody's business, so we have to consider the compareKeys level where things are meaningful, we have to say that two capabilities with identity, if they have the same identity they're the same, neither is less than or greater than the other. so at markshal layer, compareRank will put them next to each other, separate from other things you are sorting. and a further representational trick we use... the invariants between the partial order key comparison, which is one which has semantics useful to the programmer, the differences between rank order is that one of the forms of serialization is the encodePassable that richard is working on, which serializes them into strings in a rank order preserverving manner. which is why if we can demonstrate next time, which we hope to next week, if there's no unnecessary cost, then it's very nice that this has these properties. Above this is exo, which is where we define defensible remotables, which are declared with declarative patterns defined with type-like constraints. since these constraints can be somewhat more specific, messages coming in violating the type like constraints, can be rejected before coming to those methods. the nice thing about these is that they're designed for virtualization, so we can define a large number of objects, including those which spill out of the language heap. likewise we have the store package, which is for mutable tables, which have map stores and set stores. these likewise are designed for virtualization. and the mapstore is a mutable map where the keys can be anything which is passable as keys or values. again designed for virtualization.

cwebber: could you define virtualization

markm: yes, perfect segway. the swingset kernel is the OS we are running especially with on-chain computation so that within chain it connects multiple vats directly through captp message passing semantics. when you're speaking one vat to another, we don't know if one vat is on chain or elsewhere. which is why compatibility of this with inter-system protocol is so important. but within kernel we just connect without intrachain crypto protecting it, more os techniques. and virtualization is where you can have a size of data that exceeds what is reasonable to store in the langauge heap of most programming language implementation. and in particular of the js implementations running on and off chain. so endo runs on any standards conforming js. swingset uses the xs js so it can get orthogonal persistence. but part of our persistence is that the high cardinality stores we use the form of the store constructor given to us by liveslots so that the actual storage can be on disk as a form of virtual memory so the programmer can use them as if a virtual collection class as you might expect. and instances of exo classes, the total number of classes can vastly exceed what you can keep in the language. but for the programmer you can use these just as if part of lanugage heap, preserved paged in and out. and that works also with the virtualization of the c-list management in liveslottts implementation of the c-lists, for lliveslot implementation of captp. so liveslots can be bigger than size that can be in the language. so for example, probably everyone here is familiar with makeMint from E going back forever, our concrete implementation of mint, the purses are virtualized, so the total number of purses can be huge because every account for a given currency is a purse of that currency, so there's potentially a huge number of purses. so all of these need to be paged otu but without losing the capability security nature of every purse designated by a single capability which can be seperately held. that's our overall framework, that's all by motivating the properties of how we layer on top of our taxable taxonomy the patterns ane keys and in particular how to support the patterns you say, we'd like pattern based queries against these virtualized stores. via encodepassable turn into range queries of strings that the keys of the map turn into. range queries which can be efficiently looked up, and range gives you a cover. where everything that could be looked up is within the range, and that's within the range comes in... everything within inequality interval cover has all the candidates, but the candidates have to be further filtered to see if they actually match the pattern. so that's the normal thing within any such system is you have to have some kind of cover/bounding box. you need to gget rid of a vast number of irrelevant things so you can do more expensive filtering of things that may be candidates.

markm: and want to make one further point here... hopeful that all tehse representational properties, that much of what we've done here is also useful for many other systems that aren't just using it for the purposes that agoric us using it for. things that are generally useful tricks, we've taken pains to design them in very neutral ways. there's no concept of ertp or etc in here, at this level it's still all defined in terms of very widely reusable concepts I'm hoping we'll all see utility of reusing.

markm: for our system there are three levels of abstraction where notably our kindof is a finer taxonomy than passStyleOf... ordering coming from partiall ordering package...

markm: to relate to our ocapn texonomy, these are the current draft ocapn names, here are the current passStyleOf names, this is the corresonding javascript typeof, and these are just the notes of how they map on to javascript. tbd is we don't have a comparison or byteString.... precise semantics of strings not quite settled... but with surrogate issue we know what we want. ocapn capabilities divide into remotable and promises, with distinct notion of remotable pass invariant.

markm: and useful emergent classifications that are transitive through the containers. containers contain a containment tree. the tree with no remotables or promises is "just data". anything that is transitively through the containment

markm: this is where the data has some surprising properties, which is at the passStyleOf, we have a javascript object which represents a tag. and the tagged corresonds to a Tagged object. as passStyleOf it's left uninterpreted, it's left as patterns layers and etc to interpret it. so patternsl ayer then looks at the tag name and makes a determination, is this a kind of tagged data I understand. to do that it looks to see if it has a type it recognizes. simply having the name is not suffiicient, in addition the payload associated with the portion of the tag has to obey the very precise invariant that this patterns levell is looking for in order for it to be a copySet. if it doesn't satisfy invariants, then the tagged data is considered just a tag. the idea is different systems can interoperate even when they recognize different higher levell data types encoded into the tag. if alice who understands the patterns encoding, if it sees a tag it understands as a copySet and Bob does not recognize this higher level kindOf, Bob will still treat it as an opaque tagged object. if Bob passes to carol who does understand it, then the interperetation is the same that alice had. since the tagged space is a global namespace and we know the hazards of that, we take care to recognize the invariants required by the payload. the keys are defined over this classification as anything that is primitive data or remotables. remotables have identity since they have identity. keys are elements of sets or bags... keys are things you can do inequality with with comparekeys. have primitive data at leaves of heirachy. so anything transitive through keys ending in data with primitive data or remotables are just keys. and matchers, same thing, anything that's a tree of containers, anything with rprimtive data, remotables, matchers (?), matcher is encoded into a tag, each matcher type has very particular matcher invariants and has a tag name. and that's how we endcode patterns, and that's how our patterns are also passable by copy. and patterns over keys designed to be pass invariant.

markm: this is showing the comparerank, treats tag as opaque, compares rank ordering of tag without understanding what the tag encodes, can be done at all sites whether you understand the meaning or not. compareKeys ... preserves inequality so it can look

zarutian notes that cwebber had to go. As this presentation is being recorded, the record will be linked later. Transcribed by him.

jar: any objections to adjourning?

tsyekia: if you send me the recording and slides I can put it up on ocapn.org

markm notes that an open mic convo unrelatted from kriskowal inadvertantly captured.

jar: if nothing else we are adjourned.