OCapN Pre-standardization Group
- Chair: Jonathan Rees
- Scribe: Christine Lemmer Webber
Christine Lemmer-Webber (cwebber) (Spritely)
Baldur Jóhannsson (zarutian) (independent)
Mark Miller (mmark) (Agoric)
Dan Connolly (dckc) (Agoric)
Kris Kowal (Agoric)
Richard Gibson (richard) (Agoric)
David Thompson (dthompson) (Spritely)
Ian Denhardt (zenhack) (capnproto)
Jessica Tallon (tsyesika) (Spritely)
Jonathan Rees (jar) (independent)
The presentation and discussion part of this meeting was recorded, you can find that:
- Setting recurring meeting times (5 minutes)
- Issue closing procedure (5-10 minutes, dckc)
- Test suite status repo (10 minutes, tsyesika)
- Ordering guarantees in concrete syntax presentation (mmark)
markm: hoping to have this meeting focus on my and richard gibson's presentations on concrete syntax
jar: okay thank you. hope these first 3 items should go fairly quickly: regular meeting time (could take to GH if need be)...
TOPIC: Setting recurring meeting times
tsyesika: fourth tuesday conflicts with Christmas for those who celebrate, and midsummer. how about second tuesday
markm: second tuesday in july conflicts with tc39 meeting, many of us will attend. second tuesday in june and august looks good.
jar: we could handle tc39 meeting as one-off reschedule
markm: yes, one-offs are common in recurring. and usually tc39 does not happen so early
ian: I am fine tracking DST, or not
tsyesika: would rather not track DST
jar: we have some inertia towards DST
ian: I will propose not tracking DST, does anyone have a problem with that
markm: I will abstain
ian: we have one minor preference for not tracking dst
cwebber: as programers we should oppose timezones and daylight savings so we should oppose DST
Decision: 19:00 UTC on 2nd Tuesdays of the month.
TOPIC: Issue closing procedure
jar: dckc proposed structure for closing issues
dckc: we can do a test with closing one of the particular issues as a test anyone may (and should!) close an issue by making a comment that
- clearly states the resolution
- cites the position of Spritely, Agoric, and capnproto, with at least 1 in favor and none objecting
- closes the issue, subject to appeal, esp. at the next meeting
Decision: jar: issue #50 and issue #46. hearing no objections to this, they remain closed. And that's it! And we have these two good examples of how we go about this. I'll figure out someplace to put it.
tsyesika: update on where work is: draft CapTP spec, got a lot of updates, I think I've addressed all the feedback to date, though I think Agoric has some review still pending
dckc: we've concluded our review
Decision: tsyesika: does anyone have objections to merging it? (no objections were heard)
dckc: I expect editorial issues to get addressed to our satisfaction in review of this PR. some technical details that I don't expect we'll resolve now; we made issues for those
tsyesika: test suite is up on github, it is written in python, currently supports tor onion netlayers because that's what spritely does but it can be flexible for netlayers. has a bunch of tests. one thing is missing is third party handoffs, but planning on adding those in the next week. and missing op:pick. has status report, but it can be improved to show positive/negative passing status, I'm sure it can be improved if people have feedback. After that will do netlayers then implementation report. by next week hoping to have test suite updated
cwebber: congrats on getting captp test suite done, and good choice getting it done in python
zarutian silently applauds
dckc: is that a real captp implementation in python?
tsyesika: yes, just enough to run the test suite, but it would be a good starting point for implementing ocapn.
dckc: that's impressive, hadn't noticed. Has anyone else looked?
dthompson: I did look at it when porting code from racket to guile and it was useful for me and neat to look at
dckc: it would be nifty to get a few volunteers are not from spritely to take a look
ian: I can try to give a peek
tsyesika: if you do find issues anyone, please file them, will try to address them in a timely manner
markm: first of all, I'm very pleased with the progress and as you supsected, pleased the tests are written not in scheme. curious why python?
tsyesika: I can answer that, I'm just familiar with python, I've been a python dev for ten years
tsyesika: could have been lua, scheme, python, but I chose the one with the widest use
markm: good choice
TOPIC: Ordering guarantees in concrete syntax presentation (#56)
(link to recording is at the top of the meeting minutes)
jar: just created issue for this, hope it helps with tracking
markm: consent check if everyone is ok with recording?
general expressions of support
markm: begins sharing screen This is mostly to provide big picture from Agoric's point of view how Agoric got to the stance it has with its layering of abstractions that touch on all the layers of abstractions we're talking about standardizing. agoric-centered for now, but aim for rest of group. focusing not on message ordering but ordering of data types layered in our system and how we make use of it
markm: much of this started with the following use case, this is idiosynchratic to what agoric is doing, useful for orientation so you can see how we got where we are. this slide represents an option for a coveredCall within ERTP. none of that will be explicit in the rest of the talk, but this is a concrete description of an individual option. When Fred in this scenario wants to express what kind of option Fred wants so that he can give what he wants, he might want to make a more generic option such that several different concrete options might satisfy it. This comes up also for assets in general. Rather than saying I want this specific concrete asset, you might say something more general. So in particular the highlighted aspects are too specific. So we've introduced a pattern language so he can say he doesn't care what particular invitation, instance, as long as the contract itself is an instantiation of this coveredCallINstantiation code. With regarding the underlying assets instead of saying he might want to buy exactly 3 moola, he can say anything that's greater or equal to 3. likewise the strikeprice doesn't have to be exactly simoleans, if it's less than that it's fine too. and the expirationDate, rather than saying exactly 100 on the fooTime clock, it can be within this range on the entry. the important thing about these patterns is that we want to be able to turn these patterns into queries against large data stores, so we can efficiently look up options that might match this particular query.
markm: so the elements of this will come back shortly. and one additional thing, instead of the greater than or equal to 3 moola, those are for fungible assets, but our system also has nonfungible and semifungible assets. and these invitations are nonfungible or semifungible assets. so the Underlying you may be asking for is maybe greater than or equal to the inviteDescSet. So any superset of this set would also be worthwhile as the underlying set purchased. these are also partially ordered (???)
markm: so the bottom later of our system is represented by the package with the epackage name endo/pass-style. and this is the taxonomy of passables, and this is the core of what we've been negotiating. and this is the only thing where we need a very strong notion of interoperability. we need these things to round trip robustly. how much interoperation we get with the higher layers, the more the better all being equal, but those are separate negotiations than the negotiations we are currently engaged in.
markm: now some of the containers we all want like sets and maps we have postponed to the patterns level because sets and maps require a notion of comparable equality. in order for a set to have equality, the elements of the set have to be different from each other from well defined notion of equality. by postponing to this pattern level we also postpone a strong sense of comparative equality to this level. at pass-style only ideas are sequences, taggables, etc which are all well defined for comparable equality of other data types. over here in patterns we have a comparative taxonomy done through the kindOf function. so kindOf and compareKeys... compareKeys is the partial order. we have this capital M which is the namespace for anything like M.any() or M.gte().
markm: about to get into representational tricks, by which I mean many systems in addition to the abstract semantics they chooose a representation which has other nice qualities. so for example IEEE if you flip the signed bit it's order preserving. so it's not necessary but it does some nice things, and likewise UTF-8 is order preserving for UTF-16. so given many degrees of freedom, without undue cost, you can make a choice that has a nice choice that has nice algebraic properties and then you can collapse in a pleasing matter. and then another pleasing aspect of the marshal layer is that it also defines compareRank, which is a rank ordering over all passable data types. it's a full order with ties. whenever anyone is talking about a sort order being stable, they're assuming the sort function is also a sort order with ties. without that a stable sort is not a meaningful concept. For example 0 and -0 might be considered to be tied in a full order. (reconsidering example.) A great example is our remotables, capability with identity. we can't order capabilities in a permissionless system, because as messages go through c-list systems, concrete order changes. abstractly order in a permissionless system reveals info that's nobody's business, so we have to consider the compareKeys level where things are meaningful, we have to say that two capabilities with identity, if they have the same identity they're the same, neither is less than or greater than the other. so at markshal layer, compareRank will put them next to each other, separate from other things you are sorting. and a further representational trick we use... the invariants between the partial order key comparison, which is one which has semantics useful to the programmer, the differences between rank order is that one of the forms of serialization is the encodePassable that richard is working on, which serializes them into strings in a rank order preserverving manner. which is why if we can demonstrate next time, which we hope to next week, if there's no unnecessary cost, then it's very nice that this has these properties. Above this is exo, which is where we define defensible remotables, which are declared with declarative patterns defined with type-like constraints. since these constraints can be somewhat more specific, messages coming in violating the type like constraints, can be rejected before coming to those methods. the nice thing about these is that they're designed for virtualization, so we can define a large number of objects, including those which spill out of the language heap. likewise we have the store package, which is for mutable tables, which have map stores and set stores. these likewise are designed for virtualization. and the mapstore is a mutable map where the keys can be anything which is passable as keys or values. again designed for virtualization.
cwebber: could you define virtualization
markm: yes, perfect segway. the swingset kernel is the OS we are running especially with on-chain computation so that within chain it connects multiple vats directly through captp message passing semantics. when you're speaking one vat to another, we don't know if one vat is on chain or elsewhere. which is why compatibility of this with inter-system protocol is so important. but within kernel we just connect without intrachain crypto protecting it, more os techniques. and virtualization is where you can have a size of data that exceeds what is reasonable to store in the langauge heap of most programming language implementation. and in particular of the js implementations running on and off chain. so endo runs on any standards conforming js. swingset uses the xs js so it can get orthogonal persistence. but part of our persistence is that the high cardinality stores we use the form of the store constructor given to us by liveslots so that the actual storage can be on disk as a form of virtual memory so the programmer can use them as if a virtual collection class as you might expect. and instances of exo classes, the total number of classes can vastly exceed what you can keep in the language. but for the programmer you can use these just as if part of lanugage heap, preserved paged in and out. and that works also with the virtualization of the c-list management in liveslottts implementation of the c-lists, for lliveslot implementation of captp. so liveslots can be bigger than size that can be in the language. so for example, probably everyone here is familiar with makeMint from E going back forever, our concrete implementation of mint, the purses are virtualized, so the total number of purses can be huge because every account for a given currency is a purse of that currency, so there's potentially a huge number of purses. so all of these need to be paged otu but without losing the capability security nature of every purse designated by a single capability which can be seperately held. that's our overall framework, that's all by motivating the properties of how we layer on top of our taxable taxonomy the patterns ane keys and in particular how to support the patterns you say, we'd like pattern based queries against these virtualized stores. via encodepassable turn into range queries of strings that the keys of the map turn into. range queries which can be efficiently looked up, and range gives you a cover. where everything that could be looked up is within the range, and that's within the range comes in... everything within inequality interval cover has all the candidates, but the candidates have to be further filtered to see if they actually match the pattern. so that's the normal thing within any such system is you have to have some kind of cover/bounding box. you need to gget rid of a vast number of irrelevant things so you can do more expensive filtering of things that may be candidates.
markm: and want to make one further point here... hopeful that all tehse representational properties, that much of what we've done here is also useful for many other systems that aren't just using it for the purposes that agoric us using it for. things that are generally useful tricks, we've taken pains to design them in very neutral ways. there's no concept of ertp or etc in here, at this level it's still all defined in terms of very widely reusable concepts I'm hoping we'll all see utility of reusing.
markm: for our system there are three levels of abstraction where notably our kindof is a finer taxonomy than passStyleOf... ordering coming from partiall ordering package...
markm: and useful emergent classifications that are transitive through the containers. containers contain a containment tree. the tree with no remotables or promises is "just data". anything that is transitively through the containment
markm: this is showing the comparerank, treats tag as opaque, compares rank ordering of tag without understanding what the tag encodes, can be done at all sites whether you understand the meaning or not. compareKeys ... preserves inequality so it can look
zarutian notes that cwebber had to go. As this presentation is being recorded, the record will be linked later. Transcribed by him.
jar: any objections to adjourning?
tsyekia: if you send me the recording and slides I can put it up on ocapn.org
markm notes that an open mic convo unrelatted from kriskowal inadvertantly captured.
jar: if nothing else we are adjourned.