OCapN Pre-standardization Group

November 2024

Agenda

Minutes

Backpressure

https://github.com/ocapn/ocapn/issues/20

Christine: When last discussed, there was agreement between at least me and Kris.

Christine: Backpressure handling doesn't belong in the CapTP layer.

Jessica: How does this affect netlayers?

Mark: Something will happen when a buffer fills up... what is that?

Kris: Backpressure is a part of the system as a whole and there should be systems to mitigate infinite buffer bloat.

Kris: It is sometimes called "rate limiting" and can mean either drops or pacing outbound messages.

Kris: Rate limiting should occur either in the data model above CapTP, or internal to a particular netlayer (as in e.g. TCP telling the producer to stop sending).

Mark: I've always thought of backpressure as being purely a producer-side reaction to lack of capacity.

Kris: "Backpressure" is a metaphor going back to tanks connected by pipes, but we'll not get into that now.

Christine: I wonder if "backpressure" is the wrong word for what we want, which is "mitigating network overload". One thing we could do is talk about the layers responsible for handling that. Another is opening tickets for specific netlayers.

Christine: But I do think the term "backpressure" is loading in certain assumptions.

Jonathan: Can we get a volunteer comment on the issue(s)?

Christing: I will.

Kris: Does anyone object to my position?

Mark: I'm still getting oriented. But if we specify a netlayer and a CapTP layer that only depends upon a specified netlayer, then I don't think we can avoid a decision about whether or not the netlayer talks to the layer above it about pressure.

Kris: To clarify, what I'm saying is that such mitigations of unbounded message buffer growth belong entirely in the netlayer.

Mark: And would that layer communicate to the layer above?

Kris: The only signal that should be passed in that direction is to indicate session termination.

Mark: If the conditions must be reflected above the netlayer, the only observable means would then be termination.

Kris: Pacing can be accomplished between peers within the netlayer, with termination as a backstop.

Mark: I understand, but neither agree nor disagree right now.

Kris: As a hard constraint, we must not mitigate this problem by dropping or reordering messages.

Mark: When a connection is terminated at the netlayer... actually, this turns into a conversation about failure models. Does CapTP guarantee exactly once delivery, or at most once with FIFO behind a termination.

Christine: [from chat] PROPOSED: Specify that OCapN does not handle "outbound message passing" or "session termination" at the CapTP layer, but "session termination" is to be informed to CapTP, then close backpressure issue #20. Open separate issue(s) for "outbound message pacing" for individual netlayers.

Mark: Waterken and Agoric provide for exactly-once delivery, which implies infinite storage obligation. At-most-once delivery as used in E may drop messages, but no message may get delivered unless everything in front of it has been delivered. At-least-once delivery requires the recipient to implement idempotence.

Christine: This feels like a separate issue, but... the expectation is that at-least-once is not in scope for CapTP, which expects an ordered stream. A netlayer can presumably choose between exactly-once and at-most-once (if the former is even possible).

Christine: The CapTP perspective is oblivious; severance may not occur in some netlayers. But if two parties lose synchronization and cannot reconcile without disconnect/restart renegotiation, that's effectively the same thing anyway.

Christine: Exactly-once is a version of at-most-once where CapTP never sees a severance.

Christine: This is good to capture, but deserves its own issue.

Mark: Agreed. But we shouldn't close #20 until the other issue is documented.

Christine: I would like to work with Kris on this. If we agree, then I think we can probably get everyone else to agree. Let me type something up.

Kris: I do think there's something here for a meta-netlayer spec to document constraints and obligations for all netlayers.

Abstract passable data model

https://github.com/ocapn/ocapn/pull/125

Kris: I got feedback today, which I intend to address. I agree that selectors and strings should have the same constraints with respect to UTF-8. "Remotable" vocabulary still needs agreement. But it is clear that Agoric and Spritely both have two terms for their concrete manifestation.

Mark: At Agoric, "remotable" encompasses both local objects and remote presences, which is necessary for pass invariance. A system must have names for those subcategories, but I don't know that a protocol should.

Kris: I don't think the wire format makes a distinction.

Kris: A question for Spritely: do local-repr vs. remote-repr need a wire format distinction?

Christine: I think the OCapN spec talks about "remote references", but has operations desc:export vs. desc:import-promises vs. desc:import-objects.

Mark: And it's the object identified by the reference that receives and processes messages.

Kris: In the case of a third party...

Mark: References must be recognizably eq to each other, but promises are much weaker. Only fulfillment allows such comparison.

Kris: I propose fast-forwarding to consensus for renaming what I have called "remotable" to "reference".

Mark: A promise is also a reference.

Kris: That implies that one criterion for the name is not differentiating local vs. remote, but rather promise vs. not.

Mark: I like "remotable" with "imported" vs. "exported" qualifiers, particularly because "remotable" makes a good pass-invariant classification (and likewise for "promise").

Kris: Am I correct that Spritely folks don't like the term?

Christine: "Remotable" does not suggest to me exclusion of remote promises. What about "remote actor"?

Mark: From the perspective of CapTP, promises are also actors.

Kris: So we have a perfectly suitable name for promises, but not for referencable objects with convergent pass-invariant identity.

Mark: What I'm stuck on is that the name should identify a pass-invariant category. I like focusing in on the concept of identity to inspire a name.

Jessica: As I understand things, "remotable" encompasses both local objects that can be shared and references to remote objects. So I find the term confusing, and also disagree with "remote actor" for the same reason. It also suffers from encompassing promises, as previously noted.

Baldur: [agrees]

Kris: I propose that the wire protocol and abstract model use "identifier". I know this means something very different in the concrete language implementations, but it does capture the "identity" concept.

Mark: I object.

Christine: In a JavaScript environment, what term would you use for the things can receive messages but aren't promises?

Mark: E used "methodical object", which I would not suggest.

Christine: Kris suggested "target" in chat, which I like.

Kris: I think we're looking at promise vs. target or promise vs. non-promise.

Mark: Is "target" intuitive as the pass-invariant notion, or just for the exported subset?

Kris: My intention was to convey the notion that it is unique, as in the center point of concentric circles 🎯.

Kris: Are there any objections to "imported target" vs. "exported target"?

Richard: To answer Mark, I think "target" does capture the right intuitions.

Kris: I see broad approval now, but we don't need to agree now. I'll update the PR for discussion next month.

Adjournment

[Jessica and David both volunteer as potential next scribe]

Chat transcript

[18:59] Welcome to OCapN!
For help on using BigBlueButton see these (short) tutorial videos.
To join the audio bridge click the phone button.  Use a headset to avoid causing background noise for others.
To join this meeting by phone, dial:
  +1-718-247-9666
Then enter 40009 as the conference PIN number.
[19:00] Baldur: h'lo folks!
[19:00] JAR laptop: hi
[19:01] Baldur: loud and clear
[19:01] Jessica: hey
[19:01] David Thompson: hi
[19:01] Jessica: do we have a cryptpad and scribe?
[19:02] David Thompson: the dog objects :)
[19:02] Jessica: spritely did last month
[19:02] Christine Lemmer-Webber: that was me, someone threw me a treat ;)
[19:02] Jessica: I think we agreed to alternate
[19:02] Christine Lemmer-Webber: AJ Rumph, are you new?
[19:03] Christine Lemmer-Webber: or did I miss you before :)
[19:03] Baldur: cryptpad?
[19:03] Christine Lemmer-Webber: welcome regardless
[19:03] JAR laptop: Lookingf for an Agoric scribe volunteer
[19:03] Christine Lemmer-Webber: I'll make the cryptpad
[19:03] Richard Gibson: I'm on mute while another meeting wraps up, but if there's nobody else from Agoric then I guess I'm it
[19:03] AJ Rumph: My first time here...
[19:03] Jessica: welcome!
[19:03] AJ Rumph: joined listen only, so looking for the mic
[19:03] David Thompson: welcome AJ :)
[19:04] Christine Lemmer-Webber: https://cryptpad.fr/code/#/2/code/edit/LcnRQM9Hjo56PuN9xwrVNxH8/
[19:04] AJ Rumph: Agoric TPM
[19:04] AJ Rumph: technical project manager, saw this on eng calendar
[19:05] Jessica: I appologise for how long that took
[19:05] Jessica: to get those published
[19:05] Jessica: I was off all october
[19:07] Christine Lemmer-Webber: I would be okay tabling it
[19:07] Christine Lemmer-Webber: I do think kris should be here for it
[19:07] Jessica: yes
[19:08] Christine Lemmer-Webber: I have several pieces of feedback that I failed to get in beforehand ;_; but I think overall the doc looks really good.  From semantics, I have no objections, just some comments on naming
[19:08] Baldur: Markm: konnichiwa
[19:08] Christine Lemmer-Webber: I can promise to get in pretty much immediately
[19:08] Christine Lemmer-Webber: I think it's good!
[19:08] Baldur: how was japan? had any fugue fish?
[19:08] Christine Lemmer-Webber: generally
[19:10] Jessica: I think issue review is useful
[19:10] Jessica: I think we've been making great progress on those
[19:10] Jessica: +1 to jar's comment
[19:10] Baldur: I want to close #24 as it is mostly historical now
[19:11] Baldur: btw my github handle is zarutian
[19:11] Jessica: I suspect #29 could take us all meeting :)
[19:11] Jessica: didn't we speak about this last meeting?
[19:12] Jessica: I wasn't here but I read a little bit in the minutes
[19:12] Christine Lemmer-Webber: q+
[19:12] Amy Grinn: Kris just joined, hi!
[19:12] Kris Kowal: Ahoy
[19:12] Baldur: #20 backpressure being discuss3
[19:12] Kris Kowal: ty
[19:13] Jessica: netlayers are part of OCapN so netlayers aren't out of scope
[19:13] Baldur: #/3/e/
[19:13] David Thompson: oh Kris is here!
[19:14] David Thompson: Amy noticed first :)
[19:14] Jessica: q+
[19:14] Kris Kowal: Can recap
[19:14] Baldur: plus there are transports that do not have backpressure built in like tcp/tls
[19:15] Kris Kowal: Ah, well. TCP _is_ backpressure.
[19:15] Christine Lemmer-Webber: I would like to also hear confirmation from MarkM that he's happy with Kris and I's opinions
[19:15] Baldur: udp and webrtc comes to mind plus avian carriers
[19:15] Christine Lemmer-Webber: yes, netlayers may have them
[19:16] Jessica: I also missed it so a recap wouldn't hurt I think
[19:17] Baldur: inifint bufering or message load shedding
[19:17] Kris Kowal: i can answer mark
[19:17] Christine Lemmer-Webber: yeah let's hear from kris!
[19:18] Baldur: ratelimiting or even postage
[19:19] Baldur: such as tcp windowing?
[19:20] Baldur: or even xon/xoff serial link flow control
[19:20] Christine Lemmer-Webber: is exponential backoff backpressure? :)
[19:20] JAR laptop: (Kris's comments at this point echo his Oct 8 comments on the issue)
[19:21] Baldur: from garden hoses too
[19:21] Baldur: expetienced this myself firsthand
[19:21] Baldur: tymnet stuff?
[19:21] Christine Lemmer-Webber: exponential backoff is typically on the "client" side
[19:22] Christine Lemmer-Webber: I wonder if the right thing to do is to talk about "avoiding network overload"
[19:22] Christine Lemmer-Webber: and thus "backpressure" is both too vague and too specific
[19:22] Christine Lemmer-Webber: "network overload mitigation"
[19:23] Baldur: not quite as recieving machine might be too busy working through messages
[19:24] Christine Lemmer-Webber: q+
[19:26] Kris Kowal: back pressure is both ambiguous and most interpretations are not applicable
[19:26] Kris Kowal: i recommend specifically “outbound message pacing” and “session termination”
[19:26] Kris Kowal: exponential back-off is an outbound message passing implementation
[19:27] Kris Kowal: pacing
[19:27] Christine Lemmer-Webber: I could leave mine :)
[19:29] Kris Kowal: q+
[19:30] Christine Lemmer-Webber: my understanding that "above" would be something like object-level refusal or delay of messages
[19:31] Christine Lemmer-Webber: which can operate indepdently
[19:32] Jessica: I agree, we should retain FIFO and not drop messages
[19:32] Baldur: so, in order and gurantee dilver or otherwise session dropping iiuc
[19:33] Christine Lemmer-Webber: q+
[19:33] Christine Lemmer-Webber: two things to comment here
[19:33] Christine Lemmer-Webber: PROPOSED: Specify that OCapN does not handle "outbound message passing" or "session termination" at the CapTP layer, but "session termination" is to be infromed to CapTP, then close backpressure issue #20. Open separate issue(s) for "outbound message pacing" for individual netlayers.
[19:35] Jessica: I think it's not just individual netlayers, but the netlayer spec is also somewhat of a meta spec trying to convay the concerns of new netlayers
[19:35] Kris Kowal: caveat, these infinite message queue impls around Agoric also have postage
[19:35] Christine Lemmer-Webber: "exactly once" "at-most-once" "at least once"
[19:35] Jessica: so that'd be an ideal place to put the responsibilities and suggestions
[19:35] Christine Lemmer-Webber: just to help me remember
[19:35] Jessica: i.e. not just netlayer specific
[19:35] Baldur: curious might also wanto look at AmbientTalk
[19:35] Kris Kowal: I agree with Jessica that meta-Netlayer spec should speak to the design constraints for netlayers
[19:36] Kris Kowal: constraints and obligations
[19:37] Baldur: sound like data channels over webrtc connection
[19:38] Kris Kowal: session termination is certainly severe!
[19:39] Baldur: I am not going to mention SAC lines here as a netlayer building block
[19:40] Kris Kowal: provably false, baldur
[19:41] Baldur: mostly because its a weeds fertilizer as in getting into the weeds
[19:41] Jessica: I also don't think we should close it
[19:41] Jessica: fwiw
[19:41] Baldur: keep #20 open, yes?
[19:41] JAR laptop: yes
[19:43] Amy Grinn: Do you want to go back to the abstract data model?
[19:43] Christine Lemmer-Webber: I left feedback ;)
[19:44] Baldur: Far-able?
[19:45] Jessica: whatever term we agree on, I will update the spec
[19:45] Christine Lemmer-Webber: far isn't helpful if two vats are in the same... whatever we called it
[19:45] Christine Lemmer-Webber: node or whatever that was :)
[19:45] Jessica: peer
[19:45] Baldur: you know as in 'Far out man!' ;þ
[19:47] Baldur: object that is the peers export table on the session
[19:48] Jessica: not functions, they're types
[19:49] Jessica: if you say it I can probably say
[19:49] Jessica: and desc:import-promise
[19:50] Baldur: desc:import desc:export
[19:50] Christine Lemmer-Webber: q+: what about just calling it "ImportObject" and "ImportPromise"
[19:50] Jessica: in goblins that'd not be true as that'd be a handoff
[19:51] Jessica: q+
[19:51] Baldur: 3ph handoff rears its head
[19:51] JAR laptop: icu on q, christine and jessica.
[19:51] Jessica: maybe mark is just going to say what I want to say
[19:51] Baldur: eq? too? damn
[19:53] Kris Kowal: q+
[19:53] Jessica: lol
[19:54] Baldur: jessica: i can bore you death regarding the 3ph in E captp if you want
[19:55] Christine Lemmer-Webber: Kris: pls review https://github.com/ocapn/ocapn/issues/20#issuecomment-2471449037
[19:55] Baldur: I "stole" the gift table stuff to use in ActiveCapCerts
[19:56] Baldur: the local half circle in the diagramming btw
[19:56] Christine Lemmer-Webber: or "Remote Reference"
[19:56] Jessica: I don't object :)
[19:56] Christine Lemmer-Webber: ah
[19:56] Christine Lemmer-Webber: bleh
[19:56] Christine Lemmer-Webber: is it not remoteable? ;)
[19:56] Jessica: object reference?
[19:56] Christine Lemmer-Webber: RemoteActor ;)
[19:57] Jessica: but isn't remotable not always a remote actor?
[19:57] Christine Lemmer-Webber: q+
[19:57] Jessica: q+
[19:57] Jessica: unless christine says what I plan to
[19:57] Christine Lemmer-Webber: I suspect I might
[19:58] Christine Lemmer-Webber: let's both talk at the same time and see if we end up as a chorus ;)
[19:58] Christine Lemmer-Webber: jk
[19:58] Baldur: so no 'missing resolution bug' that the old wormholeop of VatTp was to solve
[19:58] Jessica: I don't love it
[19:59] Baldur: Dale Schumacer might object!
[20:00] Jessica: I still want my q+ but me and christine have similar thoughts
[20:00] Baldur: q+
[20:00] Christine Lemmer-Webber: that's settled
[20:00] JAR laptop: Jessica is still on the queue
[20:01] Baldur: its 20:00 UTC
[20:02] Christine Lemmer-Webber: q+
[20:02] Baldur: sjálfsstakað doesnt work
[20:03] Christine Lemmer-Webber: oof
[20:03] Christine Lemmer-Webber: remoteable is pretty bad then
[20:03] Baldur: s/ð//
[20:03] Christine Lemmer-Webber: invokeable!
[20:03] Amy Grinn: I have to go, but it was nice hearing everyone!
[20:03] JAR laptop: q = baldur, christine
[20:03] Christine Lemmer-Webber: invokeable!
[20:04] Baldur: ya invoke a promise
[20:04] Baldur: ya can*
[20:04] David Thompson: I'd like a noun that doesn't end in "able"
[20:04] Jessica: I would also like that dave
[20:04] Kris Kowal: q+
[20:05] Baldur: fjarvirkjað? naah
[20:06] Christine Lemmer-Webber: eekable
[20:06] Baldur: ident for short of identifier?
[20:06] Baldur: eekable? like it
[20:06] Jessica: I suspect we're not going to solve it, but I have time if we go longer
[20:06] Baldur: íkable
[20:06] Jessica: keep able
[20:06] Jessica: :)
[20:07] Baldur: concretes?
[20:07] Kris Kowal: target?
[20:07] Baldur: Methodical?
[20:07] Christine Lemmer-Webber: wait I like target
[20:08] juli (she/her): target is good
[20:08] David Thompson: Target is pretty good
[20:08] David Thompson: best suggestion so far
[20:08] Baldur: it hits the spot
[20:08] David Thompson: lol
[20:08] Christine Lemmer-Webber: haahha
[20:08] Jessica: I would love if it was more explicitly not promise, but I'll not block :)
[20:08] Baldur: /me not sorry for that pun
[20:09] Jessica: target I think is as good as any
[20:09] David Thompson: let's all go to Target after we agree on this
[20:09] David Thompson: Unpromise
[20:09] Chip Morningstar: There are lots of things that aren't either.
[20:09] Christine Lemmer-Webber: TargetActor ;)
[20:09] David Thompson: Target is also quite short which is nice
[20:10] David Thompson: 1 word better than more words
[20:10] Baldur: I took target to mean as its the reciver in a deliver and deliver only invocation message
[20:10] Christine Lemmer-Webber: no objections
[20:11] Baldur: no objection agains "imported target" or "exported target"
[20:11] Jessica: 0+
[20:11] Jessica: :)
[20:11] Christine Lemmer-Webber: +1
[20:11] Jessica: if we're voting
[20:11] David Thompson: I can think of "endpoint" as a synonym but "target" is better
[20:11] David Thompson: +1 for target
[20:11] Christine Lemmer-Webber: endpoint is good but I bet we'll get the "but promises are endpoints" argument
[20:12] Christine Lemmer-Webber: +1 to kris capturing
[20:12] Jessica: thanks for your work kris on the PR
[20:12] Baldur: captive target audience
[20:12] Christine Lemmer-Webber: "the closest shade of paint we can agree on for this bikeshed yet!"
[20:13] Jessica: scribe for next time?
[20:13] Jessica: I can do it :)
[20:13] Jessica: make it easy and quick
[20:13] David Thompson: jessica or I can do it
[20:13] Christine Lemmer-Webber: yay
[20:14] Jessica: thanks to richard for scribing
[20:14] David Thompson: ba dum
[20:14] David Thompson: tss
[20:14] Baldur: meeting adjurned
[20:14] David Thompson: byeeee
[20:14] juli (she/her): see y'all next time
[20:14] David Thompson: thanks!