-
-
Notifications
You must be signed in to change notification settings - Fork 2
feat(p2p): additional commitments #445
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
6 commits
Select commit
Hold shift + click to select a range
2e1dcb5
feat(p2p): additional commitments
richard-ramos 09905b5
chore: assign gabe to kad optimizations
richard-ramos b806446
chore: stream budget
richard-ramos 985bd9e
Merge branch 'master' into feat/p2p-more-commitments
richard-ramos 27ffd7a
Potential fix for pull request finding
richard-ramos 778bd79
Update content/p2p/ift/2026q2-nimlibp2p-maintenance.md
richard-ramos File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,77 @@ | ||
| --- | ||
|
|
||
| title: nim-libp2p Kad-DHT Optimizations | ||
| tags: | ||
| - "2026q2" | ||
| - "p2p" | ||
| - "ift" | ||
| draft: false | ||
| description: Improve Kad-DHT privacy and provider record advertisement behavior | ||
|
|
||
| --- | ||
|
|
||
| `ift-ts:p2p:ift:2026q2-nimlibp2p-kad-dht-optimizations` | ||
|
|
||
| Improve nim-libp2p Kad-DHT privacy and provider record advertisement behavior. | ||
|
|
||
| ## Description | ||
|
|
||
| This commitment covers two related Kad-DHT improvements. First, Kad-DHT response | ||
| messages should avoid disclosing peer connection type because it can reveal | ||
| network topology and increase eclipse-attack risk. This follows | ||
| [vacp2p/nim-libp2p#2050](https://github.com/vacp2p/nim-libp2p/issues/2050) | ||
| and the Logos extended Kad discovery privacy requirement that every Kad-DHT | ||
| `Peer.connection` field be encoded as `NOT_CONNECTED`. | ||
|
|
||
| Second, provider record advertisement traffic can overload nodes close to | ||
| popular content. The libp2p community has discussed Coral-style rejection, | ||
| spillover, and backtracking for provider records in | ||
| [libp2p/specs#163](https://github.com/libp2p/specs/issues/163), and | ||
| [go-libp2p-kad-dht#345](https://github.com/libp2p/go-libp2p-kad-dht/issues/345) | ||
| tracks these as part of the critical path toward DHT efficiency and | ||
| performance. This commitment brings equivalent optimization work to the | ||
| nim-libp2p Kad-DHT implementation. | ||
|
|
||
| ## Task List | ||
|
|
||
| ### Kad-DHT Connection Field Privacy | ||
|
|
||
| * fully qualified name: `ift-ts:p2p:ift:2026q2-nimlibp2p-kad-dht-optimizations:connection-field-privacy` | ||
| * owner: gabe | ||
| * status: not started | ||
| * start-date: 2026/05/01 | ||
| * end-date: 2026/06/30 | ||
|
|
||
| #### Description | ||
| Ensure Kad-DHT response messages can optionally not disclose local connection information. | ||
| When encoding Kad-DHT `Peer` records, set the `connection` field to | ||
| `NOT_CONNECTED`. Maintain backward compatibility by continuing to parse peers | ||
| that send other `ConnectionType` values. | ||
|
|
||
| #### Deliverables | ||
| - Kad-DHT encoder always emits `NOT_CONNECTED` for peer connection type depending on kademlia setup option | ||
| - Receive-side compatibility for non-compliant peers | ||
| - Tests covering encoding and parsing of `ConnectionType` values | ||
| - Documentation of the privacy behavior and compatibility policy | ||
|
|
||
|
|
||
| ### Provider Record Spillover | ||
|
|
||
| * fully qualified name: `ift-ts:p2p:ift:2026q2-nimlibp2p-kad-dht-optimizations:provider-record-spillover` | ||
| * owner: gabe | ||
| * status: not started | ||
| * start-date: 2026/04/01 | ||
| * end-date: 2026/06/30 | ||
|
|
||
| #### Description | ||
| Add provider record overload protection and advertisement spillover behavior. | ||
| Provider records should have local limits and throttling, and rejected | ||
| advertisements should be able to spill over by backtracking through the lookup | ||
| path rather than repeatedly targeting only the closest nodes to a popular key. | ||
|
|
||
| #### Deliverables | ||
| - Configurable provider record limits and throttling policy | ||
| - Rejection response handling for provider advertisements | ||
| - Spillover/backtracking behavior for popular provider records | ||
| - Metrics or simulation coverage showing advertisement load distribution | ||
| - Tests for provider record limits, rejection, spillover, and lookup behavior | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,69 @@ | ||
| --- | ||
|
|
||
| title: nim-libp2p NAT Traversal | ||
| tags: | ||
| - "2026q2" | ||
| - "p2p" | ||
| - "ift" | ||
| draft: false | ||
| description: Add reusable NAT traversal address setup to nim-libp2p | ||
|
|
||
| --- | ||
|
|
||
| `ift-ts:p2p:ift:2026q2-nimlibp2p-nat-traversal` | ||
|
|
||
| Add reusable NAT traversal and public address setup support to nim-libp2p. | ||
|
|
||
| ## Description | ||
|
|
||
| nim-libp2p has long tracked NAT traversal support in | ||
| [vacp2p/nim-libp2p#51](https://github.com/vacp2p/nim-libp2p/issues/51). | ||
| Downstream users already carry NAT address setup logic, including a configuration | ||
| option with `any`, `none`, `upnp`, `pmp`, and `extip:<IP>` modes, | ||
| and Nimbus code that calls `setupAddress(config.nat, ...)` for TCP and UDP | ||
| ports. This commitment extracts the reusable behavior into nim-libp2p so | ||
| applications can configure public address discovery and port mapping through | ||
| the library instead of duplicating efforts downstream. | ||
|
|
||
| ## Task List | ||
|
|
||
| ### NAT Address Setup API | ||
|
|
||
| * fully qualified name: `ift-ts:p2p:ift:2026q2-nimlibp2p-nat-traversal:address-setup-api` | ||
| * owner: not assigned yet | ||
| * status: not started | ||
| * start-date: 2026/05/01 | ||
| * end-date: 2026/06/30 | ||
|
|
||
| #### Description | ||
| Provide a reusable nim-libp2p API and builder/config integration for NAT public | ||
| address setup. The API should support the downstream modes already used by | ||
| Logos and Nimbus: automatic discovery, disabled NAT setup, UPnP, NAT-PMP, and | ||
| explicit external IP configuration. | ||
|
|
||
| #### Deliverables | ||
| - Builder/config option for NAT address setup | ||
| - Support for `any`, `none`, `upnp`, `pmp`, and `extip:<IP>` modes or equivalent typed configuration | ||
| - Public address and mapped port results exposed to transport/listen address handling | ||
| - Tests for mode parsing, explicit external IP handling, and disabled NAT setup | ||
|
|
||
|
|
||
| ### UPnP and NAT-PMP Integration | ||
|
|
||
| * fully qualified name: `ift-ts:p2p:ift:2026q2-nimlibp2p-nat-traversal:upnp-pmp` | ||
| * owner: not assigned yet | ||
| * status: not started | ||
| * start-date: 2026/04/01 | ||
| * end-date: 2026/06/30 | ||
|
|
||
| #### Description | ||
| Integrate UPnP and NAT-PMP address mapping through existing Nim NAT traversal | ||
| libraries where possible, reusing the behavior already proven by downstream | ||
| projects. The implementation should work for nim-libp2p TCP usage and be | ||
| structured so UDP-capable transports can use the same setup path. | ||
|
|
||
| #### Deliverables | ||
| - UPnP and NAT-PMP mapping support available from nim-libp2p | ||
| - Address propagation into identify/listen address reporting where applicable | ||
| - Downstream migration guidance for Logos and Nimbus NAT configuration | ||
| - Tests or examples covering mapped TCP address advertisement |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,118 @@ | ||
| --- | ||
|
|
||
| title: nim-libp2p Stream Budgets | ||
| tags: | ||
| - "2026q2" | ||
| - "p2p" | ||
| - "ift" | ||
| draft: false | ||
| description: Per-protocol stream budgets in nim-libp2p | ||
|
|
||
| --- | ||
|
|
||
| `ift-ts:p2p:ift:2026q2-nimlibp2p-stream-budgets` | ||
|
|
||
| Add opt-in per-protocol stream budgets to nim-libp2p so stream-count abuse can be rejected before a protocol handler is overloaded. | ||
|
|
||
| ## Description | ||
|
|
||
| nim-libp2p currently exposes `LPProtocol.maxIncomingStreams` as an inbound | ||
| per-peer limit for a protocol, but it does not provide a complete budget model | ||
| for total inbound streams, outbound streams, or protocol-wide stream pressure. | ||
| This leaves protocol handlers vulnerable to overload when one peer or many | ||
| peers consume too much stream capacity. | ||
|
|
||
| This commitment extends the existing `LPProtocol.maxIncomingStreams` model into | ||
| a small per-protocol stream budget matrix. The work should keep the V1 scope | ||
| narrow: it limits stream counts only, does not attempt memory accounting, idle | ||
| time tracking, or peer reputation, and keeps new limits opt-in unless a separate | ||
| compatibility decision changes existing defaults. | ||
|
|
||
| ## Task List | ||
|
|
||
| ### Budget API And Accounting | ||
|
|
||
| * fully qualified name: `ift-ts:p2p:ift:2026q2-nimlibp2p-stream-budgets:budget-api` | ||
| * owner: not assigned yet | ||
| * status: not started | ||
| * start-date: 2026/05/01 | ||
| * end-date: 2026/06/30 | ||
|
|
||
| #### Description | ||
| Extend `LPProtocol` with stream budget settings for inbound total streams, | ||
| outbound per-peer streams, and outbound total streams, while preserving | ||
| `maxIncomingStreams` as the existing inbound per-peer cap. Track open stream | ||
| counters per mounted protocol handler, direction, and peer where per-peer caps | ||
| need it. If one protocol handler exposes multiple compatible codec strings, all | ||
| compatible codec strings should share the same budget. | ||
|
|
||
| #### Deliverables | ||
| - `maxTotalIncomingStreams`, `maxOutgoingStreams`, and `maxTotalOutgoingStreams` | ||
| added to `LPProtocol` | ||
| - Existing `maxIncomingStreams` behavior preserved as the inbound per-peer cap | ||
| - Open stream accounting scoped by mounted protocol handler and direction | ||
|
richard-ramos marked this conversation as resolved.
|
||
| - Shared budget behavior for compatible codec strings mounted by the same protocol handler | ||
|
|
||
|
|
||
| ### Inbound Stream Enforcement | ||
|
|
||
| * fully qualified name: `ift-ts:p2p:ift:2026q2-nimlibp2p-stream-budgets:inbound-enforcement` | ||
| * owner: not assigned yet | ||
| * status: not started | ||
| * start-date: 2026/05/01 | ||
| * end-date: 2026/06/30 | ||
|
|
||
| #### Description | ||
| Enforce inbound per-peer and total protocol caps in the multistream accept path | ||
| after protocol negotiation, next to the existing `maxIncomingStreams` check. On | ||
| breach, reject or close the new stream while keeping the peer connection open. | ||
| Counters must be decremented on clean close, error close, connection drop, | ||
| handler crash, and muxer close paths. | ||
|
|
||
| #### Deliverables | ||
| - Inbound per-peer stream limit enforced through the existing `maxIncomingStreams` path | ||
| - Inbound total stream limit enforced per mounted protocol handler | ||
| - Rejection behavior that closes only the new stream and keeps the connection up | ||
| - Tests covering clean close, error close, connection drop, handler crash, and muxer close | ||
|
|
||
|
|
||
| ### Outbound Stream Enforcement | ||
|
|
||
| * fully qualified name: `ift-ts:p2p:ift:2026q2-nimlibp2p-stream-budgets:outbound-enforcement` | ||
| * owner: not assigned yet | ||
| * status: not started | ||
| * start-date: 2026/05/01 | ||
| * end-date: 2026/06/30 | ||
|
|
||
| #### Description | ||
| Enforce outbound per-peer and total protocol caps after outbound protocol | ||
| negotiation, because the muxer and connection manager stream-open path does not | ||
| know which protocol will be selected before negotiation completes. Failed | ||
| negotiation, cancellation, stream close or reset, and muxer close must roll back | ||
| any counter reservation. | ||
|
|
||
| #### Deliverables | ||
| - Outbound per-peer stream limit enforced after protocol negotiation | ||
| - Outbound total stream limit enforced per mounted protocol handler | ||
| - Counter rollback for negotiation failure, cancellation, stream close/reset, and muxer close | ||
| - Tests covering outbound negotiation failure and outbound cancellation | ||
|
|
||
|
|
||
| ### Metrics And Validation | ||
|
|
||
| * fully qualified name: `ift-ts:p2p:ift:2026q2-nimlibp2p-stream-budgets:metrics-validation` | ||
| * owner: not assigned yet | ||
| * status: not started | ||
| * start-date: 2026/05/01 | ||
| * end-date: 2026/06/30 | ||
|
|
||
| #### Description | ||
| Expose protocol stream budget metrics and validate the feature against overload | ||
| and lifecycle scenarios. Metrics should include protocol, direction, and scope | ||
| labels where useful, but should not add a per-peer label. | ||
|
|
||
| #### Deliverables | ||
| - `libp2p_protocol_stream_cap_rejections_total{protocol, direction, scope}` metric | ||
| - `libp2p_protocol_streams_open{protocol, direction}` metric | ||
| - Integration tests for inbound and outbound cap breaches | ||
| - Documentation describing the opt-in V1 behavior and compatibility expectations | ||
55 changes: 55 additions & 0 deletions
55
content/p2p/ift/2026q2-nimlibp2p-stream-connection-refactor.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,55 @@ | ||
| --- | ||
|
|
||
| title: nim-libp2p Stream and Connection Refactor | ||
| tags: | ||
| - "2026q2" | ||
| - "p2p" | ||
| - "ift" | ||
| draft: false | ||
| description: Refactor nim-libp2p stream and connection abstractions | ||
|
|
||
| --- | ||
|
|
||
| `ift-ts:p2p:ift:2026q2-nimlibp2p-stream-connection-refactor` | ||
|
|
||
| Refactor nim-libp2p stream and connection abstractions so peer connections and protocol streams are represented by distinct types. | ||
|
|
||
| ## Description | ||
|
|
||
| The current nim-libp2p implementation uses `Connection` both for actual peer | ||
| connections and for streams opened over multiplexed connections. In the current | ||
| code shape `Connection` inherits from `LPStream`, muxers return `Connection` | ||
| from `newStream`, and protocol handlers receive `Connection` even when they are | ||
| handling a single protocol stream. This makes the abstraction harder to | ||
| understand, increases the cost of onboarding contributors, and complicates | ||
| resource lifecycle work such as close/reset behavior and stream ownership. | ||
|
|
||
| This commitment separates the concepts so `Connection` refers to peer-level | ||
| connections and a new stream abstraction is used for protocol streams. The | ||
| refactor should preserve downstream compatibility during the migration and | ||
| reduce ambiguity in code, tests, metrics, and documentation. | ||
|
|
||
| ## Task List | ||
|
|
||
| ### Stream and Connection Abstractions | ||
|
|
||
| * fully qualified name: `ift-ts:p2p:ift:2026q2-nimlibp2p-stream-connection-refactor:abstractions` | ||
| * owner: not assigned yet | ||
| * status: not started | ||
| * start-date: 2026/05/01 | ||
| * end-date: 2026/06/30 | ||
|
|
||
| #### Description | ||
| Introduce a clear stream abstraction for protocol streams and narrow | ||
| `Connection` to represent actual peer connections. Update muxer and connection | ||
| manager interfaces so opening a stream returns the stream type, while peer-level | ||
| connection state remains on connection objects. | ||
|
|
||
|
|
||
| #### Deliverables | ||
| - New stream-level abstraction used by muxers and protocol handlers | ||
| - `Connection` narrowed to peer-level connection state and behavior | ||
| - Compatibility shims or migration aliases for existing downstream users | ||
| - Updated close/reset lifecycle behavior for streams and connections | ||
| - Tests covering stream creation, close/reset semantics, and protocol handler usage | ||
| - Documentation and examples using the new names consistently |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.