548396ba4a
- NodeAddr abstraction replacing SocketAddr across distribution crate (~30 files) - Core wasm32 compatibility: web-time, cfg-gated threads, wire encoding extraction - WebSocket transport (browser) with GatewayControl protocol - WebSocket gateway (native) with session routing and control protocol - BrowserRuntime API: JS actor support, connect/spawn/send/tick - Dashboard: extracted HTML to static files, gossip protocol panel with membership event log, visual fixes, click-to-filter, worker-colored rows Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
121 lines
5.3 KiB
Markdown
121 lines
5.3 KiB
Markdown
# DistributedNode
|
|
|
|
`DistributedNode` is the top-level integration type that composes SWIM
|
|
membership, Kademlia routing, the actor directory, location cache, and
|
|
repair infrastructure into a single public API.
|
|
|
|
## Internal Composition
|
|
|
|
```
|
|
┌─ DistributedNode ────────────────────────────────────────────────────────┐
|
|
│ │
|
|
│ keypair: Keypair ← ed25519 identity + signing │
|
|
│ │
|
|
│ ┌── SWIM ─────────────────┐ ┌── Kademlia ────────────────────────────┐ │
|
|
│ │ SwimNode │ │ RoutingTable (256 k-buckets, K=20) │ │
|
|
│ │ ├ MemberList (CRDT) │ │ DirectoryShard (local entries) │ │
|
|
│ │ ├ SwimProbe (FSM) │ │ RepairQueue (death-triggered) │ │
|
|
│ │ └ DisseminationQueue │ │ RepublishTracker (periodic) │ │
|
|
│ └──────────────────────────┘ └────────────────────────────────────────┘ │
|
|
│ │
|
|
│ cache: LocationCache ← LRU: ActorAddress → NodeId │
|
|
│ tick_count: u64 ← monotonic clock │
|
|
│ │
|
|
└──────────────────────────────────────────────────────────────────────────┘
|
|
```
|
|
|
|
## Node Lifecycle
|
|
|
|
```
|
|
new(config)
|
|
│
|
|
v
|
|
Generate Keypair ──► NodeId = public key bytes
|
|
│
|
|
v
|
|
Initialize subsystems (SwimNode, RoutingTable, DirectoryShard, ...)
|
|
│
|
|
v
|
|
join(seeds) ──► SendJoinRequest to each seed
|
|
│
|
|
v
|
|
handle_join_response(members) ──► populate MemberList + RoutingTable
|
|
│
|
|
v
|
|
┌─────────────┐
|
|
│ tick() loop │ ──► drives SWIM probes, processes membership changes,
|
|
│ (caller) │ runs periodic republish, drains repair queue
|
|
└─────────────┘
|
|
│
|
|
v
|
|
leave() ──► disseminate Dead for self, graceful shutdown
|
|
```
|
|
|
|
See [distribution_minor_flows.svg](distribution_minor_flows.svg) for the
|
|
join handshake, dissemination piggybacking, and membership change cascade.
|
|
|
|
## Actor Registration
|
|
|
|
`register_actor(addr, generation)`:
|
|
1. Sign a `DirectoryEntry` with the node's keypair.
|
|
2. Store in local `DirectoryShard`.
|
|
3. Insert into `LocationCache` (self is the host).
|
|
4. Register with `RepublishTracker` for periodic re-STORE.
|
|
5. Return the signed entry — caller STOREs to `r`-closest nodes.
|
|
|
|
See [actor_resolution.svg](actor_resolution.svg) for the full datapath.
|
|
|
|
## Actor Resolution
|
|
|
|
`resolve_actor(addr)` implements a 3-tier lookup chain:
|
|
1. **LRU cache** — instant, O(1).
|
|
2. **Local directory shard** — checks entries this node holds.
|
|
3. **Kademlia lookup** — returns `NeedsLookup` with closest known nodes;
|
|
caller drives iterative FIND_VALUE + quorum read.
|
|
|
|
On delivery failure, `invalidate_cache(addr)` evicts stale entries.
|
|
|
|
## Membership Change Cascade
|
|
|
|
When SWIM detects a state change, `handle_membership_change(node_id, state)`
|
|
propagates effects through all subsystems:
|
|
|
|
| State | Actions |
|
|
|-------|---------|
|
|
| **Alive** | Insert/update in `RoutingTable` |
|
|
| **Suspect** | Keep in `RoutingTable` (could downprioritize) |
|
|
| **Dead** | Remove from `RoutingTable`, invalidate cache entries for that node, queue affected directory entries in `RepairQueue` |
|
|
|
|
This cascade ensures that a single SWIM death detection triggers routing
|
|
table cleanup, cache invalidation, and directory repair in one tick.
|
|
|
|
## NodeAddr
|
|
|
|
All network addresses in the distribution crate use `NodeAddr` — an extensible
|
|
enum that currently wraps `SocketAddr`:
|
|
|
|
```rust
|
|
pub enum NodeAddr {
|
|
Tcp(SocketAddr),
|
|
// Future: Ws(String), WebRtc(String)
|
|
}
|
|
```
|
|
|
|
This abstraction allows the distribution layer to eventually support WebSocket
|
|
and WebRTC peers natively. The `TcpTransport` extracts the inner `SocketAddr`
|
|
via pattern matching. Adding a new transport variant (e.g., `Ws`) is a
|
|
compiler-guided refactor — every match arm will produce an error until handled.
|
|
|
|
## Where Things Live
|
|
|
|
| Type | File | Role |
|
|
|------|------|------|
|
|
| `DistributedNode` | `node.rs` | Top-level integration facade |
|
|
| `DistributedNodeConfig` | `node.rs` | Node configuration |
|
|
| `ResolveResult` | `node.rs` | 3-tier resolution outcomes |
|
|
| `LocationCache` | `cache.rs` | LRU actor→node cache |
|
|
| `Keypair` | `crypto.rs` | Ed25519 keypair + signing |
|
|
| `NodeId` | `types.rs` | 32-byte node identity |
|
|
| `MemberState` | `types.rs` | Alive / Suspect / Dead |
|
|
| `DirectoryEntry` | `types.rs` | Signed actor→node binding |
|
|
| `NodeRecord` | `types.rs` | Wire-format membership record |
|