zacheryasc
d817ad952f
Major feature addition. For full details read `./docs/development_history/DISTRIBUTION.md`
104 lines
4.7 KiB
Markdown
104 lines
4.7 KiB
Markdown
# DistributedNode
|
|
|
|
`DistributedNode` is the top-level integration type that composes SWIM
|
|
membership, Kademlia routing, the actor directory, location cache, and
|
|
repair infrastructure into a single public API.
|
|
|
|
## Internal Composition
|
|
|
|
```
|
|
┌─ DistributedNode ────────────────────────────────────────────────────────┐
|
|
│ │
|
|
│ keypair: Keypair ← ed25519 identity + signing │
|
|
│ │
|
|
│ ┌── SWIM ─────────────────┐ ┌── Kademlia ────────────────────────────┐ │
|
|
│ │ SwimNode │ │ RoutingTable (256 k-buckets, K=20) │ │
|
|
│ │ ├ MemberList (CRDT) │ │ DirectoryShard (local entries) │ │
|
|
│ │ ├ SwimProbe (FSM) │ │ RepairQueue (death-triggered) │ │
|
|
│ │ └ DisseminationQueue │ │ RepublishTracker (periodic) │ │
|
|
│ └──────────────────────────┘ └────────────────────────────────────────┘ │
|
|
│ │
|
|
│ cache: LocationCache ← LRU: ActorAddress → NodeId │
|
|
│ tick_count: u64 ← monotonic clock │
|
|
│ │
|
|
└──────────────────────────────────────────────────────────────────────────┘
|
|
```
|
|
|
|
## Node Lifecycle
|
|
|
|
```
|
|
new(config)
|
|
│
|
|
v
|
|
Generate Keypair ──► NodeId = public key bytes
|
|
│
|
|
v
|
|
Initialize subsystems (SwimNode, RoutingTable, DirectoryShard, ...)
|
|
│
|
|
v
|
|
join(seeds) ──► SendJoinRequest to each seed
|
|
│
|
|
v
|
|
handle_join_response(members) ──► populate MemberList + RoutingTable
|
|
│
|
|
v
|
|
┌─────────────┐
|
|
│ tick() loop │ ──► drives SWIM probes, processes membership changes,
|
|
│ (caller) │ runs periodic republish, drains repair queue
|
|
└─────────────┘
|
|
│
|
|
v
|
|
leave() ──► disseminate Dead for self, graceful shutdown
|
|
```
|
|
|
|
See [distribution_minor_flows.svg](distribution_minor_flows.svg) for the
|
|
join handshake, dissemination piggybacking, and membership change cascade.
|
|
|
|
## Actor Registration
|
|
|
|
`register_actor(addr, generation)`:
|
|
1. Sign a `DirectoryEntry` with the node's keypair.
|
|
2. Store in local `DirectoryShard`.
|
|
3. Insert into `LocationCache` (self is the host).
|
|
4. Register with `RepublishTracker` for periodic re-STORE.
|
|
5. Return the signed entry — caller STOREs to `r`-closest nodes.
|
|
|
|
See [actor_resolution.svg](actor_resolution.svg) for the full datapath.
|
|
|
|
## Actor Resolution
|
|
|
|
`resolve_actor(addr)` implements a 3-tier lookup chain:
|
|
1. **LRU cache** — instant, O(1).
|
|
2. **Local directory shard** — checks entries this node holds.
|
|
3. **Kademlia lookup** — returns `NeedsLookup` with closest known nodes;
|
|
caller drives iterative FIND_VALUE + quorum read.
|
|
|
|
On delivery failure, `invalidate_cache(addr)` evicts stale entries.
|
|
|
|
## Membership Change Cascade
|
|
|
|
When SWIM detects a state change, `handle_membership_change(node_id, state)`
|
|
propagates effects through all subsystems:
|
|
|
|
| State | Actions |
|
|
|-------|---------|
|
|
| **Alive** | Insert/update in `RoutingTable` |
|
|
| **Suspect** | Keep in `RoutingTable` (could downprioritize) |
|
|
| **Dead** | Remove from `RoutingTable`, invalidate cache entries for that node, queue affected directory entries in `RepairQueue` |
|
|
|
|
This cascade ensures that a single SWIM death detection triggers routing
|
|
table cleanup, cache invalidation, and directory repair in one tick.
|
|
|
|
## Where Things Live
|
|
|
|
| Type | File | Role |
|
|
|------|------|------|
|
|
| `DistributedNode` | `node.rs` | Top-level integration facade |
|
|
| `DistributedNodeConfig` | `node.rs` | Node configuration |
|
|
| `ResolveResult` | `node.rs` | 3-tier resolution outcomes |
|
|
| `LocationCache` | `cache.rs` | LRU actor→node cache |
|
|
| `Keypair` | `crypto.rs` | Ed25519 keypair + signing |
|
|
| `NodeId` | `types.rs` | 32-byte node identity |
|
|
| `MemberState` | `types.rs` | Alive / Suspect / Dead |
|
|
| `DirectoryEntry` | `types.rs` | Signed actor→node binding |
|
|
| `NodeRecord` | `types.rs` | Wire-format membership record |
|