swactor/docs/os-design/README.md

# Swactor Distributed OS — Design Overview

## Vision

Swactor is evolving from a local actor runtime into a **distributed operating system** for running long-lived daemons across heterogeneous, churning machines.

**Primary use case**: Wire together personal hardware today; add on-demand spot compute (vast.ai, etc.) tomorrow. Machines pop in and out of the network. The system self-heals.

**Design principles**:

- **Churn is the norm**, not the exception. Every subsystem assumes nodes can disappear at any time.
- **General and flexible**. Minimal assumptions about what a node looks like — feature-gate hardware-specific code.
- **Layered**. The core runtime stays minimal. OS features are opt-in crates. Supervision is user-space, not kernel.
- **Frontend-agnostic**. Operational interfaces (commands, inspection) work identically from CLI, TUI, REST, or future transports.

## Current Capabilities

| OS Concept | What Swactor Has Today |
|---|---|
| Processes | Actor spawn/stop, lifecycle hooks, restartable with factory + max_restarts |
| Scheduling | Worker threads, fairness budget (64 msgs/tick), load-aware placement |
| IPC | Typed send, request/reply, per-actor VecDeque mailboxes |
| Naming | String-keyed `AddressMap` — node-local only |
| Fault tolerance | `catch_unwind` for panics, factory restart, dead actor cleanup |
| Backpressure | Per-actor mailbox capacity, DropNewest/DropOldest overflow |
| Networking | SWIM membership (Lifeguard extensions), TCP transport, bincode wire protocol |
| Directory | Kademlia DHT for `ActorAddress -> NodeId` resolution |
| Monitoring | WorkerStats/RuntimeStats, TUI dashboard, REST `/api/*`, investigate REPL |
| Distribution | Multi-node cluster, 5-node Docker test suite |

## What's Missing (This Design)

| Feature | Document | Priority |
|---|---|---|
| Actor Watching | [01-actor-watching.md](./01-actor-watching.md) | Foundation for everything |
| Cluster Registry | [02-cluster-registry.md](./02-cluster-registry.md) | Actors find each other across nodes |
| Node Capabilities | [03-node-capabilities.md](./03-node-capabilities.md) | Heterogeneous placement |
| Command Interface | [04-command-interface.md](./04-command-interface.md) | Operational control |
| Supervision | [05-supervision.md](./05-supervision.md) | Self-healing (user-space) |

## Crate Structure (After This Work)

```
swactor/                           # core runtime — no network deps
  src/
    actor.rs                       # +watch/unwatch on ContextInner, ExitReason, ActorExited
    worker.rs                      # +WatchRegistry, death notification phase
    delivery.rs                    # (unchanged)
    runtime.rs                     # (unchanged)
    ...
crates/
  capabilities/                    # NEW — hardware detection + placement constraints
    src/lib.rs                     #   NodeCapabilities, CapValue, PlacementConstraint
  command/                         # NEW — frontend-agnostic command dispatch
    src/lib.rs                     #   CommandRouter, CommandHandler, CommandRequest/Response
    src/builtins/                  #   Built-in command handlers
  distribution/                    # SWIM + Kademlia + cluster registry
    src/
      registry.rs                  # NEW — ClusterRegistry, gossip-propagated naming
      node.rs                      # +register_name, resolve_name, capabilities
      messages.rs                  # +WatchRequest, UnwatchRequest, ActorExitedNotify
      ...
  runtime-dashboard/               # TUI + REST — refactored to use command crate
  python/                          # PyO3 bindings
  simulation/                      # Network simulation
  wasm/                            # WASM bindings
```

## Dependency Graph

```
                  ┌──────────────┐
                  │ capabilities │  (standalone — only serde + sysinfo)
                  └──────┬───────┘
                         │ optional
┌─────────┐      ┌──────▼────────┐
│ swactor │◄─────│ distribution  │
│ (core)  │      │ +registry     │
└────┬────┘      └──────┬────────┘
     │                  │
     │           ┌──────▼────────┐
     └──────────►│   command     │
                 └──────┬────────┘
                        │
                 ┌──────▼────────────┐
                 │ runtime-dashboard │
                 │ (TUI + REST)      │
                 └───────────────────┘
```

Key constraints:
- `capabilities` has **zero** dependency on `swactor` — it's a standalone detection library.
- `command` depends on `swactor` (needs `Runtime`, stats types) but NOT on `distribution`.
- `distribution` optionally depends on `capabilities` (for `NodeRecord` labels).
- `runtime-dashboard` depends on both `command` and optionally `distribution`.

## Feature Flags

### Core `swactor`

| Flag | Purpose |
|---|---|
| `getrandom` (default) | Cryptographic RNG for actor addresses |
| `serde` | Serialization for types |
| `tracing` | Structured logging |
| `transport` | Transport-agnostic remote messaging |
| `watching` (new) | Watch API on ContextInner/Ctx |

### `swactor-capabilities`

| Flag | Purpose |
|---|---|
| `detect` (default) | Auto-detect CPU, RAM, hostname via sysinfo |
| `gpu-nvidia` | Detect NVIDIA GPU via nvidia-smi |
| `gpu-vulkan` | Detect GPU via Vulkan API |

### `distribution`

| Flag | Purpose |
|---|---|
| `registry` (new, default) | Cluster-wide gossip-propagated naming |
| `capabilities` (new) | NodeCapabilities on NodeRecord |

## Implementation Order

Each feature is one PR, in dependency order:

```
PR 1: Actor Watching (local only)
  └──► PR 2: Command Interface
        └──► PR 3: Cluster Registry
              └──► PR 4: Node Capabilities
                    └──► PR 5: Remote Watching (cross-node)
                          └──► PR 6: Supervisor library
```

**PR 1 — Actor Watching (local)**
Adds `WatchRegistry`, `ActorExited`, `ExitReason` to core runtime. Testable without any distribution. Foundation for everything else.

**PR 2 — Command Interface**
Extracts investigate.rs into `crates/command/`. Adds write commands (spawn, stop, drain). Immediately useful for operations.

**PR 3 — Cluster Registry**
Gossip-propagated naming in `crates/distribution/src/registry.rs`. Actors can find each other by name across nodes.

**PR 4 — Node Capabilities**
`crates/capabilities/` with auto-detection and placement constraints. Integrates with distribution for capability-aware placement.

**PR 5 — Remote Watching**
Wire protocol for cross-node watches. SWIM Dead triggers `ActorExited { reason: NodeDown }` for all actors on that node.

**PR 6 — Supervisor Library**
User-space supervisor pattern. Composes watching + registry + capabilities to auto-respawn actors after churn.