swactor/docs/development_history/datastore-auth/SUMMARY.md

# Datastore Auth: Development History

**Branch:** `swactor-auth`
**Base commit:** `3d5a539` (feat: distributed datastore primitives protocol)
**Companion spec:** `DATASTORE_AUTH.md` (root)

---

## What Was Built

An ed25519 authorization layer for the datastore, spanning the full stack from crypto primitives through actor enforcement to CLI/binary wiring. Three commits of protocol work, plus uncommitted binary integration.

The auth system enforces binary access control (authorized or not) at the HTTP API boundary. Internal actors remain auth-unaware. Two auth paths exist: connection-level (iroh QUIC handshake proves NodeId) and per-request signed envelopes (for browser relay and HTTP API). This work implements Path 2 end-to-end.

---

## Commit-by-Commit

### `ebf778f` — fix: cli for datastore works

Brought up the `store_node` and `store_cli` binaries as `[[bin]]` targets with feature-gated dependencies. The node binary spawns the actor runtime, wires up BlobStoreActor/MetadataActor/DatastoreNode, and serves the HTTP API. The CLI binary talks to the node over HTTP with `ureq`. Added `tiny_http` for the API server, `clap` for arg parsing, `ctrlc` for graceful shutdown, and the `runtime-dashboard` integration.

Key files: `Cargo.toml` (features `node`/`cli`), `src/bin/store_node.rs`, `src/bin/store_cli.rs`, `src/api.rs`

### `7c1c2c1` — feat: mvp auth protocol

Core auth implementation:

- **`src/auth.rs`** — `DatastoreAction`, `SignedRequestPayload`, `SignedRequest`, `AccessControlList` (with JSON persistence), `AuthzEngine` (signature verification, timestamp window, nonce replay detection, ACL check), `sign_request()` / `verify_signed_request()` helpers, `DeniedReason` enum.
- **`src/actors/gateway.rs`** — `GatewayActor` wrapping `AuthzEngine`. Handles `Authorize` (pure auth check), `HandleSignedRequest` (auth + dispatch), `CheckConnection` (Path 1), `Grant`/`Revoke` (owner-only ACL mutations), `NonceGcTick`. Translates `DatastoreAction` to `DatastoreNodeMsg` via `action_to_node_msg()`.
- **`src/messages.rs`** — `GatewayMsg` enum, `DatastoreResponse::Denied` variant.
- **`shared-types` crate** — Extracted `ContentHash` into its own crate so both `distribution` and `datastore` can depend on it without cycles.

Tests added:
- `auth_scenario_tests.rs` (12 tests) — owner access, stranger denial, grant/revoke lifecycle, signed request happy path, tampered signature, stale timestamp, replayed nonce, nonce GC, non-owner grant/revoke rejection.
- `acl_persistence_tests.rs` (2 tests) — save/load round-trip, create-on-missing.
- `gateway_tests.rs` (4 tests) — connection allow/deny, signed request flow-through, unauthorized signed request denial.

### `6366c6b` — fix: adjust auth protocol to datastore protocol

Aligned the auth types with the content-hash-first datastore protocol:
- `DatastoreAction::Put` carries `content_hash`, `size_bytes`, and `tags` (not raw data).
- `DatastoreAction::Get`/`Delete` use `content_hash` (not name).
- `DatastoreAction::List` uses `name_filter`.
- `GatewayActor::action_to_node_msg` maps actions to the existing `DatastoreNodeMsg` variants.
- Wired `check_auth()` into the HTTP API handlers (put, get, data, delete, list) — reads `X-Signed-Request` header, sends `GatewayMsg::Authorize` to the gateway actor, denies with 401/403/504 on failure.
- `handle_status` intentionally left ungated.

### Uncommitted — Wire auth into node & CLI binaries

The auth engine and HTTP gate existed but neither binary used them. This change connects them:

**`store_node.rs`** — `--auth` and `--auth-dir <PATH>` flags:
- When `--auth`: loads or generates owner keypair from `<auth-dir>/owner.key.json` (JSON with hex-encoded keys, version field, public_key for inspection, ISO-8601 created_at).
- Owner keypair's public key becomes the `NodeId` (deterministic identity across restarts).
- Loads/creates `<auth-dir>/acl.json` with owner as sole authorized key.
- Spawns `GatewayActor` with the `AuthzEngine` and passes `Some(gateway_addr)` to `start_api_server`.
- Sends `GatewayMsg::NonceGcTick` on the same cadence as the metadata GC tick.
- Without `--auth`, behavior is unchanged (random NodeId, no gateway, `None` passed to API).

**`store_cli.rs`** — `--key <PATH>` flag:
- Loads keypair from the same JSON key file format.
- Each command (put/get/delete/list) builds the appropriate `DatastoreAction`, creates a `SignedRequestPayload` with current timestamp + `getrandom` nonce, signs it, and sends the JSON as `X-Signed-Request` header.
- `status` command never signs (always open per design).
- Without `--key`, no header is sent (backward compatible with non-auth nodes).

**`Cargo.toml`** — Added `getrandom = { version = "0.2", optional = true }` to the `cli` feature.

---

## Architecture

```
                  ┌──────────────────────────────────┐
                  │         HTTP API (api.rs)         │
                  │                                   │
                  │  /api/status ──► handle_status    │  (no auth)
                  │  /api/put    ──► check_auth ──►   │
                  │  /api/get    ──► check_auth ──►   │
                  │  /api/data   ──► check_auth ──►   │  X-Signed-Request
                  │  /api/delete ──► check_auth ──►   │  header required
                  │  /api/list   ──► check_auth ──►   │  when gateway_addr
                  │                                   │  is Some
                  └───────────┬───────────────────────┘
                              │
                    GatewayMsg::Authorize
                              │
                  ┌───────────▼───────────┐
                  │     GatewayActor      │
                  │                       │
                  │  1. verify signature  │
                  │  2. check timestamp   │
                  │  3. check nonce       │
                  │  4. check ACL         │
                  │                       │
                  │  DatastoreResponse::  │
                  │    Bool(true) or      │
                  │    Denied { reason }  │
                  └───────────────────────┘

                  ┌───────────────────────┐
                  │    CLI (store_cli)    │
                  │                       │
                  │  --key owner.key.json │
                  │                       │
                  │  sign_action():       │
                  │    timestamp + nonce  │
                  │    + DatastoreAction  │
                  │    → ed25519 sign     │
                  │    → JSON header      │
                  └───────────────────────┘
```

---

## Key File Format

`owner.key.json` / any client `key.json`:

```json
{
  "version": 1,
  "secret_key": "...64 hex chars (32 bytes)...",
  "public_key": "...64 hex chars (32 bytes)...",
  "created_at": "2026-02-15T12:00:00Z"
}
```

Shared between node and CLI. The node generates it on first `--auth` run; the CLI reads it with `--key`.

---

## Test Summary

| Test File | Count | What |
|-----------|-------|------|
| `auth_scenario_tests.rs` | 12 | AuthzEngine: signing, verification, timestamp, nonce, ACL, grant/revoke |
| `acl_persistence_tests.rs` | 2 | ACL JSON round-trip, create-on-missing |
| `gateway_tests.rs` | 4 | GatewayActor: connection check, signed request flow, denial |
| `http_auth_integration.rs` | 1 | Full HTTP stack: owner allowed, stranger gets 401/403 |
| **Auth total** | **19** | |
| **Overall total** | **107** | (93 pre-auth + 14 new auth + inherited datastore tests) |

---

## Design Decisions

1. **Key file format is JSON with hex encoding** — human-readable, inspectable with `cat`, foundation for future keystore without needing a binary format parser.

2. **Status endpoint stays open** — `/api/status` is not gated even when auth is enabled. This lets monitoring tools and health checks work without credentials.

3. **Node identity = owner keypair's public key** — when `--auth` is enabled, the keypair's `node_id()` replaces random generation. The node has a stable, cryptographic identity across restarts.

4. **Nonce source is `getrandom`** — cryptographically secure 16-byte random nonces. Already a transitive dependency via `ed25519-dalek` / `rand_core`.

5. **Backward compatible** — without `--auth` (node) or `--key` (CLI), everything works exactly as before. No breaking changes.

6. **Auth is opt-in per binary** — the auth engine, ACL, and gateway actor are always compiled (they're in the lib), but only activated when the binary flags are set. This keeps the default experience frictionless.

---

## Files Changed (Full Branch)

| File | What |
|------|------|
| `crates/shared-types/` | New crate — extracted `ContentHash` to break dependency cycles |
| `crates/datastore/src/auth.rs` | Auth engine, ACL, signing, verification |
| `crates/datastore/src/actors/gateway.rs` | GatewayActor — auth enforcement point |
| `crates/datastore/src/messages.rs` | `GatewayMsg`, `DatastoreResponse::Denied` |
| `crates/datastore/src/api.rs` | HTTP API with `check_auth` gate |
| `crates/datastore/src/bin/store_node.rs` | `--auth`, `--auth-dir`, keypair management, gateway spawn |
| `crates/datastore/src/bin/store_cli.rs` | `--key`, per-request signing |
| `crates/datastore/Cargo.toml` | `getrandom` dep, feature updates |
| `DATASTORE_AUTH.md` | Auth specification document |
| `tests/auth_scenario_tests.rs` | 12 auth engine tests |
| `tests/acl_persistence_tests.rs` | 2 ACL persistence tests |
| `tests/gateway_tests.rs` | 4 gateway actor tests |
| `tests/http_auth_integration.rs` | 1 full-stack HTTP auth test |