zacheryasc/swactor
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

swactor-auth #43

Merged
zacheryasc merged 6 commits from swactor-auth into master 2026-02-16 15:39:28 +00:00
Conversation 0 Commits 6 Files changed 38 +5409 -172
38 changed files with 5409 additions and 172 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

5
.gitignore vendored
View file

@ -13,4 +13,7 @@ docs/architecture.dot
docs/architecture.html
# Simulation traces
crates/simulation/traces
crates/simulation/traces
# xtask personal config
xtask/config.toml

24
Cargo.lock generated
View file

@ -1129,6 +1129,7 @@ dependencies = [
"rand_core 0.6.4",
"serde",
"serde_json",
"shared-types",
"swactor",
"tokio",
]
@ -4094,6 +4095,14 @@ dependencies = [
"lazy_static",
]
[[package]]
name = "shared-types"
version = "0.1.0"
dependencies = [
"blake3",
"serde",
]
[[package]]
name = "shlex"
version = "1.3.0"
@ -4375,6 +4384,13 @@ dependencies = [
"wat",
]
[[package]]
name = "swactor-crypto-wasm"
version = "0.1.0"
dependencies = [
"ed25519-dalek 2.2.0",
]
[[package]]
name = "swactor-datastore"
version = "0.1.0"
@ -4383,10 +4399,12 @@ dependencies = [
"clap",
"ctrlc",
"distribution",
"getrandom 0.2.17",
"proptest",
"runtime-dashboard",
"serde",
"serde_json",
"shared-types",
"swactor",
"swactor-std",
"tempfile",
@ -6235,6 +6253,12 @@ dependencies = [
[[package]]
name = "xtask"
version = "0.1.0"
dependencies = [
"clap",
"libc",
"serde",
"toml",
]
[[package]]
name = "yoke"

2
Cargo.toml
View file

@ -1,5 +1,5 @@
[workspace]
members = [".", "crates/python", "crates/wasm", "crates/bin-runner", "crates/simulation", "crates/runtime-dashboard", "crates/distribution", "crates/std", "crates/datastore", "tests/docker", "xtask"]
members = [".", "crates/python", "crates/wasm", "crates/bin-runner", "crates/simulation", "crates/runtime-dashboard", "crates/distribution", "crates/std", "crates/datastore", "crates/shared-types", "crates/crypto-wasm", "tests/docker", "xtask"]
exclude = ["tools/depgraph"]
[package]

305
DATASTORE_AUTH.md Normal file
View file

@ -0,0 +1,305 @@
# Swactor Datastore Auth Specification
**Version:** 0.1.0 (MVP)
**Status:** Draft
**Companion to:** `DATASTORE_PROTOCOL.md`
## 1. Overview
This document specifies the authorization layer for the Swactor Datastore. It defines how access is controlled for external clients connecting to a datastore node.
### Principles
- **Cryptographic identity** — keys, not passwords. Every participant is identified by an ed25519 public key (`NodeId`).
- **Binary access** — a client is either authorized or not. No permission tiers for MVP.
- **Owner-only administration** — only the datastore owner can grant or revoke access.
- **Transport-layer authentication** — iroh's QUIC handshake cryptographically proves a peer's `NodeId`. This spec builds authorization on top of that.
### Non-Goals (MVP)
- Per-path permission scoping.
- Permission tiers (read-only, read-write, admin).
- Capability tokens or time-limited delegated access.
- Multi-level delegation chains.
## 2. Trust Boundaries
```
┌─────────────────────────────────────────────┐
│ Cluster (SWIM mesh) │
│ │
│ Node A ◄──────────────► Node B │
│ implicitly trusted │
│ (no auth checks) │
└──────────────────┬──────────────────────────┘
│
│ auth boundary
│
┌──────────▼──────────┐
│ External Clients │
│ │
│ CLI tool │
│ Browser user │
└─────────────────────┘
```
- **Cluster-internal** (node-to-node via SWIM): implicitly trusted. Nodes that are members of the SWIM cluster communicate freely — no per-request auth checks.
- **External clients** (CLI, browser): must be authorized. Every external request is checked against the Access Control List before being dispatched to the actor system.
## 3. Identity Model
The auth layer reuses the existing ed25519 identity model from the distribution layer:
- Every client (CLI tool, browser user, node) has an ed25519 keypair.
- Identity is the 32-byte public key, represented as `NodeId`.
- The same `NodeId` type from `distribution::types` is used throughout.
There is no separate "user" concept — a keypair *is* an identity.
## 4. Access Control List
### 4.1 Structure
```
AccessControlList {
owner: NodeId, // The datastore owner's public key
authorized_keys: Set<NodeId>, // Explicitly authorized client keys
}
```
- The **owner** always has full access (implicit; never needs to be in `authorized_keys`).
- An empty `authorized_keys` set means only the owner can access the datastore.
### 4.2 Persistence
The ACL is persisted as a JSON file alongside the datastore's `storage_path`:
```
{storage_path}/
├── chunks/
├── manifests/
└── acl.json # AccessControlList
```
### 4.3 Mutations
| Operation | Signature | Who |
|-----------|-----------|-----|
| Grant access | `grant(key: NodeId)` | Owner only |
| Revoke access | `revoke(key: NodeId)` | Owner only |
- `grant` adds a `NodeId` to `authorized_keys`. Idempotent — granting an already-authorized key is a no-op.
- `revoke` removes a `NodeId` from `authorized_keys`. Idempotent — revoking a non-existent key is a no-op.
- Revoking the owner is a no-op (the owner's implicit access cannot be removed).
- Both operations persist the updated ACL to disk immediately.
## 5. Auth Path 1 — Direct iroh Connection
For clients that connect directly to the datastore node over iroh (QUIC):
```
Client (ed25519 keypair) Datastore Node
│ │
│──── iroh QUIC handshake ──────────>│
│ (proves client's NodeId) │
│ │
│ check NodeId
│ against ACL
│ │
│<─── accept / reject ──────────────│
│ │
│ (if accepted, all ops on │
│ this connection are allowed) │
```
1. The iroh QUIC handshake cryptographically proves the peer's `NodeId` (ed25519 public key).
2. On connection establishment, the node checks the peer's `NodeId` against the ACL.
3. If authorized → connection accepted. All operations on that connection are allowed with no per-message overhead.
4. If not authorized → connection rejected immediately.
This is the preferred auth path — zero overhead after the initial handshake.
## 6. Auth Path 2 — Signed Requests (Browser Relay)
For browser users who cannot establish direct iroh connections (e.g., because the browser communicates via a website backend that relays requests):
### 6.1 Threat Model
The website backend acts as an **untrusted relay**. It forwards requests between the browser and the datastore node but never sees private keys. The relay cannot forge, modify, or replay requests.
### 6.2 Signed Envelope
Each request is wrapped in a signed envelope:
```
SignedRequest {
payload: SignedRequestPayload, // The request details
public_key: NodeId, // Client's public key
signature: Signature, // ed25519 signature over serialized payload
}
SignedRequestPayload {
action: DatastoreAction, // What the client wants to do
timestamp: u64, // Unix timestamp (seconds)
nonce: [u8; 16], // 16 random bytes
}
DatastoreAction = enum {
Put { name, content_hash, size_bytes, tags },
Get { content_hash },
Delete { content_hash },
List { name_filter },
}
```
### 6.3 Verification Steps
The datastore node verifies a signed request in strict order:
1. **Signature validity** — verify the ed25519 signature over the canonical serialization of `SignedRequestPayload` using the provided `public_key`.
2. **Timestamp freshness** — reject if `|now - payload.timestamp| > 300` seconds.
3. **Nonce uniqueness** — reject if `payload.nonce` has been seen before within the time window.
4. **ACL check** — reject if `public_key` is not in the ACL.
If any step fails, the request is denied with the corresponding `DeniedReason`.
### 6.4 Put Payload Note
`DatastoreAction::Put` references a `content_hash` rather than embedding raw file data. The bulk data is uploaded separately, and its integrity is guaranteed by blake3 content addressing. The signed envelope authorizes the *operation*, not the data transfer.
## 7. Replay Protection
### 7.1 Timestamp Window
- Requests must have a `timestamp` within ±300 seconds of the node's wall clock.
- This bounds the maximum clock drift between client and server.
- Requests outside this window are rejected with `DeniedReason::RequestExpired`.
### 7.2 Nonce
- Each request includes a 16-byte random nonce.
- The node maintains a set of recently seen nonces.
- Duplicate nonces within the time window are rejected with `DeniedReason::ReplayDetected`.
### 7.3 Nonce Garbage Collection
- Nonces are stored alongside their timestamps.
- When a nonce's timestamp falls outside the ±300 second window, it is eligible for GC.
- GC runs periodically (piggy-backed on request processing or a background sweep).
## 8. Enforcement Point
Auth is enforced at the **edge** of the actor system — between external clients and the internal actors:
```
External Client
│
▼
┌─────────────┐
│ Auth Gate │◄── ACL check happens here
└──────┬──────┘
│
▼
┌──────────────┐ ┌─────────────────┐ ┌────────────────┐
│ MetadataActor│◄──►│ BlobStoreActor │ │ TransferActor │
│ │ │ │ │ │
│ (auth- │ │ (auth- │ │ (auth- │
│ unaware) │ │ unaware) │ │ unaware) │
└──────────────┘ └─────────────────┘ └────────────────┘
```
### 8.1 Direct iroh Connections
- Auth check at connection acceptance time.
- Once accepted, the connection is fully trusted for all operations.
- No per-message overhead.
### 8.2 Signed Requests (Browser Relay)
- A `GatewayActor` receives signed request envelopes.
- The GatewayActor verifies the envelope (signature, timestamp, nonce, ACL).
- If valid, the GatewayActor dispatches the inner action to the `MetadataActor`.
- If invalid, the GatewayActor returns the denial reason to the relay.
### 8.3 Internal Actors
`MetadataActor`, `BlobStoreActor`, and `TransferActor` remain **auth-unaware**. They process messages from any source within the actor system. The auth boundary is strictly external.
## 9. Key Management
### 9.1 Key Generation
- Uses `ed25519_dalek` keypairs (same as node identity).
- CLI: `swactor-store auth keygen` generates a new keypair and prints both the secret key (for the client to store) and the public key (to share with the owner).
- Browser: keypair generated client-side using WebCrypto Ed25519 or wasm-compiled ed25519. The private key never leaves the browser.
### 9.2 Grant Flow
```
1. Client generates an ed25519 keypair.
2. Client shares their public key with the datastore owner (out-of-band).
3. Owner runs: swactor-store auth grant <pubkey>
4. Client can now access the datastore.
```
The out-of-band exchange is intentional — it keeps the trust model simple. The owner explicitly decides who gets access.
### 9.3 Revocation
```
1. Owner runs: swactor-store auth revoke <pubkey>
2. Client's access is immediately revoked.
3. Existing direct iroh connections from that client remain open until disconnected.
4. Signed requests from the revoked key are rejected immediately.
```
Note: revoking a key does not forcibly disconnect an active iroh session. The revocation takes effect on the next connection attempt. For immediate disconnection, the owner should also restart the node or implement connection tracking (future extension).
## 10. CLI Extensions
The following subcommands are added under `swactor-store auth`:
```
swactor-store auth keygen
Generate a new ed25519 keypair.
Prints the public key (hex) and secret key (hex) to stdout.
swactor-store auth grant <pubkey>
Add a public key to the ACL's authorized_keys set.
Requires running on the owner's node.
swactor-store auth revoke <pubkey>
Remove a public key from the ACL's authorized_keys set.
Requires running on the owner's node.
swactor-store auth list
Show all authorized keys (including the owner).
swactor-store auth whoami
Show this node's public key (NodeId).
```
## 11. Integration with Datastore Protocol
Each protocol flow from `DATASTORE_PROTOCOL.md` §6 has a clear auth integration point:
| Protocol Flow | Auth Path 1 (Direct) | Auth Path 2 (Signed Request) |
|---------------|----------------------|------------------------------|
| §6.1 PUT | Connection-level ACL check | `SignedRequest { action: Put { name, content_hash, size_bytes, tags }, .. }` |
| §6.2 GET (Local) | Connection-level ACL check | `SignedRequest { action: Get { content_hash }, .. }` |
| §6.3 GET (Remote) | Connection-level ACL check | `SignedRequest { action: Get { content_hash }, .. }` → node handles remote fetch internally |
| §6.4 DELETE | Connection-level ACL check | `SignedRequest { action: Delete { content_hash }, .. }` |
| §6.5 LIST (Local) | Connection-level ACL check | `SignedRequest { action: List { name_filter }, .. }` |
| §6.6 LIST (Swarm-Wide) | Connection-level ACL check | `SignedRequest { action: List { name_filter }, .. }` → node handles fan-out internally |
In all cases, auth is enforced *before* the request reaches the actor system. Internal inter-node communication (DHT replication, chunk transfers between cluster members) is not subject to auth checks.
## 12. Future Extensions
These are explicitly **out of scope** for MVP but inform the design:
- **Per-path permission scoping** — restrict a key to specific path prefixes (e.g., read-only access to `photos/`).
- **Permission tiers** — read-only, read-write, admin roles.
- **Capability tokens** — time-limited, scope-limited bearer tokens for delegated access without sharing long-lived keys.
- **Multi-level delegation** — allow authorized users to grant limited access to others.
- **Connection tracking** — forcibly disconnect revoked keys from active iroh sessions.

10
crates/crypto-wasm/Cargo.toml Normal file
View file

@ -0,0 +1,10 @@
[package]
name = "swactor-crypto-wasm"
version = "0.1.0"
edition = "2021"
[lib]
crate-type = ["cdylib"]
[dependencies]
ed25519-dalek = { version = "2", default-features = false }

41
crates/crypto-wasm/src/lib.rs Normal file
View file

@ -0,0 +1,41 @@
#![no_std]
use core::ptr::addr_of_mut;
use ed25519_dalek::{SigningKey, Signer};
static mut BUF: [u8; 8192] = [0u8; 8192];
#[no_mangle]
pub extern "C" fn buffer_ptr() -> *const u8 {
addr_of_mut!(BUF).cast()
}
/// Read seed from BUF[0..32], write public key to BUF[32..64]
#[no_mangle]
pub extern "C" fn get_public_key() {
unsafe {
let buf = &mut *addr_of_mut!(BUF);
let seed: [u8; 32] = buf[0..32].try_into().unwrap_unchecked();
let sk = SigningKey::from_bytes(&seed);
buf[32..64].copy_from_slice(sk.verifying_key().as_bytes());
}
}
/// Read seed from BUF[0..32], message from BUF[128..128+msg_len].
/// Write 64-byte signature to BUF[64..128].
#[no_mangle]
pub extern "C" fn ed25519_sign(msg_len: usize) {
unsafe {
let buf = &mut *addr_of_mut!(BUF);
let seed: [u8; 32] = buf[0..32].try_into().unwrap_unchecked();
let msg = &buf[128..128 + msg_len];
let sk = SigningKey::from_bytes(&seed);
let sig = sk.sign(msg);
buf[64..128].copy_from_slice(&sig.to_bytes());
}
}
#[panic_handler]
fn panic(_: &core::panic::PanicInfo) -> ! {
core::arch::wasm32::unreachable()
}

4
crates/datastore/Cargo.toml
View file

@ -6,12 +6,14 @@ edition = "2024"
[dependencies]
swactor = { path = "../..", features = ["serde", "transport"] }
distribution = { path = "../distribution" }
shared-types = { path = "../shared-types" }
serde = { version = "1", features = ["derive"] }
serde_json = "1"
blake3 = "1"
tiny_http = { version = "0.12", optional = true }
clap = { version = "4", features = ["derive"], optional = true }
ureq = { version = "2", features = ["json"], optional = true }
getrandom = { version = "0.2", optional = true }
ctrlc = { version = "3", optional = true }
runtime-dashboard = { path = "../runtime-dashboard", optional = true }
toml = { version = "0.8", optional = true }
@ -28,7 +30,7 @@ runtime-dashboard = { path = "../runtime-dashboard" }
[features]
node = ["dep:tiny_http", "dep:clap", "dep:ctrlc", "dep:runtime-dashboard", "dep:toml"]
cli = ["dep:clap", "dep:ureq"]
cli = ["dep:clap", "dep:ureq", "dep:getrandom"]
[[bin]]
name = "swactor-store-node"

51
crates/datastore/src/actors/blob_store.rs
View file

@ -140,6 +140,54 @@ impl BlobStoreActor {
}
}
}
fn handle_write_entry(&mut self, entry: crate::types::ObjectEntry) {
if let Err(e) = self.backend.write_entry(&entry) {
eprintln!("warning: write entry failed: {e}");
}
}
fn handle_delete_entry(&mut self, hash: ContentHash) {
if let Err(e) = self.backend.delete_entry(&hash) {
eprintln!("warning: delete entry failed: {e}");
}
}
fn handle_load_all(&self, ctx: &Ctx, reply_to: swactor::actor::ActorAddress) {
match self.backend.list_entries() {
Ok(entries) => {
let mut pairs = Vec::with_capacity(entries.len());
for entry in entries {
match self.backend.read_manifest(&entry.content_hash) {
Ok(Some(manifest)) => {
pairs.push((entry, manifest));
}
Ok(None) => {
eprintln!(
"warning: entry {} has no manifest, skipping",
entry.content_hash
);
}
Err(e) => {
eprintln!(
"warning: failed to read manifest for {}: {e}",
entry.content_hash
);
}
}
}
let _ = ctx.send(reply_to, DatastoreResponse::LoadedAll { entries: pairs });
}
Err(e) => {
let _ = ctx.send(
reply_to,
DatastoreResponse::Error {
reason: format!("list entries failed: {e}"),
},
);
}
}
}
}
impl ActorInterface for BlobStoreActor {
@ -173,6 +221,9 @@ impl ActorInterface for BlobStoreActor {
BlobStoreMsg::ReadManifest { hash, reply_to } => {
self.handle_read_manifest(ctx, hash, reply_to)
}
BlobStoreMsg::WriteEntry { entry } => self.handle_write_entry(entry),
BlobStoreMsg::DeleteEntry { hash } => self.handle_delete_entry(hash),
BlobStoreMsg::LoadAll { reply_to } => self.handle_load_all(ctx, reply_to),
}
}
}

264
crates/datastore/src/actors/gateway.rs Normal file
View file

@ -0,0 +1,264 @@
//! GatewayActor — auth enforcement point for the datastore.
//!
//! Sits in front of the `DatastoreNode` coordinator. All external requests
//! pass through the gateway, which checks authorization before forwarding
//! to the internal actors.
//!
//! ```text
//! External Client → GatewayActor → DatastoreNode → MetadataActor/BlobStoreActor
//! (auth check) (dispatch) (auth-unaware)
//! ```
use std::collections::HashMap;
use std::path::PathBuf;
use std::time::{SystemTime, UNIX_EPOCH};
use swactor::actor::{ActorAddress, ActorInterface, Ctx};
use crate::auth::{AccessRequestInfo, AuthzEngine, AuthzResult, DatastoreAction, DeniedReason};
use crate::messages::{DatastoreNodeMsg, DatastoreResponse, GatewayMsg};
use distribution::types::NodeId;
/// The auth gateway actor wrapping an `AuthzEngine`.
pub struct GatewayActor {
engine: AuthzEngine,
datastore_node: ActorAddress,
acl_path: Option<PathBuf>,
pending_requests: HashMap<NodeId, AccessRequestInfo>,
}
impl GatewayActor {
pub fn new(
engine: AuthzEngine,
datastore_node: ActorAddress,
acl_path: Option<PathBuf>,
) -> Self {
Self {
engine,
datastore_node,
acl_path,
pending_requests: HashMap::new(),
}
}
fn now_secs() -> u64 {
SystemTime::now()
.duration_since(UNIX_EPOCH)
.unwrap_or_default()
.as_secs()
}
fn persist_acl(&self) {
if let Some(ref path) = self.acl_path {
let _ = self.engine.acl.save(path);
}
}
fn handle_authorize(&mut self, ctx: &Ctx, request: crate::auth::SignedRequest, reply_to: ActorAddress) {
let now = Self::now_secs();
match self.engine.check_signed_request(&request, now) {
AuthzResult::Allowed => {
let _ = ctx.send(reply_to, DatastoreResponse::Bool(true));
}
AuthzResult::Denied(reason) => {
let _ = ctx.send(reply_to, DatastoreResponse::Denied { reason });
}
}
}
fn handle_signed_request(&mut self, ctx: &Ctx, request: crate::auth::SignedRequest, reply_to: ActorAddress) {
let now = Self::now_secs();
match self.engine.check_signed_request(&request, now) {
AuthzResult::Allowed => {
let msg = action_to_node_msg(request.payload.action, reply_to);
let _ = ctx.send(self.datastore_node, msg);
}
AuthzResult::Denied(reason) => {
let _ = ctx.send(reply_to, DatastoreResponse::Denied { reason });
}
}
}
fn handle_check_connection(&self, ctx: &Ctx, node_id: NodeId, reply_to: ActorAddress) {
match self.engine.check_node(&node_id) {
AuthzResult::Allowed => {
let _ = ctx.send(reply_to, DatastoreResponse::Bool(true));
}
AuthzResult::Denied(reason) => {
let _ = ctx.send(reply_to, DatastoreResponse::Denied { reason });
}
}
}
fn handle_grant(&mut self, ctx: &Ctx, requester: NodeId, key: NodeId, label: Option<String>, reply_to: ActorAddress) {
// If the key has a pending request, use its name as the label (unless an explicit label was provided)
let resolved_label = label.or_else(|| {
self.pending_requests.remove(&key).map(|req| req.name)
});
match self.engine.grant(&requester, key, resolved_label) {
Ok(()) => {
self.persist_acl();
let _ = ctx.send(reply_to, DatastoreResponse::Bool(true));
}
Err(reason) => {
let _ = ctx.send(reply_to, DatastoreResponse::Denied { reason });
}
}
}
fn handle_revoke(&mut self, ctx: &Ctx, requester: NodeId, key: NodeId, reply_to: ActorAddress) {
match self.engine.revoke(&requester, key) {
Ok(()) => {
self.persist_acl();
let _ = ctx.send(reply_to, DatastoreResponse::Bool(true));
}
Err(reason) => {
let _ = ctx.send(reply_to, DatastoreResponse::Denied { reason });
}
}
}
fn handle_verify_signature(&mut self, ctx: &Ctx, request: crate::auth::SignedRequest, reply_to: ActorAddress) {
let now = Self::now_secs();
match self.engine.check_signature_only(&request, now) {
AuthzResult::Allowed => {
let _ = ctx.send(reply_to, DatastoreResponse::Bool(true));
}
AuthzResult::Denied(reason) => {
let _ = ctx.send(reply_to, DatastoreResponse::Denied { reason });
}
}
}
fn handle_submit_access_request(&mut self, ctx: &Ctx, key: NodeId, name: String, message: String, reply_to: ActorAddress) {
let info = AccessRequestInfo {
key,
name,
message,
requested_at: Self::now_secs(),
};
self.pending_requests.insert(key, info);
let _ = ctx.send(reply_to, DatastoreResponse::Bool(true));
}
fn handle_list_access_requests(&self, ctx: &Ctx, requester: NodeId, reply_to: ActorAddress) {
if requester != self.engine.acl.owner {
let _ = ctx.send(reply_to, DatastoreResponse::Denied { reason: DeniedReason::NotAuthorized });
return;
}
let requests: Vec<AccessRequestInfo> = self.pending_requests.values().cloned().collect();
let _ = ctx.send(reply_to, DatastoreResponse::AccessRequests { requests });
}
fn handle_deny_access_request(&mut self, ctx: &Ctx, requester: NodeId, key: NodeId, reply_to: ActorAddress) {
if requester != self.engine.acl.owner {
let _ = ctx.send(reply_to, DatastoreResponse::Denied { reason: DeniedReason::NotAuthorized });
return;
}
self.pending_requests.remove(&key);
let _ = ctx.send(reply_to, DatastoreResponse::Bool(true));
}
fn handle_list_authorized_keys(&self, ctx: &Ctx, requester: NodeId, reply_to: ActorAddress) {
if requester != self.engine.acl.owner {
let _ = ctx.send(reply_to, DatastoreResponse::Denied { reason: DeniedReason::NotAuthorized });
return;
}
let keys = self.engine.authorized_key_list();
let _ = ctx.send(reply_to, DatastoreResponse::AuthorizedKeys { keys });
}
}
impl ActorInterface for GatewayActor {
type Incoming = GatewayMsg;
type Response = DatastoreResponse;
fn handle(&mut self, ctx: &Ctx, msg: GatewayMsg) {
match msg {
GatewayMsg::HandleSignedRequest { request, reply_to } => {
self.handle_signed_request(ctx, request, reply_to);
}
GatewayMsg::CheckConnection { node_id, reply_to } => {
self.handle_check_connection(ctx, node_id, reply_to);
}
GatewayMsg::Grant {
requester,
key,
label,
reply_to,
} => {
self.handle_grant(ctx, requester, key, label, reply_to);
}
GatewayMsg::Revoke {
requester,
key,
reply_to,
} => {
self.handle_revoke(ctx, requester, key, reply_to);
}
GatewayMsg::Authorize { request, reply_to } => {
self.handle_authorize(ctx, request, reply_to);
}
GatewayMsg::VerifySignature { request, reply_to } => {
self.handle_verify_signature(ctx, request, reply_to);
}
GatewayMsg::SubmitAccessRequest { key, name, message, reply_to } => {
self.handle_submit_access_request(ctx, key, name, message, reply_to);
}
GatewayMsg::ListAccessRequests { requester, reply_to } => {
self.handle_list_access_requests(ctx, requester, reply_to);
}
GatewayMsg::DenyAccessRequest { requester, key, reply_to } => {
self.handle_deny_access_request(ctx, requester, key, reply_to);
}
GatewayMsg::ListAuthorizedKeys { requester, reply_to } => {
self.handle_list_authorized_keys(ctx, requester, reply_to);
}
GatewayMsg::NonceGcTick => {
self.engine.gc_nonces(Self::now_secs());
}
}
}
}
/// Translate a `DatastoreAction` into the corresponding `DatastoreNodeMsg`.
fn action_to_node_msg(action: DatastoreAction, reply_to: ActorAddress) -> DatastoreNodeMsg {
match action {
DatastoreAction::Get { content_hash } => DatastoreNodeMsg::Get {
content_hash,
reply_to,
},
DatastoreAction::Delete { content_hash } => DatastoreNodeMsg::Delete {
content_hash,
reply_to,
},
DatastoreAction::List { name_filter } => DatastoreNodeMsg::List {
name_filter,
all: false,
reply_to,
},
DatastoreAction::Put {
name,
content_hash: _,
size_bytes: _,
tags,
} => {
// Put via signed request is an authorization of the operation.
// The actual data upload happens separately. We forward as a
// zero-data Put — the DatastoreNode will handle the metadata.
// In the full flow, the data is uploaded separately and the
// signed request only authorizes it.
DatastoreNodeMsg::Put {
data: Vec::new(),
name,
tags,
reply_to,
}
}
DatastoreAction::Access => {
// Access is a lightweight identity proof — no content operation.
// Forward as Status to return a valid response to the caller.
DatastoreNodeMsg::Status { reply_to }
}
}
}

26
crates/datastore/src/actors/metadata.rs
View file

@ -158,6 +158,11 @@ impl MetadataActor {
entry.node_id = self.node_id;
self.entries.insert(content_hash, entry.clone());
// Persist entry to disk via BlobStoreActor.
if let Some(addr) = self.blob_store_addr {
let _ = ctx.send(addr, BlobStoreMsg::WriteEntry { entry: entry.clone() });
}
// Enqueue for DHT dissemination (include manifest for peer replication).
self.enqueue(entry, Some(manifest), 3);
@ -196,6 +201,10 @@ impl MetadataActor {
fn handle_delete_object(&mut self, ctx: &Ctx, content_hash: ContentHash, reply_to: ActorAddress) {
if self.entries.remove(&content_hash).is_some() {
self.manifests.remove(&content_hash);
// Delete persisted entry from disk.
if let Some(addr) = self.blob_store_addr {
let _ = ctx.send(addr, BlobStoreMsg::DeleteEntry { hash: content_hash });
}
let _ = ctx.send(reply_to, DatastoreResponse::DeleteOk { content_hash });
} else {
let _ = ctx.send(reply_to, DatastoreResponse::NotFound);
@ -287,7 +296,7 @@ impl MetadataActor {
}
}
fn handle_store_object(&mut self, entry: ObjectEntry, manifest: Option<ObjectManifest>) {
fn handle_store_object(&mut self, ctx: &Ctx, entry: ObjectEntry, manifest: Option<ObjectManifest>) {
// Insert if absent — content-addressed entries don't conflict.
let content_hash = entry.content_hash;
if !self.entries.contains_key(&content_hash) {
@ -295,9 +304,21 @@ impl MetadataActor {
self.manifests.insert(content_hash, m.clone());
}
self.entries.insert(content_hash, entry.clone());
// Persist entry to disk via BlobStoreActor.
if let Some(addr) = self.blob_store_addr {
let _ = ctx.send(addr, BlobStoreMsg::WriteEntry { entry: entry.clone() });
}
self.enqueue(entry, manifest, 3);
}
}
fn handle_bulk_load(&mut self, entries: Vec<(ObjectEntry, ObjectManifest)>) {
for (entry, manifest) in entries {
let hash = entry.content_hash;
self.entries.insert(hash, entry);
self.manifests.insert(hash, manifest);
}
}
}
impl ActorInterface for MetadataActor {
@ -329,11 +350,12 @@ impl ActorInterface for MetadataActor {
reply_to,
} => self.handle_find_object(ctx, from, content_hash, reply_to),
MetadataMsg::HandleStoreObject { entry, manifest } => {
self.handle_store_object(entry, manifest)
self.handle_store_object(ctx, entry, manifest)
}
MetadataMsg::SetPeers { peers } => self.handle_set_peers(peers),
MetadataMsg::DisseminateTick => self.handle_disseminate_tick(ctx),
MetadataMsg::GcTick => self.gc_tick(ctx),
MetadataMsg::BulkLoad { entries } => self.handle_bulk_load(entries),
}
}
}

2
crates/datastore/src/actors/mod.rs
View file

@ -1,9 +1,11 @@
pub mod blob_store;
pub mod datastore_node;
pub mod gateway;
pub mod metadata;
pub mod transfer;
pub use blob_store::BlobStoreActor;
pub use datastore_node::DatastoreNode;
pub use gateway::GatewayActor;
pub use metadata::MetadataActor;
pub use transfer::TransferActor;

567
crates/datastore/src/api.rs
View file

@ -12,8 +12,11 @@ use std::time::{Duration, Instant};
use swactor::actor::ActorAddress;
use swactor::runtime::{Inbox, Runtime};
use distribution::types::NodeId;
use crate::auth::SignedRequest;
use crate::chunking::reassemble_blob;
use crate::messages::{BlobStoreMsg, DatastoreNodeMsg, DatastoreResponse, MetadataMsg};
use crate::messages::{BlobStoreMsg, DatastoreNodeMsg, DatastoreResponse, GatewayMsg, MetadataMsg};
use crate::metrics::DatastoreMetrics;
use crate::types::ContentHash;
@ -30,10 +33,13 @@ struct ApiState {
datastore_addr: ActorAddress,
metadata_addr: ActorAddress,
blob_store_addr: ActorAddress,
gateway_addr: Option<ActorAddress>,
peers: Arc<Mutex<Vec<PeerInfo>>>,
metrics: Arc<DatastoreMetrics>,
}
const CRYPTO_WASM: &[u8] = include_bytes!("crypto_wasm.wasm");
const POLL_TIMEOUT: Duration = Duration::from_secs(5);
const POLL_INTERVAL: Duration = Duration::from_millis(1);
@ -51,6 +57,107 @@ fn poll_response(inbox: &Inbox<DatastoreResponse>, timeout: Duration) -> Option<
}
}
/// Check auth and return the caller's identity (public key).
/// Returns Ok(NodeId) if no gateway is configured (zero NodeId) or if authorized.
/// Returns Err((status_code, message)) if denied.
fn check_auth_identity(request: &tiny_http::Request, state: &ApiState) -> Result<NodeId, (u16, String)> {
let gateway_addr = match state.gateway_addr {
Some(addr) => addr,
None => return Ok(NodeId([0; 32])), // no auth configured
};
let header_value = request
.headers()
.iter()
.find(|h| h.field.as_str().as_str().eq_ignore_ascii_case("x-signed-request"))
.map(|h| h.value.as_str().to_string());
let header_value = match header_value {
Some(v) => v,
None => return Err((401, "missing X-Signed-Request header".to_string())),
};
let signed_request: SignedRequest = serde_json::from_str(&header_value)
.map_err(|e| (400, format!("invalid X-Signed-Request: {e}")))?;
let public_key = signed_request.public_key;
let inbox = state
.runtime
.new_inbox::<DatastoreResponse>()
.map_err(|_| (500, "failed to create inbox".to_string()))?;
let _ = state.runtime.send_to(
gateway_addr,
GatewayMsg::Authorize {
request: signed_request,
reply_to: *inbox.addr(),
},
);
match poll_response(&inbox, POLL_TIMEOUT) {
Some(DatastoreResponse::Bool(true)) => Ok(public_key),
Some(DatastoreResponse::Denied { reason }) => {
Err((403, format!("{reason:?}")))
}
_ => Err((504, "auth timeout".to_string())),
}
}
/// Check auth by sending a GatewayMsg::Authorize to the gateway actor.
/// Returns Ok(()) if no gateway is configured or if authorized.
/// Returns Err((status_code, message)) if denied.
fn check_auth(request: &tiny_http::Request, state: &ApiState) -> Result<(), (u16, String)> {
check_auth_identity(request, state).map(|_| ())
}
/// Verify the signature only (no ACL check).
/// Used for endpoints where the caller proves key ownership without needing authorization.
/// Returns Ok(NodeId) on valid signature, Err on failure.
fn check_auth_signature_only(request: &tiny_http::Request, state: &ApiState) -> Result<NodeId, (u16, String)> {
let gateway_addr = match state.gateway_addr {
Some(addr) => addr,
None => return Ok(NodeId([0; 32])),
};
let header_value = request
.headers()
.iter()
.find(|h| h.field.as_str().as_str().eq_ignore_ascii_case("x-signed-request"))
.map(|h| h.value.as_str().to_string());
let header_value = match header_value {
Some(v) => v,
None => return Err((401, "missing X-Signed-Request header".to_string())),
};
let signed_request: SignedRequest = serde_json::from_str(&header_value)
.map_err(|e| (400, format!("invalid X-Signed-Request: {e}")))?;
let public_key = signed_request.public_key;
let inbox = state
.runtime
.new_inbox::<DatastoreResponse>()
.map_err(|_| (500, "failed to create inbox".to_string()))?;
let _ = state.runtime.send_to(
gateway_addr,
GatewayMsg::VerifySignature {
request: signed_request,
reply_to: *inbox.addr(),
},
);
match poll_response(&inbox, POLL_TIMEOUT) {
Some(DatastoreResponse::Bool(true)) => Ok(public_key),
Some(DatastoreResponse::Denied { reason }) => {
Err((403, format!("{reason:?}")))
}
_ => Err((504, "auth timeout".to_string())),
}
}
fn respond_json(request: tiny_http::Request, json: &str) {
let response = tiny_http::Response::from_string(json).with_header(
"Content-Type: application/json"
@ -79,6 +186,25 @@ fn respond_html(request: tiny_http::Request) {
let _ = request.respond(response);
}
fn respond_wasm(request: tiny_http::Request) {
let response = tiny_http::Response::from_data(CRYPTO_WASM.to_vec()).with_header(
"Content-Type: application/wasm"
.parse::<tiny_http::Header>()
.unwrap(),
);
let _ = request.respond(response);
}
fn respond_admin_html(request: tiny_http::Request) {
let response =
tiny_http::Response::from_string(crate::ui_html::DATASTORE_ADMIN_HTML).with_header(
"Content-Type: text/html; charset=utf-8"
.parse::<tiny_http::Header>()
.unwrap(),
);
let _ = request.respond(response);
}
fn respond_error(request: tiny_http::Request, status: u16, msg: &str) {
let json = serde_json::json!({ "error": msg }).to_string();
let response = tiny_http::Response::from_string(json)
@ -181,6 +307,10 @@ fn entries_to_json(entries: &[crate::types::ObjectEntry]) -> Vec<serde_json::Val
// ── PUT handler ─────────────────────────────────────────────────────────
fn handle_put(mut request: tiny_http::Request, url: &str, state: &ApiState) {
if let Err((status, msg)) = check_auth(&request, state) {
respond_error(request, status, &msg);
return;
}
let params = parse_query_string(url);
let name = params.get("name").cloned();
@ -241,6 +371,10 @@ fn handle_put(mut request: tiny_http::Request, url: &str, state: &ApiState) {
// ── GET handler (metadata) ──────────────────────────────────────────────
fn handle_get(request: tiny_http::Request, url: &str, state: &ApiState) {
if let Err((status, msg)) = check_auth(&request, state) {
respond_error(request, status, &msg);
return;
}
let params = parse_query_string(url);
let hash_hex = match params.get("hash") {
Some(h) => h,
@ -299,6 +433,10 @@ fn handle_get(request: tiny_http::Request, url: &str, state: &ApiState) {
// ── DATA handler (reassembled binary) ───────────────────────────────────
fn handle_data(request: tiny_http::Request, url: &str, state: &ApiState) {
if let Err((status, msg)) = check_auth(&request, state) {
respond_error(request, status, &msg);
return;
}
let params = parse_query_string(url);
let hash_hex = match params.get("hash") {
Some(h) => h,
@ -398,6 +536,10 @@ fn handle_data(request: tiny_http::Request, url: &str, state: &ApiState) {
// ── DELETE handler ──────────────────────────────────────────────────────
fn handle_delete(request: tiny_http::Request, url: &str, state: &ApiState) {
if let Err((status, msg)) = check_auth(&request, state) {
respond_error(request, status, &msg);
return;
}
let params = parse_query_string(url);
let hash_hex = match params.get("hash") {
Some(h) => h,
@ -453,6 +595,10 @@ fn handle_delete(request: tiny_http::Request, url: &str, state: &ApiState) {
// ── LIST handler ────────────────────────────────────────────────────────
fn handle_list(request: tiny_http::Request, url: &str, state: &ApiState) {
if let Err((status, msg)) = check_auth(&request, state) {
respond_error(request, status, &msg);
return;
}
let params = parse_query_string(url);
let name_filter = params.get("name").cloned();
let all = params.get("all").map_or(false, |v| v == "true" || v == "1");
@ -724,6 +870,415 @@ fn try_remote_get(
None
}
// ── Auth grant/revoke handlers ──────────────────────────────────────────
fn parse_node_id_hex(hex: &str) -> Option<NodeId> {
if hex.len() != 64 {
return None;
}
let mut bytes = [0u8; 32];
for (i, chunk) in hex.as_bytes().chunks(2).enumerate() {
let hi = hex_val(chunk[0])?;
let lo = hex_val(chunk[1])?;
bytes[i] = (hi << 4) | lo;
}
Some(NodeId(bytes))
}
fn handle_auth_grant(request: tiny_http::Request, url: &str, state: &ApiState) {
let requester = match check_auth_identity(&request, state) {
Ok(id) => id,
Err((status, msg)) => {
respond_error(request, status, &msg);
return;
}
};
let params = parse_query_string(url);
let key_hex = match params.get("key") {
Some(k) => k,
None => {
respond_error(request, 400, "missing ?key= parameter");
return;
}
};
let key = match parse_node_id_hex(key_hex) {
Some(k) => k,
None => {
respond_error(request, 400, "invalid key hex (expected 64 hex chars)");
return;
}
};
let gateway_addr = match state.gateway_addr {
Some(addr) => addr,
None => {
respond_error(request, 400, "auth not enabled on this node");
return;
}
};
let inbox = match state.runtime.new_inbox::<DatastoreResponse>() {
Ok(i) => i,
Err(_) => {
respond_error(request, 500, "failed to create inbox");
return;
}
};
let label = params.get("name").cloned();
let _ = state.runtime.send_to(
gateway_addr,
GatewayMsg::Grant {
requester,
key,
label,
reply_to: *inbox.addr(),
},
);
match poll_response(&inbox, POLL_TIMEOUT) {
Some(DatastoreResponse::Bool(true)) => {
respond_json(request, &serde_json::json!({ "ok": true }).to_string());
}
Some(DatastoreResponse::Denied { reason }) => {
respond_error(request, 403, &format!("{reason:?}"));
}
_ => {
respond_error(request, 504, "timeout");
}
}
}
fn handle_auth_revoke(request: tiny_http::Request, url: &str, state: &ApiState) {
let requester = match check_auth_identity(&request, state) {
Ok(id) => id,
Err((status, msg)) => {
respond_error(request, status, &msg);
return;
}
};
let params = parse_query_string(url);
let key_hex = match params.get("key") {
Some(k) => k,
None => {
respond_error(request, 400, "missing ?key= parameter");
return;
}
};
let key = match parse_node_id_hex(key_hex) {
Some(k) => k,
None => {
respond_error(request, 400, "invalid key hex (expected 64 hex chars)");
return;
}
};
let gateway_addr = match state.gateway_addr {
Some(addr) => addr,
None => {
respond_error(request, 400, "auth not enabled on this node");
return;
}
};
let inbox = match state.runtime.new_inbox::<DatastoreResponse>() {
Ok(i) => i,
Err(_) => {
respond_error(request, 500, "failed to create inbox");
return;
}
};
let _ = state.runtime.send_to(
gateway_addr,
GatewayMsg::Revoke {
requester,
key,
reply_to: *inbox.addr(),
},
);
match poll_response(&inbox, POLL_TIMEOUT) {
Some(DatastoreResponse::Bool(true)) => {
respond_json(request, &serde_json::json!({ "ok": true }).to_string());
}
Some(DatastoreResponse::Denied { reason }) => {
respond_error(request, 403, &format!("{reason:?}"));
}
_ => {
respond_error(request, 504, "timeout");
}
}
}
// ── Access request handlers ──────────────────────────────────────────────
fn handle_auth_request(mut request: tiny_http::Request, state: &ApiState) {
let caller = match check_auth_signature_only(&request, state) {
Ok(id) => id,
Err((status, msg)) => {
respond_error(request, status, &msg);
return;
}
};
// Read JSON body
let mut body_bytes = Vec::new();
if request.as_reader().read_to_end(&mut body_bytes).is_err() {
return;
}
let body: serde_json::Value = match serde_json::from_slice(&body_bytes) {
Ok(v) => v,
Err(e) => {
respond_error(request, 400, &format!("invalid JSON: {e}"));
return;
}
};
let name = match body.get("name").and_then(|v| v.as_str()) {
Some(n) if !n.trim().is_empty() => n.trim().to_string(),
_ => {
respond_error(request, 400, "name is required");
return;
}
};
if name.len() > 64 {
respond_error(request, 400, "name must be 64 characters or fewer");
return;
}
let message = body
.get("message")
.and_then(|v| v.as_str())
.unwrap_or("")
.to_string();
if message.len() > 256 {
respond_error(request, 400, "message must be 256 characters or fewer");
return;
}
let gateway_addr = match state.gateway_addr {
Some(addr) => addr,
None => {
respond_error(request, 400, "auth not enabled on this node");
return;
}
};
let inbox = match state.runtime.new_inbox::<DatastoreResponse>() {
Ok(i) => i,
Err(_) => {
respond_error(request, 500, "failed to create inbox");
return;
}
};
let _ = state.runtime.send_to(
gateway_addr,
GatewayMsg::SubmitAccessRequest {
key: caller,
name,
message,
reply_to: *inbox.addr(),
},
);
match poll_response(&inbox, POLL_TIMEOUT) {
Some(DatastoreResponse::Bool(true)) => {
respond_json(request, &serde_json::json!({ "ok": true }).to_string());
}
_ => {
respond_error(request, 504, "timeout");
}
}
}
fn handle_auth_requests_list(request: tiny_http::Request, state: &ApiState) {
let requester = match check_auth_identity(&request, state) {
Ok(id) => id,
Err((status, msg)) => {
respond_error(request, status, &msg);
return;
}
};
let gateway_addr = match state.gateway_addr {
Some(addr) => addr,
None => {
respond_error(request, 400, "auth not enabled on this node");
return;
}
};
let inbox = match state.runtime.new_inbox::<DatastoreResponse>() {
Ok(i) => i,
Err(_) => {
respond_error(request, 500, "failed to create inbox");
return;
}
};
let _ = state.runtime.send_to(
gateway_addr,
GatewayMsg::ListAccessRequests {
requester,
reply_to: *inbox.addr(),
},
);
match poll_response(&inbox, POLL_TIMEOUT) {
Some(DatastoreResponse::AccessRequests { requests }) => {
let json_list: Vec<serde_json::Value> = requests
.iter()
.map(|r| {
let key_hex: String = r.key.0.iter().map(|b| format!("{b:02x}")).collect();
serde_json::json!({
"key": key_hex,
"name": r.name,
"message": r.message,
"requested_at": r.requested_at,
})
})
.collect();
respond_json(request, &serde_json::json!(json_list).to_string());
}
Some(DatastoreResponse::Denied { reason }) => {
respond_error(request, 403, &format!("{reason:?}"));
}
_ => {
respond_error(request, 504, "timeout");
}
}
}
fn handle_auth_keys_list(request: tiny_http::Request, state: &ApiState) {
let requester = match check_auth_identity(&request, state) {
Ok(id) => id,
Err((status, msg)) => {
respond_error(request, status, &msg);
return;
}
};
let gateway_addr = match state.gateway_addr {
Some(addr) => addr,
None => {
respond_error(request, 400, "auth not enabled on this node");
return;
}
};
let inbox = match state.runtime.new_inbox::<DatastoreResponse>() {
Ok(i) => i,
Err(_) => {
respond_error(request, 500, "failed to create inbox");
return;
}
};
let _ = state.runtime.send_to(
gateway_addr,
GatewayMsg::ListAuthorizedKeys {
requester,
reply_to: *inbox.addr(),
},
);
match poll_response(&inbox, POLL_TIMEOUT) {
Some(DatastoreResponse::AuthorizedKeys { keys }) => {
let json_list: Vec<serde_json::Value> = keys
.iter()
.map(|k| {
let key_hex: String = k.key.0.iter().map(|b| format!("{b:02x}")).collect();
serde_json::json!({
"key": key_hex,
"label": k.label,
})
})
.collect();
respond_json(request, &serde_json::json!(json_list).to_string());
}
Some(DatastoreResponse::Denied { reason }) => {
respond_error(request, 403, &format!("{reason:?}"));
}
_ => {
respond_error(request, 504, "timeout");
}
}
}
fn handle_auth_deny(request: tiny_http::Request, url: &str, state: &ApiState) {
let requester = match check_auth_identity(&request, state) {
Ok(id) => id,
Err((status, msg)) => {
respond_error(request, status, &msg);
return;
}
};
let params = parse_query_string(url);
let key_hex = match params.get("key") {
Some(k) => k,
None => {
respond_error(request, 400, "missing ?key= parameter");
return;
}
};
let key = match parse_node_id_hex(key_hex) {
Some(k) => k,
None => {
respond_error(request, 400, "invalid key hex (expected 64 hex chars)");
return;
}
};
let gateway_addr = match state.gateway_addr {
Some(addr) => addr,
None => {
respond_error(request, 400, "auth not enabled on this node");
return;
}
};
let inbox = match state.runtime.new_inbox::<DatastoreResponse>() {
Ok(i) => i,
Err(_) => {
respond_error(request, 500, "failed to create inbox");
return;
}
};
let _ = state.runtime.send_to(
gateway_addr,
GatewayMsg::DenyAccessRequest {
requester,
key,
reply_to: *inbox.addr(),
},
);
match poll_response(&inbox, POLL_TIMEOUT) {
Some(DatastoreResponse::Bool(true)) => {
respond_json(request, &serde_json::json!({ "ok": true }).to_string());
}
Some(DatastoreResponse::Denied { reason }) => {
respond_error(request, 403, &format!("{reason:?}"));
}
_ => {
respond_error(request, 504, "timeout");
}
}
}
// ── Server startup ──────────────────────────────────────────────────────
/// Start the HTTP API server for the datastore.
@ -735,6 +1290,7 @@ pub fn start_api_server(
datastore_addr: ActorAddress,
metadata_addr: ActorAddress,
blob_store_addr: ActorAddress,
gateway_addr: Option<ActorAddress>,
port: u16,
metrics: Arc<DatastoreMetrics>,
) -> (Arc<AtomicBool>, Arc<Mutex<Vec<PeerInfo>>>) {
@ -746,6 +1302,7 @@ pub fn start_api_server(
datastore_addr,
metadata_addr,
blob_store_addr,
gateway_addr,
peers: Arc::clone(&peers),
metrics,
});
@ -780,7 +1337,15 @@ pub fn start_api_server(
("POST", "/api/delete") => handle_delete(request, &url, &state),
("GET", "/api/list") => handle_list(request, &url, &state),
("GET", "/api/status") => handle_status(request, &state),
("POST", "/api/auth/grant") => handle_auth_grant(request, &url, &state),
("POST", "/api/auth/revoke") => handle_auth_revoke(request, &url, &state),
("POST", "/api/auth/request") => handle_auth_request(request, &state),
("GET", "/api/auth/requests") => handle_auth_requests_list(request, &state),
("GET", "/api/auth/keys") => handle_auth_keys_list(request, &state),
("POST", "/api/auth/deny") => handle_auth_deny(request, &url, &state),
("GET", "/") => respond_html(request),
("GET", "/crypto.wasm") => respond_wasm(request),
("GET", "/admin") => respond_admin_html(request),
_ => {
respond_error(request, 404, "not found");
}

324
crates/datastore/src/auth.rs Normal file
View file

@ -0,0 +1,324 @@
//! Authorization types and engine for the distributed datastore.
//!
//! Enforces binary access control (authorized or not) using ed25519 identities.
//! Two auth paths:
//! - **Path 1 (Direct iroh):** connection-level `check_node` against the ACL.
//! - **Path 2 (Browser relay):** per-request `check_signed_request` with
//! signature, timestamp, nonce, and ACL verification.
use std::collections::{BTreeMap, HashMap, HashSet};
use std::io;
use std::path::Path;
use serde::{Deserialize, Serialize};
use distribution::crypto;
use distribution::types::{NodeId, Signature};
use shared_types::ContentHash;
// ─── Access Request / Authorized Key Info ──────────────────────────────────
/// A pending access request from a browser user.
#[derive(Debug, Clone)]
pub struct AccessRequestInfo {
pub key: NodeId,
pub name: String,
pub message: String,
pub requested_at: u64,
}
/// An authorized key with its human-readable label.
#[derive(Debug, Clone)]
pub struct AuthorizedKeyInfo {
pub key: NodeId,
pub label: String,
}
// ─── DatastoreAction ────────────────────────────────────────────────────────
/// An action a client wants to perform on the datastore.
///
/// Carried inside a `SignedRequestPayload` for browser-relay auth (Auth Path 2).
/// Aligned to match `DatastoreNodeMsg` variants — content-hash-first addressing.
#[derive(Debug, Clone, Serialize, Deserialize)]
pub enum DatastoreAction {
Put {
name: Option<String>,
content_hash: ContentHash,
size_bytes: u64,
tags: BTreeMap<String, String>,
},
Get {
content_hash: ContentHash,
},
Delete {
content_hash: ContentHash,
},
List {
name_filter: Option<String>,
},
/// Browser-originated request — proves identity without binding to specific content.
Access,
}
// ─── SignedRequestPayload ───────────────────────────────────────────────────
/// The signable payload of a client request.
///
/// Serialized canonically (serde_json) and signed by the client's ed25519 key.
/// Includes timestamp and nonce for replay protection.
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct SignedRequestPayload {
pub action: DatastoreAction,
/// Unix timestamp in seconds.
pub timestamp: u64,
/// 16 random bytes — prevents replay within the timestamp window.
pub nonce: [u8; 16],
}
// ─── SignedRequest ──────────────────────────────────────────────────────────
/// A signed request envelope for browser-relay auth (Auth Path 2).
///
/// The relay forwards this opaquely — it cannot forge, modify, or replay it.
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct SignedRequest {
pub payload: SignedRequestPayload,
/// The client's ed25519 public key.
pub public_key: NodeId,
/// ed25519 signature over the canonical serialization of `payload`.
pub signature: Signature,
}
// ─── AccessControlList ──────────────────────────────────────────────────────
/// The datastore's access control list.
///
/// Persisted as `acl.json` alongside the datastore's `storage_path`.
/// The owner always has implicit full access.
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct AccessControlList {
/// The datastore owner's public key — always has full access.
pub owner: NodeId,
/// Explicitly authorized client keys.
pub authorized_keys: HashSet<NodeId>,
/// Human-readable labels for authorized keys (hex → name).
#[serde(default)]
pub key_labels: HashMap<String, String>,
}
impl AccessControlList {
/// Load an ACL from disk, or create a default one with the given owner.
pub fn load_or_create(path: &Path, owner: NodeId) -> io::Result<Self> {
if path.exists() {
let data = std::fs::read_to_string(path)?;
let acl: AccessControlList = serde_json::from_str(&data)
.map_err(|e| io::Error::new(io::ErrorKind::InvalidData, e))?;
Ok(acl)
} else {
let acl = AccessControlList {
owner,
authorized_keys: HashSet::new(),
key_labels: HashMap::new(),
};
acl.save(path)?;
Ok(acl)
}
}
/// Persist the ACL to disk as JSON.
pub fn save(&self, path: &Path) -> io::Result<()> {
if let Some(parent) = path.parent() {
std::fs::create_dir_all(parent)?;
}
let json = serde_json::to_string_pretty(self)
.map_err(|e| io::Error::new(io::ErrorKind::Other, e))?;
std::fs::write(path, json)
}
}
// ─── AuthzResult ────────────────────────────────────────────────────────────
/// The outcome of an authorization check.
#[derive(Debug, Clone, PartialEq, Eq)]
pub enum AuthzResult {
Allowed,
Denied(DeniedReason),
}
/// Why a request was denied.
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub enum DeniedReason {
/// The key is not in the ACL.
NotAuthorized,
/// The ed25519 signature is invalid.
InvalidSignature,
/// The request timestamp is outside the ±300s window.
RequestExpired,
/// The nonce has already been seen within the time window.
ReplayDetected,
}
// ─── Signing / Verification ─────────────────────────────────────────────────
/// Sign a request payload, returning a complete `SignedRequest` envelope.
pub fn sign_request(keypair: &crypto::Keypair, payload: SignedRequestPayload) -> SignedRequest {
let bytes = serde_json::to_vec(&payload).expect("SignedRequestPayload is always serializable");
let signature = keypair.sign(&bytes);
SignedRequest {
payload,
public_key: keypair.node_id(),
signature,
}
}
/// Verify a `SignedRequest`'s signature against its embedded `public_key`.
///
/// Checks only signature validity — does NOT check timestamp, nonce, or ACL.
pub fn verify_signed_request(request: &SignedRequest) -> bool {
let Ok(bytes) = serde_json::to_vec(&request.payload) else {
return false;
};
crypto::verify(&request.public_key, &bytes, &request.signature)
}
// ─── AuthzEngine ────────────────────────────────────────────────────────────
/// Authorization engine — checks requests against the ACL and replay state.
///
/// Sits at the edge of the actor system (Auth Gate / GatewayActor) and decides
/// whether to accept or reject external requests before they reach internal actors.
#[derive(Debug)]
pub struct AuthzEngine {
pub acl: AccessControlList,
seen_nonces: HashMap<[u8; 16], u64>,
timestamp_window: u64,
}
impl AuthzEngine {
/// Create a new engine with the given ACL and a default 300-second window.
pub fn new(acl: AccessControlList) -> Self {
Self {
acl,
seen_nonces: HashMap::new(),
timestamp_window: 300,
}
}
/// Check whether a `NodeId` is authorized (connection-level, Auth Path 1).
///
/// The owner always has implicit access. Other keys must be in `authorized_keys`.
pub fn check_node(&self, node_id: &NodeId) -> AuthzResult {
if *node_id == self.acl.owner || self.acl.authorized_keys.contains(node_id) {
AuthzResult::Allowed
} else {
AuthzResult::Denied(DeniedReason::NotAuthorized)
}
}
/// Verify signature, timestamp, and nonce — but skip the ACL check.
///
/// Used for endpoints where the caller proves key ownership without
/// needing to be in the ACL (e.g. submitting an access request).
pub fn check_signature_only(&mut self, request: &SignedRequest, now: u64) -> AuthzResult {
// 1. Signature
if !verify_signed_request(request) {
return AuthzResult::Denied(DeniedReason::InvalidSignature);
}
// 2. Timestamp freshness
let ts = request.payload.timestamp;
let diff = if now >= ts { now - ts } else { ts - now };
if diff > self.timestamp_window {
return AuthzResult::Denied(DeniedReason::RequestExpired);
}
// 3. Nonce uniqueness
if self.seen_nonces.contains_key(&request.payload.nonce) {
return AuthzResult::Denied(DeniedReason::ReplayDetected);
}
self.seen_nonces.insert(request.payload.nonce, ts);
AuthzResult::Allowed
}
/// Verify and authorize a signed request (Auth Path 2).
///
/// Four-step verification in strict order:
/// 1. Signature validity
/// 2. Timestamp freshness (±window)
/// 3. Nonce uniqueness
/// 4. ACL check
pub fn check_signed_request(&mut self, request: &SignedRequest, now: u64) -> AuthzResult {
// 1. Signature
if !verify_signed_request(request) {
return AuthzResult::Denied(DeniedReason::InvalidSignature);
}
// 2. Timestamp freshness
let ts = request.payload.timestamp;
let diff = if now >= ts { now - ts } else { ts - now };
if diff > self.timestamp_window {
return AuthzResult::Denied(DeniedReason::RequestExpired);
}
// 3. Nonce uniqueness
if self.seen_nonces.contains_key(&request.payload.nonce) {
return AuthzResult::Denied(DeniedReason::ReplayDetected);
}
self.seen_nonces.insert(request.payload.nonce, ts);
// 4. ACL check
self.check_node(&request.public_key)
}
/// Grant access to a `NodeId`. Owner-only, idempotent.
/// If `label` is provided, it's stored as a human-readable name for the key.
pub fn grant(&mut self, requester: &NodeId, key: NodeId, label: Option<String>) -> Result<(), DeniedReason> {
if *requester != self.acl.owner {
return Err(DeniedReason::NotAuthorized);
}
self.acl.authorized_keys.insert(key);
if let Some(name) = label {
let hex: String = key.0.iter().map(|b| format!("{b:02x}")).collect();
self.acl.key_labels.insert(hex, name);
}
Ok(())
}
/// Revoke access from a `NodeId`. Owner-only, idempotent.
/// Revoking the owner is a no-op (owner's implicit access cannot be removed).
pub fn revoke(&mut self, requester: &NodeId, key: NodeId) -> Result<(), DeniedReason> {
if *requester != self.acl.owner {
return Err(DeniedReason::NotAuthorized);
}
// Owner's implicit access cannot be removed.
if key != self.acl.owner {
self.acl.authorized_keys.remove(&key);
let hex: String = key.0.iter().map(|b| format!("{b:02x}")).collect();
self.acl.key_labels.remove(&hex);
}
Ok(())
}
/// List all authorized keys with their labels.
pub fn authorized_key_list(&self) -> Vec<AuthorizedKeyInfo> {
self.acl
.authorized_keys
.iter()
.map(|key| {
let hex: String = key.0.iter().map(|b| format!("{b:02x}")).collect();
let label = self.acl.key_labels.get(&hex).cloned().unwrap_or_default();
AuthorizedKeyInfo { key: *key, label }
})
.collect()
}
/// Evict nonces whose timestamps fall outside the current window.
pub fn gc_nonces(&mut self, now: u64) {
self.seen_nonces.retain(|_nonce, ts| {
let diff = if now >= *ts { now - *ts } else { *ts - now };
diff <= self.timestamp_window
});
}
}

616
crates/datastore/src/bin/store_cli.rs
View file

@ -2,12 +2,18 @@
//!
//! Talks to a running `swactor-store-node` over its HTTP API.
use std::collections::BTreeMap;
use std::fs;
use std::io::Read;
use std::path::PathBuf;
use std::time::{SystemTime, UNIX_EPOCH};
use clap::{Parser, Subcommand};
use distribution::crypto::Keypair;
use shared_types::ContentHash;
use swactor_datastore::auth::{sign_request, DatastoreAction, SignedRequestPayload};
#[derive(Parser)]
#[command(name = "swactor-store", about = "Swactor datastore CLI")]
struct Args {
@ -15,6 +21,10 @@ struct Args {
#[arg(long, default_value = "http://localhost:9091")]
url: String,
/// Path to key.json file for auth signing
#[arg(long)]
key: Option<PathBuf>,
#[command(subcommand)]
command: Command,
}
@ -53,22 +63,282 @@ enum Command {
},
/// Query node status
Status,
/// Authorize a public key (owner only)
Grant {
/// Public key (64 hex chars) or name to authorize
key: String,
/// Optional human-readable name for the key
#[arg(long)]
name: Option<String>,
},
/// Revoke a public key (owner only)
Revoke {
/// Public key (64 hex chars) or name to revoke
key: String,
},
/// List pending access requests (owner only)
Requests,
/// List authorized keys with names (owner only)
Keys,
/// Deny (dismiss) a pending access request (owner only)
Deny {
/// Public key (64 hex chars) or name to deny
key: String,
},
}
// ── Key file helpers ────────────────────────────────────────────────────────
fn hex_decode(hex: &str) -> Option<Vec<u8>> {
if hex.len() % 2 != 0 {
return None;
}
let mut bytes = Vec::with_capacity(hex.len() / 2);
for chunk in hex.as_bytes().chunks(2) {
let hi = hex_digit(chunk[0])?;
let lo = hex_digit(chunk[1])?;
bytes.push((hi << 4) | lo);
}
Some(bytes)
}
fn hex_digit(b: u8) -> Option<u8> {
match b {
b'0'..=b'9' => Some(b - b'0'),
b'a'..=b'f' => Some(b - b'a' + 10),
b'A'..=b'F' => Some(b - b'A' + 10),
_ => None,
}
}
fn load_keypair(path: &std::path::Path) -> Keypair {
let data = fs::read_to_string(path).unwrap_or_else(|e| {
eprintln!("Error reading key file {}: {e}", path.display());
std::process::exit(1);
});
let json: serde_json::Value = serde_json::from_str(&data).unwrap_or_else(|e| {
eprintln!("Error parsing key file: {e}");
std::process::exit(1);
});
let secret_hex = json
.get("secret_key")
.and_then(|v| v.as_str())
.unwrap_or_else(|| {
eprintln!("Key file missing secret_key field");
std::process::exit(1);
});
let secret_bytes = hex_decode(secret_hex).unwrap_or_else(|| {
eprintln!("Invalid secret_key hex in key file");
std::process::exit(1);
});
let secret: [u8; 32] = secret_bytes.try_into().unwrap_or_else(|_| {
eprintln!("secret_key must be exactly 32 bytes");
std::process::exit(1);
});
Keypair::from_bytes(&secret)
}
// ── Auth signing ────────────────────────────────────────────────────────────
fn sign_action(keypair: &Keypair, action: DatastoreAction) -> String {
let timestamp = SystemTime::now()
.duration_since(UNIX_EPOCH)
.unwrap()
.as_secs();
let mut nonce = [0u8; 16];
getrandom::getrandom(&mut nonce).expect("failed to generate random nonce");
let payload = SignedRequestPayload {
action,
timestamp,
nonce,
};
let signed = sign_request(keypair, payload);
serde_json::to_string(&signed).expect("SignedRequest is always serializable")
}
// ── Name resolution helpers ────────────────────────────────────────────────
fn is_hex_key(s: &str) -> bool {
s.len() == 64 && hex_decode(s).is_some()
}
/// Parse `"alice (c9d0e1f2)"` → `("alice", Some("c9d0e1f2"))`.
/// Returns `(input, None)` if no suffix found.
fn parse_disambiguated_name(input: &str) -> (&str, Option<&str>) {
if let Some(paren_start) = input.rfind(" (") {
if input.ends_with(')') {
let prefix = &input[paren_start + 2..input.len() - 1];
if prefix.len() == 8 && hex_decode(prefix).is_some() {
return (&input[..paren_start], Some(prefix));
}
}
}
(input, None)
}
/// Resolve a human-readable name to a hex key from the pending requests list.
fn resolve_pending_request_key(base: &str, name_input: &str, kp: &Keypair) -> String {
let url = format!("{base}/api/auth/requests");
let req = ureq::get(&url)
.set("X-Signed-Request", &sign_action(kp, DatastoreAction::Access));
let resp = match req.call() {
Ok(r) => r,
Err(e) => {
eprintln!("Error fetching requests: {e}");
std::process::exit(1);
}
};
let body: serde_json::Value = match resp.into_json() {
Ok(v) => v,
Err(e) => {
eprintln!("Error parsing requests response: {e}");
std::process::exit(1);
}
};
let requests = match body.as_array() {
Some(arr) => arr,
None => {
eprintln!("No pending request named '{name_input}'");
std::process::exit(1);
}
};
let (search_name, disambig_prefix) = parse_disambiguated_name(name_input);
let matches: Vec<&serde_json::Value> = requests
.iter()
.filter(|r| {
let name = r.get("name").and_then(|v| v.as_str()).unwrap_or("");
if !name.eq_ignore_ascii_case(search_name) {
return false;
}
if let Some(prefix) = disambig_prefix {
let key = r.get("key").and_then(|v| v.as_str()).unwrap_or("");
return key.starts_with(prefix);
}
true
})
.collect();
match matches.len() {
0 => {
eprintln!("No pending request named '{name_input}'");
std::process::exit(1);
}
1 => matches[0]
.get("key")
.and_then(|v| v.as_str())
.unwrap()
.to_string(),
_ => {
eprintln!("Multiple pending requests named '{search_name}':");
for m in &matches {
let key = m.get("key").and_then(|v| v.as_str()).unwrap_or("?");
let prefix = &key[..8];
eprintln!(" {search_name} ({prefix})");
}
eprintln!("Re-run with the disambiguated name.");
std::process::exit(1);
}
}
}
/// Resolve a human-readable name to a hex key from the authorized keys list.
fn resolve_authorized_key(base: &str, name_input: &str, kp: &Keypair) -> String {
let url = format!("{base}/api/auth/keys");
let req = ureq::get(&url)
.set("X-Signed-Request", &sign_action(kp, DatastoreAction::Access));
let resp = match req.call() {
Ok(r) => r,
Err(e) => {
eprintln!("Error fetching keys: {e}");
std::process::exit(1);
}
};
let body: serde_json::Value = match resp.into_json() {
Ok(v) => v,
Err(e) => {
eprintln!("Error parsing keys response: {e}");
std::process::exit(1);
}
};
let keys = match body.as_array() {
Some(arr) => arr,
None => {
eprintln!("No authorized key named '{name_input}'");
std::process::exit(1);
}
};
let (search_name, disambig_prefix) = parse_disambiguated_name(name_input);
let matches: Vec<&serde_json::Value> = keys
.iter()
.filter(|k| {
let label = k.get("label").and_then(|v| v.as_str()).unwrap_or("");
if !label.eq_ignore_ascii_case(search_name) {
return false;
}
if let Some(prefix) = disambig_prefix {
let key = k.get("key").and_then(|v| v.as_str()).unwrap_or("");
return key.starts_with(prefix);
}
true
})
.collect();
match matches.len() {
0 => {
eprintln!("No authorized key named '{name_input}'");
std::process::exit(1);
}
1 => matches[0]
.get("key")
.and_then(|v| v.as_str())
.unwrap()
.to_string(),
_ => {
eprintln!("Multiple authorized keys named '{search_name}':");
for m in &matches {
let key = m.get("key").and_then(|v| v.as_str()).unwrap_or("?");
let prefix = &key[..8];
eprintln!(" {search_name} ({prefix})");
}
eprintln!("Re-run with the disambiguated name.");
std::process::exit(1);
}
}
}
fn main() {
let args = Args::parse();
let base = args.url.trim_end_matches('/');
let keypair = args.key.as_deref().map(load_keypair);
match args.command {
Command::Put { path, name } => cmd_put(base, &path, name.as_deref()),
Command::Get { hash, output } => cmd_get(base, &hash, output.as_deref()),
Command::Delete { hash } => cmd_delete(base, &hash),
Command::List { name, all } => cmd_list(base, name.as_deref(), all),
Command::Put { path, name } => cmd_put(base, &path, name.as_deref(), keypair.as_ref()),
Command::Get { hash, output } => {
cmd_get(base, &hash, output.as_deref(), keypair.as_ref())
}
Command::Delete { hash } => cmd_delete(base, &hash, keypair.as_ref()),
Command::List { name, all } => cmd_list(base, name.as_deref(), all, keypair.as_ref()),
Command::Status => cmd_status(base),
Command::Grant { key, name } => cmd_grant(base, &key, name.as_deref(), keypair.as_ref()),
Command::Revoke { key } => cmd_revoke(base, &key, keypair.as_ref()),
Command::Requests => cmd_requests(base, keypair.as_ref()),
Command::Keys => cmd_keys(base, keypair.as_ref()),
Command::Deny { key } => cmd_deny(base, &key, keypair.as_ref()),
}
}
fn cmd_put(base: &str, path: &PathBuf, name: Option<&str>) {
fn cmd_put(base: &str, path: &PathBuf, name: Option<&str>, keypair: Option<&Keypair>) {
let data = match fs::read(path) {
Ok(d) => d,
Err(e) => {
@ -90,7 +360,19 @@ fn cmd_put(base: &str, path: &PathBuf, name: Option<&str>) {
url.push_str(&format!("?name={}", url_encode(n)));
}
let resp = match ureq::post(&url).send_bytes(&data) {
let mut req = ureq::post(&url);
if let Some(kp) = keypair {
let content_hash = ContentHash::of(&data);
let action = DatastoreAction::Put {
name: label.clone(),
content_hash,
size_bytes: data.len() as u64,
tags: BTreeMap::new(),
};
req = req.set("X-Signed-Request", &sign_action(kp, action));
}
let resp = match req.send_bytes(&data) {
Ok(r) => r,
Err(e) => {
eprintln!("Error: {e}");
@ -114,11 +396,19 @@ fn cmd_put(base: &str, path: &PathBuf, name: Option<&str>) {
}
}
fn cmd_get(base: &str, hash: &str, output: Option<&std::path::Path>) {
fn cmd_get(base: &str, hash: &str, output: Option<&std::path::Path>, keypair: Option<&Keypair>) {
if let Some(out_path) = output {
// Download raw data
let url = format!("{base}/api/data?hash={hash}");
let resp = match ureq::get(&url).call() {
let mut req = ureq::get(&url);
if let Some(kp) = keypair {
if let Some(ch) = ContentHash::from_hex(hash) {
let action = DatastoreAction::Get { content_hash: ch };
req = req.set("X-Signed-Request", &sign_action(kp, action));
}
}
let resp = match req.call() {
Ok(r) => r,
Err(e) => {
eprintln!("Error: {e}");
@ -146,7 +436,15 @@ fn cmd_get(base: &str, hash: &str, output: Option<&std::path::Path>) {
} else {
// Metadata only
let url = format!("{base}/api/get?hash={hash}");
let resp = match ureq::get(&url).call() {
let mut req = ureq::get(&url);
if let Some(kp) = keypair {
if let Some(ch) = ContentHash::from_hex(hash) {
let action = DatastoreAction::Get { content_hash: ch };
req = req.set("X-Signed-Request", &sign_action(kp, action));
}
}
let resp = match req.call() {
Ok(r) => r,
Err(e) => {
eprintln!("Error: {e}");
@ -203,9 +501,17 @@ fn cmd_get(base: &str, hash: &str, output: Option<&std::path::Path>) {
}
}
fn cmd_delete(base: &str, hash: &str) {
fn cmd_delete(base: &str, hash: &str, keypair: Option<&Keypair>) {
let url = format!("{base}/api/delete?hash={hash}");
let resp = match ureq::post(&url).send_bytes(&[]) {
let mut req = ureq::post(&url);
if let Some(kp) = keypair {
if let Some(ch) = ContentHash::from_hex(hash) {
let action = DatastoreAction::Delete { content_hash: ch };
req = req.set("X-Signed-Request", &sign_action(kp, action));
}
}
let resp = match req.send_bytes(&[]) {
Ok(r) => r,
Err(e) => {
eprintln!("Error: {e}");
@ -229,7 +535,7 @@ fn cmd_delete(base: &str, hash: &str) {
}
}
fn cmd_list(base: &str, name: Option<&str>, all: bool) {
fn cmd_list(base: &str, name: Option<&str>, all: bool, keypair: Option<&Keypair>) {
let mut url = format!("{base}/api/list");
let mut sep = '?';
if let Some(n) = name {
@ -240,7 +546,15 @@ fn cmd_list(base: &str, name: Option<&str>, all: bool) {
url.push_str(&format!("{sep}all=true"));
}
let resp = match ureq::get(&url).call() {
let mut req = ureq::get(&url);
if let Some(kp) = keypair {
let action = DatastoreAction::List {
name_filter: name.map(|s| s.to_string()),
};
req = req.set("X-Signed-Request", &sign_action(kp, action));
}
let resp = match req.call() {
Ok(r) => r,
Err(e) => {
eprintln!("Error: {e}");
@ -313,6 +627,282 @@ fn cmd_status(base: &str) {
}
}
fn cmd_grant(base: &str, key_input: &str, name: Option<&str>, keypair: Option<&Keypair>) {
let kp = match keypair {
Some(kp) => kp,
None => {
eprintln!("Error: --key is required for grant (must be the owner key)");
std::process::exit(1);
}
};
let key_hex = if is_hex_key(key_input) {
key_input.to_string()
} else {
resolve_pending_request_key(base, key_input, kp)
};
let mut url = format!("{base}/api/auth/grant?key={key_hex}");
if let Some(n) = name {
url.push_str(&format!("&name={}", url_encode(n)));
}
let req = ureq::post(&url)
.set("X-Signed-Request", &sign_action(kp, DatastoreAction::Access));
let resp = match req.send_bytes(&[]) {
Ok(r) => r,
Err(ureq::Error::Status(status, resp)) => {
let body = resp.into_string().unwrap_or_default();
eprintln!("Error ({status}): {body}");
std::process::exit(1);
}
Err(e) => {
eprintln!("Error: {e}");
std::process::exit(1);
}
};
let body: serde_json::Value = match resp.into_json() {
Ok(v) => v,
Err(e) => {
eprintln!("Error parsing response: {e}");
std::process::exit(1);
}
};
if body.get("ok").and_then(|v| v.as_bool()) == Some(true) {
println!("Granted {key_hex}");
} else if let Some(err) = body.get("error").and_then(|v| v.as_str()) {
eprintln!("Error: {err}");
std::process::exit(1);
}
}
fn cmd_revoke(base: &str, key_input: &str, keypair: Option<&Keypair>) {
let kp = match keypair {
Some(kp) => kp,
None => {
eprintln!("Error: --key is required for revoke (must be the owner key)");
std::process::exit(1);
}
};
let key_hex = if is_hex_key(key_input) {
key_input.to_string()
} else {
resolve_authorized_key(base, key_input, kp)
};
let url = format!("{base}/api/auth/revoke?key={key_hex}");
let req = ureq::post(&url)
.set("X-Signed-Request", &sign_action(kp, DatastoreAction::Access));
let resp = match req.send_bytes(&[]) {
Ok(r) => r,
Err(ureq::Error::Status(status, resp)) => {
let body = resp.into_string().unwrap_or_default();
eprintln!("Error ({status}): {body}");
std::process::exit(1);
}
Err(e) => {
eprintln!("Error: {e}");
std::process::exit(1);
}
};
let body: serde_json::Value = match resp.into_json() {
Ok(v) => v,
Err(e) => {
eprintln!("Error parsing response: {e}");
std::process::exit(1);
}
};
if body.get("ok").and_then(|v| v.as_bool()) == Some(true) {
println!("Revoked {key_hex}");
} else if let Some(err) = body.get("error").and_then(|v| v.as_str()) {
eprintln!("Error: {err}");
std::process::exit(1);
}
}
fn cmd_requests(base: &str, keypair: Option<&Keypair>) {
let kp = match keypair {
Some(kp) => kp,
None => {
eprintln!("Error: --key is required for requests (must be the owner key)");
std::process::exit(1);
}
};
let url = format!("{base}/api/auth/requests");
let req = ureq::get(&url)
.set("X-Signed-Request", &sign_action(kp, DatastoreAction::Access));
let resp = match req.call() {
Ok(r) => r,
Err(ureq::Error::Status(status, resp)) => {
let body = resp.into_string().unwrap_or_default();
eprintln!("Error ({status}): {body}");
std::process::exit(1);
}
Err(e) => {
eprintln!("Error: {e}");
std::process::exit(1);
}
};
let body: serde_json::Value = match resp.into_json() {
Ok(v) => v,
Err(e) => {
eprintln!("Error parsing response: {e}");
std::process::exit(1);
}
};
if let Some(err) = body.get("error").and_then(|v| v.as_str()) {
eprintln!("Error: {err}");
std::process::exit(1);
}
let requests = match body.as_array() {
Some(arr) => arr,
None => {
println!("(no pending requests)");
return;
}
};
if requests.is_empty() {
println!("(no pending requests)");
return;
}
println!("{:<64} {:<16} {}", "KEY", "NAME", "MESSAGE");
println!("{}", "-".repeat(100));
for req in requests {
let key = req.get("key").and_then(|v| v.as_str()).unwrap_or("?");
let name = req.get("name").and_then(|v| v.as_str()).unwrap_or("?");
let message = req.get("message").and_then(|v| v.as_str()).unwrap_or("");
let msg_truncated = if message.len() > 40 {
format!("{}...", &message[..37])
} else {
message.to_string()
};
println!("{key:<64} {name:<16} {msg_truncated}");
}
}
fn cmd_keys(base: &str, keypair: Option<&Keypair>) {
let kp = match keypair {
Some(kp) => kp,
None => {
eprintln!("Error: --key is required for keys (must be the owner key)");
std::process::exit(1);
}
};
let url = format!("{base}/api/auth/keys");
let req = ureq::get(&url)
.set("X-Signed-Request", &sign_action(kp, DatastoreAction::Access));
let resp = match req.call() {
Ok(r) => r,
Err(ureq::Error::Status(status, resp)) => {
let body = resp.into_string().unwrap_or_default();
eprintln!("Error ({status}): {body}");
std::process::exit(1);
}
Err(e) => {
eprintln!("Error: {e}");
std::process::exit(1);
}
};
let body: serde_json::Value = match resp.into_json() {
Ok(v) => v,
Err(e) => {
eprintln!("Error parsing response: {e}");
std::process::exit(1);
}
};
if let Some(err) = body.get("error").and_then(|v| v.as_str()) {
eprintln!("Error: {err}");
std::process::exit(1);
}
let keys = match body.as_array() {
Some(arr) => arr,
None => {
println!("(no authorized keys)");
return;
}
};
if keys.is_empty() {
println!("(no authorized keys)");
return;
}
println!("{:<64} {}", "KEY", "NAME");
println!("{}", "-".repeat(80));
for k in keys {
let key = k.get("key").and_then(|v| v.as_str()).unwrap_or("?");
let label = k.get("label").and_then(|v| v.as_str()).unwrap_or("");
println!("{key:<64} {label}");
}
}
fn cmd_deny(base: &str, key_input: &str, keypair: Option<&Keypair>) {
let kp = match keypair {
Some(kp) => kp,
None => {
eprintln!("Error: --key is required for deny (must be the owner key)");
std::process::exit(1);
}
};
let key_hex = if is_hex_key(key_input) {
key_input.to_string()
} else {
resolve_pending_request_key(base, key_input, kp)
};
let url = format!("{base}/api/auth/deny?key={key_hex}");
let req = ureq::post(&url)
.set("X-Signed-Request", &sign_action(kp, DatastoreAction::Access));
let resp = match req.send_bytes(&[]) {
Ok(r) => r,
Err(ureq::Error::Status(status, resp)) => {
let body = resp.into_string().unwrap_or_default();
eprintln!("Error ({status}): {body}");
std::process::exit(1);
}
Err(e) => {
eprintln!("Error: {e}");
std::process::exit(1);
}
};
let body: serde_json::Value = match resp.into_json() {
Ok(v) => v,
Err(e) => {
eprintln!("Error parsing response: {e}");
std::process::exit(1);
}
};
if body.get("ok").and_then(|v| v.as_bool()) == Some(true) {
println!("Denied {key_hex}");
} else if let Some(err) = body.get("error").and_then(|v| v.as_str()) {
eprintln!("Error: {err}");
std::process::exit(1);
}
}
fn url_encode(s: &str) -> String {
let mut result = String::with_capacity(s.len());
for b in s.bytes() {

249
crates/datastore/src/bin/store_node.rs
View file

@ -14,13 +14,15 @@ use serde::Deserialize;
use swactor::config::RuntimeConfig;
use swactor::runtime::Runtime;
use swactor_datastore::actors::{BlobStoreActor, DatastoreNode, MetadataActor};
use swactor_datastore::actors::{BlobStoreActor, DatastoreNode, GatewayActor, MetadataActor};
use swactor_datastore::api::start_api_server;
use swactor_datastore::messages::MetadataMsg;
use swactor_datastore::auth::{AccessControlList, AuthzEngine};
use swactor_datastore::messages::{GatewayMsg, MetadataMsg};
use swactor_datastore::metrics::DatastoreMetrics;
use swactor_datastore::storage::{FilesystemBackend, InMemoryBackend};
use swactor_datastore::DatastoreConfig;
use distribution::crypto::Keypair;
use distribution::types::NodeId;
#[derive(Parser)]
@ -53,6 +55,14 @@ struct Args {
/// Dissemination interval in ticks
#[arg(long)]
disseminate_interval: Option<u64>,
/// Enable auth (generates owner keypair if needed)
#[arg(long)]
auth: bool,
/// Directory for owner.key.json + acl.json (default: "auth")
#[arg(long, default_value = "auth")]
auth_dir: String,
}
#[derive(Deserialize, Default)]
@ -75,6 +85,94 @@ struct ResolvedConfig {
disseminate_interval: u64,
}
// ── Key file helpers ────────────────────────────────────────────────────────
fn hex_encode(bytes: &[u8]) -> String {
bytes.iter().map(|b| format!("{b:02x}")).collect()
}
fn hex_decode(hex: &str) -> Option<Vec<u8>> {
if hex.len() % 2 != 0 {
return None;
}
let mut bytes = Vec::with_capacity(hex.len() / 2);
for chunk in hex.as_bytes().chunks(2) {
let hi = hex_digit(chunk[0])?;
let lo = hex_digit(chunk[1])?;
bytes.push((hi << 4) | lo);
}
Some(bytes)
}
fn hex_digit(b: u8) -> Option<u8> {
match b {
b'0'..=b'9' => Some(b - b'0'),
b'a'..=b'f' => Some(b - b'a' + 10),
b'A'..=b'F' => Some(b - b'A' + 10),
_ => None,
}
}
fn load_or_generate_keypair(path: &std::path::Path) -> Keypair {
if path.exists() {
let data = std::fs::read_to_string(path).expect("failed to read key file");
let json: serde_json::Value = serde_json::from_str(&data).expect("invalid key file JSON");
let secret_hex = json
.get("secret_key")
.and_then(|v| v.as_str())
.expect("key file missing secret_key");
let secret_bytes = hex_decode(secret_hex).expect("invalid secret_key hex");
let secret: [u8; 32] = secret_bytes
.try_into()
.expect("secret_key must be 32 bytes");
Keypair::from_bytes(&secret)
} else {
let keypair = Keypair::generate();
let now = std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.unwrap()
.as_secs();
let json = serde_json::json!({
"version": 1,
"secret_key": hex_encode(&keypair.secret_bytes()),
"public_key": hex_encode(&keypair.node_id().0),
"created_at": format_timestamp(now),
});
if let Some(parent) = path.parent() {
std::fs::create_dir_all(parent).expect("failed to create key file directory");
}
std::fs::write(path, serde_json::to_string_pretty(&json).unwrap())
.expect("failed to write key file");
keypair
}
}
fn format_timestamp(secs: u64) -> String {
// Simple ISO-8601 UTC timestamp
let s = secs % 60;
let m = (secs / 60) % 60;
let h = (secs / 3600) % 24;
let days = secs / 86400;
// Days since epoch to Y-M-D (simplified)
let (y, mo, d) = days_to_ymd(days);
format!("{y:04}-{mo:02}-{d:02}T{h:02}:{m:02}:{s:02}Z")
}
fn days_to_ymd(mut days: u64) -> (u64, u64, u64) {
// Algorithm from http://howardhinnant.github.io/date_algorithms.html
days += 719468;
let era = days / 146097;
let doe = days - era * 146097;
let yoe = (doe - doe / 1460 + doe / 36524 - doe / 146096) / 365;
let y = yoe + era * 400;
let doy = doe - (365 * yoe + yoe / 4 - yoe / 100);
let mp = (5 * doy + 2) / 153;
let d = doy - (153 * mp + 2) / 5 + 1;
let m = if mp < 10 { mp + 3 } else { mp - 9 };
let y = if m <= 2 { y + 1 } else { y };
(y, m, d)
}
fn resolve_config(args: &Args) -> ResolvedConfig {
let file_cfg = match &args.config {
Some(path) => {
@ -101,10 +199,14 @@ fn main() {
let cfg = resolve_config(&args);
let stop = Arc::new(AtomicBool::new(false));
// Signal handler
// Signal handler — double Ctrl-C forces immediate exit
{
let stop = Arc::clone(&stop);
ctrlc::set_handler(move || {
if stop.load(Ordering::Relaxed) {
eprintln!("\nForced exit.");
std::process::exit(1);
}
stop.store(true, Ordering::Relaxed);
})
.expect("failed to set signal handler");
@ -131,25 +233,40 @@ fn main() {
});
rt.set_stats_hook(collector.clone());
// Generate node ID from random bytes
let node_id = {
let mut bytes = [0u8; 32];
for (i, b) in std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.unwrap()
.as_nanos()
.to_le_bytes()
.iter()
.enumerate()
{
bytes[i % 32] ^= *b;
}
// Mix in process id for uniqueness
let pid = std::process::id();
for (i, b) in pid.to_le_bytes().iter().enumerate() {
bytes[i + 16] ^= *b;
}
NodeId(bytes)
// Generate or load node identity
let (node_id, owner_keypair) = if args.auth {
let auth_dir = std::path::PathBuf::from(&args.auth_dir);
std::fs::create_dir_all(&auth_dir).expect("failed to create auth directory");
let key_path = auth_dir.join("owner.key.json");
let keypair = load_or_generate_keypair(&key_path);
let nid = keypair.node_id();
eprintln!(
"Auth enabled — owner key: {}",
hex_encode(&nid.0)
);
eprintln!("Key file: {}", key_path.display());
(nid, Some((keypair, auth_dir)))
} else {
let node_id = {
let mut bytes = [0u8; 32];
for (i, b) in std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.unwrap()
.as_nanos()
.to_le_bytes()
.iter()
.enumerate()
{
bytes[i % 32] ^= *b;
}
// Mix in process id for uniqueness
let pid = std::process::id();
for (i, b) in pid.to_le_bytes().iter().enumerate() {
bytes[i + 16] ^= *b;
}
NodeId(bytes)
};
(node_id, None)
};
// Datastore config
@ -190,9 +307,68 @@ fn main() {
.spawn(datastore_node)
.expect("failed to spawn DatastoreNode");
// Spawn GatewayActor if auth is enabled
let gateway_addr = if let Some((_, ref auth_dir)) = owner_keypair {
let acl_path = auth_dir.join("acl.json");
let acl = AccessControlList::load_or_create(&acl_path, node_id)
.expect("failed to load/create ACL");
let engine = AuthzEngine::new(acl);
let gateway = GatewayActor::new(engine, datastore_addr, Some(acl_path));
let addr = rt
.spawn(gateway)
.expect("failed to spawn GatewayActor");
Some(addr)
} else {
None
};
// Start runtime
let handle = rt.run().expect("failed to start runtime");
// Load persisted entries from storage
{
let inbox = handle
.runtime
.new_inbox::<swactor_datastore::DatastoreResponse>()
.expect("failed to create inbox");
let _ = handle.runtime.send_to(
blob_store_addr,
swactor_datastore::BlobStoreMsg::LoadAll {
reply_to: *inbox.addr(),
},
);
// Poll for response (up to 5 seconds)
let start = std::time::Instant::now();
let mut loaded = false;
while start.elapsed() < Duration::from_secs(5) {
if let Some(resp) = inbox.try_recv() {
match resp {
swactor_datastore::DatastoreResponse::LoadedAll { entries } => {
let n = entries.len();
let _ = handle.runtime.send_to(
metadata_addr,
swactor_datastore::MetadataMsg::BulkLoad { entries },
);
if n > 0 {
eprintln!("Loaded {n} entries from storage");
}
loaded = true;
}
swactor_datastore::DatastoreResponse::Error { reason } => {
eprintln!("Warning: failed to load entries: {reason}");
loaded = true;
}
_ => {}
}
break;
}
thread::sleep(Duration::from_millis(1));
}
if !loaded {
eprintln!("Warning: timed out loading entries from storage");
}
}
// Create datastore metrics
let node_hex: String = node_id.0.iter().map(|b| format!("{b:02x}")).collect();
let metrics = Arc::new(DatastoreMetrics::new());
@ -209,20 +385,28 @@ fn main() {
datastore_addr,
metadata_addr,
blob_store_addr,
gateway_addr,
cfg.port,
Arc::clone(&metrics),
);
eprintln!("Node {} started", &node_hex[..8]);
eprintln!("API at http://0.0.0.0:{}", cfg.port);
eprintln!("──────────────────────────────────────");
eprintln!(" swactor-store node {}", &node_hex[..16]);
eprintln!(" API: http://0.0.0.0:{}", cfg.port);
if let Some(port) = cfg.dashboard_port {
eprintln!("Dashboard at http://0.0.0.0:{port}");
eprintln!(" Dashboard: http://0.0.0.0:{port}");
}
if cfg.storage_path.is_some() {
eprintln!("Storage: {}", cfg.storage_path.as_ref().unwrap());
eprintln!(" Storage: {} (filesystem)", cfg.storage_path.as_ref().unwrap());
} else {
eprintln!("Storage: in-memory");
eprintln!(" Storage: in-memory");
}
if owner_keypair.is_some() {
eprintln!(" Auth: enabled (owner {})", &node_hex[..16]);
} else {
eprintln!(" Auth: disabled");
}
eprintln!("──────────────────────────────────────");
// Main loop
let mut round: u64 = 0;
@ -233,6 +417,10 @@ fn main() {
let _ = handle
.runtime
.send_to(metadata_addr, MetadataMsg::GcTick);
if let Some(gw) = gateway_addr {
let _ = handle.runtime.send_to(gw, GatewayMsg::NonceGcTick);
}
}
if round % cfg.disseminate_interval == 0 {
@ -250,5 +438,10 @@ fn main() {
if let Some(d) = dash {
d.shutdown();
}
handle.join();
// Brief pause for threads to flush I/O, then exit.
// No join — cargo run already died from SIGINT so there's
// no parent waiting on us; just exit cleanly.
thread::sleep(Duration::from_millis(50));
eprintln!("Shutdown complete.");
std::process::exit(0);
}

BIN
crates/datastore/src/crypto_wasm.wasm Executable file
View file

Binary file not shown.

1
crates/datastore/src/lib.rs
View file

@ -3,6 +3,7 @@ pub mod messages;
pub mod chunking;
pub mod storage;
pub mod actors;
pub mod auth;
pub mod cli;
pub mod metrics;
#[cfg(feature = "node")]

90
crates/datastore/src/messages.rs
View file

@ -16,6 +16,7 @@ use swactor::transport::NetworkMessage;
use distribution::types::NodeId;
use crate::auth::{AccessRequestInfo, AuthorizedKeyInfo, DeniedReason, SignedRequest};
use crate::types::{ContentHash, ObjectEntry, ObjectManifest};
// ═══════════════════════════════════════════════════════════════════════════
@ -185,6 +186,12 @@ pub enum BlobStoreMsg {
hash: ContentHash,
reply_to: ActorAddress,
},
/// Write an entry to disk (fire-and-forget).
WriteEntry { entry: ObjectEntry },
/// Delete an entry from disk (fire-and-forget).
DeleteEntry { hash: ContentHash },
/// Load all persisted entries + their manifests at startup.
LoadAll { reply_to: ActorAddress },
}
// ─── MetadataMsg ────────────────────────────────────────────────────────────
@ -235,6 +242,10 @@ pub enum MetadataMsg {
DisseminateTick,
/// Periodic garbage collection tick.
GcTick,
/// Bulk-load entries and manifests from storage at startup.
BulkLoad {
entries: Vec<(ObjectEntry, ObjectManifest)>,
},
}
// ─── TransferMsg ────────────────────────────────────────────────────────────
@ -365,8 +376,87 @@ pub enum DatastoreResponse {
NotFound,
/// An error occurred.
Error { reason: String },
/// Request was denied by the auth layer.
Denied { reason: DeniedReason },
/// Boolean response (e.g. HasChunk).
Bool(bool),
/// List of chunk hashes.
ChunkList { hashes: Vec<ContentHash> },
/// All persisted entries loaded at startup.
LoadedAll {
entries: Vec<(ObjectEntry, ObjectManifest)>,
},
/// List of pending access requests.
AccessRequests {
requests: Vec<AccessRequestInfo>,
},
/// List of authorized keys with labels.
AuthorizedKeys {
keys: Vec<AuthorizedKeyInfo>,
},
}
// ─── GatewayMsg ────────────────────────────────────────────────────────────
/// Messages handled by the `GatewayActor` — the auth enforcement point.
#[derive(Debug, Clone)]
pub enum GatewayMsg {
/// Auth Path 2: verify a signed request and dispatch if allowed.
HandleSignedRequest {
request: SignedRequest,
reply_to: ActorAddress,
},
/// Auth Path 1: check whether a node is authorized for connection.
CheckConnection {
node_id: NodeId,
reply_to: ActorAddress,
},
/// Owner-only: grant access to a key.
Grant {
requester: NodeId,
key: NodeId,
label: Option<String>,
reply_to: ActorAddress,
},
/// Owner-only: revoke access from a key.
Revoke {
requester: NodeId,
key: NodeId,
reply_to: ActorAddress,
},
/// Auth-only check: verify a signed request without forwarding the action.
Authorize {
request: SignedRequest,
reply_to: ActorAddress,
},
/// Verify signature only (no ACL check) — for access request submissions.
VerifySignature {
request: SignedRequest,
reply_to: ActorAddress,
},
/// Submit an access request from a browser user.
SubmitAccessRequest {
key: NodeId,
name: String,
message: String,
reply_to: ActorAddress,
},
/// List pending access requests (owner-only).
ListAccessRequests {
requester: NodeId,
reply_to: ActorAddress,
},
/// Deny (dismiss) a pending access request (owner-only).
DenyAccessRequest {
requester: NodeId,
key: NodeId,
reply_to: ActorAddress,
},
/// List all authorized keys with labels (owner-only).
ListAuthorizedKeys {
requester: NodeId,
reply_to: ActorAddress,
},
/// Periodic nonce garbage collection tick.
NonceGcTick,
}

22
crates/datastore/src/storage/in_memory.rs
View file

@ -2,7 +2,7 @@
use std::collections::HashMap;
use crate::types::{ContentHash, ObjectManifest};
use crate::types::{ContentHash, ObjectEntry, ObjectManifest};
use super::StorageBackend;
@ -13,6 +13,7 @@ use super::StorageBackend;
pub struct InMemoryBackend {
chunks: HashMap<ContentHash, Vec<u8>>,
manifests: HashMap<ContentHash, ObjectManifest>,
entries: HashMap<ContentHash, ObjectEntry>,
}
impl InMemoryBackend {
@ -20,6 +21,7 @@ impl InMemoryBackend {
Self {
chunks: HashMap::new(),
manifests: HashMap::new(),
entries: HashMap::new(),
}
}
}
@ -69,4 +71,22 @@ impl StorageBackend for InMemoryBackend {
self.manifests.remove(content_hash);
Ok(())
}
fn write_entry(&mut self, entry: &ObjectEntry) -> Result<(), std::io::Error> {
self.entries.insert(entry.content_hash, entry.clone());
Ok(())
}
fn read_entry(&self, hash: &ContentHash) -> Result<Option<ObjectEntry>, std::io::Error> {
Ok(self.entries.get(hash).cloned())
}
fn delete_entry(&mut self, hash: &ContentHash) -> Result<(), std::io::Error> {
self.entries.remove(hash);
Ok(())
}
fn list_entries(&self) -> Result<Vec<ObjectEntry>, std::io::Error> {
Ok(self.entries.values().cloned().collect())
}
}

76
crates/datastore/src/storage/mod.rs
View file

@ -10,7 +10,7 @@ use std::fs;
use std::io::Write;
use std::path::PathBuf;
use crate::types::{ContentHash, ObjectManifest};
use crate::types::{ContentHash, ObjectEntry, ObjectManifest};
pub use in_memory::InMemoryBackend;
@ -24,6 +24,10 @@ pub trait StorageBackend: Send {
fn write_manifest(&mut self, manifest: &ObjectManifest) -> Result<(), std::io::Error>;
fn read_manifest(&self, content_hash: &ContentHash) -> Result<Option<ObjectManifest>, std::io::Error>;
fn delete_manifest(&mut self, content_hash: &ContentHash) -> Result<(), std::io::Error>;
fn write_entry(&mut self, entry: &ObjectEntry) -> Result<(), std::io::Error>;
fn read_entry(&self, hash: &ContentHash) -> Result<Option<ObjectEntry>, std::io::Error>;
fn delete_entry(&mut self, hash: &ContentHash) -> Result<(), std::io::Error>;
fn list_entries(&self) -> Result<Vec<ObjectEntry>, std::io::Error>;
}
/// Filesystem-backed storage with 2-level directory sharding.
@ -32,7 +36,8 @@ pub trait StorageBackend: Send {
/// ```text
/// {root}/
/// ├── chunks/{hex[0..2]}/{hex[2..4]}/{full_hex_hash}
/// └── manifests/{hex[0..2]}/{hex[2..4]}/{full_hex_hash}
/// ├── manifests/{hex[0..2]}/{hex[2..4]}/{full_hex_hash}
/// └── entries/{hex[0..2]}/{hex[2..4]}/{full_hex_hash}
/// ```
pub struct FilesystemBackend {
root: PathBuf,
@ -67,6 +72,15 @@ impl FilesystemBackend {
.join(&hex)
}
fn entry_path(&self, hash: &ContentHash) -> PathBuf {
let hex = hash.to_hex();
self.root
.join("entries")
.join(&hex[..2])
.join(&hex[2..4])
.join(&hex)
}
fn scan_chunks(&mut self) {
let chunks_dir = self.root.join("chunks");
if !chunks_dir.exists() {
@ -171,5 +185,63 @@ impl StorageBackend for FilesystemBackend {
Err(e) => Err(e),
}
}
fn write_entry(&mut self, entry: &ObjectEntry) -> Result<(), std::io::Error> {
let path = self.entry_path(&entry.content_hash);
let data = serde_json::to_vec(entry)
.map_err(|e| std::io::Error::new(std::io::ErrorKind::InvalidData, e))?;
Self::write_and_sync(&path, &data)
}
fn read_entry(&self, hash: &ContentHash) -> Result<Option<ObjectEntry>, std::io::Error> {
let path = self.entry_path(hash);
match fs::read(&path) {
Ok(data) => {
let entry: ObjectEntry = serde_json::from_slice(&data)
.map_err(|e| std::io::Error::new(std::io::ErrorKind::InvalidData, e))?;
Ok(Some(entry))
}
Err(e) if e.kind() == std::io::ErrorKind::NotFound => Ok(None),
Err(e) => Err(e),
}
}
fn delete_entry(&mut self, hash: &ContentHash) -> Result<(), std::io::Error> {
let path = self.entry_path(hash);
match fs::remove_file(&path) {
Ok(()) => Ok(()),
Err(e) if e.kind() == std::io::ErrorKind::NotFound => Ok(()),
Err(e) => Err(e),
}
}
fn list_entries(&self) -> Result<Vec<ObjectEntry>, std::io::Error> {
let entries_dir = self.root.join("entries");
if !entries_dir.exists() {
return Ok(Vec::new());
}
let mut entries = Vec::new();
let level1 = fs::read_dir(&entries_dir)?;
for d1 in level1.flatten() {
let Ok(level2) = fs::read_dir(d1.path()) else {
continue;
};
for d2 in level2.flatten() {
let Ok(files) = fs::read_dir(d2.path()) else {
continue;
};
for file in files.flatten() {
let data = fs::read(file.path())?;
match serde_json::from_slice::<ObjectEntry>(&data) {
Ok(entry) => entries.push(entry),
Err(e) => {
eprintln!("warning: skipping corrupt entry file {}: {e}", file.path().display());
}
}
}
}
}
Ok(entries)
}
}

100
crates/datastore/src/types.rs
View file

@ -4,111 +4,13 @@
//! `blake3(blob_bytes)`. Names are optional metadata, not keys.
use std::collections::BTreeMap;
use std::fmt;
use std::path::PathBuf;
use serde::{Deserialize, Serialize};
use distribution::types::NodeId;
// ─── ContentHash ────────────────────────────────────────────────────────────
/// A blake3 content hash (32 bytes).
///
/// The primary identifier for blobs and the DHT key. Mirrors the `NodeId`
/// pattern from `distribution::types` — XOR distance for DHT routing, compact
/// Debug/Display for logging.
#[derive(Clone, Copy, PartialEq, Eq, Hash, Serialize, Deserialize)]
pub struct ContentHash(pub [u8; 32]);
impl ContentHash {
/// Compute the blake3 hash of the given data.
pub fn of(data: &[u8]) -> Self {
let hash = blake3::hash(data);
ContentHash(*hash.as_bytes())
}
/// XOR distance between two content hashes (Kademlia metric).
pub fn xor_distance(&self, other: &ContentHash) -> [u8; 32] {
let mut out = [0u8; 32];
for i in 0..32 {
out[i] = self.0[i] ^ other.0[i];
}
out
}
/// Number of leading zero bits in the XOR distance to `other`.
/// Returns 0..=256. Used to select the k-bucket index in the metadata DHT.
pub fn xor_leading_zeros(&self, other: &ContentHash) -> u32 {
let dist = self.xor_distance(other);
let mut zeros = 0u32;
for byte in dist {
if byte == 0 {
zeros += 8;
} else {
zeros += byte.leading_zeros();
break;
}
}
zeros
}
/// Parse a 64-character hex string into a ContentHash.
/// Returns `None` if the string is not exactly 64 hex characters.
pub fn from_hex(hex: &str) -> Option<Self> {
if hex.len() != 64 {
return None;
}
let mut bytes = [0u8; 32];
for (i, chunk) in hex.as_bytes().chunks(2).enumerate() {
let hi = hex_digit(chunk[0])?;
let lo = hex_digit(chunk[1])?;
bytes[i] = (hi << 4) | lo;
}
Some(ContentHash(bytes))
}
/// Encode as lowercase hex string.
pub fn to_hex(&self) -> String {
let mut s = String::with_capacity(64);
for b in &self.0 {
use fmt::Write;
write!(s, "{:02x}", b).unwrap();
}
s
}
/// The zero hash (all zeroes). Used as a sentinel.
pub const ZERO: ContentHash = ContentHash([0u8; 32]);
}
impl fmt::Debug for ContentHash {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
write!(f, "Hash(")?;
for b in &self.0[..4] {
write!(f, "{:02x}", b)?;
}
write!(f, "\u{2026})")
}
}
impl fmt::Display for ContentHash {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
for b in &self.0[..8] {
write!(f, "{:02x}", b)?;
}
write!(f, "\u{2026}")
}
}
fn hex_digit(b: u8) -> Option<u8> {
match b {
b'0'..=b'9' => Some(b - b'0'),
b'a'..=b'f' => Some(b - b'a' + 10),
b'A'..=b'F' => Some(b - b'A' + 10),
_ => None,
}
}
pub use shared_types::ContentHash;
// ─── ObjectEntry ────────────────────────────────────────────────────────────

631
crates/datastore/src/ui_html.rs
View file

@ -116,11 +116,22 @@ pub const DATASTORE_UI_HTML: &str = r##"<!DOCTYPE html>
.chunk-list { margin-top: 8px; }
.chunk-item { color: #888; font-size: 11px; padding: 2px 0; }
.auth-info { display: none; font-size: 11px; color: #888; margin-left: 12px; }
.auth-info .device-key { color: #6366f1; cursor: text; user-select: all; font-family: monospace; font-size: 10px; }
.auth-banner {
display: none; padding: 10px 16px; font-size: 12px;
background: #2a1a1a; border: 1px solid #f4433666; border-radius: 4px;
color: #f88; margin-bottom: 16px;
}
.auth-banner.show { display: block; }
@media (max-width: 640px) {
.upload-row { flex-direction: column; align-items: stretch; }
input[type="text"] { width: 100%; }
.header { flex-direction: column; align-items: flex-start; gap: 4px; }
.header .node-id { margin-left: 0; }
.auth-info { margin-left: 0; }
.actions-cell { display: flex; gap: 4px; justify-content: flex-end; }
}
</style>
@ -131,10 +142,25 @@ pub const DATASTORE_UI_HTML: &str = r##"<!DOCTYPE html>
<div style="display:flex;align-items:center;flex-wrap:wrap;">
<h1>swactor-store</h1>
<span class="node-id" id="nodeId">connecting...</span>
<span class="auth-info" id="authInfo">| device key: <span class="device-key" id="deviceKey"></span></span>
</div>
</div>
<div class="container">
<div class="auth-banner" id="authBanner">
<div id="authRequestForm">
<p style="margin-bottom:8px;">You are not authorized. Request access from the operator:</p>
<div style="display:flex;gap:8px;flex-wrap:wrap;align-items:center;">
<input type="text" id="reqName" placeholder="Your name" maxlength="64" style="width:160px;" />
<input type="text" id="reqMessage" placeholder="Why do you need access? (optional)" maxlength="256" style="width:280px;" />
<button class="primary" onclick="submitAccessRequest()">Request Access</button>
</div>
</div>
<div id="authPending" style="display:none">
<p>Access requested — waiting for operator approval. This page will refresh automatically.</p>
</div>
</div>
<!-- Upload panel -->
<div class="panel">
<h2>Upload</h2>
@ -166,6 +192,185 @@ pub const DATASTORE_UI_HTML: &str = r##"<!DOCTYPE html>
<script>
const $ = id => document.getElementById(id);
// ─── WASM Ed25519 crypto ────────────────────────────────────────
let authEnabled = false;
let deviceSeed = null;
let pubKeyBytes = null;
let wasmExports = null, bufPtr = 0;
function toHex(buf) {
return Array.from(new Uint8Array(buf)).map(b => b.toString(16).padStart(2, '0')).join('');
}
function hexToBytes(hex) {
const bytes = new Uint8Array(hex.length / 2);
for (let i = 0; i < hex.length; i += 2) bytes[i / 2] = parseInt(hex.substr(i, 2), 16);
return bytes;
}
function base64urlToBytes(b64) {
const std = b64.replace(/-/g, '+').replace(/_/g, '/');
const bin = atob(std);
const bytes = new Uint8Array(bin.length);
for (let i = 0; i < bin.length; i++) bytes[i] = bin.charCodeAt(i);
return bytes;
}
async function initCrypto() {
const { instance } = await WebAssembly.instantiate(
await (await fetch('/crypto.wasm')).arrayBuffer()
);
wasmExports = instance.exports;
bufPtr = wasmExports.buffer_ptr();
}
function derivePublicKey(seed) {
const mem = new Uint8Array(wasmExports.memory.buffer);
mem.set(seed, bufPtr);
wasmExports.get_public_key();
return new Uint8Array(wasmExports.memory.buffer, bufPtr + 32, 32).slice();
}
function signBytes(message, seed) {
const mem = new Uint8Array(wasmExports.memory.buffer);
mem.set(seed, bufPtr);
mem.set(message, bufPtr + 128);
wasmExports.ed25519_sign(message.length);
return new Uint8Array(wasmExports.memory.buffer, bufPtr + 64, 64).slice();
}
async function initKeys() {
let seed;
const storedSeed = localStorage.getItem('deviceKeySeed');
if (storedSeed) {
seed = hexToBytes(storedSeed);
} else {
const oldJwk = localStorage.getItem('deviceKey');
if (oldJwk) {
try {
const jwk = JSON.parse(oldJwk);
if (jwk.d) {
seed = base64urlToBytes(jwk.d);
localStorage.removeItem('deviceKey');
}
} catch(e) {}
}
if (!seed) {
seed = new Uint8Array(32);
crypto.getRandomValues(seed);
}
localStorage.setItem('deviceKeySeed', toHex(seed));
}
deviceSeed = seed;
pubKeyBytes = derivePublicKey(seed);
}
async function authFetch(url, opts) {
if (!authEnabled || !deviceSeed) return fetch(url, opts);
const nonce = Array.from(crypto.getRandomValues(new Uint8Array(16)));
const payload = { action: "Access", timestamp: Math.floor(Date.now() / 1000), nonce: nonce };
const payloadBytes = new TextEncoder().encode(JSON.stringify(payload));
const sigBytes = signBytes(payloadBytes, deviceSeed);
const header = JSON.stringify({
payload: payload,
public_key: Array.from(pubKeyBytes),
signature: Array.from(sigBytes)
});
opts = opts || {};
opts.headers = Object.assign({}, opts.headers || {}, { 'X-Signed-Request': header });
return fetch(url, opts);
}
async function detectAuth() {
try {
const r = await fetch('/api/list');
if (r.status === 401) {
authEnabled = true;
await initCrypto();
await initKeys();
$('deviceKey').textContent = toHex(pubKeyBytes);
$('authInfo').style.display = 'inline';
}
return r;
} catch(e) {
return null;
}
}
let pollInterval = null;
function showAuthBanner() {
$('authBanner').classList.add('show');
const pendingKey = localStorage.getItem('accessRequestPending');
const pendingName = localStorage.getItem('accessRequestName');
if (pendingKey && pubKeyBytes && pendingKey === toHex(pubKeyBytes) && pendingName) {
// Re-submit to ensure the server still has our request (survives node restart)
resubmitAccessRequest(pendingName);
}
}
async function resubmitAccessRequest(name) {
try {
const r = await authFetch('/api/auth/request', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ name, message: '' })
});
if (r.ok) {
$('authRequestForm').style.display = 'none';
$('authPending').style.display = 'block';
startPolling();
return;
}
} catch(e) { /* fall through */ }
// Failed — clear stale state, show form
localStorage.removeItem('accessRequestPending');
localStorage.removeItem('accessRequestName');
}
async function submitAccessRequest() {
const name = $('reqName').value.trim();
if (!name) { toast('Name is required', 'error'); return; }
const message = $('reqMessage').value.trim();
try {
const r = await authFetch('/api/auth/request', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ name, message })
});
if (!r.ok) {
const j = await r.json().catch(() => ({}));
throw new Error(j.error || r.statusText);
}
$('authRequestForm').style.display = 'none';
$('authPending').style.display = 'block';
localStorage.setItem('accessRequestPending', toHex(pubKeyBytes));
localStorage.setItem('accessRequestName', name);
startPolling();
} catch(e) {
toast('Request failed: ' + e.message, 'error');
}
}
function startPolling() {
if (pollInterval) return;
pollInterval = setInterval(async () => {
try {
const r = await authFetch('/api/list');
if (r.ok) {
clearInterval(pollInterval);
pollInterval = null;
localStorage.removeItem('accessRequestPending');
localStorage.removeItem('accessRequestName');
$('authBanner').classList.remove('show');
const j = await r.json();
renderTable(j.entries || []);
}
} catch(e) { /* keep polling */ }
}, 5000);
}
// ────────────────────────────────────────────────────────────────
function toast(msg, type) {
const t = $('toast');
t.textContent = msg;
@ -193,7 +398,8 @@ async function fetchStatus() {
async function refreshList() {
try {
const r = await fetch('/api/list');
const r = await authFetch('/api/list');
if (r.status === 401 || r.status === 403) { showAuthBanner(); return; }
const j = await r.json();
renderTable(j.entries || []);
} catch(e) {
@ -238,7 +444,7 @@ async function upload() {
try {
let url = '/api/put';
if (name) url += '?name=' + encodeURIComponent(name);
const r = await fetch(url, { method: 'POST', body: file });
const r = await authFetch(url, { method: 'POST', body: file });
if (!r.ok) throw new Error((await r.json()).error || r.statusText);
const j = await r.json();
toast('uploaded ' + j.content_hash.substring(0, 12), 'success');
@ -255,7 +461,7 @@ async function upload() {
async function download(hash, filename) {
try {
const r = await fetch('/api/data?hash=' + hash);
const r = await authFetch('/api/data?hash=' + hash);
if (!r.ok) throw new Error('not found');
const blob = await r.blob();
const a = document.createElement('a');
@ -271,7 +477,7 @@ async function download(hash, filename) {
async function del(hash) {
if (!confirm('Delete ' + hash.substring(0, 16) + '?')) return;
try {
const r = await fetch('/api/delete?hash=' + hash, { method: 'POST' });
const r = await authFetch('/api/delete?hash=' + hash, { method: 'POST' });
if (!r.ok) throw new Error((await r.json()).error || r.statusText);
toast('deleted', 'success');
refreshList();
@ -282,7 +488,7 @@ async function del(hash) {
async function showDetail(hash) {
try {
const r = await fetch('/api/get?hash=' + hash);
const r = await authFetch('/api/get?hash=' + hash);
if (!r.ok) throw new Error('not found');
const j = await r.json();
const e = j.entry;
@ -318,9 +524,422 @@ function row(label, value) {
function closeModal() { $('modal').classList.remove('open'); }
document.addEventListener('keydown', e => { if (e.key === 'Escape') closeModal(); });
// Init
$('fileInput').addEventListener('change', function() {
if (!$('nameInput').value.trim() && this.files.length > 0) {
$('nameInput').value = this.files[0].name;
}
});
fetchStatus();
(async function() {
await detectAuth();
refreshList();
})();
</script>
</body>
</html>"##;
pub const DATASTORE_ADMIN_HTML: &str = r##"<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>swactor-store admin</title>
<style>
* { margin: 0; padding: 0; box-sizing: border-box; }
body { font-family: 'Menlo', 'Consolas', 'Monaco', monospace; background: #0f1117; color: #e0e0e0; font-size: 13px; }
.header {
display: flex; align-items: center; justify-content: space-between;
padding: 12px 20px; background: #161822; border-bottom: 1px solid #2a2d3e;
}
.header h1 { font-size: 16px; font-weight: 600; color: #fff; }
.header .node-id { font-size: 11px; color: #888; margin-left: 12px; }
.container { max-width: 960px; margin: 0 auto; padding: 16px; }
.panel {
background: #161822; border: 1px solid #2a2d3e; border-radius: 6px;
padding: 16px; margin-bottom: 16px;
}
.panel h2 {
font-size: 12px; color: #888; text-transform: uppercase;
letter-spacing: 1px; margin-bottom: 12px;
display: flex; align-items: center; gap: 10px;
}
input[type="file"] {
background: #1c1f2e; color: #e0e0e0; border: 1px solid #2a2d3e;
border-radius: 4px; padding: 8px; font-family: inherit; font-size: 13px;
min-height: 44px; cursor: pointer;
}
input[type="file"]::file-selector-button {
background: #1e2030; color: #e0e0e0; border: 1px solid #2a2d3e;
border-radius: 4px; padding: 6px 12px; font-family: inherit;
font-size: 12px; cursor: pointer; margin-right: 8px;
}
input[type="file"]::file-selector-button:hover { border-color: #6366f1; }
input[type="text"] {
background: #1c1f2e; color: #e0e0e0; border: 1px solid #2a2d3e;
border-radius: 4px; padding: 8px 12px; font-family: inherit;
font-size: 13px; min-height: 44px;
}
input[type="text"]:focus { outline: none; border-color: #6366f1; }
button {
background: #1e2030; color: #e0e0e0; border: 1px solid #2a2d3e;
border-radius: 4px; padding: 8px 16px; font-family: inherit;
font-size: 13px; cursor: pointer; min-height: 44px;
transition: border-color 0.15s;
}
button:hover { border-color: #6366f1; color: #fff; }
button:disabled { opacity: 0.4; cursor: default; }
button.danger:hover { border-color: #f44336; }
button.primary { background: #6366f1; border-color: #6366f1; color: #fff; font-weight: 600; }
button.primary:hover { background: #5558e6; }
button.small { min-height: 32px; padding: 4px 10px; font-size: 11px; }
.toast {
position: fixed; bottom: 20px; right: 20px; padding: 10px 16px;
border-radius: 4px; font-size: 12px; z-index: 100; opacity: 0;
transition: opacity 0.3s; pointer-events: none;
}
.toast.show { opacity: 1; }
.toast.success { background: #4caf50; color: #fff; }
.toast.error { background: #f44336; color: #fff; }
table { width: 100%; border-collapse: collapse; }
th {
text-align: left; font-size: 10px; color: #888;
text-transform: uppercase; letter-spacing: 1px;
padding: 6px 8px; border-bottom: 1px solid #2a2d3e;
}
td {
padding: 8px; border-bottom: 1px solid #1c1f2e;
font-size: 13px; vertical-align: middle;
}
tr:hover td { background: #1c1f2e; }
.hash-cell { color: #6366f1; font-size: 12px; cursor: default; }
.actions-cell { white-space: nowrap; text-align: right; }
.actions-cell button { min-height: 32px; padding: 4px 10px; font-size: 11px; }
.empty-state {
text-align: center; color: #555; padding: 32px; font-size: 14px;
}
.status-badge { font-size: 11px; margin-left: 8px; }
.status-badge.ok { color: #4caf50; }
.status-badge.error { color: #f44336; }
@media (max-width: 640px) {
.header { flex-direction: column; align-items: flex-start; gap: 4px; }
.header .node-id { margin-left: 0; }
.actions-cell { display: flex; gap: 4px; justify-content: flex-end; }
}
</style>
</head>
<body>
<div class="header">
<div style="display:flex;align-items:center;flex-wrap:wrap;">
<h1>swactor-store admin</h1>
<span class="node-id" id="nodeId">connecting...</span>
</div>
</div>
<div class="container">
<!-- Key Upload Panel -->
<div class="panel" id="keyPanel">
<h2>Owner Authentication</h2>
<p style="color:#888;font-size:12px;margin-bottom:10px;">
Upload your owner <code style="color:#6366f1;">key.json</code> file to authenticate as the node owner.
</p>
<div style="display:flex;gap:10px;align-items:center;flex-wrap:wrap;">
<input type="file" id="keyFileInput" accept=".json" />
<button class="primary" onclick="loadOwnerKey()">Authenticate</button>
<span id="keyStatus"></span>
</div>
</div>
<!-- Admin Content (hidden until authenticated) -->
<div id="adminContent" style="display:none">
<div class="panel">
<h2>Pending Access Requests <button class="small" onclick="refreshAll()">Refresh</button></h2>
<div id="requestsTable"></div>
</div>
<div class="panel">
<h2>Authorized Keys</h2>
<div id="keysTable"></div>
</div>
<div class="panel">
<h2>Grant Key Manually</h2>
<p style="color:#888;font-size:12px;margin-bottom:10px;">
Authorize a public key directly, even without a pending request.
</p>
<div style="display:flex;gap:8px;flex-wrap:wrap;align-items:center;">
<input type="text" id="manualKeyInput" placeholder="Public key (64 hex chars)" style="width:320px;" />
<input type="text" id="manualNameInput" placeholder="Name (optional)" style="width:160px;" />
<button class="primary small" onclick="manualGrant()">Grant</button>
</div>
</div>
</div>
</div>
<div class="toast" id="toast"></div>
<script>
const $ = id => document.getElementById(id);
let ownerSeed = null;
let ownerPubBytes = null;
let wasmExports = null, bufPtr = 0;
function toHex(buf) {
return Array.from(new Uint8Array(buf)).map(b => b.toString(16).padStart(2, '0')).join('');
}
function hexToBytes(hex) {
const bytes = new Uint8Array(hex.length / 2);
for (let i = 0; i < hex.length; i += 2) bytes[i / 2] = parseInt(hex.substr(i, 2), 16);
return bytes;
}
function escHtml(s) { const d = document.createElement('div'); d.textContent = s; return d.innerHTML; }
function toast(msg, type) {
const t = $('toast');
t.textContent = msg;
t.className = 'toast show ' + type;
setTimeout(() => t.className = 'toast', 2500);
}
async function initCrypto() {
const { instance } = await WebAssembly.instantiate(
await (await fetch('/crypto.wasm')).arrayBuffer()
);
wasmExports = instance.exports;
bufPtr = wasmExports.buffer_ptr();
}
function derivePublicKey(seed) {
const mem = new Uint8Array(wasmExports.memory.buffer);
mem.set(seed, bufPtr);
wasmExports.get_public_key();
return new Uint8Array(wasmExports.memory.buffer, bufPtr + 32, 32).slice();
}
function signBytes(message, seed) {
const mem = new Uint8Array(wasmExports.memory.buffer);
mem.set(seed, bufPtr);
mem.set(message, bufPtr + 128);
wasmExports.ed25519_sign(message.length);
return new Uint8Array(wasmExports.memory.buffer, bufPtr + 64, 64).slice();
}
let cryptoReady = initCrypto();
async function fetchStatus() {
try {
const r = await fetch('/api/status');
const j = await r.json();
$('nodeId').textContent = j.node_id.substring(0, 16) + '...';
$('nodeId').title = j.node_id;
} catch(e) {
$('nodeId').textContent = 'offline';
}
}
async function loadOwnerKey() {
const fileInput = $('keyFileInput');
const status = $('keyStatus');
if (!fileInput.files.length) {
status.innerHTML = '<span class="status-badge error">Select a key.json file</span>';
return;
}
try {
await cryptoReady;
const text = await fileInput.files[0].text();
const json = JSON.parse(text);
const secretHex = json.secret_key;
const publicHex = json.public_key;
if (!secretHex || !publicHex) throw new Error('Missing secret_key or public_key');
ownerSeed = hexToBytes(secretHex);
ownerPubBytes = hexToBytes(publicHex);
const derived = toHex(derivePublicKey(ownerSeed));
if (derived !== publicHex) throw new Error('Key mismatch: derived public key does not match');
// Test call to verify this is the owner key
const r = await ownerAuthFetch('/api/auth/requests');
if (r.ok) {
status.innerHTML = '<span class="status-badge ok">Authenticated</span>';
$('adminContent').style.display = 'block';
refreshAll();
} else {
ownerSeed = null;
ownerPubBytes = null;
status.innerHTML = '<span class="status-badge error">Not the owner key (403)</span>';
}
} catch(e) {
ownerSeed = null;
ownerPubBytes = null;
status.innerHTML = '<span class="status-badge error">Error: ' + escHtml(e.message) + '</span>';
}
}
async function ownerAuthFetch(url, opts) {
if (!ownerSeed) return fetch(url, opts);
const nonce = Array.from(crypto.getRandomValues(new Uint8Array(16)));
const payload = { action: "Access", timestamp: Math.floor(Date.now() / 1000), nonce: nonce };
const payloadBytes = new TextEncoder().encode(JSON.stringify(payload));
const sigBytes = signBytes(payloadBytes, ownerSeed);
const header = JSON.stringify({
payload: payload,
public_key: Array.from(ownerPubBytes),
signature: Array.from(sigBytes)
});
opts = opts || {};
opts.headers = Object.assign({}, opts.headers || {}, { 'X-Signed-Request': header });
return fetch(url, opts);
}
function addDisambiguation(items, nameField) {
const counts = {};
for (const item of items) {
const name = item[nameField] || '';
counts[name] = (counts[name] || 0) + 1;
}
return items.map(item => {
const name = item[nameField] || '';
if (counts[name] > 1) {
const prefix = item.key.substring(0, 8);
return { ...item, displayName: name + ' (' + prefix + ')' };
}
return { ...item, displayName: name };
});
}
async function refreshAll() {
// Fetch requests
try {
const r = await ownerAuthFetch('/api/auth/requests');
if (!r.ok) { $('requestsTable').innerHTML = '<div class="empty-state">failed to load</div>'; return; }
const requests = await r.json();
renderRequests(requests);
} catch(e) {
$('requestsTable').innerHTML = '<div class="empty-state">failed to load</div>';
}
// Fetch keys
try {
const r = await ownerAuthFetch('/api/auth/keys');
if (!r.ok) { $('keysTable').innerHTML = '<div class="empty-state">failed to load</div>'; return; }
const keys = await r.json();
renderKeys(keys);
} catch(e) {
$('keysTable').innerHTML = '<div class="empty-state">failed to load</div>';
}
}
function renderRequests(requests) {
if (!requests || requests.length === 0) {
$('requestsTable').innerHTML = '<div class="empty-state">no pending requests</div>';
return;
}
const items = addDisambiguation(requests, 'name');
let html = '<table><thead><tr><th>Name</th><th>Message</th><th>Key</th><th style="text-align:right">Actions</th></tr></thead><tbody>';
for (const item of items) {
const short = item.key.substring(0, 16);
html += '<tr>';
html += '<td>' + escHtml(item.displayName) + '</td>';
html += '<td>' + escHtml(item.message || '') + '</td>';
html += '<td class="hash-cell" title="' + escHtml(item.key) + '">' + escHtml(short) + '</td>';
html += '<td class="actions-cell">';
html += '<button class="small primary" onclick="grantKey(\'' + item.key + '\')">grant</button> ';
html += '<button class="small danger" onclick="denyKey(\'' + item.key + '\')">deny</button>';
html += '</td></tr>';
}
html += '</tbody></table>';
$('requestsTable').innerHTML = html;
}
function renderKeys(keys) {
if (!keys || keys.length === 0) {
$('keysTable').innerHTML = '<div class="empty-state">no authorized keys</div>';
return;
}
const items = addDisambiguation(keys, 'label');
let html = '<table><thead><tr><th>Name</th><th>Key</th><th style="text-align:right">Actions</th></tr></thead><tbody>';
for (const item of items) {
const short = item.key.substring(0, 16);
html += '<tr>';
html += '<td>' + escHtml(item.displayName) + '</td>';
html += '<td class="hash-cell" title="' + escHtml(item.key) + '">' + escHtml(short) + '</td>';
html += '<td class="actions-cell">';
html += '<button class="small danger" onclick="revokeKey(\'' + item.key + '\')">revoke</button>';
html += '</td></tr>';
}
html += '</tbody></table>';
$('keysTable').innerHTML = html;
}
async function grantKey(hex) {
try {
const r = await ownerAuthFetch('/api/auth/grant?key=' + hex, { method: 'POST' });
if (!r.ok) { const j = await r.json().catch(() => ({})); throw new Error(j.error || r.statusText); }
toast('Granted ' + hex.substring(0, 12), 'success');
refreshAll();
} catch(e) {
toast('Grant failed: ' + e.message, 'error');
}
}
async function denyKey(hex) {
try {
const r = await ownerAuthFetch('/api/auth/deny?key=' + hex, { method: 'POST' });
if (!r.ok) { const j = await r.json().catch(() => ({})); throw new Error(j.error || r.statusText); }
toast('Denied ' + hex.substring(0, 12), 'success');
refreshAll();
} catch(e) {
toast('Deny failed: ' + e.message, 'error');
}
}
async function revokeKey(hex) {
if (!confirm('Revoke ' + hex.substring(0, 16) + '?')) return;
try {
const r = await ownerAuthFetch('/api/auth/revoke?key=' + hex, { method: 'POST' });
if (!r.ok) { const j = await r.json().catch(() => ({})); throw new Error(j.error || r.statusText); }
toast('Revoked ' + hex.substring(0, 12), 'success');
refreshAll();
} catch(e) {
toast('Revoke failed: ' + e.message, 'error');
}
}
async function manualGrant() {
const key = $('manualKeyInput').value.trim();
if (!key || key.length !== 64) { toast('Enter a 64-char hex public key', 'error'); return; }
const name = $('manualNameInput').value.trim();
let url = '/api/auth/grant?key=' + key;
if (name) url += '&name=' + encodeURIComponent(name);
try {
const r = await ownerAuthFetch(url, { method: 'POST' });
if (!r.ok) { const j = await r.json().catch(() => ({})); throw new Error(j.error || r.statusText); }
toast('Granted ' + key.substring(0, 12), 'success');
$('manualKeyInput').value = '';
$('manualNameInput').value = '';
refreshAll();
} catch(e) {
toast('Grant failed: ' + e.message, 'error');
}
}
// Init
fetchStatus();
refreshList();
</script>
</body>
</html>"##;

59
crates/datastore/tests/acl_persistence_tests.rs Normal file
View file

@ -0,0 +1,59 @@
//! ACL file persistence tests — roundtrip save/load.
use std::collections::{HashMap, HashSet};
use distribution::crypto::Keypair;
use swactor_datastore::auth::AccessControlList;
// ═══════════════════════════════════════════════════════════════════════════
// 1. Save + load preserves owner and authorized_keys
// ═══════════════════════════════════════════════════════════════════════════
#[test]
fn save_and_load_roundtrip() {
let dir = tempfile::tempdir().unwrap();
let path = dir.path().join("acl.json");
let owner = Keypair::generate().node_id();
let client_a = Keypair::generate().node_id();
let client_b = Keypair::generate().node_id();
let mut keys = HashSet::new();
keys.insert(client_a);
keys.insert(client_b);
let acl = AccessControlList {
owner,
authorized_keys: keys.clone(),
key_labels: HashMap::new(),
};
acl.save(&path).unwrap();
let loaded = AccessControlList::load_or_create(&path, owner).unwrap();
assert_eq!(loaded.owner, owner);
assert_eq!(loaded.authorized_keys, keys);
}
// ═══════════════════════════════════════════════════════════════════════════
// 2. load_or_create on missing file creates default
// ═══════════════════════════════════════════════════════════════════════════
#[test]
fn load_or_create_on_missing_file_creates_default() {
let dir = tempfile::tempdir().unwrap();
let path = dir.path().join("nonexistent/acl.json");
let owner = Keypair::generate().node_id();
let acl = AccessControlList::load_or_create(&path, owner).unwrap();
assert_eq!(acl.owner, owner);
assert!(acl.authorized_keys.is_empty());
// File should now exist
assert!(path.exists());
// Loading again should give same result
let acl2 = AccessControlList::load_or_create(&path, owner).unwrap();
assert_eq!(acl2.owner, owner);
assert!(acl2.authorized_keys.is_empty());
}

1
crates/datastore/tests/api_integration_test.rs
View file

@ -69,6 +69,7 @@ fn http_crud_lifecycle() {
datastore_addr,
metadata_addr,
blob_store_addr,
None,
port,
Arc::clone(&metrics),
);

292
crates/datastore/tests/auth_scenario_tests.rs Normal file
View file

@ -0,0 +1,292 @@
//! Scenario tests for AuthzEngine — no actor system, pure auth logic.
use std::collections::{HashMap, HashSet};
use distribution::crypto::Keypair;
use shared_types::ContentHash;
use swactor_datastore::auth::{
sign_request, AccessControlList, AuthzEngine, AuthzResult, DatastoreAction, DeniedReason,
SignedRequestPayload,
};
fn owner_engine() -> (Keypair, AuthzEngine) {
let owner_kp = Keypair::generate();
let acl = AccessControlList {
owner: owner_kp.node_id(),
authorized_keys: HashSet::new(),
key_labels: HashMap::new(),
};
(owner_kp, AuthzEngine::new(acl))
}
// ═══════════════════════════════════════════════════════════════════════════
// 1. Owner always allowed; random key denied
// ═══════════════════════════════════════════════════════════════════════════
#[test]
fn owner_is_always_allowed() {
let (owner_kp, engine) = owner_engine();
assert_eq!(engine.check_node(&owner_kp.node_id()), AuthzResult::Allowed);
}
#[test]
fn unknown_key_is_denied() {
let (_owner_kp, engine) = owner_engine();
let stranger = Keypair::generate().node_id();
assert_eq!(
engine.check_node(&stranger),
AuthzResult::Denied(DeniedReason::NotAuthorized)
);
}
// ═══════════════════════════════════════════════════════════════════════════
// 2. Grant → access → revoke → denied (lifecycle story)
// ═══════════════════════════════════════════════════════════════════════════
#[test]
fn grant_then_revoke_lifecycle() {
let (owner_kp, mut engine) = owner_engine();
let client = Keypair::generate().node_id();
// Initially denied
assert_eq!(
engine.check_node(&client),
AuthzResult::Denied(DeniedReason::NotAuthorized)
);
// Grant
engine.grant(&owner_kp.node_id(), client, None).unwrap();
assert_eq!(engine.check_node(&client), AuthzResult::Allowed);
// Revoke
engine.revoke(&owner_kp.node_id(), client).unwrap();
assert_eq!(
engine.check_node(&client),
AuthzResult::Denied(DeniedReason::NotAuthorized)
);
}
// ═══════════════════════════════════════════════════════════════════════════
// 3. Only owner can grant/revoke; non-owner gets NotAuthorized
// ═══════════════════════════════════════════════════════════════════════════
#[test]
fn non_owner_cannot_grant() {
let (_owner_kp, mut engine) = owner_engine();
let impostor = Keypair::generate().node_id();
let target = Keypair::generate().node_id();
assert_eq!(
engine.grant(&impostor, target, None),
Err(DeniedReason::NotAuthorized)
);
}
#[test]
fn non_owner_cannot_revoke() {
let (owner_kp, mut engine) = owner_engine();
let client = Keypair::generate().node_id();
engine.grant(&owner_kp.node_id(), client, None).unwrap();
let impostor = Keypair::generate().node_id();
assert_eq!(
engine.revoke(&impostor, client),
Err(DeniedReason::NotAuthorized)
);
// Client still authorized
assert_eq!(engine.check_node(&client), AuthzResult::Allowed);
}
// ═══════════════════════════════════════════════════════════════════════════
// 4. Cannot revoke owner's implicit access
// ═══════════════════════════════════════════════════════════════════════════
#[test]
fn revoking_owner_is_noop() {
let (owner_kp, mut engine) = owner_engine();
let owner_id = owner_kp.node_id();
// Attempt to revoke owner — should succeed (idempotent no-op) but owner remains allowed
engine.revoke(&owner_id, owner_id).unwrap();
assert_eq!(engine.check_node(&owner_id), AuthzResult::Allowed);
}
// ═══════════════════════════════════════════════════════════════════════════
// 5. Signed request happy path (sign → verify → allowed)
// ═══════════════════════════════════════════════════════════════════════════
#[test]
fn signed_request_happy_path() {
let (owner_kp, mut engine) = owner_engine();
let now = 1_000_000u64;
let payload = SignedRequestPayload {
action: DatastoreAction::Get {
content_hash: ContentHash::of(b"hello"),
},
timestamp: now,
nonce: [1; 16],
};
let request = sign_request(&owner_kp, payload);
assert_eq!(
engine.check_signed_request(&request, now),
AuthzResult::Allowed
);
}
// ═══════════════════════════════════════════════════════════════════════════
// 6. Tampered signature → InvalidSignature
// ═══════════════════════════════════════════════════════════════════════════
#[test]
fn tampered_signature_is_rejected() {
let (owner_kp, mut engine) = owner_engine();
let now = 1_000_000u64;
let payload = SignedRequestPayload {
action: DatastoreAction::Get {
content_hash: ContentHash::of(b"hello"),
},
timestamp: now,
nonce: [2; 16],
};
let mut request = sign_request(&owner_kp, payload);
// Tamper with signature
request.signature.0[0] ^= 0xFF;
assert_eq!(
engine.check_signed_request(&request, now),
AuthzResult::Denied(DeniedReason::InvalidSignature)
);
}
// ═══════════════════════════════════════════════════════════════════════════
// 7. Stale timestamp → RequestExpired
// ═══════════════════════════════════════════════════════════════════════════
#[test]
fn stale_timestamp_is_rejected() {
let (owner_kp, mut engine) = owner_engine();
let now = 1_000_000u64;
let payload = SignedRequestPayload {
action: DatastoreAction::Get {
content_hash: ContentHash::of(b"stale"),
},
timestamp: now - 400, // 400s ago, outside 300s window
nonce: [3; 16],
};
let request = sign_request(&owner_kp, payload);
assert_eq!(
engine.check_signed_request(&request, now),
AuthzResult::Denied(DeniedReason::RequestExpired)
);
}
// ═══════════════════════════════════════════════════════════════════════════
// 8. Replayed nonce → ReplayDetected
// ═══════════════════════════════════════════════════════════════════════════
#[test]
fn replayed_nonce_is_rejected() {
let (owner_kp, mut engine) = owner_engine();
let now = 1_000_000u64;
let payload = SignedRequestPayload {
action: DatastoreAction::Get {
content_hash: ContentHash::of(b"first"),
},
timestamp: now,
nonce: [4; 16],
};
let request = sign_request(&owner_kp, payload);
// First time — allowed
assert_eq!(
engine.check_signed_request(&request, now),
AuthzResult::Allowed
);
// Replay — same nonce
let payload2 = SignedRequestPayload {
action: DatastoreAction::Get {
content_hash: ContentHash::of(b"first"),
},
timestamp: now,
nonce: [4; 16],
};
let request2 = sign_request(&owner_kp, payload2);
assert_eq!(
engine.check_signed_request(&request2, now),
AuthzResult::Denied(DeniedReason::ReplayDetected)
);
}
// ═══════════════════════════════════════════════════════════════════════════
// 9. Nonce GC frees old nonces for reuse
// ═══════════════════════════════════════════════════════════════════════════
#[test]
fn nonce_gc_frees_old_nonces() {
let (owner_kp, mut engine) = owner_engine();
let t0 = 1_000_000u64;
let payload = SignedRequestPayload {
action: DatastoreAction::Get {
content_hash: ContentHash::of(b"gc-test"),
},
timestamp: t0,
nonce: [5; 16],
};
let request = sign_request(&owner_kp, payload);
assert_eq!(
engine.check_signed_request(&request, t0),
AuthzResult::Allowed
);
// Advance time past the window and GC
let t1 = t0 + 400;
engine.gc_nonces(t1);
// Same nonce but with current timestamp — no longer flagged as replay
let payload2 = SignedRequestPayload {
action: DatastoreAction::Get {
content_hash: ContentHash::of(b"gc-test"),
},
timestamp: t1,
nonce: [5; 16],
};
let request2 = sign_request(&owner_kp, payload2);
assert_eq!(
engine.check_signed_request(&request2, t1),
AuthzResult::Allowed
);
}
// ═══════════════════════════════════════════════════════════════════════════
// 10. Unauthorized key with valid signature → NotAuthorized
// ═══════════════════════════════════════════════════════════════════════════
#[test]
fn unauthorized_key_with_valid_signature_is_denied() {
let (_owner_kp, mut engine) = owner_engine();
let stranger_kp = Keypair::generate();
let now = 1_000_000u64;
let payload = SignedRequestPayload {
action: DatastoreAction::Get {
content_hash: ContentHash::of(b"intrusion"),
},
timestamp: now,
nonce: [6; 16],
};
let request = sign_request(&stranger_kp, payload);
assert_eq!(
engine.check_signed_request(&request, now),
AuthzResult::Denied(DeniedReason::NotAuthorized)
);
}

1
crates/datastore/tests/dashboard_integration_test.rs
View file

@ -94,6 +94,7 @@ fn dashboard_reflects_datastore_operations() {
datastore_addr,
metadata_addr,
blob_store_addr,
None,
api_port,
Arc::clone(&metrics),
);

203
crates/datastore/tests/gateway_tests.rs Normal file
View file

@ -0,0 +1,203 @@
//! Actor-level tests for the GatewayActor.
//!
//! Uses the swactor runtime to spawn GatewayActor + DatastoreNode and verify
//! that authorized requests flow through while unauthorized ones are denied.
mod common;
use std::collections::{HashMap, HashSet};
use std::time::{SystemTime, UNIX_EPOCH};
use common::{spawn_blob_store, spawn_metadata, test_runtime, tick_until_recv};
use distribution::crypto::Keypair;
use shared_types::ContentHash;
use swactor_datastore::actors::{DatastoreNode, GatewayActor};
use swactor_datastore::auth::{
sign_request, AccessControlList, AuthzEngine, DatastoreAction, DeniedReason,
SignedRequestPayload,
};
use swactor_datastore::messages::{DatastoreResponse, GatewayMsg};
use swactor_datastore::types::DatastoreConfig;
struct GatewayHarness {
rt: swactor::runtime::Runtime,
gateway: swactor::actor::ActorAddress,
inbox: swactor::runtime::Inbox<DatastoreResponse>,
owner_kp: Keypair,
}
impl GatewayHarness {
fn new() -> Self {
let owner_kp = Keypair::generate();
let rt = test_runtime();
let blob_store = spawn_blob_store(&rt);
let node_id = owner_kp.node_id();
let metadata = spawn_metadata(&rt, node_id);
let mut config = DatastoreConfig::default();
config.chunk_size = 64;
let datastore_node = rt
.spawn(DatastoreNode::new(node_id, blob_store, metadata, config))
.unwrap();
let acl = AccessControlList {
owner: owner_kp.node_id(),
authorized_keys: HashSet::new(),
key_labels: HashMap::new(),
};
let engine = AuthzEngine::new(acl);
let gateway = rt
.spawn(GatewayActor::new(engine, datastore_node, None))
.unwrap();
let inbox = rt.new_inbox::<DatastoreResponse>().unwrap();
// Let actors initialize
for _ in 0..3 {
rt.tick();
}
Self {
rt,
gateway,
inbox,
owner_kp,
}
}
fn reply_addr(&self) -> swactor::actor::ActorAddress {
*self.inbox.addr()
}
}
// ═══════════════════════════════════════════════════════════════════════════
// 1. Authorized signed GET dispatches and returns result
// ═══════════════════════════════════════════════════════════════════════════
#[test]
fn authorized_signed_get_flows_through_to_datastore() {
let h = GatewayHarness::new();
let now = SystemTime::now().duration_since(UNIX_EPOCH).unwrap().as_secs();
// PUT some data first via signed request
let data = b"gateway test data";
let content_hash = ContentHash::of(data);
// Store data by sending directly to datastore through a Put via gateway
// (For simplicity, store via DatastoreNode first, then GET through gateway)
// Actually, let's just do a GET for a nonexistent hash — we should get NotFound (not Denied)
let payload = SignedRequestPayload {
action: DatastoreAction::Get { content_hash },
timestamp: now,
nonce: [10; 16],
};
let request = sign_request(&h.owner_kp, payload);
h.rt.send_to(
h.gateway,
GatewayMsg::HandleSignedRequest {
request,
reply_to: h.reply_addr(),
},
)
.unwrap();
let resp = tick_until_recv(&h.rt, &h.inbox, 30).unwrap();
// Should get NotFound (authorized, but object doesn't exist) — NOT Denied
assert!(
matches!(resp, DatastoreResponse::NotFound),
"expected NotFound (authorized but missing), got {resp:?}"
);
}
// ═══════════════════════════════════════════════════════════════════════════
// 2. Unauthorized signed GET returns Denied
// ═══════════════════════════════════════════════════════════════════════════
#[test]
fn unauthorized_signed_get_returns_denied() {
let h = GatewayHarness::new();
let stranger_kp = Keypair::generate();
let now = SystemTime::now().duration_since(UNIX_EPOCH).unwrap().as_secs();
let payload = SignedRequestPayload {
action: DatastoreAction::Get {
content_hash: ContentHash::of(b"unauthorized"),
},
timestamp: now,
nonce: [11; 16],
};
let request = sign_request(&stranger_kp, payload);
h.rt.send_to(
h.gateway,
GatewayMsg::HandleSignedRequest {
request,
reply_to: h.reply_addr(),
},
)
.unwrap();
let resp = tick_until_recv(&h.rt, &h.inbox, 30).unwrap();
assert!(
matches!(
resp,
DatastoreResponse::Denied {
reason: DeniedReason::NotAuthorized
}
),
"expected Denied(NotAuthorized), got {resp:?}"
);
}
// ═══════════════════════════════════════════════════════════════════════════
// 3. Connection check allows/denies correctly
// ═══════════════════════════════════════════════════════════════════════════
#[test]
fn check_connection_allows_owner() {
let h = GatewayHarness::new();
h.rt.send_to(
h.gateway,
GatewayMsg::CheckConnection {
node_id: h.owner_kp.node_id(),
reply_to: h.reply_addr(),
},
)
.unwrap();
let resp = tick_until_recv(&h.rt, &h.inbox, 20).unwrap();
assert!(
matches!(resp, DatastoreResponse::Bool(true)),
"expected Bool(true), got {resp:?}"
);
}
#[test]
fn check_connection_denies_stranger() {
let h = GatewayHarness::new();
let stranger = Keypair::generate().node_id();
h.rt.send_to(
h.gateway,
GatewayMsg::CheckConnection {
node_id: stranger,
reply_to: h.reply_addr(),
},
)
.unwrap();
let resp = tick_until_recv(&h.rt, &h.inbox, 20).unwrap();
assert!(
matches!(
resp,
DatastoreResponse::Denied {
reason: DeniedReason::NotAuthorized
}
),
"expected Denied(NotAuthorized), got {resp:?}"
);
}

242
crates/datastore/tests/http_auth_integration.rs Normal file
View file

@ -0,0 +1,242 @@
//! Integration test: HTTP API endpoints gated behind auth.
//!
//! Spins up a full actor runtime with GatewayActor, starts the HTTP API server,
//! and uses ureq to prove that authorized requests succeed while unauthorized
//! ones get 403 and missing-auth requests get 401.
#![cfg(feature = "node")]
use std::collections::{BTreeMap, HashMap, HashSet};
use std::sync::atomic::Ordering;
use std::time::{Duration, SystemTime, UNIX_EPOCH};
use distribution::crypto::Keypair;
use shared_types::ContentHash;
use swactor::config::RuntimeConfig;
use swactor::runtime::Runtime;
use swactor_datastore::actors::{BlobStoreActor, DatastoreNode, GatewayActor, MetadataActor};
use swactor_datastore::api::start_api_server;
use swactor_datastore::auth::{
sign_request, AccessControlList, AuthzEngine, DatastoreAction, SignedRequestPayload,
};
use swactor_datastore::storage::InMemoryBackend;
use swactor_datastore::types::DatastoreConfig;
fn now_secs() -> u64 {
SystemTime::now()
.duration_since(UNIX_EPOCH)
.unwrap()
.as_secs()
}
fn random_nonce() -> [u8; 16] {
let t = SystemTime::now()
.duration_since(UNIX_EPOCH)
.unwrap()
.as_nanos();
let mut nonce = [0u8; 16];
nonce.copy_from_slice(&t.to_le_bytes());
nonce
}
fn sign_header(keypair: &Keypair, action: DatastoreAction) -> String {
let payload = SignedRequestPayload {
action,
timestamp: now_secs(),
nonce: random_nonce(),
};
let request = sign_request(keypair, payload);
serde_json::to_string(&request).unwrap()
}
/// Find an available TCP port by binding to :0.
fn available_port() -> u16 {
let listener = std::net::TcpListener::bind("127.0.0.1:0").unwrap();
listener.local_addr().unwrap().port()
}
// ═══════════════════════════════════════════════════════════════════════════
// Scenario: Owner operates over HTTP; stranger is denied
// ═══════════════════════════════════════════════════════════════════════════
#[test]
fn http_auth_owner_allowed_stranger_denied() {
let owner_kp = Keypair::generate();
let stranger_kp = Keypair::generate();
let owner_id = owner_kp.node_id();
// ── Build runtime & actors ───────────────────────────────────────────
let rt = Runtime::new(RuntimeConfig {
num_threads: 2,
max_actors: 256,
channel_buffer_size: 1024,
..Default::default()
});
let blob_store_addr = rt
.spawn(BlobStoreActor::new(Box::new(InMemoryBackend::new())))
.unwrap();
let mut metadata = MetadataActor::new(owner_id, &DatastoreConfig::default());
metadata.set_blob_store(blob_store_addr);
let metadata_addr = rt.spawn(metadata).unwrap();
let config = DatastoreConfig {
chunk_size: 1_048_576,
..Default::default()
};
let datastore_node = DatastoreNode::new(owner_id, blob_store_addr, metadata_addr, config);
let datastore_addr = rt.spawn(datastore_node).unwrap();
let acl = AccessControlList {
owner: owner_id,
authorized_keys: HashSet::new(),
key_labels: HashMap::new(),
};
let engine = AuthzEngine::new(acl);
let gateway_addr = rt
.spawn(GatewayActor::new(engine, datastore_addr, None))
.unwrap();
let handle = rt.run().expect("failed to start runtime");
// ── Start HTTP server ────────────────────────────────────────────────
let port = available_port();
let metrics = std::sync::Arc::new(swactor_datastore::metrics::DatastoreMetrics::new());
let (shutdown, _peers) = start_api_server(
handle.runtime.clone(),
datastore_addr,
metadata_addr,
blob_store_addr,
Some(gateway_addr),
port,
metrics,
);
// Give the HTTP server threads a moment to start accepting connections.
std::thread::sleep(Duration::from_millis(100));
let base = format!("http://127.0.0.1:{port}");
// ── 1. Owner PUTs data ───────────────────────────────────────────────
let test_data = b"hello from the integration test";
let expected_hash = ContentHash::of(test_data);
let put_header = sign_header(
&owner_kp,
DatastoreAction::Put {
name: Some("test.txt".to_string()),
content_hash: expected_hash,
size_bytes: test_data.len() as u64,
tags: BTreeMap::new(),
},
);
let put_resp = ureq::post(&format!("{base}/api/put?name=test.txt"))
.set("X-Signed-Request", &put_header)
.send_bytes(test_data)
.expect("PUT request failed");
assert_eq!(put_resp.status(), 200);
let put_body: serde_json::Value = put_resp.into_json().unwrap();
let returned_hash = put_body["content_hash"].as_str().unwrap();
assert_eq!(returned_hash, expected_hash.to_hex());
// ── 2. Owner GETs it back ────────────────────────────────────────────
let get_header = sign_header(
&owner_kp,
DatastoreAction::Get {
content_hash: expected_hash,
},
);
let get_resp = ureq::get(&format!("{base}/api/get?hash={}", expected_hash.to_hex()))
.set("X-Signed-Request", &get_header)
.call()
.expect("GET request failed");
assert_eq!(get_resp.status(), 200);
let get_body: serde_json::Value = get_resp.into_json().unwrap();
assert_eq!(
get_body["entry"]["content_hash"].as_str().unwrap(),
expected_hash.to_hex()
);
// ── 3. Owner LISTs ──────────────────────────────────────────────────
let list_header = sign_header(
&owner_kp,
DatastoreAction::List { name_filter: None },
);
let list_resp = ureq::get(&format!("{base}/api/list"))
.set("X-Signed-Request", &list_header)
.call()
.expect("LIST request failed");
assert_eq!(list_resp.status(), 200);
let list_body: serde_json::Value = list_resp.into_json().unwrap();
let entries = list_body["entries"].as_array().unwrap();
assert!(
entries
.iter()
.any(|e| e["content_hash"].as_str() == Some(&expected_hash.to_hex())),
"expected hash in list results"
);
// ── 4. Stranger tries GET → 403 ─────────────────────────────────────
let stranger_header = sign_header(
&stranger_kp,
DatastoreAction::Get {
content_hash: expected_hash,
},
);
let stranger_resp = ureq::get(&format!(
"{base}/api/get?hash={}",
expected_hash.to_hex()
))
.set("X-Signed-Request", &stranger_header)
.call();
match stranger_resp {
Err(ureq::Error::Status(403, _)) => {} // expected
Err(e) => panic!("expected 403, got error: {e}"),
Ok(r) => panic!("expected 403, got {}", r.status()),
}
// ── 5. No auth header → 401 ─────────────────────────────────────────
let no_auth_resp = ureq::get(&format!(
"{base}/api/get?hash={}",
expected_hash.to_hex()
))
.call();
match no_auth_resp {
Err(ureq::Error::Status(401, _)) => {} // expected
Err(e) => panic!("expected 401, got error: {e}"),
Ok(r) => panic!("expected 401, got {}", r.status()),
}
// ── 6. Owner DELETEs ─────────────────────────────────────────────────
let delete_header = sign_header(
&owner_kp,
DatastoreAction::Delete {
content_hash: expected_hash,
},
);
let delete_resp = ureq::post(&format!(
"{base}/api/delete?hash={}",
expected_hash.to_hex()
))
.set("X-Signed-Request", &delete_header)
.send_bytes(&[])
.expect("DELETE request failed");
assert_eq!(delete_resp.status(), 200);
// ── Teardown ─────────────────────────────────────────────────────────
shutdown.store(true, Ordering::Relaxed);
handle.shutdown();
handle.join();
}

1
crates/distribution/Cargo.toml
View file

@ -10,6 +10,7 @@ iroh = ["dep:iroh", "dep:tokio"]
[dependencies]
swactor = { path = "../..", features = ["serde", "transport"] }
shared-types = { path = "../shared-types" }
ed25519-dalek = { version = "2", features = ["rand_core"] }
rand_core = { version = "0.6", features = ["getrandom"] }
serde = { version = "1", features = ["derive"] }

8
crates/shared-types/Cargo.toml Normal file
View file

@ -0,0 +1,8 @@
[package]
name = "shared-types"
version = "0.1.0"
edition = "2024"
[dependencies]
serde = { version = "1", features = ["derive"] }
blake3 = "1"

106
crates/shared-types/src/lib.rs Normal file
View file

@ -0,0 +1,106 @@
//! Shared types used across the swactor crate ecosystem.
//!
//! Contains `ContentHash` — the blake3-based content address used by
//! both the datastore and distribution layers.
use std::fmt;
use serde::{Deserialize, Serialize};
// ─── ContentHash ────────────────────────────────────────────────────────────
/// A blake3 content hash (32 bytes).
///
/// The primary identifier for blobs and the DHT key. XOR distance for DHT
/// routing, compact Debug/Display for logging.
#[derive(Clone, Copy, PartialEq, Eq, Hash, Serialize, Deserialize)]
pub struct ContentHash(pub [u8; 32]);
impl ContentHash {
/// Compute the blake3 hash of the given data.
pub fn of(data: &[u8]) -> Self {
let hash = blake3::hash(data);
ContentHash(*hash.as_bytes())
}
/// XOR distance between two content hashes (Kademlia metric).
pub fn xor_distance(&self, other: &ContentHash) -> [u8; 32] {
let mut out = [0u8; 32];
for i in 0..32 {
out[i] = self.0[i] ^ other.0[i];
}
out
}
/// Number of leading zero bits in the XOR distance to `other`.
/// Returns 0..=256. Used to select the k-bucket index in the metadata DHT.
pub fn xor_leading_zeros(&self, other: &ContentHash) -> u32 {
let dist = self.xor_distance(other);
let mut zeros = 0u32;
for byte in dist {
if byte == 0 {
zeros += 8;
} else {
zeros += byte.leading_zeros();
break;
}
}
zeros
}
/// Parse a 64-character hex string into a ContentHash.
/// Returns `None` if the string is not exactly 64 hex characters.
pub fn from_hex(hex: &str) -> Option<Self> {
if hex.len() != 64 {
return None;
}
let mut bytes = [0u8; 32];
for (i, chunk) in hex.as_bytes().chunks(2).enumerate() {
let hi = hex_digit(chunk[0])?;
let lo = hex_digit(chunk[1])?;
bytes[i] = (hi << 4) | lo;
}
Some(ContentHash(bytes))
}
/// Encode as lowercase hex string.
pub fn to_hex(&self) -> String {
let mut s = String::with_capacity(64);
for b in &self.0 {
use fmt::Write;
write!(s, "{:02x}", b).unwrap();
}
s
}
/// The zero hash (all zeroes). Used as a sentinel.
pub const ZERO: ContentHash = ContentHash([0u8; 32]);
}
impl fmt::Debug for ContentHash {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
write!(f, "Hash(")?;
for b in &self.0[..4] {
write!(f, "{:02x}", b)?;
}
write!(f, "\u{2026})")
}
}
impl fmt::Display for ContentHash {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
for b in &self.0[..8] {
write!(f, "{:02x}", b)?;
}
write!(f, "\u{2026}")
}
}
fn hex_digit(b: u8) -> Option<u8> {
match b {
b'0'..=b'9' => Some(b - b'0'),
b'a'..=b'f' => Some(b - b'a' + 10),
b'A'..=b'F' => Some(b - b'A' + 10),
_ => None,
}
}

BIN
datastore/chunks/34/0f/340f67ad6e49e8aaa9e272dbcad27c38e7b32e3ff6f75741be4c1d294d8fd2dd Normal file
View file

Binary file not shown.

1
datastore/entries/34/0f/340f67ad6e49e8aaa9e272dbcad27c38e7b32e3ff6f75741be4c1d294d8fd2dd Normal file
View file

@ -0,0 +1 @@
{"content_hash":[52,15,103,173,110,73,232,170,169,226,114,219,202,210,124,56,231,179,46,63,246,247,87,65,190,76,29,41,77,143,210,221],"name":"Distributed Data Types v1.pdf","node_id":[130,74,105,87,21,9,99,141,247,147,202,184,64,55,216,254,212,34,166,19,54,124,213,5,205,134,85,35,28,219,47,170],"tags":{},"size_bytes":287176,"created_at":0}

1
datastore/manifests/34/0f/340f67ad6e49e8aaa9e272dbcad27c38e7b32e3ff6f75741be4c1d294d8fd2dd Normal file
View file

@ -0,0 +1 @@
{"content_hash":[52,15,103,173,110,73,232,170,169,226,114,219,202,210,124,56,231,179,46,63,246,247,87,65,190,76,29,41,77,143,210,221],"chunks":[{"hash":[52,15,103,173,110,73,232,170,169,226,114,219,202,210,124,56,231,179,46,63,246,247,87,65,190,76,29,41,77,143,210,221],"offset":0,"size":287176}],"total_size":287176,"chunk_size":1048576,"content_type":null}

503
docs/datastore/DATASTORE_AUTH.md Normal file
View file

@ -0,0 +1,503 @@
# Swactor Datastore Auth Specification
**Version:** 0.2.0
**Status:** Implemented (MVP)
## 1. Overview
This document specifies the authorization layer for the Swactor Datastore as implemented. It defines how access is controlled for external clients connecting to a datastore node.
### Principles
- **Cryptographic identity** — keys, not passwords. Every participant is identified by an ed25519 public key (`NodeId`).
- **Binary access** — a client is either authorized or not. No permission tiers for MVP.
- **Owner-only administration** — only the datastore owner can grant or revoke access.
- **Two auth paths** — direct iroh connections (connection-level) and signed HTTP requests (browser/CLI). This spec covers the signed request path (Auth Path 2), which is fully implemented.
### Non-Goals (MVP)
- Per-path permission scoping.
- Permission tiers (read-only, read-write, admin).
- Capability tokens or time-limited delegated access.
- Multi-level delegation chains.
## 2. Trust Boundaries
```
┌─────────────────────────────────────────────┐
│ Cluster (SWIM mesh) │
│ │
│ Node A ◄──────────────► Node B │
│ implicitly trusted │
│ (no auth checks) │
└──────────────────┬──────────────────────────┘
│
│ auth boundary
│
┌──────────▼──────────┐
│ External Clients │
│ │
│ CLI tool │
│ Browser user │
└─────────────────────┘
```
- **Cluster-internal** (node-to-node via SWIM): implicitly trusted. Nodes that are members of the SWIM cluster communicate freely — no per-request auth checks.
- **External clients** (CLI, browser): must be authorized. Every external request is checked against the Access Control List before being dispatched to the actor system.
## 3. Identity Model
The auth layer reuses the existing ed25519 identity model from the distribution layer:
- Every client (CLI tool, browser user, node) has an ed25519 keypair.
- Identity is the 32-byte public key, represented as `NodeId`.
- The same `NodeId` type from `distribution::types` is used throughout.
There is no separate "user" concept — a keypair *is* an identity.
## 4. Access Control List
### 4.1 Structure
```rust
AccessControlList {
owner: NodeId, // The datastore owner's public key
authorized_keys: HashSet<NodeId>, // Explicitly authorized client keys
key_labels: HashMap<String, String>, // hex(public_key) → human-readable name
}
```
- The **owner** always has full access (implicit; never needs to be in `authorized_keys`).
- An empty `authorized_keys` set means only the owner can access the datastore.
- `key_labels` maps the hex-encoded public key to a human-readable name. Labels are set on grant (from the access request's `name` field or an explicit `--name`/`?name=` parameter) and removed on revoke. The `#[serde(default)]` annotation ensures backward compatibility with ACL files written before labels existed.
### 4.2 Persistence
The ACL is persisted as JSON in the **auth directory**, separate from the storage path:
```
<auth-dir>/
├── owner.key.json # Owner keypair
└── acl.json # AccessControlList
```
Default `auth-dir` is `./auth` (configurable via `--auth-dir`).
### 4.3 Mutations
| Operation | Signature | Who |
|-----------|-----------|-----|
| Grant access | `grant(requester, key, label)` | Owner only |
| Revoke access | `revoke(requester, key)` | Owner only |
- `grant` adds a `NodeId` to `authorized_keys` and optionally sets a label in `key_labels`. If the key has a pending access request, the request's `name` is used as the label (unless an explicit label is provided). Idempotent.
- `revoke` removes a `NodeId` from `authorized_keys` and removes its label from `key_labels`. Idempotent.
- Revoking the owner is a no-op (the owner's implicit access cannot be removed).
- Both operations persist the updated ACL to disk immediately via `persist_acl()`.
## 5. Auth Path 1 — Direct iroh Connection
For clients that connect directly to the datastore node over iroh (QUIC):
```
Client (ed25519 keypair) Datastore Node
│ │
│──── iroh QUIC handshake ──────────>│
│ (proves client's NodeId) │
│ │
│ check NodeId
│ against ACL
│ │
│<─── accept / reject ──────────────│
│ │
│ (if accepted, all ops on │
│ this connection are allowed) │
```
1. The iroh QUIC handshake cryptographically proves the peer's `NodeId` (ed25519 public key).
2. On connection establishment, the node checks the peer's `NodeId` against the ACL via `check_node()`.
3. If authorized, connection accepted. All operations on that connection are allowed with no per-message overhead.
4. If not authorized, connection rejected immediately.
## 6. Auth Path 2 — Signed Requests (HTTP API)
For browser users and CLI clients communicating over HTTP.
### 6.1 Threat Model
The HTTP transport is treated as an **untrusted relay**. Each request is self-authenticating via a signed envelope. The relay cannot forge, modify, or replay requests.
### 6.2 Signed Envelope
Each request carries a signed envelope in the `X-Signed-Request` HTTP header:
```rust
SignedRequest {
payload: SignedRequestPayload, // The request details
public_key: NodeId, // Client's public key (as [u8; 32])
signature: Signature, // ed25519 signature over serialized payload
}
SignedRequestPayload {
action: DatastoreAction, // What the client wants to do
timestamp: u64, // Unix timestamp (seconds)
nonce: [u8; 16], // 16 random bytes
}
DatastoreAction = enum {
Put { name, content_hash, size_bytes, tags },
Get { content_hash },
Delete { content_hash },
List { name_filter },
Access, // Identity proof (no content binding)
}
```
The header value is the JSON serialization of `SignedRequest`. The `public_key` and `signature` fields are serialized as arrays of integers (e.g., `[163, 45, ...]`), matching serde's default serialization for `[u8; 32]` and `[u8; 64]`.
### 6.3 DatastoreAction::Access
The `Access` variant is a lightweight identity proof that does not bind to a specific content operation. It is used by:
- **Browser** — all API calls use `Access` (the browser proves identity, and the HTTP layer gates the actual operation).
- **CLI auth management** — `grant`, `revoke`, `requests`, `keys`, `deny` subcommands use `Access` since these admin operations don't correspond to content actions.
The CLI's data operations (`put`, `get`, `delete`, `list`) sign the corresponding specific action variants.
### 6.4 Verification Steps
The `AuthzEngine` verifies a signed request in strict order:
1. **Signature validity** — verify the ed25519 signature over the canonical JSON serialization of `SignedRequestPayload` using the provided `public_key`.
2. **Timestamp freshness** — reject if `|now - payload.timestamp| > 300` seconds.
3. **Nonce uniqueness** — reject if `payload.nonce` has been seen before within the time window.
4. **ACL check** — reject if `public_key` is not in the ACL (not owner and not in `authorized_keys`).
If any step fails, the request is denied with the corresponding `DeniedReason`:
- `InvalidSignature`
- `RequestExpired`
- `ReplayDetected`
- `NotAuthorized`
### 6.5 Signature-Only Verification
A separate `check_signature_only()` path performs steps 1-3 (signature, timestamp, nonce) but **skips** step 4 (ACL check). This is used for the access request endpoint (`POST /api/auth/request`), where an unauthorized user needs to prove they own the key they're requesting access for.
### 6.6 Put Payload Note
`DatastoreAction::Put` references a `content_hash` rather than embedding raw file data. The bulk data is uploaded separately, and its integrity is guaranteed by blake3 content addressing. The signed envelope authorizes the *operation*, not the data transfer.
## 7. Replay Protection
### 7.1 Timestamp Window
- Requests must have a `timestamp` within ±300 seconds of the node's wall clock.
- Requests outside this window are rejected with `DeniedReason::RequestExpired`.
### 7.2 Nonce
- Each request includes a 16-byte random nonce.
- The node maintains a set of recently seen nonces in `seen_nonces: HashMap<[u8; 16], u64>`.
- Duplicate nonces within the time window are rejected with `DeniedReason::ReplayDetected`.
### 7.3 Nonce Garbage Collection
- Nonces are stored alongside their timestamps.
- When a nonce's timestamp falls outside the ±300 second window, it is eligible for GC.
- `gc_nonces(now)` is called periodically via `GatewayMsg::NonceGcTick`, which piggybacks on the main loop's GC tick cadence.
## 8. Enforcement Point
Auth is enforced at the **edge** of the actor system via the `GatewayActor`:
```
External Client
│
▼
┌─────────────┐
│ GatewayActor│◄── ACL check happens here
└──────┬──────┘
│
▼
┌──────────────┐ ┌─────────────────┐ ┌────────────────┐
│ MetadataActor│◄──►│ BlobStoreActor │ │ TransferActor │
│ │ │ │ │ │
│ (auth- │ │ (auth- │ │ (auth- │
│ unaware) │ │ unaware) │ │ unaware) │
└──────────────┘ └─────────────────┘ └────────────────┘
```
### 8.1 HTTP API Route Table
| Method | Path | Auth Level | Description |
|--------|------|------------|-------------|
| `GET` | `/` | None | Browser UI page |
| `GET` | `/admin` | None | Admin page |
| `GET` | `/crypto.wasm` | None | WASM Ed25519 module |
| `GET` | `/api/status` | None | Node identity |
| `POST` | `/api/put` | Full (`check_auth`) | Store an object |
| `GET` | `/api/get` | Full (`check_auth`) | Get object metadata |
| `GET` | `/api/data` | Full (`check_auth`) | Download object data |
| `POST` | `/api/delete` | Full (`check_auth`) | Delete an object |
| `GET` | `/api/list` | Full (`check_auth`) | List objects |
| `POST` | `/api/auth/grant` | Full (`check_auth_identity`) | Grant access to a key (owner-only) |
| `POST` | `/api/auth/revoke` | Full (`check_auth_identity`) | Revoke access from a key (owner-only) |
| `GET` | `/api/auth/requests` | Full (`check_auth_identity`) | List pending access requests (owner-only) |
| `GET` | `/api/auth/keys` | Full (`check_auth_identity`) | List authorized keys (owner-only) |
| `POST` | `/api/auth/deny` | Full (`check_auth_identity`) | Deny a pending request (owner-only) |
| `POST` | `/api/auth/request` | Signature-only (`check_auth_signature_only`) | Submit an access request |
**Auth levels:**
- **None** — no `X-Signed-Request` header required.
- **Full** — `X-Signed-Request` header required; full 4-step verification (signature + timestamp + nonce + ACL).
- **Signature-only** — `X-Signed-Request` header required; 3-step verification (signature + timestamp + nonce, no ACL check).
`check_auth_identity` is like `check_auth` but also returns the caller's `NodeId`, needed for grant/revoke/deny operations to identify the requester.
### 8.2 Internal Actors
`MetadataActor`, `BlobStoreActor`, and `TransferActor` remain **auth-unaware**. They process messages from any source within the actor system. The auth boundary is strictly external.
## 9. Browser Auth Flow
### 9.1 WASM Ed25519 Crypto
Browser clients use a WASM module (`/crypto.wasm`) compiled from `crates/crypto-wasm/` — a `no_std` Rust crate using `ed25519-dalek`. This replaces the earlier Web Crypto API approach, which has inconsistent Ed25519 support across browsers.
The WASM module exports three functions through a shared 8192-byte buffer:
| Function | Input | Output |
|----------|-------|--------|
| `buffer_ptr()` | — | Pointer to shared buffer |
| `get_public_key()` | `BUF[0..32]` = seed | `BUF[32..64]` = public key |
| `ed25519_sign(msg_len)` | `BUF[0..32]` = seed, `BUF[128..128+msg_len]` = message | `BUF[64..128]` = signature |
JavaScript wrapper functions:
```javascript
async function initCrypto() {
const { instance } = await WebAssembly.instantiate(
await (await fetch('/crypto.wasm')).arrayBuffer()
);
wasmExports = instance.exports;
bufPtr = wasmExports.buffer_ptr();
}
function derivePublicKey(seed) { /* write seed → read pubkey */ }
function signBytes(message, seed) { /* write seed+message → read signature */ }
```
### 9.2 Device Key Management
On first visit (when auth is detected), the browser:
1. Generates a 32-byte random seed: `crypto.getRandomValues(new Uint8Array(32))`
2. Stores it as hex in `localStorage.deviceKeySeed`
3. Derives the public key via `derivePublicKey(seed)`
On subsequent visits, the seed is loaded from localStorage. A migration path handles legacy JWK keys (from an earlier Web Crypto implementation) by extracting the `d` parameter as the seed.
### 9.3 Auth Detection
On page load, the browser fetches `GET /api/list` without auth:
- If the response is 401, auth is enabled → initialize WASM crypto, generate/load keys, show device key in header
- If the response is 200, auth is disabled → proceed normally
### 9.4 Request Signing
All authenticated browser requests go through `authFetch()`:
```javascript
async function authFetch(url, opts) {
const nonce = Array.from(crypto.getRandomValues(new Uint8Array(16)));
const payload = {
action: "Access",
timestamp: Math.floor(Date.now() / 1000),
nonce: nonce
};
const payloadBytes = new TextEncoder().encode(JSON.stringify(payload));
const sigBytes = signBytes(payloadBytes, deviceSeed);
const header = JSON.stringify({
payload: payload,
public_key: Array.from(pubKeyBytes),
signature: Array.from(sigBytes)
});
opts.headers['X-Signed-Request'] = header;
return fetch(url, opts);
}
```
The browser always uses `DatastoreAction::Access` — it proves identity without binding to a specific content operation. The HTTP API layer handles the actual data operation gating.
### 9.5 Access Request Flow
When a browser user is not yet authorized:
1. **Auth banner appears** — shows a form with name (required, max 64 chars) and message (optional, max 256 chars) fields.
2. **User submits** — `POST /api/auth/request` with JSON body `{ name, message }` and `X-Signed-Request` header (signature-only check).
3. **Pending state** — banner switches to "waiting for operator approval" with localStorage persistence (`accessRequestPending`, `accessRequestName`).
4. **Polling** — every 5 seconds, `authFetch('/api/list')` checks if the user has been granted access.
5. **Granted** — when `/api/list` returns 200, polling stops, banner disappears, object list loads.
6. **Re-submission on reload** — if the page is reloaded while pending, the request is re-submitted to handle node restarts.
## 10. Admin Page
The admin page (`/admin`) provides a browser interface for the datastore owner to manage access.
### 10.1 Authentication
The owner authenticates by uploading their `key.json` file:
1. File is parsed for `secret_key` (hex) and `public_key` (hex).
2. Public key is derived from the secret key via WASM and compared to the stored `public_key` for integrity.
3. A test call to `GET /api/auth/requests` verifies this is actually the owner key (non-owners get 403).
### 10.2 Capabilities
- **Pending access requests** — table showing name, message, key (truncated), with grant/deny buttons per request.
- **Authorized keys** — table showing label, key (truncated), with revoke button per key.
- **Manual grant** — input fields for a 64-char hex public key + optional name, bypassing the access request flow.
- **Name disambiguation** — when multiple entries share the same name, a key prefix `(abcd1234)` is appended for disambiguation.
### 10.3 Admin Request Signing
All admin API calls use `ownerAuthFetch()`, which signs with `DatastoreAction::Access` using the owner's seed.
## 11. CLI
### 11.1 Auth Signing
The CLI uses `--key <path>` to load a key.json file. Each command signs an `X-Signed-Request` header:
- **Data operations** (`put`, `get`, `delete`, `list`) sign with the corresponding `DatastoreAction` variant (e.g., `DatastoreAction::Put { name, content_hash, size_bytes, tags }`).
- **Auth management** (`grant`, `revoke`, `requests`, `keys`, `deny`) sign with `DatastoreAction::Access`.
- **`status`** — never signed (endpoint is always open).
- Without `--key`, no header is sent (backward compatible with non-auth nodes).
### 11.2 Subcommands
```
swactor-store --key <path> put <file> [--name <label>]
Upload a file. Signs DatastoreAction::Put.
swactor-store --key <path> get <hash> [--output <path>]
Retrieve metadata (or download with --output). Signs DatastoreAction::Get.
swactor-store --key <path> delete <hash>
Delete an object. Signs DatastoreAction::Delete.
swactor-store --key <path> list [--name <filter>] [--all]
List objects. Signs DatastoreAction::List.
swactor-store status
Show node identity. No signing.
swactor-store --key <path> grant <key_or_name> [--name <label>]
Authorize a public key. Owner-only. Accepts 64 hex chars or a name.
swactor-store --key <path> revoke <key_or_name>
Revoke a public key. Owner-only. Accepts 64 hex chars or a name.
swactor-store --key <path> requests
List pending access requests. Owner-only.
swactor-store --key <path> keys
List authorized keys with labels. Owner-only.
swactor-store --key <path> deny <key_or_name>
Deny a pending access request. Owner-only. Accepts 64 hex chars or a name.
```
### 11.3 Name Resolution
`grant`, `revoke`, and `deny` accept either:
- A **64-character hex public key** — used directly.
- A **human-readable name** — resolved by fetching the pending requests (`/api/auth/requests`) or authorized keys (`/api/auth/keys`) list and matching by name.
If multiple entries match the same name, the CLI prints disambiguated names (e.g., `alice (c9d0e1f2)`) and asks the user to re-run with the disambiguated form. The `(prefix)` suffix uses the first 8 hex characters of the key.
## 12. Key Management
### 12.1 Key File Format
All keys use the same JSON format:
```json
{
"version": 1,
"secret_key": "...64 hex chars (32 bytes ed25519 seed)...",
"public_key": "...64 hex chars (32 bytes ed25519 public key)...",
"created_at": "2026-02-15T12:00:00Z"
}
```
- Generated by the node on first `--auth` run at `<auth-dir>/owner.key.json`.
- The CLI reads it via `--key`.
- The admin page accepts it via file upload for authentication.
### 12.2 Node Key Generation
When `--auth` is enabled:
1. If `<auth-dir>/owner.key.json` exists, load the keypair from it.
2. Otherwise, generate a new `Keypair`, write the key file with ISO-8601 `created_at`.
3. The keypair's `node_id()` becomes the node's `NodeId` (deterministic identity across restarts).
4. Create/load `<auth-dir>/acl.json` with this `NodeId` as owner.
### 12.3 Browser Key Generation
Browser keys are simpler — 32 random bytes stored as hex in `localStorage.deviceKeySeed`. No key file is produced. The public key is derived on each page load via the WASM `get_public_key()` function.
### 12.4 Grant Flow
Two paths to granting access:
**Via access request (browser-initiated):**
1. Browser user visits the page, generates device key, submits access request with name.
2. Owner views pending requests on `/admin` or via `swactor-store requests`.
3. Owner grants via admin page button or `swactor-store grant <name_or_key>`.
4. Pending request is removed, name becomes key label, ACL is persisted.
5. Browser's polling detects the grant and loads the object list.
**Via manual grant (out-of-band):**
1. Client generates a keypair (or uses an existing one).
2. Client shares their public key with the owner out-of-band.
3. Owner runs: `swactor-store --key owner.key.json grant <pubkey> --name <label>`
4. Or: uses the admin page's "Grant Key Manually" form.
### 12.5 Revocation
1. Owner runs: `swactor-store --key owner.key.json revoke <pubkey_or_name>`
2. Or: clicks "revoke" on the admin page's authorized keys table.
3. Client's access is immediately revoked for HTTP requests.
4. Existing direct iroh connections from that client remain open until disconnected.
## 13. Protocol Integration
Each datastore operation has a clear auth integration point:
| Operation | CLI Signing | Browser Signing |
|-----------|-------------|-----------------|
| PUT | `DatastoreAction::Put { name, content_hash, size_bytes, tags }` | `DatastoreAction::Access` |
| GET | `DatastoreAction::Get { content_hash }` | `DatastoreAction::Access` |
| DELETE | `DatastoreAction::Delete { content_hash }` | `DatastoreAction::Access` |
| LIST | `DatastoreAction::List { name_filter }` | `DatastoreAction::Access` |
| Grant/Revoke/etc. | `DatastoreAction::Access` | `DatastoreAction::Access` |
The browser uses `Access` for all operations because:
- Computing content hashes client-side would add complexity to the browser JS.
- The HTTP API already gates the actual data operation — the signed request only needs to prove identity.
- The `Access` action maps to `DatastoreNodeMsg::Status` in the gateway (a lightweight no-op that returns a valid response).
The CLI uses per-action signing for data operations because it has access to the `ContentHash` and can construct precise action payloads.
In all cases, auth is enforced *before* the request reaches the actor system. Internal inter-node communication (DHT replication, chunk transfers between cluster members) is not subject to auth checks.
## 14. Future Extensions
These are explicitly **out of scope** for MVP but inform the design:
- **Per-path permission scoping** — restrict a key to specific path prefixes.
- **Permission tiers** — read-only, read-write, admin roles.
- **Capability tokens** — time-limited, scope-limited bearer tokens for delegated access.
- **Multi-level delegation** — allow authorized users to grant limited access to others.
- **Connection tracking** — forcibly disconnect revoked keys from active iroh sessions.
- **Persistent access requests** — currently in-memory only; lost on node restart (browser re-submits on reload as mitigation).

386
docs/development_history/datastore-auth/SUMMARY.md Normal file
View file

@ -0,0 +1,386 @@
# Datastore Auth: Development History
**Branch:** `swactor-auth`
**Base commit:** `af15416` (pre-auth baseline)
**5 commits + uncommitted working tree changes**
---
## What Was Built
A complete ed25519 authorization layer for the distributed datastore, spanning:
- **Auth engine** — `AuthzEngine` with ACL, signed request verification, replay protection, nonce GC
- **GatewayActor** — actor-level enforcement point with grant/revoke, access requests, key listing
- **Browser auth flow** — WASM Ed25519 crypto, device key generation, access request/grant/deny lifecycle
- **Admin page** — owner key upload, pending request management, manual key grant, authorized key list
- **Expanded CLI** — full CRUD + auth subcommands (`grant`, `revoke`, `requests`, `keys`, `deny`) with name resolution
- **Storage persistence** — entry/manifest persistence to filesystem, startup bulk-load
- **xtask** — `node`, `cli`, `wasm` subcommands with `config.toml` support
- **WASM crypto crate** — `crates/crypto-wasm/`, a `no_std` cdylib exporting `ed25519_sign()`, `get_public_key()`, `buffer_ptr()`
The auth system enforces binary access control (authorized or not) at the HTTP API boundary. Internal actors remain auth-unaware. Two auth paths: connection-level (iroh QUIC handshake proves NodeId) and per-request signed envelopes (for browser/HTTP API). This branch implements Path 2 end-to-end, including the browser UX.
---
## Architecture
```
┌──────────────────────────────────────────────────────┐
│ HTTP API (api.rs) │
│ │
│ Ungated: │
│ GET / → browser UI (access page) │
│ GET /admin → admin page │
│ GET /crypto.wasm → WASM Ed25519 module │
│ GET /api/status → node identity │
│ │
│ Auth-gated (X-Signed-Request header): │
│ POST /api/put → check_auth → handle_put │
│ GET /api/get → check_auth → handle_get │
│ GET /api/data → check_auth → handle_data │
│ POST /api/delete → check_auth → handle_delete│
│ GET /api/list → check_auth → handle_list │
│ │
│ Auth management (owner-only): │
│ POST /api/auth/grant → check_auth_identity │
│ POST /api/auth/revoke → check_auth_identity │
│ GET /api/auth/requests→ check_auth_identity │
│ GET /api/auth/keys → check_auth_identity │
│ POST /api/auth/deny → check_auth_identity │
│ │
│ Signature-only (proves key, no ACL check): │
│ POST /api/auth/request → check_auth_sig_only │
└───────────────┬─────────────────────────────────────┘
│
GatewayMsg (various)
│
┌───────────────▼───────────────┐
│ GatewayActor │
│ │
│ AuthzEngine: │
│ 1. verify ed25519 signature │
│ 2. check timestamp ±300s │
│ 3. check nonce uniqueness │
│ 4. check ACL │
│ │
│ Access request management: │
│ pending_requests HashMap │
│ grant resolves label from │
│ pending request name │
│ │
│ ACL persistence: │
│ persist_acl() on grant/ │
│ revoke │
└───────────────┬───────────────┘
│
┌───────────────▼───────────────┐
│ DatastoreNode │
│ │
│ MetadataActor ◄──► BlobStore │
│ (auth-unaware) │
└───────────────────────────────┘
┌───────────────────────────────┐
│ Browser (WASM Ed25519) │
│ │
│ /crypto.wasm → initCrypto() │
│ deviceKeySeed in localStorage │
│ signBytes() per request │
│ Access action for all ops │
│ → X-Signed-Request header │
└───────────────────────────────┘
┌───────────────────────────────┐
│ CLI (store_cli) │
│ │
│ --key owner.key.json │
│ Per-action signing: │
│ Put/Get/Delete/List/Access │
│ Name resolution for │
│ grant/revoke/deny │
└───────────────────────────────┘
```
---
## Commit-by-Commit
### `863185e` — feat: distributed datastore primitives protocol
Foundation commit establishing the distributed datastore protocol. Defined the protocol messages (`GetChunkRequest`, `FindObjectRequest`, `StoreObjectRequest`, `ListObjectsRequest` and their responses), all implementing `NetworkMessage` with stable `type_tag()` strings. This is the wire protocol for inter-node communication over iroh/QUIC.
**Key files:** `src/messages.rs` (inter-node message types)
### `84c4408` — fix: cli for datastore works
Brought up the `store_node` and `store_cli` binaries as `[[bin]]` targets with feature-gated dependencies (`node` and `cli` features). The node binary spawns the actor runtime, wires up BlobStoreActor/MetadataActor/DatastoreNode, and serves the HTTP API via `tiny_http`. The CLI binary talks to the node over HTTP with `ureq`. Added `clap` for arg parsing, `ctrlc` for graceful shutdown, and `runtime-dashboard` integration.
**Key files:** `Cargo.toml` (features `node`/`cli`), `src/bin/store_node.rs`, `src/bin/store_cli.rs`, `src/api.rs`
### `ebee109` — feat: mvp auth protocol
Core auth implementation:
- **`src/auth.rs`** — `DatastoreAction` enum, `SignedRequestPayload`, `SignedRequest` envelope, `AccessControlList` (with JSON persistence via `save()`/`load_or_create()`), `AuthzEngine` (4-step verification: signature, timestamp, nonce, ACL), `sign_request()`/`verify_signed_request()` helpers, `AuthzResult`/`DeniedReason` enums.
- **`src/actors/gateway.rs`** — `GatewayActor` wrapping `AuthzEngine`. Handles `Authorize` (pure auth check), `HandleSignedRequest` (auth + dispatch via `action_to_node_msg()`), `CheckConnection` (Path 1), `Grant`/`Revoke` (owner-only ACL mutations), `NonceGcTick`.
- **`src/messages.rs`** — `GatewayMsg` enum, `DatastoreResponse::Denied` variant.
- **`crates/shared-types/`** — Extracted `ContentHash` into its own crate to break dependency cycles between `distribution` and `datastore`.
Tests added (18 total):
- `auth_scenario_tests.rs` (10 tests) — owner access, stranger denial, grant/revoke lifecycle, signed request happy path, tampered signature, stale timestamp, replayed nonce, nonce GC, non-owner grant/revoke rejection.
- `acl_persistence_tests.rs` (2 tests) — save/load round-trip, create-on-missing.
- `gateway_tests.rs` (4 tests) — connection allow/deny, signed request flow-through, unauthorized signed request denial.
### `8ac45e5` — fix: adjust auth protocol to datastore protocol
Aligned the auth types with the content-hash-first datastore protocol:
- `DatastoreAction::Put` carries `content_hash`, `size_bytes`, and `tags` (not raw data).
- `DatastoreAction::Get`/`Delete` use `content_hash`.
- `DatastoreAction::List` uses `name_filter`.
- `GatewayActor::action_to_node_msg()` maps actions to `DatastoreNodeMsg` variants.
- Wired `check_auth()` into the HTTP API handlers (put, get, data, delete, list) — reads `X-Signed-Request` header, sends `GatewayMsg::Authorize` to the gateway actor, denies with 401/403/504 on failure.
- `handle_status` intentionally left ungated.
### `e549eef` — feat: auth MVP with integrated tests
Wired auth into both binaries:
**`store_node.rs`** — `--auth` and `--auth-dir <PATH>` flags:
- Loads or generates owner keypair from `<auth-dir>/owner.key.json`.
- Owner keypair's public key becomes the `NodeId` (deterministic identity across restarts).
- Loads/creates `<auth-dir>/acl.json` with owner as sole authorized key.
- Spawns `GatewayActor` and passes `Some(gateway_addr)` to `start_api_server`.
- Sends `GatewayMsg::NonceGcTick` on the same cadence as the metadata GC tick.
**`store_cli.rs`** — `--key <PATH>` flag:
- Each command builds the appropriate `DatastoreAction`, signs it, sends as `X-Signed-Request` header.
- `status` never signs (always open by design).
**`http_auth_integration.rs`** — Full-stack integration test: spins up the actor runtime with GatewayActor, starts the HTTP server, proves owner is allowed (PUT/GET/LIST/DELETE), stranger gets 403, missing header gets 401.
---
## Uncommitted Working Tree Changes
The uncommitted changes represent the bulk of the user-facing work: browser UI, admin page, WASM crypto, expanded CLI, storage persistence, and xtask.
### Browser UI (`ui_html.rs` — `DATASTORE_UI_HTML`)
Complete browser access page served at `/`:
- **Upload panel** — file input + optional name, PUT via `authFetch()`
- **Object table** — list all objects with hash, name, size; download and delete buttons
- **Detail modal** — click a row to see full metadata, chunks, tags
- **Auth detection** — on load, `detectAuth()` fetches `/api/list`; if 401, enables auth mode
- **WASM crypto integration** — `initCrypto()` fetches `/crypto.wasm`, `initKeys()` generates or loads device seed from `localStorage`, derives public key via WASM
- **Auth banner** — shown when user is not authorized, with access request form (name + optional message)
- **Pending state** — after submitting request, shows "waiting for operator approval" with 5-second polling; auto-refreshes when granted
- **Device key display** — shows truncated public key hex in header when auth is active
- **JWK migration** — handles legacy `localStorage.deviceKey` (JWK format) by extracting the `d` parameter as seed
### Admin Page (`ui_html.rs` — `DATASTORE_ADMIN_HTML`)
Owner administration page served at `/admin`:
- **Owner key upload** — file input for `key.json`, loads secret/public key hex, derives via WASM to verify, test call to `/api/auth/requests` to confirm ownership
- **Pending access requests table** — name, message, key (truncated), grant/deny buttons
- **Authorized keys table** — name (label), key (truncated), revoke button
- **Manual grant form** — input for 64-char hex public key + optional name
- **Name disambiguation** — when multiple entries share the same name, appends `(key_prefix)` suffix
- **`ownerAuthFetch()`** — signs all admin API calls with `DatastoreAction::Access`
### WASM Ed25519 Crypto (`crates/crypto-wasm/`)
New `no_std` Rust crate compiled to `wasm32-unknown-unknown`:
- **`Cargo.toml`** — `swactor-crypto-wasm`, `cdylib` crate type, depends on `ed25519-dalek` (no default features)
- **`src/lib.rs`** — Three exported functions:
- `buffer_ptr()` → pointer to 8192-byte shared buffer
- `get_public_key()` — reads 32-byte seed from `BUF[0..32]`, writes public key to `BUF[32..64]`
- `ed25519_sign(msg_len)` — reads seed from `BUF[0..32]`, message from `BUF[128..128+msg_len]`, writes 64-byte signature to `BUF[64..128]`
- **`crypto_wasm.wasm`** — pre-built binary embedded in the datastore via `include_bytes!("crypto_wasm.wasm")`
- Served at `/crypto.wasm` endpoint (ungated)
- Replaces the earlier Web Crypto API approach — Web Crypto's Ed25519 support is inconsistent across browsers; WASM provides deterministic behavior using the same `ed25519-dalek` crate as the Rust backend
### Expanded GatewayActor (`actors/gateway.rs`)
New message handlers beyond the original `Authorize`/`HandleSignedRequest`/`CheckConnection`/`Grant`/`Revoke`:
- **`VerifySignature`** — calls `check_signature_only()` (no ACL check). Used for access request submissions where the caller needs to prove key ownership without being in the ACL.
- **`SubmitAccessRequest`** — stores `AccessRequestInfo { key, name, message, requested_at }` in `pending_requests: HashMap<NodeId, AccessRequestInfo>`.
- **`ListAccessRequests`** — owner-only; returns all pending requests.
- **`DenyAccessRequest`** — owner-only; removes a pending request.
- **`ListAuthorizedKeys`** — owner-only; returns `Vec<AuthorizedKeyInfo>` with labels.
Grant now resolves labels: when granting a key that has a pending request, the request's `name` field becomes the key's label (unless an explicit label is provided).
### Expanded Auth Types (`auth.rs`)
- **`AccessRequestInfo`** — `{ key: NodeId, name: String, message: String, requested_at: u64 }`
- **`AuthorizedKeyInfo`** — `{ key: NodeId, label: String }`
- **`DatastoreAction::Access`** — new variant for browser-originated requests that prove identity without binding to specific content. The browser uses `Access` for all operations (auth is at the HTTP layer).
- **`key_labels: HashMap<String, String>`** added to `AccessControlList` — maps hex public key to human-readable name. Populated by `grant()`, removed by `revoke()`.
- **`check_signature_only()`** on `AuthzEngine` — verifies signature, timestamp, and nonce but skips ACL check.
- **`authorized_key_list()`** on `AuthzEngine` — returns all authorized keys with their labels.
### Storage Persistence (`storage/mod.rs`, `storage/in_memory.rs`)
Extended `StorageBackend` trait with entry persistence:
- **`write_entry()`** / **`read_entry()`** / **`delete_entry()`** / **`list_entries()`** — persist `ObjectEntry` JSON to disk
- **`FilesystemBackend`** layout extended:
```
{root}/
├── chunks/{hex[0..2]}/{hex[2..4]}/{full_hex_hash}
├── manifests/{hex[0..2]}/{hex[2..4]}/{full_hex_hash}
└── entries/{hex[0..2]}/{hex[2..4]}/{full_hex_hash}
```
- **`BlobStoreMsg::WriteEntry`** / **`DeleteEntry`** — fire-and-forget messages for entry persistence
- **`BlobStoreMsg::LoadAll`** — startup bulk-load of all entries + their manifests
- **`MetadataMsg::BulkLoad`** — injects loaded entries into MetadataActor's index
- **`store_node.rs` startup sequence** — sends `LoadAll` to BlobStoreActor, polls for `LoadedAll` response, sends `BulkLoad` to MetadataActor
### Expanded CLI (`store_cli.rs`)
Full CRUD + auth management subcommands:
| Subcommand | Auth | Description |
|------------|------|-------------|
| `put <path> [--name]` | `--key` signs `DatastoreAction::Put` | Upload a file |
| `get <hash> [--output]` | `--key` signs `DatastoreAction::Get` | Metadata or download |
| `delete <hash>` | `--key` signs `DatastoreAction::Delete` | Delete an object |
| `list [--name] [--all]` | `--key` signs `DatastoreAction::List` | List objects |
| `status` | Never signed | Node identity |
| `grant <key_or_name> [--name]` | `--key` signs `Access` | Authorize a key (owner-only) |
| `revoke <key_or_name>` | `--key` signs `Access` | Revoke a key (owner-only) |
| `requests` | `--key` signs `Access` | List pending access requests |
| `keys` | `--key` signs `Access` | List authorized keys |
| `deny <key_or_name>` | `--key` signs `Access` | Deny a pending request |
**Name resolution:** `grant`, `revoke`, and `deny` accept either a 64-char hex key or a human-readable name. When given a name, the CLI fetches the relevant list from the API and resolves the name to a key. Disambiguated names (`"alice (c9d0e1f2)"`) are supported.
### xtask (`xtask/src/main.rs`)
Development task runner with three new subcommands beyond the existing `test`:
- **`cargo xtask node`** — builds and runs `swactor-store-node`. Flags: `--port`, `--storage-path`, `--auth` (default: true), `--auth-dir`. Builds with `--features node` first, then runs the binary directly (not via `cargo run`) to avoid SIGINT issues. Ignores SIGINT in the xtask process so the child handles Ctrl-C.
- **`cargo xtask cli`** — builds and runs `swactor-store`. Flags: `--url`, `--key`. Auto-detects `./auth/owner.key.json` if present. Passes extra args through.
- **`cargo xtask wasm`** — builds `swactor-crypto-wasm` for `wasm32-unknown-unknown --release`, copies the output to `crates/datastore/src/crypto_wasm.wasm`, optionally runs `wasm-strip`.
- **`config.toml` support** — reads `xtask/config.toml` for default values (node port, storage path, auth settings, CLI url/key).
**`xtask/Cargo.toml`** — added `toml`, `serde`, `libc` dependencies.
### HTTP API Expansion (`api.rs`)
New endpoints:
| Method | Path | Auth | Handler |
|--------|------|------|---------|
| `POST` | `/api/auth/grant?key=<hex>[&name=<label>]` | Owner (full check) | `handle_auth_grant` |
| `POST` | `/api/auth/revoke?key=<hex>` | Owner (full check) | `handle_auth_revoke` |
| `POST` | `/api/auth/request` | Signature-only | `handle_auth_request` |
| `GET` | `/api/auth/requests` | Owner (full check) | `handle_auth_requests_list` |
| `GET` | `/api/auth/keys` | Owner (full check) | `handle_auth_keys_list` |
| `POST` | `/api/auth/deny?key=<hex>` | Owner (full check) | `handle_auth_deny` |
| `GET` | `/` | None | Browser UI |
| `GET` | `/admin` | None | Admin page |
| `GET` | `/crypto.wasm` | None | WASM module |
New internal functions:
- `check_auth_identity()` — like `check_auth()` but returns the caller's `NodeId` (needed for grant/revoke to identify the requester).
- `check_auth_signature_only()` — verifies signature without ACL check (for access request submission).
- `respond_wasm()`, `respond_admin_html()` — serve the new static assets.
- `CRYPTO_WASM` constant — `include_bytes!("crypto_wasm.wasm")`.
### DatastoreResponse Expansion (`messages.rs`)
New response variants:
- `AccessRequests { requests: Vec<AccessRequestInfo> }` — response to `ListAccessRequests`
- `AuthorizedKeys { keys: Vec<AuthorizedKeyInfo> }` — response to `ListAuthorizedKeys`
- `LoadedAll { entries: Vec<(ObjectEntry, ObjectManifest)> }` — response to `BlobStoreMsg::LoadAll`
---
## Key File Format
`owner.key.json` / any client `key.json`:
```json
{
"version": 1,
"secret_key": "...64 hex chars (32 bytes)...",
"public_key": "...64 hex chars (32 bytes)...",
"created_at": "2026-02-15T12:00:00Z"
}
```
Generated by the node on first `--auth` run. The CLI reads it via `--key`. The admin page uploads it for authentication. The browser generates a simpler device seed (32 random bytes stored as hex in `localStorage.deviceKeySeed`).
---
## Test Summary
| Test File | Count | What |
|-----------|-------|------|
| `auth_scenario_tests.rs` | 10 | AuthzEngine: signing, verification, timestamp, nonce, ACL, grant/revoke |
| `acl_persistence_tests.rs` | 2 | ACL JSON round-trip, create-on-missing |
| `gateway_tests.rs` | 4 | GatewayActor: connection check, signed request flow, denial |
| `http_auth_integration.rs` | 1 | Full HTTP stack: owner PUT/GET/LIST/DELETE, stranger 403, no-header 401 |
| **Auth total** | **17** | |
Pre-existing datastore tests (blob_store, metadata, datastore_node, chunking, gc, storage, transfer, multi_node, api_integration, dashboard_integration) continue to pass.
---
## Design Decisions
1. **WASM Ed25519 over Web Crypto** — Web Crypto's Ed25519 support varies by browser (Safari lacking, Firefox gated behind flags as of early 2026). A WASM module using `ed25519-dalek` with `no_std` gives deterministic, cross-browser behavior and byte-level compatibility with the Rust backend. The compiled module is ~27KB stripped.
2. **`DatastoreAction::Access` for browser ops** — The browser signs a lightweight `Access` action for every API call rather than constructing per-operation payloads. This simplifies the browser JS (no need to compute content hashes client-side) while still proving identity. The actual data operations are auth-gated at the HTTP layer.
3. **Signature-only check for access requests** — `POST /api/auth/request` uses `check_auth_signature_only()` which verifies the signature/timestamp/nonce but skips the ACL check. This allows an unauthorized user to prove key ownership when requesting access, without being in the ACL yet.
4. **Key labels in ACL** — `key_labels: HashMap<String, String>` maps hex public key to human-readable name. Labels are set on grant (from the access request's `name` field or an explicit `--name` flag) and removed on revoke. This enables the admin page and CLI to show meaningful names instead of raw hex keys.
5. **Access request flow** — Instead of requiring out-of-band key exchange, browser users can submit an access request with their name and a message. The request is stored in-memory in the GatewayActor's `pending_requests`. The owner can grant or deny from the admin page or CLI. On grant, the pending request is removed and its name becomes the key label.
6. **Entry persistence** — `StorageBackend` trait extended with `write_entry()`/`read_entry()`/`delete_entry()`/`list_entries()`. The `FilesystemBackend` stores entries as JSON files in a `entries/` directory with the same 2-level hex sharding as chunks. On startup, `BlobStoreMsg::LoadAll` reads all entries and their manifests, then `MetadataMsg::BulkLoad` injects them into the MetadataActor's index. This means stored objects survive node restarts.
7. **xtask builds then execs** — `cargo xtask node` and `cargo xtask cli` build the binary first, then exec it directly (not via `cargo run`). This avoids cargo sitting in the process chain and dying from SIGINT before the node finishes its shutdown sequence.
8. **Status endpoint stays open** — `/api/status`, `/`, `/admin`, and `/crypto.wasm` are never auth-gated. Status enables health checks; the UI/admin pages need to be loadable before authentication; the WASM module is needed to perform authentication.
9. **ACL persisted to auth-dir** — The ACL is stored at `<auth-dir>/acl.json` (default: `./auth/acl.json`), not inside the storage path. This separates auth config from data storage.
10. **CLI name resolution** — `grant`, `revoke`, and `deny` accept human-readable names in addition to hex keys. When given a name, the CLI fetches the pending requests or authorized keys list from the API and resolves the name. If multiple entries match, it prints disambiguated names (e.g., `"alice (c9d0e1f2)"`) and asks the user to re-run.
---
## File Inventory
| File | What |
|------|------|
| `crates/shared-types/` | `ContentHash` crate (breaks dependency cycles) |
| `crates/crypto-wasm/Cargo.toml` | WASM crypto crate config |
| `crates/crypto-wasm/src/lib.rs` | `no_std` Ed25519 sign/derive/buffer exports |
| `crates/datastore/src/crypto_wasm.wasm` | Pre-built WASM binary (embedded via `include_bytes!`) |
| `crates/datastore/Cargo.toml` | Feature flags (`node`/`cli`), dependencies |
| `crates/datastore/src/auth.rs` | Auth engine, ACL, signing, verification, access request types |
| `crates/datastore/src/actors/gateway.rs` | GatewayActor — auth enforcement + access request management |
| `crates/datastore/src/actors/blob_store.rs` | BlobStoreActor — entry persistence, LoadAll |
| `crates/datastore/src/actors/metadata.rs` | MetadataActor — entry persistence writes, BulkLoad |
| `crates/datastore/src/messages.rs` | GatewayMsg, BlobStoreMsg (WriteEntry/DeleteEntry/LoadAll), DatastoreResponse extensions |
| `crates/datastore/src/api.rs` | HTTP API — auth endpoints, WASM/admin serving, auth checking functions |
| `crates/datastore/src/ui_html.rs` | Browser UI (access page) + Admin page HTML/CSS/JS |
| `crates/datastore/src/storage/mod.rs` | StorageBackend trait (entry methods), FilesystemBackend |
| `crates/datastore/src/storage/in_memory.rs` | InMemoryBackend (entry methods) |
| `crates/datastore/src/bin/store_node.rs` | Node binary — `--auth`, `--auth-dir`, keypair mgmt, gateway spawn, bulk-load |
| `crates/datastore/src/bin/store_cli.rs` | CLI binary — `--key`, all subcommands, name resolution |
| `xtask/Cargo.toml` | xtask dependencies (toml, serde, libc) |
| `xtask/src/main.rs` | `node`, `cli`, `wasm` subcommands, `config.toml` support |
| `docs/datastore/DATASTORE_AUTH.md` | Auth specification document |
| `tests/auth_scenario_tests.rs` | 10 AuthzEngine scenario tests |
| `tests/acl_persistence_tests.rs` | 2 ACL persistence tests |
| `tests/gateway_tests.rs` | 4 GatewayActor tests |
| `tests/http_auth_integration.rs` | 1 full-stack HTTP auth integration test |

6
xtask/Cargo.toml
View file

@ -2,3 +2,9 @@
name = "xtask"
version = "0.1.0"
edition = "2024"
[dependencies]
clap = { version = "4", features = ["derive"] }
toml = "0.8"
serde = { version = "1", features = ["derive"] }
libc = "0.2"

361
xtask/src/main.rs
View file

@ -1,6 +1,139 @@
use std::path::{Path, PathBuf};
use std::process::Command;
use std::time::Instant;
use clap::{Parser, Subcommand};
use serde::Deserialize;
// ── Signal handling ─────────────────────────────────────────────────
/// Ignore SIGINT in this process so the child handles Ctrl-C.
/// Without this, xtask dies immediately on Ctrl-C and the shell
/// shows a prompt before the child's shutdown messages finish.
#[cfg(unix)]
fn ignore_sigint() {
unsafe { libc::signal(libc::SIGINT, libc::SIG_IGN); }
}
#[cfg(not(unix))]
fn ignore_sigint() {}
// ── CLI ─────────────────────────────────────────────────────────────
#[derive(Parser)]
#[command(name = "xtask", about = "Development task runner")]
struct Cli {
#[command(subcommand)]
command: Cmd,
}
#[derive(Subcommand)]
enum Cmd {
/// Run test groups
Test {
/// Test group to run (core, distribution, cluster-sims, integrated, essential, all)
group: Option<String>,
/// Show all groups and the cargo commands they run
#[arg(long)]
list: bool,
},
/// Start a datastore node
#[command(trailing_var_arg = true)]
Node {
/// Port for the node
#[arg(long)]
port: Option<u16>,
/// Storage path
#[arg(long)]
storage_path: Option<String>,
/// Enable auth (bare --auth → true, --auth=false → false)
#[arg(long, num_args = 0..=1, default_missing_value = "true")]
auth: Option<bool>,
/// Auth directory
#[arg(long)]
auth_dir: Option<String>,
/// Extra arguments forwarded to the underlying binary
#[arg(allow_hyphen_values = true)]
extra: Vec<String>,
},
/// Build the crypto WASM module
Wasm,
/// Run a datastore CLI command
#[command(trailing_var_arg = true)]
Cli {
/// Node URL
#[arg(long)]
url: Option<String>,
/// Path to key file
#[arg(long)]
key: Option<String>,
/// Extra arguments forwarded to the underlying binary
#[arg(allow_hyphen_values = true)]
extra: Vec<String>,
},
}
// ── Config file ─────────────────────────────────────────────────────
#[derive(Deserialize, Default)]
struct Config {
#[serde(default)]
node: NodeConfig,
#[serde(default)]
cli: CliConfig,
}
#[derive(Deserialize, Default)]
struct NodeConfig {
port: Option<u16>,
storage_path: Option<String>,
auth: Option<bool>,
auth_dir: Option<String>,
}
#[derive(Deserialize, Default)]
struct CliConfig {
url: Option<String>,
key: Option<String>,
}
fn load_config(root: &Path) -> Config {
let path = root.join("xtask/config.toml");
match std::fs::read_to_string(&path) {
Ok(content) => toml::from_str(&content).unwrap_or_else(|e| {
eprintln!("Warning: failed to parse {}: {e}", path.display());
Config::default()
}),
Err(_) => Config::default(),
}
}
// ── Workspace root ──────────────────────────────────────────────────
fn workspace_root() -> PathBuf {
let mut dir = std::env::current_dir().expect("cannot determine current directory");
loop {
if dir.join("Cargo.toml").exists() && dir.join("xtask").is_dir() {
return dir;
}
if !dir.pop() {
panic!("could not find workspace root (Cargo.toml + xtask/ dir)");
}
}
}
// ── Test infrastructure (unchanged) ─────────────────────────────────
struct TestStep {
label: &'static str,
args: &'static [&'static str],
@ -140,30 +273,26 @@ fn print_list() {
println!(" {:<14}Every test group", "all");
}
fn main() {
let args: Vec<String> = std::env::args().collect();
// ── Dispatch ────────────────────────────────────────────────────────
if args.len() < 2 || args[1] != "test" {
print_usage();
std::process::exit(if args.len() < 2 { 1 } else { 1 });
}
if args.len() < 3 {
print_usage();
std::process::exit(1);
}
let target = &args[2];
if target == "--list" {
fn run_test(group: Option<String>, list: bool) {
if list {
print_list();
return;
}
let groups = match groups_for(target) {
let group_name = match group {
Some(g) => g,
None => {
eprintln!("Unknown test group: {target}\n");
print_usage();
std::process::exit(1);
}
};
let groups = match groups_for(&group_name) {
Some(g) => g,
None => {
eprintln!("Unknown test group: {group_name}\n");
print_usage();
std::process::exit(1);
}
@ -195,3 +324,201 @@ fn main() {
elapsed.as_secs_f64()
);
}
fn run_node(
port: Option<u16>,
storage_path: Option<String>,
auth: Option<bool>,
auth_dir: Option<String>,
extra: Vec<String>,
cfg: &NodeConfig,
) {
ignore_sigint();
let port = port.or(cfg.port).unwrap_or(9091);
let storage_path = storage_path
.or_else(|| cfg.storage_path.clone())
.unwrap_or_else(|| "./datastore".into());
let auth_enabled = auth.or(cfg.auth).unwrap_or(true);
let auth_dir = auth_dir
.or_else(|| cfg.auth_dir.clone())
.unwrap_or_else(|| "./auth".into());
// Build first, then run the binary directly (not via `cargo run`).
// This avoids cargo sitting in the middle of the process chain and
// dying from SIGINT before the node finishes its shutdown.
let build_status = Command::new("cargo")
.args([
"build", "-p", "swactor-datastore", "--features", "node",
"--bin", "swactor-store-node",
])
.status();
match build_status {
Ok(s) if !s.success() => std::process::exit(s.code().unwrap_or(1)),
Err(e) => {
eprintln!("Failed to execute cargo build: {e}");
std::process::exit(1);
}
_ => {}
}
// Locate the built binary
let root = workspace_root();
let binary = root.join("target/debug/swactor-store-node");
if !binary.exists() {
eprintln!("Binary not found at {}", binary.display());
std::process::exit(1);
}
let mut bin_args: Vec<String> = vec![
"--port".into(),
port.to_string(),
"--storage-path".into(),
storage_path,
];
if auth_enabled {
bin_args.push("--auth".into());
bin_args.push("--auth-dir".into());
bin_args.push(auth_dir);
}
bin_args.extend(extra);
let status = Command::new(&binary).args(&bin_args).status();
match status {
Ok(s) if !s.success() => std::process::exit(s.code().unwrap_or(1)),
Err(e) => {
eprintln!("Failed to execute {}: {e}", binary.display());
std::process::exit(1);
}
_ => {}
}
}
fn run_cli(
url: Option<String>,
key: Option<String>,
extra: Vec<String>,
cfg: &CliConfig,
) {
ignore_sigint();
let url = url
.or_else(|| cfg.url.clone())
.unwrap_or_else(|| "http://localhost:9091".into());
let key = key.or_else(|| cfg.key.clone()).or_else(|| {
// Only default to owner.key.json if the file exists
let default_path = "./auth/owner.key.json";
if Path::new(default_path).exists() {
Some(default_path.into())
} else {
None
}
});
// Build first, then run the binary directly.
let build_status = Command::new("cargo")
.args([
"build", "-p", "swactor-datastore", "--features", "cli",
"--bin", "swactor-store",
])
.status();
match build_status {
Ok(s) if !s.success() => std::process::exit(s.code().unwrap_or(1)),
Err(e) => {
eprintln!("Failed to execute cargo build: {e}");
std::process::exit(1);
}
_ => {}
}
let root = workspace_root();
let binary = root.join("target/debug/swactor-store");
let mut bin_args: Vec<String> = vec![
"--url".into(),
url,
];
if let Some(key) = key {
bin_args.push("--key".into());
bin_args.push(key);
}
bin_args.extend(extra);
let status = Command::new(&binary).args(&bin_args).status();
match status {
Ok(s) if !s.success() => std::process::exit(s.code().unwrap_or(1)),
Err(e) => {
eprintln!("Failed to execute {}: {e}", binary.display());
std::process::exit(1);
}
_ => {}
}
}
fn run_wasm() {
let root = workspace_root();
println!("Building crypto WASM module...");
let status = Command::new("cargo")
.args([
"build",
"--target", "wasm32-unknown-unknown",
"--release",
"-p", "swactor-crypto-wasm",
])
.status();
match status {
Ok(s) if !s.success() => {
eprintln!("WASM build failed");
std::process::exit(s.code().unwrap_or(1));
}
Err(e) => {
eprintln!("Failed to execute cargo build: {e}");
std::process::exit(1);
}
_ => {}
}
let src = root.join("target/wasm32-unknown-unknown/release/swactor_crypto_wasm.wasm");
let dst = root.join("crates/datastore/src/crypto_wasm.wasm");
std::fs::copy(&src, &dst).unwrap_or_else(|e| {
eprintln!("Failed to copy {} → {}: {e}", src.display(), dst.display());
std::process::exit(1);
});
let size = std::fs::metadata(&dst).map(|m| m.len()).unwrap_or(0);
println!("Copied {} ({} bytes)", dst.display(), size);
// Try wasm-strip if available (optional optimization)
if Command::new("wasm-strip").arg(&dst).status().is_ok() {
let stripped_size = std::fs::metadata(&dst).map(|m| m.len()).unwrap_or(0);
println!("Stripped to {} bytes", stripped_size);
}
}
fn main() {
let cli = Cli::parse();
let root = workspace_root();
let config = load_config(&root);
match cli.command {
Cmd::Test { group, list } => run_test(group, list),
Cmd::Wasm => run_wasm(),
Cmd::Node {
port,
storage_path,
auth,
auth_dir,
extra,
} => run_node(port, storage_path, auth, auth_dir, extra, &config.node),
Cmd::Cli {
url,
key,
extra,
} => run_cli(url, key, extra, &config.cli),
}
}