Move iroh_driver and the relay binary out of distribution into a dedicated
crates/iroh-driver (lib re-exports IrohDriver; relay bin renamed). Remove the node crate
and the single-gpu-inference example; drop the docker/datastream demo. Slim
pipeline-parallel vastai.
Signed-off-by: Zachery Aaron Shores-Chmielewski <zacheryasc@gmail.com>
Remove the datastore crate, the top-level design/orchestration/ring specs, the
benches, and the ci config. Add the dashboard host telemetry sampler
(cpu/disk/net/gpu/mem). Localize the pipeline-parallel e2e stub/mock paths.
Signed-off-by: Zachery Aaron Shores-Chmielewski <zacheryasc@gmail.com>
Promote pipeline-parallel-inference to a first-class app and consolidate observability on the datastream wire, decoupling the dashboard crate from `distribution`.
- apps/pipeline-parallel-inference: move the example out of `examples/` into `apps/` as its own workspace, rename binaries to `pp-worker`/`pp-orchestrator`, and strip release binaries
- cluster: add `ClusterNode`, a synchronous facade over the actorized distribution protocol (IrohDriver + per-node Runtime hosting Swim/Registry/Metadata/Directory actors with a `MembershipFanout`), replacing ad-hoc `driver.node()`/`tick()` call sites
- fleet: add per-node fleet telemetry that ships identity/resource records as `DatastreamFrame`s over the cluster transport to the orchestrator's `DatastreamSink`, folded into a `FleetView` on a 3s tick
- provision: add best-effort, opt-in SSH boot-phase telemetry (`PP_DEPLOY_KEY`) that streams rented-node boot logs onto the orchestrator's datastream as `proc.boot.<stage>.*`
- dashboard: rewire the crate dependency from `distribution` to `datastream`, drop the standalone `swactor-datastream-dashboard` binary, and rewrite `datastream_source.rs` to demux per-node frames into Overview/Distribution/Fleet views with live-node TTL filtering
- distribution: refresh dist/netmap plugin copy and README from "Kademlia routing" to gossip-directory terminology
Signed-off-by: Zachery Aaron Shores-Chmielewski <zacheryasc@gmail.com>
Strip the collector/aggregator/postproc/snapshot, vastai sampler+shipper,
host/iroh/subprocess/swim introspection, relay observability, sink/spool, and the diag
binaries; drop the t_diag_* tests. Remove DiagEvent emission from iroh_driver. Add
datastream emit/wire (mux + NoopSink/UdpFrameSink/ClusterFrameSink) and rewire the
dashboard onto datastream_source.
Signed-off-by: Zachery Aaron Shores-Chmielewski <zacheryasc@gmail.com>
Upgrades iroh/iroh-relay across datastore, distribution, node, and integration,
adapts iroh_driver to the new Endpoint::builder(Minimal).relay_mode(...) API, and
removes the vendored ed25519-dalek patch now that 0.98 resolves the upstream compile
errors. Adds the SIM_SPEC.md simulator MVP spec under pipeline-parallel-inference.
Signed-off-by: Zachery Aaron Shores-Chmielewski <zacheryasc@gmail.com>
Replaces the generic SimNode/gossip/dashboard framework with a virtual-time
discrete-event engine (priority queue ordered by time/node/fiber/seq), a TOML spec
parser, bundle writer, replay, divergence detector, lint, and postproc, plus the
parity-bar test harness with fixtures and xtask parity-lock tooling. Rewrites
transport identity/crypto and adds the SPEC/TESTING_SPEC/OBSERVABILITY/NORTH_STAR docs.
Signed-off-by: Zachery Aaron Shores-Chmielewski <zacheryasc@gmail.com>
Add a complete single-GPU distributed-inference example that rents a vast.ai GPU, boots a worker container, and runs a prompt end-to-end over iroh/SWIM.
- examples/single-gpu-inference: add the `single_gpu_inference` orchestrator binary that starts a local iroh node, waits for the remote gpu-node to register the `"inference"` SWIM name, then sends an `InferenceRequest` and prints the response
- examples/single-gpu-inference: add the `gpu_node` binary that joins the cluster via `SEED_ADDR`, spawns an `InferenceActor` over `tinygrad_worker.py`, and registers the `"inference"` bridge
- inference_actor: bridge swactor messaging to a Python child process via stdin/stdout JSON, with `ProcessBridge`/`RequestBridge` adapters that satisfy the single-`Incoming` actor constraint
- iroh_transport: add `IrohActorTransport` that sends `WireEnvelope`s over iroh QUIC uni-streams (connection-cached against early close), plus wire encode/decode and an inbound drain helper
- vastai: add a vast.ai REST client (`find_offer` with reliability/cuda/geo filters excluding CN, `create_instance`, `wait_for_running`, `destroy_instance`) parameterised by a mockable `base_url`
- worker/docs/tests: ship `tinygrad_worker.py`/`echo_worker.py` (newline-JSON, `--stub`/`--model` defaulting to llama3.2:1b), a Dockerfile, Makefile, SPEC, and actor/codec/cluster/integration/vastai test suites
Signed-off-by: Zachery Aaron Shores-Chmielewski <zacheryasc@gmail.com>
Using a bounded model checker to provide absolute guarantees on runtime behavior.
Signed-off-by: Zachery Aaron Shores-Chmielewski <zacheryasc@gmail.com>
Remove co-dependencies for different modules found in `crates` and migrate the development history to a new repository. The docs were stale, and largely not getting used, so simply deleted for now. When code stabilizes more, they will become useful again.
Signed-off-by: Zachery Aaron Shores-Chmielewski <zacheryasc@gmail.com>
Make distribution and deployment more stable. Consolidate the logic for a generic swactor node.
Signed-off-by: Zachery Aaron Shores-Chmielewski <zacheryasc@gmail.com>
Not working too well and difficult to track, but has a skeleton there to work from.
Signed-off-by: Zachery Aaron Shores-Chmielewski <zacheryasc@gmail.com>
Content addressable datastore. Allows you to configure a node to store and stream large blobs of data, and retrieve them from any swactor-connected node.
Signed-off-by: Zachery Aaron Shores-Chmielewski <zacheryasc@gmail.com>
Crates continued to grow in number, but most are still quite small and feature specific. This refactor consolidates them.
Signed-off-by: Zachery Aaron Shores-Chmielewski <zacheryasc@gmail.com>
Skeleton up for a web browser swactor engine that is capable of connecting with a cluster.
Signed-off-by: Zachery Aaron Shores-Chmielewski <zacheryasc@gmail.com>
Stand up a runnable distribution stack on top of the core node logic.
- distribution: add NodeDriver bridging DistributedNode to real TCP I/O
(TcpTransport/TcpAcceptor), translating NodeActions to/from wire messages;
refine swim probe timing and transport wiring.
- node: new swactor-node binary (crates/node) hosting a real node over TCP.
- tests/docker: multi-host LAN cluster harness (compose, run-lan-cluster.sh,
cluster + lan_cluster integration tests) exercising the full SWIM path.
- simulation: cluster_scenarios integration + distribution property coverage.
- docs: reorganize into distribution/, runtime/, diagrams/, connectome/; add
DOCKER_REALIZATION + SIMULATION_TESTING realization notes.
Signed-off-by: Zachery Aaron Shores-Chmielewski <zacheryasc@gmail.com>
Lay down the wasm-actor host, a frontend-agnostic command layer, and a
distribution registry.
- command (new crate): CommandRouter dispatching to built-in inspection handlers
(overview/workers/actors) plus user-registered handlers, with line and
query-param parsers; built for REPL/REST/TUI/WebSocket frontends.
- wasm-actor (new crate): skeleton host — WasmActor, Builder, Engine, error
types — with echo/double/silent guest fixtures and integration tests.
- distribution: add Registry (member catalog + lookups) and Snapshot, with tests.
- core: extend the worker watch API; add watch_api integration tests.
Signed-off-by: Zachery Aaron Shores-Chmielewski <zacheryasc@gmail.com>
Adds the std crate on top of the core runtime: Supervisor with RestartPolicy
(Permanent/Transient/Temporary) and SupervisorStrategy (OneForOne/OneForAll/
RestForOne), Router with RoutingStrategy, name/monitor/group registries, StdExtension,
and Ctx/Runtime extension traits. Also extends core (worker, actor, delivery identity
hashing, config, stats), adds a fuzz target, a proptest suite, expands runtime_api
tests, and adds cfuzz cycle notes + benchmarks.
Signed-off-by: Zachery Aaron Shores-Chmielewski <zacheryasc@gmail.com>
Major feature addition. For full details read `./docs/development_history/DISTRIBUTION.md`
Signed-off-by: Zachery Aaron Shores-Chmielewski <zacheryasc@gmail.com>
Runtime dashboard now features a TUI option and an interface for LLM tool use. Removed some bloat from stats collecting and replaced with a hook function to dump runtime stats into.
Signed-off-by: Zachery Aaron Shores-Chmielewski <zacheryasc@gmail.com>
Make mailbox push infallible and replace the locked tick-timing buffer with a
lock-free ring.
- channel: HybridChannel::push and Sender::send now return () — overflow always
absorbs, never rejects — dropping the Result<(), T> surface and its callers.
- stats: tick_timings moves from Mutex<VecDeque> to a lock-free crossbeam
ArrayQueue (drop-oldest-on-full), removing the per-tick lock.
- ripple the signature change through worker/runtime/config; drop worker_benchmarks.
- expand runtime_api tests around the new channel/stats shapes.
Signed-off-by: Zachery Aaron Shores-Chmielewski <zacheryasc@gmail.com>
Add a data-parallel MNIST training example where swactor actors coordinate gradient averaging across workers.
- crates/swactor-dp-mnist/worker.py: add `MnistNet` MLP and `MnistWorker` actor handling `train_batch`/`update`/`evaluate`/`save_model` over a sharded MNIST split with SGD
- crates/swactor-dp-mnist/aggregator.py: add `Aggregator` actor that buffers per-worker gradients, averages them, fans out updates, then logs/evaluates on completion
- crates/swactor-dp-mnist/run_training.py: spawn the Aggregator plus two MnistWorkers (identical initial weights, disjoint shards), run 750 rounds, and poll the inbox for `log`/`done`
- crates/swactor-dp-mnist/pyproject.toml: declare torch/torchvision/numpy deps, an editable local `swactor` source, and the PyTorch CPU index
- Cargo.toml: add a `[profile.bench]` retaining debug symbols (`debug = true`, `strip = false`) for profiling
Signed-off-by: Zachery Aaron Shores-Chmielewski <zacheryasc@gmail.com>
Add Criterion multi-threaded runtime benchmarks and a live dashboard load example.
- benches/mt_benchmarks.rs: add a Criterion suite with `mt/single_actor`, `mt/multi_actor`, `mt/ring`, and `mt/spawn` benchmarks over 2-4 threads, using `iter_custom` and a `wait_for_n_done` helper for deterministic completion
- crates/runtime-dashboard/examples/bench_dashboard.rs: add an example driving sink/ring/spawner load scenarios through `start_dashboard`/`DashboardConfig` for sustained cross-worker visualization
- Cargo.toml: register the `mt_benchmarks` Criterion bench target with `harness = false`
Signed-off-by: Zachery Aaron Shores-Chmielewski <zacheryasc@gmail.com>
Split the monolithic crate into a Cargo workspace with the Python and Wasm bindings as separate member crates.
- Cargo.toml: declare a `[workspace]` with members `.`/`crates/swactor-python`/`crates/swactor-wasm`, remove the `python` feature and pyo3 dependency, and change root crate-type from `["cdylib","rlib"]` to `["rlib"]`
- crates/swactor-python: new cdylib crate re-exporting the PyO3 bindings (Runtime/RuntimeConfig/RuntimeHandle/Inbox/Ctx/ActorAddress/RuntimeStats), depending on `swactor` + pyo3; pyproject.toml and uv.lock relocated here from the root
- crates/swactor-wasm: new cdylib crate moved from top-level `wasm/`, depending on `swactor` with `no_random` features
- src/actor.rs: widen `Actor::new`, `AnyActor`, `ContextInner`, and `Ctx::raw_inner` to `pub` so the separate binding crates can drive the runtime
- src/lib.rs: delete the in-tree `python` module and `#[pymodule]`, and gate the `no_random` RNG behind `all(feature = "no_random", not(feature = "getrandom"))`
- tools/: relocate package.json/package-lock.json; drop the now-duplicate `wasm/Cargo.lock`
Signed-off-by: Zachery Aaron Shores-Chmielewski <zacheryasc@gmail.com>
Refactoring to logically separate component modules in order to make it easier to develop tests, metrics, tracing, etc.
Signed-off-by: Zachery Aaron Shores-Chmielewski <zacheryasc@gmail.com>
Adds some basic benchmarking, stress tests. They still need to be properly examined to ensure they are testing the correct properties, but fit for "good enough". Implements the HybridChannel type, which features a channel buffer that can withstand overflows. It does so by providing a dequeue behind a mutex. Without overflow, will push messages into the lock free ArrayQueue implemented by crossbeam_queue; when that buffer fills, will use the locking portion provided by the Mutex<VecDequeue>.
In the future we can even further optimize this, perhaps with some linked list implementations of lock-free channels, but, like the benchmarks, this fits the "good enough" bar for now.
Signed-off-by: Zachery Aaron Shores-Chmielewski <zacheryasc@gmail.com>
Did not like the requirement of a tokio runtime and async/await. Rewrote
to use lock free queues (`crossbeam_queue::ArrayQueue`) as the basic
primitive to enable a runtime consisting of many pseudo-processes that
can pass messages to each other.
Reviewed-on: http://zachery.lol/code/code/zacheryasc/swactor/pulls/1
Signed-off-by: Zachery Aaron Shores-Chmielewski <zacheryasc@gmail.com>
Skeletal actor framework. Somewhat unweildy, needs a message box, a better runtime, and different channels. However, hello world example works
Signed-off-by: Zachery Aaron Shores-Chmielewski <zacheryasc@gmail.com>
