No description
Zachery Aaron Shores-Chmielewski
a2ec548134
Extend the single-GPU example into a two-node pipeline-parallel run that splits llama3.2:1b across two rented vast.ai GPUs and closes the autoregressive decode loop over iroh.
- topology: add linear-chain helpers where each stage derives its neighbours locally from `STAGE`/`NUM_STAGES`, registering `pp-entry`/`pp-exit`/`pp-stage-{i}` SWIM names
- messages: add `StageActivation` (bf16 hidden-state hand-off carrying position/seq_len/is_prefill) and `NextToken` (sampled-token feedback with a `done` flag) that close the autoregressive loop between stage 0 and stage 1
- stage_actor: add `Stage0Actor` (tokenize -> embed_and_forward -> prefill activation; decode_step on each NextToken) and `Stage1Actor` (forward_and_sample -> NextToken back; emit InferenceResponse on EOS/max_tokens)
- vastai: fork the client and add `create_pipeline_instances` (rents one instance per stage, threading `STAGE`/`NUM_STAGES`, best-effort destroys on partial failure) and `destroy_all_instances`
- pp_tinygrad_worker.py: per-stage worker slicing `model.blk[start:end]` in stub and real (GGUF) modes, plus new `pp_gpu_node`/`pp_smoke_run` binaries and ROADMAP/SPEC/TEST_SPEC docs
- reuse: build on the single-GPU example's iroh transport and process bridge unchanged; add actor/codec/topology/integration test suites
Signed-off-by: Zachery Aaron Shores-Chmielewski <zacheryasc@gmail.com>
|
||
|---|---|---|
| .cargo | ||
| .deploy | ||
| benches | ||
| ci | ||
| crates | ||
| docs | ||
| examples | ||
| fuzz | ||
| src | ||
| tests | ||
| xtask | ||
| .dockerignore | ||
| .gitignore | ||
| Cargo.lock | ||
| Cargo.toml | ||
| Dockerfile | ||
| README.md | ||
swactor
Minimal actor runtime for Rust. One trait, one message type.
Single-threaded (tick()) or multi-threaded (run()).
Description
Core runtime is src/. Actors implement ActorInterface (in actor.rs),
interact through Ctx (in runtime.rs), and run on worker threads (worker.rs).
crates/ builds upward: std adds OTP patterns (supervision, monitoring, groups),
distribution adds clustering, everything else composes from there.
Dev commands
cargo xtask --help for available test groups.
Testing
cargo check --workspace
cargo xtask test <your-feature-crate>
cargo xtask test essential