Zachery Aaron Shores-Chmielewski
ae9ca3bcf3
Promote pipeline-parallel-inference to a first-class app and consolidate observability on the datastream wire, decoupling the dashboard crate from `distribution`. - apps/pipeline-parallel-inference: move the example out of `examples/` into `apps/` as its own workspace, rename binaries to `pp-worker`/`pp-orchestrator`, and strip release binaries - cluster: add `ClusterNode`, a synchronous facade over the actorized distribution protocol (IrohDriver + per-node Runtime hosting Swim/Registry/Metadata/Directory actors with a `MembershipFanout`), replacing ad-hoc `driver.node()`/`tick()` call sites - fleet: add per-node fleet telemetry that ships identity/resource records as `DatastreamFrame`s over the cluster transport to the orchestrator's `DatastreamSink`, folded into a `FleetView` on a 3s tick - provision: add best-effort, opt-in SSH boot-phase telemetry (`PP_DEPLOY_KEY`) that streams rented-node boot logs onto the orchestrator's datastream as `proc.boot.<stage>.*` - dashboard: rewire the crate dependency from `distribution` to `datastream`, drop the standalone `swactor-datastream-dashboard` binary, and rewrite `datastream_source.rs` to demux per-node frames into Overview/Distribution/Fleet views with live-node TTL filtering - distribution: refresh dist/netmap plugin copy and README from "Kademlia routing" to gossip-directory terminology Signed-off-by: Zachery Aaron Shores-Chmielewski <zacheryasc@gmail.com>
66 lines
3 KiB
Bash
66 lines
3 KiB
Bash
# pipeline-parallel-inference — run profile (.env style)
|
|
# ----------------------------------------------------------------------------
|
|
# This is the single source of truth for the knobs that used to be scattered
|
|
# across the binaries and one operator's shell. Copy it to set up a run:
|
|
#
|
|
# cp profiles/example.env profiles/local.env # untracked; put real secrets here
|
|
# PP_PROFILE=profiles/local.env cargo run --bin pp-orchestrator -- --seed ...
|
|
#
|
|
# When PP_PROFILE is unset, profiles/local.env is loaded automatically if it
|
|
# exists. The run scripts pick this up too (the binaries load it at startup).
|
|
#
|
|
# PRECEDENCE (lowest -> highest):
|
|
# compiled default < this profile < real process env < CLI flag
|
|
# i.e. anything already exported in your shell wins over the file, and a
|
|
# --flag wins over everything. The profile only fills in what is unset.
|
|
#
|
|
# Lines are KEY=VALUE; `export ` prefix and surrounding quotes are tolerated;
|
|
# `#` comments and blank lines are ignored.
|
|
# ----------------------------------------------------------------------------
|
|
|
|
# --- Networking / relay -----------------------------------------------------
|
|
# Custom iroh relay the whole cluster homes onto (a swactor-iroh-relay on a
|
|
# VPS). Leave unset to use the iroh default relays. Placeholder below — put
|
|
# YOUR real relay only in profiles/local.env (gitignored), not here.
|
|
#SWACTOR_IROH_RELAY_URL=https://relay.example.com
|
|
|
|
# --- Topology ---------------------------------------------------------------
|
|
# Number of pipeline stages (>= 2). pp-orchestrator also accepts --num-stages.
|
|
#NUM_STAGES=2
|
|
|
|
# --- Compute target (vast.ai / docker) --------------------------------------
|
|
# Container image to run on each node. The run scripts already read PP_IMAGE;
|
|
# pp-orchestrator's --image default now reads it too. Set to your registry tag.
|
|
#PP_IMAGE=swactor-pp-gpu:latest
|
|
# GPU class requested when leasing on vast.ai (pp-orchestrator --gpu overrides).
|
|
#PP_GPU=RTX 3060
|
|
|
|
# --- Workload (what each stage computes) ------------------------------------
|
|
# Model identifier handed to the worker.
|
|
#MODEL=
|
|
# Python worker script + interpreter (per-stage compute). WORKER_SCRIPT is the
|
|
# pp-worker default; pp-orchestrator --worker ships a script to remote nodes.
|
|
#WORKER_SCRIPT=./pp_tinygrad_worker.py
|
|
#WORKER_CMD=python3
|
|
#PYTHON=python3
|
|
# Stub mode: skip the real worker, echo activations (fast local smoke runs).
|
|
#PP_WORKER_STUB=1
|
|
# Inference request prompt + token budget (pp-orchestrator --prompt/--max-tokens).
|
|
#MAX_TOKENS=64
|
|
|
|
# --- Timeouts (seconds; sane defaults baked in — override only if needed) ---
|
|
#PP_CONVERGE_TIMEOUT_SECS=1200
|
|
#PP_WORKER_READY_TIMEOUT_SECS=1800
|
|
#PP_NEIGHBOR_RESOLVE_TIMEOUT_SECS=1800
|
|
#PP_PIPELINE_WIRED_TIMEOUT_SECS=1800
|
|
#PP_AWAIT_RESPONSE_TIMEOUT_SECS=600
|
|
#PP_RESOLVE_TIMEOUT_SECS=1200
|
|
|
|
# --- Diagnostics / dashboards (optional) ------------------------------------
|
|
#PP_DASHBOARD=1
|
|
#PP_DASHBOARD_PORT=9090
|
|
#SWACTOR_DIAG_COLLECTOR_URL=http://127.0.0.1:9080
|
|
|
|
# Secrets (orchestrator / stage identity). Keep real values in local.env only.
|
|
#PP_ORCH_SECRET=
|
|
#PP_STAGE_SECRET=
|