Zachery Aaron Shores-Chmielewski
6608824cb0
Stand up an interactive end-to-end chat over a CUDA GPU, provisioning a Dockerized node that loads a GGUF model and serves prompts over TCP.
- prompt_rpc: add the newline-JSON prompt protocol (`SubmitPrompt` + `PromptEvent::{TextDelta,Done,Fault}`) carried over TCP
- mvp_chat: add an interactive REPL client connecting to the prompt RPC port (default 127.0.0.1:19777)
- mvp_orch_one_node / mvp_one_node_chat: add the single-node orchestrator that provisions a `LocalDockerPlugin` node, loads `bartowski/Llama-3.2-1B-Instruct-GGUF` (Q4_K_M), and exposes the prompt RPC listener with boot/route/weight timeouts
- mvp_node: add the GPU worker binary that spawns `tinygrad_worker.py` (default device CUDA) and ships runtime telemetry via a `ClusterFrameSink`
- vastai_provisioning / bootstrap_datastream: add the vast.ai provider adapter (`VastAiProvisioningConfig`, `VastAiLeaseClient`) wrapping `swactor_vastai`, plus a bridge that folds provision stdout onto a per-node datastream
- apps/mvp-node: add CUDA base/runtime Dockerfiles (nvidia/cuda 12.6.3, tinygrad 0.12.0, sshd), `mvp_entrypoint.sh` (sshd + mvp-node, held for postmortem), `local_docker_e2e.sh`, the GGUF tinygrad worker, and one-node-chat/bootstrap/vastai guarantee tests
Signed-off-by: Zachery Aaron Shores-Chmielewski <zacheryasc@gmail.com>
54 lines
1.9 KiB
Bash
Executable file
54 lines
1.9 KiB
Bash
Executable file
#!/usr/bin/env bash
|
|
set -euo pipefail
|
|
|
|
BASE_IMAGE=${BASE_IMAGE:-swactor-mvp-node-base:cuda12.6}
|
|
IMAGE=${IMAGE:-swactor-mvp-node:latest}
|
|
CONTAINER=${CONTAINER:-swactor-mvp-node-e2e-$$}
|
|
GPUS=${MVP_CUDA_GPUS:-all}
|
|
PROMPT=${MVP_NODE_SELF_TEST_PROMPT:-ping}
|
|
TIMEOUT_SECS=${MVP_NODE_E2E_TIMEOUT_SECS:-1800}
|
|
FRAME_LOG=${MVP_DATASTREAM_FRAME_LOG:-/var/log/mvp-datastream.ndjson}
|
|
|
|
cleanup() {
|
|
docker rm -f "$CONTAINER" >/dev/null 2>&1 || true
|
|
}
|
|
trap cleanup EXIT
|
|
|
|
cargo build --release -p mvp-system --bin mvp-node
|
|
docker build -f apps/mvp-node/Dockerfile.base -t "$BASE_IMAGE" .
|
|
docker build -f apps/mvp-node/Dockerfile --build-arg BASE_IMAGE="$BASE_IMAGE" -t "$IMAGE" .
|
|
|
|
docker run -d \
|
|
--name "$CONTAINER" \
|
|
--gpus "$GPUS" \
|
|
-e MVP_NODE_SELF_TEST_PROMPT="$PROMPT" \
|
|
-e MVP_NODE_MAX_RUNTIME_SECS=1 \
|
|
-e MVP_SELF_TEST_MAX_TOKENS="${MVP_SELF_TEST_MAX_TOKENS:-1}" \
|
|
-e MVP_MODEL_CACHE_DIR=/var/cache/mvp-models \
|
|
-e MVP_DATASTREAM_FRAME_LOG="$FRAME_LOG" \
|
|
${HF_TOKEN:+-e HF_TOKEN="$HF_TOKEN"} \
|
|
"$IMAGE" >/dev/null
|
|
|
|
deadline=$((SECONDS + TIMEOUT_SECS))
|
|
while (( SECONDS < deadline )); do
|
|
logs=$(docker logs "$CONTAINER" 2>&1 || true)
|
|
if grep -q '"type":"ready"' <<<"$logs" && grep -q '"type":"self_test_completed"' <<<"$logs"; then
|
|
frames=$(docker exec "$CONTAINER" cat "$FRAME_LOG" 2>/dev/null || true)
|
|
if grep -q '"channel":"mvp.node.ready"' <<<"$frames" &&
|
|
grep -q '"channel":"mvp.worker.weights"' <<<"$frames" &&
|
|
grep -q '"channel":"mvp.worker.prompt"' <<<"$frames"; then
|
|
printf '%s\n' "$logs"
|
|
printf '%s\n' "$frames"
|
|
exit 0
|
|
fi
|
|
fi
|
|
if grep -q 'WorkerFatal\|mvp-node: .*failed\|ModelLoadFailed\|GgufDownloadFailed' <<<"$logs"; then
|
|
printf '%s\n' "$logs" >&2
|
|
exit 1
|
|
fi
|
|
sleep 5
|
|
done
|
|
|
|
docker logs "$CONTAINER" 2>&1 || true
|
|
echo "mvp-node Docker E2E timed out after ${TIMEOUT_SECS}s" >&2
|
|
exit 1
|