swactor/docs/development_history/IN_BROWSER_RUNTIME.md

In-Browser Runtime via WebAssembly — Development History

Covers the work to make swactor run in the browser with WebSocket connectivity to a native cluster via a gateway pattern.

Branch: in-browser

---

Table of Contents

1. Overview & Motivation
2. What Was Built
3. Development Phases
4. NodeAddr Abstraction
5. Core Wasm Compatibility
6. WebSocket Transport (Browser)
7. WebSocket Gateway (Native)
8. Enhanced Browser Runtime
9. Demo & Examples
10. Design Decisions & Tradeoffs
11. Dashboard Improvements
12. Known Gaps & Future Work

---

1. Overview & Motivation

Before this work, swactor had a minimal wasm crate (crates/wasm/) that could run hardcoded Counter/Relay actors locally via tick() — no networking, no JS-defined actors, no connection to a cluster.

The goal: make a browser node that can run actors locally AND connect to a native cluster via WebSocket, with the architecture designed so the browser could eventually become a full cluster peer (WebRTC P2P).

Two design constraints guided the approach:

• Start thin, design for full. Use a gateway pattern for v1 connectivity (browser ↔ WebSocket ↔ native node), but introduce the NodeAddr abstraction now so the distribution layer can eventually support WebSocket and WebRTC peers natively.

• Both Rust and JS actors. Rust-compiled actors work by defining ActorInterface impls in the wasm crate (as the existing Counter/Relay do). JS actors work via js_sys::Function callback wrappers.

---

2. What Was Built

Component	Location	Action	Key Changes
NodeAddr abstraction	crates/distribution/	Modified (~30 files)	SocketAddr → NodeAddr across distribution, simulation, dashboard
Core wasm compat	src/	Modified (4 files)	web-time abstraction, cfg-gated threads, wire encoding extraction
WebSocket transport	crates/wasm/src/	Created (3 files)	WsTransport, GatewayControl protocol, JsActor wrapper
WebSocket gateway	crates/gateway/	Created (new crate)	WsAcceptor, WsGateway, SessionTransport
Browser runtime	crates/wasm/src/lib.rs	Rewritten	BrowserRuntime API with JS actor support
Demo	examples/, crates/wasm/www/	Created	Gateway example, HTML demo page
Dashboard improvements	crates/runtime-dashboard/static/	Created + Modified	Static file extraction, gossip panel, visual fixes, interactivity
Gossip accessors	crates/distribution/src/	Modified (5 files)	SwimProbe/SwimNode/DisseminationQueue/DistributedNode accessors, snapshot types

---

3. Development Phases

Phase 1 — NodeAddr abstraction in distribution crate

Replaced SocketAddr with an extensible NodeAddr enum across ~30 files: distribution crate (15 source files + 12 test files), simulation crate, and dashboard example. The main challenge was that NodeAddr is Clone but not Copy (unlike SocketAddr), requiring ~25 .clone() additions.

Phase 2 — Core wasm compatibility

Made the core swactor crate compile for wasm32-unknown-unknown:

• web-time behind cfg(wasm32) for Instant
• cfg-gated RuntimeHandle, run(), thread imports
• Extracted encode_wire_envelope/decode_wire_envelope into core src/transport.rs
• Distribution crate re-exports the shared wire encoding functions

Phase 3 — WebSocket transport (browser side)

Created browser-side transport wrapping web_sys::WebSocket:

• WsTransport — buffers outbound messages until connection opens, decodes inbound binary frames, implements Transport trait
• GatewayControl — control protocol for actor registration/resolution/keepalive
• Same length-prefixed binary wire format as TCP

Phase 4 — WebSocket gateway (native side)

Created crates/gateway/ — a native-side WebSocket acceptor + gateway:

• WsAcceptor — mirrors TcpAcceptor pattern (non-blocking accept, WS upgrade, non-blocking read loop, dead connection cleanup)
• WsGateway — routes inbound envelopes to runtime, handles control protocol, creates SessionTransport routes for cluster → browser forwarding
• Sync tungstenite in dedicated thread — no tokio dependency

Phase 5 — Enhanced browser runtime

Replaced the hardcoded wasm MVP with a generic runtime:

• JsActor — wraps js_sys::Function as an ActorInterface impl
• JsActorCtx — bridge object passed to JS handlers (send, self_addr)
• BrowserRuntime — JS-facing API: connect(), spawn_js_actor(), send_json(), tick(), create_inbox(), try_recv()
• JsMessage codec for JSON-based communication
• Legacy SwactorRuntime preserved for backwards compatibility

Phase 6 — Demo and testing

• examples/ws_gateway.rs — native gateway node with echo actor
• crates/wasm/www/index.html — browser demo with connection UI, actor spawning, message sending, and log panel

---

4. NodeAddr Abstraction

The Type

// crates/distribution/src/types.rs
#[derive(Debug, Clone, PartialEq, Eq, Hash, Serialize, Deserialize)]
pub enum NodeAddr {
    Tcp(SocketAddr),
    // Future: Ws(String), WebRtc(String)
}

All distribution APIs (SwimNode, DistributedNode, RoutingTable, etc.) now accept and return NodeAddr. The TcpTransport pattern-matches NodeAddr::Tcp(addr) when connecting:

fn require_tcp(addr: &NodeAddr) -> Result<SocketAddr, Error> {
    match addr {
        NodeAddr::Tcp(sa) => Ok(*sa),
    }
}

Scope of changes

The refactor touched every file in the distribution crate that previously used SocketAddr:

• types.rs — NodeRecord.addr, DirectoryEntry field type
• messages.rs — PingReq.target_addr, JoinRequest.addr, FindNodeResponse, FindValueResponse
• swim/node.rs — SwimNode.self_addr, all NodeAction variants
• swim/probe.rs — SwimAction variants, ProbePhase, target selection
• swim/member_list.rs — MemberEntry.addr, apply() signature
• swim/dissemination.rs — membership_update() function
• kademlia/routing_table.rs — NodeEntry.addr, insert() signature
• kademlia/lookup.rs — LookupAction, NodeLookup.known
• node.rs — DistributedNodeConfig.listen_addr, all handler methods
• transport.rs — TcpTransport::new(), send_to()
• snapshot.rs — addr_str()
• All 12 test files in crates/distribution/tests/
• crates/simulation/src/distribution/sim.rs
• crates/runtime-dashboard/examples/dashboard_demo.rs

---

5. Core Wasm Compatibility

Time abstraction

// src/lib.rs
pub(crate) mod time {
    #[cfg(not(target_arch = "wasm32"))]
    pub(crate) use std::time::Instant;
    #[cfg(target_arch = "wasm32")]
    pub(crate) use web_time::Instant;
}

src/runtime.rs and src/worker.rs import crate::time::Instant instead of std::time::Instant. The web-time crate provides a browser-compatible Instant backed by performance.now().

cfg-gated thread code

// src/runtime.rs
#[cfg(not(target_arch = "wasm32"))]
use std::thread::{self, JoinHandle};

#[cfg(not(target_arch = "wasm32"))]
pub struct RuntimeHandle { ... }

#[cfg(not(target_arch = "wasm32"))]
pub fn run(self) -> Result<RuntimeHandle, Error> { ... }

Same pattern in src/worker.rs for Worker::run(). On wasm32, only tick() is available.

Wire encoding extraction

encode_wire_envelope and decode_wire_envelope were moved from crates/distribution/src/transport.rs into core src/transport.rs so both the distribution crate and the wasm crate can share them. The distribution crate re-exports:

pub use swactor::transport::encode_wire_envelope;
pub use swactor::transport::decode_wire_envelope;

Worker module visibility

On wasm32, the worker module is pub(crate) (not pub) since external code shouldn't depend on thread-specific worker internals:

#[cfg(not(target_arch = "wasm32"))]
pub mod worker;
#[cfg(target_arch = "wasm32")]
pub(crate) mod worker;

---

6. WebSocket Transport (Browser)

crates/wasm/src/
├── ws_transport.rs   — WsTransport (Transport impl over web_sys::WebSocket)
├── protocol.rs       — GatewayControl enum, CONTROL_TYPE_TAG, NULL_ADDRESS
└── js_actor.rs       — JsActor, JsActorCtx, JsMessage

WsTransport

Wraps web_sys::WebSocket with outbound buffering and inbound envelope decoding:

  Browser JS                 WsTransport                    Gateway
  ──────────                 ───────────                    ───────
                     ┌─ Connecting ─┐
  rt.connect(url) ──►│ buffer sends │──── WS handshake ────►
                     └──────────────┘
                     ┌─── Open ─────┐
  rt.send_json() ───►│ encode + send│──── binary frame ────►
                     │              │
                     │ decode inbound│◄─── binary frame ────
                     └──────────────┘
  rt.tick()      ───► drain_inbound() → deliver_raw()

The transport uses Rc<RefCell<WsInner>> internally — safe because wasm32 is single-threaded. unsafe impl Send + Sync matches the pattern used by Runtime's existing unsafe impl Sync for RefCell<Vec<Worker>>.

Control Protocol

Control messages use a reserved null address ([0u8; 32]) and the type tag "swactor::GatewayControl":

pub enum GatewayControl {
    RegisterActor { addr: [u8; 32] },
    UnregisterActor { addr: [u8; 32] },
    ResolveActor { name: String },
    ActorResolved { name: String, addr: [u8; 32] },
    Ping,
    Pong,
}

---

7. WebSocket Gateway (Native)

crates/gateway/src/
├── lib.rs           — WsGateway, SessionTransport, control protocol handler
└── ws_acceptor.rs   — WsAcceptor (mirrors TcpAcceptor), WsSession, SessionId

WsAcceptor

Follows the same non-blocking pattern as TcpAcceptor in the distribution crate:

1. Non-blocking TCP accept
2. WebSocket handshake (briefly blocking per new client)
3. Switch to non-blocking for reads
4. Read binary frames from all sessions
5. Dead connection cleanup in reverse index order

WsGateway

  Browser            WsGateway              Runtime / Cluster
  ───────            ─────────              ──────────────────

  WireEnvelope ─────► route by dest:
                       │
                       ├─ control msg? → handle_control()
                       │                  ├─ RegisterActor → add route
                       │                  ├─ Ping → send Pong
                       │                  └─ ...
                       │
                       └─ regular msg → codec_registry.receive()
                                          └─ runtime.deliver_raw()

  ◄───── WireEnvelope ◄── SessionTransport.send()
                            │
                            └─ TransportRouter lookup hits
                               SessionTransport for browser-owned address

When a browser registers an actor, the gateway:

1. Stores addr → session_id mapping
2. Creates a SessionTransport route in the TransportRouter
3. Cluster actors sending to that address hit the route, which forwards via the WebSocket session

---

8. Enhanced Browser Runtime

JsActor

pub struct JsActor {
    handler: js_sys::Function,  // (ctx, type_tag, msg_json) => void
}

impl ActorInterface for JsActor {
    type Incoming = JsMessage;
    type Response = ();
    fn handle(&mut self, ctx: &Ctx, msg: JsMessage) { ... }
}

The handler receives a JsActorCtx bridge that exposes send(dest_hex, type_tag, msg_json) and self_addr() to JavaScript.

BrowserRuntime API

  JavaScript                    BrowserRuntime (Rust/wasm)
  ──────────                    ──────────────────────────

  new BrowserRuntime()     ──► creates Runtime + CodecRegistry + TransportRouter
  rt.connect(url)          ──► WsTransport::connect(url)
  rt.spawn_js_actor(fn)    ──► JsActor wrapper → rt.spawn() → register w/ gateway
  rt.send_json(hex, tag, j)──► JsMessage → rt.send_to()
  rt.tick()                ──► drain WS inbound → deliver_raw → rt.tick()
  rt.create_inbox()        ──► rt.new_inbox::<JsMessage>() → hex address
  rt.try_recv(hex)         ──► inbox.try_recv() → JSON string or undefined
  rt.is_connected()        ──► WsTransport::is_connected()
  rt.actor_count()         ──► rt.stats().actors.len()

JsMessage

A unified message type for all JS actors:

pub struct JsMessage {
    pub type_tag: String,  // wire protocol type tag (or "local")
    pub payload: String,   // JSON payload
}

Registered in the CodecRegistry with type tag "swactor::JsMessage" and a simple UTF-8 codec.

---

9. Demo & Examples

Gateway example (examples/ws_gateway.rs)

cargo run --example ws_gateway --features transport

Starts a native node on ws://127.0.0.1:9000 with an echo actor. The gateway accepts WebSocket connections and bridges messages between browser clients and the runtime.

Demo HTML page (crates/wasm/www/index.html)

Build and serve:

cd crates/wasm && wasm-pack build --target web --out-dir www/pkg
cd www && python3 -m http.server 8080
# Open http://localhost:8080

Features: connection panel with status indicator, JS actor spawning with click-to-copy addresses, message sending, and a timestamped event log.

---

10. Design Decisions & Tradeoffs

Decision	Rationale
Gateway pattern for v1	Avoids STUN/TURN complexity; one TCP connection per browser; simple to debug. NodeAddr abstraction leaves room for direct WebRTC P2P later.
Sync tungstenite (no tokio)	Matches existing TcpAcceptor pattern in the distribution crate. Keeps the dependency tree small. The gateway runs a poll loop on a dedicated thread.
NodeAddr enum (not trait)	Enums are exhaustive, serializable, and cheap to match. Adding a variant (e.g., Ws(String)) is a compiler-guided refactor.
Same wire format over WS as TCP	No protocol translation — a WireEnvelope is the same bytes on both transports. Simplifies debugging and testing.
Rc<RefCell> in WsTransport	wasm32 is single-threaded. Arc<Mutex> would compile but add unnecessary overhead. unsafe impl Send+Sync is the standard pattern for wasm-bindgen types.
JSON for JS actor messages	The browser's native format. Binary codecs could be added later via CodecRegistry.
Preserved legacy SwactorRuntime	The old Counter/Relay API still works. New code uses BrowserRuntime.

---

11. Dashboard Improvements

Alongside the in-browser work, the runtime dashboard received significant improvements: extraction to static files, visual bug fixes, interactivity, and a new gossip protocol panel on the distribution page.

Static file extraction

The dashboard HTML was originally embedded as Rust string constants (actors_html.rs, dashboard_html.rs, distribution_html.rs). These were extracted to standalone files under crates/runtime-dashboard/static/:

static/
├── css/shared.css       — shared dark-theme styles, stat cards, nav
├── js/shared.js         — SSE helper, color palette, colorWithAlpha()
├── index.html           — overview page (worker chart, actor table)
├── actors.html          — actors page (worker bars, depth chart, actor list)
└── distribution.html    — distribution page (graph, gossip, membership)

The server now serves these as static files instead of embedding them.

Visual fixes and performance

• Bar chart label clipping — clamped fillText y-position to Math.max(12, chartH - h - 4) so labels for tall bars stay visible
• Worker details click flakiness — innerHTML = '' every 200ms destroyed DOM elements and their click handlers. Replaced with persistent workerNodes map; headers and handlers are created once, text updated in-place
• Legend wrapping — added white-space: nowrap; overflow: hidden; text-overflow: ellipsis to worker legend
• Performance — data fingerprinting (JSON.stringify comparison) skips redundant redraws; differential actor table updates; requestAnimationFrame throttling

Actors page interactivity

• Click-to-filter — clicking a worker bar in the chart filters the actor table to that worker's actors (click again to clear)
• Depth chart colors — changed from green-red heatmap to purple/indigo palette to distinguish from worker bars
• Worker-colored rows — actor rows are tinted with a translucent version of their worker's color via colorWithAlpha(hex, 0.08)

Gossip protocol panel (distribution page)

Exposed SWIM gossip internals through new Rust accessors and snapshot types, then added a dedicated panel to the distribution page.

Rust changes — new accessor methods on SwimProbe (tick(), sequence(), phase_name(), probe_target(), suspicion_timers(), config()), SwimNode (probe(), dissemination()), DisseminationQueue (pending_entries()), and DistributedNode (swim_node()). New snapshot structs: GossipInfo, SuspicionInfo, DisseminationInfo, GossipConfig. Added gossip: Option<GossipInfo> to DistributionNodeSnapshot with #[serde(default)] for backward compatibility.

UI panel shows:

• Phase badge — IDLE (green), PINGING (yellow), INDIRECT (orange)
• Counters — protocol round, probes sent, incarnation, pending updates
• Suspicion timers — table with progress bars toward timeout
• Membership events — persistent rolling log of state transitions (new/alive/suspect/dead/gone) built by diffing consecutive snapshots client-side, with timestamps and node addresses
• Recently probed — list of recent probe targets
• Config — human-readable protocol parameters

All node IDs are displayed as socket addresses (via a lookup map built from the members list) rather than raw hex. When running solo with no peers, the panel shows "No peers — gossip inactive" instead of zeros.

---

12. Known Gaps & Future Work

Gap	Effort	Impact
ResolveActor control message not implemented	Low	Browser can't discover actors by name
No WebSocket reconnection logic	Medium	Browser must refresh on disconnect
No authentication on WS connections	Medium	Any client can register actors
Gateway doesn't participate in SWIM	Medium	Browser actors aren't in the cluster directory
No back-pressure from WS to actors	Low	Fast sender can overwhelm browser
WebRTC P2P (direct browser↔browser)	High	Eliminates gateway bottleneck
NodeAddr::Ws variant in distribution	Medium	Browser as full cluster peer without gateway
wasm-pack integration test	Low	Automated browser test in CI

---

Verifying

Native build + tests

# Full workspace build
cargo build

# All tests (excluding flaky simulation MT test)
cargo test --workspace --exclude simulation

# Distribution tests specifically (134 tests)
cargo test -p distribution

# Gateway crate builds
cargo build -p swactor-gateway

Wasm32 target

# Core crate compiles for wasm32
cargo build --target wasm32-unknown-unknown -p swactor --no-default-features --features "no_random,transport"

# Wasm crate compiles for wasm32
cargo build --target wasm32-unknown-unknown -p swactor-wasm

Dashboard

# Build dashboard (includes static files)
cargo build -p runtime-dashboard

# Run the distribution demo (9-node churn simulation with dashboard)
cargo run --example dashboard_demo -p runtime-dashboard --features distribution
# Open http://localhost:3000 → Overview, Actors, Distribution pages

# Verify distribution page gossip panel:
# - Solo node: "No peers — gossip inactive"
# - With peers: phase badge, counters, membership event log

Gateway example (manual)

# Terminal 1: start gateway
cargo run --example ws_gateway --features transport

# Terminal 2: build wasm + serve demo page
cd crates/wasm && wasm-pack build --target web --out-dir www/pkg
cd www && python3 -m http.server 8080
# Open http://localhost:8080 and click Connect