Claude
6197361b22
feat: cycle 6 — asymmetric block, partition-register, bidir suspicion, 3-way partition
...
4 new distribution sim tests:
- asymmetric_one_way_block_does_not_kill_node: gossip recovery through intermediates
- names_registered_during_partition_propagate_after_heal: re_disseminate_all on Alive
- bidirectional_suspicion_both_nodes_recover: mutual suspicion + refutation cycle
- three_way_partition_heals_and_converges: 9-node 3-group split recovers
35 total distribution sim tests (15 registry + 10 lifecycle + 10 property), all green.
Authored by Claude, lovingly guided by Zachery Aaron Shores-Chmielewski
2026-02-13 09:56:43 +00:00
Claude
7d55a01198
feat: cycle 6 lifecycle tests — asymmetric block, partition-register, bidirectional suspicion
...
3 new lifecycle sim tests:
- asymmetric_one_way_block_does_not_kill_node: gossip recovery through intermediate nodes
- names_registered_during_partition_propagate_after_heal: re_disseminate_all on Alive transition
- bidirectional_suspicion_both_nodes_recover: mutual suspicion + refutation via gossip
Authored by Claude, lovingly guided by Zachery Aaron Shores-Chmielewski
2026-02-13 09:55:02 +00:00
Claude
6f98b54d01
feat: cycle 5 lifecycle tests — partition+death combined, registry GC
...
New tests:
- partition_then_death_during_partition_then_heal (combined lifecycle scenario)
- registry_tombstones_gc_after_ttl (tombstone GC with short TTL + clock advancement)
Total: 15 registry + 7 lifecycle + 9 property = 31 new distribution sim tests.
Authored by Claude, lovingly guided by Zachery Aaron Shores-Chmielewski
2026-02-13 09:35:54 +00:00
Claude
fc2532ce18
feat: cycle 4 property tests — asymmetric partition, revive, monotonicity, large cluster
...
New tests:
- asymmetric_partition_registry_converges_after_heal (one-way reachability)
- revived_node_re_registration_overwrites_tombstone (death + revive + takeover)
- registry_convergence_is_monotonic_in_stable_cluster (once converged, stays converged)
- large_cluster_registry_converges (15-node stress with 2 deaths)
Total: 15 registry + 5 lifecycle + 9 property = 29 distribution sim tests.
Authored by Claude, lovingly guided by Zachery Aaron Shores-Chmielewski
2026-02-13 09:31:46 +00:00
Claude
57d9135c54
feat: cycle 3 edge case registry tests — message loss, multi-kill, churn
...
New tests:
- registry_converges_despite_message_loss (30% loss window + clean convergence)
- simultaneous_kill_of_multiple_name_owners (3 name-owning nodes killed at once)
- suspected_name_owner_recovers_and_registry_survives (Suspect→Alive path)
- registry_correct_under_rapid_churn (interleaved kills + registrations + takeover)
Total: 15 registry tests, 5 lifecycle, 5 property = 25 distribution sim tests.
Authored by Claude, lovingly guided by Zachery Aaron Shores-Chmielewski
2026-02-13 09:27:23 +00:00
Claude
9327d0734c
feat: edge case registry sim tests + GracefulLeave action
...
Cycle 2 tests:
- explicit_unregister_propagates_to_all_nodes
- re_registration_after_tombstone_succeeds
- all_names_tombstoned_when_owner_dies
- graceful_leave_tombstones_registry_names
- piggyback_contention_both_propagate
Also:
- Add GracefulLeave variant to SimAction enum
- Fix cluster_survives_brief_message_loss config (dead_reprobe + suspicion_timeout)
- Harden split_brain test config (suspicion_timeout=200 prevents cascading false deaths)
Authored by Claude, lovingly guided by Zachery Aaron Shores-Chmielewski
2026-02-13 09:22:07 +00:00
Claude
3226ba3764
feat: lifecycle and property sim tests, remove debug test
...
Phase 3 — lifecycle sim tests (distribution_lifecycle.rs):
- dead_node_triggers_repair_queue_and_cache_invalidation
- revived_node_has_empty_directory
- cache_shrinks_after_node_death
- routing_table_recovers_after_partition_heals
- routing_table_bounded_by_alive_count
Phase 4 — property-based sim tests (distribution_properties.rs):
- routing_table_bounded_across_configs (3 config variants)
- cache_bounded_across_configs (2 config variants)
- repair_queue_populates_on_death_with_directory_entries
- registry_eventually_consistent_across_configs (3 config variants)
- cascading_deaths_maintain_invariants
Also: update cluster_scenarios 10% message loss test to use
suspicion_timeout=60 + indirect_probes=3 + dead_reprobe=15
for resilience under correct death dissemination.
Remove diagnostic debug_registry.rs (superseded by distribution_registry.rs).
Authored by Claude, lovingly guided by Zachery Aaron Shores-Chmielewski
2026-02-13 08:53:07 +00:00
Claude
5d8e413db2
fix: three SWIM notification bugs — death dissemination, piggyback notifications, partition recovery
...
Bug 1 (swim/node.rs): SwimProbe::check_suspicion_timeouts() calls
members.declare_dead() before translate_probe_actions() processes the
DeclareDead action. The second declare_dead() returned false (already dead),
so MembershipChanged{Dead} was never emitted and the death was never
enqueued for dissemination. Fix: remove the redundant declare_dead() call
in translate_probe_actions since the probe already performed the mutation.
Bug 2 (swim/node.rs): apply_membership_update() — which processes piggyback
on every ping/ack/ping_req — updated the internal member list but never
emitted NodeAction::MembershipChanged. This meant DistributedNode was blind
to all state transitions learned via gossip piggyback (e.g., a dead node
refuting via incarnation bump). Fix: return MembershipChanged actions from
apply_piggyback and propagate through handle_ping/handle_ack/handle_ping_req.
Bug 3 (node.rs): DistributedNode::handle_ping/handle_ack/handle_ping_req
never processed MembershipChanged actions from SwimNode — only tick() did.
Fix: extract process_membership_changes() helper and call it from all four
message paths (tick, handle_ping, handle_ack, handle_ping_req).
Additional fixes:
- registry.rs: add re_disseminate_all() for anti-entropy on partition heal
- node.rs: call re_disseminate_all on MemberState::Alive transitions so
registry state accumulated during partition reaches recovering nodes
- cluster_scenarios: enable dead_reprobe in 10% message loss test, since
correct death dissemination (now working) causes cascading false deaths
without a recovery mechanism
Authored by Claude, lovingly guided by Zachery Aaron Shores-Chmielewski
2026-02-13 08:39:14 +00:00
Claude
c84e708846
feat: registry sim tests — 4 pass, 2 ignored (SWIM death bug)
...
Add sim infrastructure for registry testing:
- Extend DistributionSnapshot with registry_size, registry_tombstone_count
- Add SimAction enum (RegisterName, RegisterNameWithActor, UnregisterName)
- Add action_schedule to DistributionSimConfig for mid-sim registry ops
- Add run_simulation_with_nodes() returning node references for assertions
- Add property checks: registry_propagation, registry_tombstones, etc.
6 registry sim tests:
- registry_name_converges_across_cluster (PASS)
- split_brain_naming_converges_after_partition_heals (IGNORED — bug)
- tombstone_propagates_when_name_owner_dies (IGNORED — bug)
- rapid_re_registration_converges_to_latest (PASS)
- simultaneous_registration_converges_deterministically (PASS)
- multiple_names_from_different_nodes_all_propagate (PASS)
BUG FOUND: SwimProbe::check_suspicion_timeouts() calls
members.declare_dead() internally before SwimNode::translate_probe_actions()
processes the DeclareDead action, so the second declare_dead() returns false
and MembershipChanged{Dead} is never emitted. This silently breaks all
death-related side effects: RT cleanup, cache invalidation, repair queue
population, and registry tombstoning.
Authored by Claude, lovingly guided by Zachery Aaron Shores-Chmielewski
2026-02-13 08:29:37 +00:00
a7124cb3d4
distribution realization ( #33 )
...
Write tests for docker integration, as docker is how we will initially handle distribution on heterogeneous hardware.
2026-02-13 07:55:12 +00:00
d817ad952f
feat: distributed runtime ( #30 )
...
Major feature addition. For full details read `./docs/development_history/DISTRIBUTION.md`
2026-02-12 09:13:50 +00:00
