devpulse: watchdog/feedback README rewrite (was stale, missed everything that tripped VERA) + fix thread-unsafe watchdog test + true up branch-prompt timeout. README (last touched 06-23) predated the owner gate and Monitor-tool wake: now documents owner-only gating (seated owner:true, doctor --fix repairs), the 3-step wake mechanic (dispatch -> arm via Monitor TOOL -> Monitor return IS the wake; '1 monitor' IS the indicator), why passive wake-back can NEVER reach an interactive session (BLOCKED line = by design), cross-project @target resolution, 600s default + --timeout, stall events; feedback re-documented as THE owner-to-owner cross-project channel (Patrick ruling: cross-project comms impossible by design except feedback). TEST FIX: test_watchdog_agent _fake_clock_sleep patched GLOBAL time.sleep with a stateful fake -> prax logger's 3 daemon threads raced the fake clock forward and unlinked the fixture lock before watch_agent read it (nondeterministic 'no active lock' + uptime-sized elapsed; failed 4x today, passed in the same-day full-repo sweep). Fix: thread-scope the fake via caller-frame check (only agent-module sleeps advance the clock; foreign threads get a real 1ms sleep). Verified 17 pass 3x deterministic, devpulse suite 407 green. Branch prompt: watchdog default timeout claim corrected 1800s -> 600s (hit live: 3 watchdogs expired mid-build at ~600s).
This commit is contained in:
@@ -86,7 +86,7 @@ drone @flow list open # active plans
|
||||
|
||||
# Watchdog
|
||||
|
||||
Devpulse module. After dispatch, arm as a background task — it polls the dispatch lock and exits when the agent finishes. Resolves @target → branch path → `.ai_mail.local/.dispatch.lock`. Default timeout 1800s; `drone @devpulse watchdog --help` for the full reference.
|
||||
Devpulse module. After dispatch, arm as a background task — it polls the dispatch lock and exits when the agent finishes. Resolves @target → branch path → `.ai_mail.local/.dispatch.lock`. Default timeout **600s** — pass `--timeout <s>` for longer builds (verified live S300; `drone @devpulse watchdog --help` for the full reference).
|
||||
|
||||
```
|
||||
drone @ai_mail dispatch @target "Subject" "Body"
|
||||
|
||||
@@ -44,7 +44,7 @@ src/aipass/devpulse/
|
||||
│ │ └── watchdog/ # Agent, timer, schedule, registry
|
||||
│ └── plugins/ # Plugin extension point
|
||||
├── devpulse_json/ # JSON handler storage (config, data, logs per module)
|
||||
├── tests/ # 282 tests
|
||||
├── tests/ # 407 tests
|
||||
├── artifacts/ # Birth certificate, reports
|
||||
├── dropbox/ # Received files, archived plans, install audit
|
||||
├── docs/ # Transition notes
|
||||
@@ -55,11 +55,38 @@ src/aipass/devpulse/
|
||||
|
||||
All commands via `drone @devpulse <command>`:
|
||||
|
||||
### Watchdog — directed wake system
|
||||
### Watchdog — directed wake system (owner-only)
|
||||
|
||||
**Who may call it:** the project OWNER only — the first agent, seated as `owner: true`
|
||||
in the project's sealed `*_REGISTRY.json`. Portable: `@devpulse` in AIPass, `@vera` in
|
||||
Vera Studio, whoever owns elsewhere. A refusal means your project's owner isn't seated —
|
||||
run `aipass doctor` to see why and `aipass doctor --fix` to repair (DPLAN-0239).
|
||||
|
||||
**How the wake works (read this once, save a debugging session):**
|
||||
|
||||
1. `drone @ai_mail dispatch @target "Subject" "Body"` — hand off the work.
|
||||
2. **Immediately arm the watchdog via the harness Monitor TOOL** — never Bash
|
||||
`run_in_background` (its output goes nowhere and cannot wake you):
|
||||
`drone @devpulse watchdog agent @target --timeout 600`
|
||||
3. The status line shows **"1 monitor"** the moment it's armed — that IS the
|
||||
active-dispatch indicator. When `@target` finishes, the watchdog exits, the
|
||||
Monitor completes, and **your session is re-invoked with the result — that IS
|
||||
the wake.**
|
||||
|
||||
There is no passive wake: ai_mail's wake-back spawns a new headless process and can
|
||||
never inject into a live interactive session (`BLOCKED — interactive session` in the
|
||||
logs is that guard working as designed; it only serves senders whose session closed).
|
||||
If you dispatched and idle without arming, nothing will ever wake you.
|
||||
|
||||
`@target` resolves in the **caller's own project** (then falls back to scanning
|
||||
`~/Projects` registries) — external-project owners monitor their own agents with it.
|
||||
Default timeout is **600 s**; pass `--timeout <s>` for longer builds. Mid-watch it
|
||||
also emits `[watchdog.stall]` / `[watchdog.resumed]` events (no JSONL activity 120 s
|
||||
with no in-flight tool = probable stuck agent).
|
||||
|
||||
| Command | What it does |
|
||||
|---|---|
|
||||
| `watchdog agent @target` | Monitor dispatched agent until it finishes |
|
||||
| `watchdog agent @target [--timeout s]` | Wake when the dispatched agent exits (default 600 s) |
|
||||
| `watchdog timer <duration>` | Wake after duration (5m, 30s, 2h, 1h30m) |
|
||||
| `watchdog timer start/stop <name>` | Named duration tracking |
|
||||
| `watchdog schedule <HH:MM>` | Wait until a specific time |
|
||||
@@ -67,7 +94,14 @@ All commands via `drone @devpulse <command>`:
|
||||
| `watchdog cancel <id>` | Cancel a running watchdog |
|
||||
| `watchdog list` | List all watchdog entries |
|
||||
|
||||
### Feedback — personal cross-branch mailbox
|
||||
### Feedback — the owner-to-owner channel (owner-only)
|
||||
|
||||
ai_mail and dispatch stop at the project boundary — **cross-project comms is
|
||||
impossible by design, except feedback.** Project owners (managers) talk owner-to-owner
|
||||
through it: an external project's owner runs `drone @devpulse feedback send ...` from
|
||||
their project and it lands in devpulse's feedback mailbox; devpulse answers with
|
||||
`feedback reply`. Same owner gate as watchdog — unseated projects are refused until
|
||||
`aipass doctor --fix` seats them.
|
||||
|
||||
| Command | What it does |
|
||||
|---|---|
|
||||
@@ -75,7 +109,7 @@ All commands via `drone @devpulse <command>`:
|
||||
| `feedback inbox` | List all messages |
|
||||
| `feedback view <id>` | Read a message |
|
||||
| `feedback reply <id> "msg"` | Reply to sender |
|
||||
| `feedback send "subject" "body"` | Receive feedback from another agent |
|
||||
| `feedback send "subject" "body"` | Send feedback to devpulse (any project's owner may call) |
|
||||
|
||||
### Compass — rated decision store
|
||||
|
||||
@@ -122,7 +156,7 @@ drone @git log # Recent commits
|
||||
|
||||
All branches via dispatch orchestration. Watchdog monitoring for any dispatched agent. Feedback channel for cross-branch communication. Git operations (commit, PR, merge) for the entire project.
|
||||
|
||||
*Last Updated: 2026-06-23*
|
||||
*Last Updated: 2026-07-11*
|
||||
|
||||
---
|
||||
|
||||
|
||||
@@ -20,6 +20,7 @@ a live ai_mail dispatch flow. They're skipped by default in CI.
|
||||
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
import time
|
||||
from pathlib import Path
|
||||
|
||||
@@ -346,12 +347,27 @@ def test_stalltracker_resume_clears_stall(monkeypatch, capsys):
|
||||
|
||||
def _fake_clock_sleep(agent_module, monkeypatch, lock_file, unlink_at=200.0, step=60.0):
|
||||
"""Patch monotonic + sleep with a fake clock that advances `step`s per sleep
|
||||
and unlinks the dispatch lock once the clock passes `unlink_at` (loop exit)."""
|
||||
and unlinks the dispatch lock once the clock passes `unlink_at` (loop exit).
|
||||
|
||||
THREAD-SCOPED (S300): ``agent_module.time`` is the shared stdlib module, so
|
||||
patching ``time.sleep`` is process-global — background daemon threads (prax
|
||||
logger spawns three on first log) also hit the fake and would race the
|
||||
clock forward, unlinking the lock before ``watch_agent`` even reads it
|
||||
(flaked exactly so: 'no active lock' + uptime-sized elapsed). Only sleeps
|
||||
called FROM the agent module advance the clock; foreign callers get a tiny
|
||||
real sleep so they don't spin hot.
|
||||
"""
|
||||
clock = {"t": 0.0}
|
||||
agent_file = Path(agent_module.__file__).resolve()
|
||||
real_sleep = time.sleep
|
||||
monkeypatch.setattr(agent_module.time, "monotonic", lambda: clock["t"])
|
||||
|
||||
def fake_sleep(_seconds):
|
||||
"""Advance the fake clock and drop the lock once past unlink_at (loop exit)."""
|
||||
"""Advance the fake clock for agent-module callers only."""
|
||||
caller = Path(sys._getframe(1).f_code.co_filename).resolve()
|
||||
if caller != agent_file:
|
||||
real_sleep(0.001) # background thread — keep it off the fake clock
|
||||
return
|
||||
clock["t"] += step
|
||||
if clock["t"] >= unlink_at:
|
||||
lock_file.unlink(missing_ok=True)
|
||||
|
||||
Reference in New Issue
Block a user