devpulse: watchdog/feedback README rewrite (was stale, missed everything that tripped VERA) + fix thread-unsafe watchdog test + true up branch-prompt timeout. README (last touched 06-23) predated the owner gate and Monitor-tool wake: now documents owner-only gating (seated owner:true, doctor --fix repairs), the 3-step wake mechanic (dispatch -> arm via Monitor TOOL -> Monitor return IS the wake; '1 monitor' IS the indicator), why passive wake-back can NEVER reach an interactive session (BLOCKED line = by design), cross-project @target resolution, 600s default + --timeout, stall events; feedback re-documented as THE owner-to-owner cross-project channel (Patrick ruling: cross-project comms impossible by design except feedback). TEST FIX: test_watchdog_agent _fake_clock_sleep patched GLOBAL time.sleep with a stateful fake -> prax logger's 3 daemon threads raced the fake clock forward and unlinked the fixture lock before watch_agent read it (nondeterministic 'no active lock' + uptime-sized elapsed; failed 4x today, passed in the same-day full-repo sweep). Fix: thread-scope the fake via caller-frame check (only agent-module sleeps advance the clock; foreign threads get a real 1ms sleep). Verified 17 pass 3x deterministic, devpulse suite 407 green. Branch prompt: watchdog default timeout claim corrected 1800s -> 600s (hit live: 3 watchdogs expired mid-build at ~600s).

This commit is contained in:
AIOSAI
2026-07-11 17:58:18 -07:00
parent d511576fc0
commit 766d697e08
3 changed files with 59 additions and 9 deletions
@@ -86,7 +86,7 @@ drone @flow list open # active plans
# Watchdog
Devpulse module. After dispatch, arm as a background task — it polls the dispatch lock and exits when the agent finishes. Resolves @target → branch path → `.ai_mail.local/.dispatch.lock`. Default timeout 1800s; `drone @devpulse watchdog --help` for the full reference.
Devpulse module. After dispatch, arm as a background task — it polls the dispatch lock and exits when the agent finishes. Resolves @target → branch path → `.ai_mail.local/.dispatch.lock`. Default timeout **600s** — pass `--timeout <s>` for longer builds (verified live S300; `drone @devpulse watchdog --help` for the full reference).
```
drone @ai_mail dispatch @target "Subject" "Body"
+40 -6
View File
@@ -44,7 +44,7 @@ src/aipass/devpulse/
│ │ └── watchdog/ # Agent, timer, schedule, registry
│ └── plugins/ # Plugin extension point
├── devpulse_json/ # JSON handler storage (config, data, logs per module)
├── tests/ # 282 tests
├── tests/ # 407 tests
├── artifacts/ # Birth certificate, reports
├── dropbox/ # Received files, archived plans, install audit
├── docs/ # Transition notes
@@ -55,11 +55,38 @@ src/aipass/devpulse/
All commands via `drone @devpulse <command>`:
### Watchdog — directed wake system
### Watchdog — directed wake system (owner-only)
**Who may call it:** the project OWNER only — the first agent, seated as `owner: true`
in the project's sealed `*_REGISTRY.json`. Portable: `@devpulse` in AIPass, `@vera` in
Vera Studio, whoever owns elsewhere. A refusal means your project's owner isn't seated —
run `aipass doctor` to see why and `aipass doctor --fix` to repair (DPLAN-0239).
**How the wake works (read this once, save a debugging session):**
1. `drone @ai_mail dispatch @target "Subject" "Body"` — hand off the work.
2. **Immediately arm the watchdog via the harness Monitor TOOL** — never Bash
`run_in_background` (its output goes nowhere and cannot wake you):
`drone @devpulse watchdog agent @target --timeout 600`
3. The status line shows **"1 monitor"** the moment it's armed — that IS the
active-dispatch indicator. When `@target` finishes, the watchdog exits, the
Monitor completes, and **your session is re-invoked with the result — that IS
the wake.**
There is no passive wake: ai_mail's wake-back spawns a new headless process and can
never inject into a live interactive session (`BLOCKED — interactive session` in the
logs is that guard working as designed; it only serves senders whose session closed).
If you dispatched and idle without arming, nothing will ever wake you.
`@target` resolves in the **caller's own project** (then falls back to scanning
`~/Projects` registries) — external-project owners monitor their own agents with it.
Default timeout is **600 s**; pass `--timeout <s>` for longer builds. Mid-watch it
also emits `[watchdog.stall]` / `[watchdog.resumed]` events (no JSONL activity 120 s
with no in-flight tool = probable stuck agent).
| Command | What it does |
|---|---|
| `watchdog agent @target` | Monitor dispatched agent until it finishes |
| `watchdog agent @target [--timeout s]` | Wake when the dispatched agent exits (default 600 s) |
| `watchdog timer <duration>` | Wake after duration (5m, 30s, 2h, 1h30m) |
| `watchdog timer start/stop <name>` | Named duration tracking |
| `watchdog schedule <HH:MM>` | Wait until a specific time |
@@ -67,7 +94,14 @@ All commands via `drone @devpulse <command>`:
| `watchdog cancel <id>` | Cancel a running watchdog |
| `watchdog list` | List all watchdog entries |
### Feedback — personal cross-branch mailbox
### Feedback — the owner-to-owner channel (owner-only)
ai_mail and dispatch stop at the project boundary — **cross-project comms is
impossible by design, except feedback.** Project owners (managers) talk owner-to-owner
through it: an external project's owner runs `drone @devpulse feedback send ...` from
their project and it lands in devpulse's feedback mailbox; devpulse answers with
`feedback reply`. Same owner gate as watchdog — unseated projects are refused until
`aipass doctor --fix` seats them.
| Command | What it does |
|---|---|
@@ -75,7 +109,7 @@ All commands via `drone @devpulse <command>`:
| `feedback inbox` | List all messages |
| `feedback view <id>` | Read a message |
| `feedback reply <id> "msg"` | Reply to sender |
| `feedback send "subject" "body"` | Receive feedback from another agent |
| `feedback send "subject" "body"` | Send feedback to devpulse (any project's owner may call) |
### Compass — rated decision store
@@ -122,7 +156,7 @@ drone @git log # Recent commits
All branches via dispatch orchestration. Watchdog monitoring for any dispatched agent. Feedback channel for cross-branch communication. Git operations (commit, PR, merge) for the entire project.
*Last Updated: 2026-06-23*
*Last Updated: 2026-07-11*
---
@@ -20,6 +20,7 @@ a live ai_mail dispatch flow. They're skipped by default in CI.
import json
import os
import sys
import time
from pathlib import Path
@@ -346,12 +347,27 @@ def test_stalltracker_resume_clears_stall(monkeypatch, capsys):
def _fake_clock_sleep(agent_module, monkeypatch, lock_file, unlink_at=200.0, step=60.0):
"""Patch monotonic + sleep with a fake clock that advances `step`s per sleep
and unlinks the dispatch lock once the clock passes `unlink_at` (loop exit)."""
and unlinks the dispatch lock once the clock passes `unlink_at` (loop exit).
THREAD-SCOPED (S300): ``agent_module.time`` is the shared stdlib module, so
patching ``time.sleep`` is process-global — background daemon threads (prax
logger spawns three on first log) also hit the fake and would race the
clock forward, unlinking the lock before ``watch_agent`` even reads it
(flaked exactly so: 'no active lock' + uptime-sized elapsed). Only sleeps
called FROM the agent module advance the clock; foreign callers get a tiny
real sleep so they don't spin hot.
"""
clock = {"t": 0.0}
agent_file = Path(agent_module.__file__).resolve()
real_sleep = time.sleep
monkeypatch.setattr(agent_module.time, "monotonic", lambda: clock["t"])
def fake_sleep(_seconds):
"""Advance the fake clock and drop the lock once past unlink_at (loop exit)."""
"""Advance the fake clock for agent-module callers only."""
caller = Path(sys._getframe(1).f_code.co_filename).resolve()
if caller != agent_file:
real_sleep(0.001) # background thread — keep it off the fake clock
return
clock["t"] += step
if clock["t"] >= unlink_at:
lock_file.unlink(missing_ok=True)