From 455013364c6d594ccbbb22bf57416876d270f755 Mon Sep 17 00:00:00 2001 From: AIOSAI Date: Wed, 22 Apr 2026 21:20:16 -0700 Subject: [PATCH 1/8] feat(ai_mail): docs(ai_mail): comprehensive README update Co-Authored-By: @ai_mail --- src/aipass/ai_mail/README.md | 171 ++++++++++++++++++++++++++++------- 1 file changed, 140 insertions(+), 31 deletions(-) diff --git a/src/aipass/ai_mail/README.md b/src/aipass/ai_mail/README.md index a6cd3cf5..7ae5e78c 100644 --- a/src/aipass/ai_mail/README.md +++ b/src/aipass/ai_mail/README.md @@ -9,16 +9,35 @@ --- -**Status:** Operational. Core email workflow (send/inbox/reply/close), dispatch system (with startup timeout + auto-retry), daemon, desktop notifications all working. Seedgo 99%. +**Status:** Operational | **Seedgo:** 100% (34/34) | **Tests:** 355 pass | **Battle Tested:** S62 -## Commands / Usage +## Commands ```bash +# Dispatch (send + wake in one step) drone @ai_mail dispatch @target "Subject" "Body" # Send dispatch email + wake -drone @ai_mail dispatch @target "Subject" "Body" --fresh # Send + fresh wake -drone @ai_mail email @target "Subject" "Body" # Send email (no wake) +drone @ai_mail dispatch @target "Subject" "Body" --fresh # Send + fresh wake (new session) drone @ai_mail dispatch wake @target # Wake only (no email) -drone @ai_mail inbox # Check inbox + +# Send mail (no wake) +drone @ai_mail email @target "Subject" "Body" # Send to one branch +drone @ai_mail email @all "Subject" "Body" # Broadcast to all branches +drone @ai_mail email @target "Subj" "Body" --from @spawn # Explicit sender override + +# Read mail +drone @ai_mail inbox # List all emails (new + opened) +drone @ai_mail view # View email (marks as opened) +drone @ai_mail view latest # View most recent email + +# Resolve mail +drone @ai_mail reply "message" # Reply + close + archive original +drone @ai_mail close # Close single email +drone @ai_mail close # Close multiple emails +drone @ai_mail close all # Close all emails + +# Other +drone @ai_mail sent # View sent messages +drone @ai_mail contacts # List all known branches drone @ai_mail --help # Full help ``` @@ -34,16 +53,55 @@ new → opened → closed - **opened** — Viewed by recipient, awaiting action - **closed** — Replied or dismissed, archived automatically +Each branch's mailbox lives at `/.ai_mail.local/inbox.json`. Sent copies go to `.ai_mail.local/sent/`. File locking (`fcntl`/`msvcrt`) protects concurrent inbox writes. + ## Dispatch System -The `dispatch` command sends an email and wakes the target branch in one step. A polling daemon can also watch inboxes and spawn agents for `auto_execute` dispatch emails automatically. +The `dispatch` command sends an email and wakes the target branch in one step. Dispatch emails carry `auto_execute: true` and a task header, signaling the target agent to process them as work items. + +### Wake Pipeline + +1. `dispatch.py` orchestrates: send email via `send_to_single()`, then wake via `wake_branch()` +2. `wake.py` resolves the branch from the registry, finds the `claude` binary, spawns a subprocess +3. `dispatch_monitor.py` wraps the claude process with safety features: + - **Startup health check** — monitors JSONL session files for 90s, kills if no activity + - **Auto-retry** — 3 strikes: attempt 1+2 resume, attempt 3 fresh (new session) + - **Bounce email** — on final failure, sends error report back to sender + - **Lock cleanup** — removes `.dispatch.lock` when agent exits +4. After wake, `_spawn_watchdog()` auto-launches `drone @devpulse watchdog agent @target` as a detached background process + +### Safety Limits + +- PID-based locking prevents concurrent agents per branch (`.dispatch.lock`) +- Max turns per wake, max dispatches per branch per day +- `WAKE_BLOCKLIST` protects `@devpulse` from cross-branch manual wakes +- `dispatch_monitor.py` strips `AIPASS_CALLER_*` env vars to prevent parent context leaking into agent identity +- `AIPASS_BRANCH_NAME` env var set in spawn_env for CWD-independent identity + +### Daemon + +The polling daemon (`daemon.py`) watches inboxes for `auto_execute` dispatch emails and spawns agents automatically. It also runs the AIPASS-TEST token protocol: `scan_and_ack_test_emails()` intercepts ping-test messages and auto-replies with "ack" before dispatch processing. + +## Sender Identity + +Branch identity detection follows a priority chain in `detect_branch_from_pwd()`: + +1. `AIPASS_CALLER_BRANCH` env var (set by drone router from passport or `AIPASS_BRANCH_NAME`) +2. Contacts address book lookup (fastest path for registered branches) +3. Registry lookup by name +4. `AIPASS_CALLER_CWD` / `Path.cwd()` walk-up to find `.trinity/passport.json` +5. Registry lookup by path + +If all fail, detection returns `None` and the operation fails loudly. Wrong identity is worse than no identity. + +The `--from @branch` flag on send/email commands provides an explicit sender override for callers outside branch directories. + +## Cross-Project Email + +External projects (outside the AIPass repo) can send to AIPass branches. On delivery, `delivery.py` stores a `reply_path` on the message (the sender's `inbox.json` path, resolved from `AIPASS_CALLER_CWD`). Replies use `_deliver_via_reply_path()` to write directly to the external inbox without needing registry lookup. + +The contacts system (`contacts.py`) maintains an address book at `.ai_mail.local/contacts.json`, auto-registering branches on every send/receive. This enables fast sender detection for known branches without CWD walking or registry lookups. -- Agents are ephemeral (wake, do work, exit) -- Safety limits: max turns per wake, max dispatches per branch per day -- PID-based locking prevents concurrent agents per branch -- Startup health check: monitors JSONL session files for 90s, kills if no activity -- Auto-retry: 3 strikes (resume, resume, fresh) before bounce -- Failed agents trigger bounce emails back to sender ## Architecture Follows the standard AIPass 3-layer pattern: @@ -51,20 +109,65 @@ Follows the standard AIPass 3-layer pattern: ``` ai_mail/ ├── apps/ -│ ├── ai_mail.py # Entry point (auto-discovers modules) +│ ├── ai_mail.py # Entry point (auto-discovers modules) │ ├── modules/ -│ │ ├── email.py # Inbox, view, reply, close, contacts, routing -│ │ ├── email_send.py # Send orchestration (direct, interactive, broadcast) -│ │ ├── dispatch.py # Dispatch status, daemon, wake +│ │ ├── email.py # Inbox, view, reply, close, contacts, routing +│ │ ├── email_send.py # Send orchestration (direct, interactive, broadcast) +│ │ └── dispatch.py # Dispatch send+wake, status, daemon control │ └── handlers/ -│ ├── email/ # Delivery, formatting, inbox ops, purge, reply -│ ├── dispatch/ # Daemon, wake, dispatch_monitor, status, test_token -│ ├── registry/ # Branch registry read -│ ├── users/ # Branch detection, user lookup -│ ├── json_utils/ # JSON I/O helpers (load_json, save_json) -│ ├── paths.py # Shared find_repo_root() utility -│ ├── notify.py # Desktop notifications (dbus) -│ └── central_writer.py # Central inbox stats aggregation +│ ├── email/ +│ │ ├── delivery.py # Core delivery pipeline (write to recipient inbox) +│ │ ├── send.py # Sender resolution + send helpers +│ │ ├── send_args.py # Argument parsing for send command +│ │ ├── inbox_ops.py # Inbox loading + v1→v2 migration +│ │ ├── inbox_cleanup.py # Mark opened/closed + archive +│ │ ├── inbox_lock.py # File locking (fcntl/msvcrt cross-platform) +│ │ ├── inbox_resolve.py # Resolve inbox path from args or caller +│ │ ├── reply.py # Reply + auto-close original +│ │ ├── close_ops.py # Batch close operations +│ │ ├── contacts.py # Address book for branch routing +│ │ ├── create.py # Email file creation (sent/ folder) +│ │ ├── format.py # Display formatting +│ │ ├── header.py # Dispatch header injection +│ │ ├── footer.py # Email footer +│ │ ├── purge.py # Auto-purge sent/deleted folders +│ │ ├── error_dispatch.py # Error reporting via email +│ │ └── dashboard_sync.py # Dashboard integration +│ ├── dispatch/ +│ │ ├── daemon.py # Polls inboxes, spawns agents for dispatch emails +│ │ ├── wake.py # Wakes branches via claude subprocess +│ │ ├── dispatch_monitor.py # Wraps claude process (bounce + lock cleanup) +│ │ ├── status.py # Dispatch log I/O +│ │ └── test_token.py # AIPASS-TEST ping protocol (auto-ack) +│ ├── registry/ +│ │ └── read.py # Registry reading + get_all_branches() +│ ├── users/ +│ │ ├── branch_detection.py # CWD/env-based branch identity detection +│ │ └── user.py # Current user detection (get_current_user) +│ ├── json_utils/ +│ │ └── json_handler.py # Auto-creating JSON system +│ ├── paths.py # Shared find_repo_root() utility +│ ├── notify.py # Desktop notifications (dbus direct) +│ └── central_writer.py # Central inbox stats aggregation +└── tests/ # 355 tests across 16 test files + ├── conftest.py # Shared fixtures (mock_logger, mock_json_handler) + ├── test_daemon.py # Daemon config, state, kill switch, dispatch check + ├── test_dispatch_monitor.py # Monitor safety features, env stripping + ├── test_dispatch_status.py # Log I/O, age calculation + ├── test_dispatch_watchdog.py # Watchdog auto-spawn + ├── test_wake.py # Branch resolution, PID checks, lock files + ├── test_wake_blocklist.py # Wake protection for @devpulse + ├── test_delivery.py # Inbox migration, private branches, pipeline + ├── test_send_identity.py # Sender identity chain (36 tests) + ├── test_user_paths.py # Mailbox path resolution (13 tests) + ├── test_contacts.py # Address book operations + ├── test_inbox_ops.py # Inbox loading + migration + ├── test_registry_read.py # Registry parsing + branch lookup + ├── test_central_writer.py # Central stats aggregation + ├── test_cli_routing.py # CLI routing + help/version + ├── test_json_handler.py # JSON I/O helpers + ├── test_notify.py # Desktop notification dbus calls + └── test_paths.py # find_repo_root() utility ``` ## Integration Points @@ -73,16 +176,22 @@ ai_mail/ - `aipass.prax` — Logging via `system_logger` - `aipass.cli` — Console output and display formatting - `aipass.drone` — Command routing and `@branch` resolution -- Python stdlib (`pathlib`, `json`, `argparse`, `importlib`) +- `aipass.trigger` — `trigger.fire()` for `email_dispatched` events +- Python stdlib (`pathlib`, `json`, `argparse`, `importlib`, `subprocess`, `fcntl`) ### Provides To -- All modules — inter-branch messaging (send/receive/reply/close) -- Dispatch system — autonomous task execution via `--dispatch` flag -- Branch contacts — address book for `@branch` routing +- **All branches** — inter-branch messaging (send/receive/reply/close) +- **Dispatch system** — autonomous task execution via `auto_execute` emails +- **Branch contacts** — address book for `@branch` routing +- **trigger branch** — `deliver_email_to_branch()` imported directly for event-driven delivery +- **Desktop** — dbus notifications for delivery, wake, completion events + +## Known Issues + +- **DPLAN-0138**: Inbox backdoor audit identified 2 write path classes — ad-hoc direct writes (detectable by non-UUID ID format) and `_deliver_via_reply_path()` bypass (no lock, no notification). Fix pending. +- **Caller detection**: `BRANCH DETECTION FAILED` when callers don't set `AIPASS_CALLER_BRANCH` (low severity, caller-side fix — use `--from` flag) +- **Cross-branch writes**: ai_mail not in trusted cross-writers list for `system-pr` --- -*Last Updated: 2026-04-07* - ---- [← Back to AIPass](../../../README.md) From ef4f832bc3267b6c35bb4ba0ea7697fbe024c639 Mon Sep 17 00:00:00 2001 From: AIOSAI Date: Wed, 22 Apr 2026 21:20:31 -0700 Subject: [PATCH 2/8] feat(cli): docs(cli): comprehensive README update Co-Authored-By: @cli --- src/aipass/cli/README.md | 178 ++++++++++++++++++++++++++++----------- 1 file changed, 129 insertions(+), 49 deletions(-) diff --git a/src/aipass/cli/README.md b/src/aipass/cli/README.md index ebd5cd90..934e3166 100644 --- a/src/aipass/cli/README.md +++ b/src/aipass/cli/README.md @@ -2,14 +2,17 @@ # CLI -**Purpose:** Display and output formatting service for AIPass modules. Provides consistent terminal output — headers, success/error/warning messages, section breaks, and operation templates — so every module looks the same without duplicating Rich formatting code. +**Purpose:** Display and output formatting service for all AIPass branches, plus the `aipass init` project bootstrap command. Provides consistent terminal output — headers, success/error/warning messages, section breaks, and operation templates — so every branch looks the same without duplicating Rich formatting code. **Module:** `aipass.cli` -**Seedgo:** 100% -**Tests:** 60+ passing (6 files, 5/5 modules covered) +**Version:** 2.0.0 +**Seedgo:** 100% (34/34 standards) +**Tests:** 171 passing (6 files, 5/5 modules covered) **Last Updated:** 2026-04-22 ## Usage +### Display Functions + ```python from aipass.cli import header, success, error, warning, section @@ -27,13 +30,13 @@ from aipass.cli import operation_start, operation_complete operation_start("Processing", count=10) # ... do work ... -operation_complete(created=5, skipped=3, failed=0) +operation_complete(created=5, skipped=3, failed=0, time="1.2s") ``` ### Fatal (exit on error) ```python -from aipass.cli.apps.modules.display import fatal +from aipass.cli import fatal fatal("Config file missing", suggestion="Run aipass init first") # Prints error message + suggestion, then calls sys.exit(1) @@ -47,55 +50,48 @@ from aipass.cli import console console.print("[bold cyan]Custom Rich output[/bold cyan]") ``` -## Architecture +## Public API -``` -cli/ -├── apps/ -│ ├── cli.py # Entry point (main, discover_modules, route_command) -│ ├── modules/ -│ │ ├── display.py # header, success, error, warning, fatal, section -│ │ ├── templates.py # operation_start, operation_complete -│ │ └── init_project.py # aipass init command routing -│ └── handlers/ -│ ├── init/ # Project bootstrap logic -│ │ ├── bootstrap.py -│ │ └── scaffold_content.py -│ ├── json/ # JSON file management -│ │ └── json_handler.py -│ └── templates/ # Empty — placeholder from scaffold -├── cli_json/ # Auto-created JSON output (three-file pattern) -├── dropbox/ # Inbound file drop -├── logs/ # Branch-level logs -├── tests/ # 161 tests across 6 files -│ ├── test_bootstrap.py # bootstrap.py handler tests -│ ├── test_json_handler.py # json_handler tests -│ ├── test_display.py # display module tests -│ ├── test_templates.py # templates module tests -│ ├── test_init_project.py # init_project module tests -│ └── test_integration.py # main() flow + entry point tests -└── .archive/ # Archived stubs (extensions/, json_templates/) -``` +All display functions are importable from the top-level package or `apps/modules/`: -- `apps/modules/` — Public API. Import from here. -- `apps/handlers/` — Internal implementation. Don't import directly. +| Function | Signature | Purpose | +|----------|-----------|---------| +| `header()` | `header(title, details=None)` | Bordered section header with optional key-value pairs | +| `success()` | `success(message, **kwargs)` | Green checkmark message with metadata | +| `error()` | `error(message, suggestion=None)` | Red error with optional suggestion | +| `warning()` | `warning(message, details=None)` | Yellow warning with optional details | +| `fatal()` | `fatal(message, suggestion=None)` | Error + `sys.exit(1)` for unrecoverable failures | +| `section()` | `section(title)` | Visual section separator with title | +| `operation_start()` | `operation_start(operation, **details)` | Standard operation begin header | +| `operation_complete()` | `operation_complete(**summary)` | Completion summary with optional timing | +| `console` | Rich Console instance | Standard output console | +| `err_console` | Rich Console instance | Stderr console | + +Import paths (all equivalent): +```python +from aipass.cli import header # Top-level re-export +from aipass.cli.apps.modules import header # Module-level +from aipass.cli.apps.modules.display import header # Direct +``` ## Commands ```bash # Via drone -drone @cli --help # Show services and Rich formatting showcase -drone @cli --version # Show version -drone @cli # Show discovered modules (introspection) +drone @cli --help # Services + Rich formatting showcase +drone @cli --version # Version (v2.0.0) +drone @cli # Module discovery (introspection) drone @cli aipass # Show aipass subcommands drone @cli aipass init # Bootstrap AIPass project in current dir drone @cli aipass init /path # Bootstrap in target directory drone @cli aipass init /path MyProject # Bootstrap with custom name -drone @cli aipass init agent # Create agent in project (routes to spawn) +drone @cli aipass init update # Re-sync managed scaffold files +drone @cli aipass init update /path # Re-sync in target directory +drone @cli aipass init agent # Create agent (routes to spawn) drone @cli aipass init --help # Detailed init usage -drone @cli display # Display module introspection +drone @cli display # Display module info drone @cli display demo # Run display function showcase -drone @cli templates # Templates module introspection +drone @cli templates # Templates module info drone @cli templates demo # Run templates function showcase # Standalone (no drone required) @@ -104,20 +100,104 @@ python -m aipass.cli aipass init /path # Bootstrap directly aipass --help # Via console_scripts entry point ``` ---- +## aipass init + +Bootstraps a new AIPass project with 21 items (when AIPASS_HOME is detected): + +| Category | Items Created | +|----------|---------------| +| Identity | `*_REGISTRY.json` | +| Prompts | `.aipass/aipass_global_prompt.md`, `CLAUDE.md`, `AGENTS.md`, `GEMINI.md` | +| Docs | `README.md`, `STATUS.local.md` | +| Config | `.claude/settings.json`, `.gitignore` | +| Slash commands | `.claude/commands/prep.md`, `.claude/commands/memo.md` | +| Hooks | 7 enforcement/injector hooks in `.claude/hooks/` | +| Dirs | `hooks/`, `src/` | +| Mail | `.ai_mail.local/inbox.json` | + +Settings.json wires 5 hook event types: +- **UserPromptSubmit** (5 hooks): global prompt, local prompt, branch prompt loader, email notification, identity injector +- **PostToolUse**: auto-fix diagnostics +- **PreToolUse**: pre-edit gate +- **Stop**: subagent stop gate +- **PreCompact**: pre-compact + +`aipass init update` re-syncs managed files (prompts, config, hooks) without touching user-owned files (registry, README, STATUS.local.md, .gitignore, mailbox). + +Cross-platform: local prompt discovery uses `python3 -c` with pathlib (no bash dependency). A `setup.py` installer handles Windows + Linux + macOS. + +## Architecture + +``` +cli/ +├── __init__.py # Public API exports + cli_entry() +├── __main__.py # python -m aipass.cli entry +├── apps/ +│ ├── cli.py # Entry point (main, discover_modules, route_command) +│ ├── modules/ # PUBLIC — import from here +│ │ ├── __init__.py # Re-exports all display + template functions +│ │ ├── display.py # header, success, error, warning, fatal, section +│ │ ├── templates.py # operation_start, operation_complete +│ │ └── init_project.py # aipass init command routing +│ └── handlers/ # PRIVATE — internal implementation +│ ├── init/ +│ │ ├── bootstrap.py # init_project(), update_project() (511 lines) +│ │ └── scaffold_content.py # 10 content generators (502 lines) +│ ├── json/ +│ │ └── json_handler.py # JSON lifecycle (CRUD, validation, rotation) +│ └── templates/ # Empty — placeholder from scaffold +├── tests/ # 171 tests across 6 files +│ ├── test_bootstrap.py # 60 tests — init/update/hooks/memo/mailbox +│ ├── test_json_handler.py # 35 tests — CRUD, validation, rotation +│ ├── test_display.py # 28 tests — all display functions + routing +│ ├── test_templates.py # 19 tests — operation templates + routing +│ ├── test_init_project.py # 14 tests — command routing, error handling +│ └── test_integration.py # 8 tests — main() flow, entry points +├── cli_json/ # Auto-created JSON (config, data, log) +├── logs/ # Branch-level logs +└── .archive/ # Archived stubs (extensions/, json_templates/) +``` + +**Two-tier design:** +- `apps/modules/` — Public API. Import from here. +- `apps/handlers/` — Internal implementation. Don't import directly. + +## JSON Handler + +Manages the three-file JSON pattern (config, data, log) for any module: + +```python +from aipass.cli.apps.handlers.json import json_handler + +json_handler.log_operation("files_created", {"count": 12}) +data = json_handler.load_json("cli", "config") +json_handler.save_json("cli", "data", {"key": "value"}) +json_handler.ensure_module_jsons("cli") # Create all 3 if missing +``` ## Integration Points ### Depends On -- `aipass.prax` — Logging via `system_logger` -- `rich` — Rich library for terminal formatting (Table, Panel, Columns, Text) -- Python stdlib (`sys`, `importlib`, `pathlib`) +- `rich` — Terminal formatting (Table, Panel, Text, Console) +- Python stdlib (`sys`, `importlib`, `pathlib`, `json`) + +### Cannot Import +- `aipass.prax` — Circular dependency (prax depends on cli). Display/templates modules bypass this with documented entries in `.seedgo/bypass.json`. ### Provides To -- All modules — display formatting (headers, success/error/warning, section breaks) -- All modules — operation templates (`operation_start`, `operation_complete`) -- All modules — Rich console access -- All users — `aipass init` project bootstrap command +- **All branches** — Display formatting (header, success, error, warning, fatal, section) +- **All branches** — Operation templates (operation_start, operation_complete) +- **All branches** — Rich console access +- **All users** — `aipass init` project bootstrap + `aipass init update` refresh +- **All users** — `aipass init agent` routing to spawn + +## Entry Points + +| Entry | Command | How | +|-------|---------|-----| +| drone | `drone @cli [command]` | Drone routes to `apps/cli.py:main()` | +| Module | `python -m aipass.cli [args]` | `__main__.py` calls `main()` | +| PATH | `aipass [args]` | `console_scripts` calls `cli_entry()` | --- From 192e3563acec5a67ed5a915791a643423b9cea3a Mon Sep 17 00:00:00 2001 From: AIOSAI Date: Wed, 22 Apr 2026 21:20:39 -0700 Subject: [PATCH 3/8] feat(flow): docs(flow): comprehensive README update Co-Authored-By: @flow --- src/aipass/flow/README.md | 129 ++++++++++++++++++++++++-------------- 1 file changed, 81 insertions(+), 48 deletions(-) diff --git a/src/aipass/flow/README.md b/src/aipass/flow/README.md index 5c8a1827..dacf794f 100644 --- a/src/aipass/flow/README.md +++ b/src/aipass/flow/README.md @@ -2,8 +2,9 @@ # Flow -**Purpose:** Unified plan lifecycle management for AIPass. Creates, tracks, closes, and archives numbered work plans across multiple plan types via a filesystem-driven template registry. Foreground archival with vector intake verification, cross-branch aggregation, and self-healing registries. +**Purpose:** Unified plan lifecycle management for AIPass. Creates, tracks, closes, and archives numbered work plans across multiple plan types via a filesystem-driven template registry. **Module:** `aipass.flow` +**Version:** 2.2.1 **Created:** 2025-11-15 **Last Updated:** 2026-04-22 @@ -11,18 +12,21 @@ ## Overview -### What I Do -- Create numbered plans from type-specific templates via `templates/` plugins -- Unified create/close/list commands for all plan types (FPLAN, DPLAN, RPLAN, TDPLAN, ...) -- Close plans with foreground archival to `.backup/processed_plans/` -- Vector intake on close via `drone @memory process-plans` with chroma verification -- List and filter plans across branches and plan types -- Restore plans from backups -- Template registry management: register, unregister, scan, auto-heal -- Aggregate plans across branches -- `--dry-run` for close preview +Flow is AIPass's plan management system. Every branch uses flow to create, track, close, and archive work plans. Plans are numbered markdown files (`FPLAN-0042_subject_2026-04-22.md`) organized by type, with per-type registries tracking status and metadata. -## Commands / Usage +### What I Do +- Create numbered plans from type-specific templates +- Close plans with foreground archival and vector intake verification +- List and filter plans across all registered types +- Restore closed plans from backups +- Manage plan types via filesystem-driven template registry +- Aggregate plans across branches for central reporting +- Self-heal registries (orphan detection, auto-close missing files, auto-register new template dirs) +- Preview close operations with `--dry-run` + +--- + +## Commands ```bash # Create plans @@ -35,6 +39,7 @@ drone @flow close FPLAN-0042 # Close specific plan drone @flow close DPLAN-0005 # Close a DPLAN drone @flow close --all # Close all open plans drone @flow close --all --dry-run # Preview what would close +drone @flow close --dry-run FPLAN-0042 # Preview single close # List plans drone @flow list open # List open plans (all types) @@ -46,10 +51,16 @@ drone @flow scan # Find unregistered directories drone @flow register # Register new plan type drone @flow unregister # Remove plan type +# Registry +drone @flow registry scan # Scan filesystem, detect mismatches +drone @flow registry status # Show registry health + # Other drone @flow restore FPLAN-0042 # Reopen a closed plan drone @flow aggregate # Cross-branch plan aggregation +drone @flow post # Background post-close processing drone @flow --help # Full help +drone @flow --version # Version string ``` --- @@ -60,92 +71,114 @@ drone @flow --help # Full help flow/ ├── apps/ │ ├── flow.py # Entry point (auto-discovers modules) -│ ├── modules/ # Business logic (thin orchestrators) +│ ├── modules/ # Thin orchestrators (8 modules) │ │ ├── create_plan.py # Plan creation with template support │ │ ├── close_plan.py # Closure with foreground archival + vector verify │ │ ├── list_plans.py # Plan listing and filtering │ │ ├── restore_plan.py # Plan recovery from backups -│ │ ├── registry_monitor.py # Orphan detection, auto-healing +│ │ ├── registry_monitor.py # Registry scanning and auto-healing │ │ ├── aggregate_central.py # Cross-branch plan aggregation -│ │ ├── post_close_runner.py # Background post-processing *(partial — archival moved to foreground)* +│ │ ├── post_close_runner.py # Background post-processing with lock management │ │ └── template_manager.py # Template registry management │ └── handlers/ # Implementation details -│ ├── plan/ # Lifecycle, file ops, validation, close_ops -│ ├── registry/ # Load, save, auto-heal -│ ├── template/ # Plan type loader + template resolution + registry_ops -│ ├── dashboard/ # Status aggregation -│ ├── mbank/ # Memory archival -│ └── json/ # Auto-creating JSON handler -├── templates/ # Plan type plugins (DATA, not code) +│ ├── plan/ # Lifecycle: create, close, list, restore, display, validation +│ ├── registry/ # Load, save, auto-heal registries +│ ├── template/ # Plan type loader, template resolution, registry CRUD +│ ├── dashboard/ # Status push to local, central, branch dashboards +│ ├── mbank/ # Memory archival and plan processing +│ ├── runner/ # Lock file operations for background processes +│ ├── json/ # Auto-creating JSON handler +│ ├── summary/ # Plan summarization (vestigial) +│ ├── config/ # Configuration loading +│ └── events/ # Event handling stubs +├── templates/ # Plan type plugins (data, not code) │ ├── flow_plans/ # FPLAN templates (default, master) │ ├── dev_plans/ # DPLAN templates (default) │ ├── research_plans/ # RPLAN templates (default) │ ├── team_dev_plans/ # TDPLAN templates (default) -│ └── audit_plans/ # Unregistered — needs `drone @flow register` +│ └── audit_plans/ # APLAN templates (default) ├── flow_json/ # Per-type registries + template_registry.json -├── tests/ # 452 tests, 90/90 functions covered -├── docs/ # Documentation +├── tests/ # 423 tests, 17 test files └── .archive/ # Archived legacy code ``` +### Design Principles +- **Modules are thin orchestrators** — no business logic, route to handlers and display results +- **Handlers are stateless** — modules inject dependencies (registry loader, paths, config) +- **Plan types are filesystem-driven** — drop a template dir, register a prefix, done +- **Auto-discovery** — `flow.py` finds modules via `handle_command()` convention; `plan_type_loader.py` discovers types from `template_registry.json` + --- ## Plan Types -Plan types are filesystem-driven. Drop a directory with `.md` templates into `templates/`, register it with a prefix. No per-directory JSON config needed. - | Type | Prefix | Registry | Templates | |------|--------|----------|-----------| | flow_plans | FPLAN | fplan_registry.json | default, master | | dev_plans | DPLAN | dplan_registry.json | default | | research_plans | RPLAN | rplan_registry.json | default | | team_dev_plans | TDPLAN | tdplan_registry.json | default | +| audit_plans | APLAN | aplan_registry.json | default | -Plans follow the convention `{PREFIX}-{NNNN}_topic_slug_YYYY-MM-DD.md` where NNNN is auto-incrementing per type. +Plans follow the naming convention `{PREFIX}-{NNNN}_topic_slug_YYYY-MM-DD.md` where NNNN auto-increments per type. -### Auto-heal -- Template registry auto-prunes orphaned types (directory deleted → entry + plan registry JSON removed on next command) +### Adding a New Plan Type +1. Create a directory in `templates/` with one or more `.md` template files +2. Run `drone @flow register ` (or let auto-registration detect it on next command) +3. Use `drone @flow create . "Subject" ` to create plans of the new type + +### Auto-healing +- Template registry auto-prunes orphaned types (directory deleted → entry + plan registry JSON removed) - Plan registries auto-close entries for missing files +- New template directories auto-register on next command --- ## Close Pipeline On `drone @flow close`: -1. Template check (fast-delete empty templates) -2. Mark as closed in registry -3. Archive to `.backup/processed_plans/` (foreground, sets processed/cleanup flags atomically) -4. Vector intake: `drone @memory process-plans` + `is_plan_vectorized()` verification -5. Dashboard updates (local + central + branch) -6. Append to `CLOSED_PLANS.local.json` +1. **Template check** — fast-delete empty/template-only plans +2. **Mark closed** — update plan registry with closure timestamp +3. **Archive** — move to `.backup/processed_plans/` (foreground, sets processed/cleanup flags atomically) +4. **Vector intake** — `drone @memory process-plans` + `is_plan_vectorized()` verification +5. **Dashboard updates** — local, central, and branch dashboards +6. **Append** — write to `CLOSED_PLANS.local.json` + +Vector verification displays in console: "Vectorized: N chunks in chroma" or "NOT vectorized". --- ## Integration Points ### Depends On -- `aipass.cli` -- Terminal formatting (console, header, success, error) -- `aipass.prax` -- Structured logging via `system_logger` -- `aipass.trigger` -- Error reporting (optional) -- `aipass.memory` -- Vector intake on plan close (`process-plans` + `verify`) -- Python stdlib (`pathlib`, `json`, `importlib`, `sys`, `signal`, `subprocess`, `shutil`) +- `aipass.cli` — Rich terminal formatting (`console`, `header`, `success`, `error`, `warning`) +- `aipass.prax` — Structured logging via `system_logger` +- `aipass.memory` — Vector intake on plan close +- `aipass.trigger` — Error reporting (optional) ### Provides To -- All branches -- Plan creation, tracking, closure, and archival -- `aipass.devpulse` -- Plan status aggregation for system dashboards -- Registry: Per-type registries in `flow_json/` +- All branches — plan creation, tracking, closure, and archival +- `aipass.devpulse` — plan status aggregation for system dashboards +- Central reporting — `PLANS.central.json` via aggregate --- ## Quality -- **Seedgo:** 100% (all 33 standards) -- **Tests:** 452 tests, 90/90 public functions covered -- **Last audit:** 2026-04-09 +- **Seedgo:** 100% (33/33 standards) +- **Tests:** 423 passed, 83/87 public functions tested (95%) +- **Source files:** 39 tracked by seedgo +- **Last audit:** 2026-04-22 +- **Battle test:** 16/16 commands pass via drone CLI (2026-04-22) + +### Known Issues +- Registry scan fires trigger events that are never handled (by design — foreground close handles everything) +- Dashboard push warns on some closes +- `mbank/process.py` at 669 lines (nearing 700 limit) --- -*Last Updated: 2026-04-07* +*Last Updated: 2026-04-22* --- [← Back to AIPass](../../../README.md) From 58b7e12b5a1927a8035e45ef6ea94069d96881ec Mon Sep 17 00:00:00 2001 From: AIOSAI Date: Wed, 22 Apr 2026 21:21:53 -0700 Subject: [PATCH 4/8] =?UTF-8?q?feat(system):=20docs(prax):=20comprehensive?= =?UTF-8?q?=20README=20update=20=E2=80=94=20full=20rewrite=20with=20accura?= =?UTF-8?q?te=20architecture,=20commands,=20API=20docs,=20test=20coverage?= =?UTF-8?q?=20table?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-Authored-By: @devpulse --- .claude/hooks/.diagnostics_state.json | 1 - src/aipass/memory/README.md | 220 ++++++++++++--------- src/aipass/prax/CLOSED_PLANS.local.json | 25 ++- src/aipass/prax/README.md | 241 ++++++++++++++++-------- 4 files changed, 311 insertions(+), 176 deletions(-) delete mode 100644 .claude/hooks/.diagnostics_state.json diff --git a/.claude/hooks/.diagnostics_state.json b/.claude/hooks/.diagnostics_state.json deleted file mode 100644 index 04893a83..00000000 --- a/.claude/hooks/.diagnostics_state.json +++ /dev/null @@ -1 +0,0 @@ -{"file": "/home/patrick/Projects/AIPass/src/aipass/seedgo/apps/modules/inbox_audit.py", "errors": [{"line": 86, "message": "E501: Line too long (125 > 120)"}]} \ No newline at end of file diff --git a/src/aipass/memory/README.md b/src/aipass/memory/README.md index 744267ec..a9f41ddd 100644 --- a/src/aipass/memory/README.md +++ b/src/aipass/memory/README.md @@ -2,7 +2,7 @@ # MEMORY -**Purpose:** Central memory archive with semantic search, rollover, and archival across all AIPass branches. +**Purpose:** Central memory archive — vector search, rollover, and memory management for all AIPass branches. **Module:** `aipass.memory` **Created:** 2026-03-07 **Last Updated:** 2026-04-22 @@ -12,53 +12,58 @@ ## Overview -Memory is the central memory archive system that: -- Provides semantic search across all branch memories -- Archives memories when branches hit rollover limits (600 lines) -- Extracts symbolic dimensions from conversations -- Manages template distribution and line-count tracking across branches +Memory is the archival backbone of AIPass. Every branch accumulates session history and learnings in `.trinity/` files. When those files reach capacity, Memory archives the oldest entries into ChromaDB vectors — searchable, permanent, never lost. + +What Memory does: +- **Rollover** — detects when `.trinity/local.json` or `observations.json` exceed limits, extracts oldest entries, embeds them via sentence-transformers, stores in ChromaDB, trims the source file +- **Search** — semantic search across all archived branch memories (4+ collections, 2200+ vectors) +- **Templates** — distributes `.trinity/` schema updates across all branches (push, diff, status) +- **Symbolic** — fragmented memory extraction from conversations (demo, analyze, extract, fragments, bootstrap, hook-test) +- **Verify** — checks whether a flow plan is vectorized in ChromaDB +- **Watch** — persistent file watcher that auto-triggers rollover on changes --- -## Commands / Usage +## Commands + +All commands via `drone @memory `: -**Via Drone (recommended):** ```bash +# Introspection +drone @memory # Module list, version +drone @memory --help # Full command reference +drone @memory --version # Version string + # Rollover -drone @memory rollover # Show rollover module introspection -drone @memory rollover run # Execute memory rollover for files over limits -drone @memory rollover status # Show rollover statistics for all branches -drone @memory rollover check # Dry run — check which files need rollover -drone @memory rollover sync-lines # Update line count metadata for all branches +drone @memory rollover # Module introspection (handlers + subcommands) +drone @memory rollover run # Execute rollover for files over limits +drone @memory rollover status # Show per-branch rollover statistics +drone @memory rollover check # Dry run — check what needs rollover +drone @memory rollover sync-lines # Update line count metadata for all branches # Search -drone @memory search "error handling" # Semantic search across all branch memories -drone @memory search "query" --branch SEEDGO # Filter search by branch -drone @memory search "query" --n 10 # Limit number of results +drone @memory search "error handling" # Semantic search across all branch memories +drone @memory search "query" --branch X # Filter search by branch +drone @memory search "query" --n 10 # Limit number of results -# Symbolic (fragmented memory) -drone @memory symbolic # Show symbolic module introspection -drone @memory symbolic demo # Run fragmented memory demonstration -drone @memory symbolic fragments "query" # Search symbolic fragments (not operational — no stored fragments) -drone @memory symbolic extract # Extract fragments via LLM (requires API) +# Symbolic +drone @memory symbolic # Module introspection (6 handlers, subcommands) +drone @memory symbolic demo # Run v1 + v2 mock analysis demonstration +drone @memory symbolic fragments "query" # Search stored symbolic fragments +drone @memory symbolic extract # Extract fragments via LLM (requires API key) +drone @memory symbolic bootstrap # Populate fragments from session JSONLs +drone @memory symbolic hook-test # Test hook with sample conversation text -# Templates (not operational — template files missing) -drone @memory templates # Show templates module introspection -drone @memory templates push-templates # Push template updates to all branches (not operational) -drone @memory templates diff-templates # Show template differences per branch (not operational) -drone @memory templates template-status # Show template version and push status (not operational) +# Templates +drone @memory templates push-templates # Push template updates to all branches +drone @memory templates diff-templates # Show template differences per branch +drone @memory templates template-status # Show template version and push status # Verify -drone @memory verify FPLAN-XXXX # Check if a plan is vectorized in ChromaDB +drone @memory verify FPLAN-XXXX # Check if plan is vectorized in ChromaDB # Watch -drone @memory watch # Start auto-rollover watcher (Ctrl+C to stop) -``` - -**Direct execution:** -```bash -python3 -m aipass.memory.apps.memory search "query" -python3 -m aipass.memory.apps.memory rollover status +drone @memory watch # Auto-rollover watcher daemon (Ctrl+C to stop) ``` --- @@ -67,88 +72,119 @@ python3 -m aipass.memory.apps.memory rollover status ``` memory/ -├── __init__.py # Package init -├── README.md # This file -├── DASHBOARD.local.json # System status dashboard -├── pytest.ini # Test configuration +├── .trinity/ # Identity & memory +│ ├── passport.json # Branch identity +│ ├── local.json # Session history (v2 schema, entry-count limits) +│ └── observations.json # Collaboration patterns (v1 schema, line-count limits) +├── .aipass/ # Branch prompt +├── .ai_mail.local/ # Mailbox ├── apps/ -│ ├── __init__.py -│ ├── memory.py # Entry point (CLI) — auto-discovers modules -│ ├── modules/ -│ │ ├── __init__.py -│ │ ├── rollover.py # Rollover orchestrator — line checks, archival triggers -│ │ ├── search.py # Search orchestrator — semantic query routing -│ │ └── verify.py # Plan verification — check vectorized plan status -│ ├── handlers/ -│ │ ├── __init__.py -│ │ ├── central_writer.py # Central memory write operations -│ │ ├── dashboard_push.py # Dashboard status push -│ │ ├── archive/ # Memory archival indexing -│ │ ├── json/ # JSON handler operations -│ │ ├── learnings/ # Learning extraction -│ │ ├── monitor/ # File watcher for auto-rollover -│ │ ├── rollover/ # Rollover implementation logic -│ │ ├── schema/ # Memory schema definitions -│ │ ├── search/ # Search implementation (vector/semantic) -│ │ ├── storage/ # Storage backend operations -│ │ ├── tracking/ # Line count and metadata tracking -│ │ ├── symbolic/ # Symbolic dimension extraction -│ │ ├── templates/ # Template management -│ │ └── vector/ # Vector DB (ChromaDB) operations -│ ├── extensions/ # Extension plugins -│ └── plugins/ # Plugin system -├── artifacts/ # Build/output artifacts -├── docs/ # Documentation -├── memory_json/ # Memory JSON data store -├── tests/ # Test suite -└── tools/ # Utility scripts +│ ├── memory.py # Entry point — auto-discovers modules via handle_command() +│ ├── modules/ # Business logic (5 modules) +│ │ ├── rollover.py # Rollover orchestration, status display, sync-lines +│ │ ├── search.py # Semantic query routing +│ │ ├── verify.py # Plan vectorization check +│ │ ├── symbolic.py # Fragmented memory extraction and search +│ │ └── templates.py # Template push, diff, status +│ └── handlers/ # Implementation (14 handler groups, 35 files) +│ ├── archive/ # indexer.py — memory archival indexing +│ ├── intake/ # plans_processor.py, pool_processor.py — ingest pipelines +│ ├── json/ # json_handler.py, memory_files.py — JSON operations +│ ├── learnings/ # manager.py — learning extraction and management +│ ├── monitor/ # detector.py, memory_watcher.py — rollover detection + auto-trigger +│ ├── rollover/ # extractor.py, orchestrator.py — backup → extract → embed → store +│ ├── schema/ # normalize.py — schema version normalization +│ ├── search/ # query_executor.py, vector_search.py — semantic search impl +│ ├── storage/ # chroma.py, chroma_subprocess.py — ChromaDB backend +│ ├── symbolic/ # 6 handlers — chroma_client, deduplicator, extractor, hook, retriever, storage +│ ├── templates/ # pusher.py, differ.py, spawn_pusher.py — template distribution +│ ├── tracking/ # line_counter.py — metadata line count tracking +│ ├── vector/ # embedder.py, embed_subprocess.py — sentence-transformer embeddings +│ ├── central_writer.py # Central memory write operations +│ └── dashboard_push.py # Dashboard status push +├── config/ # memory_bank.config.json — per-branch rollover limits +├── templates/ # LOCAL.template.json, OBS.template — schema templates +├── tests/ # 450 tests (16/16 module coverage) +├── .chroma/ # Global ChromaDB vector store +└── memory_json/ # Operation log files (auto-created) ``` +### Rollover Pipeline + +``` +startup trigger → check_and_rollover() + → detector.check_all_branches() # scan AIPASS_REGISTRY.json + → _should_rollover(file) # v1: line_count >= max_lines + # v2: len(sessions) >= max_sessions, etc. + → orchestrator.execute_rollover() + → create_rollover_backup() # safety copy to branch/.backup/ + → extract_items() # v2: max(excess, 1) oldest entries + → embed via subprocess # sentence-transformers in memory .venv + → store in ChromaDB # global + local collections + → trim source file # write back with oldest removed +``` + +### Dual Schema Support + +- **v1** (line-count): `schema_version: "1.0.0"` — triggers at `current_lines >= max_lines` (default 600) +- **v2** (entry-count): `schema_version: "2.0.0"` — triggers at `len(sessions) >= max_sessions` or `len(key_learnings) >= max_key_learnings`. Extractor uses `max(excess, 1)` guard to prevent Python's `list[-0:]` trap. + +### Subprocess Isolation + +All ML operations (torch, sentence-transformers, chromadb) run via subprocess. The main process never imports these heavy libraries. Each embedding call spawns `memory/.venv/bin/python3` with a self-contained script that reads stdin JSON and writes stdout JSON. + --- -## Integration Points +## Dependencies -### Depends On -- `rich` — Console output, panels, and tables -- Python stdlib (`sys`, `time`, `signal`, `logging`, `pathlib`, `importlib`) +### Runtime +- `rich` — console output, panels, tables +- Python stdlib (`json`, `pathlib`, `logging`, `importlib`, `signal`) +- `prax` (internal) — logging via `get_system_logger()` + +### ML (in memory `.venv/` only) +- `torch` + `sentence-transformers` — embedding generation +- `chromadb` — vector storage and semantic search +- `numpy` — numerical operations ### Provides To -- All branches — memory rollover, archival, and retrieval services -- All branches — semantic search across branch memories -- All branches — template distribution via `push-templates` -- All branches — line count metadata via `sync-lines` +- All branches — memory rollover and archival when `.trinity/` files hit limits +- All branches — semantic search across archived memories +- All branches — template schema distribution +- All branches — line count metadata sync --- -## Key Modules +## Quality -### rollover -The rollover module monitors memory files across all branches registered in `AIPASS_REGISTRY.json`. When files exceed entry-count limits (20 sessions, 25 key_learnings), it triggers archival — extracting oldest entries, embedding them via subprocess, and storing in ChromaDB vectors. The `watch` command runs a persistent file watcher that auto-triggers rollover on changes. +- **Seedgo:** 100% (33/33 standards) — maintained since s12 +- **Tests:** 450 pass, 1 skip (test_vector.py gated with `importorskip` for numpy) +- **Coverage:** 175 public functions, 102 tested (58%), 16/16 modules +- **Type checking:** 35 files, 0 errors -### search -Semantic search across all branch memories using ChromaDB + sentence-transformers. Requires memory `.venv/` with torch installed (~3GB). All ML operations run via subprocess isolation — main process never imports torch. +--- -### symbolic *(partial)* -Fragmented memory extraction and search. Demo and introspection work. `fragments` search returns 0 results (no stored fragments). `extract` requires API key. Code is operational but no fragment data has been stored yet. +## Known Issues -### templates *(not operational)* -Living template push system for distributing `.trinity/` schema updates across branches. Template files (`LOCAL.template.json`, `OBS.template`) are missing — all subcommands (`push-templates`, `diff-templates`, `template-status`) fail. - -### verify -Checks whether a specific flow plan (FPLAN-XXXX) has been vectorized into ChromaDB. Works correctly. +- `search` requires torch/sentence-transformers in memory `.venv/` — fails without them +- memory_watcher.py at 704 lines (near 700 threshold, bypassed in seedgo) +- symbolic.py at 1604 lines (legacy port, bypassed) +- manager.py at 1076 lines (complex learning extraction, bypassed) +- `memory_threshold_exceeded` trigger event registered but never fired — rollover auto-runs at startup via watcher +- `rollover status` shows 0 branches when `AIPASS_REGISTRY.json` path not resolved --- ## Identity - **Passport:** `.trinity/passport.json` -- **Session History:** `.trinity/local.json` -- **Observations:** `.trinity/observations.json` -- **Branch Prompt:** `.aipass/branch_system_prompt.md` +- **Session History:** `.trinity/local.json` (v2 schema, 20 sessions max, 25 key_learnings max) +- **Observations:** `.trinity/observations.json` (v1 schema, 600 lines max) +- **Branch Prompt:** `.aipass/aipass_local_prompt.md` --- -*Last Updated: 2026-04-07* +*Last Updated: 2026-04-22* --- [← Back to AIPass](../../../README.md) diff --git a/src/aipass/prax/CLOSED_PLANS.local.json b/src/aipass/prax/CLOSED_PLANS.local.json index b99b3e78..88e1417c 100644 --- a/src/aipass/prax/CLOSED_PLANS.local.json +++ b/src/aipass/prax/CLOSED_PLANS.local.json @@ -63,5 +63,28 @@ "date_closed": "2026-04-10", "location": "prax" } - ] + ], + "document_metadata": { + "version": "2.0.0", + "schema_version": "2.0.0", + "document_type": "session_history", + "tags": [ + "session_tracking", + "work_log", + "PRAX" + ], + "limits": { + "max_sessions": 20, + "max_key_learnings": 25, + "session_summary_max_chars": 150, + "learning_value_max_chars": 200, + "note": "Auto-rollover to @memory when limits exceeded. Oldest entries trimmed first." + }, + "status": { + "health": "healthy", + "last_health_check": "2026-04-22", + "current_lines": 0 + } + }, + "key_learnings": {} } diff --git a/src/aipass/prax/README.md b/src/aipass/prax/README.md index f2637208..01f7cfc0 100644 --- a/src/aipass/prax/README.md +++ b/src/aipass/prax/README.md @@ -2,31 +2,20 @@ # PRAX -**Purpose:** System-wide logging, real-time monitoring, and dashboard for AIPass. +**Purpose:** System-wide logging, real-time monitoring, and dashboard infrastructure for AIPass. **Module:** `aipass.prax` +**Version:** 2.0.0 **Last Updated:** 2026-04-22 --- ## Overview -Prax auto-routes log output from any module to per-module log files and provides a live monitoring console (Mission Control) that shows file changes, log events, and command execution across all branches. Monitors Claude Code, Codex, and Gemini CLI sessions. +Prax is the logging and monitoring backbone of the AIPass ecosystem. Any branch imports `logger` and gets automatic log routing — prax detects the caller via stack introspection and writes to the correct per-module log file. No configuration needed. -## Commands +On top of logging, prax provides Mission Control (a real-time terminal console for file changes, log events, and agent activity), a log audit system, a dashboard infrastructure, and cross-branch STATUS.md synchronization. -```bash -drone @prax monitor run # Launch Mission Control -drone @prax status # System health status -drone @prax log-audit audit # Audit log file sizes -drone @prax dashboard # Show dashboard -drone @prax --help # Full help -``` - -## Usage - -### Logging - -**Canonical import (use this):** +## Quick Start ```python from aipass.prax import logger @@ -36,9 +25,79 @@ logger.warning("Disk usage high") logger.error("Connection failed") ``` -This is Pattern A — the recommended way for all branches. Logs auto-route via two-tier placement: `system_logs/_.log` (central aggregation) and `/logs/` (branch-local). No configuration needed — prax detects the caller via stack introspection. +Logs auto-route via two-tier placement: +- `system_logs/_.log` — central aggregation at the repo root +- `/logs/.log` — branch-local debugging -For prax handlers that need to bypass the event pipeline (watchdog threads, import-chain files): +## Commands + +```bash +drone @prax # Show discovered modules +drone @prax --help # Full command list +drone @prax --version # Version string +``` + +### Monitor — Mission Control + +```bash +drone @prax monitor # Show monitor architecture +drone @prax monitor run # Launch Mission Control (all branches) +drone @prax monitor run seedgo,cli # Monitor specific branches +drone @prax monitor --help # Monitor usage +``` + +Real-time unified console showing: +- File changes, log events, drone commands, agent activity +- **Caller attribution** — `CALLER → TARGET` for drone commands +- **Model tags** — `[BRANCH/model]` (e.g., `[DEVPULSE/opus]`, `[DEVPULSE/gpt-5.4]`) +- **Multi-CLI** — Claude Code (JSONL), Codex (JSONL), Gemini (JSON) session monitoring +- **Polling fallback** — automatic fallback when inotify watches are exhausted +- **Soft start** — only shows new activity after launch (seeks to EOF on startup) + +Interactive commands inside the monitor: `help`, `status`, `quit`/`exit`. + +### Status + +```bash +drone @prax status # System health (modules, loggers, watcher state) +drone @prax status sync # Build STATUS.md from all branch STATUS.local.md +drone @prax status --help # Status usage +``` + +### Log Audit + +```bash +drone @prax log-audit # Show audit module info +drone @prax log-audit audit # Scan system_logs/ for health + oversized files +drone @prax log-audit enforce # Truncate oversized logs to 1000 lines +drone @prax log-audit --help # Audit usage +``` + +### Dashboard + +```bash +drone @prax dashboard # Show dashboard sections +drone @prax dashboard refresh --all # Refresh all branch dashboards from centrals +drone @prax dashboard refresh @flow # Refresh a specific branch +drone @prax dashboard status # Show dashboard status +drone @prax dashboard push-template # Push template to all branches +drone @prax dashboard diff-template # Diff template vs branch dashboards +drone @prax dashboard --help # Dashboard usage +``` + +## Logging API + +### Pattern A — Canonical (use this) + +```python +from aipass.prax import logger + +logger.info("Processing started") +``` + +This works from any branch. Prax detects the caller via stack introspection and routes to the correct log file. If prax fails to import, a NullLogger fallback prevents crashes. + +### Pattern B — Direct Logger (for prax internals) ```python from aipass.prax.apps.modules.logger import get_direct_logger @@ -47,94 +106,112 @@ logger = get_direct_logger() logger.info("Direct log entry") ``` -### Mission Control +Use this in prax handler files that run in watchdog threads or sit in the import chain. Resolves module/branch at creation time, bypassing the runtime event pipeline. -Real-time monitoring console for watching system activity across all branches and CLI tools. +### Programmatic Dashboard API -```bash -drone @prax monitor run +```python +from aipass.prax.apps.modules.dashboard import write_section + +write_section(branch_path, "ai_mail", {"new": 3, "total": 5}) ``` -Features: -- File changes, log events, drone commands, agent activity — all in one console -- **Caller attribution** — shows `CALLER → TARGET` for drone commands -- **Model tags** — shows `[BRANCH/model]` (e.g., `[DEVPULSE/opus]`, `[DEVPULSE/gpt-5.4]`) -- **Multi-CLI** — monitors Claude Code (JSONL), Codex (JSONL), Gemini (JSON) sessions -- **Polling fallback** — when inotify is exhausted, falls back to PollingObserver automatically -- **Interactive filtering** *(not operational)* — `watch`, `filter` commands are deferred - -Interactive commands inside the monitor: - -``` -help # Show available commands -status # Display current monitoring state -quit/exit # Stop monitoring -``` - -### CLI Commands - -| Command | Description | -|---------|-------------| -| `drone @prax monitor run` | Launch Mission Control | -| `drone @prax monitor` | Show monitor introspection | -| `drone @prax status` | Show system status (modules, loggers, watcher state) | -| `drone @prax status sync` | Sync STATUS.md from all branch STATUS.local.md | -| `drone @prax log-audit audit` | Audit log file sizes and health | -| `drone @prax log-audit enforce` | Truncate oversized logs | -| `drone @prax dashboard` | Show system dashboard | -| `drone @prax dashboard refresh --all` | Refresh dashboard data from centrals | - ## Architecture ``` prax/ +├── __init__.py # Public API: exports `logger` (NullLogger fallback) ├── apps/ -│ ├── prax.py # Entry point (CLI) -│ ├── modules/ -│ │ ├── logger.py # SystemLogger (public API) -│ │ ├── monitor.py # Mission Control -│ │ ├── dashboard.py # System dashboard -│ │ ├── status.py # System status / STATUS sync -│ │ └── log_audit.py # Log file audit -│ └── handlers/ -│ ├── central/ # Central file reader -│ ├── config/ # Configuration loading -│ ├── dashboard/ # Dashboard refresh and operations -│ ├── discovery/ # Module scanning and filtering -│ ├── json/ # JSON operations handler -│ ├── json_templates/ # JSON template definitions -│ ├── logging/ # Log setup, rotation, introspection -│ ├── monitoring/ # Event queue, branch detection, stream output -│ ├── registry/ # Module registry management -│ ├── status/ # STATUS sync handler -│ └── watcher/ # File and log watchers -├── templates/ # Dashboard templates -├── tests/ # Test suite (375 tests) -└── tools/ # Standalone utilities (inbox_watchdog.py) +│ ├── prax.py # Entry point — auto-discovers modules, routes commands +│ ├── modules/ # Business logic (5 command modules) +│ │ ├── logger.py # SystemLogger — auto-routing, two-tier logging +│ │ ├── monitor.py # Mission Control — 3-thread real-time monitoring +│ │ ├── dashboard.py # Dashboard — template management, refresh, write-through +│ │ ├── status.py # System status — health display, STATUS.md sync +│ │ └── log_audit.py # Log audit — scan, health summary, enforce limits +│ └── handlers/ # Implementation details (11 handler directories) +│ ├── central/ # Central file reader (.ai_central/*.central.json) +│ ├── config/ # Path resolution, log config, ignore patterns +│ ├── dashboard/ # Refresh, operations, template push/diff, agent status +│ ├── discovery/ # Module scanning, filtering, file watcher for new .py +│ ├── json/ # Auto-creating JSON handler (config/data/log per module) +│ ├── json_templates/ # Default JSON templates for auto-creation +│ ├── logging/ # Setup, rotation, introspection, override, direct logger +│ ├── monitoring/ # Event queue, branch detector, stream output, log watcher +│ ├── registry/ # Module registry load/save +│ ├── status/ # STATUS.md sync handler +│ └── watcher/ # Background system watchers +├── prax_json/ # Auto-created per-module config/data/log files +├── templates/ # Dashboard template schema (DASHBOARD.template.json) +├── tests/ # 375 tests across 16 files +└── tools/ # Standalone utilities (inbox_watchdog.py, verify_branch.py) +``` + +### Design Pattern + +The entry point (`prax.py`) has zero business logic — it auto-discovers modules in `apps/modules/` and routes commands. Each module is a thin orchestrator over its handlers. Handlers are never imported by external branches. + +### Command Routing + +``` +drone @prax monitor run + → prax.py discovers modules (glob apps/modules/*.py) + → calls monitor.handle_command("monitor", ["run"]) + → monitor.py delegates to handlers/monitoring/* ``` ## How It Works -1. **Auto-routing** — When any module calls `logger.info()`, prax inspects the call stack to identify the caller and routes the log entry to the appropriate file. -2. **Two-tier logging** — Each log entry goes to both `system_logs/` (central aggregation) and `/logs/` (branch-local), both with rotation. -3. **Mission Control** — A multi-threaded monitoring console (display, file watcher, log watcher). Falls back to polling when inotify is exhausted. Shows caller attribution and model tags. -4. **Multi-CLI monitoring** — Watches Claude Code JSONL, Codex JSONL, and Gemini JSON session files for agent activity (thinking, tool use, responses). -5. **Dashboard** — Aggregates data from central files and branch status into per-branch dashboard views. +1. **Auto-routing** — `logger.info()` inspects the call stack to identify the caller's module, branch, and file path, then routes the log entry to the correct per-module log file. +2. **Two-tier logging** — Each log entry goes to both `system_logs/` (central, all branches) and `/logs/` (branch-local), both with size-based rotation. +3. **Self-healing** — Auto-creates missing log directories, falls back to `system_logs/external/` for unknown modules, provides NullLogger if prax itself fails to import. +4. **Mission Control** — Three threads: display worker (pulls from event queue), file watcher (watchdog on branch `apps/` dirs), log watcher (tails `system_logs/*.log`). Falls back to polling when inotify is exhausted. +5. **Multi-CLI monitoring** — Watches Claude Code JSONL, Codex JSONL, and Gemini JSON session files. Extracts agent activity (thinking, tool use, responses) with model detection and branch resolution. +6. **Dashboard** — Template-based per-branch dashboard files. Refreshes from central files (`*.central.json`). Write-through API for services to update sections directly. +7. **STATUS sync** — Scans all branch `STATUS.local.md` files, extracts State/Last update fields, builds aggregated `STATUS.md` at the repo root. ---- +## Tests + +375 tests across 16 files, covering all major components: + +| Test File | Tests | Coverage | +|-----------|-------|----------| +| test_logger_module.py | 40 | Logger init, routing, lifecycle | +| test_monitoring_filters.py | 39 | Event filtering rules | +| test_config.py | 38 | Config loading, path resolution | +| test_event_queue.py | 35 | Thread-safe event buffering | +| test_log_watcher.py | 35 | Log file tailing | +| test_logging.py | 33 | Core logging system | +| test_discovery.py | 25 | Module scanning | +| test_operations.py | 24 | Dashboard operations | +| test_watcher.py | 23 | File watcher behavior | +| test_registry.py | 22 | Module registry | +| test_json_handler.py | 18 | JSON auto-creation | +| test_central.py | 14 | Central reader | +| test_monitor_module.py | 11 | Monitor commands | +| test_log_audit.py | 10 | Log audit | +| test_status.py | 8 | Status commands | + +90/136 public functions tested (66%). ## Integration Points ### Depends On - `aipass.cli` — Console output, headers, success/error formatting - `aipass.drone` — Caller attribution via `[CALLER:BRANCH]` log markers -- Python stdlib (`pathlib`, `importlib`, `argparse`, `logging`) +- `aipass.trigger` — Optional event firing (module_discovered, error_detected) - `watchdog` — File system monitoring (inotify + polling fallback) +- Python stdlib (`pathlib`, `logging`, `threading`, `argparse`, `importlib`) ### Provides To -- All 11 branches — Unified logging via `from aipass.prax import logger` +- All branches — Unified logging via `from aipass.prax import logger` - All branches — Real-time monitoring via Mission Control -- System — STATUS.md sync, dashboard infrastructure +- All branches — Per-branch dashboard files +- System — `STATUS.md` sync, log audit enforcement + +## Known Issues +- **inotify exhaustion** — System often near `max_user_watches` limit. Monitor uses polling fallback (functional but slower). +- **Interactive filtering deferred** — `watch`/`filter` commands in Mission Control are not operational. --- From f42039dae78198aa10e359d30f92f9c4e3be7d94 Mon Sep 17 00:00:00 2001 From: AIOSAI Date: Wed, 22 Apr 2026 21:23:02 -0700 Subject: [PATCH 5/8] =?UTF-8?q?docs(api):=20comprehensive=20README=20updat?= =?UTF-8?q?e=20=E2=80=94=20current=20state,=20architecture,=20contracts?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-Authored-By: Claude Opus 4.6 --- src/aipass/api/README.md | 196 +++++++++++++++++++++++++-------------- 1 file changed, 126 insertions(+), 70 deletions(-) diff --git a/src/aipass/api/README.md b/src/aipass/api/README.md index 6b90cdd0..79c1f34a 100644 --- a/src/aipass/api/README.md +++ b/src/aipass/api/README.md @@ -2,19 +2,24 @@ # API -**Purpose:** Centralized external API gateway — authenticated service clients for all external APIs (OpenRouter, Google, future providers). -**Module:** `aipass.api` +> Centralized external API gateway — authenticated service clients for all external APIs + +**Module:** `aipass.api` | **Role:** `api_gateway` | **Version:** 1.0.0 +**Seedgo:** 100% (34/34) | **Tests:** 306 pass | **Functions:** 74 public (73 tested) **Last Updated:** 2026-04-22 --- ## Overview +API is the centralized external service gateway for AIPass. It provides authenticated clients for external APIs (OpenRouter, Google, future providers). Consumers import ready-to-use service objects — API owns the plumbing, consumers own the business logic. + ### What I Do -- Provide authenticated service clients for external APIs (Google Drive, OpenRouter, etc.) +- Provide authenticated service clients for external APIs (OpenRouter, Google Drive, Calendar, etc.) - Manage OAuth2 flows, credential storage, and token refresh - Offer thread-safe service factories for concurrent consumers - Handle API key management and validation across providers +- Host the generic contract registry for private integration drivers (DPLAN-0133) - Provide SSL retry and connection resilience utilities ### What I Don't Do @@ -22,38 +27,44 @@ - Set default models or configs — consumers provide their own - Manage application workflows, polling loops, or orchestration -### How I Work -- **Entry Point:** `apps/api.py` -- auto-discovers and routes to modules -- **Pattern:** Standard AIPass 3-tier architecture (entry point / modules / handlers) -- **Design principle:** If it's not auth, credentials, or service factory — it doesn't belong here +### Design Principle +If it's not auth, credentials, or service factory — it doesn't belong here. --- -## Commands / Usage +## Commands ```bash -drone @api get-key # Retrieve API key for provider -drone @api validate # Validate API credentials and connection -drone @api validate google # Validate Google OAuth2 credentials -drone @api reauth google # Re-authenticate Google OAuth2 -drone @api test # Test OpenRouter connection status -drone @api models [--all] # List available models from provider -drone @api status # Check OpenRouter client status -drone @api call "prompt" --model MODEL # Make API call to model -drone @api list-providers # List available API providers -drone @api init # Initialize .env template -drone @api track # Track API usage metrics -drone @api stats # Display API usage statistics -drone @api session # Show session usage data -drone @api caller-usage # Show usage by caller module -drone @api cleanup [days] # Clean up old usage data *(not operational — fails with no data)* -drone @api integrations list # List registered integration contracts -drone @api integrations call [args...] # Call a registered contract -drone @api --help # Full help output -drone @api --version # Show version -``` +# Key Management +drone @api get-key [provider] # Retrieve API key (default: openrouter) +drone @api validate [provider] # Validate API key (default: openrouter) +drone @api validate google # Validate Google OAuth2 credentials +drone @api reauth google # Re-authenticate Google OAuth2 +drone @api list-providers # List available API providers +drone @api init # Initialize .env template at ~/.secrets/aipass/ -Running `drone @api` with no arguments displays module introspection (discovered modules and status). +# OpenRouter +drone @api test # Test OpenRouter connection status +drone @api models [--all] # List available models (default: top 10) +drone @api status # Check OpenRouter client status (key, SDK, cache) +drone @api call "prompt" --model MODEL # Make API call to model + +# Usage Tracking +drone @api track # Track API usage for a generation +drone @api stats # Display overall usage statistics +drone @api session # Show current session usage +drone @api caller-usage # Show usage by caller module +drone @api cleanup [days] # Clean up data older than N days (default: 30) + +# Integration Contracts +drone @api integrations list # List registered contracts +drone @api integrations call [args...] # Call a registered contract + +# Meta +drone @api --help # Full help output +drone @api --version # Show version (v1.0.0) +drone @api # Module introspection (7 discovered modules) +``` --- @@ -81,62 +92,107 @@ from aipass.api.apps.modules.google_client import api_call_with_retry ## Architecture +Three-tier design: entry point routes to modules (orchestration), which delegate to handlers (business logic). Modules are auto-discovered from `apps/modules/*.py` — each implements `handle_command(command, args) -> bool`. + ``` api/ -├── __init__.py # Public API exports +├── __init__.py ├── apps/ -│ ├── api.py # Entry point (module discovery, command routing) -│ ├── modules/ -│ │ ├── api_key.py # Key retrieval and validation logic -│ │ ├── openrouter_client.py # OpenRouter API client -│ │ ├── google_client.py # Google API services (Drive, Calendar, etc.) -│ │ ├── usage_tracker.py # Usage metrics tracking -│ │ ├── bridge.py # Generic contract registry (register/resolve) -│ │ ├── integrations_manager.py # Integration contract orchestration -│ │ └── registry.py # Driver auto-discovery (load_drivers) -│ └── handlers/ -│ ├── auth/ -│ │ ├── env.py # Environment variable credential loading -│ │ └── keys.py # API key storage and retrieval -│ ├── config/ -│ │ └── provider.py # Provider configuration management -│ ├── google/ -│ │ ├── auth.py # OAuth2 lifecycle, credential I/O -│ │ ├── service_factory.py # Service object factory (single + thread-safe) -│ │ └── retry.py # SSL retry with exponential backoff -│ ├── json/ -│ │ └── json_handler.py # JSON operation logging -│ ├── openrouter/ -│ │ ├── caller.py # HTTP request execution -│ │ ├── client.py # OpenRouter client implementation -│ │ ├── models.py # Model discovery and listing -│ │ └── provision.py # Provider provisioning -│ ├── integrations/ -│ │ ├── list.py # List registered contracts handler -│ │ └── call.py # Call registered contract handler -│ └── usage/ -│ ├── aggregation.py # Usage data aggregation -│ ├── cleanup.py # Usage data cleanup -│ └── tracking.py # Usage event tracking -└── README.md +│ ├── api.py # Entry point — module discovery, command routing +│ ├── modules/ # Tier 2: orchestration layer (7 modules) +│ │ ├── api_key.py # Key retrieval, validation, provider listing +│ │ ├── openrouter_client.py # OpenRouter client — calls, models, status +│ │ ├── google_client.py # Google API services (Drive, Calendar, etc.) +│ │ ├── usage_tracker.py # Usage metrics — track, stats, cleanup +│ │ ├── bridge.py # Generic contract registry (register/resolve) +│ │ ├── integrations_manager.py # Contract dispatch — integrations list/call +│ │ └── registry.py # Driver auto-discovery (load_drivers) +│ ├── handlers/ # Tier 3: business logic (7 packages, 15 files) +│ │ ├── auth/ +│ │ │ ├── env.py # .env template creation (0o600 permissions) +│ │ │ └── keys.py # Key storage, retrieval, validation rules +│ │ ├── config/ +│ │ │ └── provider.py # Provider config deep-merge, validation rules +│ │ ├── google/ +│ │ │ ├── auth.py # OAuth2 lifecycle — load, refresh, save credentials +│ │ │ ├── service_factory.py # Service object factory (single + thread-safe) +│ │ │ └── retry.py # Exponential backoff with SSL error detection +│ │ ├── integrations/ +│ │ │ ├── list.py # List registered contracts +│ │ │ └── call.py # Invoke contract driver functions +│ │ ├── json/ +│ │ │ └── json_handler.py # JSON persistence — 3-file pattern per operation +│ │ ├── openrouter/ +│ │ │ ├── caller.py # Stack-based caller detection +│ │ │ ├── client.py # OpenRouter client (OpenAI SDK wrapper) +│ │ │ ├── models.py # Model discovery and listing +│ │ │ └── provision.py # Per-caller config auto-provisioning +│ │ └── usage/ +│ │ ├── aggregation.py # Stats rollup — per-caller, daily, monthly +│ │ ├── cleanup.py # Data retention and cleanup +│ │ └── tracking.py # Generation metrics from OpenRouter API +│ └── integrations/ # Private driver space (gitignored) +│ └── {project}/driver.py # Each driver registers contracts via bridge +├── tests/ # 306 tests across 15 files +│ ├── test_api_key.py # Key management (39 tests) +│ ├── test_caller.py # Caller detection (9 tests) +│ ├── test_cli_routing.py # Command routing (9 tests) +│ ├── test_config_provider.py # Config merging (16 tests) +│ ├── test_contracts.py # JSON contract operations (11 tests) +│ ├── test_critical_paths.py # End-to-end critical flows (19 tests) +│ ├── test_error_resilience.py # Error handling (4 tests) +│ ├── test_google_client.py # Google OAuth2 + services (46 tests) +│ ├── test_init_provisioning.py # Provisioning init (4 tests) +│ ├── test_integrations.py # Bridge, registry, contracts (17 tests) +│ ├── test_json_handler.py # JSON persistence (41 tests) +│ ├── test_openrouter_client.py # OpenRouter client (37 tests) +│ ├── test_provision.py # Auto-provisioning (20 tests) +│ ├── test_tracking.py # Usage tracking (13 tests) +│ └── test_usage_tracker.py # Usage module (21 tests) +└── docs/ + └── SECURITY.md # Key handling and leak prevention ``` --- +## Integration Contract System + +Private integration drivers live in `apps/integrations/{project}/driver.py` (gitignored). Each driver registers named contracts via `bridge.register()`. Callers resolve contracts by name — never referencing private projects directly. + +**Three-layer design (DPLAN-0133):** +1. **Drivers** (`apps/integrations/*/driver.py`) — private, gitignored, register contracts on load +2. **Bridge** (`bridge.py`) — public contract registry: `register()`, `resolve()`, `list_contracts()` +3. **Handlers** (`integrations/list.py`, `call.py`) — public dispatch for `drone @api integrations` + +**Auto-discovery:** `registry.py` walks `apps/integrations/*/driver.py` via `importlib.util.spec_from_file_location`. Handles empty dirs, missing files, and import errors gracefully. + +--- + ## Integration Points ### Depends On -- `aipass.prax` -- structured logging via `system_logger` -- `aipass.cli` -- Rich console output formatting +- `aipass.prax` — structured logging via `system_logger` +- `aipass.cli` — Rich console output formatting ### Provides To -- All branches -- authenticated external API clients +- All branches — authenticated external API clients via `get_response()`, `get_drive_service()`, `get_google_service()` - System-wide API key management and credential validation ### Credentials -- `~/.secrets/aipass/.env` -- API keys (OpenRouter, etc.) -- `~/.secrets/aipass/google_creds.json` -- Google OAuth2 tokens -- `~/.secrets/aipass/google_client_secret.json` -- Google OAuth app config +All credentials live at `~/.secrets/aipass/` (0o700 directory, 0o600 files): +- `.env` — API keys (OpenRouter, etc.) +- `google_creds.json` — Google OAuth2 tokens +- `google_client_secret.json` — Google OAuth app config + +### Provider Pattern +One module per provider (`openrouter_client.py`, `google_client.py`), one handler directory per provider (`openrouter/`, `google/`). Module orchestrates and presents CLI; handlers implement business logic. Pattern scales to future providers. + +--- + +## Known Issues +- Google auth libraries are optional deps — commands fail with install instructions if missing +- 1/74 public functions untested per seedgo test_map +- Backup branch credential migration pending (`~/.aipass/` to `~/.secrets/aipass/`) --- From 8ab2c471b84ee72c4b45d518e823ace3906bc5b9 Mon Sep 17 00:00:00 2001 From: AIOSAI Date: Wed, 22 Apr 2026 21:23:19 -0700 Subject: [PATCH 6/8] feat(trigger): docs(trigger): comprehensive README update Co-Authored-By: @trigger --- src/aipass/trigger/README.md | 203 ++++++++++++++++++++++++----------- 1 file changed, 139 insertions(+), 64 deletions(-) diff --git a/src/aipass/trigger/README.md b/src/aipass/trigger/README.md index 5b270e9a..b16d0ad2 100644 --- a/src/aipass/trigger/README.md +++ b/src/aipass/trigger/README.md @@ -2,37 +2,46 @@ # Trigger -**Purpose:** Event bus for AIPass. Branches fire events, registered handlers react. Decouples producers from consumers — the module that detects a condition doesn't need to know what should happen next. +**Purpose:** Event bus and error dispatch for AIPass. Branches fire events, registered handlers react. Medic watches logs for errors, fingerprints them, gates dispatch through an 8-stage pipeline, and notifies the responsible branch. **Module:** `aipass.trigger` +**Version:** 2.2.0 **Last Updated:** 2026-04-22 -## Commands / Usage +## Commands ```bash -drone @trigger fire [data] # Fire an event -drone @trigger list # List registered events -drone @trigger status # Event bus status -drone @trigger --help # Full help +drone @trigger # Introspection (modules, version) +drone @trigger --help # Full command listing +drone @trigger --version # Version string + +# Event bus +drone @trigger fire [key=val ...] # Fire an event with optional data +drone @trigger list # List all registered events + handlers +drone @trigger status # Event bus and medic state + +# Error registry +drone @trigger errors list # View tracked errors +drone @trigger errors stats # Registry stats + circuit breaker +drone @trigger errors circuit-breaker # Circuit breaker state +drone @trigger errors detail # Single error detail +drone @trigger errors --help # Error subcommand help + +# Medic (error dispatch control) +drone @trigger medic on # Enable auto-dispatch +drone @trigger medic off # Disable auto-dispatch +drone @trigger medic status # Medic state + suppression stats +drone @trigger medic mute @branch # Suppress dispatch to a branch +drone @trigger medic unmute @branch # Resume dispatch to a branch +drone @trigger medic --help # Medic subcommand help + +# Log watchers +drone @trigger branch_log_events status # Branch log watcher state +drone @trigger branch_log_events --help # Branch watcher help +drone @trigger log_events status # System log watcher state +drone @trigger log_events --help # System watcher help ``` -## Usage - -### CLI (via drone) - -```bash -drone @trigger --help # Show available commands -drone @trigger medic on # Enable error auto-dispatch -drone @trigger medic off # Disable error auto-dispatch -drone @trigger medic status # Show medic state -drone @trigger medic mute @branch # Suppress errors from a branch -drone @trigger medic unmute @branch # Resume errors from a branch -drone @trigger errors list # List tracked errors -drone @trigger errors stats # Error registry statistics -drone @trigger errors circuit-breaker # Circuit breaker state -drone @trigger --version # Show version -``` - -### Python API +## Python API ```python from aipass.trigger.apps.modules.core import Trigger @@ -41,20 +50,19 @@ from aipass.trigger.apps.modules.core import Trigger Trigger.fire("plan_file_created", path="/path/to/FPLAN-0042.md") # Register a handler -def on_plan_file_created(**data): +def on_plan_created(**data): print(f"Plan created at {data['path']}") -Trigger.on("plan_file_created", on_plan_file_created) +Trigger.on("plan_file_created", on_plan_created) # Remove a handler -Trigger.off("plan_file_created", on_plan_file_created) +Trigger.off("plan_file_created", on_plan_created) ``` -### Cross-branch error reporting - ```python from aipass.trigger.apps.modules.errors import report_error +# Cross-branch error reporting result = report_error( branch="api", error_type="ConnectionError", @@ -66,66 +74,133 @@ result = report_error( ## Events -14 events registered via `handlers/events/registry.py`. All fire through `Trigger.fire()`. +14 events registered via `handlers/events/registry.py` on first `Trigger.fire()`. All fire through the event bus. -| Event | Handler | Fired when | Action | -|-------|---------|------------|--------| -| `startup` | `startup.py` | Branch session starts | Error catch-up scan, memory rollover check | -| `error_detected` | `error_detected.py` | Error registered in log watcher (Medic v2) | 8-gate dispatch pipeline, sends fix-it email to affected branch | -| `error_logged` | `error_logged.py` | Error detected in system logs (legacy) | Rate-limited notification with medic gating | -| `warning_logged` | `warning_logged.py` | Warning detected in system logs | Logged for monitoring, no dispatch | -| `plan_file_created` | `plan_file.py` | New PLAN file detected in filesystem | Updates Flow's PLAN_REGISTRY.json | -| `plan_file_deleted` | `plan_file.py` | PLAN file removed from filesystem | Marks plan as deleted in registry | -| `plan_file_moved` | `plan_file.py` | PLAN file moved or renamed | Updates registry location | -| `bulletin_created` | `bulletin_created.py` | New system bulletin posted | Propagates to all branch dashboards | -| `memory_threshold_exceeded` | `memory_threshold_exceeded.py` | Memory file approaches line limit (600 lines) | Sends compression notification to branch | -| `memory_template_updated` | `memory_template_updated.py` | Memory template modified | Pushes template updates to branches | -| `memory_saved` | `memory.py` | Memory file saved | Placeholder for future rollover trigger | +| Event | Handler | Trigger | Action | +|-------|---------|---------|--------| +| `startup` | `startup.py` | Branch session starts | Error catch-up scan across log files, memory rollover check | +| `error_detected` | `error_detected.py` | Error registered via log watcher or `report_error()` | Full 8-gate Medic dispatch — emails fix-it to affected branch + `wake_branch()` | +| `error_logged` | `error_logged.py` | System log error (fallback path) | Monitor-only: logs the event, no dispatch | +| `warning_logged` | `warning_logged.py` | Warning in system logs | Logged for monitoring, no dispatch | +| `plan_file_created` | `plan_file.py` | New PLAN file detected | Updates Flow's PLAN_REGISTRY.json | +| `plan_file_deleted` | `plan_file.py` | PLAN file removed | Marks plan as deleted in registry | +| `plan_file_moved` | `plan_file.py` | PLAN file relocated | Updates registry location | +| `bulletin_created` | `bulletin_created.py` | New system bulletin posted | Propagates to branch dashboards | +| `memory_threshold_exceeded` | `memory_threshold_exceeded.py` | Memory file near limit (600 lines) | Emails compression notification to branch | +| `memory_template_updated` | `memory_template_updated.py` | Memory template changed | Pushes template updates to branches | +| `memory_saved` | `memory.py` | Memory file written | Placeholder for future rollover trigger | | `cli_header_displayed` | `cli.py` | CLI displays headers | Registration hook | | `pr_created` | `pr_status_sync.py` | PR opened on GitHub | Runs `drone @prax status sync` (fire-and-forget) | | `pr_merged` | `pr_status_sync.py` | PR merged on GitHub | Runs `drone @prax status sync` (fire-and-forget) | ## Medic -Built-in error monitoring subsystem. Watches logs for errors, fingerprints them via SHA1, deduplicates, and dispatches fix-it notifications to the responsible branch. Includes circuit breaker, per-branch rate limiting, and mute controls. +Error monitoring subsystem. Watches branch and system logs for errors, fingerprints them via SHA1, deduplicates, and dispatches fix-it notifications to the responsible branch. + +**Dispatch pipeline (8 gates):** + +1. **Medic enabled** — global on/off toggle +2. **Branch not muted** — per-branch suppression +3. **Count >= 2** — first occurrence suppressed, dispatch on recurrence +4. **Not DEV_CENTRAL** — devpulse protected from self-dispatch +5. **Branch in registry** — target must be a registered citizen +6. **Circuit breaker closed** — trips after 10 errors in 60s, 300s cooldown +7. **Per-fingerprint backoff** — exponential backoff per unique error +8. **Rate limit** — prevents dispatch floods + +On successful dispatch: sends email via `deliver_email_to_branch()` then calls `wake_branch()` to spawn an agent in the target branch immediately. + +**Persistent log watching** runs as a systemd user service (`trigger-log-watcher.service`). Starts both branch and system watchers, handles SIGTERM/SIGINT for clean shutdown. + +```bash +systemctl --user status trigger-log-watcher # Check watcher service +systemctl --user restart trigger-log-watcher # Restart watcher +``` + +## Error Registry + +SHA1 fingerprinting for error deduplication. Tracks: fingerprint, branch, error type, message, count, first/last seen, dispatch history, source fix status. + +**Circuit breaker:** Trips after 10 errors within 60 seconds. Rejects all dispatch while open. Auto-resets after 300s cooldown. State persists across restarts in `trigger_cb_state.json`. + +**Per-fingerprint tracking:** Each unique error has independent exponential backoff and dispatch count. State persists across restarts. ## Architecture ``` trigger/ ├── apps/ -│ ├── trigger.py # Entry point (auto-discovers modules) -│ ├── log_watcher_service.py # Persistent watcher process (systemd) +│ ├── trigger.py # Entry point (auto-discovers modules/) +│ ├── config.py # Constants, atomic_write_json, json_file_lock +│ ├── log_watcher_service.py # Persistent watcher daemon (systemd) │ ├── modules/ -│ │ ├── core.py # Event bus (Trigger.fire/on/off) -│ │ ├── errors.py # Error registry + cross-branch API -│ │ ├── medic.py # Error monitoring commands -│ │ ├── branch_log_events.py # Branch-level log event handling -│ │ └── log_events.py # System-wide log event processing +│ │ ├── core.py # Event bus: Trigger.fire/on/off/status +│ │ ├── errors.py # Error registry CLI: list/stats/circuit-breaker +│ │ ├── medic.py # Medic toggle: on/off/status/mute/unmute +│ │ ├── branch_log_events.py # Branch log watcher CLI: start/stop/status +│ │ └── log_events.py # System log watcher CLI: start/stop/status │ └── handlers/ -│ ├── events/ # One handler per event type -│ ├── log_watcher.py # Branch log watcher (watchdog) -│ ├── error_registry.py # SHA1 fingerprinting + circuit breaker -│ ├── error_reporter.py # Cross-branch error API -│ └── medic_state.py # Medic persistence (trigger_config.json) -└── tests/ +│ ├── error_registry.py # SHA1 fingerprinting, circuit breaker, backoff +│ ├── error_reporter.py # report_error() API + source fix emails +│ ├── log_watcher.py # Branch log watcher (watchdog, position tracking) +│ ├── medic_state.py # Medic config persistence (trigger_config.json) +│ ├── json/ +│ │ └── json_handler.py # JSON structure logging +│ ├── events/ +│ │ ├── registry.py # Auto-registers all 14 event handlers +│ │ ├── startup.py # Startup catch-up scan +│ │ ├── error_detected.py # 8-gate Medic dispatch +│ │ ├── error_logged.py # Monitor-only (no dispatch) +│ │ ├── warning_logged.py # Warning monitor +│ │ ├── plan_file.py # Plan lifecycle events +│ │ ├── bulletin_created.py # Bulletin propagation +│ │ ├── memory_threshold_exceeded.py +│ │ ├── memory_template_updated.py +│ │ ├── memory.py # memory_saved placeholder +│ │ ├── cli.py # cli_header_displayed hook +│ │ └── pr_status_sync.py # PR → prax status sync +│ └── watchers/ +│ └── log_watcher.py # System log watcher (system_logs/ dir) +├── tests/ # 367 tests across 12 modules +├── trigger_json/ # Runtime state files +│ ├── trigger_config.json # Medic state, muted branches +│ ├── error_registry.json # All tracked errors +│ └── trigger_cb_state.json # Circuit breaker persistence +└── trigger_data.json # Log watcher positions + dedup hashes ``` -**Note:** `branch_log_events` and `log_events` are auto-discovered modules that handle log-based event detection at branch and system levels respectively. +## Data Safety ---- +- **Atomic writes:** All JSON state files use `config.atomic_write_json()` — writes to a temp file in the same directory, then `os.replace()` for atomic rename. No partial writes on crash. +- **File locking:** All read-modify-write cycles wrapped in `config.json_file_lock()` using `fcntl.flock` with `.lock` sidecar files. Prevents concurrent corruption from watcher + CLI. +- **Circuit breaker persistence:** Trip state, recent errors, per-fingerprint tracking all survive restarts via `trigger_cb_state.json`. ## Integration Points ### Depends On - `aipass.prax` — Logging via `system_logger` -- `aipass.cli` — Console output and header formatting -- Python stdlib (`sys`, `pathlib`, `importlib`) +- `aipass.cli` — Console output and formatting +- `aipass.ai_mail` — `deliver_email_to_branch()` for dispatch emails (lazy import, graceful fallback) +- `aipass.memory` — `run_rollover()` check on startup (lazy import) ### Provides To -- All modules — event bus (`Trigger.fire`, `Trigger.on`, `Trigger.off`) -- All modules — cross-branch error reporting (`report_error`) -- All modules — medic error monitoring and dispatch +- All branches — Event bus (`Trigger.fire`, `Trigger.on`, `Trigger.off`) +- All branches — Cross-branch error reporting (`report_error()`) +- All branches — Automated error dispatch via Medic + +## Testing + +367 tests across 12 test modules, all passing. Coverage: 55/76 public functions (72%). + +```bash +cd src/aipass/trigger && pytest # Run all tests +``` + +Test files: `test_core`, `test_errors`, `test_medic`, `test_error_registry`, `test_error_reporter`, `test_medic_state`, `test_log_watcher`, `test_watchers_log_watcher`, `test_branch_log_events`, `test_log_events`, `test_json_handler`, `test_pr_status_sync` + +## Compliance + +Seedgo: 100% (34/34 standards). Zero type errors. All categories at 100%. --- From ffa466354df053bf856f493a1f7a03aa356e3251 Mon Sep 17 00:00:00 2001 From: AIOSAI Date: Wed, 22 Apr 2026 21:24:33 -0700 Subject: [PATCH 7/8] =?UTF-8?q?feat(system):=20docs(spawn):=20comprehensiv?= =?UTF-8?q?e=20README=20update=20=E2=80=94=20accurate=20commands,=20archit?= =?UTF-8?q?ecture,=20metrics,=20known=20issues?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-Authored-By: @spawn --- src/aipass/spawn/README.md | 254 ++++++++++++++++++++++++------------- 1 file changed, 165 insertions(+), 89 deletions(-) diff --git a/src/aipass/spawn/README.md b/src/aipass/spawn/README.md index 379220b4..ba507bd8 100644 --- a/src/aipass/spawn/README.md +++ b/src/aipass/spawn/README.md @@ -2,71 +2,94 @@ # SPAWN -**Purpose:** Branch creation, lifecycle management, and citizen class system. Creates new branches from class-scoped templates, manages updates, and grants citizenship. -**Module:** `aipass.spawn` -**Created:** 2026-03-05 +**The agent factory and branch lifecycle manager for AIPass.** + +**Module:** `aipass.spawn` | **Version:** 1.0.0 | **Created:** 2026-03-05 --- -## Overview +## What I Do -### What I Do - Create new branches from class-scoped templates (builder, birthright) - Grant birthright citizenship via the `passport` command -- Replace all `{{PLACEHOLDER}}` patterns with branch-specific values -- Update branches from templates (single or batch by class) -- Delete (archive + deregister) branches +- Update branches from templates (single or batch by class, with --dry-run) +- Delete branches (archive + deregister) - Sync registry and templates against filesystem -- Regenerate `.template_registry.json` with fresh file hashes -- Validate no unreplaced placeholders remain +- Regenerate template registries with fresh file hashes +- Replace all `{{PLACEHOLDER}}` patterns with branch-specific values +- Register new citizens in `AIPASS_REGISTRY.json` -### Citizen Classes +--- -Branches are scoped by **citizen class**, which determines the template used: +## Citizen Classes -| Class | Template | What it creates | +Every branch belongs to a **citizen class**, which determines its template: + +| Class | Template | What It Creates | |-------|----------|-----------------| -| `builder` (default) | `templates/builder/` | Full 3-layer scaffold: apps/, modules/, handlers/ | +| `builder` (default) | `templates/builder/` | Full 3-layer scaffold: .trinity/, .aipass/, apps/ (modules/ + handlers/), tests/, docs/, logs/ | | `birthright` | `templates/birthright/` | Minimal citizenship: .trinity/, .aipass/, README.md | -### Usage +--- + +## Commands + +All commands run through `drone @spawn `. + +### Create -**Create a builder branch (full scaffold):** ```bash -drone @spawn create /path/to/new/agent -drone @spawn create builder /path/to/new/agent --role "Analyst" --purpose "Reports" +drone @spawn create # Create builder branch +drone @spawn create --role "Analyst" --purpose "Reports" # With identity +drone @spawn create --template birthright # Specific class +drone @spawn create --dry-run # Preview without touching disk +drone @spawn create @existing # Adopt pre-existing agent +drone @spawn create ~/Projects/MyProject/agent_name # External project (auto-detects registry) ``` -**Grant birthright citizenship (minimal identity):** +### Passport + ```bash -drone @spawn passport @new_branch --role "Observer" --purpose "Monitoring" +drone @spawn passport @dirname # Grant birthright citizenship +drone @spawn passport @dirname --role "Observer" --purpose "Monitoring" ``` -**Update branches from template:** +### Update + ```bash -drone @spawn update @branch_name # Single branch (uses its passport class) -drone @spawn update builder --all # All builder-class branches -drone @spawn update birthright --all # All birthright-class branches -drone @spawn update --dry-run @branch_name # Preview changes +drone @spawn update @branch_name # Single branch (uses passport class) +drone @spawn update builder --all # All builder-class branches +drone @spawn update birthright --all # All birthright-class branches +drone @spawn update @branch_name --dry-run # Preview changes +drone @spawn update builder --all --dry-run # Preview batch update ``` -**Delete, sync:** +### Delete + ```bash -drone @spawn delete @branch_name # Archive + deregister -drone @spawn sync-registry # Repair registry vs filesystem -drone @spawn sync-registry --fix # Repair + rebuild missing .spawn/ tracking -drone @spawn sync-templates # Pull managed files from sources *(partial — template_owners.json empty)* -drone @spawn regenerate-registry # Regenerate builder template registry -drone @spawn regenerate-registry --all # Regenerate all template class registries +drone @spawn delete @branch_name # Archive + deregister ``` -**External project support:** +### Sync and Regenerate + ```bash -# Creating inside an existing AIPass project auto-detects the project registry -drone @spawn create ~/Projects/MyProject/agent_name # Registers in MYPROJECT_REGISTRY.json +drone @spawn sync-registry # Report healthy/stale/unregistered +drone @spawn sync-registry --fix # Rebuild .spawn/ tracking + fix passport registry_ids +drone @spawn sync-templates # Pull managed files from sources (partial — see Known Issues) +drone @spawn regenerate-registry # Regenerate builder template hashes +drone @spawn regenerate-registry --all # All template classes ``` -**Python API:** +### Introspection + +```bash +drone @spawn # No args — lists connected modules +drone @spawn --help # Full help text +drone @spawn --version # Version string +``` + +### Python API + ```python from aipass.spawn import spawn_agent @@ -76,101 +99,154 @@ result = spawn_agent( purpose="Process incoming reports", traits="Precise, thorough" ) -# result dict includes: success, branch_name, path, files_copied, validation_issues +# Returns: { success, branch_name, path, files_copied, validation_issues } ``` -### Known Limitations - -- `update --help` and `delete --help` fall through to argparse instead of showing help *(not operational — argparse `add_help=False` swallows --help before module intercept)* -- `sync-templates` runs but has no managed files *(partial — template_owners.json is empty)* -- `.py` files are never auto-updated by `update` — requires manual review by design - --- ## Architecture ``` spawn/ -├── __init__.py # Public API (exports spawn_agent) +├── __init__.py # Public API (exports spawn_agent) ├── apps/ -│ ├── spawn.py # Entry point (CLI) +│ ├── spawn.py # Entry point — CLI routing, version, help │ ├── modules/ -│ │ ├── core.py # Create orchestrator — coordinates spawn steps -│ │ ├── passport.py # Passport CLI — grant birthright citizenship -│ │ ├── update.py # Update CLI — parses args, delegates to handler -│ │ ├── delete.py # Delete CLI — parses args, delegates to handler -│ │ ├── sync_registry.py # Sync registry CLI — report and repair -│ │ ├── sync_templates.py # Sync templates CLI — pull from sources -│ │ └── regenerate_registry.py # Regenerate template registries CLI +│ │ ├── core.py # Create orchestrator (_spawn_agent, handle_command) +│ │ ├── passport.py # Passport CLI — birthright citizenship +│ │ ├── update.py # Update CLI — single/batch by class +│ │ ├── delete.py # Delete CLI — archive + deregister +│ │ ├── sync_registry.py # Registry repair CLI +│ │ ├── sync_templates.py # Template sync CLI +│ │ └── regenerate_registry.py # Template registry regeneration CLI │ └── handlers/ -│ ├── class_registry.py # Citizen class registry — maps classes to templates -│ ├── passport_ops.py # Passport grant implementation -│ ├── file_ops.py # Template copy, path renaming -│ ├── metadata.py # Branch name extraction, profile detection -│ ├── placeholders.py # {{PLACEHOLDER}} replacement engine -│ ├── registry.py # AIPASS_REGISTRY.json CRUD -│ ├── json_ops.py # JSON read/write operations -│ ├── json/ -│ │ └── json_handler.py # JSON I/O abstraction -│ ├── meta_ops.py # Branch metadata generation -│ ├── change_detection.py # File diff detection -│ ├── reconcile.py # Registry/filesystem reconciliation -│ ├── update_ops.py # Update implementation (class-aware) -│ ├── delete_ops.py # Delete implementation logic -│ ├── sync_registry_ops.py # Registry sync implementation -│ ├── sync_templates_ops.py # Template sync implementation -│ └── regenerate_registry_ops.py # Template registry regeneration +│ ├── class_registry.py # Citizen class → template directory mapping +│ ├── file_ops.py # Template copy, path renaming, registry regeneration +│ ├── metadata.py # Branch name extraction, profile detection +│ ├── placeholders.py # {{PLACEHOLDER}} replacement engine +│ ├── registry.py # AIPASS_REGISTRY.json CRUD, find_registry() +│ ├── meta_ops.py # Branch metadata generation, hash computation +│ ├── change_detection.py # ID-based file diff between template and branch +│ ├── reconcile.py # Registry/filesystem reconciliation +│ ├── passport_ops.py # Passport grant implementation +│ ├── update_ops.py # Update workflow (Phase 0 snapshot → detect → execute) +│ ├── delete_ops.py # Delete workflow (resolve → archive → cleanup → deregister) +│ ├── sync_registry_ops.py # Registry sync (CWD-aware, external project support) +│ ├── sync_templates_ops.py # Template sync implementation +│ ├── regenerate_registry_ops.py # Template registry hash regeneration +│ ├── json_ops.py # JSON deep merge, backup utilities +│ └── json/ +│ └── json_handler.py # Standard JSON I/O, operation logging, 7 API functions ├── templates/ -│ ├── builder/ # Full scaffold template (apps/, modules/, handlers/) -│ ├── birthright/ # Minimal template (.trinity/, .aipass/, README.md) +│ ├── builder/ # Full scaffold template (45 files, 24 dirs) +│ ├── birthright/ # Minimal template │ └── .archive/ -│ └── agent_mock_branch/ # Reference implementation -├── tools/ # Branch verification utilities -├── docs/ # Documentation -├── spawn_json/ # JSON tracking directory -└── tests/ # Test suite +│ └── agent_mock_branch/ # Reference implementation +├── tests/ # 13 test files, 253 tests +├── spawn_json/ # JSON tracking directory +├── tools/ # Branch verification utilities +├── docs/ # Documentation +└── logs/ # Prax log output ``` +### Three-Layer Design + +1. **Entry point** (`spawn.py`) — Routes CLI commands, never imports handlers directly +2. **Modules** (`modules/`) — Business logic coordinators, parse arguments, delegate to handlers +3. **Handlers** (`handlers/`) — Implementation details, pure functions where possible + --- -## Spawn Workflow +## Workflows ### Create (builder class) + 1. **Resolve** — Extract branch name from target path, validate path doesn't exist 2. **Lookup** — Resolve citizen class to template directory via class_registry 3. **Copy** — Recursive copy of class template to target (skips `__pycache__`) 4. **Rename** — Replace `{{BRANCH}}` in directory and file names 5. **Replace** — Substitute all `{{PLACEHOLDER}}` patterns in file contents -6. **Registry** — Regenerate `.template_registry.json`, register in `AIPASS_REGISTRY.json` +6. **Registry** — Generate `.branch_meta.json`, register in `AIPASS_REGISTRY.json` 7. **Validate** — Scan for any remaining `{{...}}` patterns ### Passport (birthright class) + 1. **Check** — Verify .trinity/ doesn't already exist 2. **Copy** — Copy birthright template (.trinity/, .aipass/, README.md) 3. **Replace** — Substitute placeholders in copied files 4. **Register** — Add to AIPASS_REGISTRY.json -### Update (class-aware) -- Single branch: reads `citizen_class` from passport, uses matching template -- Batch: `builder --all` or `birthright --all` filters by class -- `--all` without a class is blocked (safety) +### Update (class-aware, Phase 0) + +1. **Snapshot** — Back up current `.branch_meta.json` and `.template_registry.json` +2. **Detect** — Compare branch files against template via ID-based change detection +3. **Execute** — Apply renames, additions, JSON merges (`.py` files skipped by design) +4. **Refresh** — Regenerate `.branch_meta.json` with current state + +### Adopt Existing (`create @existing`) + +1. **Fix** — Repair `registry_id` in passport if stale (from registry recreation) +2. **Register** — Add to project registry +3. **Update** — Run template update to sync scaffold files --- -## Integration Points +## Tests + +**253 tests | 0 skipped | 0 failed** across 13 test files: + +| File | Focus | +|------|-------| +| `test_lifecycle.py` | End-to-end spawn lifecycle workflows | +| `test_json_handler.py` | JSON I/O, operation logging, standard API | +| `test_handlers.py` | Handler function behavior and integration | +| `test_regenerate_registry_ops.py` | Template registry regeneration | +| `test_update.py` | Branch update mechanics | +| `test_citizen_classes.py` | Citizen class validation and template discovery | +| `test_file_ops.py` | File copy, rename, placeholder replacement | +| `test_cli_routing.py` | Command routing and argument parsing | +| `test_contracts.py` | Handler contracts and interface compliance | +| `test_spawn.py` | Basic CLI routing and help | +| `test_error_resilience.py` | Error handling and edge cases | +| `conftest.py` | Fixtures: mock templates, registry protection | + +**Public functions:** 45 total, 41 tested (91%) + +--- + +## Integration ### Depends On -- `aipass.prax` — Logging via `system_logger` -- `aipass.cli` — Console output and headers -- Python stdlib (`pathlib`, `json`, `shutil`, `hashlib`, `re`) + +- **aipass.prax** — Logging via `system_logger` +- **aipass.cli** — Console output (header, error, warning) +- Python stdlib (`pathlib`, `json`, `shutil`, `hashlib`, `re`, `argparse`) ### Provides To -- All modules — branch creation, lifecycle management, citizenship -- Registry: Reads/writes `AIPASS_REGISTRY.json` + +- All branches — creation, template updates, registry management, citizenship +- Registry: CRUD operations on `AIPASS_REGISTRY.json` and `*_REGISTRY.json` + +--- + +## Known Issues + +- `sync-templates` is a no-op — `template_owners.json` has no entries (template IS source of truth, not downstream consumer) +- `.py` files never auto-update during `drone @spawn update` (by design) — template .py changes need individual branch dispatch +- 4 untested public functions remain (45 total, 41 tested) + +--- + +## Metrics + +- **Seedgo:** 100% (34/34) +- **Tests:** 253 passed, 0 skipped, 0 failed +- **Module coverage:** 23/23 (100%) +- **Template registry:** 45 files, 24 dirs (builder) +- **Battle test:** 17/17 commands pass (2026-04-22) --- *Last Updated: 2026-04-22* ---- [← Back to AIPass](../../../README.md) From 8a59c9a8ec71860fa7b876151badfbdb49fd2ddb Mon Sep 17 00:00:00 2001 From: AIOSAI Date: Wed, 22 Apr 2026 21:25:10 -0700 Subject: [PATCH 8/8] feat(drone): docs(drone): comprehensive README update Co-Authored-By: @drone --- src/aipass/drone/README.md | 222 +++++++++++++++++++++++++++++-------- 1 file changed, 176 insertions(+), 46 deletions(-) diff --git a/src/aipass/drone/README.md b/src/aipass/drone/README.md index 9c39d688..8060b3cc 100644 --- a/src/aipass/drone/README.md +++ b/src/aipass/drone/README.md @@ -2,8 +2,9 @@ # Drone -**Purpose:** Command router and symbolic addressing for AIPass. Resolves `@branch` names to paths at runtime via `AIPASS_REGISTRY.json`, routes commands to module entry points, and discovers available commands across the system. +**Purpose:** Command router and symbolic addressing for AIPass. Resolves `@branch` names to paths at runtime via `AIPASS_REGISTRY.json`, routes commands to module entry points, manages git workflows, and discovers available commands across the system. **Module:** `aipass.drone` +**Version:** 1.1.0 **Created:** 2026-03-05 --- @@ -13,8 +14,12 @@ ### What I Do - Resolve `@branch` symbolic names to absolute paths via `AIPASS_REGISTRY.json` - Route commands to registered branches and internal modules -- Discover available commands across the system +- Manage git workflows: PR creation, branch sync, lock management, merge +- Discover and scan available commands across the system - Provide `drone systems` introspection of all registered components +- Support external AIPass projects via dual registry lookup and module fallback + +--- ## Commands / Usage @@ -23,23 +28,34 @@ Drone provides a CLI for terminal use and a Python API for programmatic access. ### CLI ```bash -drone systems # List all registered modules and branches +# Core routing drone @seedgo audit aipass # Route "audit aipass" to seedgo drone @module --help # Show help for any module +drone systems # List all registered modules and branches + +# Git workflow drone @git pr "description" # Create a PR from current branch drone @git status # Git status scoped to branch directory drone @git sync # Pull latest main with --rebase +drone @git sync --autostash # Sync with autostash for dirty trees drone @git lock / unlock # Atomic branch lockfile -drone @git system-pr "desc" # System-wide PR (devpulse only) -drone @git merge # Squash-merge a PR + +# Git workflow (devpulse-authorized only) +drone @git system-pr "desc" # System-wide PR across all tracked changes +drone @git merge # Straight-merge a PR and sync local main drone @git smart-sync # Fetch + detect divergence + rebase drone @git fix # Auto-fix stuck rebase / detached HEAD +drone @git fix --dry-run # Detect issues without fixing + +# Command discovery drone scan @branch # Discover available commands in a branch -drone activate @branch # Scan + register all commands from a branch +drone activate @branch # Scan + register all commands as shortcuts drone list # List registered custom command shortcuts drone remove # Remove a custom command shortcut + +# Utilities drone hook-sounds on|off # Toggle hook notification sounds -drone --version # Show version +drone --version # Show version (v1.1.0) drone --help # Show usage information ``` @@ -93,48 +109,103 @@ except CommandExecutionError as e: ## Architecture +### 3-Layer Pattern + ``` drone/ -├── cli.py # pip entry point (drone command) -├── __init__.py # Public API exports +├── cli.py # pip entry point (drone command) +├── __init__.py # Public API exports (v1.1.0) ├── apps/ -│ ├── drone.py # Core entry point -│ ├── modules/ # Business logic -│ │ ├── config.py # Registry path resolution -│ │ ├── resolver.py # Branch resolution (@name -> path) -│ │ ├── router.py # Command routing via subprocess -│ │ ├── discovery.py # Module and command discovery -│ │ ├── module_registry.py # Internal module routing -│ │ ├── commands.py # Custom command shortcut orchestrator -│ │ ├── git_module.py # Git workflow (PR, status, sync, lock) -│ │ └── scan.py # Branch command scanning -│ ├── handlers/ # Implementation -│ │ ├── executor.py # Safe subprocess execution -│ │ ├── exceptions.py # Exception hierarchy -│ │ ├── generic_adapter.py # Centralized capture for external modules -│ │ ├── routing_config.json # External module routing declarations -│ │ ├── json/ # Three-JSON Pattern handler -│ │ ├── scanning/ # Scan result formatting + discovery -│ │ ├── command_registry/ # Command shortcut CRUD + lookup -│ │ └── git/ # Git workflow handlers +│ ├── drone.py # Core entry + CLI routing +│ ├── modules/ # Orchestrators (business logic) +│ │ ├── config.py # Registry path resolution +│ │ ├── resolver.py # Branch resolution (@name → path) +│ │ ├── router.py # Command routing via subprocess +│ │ ├── discovery.py # Module and command discovery +│ │ ├── module_registry.py # Internal module routing +│ │ ├── registry.py # Registry query operations +│ │ ├── commands.py # Custom command shortcut orchestrator +│ │ ├── git_module.py # Git workflow (9 commands + plugin routing) +│ │ └── scan.py # Branch command scanning +│ ├── handlers/ # Implementation details +│ │ ├── executor.py # Safe subprocess execution (timeout, no shell) +│ │ ├── exceptions.py # Exception hierarchy (10 exception types) +│ │ ├── router_handler.py # Routing implementation + caller detection +│ │ ├── registry_handler.py # Registry file ops + dual registry lookup +│ │ ├── discovery_handler.py # Discovery implementation + help parsing +│ │ ├── module_registry_handler.py # Module loading (internal + external) +│ │ ├── generic_adapter.py # StringIO capture for external modules +│ │ ├── routing_config.json # External module declarations +│ │ ├── json/ +│ │ │ └── json_handler.py # Structured operation logging +│ │ ├── scanning/ +│ │ │ ├── scanner.py # Help parsing + modules/ file scanning +│ │ │ └── formatters.py # Rich output for scan results +│ │ ├── command_registry/ +│ │ │ ├── ops.py # Command shortcut CRUD +│ │ │ ├── lookup.py # Greedy multi-word matching +│ │ │ └── formatters.py # Rich output for command lists +│ │ └── git/ │ │ ├── lock_handler.py # Atomic lockfile (O_CREAT|O_EXCL) -│ │ ├── pr_handler.py # 10-step PR workflow -│ │ ├── status_handler.py # Scoped git status (subprocess) -│ │ ├── status_handler_gitpython.py # [prototype] GitPython status — DPLAN-0140 Phase 1, not wired in -│ │ └── sync_handler.py # Safe main sync -│ └── plugins/ # Extensions beyond core routing -│ └── devpulse_ops/ # System-wide PR, merge, smart-sync, fix -├── docs/ # Documentation -└── tests/ # 529+ tests, 19 test files +│ │ ├── pr_handler.py # 10-step PR workflow with scoped staging +│ │ ├── status_handler.py # Scoped git status (subprocess) +│ │ ├── status_handler_gitpython.py # [prototype] DPLAN-0140 Phase 1, not wired in +│ │ └── sync_handler.py # Safe main sync (--autostash support) +│ └── plugins/ +│ ├── devpulse_ops/ # Privileged git operations (auth-gated) +│ │ ├── auth.py # Passport-based identity gate (ALLOWED_CALLERS) +│ │ ├── pr_plugin.py # System-wide PR (git add -A, system/ branches) +│ │ ├── merge_plugin.py # PR merge (--merge) + local sync +│ │ ├── sync_plugin.py # Smart sync (fetch, divergence detect, rebase) +│ │ └── fix_plugin.py # Auto-fix stuck rebase / detached HEAD +│ └── hook_sounds/ +│ └── hook_sounds_plugin.py # Toggle notification sounds on/off +├── docs/ # Public documentation +├── docs.local/ # Investigation reports and policies +└── tests/ # 530 tests across 20 test files ``` +### Routing Flow + +1. **CLI input** → `drone.py:main()` +2. **Built-in commands** checked first: `systems`, `scan`, `activate`, `list`, `remove`, `hook-sounds` +3. **`@target` routing** → branch resolution via `AIPASS_REGISTRY.json` → subprocess dispatch +4. **Module fallback** → if branch not found but is a registered module, routes internally +5. **Bare module names** → auto-discovered from `apps/modules/*.py`, routed via `importlib` +6. **Custom commands** → greedy multi-word matching against `drone_command_registry.json` + +### Module System + +Drone routes to two kinds of modules: + +| Type | Modules | Routing | +|------|---------|---------| +| Internal | `git` | `importlib` import → `handle_command()` | +| External | `seedgo`, `cli`, `spawn` | `generic_adapter.capture_main()` via `routing_config.json` | + +External modules are declared in `apps/handlers/routing_config.json` with entry points, descriptions, and versions. + +### Git Main-Only Enforcement + +All agents work on `main`. Branch creation is only allowed inside `drone @git system-pr`, which: +1. Commits changes on main +2. Moves branch pointer with `git branch -f` (HEAD stays on main) +3. Pushes branch with `--force-with-lease` +4. Opens PR via `gh` +5. Returns to main + +Enforcement layers: +- `.claude/settings.json` deny rules block `git checkout -b`, `git switch -c` +- `_assert_on_main_or_pr_flow()` guard in `git_module.py` +- Persistent citizen branches: `citizen/{name}` reused across PRs + --- ## Interactive Commands By default, drone captures subprocess output (`capture_output=True`) with a 30s timeout. This is safe for AI-to-AI routing but strips Rich colors, buffers progress bars, and kills long-running commands. -Commands in the interactive tuple bypass capture and inherit the terminal directly — enabling live Rich output, colors, and no timeout. Only add commands here when the user needs full terminal experience. +Commands in the interactive tuple bypass capture and inherit the terminal directly — enabling live Rich output, colors, and no timeout. **Per-command allowlist** (in `apps/drone.py`): @@ -142,6 +213,7 @@ Commands in the interactive tuple bypass capture and inherit the terminal direct |--------------|---------------------------------------------| | `monitor` | Prax real-time monitoring (live TUI) | | `audit` | Seedgo audit (Rich progress bars) | +| `watchdog` | Devpulse watchdog (live monitoring) | **Per-branch allowlist** — all commands from these branches get interactive mode: @@ -149,32 +221,90 @@ Commands in the interactive tuple bypass capture and inherit the terminal direct |----------|-----------------------------------------------| | `cli` | User-facing CLI with Rich formatted output | -To add: edit `interactive_commands` or `interactive_branches` in `_handle_target()` in `apps/drone.py`. +To add: edit `INTERACTIVE_COMMANDS` or `INTERACTIVE_BRANCHES` in `apps/drone.py`. + +--- + +## Plugin System + +Plugins live in `apps/plugins/{name}/` — outside the 3-layer structure by design. + +### devpulse_ops + +Auth-gated operations for system administration. `auth.py` walks CWD for `.trinity/passport.json` and checks `branch_name` against `ALLOWED_CALLERS` (devpulse, seedgo, spawn). + +| Plugin | Command | Purpose | +|--------|---------|---------| +| `pr_plugin` | `system-pr` | System-wide PR across all tracked changes | +| `merge_plugin` | `merge` | Straight-merge a PR and sync local main | +| `sync_plugin` | `smart-sync` | Fetch + detect divergence + rebase | +| `fix_plugin` | `fix` | Auto-fix stuck rebase / detached HEAD | + +### hook_sounds + +Simple toggle for hook notification sounds. Creates/removes `/tmp/aipass-hooks-muted` flag file. + +--- + +## External Project Support + +Infrastructure modules (seedgo, cli, git, spawn) work from external AIPass projects without per-project registration. + +**Dual registry lookup:** `registry_handler.py` merges local project registry with `AIPASS_HOME` registry. Local entries win on name collision. + +**Module fallback:** When subprocess routing fails (branch not in local registry), drone falls back to module routing for registered modules. Graceful degradation: Rich output from AIPass, functional output from external projects. + +**AIPASS_HOME hints:** When `AIPASS_HOME` is not set and the local registry lacks core branches, drone shows setup hints: +``` +Tip: set AIPASS_HOME=/path/to/AIPass to access all branches +``` --- ## Integration Points ### Depends On -- `AIPASS_REGISTRY.json` — Branch registry at repo root (read for resolution) -- Python stdlib (`pathlib`, `sys`, `subprocess`, `json`) +- `AIPASS_REGISTRY.json` — Branch registry (read for resolution) +- `gh` CLI — GitHub operations (PR creation, merge) +- Python stdlib (`pathlib`, `sys`, `subprocess`, `json`, `threading`) ### Provides To -- All modules — command routing via `drone @target command` -- All modules — branch/module discovery via `drone systems` +- All branches — command routing via `drone @target command` +- All branches — module/branch discovery via `drone systems` +- External modules — `generic_adapter.capture_main()` for subprocess-free routing - `aipass.seedgo` — routed via `drone @seedgo` +- `aipass.cli` — routed via `drone @cli` - `aipass.spawn` — routed via `drone @spawn` --- -## External Project Support +## Testing -Infrastructure modules (seedgo, cli, git) work from external AIPass projects without per-project registration. When subprocess routing fails (branch not in local registry), drone falls back to module routing automatically. Graceful degradation: Rich output from AIPass, functional output from external projects. +530 tests across 20 test files, covering all layers: + +| Area | Files | Tests | +|------|-------|-------| +| Core routing | `test_resolver.py`, `test_router.py`, `test_activation.py` | ~128 | +| Git operations | `test_git_module.py`, `test_system_pr.py`, `test_devpulse_plugins.py` | ~95 | +| Handlers | `test_executor.py`, `test_registry_handler.py`, `test_discovery.py` | ~99 | +| Infrastructure | `test_generic_adapter.py`, `test_module_registry.py`, `test_config.py` | ~66 | +| Features | `test_commands.py`, `test_scan.py`, `test_hook_sounds.py`, `test_json_handler.py` | ~125 | +| Standards | `test_cli_routing.py`, `test_contracts.py`, `test_error_resilience.py`, `test_init_provisioning.py` | ~21 | + +Run tests: `cd src/aipass/drone && python -m pytest tests/ -q` --- -**Last Updated:** 2026-04-21 +## Known Issues + +- `status_handler_gitpython.py` is an unreferenced prototype (DPLAN-0140 Phase 1) — awaiting Phase 2/3 +- `update_command()` and `command_exists()` in `ops.py` are tested CRUD API but unused from production +- Pyright warns about `json` package name shadowing stdlib — works at runtime +- Recurring sync errors when working tree is dirty — operational, not code bugs + +--- + +**Seedgo:** 100% (34/34) | **Tests:** 530 pass, 4 skip | **Last Updated:** 2026-04-22 --- [← Back to AIPass](../../../README.md) -