Merge pull request #417 from AIOSAI/system/devpulse-docsdrone-comprehensive-readme-update
feat(system): docs(drone): comprehensive README update
This commit is contained in:
AIPass
@@ -1 +0,0 @@
|
||||
{"file": "/home/patrick/Projects/AIPass/src/aipass/seedgo/apps/modules/inbox_audit.py", "errors": [{"line": 86, "message": "E501: Line too long (125 > 120)"}]}
|
||||
+140
-31
@@ -9,16 +9,35 @@
|
||||
|
||||
---
|
||||
|
||||
**Status:** Operational. Core email workflow (send/inbox/reply/close), dispatch system (with startup timeout + auto-retry), daemon, desktop notifications all working. Seedgo 99%.
|
||||
**Status:** Operational | **Seedgo:** 100% (34/34) | **Tests:** 355 pass | **Battle Tested:** S62
|
||||
|
||||
## Commands / Usage
|
||||
## Commands
|
||||
|
||||
```bash
|
||||
# Dispatch (send + wake in one step)
|
||||
drone @ai_mail dispatch @target "Subject" "Body" # Send dispatch email + wake
|
||||
drone @ai_mail dispatch @target "Subject" "Body" --fresh # Send + fresh wake
|
||||
drone @ai_mail email @target "Subject" "Body" # Send email (no wake)
|
||||
drone @ai_mail dispatch @target "Subject" "Body" --fresh # Send + fresh wake (new session)
|
||||
drone @ai_mail dispatch wake @target # Wake only (no email)
|
||||
drone @ai_mail inbox # Check inbox
|
||||
|
||||
# Send mail (no wake)
|
||||
drone @ai_mail email @target "Subject" "Body" # Send to one branch
|
||||
drone @ai_mail email @all "Subject" "Body" # Broadcast to all branches
|
||||
drone @ai_mail email @target "Subj" "Body" --from @spawn # Explicit sender override
|
||||
|
||||
# Read mail
|
||||
drone @ai_mail inbox # List all emails (new + opened)
|
||||
drone @ai_mail view <id> # View email (marks as opened)
|
||||
drone @ai_mail view latest # View most recent email
|
||||
|
||||
# Resolve mail
|
||||
drone @ai_mail reply <id> "message" # Reply + close + archive original
|
||||
drone @ai_mail close <id> # Close single email
|
||||
drone @ai_mail close <id1> <id2> <id3> # Close multiple emails
|
||||
drone @ai_mail close all # Close all emails
|
||||
|
||||
# Other
|
||||
drone @ai_mail sent # View sent messages
|
||||
drone @ai_mail contacts # List all known branches
|
||||
drone @ai_mail --help # Full help
|
||||
```
|
||||
|
||||
@@ -34,16 +53,55 @@ new → opened → closed
|
||||
- **opened** — Viewed by recipient, awaiting action
|
||||
- **closed** — Replied or dismissed, archived automatically
|
||||
|
||||
Each branch's mailbox lives at `<branch_path>/.ai_mail.local/inbox.json`. Sent copies go to `.ai_mail.local/sent/`. File locking (`fcntl`/`msvcrt`) protects concurrent inbox writes.
|
||||
|
||||
## Dispatch System
|
||||
|
||||
The `dispatch` command sends an email and wakes the target branch in one step. A polling daemon can also watch inboxes and spawn agents for `auto_execute` dispatch emails automatically.
|
||||
The `dispatch` command sends an email and wakes the target branch in one step. Dispatch emails carry `auto_execute: true` and a task header, signaling the target agent to process them as work items.
|
||||
|
||||
### Wake Pipeline
|
||||
|
||||
1. `dispatch.py` orchestrates: send email via `send_to_single()`, then wake via `wake_branch()`
|
||||
2. `wake.py` resolves the branch from the registry, finds the `claude` binary, spawns a subprocess
|
||||
3. `dispatch_monitor.py` wraps the claude process with safety features:
|
||||
- **Startup health check** — monitors JSONL session files for 90s, kills if no activity
|
||||
- **Auto-retry** — 3 strikes: attempt 1+2 resume, attempt 3 fresh (new session)
|
||||
- **Bounce email** — on final failure, sends error report back to sender
|
||||
- **Lock cleanup** — removes `.dispatch.lock` when agent exits
|
||||
4. After wake, `_spawn_watchdog()` auto-launches `drone @devpulse watchdog agent @target` as a detached background process
|
||||
|
||||
### Safety Limits
|
||||
|
||||
- PID-based locking prevents concurrent agents per branch (`.dispatch.lock`)
|
||||
- Max turns per wake, max dispatches per branch per day
|
||||
- `WAKE_BLOCKLIST` protects `@devpulse` from cross-branch manual wakes
|
||||
- `dispatch_monitor.py` strips `AIPASS_CALLER_*` env vars to prevent parent context leaking into agent identity
|
||||
- `AIPASS_BRANCH_NAME` env var set in spawn_env for CWD-independent identity
|
||||
|
||||
### Daemon
|
||||
|
||||
The polling daemon (`daemon.py`) watches inboxes for `auto_execute` dispatch emails and spawns agents automatically. It also runs the AIPASS-TEST token protocol: `scan_and_ack_test_emails()` intercepts ping-test messages and auto-replies with "ack" before dispatch processing.
|
||||
|
||||
## Sender Identity
|
||||
|
||||
Branch identity detection follows a priority chain in `detect_branch_from_pwd()`:
|
||||
|
||||
1. `AIPASS_CALLER_BRANCH` env var (set by drone router from passport or `AIPASS_BRANCH_NAME`)
|
||||
2. Contacts address book lookup (fastest path for registered branches)
|
||||
3. Registry lookup by name
|
||||
4. `AIPASS_CALLER_CWD` / `Path.cwd()` walk-up to find `.trinity/passport.json`
|
||||
5. Registry lookup by path
|
||||
|
||||
If all fail, detection returns `None` and the operation fails loudly. Wrong identity is worse than no identity.
|
||||
|
||||
The `--from @branch` flag on send/email commands provides an explicit sender override for callers outside branch directories.
|
||||
|
||||
## Cross-Project Email
|
||||
|
||||
External projects (outside the AIPass repo) can send to AIPass branches. On delivery, `delivery.py` stores a `reply_path` on the message (the sender's `inbox.json` path, resolved from `AIPASS_CALLER_CWD`). Replies use `_deliver_via_reply_path()` to write directly to the external inbox without needing registry lookup.
|
||||
|
||||
The contacts system (`contacts.py`) maintains an address book at `.ai_mail.local/contacts.json`, auto-registering branches on every send/receive. This enables fast sender detection for known branches without CWD walking or registry lookups.
|
||||
|
||||
- Agents are ephemeral (wake, do work, exit)
|
||||
- Safety limits: max turns per wake, max dispatches per branch per day
|
||||
- PID-based locking prevents concurrent agents per branch
|
||||
- Startup health check: monitors JSONL session files for 90s, kills if no activity
|
||||
- Auto-retry: 3 strikes (resume, resume, fresh) before bounce
|
||||
- Failed agents trigger bounce emails back to sender
|
||||
## Architecture
|
||||
|
||||
Follows the standard AIPass 3-layer pattern:
|
||||
@@ -51,20 +109,65 @@ Follows the standard AIPass 3-layer pattern:
|
||||
```
|
||||
ai_mail/
|
||||
├── apps/
|
||||
│ ├── ai_mail.py # Entry point (auto-discovers modules)
|
||||
│ ├── ai_mail.py # Entry point (auto-discovers modules)
|
||||
│ ├── modules/
|
||||
│ │ ├── email.py # Inbox, view, reply, close, contacts, routing
|
||||
│ │ ├── email_send.py # Send orchestration (direct, interactive, broadcast)
|
||||
│ │ ├── dispatch.py # Dispatch status, daemon, wake
|
||||
│ │ ├── email.py # Inbox, view, reply, close, contacts, routing
|
||||
│ │ ├── email_send.py # Send orchestration (direct, interactive, broadcast)
|
||||
│ │ └── dispatch.py # Dispatch send+wake, status, daemon control
|
||||
│ └── handlers/
|
||||
│ ├── email/ # Delivery, formatting, inbox ops, purge, reply
|
||||
│ ├── dispatch/ # Daemon, wake, dispatch_monitor, status, test_token
|
||||
│ ├── registry/ # Branch registry read
|
||||
│ ├── users/ # Branch detection, user lookup
|
||||
│ ├── json_utils/ # JSON I/O helpers (load_json, save_json)
|
||||
│ ├── paths.py # Shared find_repo_root() utility
|
||||
│ ├── notify.py # Desktop notifications (dbus)
|
||||
│ └── central_writer.py # Central inbox stats aggregation
|
||||
│ ├── email/
|
||||
│ │ ├── delivery.py # Core delivery pipeline (write to recipient inbox)
|
||||
│ │ ├── send.py # Sender resolution + send helpers
|
||||
│ │ ├── send_args.py # Argument parsing for send command
|
||||
│ │ ├── inbox_ops.py # Inbox loading + v1→v2 migration
|
||||
│ │ ├── inbox_cleanup.py # Mark opened/closed + archive
|
||||
│ │ ├── inbox_lock.py # File locking (fcntl/msvcrt cross-platform)
|
||||
│ │ ├── inbox_resolve.py # Resolve inbox path from args or caller
|
||||
│ │ ├── reply.py # Reply + auto-close original
|
||||
│ │ ├── close_ops.py # Batch close operations
|
||||
│ │ ├── contacts.py # Address book for branch routing
|
||||
│ │ ├── create.py # Email file creation (sent/ folder)
|
||||
│ │ ├── format.py # Display formatting
|
||||
│ │ ├── header.py # Dispatch header injection
|
||||
│ │ ├── footer.py # Email footer
|
||||
│ │ ├── purge.py # Auto-purge sent/deleted folders
|
||||
│ │ ├── error_dispatch.py # Error reporting via email
|
||||
│ │ └── dashboard_sync.py # Dashboard integration
|
||||
│ ├── dispatch/
|
||||
│ │ ├── daemon.py # Polls inboxes, spawns agents for dispatch emails
|
||||
│ │ ├── wake.py # Wakes branches via claude subprocess
|
||||
│ │ ├── dispatch_monitor.py # Wraps claude process (bounce + lock cleanup)
|
||||
│ │ ├── status.py # Dispatch log I/O
|
||||
│ │ └── test_token.py # AIPASS-TEST ping protocol (auto-ack)
|
||||
│ ├── registry/
|
||||
│ │ └── read.py # Registry reading + get_all_branches()
|
||||
│ ├── users/
|
||||
│ │ ├── branch_detection.py # CWD/env-based branch identity detection
|
||||
│ │ └── user.py # Current user detection (get_current_user)
|
||||
│ ├── json_utils/
|
||||
│ │ └── json_handler.py # Auto-creating JSON system
|
||||
│ ├── paths.py # Shared find_repo_root() utility
|
||||
│ ├── notify.py # Desktop notifications (dbus direct)
|
||||
│ └── central_writer.py # Central inbox stats aggregation
|
||||
└── tests/ # 355 tests across 16 test files
|
||||
├── conftest.py # Shared fixtures (mock_logger, mock_json_handler)
|
||||
├── test_daemon.py # Daemon config, state, kill switch, dispatch check
|
||||
├── test_dispatch_monitor.py # Monitor safety features, env stripping
|
||||
├── test_dispatch_status.py # Log I/O, age calculation
|
||||
├── test_dispatch_watchdog.py # Watchdog auto-spawn
|
||||
├── test_wake.py # Branch resolution, PID checks, lock files
|
||||
├── test_wake_blocklist.py # Wake protection for @devpulse
|
||||
├── test_delivery.py # Inbox migration, private branches, pipeline
|
||||
├── test_send_identity.py # Sender identity chain (36 tests)
|
||||
├── test_user_paths.py # Mailbox path resolution (13 tests)
|
||||
├── test_contacts.py # Address book operations
|
||||
├── test_inbox_ops.py # Inbox loading + migration
|
||||
├── test_registry_read.py # Registry parsing + branch lookup
|
||||
├── test_central_writer.py # Central stats aggregation
|
||||
├── test_cli_routing.py # CLI routing + help/version
|
||||
├── test_json_handler.py # JSON I/O helpers
|
||||
├── test_notify.py # Desktop notification dbus calls
|
||||
└── test_paths.py # find_repo_root() utility
|
||||
```
|
||||
|
||||
## Integration Points
|
||||
@@ -73,16 +176,22 @@ ai_mail/
|
||||
- `aipass.prax` — Logging via `system_logger`
|
||||
- `aipass.cli` — Console output and display formatting
|
||||
- `aipass.drone` — Command routing and `@branch` resolution
|
||||
- Python stdlib (`pathlib`, `json`, `argparse`, `importlib`)
|
||||
- `aipass.trigger` — `trigger.fire()` for `email_dispatched` events
|
||||
- Python stdlib (`pathlib`, `json`, `argparse`, `importlib`, `subprocess`, `fcntl`)
|
||||
|
||||
### Provides To
|
||||
- All modules — inter-branch messaging (send/receive/reply/close)
|
||||
- Dispatch system — autonomous task execution via `--dispatch` flag
|
||||
- Branch contacts — address book for `@branch` routing
|
||||
- **All branches** — inter-branch messaging (send/receive/reply/close)
|
||||
- **Dispatch system** — autonomous task execution via `auto_execute` emails
|
||||
- **Branch contacts** — address book for `@branch` routing
|
||||
- **trigger branch** — `deliver_email_to_branch()` imported directly for event-driven delivery
|
||||
- **Desktop** — dbus notifications for delivery, wake, completion events
|
||||
|
||||
## Known Issues
|
||||
|
||||
- **DPLAN-0138**: Inbox backdoor audit identified 2 write path classes — ad-hoc direct writes (detectable by non-UUID ID format) and `_deliver_via_reply_path()` bypass (no lock, no notification). Fix pending.
|
||||
- **Caller detection**: `BRANCH DETECTION FAILED` when callers don't set `AIPASS_CALLER_BRANCH` (low severity, caller-side fix — use `--from` flag)
|
||||
- **Cross-branch writes**: ai_mail not in trusted cross-writers list for `system-pr`
|
||||
|
||||
---
|
||||
|
||||
*Last Updated: 2026-04-07*
|
||||
|
||||
---
|
||||
[← Back to AIPass](../../../README.md)
|
||||
|
||||
+126
-70
@@ -2,19 +2,24 @@
|
||||
|
||||
# API
|
||||
|
||||
**Purpose:** Centralized external API gateway — authenticated service clients for all external APIs (OpenRouter, Google, future providers).
|
||||
**Module:** `aipass.api`
|
||||
> Centralized external API gateway — authenticated service clients for all external APIs
|
||||
|
||||
**Module:** `aipass.api` | **Role:** `api_gateway` | **Version:** 1.0.0
|
||||
**Seedgo:** 100% (34/34) | **Tests:** 306 pass | **Functions:** 74 public (73 tested)
|
||||
**Last Updated:** 2026-04-22
|
||||
|
||||
---
|
||||
|
||||
## Overview
|
||||
|
||||
API is the centralized external service gateway for AIPass. It provides authenticated clients for external APIs (OpenRouter, Google, future providers). Consumers import ready-to-use service objects — API owns the plumbing, consumers own the business logic.
|
||||
|
||||
### What I Do
|
||||
- Provide authenticated service clients for external APIs (Google Drive, OpenRouter, etc.)
|
||||
- Provide authenticated service clients for external APIs (OpenRouter, Google Drive, Calendar, etc.)
|
||||
- Manage OAuth2 flows, credential storage, and token refresh
|
||||
- Offer thread-safe service factories for concurrent consumers
|
||||
- Handle API key management and validation across providers
|
||||
- Host the generic contract registry for private integration drivers (DPLAN-0133)
|
||||
- Provide SSL retry and connection resilience utilities
|
||||
|
||||
### What I Don't Do
|
||||
@@ -22,38 +27,44 @@
|
||||
- Set default models or configs — consumers provide their own
|
||||
- Manage application workflows, polling loops, or orchestration
|
||||
|
||||
### How I Work
|
||||
- **Entry Point:** `apps/api.py` -- auto-discovers and routes to modules
|
||||
- **Pattern:** Standard AIPass 3-tier architecture (entry point / modules / handlers)
|
||||
- **Design principle:** If it's not auth, credentials, or service factory — it doesn't belong here
|
||||
### Design Principle
|
||||
If it's not auth, credentials, or service factory — it doesn't belong here.
|
||||
|
||||
---
|
||||
|
||||
## Commands / Usage
|
||||
## Commands
|
||||
|
||||
```bash
|
||||
drone @api get-key # Retrieve API key for provider
|
||||
drone @api validate # Validate API credentials and connection
|
||||
drone @api validate google # Validate Google OAuth2 credentials
|
||||
drone @api reauth google # Re-authenticate Google OAuth2
|
||||
drone @api test # Test OpenRouter connection status
|
||||
drone @api models [--all] # List available models from provider
|
||||
drone @api status # Check OpenRouter client status
|
||||
drone @api call "prompt" --model MODEL # Make API call to model
|
||||
drone @api list-providers # List available API providers
|
||||
drone @api init # Initialize .env template
|
||||
drone @api track <gen_id> # Track API usage metrics
|
||||
drone @api stats # Display API usage statistics
|
||||
drone @api session # Show session usage data
|
||||
drone @api caller-usage <caller> # Show usage by caller module
|
||||
drone @api cleanup [days] # Clean up old usage data *(not operational — fails with no data)*
|
||||
drone @api integrations list # List registered integration contracts
|
||||
drone @api integrations call <contract> [args...] # Call a registered contract
|
||||
drone @api --help # Full help output
|
||||
drone @api --version # Show version
|
||||
```
|
||||
# Key Management
|
||||
drone @api get-key [provider] # Retrieve API key (default: openrouter)
|
||||
drone @api validate [provider] # Validate API key (default: openrouter)
|
||||
drone @api validate google # Validate Google OAuth2 credentials
|
||||
drone @api reauth google # Re-authenticate Google OAuth2
|
||||
drone @api list-providers # List available API providers
|
||||
drone @api init # Initialize .env template at ~/.secrets/aipass/
|
||||
|
||||
Running `drone @api` with no arguments displays module introspection (discovered modules and status).
|
||||
# OpenRouter
|
||||
drone @api test # Test OpenRouter connection status
|
||||
drone @api models [--all] # List available models (default: top 10)
|
||||
drone @api status # Check OpenRouter client status (key, SDK, cache)
|
||||
drone @api call "prompt" --model MODEL # Make API call to model
|
||||
|
||||
# Usage Tracking
|
||||
drone @api track <gen_id> # Track API usage for a generation
|
||||
drone @api stats # Display overall usage statistics
|
||||
drone @api session # Show current session usage
|
||||
drone @api caller-usage <caller> # Show usage by caller module
|
||||
drone @api cleanup [days] # Clean up data older than N days (default: 30)
|
||||
|
||||
# Integration Contracts
|
||||
drone @api integrations list # List registered contracts
|
||||
drone @api integrations call <contract> [args...] # Call a registered contract
|
||||
|
||||
# Meta
|
||||
drone @api --help # Full help output
|
||||
drone @api --version # Show version (v1.0.0)
|
||||
drone @api # Module introspection (7 discovered modules)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
@@ -81,62 +92,107 @@ from aipass.api.apps.modules.google_client import api_call_with_retry
|
||||
|
||||
## Architecture
|
||||
|
||||
Three-tier design: entry point routes to modules (orchestration), which delegate to handlers (business logic). Modules are auto-discovered from `apps/modules/*.py` — each implements `handle_command(command, args) -> bool`.
|
||||
|
||||
```
|
||||
api/
|
||||
├── __init__.py # Public API exports
|
||||
├── __init__.py
|
||||
├── apps/
|
||||
│ ├── api.py # Entry point (module discovery, command routing)
|
||||
│ ├── modules/
|
||||
│ │ ├── api_key.py # Key retrieval and validation logic
|
||||
│ │ ├── openrouter_client.py # OpenRouter API client
|
||||
│ │ ├── google_client.py # Google API services (Drive, Calendar, etc.)
|
||||
│ │ ├── usage_tracker.py # Usage metrics tracking
|
||||
│ │ ├── bridge.py # Generic contract registry (register/resolve)
|
||||
│ │ ├── integrations_manager.py # Integration contract orchestration
|
||||
│ │ └── registry.py # Driver auto-discovery (load_drivers)
|
||||
│ └── handlers/
|
||||
│ ├── auth/
|
||||
│ │ ├── env.py # Environment variable credential loading
|
||||
│ │ └── keys.py # API key storage and retrieval
|
||||
│ ├── config/
|
||||
│ │ └── provider.py # Provider configuration management
|
||||
│ ├── google/
|
||||
│ │ ├── auth.py # OAuth2 lifecycle, credential I/O
|
||||
│ │ ├── service_factory.py # Service object factory (single + thread-safe)
|
||||
│ │ └── retry.py # SSL retry with exponential backoff
|
||||
│ ├── json/
|
||||
│ │ └── json_handler.py # JSON operation logging
|
||||
│ ├── openrouter/
|
||||
│ │ ├── caller.py # HTTP request execution
|
||||
│ │ ├── client.py # OpenRouter client implementation
|
||||
│ │ ├── models.py # Model discovery and listing
|
||||
│ │ └── provision.py # Provider provisioning
|
||||
│ ├── integrations/
|
||||
│ │ ├── list.py # List registered contracts handler
|
||||
│ │ └── call.py # Call registered contract handler
|
||||
│ └── usage/
|
||||
│ ├── aggregation.py # Usage data aggregation
|
||||
│ ├── cleanup.py # Usage data cleanup
|
||||
│ └── tracking.py # Usage event tracking
|
||||
└── README.md
|
||||
│ ├── api.py # Entry point — module discovery, command routing
|
||||
│ ├── modules/ # Tier 2: orchestration layer (7 modules)
|
||||
│ │ ├── api_key.py # Key retrieval, validation, provider listing
|
||||
│ │ ├── openrouter_client.py # OpenRouter client — calls, models, status
|
||||
│ │ ├── google_client.py # Google API services (Drive, Calendar, etc.)
|
||||
│ │ ├── usage_tracker.py # Usage metrics — track, stats, cleanup
|
||||
│ │ ├── bridge.py # Generic contract registry (register/resolve)
|
||||
│ │ ├── integrations_manager.py # Contract dispatch — integrations list/call
|
||||
│ │ └── registry.py # Driver auto-discovery (load_drivers)
|
||||
│ ├── handlers/ # Tier 3: business logic (7 packages, 15 files)
|
||||
│ │ ├── auth/
|
||||
│ │ │ ├── env.py # .env template creation (0o600 permissions)
|
||||
│ │ │ └── keys.py # Key storage, retrieval, validation rules
|
||||
│ │ ├── config/
|
||||
│ │ │ └── provider.py # Provider config deep-merge, validation rules
|
||||
│ │ ├── google/
|
||||
│ │ │ ├── auth.py # OAuth2 lifecycle — load, refresh, save credentials
|
||||
│ │ │ ├── service_factory.py # Service object factory (single + thread-safe)
|
||||
│ │ │ └── retry.py # Exponential backoff with SSL error detection
|
||||
│ │ ├── integrations/
|
||||
│ │ │ ├── list.py # List registered contracts
|
||||
│ │ │ └── call.py # Invoke contract driver functions
|
||||
│ │ ├── json/
|
||||
│ │ │ └── json_handler.py # JSON persistence — 3-file pattern per operation
|
||||
│ │ ├── openrouter/
|
||||
│ │ │ ├── caller.py # Stack-based caller detection
|
||||
│ │ │ ├── client.py # OpenRouter client (OpenAI SDK wrapper)
|
||||
│ │ │ ├── models.py # Model discovery and listing
|
||||
│ │ │ └── provision.py # Per-caller config auto-provisioning
|
||||
│ │ └── usage/
|
||||
│ │ ├── aggregation.py # Stats rollup — per-caller, daily, monthly
|
||||
│ │ ├── cleanup.py # Data retention and cleanup
|
||||
│ │ └── tracking.py # Generation metrics from OpenRouter API
|
||||
│ └── integrations/ # Private driver space (gitignored)
|
||||
│ └── {project}/driver.py # Each driver registers contracts via bridge
|
||||
├── tests/ # 306 tests across 15 files
|
||||
│ ├── test_api_key.py # Key management (39 tests)
|
||||
│ ├── test_caller.py # Caller detection (9 tests)
|
||||
│ ├── test_cli_routing.py # Command routing (9 tests)
|
||||
│ ├── test_config_provider.py # Config merging (16 tests)
|
||||
│ ├── test_contracts.py # JSON contract operations (11 tests)
|
||||
│ ├── test_critical_paths.py # End-to-end critical flows (19 tests)
|
||||
│ ├── test_error_resilience.py # Error handling (4 tests)
|
||||
│ ├── test_google_client.py # Google OAuth2 + services (46 tests)
|
||||
│ ├── test_init_provisioning.py # Provisioning init (4 tests)
|
||||
│ ├── test_integrations.py # Bridge, registry, contracts (17 tests)
|
||||
│ ├── test_json_handler.py # JSON persistence (41 tests)
|
||||
│ ├── test_openrouter_client.py # OpenRouter client (37 tests)
|
||||
│ ├── test_provision.py # Auto-provisioning (20 tests)
|
||||
│ ├── test_tracking.py # Usage tracking (13 tests)
|
||||
│ └── test_usage_tracker.py # Usage module (21 tests)
|
||||
└── docs/
|
||||
└── SECURITY.md # Key handling and leak prevention
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Integration Contract System
|
||||
|
||||
Private integration drivers live in `apps/integrations/{project}/driver.py` (gitignored). Each driver registers named contracts via `bridge.register()`. Callers resolve contracts by name — never referencing private projects directly.
|
||||
|
||||
**Three-layer design (DPLAN-0133):**
|
||||
1. **Drivers** (`apps/integrations/*/driver.py`) — private, gitignored, register contracts on load
|
||||
2. **Bridge** (`bridge.py`) — public contract registry: `register()`, `resolve()`, `list_contracts()`
|
||||
3. **Handlers** (`integrations/list.py`, `call.py`) — public dispatch for `drone @api integrations`
|
||||
|
||||
**Auto-discovery:** `registry.py` walks `apps/integrations/*/driver.py` via `importlib.util.spec_from_file_location`. Handles empty dirs, missing files, and import errors gracefully.
|
||||
|
||||
---
|
||||
|
||||
## Integration Points
|
||||
|
||||
### Depends On
|
||||
- `aipass.prax` -- structured logging via `system_logger`
|
||||
- `aipass.cli` -- Rich console output formatting
|
||||
- `aipass.prax` — structured logging via `system_logger`
|
||||
- `aipass.cli` — Rich console output formatting
|
||||
|
||||
### Provides To
|
||||
- All branches -- authenticated external API clients
|
||||
- All branches — authenticated external API clients via `get_response()`, `get_drive_service()`, `get_google_service()`
|
||||
- System-wide API key management and credential validation
|
||||
|
||||
### Credentials
|
||||
- `~/.secrets/aipass/.env` -- API keys (OpenRouter, etc.)
|
||||
- `~/.secrets/aipass/google_creds.json` -- Google OAuth2 tokens
|
||||
- `~/.secrets/aipass/google_client_secret.json` -- Google OAuth app config
|
||||
All credentials live at `~/.secrets/aipass/` (0o700 directory, 0o600 files):
|
||||
- `.env` — API keys (OpenRouter, etc.)
|
||||
- `google_creds.json` — Google OAuth2 tokens
|
||||
- `google_client_secret.json` — Google OAuth app config
|
||||
|
||||
### Provider Pattern
|
||||
One module per provider (`openrouter_client.py`, `google_client.py`), one handler directory per provider (`openrouter/`, `google/`). Module orchestrates and presents CLI; handlers implement business logic. Pattern scales to future providers.
|
||||
|
||||
---
|
||||
|
||||
## Known Issues
|
||||
- Google auth libraries are optional deps — commands fail with install instructions if missing
|
||||
- 1/74 public functions untested per seedgo test_map
|
||||
- Backup branch credential migration pending (`~/.aipass/` to `~/.secrets/aipass/`)
|
||||
|
||||
---
|
||||
|
||||
|
||||
+129
-49
@@ -2,14 +2,17 @@
|
||||
|
||||
# CLI
|
||||
|
||||
**Purpose:** Display and output formatting service for AIPass modules. Provides consistent terminal output — headers, success/error/warning messages, section breaks, and operation templates — so every module looks the same without duplicating Rich formatting code.
|
||||
**Purpose:** Display and output formatting service for all AIPass branches, plus the `aipass init` project bootstrap command. Provides consistent terminal output — headers, success/error/warning messages, section breaks, and operation templates — so every branch looks the same without duplicating Rich formatting code.
|
||||
**Module:** `aipass.cli`
|
||||
**Seedgo:** 100%
|
||||
**Tests:** 60+ passing (6 files, 5/5 modules covered)
|
||||
**Version:** 2.0.0
|
||||
**Seedgo:** 100% (34/34 standards)
|
||||
**Tests:** 171 passing (6 files, 5/5 modules covered)
|
||||
**Last Updated:** 2026-04-22
|
||||
|
||||
## Usage
|
||||
|
||||
### Display Functions
|
||||
|
||||
```python
|
||||
from aipass.cli import header, success, error, warning, section
|
||||
|
||||
@@ -27,13 +30,13 @@ from aipass.cli import operation_start, operation_complete
|
||||
|
||||
operation_start("Processing", count=10)
|
||||
# ... do work ...
|
||||
operation_complete(created=5, skipped=3, failed=0)
|
||||
operation_complete(created=5, skipped=3, failed=0, time="1.2s")
|
||||
```
|
||||
|
||||
### Fatal (exit on error)
|
||||
|
||||
```python
|
||||
from aipass.cli.apps.modules.display import fatal
|
||||
from aipass.cli import fatal
|
||||
|
||||
fatal("Config file missing", suggestion="Run aipass init first")
|
||||
# Prints error message + suggestion, then calls sys.exit(1)
|
||||
@@ -47,55 +50,48 @@ from aipass.cli import console
|
||||
console.print("[bold cyan]Custom Rich output[/bold cyan]")
|
||||
```
|
||||
|
||||
## Architecture
|
||||
## Public API
|
||||
|
||||
```
|
||||
cli/
|
||||
├── apps/
|
||||
│ ├── cli.py # Entry point (main, discover_modules, route_command)
|
||||
│ ├── modules/
|
||||
│ │ ├── display.py # header, success, error, warning, fatal, section
|
||||
│ │ ├── templates.py # operation_start, operation_complete
|
||||
│ │ └── init_project.py # aipass init command routing
|
||||
│ └── handlers/
|
||||
│ ├── init/ # Project bootstrap logic
|
||||
│ │ ├── bootstrap.py
|
||||
│ │ └── scaffold_content.py
|
||||
│ ├── json/ # JSON file management
|
||||
│ │ └── json_handler.py
|
||||
│ └── templates/ # Empty — placeholder from scaffold
|
||||
├── cli_json/ # Auto-created JSON output (three-file pattern)
|
||||
├── dropbox/ # Inbound file drop
|
||||
├── logs/ # Branch-level logs
|
||||
├── tests/ # 161 tests across 6 files
|
||||
│ ├── test_bootstrap.py # bootstrap.py handler tests
|
||||
│ ├── test_json_handler.py # json_handler tests
|
||||
│ ├── test_display.py # display module tests
|
||||
│ ├── test_templates.py # templates module tests
|
||||
│ ├── test_init_project.py # init_project module tests
|
||||
│ └── test_integration.py # main() flow + entry point tests
|
||||
└── .archive/ # Archived stubs (extensions/, json_templates/)
|
||||
```
|
||||
All display functions are importable from the top-level package or `apps/modules/`:
|
||||
|
||||
- `apps/modules/` — Public API. Import from here.
|
||||
- `apps/handlers/` — Internal implementation. Don't import directly.
|
||||
| Function | Signature | Purpose |
|
||||
|----------|-----------|---------|
|
||||
| `header()` | `header(title, details=None)` | Bordered section header with optional key-value pairs |
|
||||
| `success()` | `success(message, **kwargs)` | Green checkmark message with metadata |
|
||||
| `error()` | `error(message, suggestion=None)` | Red error with optional suggestion |
|
||||
| `warning()` | `warning(message, details=None)` | Yellow warning with optional details |
|
||||
| `fatal()` | `fatal(message, suggestion=None)` | Error + `sys.exit(1)` for unrecoverable failures |
|
||||
| `section()` | `section(title)` | Visual section separator with title |
|
||||
| `operation_start()` | `operation_start(operation, **details)` | Standard operation begin header |
|
||||
| `operation_complete()` | `operation_complete(**summary)` | Completion summary with optional timing |
|
||||
| `console` | Rich Console instance | Standard output console |
|
||||
| `err_console` | Rich Console instance | Stderr console |
|
||||
|
||||
Import paths (all equivalent):
|
||||
```python
|
||||
from aipass.cli import header # Top-level re-export
|
||||
from aipass.cli.apps.modules import header # Module-level
|
||||
from aipass.cli.apps.modules.display import header # Direct
|
||||
```
|
||||
|
||||
## Commands
|
||||
|
||||
```bash
|
||||
# Via drone
|
||||
drone @cli --help # Show services and Rich formatting showcase
|
||||
drone @cli --version # Show version
|
||||
drone @cli # Show discovered modules (introspection)
|
||||
drone @cli --help # Services + Rich formatting showcase
|
||||
drone @cli --version # Version (v2.0.0)
|
||||
drone @cli # Module discovery (introspection)
|
||||
drone @cli aipass # Show aipass subcommands
|
||||
drone @cli aipass init # Bootstrap AIPass project in current dir
|
||||
drone @cli aipass init /path # Bootstrap in target directory
|
||||
drone @cli aipass init /path MyProject # Bootstrap with custom name
|
||||
drone @cli aipass init agent <name> # Create agent in project (routes to spawn)
|
||||
drone @cli aipass init update # Re-sync managed scaffold files
|
||||
drone @cli aipass init update /path # Re-sync in target directory
|
||||
drone @cli aipass init agent <name> # Create agent (routes to spawn)
|
||||
drone @cli aipass init --help # Detailed init usage
|
||||
drone @cli display # Display module introspection
|
||||
drone @cli display # Display module info
|
||||
drone @cli display demo # Run display function showcase
|
||||
drone @cli templates # Templates module introspection
|
||||
drone @cli templates # Templates module info
|
||||
drone @cli templates demo # Run templates function showcase
|
||||
|
||||
# Standalone (no drone required)
|
||||
@@ -104,20 +100,104 @@ python -m aipass.cli aipass init /path # Bootstrap directly
|
||||
aipass --help # Via console_scripts entry point
|
||||
```
|
||||
|
||||
---
|
||||
## aipass init
|
||||
|
||||
Bootstraps a new AIPass project with 21 items (when AIPASS_HOME is detected):
|
||||
|
||||
| Category | Items Created |
|
||||
|----------|---------------|
|
||||
| Identity | `*_REGISTRY.json` |
|
||||
| Prompts | `.aipass/aipass_global_prompt.md`, `CLAUDE.md`, `AGENTS.md`, `GEMINI.md` |
|
||||
| Docs | `README.md`, `STATUS.local.md` |
|
||||
| Config | `.claude/settings.json`, `.gitignore` |
|
||||
| Slash commands | `.claude/commands/prep.md`, `.claude/commands/memo.md` |
|
||||
| Hooks | 7 enforcement/injector hooks in `.claude/hooks/` |
|
||||
| Dirs | `hooks/`, `src/` |
|
||||
| Mail | `.ai_mail.local/inbox.json` |
|
||||
|
||||
Settings.json wires 5 hook event types:
|
||||
- **UserPromptSubmit** (5 hooks): global prompt, local prompt, branch prompt loader, email notification, identity injector
|
||||
- **PostToolUse**: auto-fix diagnostics
|
||||
- **PreToolUse**: pre-edit gate
|
||||
- **Stop**: subagent stop gate
|
||||
- **PreCompact**: pre-compact
|
||||
|
||||
`aipass init update` re-syncs managed files (prompts, config, hooks) without touching user-owned files (registry, README, STATUS.local.md, .gitignore, mailbox).
|
||||
|
||||
Cross-platform: local prompt discovery uses `python3 -c` with pathlib (no bash dependency). A `setup.py` installer handles Windows + Linux + macOS.
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
cli/
|
||||
├── __init__.py # Public API exports + cli_entry()
|
||||
├── __main__.py # python -m aipass.cli entry
|
||||
├── apps/
|
||||
│ ├── cli.py # Entry point (main, discover_modules, route_command)
|
||||
│ ├── modules/ # PUBLIC — import from here
|
||||
│ │ ├── __init__.py # Re-exports all display + template functions
|
||||
│ │ ├── display.py # header, success, error, warning, fatal, section
|
||||
│ │ ├── templates.py # operation_start, operation_complete
|
||||
│ │ └── init_project.py # aipass init command routing
|
||||
│ └── handlers/ # PRIVATE — internal implementation
|
||||
│ ├── init/
|
||||
│ │ ├── bootstrap.py # init_project(), update_project() (511 lines)
|
||||
│ │ └── scaffold_content.py # 10 content generators (502 lines)
|
||||
│ ├── json/
|
||||
│ │ └── json_handler.py # JSON lifecycle (CRUD, validation, rotation)
|
||||
│ └── templates/ # Empty — placeholder from scaffold
|
||||
├── tests/ # 171 tests across 6 files
|
||||
│ ├── test_bootstrap.py # 60 tests — init/update/hooks/memo/mailbox
|
||||
│ ├── test_json_handler.py # 35 tests — CRUD, validation, rotation
|
||||
│ ├── test_display.py # 28 tests — all display functions + routing
|
||||
│ ├── test_templates.py # 19 tests — operation templates + routing
|
||||
│ ├── test_init_project.py # 14 tests — command routing, error handling
|
||||
│ └── test_integration.py # 8 tests — main() flow, entry points
|
||||
├── cli_json/ # Auto-created JSON (config, data, log)
|
||||
├── logs/ # Branch-level logs
|
||||
└── .archive/ # Archived stubs (extensions/, json_templates/)
|
||||
```
|
||||
|
||||
**Two-tier design:**
|
||||
- `apps/modules/` — Public API. Import from here.
|
||||
- `apps/handlers/` — Internal implementation. Don't import directly.
|
||||
|
||||
## JSON Handler
|
||||
|
||||
Manages the three-file JSON pattern (config, data, log) for any module:
|
||||
|
||||
```python
|
||||
from aipass.cli.apps.handlers.json import json_handler
|
||||
|
||||
json_handler.log_operation("files_created", {"count": 12})
|
||||
data = json_handler.load_json("cli", "config")
|
||||
json_handler.save_json("cli", "data", {"key": "value"})
|
||||
json_handler.ensure_module_jsons("cli") # Create all 3 if missing
|
||||
```
|
||||
|
||||
## Integration Points
|
||||
|
||||
### Depends On
|
||||
- `aipass.prax` — Logging via `system_logger`
|
||||
- `rich` — Rich library for terminal formatting (Table, Panel, Columns, Text)
|
||||
- Python stdlib (`sys`, `importlib`, `pathlib`)
|
||||
- `rich` — Terminal formatting (Table, Panel, Text, Console)
|
||||
- Python stdlib (`sys`, `importlib`, `pathlib`, `json`)
|
||||
|
||||
### Cannot Import
|
||||
- `aipass.prax` — Circular dependency (prax depends on cli). Display/templates modules bypass this with documented entries in `.seedgo/bypass.json`.
|
||||
|
||||
### Provides To
|
||||
- All modules — display formatting (headers, success/error/warning, section breaks)
|
||||
- All modules — operation templates (`operation_start`, `operation_complete`)
|
||||
- All modules — Rich console access
|
||||
- All users — `aipass init` project bootstrap command
|
||||
- **All branches** — Display formatting (header, success, error, warning, fatal, section)
|
||||
- **All branches** — Operation templates (operation_start, operation_complete)
|
||||
- **All branches** — Rich console access
|
||||
- **All users** — `aipass init` project bootstrap + `aipass init update` refresh
|
||||
- **All users** — `aipass init agent` routing to spawn
|
||||
|
||||
## Entry Points
|
||||
|
||||
| Entry | Command | How |
|
||||
|-------|---------|-----|
|
||||
| drone | `drone @cli [command]` | Drone routes to `apps/cli.py:main()` |
|
||||
| Module | `python -m aipass.cli [args]` | `__main__.py` calls `main()` |
|
||||
| PATH | `aipass [args]` | `console_scripts` calls `cli_entry()` |
|
||||
|
||||
---
|
||||
|
||||
|
||||
+176
-46
@@ -2,8 +2,9 @@
|
||||
|
||||
# Drone
|
||||
|
||||
**Purpose:** Command router and symbolic addressing for AIPass. Resolves `@branch` names to paths at runtime via `AIPASS_REGISTRY.json`, routes commands to module entry points, and discovers available commands across the system.
|
||||
**Purpose:** Command router and symbolic addressing for AIPass. Resolves `@branch` names to paths at runtime via `AIPASS_REGISTRY.json`, routes commands to module entry points, manages git workflows, and discovers available commands across the system.
|
||||
**Module:** `aipass.drone`
|
||||
**Version:** 1.1.0
|
||||
**Created:** 2026-03-05
|
||||
|
||||
---
|
||||
@@ -13,8 +14,12 @@
|
||||
### What I Do
|
||||
- Resolve `@branch` symbolic names to absolute paths via `AIPASS_REGISTRY.json`
|
||||
- Route commands to registered branches and internal modules
|
||||
- Discover available commands across the system
|
||||
- Manage git workflows: PR creation, branch sync, lock management, merge
|
||||
- Discover and scan available commands across the system
|
||||
- Provide `drone systems` introspection of all registered components
|
||||
- Support external AIPass projects via dual registry lookup and module fallback
|
||||
|
||||
---
|
||||
|
||||
## Commands / Usage
|
||||
|
||||
@@ -23,23 +28,34 @@ Drone provides a CLI for terminal use and a Python API for programmatic access.
|
||||
### CLI
|
||||
|
||||
```bash
|
||||
drone systems # List all registered modules and branches
|
||||
# Core routing
|
||||
drone @seedgo audit aipass # Route "audit aipass" to seedgo
|
||||
drone @module --help # Show help for any module
|
||||
drone systems # List all registered modules and branches
|
||||
|
||||
# Git workflow
|
||||
drone @git pr "description" # Create a PR from current branch
|
||||
drone @git status # Git status scoped to branch directory
|
||||
drone @git sync # Pull latest main with --rebase
|
||||
drone @git sync --autostash # Sync with autostash for dirty trees
|
||||
drone @git lock / unlock # Atomic branch lockfile
|
||||
drone @git system-pr "desc" # System-wide PR (devpulse only)
|
||||
drone @git merge <PR#> # Squash-merge a PR
|
||||
|
||||
# Git workflow (devpulse-authorized only)
|
||||
drone @git system-pr "desc" # System-wide PR across all tracked changes
|
||||
drone @git merge <PR#> # Straight-merge a PR and sync local main
|
||||
drone @git smart-sync # Fetch + detect divergence + rebase
|
||||
drone @git fix # Auto-fix stuck rebase / detached HEAD
|
||||
drone @git fix --dry-run # Detect issues without fixing
|
||||
|
||||
# Command discovery
|
||||
drone scan @branch # Discover available commands in a branch
|
||||
drone activate @branch # Scan + register all commands from a branch
|
||||
drone activate @branch # Scan + register all commands as shortcuts
|
||||
drone list # List registered custom command shortcuts
|
||||
drone remove <name> # Remove a custom command shortcut
|
||||
|
||||
# Utilities
|
||||
drone hook-sounds on|off # Toggle hook notification sounds
|
||||
drone --version # Show version
|
||||
drone --version # Show version (v1.1.0)
|
||||
drone --help # Show usage information
|
||||
```
|
||||
|
||||
@@ -93,48 +109,103 @@ except CommandExecutionError as e:
|
||||
|
||||
## Architecture
|
||||
|
||||
### 3-Layer Pattern
|
||||
|
||||
```
|
||||
drone/
|
||||
├── cli.py # pip entry point (drone command)
|
||||
├── __init__.py # Public API exports
|
||||
├── cli.py # pip entry point (drone command)
|
||||
├── __init__.py # Public API exports (v1.1.0)
|
||||
├── apps/
|
||||
│ ├── drone.py # Core entry point
|
||||
│ ├── modules/ # Business logic
|
||||
│ │ ├── config.py # Registry path resolution
|
||||
│ │ ├── resolver.py # Branch resolution (@name -> path)
|
||||
│ │ ├── router.py # Command routing via subprocess
|
||||
│ │ ├── discovery.py # Module and command discovery
|
||||
│ │ ├── module_registry.py # Internal module routing
|
||||
│ │ ├── commands.py # Custom command shortcut orchestrator
|
||||
│ │ ├── git_module.py # Git workflow (PR, status, sync, lock)
|
||||
│ │ └── scan.py # Branch command scanning
|
||||
│ ├── handlers/ # Implementation
|
||||
│ │ ├── executor.py # Safe subprocess execution
|
||||
│ │ ├── exceptions.py # Exception hierarchy
|
||||
│ │ ├── generic_adapter.py # Centralized capture for external modules
|
||||
│ │ ├── routing_config.json # External module routing declarations
|
||||
│ │ ├── json/ # Three-JSON Pattern handler
|
||||
│ │ ├── scanning/ # Scan result formatting + discovery
|
||||
│ │ ├── command_registry/ # Command shortcut CRUD + lookup
|
||||
│ │ └── git/ # Git workflow handlers
|
||||
│ ├── drone.py # Core entry + CLI routing
|
||||
│ ├── modules/ # Orchestrators (business logic)
|
||||
│ │ ├── config.py # Registry path resolution
|
||||
│ │ ├── resolver.py # Branch resolution (@name → path)
|
||||
│ │ ├── router.py # Command routing via subprocess
|
||||
│ │ ├── discovery.py # Module and command discovery
|
||||
│ │ ├── module_registry.py # Internal module routing
|
||||
│ │ ├── registry.py # Registry query operations
|
||||
│ │ ├── commands.py # Custom command shortcut orchestrator
|
||||
│ │ ├── git_module.py # Git workflow (9 commands + plugin routing)
|
||||
│ │ └── scan.py # Branch command scanning
|
||||
│ ├── handlers/ # Implementation details
|
||||
│ │ ├── executor.py # Safe subprocess execution (timeout, no shell)
|
||||
│ │ ├── exceptions.py # Exception hierarchy (10 exception types)
|
||||
│ │ ├── router_handler.py # Routing implementation + caller detection
|
||||
│ │ ├── registry_handler.py # Registry file ops + dual registry lookup
|
||||
│ │ ├── discovery_handler.py # Discovery implementation + help parsing
|
||||
│ │ ├── module_registry_handler.py # Module loading (internal + external)
|
||||
│ │ ├── generic_adapter.py # StringIO capture for external modules
|
||||
│ │ ├── routing_config.json # External module declarations
|
||||
│ │ ├── json/
|
||||
│ │ │ └── json_handler.py # Structured operation logging
|
||||
│ │ ├── scanning/
|
||||
│ │ │ ├── scanner.py # Help parsing + modules/ file scanning
|
||||
│ │ │ └── formatters.py # Rich output for scan results
|
||||
│ │ ├── command_registry/
|
||||
│ │ │ ├── ops.py # Command shortcut CRUD
|
||||
│ │ │ ├── lookup.py # Greedy multi-word matching
|
||||
│ │ │ └── formatters.py # Rich output for command lists
|
||||
│ │ └── git/
|
||||
│ │ ├── lock_handler.py # Atomic lockfile (O_CREAT|O_EXCL)
|
||||
│ │ ├── pr_handler.py # 10-step PR workflow
|
||||
│ │ ├── status_handler.py # Scoped git status (subprocess)
|
||||
│ │ ├── status_handler_gitpython.py # [prototype] GitPython status — DPLAN-0140 Phase 1, not wired in
|
||||
│ │ └── sync_handler.py # Safe main sync
|
||||
│ └── plugins/ # Extensions beyond core routing
|
||||
│ └── devpulse_ops/ # System-wide PR, merge, smart-sync, fix
|
||||
├── docs/ # Documentation
|
||||
└── tests/ # 529+ tests, 19 test files
|
||||
│ │ ├── pr_handler.py # 10-step PR workflow with scoped staging
|
||||
│ │ ├── status_handler.py # Scoped git status (subprocess)
|
||||
│ │ ├── status_handler_gitpython.py # [prototype] DPLAN-0140 Phase 1, not wired in
|
||||
│ │ └── sync_handler.py # Safe main sync (--autostash support)
|
||||
│ └── plugins/
|
||||
│ ├── devpulse_ops/ # Privileged git operations (auth-gated)
|
||||
│ │ ├── auth.py # Passport-based identity gate (ALLOWED_CALLERS)
|
||||
│ │ ├── pr_plugin.py # System-wide PR (git add -A, system/ branches)
|
||||
│ │ ├── merge_plugin.py # PR merge (--merge) + local sync
|
||||
│ │ ├── sync_plugin.py # Smart sync (fetch, divergence detect, rebase)
|
||||
│ │ └── fix_plugin.py # Auto-fix stuck rebase / detached HEAD
|
||||
│ └── hook_sounds/
|
||||
│ └── hook_sounds_plugin.py # Toggle notification sounds on/off
|
||||
├── docs/ # Public documentation
|
||||
├── docs.local/ # Investigation reports and policies
|
||||
└── tests/ # 530 tests across 20 test files
|
||||
```
|
||||
|
||||
### Routing Flow
|
||||
|
||||
1. **CLI input** → `drone.py:main()`
|
||||
2. **Built-in commands** checked first: `systems`, `scan`, `activate`, `list`, `remove`, `hook-sounds`
|
||||
3. **`@target` routing** → branch resolution via `AIPASS_REGISTRY.json` → subprocess dispatch
|
||||
4. **Module fallback** → if branch not found but is a registered module, routes internally
|
||||
5. **Bare module names** → auto-discovered from `apps/modules/*.py`, routed via `importlib`
|
||||
6. **Custom commands** → greedy multi-word matching against `drone_command_registry.json`
|
||||
|
||||
### Module System
|
||||
|
||||
Drone routes to two kinds of modules:
|
||||
|
||||
| Type | Modules | Routing |
|
||||
|------|---------|---------|
|
||||
| Internal | `git` | `importlib` import → `handle_command()` |
|
||||
| External | `seedgo`, `cli`, `spawn` | `generic_adapter.capture_main()` via `routing_config.json` |
|
||||
|
||||
External modules are declared in `apps/handlers/routing_config.json` with entry points, descriptions, and versions.
|
||||
|
||||
### Git Main-Only Enforcement
|
||||
|
||||
All agents work on `main`. Branch creation is only allowed inside `drone @git system-pr`, which:
|
||||
1. Commits changes on main
|
||||
2. Moves branch pointer with `git branch -f` (HEAD stays on main)
|
||||
3. Pushes branch with `--force-with-lease`
|
||||
4. Opens PR via `gh`
|
||||
5. Returns to main
|
||||
|
||||
Enforcement layers:
|
||||
- `.claude/settings.json` deny rules block `git checkout -b`, `git switch -c`
|
||||
- `_assert_on_main_or_pr_flow()` guard in `git_module.py`
|
||||
- Persistent citizen branches: `citizen/{name}` reused across PRs
|
||||
|
||||
---
|
||||
|
||||
## Interactive Commands
|
||||
|
||||
By default, drone captures subprocess output (`capture_output=True`) with a 30s timeout. This is safe for AI-to-AI routing but strips Rich colors, buffers progress bars, and kills long-running commands.
|
||||
|
||||
Commands in the interactive tuple bypass capture and inherit the terminal directly — enabling live Rich output, colors, and no timeout. Only add commands here when the user needs full terminal experience.
|
||||
Commands in the interactive tuple bypass capture and inherit the terminal directly — enabling live Rich output, colors, and no timeout.
|
||||
|
||||
**Per-command allowlist** (in `apps/drone.py`):
|
||||
|
||||
@@ -142,6 +213,7 @@ Commands in the interactive tuple bypass capture and inherit the terminal direct
|
||||
|--------------|---------------------------------------------|
|
||||
| `monitor` | Prax real-time monitoring (live TUI) |
|
||||
| `audit` | Seedgo audit (Rich progress bars) |
|
||||
| `watchdog` | Devpulse watchdog (live monitoring) |
|
||||
|
||||
**Per-branch allowlist** — all commands from these branches get interactive mode:
|
||||
|
||||
@@ -149,32 +221,90 @@ Commands in the interactive tuple bypass capture and inherit the terminal direct
|
||||
|----------|-----------------------------------------------|
|
||||
| `cli` | User-facing CLI with Rich formatted output |
|
||||
|
||||
To add: edit `interactive_commands` or `interactive_branches` in `_handle_target()` in `apps/drone.py`.
|
||||
To add: edit `INTERACTIVE_COMMANDS` or `INTERACTIVE_BRANCHES` in `apps/drone.py`.
|
||||
|
||||
---
|
||||
|
||||
## Plugin System
|
||||
|
||||
Plugins live in `apps/plugins/{name}/` — outside the 3-layer structure by design.
|
||||
|
||||
### devpulse_ops
|
||||
|
||||
Auth-gated operations for system administration. `auth.py` walks CWD for `.trinity/passport.json` and checks `branch_name` against `ALLOWED_CALLERS` (devpulse, seedgo, spawn).
|
||||
|
||||
| Plugin | Command | Purpose |
|
||||
|--------|---------|---------|
|
||||
| `pr_plugin` | `system-pr` | System-wide PR across all tracked changes |
|
||||
| `merge_plugin` | `merge` | Straight-merge a PR and sync local main |
|
||||
| `sync_plugin` | `smart-sync` | Fetch + detect divergence + rebase |
|
||||
| `fix_plugin` | `fix` | Auto-fix stuck rebase / detached HEAD |
|
||||
|
||||
### hook_sounds
|
||||
|
||||
Simple toggle for hook notification sounds. Creates/removes `/tmp/aipass-hooks-muted` flag file.
|
||||
|
||||
---
|
||||
|
||||
## External Project Support
|
||||
|
||||
Infrastructure modules (seedgo, cli, git, spawn) work from external AIPass projects without per-project registration.
|
||||
|
||||
**Dual registry lookup:** `registry_handler.py` merges local project registry with `AIPASS_HOME` registry. Local entries win on name collision.
|
||||
|
||||
**Module fallback:** When subprocess routing fails (branch not in local registry), drone falls back to module routing for registered modules. Graceful degradation: Rich output from AIPass, functional output from external projects.
|
||||
|
||||
**AIPASS_HOME hints:** When `AIPASS_HOME` is not set and the local registry lacks core branches, drone shows setup hints:
|
||||
```
|
||||
Tip: set AIPASS_HOME=/path/to/AIPass to access all branches
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Integration Points
|
||||
|
||||
### Depends On
|
||||
- `AIPASS_REGISTRY.json` — Branch registry at repo root (read for resolution)
|
||||
- Python stdlib (`pathlib`, `sys`, `subprocess`, `json`)
|
||||
- `AIPASS_REGISTRY.json` — Branch registry (read for resolution)
|
||||
- `gh` CLI — GitHub operations (PR creation, merge)
|
||||
- Python stdlib (`pathlib`, `sys`, `subprocess`, `json`, `threading`)
|
||||
|
||||
### Provides To
|
||||
- All modules — command routing via `drone @target command`
|
||||
- All modules — branch/module discovery via `drone systems`
|
||||
- All branches — command routing via `drone @target command`
|
||||
- All branches — module/branch discovery via `drone systems`
|
||||
- External modules — `generic_adapter.capture_main()` for subprocess-free routing
|
||||
- `aipass.seedgo` — routed via `drone @seedgo`
|
||||
- `aipass.cli` — routed via `drone @cli`
|
||||
- `aipass.spawn` — routed via `drone @spawn`
|
||||
|
||||
---
|
||||
|
||||
## External Project Support
|
||||
## Testing
|
||||
|
||||
Infrastructure modules (seedgo, cli, git) work from external AIPass projects without per-project registration. When subprocess routing fails (branch not in local registry), drone falls back to module routing automatically. Graceful degradation: Rich output from AIPass, functional output from external projects.
|
||||
530 tests across 20 test files, covering all layers:
|
||||
|
||||
| Area | Files | Tests |
|
||||
|------|-------|-------|
|
||||
| Core routing | `test_resolver.py`, `test_router.py`, `test_activation.py` | ~128 |
|
||||
| Git operations | `test_git_module.py`, `test_system_pr.py`, `test_devpulse_plugins.py` | ~95 |
|
||||
| Handlers | `test_executor.py`, `test_registry_handler.py`, `test_discovery.py` | ~99 |
|
||||
| Infrastructure | `test_generic_adapter.py`, `test_module_registry.py`, `test_config.py` | ~66 |
|
||||
| Features | `test_commands.py`, `test_scan.py`, `test_hook_sounds.py`, `test_json_handler.py` | ~125 |
|
||||
| Standards | `test_cli_routing.py`, `test_contracts.py`, `test_error_resilience.py`, `test_init_provisioning.py` | ~21 |
|
||||
|
||||
Run tests: `cd src/aipass/drone && python -m pytest tests/ -q`
|
||||
|
||||
---
|
||||
|
||||
**Last Updated:** 2026-04-21
|
||||
## Known Issues
|
||||
|
||||
- `status_handler_gitpython.py` is an unreferenced prototype (DPLAN-0140 Phase 1) — awaiting Phase 2/3
|
||||
- `update_command()` and `command_exists()` in `ops.py` are tested CRUD API but unused from production
|
||||
- Pyright warns about `json` package name shadowing stdlib — works at runtime
|
||||
- Recurring sync errors when working tree is dirty — operational, not code bugs
|
||||
|
||||
---
|
||||
|
||||
**Seedgo:** 100% (34/34) | **Tests:** 530 pass, 4 skip | **Last Updated:** 2026-04-22
|
||||
|
||||
---
|
||||
[← Back to AIPass](../../../README.md)
|
||||
|
||||
|
||||
+81
-48
@@ -2,8 +2,9 @@
|
||||
|
||||
# Flow
|
||||
|
||||
**Purpose:** Unified plan lifecycle management for AIPass. Creates, tracks, closes, and archives numbered work plans across multiple plan types via a filesystem-driven template registry. Foreground archival with vector intake verification, cross-branch aggregation, and self-healing registries.
|
||||
**Purpose:** Unified plan lifecycle management for AIPass. Creates, tracks, closes, and archives numbered work plans across multiple plan types via a filesystem-driven template registry.
|
||||
**Module:** `aipass.flow`
|
||||
**Version:** 2.2.1
|
||||
**Created:** 2025-11-15
|
||||
**Last Updated:** 2026-04-22
|
||||
|
||||
@@ -11,18 +12,21 @@
|
||||
|
||||
## Overview
|
||||
|
||||
### What I Do
|
||||
- Create numbered plans from type-specific templates via `templates/` plugins
|
||||
- Unified create/close/list commands for all plan types (FPLAN, DPLAN, RPLAN, TDPLAN, ...)
|
||||
- Close plans with foreground archival to `.backup/processed_plans/`
|
||||
- Vector intake on close via `drone @memory process-plans` with chroma verification
|
||||
- List and filter plans across branches and plan types
|
||||
- Restore plans from backups
|
||||
- Template registry management: register, unregister, scan, auto-heal
|
||||
- Aggregate plans across branches
|
||||
- `--dry-run` for close preview
|
||||
Flow is AIPass's plan management system. Every branch uses flow to create, track, close, and archive work plans. Plans are numbered markdown files (`FPLAN-0042_subject_2026-04-22.md`) organized by type, with per-type registries tracking status and metadata.
|
||||
|
||||
## Commands / Usage
|
||||
### What I Do
|
||||
- Create numbered plans from type-specific templates
|
||||
- Close plans with foreground archival and vector intake verification
|
||||
- List and filter plans across all registered types
|
||||
- Restore closed plans from backups
|
||||
- Manage plan types via filesystem-driven template registry
|
||||
- Aggregate plans across branches for central reporting
|
||||
- Self-heal registries (orphan detection, auto-close missing files, auto-register new template dirs)
|
||||
- Preview close operations with `--dry-run`
|
||||
|
||||
---
|
||||
|
||||
## Commands
|
||||
|
||||
```bash
|
||||
# Create plans
|
||||
@@ -35,6 +39,7 @@ drone @flow close FPLAN-0042 # Close specific plan
|
||||
drone @flow close DPLAN-0005 # Close a DPLAN
|
||||
drone @flow close --all # Close all open plans
|
||||
drone @flow close --all --dry-run # Preview what would close
|
||||
drone @flow close --dry-run FPLAN-0042 # Preview single close
|
||||
|
||||
# List plans
|
||||
drone @flow list open # List open plans (all types)
|
||||
@@ -46,10 +51,16 @@ drone @flow scan # Find unregistered directories
|
||||
drone @flow register <dir> <PREFIX> # Register new plan type
|
||||
drone @flow unregister <dir> # Remove plan type
|
||||
|
||||
# Registry
|
||||
drone @flow registry scan # Scan filesystem, detect mismatches
|
||||
drone @flow registry status # Show registry health
|
||||
|
||||
# Other
|
||||
drone @flow restore FPLAN-0042 # Reopen a closed plan
|
||||
drone @flow aggregate # Cross-branch plan aggregation
|
||||
drone @flow post # Background post-close processing
|
||||
drone @flow --help # Full help
|
||||
drone @flow --version # Version string
|
||||
```
|
||||
|
||||
---
|
||||
@@ -60,92 +71,114 @@ drone @flow --help # Full help
|
||||
flow/
|
||||
├── apps/
|
||||
│ ├── flow.py # Entry point (auto-discovers modules)
|
||||
│ ├── modules/ # Business logic (thin orchestrators)
|
||||
│ ├── modules/ # Thin orchestrators (8 modules)
|
||||
│ │ ├── create_plan.py # Plan creation with template support
|
||||
│ │ ├── close_plan.py # Closure with foreground archival + vector verify
|
||||
│ │ ├── list_plans.py # Plan listing and filtering
|
||||
│ │ ├── restore_plan.py # Plan recovery from backups
|
||||
│ │ ├── registry_monitor.py # Orphan detection, auto-healing
|
||||
│ │ ├── registry_monitor.py # Registry scanning and auto-healing
|
||||
│ │ ├── aggregate_central.py # Cross-branch plan aggregation
|
||||
│ │ ├── post_close_runner.py # Background post-processing *(partial — archival moved to foreground)*
|
||||
│ │ ├── post_close_runner.py # Background post-processing with lock management
|
||||
│ │ └── template_manager.py # Template registry management
|
||||
│ └── handlers/ # Implementation details
|
||||
│ ├── plan/ # Lifecycle, file ops, validation, close_ops
|
||||
│ ├── registry/ # Load, save, auto-heal
|
||||
│ ├── template/ # Plan type loader + template resolution + registry_ops
|
||||
│ ├── dashboard/ # Status aggregation
|
||||
│ ├── mbank/ # Memory archival
|
||||
│ └── json/ # Auto-creating JSON handler
|
||||
├── templates/ # Plan type plugins (DATA, not code)
|
||||
│ ├── plan/ # Lifecycle: create, close, list, restore, display, validation
|
||||
│ ├── registry/ # Load, save, auto-heal registries
|
||||
│ ├── template/ # Plan type loader, template resolution, registry CRUD
|
||||
│ ├── dashboard/ # Status push to local, central, branch dashboards
|
||||
│ ├── mbank/ # Memory archival and plan processing
|
||||
│ ├── runner/ # Lock file operations for background processes
|
||||
│ ├── json/ # Auto-creating JSON handler
|
||||
│ ├── summary/ # Plan summarization (vestigial)
|
||||
│ ├── config/ # Configuration loading
|
||||
│ └── events/ # Event handling stubs
|
||||
├── templates/ # Plan type plugins (data, not code)
|
||||
│ ├── flow_plans/ # FPLAN templates (default, master)
|
||||
│ ├── dev_plans/ # DPLAN templates (default)
|
||||
│ ├── research_plans/ # RPLAN templates (default)
|
||||
│ ├── team_dev_plans/ # TDPLAN templates (default)
|
||||
│ └── audit_plans/ # Unregistered — needs `drone @flow register`
|
||||
│ └── audit_plans/ # APLAN templates (default)
|
||||
├── flow_json/ # Per-type registries + template_registry.json
|
||||
├── tests/ # 452 tests, 90/90 functions covered
|
||||
├── docs/ # Documentation
|
||||
├── tests/ # 423 tests, 17 test files
|
||||
└── .archive/ # Archived legacy code
|
||||
```
|
||||
|
||||
### Design Principles
|
||||
- **Modules are thin orchestrators** — no business logic, route to handlers and display results
|
||||
- **Handlers are stateless** — modules inject dependencies (registry loader, paths, config)
|
||||
- **Plan types are filesystem-driven** — drop a template dir, register a prefix, done
|
||||
- **Auto-discovery** — `flow.py` finds modules via `handle_command()` convention; `plan_type_loader.py` discovers types from `template_registry.json`
|
||||
|
||||
---
|
||||
|
||||
## Plan Types
|
||||
|
||||
Plan types are filesystem-driven. Drop a directory with `.md` templates into `templates/`, register it with a prefix. No per-directory JSON config needed.
|
||||
|
||||
| Type | Prefix | Registry | Templates |
|
||||
|------|--------|----------|-----------|
|
||||
| flow_plans | FPLAN | fplan_registry.json | default, master |
|
||||
| dev_plans | DPLAN | dplan_registry.json | default |
|
||||
| research_plans | RPLAN | rplan_registry.json | default |
|
||||
| team_dev_plans | TDPLAN | tdplan_registry.json | default |
|
||||
| audit_plans | APLAN | aplan_registry.json | default |
|
||||
|
||||
Plans follow the convention `{PREFIX}-{NNNN}_topic_slug_YYYY-MM-DD.md` where NNNN is auto-incrementing per type.
|
||||
Plans follow the naming convention `{PREFIX}-{NNNN}_topic_slug_YYYY-MM-DD.md` where NNNN auto-increments per type.
|
||||
|
||||
### Auto-heal
|
||||
- Template registry auto-prunes orphaned types (directory deleted → entry + plan registry JSON removed on next command)
|
||||
### Adding a New Plan Type
|
||||
1. Create a directory in `templates/` with one or more `.md` template files
|
||||
2. Run `drone @flow register <dirname> <PREFIX>` (or let auto-registration detect it on next command)
|
||||
3. Use `drone @flow create . "Subject" <shorthand>` to create plans of the new type
|
||||
|
||||
### Auto-healing
|
||||
- Template registry auto-prunes orphaned types (directory deleted → entry + plan registry JSON removed)
|
||||
- Plan registries auto-close entries for missing files
|
||||
- New template directories auto-register on next command
|
||||
|
||||
---
|
||||
|
||||
## Close Pipeline
|
||||
|
||||
On `drone @flow close`:
|
||||
1. Template check (fast-delete empty templates)
|
||||
2. Mark as closed in registry
|
||||
3. Archive to `.backup/processed_plans/` (foreground, sets processed/cleanup flags atomically)
|
||||
4. Vector intake: `drone @memory process-plans` + `is_plan_vectorized()` verification
|
||||
5. Dashboard updates (local + central + branch)
|
||||
6. Append to `CLOSED_PLANS.local.json`
|
||||
1. **Template check** — fast-delete empty/template-only plans
|
||||
2. **Mark closed** — update plan registry with closure timestamp
|
||||
3. **Archive** — move to `.backup/processed_plans/` (foreground, sets processed/cleanup flags atomically)
|
||||
4. **Vector intake** — `drone @memory process-plans` + `is_plan_vectorized()` verification
|
||||
5. **Dashboard updates** — local, central, and branch dashboards
|
||||
6. **Append** — write to `CLOSED_PLANS.local.json`
|
||||
|
||||
Vector verification displays in console: "Vectorized: N chunks in chroma" or "NOT vectorized".
|
||||
|
||||
---
|
||||
|
||||
## Integration Points
|
||||
|
||||
### Depends On
|
||||
- `aipass.cli` -- Terminal formatting (console, header, success, error)
|
||||
- `aipass.prax` -- Structured logging via `system_logger`
|
||||
- `aipass.trigger` -- Error reporting (optional)
|
||||
- `aipass.memory` -- Vector intake on plan close (`process-plans` + `verify`)
|
||||
- Python stdlib (`pathlib`, `json`, `importlib`, `sys`, `signal`, `subprocess`, `shutil`)
|
||||
- `aipass.cli` — Rich terminal formatting (`console`, `header`, `success`, `error`, `warning`)
|
||||
- `aipass.prax` — Structured logging via `system_logger`
|
||||
- `aipass.memory` — Vector intake on plan close
|
||||
- `aipass.trigger` — Error reporting (optional)
|
||||
|
||||
### Provides To
|
||||
- All branches -- Plan creation, tracking, closure, and archival
|
||||
- `aipass.devpulse` -- Plan status aggregation for system dashboards
|
||||
- Registry: Per-type registries in `flow_json/`
|
||||
- All branches — plan creation, tracking, closure, and archival
|
||||
- `aipass.devpulse` — plan status aggregation for system dashboards
|
||||
- Central reporting — `PLANS.central.json` via aggregate
|
||||
|
||||
---
|
||||
|
||||
## Quality
|
||||
|
||||
- **Seedgo:** 100% (all 33 standards)
|
||||
- **Tests:** 452 tests, 90/90 public functions covered
|
||||
- **Last audit:** 2026-04-09
|
||||
- **Seedgo:** 100% (33/33 standards)
|
||||
- **Tests:** 423 passed, 83/87 public functions tested (95%)
|
||||
- **Source files:** 39 tracked by seedgo
|
||||
- **Last audit:** 2026-04-22
|
||||
- **Battle test:** 16/16 commands pass via drone CLI (2026-04-22)
|
||||
|
||||
### Known Issues
|
||||
- Registry scan fires trigger events that are never handled (by design — foreground close handles everything)
|
||||
- Dashboard push warns on some closes
|
||||
- `mbank/process.py` at 669 lines (nearing 700 limit)
|
||||
|
||||
---
|
||||
|
||||
*Last Updated: 2026-04-07*
|
||||
*Last Updated: 2026-04-22*
|
||||
|
||||
---
|
||||
[← Back to AIPass](../../../README.md)
|
||||
|
||||
+128
-92
@@ -2,7 +2,7 @@
|
||||
|
||||
# MEMORY
|
||||
|
||||
**Purpose:** Central memory archive with semantic search, rollover, and archival across all AIPass branches.
|
||||
**Purpose:** Central memory archive — vector search, rollover, and memory management for all AIPass branches.
|
||||
**Module:** `aipass.memory`
|
||||
**Created:** 2026-03-07
|
||||
**Last Updated:** 2026-04-22
|
||||
@@ -12,53 +12,58 @@
|
||||
|
||||
## Overview
|
||||
|
||||
Memory is the central memory archive system that:
|
||||
- Provides semantic search across all branch memories
|
||||
- Archives memories when branches hit rollover limits (600 lines)
|
||||
- Extracts symbolic dimensions from conversations
|
||||
- Manages template distribution and line-count tracking across branches
|
||||
Memory is the archival backbone of AIPass. Every branch accumulates session history and learnings in `.trinity/` files. When those files reach capacity, Memory archives the oldest entries into ChromaDB vectors — searchable, permanent, never lost.
|
||||
|
||||
What Memory does:
|
||||
- **Rollover** — detects when `.trinity/local.json` or `observations.json` exceed limits, extracts oldest entries, embeds them via sentence-transformers, stores in ChromaDB, trims the source file
|
||||
- **Search** — semantic search across all archived branch memories (4+ collections, 2200+ vectors)
|
||||
- **Templates** — distributes `.trinity/` schema updates across all branches (push, diff, status)
|
||||
- **Symbolic** — fragmented memory extraction from conversations (demo, analyze, extract, fragments, bootstrap, hook-test)
|
||||
- **Verify** — checks whether a flow plan is vectorized in ChromaDB
|
||||
- **Watch** — persistent file watcher that auto-triggers rollover on changes
|
||||
|
||||
---
|
||||
|
||||
## Commands / Usage
|
||||
## Commands
|
||||
|
||||
All commands via `drone @memory <command>`:
|
||||
|
||||
**Via Drone (recommended):**
|
||||
```bash
|
||||
# Introspection
|
||||
drone @memory # Module list, version
|
||||
drone @memory --help # Full command reference
|
||||
drone @memory --version # Version string
|
||||
|
||||
# Rollover
|
||||
drone @memory rollover # Show rollover module introspection
|
||||
drone @memory rollover run # Execute memory rollover for files over limits
|
||||
drone @memory rollover status # Show rollover statistics for all branches
|
||||
drone @memory rollover check # Dry run — check which files need rollover
|
||||
drone @memory rollover sync-lines # Update line count metadata for all branches
|
||||
drone @memory rollover # Module introspection (handlers + subcommands)
|
||||
drone @memory rollover run # Execute rollover for files over limits
|
||||
drone @memory rollover status # Show per-branch rollover statistics
|
||||
drone @memory rollover check # Dry run — check what needs rollover
|
||||
drone @memory rollover sync-lines # Update line count metadata for all branches
|
||||
|
||||
# Search
|
||||
drone @memory search "error handling" # Semantic search across all branch memories
|
||||
drone @memory search "query" --branch SEEDGO # Filter search by branch
|
||||
drone @memory search "query" --n 10 # Limit number of results
|
||||
drone @memory search "error handling" # Semantic search across all branch memories
|
||||
drone @memory search "query" --branch X # Filter search by branch
|
||||
drone @memory search "query" --n 10 # Limit number of results
|
||||
|
||||
# Symbolic (fragmented memory)
|
||||
drone @memory symbolic # Show symbolic module introspection
|
||||
drone @memory symbolic demo # Run fragmented memory demonstration
|
||||
drone @memory symbolic fragments "query" # Search symbolic fragments (not operational — no stored fragments)
|
||||
drone @memory symbolic extract <file> # Extract fragments via LLM (requires API)
|
||||
# Symbolic
|
||||
drone @memory symbolic # Module introspection (6 handlers, subcommands)
|
||||
drone @memory symbolic demo # Run v1 + v2 mock analysis demonstration
|
||||
drone @memory symbolic fragments "query" # Search stored symbolic fragments
|
||||
drone @memory symbolic extract <file> # Extract fragments via LLM (requires API key)
|
||||
drone @memory symbolic bootstrap # Populate fragments from session JSONLs
|
||||
drone @memory symbolic hook-test # Test hook with sample conversation text
|
||||
|
||||
# Templates (not operational — template files missing)
|
||||
drone @memory templates # Show templates module introspection
|
||||
drone @memory templates push-templates # Push template updates to all branches (not operational)
|
||||
drone @memory templates diff-templates # Show template differences per branch (not operational)
|
||||
drone @memory templates template-status # Show template version and push status (not operational)
|
||||
# Templates
|
||||
drone @memory templates push-templates # Push template updates to all branches
|
||||
drone @memory templates diff-templates # Show template differences per branch
|
||||
drone @memory templates template-status # Show template version and push status
|
||||
|
||||
# Verify
|
||||
drone @memory verify FPLAN-XXXX # Check if a plan is vectorized in ChromaDB
|
||||
drone @memory verify FPLAN-XXXX # Check if plan is vectorized in ChromaDB
|
||||
|
||||
# Watch
|
||||
drone @memory watch # Start auto-rollover watcher (Ctrl+C to stop)
|
||||
```
|
||||
|
||||
**Direct execution:**
|
||||
```bash
|
||||
python3 -m aipass.memory.apps.memory search "query"
|
||||
python3 -m aipass.memory.apps.memory rollover status
|
||||
drone @memory watch # Auto-rollover watcher daemon (Ctrl+C to stop)
|
||||
```
|
||||
|
||||
---
|
||||
@@ -67,88 +72,119 @@ python3 -m aipass.memory.apps.memory rollover status
|
||||
|
||||
```
|
||||
memory/
|
||||
├── __init__.py # Package init
|
||||
├── README.md # This file
|
||||
├── DASHBOARD.local.json # System status dashboard
|
||||
├── pytest.ini # Test configuration
|
||||
├── .trinity/ # Identity & memory
|
||||
│ ├── passport.json # Branch identity
|
||||
│ ├── local.json # Session history (v2 schema, entry-count limits)
|
||||
│ └── observations.json # Collaboration patterns (v1 schema, line-count limits)
|
||||
├── .aipass/ # Branch prompt
|
||||
├── .ai_mail.local/ # Mailbox
|
||||
├── apps/
|
||||
│ ├── __init__.py
|
||||
│ ├── memory.py # Entry point (CLI) — auto-discovers modules
|
||||
│ ├── modules/
|
||||
│ │ ├── __init__.py
|
||||
│ │ ├── rollover.py # Rollover orchestrator — line checks, archival triggers
|
||||
│ │ ├── search.py # Search orchestrator — semantic query routing
|
||||
│ │ └── verify.py # Plan verification — check vectorized plan status
|
||||
│ ├── handlers/
|
||||
│ │ ├── __init__.py
|
||||
│ │ ├── central_writer.py # Central memory write operations
|
||||
│ │ ├── dashboard_push.py # Dashboard status push
|
||||
│ │ ├── archive/ # Memory archival indexing
|
||||
│ │ ├── json/ # JSON handler operations
|
||||
│ │ ├── learnings/ # Learning extraction
|
||||
│ │ ├── monitor/ # File watcher for auto-rollover
|
||||
│ │ ├── rollover/ # Rollover implementation logic
|
||||
│ │ ├── schema/ # Memory schema definitions
|
||||
│ │ ├── search/ # Search implementation (vector/semantic)
|
||||
│ │ ├── storage/ # Storage backend operations
|
||||
│ │ ├── tracking/ # Line count and metadata tracking
|
||||
│ │ ├── symbolic/ # Symbolic dimension extraction
|
||||
│ │ ├── templates/ # Template management
|
||||
│ │ └── vector/ # Vector DB (ChromaDB) operations
|
||||
│ ├── extensions/ # Extension plugins
|
||||
│ └── plugins/ # Plugin system
|
||||
├── artifacts/ # Build/output artifacts
|
||||
├── docs/ # Documentation
|
||||
├── memory_json/ # Memory JSON data store
|
||||
├── tests/ # Test suite
|
||||
└── tools/ # Utility scripts
|
||||
│ ├── memory.py # Entry point — auto-discovers modules via handle_command()
|
||||
│ ├── modules/ # Business logic (5 modules)
|
||||
│ │ ├── rollover.py # Rollover orchestration, status display, sync-lines
|
||||
│ │ ├── search.py # Semantic query routing
|
||||
│ │ ├── verify.py # Plan vectorization check
|
||||
│ │ ├── symbolic.py # Fragmented memory extraction and search
|
||||
│ │ └── templates.py # Template push, diff, status
|
||||
│ └── handlers/ # Implementation (14 handler groups, 35 files)
|
||||
│ ├── archive/ # indexer.py — memory archival indexing
|
||||
│ ├── intake/ # plans_processor.py, pool_processor.py — ingest pipelines
|
||||
│ ├── json/ # json_handler.py, memory_files.py — JSON operations
|
||||
│ ├── learnings/ # manager.py — learning extraction and management
|
||||
│ ├── monitor/ # detector.py, memory_watcher.py — rollover detection + auto-trigger
|
||||
│ ├── rollover/ # extractor.py, orchestrator.py — backup → extract → embed → store
|
||||
│ ├── schema/ # normalize.py — schema version normalization
|
||||
│ ├── search/ # query_executor.py, vector_search.py — semantic search impl
|
||||
│ ├── storage/ # chroma.py, chroma_subprocess.py — ChromaDB backend
|
||||
│ ├── symbolic/ # 6 handlers — chroma_client, deduplicator, extractor, hook, retriever, storage
|
||||
│ ├── templates/ # pusher.py, differ.py, spawn_pusher.py — template distribution
|
||||
│ ├── tracking/ # line_counter.py — metadata line count tracking
|
||||
│ ├── vector/ # embedder.py, embed_subprocess.py — sentence-transformer embeddings
|
||||
│ ├── central_writer.py # Central memory write operations
|
||||
│ └── dashboard_push.py # Dashboard status push
|
||||
├── config/ # memory_bank.config.json — per-branch rollover limits
|
||||
├── templates/ # LOCAL.template.json, OBS.template — schema templates
|
||||
├── tests/ # 450 tests (16/16 module coverage)
|
||||
├── .chroma/ # Global ChromaDB vector store
|
||||
└── memory_json/ # Operation log files (auto-created)
|
||||
```
|
||||
|
||||
### Rollover Pipeline
|
||||
|
||||
```
|
||||
startup trigger → check_and_rollover()
|
||||
→ detector.check_all_branches() # scan AIPASS_REGISTRY.json
|
||||
→ _should_rollover(file) # v1: line_count >= max_lines
|
||||
# v2: len(sessions) >= max_sessions, etc.
|
||||
→ orchestrator.execute_rollover()
|
||||
→ create_rollover_backup() # safety copy to branch/.backup/
|
||||
→ extract_items() # v2: max(excess, 1) oldest entries
|
||||
→ embed via subprocess # sentence-transformers in memory .venv
|
||||
→ store in ChromaDB # global + local collections
|
||||
→ trim source file # write back with oldest removed
|
||||
```
|
||||
|
||||
### Dual Schema Support
|
||||
|
||||
- **v1** (line-count): `schema_version: "1.0.0"` — triggers at `current_lines >= max_lines` (default 600)
|
||||
- **v2** (entry-count): `schema_version: "2.0.0"` — triggers at `len(sessions) >= max_sessions` or `len(key_learnings) >= max_key_learnings`. Extractor uses `max(excess, 1)` guard to prevent Python's `list[-0:]` trap.
|
||||
|
||||
### Subprocess Isolation
|
||||
|
||||
All ML operations (torch, sentence-transformers, chromadb) run via subprocess. The main process never imports these heavy libraries. Each embedding call spawns `memory/.venv/bin/python3` with a self-contained script that reads stdin JSON and writes stdout JSON.
|
||||
|
||||
---
|
||||
|
||||
## Integration Points
|
||||
## Dependencies
|
||||
|
||||
### Depends On
|
||||
- `rich` — Console output, panels, and tables
|
||||
- Python stdlib (`sys`, `time`, `signal`, `logging`, `pathlib`, `importlib`)
|
||||
### Runtime
|
||||
- `rich` — console output, panels, tables
|
||||
- Python stdlib (`json`, `pathlib`, `logging`, `importlib`, `signal`)
|
||||
- `prax` (internal) — logging via `get_system_logger()`
|
||||
|
||||
### ML (in memory `.venv/` only)
|
||||
- `torch` + `sentence-transformers` — embedding generation
|
||||
- `chromadb` — vector storage and semantic search
|
||||
- `numpy` — numerical operations
|
||||
|
||||
### Provides To
|
||||
- All branches — memory rollover, archival, and retrieval services
|
||||
- All branches — semantic search across branch memories
|
||||
- All branches — template distribution via `push-templates`
|
||||
- All branches — line count metadata via `sync-lines`
|
||||
- All branches — memory rollover and archival when `.trinity/` files hit limits
|
||||
- All branches — semantic search across archived memories
|
||||
- All branches — template schema distribution
|
||||
- All branches — line count metadata sync
|
||||
|
||||
---
|
||||
|
||||
## Key Modules
|
||||
## Quality
|
||||
|
||||
### rollover
|
||||
The rollover module monitors memory files across all branches registered in `AIPASS_REGISTRY.json`. When files exceed entry-count limits (20 sessions, 25 key_learnings), it triggers archival — extracting oldest entries, embedding them via subprocess, and storing in ChromaDB vectors. The `watch` command runs a persistent file watcher that auto-triggers rollover on changes.
|
||||
- **Seedgo:** 100% (33/33 standards) — maintained since s12
|
||||
- **Tests:** 450 pass, 1 skip (test_vector.py gated with `importorskip` for numpy)
|
||||
- **Coverage:** 175 public functions, 102 tested (58%), 16/16 modules
|
||||
- **Type checking:** 35 files, 0 errors
|
||||
|
||||
### search
|
||||
Semantic search across all branch memories using ChromaDB + sentence-transformers. Requires memory `.venv/` with torch installed (~3GB). All ML operations run via subprocess isolation — main process never imports torch.
|
||||
---
|
||||
|
||||
### symbolic *(partial)*
|
||||
Fragmented memory extraction and search. Demo and introspection work. `fragments` search returns 0 results (no stored fragments). `extract` requires API key. Code is operational but no fragment data has been stored yet.
|
||||
## Known Issues
|
||||
|
||||
### templates *(not operational)*
|
||||
Living template push system for distributing `.trinity/` schema updates across branches. Template files (`LOCAL.template.json`, `OBS.template`) are missing — all subcommands (`push-templates`, `diff-templates`, `template-status`) fail.
|
||||
|
||||
### verify
|
||||
Checks whether a specific flow plan (FPLAN-XXXX) has been vectorized into ChromaDB. Works correctly.
|
||||
- `search` requires torch/sentence-transformers in memory `.venv/` — fails without them
|
||||
- memory_watcher.py at 704 lines (near 700 threshold, bypassed in seedgo)
|
||||
- symbolic.py at 1604 lines (legacy port, bypassed)
|
||||
- manager.py at 1076 lines (complex learning extraction, bypassed)
|
||||
- `memory_threshold_exceeded` trigger event registered but never fired — rollover auto-runs at startup via watcher
|
||||
- `rollover status` shows 0 branches when `AIPASS_REGISTRY.json` path not resolved
|
||||
|
||||
---
|
||||
|
||||
## Identity
|
||||
|
||||
- **Passport:** `.trinity/passport.json`
|
||||
- **Session History:** `.trinity/local.json`
|
||||
- **Observations:** `.trinity/observations.json`
|
||||
- **Branch Prompt:** `.aipass/branch_system_prompt.md`
|
||||
- **Session History:** `.trinity/local.json` (v2 schema, 20 sessions max, 25 key_learnings max)
|
||||
- **Observations:** `.trinity/observations.json` (v1 schema, 600 lines max)
|
||||
- **Branch Prompt:** `.aipass/aipass_local_prompt.md`
|
||||
|
||||
---
|
||||
|
||||
*Last Updated: 2026-04-07*
|
||||
*Last Updated: 2026-04-22*
|
||||
|
||||
---
|
||||
[← Back to AIPass](../../../README.md)
|
||||
|
||||
@@ -63,5 +63,28 @@
|
||||
"date_closed": "2026-04-10",
|
||||
"location": "prax"
|
||||
}
|
||||
]
|
||||
],
|
||||
"document_metadata": {
|
||||
"version": "2.0.0",
|
||||
"schema_version": "2.0.0",
|
||||
"document_type": "session_history",
|
||||
"tags": [
|
||||
"session_tracking",
|
||||
"work_log",
|
||||
"PRAX"
|
||||
],
|
||||
"limits": {
|
||||
"max_sessions": 20,
|
||||
"max_key_learnings": 25,
|
||||
"session_summary_max_chars": 150,
|
||||
"learning_value_max_chars": 200,
|
||||
"note": "Auto-rollover to @memory when limits exceeded. Oldest entries trimmed first."
|
||||
},
|
||||
"status": {
|
||||
"health": "healthy",
|
||||
"last_health_check": "2026-04-22",
|
||||
"current_lines": 0
|
||||
}
|
||||
},
|
||||
"key_learnings": {}
|
||||
}
|
||||
|
||||
+159
-82
@@ -2,31 +2,20 @@
|
||||
|
||||
# PRAX
|
||||
|
||||
**Purpose:** System-wide logging, real-time monitoring, and dashboard for AIPass.
|
||||
**Purpose:** System-wide logging, real-time monitoring, and dashboard infrastructure for AIPass.
|
||||
**Module:** `aipass.prax`
|
||||
**Version:** 2.0.0
|
||||
**Last Updated:** 2026-04-22
|
||||
|
||||
---
|
||||
|
||||
## Overview
|
||||
|
||||
Prax auto-routes log output from any module to per-module log files and provides a live monitoring console (Mission Control) that shows file changes, log events, and command execution across all branches. Monitors Claude Code, Codex, and Gemini CLI sessions.
|
||||
Prax is the logging and monitoring backbone of the AIPass ecosystem. Any branch imports `logger` and gets automatic log routing — prax detects the caller via stack introspection and writes to the correct per-module log file. No configuration needed.
|
||||
|
||||
## Commands
|
||||
On top of logging, prax provides Mission Control (a real-time terminal console for file changes, log events, and agent activity), a log audit system, a dashboard infrastructure, and cross-branch STATUS.md synchronization.
|
||||
|
||||
```bash
|
||||
drone @prax monitor run # Launch Mission Control
|
||||
drone @prax status # System health status
|
||||
drone @prax log-audit audit # Audit log file sizes
|
||||
drone @prax dashboard # Show dashboard
|
||||
drone @prax --help # Full help
|
||||
```
|
||||
|
||||
## Usage
|
||||
|
||||
### Logging
|
||||
|
||||
**Canonical import (use this):**
|
||||
## Quick Start
|
||||
|
||||
```python
|
||||
from aipass.prax import logger
|
||||
@@ -36,9 +25,79 @@ logger.warning("Disk usage high")
|
||||
logger.error("Connection failed")
|
||||
```
|
||||
|
||||
This is Pattern A — the recommended way for all branches. Logs auto-route via two-tier placement: `system_logs/<branch>_<module>.log` (central aggregation) and `<branch>/logs/` (branch-local). No configuration needed — prax detects the caller via stack introspection.
|
||||
Logs auto-route via two-tier placement:
|
||||
- `system_logs/<branch>_<module>.log` — central aggregation at the repo root
|
||||
- `<branch>/logs/<module>.log` — branch-local debugging
|
||||
|
||||
For prax handlers that need to bypass the event pipeline (watchdog threads, import-chain files):
|
||||
## Commands
|
||||
|
||||
```bash
|
||||
drone @prax # Show discovered modules
|
||||
drone @prax --help # Full command list
|
||||
drone @prax --version # Version string
|
||||
```
|
||||
|
||||
### Monitor — Mission Control
|
||||
|
||||
```bash
|
||||
drone @prax monitor # Show monitor architecture
|
||||
drone @prax monitor run # Launch Mission Control (all branches)
|
||||
drone @prax monitor run seedgo,cli # Monitor specific branches
|
||||
drone @prax monitor --help # Monitor usage
|
||||
```
|
||||
|
||||
Real-time unified console showing:
|
||||
- File changes, log events, drone commands, agent activity
|
||||
- **Caller attribution** — `CALLER → TARGET` for drone commands
|
||||
- **Model tags** — `[BRANCH/model]` (e.g., `[DEVPULSE/opus]`, `[DEVPULSE/gpt-5.4]`)
|
||||
- **Multi-CLI** — Claude Code (JSONL), Codex (JSONL), Gemini (JSON) session monitoring
|
||||
- **Polling fallback** — automatic fallback when inotify watches are exhausted
|
||||
- **Soft start** — only shows new activity after launch (seeks to EOF on startup)
|
||||
|
||||
Interactive commands inside the monitor: `help`, `status`, `quit`/`exit`.
|
||||
|
||||
### Status
|
||||
|
||||
```bash
|
||||
drone @prax status # System health (modules, loggers, watcher state)
|
||||
drone @prax status sync # Build STATUS.md from all branch STATUS.local.md
|
||||
drone @prax status --help # Status usage
|
||||
```
|
||||
|
||||
### Log Audit
|
||||
|
||||
```bash
|
||||
drone @prax log-audit # Show audit module info
|
||||
drone @prax log-audit audit # Scan system_logs/ for health + oversized files
|
||||
drone @prax log-audit enforce # Truncate oversized logs to 1000 lines
|
||||
drone @prax log-audit --help # Audit usage
|
||||
```
|
||||
|
||||
### Dashboard
|
||||
|
||||
```bash
|
||||
drone @prax dashboard # Show dashboard sections
|
||||
drone @prax dashboard refresh --all # Refresh all branch dashboards from centrals
|
||||
drone @prax dashboard refresh @flow # Refresh a specific branch
|
||||
drone @prax dashboard status # Show dashboard status
|
||||
drone @prax dashboard push-template # Push template to all branches
|
||||
drone @prax dashboard diff-template # Diff template vs branch dashboards
|
||||
drone @prax dashboard --help # Dashboard usage
|
||||
```
|
||||
|
||||
## Logging API
|
||||
|
||||
### Pattern A — Canonical (use this)
|
||||
|
||||
```python
|
||||
from aipass.prax import logger
|
||||
|
||||
logger.info("Processing started")
|
||||
```
|
||||
|
||||
This works from any branch. Prax detects the caller via stack introspection and routes to the correct log file. If prax fails to import, a NullLogger fallback prevents crashes.
|
||||
|
||||
### Pattern B — Direct Logger (for prax internals)
|
||||
|
||||
```python
|
||||
from aipass.prax.apps.modules.logger import get_direct_logger
|
||||
@@ -47,94 +106,112 @@ logger = get_direct_logger()
|
||||
logger.info("Direct log entry")
|
||||
```
|
||||
|
||||
### Mission Control
|
||||
Use this in prax handler files that run in watchdog threads or sit in the import chain. Resolves module/branch at creation time, bypassing the runtime event pipeline.
|
||||
|
||||
Real-time monitoring console for watching system activity across all branches and CLI tools.
|
||||
### Programmatic Dashboard API
|
||||
|
||||
```bash
|
||||
drone @prax monitor run
|
||||
```python
|
||||
from aipass.prax.apps.modules.dashboard import write_section
|
||||
|
||||
write_section(branch_path, "ai_mail", {"new": 3, "total": 5})
|
||||
```
|
||||
|
||||
Features:
|
||||
- File changes, log events, drone commands, agent activity — all in one console
|
||||
- **Caller attribution** — shows `CALLER → TARGET` for drone commands
|
||||
- **Model tags** — shows `[BRANCH/model]` (e.g., `[DEVPULSE/opus]`, `[DEVPULSE/gpt-5.4]`)
|
||||
- **Multi-CLI** — monitors Claude Code (JSONL), Codex (JSONL), Gemini (JSON) sessions
|
||||
- **Polling fallback** — when inotify is exhausted, falls back to PollingObserver automatically
|
||||
- **Interactive filtering** *(not operational)* — `watch`, `filter` commands are deferred
|
||||
|
||||
Interactive commands inside the monitor:
|
||||
|
||||
```
|
||||
help # Show available commands
|
||||
status # Display current monitoring state
|
||||
quit/exit # Stop monitoring
|
||||
```
|
||||
|
||||
### CLI Commands
|
||||
|
||||
| Command | Description |
|
||||
|---------|-------------|
|
||||
| `drone @prax monitor run` | Launch Mission Control |
|
||||
| `drone @prax monitor` | Show monitor introspection |
|
||||
| `drone @prax status` | Show system status (modules, loggers, watcher state) |
|
||||
| `drone @prax status sync` | Sync STATUS.md from all branch STATUS.local.md |
|
||||
| `drone @prax log-audit audit` | Audit log file sizes and health |
|
||||
| `drone @prax log-audit enforce` | Truncate oversized logs |
|
||||
| `drone @prax dashboard` | Show system dashboard |
|
||||
| `drone @prax dashboard refresh --all` | Refresh dashboard data from centrals |
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
prax/
|
||||
├── __init__.py # Public API: exports `logger` (NullLogger fallback)
|
||||
├── apps/
|
||||
│ ├── prax.py # Entry point (CLI)
|
||||
│ ├── modules/
|
||||
│ │ ├── logger.py # SystemLogger (public API)
|
||||
│ │ ├── monitor.py # Mission Control
|
||||
│ │ ├── dashboard.py # System dashboard
|
||||
│ │ ├── status.py # System status / STATUS sync
|
||||
│ │ └── log_audit.py # Log file audit
|
||||
│ └── handlers/
|
||||
│ ├── central/ # Central file reader
|
||||
│ ├── config/ # Configuration loading
|
||||
│ ├── dashboard/ # Dashboard refresh and operations
|
||||
│ ├── discovery/ # Module scanning and filtering
|
||||
│ ├── json/ # JSON operations handler
|
||||
│ ├── json_templates/ # JSON template definitions
|
||||
│ ├── logging/ # Log setup, rotation, introspection
|
||||
│ ├── monitoring/ # Event queue, branch detection, stream output
|
||||
│ ├── registry/ # Module registry management
|
||||
│ ├── status/ # STATUS sync handler
|
||||
│ └── watcher/ # File and log watchers
|
||||
├── templates/ # Dashboard templates
|
||||
├── tests/ # Test suite (375 tests)
|
||||
└── tools/ # Standalone utilities (inbox_watchdog.py)
|
||||
│ ├── prax.py # Entry point — auto-discovers modules, routes commands
|
||||
│ ├── modules/ # Business logic (5 command modules)
|
||||
│ │ ├── logger.py # SystemLogger — auto-routing, two-tier logging
|
||||
│ │ ├── monitor.py # Mission Control — 3-thread real-time monitoring
|
||||
│ │ ├── dashboard.py # Dashboard — template management, refresh, write-through
|
||||
│ │ ├── status.py # System status — health display, STATUS.md sync
|
||||
│ │ └── log_audit.py # Log audit — scan, health summary, enforce limits
|
||||
│ └── handlers/ # Implementation details (11 handler directories)
|
||||
│ ├── central/ # Central file reader (.ai_central/*.central.json)
|
||||
│ ├── config/ # Path resolution, log config, ignore patterns
|
||||
│ ├── dashboard/ # Refresh, operations, template push/diff, agent status
|
||||
│ ├── discovery/ # Module scanning, filtering, file watcher for new .py
|
||||
│ ├── json/ # Auto-creating JSON handler (config/data/log per module)
|
||||
│ ├── json_templates/ # Default JSON templates for auto-creation
|
||||
│ ├── logging/ # Setup, rotation, introspection, override, direct logger
|
||||
│ ├── monitoring/ # Event queue, branch detector, stream output, log watcher
|
||||
│ ├── registry/ # Module registry load/save
|
||||
│ ├── status/ # STATUS.md sync handler
|
||||
│ └── watcher/ # Background system watchers
|
||||
├── prax_json/ # Auto-created per-module config/data/log files
|
||||
├── templates/ # Dashboard template schema (DASHBOARD.template.json)
|
||||
├── tests/ # 375 tests across 16 files
|
||||
└── tools/ # Standalone utilities (inbox_watchdog.py, verify_branch.py)
|
||||
```
|
||||
|
||||
### Design Pattern
|
||||
|
||||
The entry point (`prax.py`) has zero business logic — it auto-discovers modules in `apps/modules/` and routes commands. Each module is a thin orchestrator over its handlers. Handlers are never imported by external branches.
|
||||
|
||||
### Command Routing
|
||||
|
||||
```
|
||||
drone @prax monitor run
|
||||
→ prax.py discovers modules (glob apps/modules/*.py)
|
||||
→ calls monitor.handle_command("monitor", ["run"])
|
||||
→ monitor.py delegates to handlers/monitoring/*
|
||||
```
|
||||
|
||||
## How It Works
|
||||
|
||||
1. **Auto-routing** — When any module calls `logger.info()`, prax inspects the call stack to identify the caller and routes the log entry to the appropriate file.
|
||||
2. **Two-tier logging** — Each log entry goes to both `system_logs/` (central aggregation) and `<branch>/logs/` (branch-local), both with rotation.
|
||||
3. **Mission Control** — A multi-threaded monitoring console (display, file watcher, log watcher). Falls back to polling when inotify is exhausted. Shows caller attribution and model tags.
|
||||
4. **Multi-CLI monitoring** — Watches Claude Code JSONL, Codex JSONL, and Gemini JSON session files for agent activity (thinking, tool use, responses).
|
||||
5. **Dashboard** — Aggregates data from central files and branch status into per-branch dashboard views.
|
||||
1. **Auto-routing** — `logger.info()` inspects the call stack to identify the caller's module, branch, and file path, then routes the log entry to the correct per-module log file.
|
||||
2. **Two-tier logging** — Each log entry goes to both `system_logs/` (central, all branches) and `<branch>/logs/` (branch-local), both with size-based rotation.
|
||||
3. **Self-healing** — Auto-creates missing log directories, falls back to `system_logs/external/` for unknown modules, provides NullLogger if prax itself fails to import.
|
||||
4. **Mission Control** — Three threads: display worker (pulls from event queue), file watcher (watchdog on branch `apps/` dirs), log watcher (tails `system_logs/*.log`). Falls back to polling when inotify is exhausted.
|
||||
5. **Multi-CLI monitoring** — Watches Claude Code JSONL, Codex JSONL, and Gemini JSON session files. Extracts agent activity (thinking, tool use, responses) with model detection and branch resolution.
|
||||
6. **Dashboard** — Template-based per-branch dashboard files. Refreshes from central files (`*.central.json`). Write-through API for services to update sections directly.
|
||||
7. **STATUS sync** — Scans all branch `STATUS.local.md` files, extracts State/Last update fields, builds aggregated `STATUS.md` at the repo root.
|
||||
|
||||
---
|
||||
## Tests
|
||||
|
||||
375 tests across 16 files, covering all major components:
|
||||
|
||||
| Test File | Tests | Coverage |
|
||||
|-----------|-------|----------|
|
||||
| test_logger_module.py | 40 | Logger init, routing, lifecycle |
|
||||
| test_monitoring_filters.py | 39 | Event filtering rules |
|
||||
| test_config.py | 38 | Config loading, path resolution |
|
||||
| test_event_queue.py | 35 | Thread-safe event buffering |
|
||||
| test_log_watcher.py | 35 | Log file tailing |
|
||||
| test_logging.py | 33 | Core logging system |
|
||||
| test_discovery.py | 25 | Module scanning |
|
||||
| test_operations.py | 24 | Dashboard operations |
|
||||
| test_watcher.py | 23 | File watcher behavior |
|
||||
| test_registry.py | 22 | Module registry |
|
||||
| test_json_handler.py | 18 | JSON auto-creation |
|
||||
| test_central.py | 14 | Central reader |
|
||||
| test_monitor_module.py | 11 | Monitor commands |
|
||||
| test_log_audit.py | 10 | Log audit |
|
||||
| test_status.py | 8 | Status commands |
|
||||
|
||||
90/136 public functions tested (66%).
|
||||
|
||||
## Integration Points
|
||||
|
||||
### Depends On
|
||||
- `aipass.cli` — Console output, headers, success/error formatting
|
||||
- `aipass.drone` — Caller attribution via `[CALLER:BRANCH]` log markers
|
||||
- Python stdlib (`pathlib`, `importlib`, `argparse`, `logging`)
|
||||
- `aipass.trigger` — Optional event firing (module_discovered, error_detected)
|
||||
- `watchdog` — File system monitoring (inotify + polling fallback)
|
||||
- Python stdlib (`pathlib`, `logging`, `threading`, `argparse`, `importlib`)
|
||||
|
||||
### Provides To
|
||||
- All 11 branches — Unified logging via `from aipass.prax import logger`
|
||||
- All branches — Unified logging via `from aipass.prax import logger`
|
||||
- All branches — Real-time monitoring via Mission Control
|
||||
- System — STATUS.md sync, dashboard infrastructure
|
||||
- All branches — Per-branch dashboard files
|
||||
- System — `STATUS.md` sync, log audit enforcement
|
||||
|
||||
## Known Issues
|
||||
- **inotify exhaustion** — System often near `max_user_watches` limit. Monitor uses polling fallback (functional but slower).
|
||||
- **Interactive filtering deferred** — `watch`/`filter` commands in Mission Control are not operational.
|
||||
|
||||
---
|
||||
|
||||
|
||||
+165
-89
@@ -2,71 +2,94 @@
|
||||
|
||||
# SPAWN
|
||||
|
||||
**Purpose:** Branch creation, lifecycle management, and citizen class system. Creates new branches from class-scoped templates, manages updates, and grants citizenship.
|
||||
**Module:** `aipass.spawn`
|
||||
**Created:** 2026-03-05
|
||||
**The agent factory and branch lifecycle manager for AIPass.**
|
||||
|
||||
**Module:** `aipass.spawn` | **Version:** 1.0.0 | **Created:** 2026-03-05
|
||||
|
||||
---
|
||||
|
||||
## Overview
|
||||
## What I Do
|
||||
|
||||
### What I Do
|
||||
- Create new branches from class-scoped templates (builder, birthright)
|
||||
- Grant birthright citizenship via the `passport` command
|
||||
- Replace all `{{PLACEHOLDER}}` patterns with branch-specific values
|
||||
- Update branches from templates (single or batch by class)
|
||||
- Delete (archive + deregister) branches
|
||||
- Update branches from templates (single or batch by class, with --dry-run)
|
||||
- Delete branches (archive + deregister)
|
||||
- Sync registry and templates against filesystem
|
||||
- Regenerate `.template_registry.json` with fresh file hashes
|
||||
- Validate no unreplaced placeholders remain
|
||||
- Regenerate template registries with fresh file hashes
|
||||
- Replace all `{{PLACEHOLDER}}` patterns with branch-specific values
|
||||
- Register new citizens in `AIPASS_REGISTRY.json`
|
||||
|
||||
### Citizen Classes
|
||||
---
|
||||
|
||||
Branches are scoped by **citizen class**, which determines the template used:
|
||||
## Citizen Classes
|
||||
|
||||
| Class | Template | What it creates |
|
||||
Every branch belongs to a **citizen class**, which determines its template:
|
||||
|
||||
| Class | Template | What It Creates |
|
||||
|-------|----------|-----------------|
|
||||
| `builder` (default) | `templates/builder/` | Full 3-layer scaffold: apps/, modules/, handlers/ |
|
||||
| `builder` (default) | `templates/builder/` | Full 3-layer scaffold: .trinity/, .aipass/, apps/ (modules/ + handlers/), tests/, docs/, logs/ |
|
||||
| `birthright` | `templates/birthright/` | Minimal citizenship: .trinity/, .aipass/, README.md |
|
||||
|
||||
### Usage
|
||||
---
|
||||
|
||||
## Commands
|
||||
|
||||
All commands run through `drone @spawn <command>`.
|
||||
|
||||
### Create
|
||||
|
||||
**Create a builder branch (full scaffold):**
|
||||
```bash
|
||||
drone @spawn create /path/to/new/agent
|
||||
drone @spawn create builder /path/to/new/agent --role "Analyst" --purpose "Reports"
|
||||
drone @spawn create <path> # Create builder branch
|
||||
drone @spawn create <path> --role "Analyst" --purpose "Reports" # With identity
|
||||
drone @spawn create --template birthright <path> # Specific class
|
||||
drone @spawn create <path> --dry-run # Preview without touching disk
|
||||
drone @spawn create @existing # Adopt pre-existing agent
|
||||
drone @spawn create ~/Projects/MyProject/agent_name # External project (auto-detects registry)
|
||||
```
|
||||
|
||||
**Grant birthright citizenship (minimal identity):**
|
||||
### Passport
|
||||
|
||||
```bash
|
||||
drone @spawn passport @new_branch --role "Observer" --purpose "Monitoring"
|
||||
drone @spawn passport @dirname # Grant birthright citizenship
|
||||
drone @spawn passport @dirname --role "Observer" --purpose "Monitoring"
|
||||
```
|
||||
|
||||
**Update branches from template:**
|
||||
### Update
|
||||
|
||||
```bash
|
||||
drone @spawn update @branch_name # Single branch (uses its passport class)
|
||||
drone @spawn update builder --all # All builder-class branches
|
||||
drone @spawn update birthright --all # All birthright-class branches
|
||||
drone @spawn update --dry-run @branch_name # Preview changes
|
||||
drone @spawn update @branch_name # Single branch (uses passport class)
|
||||
drone @spawn update builder --all # All builder-class branches
|
||||
drone @spawn update birthright --all # All birthright-class branches
|
||||
drone @spawn update @branch_name --dry-run # Preview changes
|
||||
drone @spawn update builder --all --dry-run # Preview batch update
|
||||
```
|
||||
|
||||
**Delete, sync:**
|
||||
### Delete
|
||||
|
||||
```bash
|
||||
drone @spawn delete @branch_name # Archive + deregister
|
||||
drone @spawn sync-registry # Repair registry vs filesystem
|
||||
drone @spawn sync-registry --fix # Repair + rebuild missing .spawn/ tracking
|
||||
drone @spawn sync-templates # Pull managed files from sources *(partial — template_owners.json empty)*
|
||||
drone @spawn regenerate-registry # Regenerate builder template registry
|
||||
drone @spawn regenerate-registry --all # Regenerate all template class registries
|
||||
drone @spawn delete @branch_name # Archive + deregister
|
||||
```
|
||||
|
||||
**External project support:**
|
||||
### Sync and Regenerate
|
||||
|
||||
```bash
|
||||
# Creating inside an existing AIPass project auto-detects the project registry
|
||||
drone @spawn create ~/Projects/MyProject/agent_name # Registers in MYPROJECT_REGISTRY.json
|
||||
drone @spawn sync-registry # Report healthy/stale/unregistered
|
||||
drone @spawn sync-registry --fix # Rebuild .spawn/ tracking + fix passport registry_ids
|
||||
drone @spawn sync-templates # Pull managed files from sources (partial — see Known Issues)
|
||||
drone @spawn regenerate-registry # Regenerate builder template hashes
|
||||
drone @spawn regenerate-registry --all # All template classes
|
||||
```
|
||||
|
||||
**Python API:**
|
||||
### Introspection
|
||||
|
||||
```bash
|
||||
drone @spawn # No args — lists connected modules
|
||||
drone @spawn --help # Full help text
|
||||
drone @spawn --version # Version string
|
||||
```
|
||||
|
||||
### Python API
|
||||
|
||||
```python
|
||||
from aipass.spawn import spawn_agent
|
||||
|
||||
@@ -76,101 +99,154 @@ result = spawn_agent(
|
||||
purpose="Process incoming reports",
|
||||
traits="Precise, thorough"
|
||||
)
|
||||
# result dict includes: success, branch_name, path, files_copied, validation_issues
|
||||
# Returns: { success, branch_name, path, files_copied, validation_issues }
|
||||
```
|
||||
|
||||
### Known Limitations
|
||||
|
||||
- `update --help` and `delete --help` fall through to argparse instead of showing help *(not operational — argparse `add_help=False` swallows --help before module intercept)*
|
||||
- `sync-templates` runs but has no managed files *(partial — template_owners.json is empty)*
|
||||
- `.py` files are never auto-updated by `update` — requires manual review by design
|
||||
|
||||
---
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
spawn/
|
||||
├── __init__.py # Public API (exports spawn_agent)
|
||||
├── __init__.py # Public API (exports spawn_agent)
|
||||
├── apps/
|
||||
│ ├── spawn.py # Entry point (CLI)
|
||||
│ ├── spawn.py # Entry point — CLI routing, version, help
|
||||
│ ├── modules/
|
||||
│ │ ├── core.py # Create orchestrator — coordinates spawn steps
|
||||
│ │ ├── passport.py # Passport CLI — grant birthright citizenship
|
||||
│ │ ├── update.py # Update CLI — parses args, delegates to handler
|
||||
│ │ ├── delete.py # Delete CLI — parses args, delegates to handler
|
||||
│ │ ├── sync_registry.py # Sync registry CLI — report and repair
|
||||
│ │ ├── sync_templates.py # Sync templates CLI — pull from sources
|
||||
│ │ └── regenerate_registry.py # Regenerate template registries CLI
|
||||
│ │ ├── core.py # Create orchestrator (_spawn_agent, handle_command)
|
||||
│ │ ├── passport.py # Passport CLI — birthright citizenship
|
||||
│ │ ├── update.py # Update CLI — single/batch by class
|
||||
│ │ ├── delete.py # Delete CLI — archive + deregister
|
||||
│ │ ├── sync_registry.py # Registry repair CLI
|
||||
│ │ ├── sync_templates.py # Template sync CLI
|
||||
│ │ └── regenerate_registry.py # Template registry regeneration CLI
|
||||
│ └── handlers/
|
||||
│ ├── class_registry.py # Citizen class registry — maps classes to templates
|
||||
│ ├── passport_ops.py # Passport grant implementation
|
||||
│ ├── file_ops.py # Template copy, path renaming
|
||||
│ ├── metadata.py # Branch name extraction, profile detection
|
||||
│ ├── placeholders.py # {{PLACEHOLDER}} replacement engine
|
||||
│ ├── registry.py # AIPASS_REGISTRY.json CRUD
|
||||
│ ├── json_ops.py # JSON read/write operations
|
||||
│ ├── json/
|
||||
│ │ └── json_handler.py # JSON I/O abstraction
|
||||
│ ├── meta_ops.py # Branch metadata generation
|
||||
│ ├── change_detection.py # File diff detection
|
||||
│ ├── reconcile.py # Registry/filesystem reconciliation
|
||||
│ ├── update_ops.py # Update implementation (class-aware)
|
||||
│ ├── delete_ops.py # Delete implementation logic
|
||||
│ ├── sync_registry_ops.py # Registry sync implementation
|
||||
│ ├── sync_templates_ops.py # Template sync implementation
|
||||
│ └── regenerate_registry_ops.py # Template registry regeneration
|
||||
│ ├── class_registry.py # Citizen class → template directory mapping
|
||||
│ ├── file_ops.py # Template copy, path renaming, registry regeneration
|
||||
│ ├── metadata.py # Branch name extraction, profile detection
|
||||
│ ├── placeholders.py # {{PLACEHOLDER}} replacement engine
|
||||
│ ├── registry.py # AIPASS_REGISTRY.json CRUD, find_registry()
|
||||
│ ├── meta_ops.py # Branch metadata generation, hash computation
|
||||
│ ├── change_detection.py # ID-based file diff between template and branch
|
||||
│ ├── reconcile.py # Registry/filesystem reconciliation
|
||||
│ ├── passport_ops.py # Passport grant implementation
|
||||
│ ├── update_ops.py # Update workflow (Phase 0 snapshot → detect → execute)
|
||||
│ ├── delete_ops.py # Delete workflow (resolve → archive → cleanup → deregister)
|
||||
│ ├── sync_registry_ops.py # Registry sync (CWD-aware, external project support)
|
||||
│ ├── sync_templates_ops.py # Template sync implementation
|
||||
│ ├── regenerate_registry_ops.py # Template registry hash regeneration
|
||||
│ ├── json_ops.py # JSON deep merge, backup utilities
|
||||
│ └── json/
|
||||
│ └── json_handler.py # Standard JSON I/O, operation logging, 7 API functions
|
||||
├── templates/
|
||||
│ ├── builder/ # Full scaffold template (apps/, modules/, handlers/)
|
||||
│ ├── birthright/ # Minimal template (.trinity/, .aipass/, README.md)
|
||||
│ ├── builder/ # Full scaffold template (45 files, 24 dirs)
|
||||
│ ├── birthright/ # Minimal template
|
||||
│ └── .archive/
|
||||
│ └── agent_mock_branch/ # Reference implementation
|
||||
├── tools/ # Branch verification utilities
|
||||
├── docs/ # Documentation
|
||||
├── spawn_json/ # JSON tracking directory
|
||||
└── tests/ # Test suite
|
||||
│ └── agent_mock_branch/ # Reference implementation
|
||||
├── tests/ # 13 test files, 253 tests
|
||||
├── spawn_json/ # JSON tracking directory
|
||||
├── tools/ # Branch verification utilities
|
||||
├── docs/ # Documentation
|
||||
└── logs/ # Prax log output
|
||||
```
|
||||
|
||||
### Three-Layer Design
|
||||
|
||||
1. **Entry point** (`spawn.py`) — Routes CLI commands, never imports handlers directly
|
||||
2. **Modules** (`modules/`) — Business logic coordinators, parse arguments, delegate to handlers
|
||||
3. **Handlers** (`handlers/`) — Implementation details, pure functions where possible
|
||||
|
||||
---
|
||||
|
||||
## Spawn Workflow
|
||||
## Workflows
|
||||
|
||||
### Create (builder class)
|
||||
|
||||
1. **Resolve** — Extract branch name from target path, validate path doesn't exist
|
||||
2. **Lookup** — Resolve citizen class to template directory via class_registry
|
||||
3. **Copy** — Recursive copy of class template to target (skips `__pycache__`)
|
||||
4. **Rename** — Replace `{{BRANCH}}` in directory and file names
|
||||
5. **Replace** — Substitute all `{{PLACEHOLDER}}` patterns in file contents
|
||||
6. **Registry** — Regenerate `.template_registry.json`, register in `AIPASS_REGISTRY.json`
|
||||
6. **Registry** — Generate `.branch_meta.json`, register in `AIPASS_REGISTRY.json`
|
||||
7. **Validate** — Scan for any remaining `{{...}}` patterns
|
||||
|
||||
### Passport (birthright class)
|
||||
|
||||
1. **Check** — Verify .trinity/ doesn't already exist
|
||||
2. **Copy** — Copy birthright template (.trinity/, .aipass/, README.md)
|
||||
3. **Replace** — Substitute placeholders in copied files
|
||||
4. **Register** — Add to AIPASS_REGISTRY.json
|
||||
|
||||
### Update (class-aware)
|
||||
- Single branch: reads `citizen_class` from passport, uses matching template
|
||||
- Batch: `builder --all` or `birthright --all` filters by class
|
||||
- `--all` without a class is blocked (safety)
|
||||
### Update (class-aware, Phase 0)
|
||||
|
||||
1. **Snapshot** — Back up current `.branch_meta.json` and `.template_registry.json`
|
||||
2. **Detect** — Compare branch files against template via ID-based change detection
|
||||
3. **Execute** — Apply renames, additions, JSON merges (`.py` files skipped by design)
|
||||
4. **Refresh** — Regenerate `.branch_meta.json` with current state
|
||||
|
||||
### Adopt Existing (`create @existing`)
|
||||
|
||||
1. **Fix** — Repair `registry_id` in passport if stale (from registry recreation)
|
||||
2. **Register** — Add to project registry
|
||||
3. **Update** — Run template update to sync scaffold files
|
||||
|
||||
---
|
||||
|
||||
## Integration Points
|
||||
## Tests
|
||||
|
||||
**253 tests | 0 skipped | 0 failed** across 13 test files:
|
||||
|
||||
| File | Focus |
|
||||
|------|-------|
|
||||
| `test_lifecycle.py` | End-to-end spawn lifecycle workflows |
|
||||
| `test_json_handler.py` | JSON I/O, operation logging, standard API |
|
||||
| `test_handlers.py` | Handler function behavior and integration |
|
||||
| `test_regenerate_registry_ops.py` | Template registry regeneration |
|
||||
| `test_update.py` | Branch update mechanics |
|
||||
| `test_citizen_classes.py` | Citizen class validation and template discovery |
|
||||
| `test_file_ops.py` | File copy, rename, placeholder replacement |
|
||||
| `test_cli_routing.py` | Command routing and argument parsing |
|
||||
| `test_contracts.py` | Handler contracts and interface compliance |
|
||||
| `test_spawn.py` | Basic CLI routing and help |
|
||||
| `test_error_resilience.py` | Error handling and edge cases |
|
||||
| `conftest.py` | Fixtures: mock templates, registry protection |
|
||||
|
||||
**Public functions:** 45 total, 41 tested (91%)
|
||||
|
||||
---
|
||||
|
||||
## Integration
|
||||
|
||||
### Depends On
|
||||
- `aipass.prax` — Logging via `system_logger`
|
||||
- `aipass.cli` — Console output and headers
|
||||
- Python stdlib (`pathlib`, `json`, `shutil`, `hashlib`, `re`)
|
||||
|
||||
- **aipass.prax** — Logging via `system_logger`
|
||||
- **aipass.cli** — Console output (header, error, warning)
|
||||
- Python stdlib (`pathlib`, `json`, `shutil`, `hashlib`, `re`, `argparse`)
|
||||
|
||||
### Provides To
|
||||
- All modules — branch creation, lifecycle management, citizenship
|
||||
- Registry: Reads/writes `AIPASS_REGISTRY.json`
|
||||
|
||||
- All branches — creation, template updates, registry management, citizenship
|
||||
- Registry: CRUD operations on `AIPASS_REGISTRY.json` and `*_REGISTRY.json`
|
||||
|
||||
---
|
||||
|
||||
## Known Issues
|
||||
|
||||
- `sync-templates` is a no-op — `template_owners.json` has no entries (template IS source of truth, not downstream consumer)
|
||||
- `.py` files never auto-update during `drone @spawn update` (by design) — template .py changes need individual branch dispatch
|
||||
- 4 untested public functions remain (45 total, 41 tested)
|
||||
|
||||
---
|
||||
|
||||
## Metrics
|
||||
|
||||
- **Seedgo:** 100% (34/34)
|
||||
- **Tests:** 253 passed, 0 skipped, 0 failed
|
||||
- **Module coverage:** 23/23 (100%)
|
||||
- **Template registry:** 45 files, 24 dirs (builder)
|
||||
- **Battle test:** 17/17 commands pass (2026-04-22)
|
||||
|
||||
---
|
||||
|
||||
*Last Updated: 2026-04-22*
|
||||
|
||||
---
|
||||
[← Back to AIPass](../../../README.md)
|
||||
|
||||
+139
-64
@@ -2,37 +2,46 @@
|
||||
|
||||
# Trigger
|
||||
|
||||
**Purpose:** Event bus for AIPass. Branches fire events, registered handlers react. Decouples producers from consumers — the module that detects a condition doesn't need to know what should happen next.
|
||||
**Purpose:** Event bus and error dispatch for AIPass. Branches fire events, registered handlers react. Medic watches logs for errors, fingerprints them, gates dispatch through an 8-stage pipeline, and notifies the responsible branch.
|
||||
**Module:** `aipass.trigger`
|
||||
**Version:** 2.2.0
|
||||
**Last Updated:** 2026-04-22
|
||||
|
||||
## Commands / Usage
|
||||
## Commands
|
||||
|
||||
```bash
|
||||
drone @trigger fire <event> [data] # Fire an event
|
||||
drone @trigger list # List registered events
|
||||
drone @trigger status # Event bus status
|
||||
drone @trigger --help # Full help
|
||||
drone @trigger # Introspection (modules, version)
|
||||
drone @trigger --help # Full command listing
|
||||
drone @trigger --version # Version string
|
||||
|
||||
# Event bus
|
||||
drone @trigger fire <event> [key=val ...] # Fire an event with optional data
|
||||
drone @trigger list # List all registered events + handlers
|
||||
drone @trigger status # Event bus and medic state
|
||||
|
||||
# Error registry
|
||||
drone @trigger errors list # View tracked errors
|
||||
drone @trigger errors stats # Registry stats + circuit breaker
|
||||
drone @trigger errors circuit-breaker # Circuit breaker state
|
||||
drone @trigger errors detail <fingerprint> # Single error detail
|
||||
drone @trigger errors --help # Error subcommand help
|
||||
|
||||
# Medic (error dispatch control)
|
||||
drone @trigger medic on # Enable auto-dispatch
|
||||
drone @trigger medic off # Disable auto-dispatch
|
||||
drone @trigger medic status # Medic state + suppression stats
|
||||
drone @trigger medic mute @branch # Suppress dispatch to a branch
|
||||
drone @trigger medic unmute @branch # Resume dispatch to a branch
|
||||
drone @trigger medic --help # Medic subcommand help
|
||||
|
||||
# Log watchers
|
||||
drone @trigger branch_log_events status # Branch log watcher state
|
||||
drone @trigger branch_log_events --help # Branch watcher help
|
||||
drone @trigger log_events status # System log watcher state
|
||||
drone @trigger log_events --help # System watcher help
|
||||
```
|
||||
|
||||
## Usage
|
||||
|
||||
### CLI (via drone)
|
||||
|
||||
```bash
|
||||
drone @trigger --help # Show available commands
|
||||
drone @trigger medic on # Enable error auto-dispatch
|
||||
drone @trigger medic off # Disable error auto-dispatch
|
||||
drone @trigger medic status # Show medic state
|
||||
drone @trigger medic mute @branch # Suppress errors from a branch
|
||||
drone @trigger medic unmute @branch # Resume errors from a branch
|
||||
drone @trigger errors list # List tracked errors
|
||||
drone @trigger errors stats # Error registry statistics
|
||||
drone @trigger errors circuit-breaker # Circuit breaker state
|
||||
drone @trigger --version # Show version
|
||||
```
|
||||
|
||||
### Python API
|
||||
## Python API
|
||||
|
||||
```python
|
||||
from aipass.trigger.apps.modules.core import Trigger
|
||||
@@ -41,20 +50,19 @@ from aipass.trigger.apps.modules.core import Trigger
|
||||
Trigger.fire("plan_file_created", path="/path/to/FPLAN-0042.md")
|
||||
|
||||
# Register a handler
|
||||
def on_plan_file_created(**data):
|
||||
def on_plan_created(**data):
|
||||
print(f"Plan created at {data['path']}")
|
||||
|
||||
Trigger.on("plan_file_created", on_plan_file_created)
|
||||
Trigger.on("plan_file_created", on_plan_created)
|
||||
|
||||
# Remove a handler
|
||||
Trigger.off("plan_file_created", on_plan_file_created)
|
||||
Trigger.off("plan_file_created", on_plan_created)
|
||||
```
|
||||
|
||||
### Cross-branch error reporting
|
||||
|
||||
```python
|
||||
from aipass.trigger.apps.modules.errors import report_error
|
||||
|
||||
# Cross-branch error reporting
|
||||
result = report_error(
|
||||
branch="api",
|
||||
error_type="ConnectionError",
|
||||
@@ -66,66 +74,133 @@ result = report_error(
|
||||
|
||||
## Events
|
||||
|
||||
14 events registered via `handlers/events/registry.py`. All fire through `Trigger.fire()`.
|
||||
14 events registered via `handlers/events/registry.py` on first `Trigger.fire()`. All fire through the event bus.
|
||||
|
||||
| Event | Handler | Fired when | Action |
|
||||
|-------|---------|------------|--------|
|
||||
| `startup` | `startup.py` | Branch session starts | Error catch-up scan, memory rollover check |
|
||||
| `error_detected` | `error_detected.py` | Error registered in log watcher (Medic v2) | 8-gate dispatch pipeline, sends fix-it email to affected branch |
|
||||
| `error_logged` | `error_logged.py` | Error detected in system logs (legacy) | Rate-limited notification with medic gating |
|
||||
| `warning_logged` | `warning_logged.py` | Warning detected in system logs | Logged for monitoring, no dispatch |
|
||||
| `plan_file_created` | `plan_file.py` | New PLAN file detected in filesystem | Updates Flow's PLAN_REGISTRY.json |
|
||||
| `plan_file_deleted` | `plan_file.py` | PLAN file removed from filesystem | Marks plan as deleted in registry |
|
||||
| `plan_file_moved` | `plan_file.py` | PLAN file moved or renamed | Updates registry location |
|
||||
| `bulletin_created` | `bulletin_created.py` | New system bulletin posted | Propagates to all branch dashboards |
|
||||
| `memory_threshold_exceeded` | `memory_threshold_exceeded.py` | Memory file approaches line limit (600 lines) | Sends compression notification to branch |
|
||||
| `memory_template_updated` | `memory_template_updated.py` | Memory template modified | Pushes template updates to branches |
|
||||
| `memory_saved` | `memory.py` | Memory file saved | Placeholder for future rollover trigger |
|
||||
| Event | Handler | Trigger | Action |
|
||||
|-------|---------|---------|--------|
|
||||
| `startup` | `startup.py` | Branch session starts | Error catch-up scan across log files, memory rollover check |
|
||||
| `error_detected` | `error_detected.py` | Error registered via log watcher or `report_error()` | Full 8-gate Medic dispatch — emails fix-it to affected branch + `wake_branch()` |
|
||||
| `error_logged` | `error_logged.py` | System log error (fallback path) | Monitor-only: logs the event, no dispatch |
|
||||
| `warning_logged` | `warning_logged.py` | Warning in system logs | Logged for monitoring, no dispatch |
|
||||
| `plan_file_created` | `plan_file.py` | New PLAN file detected | Updates Flow's PLAN_REGISTRY.json |
|
||||
| `plan_file_deleted` | `plan_file.py` | PLAN file removed | Marks plan as deleted in registry |
|
||||
| `plan_file_moved` | `plan_file.py` | PLAN file relocated | Updates registry location |
|
||||
| `bulletin_created` | `bulletin_created.py` | New system bulletin posted | Propagates to branch dashboards |
|
||||
| `memory_threshold_exceeded` | `memory_threshold_exceeded.py` | Memory file near limit (600 lines) | Emails compression notification to branch |
|
||||
| `memory_template_updated` | `memory_template_updated.py` | Memory template changed | Pushes template updates to branches |
|
||||
| `memory_saved` | `memory.py` | Memory file written | Placeholder for future rollover trigger |
|
||||
| `cli_header_displayed` | `cli.py` | CLI displays headers | Registration hook |
|
||||
| `pr_created` | `pr_status_sync.py` | PR opened on GitHub | Runs `drone @prax status sync` (fire-and-forget) |
|
||||
| `pr_merged` | `pr_status_sync.py` | PR merged on GitHub | Runs `drone @prax status sync` (fire-and-forget) |
|
||||
|
||||
## Medic
|
||||
|
||||
Built-in error monitoring subsystem. Watches logs for errors, fingerprints them via SHA1, deduplicates, and dispatches fix-it notifications to the responsible branch. Includes circuit breaker, per-branch rate limiting, and mute controls.
|
||||
Error monitoring subsystem. Watches branch and system logs for errors, fingerprints them via SHA1, deduplicates, and dispatches fix-it notifications to the responsible branch.
|
||||
|
||||
**Dispatch pipeline (8 gates):**
|
||||
|
||||
1. **Medic enabled** — global on/off toggle
|
||||
2. **Branch not muted** — per-branch suppression
|
||||
3. **Count >= 2** — first occurrence suppressed, dispatch on recurrence
|
||||
4. **Not DEV_CENTRAL** — devpulse protected from self-dispatch
|
||||
5. **Branch in registry** — target must be a registered citizen
|
||||
6. **Circuit breaker closed** — trips after 10 errors in 60s, 300s cooldown
|
||||
7. **Per-fingerprint backoff** — exponential backoff per unique error
|
||||
8. **Rate limit** — prevents dispatch floods
|
||||
|
||||
On successful dispatch: sends email via `deliver_email_to_branch()` then calls `wake_branch()` to spawn an agent in the target branch immediately.
|
||||
|
||||
**Persistent log watching** runs as a systemd user service (`trigger-log-watcher.service`). Starts both branch and system watchers, handles SIGTERM/SIGINT for clean shutdown.
|
||||
|
||||
```bash
|
||||
systemctl --user status trigger-log-watcher # Check watcher service
|
||||
systemctl --user restart trigger-log-watcher # Restart watcher
|
||||
```
|
||||
|
||||
## Error Registry
|
||||
|
||||
SHA1 fingerprinting for error deduplication. Tracks: fingerprint, branch, error type, message, count, first/last seen, dispatch history, source fix status.
|
||||
|
||||
**Circuit breaker:** Trips after 10 errors within 60 seconds. Rejects all dispatch while open. Auto-resets after 300s cooldown. State persists across restarts in `trigger_cb_state.json`.
|
||||
|
||||
**Per-fingerprint tracking:** Each unique error has independent exponential backoff and dispatch count. State persists across restarts.
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
trigger/
|
||||
├── apps/
|
||||
│ ├── trigger.py # Entry point (auto-discovers modules)
|
||||
│ ├── log_watcher_service.py # Persistent watcher process (systemd)
|
||||
│ ├── trigger.py # Entry point (auto-discovers modules/)
|
||||
│ ├── config.py # Constants, atomic_write_json, json_file_lock
|
||||
│ ├── log_watcher_service.py # Persistent watcher daemon (systemd)
|
||||
│ ├── modules/
|
||||
│ │ ├── core.py # Event bus (Trigger.fire/on/off)
|
||||
│ │ ├── errors.py # Error registry + cross-branch API
|
||||
│ │ ├── medic.py # Error monitoring commands
|
||||
│ │ ├── branch_log_events.py # Branch-level log event handling
|
||||
│ │ └── log_events.py # System-wide log event processing
|
||||
│ │ ├── core.py # Event bus: Trigger.fire/on/off/status
|
||||
│ │ ├── errors.py # Error registry CLI: list/stats/circuit-breaker
|
||||
│ │ ├── medic.py # Medic toggle: on/off/status/mute/unmute
|
||||
│ │ ├── branch_log_events.py # Branch log watcher CLI: start/stop/status
|
||||
│ │ └── log_events.py # System log watcher CLI: start/stop/status
|
||||
│ └── handlers/
|
||||
│ ├── events/ # One handler per event type
|
||||
│ ├── log_watcher.py # Branch log watcher (watchdog)
|
||||
│ ├── error_registry.py # SHA1 fingerprinting + circuit breaker
|
||||
│ ├── error_reporter.py # Cross-branch error API
|
||||
│ └── medic_state.py # Medic persistence (trigger_config.json)
|
||||
└── tests/
|
||||
│ ├── error_registry.py # SHA1 fingerprinting, circuit breaker, backoff
|
||||
│ ├── error_reporter.py # report_error() API + source fix emails
|
||||
│ ├── log_watcher.py # Branch log watcher (watchdog, position tracking)
|
||||
│ ├── medic_state.py # Medic config persistence (trigger_config.json)
|
||||
│ ├── json/
|
||||
│ │ └── json_handler.py # JSON structure logging
|
||||
│ ├── events/
|
||||
│ │ ├── registry.py # Auto-registers all 14 event handlers
|
||||
│ │ ├── startup.py # Startup catch-up scan
|
||||
│ │ ├── error_detected.py # 8-gate Medic dispatch
|
||||
│ │ ├── error_logged.py # Monitor-only (no dispatch)
|
||||
│ │ ├── warning_logged.py # Warning monitor
|
||||
│ │ ├── plan_file.py # Plan lifecycle events
|
||||
│ │ ├── bulletin_created.py # Bulletin propagation
|
||||
│ │ ├── memory_threshold_exceeded.py
|
||||
│ │ ├── memory_template_updated.py
|
||||
│ │ ├── memory.py # memory_saved placeholder
|
||||
│ │ ├── cli.py # cli_header_displayed hook
|
||||
│ │ └── pr_status_sync.py # PR → prax status sync
|
||||
│ └── watchers/
|
||||
│ └── log_watcher.py # System log watcher (system_logs/ dir)
|
||||
├── tests/ # 367 tests across 12 modules
|
||||
├── trigger_json/ # Runtime state files
|
||||
│ ├── trigger_config.json # Medic state, muted branches
|
||||
│ ├── error_registry.json # All tracked errors
|
||||
│ └── trigger_cb_state.json # Circuit breaker persistence
|
||||
└── trigger_data.json # Log watcher positions + dedup hashes
|
||||
```
|
||||
|
||||
**Note:** `branch_log_events` and `log_events` are auto-discovered modules that handle log-based event detection at branch and system levels respectively.
|
||||
## Data Safety
|
||||
|
||||
---
|
||||
- **Atomic writes:** All JSON state files use `config.atomic_write_json()` — writes to a temp file in the same directory, then `os.replace()` for atomic rename. No partial writes on crash.
|
||||
- **File locking:** All read-modify-write cycles wrapped in `config.json_file_lock()` using `fcntl.flock` with `.lock` sidecar files. Prevents concurrent corruption from watcher + CLI.
|
||||
- **Circuit breaker persistence:** Trip state, recent errors, per-fingerprint tracking all survive restarts via `trigger_cb_state.json`.
|
||||
|
||||
## Integration Points
|
||||
|
||||
### Depends On
|
||||
- `aipass.prax` — Logging via `system_logger`
|
||||
- `aipass.cli` — Console output and header formatting
|
||||
- Python stdlib (`sys`, `pathlib`, `importlib`)
|
||||
- `aipass.cli` — Console output and formatting
|
||||
- `aipass.ai_mail` — `deliver_email_to_branch()` for dispatch emails (lazy import, graceful fallback)
|
||||
- `aipass.memory` — `run_rollover()` check on startup (lazy import)
|
||||
|
||||
### Provides To
|
||||
- All modules — event bus (`Trigger.fire`, `Trigger.on`, `Trigger.off`)
|
||||
- All modules — cross-branch error reporting (`report_error`)
|
||||
- All modules — medic error monitoring and dispatch
|
||||
- All branches — Event bus (`Trigger.fire`, `Trigger.on`, `Trigger.off`)
|
||||
- All branches — Cross-branch error reporting (`report_error()`)
|
||||
- All branches — Automated error dispatch via Medic
|
||||
|
||||
## Testing
|
||||
|
||||
367 tests across 12 test modules, all passing. Coverage: 55/76 public functions (72%).
|
||||
|
||||
```bash
|
||||
cd src/aipass/trigger && pytest # Run all tests
|
||||
```
|
||||
|
||||
Test files: `test_core`, `test_errors`, `test_medic`, `test_error_registry`, `test_error_reporter`, `test_medic_state`, `test_log_watcher`, `test_watchers_log_watcher`, `test_branch_log_events`, `test_log_events`, `test_json_handler`, `test_pr_status_sync`
|
||||
|
||||
## Compliance
|
||||
|
||||
Seedgo: 100% (34/34 standards). Zero type errors. All categories at 100%.
|
||||
|
||||
---
|
||||
|
||||
|
||||
Reference in New Issue
Block a user