diff --git a/src/aipass/api/README.md b/src/aipass/api/README.md index 79c1f34a..4af64b9b 100644 --- a/src/aipass/api/README.md +++ b/src/aipass/api/README.md @@ -4,102 +4,51 @@ > Centralized external API gateway — authenticated service clients for all external APIs -**Module:** `aipass.api` | **Role:** `api_gateway` | **Version:** 1.0.0 -**Seedgo:** 100% (34/34) | **Tests:** 306 pass | **Functions:** 74 public (73 tested) -**Last Updated:** 2026-04-22 +**Module:** `aipass.api` | **Role:** `api_gateway` +**Seedgo:** 99% (35/36 at 100%) | **Tests:** 447 pass | **Functions:** 77 public (77 tested) +**Last Updated:** 2026-05-16 --- -## Overview +## Invoke -API is the centralized external service gateway for AIPass. It provides authenticated clients for external APIs (OpenRouter, Google, future providers). Consumers import ready-to-use service objects — API owns the plumbing, consumers own the business logic. - -### What I Do -- Provide authenticated service clients for external APIs (OpenRouter, Google Drive, Calendar, etc.) -- Manage OAuth2 flows, credential storage, and token refresh -- Offer thread-safe service factories for concurrent consumers -- Handle API key management and validation across providers -- Host the generic contract registry for private integration drivers (DPLAN-0133) -- Provide SSL retry and connection resilience utilities - -### What I Don't Do -- Host business logic — consumers own what they do with the service -- Set default models or configs — consumers provide their own -- Manage application workflows, polling loops, or orchestration - -### Design Principle -If it's not auth, credentials, or service factory — it doesn't belong here. +```bash +drone @api [args] +``` --- ## Commands -```bash -# Key Management -drone @api get-key [provider] # Retrieve API key (default: openrouter) -drone @api validate [provider] # Validate API key (default: openrouter) -drone @api validate google # Validate Google OAuth2 credentials -drone @api reauth google # Re-authenticate Google OAuth2 -drone @api list-providers # List available API providers -drone @api init # Initialize .env template at ~/.secrets/aipass/ - -# OpenRouter -drone @api test # Test OpenRouter connection status -drone @api models [--all] # List available models (default: top 10) -drone @api status # Check OpenRouter client status (key, SDK, cache) -drone @api call "prompt" --model MODEL # Make API call to model - -# Usage Tracking -drone @api track # Track API usage for a generation -drone @api stats # Display overall usage statistics -drone @api session # Show current session usage -drone @api caller-usage # Show usage by caller module -drone @api cleanup [days] # Clean up data older than N days (default: 30) - -# Integration Contracts -drone @api integrations list # List registered contracts -drone @api integrations call [args...] # Call a registered contract - -# Meta -drone @api --help # Full help output -drone @api --version # Show version (v1.0.0) -drone @api # Module introspection (7 discovered modules) -``` - ---- - -## Cross-Branch API - -```python -# LLM access (OpenRouter) -from aipass.api.apps.modules.openrouter_client import get_response -response = get_response(prompt="...", model="anthropic/claude-3.5-sonnet", caller="flow") - -# Google Drive (or any Google API) -from aipass.api.apps.modules.google_client import get_drive_service -service = get_drive_service() # Single-threaded -service = get_drive_service(thread_safe=True) # For concurrent workers - -# Any Google service -from aipass.api.apps.modules.google_client import get_google_service -service = get_google_service("calendar", "v3") - -# Retry utility for raw API calls -from aipass.api.apps.modules.google_client import api_call_with_retry -``` +| Command | Description | +|---|---| +| `get-key [provider]` | Retrieve API key (default: openrouter) | +| `validate [provider]` | Validate API key (default: openrouter) | +| `validate google` | Validate Google OAuth2 credentials | +| `reauth google` | Re-authenticate Google OAuth2 | +| `list-providers` | List available API providers | +| `init` | Initialize .env template at ~/.secrets/aipass/ | +| `test` | Test OpenRouter connection status | +| `models [--all]` | List available models (default: top 10) | +| `status` | Check OpenRouter client status (key, SDK, cache) | +| `call "prompt" --model MODEL` | Make API call to model | +| `track [caller]` | Track API usage for a generation | +| `stats` | Display overall usage statistics | +| `session` | Show current session usage | +| `caller-usage ` | Show usage by caller module | +| `cleanup [days]` | Clean up data older than N days (default: 30) | +| `integrations list` | List registered contracts | +| `integrations call [args...]` | Call a registered contract | --- ## Architecture -Three-tier design: entry point routes to modules (orchestration), which delegate to handlers (business logic). Modules are auto-discovered from `apps/modules/*.py` — each implements `handle_command(command, args) -> bool`. - ``` api/ -├── __init__.py ├── apps/ │ ├── api.py # Entry point — module discovery, command routing -│ ├── modules/ # Tier 2: orchestration layer (7 modules) +│ ├── modules/ # Orchestration layer (7 modules) │ │ ├── api_key.py # Key retrieval, validation, provider listing │ │ ├── openrouter_client.py # OpenRouter client — calls, models, status │ │ ├── google_client.py # Google API services (Drive, Calendar, etc.) @@ -107,96 +56,70 @@ api/ │ │ ├── bridge.py # Generic contract registry (register/resolve) │ │ ├── integrations_manager.py # Contract dispatch — integrations list/call │ │ └── registry.py # Driver auto-discovery (load_drivers) -│ ├── handlers/ # Tier 3: business logic (7 packages, 15 files) -│ │ ├── auth/ -│ │ │ ├── env.py # .env template creation (0o600 permissions) -│ │ │ └── keys.py # Key storage, retrieval, validation rules -│ │ ├── config/ -│ │ │ └── provider.py # Provider config deep-merge, validation rules -│ │ ├── google/ -│ │ │ ├── auth.py # OAuth2 lifecycle — load, refresh, save credentials -│ │ │ ├── service_factory.py # Service object factory (single + thread-safe) -│ │ │ └── retry.py # Exponential backoff with SSL error detection -│ │ ├── integrations/ -│ │ │ ├── list.py # List registered contracts -│ │ │ └── call.py # Invoke contract driver functions -│ │ ├── json/ -│ │ │ └── json_handler.py # JSON persistence — 3-file pattern per operation -│ │ ├── openrouter/ -│ │ │ ├── caller.py # Stack-based caller detection -│ │ │ ├── client.py # OpenRouter client (OpenAI SDK wrapper) -│ │ │ ├── models.py # Model discovery and listing -│ │ │ └── provision.py # Per-caller config auto-provisioning -│ │ └── usage/ -│ │ ├── aggregation.py # Stats rollup — per-caller, daily, monthly -│ │ ├── cleanup.py # Data retention and cleanup -│ │ └── tracking.py # Generation metrics from OpenRouter API +│ ├── handlers/ # Business logic (7 packages, 15 files) +│ │ ├── auth/env.py, keys.py +│ │ ├── config/provider.py +│ │ ├── google/auth.py, service_factory.py, retry.py +│ │ ├── integrations/list.py, call.py +│ │ ├── json/json_handler.py +│ │ ├── openrouter/caller.py, client.py, models.py, provision.py +│ │ └── usage/aggregation.py, cleanup.py, tracking.py │ └── integrations/ # Private driver space (gitignored) -│ └── {project}/driver.py # Each driver registers contracts via bridge -├── tests/ # 306 tests across 15 files -│ ├── test_api_key.py # Key management (39 tests) -│ ├── test_caller.py # Caller detection (9 tests) -│ ├── test_cli_routing.py # Command routing (9 tests) -│ ├── test_config_provider.py # Config merging (16 tests) -│ ├── test_contracts.py # JSON contract operations (11 tests) -│ ├── test_critical_paths.py # End-to-end critical flows (19 tests) -│ ├── test_error_resilience.py # Error handling (4 tests) -│ ├── test_google_client.py # Google OAuth2 + services (46 tests) -│ ├── test_init_provisioning.py # Provisioning init (4 tests) -│ ├── test_integrations.py # Bridge, registry, contracts (17 tests) -│ ├── test_json_handler.py # JSON persistence (41 tests) -│ ├── test_openrouter_client.py # OpenRouter client (37 tests) -│ ├── test_provision.py # Auto-provisioning (20 tests) -│ ├── test_tracking.py # Usage tracking (13 tests) -│ └── test_usage_tracker.py # Usage module (21 tests) -└── docs/ - └── SECURITY.md # Key handling and leak prevention +│ └── {project}/driver.py +└── tests/ # 447 tests across 27 files ``` +Three-tier: entry point routes to modules (orchestration), modules delegate to handlers (business logic). Modules auto-discovered from `apps/modules/*.py` via `handle_command()`. + --- -## Integration Contract System +## Cross-Branch API -Private integration drivers live in `apps/integrations/{project}/driver.py` (gitignored). Each driver registers named contracts via `bridge.register()`. Callers resolve contracts by name — never referencing private projects directly. +```python +from aipass.api.apps.modules.openrouter_client import get_response +response = get_response(prompt="...", model="anthropic/claude-3.5-sonnet", caller="flow") -**Three-layer design (DPLAN-0133):** -1. **Drivers** (`apps/integrations/*/driver.py`) — private, gitignored, register contracts on load -2. **Bridge** (`bridge.py`) — public contract registry: `register()`, `resolve()`, `list_contracts()` -3. **Handlers** (`integrations/list.py`, `call.py`) — public dispatch for `drone @api integrations` +from aipass.api.apps.modules.google_client import get_drive_service +service = get_drive_service() # Single-threaded +service = get_drive_service(thread_safe=True) # For concurrent workers -**Auto-discovery:** `registry.py` walks `apps/integrations/*/driver.py` via `importlib.util.spec_from_file_location`. Handles empty dirs, missing files, and import errors gracefully. +from aipass.api.apps.modules.google_client import get_google_service +service = get_google_service("calendar", "v3") +``` --- ## Integration Points -### Depends On +**Depends On:** - `aipass.prax` — structured logging via `system_logger` - `aipass.cli` — Rich console output formatting -### Provides To -- All branches — authenticated external API clients via `get_response()`, `get_drive_service()`, `get_google_service()` +**Provides To:** +- All branches — authenticated API clients (`get_response()`, `get_drive_service()`, `get_google_service()`) - System-wide API key management and credential validation -### Credentials -All credentials live at `~/.secrets/aipass/` (0o700 directory, 0o600 files): +**Credentials** (`~/.secrets/aipass/`, 0o700 dir, 0o600 files): - `.env` — API keys (OpenRouter, etc.) - `google_creds.json` — Google OAuth2 tokens - `google_client_secret.json` — Google OAuth app config -### Provider Pattern -One module per provider (`openrouter_client.py`, `google_client.py`), one handler directory per provider (`openrouter/`, `google/`). Module orchestrates and presents CLI; handlers implement business logic. Pattern scales to future providers. +--- + +## Integration Contract System (DPLAN-0133) + +Private drivers in `apps/integrations/{project}/driver.py` (gitignored) register named contracts via `bridge.register()`. Callers resolve by name — never referencing private projects directly. `registry.py` handles auto-discovery via importlib. --- ## Known Issues + - Google auth libraries are optional deps — commands fail with install instructions if missing -- 1/74 public functions untested per seedgo test_map -- Backup branch credential migration pending (`~/.aipass/` to `~/.secrets/aipass/`) +- Backup branch credential migration pending (`~/.aipass/` → `~/.secrets/aipass/`) +- No rate limiting on OpenRouter calls (S117 finding) --- -*Last Updated: 2026-04-22* +*Last Updated: 2026-05-16* ---- [← Back to AIPass](../../../README.md) diff --git a/src/aipass/cli/README.md b/src/aipass/cli/README.md index d4909b60..30b3bac9 100644 --- a/src/aipass/cli/README.md +++ b/src/aipass/cli/README.md @@ -5,9 +5,9 @@ **Purpose:** Display and output formatting service for all AIPass branches. Provides consistent terminal output — headers, success/error/warning messages, section breaks, and operation templates — so every branch looks the same without duplicating Rich formatting code. **Module:** `aipass.cli` **Version:** 2.1.0 -**Seedgo:** 100% (34/34 standards) -**Tests:** 203 passing (6 files, 5/5 modules covered) -**Last Updated:** 2026-05-04 +**Seedgo:** 99% +**Tests:** 127 passing (5 files) +**Last Updated:** 2026-05-16 ## Usage @@ -28,7 +28,7 @@ section("Results") ### Operation Templates ```python -from aipass.cli import operation_start, operation_complete +from aipass.cli.apps.modules import operation_start, operation_complete operation_start("Processing", count=10) # ... do work ... @@ -38,7 +38,7 @@ operation_complete(created=5, skipped=3, failed=0, time="1.2s") ### Fatal (exit on error) ```python -from aipass.cli import fatal +from aipass.cli.apps.modules import fatal fatal("Config file missing", suggestion="Run aipass init first") # Prints error message + suggestion, then calls sys.exit(1) @@ -54,10 +54,12 @@ console.print("[bold cyan]Custom Rich output[/bold cyan]") ## Public API -All display functions are importable from the top-level package or `apps/modules/`: +Exported from `apps/modules/__init__.py` (10 symbols): | Function | Signature | Purpose | |----------|-----------|---------| +| `console` | Rich Console instance | Standard output console | +| `err_console` | Rich Console instance | Stderr console | | `header()` | `header(title, details=None)` | Bordered section header with optional key-value pairs | | `success()` | `success(message, **kwargs)` | Green checkmark message with metadata | | `error()` | `error(message, suggestion=None)` | Red error with optional suggestion | @@ -66,62 +68,54 @@ All display functions are importable from the top-level package or `apps/modules | `section()` | `section(title)` | Visual section separator with title | | `operation_start()` | `operation_start(operation, **details)` | Standard operation begin header | | `operation_complete()` | `operation_complete(**summary)` | Completion summary with optional timing | -| `console` | Rich Console instance | Standard output console | -| `err_console` | Rich Console instance | Stderr console | -Import paths (all equivalent): +Import paths: ```python -from aipass.cli import header # Top-level re-export -from aipass.cli.apps.modules import header # Module-level -from aipass.cli.apps.modules.display import header # Direct +from aipass.cli import console, header, success, error, warning, section # Top-level (6 symbols) +from aipass.cli.apps.modules import header, fatal, operation_start # Full set (10 symbols) +from aipass.cli.apps.modules.display import header # Direct module ``` ## Commands ```bash -# Via drone -drone @cli --help # Services + Rich formatting showcase -drone @cli --version # Version (v2.1.0) -drone @cli # Module discovery (introspection) -drone @cli display # Display module info -drone @cli display demo # Run display function showcase -drone @cli templates # Templates module info -drone @cli templates demo # Run templates function showcase - -# Standalone (no drone required) -python -m aipass.cli --help # Same help output +drone @cli --help # Full help + architecture overview +drone @cli --version # v2.1.0 +drone @cli # Module discovery (introspection) +drone @cli display # Display module info +drone @cli display demo # Run display function showcase +drone @cli templates # Templates module info +drone @cli templates demo # Run templates function showcase +python -m aipass.cli --help # Same help (no drone required) ``` -> **Note:** The `aipass init` command is now owned by the @aipass branch (`aipass.aipass.apps.aipass:main`). - -## aipass init (moved) - -The `aipass init` command has been moved to the @aipass branch. CLI no longer owns project bootstrapping. See `aipass.aipass.apps.aipass:main` for the current implementation. - ## Architecture ``` cli/ -├── __init__.py # Public API exports + cli_entry() +├── __init__.py # Top-level exports (6 symbols) + cli_entry() ├── __main__.py # python -m aipass.cli entry ├── apps/ │ ├── cli.py # Entry point (main, discover_modules, route_command) │ ├── modules/ # PUBLIC — import from here -│ │ ├── __init__.py # Re-exports all display + template functions +│ │ ├── __init__.py # Re-exports all 10 display + template symbols │ │ ├── display.py # header, success, error, warning, fatal, section │ │ └── templates.py # operation_start, operation_complete -│ └── handlers/ # PRIVATE — internal implementation -│ ├── json/ -│ │ └── json_handler.py # JSON lifecycle (CRUD, validation, rotation) -│ └── templates/ # Empty — placeholder from scaffold -├── tests/ # Test suite -│ ├── test_json_handler.py # 35 tests — CRUD, validation, rotation -│ ├── test_display.py # 28 tests — all display functions + routing -│ ├── test_templates.py # 19 tests — operation templates + routing -│ └── test_integration.py # 8 tests — main() flow, entry points +│ ├── handlers/ # PRIVATE — internal implementation +│ │ ├── json/ +│ │ │ └── json_handler.py # JSON lifecycle (CRUD, validation, rotation) +│ │ └── templates/ # Scaffold placeholder +│ ├── integrations/ # Scaffold placeholder +│ └── plugins/ # Required by spawn builder template +├── tests/ # 127 tests across 5 files +│ ├── test_display.py # 45 tests — display functions + routing + triggers +│ ├── test_json_handler.py # 39 tests — CRUD, validation, rotation +│ ├── test_templates.py # 28 tests — operation templates + routing +│ ├── test_handler_guard.py # 8 tests — cross-branch import guard +│ └── test_integration.py # 7 tests — main() flow, entry points ├── cli_json/ # Auto-created JSON (config, data, log) ├── logs/ # Branch-level logs -└── .archive/ # Archived stubs (extensions/, json_templates/) +└── .archive/ # Archived stubs (extensions/, json_templates/, drone_adapter) ``` **Two-tier design:** @@ -145,10 +139,11 @@ json_handler.ensure_module_jsons("cli") # Create all 3 if missing ### Depends On - `rich` — Terminal formatting (Table, Panel, Text, Console) +- `aipass.prax` — Logging (imported in cli.py only, not in modules/) - Python stdlib (`sys`, `importlib`, `pathlib`, `json`) -### Cannot Import -- `aipass.prax` — Circular dependency (prax depends on cli). Display/templates modules bypass this with documented entries in `.seedgo/bypass.json`. +### Cannot Import (in modules/) +- `aipass.prax` — Circular dependency (prax depends on cli). Bypassed in `.seedgo/bypass.json`. ### Provides To - **All branches** — Display formatting (header, success, error, warning, fatal, section) @@ -159,12 +154,13 @@ json_handler.ensure_module_jsons("cli") # Create all 3 if missing | Entry | Command | How | |-------|---------|-----| -| drone | `drone @cli [command]` | Drone routes to `apps/cli.py:main()` | +| drone | `drone @cli [command]` | Routes to `apps/cli.py:main()` | | Module | `python -m aipass.cli [args]` | `__main__.py` calls `main()` | +| console_scripts | `aipass` on PATH | `cli_entry()` in `__init__.py` | --- -*Last Updated: 2026-05-04* +*Last Updated: 2026-05-16* --- [← Back to AIPass](../../../README.md) diff --git a/src/aipass/memory/README.md b/src/aipass/memory/README.md index 32a4a054..5991065c 100644 --- a/src/aipass/memory/README.md +++ b/src/aipass/memory/README.md @@ -2,67 +2,35 @@ # MEMORY -**Purpose:** Central memory archive — vector search, rollover, and memory management for all AIPass branches. -**Module:** `aipass.memory` -**Created:** 2026-03-07 -**Last Updated:** 2026-05-10 -**Citizen Class:** builder +**Central memory archive — vector search, rollover, and memory management for all AIPass branches.** ---- - -## Overview - -Memory is the archival backbone of AIPass. Every branch accumulates session history and learnings in `.trinity/` files. When those files reach capacity, Memory archives the oldest entries into ChromaDB vectors — searchable, permanent, never lost. - -What Memory does: -- **Rollover** — detects when `.trinity/local.json` or `observations.json` exceed limits, extracts oldest entries, embeds them via fastembed, stores in ChromaDB, trims the source file -- **Search** — semantic search across all archived branch memories (4+ collections, 2200+ vectors) -- **Templates** — distributes `.trinity/` schema updates across all branches (push, diff, status) -- **Symbolic** — fragmented memory extraction from conversations (demo, analyze, extract, fragments, bootstrap, hook-test) -- **Verify** — checks whether a flow plan is vectorized in ChromaDB -- **Watch** — persistent file watcher that auto-triggers rollover on changes +`drone @memory ` | Module: `aipass.memory` | Created: 2026-03-07 --- ## Commands -All commands via `drone @memory `: - ```bash -# Introspection -drone @memory # Module list, version -drone @memory --help # Full command reference -drone @memory --version # Version string - -# Rollover -drone @memory rollover # Module introspection (handlers + subcommands) drone @memory rollover run # Execute rollover for files over limits drone @memory rollover status # Show per-branch rollover statistics -drone @memory rollover check # Dry run — check what needs rollover -drone @memory rollover sync-lines # Update line count metadata for all branches +drone @memory rollover check # Dry run — what needs rollover +drone @memory rollover sync-lines # Update line count metadata -# Search -drone @memory search "error handling" # Semantic search across all branch memories -drone @memory search "query" --branch X # Filter search by branch -drone @memory search "query" --n 10 # Limit number of results +drone @memory search "query" # Semantic search across all branch memories +drone @memory search "query" --branch X # Filter by branch +drone @memory search "query" --n 10 # Limit results -# Symbolic -drone @memory symbolic # Module introspection (6 handlers, subcommands) -drone @memory symbolic demo # Run v1 + v2 mock analysis demonstration +drone @memory symbolic demo # Mock analysis demonstration drone @memory symbolic fragments "query" # Search stored symbolic fragments drone @memory symbolic extract # Extract fragments via LLM (requires API key) drone @memory symbolic bootstrap # Populate fragments from session JSONLs drone @memory symbolic hook-test # Test hook with sample conversation text -# Templates drone @memory templates push-templates # Push template updates to all branches drone @memory templates diff-templates # Show template differences per branch drone @memory templates template-status # Show template version and push status -# Verify drone @memory verify FPLAN-XXXX # Check if plan is vectorized in ChromaDB - -# Watch drone @memory watch # Auto-rollover watcher daemon (Ctrl+C to stop) ``` @@ -72,118 +40,93 @@ drone @memory watch # Auto-rollover watcher daemon (Ctrl+ ``` memory/ -├── .trinity/ # Identity & memory -│ ├── passport.json # Branch identity -│ ├── local.json # Session history (v2 schema, entry-count limits) -│ └── observations.json # Collaboration patterns (v1 schema, line-count limits) -├── .aipass/ # Branch prompt -├── .ai_mail.local/ # Mailbox ├── apps/ -│ ├── memory.py # Entry point — auto-discovers modules via handle_command() -│ ├── modules/ # Business logic (5 modules) -│ │ ├── rollover.py # Rollover orchestration, status display, sync-lines +│ ├── memory.py # Entry point — auto-discovers modules +│ ├── modules/ # 5 modules +│ │ ├── rollover.py # Rollover orchestration, status, sync-lines │ │ ├── search.py # Semantic query routing -│ │ ├── verify.py # Plan vectorization check │ │ ├── symbolic.py # Fragmented memory extraction and search -│ │ └── templates.py # Template push, diff, status -│ └── handlers/ # Implementation (14 handler groups, 35 files) -│ ├── archive/ # indexer.py — memory archival indexing -│ ├── intake/ # plans_processor.py, pool_processor.py — ingest pipelines -│ ├── json/ # json_handler.py, memory_files.py — JSON operations -│ ├── learnings/ # manager.py — learning extraction and management -│ ├── monitor/ # detector.py, memory_watcher.py — rollover detection + auto-trigger -│ ├── rollover/ # extractor.py, orchestrator.py — backup → extract → embed → store -│ ├── schema/ # normalize.py — schema version normalization -│ ├── search/ # query_executor.py, vector_search.py — semantic search impl -│ ├── storage/ # chroma.py, chroma_subprocess.py — ChromaDB backend -│ ├── symbolic/ # 6 handlers — chroma_client, deduplicator, extractor, hook, retriever, storage -│ ├── templates/ # pusher.py, differ.py, spawn_pusher.py — template distribution -│ ├── tracking/ # line_counter.py — metadata line count tracking -│ ├── vector/ # embedder.py, embed_subprocess.py — fastembed embeddings -│ └── central_writer.py # Central memory write operations +│ │ ├── templates.py # Template push, diff, status +│ │ └── verify.py # Plan vectorization check +│ └── handlers/ # 14 handler groups +│ ├── archive/ # indexer.py +│ ├── intake/ # plans_processor.py, pool_processor.py +│ ├── json/ # json_handler.py, memory_files.py +│ ├── learnings/ # manager.py +│ ├── monitor/ # detector.py, memory_watcher.py +│ ├── rollover/ # extractor.py, orchestrator.py +│ ├── schema/ # normalize.py +│ ├── search/ # query_executor.py, vector_search.py +│ ├── storage/ # chroma.py, chroma_subprocess.py +│ ├── symbolic/ # chroma_client, deduplicator, extractor, hook, retriever, storage +│ ├── templates/ # pusher.py, differ.py, spawn_pusher.py +│ ├── tracking/ # line_counter.py +│ ├── vector/ # embedder.py, embed_subprocess.py +│ └── central_writer.py ├── config/ # memory.config.json — per-branch rollover limits -├── templates/ # LOCAL.template.json, OBS.template — schema templates -├── tests/ # 450 tests (16/16 module coverage) -├── .chroma/ # Global ChromaDB vector store +├── templates/ # LOCAL.template.json, OBSERVATIONS.template.json +├── tests/ # 839 tests (28 test files) +├── .chroma/ # ChromaDB vector store └── memory_json/ # Operation log files (auto-created) ``` ### Rollover Pipeline ``` -startup trigger → check_and_rollover() - → detector.check_all_branches() # scan AIPASS_REGISTRY.json - → _should_rollover(file) # v1: line_count >= max_lines - # v2: len(sessions) >= max_sessions, etc. - → orchestrator.execute_rollover() - → create_rollover_backup() # safety copy to branch/.backup/ - → extract_items() # v2: max(excess, 1) oldest entries - → embed via subprocess # fastembed (ONNX) in memory .venv - → store in ChromaDB # global + local collections - → trim source file # write back with oldest removed +detector.check_all_branches() # scan AIPASS_REGISTRY.json + external registries +→ _should_rollover(file) # v1: line_count >= max_lines (600) + # v2: len(sessions) >= max_sessions (20) +→ orchestrator.execute_rollover() + → create_rollover_backup() # safety copy to branch/.backup/ + → extract_items() # v2: max(excess, 1) oldest entries + → embed via subprocess # fastembed (ONNX) in memory .venv + → upsert in ChromaDB # content-hash IDs (sha256[:16]), no duplicates + → trim source file # write back with oldest removed ``` -### Dual Schema Support - -- **v1** (line-count): `schema_version: "1.0.0"` — triggers at `current_lines >= max_lines` (default 600) -- **v2** (entry-count): `schema_version: "2.0.0"` — triggers at `len(sessions) >= max_sessions` or `len(key_learnings) >= max_key_learnings`. Extractor uses `max(excess, 1)` guard to prevent Python's `list[-0:]` trap. - ### Subprocess Isolation -All ML operations (fastembed, chromadb) run via subprocess. The main process never imports these libraries. Each embedding call resolves a Python interpreter via `_get_memory_python()` (env var `AIPASS_MEMORY_PYTHON` → `memory/.venv/bin/python` → `sys.executable`) and runs a self-contained script that reads stdin JSON and writes stdout JSON. +All ML operations (fastembed, chromadb) run via subprocess. The main process never imports these libraries. Python interpreter resolved via `_get_memory_python()` (env var `AIPASS_MEMORY_PYTHON` → `memory/.venv/bin/python` → `sys.executable`). --- -## Dependencies +## Integration Points -### Runtime -- `rich` — console output, panels, tables -- Python stdlib (`json`, `pathlib`, `logging`, `importlib`, `signal`) -- `prax` (internal) — logging via `get_system_logger()` +**Depends on:** +- `prax` — logging via `get_system_logger()` +- `api` — API key for symbolic extraction (`get_api_key()`) +- `AIPASS_REGISTRY.json` — branch discovery for rollover scanning +- External `*_REGISTRY.json` — scanned via `AIPASS_CALLER_CWD` -### ML (in memory `.venv/` only) -- `fastembed` — embedding generation (ONNX, no torch required) -- `chromadb` — vector storage and semantic search -- `numpy` — numerical operations - -### Provides To -- All branches — memory rollover and archival when `.trinity/` files hit limits +**Provides to:** +- All branches — rollover archival when `.trinity/` files hit limits - All branches — semantic search across archived memories -- All branches — template schema distribution -- All branches — line count metadata sync +- All branches — `.trinity/` template distribution and sync + +**ML dependencies (memory `.venv/` only):** +- `fastembed` — ONNX embeddings (model: `sentence-transformers/all-MiniLM-L6-v2`) +- `chromadb` — vector storage and semantic search +- `numpy` --- ## Quality -- **Seedgo:** 100% (33/33 standards) — maintained since s12 -- **Tests:** 450 pass, 1 skip (test_vector.py gated with `importorskip` for numpy) -- **Coverage:** 175 public functions, 102 tested (58%), 16/16 modules -- **Type checking:** 35 files, 0 errors +- **Tests:** 839 passed, 0 failures, 0 skips +- **Test files:** 28 +- **Seedgo:** 100% — maintained since s12 --- ## Known Issues - `search` requires fastembed in memory `.venv/` — fails without it -- memory_watcher.py at 704 lines (near 700 threshold, bypassed in seedgo) -- symbolic.py at 1604 lines (legacy port, bypassed) -- manager.py at 1076 lines (complex learning extraction, bypassed) -- `memory_threshold_exceeded` trigger event registered but never fired — rollover auto-runs at startup via watcher -- `rollover status` shows 0 branches when `AIPASS_REGISTRY.json` path not resolved +- `rollover status` shows 0 branches when registry path not resolved +- `memory_threshold_exceeded` trigger event registered but never fired --- -## Identity - -- **Passport:** `.trinity/passport.json` -- **Session History:** `.trinity/local.json` (v2 schema, 20 sessions max, 25 key_learnings max) -- **Observations:** `.trinity/observations.json` (v1 schema, 600 lines max) -- **Branch Prompt:** `.aipass/aipass_local_prompt.md` - ---- - -*Last Updated: 2026-05-10* +*Last Updated: 2026-05-16* --- [← Back to AIPass](../../../README.md)