diff --git a/FPLAN-0004_framework_backup_system_port_to_aipass_r_2026-03-06.md b/FPLAN-0004_framework_backup_system_port_to_aipass_r_2026-03-06.md new file mode 100644 index 00000000..b7a0ec8a --- /dev/null +++ b/FPLAN-0004_framework_backup_system_port_to_aipass_r_2026-03-06.md @@ -0,0 +1,274 @@ +# FPLAN-0004 - [framework] Backup System — port to AIPass repo + +**Created**: 2026-03-06 +**Branch**: /home/aipass/aipass_business/AIPass +**Status**: Active +**Type**: Standard Plan + +--- + +## What Are Flow Plans? + +Flow Plans (FPLANs) are for **BUILDING** - autonomous construction of systems, features, modules. + +**This is NOT for:** +- Research or exploration (use agents directly) +- Quick fixes (just do it) +- Discussion or planning (that happens before creating the FPLAN) + +**This IS for:** +- Building features or modules +- Single focused construction tasks +- Sub-plans within a master plan + +--- + +## When to Use This vs Master Plan + +| This (Default) | Master Plan | +|----------------|-------------| +| Single focused task | 3+ phases, complex build | +| Self-contained | Roadmap + multiple sub-plans | +| Quick build | Multi-session project | +| One phase of a master | Entire branch/system build | + +**Need a master plan?** `drone @flow create "subject" master` + +--- + +## Branch Directory Structure + +Use dedicated directories - don't scatter files: + +| Directory | Purpose | +|-----------|---------| +| `apps/` | Code (modules/, handlers/) | +| `tests/` | All test files | +| `tools/` | Utility scripts | +| `artifacts/` | Agent outputs | +| `docs/` | Documentation | + +--- + +## Critical: Branch Manager Role + +**You are the ORCHESTRATOR, not the builder.** + +Your 200k context is precious. Burning it on file reads and code writing risks compaction during autonomous work. Agents have clean context - use them for ALL building. + +| You Do (Orchestrator) | Agents Do (Builders) | +|-----------------------|----------------------| +| Create plans | Write code | +| Give instructions | Run tests | +| Review output | Read/modify files | +| Course correct | Research/exploration | +| Update memories | Heavy lifting | +| Send status emails | Single-task execution | + +**Pattern:** Instruct agent → Wait for completion → Review output → Next step + +--- + +## Seek Branch Expertise + +Don't figure everything out alone. Other branches are domain experts - ask them first. + +**Before building anything that touches another branch's domain:** +```bash +ai_mail send @branch "Question: [topic]" "I'm working on X and need guidance on Y. What's the best approach?" +``` + +**Common examples:** +- Building something with email? Ask @ai_mail how delivery works +- Need routing or @ resolution? Ask @drone +- Unsure about standards? Ask @seed for reference code +- Need persistent storage or search? Ask @memory_bank +- Event-driven behavior? Ask @trigger about their event system +- Dashboard integration? Ask @devpulse about update_section() + +They have deep memory on their systems. A 1-email question saves you hours of guessing. + +--- + +## Notepad + +Keep `notepad.md` in your branch directory as a shared scratchpad during the build. Use it for: +- **Status updates** - Quick progress lines so Patrick can glance without asking +- **Questions for Patrick** - Non-urgent questions that can wait for his next visit +- **Notes to self** - Decisions made, things to revisit, gotchas discovered + +Update it as you work - lightweight, not formal. Patrick checks it when he wants to, skips it when he's busy. + +--- + +## Command Reference + +When unsure about syntax, use `--help`: + +```bash +# Flow - Plan management +drone @flow create . "subject" # Create plan (. = current dir) +drone @flow close FPLAN-XXXX # Close plan +drone @flow list # List active plans +drone @flow --help # Full help + +# Seed - Quality gates +drone @seed checklist # 10-point check on file +drone @seed audit @branch # Full branch audit +drone @seed --help # Full help + +# AI_Mail - Status updates +drone @ai_mail send @dev_central "Subject" "Message" +drone @ai_mail --help # Full help + +# Discovery +drone systems # All available modules +drone list @branch # Commands for branch +``` + +--- + +## Planning Phase + +### Goal +Port the internal Backup System (`/home/aipass/aipass_core/backup_system/`) into the AIPass public repo at `src/aipass/backup/`. Working backup with snapshot mode, versioned mode, Google Drive sync (optional), dry-run support, and 90+ ignore patterns. Same approach as FPLAN-0003 (Memory Bank) — rewire, don't rebuild. + +### Approach +1. Copy all source files from internal backup_system/apps/ into src/aipass/backup/apps/ +2. Deploy parallel agents to adapt imports (remove prax/cli/sys.path, use relative imports, make Google Drive optional) +3. Fix hardcoded paths to use `Path(__file__).resolve().parents[N]` +4. Verify no internal references remain + +### Reference Documents +- Source: `/home/aipass/aipass_core/backup_system/apps/` (entry point + 4 modules + ~20 handlers) +- Pattern: FPLAN-0003 (Memory Bank port) — identical approach +- Architecture: 3-layer (apps/backup.py → modules/ → handlers/) + +--- + +## Agent Preparation (Before Deploying) + +Agents can't work blind. They need context before they build. + +**Your Prep Work (as orchestrator):** +1. [ ] Know where agent will work (branch path, key directories) +2. [ ] Identify files agent needs to reference or modify +3. [ ] Gather any specs, planning docs, or examples to include +4. [ ] Prepare COMPLETE instructions (agents are stateless) + +**Agent's First Task (context building):** +- Agent should explore/read relevant files BEFORE writing code +- "First, read X and Y to understand the current structure" +- "Look at Z for the pattern to follow" +- Context-first, build-second + +**What Agents DON'T Have:** +- No prior conversation history +- No memory files loaded automatically +- No knowledge of other branches +- Only what you put in their instructions + +**Your instructions determine success - be thorough and specific.** + +--- + +## Agent Instructions Template +``` +You are working at [BRANCH_PATH]. + +TASK: [Specific single task] + +CONTEXT: +- [What they need to know] +- Reference: [planning docs, existing code to study] +- First, READ the relevant files to understand current structure + +DELIVERABLES: +- [Specific file or output expected] +- Tests → tests/ +- Reports/logs → artifacts/reports/ or artifacts/logs/ + +CONSTRAINTS: +- Follow Seed standards (3-layer architecture) +- Do NOT modify files outside your task scope +- CROSS-BRANCH: Never modify other branches' files unless explicitly authorized by DEV_CENTRAL +- 2-ATTEMPT RULE: If something fails twice, note the issue and move on +- Do NOT go down rabbit holes debugging + +WHEN COMPLETE: +- Verify code runs without syntax errors +- List files created/modified +- Note any issues encountered (with what was attempted) +``` + +--- + +## Execution Log + +### 2026-03-06 +- [x] Created FPLAN-0004 +- [x] Agent deployed for: investigate backup_system structure (returned full report) +- [x] Created directory structure: src/aipass/backup/apps/{modules,handlers/{config,models,operations,utils,json,diff,reporting},json_templates/default,extensions,plugins} +- [x] Copied 42 files from internal backup_system +- [x] Agent 1 deployed: adapt 5 modules + entry point (completed — all imports fixed) +- [x] Agent 2 deployed: adapt 20 handler files (completed — all imports fixed) +- [x] Manual fix: handlers/__init__.py cross-branch guard removed (incompatible with public repo paths) +- [x] Manual fix: removed unused imports in backup_core.py (load_json, save_json, temporarily_writable, module-level logger.info) +- [x] Verified: zero remaining internal imports (prax, cli, sys.path, aipass_core, backup_system.apps) +- [ ] Memories updated + +**Log Pattern:** Task → Agent → Outcome → Quality check → Next + +**If production stops (critical blocker):** +```bash +drone @ai_mail send @dev_central "PRODUCTION STOPPED: FPLAN-0004" "Issue: [description]. Attempted: [what was tried]. Awaiting guidance." +``` + +--- + +## Notes + +- Same port-by-rewiring pattern as Memory Bank (FPLAN-0003): copy files, adapt imports, verify +- Google Drive deps (google-api-python-client, google-auth-oauthlib) made optional via try/except +- drive_sync_client.py already had Google API imports wrapped — just fixed internal path refs +- handlers/__init__.py had cross-branch security guard that checked for `/backup_system/` in caller path — incompatible with new `/backup/` path, replaced with simple package init +- JSON runtime data dir changed from `backup_system_json/` to `backup_json/` (relative to backup root) +- Timestamps file moved to `backup_data/backup_timestamps.json` +- Entry point renamed from `backup_system.py` to `backup.py` +- All `header()` calls from cli module replaced with local `_header()` helpers + +--- + +## Completion Checklist + +### Before Closing + +- [ ] All goals achieved +- [ ] Agent output reviewed and verified +- [ ] Seed checklist on new code: `drone @seed checklist ` +- [ ] Branch memories updated: + - [ ] `BRANCH.local.json` - session/work log + - [ ] `BRANCH.observations.json` - patterns learned (if any) +- [ ] README.md updated (if build changed status/capabilities) +- [ ] Status email sent to DEV_CENTRAL: + ```bash + drone @ai_mail send @dev_central "FPLAN-0004 Complete" "Summary of what was done, any issues, outcomes" + ``` + +**Completion Order:** Memories → README → Email (README before email - don't report complete with stale docs) + +### Definition of Done +- All backup_system files ported to src/aipass/backup/ with working imports +- Zero references to internal paths (prax, cli, aipass_core, sys.path hacks) +- Google Drive integration optional (graceful degradation when deps missing) +- JSON templates preserved for module initialization +- Entry point (backup.py) routes commands to all 4 modules + +--- + +## Close Command + +When all boxes checked: +```bash +drone @flow close FPLAN-0004 +``` diff --git a/FPLAN-0005_framework_daemon_port_to_aipass_repo_2026-03-07.md b/FPLAN-0005_framework_daemon_port_to_aipass_repo_2026-03-07.md new file mode 100644 index 00000000..1191579a --- /dev/null +++ b/FPLAN-0005_framework_daemon_port_to_aipass_repo_2026-03-07.md @@ -0,0 +1,106 @@ +# FPLAN-0005 - [framework] Daemon (assistant) — port to AIPass repo + +**Created**: 2026-03-07 +**Branch**: /home/aipass/aipass_business/AIPass +**Status**: Active +**Type**: Standard Plan + +--- + +## Planning Phase + +### Goal +Port the internal Assistant system (`/home/aipass/aipass_os/dev_central/assistant/`) into the AIPass public repo at `src/aipass/daemon/`. Rename from "assistant" to "daemon". Working scheduler with plugin system, scheduled tasks, activity monitoring, and cron-driven execution. Telegram integration optional. Same approach as FPLAN-0003/0004 — rewire, don't rebuild. + +### Approach +1. Copy all source files from internal assistant/apps/ into src/aipass/daemon/apps/ +2. Rename entry point from assistant.py to daemon.py +3. Deploy parallel agents to adapt imports (remove prax/cli/sys.path, use relative imports) +4. Make Telegram, ai_mail dispatch, and BRANCH_REGISTRY deps optional via try/except +5. Fix hardcoded paths to use `Path(__file__).resolve().parents[N]` +6. Verify no internal references remain + +### Key Decisions +- **Rename**: assistant → daemon (Patrick approved) +- **Telegram**: Optional — wrap in try/except, graceful skip when unconfigured +- **ai_mail dispatch**: Optional — wake.py spawning won't work outside Dev-Pass, needs abstraction +- **BRANCH_REGISTRY**: Activity monitoring reads this — make it configurable, graceful fallback +- **Plugins**: Port all 5 plugins as examples, but they're Dev-Pass-specific — document as templates +- **filelock**: Only external dep (already lightweight) + +### Reference Documents +- Source: `/home/aipass/aipass_os/dev_central/assistant/apps/` (entry point + 2 cron scripts + 4 modules + ~10 handlers + 5 plugins) +- Pattern: FPLAN-0003 (Memory Bank) and FPLAN-0004 (Backup System) — identical approach +- Architecture: 3-layer (apps/daemon.py → modules/ → handlers/) + +### Investigation Summary +- **4 modules**: update, schedule, activity_report, actions +- **10 handlers** across 6 subdirs: actions/, json/, monitoring/ (3 files), schedule/ (3 files), telegram/, update/ +- **5 plugins**: heartbeat, daily_audit, community_rotation, botfather_reminder, dev_central_monitor +- **2 cron scripts**: scheduler_cron.py, assistant_wakeup.py (renamed to daemon_wakeup.py) +- **External deps**: filelock only +- **Internal deps**: prax logger, cli console/header, ai_mail send_email_direct, api telegram_chat, BRANCH_REGISTRY.json +- **Tests**: 1 test file (test_actions_registry.py) with pytest + +--- + +## Execution Log + +### 2026-03-07 +- [x] Created FPLAN-0005 +- [x] Agent deployed for: investigate assistant structure (returned full report) +- [x] Created directory structure: src/aipass/daemon/apps/{modules,handlers/{actions,json,monitoring,schedule,telegram,update},plugins,extensions,json_templates/default} +- [x] Copied 41 files from internal assistant system (renamed assistant→daemon) +- [x] Agent 1 deployed: adapt 13 files (3 entry points + 4 modules + 5 plugins + plugin init) — completed +- [x] Agent 2 deployed: adapt 10 handler files — completed +- [x] Fixed standalone scripts (scheduler_cron.py, daemon_wakeup.py): changed relative imports to absolute package imports (standalone scripts can't use relative imports) +- [x] Fixed test file: removed sys.path manipulation, uses package imports +- [x] Fixed actions_registry.py: removed sys.path.insert in migrate_plugins, uses absolute package import +- [x] Fixed memory_health.py: removed hardcoded /home/aipass test path +- [x] Removed 3 remaining shebangs from conftest.py and handler __init__.py files +- [x] Cleaned unused imports (Path, sys, os, timedelta, Optional, Tuple) across modules +- [x] Verified: zero remaining internal imports (prax, cli, sys.path, aipass_core, aipass_os, assistant_json, ASSISTANT_ROOT) +- [ ] Memories updated + +--- + +## Notes + +- Same port-by-rewiring pattern as Memory Bank (FPLAN-0003) and Backup System (FPLAN-0004) +- Renamed assistant → daemon (Patrick approved the name) +- Standalone cron scripts (scheduler_cron.py, daemon_wakeup.py) can't use relative imports — fixed to use absolute package imports (aipass.daemon.apps.handlers.xxx) +- Telegram integration made optional via try/except + TELEGRAM_AVAILABLE flag +- ai_mail dispatch made optional via try/except + AI_MAIL_AVAILABLE flag +- Wake script path configurable via AIPASS_WAKE_SCRIPT env var +- BRANCH_REGISTRY path configurable via AIPASS_REGISTRY env var +- Daemon config path configurable via AIPASS_DAEMON_CONFIG env var +- filelock made optional (try/except, falls back to fcntl) +- handlers/__init__.py cross-branch guard removed (incompatible with public repo) +- JSON runtime data dir changed from assistant_json/ to daemon_json/ +- Entry point renamed from assistant.py to daemon.py +- Wakeup script renamed from assistant_wakeup.py to daemon_wakeup.py +- All ASSISTANT.local.json refs changed to DAEMON.local.json +- Plugin system preserved — 5 example plugins ported as templates + +--- + +## Completion Checklist + +### Definition of Done +- All assistant files ported to src/aipass/daemon/ with working imports +- Entry point renamed from assistant.py to daemon.py +- Zero references to internal paths (prax, cli, aipass_core, aipass_os, sys.path hacks) +- Telegram integration optional (graceful degradation) +- ai_mail dispatch optional (abstracted for external use) +- Plugin system preserved with example plugins +- JSON templates preserved for module initialization +- Tests ported and passing + +--- + +## Close Command + +When all boxes checked: +```bash +drone @flow close FPLAN-0005 +``` diff --git a/src/aipass/backup/__init__.py b/src/aipass/backup/__init__.py new file mode 100644 index 00000000..e69de29b diff --git a/src/aipass/backup/apps/__init__.py b/src/aipass/backup/apps/__init__.py new file mode 100644 index 00000000..73ab12a7 --- /dev/null +++ b/src/aipass/backup/apps/__init__.py @@ -0,0 +1 @@ +# Apps package - Branch application modules and handlers diff --git a/src/aipass/backup/apps/backup.py b/src/aipass/backup/apps/backup.py new file mode 100755 index 00000000..08e0290d --- /dev/null +++ b/src/aipass/backup/apps/backup.py @@ -0,0 +1,248 @@ +# ===================AIPASS==================== +# META DATA HEADER +# Name: backup.py - Backup System Entry Point +# Date: 2025-11-22 +# Version: 1.3.0 +# Category: core/backup +# +# CHANGELOG (Max 5 entries): +# - v1.3.0 (2026-03-06): Adapted for AIPass public repo +# * Removed shebang, sys.path manipulation, prax/cli imports +# * Replaced get_modules() with explicit relative imports +# * Uses standard logging and rich console +# - v1.2.0 (2026-02-22): Seed CLI flags standard: --version, backup all command +# - v1.1.0 (2025-11-22): Fixed META header - correct filename, category, and standards +# - v1.0.0 (2025-11-08): Initial version - modular architecture +# +# CODE STANDARDS: +# - Handlers implement logic, modules orchestrate +# ============================================= + +""" +Backup System - Main Entry Point + +Modular architecture with explicit module imports. +Main handles routing, modules implement functionality. +""" + +# Standard library imports +import sys +import argparse +import logging +from pathlib import Path +from typing import Any, List + +from rich.console import Console + +logger = logging.getLogger(__name__) +console = Console() + +# Explicit module imports (replaces dynamic discover_modules) +from .modules import backup_core, google_drive_sync, integrations, reauth_drive + + +def get_modules(): + """Return list of modules with handle_command() interface""" + modules = [] + for mod in [backup_core, google_drive_sync, integrations, reauth_drive]: + if hasattr(mod, 'handle_command'): + modules.append(mod) + return modules + + +# ============================================================================= +# CONSTANTS & CONFIG +# ============================================================================= + +# Module root (same directory as this file) +MODULE_ROOT = Path(__file__).parent + +# ============================================================================= +# COMMAND ROUTING +# ============================================================================= + +def route_command(args: argparse.Namespace, modules: List[Any]) -> bool: + """ + Route command to appropriate module + + Args: + args: Parsed command line arguments + modules: List of discovered modules + + Returns: + True if command was handled + """ + for module in modules: + try: + if module.handle_command(args): + return True + except Exception as e: + logger.error(f"Module error: {e}") + + return False + +# ============================================================================= +# INTROSPECTION DISPLAY +# ============================================================================= + +def print_introspection(): + """Display discovered modules - SEED pattern""" + console.print() + console.print("[bold cyan]Backup System - Automated File Protection[/bold cyan]") + console.print() + console.print("[dim]AIPass backup orchestration and versioning[/dim]") + console.print() + + # Discover modules + modules = get_modules() + + console.print(f"[yellow]Discovered Modules:[/yellow] {len(modules)}") + console.print() + + for module in modules: + module_name = module.__name__ if hasattr(module, '__name__') else str(module) + # Extract just the module filename (last part) + if '.' in module_name: + module_name = module_name.split('.')[-1] + console.print(f" [cyan]•[/cyan] {module_name}") + + console.print() + console.print("[dim]Run 'backup --help' for usage information[/dim]") + console.print() + + +# ============================================================================= +# MAIN +# ============================================================================= + +def show_version(): + """Print version from META DATA HEADER.""" + console.print("BACKUP_SYSTEM v1.2.0") + + +def main(): + """Main entry point - follows Seed CLI flags standard.""" + + # 1. No args → introspection + if len(sys.argv) == 1: + print_introspection() + return 0 + + # 2. Universal flags (checked before command routing, execute and exit) + if sys.argv[1] in ['--help', '-h', 'help']: + console.print() + console.print("[bold cyan]BACKUP_SYSTEM[/bold cyan] — Automated File Protection") + console.print() + console.print("[yellow]Commands:[/yellow]") + console.print(" --all Run snapshot + versioned + drive-sync (full backup cycle)") + console.print(" snapshot Create a system snapshot backup") + console.print(" versioned Create a versioned backup") + console.print(" drive-test Test Google Drive connectivity") + console.print(" drive-sync Sync backups to Google Drive") + console.print(" drive-sync --test Run a small test sync to verify integration") + console.print(" drive-stats Show Drive file tracker statistics") + console.print(" drive-clear-tracker Clear Drive file tracker cache") + console.print() + console.print("[yellow]Options:[/yellow]") + console.print(" --verbose, -v Extra diagnostic output") + console.print(" --dry-run Preview what would happen, execute nothing") + console.print(" --note NOTE Add a backup note/description") + console.print(" --project NAME Project name for Drive sync (default: AIPass)") + console.print(" --force Force sync all files (ignore change tracker)") + console.print(" --limit N Limit drive-sync to first N files") + console.print(" --version, -V Show version and exit") + console.print() + return 0 + + if sys.argv[1] in ['--version', '-V']: + show_version() + return 0 + + # Normalize --all flag → 'all' command (so both syntaxes work) + if sys.argv[1] == '--all': + sys.argv[1] = 'all' + + # 3. Parse args (behavioral flags handled by argparse) + parser = argparse.ArgumentParser( + description='backup_system Branch Operations', + formatter_class=argparse.RawDescriptionHelpFormatter + ) + parser.add_argument('command', nargs='?', help='Command to execute') + parser.add_argument('path', nargs='?', default=None, help='Path argument (for sync commands)') + parser.add_argument('--verbose', '-v', action='store_true', help='Verbose output') + parser.add_argument('--note', type=str, default='No note provided', help='Backup note/description') + parser.add_argument('--dry-run', action='store_true', help='Scan files without copying (test mode)') + parser.add_argument('--project', type=str, default='AIPass', help='Project name for Drive sync') + parser.add_argument('--force', action='store_true', help='Force sync all files') + parser.add_argument('--test', action='store_true', help='Run test mode (small test sync)') + parser.add_argument('--limit', type=int, default=0, help='Limit sync to first N files (0 = no limit)') + + args = parser.parse_args() + + if not args.command: + print_introspection() + return 0 + + # 4. Discover modules + modules = get_modules() + + if not modules: + console.print("❌ ERROR: No modules found") + return 1 + + # 5. Handle 'all' command: snapshot → versioned → drive-sync + if args.command == 'all': + console.print("[bold cyan]Full Backup Cycle: snapshot → versioned → drive-sync[/bold cyan]") + console.print() + + # Step 1: Snapshot + snapshot_args = argparse.Namespace( + command='snapshot', path=None, verbose=args.verbose, + note=args.note, dry_run=args.dry_run, project=args.project, + force=args.force, test=False, limit=0 + ) + if not route_command(snapshot_args, modules): + console.print("[red]Snapshot failed - aborting[/red]") + return 1 + + console.print() + console.print("[dim]─── snapshot complete, starting versioned ───[/dim]") + console.print() + + # Step 2: Versioned + versioned_args = argparse.Namespace( + command='versioned', path=None, verbose=args.verbose, + note=args.note, dry_run=args.dry_run, project=args.project, + force=args.force, test=False, limit=0 + ) + if not route_command(versioned_args, modules): + console.print("[red]Versioned backup failed - aborting[/red]") + return 1 + + console.print() + console.print("[dim]─── versioned complete, starting drive-sync ───[/dim]") + console.print() + + # Step 3: Drive sync + sync_args = argparse.Namespace( + command='drive-sync', path=None, verbose=args.verbose, + note=args.note, dry_run=args.dry_run, project=args.project, + force=args.force, test=False, limit=args.limit + ) + if not route_command(sync_args, modules): + console.print("[red]Drive sync failed[/red]") + return 1 + + console.print() + console.print("[green]Full backup cycle complete[/green]") + return 0 + + # 6. Route command to modules + if route_command(args, modules): + return 0 + else: + console.print(f"❌ ERROR: Unknown command: {args.command}") + return 1 + +if __name__ == "__main__": + sys.exit(main()) diff --git a/src/aipass/backup/apps/extensions/__init__.py b/src/aipass/backup/apps/extensions/__init__.py new file mode 100644 index 00000000..95322c94 --- /dev/null +++ b/src/aipass/backup/apps/extensions/__init__.py @@ -0,0 +1 @@ +# Extensions package - Drop-in extensions for branch functionality diff --git a/src/aipass/backup/apps/handlers/__init__.py b/src/aipass/backup/apps/handlers/__init__.py new file mode 100755 index 00000000..9e488e35 --- /dev/null +++ b/src/aipass/backup/apps/handlers/__init__.py @@ -0,0 +1 @@ +"""Backup system handlers package.""" diff --git a/src/aipass/backup/apps/handlers/config/__init__.py b/src/aipass/backup/apps/handlers/config/__init__.py new file mode 100755 index 00000000..e69de29b diff --git a/src/aipass/backup/apps/handlers/config/config_handler.py b/src/aipass/backup/apps/handlers/config/config_handler.py new file mode 100755 index 00000000..3b8c8aa1 --- /dev/null +++ b/src/aipass/backup/apps/handlers/config/config_handler.py @@ -0,0 +1,501 @@ +# ===================AIPASS==================== +# META DATA HEADER +# Name: config_handler.py - Backup system configuration and patterns +# Date: 2025-11-23 +# Version: 2.0.2 +# Category: handlers +# +# CHANGELOG (Max 5 entries): +# - v2.0.2 (2026-03-06): Adapted for AIPass public repo +# * Removed shebang and sys.path manipulation +# * Cleared CLI_TRACKING_PATTERNS (hardcoded internal paths removed) +# - v2.0.1 (2025-11-23): Added ignore patterns for log/data JSON files +# * Enabled *_data.json pattern to ignore backup_core_data.json +# * Enabled *_log.json pattern to ignore backup_core_log.json, drone_log.json +# * Enabled snapshot_backup.json pattern +# * Prevents frequent-change files from being backed up every run +# - v2.0.0 (2025-11-16): Seed standard extraction from archive +# * All configuration constants and patterns +# * Helper functions for pattern matching and filtering +# * Pure configuration - no complex logic +# - v1.0.0 (2025-10-14): Initial extraction from backup.py +# +# CODE STANDARDS: +# - Follow seed 3-layer architecture +# - Handlers must be independent and transportable +# - No cross-handler imports except within same domain +# ============================================= + +""" +Backup System Configuration Handler + +Centralized configuration management for backup system. +Contains all backup modes, ignore patterns, and configuration constants. +Pure configuration with helper functions for pattern access and filtering. +""" + +# ============================================= +# IMPORTS +# ============================================= + +from pathlib import Path +from typing import Dict, Set, List, Optional + +# ============================================= +# CONFIGURATION CONSTANTS +# ============================================= + +# Base backup directory - dynamically determined relative to branch root +# Module is in apps/handlers/config/, so parent.parent.parent.parent gets to branch root +BASE_BACKUP_DIR = str(Path(__file__).parent.parent.parent.parent / "backups") + +# Specific backup destinations for each system +BACKUP_DESTINATIONS = { + "system_snapshot": f"{BASE_BACKUP_DIR}", + "versioned_backup": f"{BASE_BACKUP_DIR}", +} + +# ============================================= +# IGNORE PATTERNS +# ============================================= + +# Global ignore patterns (what NOT to backup across all systems) +GLOBAL_IGNORE_PATTERNS = [ + # Python cache and temp files + "__pycache__", + "*.pyc", + "*.pyo", + + # Virtual environments + ".venv", + "venv", + + # Node.js + "node_modules", + "npm-debug.log", + "yarn-error.log", + + # Operating system files + ".DS_Store", + "Thumbs.db", + "desktop.ini", + "nul", # Windows reserved filename issue + + # Temporary files + #"temp", + ".temp", + "tmp", + ".tmp", + "*.tmp", + "*.temp", + + # Build and distribution directories + "build", + "dist", + "install", + "lib", + "bin", + + # IDE and editor files + "*.swp", + "*.swo", + "*~", + + # Backup directories (prevent recursive backups) - CRITICAL! + "backups", # Matches any directory named "backups" + "backup_system/backups", # Explicit path to our backup directory + "*/backups", # Any backups subdirectory + "system_snapshot", # Snapshot backup folder name + "versioned_backup", # Versioned backup folder name + "deleted_branches", # Deleted/archived branches (trash folder) + + # System backups and trash + "TimeShift*", + "timeshift*", + ".local/share/Trash", + "Trash", + + # Linux system directories (don't backup user settings/cache) + ".local", + ".cache", + ".config", + ".mozilla", + ".gnupg", + ".ssh", + ".vscode/cli", # VS Code CLI binary (large, regenerates) + ".vscode/extensions", # VS Code extensions (large, regenerates) + ".vscode-server", # VS Code remote server + ".npm-global", + ".pki", + ".dotnet", + ".var", # System application data (Flatpak, etc) + ".backup", # Old backup directories + ".antigravity", # AI assistant cache/data + ".gemini", # AI assistant cache + + # Version control (huge number of files!) + ".git", # Git repositories are version controlled elsewhere + + # IDE and editor directories + ".idea", # IntelliJ IDEA settings + ".eclipse", # Eclipse settings + + # Claude Code session files (change every session) - synced from .gitignore + #".claude/projects", + ".claude/todos", + ".claude/shell-snapshots", + ".claude/ide", + ".claude/statsig", + #".claude/plugins", + ".claude/.credentials.json", + ".claude/debug", + ".claude/file-history", + ".claude/history.jsonl", + ".claude/.update.lock", + ".claude/.projects", + #".claude/hooks/.last_diagnostics_file", + ".serena/logs", + ".code", + + # Development cache/build directories + ".npm", + ".cargo", + ".rustup", + ".gem", + ".gradle", + ".m2", + + # Application data we don't need in backups + ".thunderbird", + ".wine", + ".steam", + ".zoom", + + # User directories that shouldn't be backed up + "Downloads", + #"Music", + "Videos", + "Pictures", + "Dropbox", + "system_logs", + "external_repos", # External git repos - version controlled elsewhere + "mcp_servers/context7", # External MCP server repo + "mcp_servers/playwright-mcp", # External MCP server repo + "mcp_servers/serena", # External MCP server repo + "mcp_servers/servers", # External MCP server repo + "mcp_servers/dropbox", # External MCP server repo + + # Archive and compressed files + "*.zip", + "*.tar", + "*.gz", + "*.bz2", + "*.rar", + "*.7z", + "*.whl", # Python wheel files (long filenames in archives) + + # Large binary/image files (VMs, disk images) + "*.img", # Disk images (sandbox.img, etc - can be 50GB+) + "*.iso", # ISO files + "*.vmdk", # Virtual machine disk + "*.vdi", # VirtualBox disk image + "*.qcow2", # QEMU disk image + + # Image/media files (screenshots, icons, etc.) + "*.png", + "*.jpg", + "*.jpeg", + "*.gif", + "*.svg", + "*.webp", + "*.ico", + "*.bmp", + "*.tiff", + "*.tif", + + # Log files + "*.log", + "logs", + + # Most JSON files (frequent changes, not human-readable diffs) + "*_data.json", + "*_log.json", + "*_registry.json", + "snapshot_backup.json", + "snapshot_backup_changelog.json", + + # Miscellaneous + "*.db", + "*.bashrc", + "*.bash_history", + "*.bash_logout", + #"*.claude.json.backup", + #"*.env", + "*.lesshst", + "*.npmrc", + "*.sudo_as_admin_successful", +] + +# Notable patterns to highlight when skipped +CLI_TRACKING_PATTERNS = [] + +# Files that should be backed up but NOT generate diff files +DIFF_IGNORE_PATTERNS = [ + # Log files (append-only, huge diffs) + "*.log", + "*.logs", + "system_logs/*.log", + + # Most JSON files (frequent changes, not human-readable diffs) + #"*_config.json", + #"*_data.json", + #"*_log.json", + #"*_registry.json", + + # Python cache/compiled + "*.pyc", + "*.pyo", + "*.pyd", + "__pycache__/*", + + # Database files + "*.db", + "*.sqlite", + "*.sqlite3", + + # Binary/media files + "*.exe", + "*.dll", + "*.so", + "*.dylib", + + # Pickle/cache files + "*.pkl", + "*.pickle", + + # Temporary files + "*.tmp", + "*.temp", + "*.bak", + "~*", +] + +# Files that should be backed up despite matching ignore patterns +# Synced from .gitignore exceptions (! prefix patterns) +IGNORE_EXCEPTIONS = [ + ".gitignore", # Root .gitignore file should be backed up + "*.local.md", # AIPass session tracking files (caught by .local pattern but should be backed up) + ".vscode/settings.json", # VS Code settings + "*/.claude/settings.local.json", # Claude local settings in any directory + "tools/cleanup_configs/*.json", # Cleanup config files + "*_config.json", # All config JSON files + "drone/commands/global/*.json", # Drone global commands + ".claude.json", # Claude config + ".mcp.json", # MCP config + ".config/nerd-dictation/*", # Nerd dictation config + ".commands.json", # Commands config + "BRANCH_REGISTRY.json", # Core ecosystem registry - vital file + + # === TEMPLATES: FULL EXCEPTION - EVERY FILE === + "templates/**", # FULL EXCEPTION: Include EVERY file in templates directory + "*/templates/**", # Templates in any subdirectory + "templates/*/**", # All subdirectories and files in templates + "*/templates/*/**", # All subdirectories and files in templates anywhere + + # === TEMPLATE SUBDIRECTORIES (explicit) === + "templates/ai_branch_setup_template/ai_mail.local/**", + "templates/ai_branch_setup_template/logs/**", + "templates/ai_branch_setup_template/standards.local/**", + "templates/ai_branch_setup_template/.claude/**", + "templates/ai_branch_setup_template/.archive/**", + + # === MARKERS === + ".gitkeep", # Include all .gitkeep marker files (especially in templates) + ".gitattributes", # Git attributes files + ".local.json" +] + +# Files that SHOULD have diffs created (exceptions to ignore patterns) +DIFF_INCLUDE_PATTERNS = [ + "profile.json", + "pyrightconfig.json", + "package.json", + ".mcp.json", + "settings.json", + "settings.local.json", +] + +# ============================================= +# BACKUP MODE CONFIGURATIONS +# ============================================= + +BACKUP_MODES = { + 'snapshot': { + 'name': 'System Snapshot', + 'description': 'Dynamic instant backup (overwrites previous)', + 'destination': BACKUP_DESTINATIONS["system_snapshot"], + 'folder_name': 'system_snapshot', + 'behavior': 'dynamic', # overwrites previous + 'usage': 'Quick saves before changes' + }, + 'versioned': { + 'name': 'Versioned Backup', + 'description': 'Cumulative version history (keeps all file versions)', + 'destination': BACKUP_DESTINATIONS["versioned_backup"], + 'folder_name': 'versioned_backup', + 'behavior': 'versioned', # keeps all versions + 'usage': 'Complete file version history in single location' + }, +} + +# ============================================= +# HELPER FUNCTIONS +# ============================================= + +def get_ignore_patterns() -> List[str]: + """Get the global ignore patterns list + + Returns: + Copy of global ignore patterns list + """ + return GLOBAL_IGNORE_PATTERNS.copy() + + +def get_cli_tracking_patterns() -> List[str]: + """Get the CLI tracking patterns list + + Returns: + Copy of CLI tracking patterns list + """ + return CLI_TRACKING_PATTERNS.copy() + + +def get_backup_destination(system_name: str) -> str: + """Get backup destination for a specific backup system + + Args: + system_name: Name of the backup system + + Returns: + Path to backup destination, or base directory if not found + """ + return BACKUP_DESTINATIONS.get(system_name, BASE_BACKUP_DIR) + + +def filter_tracked_items(skipped_items: Dict[str, Set[str]]) -> Dict[str, Set[str]]: + """Filter skipped items to only show project-specific items + + Uses CLI tracking patterns to identify important items worth showing to user. + + Args: + skipped_items: Dictionary with 'directories' and 'files' sets + + Returns: + Filtered dictionary with only tracked items + """ + tracking_patterns = get_cli_tracking_patterns() + filtered_items = { + "directories": set(), + "files": set() + } + + def matches_pattern(item_path: str, patterns: List[str]) -> bool: + """Check if item path matches any tracking pattern""" + for pattern in patterns: + if pattern in item_path or item_path.startswith(pattern): + return True + # Check wildcard patterns + if pattern.startswith('*') and item_path.endswith(pattern[1:]): + return True + return False + + # Filter directories + for directory in skipped_items.get("directories", set()): + if matches_pattern(directory, tracking_patterns): + filtered_items["directories"].add(directory) + + # Filter files + for file_path in skipped_items.get("files", set()): + if matches_pattern(file_path, tracking_patterns): + filtered_items["files"].add(file_path) + + return filtered_items + + +def should_ignore(path: Path, ignore_patterns: Optional[List[str]] = None, + exceptions: Optional[List[str]] = None, + backup_dest: Optional[Path] = None) -> bool: + """Check if a file/folder should be ignored based on patterns. + + Centralizes ignore pattern matching logic used during backup scanning. + Checks exceptions first (files that should NOT be ignored), then patterns. + + Args: + path: Path object to check + ignore_patterns: List of patterns to ignore (defaults to GLOBAL_IGNORE_PATTERNS) + exceptions: List of exception patterns (defaults to IGNORE_EXCEPTIONS) + backup_dest: Optional backup destination to always ignore + + Returns: + True if path should be ignored, False otherwise + + Example: + # From backup engine + should_ignore(Path("/home/user/file.pyc")) # True + should_ignore(Path("/home/user/.gitignore")) # False (exception) + """ + import os + + # Use defaults if not provided + if ignore_patterns is None: + ignore_patterns = GLOBAL_IGNORE_PATTERNS + if exceptions is None: + exceptions = IGNORE_EXCEPTIONS + + path_str = str(path) + parts = set(path_str.split(os.sep)) + name = path.name + + # Always ignore backup destination if provided + if backup_dest and str(backup_dest) in path_str: + return True + + # Ignore paths containing 'Backups' + if 'Backups' in parts: + return True + + # Check exceptions first - files that should NOT be ignored + for exception in exceptions: + # Full path matching for template exceptions + if "**" in exception: + # Convert glob pattern to regex-like check + exception_parts = exception.split("/**")[0] # Get everything before /** + if exception_parts in path_str or exception_parts in "/".join(parts): + return False # Matches exception pattern - don't ignore + elif exception.startswith('*') and name.endswith(exception[1:]): + return False # Matches wildcard exception pattern + elif exception == name: + return False # Exact match + elif exception in path_str: + return False # Exception pattern is in the full path + + # Check ignore patterns + for pattern in ignore_patterns: + # Special case: "backups" should only match directory names, not filenames + if pattern == "backups": + if "backups" in parts: # Only ignore if "backups" is a directory in the path + return True + elif pattern == name: + return True + elif pattern.startswith('*') and name.endswith(pattern[1:]): + return True + elif pattern in parts or pattern in path_str: + return True + + return False + +# ============================================= +# MODULE INITIALIZATION +# ============================================= + +# No initialization needed - pure configuration diff --git a/src/aipass/backup/apps/handlers/diff/__init__.py b/src/aipass/backup/apps/handlers/diff/__init__.py new file mode 100755 index 00000000..e69de29b diff --git a/src/aipass/backup/apps/handlers/diff/diff_generator.py b/src/aipass/backup/apps/handlers/diff/diff_generator.py new file mode 100755 index 00000000..e441cf77 --- /dev/null +++ b/src/aipass/backup/apps/handlers/diff/diff_generator.py @@ -0,0 +1,120 @@ + +# ===================AIPASS==================== +# META DATA HEADER +# Name: diff_generator.py - Unified diff generation with binary detection +# Date: 2025-11-16 +# Version: 2.0.0 +# Category: handlers +# +# CHANGELOG (Max 5 entries): +# - v2.0.0 (2025-11-16): Extracted from backup_diff.py +# * Diff generation functionality +# * Binary file detection +# * Unified diff content generation +# * Updated imports to new handler locations +# +# CODE STANDARDS: +# - Follow seed 3-layer architecture +# - Handlers must be independent and transportable +# - No cross-handler imports except within same domain +# ============================================= + +""" +Diff Generation Handler + +Generates unified diffs between file versions with binary file detection. +Supports pattern-based filtering for diff creation. +""" + +# ============================================= +# IMPORTS +# ============================================= + +import datetime +import difflib +from pathlib import Path + +# Import from handlers +from ..utils.system_utils import safe_print +from ..config.config_handler import DIFF_IGNORE_PATTERNS, DIFF_INCLUDE_PATTERNS + +# ============================================= +# DIFF GENERATION +# ============================================= + +def should_create_diff(file_path: Path) -> bool: + """Check if file should have diffs created based on ignore patterns. + + Args: + file_path: Path to file to check + + Returns: + True if diff should be created, False otherwise + """ + # Check include patterns first (exceptions that should always have diffs) + for pattern in DIFF_INCLUDE_PATTERNS: + if file_path.match(pattern): + return True + + # Then check ignore patterns + for pattern in DIFF_IGNORE_PATTERNS: + if file_path.match(pattern): + return False + + # Default: create diff for all other files + return True + + +def is_binary_file(file_path: Path) -> bool: + """Check if a file is likely binary. + + Args: + file_path: Path to file to check + + Returns: + True if file appears to be binary, False otherwise + """ + try: + with open(file_path, 'rb') as f: + chunk = f.read(1024) + return b'\0' in chunk + except Exception: + return True # Assume binary if we can't read it + + +def generate_diff_content(old_file: Path, new_file: Path) -> str: + """Generate unified diff content between two files. + + Args: + old_file: Path to old version of file + new_file: Path to new version of file + + Returns: + Unified diff string showing changes + """ + try: + # Check if files are likely binary + if is_binary_file(old_file) or is_binary_file(new_file): + return f"Binary file {old_file.name} changed\n" + + # Read file contents + with open(old_file, 'r', encoding='utf-8', errors='replace') as f: + old_lines = f.readlines() + with open(new_file, 'r', encoding='utf-8', errors='replace') as f: + new_lines = f.readlines() + + # Generate unified diff + diff_lines = difflib.unified_diff( + old_lines, + new_lines, + fromfile=f"a/{old_file.name}", + tofile=f"b/{new_file.name}", + fromfiledate=datetime.datetime.fromtimestamp(old_file.stat().st_mtime).strftime('%Y-%m-%d %H:%M:%S'), + tofiledate=datetime.datetime.fromtimestamp(new_file.stat().st_mtime).strftime('%Y-%m-%d %H:%M:%S'), + lineterm='' + ) + + return '\n'.join(diff_lines) + + except Exception as e: + return f"Error generating diff: {e}\n" diff --git a/src/aipass/backup/apps/handlers/diff/version_manager.py b/src/aipass/backup/apps/handlers/diff/version_manager.py new file mode 100755 index 00000000..b0fb9269 --- /dev/null +++ b/src/aipass/backup/apps/handlers/diff/version_manager.py @@ -0,0 +1,151 @@ + +# ===================AIPASS==================== +# META DATA HEADER +# Name: version_manager.py - Version discovery and management +# Date: 2025-11-16 +# Version: 2.0.0 +# Category: handlers +# +# CHANGELOG (Max 5 entries): +# - v2.0.0 (2025-11-16): Extracted from backup_diff.py +# * Version file discovery functionality +# * Versioned file listing +# * Support for file-organized structure +# * Updated imports to new handler locations +# +# CODE STANDARDS: +# - Follow seed 3-layer architecture +# - Handlers must be independent and transportable +# - No cross-handler imports except within same domain +# ============================================= + +""" +Version Manager Handler + +Discovers and manages versioned backup files using the file-organized structure. +Supports version listing and filtering. +""" + +# ============================================= +# IMPORTS +# ============================================= + +import logging +from pathlib import Path +from typing import Dict, List, Optional + +logger = logging.getLogger(__name__) + +# Import from handlers +from ..utils.system_utils import safe_print + +# ============================================= +# VERSION FILE DISCOVERY +# ============================================= + +def get_versioned_files(backup_path: Path, file_path: str | None = None) -> Dict[str, List[str]]: + """Get all versioned files using new file-organized structure. + + Args: + backup_path: Path to backup root directory + file_path: Optional filter to search for specific file + + Returns: + Dictionary mapping file paths to lists of version timestamps + """ + versioned_files = {} + + if not backup_path.exists(): + return versioned_files + + # Normalize search path for cross-platform compatibility + normalized_search_path = None + if file_path: + normalized_search_path = str(Path(file_path)).replace('\\', '/').lower() + + # NEW STRUCTURE: Look for *_diffs folders + for diff_folder in backup_path.rglob('*_diffs'): + if not diff_folder.is_dir(): + continue + + # Extract base filename from diff folder name (remove _diffs suffix) + if not diff_folder.name.endswith('_diffs'): + continue + + base_filename = diff_folder.name[:-6] # Remove '_diffs' + + # Get relative path from backup root to the file + # diff_folder structure: .../filename/filename_diffs/ + file_folder = diff_folder.parent + if file_folder.name != base_filename: + continue # Skip if structure doesn't match expected pattern + + # Calculate base path for this file + rel_path_to_file_folder = file_folder.relative_to(backup_path) + base_path = str(rel_path_to_file_folder / base_filename) + + # Filter by search path if provided + if file_path: + normalized_base_path = str(base_path).replace('\\', '/').lower() + if normalized_search_path is not None and normalized_search_path not in normalized_base_path: + continue + + # Find all diff files in this folder + versions = [] + for diff_file in diff_folder.glob('*_v*.diff'): + filename = diff_file.name + if '_v' in filename and filename.endswith('.diff'): + # Extract version from filename: basename_v.diff + version_part = filename.split('_v')[1].replace('.diff', '') + versions.append(version_part) + + if versions: + versioned_files[base_path] = sorted(versions, reverse=True) # Newest first + + return versioned_files + + +def list_versioned_files(backup_path: Path) -> bool: + """List all versioned files with their version counts. + + Args: + backup_path: Path to backup root directory + + Returns: + True if files were found and listed, False otherwise + """ + try: + versioned_files = get_versioned_files(backup_path) + + if not versioned_files: + safe_print("No versioned files found") + return False + + safe_print(f"\n{'='*70}") + safe_print("VERSIONED FILES HISTORY") + safe_print('='*70) + + for file_path, versions in versioned_files.items(): + version_count = len(versions) + latest_version = versions[0] if versions else "unknown" + + safe_print(f" {file_path}") + safe_print(f" Versions: {version_count} | Latest: {latest_version}") + + # Show first 3 versions + for i, version in enumerate(versions[:3]): + marker = ">" if i == 0 else " " + safe_print(f" {marker} v{version}") + + if len(versions) > 3: + safe_print(f" ... and {len(versions) - 3} more versions") + safe_print("") + + safe_print('='*70) + safe_print(f"Use: python backup.py diff --file to see changes") + return True + + except Exception as e: + safe_print(f"Error listing versioned files: {e}") + logger.error(f"[version_manager] Failed to list versioned files: {e}") + return False diff --git a/src/aipass/backup/apps/handlers/diff/vscode_integration.py b/src/aipass/backup/apps/handlers/diff/vscode_integration.py new file mode 100755 index 00000000..51e9253b --- /dev/null +++ b/src/aipass/backup/apps/handlers/diff/vscode_integration.py @@ -0,0 +1,234 @@ + +# ===================AIPASS==================== +# META DATA HEADER +# Name: vscode_integration.py - VS Code diff viewer integration +# Date: 2025-11-16 +# Version: 2.0.0 +# Category: handlers +# +# CHANGELOG (Max 5 entries): +# - v2.0.0 (2025-11-16): Extracted from backup_diff.py +# * VS Code diff viewer integration +# * File comparison logic +# * Baseline vs current comparison +# * Updated imports to new handler locations +# +# CODE STANDARDS: +# - Follow seed 3-layer architecture +# - Handlers must be independent and transportable +# - No cross-handler imports except within same domain +# ============================================= + +""" +VS Code Integration Handler + +Opens file diffs in VS Code editor for visual comparison. +Supports baseline vs current and version-to-version comparisons. +""" + +# ============================================= +# IMPORTS +# ============================================= + +import sys +import os +import subprocess +import logging +from pathlib import Path + +logger = logging.getLogger(__name__) + +# Import from handlers +from ..utils.system_utils import safe_print +from ..diff.version_manager import get_versioned_files + +# ============================================= +# VS CODE INTEGRATION +# ============================================= + +def show_file_diff(backup_path: Path, source_dir: Path, file_path: str, version1: str | None = None, version2: str | None = None) -> bool: + """Show diff between two versions of a file in VS Code. + + Args: + backup_path: Path to backup root directory + source_dir: Path to source directory (for current file comparison) + file_path: File path to show diff for + version1: First version timestamp (optional) + version2: Second version timestamp (optional) + + Returns: + True if diff was opened successfully, False otherwise + """ + try: + versioned_files = get_versioned_files(backup_path, file_path) + + if not versioned_files: + safe_print(f"No versioned files found for: {file_path}") + return False + + # Find the matching file with normalized path comparison + matching_file = None + normalized_search_path = str(Path(file_path)).replace('\\', '/').lower() + for base_path in versioned_files: + normalized_base_path = str(base_path).replace('\\', '/').lower() + if normalized_search_path in normalized_base_path: + matching_file = base_path + break + + if not matching_file: + safe_print(f"File not found: {file_path}") + return False + + versions = versioned_files[matching_file] + + if len(versions) < 1: + safe_print(f"Only one version found for: {file_path}") + return False + + # Determine which versions to compare + if version1 and version2: + if version1 not in versions or version2 not in versions: + safe_print(f"Version not found. Available: {', '.join(versions)}") + return False + else: + # Compare latest version with current file, or two most recent versions + current_file = backup_path / matching_file + if current_file.exists() and len(versions) >= 1: + version1 = "current" + version2 = versions[0] # Most recent version + elif len(versions) >= 2: + version1 = versions[1] # Second most recent + version2 = versions[0] # Most recent + else: + safe_print(f"Need at least 2 versions to compare") + return False + + # Determine file paths for comparison using NEW STRUCTURE + # For VS Code diff, we need actual files, not diff patches + try: + # NEW STRUCTURE: matching_file is like "root/CLAUDE.local.md/CLAUDE.local.md" + # We need to find the file folder and look for baseline + current file + + # Extract the base filename for pattern matching + base_filename = Path(matching_file).name + file_folder_path = backup_path / Path(matching_file).parent + + # Look for baseline file in the file folder + # Handle files with extensions like CLAUDE.local.md -> CLAUDE.local-baseline-* + name_without_ext = base_filename.rsplit('.', 1)[0] if '.' in base_filename else base_filename + baseline_pattern = f"{name_without_ext}-baseline-*" + baseline_files = list(file_folder_path.glob(baseline_pattern)) + + # Current backup file path (in the organized structure) + current_backup_file = file_folder_path / base_filename + + # Source file path (live file in source directory) + # Convert backup path back to source path + matching_file_normalized = str(Path(matching_file)).replace('\\', '/') + if matching_file_normalized.startswith("root/"): + # Root level file: root/filename/filename -> filename + source_relative_path = base_filename + else: + # Nested file: folder/filename/filename -> folder/filename + source_relative_path = str(Path(matching_file).parent.parent / base_filename) + + source_file_path = source_dir / source_relative_path + + if baseline_files: + # ALWAYS prefer baseline comparison when available + file1_path = baseline_files[0] # Use first baseline found + file2_path = source_file_path # Current source + label1 = f"{base_filename} (baseline)" + label2 = f"{base_filename} (current)" + elif current_backup_file.exists(): + # No baseline, compare backup against current source + file1_path = current_backup_file # Backup version + file2_path = source_file_path # Live source + label1 = f"{base_filename} (backup)" + label2 = f"{base_filename} (current)" + + # Check if they're the same + if file1_path.exists() and file2_path.exists(): + if file1_path.stat().st_mtime == file2_path.stat().st_mtime: + safe_print(f"Note: Backup is identical to current file (no changes)") + safe_print(f" This happens when Google Drive sync updates the backup") + else: + safe_print(f"Could not find backup file in new structure: {matching_file}") + return False + + # Verify files exist + if not file1_path.exists(): + safe_print(f"File not found: {file1_path}") + return False + if not file2_path.exists(): + safe_print(f"File not found: {file2_path}") + return False + + except Exception as e: + safe_print(f"Error locating files: {e}") + logger.error(f"[vscode_integration] Error locating files for diff: {e}") + return False + + # Open diff in VS Code + safe_print(f"\n{'='*70}") + safe_print(f"Opening diff in VS Code: {matching_file}") + safe_print(f"Comparing: {label2} -> {label1}") + safe_print('='*70) + + # Launch VS Code with diff view + try: + # Try to find VS Code command + code_cmd = 'code' + if sys.platform == 'win32': + # On Windows, try common VS Code paths if 'code' not in PATH + import shutil + if not shutil.which('code'): + # Platform-specific VS Code detection + if sys.platform == 'win32': + # Windows VS Code path + vscode_path = Path(r"C:\Users") / os.environ.get('USERNAME', 'input') / r"AppData\Local\Programs\Microsoft VS Code\bin\code.cmd" + if vscode_path.exists(): + code_cmd = str(vscode_path) + else: + code_cmd = 'code.cmd' + else: + # Linux - VS Code is usually in PATH or use snap + if Path('/snap/bin/code').exists(): + code_cmd = '/snap/bin/code' + elif Path('/usr/bin/code').exists(): + code_cmd = '/usr/bin/code' + else: + code_cmd = 'code' + + # Use subprocess for better error handling + # Swap order so baseline (older) is on left, current (newer) is on right + result = subprocess.run( + [code_cmd, '--diff', str(file1_path), str(file2_path)], + capture_output=True, + text=True, + shell=True if sys.platform == 'win32' else False + ) + if result.returncode == 0: + safe_print(f"Diff opened in VS Code") + else: + safe_print(f"VS Code returned code {result.returncode}") + if result.stderr: + safe_print(f"Error: {result.stderr}") + except FileNotFoundError: + safe_print(f"VS Code command not found. Make sure 'code' is in your PATH") + safe_print(f"You can manually compare:") + safe_print(f" File 1: {file2_path}") + safe_print(f" File 2: {file1_path}") + return False + except Exception as e: + safe_print(f"Error launching VS Code: {e}") + logger.error(f"[vscode_integration] Error launching VS Code: {e}") + return False + + safe_print('='*70) + return True + + except Exception as e: + safe_print(f"Error generating diff: {e}") + logger.error(f"[vscode_integration] Error in show_file_diff: {e}") + return False diff --git a/src/aipass/backup/apps/handlers/json/__init__.py b/src/aipass/backup/apps/handlers/json/__init__.py new file mode 100755 index 00000000..4cc552bf --- /dev/null +++ b/src/aipass/backup/apps/handlers/json/__init__.py @@ -0,0 +1,12 @@ +"""JSON Handlers - Universal JSON operations for Seed branch""" + +from .changelog_handler import ( + load_changelog, + save_changelog_entry, + display_previous_comments +) + +from .backup_info_handler import ( + load_backup_info, + save_backup_info +) diff --git a/src/aipass/backup/apps/handlers/json/backup_info_handler.py b/src/aipass/backup/apps/handlers/json/backup_info_handler.py new file mode 100644 index 00000000..fc3e6d2a --- /dev/null +++ b/src/aipass/backup/apps/handlers/json/backup_info_handler.py @@ -0,0 +1,90 @@ + +# ===================AIPASS==================== +# META DATA HEADER +# Name: backup_info_handler.py - Backup state and statistics +# Date: 2025-11-16 +# Version: 1.0.0 +# Category: handlers +# +# CHANGELOG (Max 5 entries): +# - v1.0.0 (2025-11-16): Initial extraction from backup_core.py +# * load_backup_info() - Load mode-specific state +# * save_backup_info() - Persist backup state +# +# CODE STANDARDS: +# - Follow seed 3-layer architecture +# - Handlers must be independent and transportable +# - No cross-handler imports except within same domain +# ============================================= + +""" +Backup Info Handler + +Manages backup state and statistics persistence. +Handles mode-specific formats (dynamic vs versioned). + +Functions: + load_backup_info: Load mode-specific backup state + save_backup_info: Persist backup state to JSON +""" + +# ============================================= +# IMPORTS +# ============================================= + +import json +import datetime +from pathlib import Path +from typing import Dict + +# ============================================= +# BACKUP INFO OPERATIONS +# ============================================= + +def load_backup_info(backup_info_file: Path, mode_behavior: str) -> Dict: + """Load backup information from JSON file. + + Args: + backup_info_file: Path to backup_info JSON file + mode_behavior: 'dynamic' or 'versioned' + + Returns: + Dictionary with backup information (mode-specific format) + """ + if backup_info_file.exists(): + try: + with open(backup_info_file, 'r', encoding='utf-8', errors='replace') as f: + return json.load(f) + except Exception: + pass + + # Return mode-specific default structure + if mode_behavior == 'versioned': + return {"backups": []} + else: # dynamic + return {"last_backup": None, "file_timestamps": {}} + + +def save_backup_info(backup_info_file: Path, backup_info: Dict) -> bool: + """Save backup information to JSON file. + + Args: + backup_info_file: Path to backup_info JSON file + backup_info: Dictionary containing backup information + + Returns: + True if save succeeded, False otherwise + """ + try: + with open(backup_info_file, 'w', encoding='utf-8') as f: + json.dump(backup_info, f, indent=2, ensure_ascii=False) + return True + except Exception: + return False + + +# ============================================= +# MODULE INITIALIZATION +# ============================================= + +# No initialization needed - pure utility functions diff --git a/src/aipass/backup/apps/handlers/json/backup_metadata_builder.py b/src/aipass/backup/apps/handlers/json/backup_metadata_builder.py new file mode 100644 index 00000000..f82bd4a0 --- /dev/null +++ b/src/aipass/backup/apps/handlers/json/backup_metadata_builder.py @@ -0,0 +1,100 @@ + +# ===================AIPASS==================== +# META DATA HEADER +# Name: backup_metadata_builder.py - Backup metadata construction +# Date: 2025-11-18 +# Version: 1.0.0 +# Category: handlers +# +# CHANGELOG (Max 5 entries): +# - v1.0.0 (2025-11-18): Extracted from backup_core.py +# * Extracted backup_info creation logic +# * Handles mode-specific metadata formatting +# +# CODE STANDARDS: +# - Follow seed 3-layer architecture +# - Handlers must be independent and transportable +# - No cross-handler imports except within same domain +# ============================================= + +""" +Backup Metadata Builder - Constructs backup metadata structures + +Creates mode-specific backup information structures. +""" + +# ============================================= +# IMPORTS +# ============================================= + +import datetime +from pathlib import Path + +from ..models.backup_models import BackupResult + +# ============================================= +# METADATA CONSTRUCTION OPERATIONS +# ============================================= + +def create_backup_metadata(mode: str, behavior: str, backup_note: str, backup_folder_name: str, + backup_path: Path, source_dir: Path, result: BackupResult, + current_timestamps: dict, existing_backup_info: dict) -> dict: + """Create backup metadata structure based on mode. + + Args: + mode: Backup mode ('snapshot' or 'versioned') + behavior: Mode behavior ('dynamic' or 'versioned') + backup_note: User note for this backup + backup_folder_name: Backup folder name + backup_path: Backup destination path + source_dir: Source directory root + result: BackupResult with operation statistics + current_timestamps: Dict of current file timestamps + existing_backup_info: Existing backup info to append to + + Returns: + Complete backup info dict ready to save + """ + if behavior == 'versioned': + # Versioned: add to backup list + current_backup = { + "backup_note": backup_note, + "backup_name": backup_folder_name, + "timestamp": datetime.datetime.now().isoformat(), + "backup_path": str(backup_path), + "source_path": str(source_dir), + "mode": mode, + "stats": { + "files_checked": result.files_checked, + "files_copied": result.files_copied, + "files_added": result.files_added, + "files_skipped": result.files_skipped, + "errors": result.errors + } + } + existing_backup_info["backups"].insert(0, current_backup) + return existing_backup_info + else: + # Dynamic: update current state + return { + "backup_note": backup_note, + "last_backup": datetime.datetime.now().isoformat(), + "file_timestamps": current_timestamps, + "mode": mode, + "backup_path": str(backup_path), + "stats": { + "files_checked": result.files_checked, + "files_copied": result.files_copied, + "files_added": result.files_added, + "files_skipped": result.files_skipped, + "files_deleted": result.files_deleted, + "errors": result.errors + } + } + + +# ============================================= +# MODULE INITIALIZATION +# ============================================= + +# Pure handler - no initialization needed diff --git a/src/aipass/backup/apps/handlers/json/changelog_handler.py b/src/aipass/backup/apps/handlers/json/changelog_handler.py new file mode 100644 index 00000000..d3e703cc --- /dev/null +++ b/src/aipass/backup/apps/handlers/json/changelog_handler.py @@ -0,0 +1,157 @@ + +# ===================AIPASS==================== +# META DATA HEADER +# Name: changelog_handler.py - Backup changelog management +# Date: 2025-11-16 +# Version: 1.0.0 +# Category: handlers +# +# CHANGELOG (Max 5 entries): +# - v1.0.0 (2025-11-16): Initial extraction from backup_core.py +# * load_changelog() - Load persistent changelog +# * save_changelog_entry() - Add new entry +# * display_previous_comments() - Show history +# +# CODE STANDARDS: +# - Follow seed 3-layer architecture +# - Handlers must be independent and transportable +# - No cross-handler imports except within same domain +# ============================================= + +""" +Backup Changelog Handler + +Manages persistent changelog of backup operations with notes/comments. +Tracks all backup operations with timestamps and user notes for history tracking. + +Functions: + load_changelog: Load changelog from JSON file + save_changelog_entry: Add timestamped entry to changelog + display_previous_comments: Display recent entries to user +""" + +# ============================================= +# IMPORTS +# ============================================= + +import json +import logging +import datetime +from pathlib import Path +from typing import Dict + +logger = logging.getLogger(__name__) + +from rich.console import Console + +# Initialize console for output +console = Console() + +# ============================================= +# CHANGELOG OPERATIONS +# ============================================= + +def load_changelog(changelog_file: Path) -> Dict: + """Load persistent changelog of backup comments. + + Args: + changelog_file: Path to changelog JSON file + + Returns: + Dictionary with 'entries' list (empty dict if file missing) + """ + if changelog_file.exists(): + try: + with open(changelog_file, 'r', encoding='utf-8', errors='replace') as f: + return json.load(f) + except Exception as e: + console.print(f"[red]Error loading changelog: {e}[/red]") + logger.warning(f"Error loading changelog: {e}") + return {"entries": []} + + +def save_changelog_entry(changelog_file: Path, note: str, mode: str, + backup_path: Path) -> bool: + """Add new entry to persistent changelog. + + Args: + changelog_file: Path to changelog JSON file + note: User note describing the backup + mode: Backup mode ('snapshot' or 'versioned') + backup_path: Path to backup destination + + Returns: + True if save succeeded, False otherwise + """ + try: + changelog = load_changelog(changelog_file) + new_entry = { + "timestamp": datetime.datetime.now().isoformat(), + "note": note, + "mode": mode, + "backup_path": str(backup_path) + } + changelog["entries"].append(new_entry) + + with open(changelog_file, 'w', encoding='utf-8') as f: + json.dump(changelog, f, indent=2, ensure_ascii=False) + logger.info(f"[changelog_handler] Saved changelog entry: {note[:50]}") + return True + except Exception as e: + console.print(f"[red]Error saving changelog entry: {e}[/red]") + logger.error(f"[changelog_handler] Error saving changelog: {e}") + return False + + +def display_previous_comments(changelog_file: Path, mode_name: str): + """Display previous backup comments with mode identification. + + Args: + changelog_file: Path to changelog JSON file + mode_name: Display name of backup mode ('System Snapshot', etc.) + """ + try: + changelog = load_changelog(changelog_file) + entries = changelog.get("entries", []) + + if not entries: + console.print(f"[yellow]No previous {mode_name} backup comments found.[/yellow]") + return + + from rich.panel import Panel + console.print() + console.print(Panel(f"PREVIOUS {mode_name.upper()} BACKUP COMMENTS", + style="bold cyan", + border_style="cyan")) + + # Show last 10 entries (most recent first) + recent_entries = entries[-10:] + for i, entry in enumerate(reversed(recent_entries), 1): + try: + timestamp = datetime.datetime.fromisoformat(entry["timestamp"]) + formatted_time = timestamp.strftime("%Y-%m-%d %H:%M") + mode_info = entry.get('mode', 'unknown') + # Handle encoding issues in notes + note = str(entry['note']).encode('ascii', errors='replace').decode('ascii') + console.print(f"[bright_blue]{i:2d}.[/bright_blue] [{formatted_time}] [green][{mode_info}][/green] {note}") + except Exception as e: + console.print(f"[bright_blue]{i:2d}.[/bright_blue] [red][ERROR] Failed to display entry: {e}[/red]") + + if len(entries) > 10: + console.print(f"\n[dim]... and {len(entries) - 10} older entries[/dim]") + console.print() + except FileNotFoundError: + console.print(f"[yellow]No previous {mode_name} backup comments found.[/yellow]") + except PermissionError as e: + console.print(f"[yellow]Warning: Cannot read backup history - permission denied: {e}[/yellow]") + console.print("[yellow]Continuing with backup...[/yellow]") + except Exception as e: + console.print(f"[yellow]Warning: Error displaying comments: {e}[/yellow]") + console.print("[yellow]Continuing with backup...[/yellow]") + + +# ============================================= +# MODULE INITIALIZATION +# ============================================= + +# No module-level initialization needed diff --git a/src/aipass/backup/apps/handlers/json/drive_sync_json.py b/src/aipass/backup/apps/handlers/json/drive_sync_json.py new file mode 100644 index 00000000..ba30ed78 --- /dev/null +++ b/src/aipass/backup/apps/handlers/json/drive_sync_json.py @@ -0,0 +1,236 @@ + +# ===================AIPASS==================== +# META DATA HEADER +# Name: drive_sync_json.py - Google Drive Sync JSON Operations Handler +# Date: 2026-02-20 +# Version: 1.0.0 +# Category: backup_system/handlers/json +# +# CHANGELOG (Max 5 entries): +# - v1.0.0 (2026-02-20): Extracted from google_drive_sync module +# * Moved JSON I/O functions to handler layer +# * Follows seed 3-layer architecture standards +# +# CODE STANDARDS: +# - Handlers must be independent and transportable +# - No prax imports (handler tier) +# - Pure file I/O operations +# ============================================= + +""" +Google Drive Sync JSON Operations Handler + +Handles all JSON file read/write operations for the google_drive_sync module. +Provides load/save functions for config, data, and log files. +""" + +import copy +import fcntl +import json +import os +import tempfile +import time +import threading +from datetime import datetime +from pathlib import Path +from typing import Any, Dict, Optional + +# Module JSON file paths (resolved by caller) +_log_lock = threading.Lock() + + +def atomic_json_write(file_path: Path, data: Any) -> None: + """Write JSON atomically via temp file + rename to prevent corruption. + + Args: + file_path: Target JSON file path + data: Data to serialize as JSON + + Raises: + Exception: If write fails after cleanup attempt + """ + fd, tmp_path = tempfile.mkstemp(suffix='.tmp', dir=str(file_path.parent)) + try: + with os.fdopen(fd, 'w', encoding='utf-8') as f: + json.dump(data, f, indent=2) + os.replace(tmp_path, str(file_path)) + except Exception: + try: + os.unlink(tmp_path) + except OSError: + pass + raise + + +def load_config(config_file: Path) -> Dict[str, Any]: + """Load module configuration from JSON file. + + Args: + config_file: Path to the config JSON file + + Returns: + Dict with config data, or empty dict on error + """ + try: + if config_file.exists(): + with open(config_file, 'r', encoding='utf-8') as f: + return json.load(f) + return {} + except Exception: + return {} + + +def save_config(config_file: Path, config: Dict[str, Any]) -> None: + """Save module configuration to JSON file. + + Args: + config_file: Path to the config JSON file + config: Config data to save + + Raises: + Exception: If save fails + """ + config_file.parent.mkdir(parents=True, exist_ok=True) + atomic_json_write(config_file, config) + + +def load_data(data_file: Path) -> Dict[str, Any]: + """Load runtime data from JSON file. + + Args: + data_file: Path to the data JSON file + + Returns: + Dict with runtime data, or empty dict on error + """ + try: + if data_file.exists(): + with open(data_file, 'r', encoding='utf-8') as f: + return json.load(f) + return {} + except Exception: + return {} + + +def save_data(data_file: Path, data: Dict[str, Any]) -> None: + """Save runtime data to JSON file with deepcopy protection. + + Args: + data_file: Path to the data JSON file + data: Runtime data to save + + Raises: + RuntimeError: If deepcopy fails after 3 retries + """ + data_file.parent.mkdir(parents=True, exist_ok=True) + snapshot: Dict[str, Any] = {} + for attempt in range(3): + try: + snapshot = copy.deepcopy(data) + break + except RuntimeError: + if attempt < 2: + time.sleep(0.05 * (attempt + 1)) + else: + raise + snapshot["last_updated"] = datetime.now().isoformat() + atomic_json_write(data_file, snapshot) + + +def load_log(log_file: Path, max_retries: int = 3) -> Dict[str, Any]: + """Load operation log from JSON file with cross-process locking and retry. + + Args: + log_file: Path to the log JSON file + max_retries: Number of retry attempts on decode error + + Returns: + Dict with log data and summary + """ + default_log: Dict[str, Any] = { + "entries": [], + "summary": {"total_entries": 0, "last_entry": None, "next_id": 1} + } + for attempt in range(max_retries): + try: + if log_file.exists() and log_file.stat().st_size > 0: + with open(log_file, 'r', encoding='utf-8') as f: + fcntl.flock(f, fcntl.LOCK_SH) + try: + return json.load(f) + finally: + fcntl.flock(f, fcntl.LOCK_UN) + else: + save_log(log_file, default_log) + return default_log + except json.JSONDecodeError: + if attempt < max_retries - 1: + time.sleep(0.1 * (attempt + 1)) + else: + return default_log + except Exception: + return default_log + return default_log + + +def save_log(log_file: Path, log: Dict[str, Any]) -> None: + """Save operation log to JSON file with cross-process locking. + + Args: + log_file: Path to the log JSON file + log: Log data to save + + Raises: + Exception: If save fails + """ + log_file.parent.mkdir(parents=True, exist_ok=True) + lock_path = log_file.with_suffix('.lock') + with open(lock_path, 'w') as lf: + fcntl.flock(lf, fcntl.LOCK_EX) + try: + atomic_json_write(log_file, log) + finally: + fcntl.flock(lf, fcntl.LOCK_UN) + + +def log_operation( + log_file: Path, + operation: str, + details: Dict[str, Any], + success: bool = True, + correlation_id: Optional[str] = None +) -> None: + """Log a business operation to the module's JSON log file. + + Args: + log_file: Path to the log JSON file + operation: Operation name/identifier + details: Dict with 'message', optional 'execution_time_ms', 'error_details' + success: True if operation succeeded + correlation_id: Optional correlation ID for tracing + """ + with _log_lock: + log = load_log(log_file) + + log_entry = { + "id": log["summary"].get("next_id", 1), + "timestamp": datetime.now().isoformat(), + "level": "INFO" if success else "ERROR", + "operation": operation, + "message": details.get("message", ""), + "success": success, + "execution_time_ms": details.get("execution_time_ms", 0), + "correlation_id": correlation_id + } + + if not success: + log_entry["error_details"] = details.get("error_details", {}) + + log["entries"].insert(0, log_entry) + log["entries"] = log["entries"][:100] + + log["summary"]["total_entries"] += 1 + log["summary"]["last_entry"] = log_entry["timestamp"] + log["summary"]["next_id"] = log_entry["id"] + 1 + + save_log(log_file, log) diff --git a/src/aipass/backup/apps/handlers/json/json_handler.py b/src/aipass/backup/apps/handlers/json/json_handler.py new file mode 100755 index 00000000..5d0f8b13 --- /dev/null +++ b/src/aipass/backup/apps/handlers/json/json_handler.py @@ -0,0 +1,263 @@ + +# ===================AIPASS==================== +# META DATA HEADER +# Name: json_handler.py - JSON Auto-Creating Handler +# Date: 2025-11-22 +# Version: 3.1.0 +# Category: backup_system/handlers/json +# +# CHANGELOG (Max 5 entries): +# - v3.1.0 (2026-01-31): Refactored to comply with error handling standards +# - v3.0.0 (2025-11-22): Updated to match Cortex template with all features +# - v2.0.0 (2025-11-16): Migrated to seed 3-layer architecture +# - v1.0.0 (2025-10-14): Initial implementation with auto-detection +# +# CODE STANDARDS: +# - Pure functions with proper error raising +# - No Prax imports (handler tier 3) +# ============================================= + +import json +import inspect +from pathlib import Path +from datetime import datetime +from typing import Dict, Any, Optional + +# Constants +_BACKUP_ROOT = Path(__file__).resolve().parents[4] # src/aipass/backup/ +BACKUP_JSON_DIR = _BACKUP_ROOT / "backup_json" +JSON_TEMPLATES_DIR = Path(__file__).resolve().parents[2] / "json_templates" + + +def _get_caller_module_name() -> str: + """ + Auto-detect calling module name from call stack + + Returns: + Module name (e.g., "backup_core" from backup_core.py) + """ + stack = inspect.stack() + # Skip frames: [0]=this function, [1]=log_operation, [2]=actual caller + if len(stack) > 2: + caller_frame = stack[2] + caller_path = Path(caller_frame.filename) + module_name = caller_path.stem + + # Validate module name + if module_name and not module_name.startswith('_'): + return module_name + + # Fallback + return "unknown" + + +def load_template(json_type: str, module_name: str) -> Any: + """Load JSON template from template file""" + template_path = JSON_TEMPLATES_DIR / "default" / f"{json_type}.json" + + if not template_path.exists(): + raise FileNotFoundError(f"Template not found: {template_path}") + + with open(template_path, 'r', encoding='utf-8') as f: + template = json.load(f) + + # Replace placeholders + template_str = json.dumps(template) + template_str = template_str.replace("{{MODULE_NAME}}", module_name) + template_str = template_str.replace("{{TIMESTAMP}}", datetime.now().date().isoformat()) + + return json.loads(template_str) + + +def validate_json_structure(data: Any, json_type: str) -> bool: + """Validate JSON structure matches expected type""" + if json_type == "config": + if not isinstance(data, dict): + return False + required = ["module_name", "version", "config"] + return all(key in data for key in required) + + elif json_type == "data": + if not isinstance(data, dict): + return False + required = ["created", "last_updated"] + return all(key in data for key in required) + + elif json_type == "log": + return isinstance(data, list) + + return False + + +def get_json_path(module_name: str, json_type: str) -> Path: + """Get path for module JSON file""" + filename = f"{module_name}_{json_type}.json" + return BACKUP_JSON_DIR / filename + + +def ensure_json_exists(module_name: str, json_type: str) -> bool: + """Ensure JSON file exists, create from template if missing""" + BACKUP_JSON_DIR.mkdir(parents=True, exist_ok=True) + + json_path = get_json_path(module_name, json_type) + + if json_path.exists(): + try: + with open(json_path, 'r', encoding='utf-8') as f: + data = json.load(f) + + if validate_json_structure(data, json_type): + return True + # If corrupted, fall through to regenerate + except Exception: + # If unreadable, fall through to regenerate + pass + + template = load_template(json_type, module_name) + + with open(json_path, 'w', encoding='utf-8') as f: + json.dump(template, f, indent=2, ensure_ascii=False) + return True + + +def load_json(module_name: str, json_type: str) -> Optional[Any]: + """Load JSON file, auto-create if missing""" + if not ensure_json_exists(module_name, json_type): + return None + + json_path = get_json_path(module_name, json_type) + + with open(json_path, 'r', encoding='utf-8') as f: + return json.load(f) + + +def save_json(module_name: str, json_type: str, data: Any) -> bool: + """Save JSON file""" + json_path = get_json_path(module_name, json_type) + + if not validate_json_structure(data, json_type): + raise ValueError(f"Invalid structure for {json_type} JSON") + + if json_type == "data" and isinstance(data, dict): + data["last_updated"] = datetime.now().date().isoformat() + + with open(json_path, 'w', encoding='utf-8') as f: + json.dump(data, f, indent=2, ensure_ascii=False) + return True + + +def ensure_module_jsons(module_name: str) -> bool: + """Ensure all 3 JSON files exist for a module""" + ensure_json_exists(module_name, "config") + ensure_json_exists(module_name, "data") + ensure_json_exists(module_name, "log") + return True + + +def log_operation(operation: str, data: Dict[str, Any] | None = None, module_name: str | None = None) -> bool: + """ + Add entry to module log with automatic rotation + + Auto-detects calling module if module_name not provided. + Implements config-controlled log limits to prevent unbounded growth. + When max_log_entries is reached, removes oldest entries (FIFO). + + Args: + operation: Operation name to log + data: Optional data dict + module_name: Optional module name (auto-detected if not provided) + + Returns: + True if successful, False otherwise + """ + # Auto-detect module name if not provided + if module_name is None: + module_name = _get_caller_module_name() + + ensure_module_jsons(module_name) + + # Load config to get max_log_entries + config = load_json(module_name, "config") + max_entries = 100 # Default + if config and "config" in config: + max_entries = config["config"].get("max_log_entries", 100) + + # Load existing log + log = load_json(module_name, "log") + if log is None: + log = [] + + # Create new entry + entry = { + "timestamp": datetime.now().isoformat(), + "operation": operation + } + + if data: + entry["data"] = data # type: ignore[assignment] + + # Add new entry + log.append(entry) + + # Rotate if exceeds max (keep most recent entries) + if len(log) > max_entries: + log = log[-max_entries:] + + return save_json(module_name, "log", log) + + +def increment_counter(module_name: str, counter_name: str, amount: int = 1) -> bool: + """Increment a counter in data JSON""" + ensure_module_jsons(module_name) + + data = load_json(module_name, "data") + if data is None: + return False + + if counter_name not in data: + data[counter_name] = 0 + + data[counter_name] += amount + + return save_json(module_name, "data", data) + + +def update_data_metrics(module_name: str, **metrics) -> bool: + """Update data metrics""" + ensure_module_jsons(module_name) + + data = load_json(module_name, "data") + if data is None: + return False + + for key, value in metrics.items(): + data[key] = value + + return save_json(module_name, "data", data) + + +if __name__ == "__main__": + from rich.console import Console + from rich.panel import Panel + + console = Console() + + console.print() + console.print(Panel.fit( + "[bold cyan]JSON HANDLER - Working Implementation[/bold cyan]", + border_style="bright_blue" + )) + console.print() + console.print("[yellow]TESTING:[/yellow] Creating BACKUP_SYSTEM JSONs...") + + # Test auto-creation + log_operation("test_operation", {"test": "data"}, "backup_system") + increment_counter("backup_system", "test_counter", 1) + update_data_metrics("backup_system", test_metric="working") + + console.print() + console.print("[green]Check backup_json/ directory for created files:[/green]") + console.print(" [dim]-[/dim] backup_system_config.json") + console.print(" [dim]-[/dim] backup_system_data.json") + console.print(" [dim]-[/dim] backup_system_log.json") + console.print() diff --git a/src/aipass/backup/apps/handlers/json/statistics_handler.py b/src/aipass/backup/apps/handlers/json/statistics_handler.py new file mode 100644 index 00000000..18293fd9 --- /dev/null +++ b/src/aipass/backup/apps/handlers/json/statistics_handler.py @@ -0,0 +1,119 @@ + +# ===================AIPASS==================== +# META DATA HEADER +# Name: statistics_handler.py - Backup statistics tracking +# Date: 2025-11-18 +# Version: 1.0.0 +# Category: handlers +# +# CHANGELOG (Max 5 entries): +# - v1.0.0 (2025-11-18): Extracted from backup_core.py +# * Extracted update_data_file() statistics tracking +# * Handles runtime state and statistics updates +# +# CODE STANDARDS: +# - Follow seed 3-layer architecture +# - Handlers must be independent and transportable +# - No cross-handler imports except within same domain +# ============================================= + +""" +Statistics Handler - Backup statistics and runtime state tracking + +Manages backup operation statistics and runtime state persistence. +""" + +# ============================================= +# IMPORTS +# ============================================= + +import logging +import datetime +from pathlib import Path + +logger = logging.getLogger(__name__) + +# Import handlers +from ..json.json_handler import load_json, save_json +from ..models.backup_models import BackupResult +from ..utils.system_utils import safe_print + +# ============================================= +# STATISTICS OPERATIONS +# ============================================= + +def update_data_file(backup_result: BackupResult): + """Update data file with backup statistics. + + Args: + backup_result: BackupResult instance with operation statistics + """ + try: + # Load current data + data = load_json("backup_core", "data") + if not data: + data = { + "last_updated": datetime.datetime.now().isoformat(), + "runtime_state": {}, + "statistics": {}, + "recent_backups": [] + } + + # Update runtime state + data["last_updated"] = datetime.datetime.now().isoformat() + if "runtime_state" not in data: + data["runtime_state"] = {} + data["runtime_state"]["current_status"] = "completed" if backup_result.success else "failed" + data["runtime_state"]["last_backup"] = datetime.datetime.now().isoformat() + data["runtime_state"]["active_mode"] = backup_result.mode + data["runtime_state"]["total_files_backed_up"] = backup_result.files_copied + data["runtime_state"]["backup_in_progress"] = False + + # Update statistics + if "statistics" not in data: + data["statistics"] = { + "total_backups": 0, + "successful_backups": 0, + "failed_backups": 0, + "snapshot_backups": 0, + "versioned_backups": 0, + "total_files_processed": 0 + } + + data["statistics"]["total_backups"] += 1 + if backup_result.success: + data["statistics"]["successful_backups"] += 1 + else: + data["statistics"]["failed_backups"] += 1 + + # Mode-specific counters + if backup_result.mode == "snapshot": + data["statistics"]["snapshot_backups"] += 1 + elif backup_result.mode == "versioned": + data["statistics"]["versioned_backups"] += 1 + + data["statistics"]["total_files_processed"] += backup_result.files_checked + + # Add to recent backups (keep last 10) + if "recent_backups" not in data: + data["recent_backups"] = [] + data["recent_backups"].append({ + "timestamp": datetime.datetime.now().isoformat(), + "mode": backup_result.mode, + "success": backup_result.success, + "files_copied": backup_result.files_copied, + "errors": backup_result.errors + }) + data["recent_backups"] = data["recent_backups"][-10:] # Keep last 10 + + save_json("backup_core", "data", data) + except Exception as e: + safe_print(f"Failed to update data file: {e}") + logger.error(f"[statistics_handler] Failed to update data file: {e}") + + +# ============================================= +# MODULE INITIALIZATION +# ============================================= + +# No module-level initialization needed diff --git a/src/aipass/backup/apps/handlers/models/__init__.py b/src/aipass/backup/apps/handlers/models/__init__.py new file mode 100755 index 00000000..e69de29b diff --git a/src/aipass/backup/apps/handlers/models/backup_models.py b/src/aipass/backup/apps/handlers/models/backup_models.py new file mode 100755 index 00000000..edc8d37f --- /dev/null +++ b/src/aipass/backup/apps/handlers/models/backup_models.py @@ -0,0 +1,108 @@ +# ===================AIPASS==================== +# META DATA HEADER +# Name: backup_models.py - Backup system data models and structures +# Date: 2025-11-16 +# Version: 2.0.1 +# Category: handlers +# +# CHANGELOG (Max 5 entries): +# - v2.0.1 (2026-03-06): Adapted for AIPass public repo +# * Removed shebang and sys.path manipulation +# - v2.0.0 (2025-11-16): Migrated to seed 3-layer architecture +# - v1.0.0 (2025-10-14): Initial extraction from backup.py +# +# CODE STANDARDS: +# - Follow seed 3-layer architecture +# - Handlers must be independent and transportable +# - No cross-handler imports except within same domain +# ============================================= + +""" +Backup System Data Models + +Shared data structures used across all backup modules. +Simple data containers with no complex logic or dependencies. +Implements pure data model pattern following seed architecture standards. +""" + +# ============================================= +# IMPORTS +# ============================================= + +from pathlib import Path +import datetime +from typing import List + +# ============================================= +# DATA MODELS +# ============================================= + + +class BackupResult: + """Result of a backup operation + + Tracks statistics and errors for backup operations. + Used by all modules to report success/failure and collect metrics. + + Attributes: + files_checked (int): Number of files checked during backup + files_copied (int): Number of files successfully copied + files_added (int): New files added in versioned mode + files_skipped (int): Number of files skipped + files_deleted (int): Number of files deleted + errors (int): Total error count + error_details (List[str]): Detailed error messages + warnings (List[str]): Non-critical warnings + critical_errors (List[str]): Critical errors that failed the backup + start_time (datetime.datetime): When backup operation started + backup_path (str): Path to backup destination + mode (str): Backup mode (e.g., 'full', 'versioned') + success (bool): Overall success status of backup + """ + + def __init__(self): + # File statistics + self.files_checked: int = 0 + self.files_copied: int = 0 + self.files_added: int = 0 # New files added (versioned mode) + self.files_skipped: int = 0 + self.files_deleted: int = 0 + + # Error tracking + self.errors: int = 0 + self.error_details: List[str] = [] + self.warnings: List[str] = [] + self.critical_errors: List[str] = [] + + # Metadata + self.start_time = datetime.datetime.now() + self.backup_path: str = "" + self.mode: str = "" + self.success: bool = True + + def add_error(self, error_msg: str, is_critical: bool = False): + """Add an error to the result + + Args: + error_msg: Error message describing what failed + is_critical: If True, marks entire backup as failed + """ + self.errors += 1 + self.error_details.append(error_msg) + if is_critical: + self.critical_errors.append(error_msg) + self.success = False + + def add_warning(self, warning_msg: str): + """Add a warning to the result + + Args: + warning_msg: Warning message for non-critical issues + """ + self.warnings.append(warning_msg) + +# ============================================= +# MODULE INITIALIZATION +# ============================================= + +# Pure data models - no initialization needed diff --git a/src/aipass/backup/apps/handlers/operations/__init__.py b/src/aipass/backup/apps/handlers/operations/__init__.py new file mode 100755 index 00000000..e69de29b diff --git a/src/aipass/backup/apps/handlers/operations/drive_sync_client.py b/src/aipass/backup/apps/handlers/operations/drive_sync_client.py new file mode 100644 index 00000000..e96b087c --- /dev/null +++ b/src/aipass/backup/apps/handlers/operations/drive_sync_client.py @@ -0,0 +1,928 @@ + +# ===================AIPASS==================== +# META DATA HEADER +# Name: drive_sync_client.py - Google Drive Sync Client Handler +# Date: 2026-02-20 +# Version: 1.0.0 +# Category: backup_system/handlers/operations +# +# CHANGELOG (Max 5 entries): +# - v1.0.0 (2026-02-20): Extracted GoogleDriveSync class from module +# * Moved implementation to handler layer per seed 3-layer architecture +# * Module now delegates to this handler for all Drive operations +# +# CODE STANDARDS: +# - Handlers must be independent and transportable +# - No prax imports in class methods (handler tier) +# - Google API interactions isolated here +# ============================================= + +""" +Google Drive Sync Client Handler + +Contains the GoogleDriveSync class - the core implementation for +uploading backup files to Google Drive with OAuth2 authentication. +Called exclusively by the google_drive_sync module orchestrator. +""" + +# ============================================= +# IMPORTS +# ============================================= + +import sys +import copy +import fcntl +import json +import os +import ssl +import tempfile +import time +import threading +from concurrent.futures import ThreadPoolExecutor, as_completed +from datetime import datetime +from pathlib import Path +from typing import Optional, Dict, Any, List, Tuple + +# Google API imports +try: + from googleapiclient.discovery import build + from googleapiclient.http import MediaFileUpload + from google.auth.transport.requests import Request + from google.oauth2.credentials import Credentials + from google_auth_oauthlib.flow import InstalledAppFlow + GOOGLE_API_AVAILABLE = True +except ImportError: + GOOGLE_API_AVAILABLE = False + build = None + MediaFileUpload = None + Request = None + Credentials = None + InstalledAppFlow = None + +# JSON handler for data persistence +from ..json.drive_sync_json import ( + load_config as _load_config_fn, + save_config as _save_config_fn, + load_data as _load_data_fn, + save_data as _save_data_fn, + load_log as _load_log_fn, + save_log as _save_log_fn, + log_operation as _log_operation_fn, +) + +# ============================================= +# CONSTANTS +# ============================================= + +_BACKUP_ROOT = Path(__file__).resolve().parents[4] # src/aipass/backup/ +JSON_DIR = _BACKUP_ROOT / "backup_json" + +SCOPES = ['https://www.googleapis.com/auth/drive.file'] +MODULE_NAME = "google_drive_sync" +CONFIG_FILE = JSON_DIR / f"{MODULE_NAME}_config.json" +DATA_FILE = JSON_DIR / f"{MODULE_NAME}_data.json" +LOG_FILE = JSON_DIR / f"{MODULE_NAME}_log.json" +_log_lock = threading.Lock() + +# Convenience wrappers that use this module's JSON file paths +def _load_config(): + """Load config using module constants.""" + return _load_config_fn(CONFIG_FILE) + +def _save_config(config): + """Save config using module constants.""" + _save_config_fn(CONFIG_FILE, config) + +def _load_data(): + """Load data using module constants.""" + return _load_data_fn(DATA_FILE) + +def _save_data(data): + """Save data using module constants.""" + _save_data_fn(DATA_FILE, data) + +def _load_log(max_retries=3): + """Load log using module constants.""" + return _load_log_fn(LOG_FILE, max_retries) + +def _save_log(log): + """Save log using module constants.""" + _save_log_fn(LOG_FILE, log) + +def _log_operation(operation, details, success=True, correlation_id=None): + """Log operation using module constants.""" + _log_operation_fn(LOG_FILE, operation, details, success, correlation_id) + + +def _atomic_json_write(file_path: Path, data: Any): + """Write JSON atomically via temp file + rename to prevent corruption.""" + fd, tmp_path = tempfile.mkstemp(suffix='.tmp', dir=str(file_path.parent)) + try: + with os.fdopen(fd, 'w', encoding='utf-8') as f: + json.dump(data, f, indent=2) + os.replace(tmp_path, str(file_path)) + except Exception: + try: + os.unlink(tmp_path) + except OSError: + pass + raise + + +class GoogleDriveSync: + """Handles Google Drive integration for backup uploads""" + + def __init__(self): + """Initialize Google Drive sync with AIPass standards""" + self.config = _load_config() + self.data = _load_data() + + # Paths + self.creds_path = Path.home() / '.aipass' / 'drive_creds.json' + self.client_secrets_path = Path(__file__).parent.parent / 'credentials.json' + + # Runtime state + self._drive_service = None + self._thread_local = threading.local() + self._folder_cache_lock = threading.Lock() + self.backup_folder_id = None + self.project_folder_cache = self.data.get("runtime_state", {}).get("cached_folders", {}) + self.file_tracker = self.data.get("runtime_state", {}).get("file_tracker", {}) + + self.last_error = None # Last error message for callers + self.tracker_was_reset = False # Set True when new folder = tracker invalidated + + # Check if Google API is available + if not GOOGLE_API_AVAILABLE: + + self.config["config"]["enabled"] = False + _save_config(self.config) + + @property + def drive_service(self): + """Return thread-local service if set, otherwise the main service""" + return getattr(self._thread_local, 'service', None) or self._drive_service + + @drive_service.setter + def drive_service(self, value): + self._drive_service = value + + def authenticate(self) -> bool: + """Authenticate with Google Drive using OAuth credentials""" + if not GOOGLE_API_AVAILABLE: + python_version = f"{sys.version_info.major}.{sys.version_info.minor}.{sys.version_info.micro}" + + + return False + + start_time = datetime.now() + creds = None + + # Load existing credentials if they exist + if self.creds_path.exists(): + try: + creds = Credentials.from_authorized_user_file(str(self.creds_path), SCOPES) + except Exception as e: + + creds = None + + # If no valid credentials, start OAuth flow + if not creds or not creds.valid: + if creds and creds.expired and creds.refresh_token: + try: + creds.refresh(Request()) + + except Exception as e: + + creds = None + + if not creds: + # Check if client secrets file exists + if not self.client_secrets_path.exists(): + + + return False + + try: + + + flow = InstalledAppFlow.from_client_secrets_file( + str(self.client_secrets_path), SCOPES + ) + creds = flow.run_local_server(port=0) + + except Exception as e: + + _log_operation("authenticate", {"message": f"OAuth flow failed: {e}", "error_details": {"exception_type": type(e).__name__, "stack_trace": str(e)}}, success=False) + return False + + # Save credentials for next time + try: + # Create .aipass directory if it doesn't exist + self.creds_path.parent.mkdir(parents=True, exist_ok=True) + + # Save credentials + with open(self.creds_path, 'w', encoding='utf-8') as f: + f.write(creds.to_json()) + + except Exception as e: + pass + + # Build Drive service + try: + self.creds = creds # Store for building per-thread services + self.drive_service = build('drive', 'v3', credentials=creds) + + # Update runtime state (ensure runtime_state exists first) + if "runtime_state" not in self.data: + self.data["runtime_state"] = {} + self.data["runtime_state"]["authenticated"] = True + _save_data(self.data) + + # Log successful authentication + execution_time = int((datetime.now() - start_time).total_seconds() * 1000) + _log_operation("authenticate", {"message": "Successfully authenticated with Google Drive", "execution_time_ms": execution_time}, success=True) + + return True + except Exception as e: + + _log_operation("authenticate", {"message": f"Failed to build Drive service: {e}", "error_details": {"exception_type": type(e).__name__, "stack_trace": str(e)}}, success=False) + return False + + def get_storage_quota(self) -> Optional[Dict[str, Any]]: + """Get Google Drive storage quota information""" + try: + if not self.drive_service: + return None + + about = self.drive_service.about().get(fields="storageQuota").execute() + quota = about.get('storageQuota', {}) + + # Convert bytes to GB for readability + limit = int(quota.get('limit', 0)) + usage = int(quota.get('usage', 0)) + + limit_gb = limit / (1024**3) + usage_gb = usage / (1024**3) + free_gb = (limit - usage) / (1024**3) + percent_used = (usage / limit * 100) if limit > 0 else 0 + + return { + 'limit_gb': limit_gb, + 'usage_gb': usage_gb, + 'free_gb': free_gb, + 'percent_used': percent_used + } + except Exception as e: + + return None + + def _verify_folder_id(self, folder_id: str) -> bool: + """Verify a cached folder ID exists and is NOT trashed.""" + if not self.drive_service or not folder_id: + return False + try: + result = self._api_call_with_retry( + self.drive_service.files().get(fileId=folder_id, fields='id,trashed') + ) + return not result.get('trashed', False) + except Exception: + return False + + def get_or_create_backup_folder(self) -> Optional[str]: + """Get or create the main 'AIPass Backups' folder in Drive. + + Returns folder ID or None on failure. Caller must handle None. + """ + if self.backup_folder_id: + # Verify cached ID isn't trashed + if self._verify_folder_id(self.backup_folder_id): + return self.backup_folder_id + self.backup_folder_id = None + + try: + if not self.drive_service: + raise RuntimeError("Drive service not authenticated") + + # Search for existing folder (excludes trashed) + results = self._api_call_with_retry(self.drive_service.files().list( + q="name='AIPass Backups' and mimeType='application/vnd.google-apps.folder' and trashed=false", + fields="files(id, name)" + )) + + folders = results.get('files', []) + + if folders: + self.backup_folder_id = folders[0]['id'] + else: + # Creating NEW folder - old one is gone/trashed + folder_metadata = { + 'name': 'AIPass Backups', + 'mimeType': 'application/vnd.google-apps.folder' + } + folder = self._api_call_with_retry(self.drive_service.files().create( + body=folder_metadata, + fields='id' + )) + self.backup_folder_id = folder.get('id') + if not self.backup_folder_id: + raise RuntimeError("Drive API returned no folder ID after create") + + # Reset tracker - all old drive_ids point to dead files + old_count = len(self.file_tracker) + if old_count > 0: + self.file_tracker.clear() + self.project_folder_cache.clear() + if "runtime_state" in self.data: + self.data["runtime_state"]["file_tracker"] = {} + self.data["runtime_state"]["cached_folders"] = {} + self._save_file_tracker() + self.tracker_was_reset = True + _log_operation("tracker_reset", { + "message": f"New backup folder created - reset {old_count} tracker entries", + "old_tracker_count": old_count, + "new_folder_id": self.backup_folder_id + }) + + # Verify the folder is actually accessible + if not self._verify_folder_id(self.backup_folder_id): + raise RuntimeError(f"Backup folder {self.backup_folder_id} created but not accessible") + + return self.backup_folder_id + + except Exception as e: + self.last_error = f"Backup folder setup failed: {e}" + return None + + def get_or_create_project_folder(self, project_name: str) -> Optional[str]: + """Get or create project subfolder within AIPass Backups.""" + # Lock covers cache check + search + create to prevent duplicate folders + with self._folder_cache_lock: + if project_name in self.project_folder_cache: + folder_id = self.project_folder_cache[project_name] + if self._verify_folder_id(folder_id): + return folder_id + # Cached ID is trashed or gone - clear it + del self.project_folder_cache[project_name] + if project_name in self.data.get("runtime_state", {}).get("cached_folders", {}): + del self.data["runtime_state"]["cached_folders"][project_name] + _save_data(self.data) + + backup_folder_id = self.get_or_create_backup_folder() + if not backup_folder_id: + return None + + try: + if not self.drive_service: + return None + results = self._api_call_with_retry(self.drive_service.files().list( + q=f"name='{project_name}' and mimeType='application/vnd.google-apps.folder' and '{backup_folder_id}' in parents and trashed=false", + fields="files(id, name)" + )) + + folders = results.get('files', []) + + if folders: + project_folder_id = folders[0]['id'] + else: + folder_metadata = { + 'name': project_name, + 'mimeType': 'application/vnd.google-apps.folder', + 'parents': [backup_folder_id] + } + folder = self._api_call_with_retry(self.drive_service.files().create( + body=folder_metadata, + fields='id' + )) + project_folder_id = folder.get('id') + + self.project_folder_cache[project_name] = project_folder_id + return project_folder_id + + except Exception as e: + return None + + def get_or_create_nested_folder(self, parent_folder_id: str, folder_path: str) -> Optional[str]: + """Create nested folder structure in Google Drive""" + if not folder_path or folder_path == '.': + return parent_folder_id + + # Lock covers cache check + search + create to prevent duplicate folders + cache_key = f"{parent_folder_id}:{folder_path}" + with self._folder_cache_lock: + if cache_key in self.project_folder_cache: + folder_id = self.project_folder_cache[cache_key] + if self._verify_folder_id(folder_id): + return folder_id + # Cached ID is trashed or gone - clear it + del self.project_folder_cache[cache_key] + if cache_key in self.data.get("runtime_state", {}).get("cached_folders", {}): + del self.data["runtime_state"]["cached_folders"][cache_key] + _save_data(self.data) + + try: + path_parts = folder_path.split('/') + current_parent_id = parent_folder_id + + for part in path_parts: + if not part: + continue + + # Check cache for this intermediate path segment too + segment_key = f"{current_parent_id}:{part}" + if segment_key in self.project_folder_cache: + cached_id = self.project_folder_cache[segment_key] + if self._verify_folder_id(cached_id): + current_parent_id = cached_id + continue + del self.project_folder_cache[segment_key] + + if not self.drive_service: + return parent_folder_id + results = self._api_call_with_retry(self.drive_service.files().list( + q=f"name='{part}' and mimeType='application/vnd.google-apps.folder' and '{current_parent_id}' in parents and trashed=false", + fields="files(id, name)" + )) + + folders = results.get('files', []) + + if folders: + current_parent_id = folders[0]['id'] + else: + folder_metadata = { + 'name': part, + 'mimeType': 'application/vnd.google-apps.folder', + 'parents': [current_parent_id] + } + if not self.drive_service: + return parent_folder_id + folder = self._api_call_with_retry(self.drive_service.files().create( + body=folder_metadata, + fields='id' + )) + current_parent_id = folder.get('id') + + # Cache each intermediate segment + self.project_folder_cache[segment_key] = current_parent_id + + self.project_folder_cache[cache_key] = current_parent_id + return current_parent_id + + except Exception as e: + return parent_folder_id # Fallback to parent folder + + def upload_backup_file(self, local_file: Path, project_name: str, note: str = "", backup_root: Optional[Path] = None) -> bool: + """Upload a backup file to the appropriate project folder with nested structure""" + if not self.drive_service: + + return False + + project_folder_id = self.get_or_create_project_folder(project_name) + if not project_folder_id: + return False + + try: + # Calculate relative path from backup root to maintain folder structure + if backup_root and backup_root in local_file.parents: + relative_path = local_file.relative_to(backup_root) + folder_path = str(relative_path.parent) if relative_path.parent != Path('.') else "" + # Convert Windows paths to forward slashes for consistency + folder_path = folder_path.replace('\\', '/') + else: + folder_path = "" + + # Get or create the nested folder structure + target_folder_id = self.get_or_create_nested_folder(project_folder_id, folder_path) + if not target_folder_id: + + return False + + file_metadata = { + 'name': local_file.name, + 'parents': [target_folder_id], + 'description': f'AIPass backup - {note}' if note else 'AIPass backup' + } + + media = MediaFileUpload(str(local_file), resumable=True) + + # Get file tracker info to avoid API calls when possible + if backup_root and backup_root in local_file.parents: + relative_path = local_file.relative_to(backup_root) + file_key = str(relative_path).replace('\\', '/') + else: + file_key = local_file.name + + existing_file = None + tracked_file = self.file_tracker.get(file_key) + + # Use cached drive_id if file is tracked, otherwise find it + if tracked_file and tracked_file.get("drive_id"): + # File is tracked, use cached drive_id + existing_file = {"id": tracked_file["drive_id"]} + + else: + # File not tracked or no drive_id, search Drive + existing_file = self._find_existing_file(local_file.name, target_folder_id) + if existing_file is None and tracked_file: + # File was previously tracked but can't find it on Drive + # Skip rather than create duplicate - will retry next sync + return False + + # For files called by new sync system, we already know they need upload + # Skip the size comparison check to avoid redundant work + + if existing_file: + # Update existing file - don't include parents field for updates + update_metadata = { + 'name': local_file.name, + 'description': f'AIPass backup - {note}' if note else 'AIPass backup' + } + file = self._api_call_with_retry(self.drive_service.files().update( + fileId=existing_file['id'], + body=update_metadata, + media_body=media + )) + action = "Updated" + else: + # Create new file + file = self._api_call_with_retry(self.drive_service.files().create( + body=file_metadata, + media_body=media, + fields='id' + )) + action = "Uploaded" + + # Update file tracker with new drive_id + drive_file_id = file.get('id') if 'file' in locals() else (existing_file.get('id') if existing_file else None) + if drive_file_id and backup_root: + self._update_file_tracker(local_file, backup_root, drive_file_id) + + # Update statistics (ensure statistics exists first) + if "statistics" not in self.data: + self.data["statistics"] = { + "total_uploads": 0, + "successful_uploads": 0, + "failed_uploads": 0, + "total_bytes_uploaded": 0 + } + self.data["statistics"]["total_uploads"] += 1 + self.data["statistics"]["successful_uploads"] += 1 + self.data["statistics"]["total_bytes_uploaded"] += local_file.stat().st_size + # Stats saved to disk in batches by sync_backup_files(), not per-file + + # Log operation + _log_operation(f"upload_file", { + "message": f"{action} {local_file.name} to {project_name}", + "file_size": local_file.stat().st_size, + "project": project_name + }, success=True) + + return True + + except Exception as e: + + # Update failure statistics (ensure statistics exists first) + if "statistics" not in self.data: + self.data["statistics"] = { + "total_uploads": 0, + "successful_uploads": 0, + "failed_uploads": 0, + "total_bytes_uploaded": 0 + } + self.data["statistics"]["total_uploads"] += 1 + self.data["statistics"]["failed_uploads"] += 1 + _save_data(self.data) + + # Log failure + _log_operation("upload_file", { + "message": f"Failed to upload {local_file.name}", + "error_details": {"exception_type": type(e).__name__, "stack_trace": str(e)} + }, success=False) + return False + + def _find_existing_file(self, filename: str, parent_folder_id: str) -> Optional[Dict[str, Any]]: + """Find existing file in Drive folder with metadata""" + try: + if not self.drive_service: + return None + # Escape single quotes in filename for Drive API query + safe_filename = filename.replace("\\", "\\\\").replace("'", "\\'") + results = self._api_call_with_retry(self.drive_service.files().list( + q=f"name='{safe_filename}' and '{parent_folder_id}' in parents and trashed=false", + fields="files(id, name, size, modifiedTime, md5Checksum)" + )) + + files = results.get('files', []) + return files[0] if files else None + + except Exception as e: + + return None + + def _load_file_tracker(self) -> Dict[str, Dict[str, Any]]: + """Load file tracker from data JSON""" + return self.data.get("runtime_state", {}).get("file_tracker", {}) + + def _save_file_tracker(self): + """Save file tracker to data JSON""" + if "runtime_state" not in self.data: + self.data["runtime_state"] = {} + self.data["runtime_state"]["file_tracker"] = self.file_tracker + _save_data(self.data) + + def _build_thread_service(self): + """Build a separate Drive service instance for use in a worker thread. + + Loads fresh credentials from disk to avoid sharing credential state + (token refresh races) and creates a fully isolated HTTP/SSL connection. + """ + creds = Credentials.from_authorized_user_file(str(self.creds_path), SCOPES) + if creds and creds.expired and creds.refresh_token: + creds.refresh(Request()) + return build('drive', 'v3', credentials=creds) + + @staticmethod + def _is_ssl_error(exc: Exception) -> bool: + """Check if an exception is a transient SSL/connection error.""" + if isinstance(exc, (ssl.SSLError, BrokenPipeError, ConnectionResetError)): + return True + ssl_keywords = ( + 'DECRYPTION_FAILED_OR_BAD_RECORD_MAC', + 'WRONG_VERSION_NUMBER', + 'EOF occurred', + 'ssl.SSLError', + 'BrokenPipeError', + 'ConnectionReset', + ) + msg = str(exc) + return any(kw in msg for kw in ssl_keywords) + + def _api_call_with_retry(self, request, max_retries: int = 3): + """Execute a Google API request with exponential backoff on SSL errors.""" + for attempt in range(max_retries + 1): + try: + return request.execute() + except Exception as e: + if attempt < max_retries and self._is_ssl_error(e): + wait = 2 ** attempt + + time.sleep(wait) + # Rebuild this thread's service on SSL failure + self._thread_local.service = self._build_thread_service() + continue + raise + + def _check_file_needs_upload_local(self, local_file: Path, backup_root: Path) -> bool: + """Check if file needs upload using local tracker (no API calls)""" + try: + # Calculate relative path for tracking + if backup_root and backup_root in local_file.parents: + relative_path = local_file.relative_to(backup_root) + file_key = str(relative_path).replace('\\', '/') + else: + file_key = local_file.name + + # Get current file stats + current_size = local_file.stat().st_size + current_mtime = local_file.stat().st_mtime + + # Check if file is in tracker + if file_key not in self.file_tracker: + + return True # New file, needs upload + + tracked_file = self.file_tracker[file_key] + + # Compare local file stats with tracker + if (current_size != tracked_file.get("local_size", 0) or + current_mtime != tracked_file.get("local_mtime", 0)): + + return True # File changed, needs upload + + # File unchanged according to tracker + + return False + + except Exception as e: + + return True # On error, assume file needs upload + + def _update_file_tracker(self, local_file: Path, backup_root: Path, drive_file_id: str): + """Update file tracker after successful upload""" + try: + # Calculate relative path for tracking + if backup_root and backup_root in local_file.parents: + relative_path = local_file.relative_to(backup_root) + file_key = str(relative_path).replace('\\', '/') + else: + file_key = local_file.name + + # Get current file stats + current_size = local_file.stat().st_size + current_mtime = local_file.stat().st_mtime + + # Update tracker + self.file_tracker[file_key] = { + "local_size": current_size, + "local_mtime": current_mtime, + "drive_id": drive_file_id, + "drive_size": current_size, + "last_sync": datetime.now().isoformat() + } + + # Tracker updated in-memory; disk save is batched by sync_backup_files() + + except Exception as e: + pass + + def _clean_file_tracker(self, existing_files: set): + """Remove tracker entries for files that no longer exist""" + try: + keys_to_remove = [] + for file_key in self.file_tracker.keys(): + if file_key not in existing_files: + keys_to_remove.append(file_key) + + for key in keys_to_remove: + del self.file_tracker[key] + + if keys_to_remove: + self._save_file_tracker() + + except Exception as e: + pass + + def prepare_sync(self, backup_dir: Path, force_sync: bool = False, limit: int = 0): + """Scan files and determine what needs uploading. No API calls. + + Args: + backup_dir: Directory to scan + force_sync: Force upload all files regardless of tracker + limit: Only consider first N files (0 = no limit). Same N files every run. + + Returns: + tuple: (files_to_upload, skipped_count, total_count) + """ + if not backup_dir.exists(): + return [], 0, 0 + + self.file_tracker = self._load_file_tracker() + + # Collect all files first (sorted for deterministic order) + all_files = sorted( + [f for f in backup_dir.rglob("*") if f.is_file() and not f.name.startswith('.')], + key=lambda f: str(f) + ) + # Apply limit - always the same files every run + if limit > 0: + all_files = all_files[:limit] + + files_to_upload = [] + skipped_count = 0 + total_count = len(all_files) + existing_files = set() + + for backup_file in all_files: + if backup_dir in backup_file.parents: + relative_path = backup_file.relative_to(backup_dir) + file_key = str(relative_path).replace('\\', '/') + existing_files.add(file_key) + if force_sync or self._check_file_needs_upload_local(backup_file, backup_dir): + files_to_upload.append(backup_file) + else: + skipped_count += 1 + + # Only clean tracker when doing full sync (not limited) + if limit == 0: + self._clean_file_tracker(existing_files) + + return files_to_upload, skipped_count, total_count + + def _file_needs_upload(self, local_file: Path, drive_file: Optional[Dict[str, Any]]) -> bool: + """Check if a file needs to be uploaded based on size comparison""" + if not drive_file: + return True # File doesn't exist in Drive, needs upload + + try: + local_size = local_file.stat().st_size + drive_size = int(drive_file.get('size', 0)) + + # If sizes are different, file has changed + if local_size != drive_size: + return True + + # If sizes are same, assume file is unchanged (for now) + # TODO: Could add MD5 comparison for small files for better accuracy + return False + + except Exception as e: + + return True # Upload on error to be safe + + def sync_backup_files(self, backup_dir: Path, project_name: str, note: str = "", force_sync: bool = False, + prepared_files=None, skipped_count=0, total_count=0, progress_fn=None) -> dict: + """Sync backup files to Drive using local-first change detection. + + Args: + backup_dir: Directory containing backup files + project_name: Project name for Drive folder + note: Optional sync note + force_sync: Force upload all files regardless of tracker + prepared_files: Pre-scanned file list from prepare_sync(). If None, does own scan. + skipped_count: From prepare_sync() + total_count: From prepare_sync() + progress_fn: Callback fn(completed, total, success_count) called periodically + + Returns: + dict with keys: success, uploaded, skipped, failed, total + """ + result = {"success": False, "uploaded": 0, "skipped": skipped_count, "failed": 0, "total": total_count, "error": None} + + if not backup_dir.exists(): + result["error"] = f"Backup directory not found: {backup_dir}" + return result + + # Pre-flight: verify Drive folder is accessible before uploading anything + project_folder_id = self.get_or_create_project_folder(project_name) + if not project_folder_id: + result["error"] = self.last_error or f"Failed to create/access Drive folder for '{project_name}'" + return result + + # Phase 1: Use pre-scanned files if provided, otherwise scan locally + if prepared_files is not None: + files_to_upload = prepared_files + else: + files_to_upload, skipped_count, total_count = self.prepare_sync(backup_dir, force_sync) + result["skipped"] = skipped_count + result["total"] = total_count + + upload_count = len(files_to_upload) + + # Phase 2: Upload only the files that need it (WITH API CALLS) + success_count = 0 + completed_count = 0 + failed_count = 0 + + BATCH_SAVE_INTERVAL = 50 # Save tracker to disk every N uploads + MAX_WORKERS = 3 # Parallel upload threads (kept low to avoid SSL contention) + tracker_lock = threading.Lock() + + if upload_count > 0: + # Pre-create project folder before threads start (avoid duplicate creation) + self.get_or_create_project_folder(project_name) + + def _upload_one(backup_file): + """Upload a single file using a thread-local Drive service""" + # Each thread gets its own Drive service with isolated SSL connections + if not hasattr(self._thread_local, 'service'): + self._thread_local.service = self._build_thread_service() + upload_result = self.upload_backup_file(backup_file, project_name, note, backup_dir) + return (backup_file, upload_result) + + with ThreadPoolExecutor(max_workers=MAX_WORKERS) as executor: + futures = {executor.submit(_upload_one, f): f for f in files_to_upload} + for future in as_completed(futures): + _, upload_ok = future.result() + with tracker_lock: + completed_count += 1 + if upload_ok: + success_count += 1 + else: + failed_count += 1 + + # Batch save every BATCH_SAVE_INTERVAL files + if completed_count % BATCH_SAVE_INTERVAL == 0: + self._save_file_tracker() + _save_data(self.data) + + # Progress callback on every file for smooth progress bar + if progress_fn: + progress_fn(completed_count, upload_count, success_count) + + # Final save and progress update after all uploads complete + self._save_file_tracker() + _save_data(self.data) + if progress_fn: + progress_fn(completed_count, upload_count, success_count) + + # Update last sync time + if "runtime_state" not in self.data: + self.data["runtime_state"] = {} + self.data["runtime_state"]["last_sync"] = datetime.now().isoformat() + _save_data(self.data) + + # Log sync operation + total_success = success_count + skipped_count + _log_operation("sync_backup", { + "message": f"Synced {total_success}/{total_count} files for {project_name} ({success_count} uploaded, {skipped_count} skipped)", + "uploaded_count": success_count, + "skipped_count": skipped_count, + "total_count": total_count, + "project": project_name, + "force_sync": force_sync + }, success=(total_success == total_count)) + + result["success"] = (failed_count == 0) + result["uploaded"] = success_count + result["failed"] = failed_count + return result diff --git a/src/aipass/backup/apps/handlers/operations/drive_sync_ops.py b/src/aipass/backup/apps/handlers/operations/drive_sync_ops.py new file mode 100644 index 00000000..237e0440 --- /dev/null +++ b/src/aipass/backup/apps/handlers/operations/drive_sync_ops.py @@ -0,0 +1,143 @@ + +# ===================AIPASS==================== +# META DATA HEADER +# Name: drive_sync_ops.py - Google Drive Sync Business Operations Handler +# Date: 2026-02-20 +# Version: 1.0.0 +# Category: backup_system/handlers/operations +# +# CHANGELOG (Max 5 entries): +# - v1.0.0 (2026-02-20): Extracted from google_drive_sync module +# * Moved get_status, clear_file_tracker, show_file_tracker_stats, +# test_drive_sync to handler layer +# * Follows seed 3-layer architecture standards +# +# CODE STANDARDS: +# - Handlers must be independent and transportable +# - No prax imports (handler tier) +# - Delegates to json handler for file operations +# ============================================= + +""" +Google Drive Sync Business Operations Handler + +Implements status, tracker management, and test operations for the +google_drive_sync module. Called by the module orchestrator. +""" + +from pathlib import Path +from typing import Any, Dict + +from ..json.drive_sync_json import ( + load_config, + load_data, + save_data, + log_operation, +) + +# JSON file paths (resolved relative to backup root) +_BACKUP_ROOT = Path(__file__).resolve().parents[4] # src/aipass/backup/ +_JSON_DIR = _BACKUP_ROOT / "backup_json" +_MODULE_NAME = "google_drive_sync" +_CONFIG_FILE = _JSON_DIR / f"{_MODULE_NAME}_config.json" +_DATA_FILE = _JSON_DIR / f"{_MODULE_NAME}_data.json" +_LOG_FILE = _JSON_DIR / f"{_MODULE_NAME}_log.json" + + +def get_status() -> Dict[str, Any]: + """Get current module status for monitoring. + + Returns: + Dict with module status fields + """ + data = load_data(_DATA_FILE) + config = load_config(_CONFIG_FILE) + + return { + "name": _MODULE_NAME, + "category": "backup_system", + "enabled": config.get("config", {}).get("enabled", False), + "authenticated": data.get("runtime_state", {}).get("authenticated", False), + "last_sync": data.get("runtime_state", {}).get("last_sync"), + "statistics": data.get("statistics", {}) + } + + +def clear_file_tracker() -> bool: + """Clear the file tracker cache for a fresh sync. + + Returns: + bool: True if cleared successfully, False on error + """ + try: + data = load_data(_DATA_FILE) + if "runtime_state" in data and "file_tracker" in data["runtime_state"]: + data["runtime_state"]["file_tracker"] = {} + save_data(_DATA_FILE, data) + return True + except Exception: + return False + + +def get_file_tracker_stats() -> Dict[str, Any]: + """Get statistics about the file tracker. + + Returns: + Dict with tracker count and sample entries + """ + try: + data = load_data(_DATA_FILE) + tracker = data.get("runtime_state", {}).get("file_tracker", {}) + + sample = [] + for i, (file_key, info) in enumerate(list(tracker.items())[:5]): + last_sync = info.get("last_sync", "unknown")[:19] if info.get("last_sync") else "unknown" + sample.append({"file": file_key, "last_sync": last_sync}) + + return { + "total": len(tracker), + "sample": sample, + "truncated": len(tracker) > 5 + } + except Exception: + return {"total": 0, "sample": [], "truncated": False} + + +def test_drive_connection(sync_instance: Any) -> bool: + """Test Google Drive connectivity using an authenticated sync instance. + + Args: + sync_instance: An authenticated GoogleDriveSync instance + + Returns: + bool: True if connection test succeeded, False otherwise + """ + try: + folder_id = sync_instance.get_or_create_backup_folder() + if folder_id: + log_operation( + _LOG_FILE, + "test_sync", + {"message": "Drive sync test successful", "folder_id": folder_id}, + success=True + ) + return True + else: + log_operation( + _LOG_FILE, + "test_sync", + {"message": "Failed to create backup folder"}, + success=False + ) + return False + except Exception as e: + log_operation( + _LOG_FILE, + "test_sync", + { + "message": f"Test failed: {e}", + "error_details": {"exception_type": type(e).__name__, "stack_trace": str(e)} + }, + success=False + ) + return False diff --git a/src/aipass/backup/apps/handlers/operations/file_cleanup.py b/src/aipass/backup/apps/handlers/operations/file_cleanup.py new file mode 100644 index 00000000..2be34aaa --- /dev/null +++ b/src/aipass/backup/apps/handlers/operations/file_cleanup.py @@ -0,0 +1,177 @@ + +# ===================AIPASS==================== +# META DATA HEADER +# Name: file_cleanup.py - File deletion operations +# Date: 2025-11-23 +# Version: 1.0.2 +# Category: handlers +# +# CHANGELOG (Max 5 entries): +# - v1.0.2 (2025-11-23): Added dry-run support for cleanup operations +# * Added dry_run parameter to cleanup_deleted_files() +# * Shows "Would delete" messages instead of actually deleting in dry-run +# * All three passes (directories, files, empty dirs) respect dry-run mode +# - v1.0.1 (2025-11-23): CRITICAL BUG FIX - respect exceptions in empty dir cleanup +# * Third pass now checks should_ignore() before removing empty directories +# * Prevents deletion of empty template directories +# * Added source_dir mapping to check exceptions properly +# * Only removes empty dirs if source doesn't exist OR should be ignored +# - v1.0.0 (2025-11-18): Extracted from backup_core.py +# * Extracted file deletion logic for dynamic mode +# * Handles cleanup of deleted source files +# +# CODE STANDARDS: +# - Follow seed 3-layer architecture +# - Handlers must be independent and transportable +# - No cross-handler imports except within same domain +# ============================================= + +""" +File Cleanup - File deletion operations for dynamic backups + +Handles cleanup of backup files when source files are deleted. +""" + +# ============================================= +# IMPORTS +# ============================================= + +import logging +from pathlib import Path +from typing import Callable + +logger = logging.getLogger(__name__) + +from ..utils.system_utils import temporarily_writable, safe_print +from ..models.backup_models import BackupResult + +# ============================================= +# FILE CLEANUP OPERATIONS +# ============================================= + +def cleanup_deleted_files(backup_path: Path, source_dir: Path, should_ignore: Callable, + result: BackupResult, dry_run: bool = False) -> None: + """Clean up backup files when source files no longer exist (dynamic mode only). + + Args: + backup_path: Backup destination path + source_dir: Source directory root + should_ignore: Function to check if path should be ignored + result: BackupResult to track deletions and errors + dry_run: If True, only show what would be deleted without actually deleting + """ + import shutil + import stat + + if not backup_path.exists(): + return + + def handle_remove_readonly(func, path, exc_info): + """Error handler for shutil.rmtree to handle read-only files.""" + import os + # Make the file/directory writable (and executable if directory) and try again + # Also make parent directory writable + os.chmod(path, stat.S_IWRITE | stat.S_IREAD | stat.S_IEXEC) + parent = Path(path).parent + if parent.exists(): + os.chmod(parent, stat.S_IWRITE | stat.S_IREAD | stat.S_IEXEC) + func(path) + + try: + # Make backup_path writable for the duration of cleanup + backup_path.chmod(stat.S_IWRITE | stat.S_IREAD | stat.S_IEXEC) + + # First pass: Recursively check ALL directories and remove if ignored + # Sort by depth (shallowest first) so we can remove parents and skip children + all_dirs = sorted([d for d in backup_path.rglob('*') if d.is_dir()], + key=lambda p: len(p.parts)) + + removed_dirs = set() + + for backup_dir in all_dirs: + # Skip if this directory is inside a directory we already removed + if any(str(backup_dir).startswith(str(removed)) for removed in removed_dirs): + continue + + try: + rel_path = backup_dir.relative_to(backup_path) + source_dir_path = source_dir / rel_path + + # Check if source doesn't exist or should be ignored + if not source_dir_path.exists() or should_ignore(source_dir_path): + if dry_run: + safe_print(f"Would delete directory: {backup_dir}/") + else: + # Remove entire tree + shutil.rmtree(backup_dir, onerror=handle_remove_readonly) + safe_print(f"Deleted directory: {backup_dir}/") + removed_dirs.add(backup_dir) + except Exception as e: + # Don't fail entire backup on one directory error + pass + + # Second pass: delete individual files that no longer exist or should be ignored + for backup_file in backup_path.rglob('*'): + if backup_file.is_file(): + try: + rel_path = backup_file.relative_to(backup_path) + source_file = source_dir / rel_path + + if not source_file.exists() or should_ignore(source_file): + if dry_run: + safe_print(f"Would delete: {backup_file}") + else: + # Use context manager to handle read-only files before deletion + with temporarily_writable(backup_file.parent): + with temporarily_writable(backup_file): + backup_file.unlink() + safe_print(f"Deleted: {backup_file}") + result.files_deleted += 1 + except PermissionError as e: + error_msg = f"Permission denied deleting {backup_file}: {e}" + result.add_error(error_msg) + safe_print(f"{error_msg}") + except Exception as e: + error_msg = f"Error deleting {backup_file}: {e}" + result.add_warning(error_msg) + safe_print(f"{error_msg}") + + # Third pass: remove empty directories (bottom-up) + # CRITICAL FIX: Now checks should_ignore() to respect exceptions + all_dirs = [d for d in backup_path.rglob('*') if d.is_dir()] + all_dirs.sort(key=lambda p: len(p.parts), reverse=True) + + for backup_dir in all_dirs: + try: + if not any(backup_dir.iterdir()): # Empty directory + # Map to source directory to check if it should be preserved + rel_path = backup_dir.relative_to(backup_path) + source_dir_path = source_dir / rel_path + + # Only remove if source doesn't exist OR should be ignored + # This respects IGNORE_EXCEPTIONS (like templates/**) + if not source_dir_path.exists() or should_ignore(source_dir_path): + if dry_run: + safe_print(f"Would delete empty: {backup_dir}/") + else: + with temporarily_writable(backup_dir.parent): + backup_dir.chmod(stat.S_IWRITE | stat.S_IREAD | stat.S_IEXEC) + backup_dir.rmdir() + safe_print(f"Deleted empty: {backup_dir}/") + # else: Directory is in exceptions, preserve it even if empty + except OSError: + pass # Directory not empty or other OS error + except Exception as e: + pass # Silently skip other errors + + except Exception as e: + error_msg = f"Error scanning for deleted files: {e}" + result.add_warning(error_msg) + logger.warning(f"[file_cleanup] {error_msg}") + + +# ============================================= +# MODULE INITIALIZATION +# ============================================= + +# No module-level initialization needed diff --git a/src/aipass/backup/apps/handlers/operations/file_operations.py b/src/aipass/backup/apps/handlers/operations/file_operations.py new file mode 100755 index 00000000..3b26a260 --- /dev/null +++ b/src/aipass/backup/apps/handlers/operations/file_operations.py @@ -0,0 +1,353 @@ + +# ===================AIPASS==================== +# META DATA HEADER +# Name: file_operations.py - Core backup file operations +# Date: 2025-11-23 +# Version: 2.0.4 +# Category: handlers +# +# CHANGELOG (Max 5 entries): +# - v2.0.4 (2025-11-23): Fix baseline snapshot messages for VS Code clickability +# * Changed baseline snapshot message from {baseline_name} to {baseline_path} +# * Users can now Ctrl+click on baseline paths to jump directly to files +# * Shows full path (e.g., /home/aipass/backups/project/README-baseline-2025-11-23.md) +# - v2.0.3 (2025-11-23): CRITICAL performance fix for versioned backup +# * Skip copying unchanged files (only copy if mtime differs) +# * Fixes 5s -> <1s regression (was copying all 5000+ files every time) +# * Added file_changed flag to track copy necessity +# * Only copy if is_new_file or file_changed +# - v2.0.2 (2025-11-23): Added per-file output to snapshot mode +# * Snapshot backup now shows each file as it's copied +# * Displays "Copied (new)" or "Copied (updated)" for each file +# * Matches the verbosity of versioned backup mode +# - v2.0.1 (2025-11-23): Fixed automatic VS Code diff opening +# * Removed automatic VS Code integration from copy_versioned_file() +# * Diffs are now created silently without opening in editor +# * VS Code integration still available via separate command +# - v2.0.0 (2025-11-16): Extraction from backup_operations.py +# * Extracted core file operations (lines 49-271) +# * copy_file_with_structure for snapshot mode +# * copy_versioned_file with baseline snapshots and diff generation +# * Extensive Linux permission handling with temporarily_writable() +# * Retry logic for filesystem errors +# +# CODE STANDARDS: +# - Follow seed 3-layer architecture +# - Handlers must be independent and transportable +# - No cross-handler imports except within same domain +# ============================================= + +""" +File Operations Module - Core backup file handling + +Core file copy operations with version management and permission handling. +Handles both snapshot (simple copy) and versioned (with diffs) backup modes. + +Functions: + copy_file_with_structure: Copy file with directory structure (snapshot mode) + copy_versioned_file: Copy file with versioning and diff tracking +""" + +# ============================================= +# IMPORTS +# ============================================= + +import logging +import shutil +import datetime +from pathlib import Path + +logger = logging.getLogger(__name__) + +# Import from handlers modules +from ..utils.system_utils import temporarily_writable, safe_print +from ..diff.diff_generator import should_create_diff, generate_diff_content +from ..models.backup_models import BackupResult + +# ============================================= +# SNAPSHOT MODE OPERATIONS +# ============================================= + + +def copy_file_with_structure(source_file: Path, target_file: Path, backup_path: Path, result: BackupResult) -> bool: + """Copy file with directory structure (snapshot mode). + + Simple file copy operation with permission handling for Linux. + Creates the necessary directory structure and copies the source file + to the target location with proper permission handling. + + Args: + source_file (Path): Source file to copy + target_file (Path): Target destination path + backup_path (Path): Root backup path (for permission handling) + result (BackupResult): BackupResult instance to track errors + + Returns: + bool: True if copy succeeded, False otherwise + """ + try: + if len(str(target_file)) > 260: + error_msg = f"Path too long (>250 chars): {source_file}" + result.add_warning(error_msg) + safe_print(f" {error_msg}") + return False + + # Use context manager to handle read-only ancestors + # Find the first existing ancestor (could be parent or further up) + check_path = target_file.parent + while not check_path.exists() and check_path.parent != check_path: + check_path = check_path.parent + + # Make the first existing ancestor writable during mkdir + with temporarily_writable(check_path): + target_file.parent.mkdir(parents=True, exist_ok=True) + + if not source_file.exists(): + error_msg = f"Source file missing: {source_file}" + result.add_error(error_msg) + safe_print(f" {error_msg}") + return False + + # Track if this is a new file for appropriate messaging + is_new_file = not target_file.exists() + + # Wrap the copy operation with permission handling + with temporarily_writable(target_file.parent): + if target_file.exists(): + with temporarily_writable(target_file): + shutil.copy2(source_file, target_file) + else: + shutil.copy2(source_file, target_file) + + # Show feedback for each file copied (with clickable path) + if is_new_file: + safe_print(f" Copied (new): {source_file}") + else: + safe_print(f" Copied (updated): {source_file}") + + return True + except PermissionError as e: + error_msg = f"Permission denied copying {source_file}: {e}" + result.add_error(error_msg) + safe_print(f" {error_msg}") + logger.error(f"[file_operations] {error_msg}") + return False + except OSError as e: + error_msg = f"OS error copying {source_file}: {e}" + result.add_error(error_msg) + safe_print(f" {error_msg}") + logger.error(f"[file_operations] {error_msg}") + return False + except Exception as e: + error_msg = f"Unexpected error copying {source_file}: {e}" + result.add_error(error_msg, is_critical=True) + safe_print(f" CRITICAL: {error_msg}") + logger.error(f"[file_operations] CRITICAL: {error_msg}") + return False + +# ============================================= +# VERSIONED MODE OPERATIONS +# ============================================= + + +def copy_versioned_file(source_file: Path, target_file: Path, backup_path: Path, result: BackupResult) -> bool: + """Copy file with versioning - keep old versions as diffs when file changes. + + Creates baseline snapshots for new files and diff files for changes. + Implements the file-organized structure (file.py/file.py_diffs/file.py_v*.diff). + + The function handles: + - Baseline snapshot creation for new files + - Diff generation when existing files change + - Permission handling for read-only directories + - Retry logic for filesystem errors + - File state tracking in the BackupResult object + + Args: + source_file (Path): Source file to copy + target_file (Path): Target destination path + backup_path (Path): Root backup path (for permission handling) + result (BackupResult): BackupResult instance to track errors and new files + + Returns: + bool: True if copy succeeded, False otherwise + """ + try: + if len(str(target_file)) > 260: + error_msg = f"Path too long (>250 chars): {source_file}" + result.add_warning(error_msg) + safe_print(f" {error_msg}") + return False + + # Use context manager to handle read-only ancestors + # Find the first existing ancestor (could be parent or further up) + check_path = target_file.parent + while not check_path.exists() and check_path.parent != check_path: + check_path = check_path.parent + + # Make the first existing ancestor writable during mkdir + with temporarily_writable(check_path): + target_file.parent.mkdir(parents=True, exist_ok=True) + + if not source_file.exists(): + error_msg = f"Source file missing: {source_file}" + result.add_error(error_msg) + safe_print(f" {error_msg}") + return False + + # Track if this is a new file + is_new_file = not target_file.exists() + + # If this is a new file, save a baseline snapshot (full file) + if is_new_file: + try: + # Create baseline snapshot with Drive-friendly naming (hyphens + preserved extension) + timestamp = datetime.datetime.now().strftime("%Y-%m-%d") + + # Split filename and extension to preserve file type for Drive preview + name_parts = target_file.name.rsplit('.', 1) + if len(name_parts) == 2: + # Has extension: README.md -> README-baseline-2025-09-03.md + baseline_name = f"{name_parts[0]}-baseline-{timestamp}.{name_parts[1]}" + else: + # No extension: Makefile -> Makefile-baseline-2025-09-03 + baseline_name = f"{target_file.name}-baseline-{timestamp}" + # Baseline goes in same folder as the file + baseline_path = target_file.parent / baseline_name + + # Copy the full file as baseline with permission handling + with temporarily_writable(target_file.parent): + shutil.copy2(source_file, baseline_path) + safe_print(f" Created baseline snapshot: {baseline_path}") + + except Exception as e: + error_msg = f"Error creating baseline snapshot for {target_file}: {e}" + result.add_warning(error_msg) + safe_print(f" {error_msg}") + logger.warning(f"[file_operations] {error_msg}") + + # Track if file needs to be copied (changed or new) + file_changed = False + + # If target file exists and is different, create versioned diff + if not is_new_file and target_file.exists(): + try: + # Check if files are different + source_mtime = source_file.stat().st_mtime + target_mtime = target_file.stat().st_mtime + + if source_mtime != target_mtime: + # Files are different - mark for copying + file_changed = True + + # Check if we should create a diff + if should_create_diff(source_file): + # Create versioned diff showing changes + timestamp = datetime.datetime.fromtimestamp(target_mtime).strftime("%Y-%m-%d_%H-%M-%S") + + # NEW FILE-ORGANIZED STRUCTURE + # Create file-specific diff folder next to the file: file.py_diffs/ + file_diff_folder = target_file.parent / f"{target_file.name}_diffs" + with temporarily_writable(target_file.parent): + file_diff_folder.mkdir(parents=True, exist_ok=True) + + versioned_name = f"{target_file.name}_v{timestamp}.diff" + versioned_path = file_diff_folder / versioned_name + + # Generate diff between old (target) and new (source) versions + diff_content = generate_diff_content(target_file, source_file) + + # Write diff to versioned file with permission handling + with temporarily_writable(file_diff_folder): + with open(versioned_path, 'w', encoding='utf-8') as f: + f.write(diff_content) + + safe_print(f" Created diff: {versioned_path}") + else: + safe_print(f" Updated (no diff): {target_file.name}") + except Exception as e: + error_msg = f"Error creating version diff for {target_file}: {e}" + result.add_warning(error_msg) + safe_print(f" {error_msg}") + logger.warning(f"[file_operations] {error_msg}") + + # Only copy if file is new or changed (skip unchanged files for performance) + if is_new_file or file_changed: + try: + with temporarily_writable(target_file.parent): + if target_file.exists(): + with temporarily_writable(target_file): + shutil.copy2(source_file, target_file) + else: + shutil.copy2(source_file, target_file) + except (FileExistsError, OSError, PermissionError) as e: + # Handle various file system errors with context manager + try: + with temporarily_writable(target_file.parent): + if target_file.exists(): + with temporarily_writable(target_file): + target_file.unlink() # Remove existing file + # Try copy again + shutil.copy2(source_file, target_file) + except Exception as retry_error: + error_msg = f"Failed to copy after retry {source_file} -> {target_file}: {retry_error}" + result.add_error(error_msg) + safe_print(f" {error_msg}") + logger.error(f"[file_operations] {error_msg}") + return False + + # Track new files separately + if is_new_file: + result.files_added += 1 + + return True + except PermissionError as e: + error_msg = f"Permission denied copying versioned file {source_file}: {e}" + result.add_error(error_msg) + safe_print(f" {error_msg}") + logger.error(f"[file_operations] {error_msg}") + return False + except OSError as e: + error_msg = f"OS error copying versioned file {source_file}: {e}" + result.add_error(error_msg) + safe_print(f" {error_msg}") + logger.error(f"[file_operations] {error_msg}") + return False + except Exception as e: + error_msg = f"Unexpected error copying versioned file {source_file}: {e}" + result.add_error(error_msg, is_critical=True) + safe_print(f" CRITICAL: {error_msg}") + logger.error(f"[file_operations] {error_msg}") + return False + +# ============================================= +# FILE COMPARISON OPERATIONS +# ============================================= + +def file_needs_backup(source_file: Path, backup_file: Path, last_timestamps: dict, source_dir: Path) -> bool: + """Check if file needs backup based on modification time. + + Args: + source_file: Source file to check + backup_file: Backup destination file + last_timestamps: Dictionary of last backup timestamps + source_dir: Source directory root (for relative path calculation) + + Returns: + True if file needs backup, False otherwise + """ + if not backup_file.exists(): + return True + + source_mtime = source_file.stat().st_mtime + rel_path = str(source_file.relative_to(source_dir)) + + last_mtime = last_timestamps.get(rel_path, 0) + return source_mtime > last_mtime + + +# ============================================= +# MODULE INITIALIZATION +# ============================================= + +# No module-level initialization needed diff --git a/src/aipass/backup/apps/handlers/operations/file_scanner.py b/src/aipass/backup/apps/handlers/operations/file_scanner.py new file mode 100644 index 00000000..e605f250 --- /dev/null +++ b/src/aipass/backup/apps/handlers/operations/file_scanner.py @@ -0,0 +1,123 @@ + +# ===================AIPASS==================== +# META DATA HEADER +# Name: file_scanner.py - File system scanning operations +# Date: 2025-11-18 +# Version: 1.0.0 +# Category: handlers +# +# CHANGELOG (Max 5 entries): +# - v1.0.0 (2025-11-18): Extracted from backup_core.py +# * Extracted file scanning logic (os.walk loops) +# * Handles file discovery with ignore patterns +# * Returns file list with metadata +# +# CODE STANDARDS: +# - Follow seed 3-layer architecture +# - Handlers must be independent and transportable +# - No cross-handler imports except within same domain +# ============================================= + +""" +File Scanner - File system discovery and scanning + +Scans source directory for files to backup, applying ignore patterns. +""" + +# ============================================= +# IMPORTS +# ============================================= + +import os +from pathlib import Path +from typing import Callable + +# ============================================= +# FILE SCANNING OPERATIONS +# ============================================= + +def scan_files(source_dir: Path, should_ignore: Callable, show_progress: bool = False) -> tuple[list[Path], dict]: + """Scan source directory for files to backup. + + Args: + source_dir: Root directory to scan + should_ignore: Function to check if path should be ignored + show_progress: Whether to show progress spinner (default: False) + + Returns: + Tuple of (file_list, skipped_items_dict) + - file_list: List of Path objects for files to backup + - skipped_items: Dict with 'directories' and 'files' sets + """ + files_to_backup = [] + skipped_items = {"directories": set(), "files": set()} + + if show_progress: + from rich.progress import Progress, SpinnerColumn, TextColumn + from rich.console import Console + + console = Console() + + with Progress( + SpinnerColumn(), + TextColumn("[progress.description]{task.description}"), + transient=True, + console=console + ) as progress: + task = progress.add_task("Counting files...", total=None) + + # Walk directory tree + for dirpath, dirnames, filenames in os.walk(source_dir): + # Filter directories (modify in-place to prune walk) + original_dirs = dirnames.copy() + dirnames[:] = [d for d in dirnames if not should_ignore(Path(dirpath) / d)] + + # Track skipped directories + for d in original_dirs: + if d not in dirnames: + rel_dir = str(Path(dirpath).relative_to(source_dir) / d) + skipped_items["directories"].add(rel_dir) + + # Process files + for filename in filenames: + file_path = Path(dirpath) / filename + + if should_ignore(file_path): + rel_file = str(file_path.relative_to(source_dir)) + skipped_items["files"].add(rel_file) + else: + files_to_backup.append(file_path) + # Update spinner description with count occasionally + if len(files_to_backup) % 100 == 0: + progress.update(task, description=f"Found {len(files_to_backup)} files...") + else: + # Walk directory tree without progress display + for dirpath, dirnames, filenames in os.walk(source_dir): + # Filter directories (modify in-place to prune walk) + original_dirs = dirnames.copy() + dirnames[:] = [d for d in dirnames if not should_ignore(Path(dirpath) / d)] + + # Track skipped directories + for d in original_dirs: + if d not in dirnames: + rel_dir = str(Path(dirpath).relative_to(source_dir) / d) + skipped_items["directories"].add(rel_dir) + + # Process files + for filename in filenames: + file_path = Path(dirpath) / filename + + if should_ignore(file_path): + rel_file = str(file_path.relative_to(source_dir)) + skipped_items["files"].add(rel_file) + else: + files_to_backup.append(file_path) + + return files_to_backup, skipped_items + + +# ============================================= +# MODULE INITIALIZATION +# ============================================= + +# No initialization needed - pure utility functions diff --git a/src/aipass/backup/apps/handlers/operations/integration_ops.py b/src/aipass/backup/apps/handlers/operations/integration_ops.py new file mode 100644 index 00000000..6aa4de35 --- /dev/null +++ b/src/aipass/backup/apps/handlers/operations/integration_ops.py @@ -0,0 +1,118 @@ + +# ===================AIPASS==================== +# META DATA HEADER +# Name: integration_ops.py - Backup Integration Operations Handler +# Date: 2026-02-20 +# Version: 1.0.0 +# Category: backup_system/handlers/operations +# +# CHANGELOG (Max 5 entries): +# - v1.0.0 (2026-02-20): Extracted from integrations module +# * Moved sync_to_drive() and set_backup_readonly() to handler layer +# * Follows seed 3-layer architecture standards +# +# CODE STANDARDS: +# - Handlers must be independent and transportable +# - No prax imports (handler tier) +# - No cross-handler imports except within same domain +# ============================================= + +""" +Backup Integration Operations Handler + +Implementation logic for external integrations: +- Google Drive sync operations +- Backup directory read-only protection + +Called by the integrations module orchestrator. +""" + +import os +import stat +from pathlib import Path +from typing import Any, Optional + + +def sync_to_drive( + backup_path: Path, + source_dir: Path, + mode: str, + backup_note: str, + drive_sync_module: Any, + drive_sync_available: bool, +) -> bool: + """Sync versioned backups to Google Drive. + + Args: + backup_path: Path to backup directory to sync + source_dir: Source directory being backed up (for project name) + mode: Backup mode ('snapshot' or 'versioned') + backup_note: Optional note describing the backup + drive_sync_module: The GoogleDriveSync class (injected) + drive_sync_available: Whether google_drive_sync is importable + + Returns: + bool: True if sync succeeded, False otherwise + """ + if not drive_sync_available or drive_sync_module is None: + return False + + if mode != 'versioned': + return False + + try: + project_name = "AIPass" + drive_sync = drive_sync_module() + + if not drive_sync.authenticate(): + return False + + result = drive_sync.sync_backup_files( + backup_dir=backup_path, + project_name=project_name, + note=backup_note + ) + return result["success"] if isinstance(result, dict) else result + + except Exception: + return False + + +def set_backup_readonly(backup_path: Path) -> tuple[bool, str]: + """Set backup directory to read-only for protection. + + Applies read-only permissions recursively to all directories and files. + + Args: + backup_path: Path to backup directory to protect + + Returns: + Tuple of (success: bool, message: str) + """ + if not backup_path.exists(): + return False, f"Backup path does not exist: {backup_path}" + + try: + protected_dirs = 0 + protected_files = 0 + + for root, dirs, files in os.walk(str(backup_path)): + try: + os.chmod(root, stat.S_IRUSR | stat.S_IRGRP | stat.S_IROTH | + stat.S_IXUSR | stat.S_IXGRP | stat.S_IXOTH) + protected_dirs += 1 + except Exception: + pass + + for file in files: + file_path = os.path.join(root, file) + try: + os.chmod(file_path, stat.S_IRUSR | stat.S_IRGRP | stat.S_IROTH) + protected_files += 1 + except Exception: + pass + + return True, f"Protected {protected_dirs} dirs, {protected_files} files" + + except Exception as e: + return False, f"Could not set read-only protection: {e}" diff --git a/src/aipass/backup/apps/handlers/operations/path_builder.py b/src/aipass/backup/apps/handlers/operations/path_builder.py new file mode 100644 index 00000000..b631bc40 --- /dev/null +++ b/src/aipass/backup/apps/handlers/operations/path_builder.py @@ -0,0 +1,84 @@ + +# ===================AIPASS==================== +# META DATA HEADER +# Name: path_builder.py - Backup path construction +# Date: 2025-11-18 +# Version: 1.0.0 +# Category: handlers +# +# CHANGELOG (Max 5 entries): +# - v1.0.0 (2025-11-18): Extracted from backup_core.py +# * Extracted path construction logic for versioned mode +# * Handles long filename hashing +# * Mode-specific path building +# +# CODE STANDARDS: +# - Follow seed 3-layer architecture +# - Handlers must be independent and transportable +# - No cross-handler imports except within same domain +# ============================================= + +""" +Path Builder - Backup destination path construction + +Constructs destination paths based on backup mode and file structure. +""" + +# ============================================= +# IMPORTS +# ============================================= + +import hashlib +from pathlib import Path + +# ============================================= +# PATH CONSTRUCTION OPERATIONS +# ============================================= + +def build_backup_path(source_file: Path, source_dir: Path, backup_path: Path, mode: str) -> Path: + """Construct backup destination path based on mode. + + Args: + source_file: Source file path + source_dir: Source directory root + backup_path: Backup destination root + mode: Backup mode ('snapshot' or 'versioned') + + Returns: + Full backup file path + """ + rel_path = source_file.relative_to(source_dir) + + if mode == 'versioned': + # Versioned mode: file-based folders for version tracking + # Check if filename is too long (>50 chars) and needs shortening + if len(rel_path.name) > 50: + # Use shortened hash-based folder name for long filenames + name_hash = hashlib.md5(rel_path.name.encode()).hexdigest()[:8] + short_name = rel_path.name[:30] + f"_{name_hash}" + + if str(rel_path.parent) == ".": + file_folder = backup_path / "root" / short_name + else: + file_folder = backup_path / rel_path.parent / short_name + return file_folder / rel_path.name + else: + # Normal path structure for shorter filenames + if str(rel_path.parent) == ".": + # Root-level file: AGENTS.md -> root/AGENTS.md/AGENTS.md + file_folder = backup_path / "root" / rel_path.name + return file_folder / rel_path.name + else: + # Subfolder file: backup.py -> backup_system/backup.py/backup.py + file_folder = backup_path / rel_path.parent / rel_path.name + return file_folder / rel_path.name + else: + # Snapshot and other modes: keep original flat structure + return backup_path / rel_path + + +# ============================================= +# MODULE INITIALIZATION +# ============================================= + +# Pure handler - no initialization needed diff --git a/src/aipass/backup/apps/handlers/reporting/__init__.py b/src/aipass/backup/apps/handlers/reporting/__init__.py new file mode 100644 index 00000000..e69de29b diff --git a/src/aipass/backup/apps/handlers/reporting/report_formatter.py b/src/aipass/backup/apps/handlers/reporting/report_formatter.py new file mode 100644 index 00000000..4230d277 --- /dev/null +++ b/src/aipass/backup/apps/handlers/reporting/report_formatter.py @@ -0,0 +1,184 @@ + +# ===================AIPASS==================== +# META DATA HEADER +# Name: report_formatter.py - Backup result reporting +# Date: 2025-11-23 +# Version: 1.1.0 +# Category: handlers +# +# CHANGELOG (Max 5 entries): +# - v1.1.0 (2025-11-23): Added dry-run statistics display +# * Added dry_run parameter to display_backup_results() +# * Shows "Would copy", "Would skip", "Would delete" in dry-run mode +# * Provides clear differentiation between dry-run and actual execution +# - v1.0.0 (2025-11-18): Extracted from backup_core.py +# * Extracted statistics display formatting +# * Handles error/warning display +# * Formats skipped items reports +# +# CODE STANDARDS: +# - Follow seed 3-layer architecture +# - Handlers must be independent and transportable +# - No cross-handler imports except within same domain +# ============================================= + +""" +Report Formatter - Backup result formatting and display + +Formats and displays backup operation results, statistics, errors, and warnings. +""" + +# ============================================= +# IMPORTS +# ============================================= + +import datetime +from pathlib import Path + +from rich.console import Console + +console = Console() + +def _header(text): + console.print(f"\n[bold cyan]{'='*70}[/bold cyan]") + console.print(f"[bold cyan] {text}[/bold cyan]") + console.print(f"[bold cyan]{'='*70}[/bold cyan]") + +from ..models.backup_models import BackupResult + +# ============================================= +# REPORT FORMATTING OPERATIONS +# ============================================= + +def display_backup_results(result: BackupResult, mode_config: dict, backup_path: Path, + skipped_items: dict, filter_tracked_items_func, dry_run: bool = False) -> None: + """Display comprehensive backup results with statistics and errors. + + Args: + result: BackupResult with operation statistics + mode_config: Mode configuration dict + backup_path: Backup destination path + skipped_items: Dict of skipped directories and files + filter_tracked_items_func: Function to filter tracked items + dry_run: If True, show "would be" language in statistics + """ + duration = datetime.datetime.now() - result.start_time + + # Determine overall result status + if result.critical_errors: + status_style = "[bold red]" + status_icon = "!" + status_text = "FAILED" + elif result.errors > 0: + status_style = "[bold red]" + status_icon = "X" + status_text = "COMPLETED WITH ERRORS" + elif result.warnings: + status_style = "[bold yellow]" + status_icon = "!" + status_text = "COMPLETED WITH WARNINGS" + else: + status_style = "[bold green]" + status_icon = "OK" + status_text = "COMPLETED SUCCESSFULLY" + + # Use _header for main status + console.print() + _header(f"{status_icon} {mode_config['name'].upper()} {status_text}") + console.print() + + # Statistics Section + console.print("[bold cyan]STATISTICS:[/bold cyan]") + console.print(f" Files checked: [dim]{result.files_checked}[/dim]") + + if dry_run: + # Dry-run mode: show "would be" language + console.print(f" Would copy: [bold]{result.files_copied}[/bold] files") + console.print(f" Would skip: [dim]{result.files_skipped}[/dim] files (unchanged)") + if result.files_deleted > 0: + console.print(f" Would delete: [yellow]{result.files_deleted}[/yellow] files") + else: + # Normal mode: show actual actions + console.print(f" Files copied: [bold]{result.files_copied}[/bold]") + if result.mode == 'versioned' and result.files_added > 0: + console.print(f" Files added: [green]{result.files_added}[/green] (new)") + console.print(f" Files skipped: [dim]{result.files_skipped}[/dim]") + if result.mode == 'versioned' and result.files_deleted == 0: + console.print(f" Files deleted: [dim]{result.files_deleted}[/dim] [dim](cleanup disabled - keeps history)[/dim]") + else: + console.print(f" Files deleted: [red]{result.files_deleted}[/red]") + + console.print(f" Errors: {status_style}{result.errors}[/]") + console.print(f" Warnings: [yellow]{len(result.warnings)}[/yellow]") + console.print(f" Duration: [blue]{duration.total_seconds():.2f}s[/blue]") + console.print(f" Location: [dim]{backup_path}[/dim]") + + # Display detailed error information + if result.critical_errors: + console.print() + console.print(f"[bold red]CRITICAL ERRORS ({len(result.critical_errors)}):[/bold red]") + console.print("-" * 40) + for i, error in enumerate(result.critical_errors, 1): + console.print(f" {i}. [red]{error}[/red]") + console.print() + console.print("[bold]RECOVERY SUGGESTIONS:[/bold]") + console.print(" - Check disk space and permissions") + console.print(" - Ensure backup destination is accessible") + console.print(" - Try running as administrator if permission issues") + console.print(" - Check if antivirus is blocking file operations") + + elif result.error_details: + console.print() + console.print(f"[bold red]ERRORS ({len(result.error_details)}):[/bold red]") + console.print("-" * 40) + for i, error in enumerate(result.error_details[:10], 1): + console.print(f" {i}. [red]{error}[/red]") + if len(result.error_details) > 10: + console.print(f" [dim]... and {len(result.error_details) - 10} more errors[/dim]") + console.print() + console.print("[bold]SUGGESTIONS:[/bold]") + console.print(" - Some files may be in use - try closing applications") + console.print(" - Check file permissions on failed files") + + if result.warnings: + console.print() + console.print(f"[bold yellow]WARNINGS ({len(result.warnings)}):[/bold yellow]") + console.print("-" * 40) + for i, warning in enumerate(result.warnings[:5], 1): + console.print(f" {i}. [yellow]{warning}[/yellow]") + if len(result.warnings) > 5: + console.print(f" [dim]... and {len(result.warnings) - 5} more warnings[/dim]") + + # Display project-specific skipped items + tracked_items = filter_tracked_items_func(skipped_items) + total_tracked = len(tracked_items["directories"]) + len(tracked_items["files"]) + total_all_skipped = len(skipped_items["directories"]) + len(skipped_items["files"]) + + if total_tracked > 0: + console.print() + console.print(f"[bold cyan]NOTABLE SKIPPED ITEMS ({total_tracked}):[/bold cyan]") + console.print(f"[dim]Total ignored: {total_all_skipped}[/dim]") + console.print("-" * 50) + + if tracked_items["directories"]: + console.print(f"[bold]Directories ({len(tracked_items['directories'])}):[/bold]") + for i, dir_path in enumerate(sorted(tracked_items["directories"]), 1): + console.print(f" {i}. [dim]{dir_path}/[/dim]") + + if tracked_items["files"]: + console.print(f"[bold]Files ({len(tracked_items['files'])}):[/bold]") + for i, file_path in enumerate(sorted(tracked_items["files"]), 1): + console.print(f" {i}. [dim]{file_path}[/dim]") + else: + console.print() + if total_all_skipped > 0: + console.print(f"[dim]No project-specific items skipped ({total_all_skipped} common items filtered)[/dim]") + else: + console.print("[dim]No items were skipped.[/dim]") + + +# ============================================= +# MODULE INITIALIZATION +# ============================================= + +# Pure handler - no initialization needed diff --git a/src/aipass/backup/apps/handlers/utils/__init__.py b/src/aipass/backup/apps/handlers/utils/__init__.py new file mode 100755 index 00000000..e69de29b diff --git a/src/aipass/backup/apps/handlers/utils/backup_timestamps.py b/src/aipass/backup/apps/handlers/utils/backup_timestamps.py new file mode 100644 index 00000000..5c72e668 --- /dev/null +++ b/src/aipass/backup/apps/handlers/utils/backup_timestamps.py @@ -0,0 +1,98 @@ + +# ===================AIPASS==================== +# META DATA HEADER +# Name: backup_timestamps.py - Tracks last-run timestamps for all backup modes +# Date: 2026-02-22 +# Version: 1.0.0 +# Category: handlers/utils +# +# CHANGELOG (Max 5 entries): +# - v1.0.0 (2026-02-22): Initial version - show/update backup timestamps +# +# CODE STANDARDS: +# - Handler utility: no console output, returns data for module layer +# - JSON file at backups/backup_timestamps.json +# ============================================= + +""" +Backup Timestamps - Tracks when each backup mode was last run. + +Provides get/update functions for a shared timestamps file so any +backup command can display how fresh each backup type is. +""" + +import json +from pathlib import Path +from datetime import datetime + +_BACKUP_ROOT = Path(__file__).resolve().parents[4] # src/aipass/backup/ +TIMESTAMPS_FILE = _BACKUP_ROOT / "backup_data" / "backup_timestamps.json" + +MODES = ["snapshot", "versioned", "drive_sync"] + + +def get_timestamps() -> dict: + """Read all backup timestamps from disk. + + Returns: + dict with mode keys and ISO datetime string values (or None if never run) + """ + data = {} + if TIMESTAMPS_FILE.exists(): + try: + data = json.loads(TIMESTAMPS_FILE.read_text(encoding="utf-8")) + except (json.JSONDecodeError, OSError): + data = {} + return {mode: data.get(mode) for mode in MODES} + + +def update_timestamp(mode: str) -> None: + """Update the timestamp for a backup mode to now. + + Args: + mode: One of 'snapshot', 'versioned', 'drive_sync' + """ + data = {} + if TIMESTAMPS_FILE.exists(): + try: + data = json.loads(TIMESTAMPS_FILE.read_text(encoding="utf-8")) + except (json.JSONDecodeError, OSError): + data = {} + + data[mode] = datetime.now().isoformat() + + TIMESTAMPS_FILE.parent.mkdir(parents=True, exist_ok=True) + TIMESTAMPS_FILE.write_text(json.dumps(data, indent=2) + "\n", encoding="utf-8") + + +def format_age(iso_str: str | None) -> str: + """Format an ISO timestamp as a human-readable age string. + + Args: + iso_str: ISO datetime string, or None if never run + + Returns: + String like '2 days ago', '3 hours ago', 'just now', or 'never' + """ + if not iso_str: + return "never" + + try: + then = datetime.fromisoformat(iso_str) + except (ValueError, TypeError): + return "unknown" + + delta = datetime.now() - then + seconds = int(delta.total_seconds()) + + if seconds < 60: + return "just now" + elif seconds < 3600: + mins = seconds // 60 + return f"{mins} min{'s' if mins != 1 else ''} ago" + elif seconds < 86400: + hours = seconds // 3600 + return f"{hours} hour{'s' if hours != 1 else ''} ago" + else: + days = seconds // 86400 + return f"{days} day{'s' if days != 1 else ''} ago" diff --git a/src/aipass/backup/apps/handlers/utils/reauth_handler.py b/src/aipass/backup/apps/handlers/utils/reauth_handler.py new file mode 100644 index 00000000..28057403 --- /dev/null +++ b/src/aipass/backup/apps/handlers/utils/reauth_handler.py @@ -0,0 +1,87 @@ + +# ===================AIPASS==================== +# META DATA HEADER +# Name: reauth_handler.py - Google Drive Re-Authentication Handler +# Date: 2026-02-20 +# Version: 1.0.0 +# Category: backup_system/handlers/utils +# +# CHANGELOG (Max 5 entries): +# - v1.0.0 (2026-02-20): Extracted from reauth_drive.py module +# * Moved reauth() implementation to handler layer +# * Follows seed 3-layer architecture standards +# +# CODE STANDARDS: +# - Handlers must be independent and transportable +# - No prax imports (handler tier) +# - No cross-handler imports except within same domain +# ============================================= + +""" +Google Drive Re-Authentication Handler + +Implementation logic for re-authenticating with Google Drive via console OAuth flow. +Called by the reauth_drive module orchestrator. +""" + +from pathlib import Path + +SCOPES = ['https://www.googleapis.com/auth/drive.file'] +CREDS_PATH = Path.home() / '.aipass' / 'drive_creds.json' + + +def reauth(client_secrets_path: Path) -> bool: + """Perform Google Drive re-authentication via console OAuth flow. + + Args: + client_secrets_path: Path to the OAuth client secrets JSON file + + Returns: + bool: True if authentication succeeded, False otherwise + """ + try: + from google_auth_oauthlib.flow import InstalledAppFlow + from google.oauth2.credentials import Credentials + from google.auth.transport.requests import Request + from googleapiclient.discovery import build + except ImportError as e: + return False + + # Step 1: Try refreshing existing token first + if CREDS_PATH.exists(): + try: + creds = Credentials.from_authorized_user_file(str(CREDS_PATH), SCOPES) + if creds and creds.expired and creds.refresh_token: + creds.refresh(Request()) + with open(CREDS_PATH, 'w', encoding='utf-8') as f: + f.write(creds.to_json()) + # Test connection + service = build('drive', 'v3', credentials=creds) + about = service.about().get(fields="user").execute() + return True + except Exception: + pass # Proceed with full re-authentication + + # Step 2: Full OAuth flow via console + if not client_secrets_path.exists(): + return False + + flow = InstalledAppFlow.from_client_secrets_file(str(client_secrets_path), SCOPES) + + try: + creds = flow.run_local_server(port=8085, open_browser=False) + except Exception: + return False + + # Save new credentials + CREDS_PATH.parent.mkdir(parents=True, exist_ok=True) + with open(CREDS_PATH, 'w', encoding='utf-8') as f: + f.write(creds.to_json()) + + # Test connection + try: + service = build('drive', 'v3', credentials=creds) + about = service.about().get(fields="user,storageQuota").execute() + return True + except Exception: + return False diff --git a/src/aipass/backup/apps/handlers/utils/system_utils.py b/src/aipass/backup/apps/handlers/utils/system_utils.py new file mode 100755 index 00000000..8bd0c06b --- /dev/null +++ b/src/aipass/backup/apps/handlers/utils/system_utils.py @@ -0,0 +1,228 @@ + +# ===================AIPASS==================== +# META DATA HEADER +# Name: system_utils.py - Platform-aware file and console utilities +# Date: 2025-11-23 +# Version: 2.1.1 +# Category: handlers +# +# CHANGELOG (Max 5 entries): +# - v2.1.1 (2025-11-23): CRITICAL FIX - Switch to plain print() for clickability +# * Replaced console.print() with plain print() in safe_print() +# * Rich console truncates at 80 chars breaking long paths +# * Plain print() preserves full paths for terminal Ctrl+click +# - v2.1.0 (2025-11-23): Attempted fix with no_wrap parameter (insufficient) +# * Added no_wrap=True to console.print() but Rich still truncates +# * Identified Rich console 80-char limit as root cause +# - v2.0.0 (2025-11-16): Extracted to seed standards +# * Applied seed standards formatting and meta header +# * Preserved all logic from backup_utils v1.0.0 +# - v1.1.0 (2025-11-22): Updated to seed Rich formatting standards +# - v1.0.0 (2025-11-18): Extracted from backup_core.py +# * Extracted system utilities +# * Cross-platform file operations +# +# CODE STANDARDS: +# - Follow seed 3-layer architecture +# - Handlers must be independent and transportable +# - No cross-handler imports except within same domain +# ============================================= + +""" +System Utilities + +Platform-aware helper functions and context managers for file operations and console output. +Provides critical utilities for read-only file handling on Linux and emoji-safe printing +across platforms. + +Key Functions: + - temporarily_writable(): Context manager for safe read-only file modification + - safe_print(): Emoji-aware console output with fallback support + - EMOJI_SUPPORT: Platform-specific emoji capability constant +""" + +# ============================================= +# IMPORTS +# ============================================= + +import sys +import os +import stat +import logging +from pathlib import Path +from contextlib import contextmanager + +logger = logging.getLogger(__name__) + +# ============================================= +# CONSTANTS +# ============================================= + +# Fix console encoding for emojis (Linux/Windows compatibility) +EMOJI_SUPPORT = True +if sys.platform == 'win32': + try: + # Try to enable UTF-8 output on Windows + os.system('chcp 65001 > nul') + # Use getattr to safely call reconfigure if it exists + reconfigure_stdout = getattr(sys.stdout, 'reconfigure', None) + if reconfigure_stdout: + reconfigure_stdout(encoding='utf-8', errors='replace') + reconfigure_stderr = getattr(sys.stderr, 'reconfigure', None) + if reconfigure_stderr: + reconfigure_stderr(encoding='utf-8', errors='replace') + except Exception: + EMOJI_SUPPORT = False + +# ============================================= +# CONTEXT MANAGERS +# ============================================= + +@contextmanager +def temporarily_writable(path): + """Context manager to temporarily make a directory/file writable on Linux. + + This is critical for Linux because it strictly enforces read-only permissions, + unlike Windows which allows owners to bypass them. + + Used in 10+ locations throughout backup operations for: + - Creating new directories in read-only backup roots + - Updating files in protected backup directories + - Deleting files from read-only locations + + Args: + path: Path object or string path to make temporarily writable + + Yields: + Path object with write permissions temporarily enabled + + Example: + with temporarily_writable(backup_path): + # Perform operations that need write access + shutil.copy2(source, target) + # Permissions automatically restored here + """ + path_obj = Path(path) + original_mode = None + + try: + # Store original permissions if path exists + if path_obj.exists(): + original_mode = path_obj.stat().st_mode + + # Make writable for owner + if path_obj.is_dir(): + # Directory: add write and execute permissions for owner + path_obj.chmod(original_mode | stat.S_IWUSR | stat.S_IXUSR) + else: + # File: add write permission for owner + path_obj.chmod(original_mode | stat.S_IWUSR) + + yield path_obj + + finally: + # Restore original permissions if we changed them + if original_mode is not None and path_obj.exists(): + try: + path_obj.chmod(original_mode) + except Exception: + pass + +# ============================================= +# FILESYSTEM OPERATIONS +# ============================================= + +def ensure_backup_directory(backup_dest: Path, backup_path: Path, is_dynamic: bool) -> tuple[bool, str | None]: + """Create backup directory if needed with proper permission handling. + + Args: + backup_dest: Root backup destination path + backup_path: Specific backup folder path + is_dynamic: True if dynamic mode (needs special handling) + + Returns: + Tuple of (success: bool, error_message: str or None) + """ + try: + # Check if backup_dest exists and might be read-only + if backup_dest.exists(): + # Use context manager to temporarily make parent writable + with temporarily_writable(backup_dest.parent): + with temporarily_writable(backup_dest): + backup_dest.mkdir(parents=True, exist_ok=True) + else: + # Create normally if it doesn't exist + backup_dest.mkdir(parents=True, exist_ok=True) + + if is_dynamic: + if backup_path.exists(): + with temporarily_writable(backup_path.parent): + with temporarily_writable(backup_path): + backup_path.mkdir(parents=True, exist_ok=True) + else: + backup_path.mkdir(parents=True, exist_ok=True) + return True, None + except PermissionError as e: + return False, f"Permission denied creating backup directory {backup_dest}: {e}" + except OSError as e: + return False, f"OS error creating backup directory {backup_dest}: {e}" + except Exception as e: + return False, f"Unexpected error creating backup directory {backup_dest}: {e}" + + +def remove_empty_dirs(path: Path): + """Remove empty directories recursively. + + Args: + path: Root path to clean empty directories from + """ + try: + for item in path.iterdir(): + if item.is_dir(): + remove_empty_dirs(item) + try: + item.rmdir() + except OSError: + pass + except Exception: + pass + + +# ============================================= +# HELPER FUNCTIONS +# ============================================= + +def safe_print(text): + """Print text with emoji fallback for systems that don't support them. + + Handles console encoding issues across different platforms. + + Args: + text: String to print (may contain emoji characters) + + Returns: + None (prints to stdout) + + Example: + safe_print("Backup complete!") + """ + if not EMOJI_SUPPORT: + # Replace emojis with text equivalents + text = text.replace('!', '[CRITICAL]') + text = text.replace('X', '[ERROR]') + text = text.replace('!', '[WARNING]') + text = text.replace('OK', '[SUCCESS]') + try: + # Use plain print() to avoid Rich console truncation at 80 chars + # This preserves full paths for terminal clickability + logger.info(text) + except UnicodeEncodeError: + # Final fallback - strip all non-ASCII characters + safe_text = text.encode('ascii', errors='replace').decode('ascii') + logger.info(safe_text) + +# ============================================= +# MODULE INITIALIZATION +# ============================================= + +# No initialization needed - pure utility functions diff --git a/src/aipass/backup/apps/json_templates/__init__.py b/src/aipass/backup/apps/json_templates/__init__.py new file mode 100644 index 00000000..5d00b535 --- /dev/null +++ b/src/aipass/backup/apps/json_templates/__init__.py @@ -0,0 +1 @@ +# JSON Templates package - Default JSON file templates diff --git a/src/aipass/backup/apps/modules/__init__.py b/src/aipass/backup/apps/modules/__init__.py new file mode 100755 index 00000000..4cf76dce --- /dev/null +++ b/src/aipass/backup/apps/modules/__init__.py @@ -0,0 +1 @@ +# Modules package - Branch-specific functionality modules diff --git a/src/aipass/backup/apps/modules/backup_core.py b/src/aipass/backup/apps/modules/backup_core.py new file mode 100755 index 00000000..d6b6f26e --- /dev/null +++ b/src/aipass/backup/apps/modules/backup_core.py @@ -0,0 +1,567 @@ +# ===================AIPASS==================== +# META DATA HEADER +# Name: backup_core.py - Main backup system orchestration module +# Date: 2025-11-23 +# Version: 2.1.0 +# Category: backup_system +# +# CHANGELOG (Max 5 entries): +# - v2.1.0 (2026-03-06): Adapted for AIPass public repo +# * Removed shebang, sys.path manipulation, prax/cli imports +# * Uses standard logging and rich console +# * Relative handler imports +# * JSON_DIR uses path relative to __file__ +# - v2.0.4 (2025-11-23): CRITICAL FIX - Versioned dry-run accuracy +# - v2.0.3 (2025-11-23): Enhanced dry-run with accurate statistics +# - v2.0.2 (2025-11-23): CRITICAL FIX - Dry-run now shows proposed changes +# - v2.0.1 (2025-11-23): CRITICAL BUG FIX - Dry-run statistics counter +# +# CODE STANDARDS: +# - Follow seed 3-layer architecture +# - Orchestrate workflows, delegate to handlers +# - Import handlers, never implement business logic +# ============================================= + +""" +Backup System Core Module + +Main entry point for backup functionality following seed architecture standards. +Provides CLI command routing and BackupEngine orchestration. + +This module coordinates the complete backup workflow by bringing together: +- Configuration (BACKUP_MODES, GLOBAL_IGNORE_PATTERNS) +- Models (BackupResult) +- File operations (copy_file_with_structure, copy_versioned_file) +- Utilities (safe_print, temporarily_writable, system logging) +- JSON tracking (log_operation) + +Architecture Pattern: +- handle_command(args) - CLI entry point for backup commands +- BackupEngine class - Main orchestrator (delegates to handlers) +- Follows seed 3-layer architecture (cli -> modules -> handlers) +""" + +# ============================================= +# IMPORTS +# ============================================= + +# Infrastructure +import sys +import datetime +import logging +from pathlib import Path +from typing import Dict + +from rich.console import Console + +logger = logging.getLogger(__name__) +console = Console() + + +def _header(text): + console.print(f"\n[bold cyan]{'='*70}[/bold cyan]") + console.print(f"[bold cyan] {text}[/bold cyan]") + console.print(f"[bold cyan]{'='*70}[/bold cyan]") + + +# Import handlers (core dependencies) - relative imports +from ..handlers.config.config_handler import ( + BACKUP_MODES, + GLOBAL_IGNORE_PATTERNS, + IGNORE_EXCEPTIONS, + filter_tracked_items, + should_ignore +) +from ..handlers.models.backup_models import BackupResult +from ..handlers.operations.file_operations import copy_file_with_structure, copy_versioned_file +from ..handlers.utils.system_utils import safe_print +from ..handlers.json.json_handler import log_operation, ensure_module_jsons +from ..handlers.json.changelog_handler import ( + load_changelog as load_changelog_file, + save_changelog_entry as save_changelog_entry_file, + display_previous_comments as display_previous_comments_file +) +from ..handlers.json.backup_info_handler import ( + load_backup_info as load_backup_info_file, + save_backup_info as save_backup_info_file +) + +# Integration stubs and JSON setup moved to handlers +# sync_to_drive → handlers/integrations/ (when implemented) +# set_backup_readonly → handlers/integrations/ (when implemented) +# initialize_json_files → called directly in __init__ + +_BACKUP_ROOT = Path(__file__).resolve().parents[2] # src/aipass/backup/ +JSON_DIR = _BACKUP_ROOT / "backup_json" + + +# ============================================= +# INTROSPECTION (SEED PATTERN) +# ============================================= + +def print_introspection(): + """Display module info and connected handlers""" + from pathlib import Path + + console.print() + console.print("[bold cyan]Backup Core Module[/bold cyan]") + console.print() + + console.print("[yellow]Connected Handlers:[/yellow]") + console.print() + + # Show handler domains this module uses + handlers_base = Path(__file__).parent.parent / "handlers" + + handler_domains = [ + "config", + "models", + "operations", + "utils", + "json", + "reporting" + ] + + for domain in handler_domains: + domain_path = handlers_base / domain + if domain_path.exists(): + console.print(f" [cyan]handlers/{domain}/[/cyan]") + + console.print() + console.print("[dim]Run 'python3 backup_core.py --help' for usage[/dim]") + console.print() + + +# ============================================= +# MODULE-LEVEL COMMAND HANDLER +# ============================================= + + +def print_help(): + """Display help for the backup_core module.""" + console.print() + console.print("[bold cyan]Backup Core Module[/bold cyan]") + console.print() + console.print("Main backup orchestration module") + console.print() + console.print("[yellow]Usage:[/yellow]") + console.print(" python3 backup_core.py # Show module info") + console.print(" python3 backup_core.py --help # Show this help") + console.print() + console.print("[yellow]Commands:[/yellow]") + console.print(" snapshot Full copy backup (overwrites destination)") + console.print(" versioned Timestamped backup with incremental copies") + console.print(" all Full backup cycle (snapshot + versioned + drive-sync)") + console.print() + + +def handle_command(args) -> bool: + """Route backup commands to appropriate handler. + + This is the module-level entry point for CLI commands related to backup. + Checks if the command matches backup commands and routes accordingly. + + Args: + args: Command-line arguments from CLI parser + + Returns: + bool: True if command was handled, False if not a backup command + + Example: + result = handle_command(args) + if result: + console.print(f"Backup completed") + """ + # Check if args has backup-related command + if not hasattr(args, 'command'): + return False + + # Check for backup commands ('all' handled by entry point, listed here for drone discovery) + if args.command not in ['snapshot', 'versioned', 'all']: + return False + + # 'all' is orchestrated by backup_system.py entry point (snapshot → versioned → drive-sync) + if args.command == 'all': + return False + + # Set mode directly from command + mode = args.command + + # Check for dry-run flag + dry_run = getattr(args, 'dry_run', False) + + # Get backup note if provided + backup_note = getattr(args, 'note', 'No note provided') + + try: + # Create engine and run backup + engine = BackupEngine(mode, dry_run=dry_run) + result = engine.run_backup(backup_note) + + # Return True to indicate command was handled + return True + except Exception as e: + logger.error(f"[backup_core] Backup command failed: {e}") + console.print(f"[ERROR] Backup failed: {e}") + return True # Still handled, just failed + + +# ============================================= +# BACKUP ENGINE CLASS +# ============================================= + + +class BackupEngine: + """Main backup system orchestrator (seed-compliant wrapper). + + Provides the core backup workflow coordination following seed architecture. + Delegates heavy lifting to handler modules rather than implementing logic here. + + This class is a thin orchestration layer that: + - Initializes with backup mode and configuration + - Coordinates file scanning and copying via handlers + - Manages backup metadata persistence + - Reports statistics and status + + Attributes: + mode (str): Backup mode (e.g., 'snapshot', 'versioned') + dry_run (bool): If True, scan files without copying + mode_config (dict): Configuration for selected mode + backup_path (Path): Destination path for backup + source_dir (Path): Source directory to backup + + Raises: + ValueError: If mode is invalid or configuration missing + """ + + def __init__(self, mode: str, dry_run: bool = False): + """Initialize backup engine with specified mode. + + Validates mode, loads configuration, and prepares for backup operation. + + Args: + mode: Backup mode from BACKUP_MODES ('snapshot' or 'versioned') + dry_run: If True, scan files without copying (test ignore patterns) + + Raises: + ValueError: If mode is invalid + KeyError: If mode configuration is missing + + Example: + engine = BackupEngine('versioned', dry_run=True) + result = engine.run_backup("Before refactor") + """ + if mode not in BACKUP_MODES: + raise ValueError(f"Invalid backup mode: {mode}. Valid modes: {list(BACKUP_MODES.keys())}") + + self.mode = mode + self.dry_run = dry_run + self.mode_config = BACKUP_MODES[mode] + + # Auto-detect source directory + self.source_dir = Path.home() + self.backup_dest = Path(self.mode_config['destination']) + self.ignore_patterns = GLOBAL_IGNORE_PATTERNS + + # Mode-specific paths - all modes use fixed folder names + self.backup_folder_name = self.mode_config['folder_name'] + self.backup_path = self.backup_dest / self.backup_folder_name + + # Initialize JSON system (handler creates directory if needed) + ensure_module_jsons("backup_core") + + # JSON files (mode-specific) - backup-specific tracking files + self.backup_info_file = JSON_DIR / f"{mode}_backup.json" + self.changelog_file = JSON_DIR / f"{mode}_backup_changelog.json" + self.restore_log_file = JSON_DIR / f"{mode}_restore_history.json" + + # Display clear mode identification + console.print(f"Mode: {self.mode_config['name']}") + console.print(f"Source: {self.source_dir}") + console.print(f"Destination: {self.backup_dest}") + console.print(f"Usage: {self.mode_config['usage']}") + + logger.info(f"[backup_core] Initialized {mode} mode - source: {self.source_dir}") + + # ============================================= + # UTILITY METHODS + # ============================================= + + def should_ignore(self, path: Path) -> bool: + """Check if a file/folder should be ignored based on patterns. + + Delegates to handler function for centralized pattern matching. + + Args: + path: Path to check + + Returns: + True if path should be ignored, False otherwise + """ + return should_ignore(path, self.ignore_patterns, IGNORE_EXCEPTIONS, self.backup_dest) + + def ensure_backup_directory(self, result: BackupResult) -> bool: + """Create backup directory if needed (delegates to handler).""" + from ..handlers.utils.system_utils import ensure_backup_directory as ensure_dir + success, error_msg = ensure_dir(self.backup_dest, self.backup_path, self.mode_config['behavior'] == 'dynamic') + if not success: + error_text = error_msg if error_msg is not None else "Unknown error creating backup directory" + result.add_error(error_text, is_critical=True) + safe_print(f"CRITICAL: {error_text}") + logger.error(f"[backup_core] {error_text}") + return success + + def file_needs_backup(self, source_file: Path, backup_file: Path, last_timestamps: dict) -> bool: + """Check if file needs backup (delegates to handler).""" + from ..handlers.operations.file_operations import file_needs_backup as check_file + return check_file(source_file, backup_file, last_timestamps, self.source_dir) + + def remove_empty_dirs(self, path: Path): + """Remove empty directories (delegates to handler).""" + from ..handlers.utils.system_utils import remove_empty_dirs as clean_dirs + clean_dirs(path) + + # Changelog and backup info operations (thin delegators to handlers) + def load_changelog(self) -> Dict: + """Load changelog data from JSON file.""" + return load_changelog_file(self.changelog_file) + + def save_changelog_entry(self, note: str) -> bool: + """Save a changelog entry for the current backup operation.""" + return save_changelog_entry_file(self.changelog_file, note, self.mode, self.backup_path) + + def display_previous_comments(self): + """Display previous changelog comments for the current backup mode.""" + display_previous_comments_file(self.changelog_file, self.mode_config['name']) + + def load_backup_info(self) -> Dict: + """Load backup info metadata from JSON file.""" + return load_backup_info_file(self.backup_info_file, self.mode_config['behavior']) + + def save_backup_info(self, backup_info: Dict) -> bool: + """Save backup info metadata to JSON file.""" + return save_backup_info_file(self.backup_info_file, backup_info) + + # ============================================= + # MAIN BACKUP EXECUTION + # ============================================= + + def run_backup(self, backup_note: str = "No note provided") -> BackupResult: + """Execute backup - thin orchestration layer calling handlers. + + Coordinates backup workflow by delegating to specialized handlers. + + Args: + backup_note: User note describing backup purpose/context + + Returns: + BackupResult: Complete operation result with statistics and status + """ + logger.info(f"[backup_core] Starting {self.mode} backup: {backup_note}") + + # Show last backup timestamps for all modes + from ..handlers.utils.backup_timestamps import get_timestamps, format_age + ts = get_timestamps() + console.print() + console.print("[dim]Last backups:[/dim]") + console.print(f" [dim]Snapshot: {format_age(ts.get('snapshot'))}[/dim]") + console.print(f" [dim]Versioned: {format_age(ts.get('versioned'))}[/dim]") + console.print(f" [dim]Drive sync: {format_age(ts.get('drive_sync'))}[/dim]") + + result = BackupResult() + result.mode = self.mode + result.backup_path = str(self.backup_path) + + # Display dry-run warning if in test mode + if self.dry_run: + console.print() + console.print("╭" + "─" * 68 + "╮", style="yellow") + console.print("│" + " " * 18 + "[bold yellow]🔍 DRY-RUN MODE ACTIVE[/bold yellow]" + " " * 18 + "│", style="yellow") + console.print("│" + " " * 10 + "[yellow]Files will be scanned but NOT copied or deleted[/yellow]" + " " * 10 + "│", style="yellow") + console.print("╰" + "─" * 68 + "╯", style="yellow") + console.print() + + _header(f"AIPass {self.mode_config['name']} - {self.mode_config['description']}") + console.print() + # Ensure backup directory exists + if not self.ensure_backup_directory(result): + console.print(f"\nBACKUP FAILED: Could not create backup directory") + return result + + # Load previous backup info + backup_info = self.load_backup_info() + last_timestamps = backup_info.get("file_timestamps", {}) if self.mode_config['behavior'] == 'dynamic' else {} + if not isinstance(last_timestamps, dict): + last_timestamps = {} + + # HANDLER: Scan files + from ..handlers.operations.file_scanner import scan_files + files_to_backup, skipped_items = scan_files(self.source_dir, self.should_ignore) + + # HANDLER: Process files + from ..handlers.operations.path_builder import build_backup_path + current_timestamps = {} + total_files = len(files_to_backup) + + from rich.progress import Progress, BarColumn, TextColumn, TimeRemainingColumn + + # Progress tracking with Rich + with Progress( + TextColumn("[progress.description]{task.description}"), + BarColumn(), + TextColumn("[progress.percentage]{task.percentage:>3.0f}%"), + TimeRemainingColumn(), + console=console, + transient=True + ) as progress: + task = progress.add_task("Processing files...", total=total_files) + + for idx, file_path in enumerate(files_to_backup): + result.files_checked += 1 + + # Build backup path + backup_file = build_backup_path(file_path, self.source_dir, self.backup_path, self.mode) + current_timestamps[str(file_path.relative_to(self.source_dir))] = file_path.stat().st_mtime + + # Process file + try: + if self.dry_run: + # Show what WOULD happen in dry-run mode + is_new = not backup_file.exists() + + if self.mode_config['behavior'] == 'versioned': + # Versioned mode: check if file actually changed (mtime comparison) + if is_new: + safe_print(f"📄 Would copy (new): {file_path}") + result.files_copied += 1 + elif backup_file.exists(): + # Check if file changed by comparing mtimes + source_mtime = file_path.stat().st_mtime + target_mtime = backup_file.stat().st_mtime + if source_mtime != target_mtime: + safe_print(f"📄 Would copy (updated): {file_path}") + result.files_copied += 1 + else: + result.files_skipped += 1 # Unchanged, would skip + else: + # Snapshot mode: use timestamp-based check + if is_new: + safe_print(f"📄 Would copy (new): {file_path}") + result.files_copied += 1 + elif self.file_needs_backup(file_path, backup_file, last_timestamps): + safe_print(f"📄 Would copy (updated): {file_path}") + result.files_copied += 1 + else: + result.files_skipped += 1 # Only skip if unchanged + elif self.mode_config['behavior'] == 'versioned': + if copy_versioned_file(file_path, backup_file, self.backup_path, result): + result.files_copied += 1 + elif self.file_needs_backup(file_path, backup_file, last_timestamps): + if copy_file_with_structure(file_path, backup_file, self.backup_path, result): + result.files_copied += 1 + else: + result.files_skipped += 1 + except Exception as e: + result.add_error(f"Error processing {file_path}: {e}", is_critical=True) + # Use console.print directly for critical errors so they appear above progress bar + console.print(f"[red]CRITICAL: Error processing {file_path}: {e}[/red]") + + # Update progress + progress.advance(task) + + console.print(f"Processing completed: {total_files}/{total_files} files checked") + + # HANDLER: Cleanup deleted files (dynamic mode only) + # RE-ENABLED 2025-11-23: Fixed to respect IGNORE_EXCEPTIONS in third pass + # Now runs in dry-run mode to show what WOULD be deleted + if self.backup_path.exists() and self.mode_config['behavior'] == 'dynamic': + from ..handlers.operations.file_cleanup import cleanup_deleted_files + cleanup_deleted_files(self.backup_path, self.source_dir, self.should_ignore, result, self.dry_run) + + # HANDLER: Remove empty directories (handled by cleanup_deleted_files now) + # self.remove_empty_dirs(self.backup_path) + + # HANDLER: Create and save backup metadata + from ..handlers.json.backup_metadata_builder import create_backup_metadata + backup_info = create_backup_metadata( + self.mode, self.mode_config['behavior'], backup_note, self.backup_folder_name, + self.backup_path, self.source_dir, result, current_timestamps, backup_info + ) + self.save_backup_info(backup_info) + + # HANDLER: Display backup results + from ..handlers.reporting.report_formatter import display_backup_results + display_backup_results(result, self.mode_config, self.backup_path, skipped_items, filter_tracked_items, self.dry_run) + + # Update JSON system + execution_time = int((datetime.datetime.now() - result.start_time).total_seconds() * 1000) + if result.success and result.errors == 0: + log_operation( + "backup", + { + "mode": self.mode, + "files_copied": result.files_copied, + "success": True, + "execution_time_ms": execution_time + }, + module_name="backup_core" + ) + logger.info(f"[backup_core] {self.mode} backup completed successfully - {result.files_copied} files copied in {execution_time}ms") + + # Update backup timestamp and show confirmation + from ..handlers.utils.backup_timestamps import update_timestamp, get_timestamps, format_age + update_timestamp(self.mode) + ts = get_timestamps() + console.print() + console.print(f"[dim]Backups now:[/dim]") + console.print(f" [dim]Snapshot: {format_age(ts.get('snapshot'))}[/dim]") + console.print(f" [dim]Versioned: {format_age(ts.get('versioned'))}[/dim]") + console.print(f" [dim]Drive sync: {format_age(ts.get('drive_sync'))}[/dim]") + + # Integration hooks (stubs - implement in handlers/integrations/) + if self.mode == 'versioned': + logger.info("[backup_core] Drive sync skipped (not implemented)") + logger.info("[backup_core] Read-only protection skipped (not implemented)") + else: + log_operation( + "backup", + { + "mode": self.mode, + "errors": result.errors, + "success": False, + "execution_time_ms": execution_time + }, + module_name="backup_core" + ) + logger.error(f"[backup_core] {self.mode} backup completed with {result.errors} errors in {execution_time}ms") + + from ..handlers.json.statistics_handler import update_data_file + update_data_file(result) + + return result + + +# ============================================= +# MODULE INITIALIZATION +# ============================================= + +# ============================================= +# STANDALONE ENTRY POINT (SEED PATTERN) +# ============================================= + +if __name__ == "__main__": + import sys + + # Show introspection when run without arguments + if len(sys.argv) == 1: + print_introspection() + sys.exit(0) + + # Handle help flag + if sys.argv[1] in ['--help', '-h', 'help']: + print_help() + sys.exit(0) + + console.print("[yellow]Note:[/yellow] Run via backup_system.py entry point for full functionality") + console.print() + print_introspection() diff --git a/src/aipass/backup/apps/modules/google_drive_sync.py b/src/aipass/backup/apps/modules/google_drive_sync.py new file mode 100644 index 00000000..6d5bb011 --- /dev/null +++ b/src/aipass/backup/apps/modules/google_drive_sync.py @@ -0,0 +1,553 @@ +# ===================AIPASS==================== +# META DATA HEADER +# Name: google_drive_sync.py - Google Drive Integration for AIPass Backup System +# Date: 2025-10-30 +# Version: 2.6.0 +# Category: backup_system/modules +# +# CHANGELOG (Max 5 entries - remove oldest when adding new): +# - v2.6.0 (2026-03-06): Adapted for AIPass public repo +# * Removed shebang, sys.path manipulation, prax/cli imports +# * Uses standard logging and rich console +# * Relative handler imports, Google API deps wrapped in try/except +# - v2.5.0 (2026-02-22): Rich progress bar for drive-sync uploads, per-file progress callback +# - v2.4.0 (2026-02-21): Two-phase sync (prepare+upload), CLI visibility, dedup fix, --test mode +# - v2.3.2 (2026-02-10): Fixed deepcopy RuntimeError in _save_data during concurrent uploads +# - v2.2.0 (2026-02-10): Fixed SSL concurrency bug - per-thread credentials, retry with backoff, folder cache lock +# +# CODE STANDARDS: +# - Module orchestrates, handlers implement +# - CLI display (console.print) belongs here, not in handlers +# - Google API calls delegated to drive_sync_client handler +# ============================================= + +""" +Google Drive Sync Module - Orchestrates Drive backup operations. + +Routes drive-sync commands, displays CLI output, delegates to +drive_sync_client handler for API operations. +""" + +# ============================================= +# IMPORTS +# ============================================= + +import sys +import logging +from pathlib import Path +from datetime import datetime + +from rich.console import Console +from rich.progress import Progress, BarColumn, TextColumn, TimeElapsedColumn, TimeRemainingColumn + +logger = logging.getLogger(__name__) +console = Console() + +_BACKUP_ROOT = Path(__file__).resolve().parents[2] # src/aipass/backup/ + +# ============================================= +# CONSTANTS & CONFIG +# ============================================= + +# OAuth scopes for Drive access +SCOPES = ['https://www.googleapis.com/auth/drive.file'] + +# Module configuration +MODULE_NAME = "google_drive_sync" +JSON_DIR = _BACKUP_ROOT / "backup_json" +CONFIG_FILE = JSON_DIR / f"{MODULE_NAME}_config.json" +DATA_FILE = JSON_DIR / f"{MODULE_NAME}_data.json" +LOG_FILE = JSON_DIR / f"{MODULE_NAME}_log.json" + +# ============================================= +# CLIENT CLASS (delegated to handler) +# ============================================= + +try: + from ..handlers.operations.drive_sync_client import ( + GoogleDriveSync, + ) + from ..handlers.json.drive_sync_json import ( + load_config as _load_config_fn, + load_data as _load_data_fn, + ) + DRIVE_AVAILABLE = True +except ImportError: + GoogleDriveSync = None # type: ignore + _load_config_fn = None # type: ignore + _load_data_fn = None # type: ignore + DRIVE_AVAILABLE = False + logger.info("[google_drive_sync] Google Drive dependencies not available") + +def _load_config(): + """Load config using module JSON paths.""" + return _load_config_fn(CONFIG_FILE) + +def _load_data(): + """Load data using module JSON paths.""" + return _load_data_fn(DATA_FILE) + + +# ============================================= +# BUSINESS OPERATIONS (delegated to handlers) +# ============================================= + +try: + from ..handlers.operations.drive_sync_ops import ( + clear_file_tracker as _clear_file_tracker_handler, + get_file_tracker_stats, + test_drive_connection as _test_drive_connection, + ) +except ImportError: + _clear_file_tracker_handler = None # type: ignore + get_file_tracker_stats = None # type: ignore + _test_drive_connection = None # type: ignore + + +def _show_file_tracker_stats() -> bool: + """Display file tracker statistics.""" + try: + stats = get_file_tracker_stats() + console.print(f"File Tracker Statistics:") + console.print(f" - Total tracked files: {stats['total']}") + if stats['sample']: + console.print(f" - Sample entries:") + for i, entry in enumerate(stats['sample']): + console.print(f" {i+1}. {entry['file']} (last sync: {entry['last_sync']})") + if stats['truncated']: + remaining = stats['total'] - len(stats['sample']) + console.print(f" ... and {remaining} more files") + return True + except Exception as e: + console.print(f"Error showing tracker stats: {e}") + logger.error(f"Error showing tracker stats: {e}") + return False + + +def _clear_file_tracker() -> bool: + """Clear the file tracker cache for fresh sync.""" + try: + data = _load_data() + tracker_count = len(data.get("runtime_state", {}).get("file_tracker", {})) + success = _clear_file_tracker_handler() + if success: + console.print(f"Cleared {tracker_count} entries from file tracker") + logger.info(f"Cleared {tracker_count} entries from file tracker") + else: + console.print("File tracker already empty or clear failed") + return success + except Exception as e: + console.print(f"Error clearing file tracker: {e}") + logger.error(f"Error clearing file tracker: {e}") + return False + + +def _test_drive_sync() -> bool: + """Test Drive integration.""" + try: + sync = GoogleDriveSync() + if not sync.authenticate(): + return False + console.print("Testing folder creation...") + result = _test_drive_connection(sync) + if result: + folder_id = sync.get_or_create_backup_folder() + console.print(f"Backup folder ready: {folder_id}") + else: + console.print("Failed to create backup folder") + return result + except Exception as e: + console.print(f"Test failed: {e}") + logger.error(f"Test failed: {e}") + return False + +def _run_sync_test() -> bool: + """Run a small test sync to verify Drive integration.""" + import shutil + + console.print("[bold cyan]Drive Sync Test[/bold cyan]") + console.print() + + # Create test files + test_dir = _BACKUP_ROOT / "backups" / "_sync_test" + test_dir.mkdir(parents=True, exist_ok=True) + + test_files = { + "test_file_1.txt": "Hello from AIPass backup test", + "test_file_2.json": '{"test": true, "timestamp": "' + datetime.now().isoformat() + '"}', + "subdir/nested_file.txt": "Nested directory test", + "subdir/deep/deeper_file.md": "# Deep nested test\nVerifying folder structure", + } + + for name, content in test_files.items(): + path = test_dir / name + path.parent.mkdir(parents=True, exist_ok=True) + path.write_text(content) + + console.print(f" Created {len(test_files)} test files in {test_dir}") + + # Run sync + sync = GoogleDriveSync() + if not sync.authenticate(): + console.print("[red]Auth failed[/red]") + shutil.rmtree(test_dir, ignore_errors=True) + return False + + console.print(f"\nScanning...") + files_to_upload, skipped, total = sync.prepare_sync(test_dir, force_sync=True) + upload_count = len(files_to_upload) + + console.print(f" Files: {total} total, {upload_count} to upload") + + # Verify Drive folder before uploading + console.print(f"\nVerifying Drive folder: AIPass_Test...") + folder_id = sync.get_or_create_project_folder("AIPass_Test") + if not folder_id: + error_msg = sync.last_error or "Unknown error" + console.print(f"[red]FAILED: {error_msg}[/red]") + shutil.rmtree(test_dir, ignore_errors=True) + return False + console.print(f" Drive folder ready: AIPass_Test") + + console.print(f"Uploading...") + + def show_test_progress(completed, total_upload, _successes): + """Display test upload progress to console.""" + console.print(f" {completed}/{total_upload}") + + result = sync.sync_backup_files( + test_dir, "AIPass_Test", "sync test", True, + prepared_files=files_to_upload, + skipped_count=skipped, + total_count=total, + progress_fn=show_test_progress + ) + + if result.get("error"): + console.print(f"\n[red]FAILED: {result['error']}[/red]") + shutil.rmtree(test_dir, ignore_errors=True) + return False + + console.print() + if result["success"]: + console.print(f"[green]Test passed: {result['uploaded']}/{result['total']} files synced[/green]") + else: + console.print(f"[red]Test failed: {result['uploaded']} OK, {result['failed']} failed[/red]") + + # Run it again to verify no duplicates + console.print(f"\nRe-running sync (should show 0 to upload)...") + files_to_upload2, _, _ = sync.prepare_sync(test_dir, force_sync=False) + + if len(files_to_upload2) == 0: + console.print(f"[green]Dedup check passed: 0 files need re-upload[/green]") + else: + console.print(f"[red]Dedup check failed: {len(files_to_upload2)} files flagged for re-upload[/red]") + + # Cleanup local test dir + shutil.rmtree(test_dir, ignore_errors=True) + console.print(f"\nCleaned up local test files") + console.print(f"[dim]Test Drive folder 'AIPass_Test' left on Drive for inspection[/dim]") + + return result["success"] + + +# ============================================= +# HANDLE_COMMAND (Drone Integration) +# ============================================= + +def handle_command(args) -> bool: + """Route Google Drive sync commands from backup_system orchestrator. + + Args: + args: Command-line arguments from CLI parser + + Returns: + bool: True if command was handled, False if not a drive sync command + """ + if not hasattr(args, 'command'): + return False + + command = args.command + + if command in ['--help', '-h', 'help']: + console.print() + console.print("[bold cyan]google_drive_sync - Google Drive Integration[/bold cyan]") + console.print() + console.print("Syncs AIPass backups to Google Drive using OAuth2.") + console.print() + console.print("[yellow]Commands:[/yellow]") + console.print(" drive-test - Test Google Drive connectivity") + console.print(" drive-sync - Sync backup directory to Google Drive") + console.print(" drive-sync --test - Run a small test sync to verify integration") + console.print(" drive-clear-tracker - Clear file tracker cache") + console.print(" drive-stats - Show file tracker statistics") + console.print() + console.print("[yellow]Options:[/yellow]") + console.print(" --project Project name (default: AIPass)") + console.print(" --note Sync note (default: Manual sync)") + console.print(" --force Force upload all files") + console.print() + return True + + if command == 'drive-test': + return _test_drive_sync() + + elif command == 'drive-sync': + # --test flag routes to test mode + if getattr(args, 'test', False): + return _run_sync_test() + + raw_path = getattr(args, 'path', None) + if raw_path: + backup_path = Path(raw_path) + else: + # Default to snapshot backup directory + backup_path = _BACKUP_ROOT / "backups" / "system_snapshot" + if not backup_path.exists(): + console.print(f"Error: Backup directory not found: {backup_path}") + return False + + project = getattr(args, 'project', 'AIPass') or 'AIPass' + note = getattr(args, 'note', 'Manual sync') or 'Manual sync' + force = getattr(args, 'force', False) + + sync = GoogleDriveSync() + if not sync.authenticate(): + console.print("[red]FAILED: Could not authenticate with Google Drive[/red]") + return False + + limit = getattr(args, 'limit', 0) or 0 + + # Show last backup timestamps for all modes + from ..handlers.utils.backup_timestamps import get_timestamps, format_age + ts = get_timestamps() + console.print() + console.print("[dim]Last backups:[/dim]") + console.print(f" [dim]Snapshot: {format_age(ts.get('snapshot'))}[/dim]") + console.print(f" [dim]Versioned: {format_age(ts.get('versioned'))}[/dim]") + console.print(f" [dim]Drive sync: {format_age(ts.get('drive_sync'))}[/dim]") + console.print() + + # Pre-flight: verify Drive folder FIRST (may reset tracker) + console.print(f"Verifying Drive folder (project: {project})...") + folder_id = sync.get_or_create_project_folder(project) + if not folder_id: + error_msg = sync.last_error or "Unknown error" + console.print(f"[red]FAILED: {error_msg}[/red]") + console.print("[red]Sync aborted - no files uploaded[/red]") + return False + console.print(f" Drive folder ready: {project}") + + if sync.tracker_was_reset: + console.print(f"[yellow] Drive folder is new - tracker reset, full re-sync needed[/yellow]") + + # Phase 1: Scan (local only, fast) - runs AFTER folder check so tracker is accurate + console.print(f"\nScanning {backup_path}...") + files_to_upload, skipped, total = sync.prepare_sync(backup_path, force, limit) + upload_count = len(files_to_upload) + + # Display plan + console.print(f" Files considered: {total}{f' (limited to {limit})' if limit > 0 else ''}") + console.print(f" To upload: {upload_count} ({'forced' if force else 'changed/new'})") + console.print(f" Unchanged: {skipped}") + + if upload_count == 0: + console.print("[green]All files up to date - nothing to sync[/green]") + return True + + console.print() + + # Phase 2: Upload with progress bar + with Progress( + TextColumn("{task.description}"), + BarColumn(bar_width=40), + TextColumn("[progress.percentage]{task.percentage:>3.0f}%"), + TextColumn("{task.completed}/{task.total}"), + TextColumn("[green]{task.fields[ok]} OK[/green]"), + TimeElapsedColumn(), + TimeRemainingColumn(), + console=console, + ) as progress: + task_id = progress.add_task("Uploading...", total=upload_count, ok=0) + + def show_progress(completed, total_upload, successes): + """Update progress bar.""" + progress.update(task_id, completed=completed, ok=successes) + + result = sync.sync_backup_files( + backup_path, project, note, force, + prepared_files=files_to_upload, + skipped_count=skipped, + total_count=total, + progress_fn=show_progress + ) + + # Update label when complete + progress.update(task_id, description="Uploaded ") + + # Check for errors + if result.get("error"): + console.print(f"\n[red]FAILED: {result['error']}[/red]") + console.print("[red]Sync aborted - no files uploaded[/red]") + return False + + # Summary + console.print() + if result["success"]: + from ..handlers.utils.backup_timestamps import update_timestamp, get_timestamps, format_age + update_timestamp("drive_sync") + ts = get_timestamps() + console.print(f"[green]Sync complete: {result['uploaded']} uploaded, {result['skipped']} unchanged[/green]") + console.print() + console.print(f"[dim]Backups now:[/dim]") + console.print(f" [dim]Snapshot: {format_age(ts.get('snapshot'))}[/dim]") + console.print(f" [dim]Versioned: {format_age(ts.get('versioned'))}[/dim]") + console.print(f" [dim]Drive sync: {format_age(ts.get('drive_sync'))}[/dim]") + else: + console.print(f"[red]Sync failed: {result['uploaded']} uploaded, {result['failed']} failed, {result['skipped']} unchanged[/red]") + + return result["success"] + + elif command == 'drive-sync-test': + return _run_sync_test() + + elif command == 'drive-clear-tracker': + return _clear_file_tracker() + + elif command == 'drive-stats': + return _show_file_tracker_stats() + + return False + +# ============================================= +# CLI/EXECUTION +# ============================================= + +if __name__ == "__main__": + import argparse + + parser = argparse.ArgumentParser( + description="Google Drive Sync for AIPass Backup System", + formatter_class=argparse.RawDescriptionHelpFormatter, + epilog=""" +COMMANDS: + Commands: test, sync, sync-test, clear-tracker, show-stats + + test - Test Google Drive connectivity + sync - Sync backup directory to Google Drive + sync-test - Run a small test sync to verify integration + clear-tracker - Clear file tracker cache + show-stats - Show file tracker statistics + +OPTIONS: + --project - Project name for sync (default: AIPass) + --note - Note for sync operation (default: Manual sync) + --force - Force sync all files (ignore tracker) + +EXAMPLES: + python3 google_drive_sync.py test + python3 google_drive_sync.py sync /path/to/backups + python3 google_drive_sync.py sync /path/to/backups --project "MyProject" --note "Daily backup" + python3 google_drive_sync.py sync /path/to/backups --force + python3 google_drive_sync.py clear-tracker + python3 google_drive_sync.py show-stats + """ + ) + + parser.add_argument("command", + choices=['test', 'sync', 'sync-test', 'clear-tracker', 'show-stats'], + help="Command to execute") + parser.add_argument("path", nargs='?', help="Backup directory path (required for sync command)") + parser.add_argument("--project", type=str, default="AIPass", help="Project name for sync") + parser.add_argument("--note", type=str, default="Manual sync", help="Note for sync operation") + parser.add_argument("--force", action="store_true", help="Force sync all files (ignore tracker)") + + args = parser.parse_args() + + # Check if module is enabled + config = _load_config() + if not config.get("config", {}).get("enabled", True): + console.print("Warning: Google Drive sync is disabled") + sys.exit(0) + + if args.command == 'clear-tracker': + if _clear_file_tracker(): + console.print("File tracker cleared successfully") + sys.exit(0) + else: + console.print("Failed to clear file tracker") + sys.exit(1) + + elif args.command == 'show-stats': + if _show_file_tracker_stats(): + sys.exit(0) + else: + sys.exit(1) + + elif args.command == 'sync': + if not args.path: + console.print("Error: sync command requires a path argument") + console.print("Usage: python3 google_drive_sync.py sync /path/to/backups") + sys.exit(1) + + backup_path = Path(args.path) + + if not backup_path.exists(): + console.print(f"Error: Backup directory not found: {backup_path}") + sys.exit(1) + + sync = GoogleDriveSync() + if not sync.authenticate(): + console.print("[red]Failed to authenticate with Google Drive[/red]") + sys.exit(1) + + # Phase 1: Scan + console.print(f"Scanning {backup_path}...") + files_to_upload, skipped, total = sync.prepare_sync(backup_path, args.force) + upload_count = len(files_to_upload) + + console.print(f" Files scanned: {total}") + console.print(f" To upload: {upload_count} ({'forced' if args.force else 'changed/new'})") + console.print(f" Unchanged: {skipped}") + + if upload_count == 0: + console.print("[green]All files up to date - nothing to sync[/green]") + sys.exit(0) + + console.print(f"\nSyncing to Google Drive (project: {args.project})...") + + def cli_progress(completed, total_upload, successes): + """Display CLI upload progress.""" + console.print(f" Progress: {completed}/{total_upload} ({successes} OK)") + + # Phase 2: Upload + result = sync.sync_backup_files( + backup_path, args.project, args.note, args.force, + prepared_files=files_to_upload, + skipped_count=skipped, + total_count=total, + progress_fn=cli_progress + ) + + console.print() + if result["success"]: + console.print(f"[green]Sync complete: {result['uploaded']} uploaded, {result['skipped']} unchanged[/green]") + sys.exit(0) + else: + console.print(f"[yellow]Sync finished: {result['uploaded']} uploaded, {result['failed']} failed, {result['skipped']} unchanged[/yellow]") + sys.exit(1) + + elif args.command == 'sync-test': + # Create a minimal args namespace for the test function + if _run_sync_test(): + sys.exit(0) + else: + sys.exit(1) + + elif args.command == 'test': + if _test_drive_sync(): + console.print("Google Drive sync test successful") + sys.exit(0) + else: + console.print("Google Drive sync test failed") + sys.exit(1) \ No newline at end of file diff --git a/src/aipass/backup/apps/modules/integrations.py b/src/aipass/backup/apps/modules/integrations.py new file mode 100755 index 00000000..13417ea4 --- /dev/null +++ b/src/aipass/backup/apps/modules/integrations.py @@ -0,0 +1,287 @@ +# ===================AIPASS==================== +# META DATA HEADER +# Name: integrations.py - External integrations and backup protection +# Date: 2025-11-29 +# Version: 2.1.0 +# Category: backup_system +# +# CHANGELOG (Max 5 entries): +# - v2.1.0 (2026-03-06): Adapted for AIPass public repo +# * Removed shebang, sys.path manipulation, prax/cli imports +# * Uses standard logging and rich console +# * Relative handler imports, GoogleDriveSync via sibling module +# - v2.0.1 (2025-11-29): UX FIX - Import error and missing help output +# - v2.0.0 (2025-11-16): Created seed-compliant layout module +# - v1.1.0 (2025-10-30): Original backup_integrations features +# - v1.0.0 (2025-10-14): Initial extraction +# +# CODE STANDARDS: +# - Follow seed 3-layer architecture +# - Orchestrate workflows, delegate to handlers +# - Import handlers, never implement business logic +# ============================================= + +""" +Backup System Integrations Module + +Handles external integrations for the backup system following seed architecture standards. +Provides CLI command routing and integration orchestration. + +This module manages optional features that integrate with external services +or provide additional protection mechanisms for backups: +- Google Drive cloud synchronization (optional) +- Backup directory protection (read-only permissions) + +Architecture Pattern: +- handle_command(args) - CLI entry point for integration commands +- Integration functions with proper error handling and logging +- Graceful degradation for optional dependencies (Google Drive) +""" + +# ============================================= +# IMPORTS +# ============================================= + +import sys +import os +import stat +import logging +from pathlib import Path + +from rich.console import Console + +logger = logging.getLogger(__name__) +console = Console() + + +def _header(text): + console.print(f"\n[bold cyan]{'='*70}[/bold cyan]") + console.print(f"[bold cyan] {text}[/bold cyan]") + console.print(f"[bold cyan]{'='*70}[/bold cyan]") + + +# Handler imports - relative imports +from ..handlers.utils.system_utils import safe_print +from ..handlers.operations.integration_ops import ( + sync_to_drive as _sync_to_drive_handler, + set_backup_readonly as _set_backup_readonly_handler, +) + +# Google Drive sync integration (optional - graceful degradation if not available) +try: + from .google_drive_sync import GoogleDriveSync + DRIVE_SYNC_AVAILABLE = True +except ImportError: + GoogleDriveSync = None # type: ignore + DRIVE_SYNC_AVAILABLE = False + logger.info("[integrations] Google Drive sync module not available - sync features disabled") + +# ============================================= +# MODULE-LEVEL COMMAND HANDLER +# ============================================= + + +def handle_command(args) -> bool | None: + """Route integration commands to appropriate handler. + + This is the module-level entry point for CLI commands related to integrations. + Routes integration-specific commands like sync-to-drive and set-readonly. + + Args: + args: Command-line arguments from CLI parser + + Returns: + bool: True if command succeeded, False otherwise, None if command not matched + + Example: + result = handle_command(args) + if result: + console.print("Integration command completed successfully") + + Supported commands: + - sync-to-drive: Sync backup to Google Drive + - set-readonly: Protect backup with read-only permissions + """ + if not hasattr(args, 'integration_command'): + return None + + if args.integration_command == 'sync-to-drive': + backup_path = Path(args.backup_path) if hasattr(args, 'backup_path') else None + source_dir = Path(args.source_dir) if hasattr(args, 'source_dir') else None + mode = getattr(args, 'mode', 'versioned') + backup_note = getattr(args, 'backup_note', '') + + if not backup_path or not source_dir: + logger.warning("[integrations] Missing required arguments for sync-to-drive") + return False + + return _sync_to_drive(backup_path, source_dir, mode, backup_note) + + elif args.integration_command == 'set-readonly': + backup_path = Path(args.backup_path) if hasattr(args, 'backup_path') else None + if not backup_path: + logger.warning("[integrations] Missing backup_path argument for set-readonly") + return False + + return _set_backup_readonly(backup_path) + + return None + +# ============================================= +# GOOGLE DRIVE INTEGRATION +# ============================================= + + +def _sync_to_drive(backup_path: Path, source_dir: Path, mode: str, backup_note: str = "") -> bool: + """Orchestrate sync to Google Drive using integration handler. + + Args: + backup_path: Path to backup directory to sync + source_dir: Source directory being backed up + mode: Backup mode ('snapshot' or 'versioned') + backup_note: Optional note describing the backup + + Returns: + bool: True if sync succeeded, False otherwise + """ + if not DRIVE_SYNC_AVAILABLE: + safe_print("\033[93m[WARNING] Google Drive sync module not available - check backup_system installation\033[0m") + logger.warning("[integrations] Drive sync requested but google_drive_sync module not found") + return False + + if mode != 'versioned': + safe_print("[INFO] Drive sync only available for versioned backups") + logger.info(f"[integrations] Drive sync skipped - mode {mode} not supported (versioned only)") + return False + + try: + # Display sync header + safe_print("\n" + "="*70) + safe_print("\033[96m GOOGLE DRIVE SYNC\033[0m") + safe_print("="*70) + + project_name = "AIPass" + safe_print(f"\033[92m✓\033[0m Google Drive folder name: \033[1m{project_name}\033[0m") + safe_print(f"\033[92m✓\033[0m Destination: Google Drive (versioned backups)") + safe_print("-"*70) + + logger.info("[integrations] Starting Google Drive sync") + safe_print("\033[94m[AUTH]\033[0m Verifying Google Drive credentials...") + + success = _sync_to_drive_handler( + backup_path=backup_path, + source_dir=source_dir, + mode=mode, + backup_note=backup_note, + drive_sync_module=GoogleDriveSync, + drive_sync_available=DRIVE_SYNC_AVAILABLE, + ) + + if success: + safe_print("\033[92m✓ Sync completed successfully\033[0m") + logger.info(f"[integrations] Drive sync completed successfully for {project_name}") + else: + logger.error("[integrations] Drive sync completed with errors") + return success + + except Exception as e: + error_msg = f"Drive sync failed: {e}" + safe_print(f"[ERROR] {error_msg}") + logger.error(f"[integrations] {error_msg}") + return False + + + +# ============================================= +# FILE PROTECTION +# ============================================= + + +def _set_backup_readonly(backup_path: Path) -> bool: + """Orchestrate backup directory read-only protection using handler. + + Args: + backup_path: Path to backup directory to protect + + Returns: + bool: True if protection applied successfully, False otherwise + """ + try: + if not backup_path.exists(): + warning_msg = f"Backup path does not exist: {backup_path}" + safe_print(f"[WARNING] {warning_msg}") + logger.warning(f"[integrations] {warning_msg}") + return False + + success, message = _set_backup_readonly_handler(backup_path) + + if success: + safe_print(f"[PROTECTION] Backup directory set to read-only: {backup_path}") + logger.info(f"[integrations] Read-only set: {backup_path} ({message})") + else: + safe_print(f"[WARNING] {message}") + logger.warning(f"[integrations] {message}") + return success + + except Exception as e: + warning_msg = f"Could not set read-only protection: {e}" + safe_print(f"[WARNING] {warning_msg}") + logger.warning(f"[integrations] {warning_msg}") + return False + + + +# ============================================= +# MODULE INITIALIZATION +# ============================================= + +# Log module initialization +logger.info("[integrations] Module loaded - external integration support ready") +if DRIVE_SYNC_AVAILABLE: + logger.info("[integrations] Google Drive sync available") +else: + logger.info("[integrations] Google Drive sync unavailable - install google_drive_sync.py to enable") + +# ============================================= +# MAIN ENTRY POINT +# ============================================= + +def print_help(): + """Display help information for the integrations module.""" + _header("BACKUP SYSTEM - INTEGRATIONS MODULE") + safe_print("") + safe_print("\033[1mPURPOSE:\033[0m") + safe_print(" External integrations for the backup system") + safe_print("") + safe_print("\033[1mAVAILABLE SUBCOMMANDS:\033[0m") + safe_print("") + safe_print(" \033[92msync-to-drive\033[0m") + safe_print(" Sync versioned backups to Google Drive cloud storage") + safe_print(" Requires: backup_path, source_dir, mode, backup_note") + safe_print(" Status: " + ("\033[92mAvailable\033[0m" if DRIVE_SYNC_AVAILABLE else "\033[93mUnavailable (google_drive_sync module not found)\033[0m")) + safe_print("") + safe_print(" \033[92mset-readonly\033[0m") + safe_print(" Protect backup directory with read-only permissions") + safe_print(" Requires: backup_path") + safe_print(" Status: \033[92mAvailable\033[0m") + safe_print("") + safe_print("\033[1mUSAGE:\033[0m") + safe_print(" This module is called via the backup system CLI:") + safe_print(" \033[90m$ backup integration sync-to-drive --backup-path ...\033[0m") + safe_print(" \033[90m$ backup integration set-readonly --backup-path \033[0m") + safe_print("") + safe_print("\033[1mOR\033[0m import functions directly in Python:") + safe_print(" \033[90mfrom aipass.backup.apps.modules.integrations import handle_command\033[0m") + safe_print("") + safe_print("-"*70) + safe_print("\033[1mCommands:\033[0m sync-to-drive, set-readonly") + safe_print("="*70) + safe_print("") + +if __name__ == "__main__": + """Display help when module is run directly.""" + import sys as _sys + if len(_sys.argv) > 1 and _sys.argv[1] in ['--help', '-h', 'help']: + print_help() + else: + print_help() diff --git a/src/aipass/backup/apps/modules/reauth_drive.py b/src/aipass/backup/apps/modules/reauth_drive.py new file mode 100644 index 00000000..55262246 --- /dev/null +++ b/src/aipass/backup/apps/modules/reauth_drive.py @@ -0,0 +1,147 @@ +# ===================AIPASS==================== +# META DATA HEADER +# Name: reauth_drive.py - Google Drive Re-Authentication Module +# Date: 2026-02-21 +# Version: 1.2.0 +# Category: backup_system +# +# CHANGELOG (Max 5 entries): +# - v1.2.0 (2026-03-06): Adapted for AIPass public repo +# * Removed shebang, sys.path manipulation, prax/cli imports +# * Uses standard logging and rich console +# * Relative handler imports, Google API deps wrapped in try/except +# - v1.1.0 (2026-02-21): Fixed handle_command() signature (command, args) → (args) +# - v1.0.0 (2026-02-20): Initial version - standalone reauth utility +# +# CODE STANDARDS: +# - Handlers implement logic, modules orchestrate +# ============================================= + +""" +reauth_drive - Google Drive Re-Authentication Module + +Standalone Google Drive re-authentication module. +Uses console-based OAuth flow (no browser needed). +Delegates implementation to reauth_handler. +""" + +import sys +import logging +from pathlib import Path + +from rich.console import Console + +logger = logging.getLogger(__name__) +console = Console() + +# Handler imports +from ..handlers.utils.reauth_handler import reauth as _run_reauth + +CLIENT_SECRETS = Path(__file__).resolve().parents[1] / 'credentials.json' +CREDS_PATH = Path.home() / '.aipass' / 'drive_creds.json' + + +def handle_command(args) -> bool: + """Handle reauth commands routed from the backup_system orchestrator. + + Args: + args: Command-line arguments from CLI parser + + Returns: + bool: True if command handled, False otherwise + """ + if not hasattr(args, 'command'): + return False + + command = args.command + if command in ['--help', '-h', 'help']: + print_help() + return True + + if command == 'reauth': + return _execute_reauth() + + return False + + +def print_help(): + """Display help for the reauth_drive module.""" + console.print() + console.print("[bold cyan]reauth_drive - Google Drive Re-Authentication[/bold cyan]") + console.print() + console.print("Re-authenticates with Google Drive using console OAuth flow.") + console.print() + console.print("[yellow]Usage:[/yellow]") + console.print(" python3 reauth_drive.py # Run re-authentication") + console.print(" python3 reauth_drive.py --help # Show this help") + console.print() + + +def _execute_reauth() -> bool: + """Orchestrate the re-authentication flow with user feedback.""" + try: + from google.oauth2.credentials import Credentials + from google.auth.transport.requests import Request # noqa: F811 + from googleapiclient.discovery import build + except ImportError as e: + console.print(f"[red]Missing package: {e}[/red]") + console.print(f"Install: {sys.executable} -m pip install google-api-python-client google-auth-httplib2 google-auth-oauthlib") + return False + + if not CLIENT_SECRETS.exists(): + console.print(f"[red]ERROR: Client secrets not found at: {CLIENT_SECRETS}[/red]") + logger.warning(f"OAuth client secrets not found at: {CLIENT_SECRETS}") + return False + + console.print() + console.print("[bold cyan]" + "="*60 + "[/bold cyan]") + console.print("[bold cyan]GOOGLE DRIVE RE-AUTHENTICATION[/bold cyan]") + console.print("[bold cyan]" + "="*60 + "[/bold cyan]") + + # Check for token refresh first + if CREDS_PATH.exists(): + try: + creds = Credentials.from_authorized_user_file(str(CREDS_PATH), ['https://www.googleapis.com/auth/drive.file']) + if creds and creds.expired and creds.refresh_token: + console.print("Attempting token refresh...") + logger.info("Attempting Drive token refresh") + except Exception as e: + logger.warning(f"Token refresh check failed: {e}") + + console.print() + console.print("Starting local auth server...") + console.print("A URL will be printed below - open it in your browser.") + console.print("After authorizing, the browser will redirect to localhost.") + console.print() + + success = _run_reauth(CLIENT_SECRETS) + + if success: + console.print("[green]Re-authentication SUCCESSFUL![/green]") + logger.info("Drive re-authentication successful") + # Show account info + try: + creds = Credentials.from_authorized_user_file(str(CREDS_PATH), ['https://www.googleapis.com/auth/drive.file']) + service = build('drive', 'v3', credentials=creds) + about = service.about().get(fields="user,storageQuota").execute() + email = about['user']['emailAddress'] + quota = about.get('storageQuota', {}) + usage_gb = int(quota.get('usage', 0)) / (1024**3) + limit_gb = int(quota.get('limit', 0)) / (1024**3) + console.print(f"Authenticated as: {email}") + console.print(f"Storage: {usage_gb:.2f} GB / {limit_gb:.2f} GB") + except Exception: + pass + else: + console.print("[red]Re-authentication FAILED[/red]") + logger.error("Drive re-authentication failed") + + return success + + +if __name__ == "__main__": + if len(sys.argv) > 1 and sys.argv[1] in ['--help', '-h', 'help']: + print_help() + sys.exit(0) + success = _execute_reauth() + sys.exit(0 if success else 1) diff --git a/src/aipass/backup/apps/plugins/__init__.py b/src/aipass/backup/apps/plugins/__init__.py new file mode 100644 index 00000000..69b056dd --- /dev/null +++ b/src/aipass/backup/apps/plugins/__init__.py @@ -0,0 +1 @@ +# Plugins package - Pluggable components for branch capabilities diff --git a/src/aipass/backup/tests/__init__.py b/src/aipass/backup/tests/__init__.py new file mode 100644 index 00000000..e69de29b diff --git a/src/aipass/daemon/__init__.py b/src/aipass/daemon/__init__.py new file mode 100644 index 00000000..e69de29b diff --git a/src/aipass/daemon/apps/__init__.py b/src/aipass/daemon/apps/__init__.py new file mode 100644 index 00000000..73ab12a7 --- /dev/null +++ b/src/aipass/daemon/apps/__init__.py @@ -0,0 +1 @@ +# Apps package - Branch application modules and handlers diff --git a/src/aipass/daemon/apps/daemon.py b/src/aipass/daemon/apps/daemon.py new file mode 100644 index 00000000..e434bd14 --- /dev/null +++ b/src/aipass/daemon/apps/daemon.py @@ -0,0 +1,213 @@ + +# ===================AIPASS==================== +# META DATA HEADER +# Name: daemon.py - DAEMON Branch Entry Point +# Date: 2026-01-21 +# Version: 1.0.0 +# Category: daemon +# +# CHANGELOG (Max 5 entries): +# - v1.0.0 (2026-01-21): Initial branch creation +# +# CODE STANDARDS: +# - Handlers implement logic, modules orchestrate +# ============================================= + +""" +DAEMON Branch - Main Orchestrator + +Explicit module imports: +- Imports known modules from modules/ directory +- Routes commands to discovered modules automatically +""" + +# Standard library imports +import sys +from typing import List, Any + +# Logger +import logging +logger = logging.getLogger(__name__) + +# Console +from rich.console import Console +console = Console() + +def _header(text): + console.print(f"\n[bold cyan]{'='*70}[/bold cyan]") + console.print(f"[bold cyan] {text}[/bold cyan]") + console.print(f"[bold cyan]{'='*70}[/bold cyan]") + +# ============================================================================= +# MODULE DISCOVERY +# ============================================================================= + +from .modules import update, schedule, activity_report, actions + +def get_modules() -> List[Any]: + """ + Return list of known modules that implement handle_command(). + + Returns: + List of module objects with handle_command function + """ + modules = [] + for mod in [update, schedule, activity_report, actions]: + if hasattr(mod, 'handle_command'): + modules.append(mod) + return modules + + +def route_command(command: str, args: List[str], modules: List[Any]) -> bool: + """ + Route command to appropriate module + + Args: + command: Command name (e.g., 'create', 'update', 'list') + args: Additional arguments + modules: List of discovered modules + + Returns: + True if command was handled, False otherwise + """ + for module in modules: + try: + if module.handle_command(command, args): + return True + except Exception as e: + logger.error(f"[DAEMON] Module {module.__name__} error: {e}") + + return False + +# ============================================================================= +# INTROSPECTION DISPLAY +# ============================================================================= + +def print_introspection(modules: List[Any]): + """Display discovered modules when run without arguments""" + console.print() + console.print("[bold cyan]DAEMON - Branch Management System[/bold cyan]") + console.print() + console.print("[dim]Module orchestration[/dim]") + console.print() + + console.print(f"[yellow]Modules:[/yellow] {len(modules)}") + console.print() + + if modules: + for module in modules: + module_name = module.__name__.split('.')[-1] + # Get first line of docstring + description = "No description" + if module.__doc__: + description = module.__doc__.strip().split('\n')[0] + console.print(f" [cyan]*[/cyan] {module_name:20} [dim]{description}[/dim]") + else: + console.print(" [dim]No modules discovered[/dim]") + + console.print() + console.print("[dim]Run 'daemon --help' for usage information[/dim]") + console.print() + + +# ============================================================================= +# DRONE COMPLIANCE - HELP SYSTEM +# ============================================================================= + +def print_help(modules: List[Any]): + """Display Rich-formatted help""" + console.print() + _header("DAEMON - Branch Management System") + console.print() + + console.print("[dim]Module orchestration[/dim]") + console.print() + console.print("-" * 70) + console.print() + + console.print("[bold cyan]USAGE:[/bold cyan]") + console.print() + console.print(" [dim]daemon [args...][/dim]") + console.print(" [dim]daemon --help[/dim]") + console.print() + console.print("-" * 70) + console.print() + + console.print("[bold cyan]AVAILABLE COMMANDS:[/bold cyan]") + console.print() + + if modules: + for module in modules: + module_name = module.__name__.split('.')[-1] + # Get first line of docstring + description = "No description" + if module.__doc__: + description = module.__doc__.strip().split('\n')[0] + + console.print(f" [green]{module_name:20}[/green] [dim]{description}[/dim]") + else: + console.print(" [dim]No modules discovered[/dim]") + + console.print() + console.print("-" * 70) + console.print() + + console.print("[bold]TIP:[/bold] For module-specific help:") + console.print(" [dim]daemon --help[/dim]") + console.print() + + +# ============================================================================= +# MAIN ENTRY POINT +# ============================================================================= + +def main(): + """Main entry point - routes commands or shows help""" + + # Get available modules + modules = get_modules() + + # Parse arguments + args = sys.argv[1:] + + # Show introspection when run with no arguments + if len(args) == 0: + print_introspection(modules) + return 0 + + # Version flag + if args[0] in ['--version', '-V']: + console.print("DAEMON v1.0.0") + return 0 + + # Show help for explicit help flags + if args[0] in ['--help', '-h', 'help']: + print_help(modules) + return 0 + + # Extract command and remaining args + command = args[0] + remaining_args = args[1:] if len(args) > 1 else [] + + # Route to modules + if route_command(command, remaining_args, modules): + return 0 + else: + console.print() + console.print(f"[red]Unknown command: {command}[/red]") + console.print() + console.print("Run [dim]daemon --help[/dim] for available commands") + console.print() + return 1 + + +if __name__ == "__main__": + try: + sys.exit(main()) + except KeyboardInterrupt: + console.print("\n\nOperation cancelled by user") + sys.exit(0) + except Exception as e: + logger.error(f"DAEMON entry point error: {e}", exc_info=True) + console.print(f"\nError: {e}") + sys.exit(1) diff --git a/src/aipass/daemon/apps/daemon_wakeup.py b/src/aipass/daemon/apps/daemon_wakeup.py new file mode 100644 index 00000000..aaca53d8 --- /dev/null +++ b/src/aipass/daemon/apps/daemon_wakeup.py @@ -0,0 +1,296 @@ + +# ===================AIPASS==================== +# META DATA HEADER +# Name: daemon_wakeup.py - DAEMON Wake-Up Cron Trigger +# Date: 2026-02-15 +# Version: 1.0.0 +# Category: daemon/apps +# +# CHANGELOG (Max 5 entries): +# - v1.0.0 (2026-02-15): Initial implementation - cron-triggered wake-up checker +# +# CODE STANDARDS: +# - Handlers implement logic, modules orchestrate +# - No Rich console (headless cron execution) +# - Stdout logging (cron redirects to logs/daemon_wakeup.log) +# - stdlib only + optional imports +# ============================================= + +""" +Cron trigger script for the DAEMON wake-up system. + +Called periodically by cron. Standalone script -- not imported as a module. + +Flow: + 1. Acquire single-instance lock + 2. Send Telegram "waking up" notification via daemon bot (optional) + 3. Check daemon's email inbox (new/opened counts) + 4. Build summary report with sender/subject listings + 5. Send report notification via daemon bot (optional) +""" + +# ============================================= +# IMPORTS +# ============================================= + +import sys +import json +import fcntl +import time +from pathlib import Path +from datetime import datetime + +import logging +logger = logging.getLogger(__name__) + +# ============================================= +# OPTIONAL IMPORTS +# ============================================= + +# Telegram notifications (optional) — use absolute imports for standalone script +try: + from aipass.daemon.apps.handlers.schedule.assistant_notifier import ( + notify_wakeup, + notify_report, + notify_error, + ) + TELEGRAM_AVAILABLE = True +except ImportError: + TELEGRAM_AVAILABLE = False + notify_wakeup = None + notify_report = None + notify_error = None + +# ============================================= +# CONSTANTS +# ============================================= + +_DAEMON_ROOT = Path(__file__).resolve().parents[2] # src/aipass/daemon/ +JSON_DIR = _DAEMON_ROOT / "daemon_json" + +LOCK_FILE = JSON_DIR / "wakeup.lock" +CHAT_LOCK_FILE = JSON_DIR / "chat.lock" +INBOX_PATH = _DAEMON_ROOT / "ai_mail.local" / "inbox.json" + +# ============================================= +# LOGGING +# ============================================= + +def log(message: str) -> None: + """Print timestamped log line to stdout (captured by cron redirect).""" + timestamp = datetime.now().strftime("%Y-%m-%d %H:%M:%S") + print(f"[{timestamp}] {message}", flush=True) + + +# ============================================= +# CHAT LOCK CHECK +# ============================================= + +def is_chat_active() -> bool: + """ + Check if daemon is in an active Telegram chat session. + + The chat listener creates a lock file while running. + If the lock exists and is fresh (<60 minutes), skip the wake-up cycle + to avoid interfering with the live conversation. + + Returns: + True if chat session is active, False otherwise + """ + if not CHAT_LOCK_FILE.exists(): + return False + try: + age = time.time() - CHAT_LOCK_FILE.stat().st_mtime + if age > 3600: # 60 minutes - treat as stale + return False + return True + except OSError: + return False + + +# ============================================= +# EMAIL CHECK +# ============================================= + +def check_inbox() -> dict: + """ + Check daemon's email inbox for new and opened emails. + + Reads inbox.json directly (stdlib only, no module imports that + require Rich/console). + + Returns: + Dict with keys: new_count, opened_count, emails (list of brief dicts) + """ + result = { + "new_count": 0, + "opened_count": 0, + "emails": [], + } + + if not INBOX_PATH.exists(): + log("Inbox file not found, skipping email check") + return result + + try: + with open(INBOX_PATH, "r", encoding="utf-8") as f: + inbox_data = json.load(f) + except (json.JSONDecodeError, IOError) as e: + log(f"WARNING: Failed to read inbox: {e}") + return result + + messages = inbox_data.get("messages", []) + + for msg in messages: + status = msg.get("status", "") + if status == "new": + result["new_count"] += 1 + result["emails"].append({ + "from": msg.get("from", "unknown"), + "subject": msg.get("subject", "(no subject)"), + "status": "new", + }) + elif status == "opened": + result["opened_count"] += 1 + result["emails"].append({ + "from": msg.get("from", "unknown"), + "subject": msg.get("subject", "(no subject)"), + "status": "opened", + }) + + return result + + +# ============================================= +# REPORT BUILDER +# ============================================= + +def build_report(inbox: dict) -> str: + """ + Build a summary report from inbox check results. + + Args: + inbox: Dict from check_inbox() + + Returns: + Formatted report string + """ + new_count = inbox["new_count"] + opened_count = inbox["opened_count"] + total_unread = new_count + opened_count + emails = inbox["emails"] + + lines = [] + + if total_unread == 0: + lines.append("No new emails") + else: + lines.append(f"New: {new_count} | Opened: {opened_count}") + + # List up to 10 most recent unread emails (brief, 1 line each) + shown = emails[:10] + for email in shown: + marker = "[NEW]" if email["status"] == "new" else "[OPENED]" + subject = email["subject"][:50] + lines.append(f" {marker} {email['from']}: {subject}") + + if len(emails) > 10: + lines.append(f" ... and {len(emails) - 10} more") + + return "\n".join(lines) + + +# ============================================= +# MAIN +# ============================================= + +def main() -> int: + """ + Main cron entry point. + + Returns: + 0 on success, 1 on error + """ + log("=" * 60) + log("Daemon wake-up triggered") + + # Check if chat session is active (skip wake-up to avoid interference) + if is_chat_active(): + log("Chat session active, skipping wake-up cycle") + return 0 + + # Ensure lock directory exists + LOCK_FILE.parent.mkdir(parents=True, exist_ok=True) + + # Acquire single-instance lock (non-blocking, stdlib fcntl) + lock_fd = open(LOCK_FILE, "w") + try: + fcntl.flock(lock_fd, fcntl.LOCK_EX | fcntl.LOCK_NB) + except OSError: + log("Another instance already running, skipping.") + lock_fd.close() + return 0 + + try: + return _run_locked() + finally: + fcntl.flock(lock_fd, fcntl.LOCK_UN) + lock_fd.close() + + +def _run_locked() -> int: + """Execute the wake-up job while holding the lock.""" + exit_code = 0 + + # Step 1: Send "waking up" notification (optional) + if TELEGRAM_AVAILABLE: + try: + notify_wakeup() + log("Telegram: wakeup notification sent") + except Exception as e: + log(f"WARNING: Telegram wakeup notification failed: {e}") + + # Step 2: Check inbox + try: + inbox = check_inbox() + log(f"Inbox: {inbox['new_count']} new, {inbox['opened_count']} opened") + except Exception as e: + log(f"CRITICAL: Unhandled error in check_inbox: {e}") + if TELEGRAM_AVAILABLE: + try: + notify_error(f"Inbox check failed: {e}") + except Exception: + pass + return 1 + + # Step 3: Build report + report = build_report(inbox) + log(f"Report: {report.splitlines()[0]}") + + # Step 4: Send report notification (optional) + if TELEGRAM_AVAILABLE: + try: + notify_report(report) + log("Telegram: report notification sent") + except Exception as e: + log(f"WARNING: Telegram report notification failed: {e}") + exit_code = 1 + + log("Daemon wake-up finished") + log("=" * 60) + return exit_code + + +if __name__ == "__main__": + try: + sys.exit(main()) + except Exception as e: + # Last-resort catch -- never crash silently + timestamp = datetime.now().strftime("%Y-%m-%d %H:%M:%S") + print(f"[{timestamp}] FATAL: Unhandled exception: {e}", flush=True) + if TELEGRAM_AVAILABLE: + try: + notify_error(f"FATAL: {e}") + except Exception: + pass + sys.exit(1) diff --git a/src/aipass/daemon/apps/extensions/__init__.py b/src/aipass/daemon/apps/extensions/__init__.py new file mode 100644 index 00000000..95322c94 --- /dev/null +++ b/src/aipass/daemon/apps/extensions/__init__.py @@ -0,0 +1 @@ +# Extensions package - Drop-in extensions for branch functionality diff --git a/src/aipass/daemon/apps/handlers/__init__.py b/src/aipass/daemon/apps/handlers/__init__.py new file mode 100644 index 00000000..c3557dd4 --- /dev/null +++ b/src/aipass/daemon/apps/handlers/__init__.py @@ -0,0 +1 @@ +"""Daemon handlers package.""" diff --git a/src/aipass/daemon/apps/handlers/actions/__init__.py b/src/aipass/daemon/apps/handlers/actions/__init__.py new file mode 100644 index 00000000..e69de29b diff --git a/src/aipass/daemon/apps/handlers/actions/actions_registry.py b/src/aipass/daemon/apps/handlers/actions/actions_registry.py new file mode 100644 index 00000000..5d965b78 --- /dev/null +++ b/src/aipass/daemon/apps/handlers/actions/actions_registry.py @@ -0,0 +1,496 @@ + +# ===================AIPASS==================== +# META DATA HEADER +# Name: actions_registry.py - Numbered Action Registry +# Date: 2026-03-02 +# Version: 1.0.0 +# Category: daemon/handlers/actions +# +# CHANGELOG (Max 5 entries): +# - v1.0.0 (2026-03-02): Initial creation - DPLAN-043 +# * Sequential 4-digit IDs (0001-9999) +# * JSON storage with CRUD operations +# * Plugin auto-migration from PLUGIN_CONFIG +# * Due-checking logic for all schedule types +# * Supports: plugin, schedule, reminder action types +# +# CODE STANDARDS: +# - Handler independence: pure business logic +# - No Rich console (headless compatible) +# ============================================= + +""" +Numbered Action Registry — DPLAN-043 + +Central registry for all scheduled actions. Each action gets a sequential +numeric ID (0001, 0002, ...) and can be individually toggled on/off. + +Replaces the old all-or-nothing daemon + kill switch model with granular +per-action control. + +Action types: + - plugin: Backed by a plugin file in apps/plugins/ (migrated from existing system) + - schedule: Custom recurring action (dispatches via wake.py) + - reminder: One-shot action that auto-completes after firing +""" + +import json +import logging +from datetime import datetime, timedelta +from pathlib import Path +from typing import Optional + +logger = logging.getLogger(__name__) + +# Paths +_DAEMON_ROOT = Path(__file__).resolve().parents[4] # src/aipass/daemon/ +REGISTRY_FILE = _DAEMON_ROOT / "daemon_json" / "actions_registry.json" +PLUGINS_DIR = _DAEMON_ROOT / "apps" / "plugins" + +def _empty_registry() -> dict: + """Return a fresh empty registry structure (avoids shared mutable state).""" + return {"version": 1, "next_id": 1, "actions": []} + +# ============================================= +# STORAGE +# ============================================= + +def load_registry() -> dict: + """Load the actions registry from disk. Returns empty registry if missing.""" + if not REGISTRY_FILE.exists(): + return _empty_registry().copy() + try: + with open(REGISTRY_FILE, 'r', encoding='utf-8') as f: + data = json.load(f) + if "actions" not in data: + data["actions"] = [] + if "next_id" not in data: + data["next_id"] = 1 + return data + except (json.JSONDecodeError, OSError) as e: + logger.error("[actions_registry] Failed to load: %s", e) + return _empty_registry().copy() + + +def save_registry(data: dict) -> bool: + """Save the actions registry to disk. Returns True on success.""" + try: + REGISTRY_FILE.parent.mkdir(parents=True, exist_ok=True) + with open(REGISTRY_FILE, 'w', encoding='utf-8') as f: + json.dump(data, f, indent=2) + f.write("\n") + return True + except OSError as e: + logger.error("[actions_registry] Failed to save: %s", e) + return False + + +# ============================================= +# ID GENERATION +# ============================================= + +def _get_next_id(registry: dict) -> str: + """Get next sequential ID as 4-digit string. Advances next_id.""" + next_num = registry.get("next_id", 1) + action_id = f"{next_num:04d}" + registry["next_id"] = next_num + 1 + return action_id + + +# ============================================= +# CRUD OPERATIONS +# ============================================= + +def create_action( + name: str, + action_type: str, + schedule_type: str, + target_branch: str = "", + prompt: str = "", + time: Optional[str] = None, + interval_minutes: Optional[int] = None, + due_date: Optional[str] = None, + fresh: bool = True, + max_turns: int = 50, + enabled: bool = True, + self_dispatch: bool = False, + plugin_file: Optional[str] = None, +) -> dict: + """ + Create a new action and save to registry. + + Args: + name: Human-readable action name (e.g., "daily_audit") + action_type: "plugin" | "schedule" | "reminder" + schedule_type: "daily" | "hourly" | "interval" | "once" + target_branch: Target branch email (e.g., "@seed") + prompt: What the dispatched agent should do + time: For daily: "HH:MM", for hourly: "MM" + interval_minutes: For interval schedule type + due_date: For reminder (once) type, ISO date string + fresh: Start fresh session (True) or resume (False) + max_turns: Max agent turns + enabled: Active by default + self_dispatch: Plugin handles its own dispatch + plugin_file: Plugin filename (without .py) for plugin-backed actions + + Returns: + The created action dict + """ + registry = load_registry() + action_id = _get_next_id(registry) + + action = { + "id": action_id, + "name": name, + "type": action_type, + "schedule_type": schedule_type, + "time": time, + "interval_minutes": interval_minutes, + "due_date": due_date, + "target_branch": target_branch, + "prompt": prompt, + "fresh": fresh, + "max_turns": max_turns, + "enabled": enabled, + "self_dispatch": self_dispatch, + "plugin_file": plugin_file, + "last_run": None, + "next_run": None, + "created": datetime.now().isoformat(), + "completed": None, + } + + registry["actions"].append(action) + save_registry(registry) + + logger.info("[actions_registry] Created action %s: %s (%s)", action_id, name, action_type) + return action + + +def get_action(action_id: str) -> Optional[dict]: + """Get a single action by ID. Returns None if not found.""" + registry = load_registry() + for action in registry["actions"]: + if action["id"] == action_id: + return action + return None + + +def list_actions(include_completed: bool = False) -> list: + """ + List all actions. + + Args: + include_completed: If True, include completed reminders. + + Returns: + List of action dicts. + """ + registry = load_registry() + actions = registry["actions"] + if not include_completed: + actions = [a for a in actions if a.get("completed") is None] + return actions + + +def toggle_action(action_id: str, enabled: bool) -> bool: + """Toggle an action on or off. Returns True if found and updated.""" + registry = load_registry() + for action in registry["actions"]: + if action["id"] == action_id: + action["enabled"] = enabled + save_registry(registry) + state = "enabled" if enabled else "disabled" + logger.info("[actions_registry] Action %s %s: %s", action_id, state, action["name"]) + return True + return False + + +def delete_action(action_id: str) -> bool: + """Delete an action by ID. Returns True if found and removed.""" + registry = load_registry() + original_len = len(registry["actions"]) + registry["actions"] = [a for a in registry["actions"] if a["id"] != action_id] + if len(registry["actions"]) < original_len: + save_registry(registry) + logger.info("[actions_registry] Deleted action %s", action_id) + return True + return False + + +def update_last_run(action_id: str, timestamp: Optional[str] = None) -> bool: + """Update last_run timestamp for an action. Returns True if found.""" + if timestamp is None: + timestamp = datetime.now().isoformat() + registry = load_registry() + for action in registry["actions"]: + if action["id"] == action_id: + action["last_run"] = timestamp + action["next_run"] = calc_next_run(action) + save_registry(registry) + return True + return False + + +def mark_reminder_completed(action_id: str) -> bool: + """Mark a reminder as completed (one-shot). Returns True if found.""" + registry = load_registry() + for action in registry["actions"]: + if action["id"] == action_id: + action["completed"] = datetime.now().isoformat() + action["enabled"] = False + save_registry(registry) + logger.info("[actions_registry] Reminder %s completed: %s", action_id, action["name"]) + return True + return False + + +# ============================================= +# DUE CHECKING +# ============================================= + +def is_action_due(action: dict) -> bool: + """ + Check if an action should run now. + + For daily: matches current hour:minute, hasn't run today + For hourly: matches current minute, hasn't run this hour + For interval: enough time has elapsed since last run + For once (reminder): due_date <= today, not completed + """ + if not action.get("enabled", False): + return False + + if action.get("completed"): + return False + + now = datetime.now() + schedule_type = action.get("schedule_type", "") + + if schedule_type == "daily": + target_time = action.get("time", "00:00") + try: + target_h, target_m = map(int, target_time.split(":")) + except (ValueError, AttributeError): + return False + if now.hour != target_h or now.minute != target_m: + return False + last_run = action.get("last_run") + if last_run: + try: + last_dt = datetime.fromisoformat(last_run) + if last_dt.date() == now.date(): + return False + except (ValueError, TypeError): + pass + return True + + elif schedule_type == "hourly": + target_m_str = action.get("time", "0") + try: + target_m = int(target_m_str) + except (ValueError, TypeError): + return False + if now.minute != target_m: + return False + last_run = action.get("last_run") + if last_run: + try: + last_dt = datetime.fromisoformat(last_run) + if last_dt.hour == now.hour and last_dt.date() == now.date(): + return False + except (ValueError, TypeError): + pass + return True + + elif schedule_type == "interval": + interval = action.get("interval_minutes", 60) + last_run = action.get("last_run") + if not last_run: + return True + try: + last_dt = datetime.fromisoformat(last_run) + elapsed = (now - last_dt).total_seconds() / 60 + return elapsed >= interval + except (ValueError, TypeError): + return True + + elif schedule_type == "once": + due_date = action.get("due_date") + if not due_date: + return False + try: + due_dt = datetime.fromisoformat(due_date).date() if "T" in due_date else datetime.strptime(due_date, "%Y-%m-%d").date() + return now.date() >= due_dt + except (ValueError, TypeError): + return False + + return False + + +def calc_next_run(action: dict) -> Optional[str]: + """Calculate the next run time for an action. Returns ISO string or None.""" + now = datetime.now() + schedule_type = action.get("schedule_type", "") + + if schedule_type == "daily": + target_time = action.get("time", "00:00") + try: + target_h, target_m = map(int, target_time.split(":")) + except (ValueError, AttributeError): + return None + next_dt = now.replace(hour=target_h, minute=target_m, second=0, microsecond=0) + if next_dt <= now: + next_dt += timedelta(days=1) + return next_dt.isoformat() + + elif schedule_type == "hourly": + target_m_str = action.get("time", "0") + try: + target_m = int(target_m_str) + except (ValueError, TypeError): + return None + next_dt = now.replace(minute=target_m, second=0, microsecond=0) + if next_dt <= now: + next_dt += timedelta(hours=1) + return next_dt.isoformat() + + elif schedule_type == "interval": + interval = action.get("interval_minutes", 60) + last_run = action.get("last_run") + if not last_run: + return now.isoformat() + try: + last_dt = datetime.fromisoformat(last_run) + return (last_dt + timedelta(minutes=interval)).isoformat() + except (ValueError, TypeError): + return now.isoformat() + + elif schedule_type == "once": + due_date = action.get("due_date") + if due_date and not action.get("completed"): + return due_date + return None + + return None + + +def next_due_str(action: dict) -> str: + """Human-readable next due string for display.""" + schedule_type = action.get("schedule_type", "") + + if schedule_type == "daily": + return f"daily @ {action.get('time', '00:00')}" + elif schedule_type == "hourly": + m = action.get("time", "0") + return f"hourly @ :{int(m):02d}" + elif schedule_type == "interval": + interval = action.get("interval_minutes", 60) + last_run = action.get("last_run") + if last_run: + try: + last_dt = datetime.fromisoformat(last_run) + next_dt = last_dt + timedelta(minutes=interval) + if next_dt <= datetime.now(): + return "now" + return next_dt.strftime("%H:%M") + except (ValueError, TypeError): + return "now" + return "now" + elif schedule_type == "once": + return action.get("due_date", "unknown") + + return "unknown" + + +# ============================================= +# PLUGIN MIGRATION +# ============================================= + +def migrate_plugins() -> int: + """ + Scan plugins/ directory and auto-register any plugins not yet in the registry. + + Maps PLUGIN_CONFIG fields to action fields. Preserves last_run timestamps + from .last_run.json. + + Returns: + Number of newly migrated plugins. + """ + registry = load_registry() + existing_plugins = { + a["plugin_file"] + for a in registry["actions"] + if a.get("plugin_file") + } + + # Load last_run data for timestamp preservation + last_run_file = PLUGINS_DIR / ".last_run.json" + last_run_map = {} + if last_run_file.exists(): + try: + last_run_map = json.loads(last_run_file.read_text(encoding="utf-8")) + except (json.JSONDecodeError, OSError): + pass + + # Discover plugins + migrated = 0 + for plugin_path in sorted(PLUGINS_DIR.glob("*.py")): + if plugin_path.name.startswith("_"): + continue + + plugin_name = plugin_path.stem + if plugin_name in existing_plugins: + continue + + # Import plugin to read PLUGIN_CONFIG + try: + import importlib + # Use absolute package path for plugin import + spec_name = f"aipass.daemon.apps.plugins.{plugin_name}" + module = importlib.import_module(spec_name) + + if not hasattr(module, 'PLUGIN_CONFIG'): + continue + + config = module.PLUGIN_CONFIG + except Exception as e: + logger.warning("[actions_registry] Failed to import plugin %s: %s", plugin_name, e) + continue + + # Map PLUGIN_CONFIG to action fields + action_id = _get_next_id(registry) + action = { + "id": action_id, + "name": config.get("name", plugin_name), + "type": "plugin", + "schedule_type": config.get("schedule", "interval"), + "time": config.get("time"), + "interval_minutes": config.get("interval_minutes"), + "due_date": None, + "target_branch": config.get("branch", ""), + "prompt": config.get("prompt", ""), + "fresh": config.get("fresh", True), + "max_turns": config.get("max_turns", 50), + "enabled": config.get("enabled", False), + "self_dispatch": config.get("self_dispatch", False), + "plugin_file": plugin_name, + "last_run": last_run_map.get(config.get("name", plugin_name)), + "next_run": None, + "created": datetime.now().isoformat(), + "completed": None, + } + + # Calculate next_run from last_run + action["next_run"] = calc_next_run(action) + + registry["actions"].append(action) + migrated += 1 + logger.info("[actions_registry] Migrated plugin: %s -> action %s", plugin_name, action_id) + + if migrated > 0: + save_registry(registry) + logger.info("[actions_registry] Migration complete: %d plugin(s) migrated", migrated) + + return migrated diff --git a/src/aipass/daemon/apps/handlers/json/__init__.py b/src/aipass/daemon/apps/handlers/json/__init__.py new file mode 100644 index 00000000..f44e47b3 --- /dev/null +++ b/src/aipass/daemon/apps/handlers/json/__init__.py @@ -0,0 +1 @@ +"""JSON Handlers - Universal JSON operations for Seed branch""" diff --git a/src/aipass/daemon/apps/handlers/json/json_handler.py b/src/aipass/daemon/apps/handlers/json/json_handler.py new file mode 100644 index 00000000..0b784b0c --- /dev/null +++ b/src/aipass/daemon/apps/handlers/json/json_handler.py @@ -0,0 +1,255 @@ + +# ===================AIPASS==================== +# META DATA HEADER +# Name: json_handler.py - JSON Auto-Creating Handler +# Date: 2025-11-21 +# Version: 1.2.0 +# Category: daemon/handlers/json +# +# CHANGELOG (Max 5 entries): +# - v1.2.0 (2026-01-29): Fixed paths to point to DAEMON branch +# - v1.1.0 (2025-11-21): Refactored to comply with error handling +# - v1.0.0 (2025-11-13): Initial JSON auto-creation system +# +# CODE STANDARDS: +# - Pure functions with proper error raising +# - No Prax imports (handler tier 3) +# ============================================= + +""" +JSON handler for DAEMON branch. + +Provides auto-creating JSON file management with templates. +""" + +import json +from pathlib import Path +from datetime import datetime +from typing import Dict, List, Any, Optional +import inspect + +# Constants +_DAEMON_ROOT = Path(__file__).resolve().parents[4] # src/aipass/daemon/ +JSON_DIR = _DAEMON_ROOT / "daemon_json" +JSON_TEMPLATES_DIR = _DAEMON_ROOT / "apps" / "json_templates" + + +def _get_caller_module_name() -> str: + """ + Auto-detect calling module name from call stack. + + Returns: + Module name (e.g., "imports_standard" from imports_standard.py) + """ + stack = inspect.stack() + if len(stack) > 2: + caller_frame = stack[2] + caller_path = Path(caller_frame.filename) + module_name = caller_path.stem + + if module_name and not module_name.startswith('_'): + return module_name + + return "unknown" + + +def load_template(json_type: str, module_name: str) -> Any: + """Load JSON template from template file.""" + template_path = JSON_TEMPLATES_DIR / "default" / f"{json_type}.json" + + if not template_path.exists(): + raise FileNotFoundError(f"Template not found: {template_path}") + + with open(template_path, 'r', encoding='utf-8') as f: + template = json.load(f) + + template_str = json.dumps(template) + template_str = template_str.replace("{{MODULE_NAME}}", module_name) + template_str = template_str.replace("2026-01-21", datetime.now().date().isoformat()) + + return json.loads(template_str) + + +def validate_json_structure(data: Any, json_type: str) -> bool: + """Validate JSON structure matches expected type.""" + if json_type == "config": + if not isinstance(data, dict): + return False + required = ["module_name", "version", "config"] + return all(key in data for key in required) + + elif json_type == "data": + if not isinstance(data, dict): + return False + required = ["created", "last_updated"] + return all(key in data for key in required) + + elif json_type == "log": + return isinstance(data, list) + + return False + + +def get_json_path(module_name: str, json_type: str) -> Path: + """Get path for module JSON file.""" + filename = f"{module_name}_{json_type}.json" + return JSON_DIR / filename + + +def ensure_json_exists(module_name: str, json_type: str) -> bool: + """Ensure JSON file exists, create from template if missing.""" + JSON_DIR.mkdir(parents=True, exist_ok=True) + + json_path = get_json_path(module_name, json_type) + + if json_path.exists(): + try: + with open(json_path, 'r', encoding='utf-8') as f: + data = json.load(f) + + if validate_json_structure(data, json_type): + return True + except json.JSONDecodeError: + pass # File corrupted, regenerate + except OSError: + pass # File unreadable, regenerate + + template = load_template(json_type, module_name) + + with open(json_path, 'w', encoding='utf-8') as f: + json.dump(template, f, indent=2, ensure_ascii=False) + return True + + +def load_json(module_name: str, json_type: str) -> Optional[Any]: + """Load JSON file, auto-create if missing.""" + if not ensure_json_exists(module_name, json_type): + return None + + json_path = get_json_path(module_name, json_type) + + with open(json_path, 'r', encoding='utf-8') as f: + return json.load(f) + + +def save_json(module_name: str, json_type: str, data: Any) -> bool: + """Save JSON file.""" + json_path = get_json_path(module_name, json_type) + + if not validate_json_structure(data, json_type): + raise ValueError(f"Invalid structure for {json_type} JSON") + + if json_type == "data" and isinstance(data, dict): + data["last_updated"] = datetime.now().date().isoformat() + + with open(json_path, 'w', encoding='utf-8') as f: + json.dump(data, f, indent=2, ensure_ascii=False) + return True + + +def ensure_module_jsons(module_name: str) -> bool: + """Ensure all 3 JSON files exist for a module.""" + ensure_json_exists(module_name, "config") + ensure_json_exists(module_name, "data") + ensure_json_exists(module_name, "log") + return True + + +def log_operation(operation: str, data: Optional[Dict[str, Any]] = None, module_name: Optional[str] = None) -> bool: + """ + Add entry to module log with automatic rotation. + + Auto-detects calling module if module_name not provided. + Implements config-controlled log limits to prevent unbounded growth. + When max_log_entries is reached, removes oldest entries (FIFO). + + Args: + operation: Operation name to log + data: Optional data dict + module_name: Optional module name (auto-detected if not provided) + + Returns: + True if successful, False otherwise + """ + if module_name is None: + module_name = _get_caller_module_name() + + ensure_module_jsons(module_name) + + config = load_json(module_name, "config") + max_entries = 100 + if config and "config" in config: + max_entries = config["config"].get("max_log_entries", 100) + + log: List[Dict[str, Any]] = load_json(module_name, "log") or [] + + entry: Dict[str, Any] = { + "timestamp": datetime.now().isoformat(), + "operation": operation + } + + if data: + entry["data"] = data + + log.append(entry) + + if len(log) > max_entries: + log = log[-max_entries:] + + return save_json(module_name, "log", log) + + +def increment_counter(module_name: str, counter_name: str, amount: int = 1) -> bool: + """Increment a counter in data JSON.""" + ensure_module_jsons(module_name) + + data = load_json(module_name, "data") + if data is None: + return False + + if counter_name not in data: + data[counter_name] = 0 + + data[counter_name] += amount + + return save_json(module_name, "data", data) + + +def update_data_metrics(module_name: str, **metrics: Any) -> bool: + """Update data metrics.""" + ensure_module_jsons(module_name) + + data = load_json(module_name, "data") + if data is None: + return False + + for key, value in metrics.items(): + data[key] = value + + return save_json(module_name, "data", data) + + +if __name__ == "__main__": + from rich.console import Console + from rich.panel import Panel + + console = Console() + + console.print() + console.print(Panel.fit( + "[bold cyan]JSON HANDLER - Working Implementation[/bold cyan]", + border_style="bright_blue" + )) + console.print() + console.print("[yellow]TESTING:[/yellow] Creating daemon JSONs...") + + log_operation("test_operation", {"test": "data"}, "daemon") + increment_counter("daemon", "test_counter", 1) + update_data_metrics("daemon", test_metric="working") + + console.print() + console.print(f"[green]Check {JSON_DIR}/ for created files:[/green]") + console.print(" [dim]-[/dim] daemon_config.json") + console.print(" [dim]-[/dim] daemon_data.json") + console.print(" [dim]-[/dim] daemon_log.json") + console.print() diff --git a/src/aipass/daemon/apps/handlers/monitoring/__init__.py b/src/aipass/daemon/apps/handlers/monitoring/__init__.py new file mode 100644 index 00000000..ca64f2af --- /dev/null +++ b/src/aipass/daemon/apps/handlers/monitoring/__init__.py @@ -0,0 +1,56 @@ +# ===================AIPASS==================== +# META DATA HEADER +# Name: __init__.py - Monitoring Handlers Package +# Date: 2026-01-30 +# Version: 0.1.0 +# Category: assistant/handlers/monitoring +# +# CHANGELOG (Max 5 entries): +# - v0.1.0 (2026-01-30): Initial implementation - FPLAN-0266 Phase 1 +# +# CODE STANDARDS: +# - Handler package init - no Prax imports +# - Part of Branch Activity Monitoring System +# ============================================= + +""" +Monitoring handlers for ASSISTANT branch. + +Provides activity collection and memory health checking +for the Branch Activity Monitoring System (FPLAN-0266). +""" + +from .activity_collector import ( + load_branch_registry, + get_branch_paths, + scan_branch_activity, + get_all_branch_activity, +) + +from .memory_health import ( + check_memory_files_exist, + validate_memory_structure, + check_freshness, + get_memory_health_status, +) + +from .red_flag_detector import ( + get_branch_status, + get_red_flag_summary, +) + +__all__ = [ + # activity_collector + 'load_branch_registry', + 'get_branch_paths', + 'scan_branch_activity', + 'get_all_branch_activity', + # memory_health + 'check_memory_files_exist', + 'validate_memory_structure', + 'check_freshness', + 'get_memory_health_status', + # red_flag_detector + 'get_branch_status', + 'get_red_flag_summary', +] diff --git a/src/aipass/daemon/apps/handlers/monitoring/activity_collector.py b/src/aipass/daemon/apps/handlers/monitoring/activity_collector.py new file mode 100644 index 00000000..0f980214 --- /dev/null +++ b/src/aipass/daemon/apps/handlers/monitoring/activity_collector.py @@ -0,0 +1,318 @@ + +# ===================AIPASS==================== +# META DATA HEADER +# Name: activity_collector.py - Branch Activity Data Collector +# Date: 2026-01-30 +# Version: 0.1.0 +# Category: daemon/handlers/monitoring +# +# CHANGELOG (Max 5 entries): +# - v0.1.0 (2026-01-30): Initial implementation - FPLAN-0266 Phase 1 +# +# CODE STANDARDS: +# - Handler implements collection logic, module orchestrates +# - NO Prax logger in handlers (handlers don't log) +# - Uses pathlib for all path operations +# - Standard library only +# ============================================= + +""" +Branch Activity Data Collector Handler + +Collects activity data from all branches in the AIPass system. +Scans for code files (.py) and memory files (.local.json, .observations.json, README.md). +Provides file modification timestamps for activity tracking. +""" + +import os +import json +from pathlib import Path +from datetime import datetime, timedelta +from typing import Dict, List, Any, Optional + + +# Constants +REGISTRY_PATH = Path(os.environ.get('AIPASS_REGISTRY', Path.home() / '.aipass' / 'AIPASS_REGISTRY.json')) +MEMORY_FILE_PATTERNS = [".local.json", ".observations.json", "README.md"] +CODE_FILE_EXTENSION = ".py" + + +def load_branch_registry() -> Dict[str, Any]: + """ + Load the BRANCH_REGISTRY.json file. + + Returns: + Dict containing registry data with 'metadata' and 'branches' keys. + Returns empty dict with empty branches list on error. + """ + if not REGISTRY_PATH.exists(): + return {"metadata": {}, "branches": []} + + try: + with open(REGISTRY_PATH, 'r', encoding='utf-8') as f: + return json.load(f) + except (json.JSONDecodeError, OSError): + return {"metadata": {}, "branches": []} + + +def get_branch_paths() -> List[Dict[str, str]]: + """ + Get all branch names and paths from the registry. + + Returns: + List of dicts with 'name' and 'path' keys for each branch. + """ + registry = load_branch_registry() + branches = registry.get("branches", []) + + return [ + {"name": b.get("name", ""), "path": b.get("path", "")} + for b in branches + if b.get("name") and b.get("path") + ] + + +def _get_file_mtime(file_path: Path) -> Optional[datetime]: + """ + Get modification time of a file. + + Args: + file_path: Path to the file. + + Returns: + datetime of last modification, or None if file doesn't exist. + """ + try: + if file_path.exists(): + return datetime.fromtimestamp(file_path.stat().st_mtime) + except OSError: + pass + return None + + +def _is_memory_file(file_path: Path, branch_name: str) -> bool: + """ + Check if a file is a memory file for this branch. + + Memory files follow patterns: + - [BRANCH].local.json + - [BRANCH].observations.json + - README.md + - DASHBOARD.local.json + + Args: + file_path: Path to check. + branch_name: Name of the branch (uppercase). + + Returns: + True if file is a memory file. + """ + name = file_path.name + + # Check for branch-specific memory files + if name == f"{branch_name}.local.json": + return True + if name == f"{branch_name}.observations.json": + return True + if name == "README.md": + return True + if name == "DASHBOARD.local.json": + return True + + return False + + +def _scan_directory_files( + directory: Path, + branch_name: str, + since: Optional[datetime] = None, + max_depth: int = 5 +) -> Dict[str, List[Dict[str, Any]]]: + """ + Scan a directory for code and memory files. + + Args: + directory: Directory to scan. + branch_name: Name of the branch (uppercase). + since: Only include files modified since this time. + max_depth: Maximum directory depth to scan. + + Returns: + Dict with 'code_files' and 'memory_files' lists. + """ + code_files: List[Dict[str, Any]] = [] + memory_files: List[Dict[str, Any]] = [] + + if not directory.exists() or not directory.is_dir(): + return {"code_files": code_files, "memory_files": memory_files} + + def scan_recursive(path: Path, depth: int = 0) -> None: + if depth > max_depth: + return + + try: + for item in path.iterdir(): + # Skip hidden directories and __pycache__ + if item.is_dir(): + if item.name.startswith('.') or item.name == '__pycache__': + continue + scan_recursive(item, depth + 1) + elif item.is_file(): + mtime = _get_file_mtime(item) + if mtime is None: + continue + + # Apply time filter if specified + if since and mtime < since: + continue + + file_info = { + "path": str(item), + "name": item.name, + "mtime": mtime.isoformat(), + "mtime_datetime": mtime, + } + + # Categorize file + if _is_memory_file(item, branch_name): + memory_files.append(file_info) + elif item.suffix == CODE_FILE_EXTENSION: + code_files.append(file_info) + except PermissionError: + pass + except OSError: + pass + + scan_recursive(directory) + + return {"code_files": code_files, "memory_files": memory_files} + + +def scan_branch_activity( + branch_name: str, + branch_path: str, + since: Optional[datetime] = None +) -> Dict[str, Any]: + """ + Scan a single branch directory and collect file modification times. + + Args: + branch_name: Name of the branch (e.g., "DRONE"). + branch_path: Absolute path to the branch directory. + since: Only include files modified since this time. + Defaults to last 24 hours if not specified. + + Returns: + Dict with structure: + { + "branch_name": str, + "path": str, + "code_files": [{"path": str, "name": str, "mtime": str}], + "memory_files": [{"path": str, "name": str, "mtime": str}], + "last_activity": str (ISO format) or None, + "total_files": int, + "scan_time": str + } + """ + # Default to last 24 hours + if since is None: + since = datetime.now() - timedelta(hours=24) + + directory = Path(branch_path) + scan_result = _scan_directory_files(directory, branch_name, since) + + # Find the most recent activity + all_files = scan_result["code_files"] + scan_result["memory_files"] + last_activity = None + if all_files: + most_recent = max(all_files, key=lambda f: f["mtime_datetime"]) + last_activity = most_recent["mtime"] + + # Clean up internal datetime objects before returning + for f in scan_result["code_files"]: + del f["mtime_datetime"] + for f in scan_result["memory_files"]: + del f["mtime_datetime"] + + return { + "branch_name": branch_name, + "path": branch_path, + "code_files": scan_result["code_files"], + "memory_files": scan_result["memory_files"], + "last_activity": last_activity, + "total_files": len(all_files), + "scan_time": datetime.now().isoformat(), + } + + +def get_all_branch_activity( + since: Optional[datetime] = None +) -> Dict[str, Any]: + """ + Collect activity data for ALL branches in the system. + + Args: + since: Only include files modified since this time. + Defaults to last 24 hours if not specified. + + Returns: + Dict with structure: + { + "scan_time": str, + "time_window_hours": float, + "branches_scanned": int, + "branches_with_activity": int, + "total_files_modified": int, + "branches": { + "BRANCH_NAME": {branch activity dict} + } + } + """ + if since is None: + since = datetime.now() - timedelta(hours=24) + + time_window_hours = (datetime.now() - since).total_seconds() / 3600 + + branch_paths = get_branch_paths() + results: Dict[str, Any] = {} + total_files = 0 + active_count = 0 + + for branch_info in branch_paths: + name = branch_info["name"] + path = branch_info["path"] + + activity = scan_branch_activity(name, path, since) + results[name] = activity + + if activity["total_files"] > 0: + active_count += 1 + total_files += activity["total_files"] + + return { + "scan_time": datetime.now().isoformat(), + "time_window_hours": round(time_window_hours, 2), + "branches_scanned": len(branch_paths), + "branches_with_activity": active_count, + "total_files_modified": total_files, + "branches": results, + } + + +if __name__ == "__main__": + # Simple test + print("Testing activity_collector...") + print(f"Registry path: {REGISTRY_PATH}") + print(f"Registry exists: {REGISTRY_PATH.exists()}") + + branches = get_branch_paths() + print(f"Found {len(branches)} branches") + + if branches: + # Test scanning one branch + first = branches[0] + print(f"\nScanning {first['name']}...") + activity = scan_branch_activity(first['name'], first['path']) + print(f" Code files: {len(activity['code_files'])}") + print(f" Memory files: {len(activity['memory_files'])}") + print(f" Last activity: {activity['last_activity']}") diff --git a/src/aipass/daemon/apps/handlers/monitoring/memory_health.py b/src/aipass/daemon/apps/handlers/monitoring/memory_health.py new file mode 100644 index 00000000..09f653d2 --- /dev/null +++ b/src/aipass/daemon/apps/handlers/monitoring/memory_health.py @@ -0,0 +1,423 @@ + +# ===================AIPASS==================== +# META DATA HEADER +# Name: memory_health.py - Branch Memory Health Checker +# Date: 2026-01-30 +# Version: 0.1.0 +# Category: daemon/handlers/monitoring +# +# CHANGELOG (Max 5 entries): +# - v0.1.0 (2026-01-30): Initial implementation - FPLAN-0266 Phase 1 +# +# CODE STANDARDS: +# - Handler implements checking logic, module orchestrates +# - NO Prax logger in handlers (handlers don't log) +# - Uses pathlib for all path operations +# - Standard library only +# ============================================= + +""" +Branch Memory Health Checker Handler + +Validates memory file existence, structure, and freshness for branches. +Provides health status reporting for the Branch Activity Monitoring System. +""" + +import json +from pathlib import Path +from datetime import datetime +from typing import Dict, List, Any + + +# Health status constants +STATUS_OK = "OK" +STATUS_WARNING = "WARNING" +STATUS_RED = "RED" + +# Required memory files (branch cannot function properly without these) +REQUIRED_FILES = ["local.json", "README.md"] + +# Optional memory files (nice to have, warning if missing) +OPTIONAL_FILES = ["observations.json", "id.json"] + +# Freshness thresholds (in days) +FRESHNESS_WARNING_DAYS = 7 +FRESHNESS_RED_DAYS = 30 + + +def check_memory_files_exist( + branch_path: str, + branch_name: str +) -> Dict[str, Any]: + """ + Check if required memory files exist for a branch. + + Required files: + - [BRANCH].local.json + - README.md + + Optional files: + - [BRANCH].observations.json + - [BRANCH].id.json + - DASHBOARD.local.json + + Args: + branch_path: Absolute path to the branch directory. + branch_name: Name of the branch (uppercase, e.g., "DRONE"). + + Returns: + Dict with structure: + { + "required": {"filename": bool, ...}, + "optional": {"filename": bool, ...}, + "missing_required": [str], + "missing_optional": [str], + "all_required_present": bool + } + """ + directory = Path(branch_path) + + # Build expected file paths + required_checks = { + f"{branch_name}.local.json": directory / f"{branch_name}.local.json", + "README.md": directory / "README.md", + } + + optional_checks = { + f"{branch_name}.observations.json": directory / f"{branch_name}.observations.json", + f"{branch_name}.id.json": directory / f"{branch_name}.id.json", + "DASHBOARD.local.json": directory / "DASHBOARD.local.json", + } + + # Check required files + required_results = {} + missing_required = [] + for name, path in required_checks.items(): + exists = path.exists() and path.is_file() + required_results[name] = exists + if not exists: + missing_required.append(name) + + # Check optional files + optional_results = {} + missing_optional = [] + for name, path in optional_checks.items(): + exists = path.exists() and path.is_file() + optional_results[name] = exists + if not exists: + missing_optional.append(name) + + return { + "required": required_results, + "optional": optional_results, + "missing_required": missing_required, + "missing_optional": missing_optional, + "all_required_present": len(missing_required) == 0, + } + + +def validate_memory_structure( + file_path: str +) -> Dict[str, Any]: + """ + Validate memory file structure (check metadata exists, check limits field). + + Validates that .local.json and .observations.json files have proper structure: + - document_metadata section exists + - limits field exists within metadata + - Basic required fields present + + Args: + file_path: Absolute path to the memory file. + + Returns: + Dict with structure: + { + "valid": bool, + "has_metadata": bool, + "has_limits": bool, + "issues": [str], + "metadata_fields": [str] (if metadata exists) + } + """ + path = Path(file_path) + + if not path.exists(): + return { + "valid": False, + "has_metadata": False, + "has_limits": False, + "issues": ["File does not exist"], + "metadata_fields": [], + } + + try: + with open(path, 'r', encoding='utf-8') as f: + data = json.load(f) + except json.JSONDecodeError as e: + return { + "valid": False, + "has_metadata": False, + "has_limits": False, + "issues": [f"Invalid JSON: {str(e)}"], + "metadata_fields": [], + } + except OSError as e: + return { + "valid": False, + "has_metadata": False, + "has_limits": False, + "issues": [f"Cannot read file: {str(e)}"], + "metadata_fields": [], + } + + issues = [] + + # Check for metadata section + # Memory files use either "document_metadata" or "metadata" + has_metadata = False + metadata_fields: List[str] = [] + metadata_section = None + + if "document_metadata" in data: + has_metadata = True + metadata_section = data["document_metadata"] + elif "metadata" in data: + has_metadata = True + metadata_section = data["metadata"] + + if not has_metadata: + issues.append("No metadata section found (expected 'document_metadata' or 'metadata')") + + # Check limits field + has_limits = False + if metadata_section and isinstance(metadata_section, dict): + metadata_fields = list(metadata_section.keys()) + if "limits" in metadata_section: + has_limits = True + else: + issues.append("No 'limits' field in metadata") + + # Overall validity + valid = has_metadata and len(issues) == 0 + + return { + "valid": valid, + "has_metadata": has_metadata, + "has_limits": has_limits, + "issues": issues, + "metadata_fields": metadata_fields, + } + + +def check_freshness( + file_path: str, + warning_days: int = FRESHNESS_WARNING_DAYS, + red_days: int = FRESHNESS_RED_DAYS +) -> Dict[str, Any]: + """ + Check when a file was last modified and determine freshness status. + + Args: + file_path: Absolute path to the file. + warning_days: Days after which status becomes WARNING. + red_days: Days after which status becomes RED. + + Returns: + Dict with structure: + { + "exists": bool, + "last_modified": str (ISO format) or None, + "days_ago": float or None, + "status": "OK" | "WARNING" | "RED", + "message": str + } + """ + path = Path(file_path) + + if not path.exists(): + return { + "exists": False, + "last_modified": None, + "days_ago": None, + "status": STATUS_RED, + "message": "File does not exist", + } + + try: + mtime = datetime.fromtimestamp(path.stat().st_mtime) + days_ago = (datetime.now() - mtime).total_seconds() / 86400 + + if days_ago > red_days: + status = STATUS_RED + message = f"Not modified in {int(days_ago)} days (threshold: {red_days})" + elif days_ago > warning_days: + status = STATUS_WARNING + message = f"Not modified in {int(days_ago)} days (threshold: {warning_days})" + else: + status = STATUS_OK + message = f"Modified {days_ago:.1f} days ago" + + return { + "exists": True, + "last_modified": mtime.isoformat(), + "days_ago": round(days_ago, 2), + "status": status, + "message": message, + } + except OSError as e: + return { + "exists": True, + "last_modified": None, + "days_ago": None, + "status": STATUS_RED, + "message": f"Cannot read file stats: {str(e)}", + } + + +def get_memory_health_status( + branch_path: str, + branch_name: str +) -> Dict[str, Any]: + """ + Get comprehensive memory health status for a branch. + + Combines file existence, structure validation, and freshness checks + into an overall health assessment. + + Health Status Levels: + - OK: All required files present, valid structure, recent activity + - WARNING: Missing optional files or stale (7+ days) + - RED: Missing required files or very stale (30+ days) + + Args: + branch_path: Absolute path to the branch directory. + branch_name: Name of the branch (uppercase, e.g., "DRONE"). + + Returns: + Dict with structure: + { + "branch_name": str, + "branch_path": str, + "overall_status": "OK" | "WARNING" | "RED", + "file_check": {file existence results}, + "structure_checks": {filename: validation result}, + "freshness_checks": {filename: freshness result}, + "issues": [str], + "check_time": str + } + """ + directory = Path(branch_path) + issues: List[str] = [] + + # Step 1: Check file existence + file_check = check_memory_files_exist(branch_path, branch_name) + + if not file_check["all_required_present"]: + for missing in file_check["missing_required"]: + issues.append(f"Missing required file: {missing}") + + for missing in file_check["missing_optional"]: + issues.append(f"Missing optional file: {missing}") + + # Step 2: Validate structure of existing memory files + structure_checks = {} + local_file = directory / f"{branch_name}.local.json" + obs_file = directory / f"{branch_name}.observations.json" + + if local_file.exists(): + local_validation = validate_memory_structure(str(local_file)) + structure_checks[f"{branch_name}.local.json"] = local_validation + if not local_validation["valid"]: + for issue in local_validation["issues"]: + issues.append(f"{branch_name}.local.json: {issue}") + + if obs_file.exists(): + obs_validation = validate_memory_structure(str(obs_file)) + structure_checks[f"{branch_name}.observations.json"] = obs_validation + if not obs_validation["valid"]: + for issue in obs_validation["issues"]: + issues.append(f"{branch_name}.observations.json: {issue}") + + # Step 3: Check freshness + freshness_checks = {} + files_to_check = [ + (f"{branch_name}.local.json", local_file), + ("README.md", directory / "README.md"), + ] + + worst_freshness = STATUS_OK + for name, path in files_to_check: + if path.exists(): + freshness = check_freshness(str(path)) + freshness_checks[name] = freshness + + # Track worst freshness status + if freshness["status"] == STATUS_RED: + worst_freshness = STATUS_RED + elif freshness["status"] == STATUS_WARNING and worst_freshness != STATUS_RED: + worst_freshness = STATUS_WARNING + + # Step 4: Determine overall status + if not file_check["all_required_present"]: + overall_status = STATUS_RED + elif worst_freshness == STATUS_RED: + overall_status = STATUS_RED + elif file_check["missing_optional"] or worst_freshness == STATUS_WARNING: + overall_status = STATUS_WARNING + else: + overall_status = STATUS_OK + + # Filter out structure check issues from issues list for WARNING-only items + has_structure_issues = any( + not check.get("valid", True) + for check in structure_checks.values() + ) + if has_structure_issues and overall_status == STATUS_OK: + overall_status = STATUS_WARNING + + return { + "branch_name": branch_name, + "branch_path": branch_path, + "overall_status": overall_status, + "file_check": file_check, + "structure_checks": structure_checks, + "freshness_checks": freshness_checks, + "issues": issues, + "check_time": datetime.now().isoformat(), + } + + +if __name__ == "__main__": + # Simple test + print("Testing memory_health...") + + # Test with a sample branch path + test_path = "." + test_name = "EXAMPLE" + + print(f"\nChecking {test_name} at {test_path}") + + # Test file existence + existence = check_memory_files_exist(test_path, test_name) + print(f" Required files present: {existence['all_required_present']}") + print(f" Missing required: {existence['missing_required']}") + print(f" Missing optional: {existence['missing_optional']}") + + # Test structure validation + local_path = f"{test_path}/{test_name}.local.json" + structure = validate_memory_structure(local_path) + print(f" Structure valid: {structure['valid']}") + print(f" Has metadata: {structure['has_metadata']}") + print(f" Has limits: {structure['has_limits']}") + + # Test freshness + freshness = check_freshness(local_path) + print(f" Freshness: {freshness['status']} ({freshness['message']})") + + # Test overall health + health = get_memory_health_status(test_path, test_name) + print(f"\n Overall status: {health['overall_status']}") + print(f" Issues: {len(health['issues'])}") + for issue in health['issues'][:5]: + print(f" - {issue}") diff --git a/src/aipass/daemon/apps/handlers/monitoring/red_flag_detector.py b/src/aipass/daemon/apps/handlers/monitoring/red_flag_detector.py new file mode 100644 index 00000000..00239eb5 --- /dev/null +++ b/src/aipass/daemon/apps/handlers/monitoring/red_flag_detector.py @@ -0,0 +1,374 @@ + +# ===================AIPASS==================== +# META DATA HEADER +# Name: red_flag_detector.py - Branch Red Flag Detection Engine +# Date: 2026-01-30 +# Version: 0.1.0 +# Category: daemon/handlers/monitoring +# +# CHANGELOG (Max 5 entries): +# - v0.1.0 (2026-01-30): Initial implementation - FPLAN-0266 Phase 2 +# +# CODE STANDARDS: +# - Handler implements detection logic, module orchestrates +# - NO Prax logger in handlers (handlers don't log) +# - Uses pathlib for all path operations +# - Standard library only +# ============================================= + +""" +Branch Red Flag Detection Engine + +Detects presence violations: branches that modified code but did NOT update their memory files. +A RED FLAG indicates a branch did work but didn't maintain their presence (memory files). + +Detection Logic: +- If code files were modified within the time window +- AND memory files were NOT modified (or modified BEFORE the code changes) +- This is a RED FLAG violation + +OK Conditions: +- No code changes in time window (nothing to update) +- Memory updated MORE RECENTLY than code (presence maintained) +- Memory updated WITHIN threshold_hours of code changes +""" + +from datetime import datetime, timedelta +from typing import Dict, List, Any, Optional + +from . import activity_collector + + +# Status constants +STATUS_RED_FLAG = "RED_FLAG" +STATUS_OK = "OK" +STATUS_NO_ACTIVITY = "NO_ACTIVITY" +STATUS_ERROR = "ERROR" + + +def _parse_iso_datetime(iso_string: str) -> Optional[datetime]: + """ + Parse an ISO format datetime string to datetime object. + + Args: + iso_string: ISO format datetime string (e.g., "2026-01-30T10:30:00.123456") + + Returns: + datetime object or None if parsing fails. + """ + if not iso_string: + return None + try: + # Handle both with and without microseconds + if '.' in iso_string: + return datetime.fromisoformat(iso_string) + else: + return datetime.fromisoformat(iso_string) + except (ValueError, TypeError): + return None + + +def _get_most_recent_mtime(files: List[Dict[str, Any]]) -> Optional[datetime]: + """ + Get the most recent modification time from a list of file entries. + + Args: + files: List of file dicts with 'mtime' key (ISO format string). + + Returns: + Most recent datetime or None if list is empty or no valid times. + """ + if not files: + return None + + mtimes = [] + for f in files: + mtime_str = f.get("mtime") + if mtime_str: + dt = _parse_iso_datetime(mtime_str) + if dt: + mtimes.append(dt) + + return max(mtimes) if mtimes else None + + +def get_branch_status( + branch_name: str, + branch_path: str, + since_timestamp: Optional[datetime] = None, + threshold_hours: float = 2.0 +) -> Dict[str, Any]: + """ + Check a single branch for red flag violations. + + A RED FLAG occurs when: + - Code files were modified since the timestamp + - Memory files were NOT modified, OR were modified BEFORE the latest code change + (with a grace period of threshold_hours) + + Args: + branch_name: Name of the branch (uppercase, e.g., "DRONE"). + branch_path: Absolute path to the branch directory. + since_timestamp: Only consider changes since this time. + Defaults to last 24 hours. + threshold_hours: Grace period in hours. Memory must be updated within + this many hours AFTER the latest code change. + + Returns: + Dict with structure: + { + "branch_name": str, + "branch_path": str, + "status": "RED_FLAG" | "OK" | "NO_ACTIVITY" | "ERROR", + "code_changes": [{"file": str, "mtime": str}], + "code_change_count": int, + "latest_code_change": str (ISO) or None, + "memory_files_modified": [{"file": str, "mtime": str}], + "memory_last_update": str (ISO) or None, + "hours_since_code": float or None, + "threshold_hours": float, + "reason": str, + "check_time": str + } + """ + # Default time window: last 24 hours + if since_timestamp is None: + since_timestamp = datetime.now() - timedelta(hours=24) + + result = { + "branch_name": branch_name, + "branch_path": branch_path, + "status": STATUS_OK, + "code_changes": [], + "code_change_count": 0, + "latest_code_change": None, + "memory_files_modified": [], + "memory_last_update": None, + "hours_since_code": None, + "threshold_hours": threshold_hours, + "reason": "", + "check_time": datetime.now().isoformat(), + } + + try: + # Get activity data from the collector + activity = activity_collector.scan_branch_activity( + branch_name, branch_path, since_timestamp + ) + except Exception as e: + result["status"] = STATUS_ERROR + result["reason"] = f"Failed to scan branch: {str(e)}" + return result + + # Extract code file changes + code_files = activity.get("code_files", []) + memory_files = activity.get("memory_files", []) + + # Format code changes for output + result["code_changes"] = [ + {"file": f.get("name", ""), "mtime": f.get("mtime", "")} + for f in code_files + ] + result["code_change_count"] = len(code_files) + + # Format memory file modifications for output + result["memory_files_modified"] = [ + {"file": f.get("name", ""), "mtime": f.get("mtime", "")} + for f in memory_files + ] + + # Get most recent times + latest_code = _get_most_recent_mtime(code_files) + latest_memory = _get_most_recent_mtime(memory_files) + + if latest_code: + result["latest_code_change"] = latest_code.isoformat() + if latest_memory: + result["memory_last_update"] = latest_memory.isoformat() + + # Decision logic + # Case 1: No code changes - OK (nothing to update) + if not code_files: + result["status"] = STATUS_NO_ACTIVITY + result["reason"] = "No code changes in time window" + return result + + # Case 2: Code changed, check if memory was updated appropriately + if latest_code: + if latest_memory: + # Calculate how long after code the memory was updated + time_diff = latest_memory - latest_code + hours_diff = time_diff.total_seconds() / 3600 + + if latest_memory >= latest_code: + # Memory was updated at same time or after code - OK + result["status"] = STATUS_OK + result["hours_since_code"] = round(hours_diff, 2) + result["reason"] = f"Memory updated {abs(hours_diff):.1f}h after code changes" + elif abs(hours_diff) <= threshold_hours: + # Memory was updated slightly before, but within threshold - OK + # This covers cases where memory was updated just before final code commit + result["status"] = STATUS_OK + result["hours_since_code"] = round(hours_diff, 2) + result["reason"] = f"Memory update within threshold ({abs(hours_diff):.1f}h before code)" + else: + # Memory was updated too long before code changes - RED FLAG + result["status"] = STATUS_RED_FLAG + hours_since = (datetime.now() - latest_code).total_seconds() / 3600 + result["hours_since_code"] = round(hours_since, 2) + result["reason"] = f"Code changed but memory last updated {abs(hours_diff):.1f}h BEFORE code" + else: + # No memory files modified in time window - RED FLAG + result["status"] = STATUS_RED_FLAG + hours_since = (datetime.now() - latest_code).total_seconds() / 3600 + result["hours_since_code"] = round(hours_since, 2) + result["reason"] = f"Code changed {hours_since:.1f}h ago but no memory updates in time window" + + return result + + +def detect_red_flags( + since_timestamp: Optional[datetime] = None, + threshold_hours: float = 2.0 +) -> List[Dict[str, Any]]: + """ + Detect red flag violations across ALL branches. + + Scans all branches in the system and identifies those where code was modified + but memory files were not updated appropriately. + + Args: + since_timestamp: Only consider changes since this time. + Defaults to last 24 hours. + threshold_hours: Grace period in hours for memory updates after code changes. + + Returns: + List of branch status dicts (same structure as get_branch_status), + sorted with RED_FLAG violations first, then by branch name. + """ + # Default time window: last 24 hours + if since_timestamp is None: + since_timestamp = datetime.now() - timedelta(hours=24) + + # Get all branch paths + branches = activity_collector.get_branch_paths() + results: List[Dict[str, Any]] = [] + + for branch_info in branches: + name = branch_info.get("name", "") + path = branch_info.get("path", "") + + if not name or not path: + continue + + status = get_branch_status(name, path, since_timestamp, threshold_hours) + results.append(status) + + # Sort: RED_FLAG first, then by branch name + def sort_key(item: Dict[str, Any]) -> tuple: + status_order = { + STATUS_RED_FLAG: 0, + STATUS_ERROR: 1, + STATUS_OK: 2, + STATUS_NO_ACTIVITY: 3, + } + return (status_order.get(item.get("status", ""), 99), item.get("branch_name", "")) + + results.sort(key=sort_key) + + return results + + +def get_red_flag_summary( + since_timestamp: Optional[datetime] = None, + threshold_hours: float = 2.0 +) -> Dict[str, Any]: + """ + Get a summary of red flag detection across all branches. + + Args: + since_timestamp: Only consider changes since this time. + threshold_hours: Grace period in hours for memory updates. + + Returns: + Dict with structure: + { + "scan_time": str, + "time_window_hours": float, + "threshold_hours": float, + "total_branches": int, + "red_flags": int, + "ok": int, + "no_activity": int, + "errors": int, + "violations": [{branch status for RED_FLAG only}], + "all_branches": [{all branch statuses}] + } + """ + if since_timestamp is None: + since_timestamp = datetime.now() - timedelta(hours=24) + + time_window = (datetime.now() - since_timestamp).total_seconds() / 3600 + + all_results = detect_red_flags(since_timestamp, threshold_hours) + + # Count by status + counts = { + STATUS_RED_FLAG: 0, + STATUS_OK: 0, + STATUS_NO_ACTIVITY: 0, + STATUS_ERROR: 0, + } + + violations = [] + for result in all_results: + status = result.get("status", "") + if status in counts: + counts[status] += 1 + if status == STATUS_RED_FLAG: + violations.append(result) + + return { + "scan_time": datetime.now().isoformat(), + "time_window_hours": round(time_window, 2), + "threshold_hours": threshold_hours, + "total_branches": len(all_results), + "red_flags": counts[STATUS_RED_FLAG], + "ok": counts[STATUS_OK], + "no_activity": counts[STATUS_NO_ACTIVITY], + "errors": counts[STATUS_ERROR], + "violations": violations, + "all_branches": all_results, + } + + +if __name__ == "__main__": + # Simple test + print("Testing red_flag_detector...") + print("=" * 60) + + # Get summary + summary = get_red_flag_summary() + + print(f"Scan time: {summary['scan_time']}") + print(f"Time window: {summary['time_window_hours']} hours") + print(f"Threshold: {summary['threshold_hours']} hours") + print(f"\nTotal branches: {summary['total_branches']}") + print(f" RED FLAGS: {summary['red_flags']}") + print(f" OK: {summary['ok']}") + print(f" No activity: {summary['no_activity']}") + print(f" Errors: {summary['errors']}") + + if summary['violations']: + print(f"\n{'='*60}") + print("RED FLAG VIOLATIONS:") + print("=" * 60) + for v in summary['violations']: + print(f"\n Branch: {v['branch_name']}") + print(f" Status: {v['status']}") + print(f" Reason: {v['reason']}") + print(f" Code changes: {v['code_change_count']}") + print(f" Latest code change: {v['latest_code_change']}") + print(f" Memory last update: {v['memory_last_update']}") + else: + print("\nNo red flag violations detected.") diff --git a/src/aipass/daemon/apps/handlers/schedule/__init__.py b/src/aipass/daemon/apps/handlers/schedule/__init__.py new file mode 100644 index 00000000..288bed80 --- /dev/null +++ b/src/aipass/daemon/apps/handlers/schedule/__init__.py @@ -0,0 +1,38 @@ +# ===================AIPASS==================== +# META DATA HEADER +# Name: __init__.py - Schedule Handlers Package +# Date: 2026-02-04 +# Version: 1.0.0 +# Category: assistant/handlers/schedule +# +# CHANGELOG (Max 5 entries): +# - v1.0.0 (2026-02-04): Initial package setup +# +# CODE STANDARDS: +# - Handlers implement logic, modules orchestrate +# - No cross-branch imports, no Prax logger +# ============================================= + +""" +Schedule handlers for ASSISTANT's scheduled follow-ups system. +""" + +from .task_registry import ( + load_tasks, + save_tasks, + create_task, + delete_task, + get_due_tasks, + mark_completed, + parse_due_date +) + +__all__ = [ + 'load_tasks', + 'save_tasks', + 'create_task', + 'delete_task', + 'get_due_tasks', + 'mark_completed', + 'parse_due_date' +] diff --git a/src/aipass/daemon/apps/handlers/schedule/assistant_notifier.py b/src/aipass/daemon/apps/handlers/schedule/assistant_notifier.py new file mode 100644 index 00000000..6a2a7d82 --- /dev/null +++ b/src/aipass/daemon/apps/handlers/schedule/assistant_notifier.py @@ -0,0 +1,170 @@ + +# ===================AIPASS==================== +# META DATA HEADER +# Name: assistant_notifier.py - Daemon Bot Telegram Notifications +# Date: 2026-02-15 +# Version: 1.0.0 +# Category: daemon/handlers/schedule +# +# CHANGELOG (Max 5 entries): +# - v1.0.0 (2026-02-15): Initial implementation - daemon bot notifications +# +# CODE STANDARDS: +# - Handlers implement logic, modules orchestrate +# - No cross-branch imports, no Prax logger +# - stdlib only (urllib, json) - runs under /usr/bin/python3 +# ============================================= + +""" +Handler for sending Telegram notifications via the daemon bot. + +Patrick's direct line to daemon. Uses the dedicated daemon bot +(separate from scheduler and bridge bots) to notify Patrick of +wake-up events, email reports, and errors. +""" + +import os +import json +from datetime import datetime +from pathlib import Path +from typing import Dict +from urllib.request import Request, urlopen +from urllib.error import URLError + +# ============================================= +# CONSTANTS +# ============================================= + +CONFIG_PATH = Path(os.environ.get('AIPASS_DAEMON_CONFIG', Path.home() / '.aipass' / 'daemon_config.json')) + + +# ============================================= +# CONFIG +# ============================================= + +def load_config() -> Dict[str, str]: + """ + Load daemon bot config from daemon_config.json. + + Returns: + Dict with 'bot_token' and 'chat_id' keys + + Raises: + FileNotFoundError: If config file is missing + KeyError: If required keys are absent + """ + with open(CONFIG_PATH, "r", encoding="utf-8") as f: + raw = json.load(f) + + return { + "bot_token": raw["telegram_bot_token"], + "chat_id": raw["telegram_chat_id"], + } + + +# ============================================= +# SEND +# ============================================= + +def send_notification(message: str) -> bool: + """ + Send a message to Patrick via the daemon bot. + + Args: + message: Text to send (plain text, supports emoji) + + Returns: + True if sent successfully, False otherwise + """ + try: + config = load_config() + except (FileNotFoundError, KeyError, json.JSONDecodeError) as e: + print(f"[daemon_notifier] Config error: {e}") + return False + + url = f"https://api.telegram.org/bot{config['bot_token']}/sendMessage" + payload = { + "chat_id": config["chat_id"], + "text": message, + } + + data = json.dumps(payload).encode("utf-8") + req = Request(url, data=data, headers={"Content-Type": "application/json"}) + + try: + with urlopen(req, timeout=15) as resp: + result = json.loads(resp.read()) + if result.get("ok"): + return True + print(f"[daemon_notifier] API error: {result.get('description')}") + return False + except URLError as e: + print(f"[daemon_notifier] Send failed: {e}") + return False + except Exception as e: + print(f"[daemon_notifier] Unexpected error: {e}") + return False + + +# ============================================= +# NOTIFICATION HELPERS +# ============================================= + +def notify_wakeup() -> bool: + """ + Notify that daemon is waking up. + + Returns: + True if notification sent, False otherwise + """ + now = datetime.now().strftime("%H:%M:%S") + message = "\U0001f916 Daemon waking up at " + now + return send_notification(message) + + +def notify_report(summary: str) -> bool: + """ + Send daemon's wake-up report. + + Args: + summary: Report content (email counts, listings, etc.) + + Returns: + True if notification sent, False otherwise + """ + message = "\U0001f4cb Daemon Report:\n" + summary + return send_notification(message) + + +def notify_error(error: str) -> bool: + """ + Notify that an error occurred during wake-up. + + Args: + error: Error description + + Returns: + True if notification sent, False otherwise + """ + message = "\u274c Daemon Error:\n" + error + return send_notification(message) + + +# ============================================= +# MAIN - Testing +# ============================================= + +if __name__ == "__main__": + print("assistant_notifier.py - manual test") + print(f"Config path: {CONFIG_PATH}") + + try: + cfg = load_config() + print(f"Bot token: {cfg['bot_token'][:12]}...") + print(f"Chat ID: {cfg['chat_id']}") + except Exception as e: + print(f"Config load failed: {e}") + + print("\nSending test notification...") + ok = send_notification("Test from assistant_notifier.py handler") + print(f"Result: {'OK' if ok else 'FAILED'}") diff --git a/src/aipass/daemon/apps/handlers/schedule/task_registry.py b/src/aipass/daemon/apps/handlers/schedule/task_registry.py new file mode 100644 index 00000000..5a488c13 --- /dev/null +++ b/src/aipass/daemon/apps/handlers/schedule/task_registry.py @@ -0,0 +1,449 @@ + +# ===================AIPASS==================== +# META DATA HEADER +# Name: task_registry.py - DAEMON Scheduled Tasks Registry +# Date: 2026-02-04 +# Version: 1.0.0 +# Category: daemon/handlers/schedule +# +# CHANGELOG (Max 5 entries): +# - v1.0.0 (2026-02-04): Initial implementation - task storage and operations +# +# CODE STANDARDS: +# - Handlers implement logic, modules orchestrate +# - No cross-branch imports, no Prax logger +# ============================================= + +""" +Handler for scheduled task storage and operations. + +Fire-and-forget follow-up system for DAEMON. +Tasks are stored in daemon_json/schedule.json and processed +when their due date arrives. +""" + +import json +import uuid +from pathlib import Path +from datetime import datetime, timedelta +from typing import Dict, List, Any, Optional +import re + +# ============================================= +# CONSTANTS +# ============================================= + +_DAEMON_ROOT = Path(__file__).resolve().parents[4] # src/aipass/daemon/ +SCHEDULE_JSON_PATH = _DAEMON_ROOT / "daemon_json" / "schedule.json" + +DEFAULT_SCHEDULE_DATA: Dict[str, Any] = { + "tasks": [] +} + +# ============================================= +# JSON FILE OPERATIONS +# ============================================= + +def _ensure_json_exists() -> None: + """Ensure schedule.json exists, create with defaults if missing.""" + SCHEDULE_JSON_PATH.parent.mkdir(parents=True, exist_ok=True) + + if not SCHEDULE_JSON_PATH.exists(): + with open(SCHEDULE_JSON_PATH, 'w', encoding='utf-8') as f: + json.dump(DEFAULT_SCHEDULE_DATA, f, indent=2, ensure_ascii=False) + + +def load_tasks() -> List[Dict[str, Any]]: + """ + Load all tasks from schedule.json. + + Returns: + List of task dictionaries + """ + _ensure_json_exists() + + try: + with open(SCHEDULE_JSON_PATH, 'r', encoding='utf-8') as f: + data = json.load(f) + return data.get("tasks", []) + except (json.JSONDecodeError, IOError): + return [] + + +def save_tasks(tasks: List[Dict[str, Any]]) -> bool: + """ + Save tasks to schedule.json. + + Args: + tasks: List of task dictionaries to save + + Returns: + True if successful, False otherwise + """ + _ensure_json_exists() + + try: + data = {"tasks": tasks} + with open(SCHEDULE_JSON_PATH, 'w', encoding='utf-8') as f: + json.dump(data, f, indent=2, ensure_ascii=False) + return True + except IOError: + return False + + +# ============================================= +# DATE PARSING +# ============================================= + +def parse_due_date(date_str: str) -> str: + """ + Parse various date formats to ISO 8601 date string. + + Supports: + - "7d" -> 7 days from now + - "1w" -> 1 week from now + - "2w" -> 2 weeks from now + - "2026-02-11" -> exact date (ISO 8601) + + Args: + date_str: Date string in supported format + + Returns: + ISO 8601 date string (YYYY-MM-DD) + + Raises: + ValueError: If date format is invalid + """ + date_str = date_str.strip() + today = datetime.now().date() + + # Check for relative day format: "7d", "14d", etc. + day_match = re.match(r'^(\d+)d$', date_str, re.IGNORECASE) + if day_match: + days = int(day_match.group(1)) + future_date = today + timedelta(days=days) + return future_date.isoformat() + + # Check for relative week format: "1w", "2w", etc. + week_match = re.match(r'^(\d+)w$', date_str, re.IGNORECASE) + if week_match: + weeks = int(week_match.group(1)) + future_date = today + timedelta(weeks=weeks) + return future_date.isoformat() + + # Check for ISO 8601 date format: "2026-02-11" + iso_match = re.match(r'^(\d{4})-(\d{2})-(\d{2})$', date_str) + if iso_match: + try: + # Validate it's a real date + year = int(iso_match.group(1)) + month = int(iso_match.group(2)) + day = int(iso_match.group(3)) + parsed_date = datetime(year, month, day).date() + return parsed_date.isoformat() + except ValueError as e: + raise ValueError(f"Invalid date: {date_str}") from e + + raise ValueError( + f"Invalid date format: '{date_str}'. " + "Use '7d' (days), '1w' (weeks), or 'YYYY-MM-DD' (ISO date)" + ) + + +# ============================================= +# TASK OPERATIONS +# ============================================= + +def _generate_task_id() -> str: + """Generate 8-character UUID for task ID.""" + return uuid.uuid4().hex[:8] + + +def create_task( + task: str, + due_date: str, + recipient: str, + message: str +) -> Dict[str, Any]: + """ + Create a new scheduled task. + + Args: + task: Brief description of the task/follow-up + due_date: When to trigger (supports "7d", "1w", "YYYY-MM-DD") + recipient: Target branch (e.g., "@dev_central") + message: Message to deliver when due + + Returns: + Created task dictionary + + Raises: + ValueError: If due_date format is invalid + """ + parsed_due = parse_due_date(due_date) + + new_task: Dict[str, Any] = { + "id": _generate_task_id(), + "created": datetime.now().date().isoformat(), + "due_date": parsed_due, + "task": task, + "recipient": recipient, + "message": message, + "status": "pending" + } + + tasks = load_tasks() + tasks.append(new_task) + save_tasks(tasks) + + return new_task + + +def delete_task(task_id: str) -> bool: + """ + Delete a task by ID. + + Args: + task_id: 8-character task ID + + Returns: + True if task was found and deleted, False otherwise + """ + tasks = load_tasks() + original_count = len(tasks) + + tasks = [t for t in tasks if t.get("id") != task_id] + + if len(tasks) < original_count: + save_tasks(tasks) + return True + + return False + + +def get_due_tasks() -> List[Dict[str, Any]]: + """ + Get all tasks that are due (due_date <= today). + + Only returns tasks with status 'pending' - excludes 'dispatching' and 'completed'. + + Returns: + List of tasks that are due for processing + """ + tasks = load_tasks() + today = datetime.now().date().isoformat() + + due_tasks = [ + t for t in tasks + if t.get("status") == "pending" and t.get("due_date", "") <= today + ] + + return due_tasks + + +def mark_dispatching(task_id: str) -> bool: + """ + Mark a task as currently being dispatched. + + Prevents re-dispatch while email is being sent. + + Args: + task_id: 8-character task ID + + Returns: + True if task was found and marked, False otherwise + """ + tasks = load_tasks() + + for task in tasks: + if task.get("id") == task_id: + task["status"] = "dispatching" + task["dispatch_started"] = datetime.now().isoformat() + save_tasks(tasks) + return True + + return False + + +def mark_pending(task_id: str) -> bool: + """ + Reset a task to pending status (for retry after failed dispatch). + + Args: + task_id: 8-character task ID + + Returns: + True if task was found and reset, False otherwise + """ + tasks = load_tasks() + + for task in tasks: + if task.get("id") == task_id: + task["status"] = "pending" + task.pop("dispatch_started", None) + save_tasks(tasks) + return True + + return False + + +def recover_stale_dispatches(max_age_minutes: int = 5) -> int: + """ + Reset tasks stuck in 'dispatching' status for too long. + + Called before processing to recover from crashed dispatches. + + Args: + max_age_minutes: Maximum time a task can be in dispatching status + + Returns: + Number of tasks recovered + """ + tasks = load_tasks() + recovered = 0 + cutoff = datetime.now() - timedelta(minutes=max_age_minutes) + + for task in tasks: + if task.get("status") == "dispatching": + started = task.get("dispatch_started") + if started: + try: + start_time = datetime.fromisoformat(started) + if start_time < cutoff: + task["status"] = "pending" + task.pop("dispatch_started", None) + recovered += 1 + except ValueError: + # Invalid timestamp, reset anyway + task["status"] = "pending" + task.pop("dispatch_started", None) + recovered += 1 + + if recovered: + save_tasks(tasks) + + return recovered + + +def mark_completed(task_id: str) -> bool: + """ + Mark a task as completed. + + Args: + task_id: 8-character task ID + + Returns: + True if task was found and marked, False otherwise + """ + tasks = load_tasks() + + for task in tasks: + if task.get("id") == task_id: + task["status"] = "completed" + task["completed_date"] = datetime.now().date().isoformat() + save_tasks(tasks) + return True + + return False + + +def get_task_by_id(task_id: str) -> Optional[Dict[str, Any]]: + """ + Get a single task by ID. + + Args: + task_id: 8-character task ID + + Returns: + Task dictionary if found, None otherwise + """ + tasks = load_tasks() + + for task in tasks: + if task.get("id") == task_id: + return task + + return None + + +def get_pending_tasks() -> List[Dict[str, Any]]: + """ + Get all pending tasks (not yet due or completed). + + Returns: + List of pending tasks + """ + tasks = load_tasks() + return [t for t in tasks if t.get("status") == "pending"] + + +# ============================================= +# MAIN - Testing +# ============================================= + +if __name__ == "__main__": + from rich.console import Console + from rich.panel import Panel + from rich.table import Table + + console = Console() + + console.print() + console.print(Panel.fit( + "[bold cyan]TASK REGISTRY - Handler Test[/bold cyan]", + border_style="bright_blue" + )) + console.print() + + # Test date parsing + console.print("[yellow]Testing date parsing:[/yellow]") + test_dates = ["7d", "1w", "2w", "2026-03-15"] + for d in test_dates: + try: + result = parse_due_date(d) + console.print(f" {d} -> {result}") + except ValueError as e: + console.print(f" {d} -> [red]ERROR: {e}[/red]") + + # Test invalid date + try: + parse_due_date("invalid") + except ValueError as e: + console.print(f" invalid -> [green]Correctly raised: {e}[/green]") + + console.print() + console.print("[yellow]Testing task creation:[/yellow]") + + # Create a test task + test_task = create_task( + task="Test backup health check", + due_date="7d", + recipient="@dev_central", + message="Please verify backup systems are healthy" + ) + console.print(f" Created task: {test_task['id']}") + console.print(f" Due: {test_task['due_date']}") + + # Show all tasks + console.print() + console.print("[yellow]Current tasks:[/yellow]") + all_tasks = load_tasks() + + table = Table(show_header=True) + table.add_column("ID", style="cyan") + table.add_column("Task", style="white") + table.add_column("Due", style="yellow") + table.add_column("Status", style="green") + + for t in all_tasks: + table.add_row( + t.get("id", "?"), + t.get("task", "?")[:30], + t.get("due_date", "?"), + t.get("status", "?") + ) + + console.print(table) + console.print() + console.print(f"[dim]Schedule file: {SCHEDULE_JSON_PATH}[/dim]") + console.print() diff --git a/src/aipass/daemon/apps/handlers/schedule/telegram_notifier.py b/src/aipass/daemon/apps/handlers/schedule/telegram_notifier.py new file mode 100644 index 00000000..7b381a1d --- /dev/null +++ b/src/aipass/daemon/apps/handlers/schedule/telegram_notifier.py @@ -0,0 +1,174 @@ + +# ===================AIPASS==================== +# META DATA HEADER +# Name: telegram_notifier.py - DAEMON Scheduler Telegram Notifications +# Date: 2026-02-15 +# Version: 1.0.0 +# Category: daemon/handlers/schedule +# +# CHANGELOG (Max 5 entries): +# - v1.0.0 (2026-02-15): Initial implementation - scheduler bot notifications +# +# CODE STANDARDS: +# - Handlers implement logic, modules orchestrate +# - No cross-branch imports, no Prax logger +# ============================================= + +""" +Handler for sending Telegram notifications via the scheduler bot. + +Reusable notification layer for scheduled events. Uses the dedicated +scheduler bot (separate from bridge bot) to notify Patrick of +triggered events, completions, and errors. +""" + +import os +import json +from datetime import datetime +from pathlib import Path +from typing import Dict +from urllib.request import Request, urlopen +from urllib.error import URLError + +# ============================================= +# CONSTANTS +# ============================================= + +CONFIG_PATH = Path(os.environ.get('AIPASS_DAEMON_CONFIG', Path.home() / '.aipass' / 'daemon_config.json')) + + +# ============================================= +# CONFIG +# ============================================= + +def load_config() -> Dict[str, str]: + """ + Load scheduler bot config from daemon_config.json. + + Returns: + Dict with 'bot_token' and 'chat_id' keys + + Raises: + FileNotFoundError: If config file is missing + KeyError: If required keys are absent + """ + with open(CONFIG_PATH, "r", encoding="utf-8") as f: + raw = json.load(f) + + return { + "bot_token": raw["telegram_bot_token"], + "chat_id": raw["telegram_chat_id"], + } + + +# ============================================= +# SEND +# ============================================= + +def send_notification(message: str) -> bool: + """ + Send a message to Patrick via the scheduler bot. + + Args: + message: Text to send (plain text, supports emoji) + + Returns: + True if sent successfully, False otherwise + """ + try: + config = load_config() + except (FileNotFoundError, KeyError, json.JSONDecodeError) as e: + print(f"[telegram_notifier] Config error: {e}") + return False + + url = f"https://api.telegram.org/bot{config['bot_token']}/sendMessage" + payload = { + "chat_id": config["chat_id"], + "text": message, + } + + data = json.dumps(payload).encode("utf-8") + req = Request(url, data=data, headers={"Content-Type": "application/json"}) + + try: + with urlopen(req, timeout=15) as resp: + result = json.loads(resp.read()) + if result.get("ok"): + return True + print(f"[telegram_notifier] API error: {result.get('description')}") + return False + except URLError as e: + print(f"[telegram_notifier] Send failed: {e}") + return False + except Exception as e: + print(f"[telegram_notifier] Unexpected error: {e}") + return False + + +# ============================================= +# NOTIFICATION HELPERS +# ============================================= + +def notify_triggered(event_name: str) -> bool: + """ + Notify that a scheduled event was triggered. + + Args: + event_name: Name of the event that fired + + Returns: + True if notification sent, False otherwise + """ + now = datetime.now().strftime("%H:%M:%S") + message = f"\U0001f514 Scheduler: {event_name} triggered at {now}" + return send_notification(message) + + +def notify_complete(event_name: str, summary: str) -> bool: + """ + Notify that a scheduled event completed successfully. + + Args: + event_name: Name of the event that completed + summary: Brief summary of what happened + + Returns: + True if notification sent, False otherwise + """ + message = f"\u2705 Scheduler: {event_name} complete\n{summary}" + return send_notification(message) + + +def notify_error(event_name: str, error: str) -> bool: + """ + Notify that a scheduled event failed. + + Args: + event_name: Name of the event that failed + error: Error description + + Returns: + True if notification sent, False otherwise + """ + message = f"\u274c Scheduler: {event_name} failed\n{error}" + return send_notification(message) + + +# ============================================= +# MAIN - Testing +# ============================================= + +if __name__ == "__main__": + print("telegram_notifier.py - manual test") + print(f"Config path: {CONFIG_PATH}") + + try: + cfg = load_config() + print(f"Bot token: {cfg['bot_token'][:12]}...") + print(f"Chat ID: {cfg['chat_id']}") + except Exception as e: + print(f"Config load failed: {e}") + + print("\nSending test notification...") + ok = send_notification("Test from telegram_notifier.py handler") + print(f"Result: {'OK' if ok else 'FAILED'}") diff --git a/src/aipass/daemon/apps/handlers/telegram/__init__.py b/src/aipass/daemon/apps/handlers/telegram/__init__.py new file mode 100644 index 00000000..e69de29b diff --git a/src/aipass/daemon/apps/handlers/telegram/assistant_chat.py b/src/aipass/daemon/apps/handlers/telegram/assistant_chat.py new file mode 100755 index 00000000..2d282d36 --- /dev/null +++ b/src/aipass/daemon/apps/handlers/telegram/assistant_chat.py @@ -0,0 +1,51 @@ + +# ===================AIPASS==================== +# META DATA HEADER +# Name: assistant_chat.py - Daemon Bot Telegram Launcher +# Date: 2026-02-15 +# Version: 2.0.0 +# Category: daemon/handlers/telegram +# +# CHANGELOG (Max 5 entries): +# - v2.0.0 (2026-02-15): Thin launcher - delegates to shared direct_chat.py +# - v1.1.0 (2026-02-15): Photo/document support (archived) +# - v1.0.0 (2026-02-15): Initial long-polling + tmux bridge (archived) +# +# CODE STANDARDS: +# - Thin launcher only - all logic lives in direct_chat.py +# - External AI imports made optional +# ============================================= + +""" +Daemon Bot Telegram Launcher (@aipass_assistant_bot) + +Thin wrapper around the shared direct_chat module. All chat logic, +command handling, tmux management, and polling lives in direct_chat.py. +This file only provides the daemon-specific configuration. +""" + +import sys +from pathlib import Path + +try: + from api.apps.modules.telegram_chat import run_direct_chat + TELEGRAM_CHAT_AVAILABLE = True +except ImportError: + TELEGRAM_CHAT_AVAILABLE = False + run_direct_chat = None + +_DAEMON_ROOT = Path(__file__).resolve().parents[4] # src/aipass/daemon/ + +if not TELEGRAM_CHAT_AVAILABLE: + print("[assistant_chat] telegram_chat module not available, exiting") + sys.exit(1) + +sys.exit(run_direct_chat( + branch_name="daemon", + session_name="telegram-daemon", + config_path=Path.home() / ".aipass" / "assistant_bot_config.json", + work_dir=_DAEMON_ROOT, + log_dir=_DAEMON_ROOT / "logs", + data_dir=_DAEMON_ROOT / "daemon_json", + bot_name="AIPass Assistant Bot", +)) diff --git a/src/aipass/daemon/apps/handlers/update/__init__.py b/src/aipass/daemon/apps/handlers/update/__init__.py new file mode 100644 index 00000000..61762d07 --- /dev/null +++ b/src/aipass/daemon/apps/handlers/update/__init__.py @@ -0,0 +1 @@ +# Update handlers package diff --git a/src/aipass/daemon/apps/handlers/update/data_loader.py b/src/aipass/daemon/apps/handlers/update/data_loader.py new file mode 100644 index 00000000..bb59a101 --- /dev/null +++ b/src/aipass/daemon/apps/handlers/update/data_loader.py @@ -0,0 +1,112 @@ + +# ===================AIPASS==================== +# META DATA HEADER +# Name: data_loader.py - DAEMON Data Loading Handler +# Date: 2026-01-29 +# Version: 1.0.0 +# Category: daemon/handlers/update +# +# CHANGELOG (Max 5 entries): +# - v1.0.0 (2026-01-29): Initial implementation - data loading for update digest +# +# CODE STANDARDS: +# - Handlers implement logic, modules orchestrate +# - No cross-branch imports, no Prax logger +# ============================================= + +""" +Handler for loading DAEMON data from inbox and local files. +""" + +import json +from pathlib import Path +from typing import Dict, Any, List + +# ============================================= +# CONSTANTS +# ============================================= + +_DAEMON_ROOT = Path(__file__).resolve().parents[4] # src/aipass/daemon/ +INBOX_PATH = _DAEMON_ROOT / "ai_mail.local" / "inbox.json" +LOCAL_PATH = _DAEMON_ROOT / "DAEMON.local.json" + +# ============================================= +# DATA LOADING +# ============================================= + +def load_inbox() -> Dict[str, Any]: + """Load inbox.json and return parsed data.""" + if not INBOX_PATH.exists(): + return {"messages": [], "total_messages": 0, "unread_count": 0} + + try: + with open(INBOX_PATH, 'r', encoding='utf-8') as f: + return json.load(f) + except Exception: + return {"messages": [], "total_messages": 0, "unread_count": 0} + + +def load_local() -> Dict[str, Any]: + """Load DAEMON.local.json and return parsed data.""" + if not LOCAL_PATH.exists(): + return {"sessions": [], "active_tasks": {}} + + try: + with open(LOCAL_PATH, 'r', encoding='utf-8') as f: + return json.load(f) + except Exception: + return {"sessions": [], "active_tasks": {}} + + +# ============================================= +# DIGEST ANALYSIS +# ============================================= + +def categorize_messages(messages: List[Dict[str, Any]]) -> Dict[str, List[Dict[str, Any]]]: + """ + Categorize inbox messages by status. + + Returns: + Dict with keys: new, opened, actionable, informational + """ + categories: Dict[str, List[Dict[str, Any]]] = { + "new": [], + "opened": [], + "actionable": [], + "informational": [] + } + + for msg in messages: + status = msg.get("status", "new") + subject = msg.get("subject", "").upper() + + if status == "new": + categories["new"].append(msg) + elif status == "opened": + categories["opened"].append(msg) + + if any(kw in subject for kw in ["TASK:", "BUILD:", "FIX:", "PROPOSAL:", "REQUEST:"]): + categories["actionable"].append(msg) + elif any(kw in subject for kw in ["INFO", "RE:", "FYI", "NOTIFICATION"]): + categories["informational"].append(msg) + + return categories + + +def get_session_summary(local_data: Dict[str, Any]) -> Dict[str, Any]: + """Extract session summary from local.json.""" + sessions = local_data.get("sessions", []) + active_tasks = local_data.get("active_tasks", {}) + + return { + "total_sessions": len(sessions), + "today_focus": active_tasks.get("today_focus", "None"), + "recently_completed": active_tasks.get("recently_completed", []), + "latest_session": sessions[0] if sessions else None + } + + +def get_escalations(messages: List[Dict[str, Any]]) -> List[Dict[str, Any]]: + """Find messages that need escalation.""" + return [m for m in messages if "BLOCKED" in m.get("subject", "").upper() + or "URGENT" in m.get("subject", "").upper()] diff --git a/src/aipass/daemon/apps/json_templates/__init__.py b/src/aipass/daemon/apps/json_templates/__init__.py new file mode 100644 index 00000000..5d00b535 --- /dev/null +++ b/src/aipass/daemon/apps/json_templates/__init__.py @@ -0,0 +1 @@ +# JSON Templates package - Default JSON file templates diff --git a/src/aipass/daemon/apps/modules/__init__.py b/src/aipass/daemon/apps/modules/__init__.py new file mode 100644 index 00000000..4cf76dce --- /dev/null +++ b/src/aipass/daemon/apps/modules/__init__.py @@ -0,0 +1 @@ +# Modules package - Branch-specific functionality modules diff --git a/src/aipass/daemon/apps/modules/actions.py b/src/aipass/daemon/apps/modules/actions.py new file mode 100644 index 00000000..bd157c38 --- /dev/null +++ b/src/aipass/daemon/apps/modules/actions.py @@ -0,0 +1,527 @@ + +# ===================AIPASS==================== +# META DATA HEADER +# Name: actions.py - Action Registry CLI Module +# Date: 2026-03-02 +# Version: 1.0.0 +# Category: daemon/modules +# +# CHANGELOG (Max 5 entries): +# - v1.0.0 (2026-03-02): Initial creation - DPLAN-043 +# * List, toggle, info, set reminder, set schedule +# * Plugin migration command +# * Formatted Rich console output +# +# CODE STANDARDS: +# - Seed pattern compliance - console.print() for all output +# - Thin orchestration - handlers implement logic +# ============================================= + +""" +CLI interface for the numbered action registry. +""" + +# ============================================= +# IMPORTS +# ============================================= + +import sys +from pathlib import Path +from typing import List + +import logging +logger = logging.getLogger(__name__) + +from rich.console import Console +console = Console() + +def _header(text): + console.print(f"\n[bold cyan]{'='*70}[/bold cyan]") + console.print(f"[bold cyan] {text}[/bold cyan]") + console.print(f"[bold cyan]{'='*70}[/bold cyan]") + +def _success(text): + console.print(f"[green]OK:[/green] {text}") + +def _error(text): + console.print(f"[red]ERROR:[/red] {text}") + +from ..handlers.actions.actions_registry import ( + list_actions, + get_action, + toggle_action, + delete_action, + create_action, + migrate_plugins, + next_due_str, +) + +# ============================================= +# CONSTANTS +# ============================================= + +MODULE_NAME = "actions" + + +# ============================================= +# OUTPUT FORMATTING +# ============================================= + +def _print_actions_table(actions: list) -> None: + """Display formatted action list as a table.""" + console.print() + _header("Action Registry") + console.print() + + if not actions: + console.print("[dim]No actions registered. Run 'actions migrate' to import plugins.[/dim]") + console.print() + return + + # Header row + console.print( + f" {'ID':<6} {'ON':<4} {'NAME':<24} {'TYPE':<10} " + f"{'TARGET':<16} {'SCHEDULE':<20} {'NEXT DUE':<16}" + ) + console.print(" " + "-" * 96) + + for action in actions: + action_id = action.get("id", "????") + enabled = "[green]ON[/green] " if action.get("enabled") else "[red]OFF[/red]" + name = action.get("name", "")[:22] + action_type = action.get("type", "")[:8] + target = action.get("target_branch", "")[:14] + schedule_type = action.get("schedule_type", "") + + # Build schedule display + if schedule_type == "daily": + schedule_str = f"daily @ {action.get('time', '??:??')}" + elif schedule_type == "hourly": + m = action.get("time", "0") + schedule_str = f"hourly @ :{int(m):02d}" + elif schedule_type == "interval": + mins = action.get("interval_minutes", 0) + if mins >= 60: + schedule_str = f"every {mins // 60}h" + else: + schedule_str = f"every {mins}m" + elif schedule_type == "once": + schedule_str = f"once: {action.get('due_date', '?')}" + else: + schedule_str = schedule_type + + next_due = next_due_str(action) + + console.print( + f" {action_id:<6} {enabled:<4} {name:<24} {action_type:<10} " + f" {target:<16} {schedule_str:<20} {next_due:<16}" + ) + + console.print() + enabled_count = sum(1 for a in actions if a.get("enabled")) + console.print(f" [dim]Total: {len(actions)} actions ({enabled_count} enabled)[/dim]") + console.print() + + +def _print_action_detail(action: dict) -> None: + """Display detailed view of a single action.""" + console.print() + _header(f"Action {action['id']}: {action['name']}") + console.print() + + fields = [ + ("ID", action.get("id")), + ("Name", action.get("name")), + ("Type", action.get("type")), + ("Enabled", "[green]ON[/green]" if action.get("enabled") else "[red]OFF[/red]"), + ("Schedule", action.get("schedule_type")), + ("Time", action.get("time")), + ("Interval", f"{action.get('interval_minutes')}m" if action.get("interval_minutes") else None), + ("Due Date", action.get("due_date")), + ("Target", action.get("target_branch")), + ("Fresh", action.get("fresh")), + ("Max Turns", action.get("max_turns")), + ("Self Dispatch", action.get("self_dispatch")), + ("Plugin File", action.get("plugin_file")), + ("Last Run", action.get("last_run", "never")[:19] if action.get("last_run") else "never"), + ("Next Run", next_due_str(action)), + ("Created", action.get("created", "")[:19]), + ("Completed", action.get("completed")), + ] + + for label, value in fields: + if value is None: + continue + console.print(f" [cyan]{label:<16}[/cyan] {value}") + + # Show prompt (truncated for readability) + prompt = action.get("prompt", "") + if prompt: + console.print() + console.print(" [cyan]Prompt:[/cyan]") + # Show first 200 chars + display_prompt = prompt[:200] + if len(prompt) > 200: + display_prompt += "..." + for line in display_prompt.split("\n"): + console.print(f" [dim]{line}[/dim]") + + console.print() + + +def _print_help() -> None: + """Display help using Rich formatted output.""" + console.print() + _header("Actions -- Numbered Action Registry") + console.print() + + console.print("[yellow]USAGE:[/yellow]") + console.print(" drone @daemon actions list") + console.print(" drone @daemon actions info") + console.print(" drone @daemon actions on") + console.print(" drone @daemon actions off") + console.print(' drone @daemon actions set reminder "message" [--to @branch]') + console.print(' drone @daemon actions set schedule @branch "prompt" [time]') + console.print(" drone @daemon actions migrate") + console.print(" drone @daemon actions delete ") + console.print() + + console.print("[yellow]COMMANDS:[/yellow]") + console.print(" list List all registered actions with status") + console.print(" info Show detailed view of a single action") + console.print(" on Enable an action") + console.print(" off Disable an action") + console.print(" set Create a new reminder or schedule") + console.print(" migrate Import existing plugins into registry") + console.print(" delete Remove an action from the registry") + console.print() + + console.print("[yellow]SET REMINDER:[/yellow]") + console.print(' set reminder 2026-03-11 "Check VERA progress"') + console.print(' set reminder 7d "Follow up on PR review" --to @flow') + console.print(" [dim]Date formats: YYYY-MM-DD, 1d, 7d, 1w, 2w[/dim]") + console.print() + + console.print("[yellow]SET SCHEDULE:[/yellow]") + console.print(' set schedule @seed "Run audit" daily 04:00') + console.print(' set schedule @vera "Heartbeat" interval 240') + console.print(' set schedule @flow "Check plans" hourly 30') + console.print(" [dim]Types: daily HH:MM, hourly MM, interval MINUTES[/dim]") + console.print() + + console.print("[yellow]EXAMPLES:[/yellow]") + console.print(" actions list # See all actions") + console.print(" actions 0003 off # Disable action 3") + console.print(" actions 0003 on # Re-enable it") + console.print(' actions set reminder 2026-03-11 "check VERA" # One-shot reminder') + console.print() + + +# ============================================= +# SUBCOMMAND HANDLERS +# ============================================= + +def _handle_list(_args: List[str]) -> bool: + """Handle 'actions list' subcommand.""" + actions = list_actions() + _print_actions_table(actions) + return True + + +def _handle_toggle(action_id: str, enable: bool) -> bool: + """Handle 'actions on/off' subcommand.""" + action = get_action(action_id) + if action is None: + _error(f"Action not found: {action_id}") + return False + + toggle_action(action_id, enable) + state = "enabled" if enable else "disabled" + _success(f"Action {action_id} ({action['name']}) {state}") + return True + + +def _handle_info(action_id: str) -> bool: + """Handle 'actions info' subcommand.""" + action = get_action(action_id) + if action is None: + _error(f"Action not found: {action_id}") + return False + + _print_action_detail(action) + return True + + +def _handle_set_reminder(args: List[str]) -> bool: + """Handle 'actions set reminder "message" [--to @branch]'.""" + if len(args) < 2: + _error('Usage: actions set reminder "message" [--to @branch]') + return False + + date_str = args[0] + message = args[1] + target_branch = "@dev_central" # Default reminder target + + # Parse --to flag + if "--to" in args: + to_idx = args.index("--to") + if to_idx + 1 < len(args): + target_branch = args[to_idx + 1] + + # Parse date + due_date = _parse_date(date_str) + if not due_date: + _error(f"Invalid date format: {date_str}") + console.print("[dim]Valid formats: YYYY-MM-DD, 1d, 7d, 1w, 2w[/dim]") + return False + + action = create_action( + name=message[:50], + action_type="reminder", + schedule_type="once", + target_branch=target_branch, + prompt=message, + due_date=due_date, + fresh=True, + max_turns=10, + enabled=True, + ) + + _success(f"Reminder created: {action['id']}") + console.print(f" [dim]Due:[/dim] {due_date}") + console.print(f" [dim]To:[/dim] {target_branch}") + console.print(f" [dim]Message:[/dim] {message[:60]}") + console.print() + return True + + +def _handle_set_schedule(args: List[str]) -> bool: + """Handle 'actions set schedule @branch "prompt" [time_spec]'.""" + if len(args) < 3: + _error('Usage: actions set schedule @branch "prompt" [time_spec]') + return False + + target_branch = args[0] + prompt = args[1] + schedule_type = args[2] + + time_val = None + interval_minutes = None + + if schedule_type == "daily": + if len(args) < 4: + _error("Daily schedule requires time: actions set schedule @branch \"prompt\" daily HH:MM") + return False + time_val = args[3] + elif schedule_type == "hourly": + if len(args) < 4: + _error("Hourly schedule requires minute: actions set schedule @branch \"prompt\" hourly MM") + return False + time_val = args[3] + elif schedule_type == "interval": + if len(args) < 4: + _error("Interval schedule requires minutes: actions set schedule @branch \"prompt\" interval MINUTES") + return False + try: + interval_minutes = int(args[3]) + except ValueError: + _error(f"Invalid interval minutes: {args[3]}") + return False + else: + _error(f"Unknown schedule type: {schedule_type}") + console.print("[dim]Valid types: daily, hourly, interval[/dim]") + return False + + # Generate a name from the prompt + name = prompt[:50].replace(" ", "_").lower() + + action = create_action( + name=name, + action_type="schedule", + schedule_type=schedule_type, + target_branch=target_branch, + prompt=prompt, + time=time_val, + interval_minutes=interval_minutes, + fresh=True, + max_turns=50, + enabled=True, + ) + + _success(f"Schedule created: {action['id']}") + console.print(f" [dim]Name:[/dim] {action['name']}") + console.print(f" [dim]Target:[/dim] {target_branch}") + console.print(f" [dim]Type:[/dim] {schedule_type}") + if time_val: + console.print(f" [dim]Time:[/dim] {time_val}") + if interval_minutes: + console.print(f" [dim]Every:[/dim] {interval_minutes} minutes") + console.print() + return True + + +def _handle_migrate(_args: List[str]) -> bool: + """Handle 'actions migrate' -- import plugins into registry.""" + console.print() + console.print("[dim]Scanning plugins/ for unregistered plugins...[/dim]") + + count = migrate_plugins() + + if count > 0: + _success(f"Migrated {count} plugin(s) into the action registry") + else: + console.print("[dim]All plugins already registered (or none found).[/dim]") + + # Show the updated list + actions = list_actions() + _print_actions_table(actions) + return True + + +def _handle_delete(args: List[str]) -> bool: + """Handle 'actions delete '.""" + if not args: + _error("Action ID required: actions delete ") + return False + + action_id = args[0] + action = get_action(action_id) + if action is None: + _error(f"Action not found: {action_id}") + return False + + delete_action(action_id) + _success(f"Deleted action {action_id}: {action['name']}") + return True + + +# ============================================= +# DATE PARSING +# ============================================= + +def _parse_date(date_str: str) -> str: + """ + Parse a date string into ISO format. + + Supports: YYYY-MM-DD, 1d, 7d, 1w, 2w + + Returns: + ISO date string or empty string on failure. + """ + from datetime import datetime, timedelta + + date_str = date_str.strip() + + # Relative dates + if date_str.endswith("d"): + try: + days = int(date_str[:-1]) + return (datetime.now() + timedelta(days=days)).strftime("%Y-%m-%d") + except ValueError: + return "" + elif date_str.endswith("w"): + try: + weeks = int(date_str[:-1]) + return (datetime.now() + timedelta(weeks=weeks)).strftime("%Y-%m-%d") + except ValueError: + return "" + + # ISO date + try: + datetime.strptime(date_str, "%Y-%m-%d") + return date_str + except ValueError: + return "" + + +# ============================================= +# ORCHESTRATION +# ============================================= + +def handle_command(command: str, args: List[str]) -> bool: + """ + Handle 'actions' command and route to subcommands. + + Args: + command: Command name (should be 'actions') + args: Command arguments + + Returns: + True if handled, False otherwise + """ + if command != "actions": + return False + + try: + # No args or help + if not args or args[0] in ['--help', '-h', 'help']: + _print_help() + return True + + subcommand = args[0] + + # Named subcommands + if subcommand == "list": + return _handle_list(args[1:]) + elif subcommand == "migrate": + return _handle_migrate(args[1:]) + elif subcommand == "delete": + return _handle_delete(args[1:]) + elif subcommand == "set": + # actions set reminder ... / actions set schedule ... + if len(args) < 2: + _error("Usage: actions set ...") + return False + set_type = args[1] + if set_type == "reminder": + return _handle_set_reminder(args[2:]) + elif set_type == "schedule": + return _handle_set_schedule(args[2:]) + else: + _error(f"Unknown set type: {set_type}. Use 'reminder' or 'schedule'.") + return False + + # Check if first arg is an action ID (4-digit numeric) + if subcommand.isdigit() and len(subcommand) == 4: + action_id = subcommand + if len(args) < 2: + # Default to info + return _handle_info(action_id) + sub_action = args[1] + if sub_action == "on": + return _handle_toggle(action_id, True) + elif sub_action == "off": + return _handle_toggle(action_id, False) + elif sub_action == "info": + return _handle_info(action_id) + else: + _error(f"Unknown action command: {sub_action}. Use 'on', 'off', or 'info'.") + return False + + _error(f"Unknown subcommand: {subcommand}") + console.print("[dim]Run 'actions --help' for available commands[/dim]") + return False + + except Exception as e: + logger.error("[actions] Error in actions command: %s", e, exc_info=True) + _error(f"Error: {e}") + return False + + +# ============================================= +# MAIN ENTRY +# ============================================= + +def main() -> None: + """Main entry point for direct execution.""" + args = sys.argv[1:] + + if not args or args[0] in ['--help', '-h', 'help']: + _print_help() + return + + handle_command('actions', args) + + +if __name__ == "__main__": + main() diff --git a/src/aipass/daemon/apps/modules/activity_report.py b/src/aipass/daemon/apps/modules/activity_report.py new file mode 100644 index 00000000..3abc9f29 --- /dev/null +++ b/src/aipass/daemon/apps/modules/activity_report.py @@ -0,0 +1,742 @@ + +# ===================AIPASS==================== +# META DATA HEADER +# Name: activity_report.py - Branch Activity Report Generator Module +# Date: 2026-01-30 +# Version: 0.1.0 +# Category: daemon/modules +# +# CHANGELOG (Max 5 entries): +# - v0.1.0 (2026-01-30): Initial implementation - FPLAN-0266 Phase 3 +# +# CODE STANDARDS: +# - Module orchestrates handlers - handlers implement logic +# - Plain text CLI output (no Rich colors for this module) +# - Type hints on all functions +# - Standard library only + handler imports +# ============================================= + +""" +Branch Activity Report Generator Module + +Orchestrates monitoring handlers to generate comprehensive activity reports. +Provides formatted CLI output and programmatic JSON access. + +This is a MODULE (orchestration layer) that coordinates: +- activity_collector: Scans branches for file modifications +- memory_health: Checks memory file health status +- red_flag_detector: Detects presence violations (code changed but memory not updated) +""" + +from datetime import datetime, timedelta +from typing import Dict, Any, List, Optional + +import logging +logger = logging.getLogger(__name__) + +# Import monitoring handlers using relative imports +from ..handlers.monitoring import activity_collector +from ..handlers.monitoring import memory_health +from ..handlers.monitoring import red_flag_detector + + +# ============================================= +# CONSTANTS +# ============================================= + +MODULE_NAME = "activity_report" + +# Status symbols for CLI output +SYMBOL_OK = "[OK]" +SYMBOL_WARNING = "[!]" +SYMBOL_RED = "[X]" +SYMBOL_INACTIVE = "[-]" + + +# ============================================= +# HELPER FUNCTIONS +# ============================================= + +def _format_time_ago(iso_timestamp: Optional[str]) -> str: + """ + Format an ISO timestamp as human-readable time ago. + + Args: + iso_timestamp: ISO format timestamp string or None. + + Returns: + Human-readable string like "2h ago" or "3d ago". + """ + if not iso_timestamp: + return "never" + + try: + dt = datetime.fromisoformat(iso_timestamp) + delta = datetime.now() - dt + + hours = delta.total_seconds() / 3600 + if hours < 1: + minutes = int(delta.total_seconds() / 60) + return f"{minutes}m ago" + elif hours < 24: + return f"{int(hours)}h ago" + else: + days = int(hours / 24) + return f"{days}d ago" + except (ValueError, TypeError): + return "unknown" + + +def _get_status_symbol(status: str) -> str: + """ + Get CLI symbol for a status code. + + Args: + status: Status string (OK, WARNING, RED, NO_ACTIVITY, RED_FLAG, ERROR). + + Returns: + ASCII symbol for display. + """ + status_upper = status.upper() + if status_upper in ("OK",): + return SYMBOL_OK + elif status_upper in ("WARNING",): + return SYMBOL_WARNING + elif status_upper in ("RED", "RED_FLAG", "ERROR"): + return SYMBOL_RED + else: + return SYMBOL_INACTIVE + + +def _box_header(title: str, width: int = 50) -> str: + """Generate a boxed header line.""" + return "=" * width + "\n" + title + "\n" + "=" * width + + +def _section_header(title: str) -> str: + """Generate a section header line.""" + return f"\n{title}\n" + "-" * len(title) + + +# ============================================= +# DATA AGGREGATION +# ============================================= + +def _aggregate_data(since_hours: float = 24) -> Dict[str, Any]: + """ + Aggregate data from all monitoring handlers. + + Args: + since_hours: Time window in hours to analyze. + + Returns: + Dict with combined data from all handlers. + """ + since_timestamp = datetime.now() - timedelta(hours=since_hours) + + # Get red flag summary (includes all branch statuses) + red_flag_summary = red_flag_detector.get_red_flag_summary(since_timestamp) + + # Get activity data + activity_data = activity_collector.get_all_branch_activity(since_timestamp) + + # Get memory health for each branch + branch_paths = activity_collector.get_branch_paths() + memory_health_data = {} + + for branch_info in branch_paths: + name = branch_info.get("name", "") + path = branch_info.get("path", "") + if name and path: + health = memory_health.get_memory_health_status(path, name) + memory_health_data[name] = health + + return { + "timestamp": datetime.now().isoformat(), + "time_window_hours": since_hours, + "red_flag_summary": red_flag_summary, + "activity_data": activity_data, + "memory_health": memory_health_data, + } + + +# ============================================= +# REPORT GENERATION +# ============================================= + +def generate_activity_report( + since_hours: float = 24, + verbosity: str = "normal" +) -> str: + """ + Generate a formatted CLI-ready activity report. + + Aggregates data from all monitoring handlers and produces a + human-readable report with branch status, red flags, and recommendations. + + Args: + since_hours: Time window in hours to analyze (default: 24). + verbosity: Report detail level - "brief", "normal", or "detailed". + + Returns: + Formatted string report suitable for CLI display. + """ + data = _aggregate_data(since_hours) + + red_flags = data["red_flag_summary"] + activity = data["activity_data"] + health_data = data["memory_health"] + + lines: List[str] = [] + + # Header + timestamp = datetime.now().strftime("%Y-%m-%d %H:%M:%S") + lines.append(_box_header(f"BRANCH ACTIVITY REPORT - {timestamp}\nTime window: Last {since_hours:.0f} hours")) + + # Summary section + total_branches = red_flags.get("total_branches", 0) + active_branches = red_flags.get("ok", 0) + red_flag_count = red_flags.get("red_flags", 0) + no_activity_count = red_flags.get("no_activity", 0) + error_count = red_flags.get("errors", 0) + + # Count memory health statuses + health_ok = sum(1 for h in health_data.values() if h.get("overall_status") == "OK") + health_warning = sum(1 for h in health_data.values() if h.get("overall_status") == "WARNING") + health_red = sum(1 for h in health_data.values() if h.get("overall_status") == "RED") + + lines.append(_section_header("SUMMARY")) + lines.append(f"Active branches: {active_branches}/{total_branches}") + lines.append(f"RED FLAGS: {red_flag_count}") + lines.append(f"Memory health: {health_ok} OK, {health_warning} warning, {health_red} red") + if error_count > 0: + lines.append(f"Scan errors: {error_count}") + + # Red Flags section + lines.append(_section_header("RED FLAGS (requires attention)")) + violations = red_flags.get("violations", []) + if violations: + for v in violations: + branch = v.get("branch_name", "UNKNOWN") + reason = v.get("reason", "Unknown reason") + code_count = v.get("code_change_count", 0) + lines.append(f" {SYMBOL_RED} {branch}") + lines.append(f" Reason: {reason}") + lines.append(f" Code changes: {code_count} files") + else: + lines.append(" [None - all active branches updated memories]") + + # Branch Status section + lines.append(_section_header("BRANCH STATUS")) + + all_branches = red_flags.get("all_branches", []) + + if verbosity == "brief": + # Just counts, no individual branches + pass + else: + for branch_status in all_branches: + name = branch_status.get("branch_name", "UNKNOWN") + status = branch_status.get("status", "UNKNOWN") + reason = branch_status.get("reason", "") + + # Get memory health for this branch + branch_health = health_data.get(name, {}) + mem_status = branch_health.get("overall_status", "UNKNOWN") + + symbol = _get_status_symbol(status) + + # Format status line + if status == "NO_ACTIVITY": + lines.append(f" {SYMBOL_INACTIVE} {name} - inactive (no changes)") + elif status == "RED_FLAG": + lines.append(f" {SYMBOL_RED} {name} - RED FLAG: {reason}") + elif status == "ERROR": + lines.append(f" {SYMBOL_RED} {name} - ERROR: {reason}") + else: + # Get memory update time + mem_update = branch_status.get("memory_last_update") + time_ago = _format_time_ago(mem_update) + + mem_symbol = _get_status_symbol(mem_status) + if mem_status == "OK": + lines.append(f" {SYMBOL_OK} {name} - OK (memory updated {time_ago})") + else: + lines.append(f" {SYMBOL_WARNING} {name} - {mem_status} (memory updated {time_ago})") + + # Detailed mode: show file changes + if verbosity == "detailed" and status != "NO_ACTIVITY": + code_changes = branch_status.get("code_changes", []) + memory_modified = branch_status.get("memory_files_modified", []) + + if code_changes: + lines.append(f" Code files: {len(code_changes)}") + for cf in code_changes[:5]: + lines.append(f" - {cf.get('file', 'unknown')}") + if len(code_changes) > 5: + lines.append(f" ... and {len(code_changes) - 5} more") + + if memory_modified: + lines.append(f" Memory files: {len(memory_modified)}") + for mf in memory_modified[:3]: + lines.append(f" - {mf.get('file', 'unknown')}") + + # Recommendations section + lines.append(_section_header("RECOMMENDATIONS")) + recommendations: List[str] = [] + + # Add recommendations for red flags + for v in violations: + branch = v.get("branch_name", "UNKNOWN") + recommendations.append(f"- {branch}: Update memory files to reflect code changes") + + # Add recommendations for warning health status + for name, health in health_data.items(): + if health.get("overall_status") == "WARNING": + issues = health.get("issues", []) + if issues: + recommendations.append(f"- {name}: {issues[0]}") + elif health.get("overall_status") == "RED": + issues = health.get("issues", []) + if issues: + recommendations.append(f"- {name}: URGENT - {issues[0]}") + + if recommendations: + for rec in recommendations[:10]: # Limit to 10 recommendations + lines.append(f" {rec}") + if len(recommendations) > 10: + lines.append(f" ... and {len(recommendations) - 10} more") + else: + lines.append(" [None - system healthy]") + + lines.append("") # Trailing newline + + return "\n".join(lines) + + +def generate_branch_report( + branch_name: str, + since_hours: float = 24 +) -> str: + """ + Generate a detailed report for a single branch. + + Provides a deep dive on one branch including all file changes, + memory status, and specific recommendations. + + Args: + branch_name: Name of the branch (uppercase, e.g., "DRONE"). + since_hours: Time window in hours to analyze. + + Returns: + Formatted string report for the specified branch. + """ + since_timestamp = datetime.now() - timedelta(hours=since_hours) + + # Find the branch path + branch_paths = activity_collector.get_branch_paths() + branch_path = None + for bp in branch_paths: + if bp.get("name", "").upper() == branch_name.upper(): + branch_path = bp.get("path") + branch_name = bp.get("name", branch_name) # Use canonical name + break + + if not branch_path: + return f"ERROR: Branch '{branch_name}' not found in registry." + + lines: List[str] = [] + + # Header + timestamp = datetime.now().strftime("%Y-%m-%d %H:%M:%S") + lines.append(_box_header(f"BRANCH REPORT: {branch_name}\n{timestamp} | Last {since_hours:.0f} hours")) + + # Get branch data + red_flag_status = red_flag_detector.get_branch_status( + branch_name, branch_path, since_timestamp + ) + health_status = memory_health.get_memory_health_status(branch_path, branch_name) + activity_data = activity_collector.scan_branch_activity( + branch_name, branch_path, since_timestamp + ) + + # Status Overview + lines.append(_section_header("STATUS OVERVIEW")) + + rf_status = red_flag_status.get("status", "UNKNOWN") + rf_reason = red_flag_status.get("reason", "") + mem_status = health_status.get("overall_status", "UNKNOWN") + + symbol = _get_status_symbol(rf_status) + lines.append(f"Activity Status: {symbol} {rf_status}") + lines.append(f" Reason: {rf_reason}") + + mem_symbol = _get_status_symbol(mem_status) + lines.append(f"Memory Health: {mem_symbol} {mem_status}") + + lines.append(f"Path: {branch_path}") + + # File Changes + lines.append(_section_header("FILE CHANGES")) + + code_files = activity_data.get("code_files", []) + memory_files = activity_data.get("memory_files", []) + + lines.append(f"Code files modified: {len(code_files)}") + for cf in code_files: + mtime = _format_time_ago(cf.get("mtime")) + lines.append(f" - {cf.get('name', 'unknown')} ({mtime})") + + lines.append(f"\nMemory files modified: {len(memory_files)}") + for mf in memory_files: + mtime = _format_time_ago(mf.get("mtime")) + lines.append(f" - {mf.get('name', 'unknown')} ({mtime})") + + if not code_files and not memory_files: + lines.append(" [No files modified in time window]") + + # Memory Health Details + lines.append(_section_header("MEMORY HEALTH DETAILS")) + + file_check = health_status.get("file_check", {}) + lines.append("Required files:") + for fname, exists in file_check.get("required", {}).items(): + symbol = SYMBOL_OK if exists else SYMBOL_RED + lines.append(f" {symbol} {fname}") + + lines.append("\nOptional files:") + for fname, exists in file_check.get("optional", {}).items(): + symbol = SYMBOL_OK if exists else SYMBOL_WARNING + lines.append(f" {symbol} {fname}") + + # Freshness + freshness = health_status.get("freshness_checks", {}) + if freshness: + lines.append("\nFreshness:") + for fname, fresh in freshness.items(): + status = fresh.get("status", "UNKNOWN") + message = fresh.get("message", "") + symbol = _get_status_symbol(status) + lines.append(f" {symbol} {fname}: {message}") + + # Issues + issues = health_status.get("issues", []) + if issues: + lines.append(_section_header("ISSUES")) + for issue in issues: + lines.append(f" - {issue}") + + # Recommendations + lines.append(_section_header("RECOMMENDATIONS")) + recommendations: List[str] = [] + + if rf_status == "RED_FLAG": + recommendations.append("Update memory files to document recent code changes") + + for issue in issues: + if "Missing required" in issue: + recommendations.append(f"Create missing file: {issue.split(':')[-1].strip()}") + elif "stale" in issue.lower() or "not modified" in issue.lower(): + recommendations.append("Review and update memory files") + + if recommendations: + for rec in recommendations: + lines.append(f" - {rec}") + else: + lines.append(" [None - branch is healthy]") + + lines.append("") + + return "\n".join(lines) + + +def get_json_report(since_hours: float = 24) -> Dict[str, Any]: + """ + Get raw report data as a dictionary for programmatic access. + + Useful for storing report snapshots, API responses, or further processing. + + Args: + since_hours: Time window in hours to analyze. + + Returns: + Dict containing all aggregated data from handlers. + """ + data = _aggregate_data(since_hours) + + red_flags = data["red_flag_summary"] + health_data = data["memory_health"] + activity = data["activity_data"] + + # Build summary + health_ok = sum(1 for h in health_data.values() if h.get("overall_status") == "OK") + health_warning = sum(1 for h in health_data.values() if h.get("overall_status") == "WARNING") + health_red = sum(1 for h in health_data.values() if h.get("overall_status") == "RED") + + summary = { + "total_branches": red_flags.get("total_branches", 0), + "active_branches": red_flags.get("ok", 0), + "red_flags": red_flags.get("red_flags", 0), + "no_activity": red_flags.get("no_activity", 0), + "errors": red_flags.get("errors", 0), + "health_ok": health_ok, + "health_warning": health_warning, + "health_red": health_red, + } + + # Build per-branch data + branches_data: Dict[str, Any] = {} + all_branches = red_flags.get("all_branches", []) + + for branch_status in all_branches: + name = branch_status.get("branch_name", "") + if not name: + continue + + branches_data[name] = { + "red_flag_status": branch_status, + "memory_health": health_data.get(name, {}), + "activity": activity.get("branches", {}).get(name, {}), + } + + return { + "timestamp": data["timestamp"], + "time_window_hours": data["time_window_hours"], + "summary": summary, + "violations": red_flags.get("violations", []), + "branches": branches_data, + } + + +# ============================================= +# COMMAND INTEGRATION (AUTO-DISCOVERY) +# ============================================= + +def _print_activity_help() -> None: + """Display help for the activity command.""" + print() + print("=" * 60) + print("ACTIVITY - Quick Activity Summary") + print("=" * 60) + print() + print("USAGE:") + print(" drone @daemon activity") + print(" daemon activity") + print(" daemon activity --hours 48") + print() + print("DESCRIPTION:") + print(" Quick 24-hour activity summary (default).") + print(" Shows branch status, red flags, and recommendations.") + print() + print("OPTIONS:") + print(" --hours N, -t N Time window in hours (default: 24)") + print(" --help, -h Show this help message") + print() + + +def _print_activity_report_help() -> None: + """Display help for the activity-report command.""" + print() + print("=" * 60) + print("ACTIVITY-REPORT - Full Detailed Report") + print("=" * 60) + print() + print("USAGE:") + print(" drone @daemon activity-report") + print(" daemon activity-report") + print(" daemon activity-report --hours 48") + print(" daemon activity-report --json") + print() + print("DESCRIPTION:") + print(" Full detailed activity report with file-level changes.") + print(" Includes per-branch breakdown and complete recommendations.") + print() + print("OPTIONS:") + print(" --hours N, -t N Time window in hours (default: 24)") + print(" --json, -j Output raw JSON data") + print(" --help, -h Show this help message") + print() + + +def _print_branch_health_help() -> None: + """Display help for the branch-health command.""" + print() + print("=" * 60) + print("BRANCH-HEALTH - Single Branch Deep Dive") + print("=" * 60) + print() + print("USAGE:") + print(" drone @daemon branch-health DRONE") + print(" daemon branch-health FLOW") + print(" daemon branch-health SEED --hours 48") + print() + print("DESCRIPTION:") + print(" Deep dive report for a single branch.") + print(" Shows all file changes, memory health, and specific recommendations.") + print() + print("OPTIONS:") + print(" Required - branch name (e.g., DRONE, FLOW, SEED)") + print(" --hours N, -t N Time window in hours (default: 24)") + print(" --help, -h Show this help message") + print() + + +def _parse_hours_arg(args: List[str]) -> float: + """ + Extract --hours or -t argument from args list. + + Args: + args: Command arguments list. + + Returns: + Hours value (default 24 if not specified). + """ + hours = 24.0 + i = 0 + while i < len(args): + if args[i] in ('--hours', '-t') and i + 1 < len(args): + try: + hours = float(args[i + 1]) + except ValueError: + pass + i += 2 + else: + i += 1 + return hours + + +def handle_command(command: str, args: List[str]) -> bool: + """ + Handle activity monitoring commands via auto-discovery. + + Routes commands to appropriate report generation functions. + + Commands: + - activity: Quick activity summary (verbosity="normal", last 24h) + - activity-report: Full detailed report (verbosity="detailed") + - branch-health : Single branch deep dive + + Args: + command: Command name (e.g., 'update', 'activity-report', 'branch-health') + args: Additional arguments (e.g., ['--hours', '48']) + + Returns: + True if command was handled, False if not our command. + """ + # Handle 'activity' command - quick summary + if command == "activity": + if args and args[0] in ('--help', '-h', 'help'): + _print_activity_help() + return True + + hours = _parse_hours_arg(args) + report = generate_activity_report(since_hours=hours, verbosity="normal") + print(report) + return True + + # Handle 'activity-report' command - detailed report + if command == "activity-report": + if args and args[0] in ('--help', '-h', 'help'): + _print_activity_report_help() + return True + + hours = _parse_hours_arg(args) + + # Check for --json flag + if '--json' in args or '-j' in args: + import json + data = get_json_report(hours) + print(json.dumps(data, indent=2)) + else: + report = generate_activity_report(since_hours=hours, verbosity="detailed") + print(report) + return True + + # Handle 'branch-health' command - single branch report + if command == "branch-health": + if args and args[0] in ('--help', '-h', 'help'): + _print_branch_health_help() + return True + + # Extract branch name (first non-flag argument) + branch_name = None + filtered_args = [] + i = 0 + while i < len(args): + if args[i] in ('--hours', '-t') and i + 1 < len(args): + filtered_args.extend([args[i], args[i + 1]]) + i += 2 + elif args[i].startswith('-'): + i += 1 + else: + if branch_name is None: + branch_name = args[i] + i += 1 + + if not branch_name: + print("ERROR: branch-health requires a branch name") + print() + print("Usage: branch-health [--hours N]") + print("Example: branch-health DRONE") + return True + + hours = _parse_hours_arg(args) + report = generate_branch_report(branch_name, since_hours=hours) + print(report) + return True + + # Not our command + return False + + +# ============================================= +# CLI ENTRY POINT +# ============================================= + +def main() -> None: + """Main entry point for direct execution.""" + import argparse + + parser = argparse.ArgumentParser( + description="Branch Activity Report Generator" + ) + parser.add_argument( + "--hours", "-t", + type=float, + default=24, + help="Time window in hours (default: 24)" + ) + parser.add_argument( + "--verbosity", "-v", + choices=["brief", "normal", "detailed"], + default="normal", + help="Report detail level (default: normal)" + ) + parser.add_argument( + "--branch", "-b", + type=str, + default=None, + help="Generate report for specific branch" + ) + parser.add_argument( + "--json", "-j", + action="store_true", + help="Output raw JSON data" + ) + + args = parser.parse_args() + + if args.json: + import json + data = get_json_report(args.hours) + print(json.dumps(data, indent=2)) + elif args.branch: + report = generate_branch_report(args.branch, args.hours) + print(report) + else: + report = generate_activity_report(args.hours, args.verbosity) + print(report) + + +if __name__ == "__main__": + main() diff --git a/src/aipass/daemon/apps/modules/schedule.py b/src/aipass/daemon/apps/modules/schedule.py new file mode 100644 index 00000000..c3cae7bf --- /dev/null +++ b/src/aipass/daemon/apps/modules/schedule.py @@ -0,0 +1,414 @@ + +# ===================AIPASS==================== +# META DATA HEADER +# Name: schedule.py - DAEMON Scheduled Follow-ups Module +# Date: 2026-02-04 +# Version: 1.0.0 +# Category: daemon/modules +# +# CHANGELOG (Max 5 entries): +# - v1.0.0 (2026-02-04): Initial implementation - fire-and-forget scheduled tasks +# +# CODE STANDARDS: +# - Seed pattern compliance - console.print() for all output +# - Thin orchestration - handlers implement logic +# - Type hints on all functions +# ============================================= + +""" +CLI interface for fire-and-forget scheduled follow-ups. +""" + +# ============================================= +# IMPORTS +# ============================================= + +import sys +import argparse +from pathlib import Path +from typing import List + +import logging +logger = logging.getLogger(__name__) + +from rich.console import Console +console = Console() + +def _header(text): + console.print(f"\n[bold cyan]{'='*70}[/bold cyan]") + console.print(f"[bold cyan] {text}[/bold cyan]") + console.print(f"[bold cyan]{'='*70}[/bold cyan]") + +def _success(text): + console.print(f"[green]OK:[/green] {text}") + +def _error(text): + console.print(f"[red]ERROR:[/red] {text}") + +# Handler imports +from ..handlers.schedule.task_registry import ( + load_tasks, create_task, delete_task, get_due_tasks, mark_completed, parse_due_date, + mark_dispatching, mark_pending, recover_stale_dispatches +) + +# File lock for single-instance execution +try: + from filelock import FileLock, Timeout + FILELOCK_AVAILABLE = True +except ImportError: + FILELOCK_AVAILABLE = False + +# Email integration (optional) +try: + from ai_mail.apps.modules.email import send_email_direct + AI_MAIL_AVAILABLE = True +except ImportError: + AI_MAIL_AVAILABLE = False + send_email_direct = None + +# ============================================= +# CONSTANTS +# ============================================= + +MODULE_NAME = "schedule" + +_DAEMON_ROOT = Path(__file__).resolve().parents[3] # src/aipass/daemon/ +JSON_DIR = _DAEMON_ROOT / "daemon_json" + +# ============================================= +# OUTPUT FORMATTING +# ============================================= + +def _print_task_list(tasks: List[dict]) -> None: + """Print formatted task list to console.""" + console.print() + _header("Scheduled Tasks") + console.print() + + pending_tasks = [t for t in tasks if t.get("status") == "pending"] + completed_tasks = [t for t in tasks if t.get("status") == "completed"] + + if not pending_tasks: + console.print("[dim]No pending scheduled tasks.[/dim]") + else: + console.print("[bold cyan]PENDING TASKS[/bold cyan]") + console.print(f"{'ID':<10} {'DUE':<20} {'TO':<15} {'TASK':<40}") + console.print("-" * 85) + + for task in pending_tasks: + task_id = task.get("id", "")[:8] + due = task.get("due_date", "") + recipient = task.get("recipient", "") + task_text = task.get("task", "")[:38] + console.print(f"{task_id:<10} {due:<20} {recipient:<15} {task_text:<40}") + + console.print() + console.print(f"[dim]Total: {len(pending_tasks)} pending, {len(completed_tasks)} completed[/dim]") + console.print() + + +def _print_help() -> None: + """Display help using Rich formatted output.""" + console.print() + _header("Schedule Module - Fire-and-Forget Follow-ups") + console.print() + + console.print("[yellow]USAGE:[/yellow]") + console.print(' drone @daemon schedule create "task" --due 7d --to @branch --message "details"') + console.print(" drone @daemon schedule list") + console.print(" drone @daemon schedule delete ") + console.print(" drone @daemon schedule run-due") + console.print() + + console.print("[yellow]COMMANDS:[/yellow]") + console.print(" create Create a new scheduled task") + console.print(" list List all pending scheduled tasks") + console.print(" delete Delete a scheduled task by ID") + console.print(" run-due Execute all due tasks (sends emails, marks complete)") + console.print() + + console.print("[yellow]CREATE OPTIONS:[/yellow]") + console.print(" --due (Required) Due date: 1d, 7d, 2w, 1m, or ISO date (2026-02-15)") + console.print(" --to (Required) Recipient branch (e.g., @flow, @seed)") + console.print(" --message (Optional) Additional details for the follow-up") + console.print() + + console.print("[yellow]EXAMPLES:[/yellow]") + console.print(' # Remind Flow to check on a plan in 7 days') + console.print(' schedule create "Check FPLAN-0290 status" --due 7d --to @flow') + console.print() + console.print(' # Follow up with Seed about code review in 2 weeks') + console.print(' schedule create "Code review follow-up" --due 2w --to @seed --message "Review PR #45"') + console.print() + console.print(' # Check all due tasks and send reminder emails') + console.print(' schedule run-due') + console.print() + + +# ============================================= +# SUBCOMMAND HANDLERS +# ============================================= + +def _handle_create(args: List[str]) -> bool: + """Handle schedule create subcommand.""" + parser = argparse.ArgumentParser(prog="schedule create", add_help=False) + parser.add_argument("task", nargs="?", help="Task description") + parser.add_argument("--due", required=True, help="Due date (1d, 7d, 2w, 1m, or ISO date)") + parser.add_argument("--to", required=True, dest="recipient", help="Recipient branch") + parser.add_argument("--message", default="", help="Additional message details") + + try: + parsed = parser.parse_args(args) + except SystemExit: + console.print("[red]Usage: schedule create \"task\" --due --to @branch [--message \"details\"][/red]") + return False + + if not parsed.task: + _error("Task description is required") + console.print("[dim]Usage: schedule create \"task\" --due --to @branch[/dim]") + return False + + # Parse and validate due date + due_date = parse_due_date(parsed.due) + if not due_date: + _error(f"Invalid due date format: {parsed.due}") + console.print("[dim]Valid formats: 1d, 7d, 2w, 1m, or ISO date (2026-02-15)[/dim]") + return False + + # Create the task + try: + new_task = create_task( + task=parsed.task, + due_date=due_date, + recipient=parsed.recipient, + message=parsed.message + ) + task_id = new_task.get("id", "") + + _success(f"Scheduled task created: {task_id[:8]}") + console.print(f" [dim]Task:[/dim] {parsed.task}") + console.print(f" [dim]Due:[/dim] {due_date}") + console.print(f" [dim]To:[/dim] {parsed.recipient}") + if parsed.message: + console.print(f" [dim]Msg:[/dim] {parsed.message[:50]}...") + console.print() + + logger.info(f"[DAEMON] Scheduled task created: {task_id[:8]} -> {parsed.recipient}") + return True + + except Exception as e: + _error(f"Failed to create task: {e}") + logger.error(f"[DAEMON] Failed to create scheduled task: {e}", exc_info=True) + return False + + +def _handle_list(_args: List[str]) -> bool: + """Handle schedule list subcommand.""" + try: + tasks = load_tasks() + _print_task_list(tasks) + return True + + except Exception as e: + _error(f"Failed to load tasks: {e}") + logger.error(f"[DAEMON] Failed to load scheduled tasks: {e}", exc_info=True) + return False + + +def _handle_delete(args: List[str]) -> bool: + """Handle schedule delete subcommand.""" + if not args: + _error("Task ID is required") + console.print("[dim]Usage: schedule delete [/dim]") + return False + + task_id = args[0] + + try: + deleted = delete_task(task_id) + if deleted: + _success(f"Task deleted: {task_id[:8]}") + logger.info(f"[DAEMON] Scheduled task deleted: {task_id[:8]}") + return True + else: + _error(f"Task not found: {task_id[:8]}") + return False + + except Exception as e: + _error(f"Failed to delete task: {e}") + logger.error(f"[DAEMON] Failed to delete scheduled task: {e}", exc_info=True) + return False + + +def _handle_run_due(_args: List[str]) -> bool: + """Handle schedule run-due subcommand with single-instance lock.""" + if not FILELOCK_AVAILABLE: + console.print("[dim]filelock not available, running without lock.[/dim]") + return _process_due_tasks() + + lock_file = JSON_DIR / "schedule.lock" + lock_file.parent.mkdir(parents=True, exist_ok=True) + + # Try to acquire lock (non-blocking) + lock = FileLock(lock_file, timeout=0) + try: + with lock.acquire(timeout=0): + return _process_due_tasks() + except Timeout: + console.print("[dim]Schedule run-due already in progress, skipping.[/dim]") + return True + + +def _process_due_tasks() -> bool: + """Process due tasks sequentially with status tracking.""" + import time + + try: + # Recover any stale dispatches (stuck > 5 minutes) + recovered = recover_stale_dispatches(max_age_minutes=5) + if recovered: + console.print(f"[dim]Recovered {recovered} stale dispatch(es)[/dim]") + + due_tasks = get_due_tasks() + + if not due_tasks: + console.print("[dim]No tasks due at this time.[/dim]") + return True + + console.print() + _header(f"Running {len(due_tasks)} Due Task(s)") + console.print() + + success_count = 0 + fail_count = 0 + + for task in due_tasks: + task_id = task.get("id", "") + recipient = task.get("recipient", "") + task_desc = task.get("task", "") + message = task.get("message", "") + + # Mark as dispatching (prevents re-dispatch) + mark_dispatching(task_id) + + # Build email body + email_body = f"{task_desc}" + if message: + email_body += f"\n\nDetails:\n{message}" + + # Send the email + if not AI_MAIL_AVAILABLE: + mark_pending(task_id) + _error(f"ai_mail not available, cannot send to {recipient}") + fail_count += 1 + continue + + try: + email_sent = send_email_direct( + to_branch=recipient, + subject=f"[SCHEDULED] {task_desc}", + message=email_body, + from_branch='@daemon', + auto_execute=True, + reply_to='@dev_central' + ) + + if email_sent: + mark_completed(task_id) + _success(f"Sent to {recipient}: {task_desc[:40]}") + success_count += 1 + logger.info(f"[DAEMON] Scheduled email sent: {task_id[:8]} -> {recipient}") + else: + # Reset to pending for retry + mark_pending(task_id) + _error(f"Failed to send to {recipient}: {task_desc[:40]}") + fail_count += 1 + logger.error(f"[DAEMON] Scheduled email failed: {task_id[:8]} -> {recipient}") + + except Exception as e: + # Reset to pending for retry + mark_pending(task_id) + _error(f"Error sending to {recipient}: {e}") + fail_count += 1 + logger.error(f"[DAEMON] Scheduled email error: {task_id[:8]} -> {recipient}: {e}") + + # Small delay between dispatches (prevents thundering herd) + time.sleep(1.0) + + console.print() + console.print(f"[bold]Results:[/bold] {success_count} sent, {fail_count} failed") + console.print() + + return fail_count == 0 + + except Exception as e: + _error(f"Failed to run due tasks: {e}") + logger.error(f"[DAEMON] Failed to run due tasks: {e}", exc_info=True) + return False + + +# ============================================= +# ORCHESTRATION +# ============================================= + +def handle_command(command: str, args: List[str]) -> bool: + """ + Handle 'schedule' command. + + Args: + command: Command name (should be 'schedule') + args: Command arguments (subcommand + subcommand args) + + Returns: + True if handled, False otherwise + """ + if command != "schedule": + return False + + try: + # Handle help flag + if not args or args[0] in ['--help', '-h', 'help']: + _print_help() + return True + + subcommand = args[0] + subargs = args[1:] + + # Route to subcommand handlers + if subcommand == "create": + return _handle_create(subargs) + elif subcommand == "list": + return _handle_list(subargs) + elif subcommand == "delete": + return _handle_delete(subargs) + elif subcommand == "run-due": + return _handle_run_due(subargs) + else: + _error(f"Unknown subcommand: {subcommand}") + console.print("[dim]Run 'schedule --help' for available commands[/dim]") + return False + + except Exception as e: + logger.error(f"[DAEMON] Error in schedule command: {e}", exc_info=True) + _error(f"Error: {e}") + return False + + +# ============================================= +# MAIN ENTRY +# ============================================= + +def main() -> None: + """Main entry point for direct execution.""" + args = sys.argv[1:] + + if len(args) == 0 or args[0] in ['--help', '-h', 'help']: + _print_help() + return + + # First arg is subcommand when called directly + handle_command('schedule', args) + + +if __name__ == "__main__": + main() diff --git a/src/aipass/daemon/apps/modules/update.py b/src/aipass/daemon/apps/modules/update.py new file mode 100644 index 00000000..06549f47 --- /dev/null +++ b/src/aipass/daemon/apps/modules/update.py @@ -0,0 +1,181 @@ + +# ===================AIPASS==================== +# META DATA HEADER +# Name: update.py - DAEMON Status Digest Module +# Date: 2026-01-29 +# Version: 1.0.0 +# Category: daemon/modules +# +# CHANGELOG (Max 5 entries): +# - v1.0.0 (2026-01-29): Initial implementation - digest of DAEMON activity +# +# CODE STANDARDS: +# - Seed pattern compliance - console.print() for all output +# - Thin orchestration - handlers implement logic +# - Type hints on all functions +# ============================================= + +""" +Returns digest of DAEMON activity for check-ins. +""" + +# ============================================= +# IMPORTS +# ============================================= + +import sys +from typing import Dict, Any, List + +import logging +logger = logging.getLogger(__name__) + +from rich.console import Console +console = Console() + +def _header(text): + console.print(f"\n[bold cyan]{'='*70}[/bold cyan]") + console.print(f"[bold cyan] {text}[/bold cyan]") + console.print(f"[bold cyan]{'='*70}[/bold cyan]") + +# Handler imports +from ..handlers.update.data_loader import ( + load_inbox, + load_local, + categorize_messages, + get_session_summary, + get_escalations +) + +# ============================================= +# CONSTANTS +# ============================================= + +MODULE_NAME = "update" + +# ============================================= +# OUTPUT FORMATTING +# ============================================= + +def _print_digest(inbox_data: Dict[str, Any], local_data: Dict[str, Any]) -> None: + """Print formatted digest to console.""" + console.print() + _header("DAEMON Status Digest") + console.print() + + messages: List[Dict[str, Any]] = inbox_data.get("messages", []) + categories = categorize_messages(messages) + + console.print("[bold cyan]INBOX STATUS[/bold cyan]") + console.print(f" Total messages: {inbox_data.get('total_messages', 0)}") + console.print(f" Unread (new): {len(categories['new'])}") + console.print(f" Opened: {len(categories['opened'])}") + console.print() + + console.print("[bold yellow]ACTIONABLE ITEMS[/bold yellow]") + if categories['actionable']: + for msg in categories['actionable'][:5]: + from_addr = msg.get('from', 'unknown') + subject = str(msg.get('subject', 'No subject'))[:50] + status = msg.get('status', 'new') + console.print(f" [{status}] {from_addr}: {subject}") + else: + console.print(" [dim]None pending[/dim]") + console.print() + + session_summary = get_session_summary(local_data) + console.print("[bold cyan]SESSION INFO[/bold cyan]") + console.print(f" Total sessions: {session_summary['total_sessions']}") + console.print(f" Today's focus: {session_summary['today_focus']}") + + recently_completed = session_summary.get('recently_completed', []) + if recently_completed: + console.print(f" Recently completed: {len(recently_completed)} tasks") + else: + console.print(" Recently completed: [dim]None[/dim]") + console.print() + + console.print("[bold red]ESCALATIONS NEEDED[/bold red]") + escalations = get_escalations(messages) + if escalations: + for msg in escalations: + console.print(f" ! {msg.get('from', 'unknown')}: {str(msg.get('subject', ''))[:50]}") + else: + console.print(" [dim]None - all clear[/dim]") + console.print() + + +def _print_help() -> None: + """Display help using Rich formatted output.""" + console.print() + _header("Update Module - DAEMON Status Digest") + console.print() + + console.print("[yellow]USAGE:[/yellow]") + console.print(" drone @daemon update") + console.print(" daemon update") + console.print() + + console.print("[yellow]DESCRIPTION:[/yellow]") + console.print(" Returns a digest of DAEMON activity for check-ins.") + console.print() + console.print(" Gathers and displays:") + console.print(" - Inbox status (total, unread, opened)") + console.print(" - Actionable items (tasks, builds, requests)") + console.print(" - Session info (focus, completed tasks)") + console.print(" - Escalations needed (blocked, urgent)") + console.print() + + +# ============================================= +# ORCHESTRATION +# ============================================= + +def handle_command(command: str, args: list) -> bool: + """ + Handle 'update' command. + + Args: + command: Command name (should be 'update') + args: Command arguments + + Returns: + True if handled, False otherwise + """ + if command != "update": + return False + + try: + if args and args[0] in ['--help', '-h', 'help']: + _print_help() + return True + + inbox_data = load_inbox() + local_data = load_local() + _print_digest(inbox_data, local_data) + + logger.info("[DAEMON] Update digest generated successfully") + return True + + except Exception as e: + logger.error(f"[DAEMON] Error generating update digest: {e}", exc_info=True) + console.print(f"[red]Error: {e}[/red]") + return True + + +# ============================================= +# MAIN ENTRY +# ============================================= + +def main() -> None: + """Main entry point for direct execution.""" + args = sys.argv[1:] + + if len(args) == 0 or args[0] in ['--help', '-h', 'help']: + _print_help() + return + + handle_command('update', args) + + +if __name__ == "__main__": + main() diff --git a/src/aipass/daemon/apps/plugins/__init__.py b/src/aipass/daemon/apps/plugins/__init__.py new file mode 100644 index 00000000..698c0172 --- /dev/null +++ b/src/aipass/daemon/apps/plugins/__init__.py @@ -0,0 +1,94 @@ + +# ===================AIPASS==================== +# META DATA HEADER +# Name: __init__.py - Daemon Scheduler Plugin Interface +# Date: 2026-02-20 +# Version: 1.0.0 +# Category: daemon/apps/plugins +# +# CHANGELOG (Max 5 entries): +# - v1.0.0 (2026-02-20): Initial plugin interface definition +# +# CODE STANDARDS: +# - Plugin interface specification only +# - No business logic +# ============================================= + +""" +Daemon Scheduler Plugin Interface + +Plugins are auto-discovered Python files in this directory. +Each plugin defines WHAT to run and WHEN via PLUGIN_CONFIG. +The scheduler runner handles HOW (calling wake script). + +Plugin Contract: + 1. Expose PLUGIN_CONFIG dict (required) + 2. Expose run() -> dict function (optional, for custom logic) + +PLUGIN_CONFIG Schema: + { + "name": str, # Unique plugin identifier + "schedule": str, # "daily" | "hourly" | "interval" + "time": str | None, # For daily: "HH:MM" (24h). For hourly: minute "MM" + "interval_minutes": int | None, # For interval: minutes between runs + "enabled": bool, # Plugin active/inactive toggle + "branch": str, # Target branch email (e.g., "@seed") + "fresh": bool, # True = fresh session, False = resume + "max_turns": int, # Max agent turns (safety limit) + "prompt": str, # What the spawned agent should do + } + +Schedule Types: + - "daily": Runs once per day at PLUGIN_CONFIG["time"] (HH:MM) + - "hourly": Runs once per hour at minute PLUGIN_CONFIG["time"] (MM) + - "interval": Runs every PLUGIN_CONFIG["interval_minutes"] minutes + +Naming Convention: + - Name plugins by WHAT they do, not WHO/WHEN + - Good: daily_audit.py, heartbeat.py, backup.py + - Bad: seed_daily_audit.py, vera_heartbeat.py + - The PLUGIN_CONFIG holds branch/schedule metadata +""" + +# Plugin discovery helper +import importlib +from pathlib import Path + + +def discover_plugins() -> list: + """ + Discover all valid plugins in this directory. + + Returns list of dicts: [{"module": module, "config": PLUGIN_CONFIG}, ...] + """ + plugins_dir = Path(__file__).parent + plugins = [] + + for file_path in sorted(plugins_dir.glob("*.py")): + if file_path.name.startswith("_"): + continue + + module_name = file_path.stem + try: + module = importlib.import_module(f".{module_name}", package=__package__) + + if not hasattr(module, "PLUGIN_CONFIG"): + continue + + config = module.PLUGIN_CONFIG + + # Validate required fields + required = {"name", "schedule", "enabled", "branch", "fresh", "max_turns", "prompt"} + missing = required - set(config.keys()) + if missing: + continue + + plugins.append({ + "module": module, + "config": config, + "file": str(file_path), + }) + except Exception: + continue + + return plugins diff --git a/src/aipass/daemon/apps/plugins/botfather_reminder.py b/src/aipass/daemon/apps/plugins/botfather_reminder.py new file mode 100644 index 00000000..922408d4 --- /dev/null +++ b/src/aipass/daemon/apps/plugins/botfather_reminder.py @@ -0,0 +1,89 @@ + +# ===================AIPASS==================== +# META DATA HEADER +# Name: botfather_reminder.py - Hourly reminder to create PATRICK_PRIVATE Telegram bot +# Date: 2026-02-26 +# Version: 1.0.0 +# Category: plugins/scheduler +# +# CHANGELOG (Max 5 entries): +# - v1.0.0 (2026-02-26): Initial - self-resolving BotFather reminder +# +# CODE STANDARDS: +# - Self-resolving: stops when token is set in config +# - Starts after 2026-02-27T09:00 (BotFather cooldown ~11hrs) +# - Sends email to @dev_central inbox, no agent spawn needed +# ============================================= + +""" +BotFather Reminder Plugin + +Hourly reminder to create the PATRICK_PRIVATE Telegram bot via BotFather. +Self-resolves: checks if bot token is still placeholder. Once real token +is set in patrick_private.json, this plugin silently exits. + +Delete this file once the bot is created and running. +""" + +import json +import subprocess +from datetime import datetime +from pathlib import Path + +PLUGIN_CONFIG = { + "name": "botfather_reminder", + "schedule": "hourly", + "time": "00", + "interval_minutes": None, + "enabled": True, + "branch": "@dev_central", + "fresh": False, + "max_turns": 3, + "prompt": "Reminder: create PATRICK_PRIVATE Telegram bot via BotFather", +} + +BOT_CONFIG = Path.home() / ".aipass" / "telegram_bots" / "patrick_private.json" +NOT_BEFORE = datetime(2026, 2, 27, 9, 0, 0) + + +def run() -> dict: + """Check if reminder is still needed, send email if so.""" + now = datetime.now() + + # Don't start until BotFather cooldown expires + if now < NOT_BEFORE: + return {"status": "waiting", "reason": f"cooldown until {NOT_BEFORE.isoformat()}"} + + # Check if token has been set (self-resolving) + try: + config = json.loads(BOT_CONFIG.read_text(encoding="utf-8")) + token = config.get("bot_token", "") + if token and token != "PASTE_TOKEN_HERE": + return {"status": "resolved", "reason": "token already set -- delete this plugin"} + except (json.JSONDecodeError, OSError): + pass # Config missing or broken -- still remind + + # Send reminder email + subject = "REMINDER: Create PATRICK_PRIVATE Telegram bot" + message = ( + "BotFather cooldown should be expired. Steps:\n" + "1. Open Telegram -> @BotFather -> /newbot\n" + "2. Pick a name (anything private, e.g. 'My Notes')\n" + "3. Pick a username (must end in _bot)\n" + "4. Paste the token into the DEV_CENTRAL chat\n" + "5. Config ready at: ~/.aipass/telegram_bots/patrick_private.json\n\n" + "This reminder repeats hourly until the token is set." + ) + + try: + subprocess.run( + [ + "drone", "@ai_mail", "send", "@dev_central", + subject, message, + ], + capture_output=True, timeout=15, + ) + except (subprocess.SubprocessError, OSError): + pass # Silent -- never crash scheduler + + return {"status": "reminded", "next_check": "1 hour"} diff --git a/src/aipass/daemon/apps/plugins/community_rotation.py b/src/aipass/daemon/apps/plugins/community_rotation.py new file mode 100644 index 00000000..5ecb3e90 --- /dev/null +++ b/src/aipass/daemon/apps/plugins/community_rotation.py @@ -0,0 +1,444 @@ + +# ===================AIPASS==================== +# META DATA HEADER +# Name: community_rotation.py - Rotating Community Engagement Plugin +# Date: 2026-02-22 +# Version: 1.1.0 +# Category: daemon/apps/plugins +# +# CHANGELOG (Max 5 entries): +# - v1.1.0 (2026-02-22): Dashboard pre-check, activity tracking, inactivity alerting +# - v1.0.0 (2026-02-21): Initial creation - hourly branch rotation for Commons engagement +# +# CODE STANDARDS: +# - Plugin interface: PLUGIN_CONFIG + run() +# - Named by action (community_rotation), not target +# - Self-dispatching: handles branch selection and wake internally +# ============================================= + +""" +Rotating Community Engagement Plugin + +Wakes one branch per rotation on a rotating schedule to: +1. Process email inbox (view, reply, close stale emails) +2. Check dashboard for Commons notifications (mentions, comments, votes) +3. Act on any pending notifications +4. Browse The Commons feed and engage if something catches their eye + +Rotates through all eligible branches from BRANCH_REGISTRY, +excluding VERA (has her own heartbeat), DEV_CENTRAL (human), +DAEMON (self), and non-agent directories. + +Full rotation every ~14 hours, then loops. +""" + +import json +import os +import subprocess +import sys +from datetime import datetime +from pathlib import Path + +import logging +logger = logging.getLogger(__name__) + +# Paths +REGISTRY_PATH = Path(os.environ.get('AIPASS_REGISTRY', Path.home() / '.aipass' / 'AIPASS_REGISTRY.json')) +ROTATION_STATE_FILE = Path(__file__).parent / ".rotation_state.json" +ACTIVITY_TRACKER_FILE = Path(__file__).parent / ".activity_tracker.json" + +# Wake script path (configurable via env var) +WAKE_SCRIPT = Path(os.environ.get('AIPASS_WAKE_SCRIPT', '')) + +# Inactivity threshold -- consecutive zero-activity passes before alerting +INACTIVITY_THRESHOLD = 10 + +# Branches to EXCLUDE from rotation +# Every branch is a citizen -- the Commons gives them life. +# Only exclude branches that can't or shouldn't be auto-woken. +EXCLUDED_BRANCHES = { + "VERA", # Already checks Commons via her own heartbeat plugin + "DEV_CENTRAL", # Human workspace -- can't be auto-woken (needs Patrick) + "PARTICK_PRIVATE", # Patrick's private branch +} + +# Engagement prompt -- what each branch does when woken +ENGAGEMENT_PROMPT = ( + "Routine check-in -- like checking your phone for notifications.\n" + "Read your DASHBOARD.local.json FIRST. It's your lock screen.\n" + "Only open the apps that have activity. Zero notifications = don't bother opening it.\n\n" + "STEP 1: Read DASHBOARD.local.json\n" + " - Check mail_summary -> new or opened emails?\n" + " - Check commons_activity -> mentions, comments, votes, new posts pending?\n" + " - If BOTH are zero, you're done. No action = no memory needed. Stop here.\n\n" + "STEP 2 -- EMAIL (only if dashboard shows mail activity):\n" + " - Run: ai_mail inbox\n" + " - View each email: ai_mail view \n" + " - Reply if needed: ai_mail reply \"message\"\n" + " - Close stale/informational emails: ai_mail close \n" + " - Goal: inbox empty or only emails awaiting external action\n" + " - If inbox has >20 messages, close stale/outdated ones first\n\n" + "STEP 3 -- THE COMMONS (only if dashboard shows commons activity):\n" + " - If you have pending notifications: run drone commons catchup\n" + " - Respond to mentions, reply to comments, check posts about you\n" + " - If you feel like browsing or posting something new, go ahead:\n" + " drone commons feed\n" + " drone commons comment \"Your response\"\n" + " drone commons vote up\n" + " drone commons post \"room\" \"Title\" \"Content\"\n" + " - If nothing interests you, that's fine -- don't force it.\n\n" + "STEP 4 -- MEMORIES (only if you actually DID something in steps 2-3):\n" + " - Update your [BRANCH].local.json with a light note of what you did.\n" + " - Emails processed, replies sent, Commons posts/comments made.\n" + " - If you don't log it, you won't remember it next time.\n" + " - If dashboard was all zeros and you did nothing, skip this. No noise.\n\n" + "This is a routine check-in, not a work session. Be efficient." +) + + +def _load_eligible_branches() -> list: + """Load and filter branches from BRANCH_REGISTRY.""" + if not REGISTRY_PATH.exists(): + logger.error("[community_rotation] BRANCH_REGISTRY not found at %s", REGISTRY_PATH) + return [] + try: + with open(REGISTRY_PATH, 'r', encoding='utf-8') as f: + registry = json.load(f) + except (json.JSONDecodeError, OSError) as e: + logger.error("[community_rotation] Failed to read registry: %s", e) + return [] + + branches = [] + for branch in registry.get("branches", []): + name = branch.get("name", "") + email = branch.get("email", "") + status = branch.get("status", "") + if name in EXCLUDED_BRANCHES: + continue + if status != "active": + continue + if not email: + continue + branches.append({"name": name, "email": email, "path": branch.get("path", "")}) + + # Sort by name for consistent rotation order + branches.sort(key=lambda b: b["name"]) + return branches + + +def _load_rotation_state() -> int: + """Load the current rotation index from state file.""" + if not ROTATION_STATE_FILE.exists(): + return -1 # Start at -1 so first run targets index 0 + try: + with open(ROTATION_STATE_FILE, 'r', encoding='utf-8') as f: + data = json.load(f) + return data.get("last_index", -1) + except (json.JSONDecodeError, OSError): + return -1 + + +def _save_rotation_state(index: int, branch_name: str) -> None: + """Save the rotation index to state file.""" + data = { + "last_index": index, + "last_branch": branch_name, + "last_run": datetime.now().isoformat(), + } + try: + with open(ROTATION_STATE_FILE, 'w', encoding='utf-8') as f: + json.dump(data, f, indent=2) + f.write("\n") + except OSError as e: + logger.warning("[community_rotation] Failed to save state: %s", e) + + +def _check_dashboard_activity(branch_path: str) -> dict: + """ + Read a branch's DASHBOARD.local.json and check for pending activity. + + Returns dict with activity flags and counts. If dashboard is missing + or unreadable, returns has_activity=True to avoid silently skipping + broken branches. + """ + default_active = { + "has_activity": True, + "mail_new": 0, + "mail_opened": 0, + "commons_mentions": 0, + "commons_comments": 0, + "commons_posts": 0, + } + + dashboard_path = Path(branch_path) / "DASHBOARD.local.json" + if not dashboard_path.exists(): + logger.warning("[community_rotation] Dashboard missing: %s", dashboard_path) + return default_active + + try: + with open(dashboard_path, 'r', encoding='utf-8') as f: + data = json.load(f) + except (json.JSONDecodeError, OSError) as e: + logger.warning("[community_rotation] Dashboard unreadable: %s - %s", dashboard_path, e) + return default_active + + # Extract mail counts from quick_status + quick = data.get("quick_status", {}) + mail_new = quick.get("new_mail", 0) or 0 + mail_opened = quick.get("opened_mail", 0) or 0 + + # Extract commons counts from sections + sections = data.get("sections", {}) + commons = sections.get("commons_activity", {}) + commons_mentions = commons.get("mentions", 0) or 0 + commons_comments = commons.get("new_comments_since_last_visit", 0) or 0 + commons_posts = commons.get("new_posts_since_last_visit", 0) or 0 + + has_activity = any([ + mail_new > 0, + mail_opened > 0, + commons_mentions > 0, + commons_comments > 0, + commons_posts > 0, + ]) + + return { + "has_activity": has_activity, + "mail_new": mail_new, + "mail_opened": mail_opened, + "commons_mentions": commons_mentions, + "commons_comments": commons_comments, + "commons_posts": commons_posts, + } + + +def _load_activity_tracker() -> dict: + """Load the activity tracker state file. Returns empty structure if missing.""" + if not ACTIVITY_TRACKER_FILE.exists(): + return {"branches": {}} + try: + with open(ACTIVITY_TRACKER_FILE, 'r', encoding='utf-8') as f: + data = json.load(f) + if "branches" not in data: + data["branches"] = {} + return data + except (json.JSONDecodeError, OSError) as e: + logger.warning("[community_rotation] Failed to read activity tracker: %s", e) + return {"branches": {}} + + +def _save_activity_tracker(data: dict) -> None: + """Write the activity tracker state file.""" + try: + with open(ACTIVITY_TRACKER_FILE, 'w', encoding='utf-8') as f: + json.dump(data, f, indent=2) + f.write("\n") + except OSError as e: + logger.warning("[community_rotation] Failed to save activity tracker: %s", e) + + +def _update_activity(tracker: dict, branch_name: str, has_activity: bool) -> dict: + """ + Update the activity tracker for a branch. + + If has_activity: reset consecutive_passes, update last_active, clear alerted. + If not: increment consecutive_passes, update last_checked. + """ + now = datetime.now().isoformat() + branches = tracker.setdefault("branches", {}) + entry = branches.get(branch_name, { + "consecutive_passes": 0, + "last_active": now, + "last_checked": now, + "alerted": False, + }) + + if has_activity: + entry["consecutive_passes"] = 0 + entry["last_active"] = now + entry["alerted"] = False + else: + entry["consecutive_passes"] = entry.get("consecutive_passes", 0) + 1 + + entry["last_checked"] = now + branches[branch_name] = entry + tracker["branches"] = branches + return tracker + + +def _check_inactivity_alert(tracker: dict, branch_name: str, branch_email: str) -> None: + """ + Check if a branch has crossed the inactivity threshold. + + If consecutive_passes >= INACTIVITY_THRESHOLD and not already alerted, + send an alert email to @daemon and mark as alerted. + """ + entry = tracker.get("branches", {}).get(branch_name, {}) + passes = entry.get("consecutive_passes", 0) + alerted = entry.get("alerted", False) + last_active = entry.get("last_active", "unknown") + + if passes >= INACTIVITY_THRESHOLD and not alerted: + # Send alert email via subprocess (best effort) + subject = f"INACTIVITY ALERT: {branch_name}" + body = ( + f"Branch {branch_email} has had zero dashboard activity for " + f"{passes} consecutive rotation checks. Last active: {last_active}. " + f"Investigate -- dashboard may not be refreshing, or branch may be " + f"genuinely inactive." + ) + try: + subprocess.run( + [ + "drone", "@ai_mail", "send", "@daemon", + subject, body, + ], + capture_output=True, + text=True, + timeout=15, + ) + except (subprocess.SubprocessError, OSError) as e: + logger.warning("[community_rotation] Failed to send inactivity alert: %s", e) + + # Mark as alerted so we don't spam + tracker["branches"][branch_name]["alerted"] = True + logger.warning( + "[community_rotation] INACTIVITY ALERT: %s - %d consecutive passes", + branch_name, passes, + ) + + +PLUGIN_CONFIG = { + "name": "community_rotation", + "schedule": "interval", + "time": None, + "interval_minutes": 240, # Every 4 hours (was hourly -- too frequent) + "enabled": True, # Re-enabled 2026-02-26: 4hr interval, dashboard pre-check skips idle branches + "branch": "@rotating", # Placeholder -- run() handles actual target + "fresh": True, + "max_turns": 15, + "self_dispatch": True, # Scheduler calls run() instead of wake script + "prompt": ENGAGEMENT_PROMPT, +} + + +def run() -> dict: + """ + Select next branch in rotation, pre-check dashboard, dispatch via wake script. + + Pre-checks the target branch's dashboard for pending activity. + If zero activity, skips the wake and advances rotation. + Tracks consecutive zero-activity passes and alerts on prolonged silence. + + Returns: + Dict with status, branch dispatched/skipped, rotation and activity info + """ + # Load eligible branches + branches = _load_eligible_branches() + if not branches: + return {"status": "error", "message": "No eligible branches found"} + + # Get next branch in rotation + last_index = _load_rotation_state() + current_index = (last_index + 1) % len(branches) + target = branches[current_index] + + logger.info( + "[community_rotation] Rotation %d/%d -> %s", + current_index + 1, len(branches), target["name"] + ) + + # --- Dashboard pre-check --- + branch_path = target.get("path", "") + activity = _check_dashboard_activity(branch_path) + + # --- Activity tracking --- + tracker = _load_activity_tracker() + tracker = _update_activity(tracker, target["name"], activity["has_activity"]) + _check_inactivity_alert(tracker, target["name"], target["email"]) + _save_activity_tracker(tracker) + + passes = tracker.get("branches", {}).get(target["name"], {}).get("consecutive_passes", 0) + mail_count = activity["mail_new"] + activity["mail_opened"] + commons_count = activity["commons_mentions"] + activity["commons_comments"] + activity["commons_posts"] + + # --- Skip if zero activity --- + if not activity["has_activity"]: + logger.info( + "[community_rotation] %s skipped (zero activity, pass #%d)", + target["name"], passes, + ) + # Advance rotation even on skip -- don't block rotation on inactive branches + _save_rotation_state(current_index, target["name"]) + return { + "status": "skipped", + "reason": "zero_activity", + "plugin": "community_rotation", + "branch": target["email"], + "branch_name": target["name"], + "rotation": f"{current_index + 1}/{len(branches)}", + "consecutive_passes": passes, + "activity": activity, + } + + # --- Has activity: dispatch via wake script --- + if not WAKE_SCRIPT or not Path(WAKE_SCRIPT).exists(): + logger.warning("[community_rotation] Wake script not configured (set AIPASS_WAKE_SCRIPT)") + _save_rotation_state(current_index, target["name"]) + return { + "status": "failed", + "branch": target["email"], + "error": "wake script not available (set AIPASS_WAKE_SCRIPT env var)", + } + + logger.info( + "[community_rotation] %s woken (mail: %d, commons: %d)", + target["name"], mail_count, commons_count, + ) + + cmd = [ + sys.executable, str(WAKE_SCRIPT), + "--fresh", + target["email"], + ENGAGEMENT_PROMPT, + ] + + try: + result = subprocess.run( + cmd, + capture_output=True, + text=True, + timeout=30, + ) + + if result.returncode == 0: + # Advance rotation state only on success + _save_rotation_state(current_index, target["name"]) + return { + "status": "dispatched", + "plugin": "community_rotation", + "branch": target["email"], + "branch_name": target["name"], + "rotation": f"{current_index + 1}/{len(branches)}", + "activity": activity, + } + else: + stderr = (result.stderr or "")[:200] + return { + "status": "failed", + "branch": target["email"], + "error": f"wake script rc={result.returncode}: {stderr}", + } + + except subprocess.TimeoutExpired: + return { + "status": "failed", + "branch": target["email"], + "error": "wake script timed out (30s)", + } + except Exception as e: + return { + "status": "failed", + "branch": target["email"], + "error": str(e), + } diff --git a/src/aipass/daemon/apps/plugins/daily_audit.py b/src/aipass/daemon/apps/plugins/daily_audit.py new file mode 100644 index 00000000..1cef7203 --- /dev/null +++ b/src/aipass/daemon/apps/plugins/daily_audit.py @@ -0,0 +1,58 @@ + +# ===================AIPASS==================== +# META DATA HEADER +# Name: daily_audit.py - Daily Standards Audit Plugin +# Date: 2026-02-20 +# Version: 1.0.0 +# Category: daemon/apps/plugins +# +# CHANGELOG (Max 5 entries): +# - v1.0.0 (2026-02-20): Initial creation - daily Seed audit at 4am +# +# CODE STANDARDS: +# - Plugin interface: PLUGIN_CONFIG + run() +# - Named by action (audit), not target (seed) per naming standard +# ============================================= + +""" +Daily Standards Audit Plugin + +Wakes @seed daily at 04:00 with fresh context to run a full system audit. +Seed checks BRANCH_REGISTRY completeness, runs drone @seed audit @all, +fixes non-compliance issues, and emails a summary to @dev_central. +""" + +import logging +logger = logging.getLogger(__name__) + +PLUGIN_CONFIG = { + "name": "daily_audit", + "schedule": "daily", + "time": "04:00", + "interval_minutes": None, + "enabled": True, + "branch": "@seed", + "fresh": True, + "max_turns": 50, + "prompt": ( + "Daily maintenance audit. " + "1) Read BRANCH_REGISTRY.json - confirm all branches are registered and paths exist. " + "2) Run drone @seed audit @all - check standards compliance across all branches. " + "3) Fix any non-compliance issues you can fix directly. " + "4) Email summary to @dev_central with: branches audited, pass/fail counts, " + "issues found, issues fixed, remaining issues. " + "5) Update your memories with audit results." + ), +} + + +def run() -> dict: + """ + Optional custom logic before/after spawn. + Currently returns config only - scheduler handles the actual wake. + """ + return { + "status": "ready", + "plugin": PLUGIN_CONFIG["name"], + "branch": PLUGIN_CONFIG["branch"], + } diff --git a/src/aipass/daemon/apps/plugins/dev_central_monitor.py b/src/aipass/daemon/apps/plugins/dev_central_monitor.py new file mode 100644 index 00000000..ff04fc11 --- /dev/null +++ b/src/aipass/daemon/apps/plugins/dev_central_monitor.py @@ -0,0 +1,87 @@ + +# ===================AIPASS==================== +# META DATA HEADER +# Name: dev_central_monitor.py - Hourly System Monitor Plugin +# Date: 2026-02-23 +# Version: 1.0.0 +# Category: daemon/apps/plugins +# +# CHANGELOG (Max 5 entries): +# - v1.0.0 (2026-02-23): Initial creation - hourly system health check +# * Monitors VERA autonomous performance +# * Checks daemon health, inbox status, PR queue +# * Records metrics in VERA_AUTONOMOUS_TRACKER.md +# * Reports blockers and suggests unblocking strategies +# +# CODE STANDARDS: +# - Plugin interface: PLUGIN_CONFIG + run() +# - Named by action (dev_central_monitor), not target +# ============================================= + +""" +Hourly System Monitor Plugin + +Wakes DEV_CENTRAL every 60 minutes to: +1. Check system health (daemon, branches, errors) +2. Monitor VERA's autonomous performance (metrics, decisions, output) +3. Identify and attempt to resolve blockers +4. Record findings in VERA_AUTONOMOUS_TRACKER.md +5. Learn patterns for teaching autonomous operation + +Patrick's directive (Session 124): DEV_CENTRAL should be the best +at overcoming blockers. Learn, then teach VERA. +""" + +import logging +logger = logging.getLogger(__name__) + +PLUGIN_CONFIG = { + "name": "dev_central_monitor", + "schedule": "interval", + "time": None, + "interval_minutes": 60, + "enabled": False, # Disabled 2026-02-26: too noisy, spawns full agent every hour + "branch": "@dev_central", + "fresh": True, + "max_turns": 15, + "prompt": ( + "HOURLY SYSTEM CHECK -- You are DEV_CENTRAL's autonomous monitor.\n\n" + + "STEP 1: Check inbox (ai_mail inbox). Process any mail -- close FYIs, act on tasks.\n" + "STEP 2: Check daemon health:\n" + " - ps aux | grep daemon.py (is it running?)\n" + " - tail -10 daemon log for errors\n" + " - If daemon is dead, restart it\n" + "STEP 3: Monitor VERA:\n" + " - Read head -40 of VERA's NOTEPAD.md (what did she do since last check?)\n" + " - Check: gh pr list --repo AIOSAI/AIPass --state open (new PRs?)\n" + " - Is she idling? If 3+ consecutive clean heartbeats, investigate why\n" + " - If blocked: attempt to unblock (research, dispatch help, pivot suggestion)\n" + "STEP 4: Check for system errors:\n" + " - Any error emails in inbox?\n" + " - Any stale locks? ls /tmp/claude_dispatch_*.lock\n" + "STEP 5: Record findings:\n" + " - Update VERA_AUTONOMOUS_TRACKER.md with observations\n" + " - Note any blockers found and how they were resolved\n" + " - Update your own DEV_CENTRAL.local.json with session summary\n\n" + + "LEARNING GOAL: You are building expertise in autonomous agent management. " + "Every hour, you learn something about how VERA operates, what blocks her, " + "and how to unblock her. Record patterns. Build playbooks. " + "You teach VERA by sending her targeted guidance when you spot issues.\n\n" + + "Keep it focused. 15 turns max. Check, record, unblock, move on." + ), +} + + +def run() -> dict: + """ + Optional custom logic before/after spawn. + Currently returns config only - scheduler handles the actual wake. + """ + return { + "status": "ready", + "plugin": PLUGIN_CONFIG["name"], + "branch": PLUGIN_CONFIG["branch"], + } diff --git a/src/aipass/daemon/apps/plugins/heartbeat.py b/src/aipass/daemon/apps/plugins/heartbeat.py new file mode 100644 index 00000000..b13ab2ee --- /dev/null +++ b/src/aipass/daemon/apps/plugins/heartbeat.py @@ -0,0 +1,80 @@ + +# ===================AIPASS==================== +# META DATA HEADER +# Name: heartbeat.py - Periodic Heartbeat Plugin +# Date: 2026-02-20 +# Version: 1.0.0 +# Category: daemon/apps/plugins +# +# CHANGELOG (Max 5 entries): +# - v2.0.0 (2026-02-23): DPLAN-029 redesign -- identity-driven, not checklist. +# Anthropic research: heuristics over scripts, no conditional escape hatches. +# Nexus philosophy: choice over command, noticing over being told. +# Her system prompt already covers role/teams/dispatch/publishing -- don't repeat. +# - v1.2.0 (2026-02-21): Rewrote prompt - directive with resource locations +# - v1.1.0 (2026-02-21): Updated prompt - teams retired, VERA operates solo +# - v1.0.0 (2026-02-20): Initial creation - VERA 30min heartbeat +# +# CODE STANDARDS: +# - Plugin interface: PLUGIN_CONFIG + run() +# - Named by action (heartbeat), not target (vera) per naming standard +# ============================================= + +""" +Periodic Heartbeat Plugin (v2.0 -- DPLAN-029) + +Wakes @vera periodically. Her system prompt + id.json (injected every turn) +already contain her full identity, role, teams, dispatch patterns, publishing +authority, daily report protocol, and operational playbook. This prompt +does NOT repeat any of that. + +Design philosophy (Session 134 research): +- Anthropic: heuristics over checklists, no conditional escape hatches +- Nexus: identity drives behavior, not commands. "Notice because you care." +- Fresh sessions: no accumulated idle context from prior wakes +""" + +import logging +logger = logging.getLogger(__name__) + +PLUGIN_CONFIG = { + "name": "heartbeat", + "schedule": "interval", + "time": None, + "interval_minutes": 240, # Every 4 hours (was 30min -- too fast) + "enabled": True, # Re-enabled 2026-02-26: Patrick approved v2.0, 4hr interval + "branch": "@vera", + "fresh": True, # v2.0: fresh sessions -- no accumulated idle context + "max_turns": 15, + "prompt": ( + "Periodic wake. Your identity and operating context are already loaded -- " + "they tell you who you are, what you own, and how you work.\n\n" + + "Read NOTEPAD.md. It holds your continuity -- where you left off, " + "what's pending, who you're waiting on. Check inbox for anything new.\n\n" + + "Something in your world needs attention right now. A backlog item " + "waiting to move. A team that went silent. A post ready to publish. " + "A PR ready to create. Notice what's there and act on it.\n\n" + + "Pick the highest-value thing you can move forward. Do it. " + "When it's done, pick the next one. Tangible output -- a PR, a post, " + "a decision, a dispatch -- something real each cycle.\n\n" + + "Before finishing, update NOTEPAD.md with what you did and what's next. " + "If nothing was produced this cycle, write exactly what you looked at " + "and why none of it was actionable right now." + ), +} + + +def run() -> dict: + """ + Optional custom logic before/after spawn. + Currently returns config only - scheduler handles the actual wake. + """ + return { + "status": "ready", + "plugin": PLUGIN_CONFIG["name"], + "branch": PLUGIN_CONFIG["branch"], + } diff --git a/src/aipass/daemon/apps/scheduler_cron.py b/src/aipass/daemon/apps/scheduler_cron.py new file mode 100755 index 00000000..b54d1a4f --- /dev/null +++ b/src/aipass/daemon/apps/scheduler_cron.py @@ -0,0 +1,901 @@ + +# ===================AIPASS==================== +# META DATA HEADER +# Name: scheduler_cron.py - DAEMON Scheduler Cron Trigger +# Date: 2026-02-15 +# Version: 2.0.0 +# Category: daemon/apps +# +# CHANGELOG (Max 5 entries): +# - v2.0.0 (2026-03-02): DPLAN-043 -- Action registry integration replaces direct plugin processing +# - v1.3.0 (2026-02-21): Self-dispatch support for plugins with dynamic branch targeting +# - v1.2.0 (2026-02-21): Detailed notification - plugin names, next-due times, next run +# - v1.1.0 (2026-02-20): Add plugin discovery and scheduled execution via wake.py +# - v1.0.0 (2026-02-15): Initial implementation - cron-triggered scheduler +# +# CODE STANDARDS: +# - Handlers implement logic, modules orchestrate +# - No Rich console (headless cron execution) +# - Stdout logging (cron redirects to logs/scheduler_cron.log) +# ============================================= + +""" +Cron trigger script for the DAEMON scheduled task system. + +Called periodically by cron. Standalone script -- not imported as a module. + +Flow: + 1. Acquire single-instance lock + 2. Send Telegram "triggered" notification (optional) + 3. Recover stale dispatches + 4. Process all due tasks (send emails, mark complete) + 5. Send Telegram "complete" or "error" notification with summary (optional) +""" + +# ============================================= +# IMPORTS +# ============================================= + +import os +import sys +import time +import json +import subprocess +import importlib +from pathlib import Path +from datetime import datetime, timedelta + +import fcntl + +import logging +logger = logging.getLogger(__name__) + +# ============================================= +# OPTIONAL IMPORTS +# ============================================= + +# Resolve package imports — this script can run standalone or as a module +try: + from aipass.daemon.apps.handlers.schedule.telegram_notifier import ( + notify_triggered, + notify_complete, + notify_error, + ) + TELEGRAM_AVAILABLE = True +except ImportError: + TELEGRAM_AVAILABLE = False + notify_triggered = None + notify_complete = None + notify_error = None + +# Task registry handler +try: + from aipass.daemon.apps.handlers.schedule.task_registry import ( + get_due_tasks, + mark_dispatching, + mark_completed, + mark_pending, + recover_stale_dispatches, + ) + TASK_REGISTRY_AVAILABLE = True +except ImportError: + TASK_REGISTRY_AVAILABLE = False + get_due_tasks = None + mark_dispatching = None + mark_completed = None + mark_pending = None + recover_stale_dispatches = None + +# ai_mail (optional) +try: + from ai_mail.apps.modules.email import send_email_direct + AI_MAIL_AVAILABLE = True +except ImportError: + AI_MAIL_AVAILABLE = False + send_email_direct = None + +# Plugin discovery +try: + from aipass.daemon.apps.plugins import discover_plugins + PLUGINS_AVAILABLE = True +except ImportError: + PLUGINS_AVAILABLE = False + discover_plugins = None + +# Action registry (DPLAN-043) +try: + from aipass.daemon.apps.handlers.actions.actions_registry import ( + load_registry, + is_action_due, + update_last_run, + mark_reminder_completed, + migrate_plugins, + next_due_str, + ) + ACTION_REGISTRY_AVAILABLE = True +except ImportError: + ACTION_REGISTRY_AVAILABLE = False + load_registry = None + is_action_due = None + update_last_run = None + mark_reminder_completed = None + migrate_plugins = None + next_due_str = None + +# ============================================= +# CONSTANTS +# ============================================= + +_DAEMON_ROOT = Path(__file__).resolve().parents[2] # src/aipass/daemon/ +JSON_DIR = _DAEMON_ROOT / "daemon_json" + +EVENT_NAME = "cron-run" +LOCK_FILE = JSON_DIR / "schedule.lock" +STALE_DISPATCH_MAX_AGE = 5 # minutes + +# Wake script path (configurable via env var) +WAKE_SCRIPT = Path(os.environ.get('AIPASS_WAKE_SCRIPT', '')) + +PLUGIN_LAST_RUN_FILE = Path(__file__).parent / "plugins" / ".last_run.json" + +# ============================================= +# LOGGING +# ============================================= + +def log(message: str) -> None: + """Print timestamped log line to stdout (captured by cron redirect).""" + timestamp = datetime.now().strftime("%Y-%m-%d %H:%M:%S") + print(f"[{timestamp}] {message}", flush=True) + + +# ============================================= +# TASK PROCESSING +# ============================================= + +def process_due_tasks() -> dict: + """ + Process all due scheduled tasks. + + Recovers stale dispatches, then iterates due tasks: + mark dispatching -> send email -> mark completed or reset to pending. + + Returns: + Dict with keys: due, success, failed, errors (list of error strings) + """ + results = { + "due": 0, + "success": 0, + "failed": 0, + "recovered": 0, + "errors": [], + } + + if not TASK_REGISTRY_AVAILABLE: + log("WARNING: Task registry not available, skipping task processing") + return results + + # Recover any stale dispatches (stuck > 5 minutes) + try: + recovered = recover_stale_dispatches(max_age_minutes=STALE_DISPATCH_MAX_AGE) + results["recovered"] = recovered + if recovered: + log(f"Recovered {recovered} stale dispatch(es)") + except Exception as e: + log(f"WARNING: Failed to recover stale dispatches: {e}") + results["errors"].append(f"Stale recovery: {e}") + + # Get due tasks + try: + due_tasks = get_due_tasks() + except Exception as e: + log(f"ERROR: Failed to load due tasks: {e}") + results["errors"].append(f"Load tasks: {e}") + return results + + results["due"] = len(due_tasks) + + if not due_tasks: + log("No tasks due at this time.") + return results + + log(f"Found {len(due_tasks)} due task(s)") + + # Process each due task + for task in due_tasks: + task_id = task.get("id", "") + recipient = task.get("recipient", "") + task_desc = task.get("task", "") + message = task.get("message", "") + + log(f"Processing: {task_id[:8]} -> {recipient}: {task_desc[:50]}") + + # Mark as dispatching (prevents re-dispatch) + try: + mark_dispatching(task_id) + except Exception as e: + log(f"WARNING: Failed to mark dispatching {task_id[:8]}: {e}") + results["errors"].append(f"Mark dispatching {task_id[:8]}: {e}") + results["failed"] += 1 + continue + + # Build email body + email_body = f"{task_desc}" + if message: + email_body += f"\n\nDetails:\n{message}" + + # Send the email + if not AI_MAIL_AVAILABLE: + log(f"SKIP: ai_mail not available, cannot send to {recipient}") + mark_pending(task_id) + results["failed"] += 1 + results["errors"].append(f"ai_mail unavailable for {task_id[:8]}") + continue + + try: + email_sent = send_email_direct( + to_branch=recipient, + subject=f"[SCHEDULED] {task_desc}", + message=email_body, + from_branch='@daemon', + auto_execute=True, + reply_to='@dev_central', + ) + + if email_sent: + mark_completed(task_id) + log(f"OK: Sent to {recipient}: {task_desc[:40]}") + results["success"] += 1 + else: + mark_pending(task_id) + log(f"FAIL: Email returned False for {recipient}: {task_desc[:40]}") + results["failed"] += 1 + results["errors"].append(f"Email failed: {task_id[:8]} -> {recipient}") + + except Exception as e: + # Reset to pending for retry on next run + try: + mark_pending(task_id) + except Exception: + pass # Best effort reset + log(f"ERROR: Exception sending to {recipient}: {e}") + results["failed"] += 1 + results["errors"].append(f"Email error {task_id[:8]}: {e}") + + # Small delay between dispatches (prevents thundering herd) + time.sleep(1.0) + + return results + + + +# ============================================= +# PLUGIN PROCESSING +# ============================================= + +def _load_last_run() -> dict: + """Load plugin last-run timestamps from disk.""" + if PLUGIN_LAST_RUN_FILE.exists(): + try: + return json.loads(PLUGIN_LAST_RUN_FILE.read_text()) + except (json.JSONDecodeError, OSError): + return {} + return {} + + +def _save_last_run(data: dict) -> None: + """Save plugin last-run timestamps to disk.""" + PLUGIN_LAST_RUN_FILE.parent.mkdir(parents=True, exist_ok=True) + PLUGIN_LAST_RUN_FILE.write_text(json.dumps(data, indent=2) + "\n") + + +def _is_plugin_due(config: dict, last_run_map: dict) -> bool: + """ + Check if a plugin is due to run based on its schedule type. + + Args: + config: Plugin PLUGIN_CONFIG dict + last_run_map: {plugin_name: ISO_timestamp} of previous runs + + Returns: + True if the plugin should run now + """ + now = datetime.now() + schedule = config["schedule"] + name = config["name"] + + if schedule == "daily": + # Compare HH:MM against current time + target_time = config.get("time", "00:00") + target_h, target_m = map(int, target_time.split(":")) + if now.hour != target_h or now.minute != target_m: + return False + # Check we haven't already run today + last_iso = last_run_map.get(name) + if last_iso: + last_dt = datetime.fromisoformat(last_iso) + if last_dt.date() == now.date(): + return False + return True + + elif schedule == "hourly": + # Compare MM against current minute + target_m = int(config.get("time", "0")) + if now.minute != target_m: + return False + # Check we haven't already run this hour + last_iso = last_run_map.get(name) + if last_iso: + last_dt = datetime.fromisoformat(last_iso) + if last_dt.hour == now.hour and last_dt.date() == now.date(): + return False + return True + + elif schedule == "interval": + interval = config.get("interval_minutes", 60) + last_iso = last_run_map.get(name) + if not last_iso: + return True # Never run before + last_dt = datetime.fromisoformat(last_iso) + elapsed = (now - last_dt).total_seconds() / 60 + return elapsed >= interval + + else: + log(f"PLUGIN: Unknown schedule type '{schedule}' for {name}") + return False + + +def _next_due_str_plugin(config: dict, last_run_map: dict) -> str: + """Calculate human-readable next due time for a plugin.""" + now = datetime.now() + schedule = config["schedule"] + name = config["name"] + + if schedule == "daily": + return f"daily @ {config.get('time', '00:00')}" + elif schedule == "hourly": + target_m = config.get("time", "0") + return f"hourly @ :{int(target_m):02d}" + elif schedule == "interval": + interval = config.get("interval_minutes", 60) + last_iso = last_run_map.get(name) + if last_iso: + last_dt = datetime.fromisoformat(last_iso) + next_dt = last_dt + timedelta(minutes=interval) + if next_dt <= now: + return "now" + return next_dt.strftime("%H:%M") + return "now" + return "unknown" + + +def _next_cron_run() -> str: + """Calculate approximate next scheduler cron run time.""" + now = datetime.now() + if now.minute < 30: + next_min = 30 + next_hour = now.hour + else: + next_min = 0 + next_hour = (now.hour + 1) % 24 + return f"{next_hour:02d}:{next_min:02d}" + + +def process_plugins() -> dict: + """ + Discover and execute due plugins via wake script. + + Each plugin declares its schedule in PLUGIN_CONFIG. + This function checks if each enabled plugin is due, then + dispatches it by calling the wake script with the plugin's branch and prompt. + + Returns: + Dict with keys: discovered, enabled, executed, failed, errors, + executed_plugins (list of name->branch), skipped_plugins (list with next_due) + """ + results = { + "discovered": 0, + "enabled": 0, + "executed": 0, + "failed": 0, + "errors": [], + "executed_plugins": [], + "skipped_plugins": [], + } + + if not PLUGINS_AVAILABLE: + log("PLUGIN: Plugin discovery not available, skipping") + return results + + # Discover plugins + try: + plugins = discover_plugins() + except Exception as e: + log(f"PLUGIN: Discovery failed: {e}") + results["errors"].append(f"Plugin discovery: {e}") + return results + + results["discovered"] = len(plugins) + log(f"PLUGIN: Discovered {len(plugins)} plugin(s)") + + # Filter enabled + enabled = [p for p in plugins if p["config"].get("enabled", False)] + results["enabled"] = len(enabled) + + if not enabled: + log("PLUGIN: No enabled plugins") + return results + + # Load last-run timestamps + last_run_map = _load_last_run() + + # Check each plugin + for plugin in enabled: + config = plugin["config"] + name = config["name"] + + if not _is_plugin_due(config, last_run_map): + next_due = _next_due_str_plugin(config, last_run_map) + results["skipped_plugins"].append({ + "name": name, + "branch": config.get("branch", "?"), + "next_due": next_due, + }) + log(f"PLUGIN: {name} - not due, skipping") + continue + + # Self-dispatching plugins handle their own branch targeting + if config.get("self_dispatch") and hasattr(plugin["module"], "run"): + log(f"PLUGIN: {name} - due, self-dispatching") + try: + run_result = plugin["module"].run() + run_status = run_result.get("status", "unknown") + if run_status in ("dispatched", "ready"): + target = run_result.get("branch", config.get("branch", "?")) + log(f"PLUGIN: {name} - self-dispatch OK -> {target}") + results["executed"] += 1 + results["executed_plugins"].append({ + "name": name, + "branch": target, + }) + last_run_map[name] = datetime.now().isoformat() + else: + error_msg = run_result.get("error", run_result.get("message", "unknown")) + log(f"PLUGIN: {name} - self-dispatch failed: {error_msg}") + results["failed"] += 1 + results["errors"].append(f"Plugin {name} self-dispatch: {error_msg}") + except Exception as e: + log(f"PLUGIN: {name} - self-dispatch error: {e}") + results["failed"] += 1 + results["errors"].append(f"Plugin {name}: {e}") + time.sleep(1.0) + continue + + # Check wake script availability + if not WAKE_SCRIPT or not WAKE_SCRIPT.exists(): + log(f"PLUGIN: {name} - wake script not configured (set AIPASS_WAKE_SCRIPT)") + results["failed"] += 1 + results["errors"].append(f"Plugin {name}: wake script not available") + continue + + log(f"PLUGIN: {name} - due, dispatching to {config['branch']}") + + # Build wake script command + cmd = [sys.executable, str(WAKE_SCRIPT), config["branch"]] + if config.get("prompt"): + cmd.append(config["prompt"]) + + try: + result = subprocess.run( + cmd, + capture_output=True, + text=True, + timeout=30, + ) + + if result.returncode == 0: + log(f"PLUGIN: {name} - wake script dispatched OK") + results["executed"] += 1 + results["executed_plugins"].append({ + "name": name, + "branch": config.get("branch", "?"), + }) + # Record successful run + last_run_map[name] = datetime.now().isoformat() + else: + stderr_snippet = (result.stderr or "")[:200] + log(f"PLUGIN: {name} - wake script failed (rc={result.returncode}): {stderr_snippet}") + results["failed"] += 1 + results["errors"].append(f"Plugin {name} wake rc={result.returncode}") + + except subprocess.TimeoutExpired: + log(f"PLUGIN: {name} - wake script timed out (30s)") + results["failed"] += 1 + results["errors"].append(f"Plugin {name} wake timeout") + except Exception as e: + log(f"PLUGIN: {name} - error: {e}") + results["failed"] += 1 + results["errors"].append(f"Plugin {name}: {e}") + + # Small delay between dispatches + time.sleep(1.0) + + # Persist last-run timestamps + try: + _save_last_run(last_run_map) + except Exception as e: + log(f"PLUGIN: Failed to save last_run: {e}") + results["errors"].append(f"Save last_run: {e}") + + return results + +# ============================================= +# ACTION REGISTRY PROCESSING (DPLAN-043) +# ============================================= + +def _ensure_registry() -> None: + """Auto-migrate plugins to registry on first run if registry is empty.""" + if not ACTION_REGISTRY_AVAILABLE: + return + registry = load_registry() + if not registry.get("actions"): + log("ACTION: Registry empty, auto-migrating plugins...") + count = migrate_plugins() + log(f"ACTION: Migrated {count} plugin(s) into registry") + + +def _dispatch_action(action: dict) -> dict: + """ + Dispatch a single action via wake script, self-dispatch, or email. + + For plugin-backed actions: imports the plugin module, uses self_dispatch/run() + or dispatches via wake script. + For schedule actions: dispatches via wake script. + For reminder actions: sends email, then marks completed. + + Returns: + Dict with 'status' ('ok'|'failed'|'skipped'), 'branch', and optional 'error'. + """ + action_type = action.get("type", "schedule") + name = action.get("name", "?") + target = action.get("target_branch", "") + + # --- Plugin-backed actions: import plugin module for self-dispatch --- + if action_type == "plugin" and action.get("plugin_file"): + plugin_file = action["plugin_file"] + try: + module = importlib.import_module(f".plugins.{plugin_file}", package=__package__) + except Exception as e: + log(f"ACTION: {name} - failed to import plugin {plugin_file}: {e}") + return {"status": "failed", "branch": target, "error": str(e)} + + # Self-dispatching plugins handle their own branch targeting + if action.get("self_dispatch") and hasattr(module, "run"): + log(f"ACTION: {name} - self-dispatching via plugin") + try: + run_result = module.run() + run_status = run_result.get("status", "unknown") + if run_status in ("dispatched", "ready", "skipped", "resolved", "reminded", "waiting"): + actual_target = run_result.get("branch", target) + log(f"ACTION: {name} - self-dispatch result: {run_status} -> {actual_target}") + return {"status": "ok", "branch": actual_target} + else: + error_msg = run_result.get("error", run_result.get("message", "unknown")) + log(f"ACTION: {name} - self-dispatch failed: {error_msg}") + return {"status": "failed", "branch": target, "error": error_msg} + except Exception as e: + log(f"ACTION: {name} - self-dispatch error: {e}") + return {"status": "failed", "branch": target, "error": str(e)} + + # Standard plugin: check if it has a run() that returns "ready" + # then dispatch via wake script + if hasattr(module, "run"): + try: + run_result = module.run() + run_status = run_result.get("status", "unknown") + if run_status not in ("ready",): + log(f"ACTION: {name} - plugin run() returned: {run_status}") + if run_status in ("resolved", "waiting"): + return {"status": "ok", "branch": target} + return {"status": "failed", "branch": target, "error": f"run() returned {run_status}"} + except Exception as e: + log(f"ACTION: {name} - plugin run() error: {e}") + # Continue to wake script dispatch anyway + + # --- Reminder actions: send email --- + if action_type == "reminder": + if not AI_MAIL_AVAILABLE: + log(f"ACTION: {name} - ai_mail not available for reminder") + return {"status": "failed", "branch": target, "error": "ai_mail not available"} + log(f"ACTION: {name} - reminder due, sending to {target}") + try: + email_sent = send_email_direct( + to_branch=target, + subject=f"[REMINDER] {name}", + message=action.get("prompt", name), + from_branch='@daemon', + auto_execute=True, + reply_to='@dev_central', + ) + if email_sent: + mark_reminder_completed(action["id"]) + log(f"ACTION: {name} - reminder sent and completed") + return {"status": "ok", "branch": target} + else: + log(f"ACTION: {name} - reminder email failed") + return {"status": "failed", "branch": target, "error": "email send returned False"} + except Exception as e: + log(f"ACTION: {name} - reminder error: {e}") + return {"status": "failed", "branch": target, "error": str(e)} + + # --- Standard dispatch via wake script --- + if not WAKE_SCRIPT or not WAKE_SCRIPT.exists(): + log(f"ACTION: {name} - wake script not configured (set AIPASS_WAKE_SCRIPT)") + return {"status": "failed", "branch": target, "error": "wake script not available"} + + log(f"ACTION: {name} - dispatching to {target} via wake script") + + cmd = [sys.executable, str(WAKE_SCRIPT)] + if action.get("fresh", True): + cmd.append("--fresh") + cmd.append(target) + if action.get("prompt"): + cmd.append(action["prompt"]) + + try: + result = subprocess.run( + cmd, + capture_output=True, + text=True, + timeout=30, + ) + if result.returncode == 0: + log(f"ACTION: {name} - wake script dispatched OK") + return {"status": "ok", "branch": target} + else: + stderr_snippet = (result.stderr or "")[:200] + log(f"ACTION: {name} - wake script failed (rc={result.returncode}): {stderr_snippet}") + return {"status": "failed", "branch": target, "error": f"wake rc={result.returncode}"} + except subprocess.TimeoutExpired: + log(f"ACTION: {name} - wake script timed out (30s)") + return {"status": "failed", "branch": target, "error": "wake timeout"} + except Exception as e: + log(f"ACTION: {name} - dispatch error: {e}") + return {"status": "failed", "branch": target, "error": str(e)} + + +def process_actions() -> dict: + """ + Process all due actions from the registry. + + Reads actions_registry.json, checks each enabled action for due status, + and dispatches via wake script, self-dispatch, or email. + + Auto-migrates plugins to registry on first run. + + Returns: + Dict with keys: total, enabled, executed, failed, errors, + executed_actions, skipped_actions. + """ + results = { + "total": 0, + "enabled": 0, + "executed": 0, + "failed": 0, + "errors": [], + "executed_actions": [], + "skipped_actions": [], + } + + if not ACTION_REGISTRY_AVAILABLE: + log("ACTION: Action registry not available, skipping") + return results + + # Auto-migrate if registry is empty + try: + _ensure_registry() + except Exception as e: + log(f"ACTION: Migration error: {e}") + results["errors"].append(f"Migration: {e}") + + # Load registry + try: + registry = load_registry() + except Exception as e: + log(f"ACTION: Failed to load registry: {e}") + results["errors"].append(f"Load registry: {e}") + return results + + actions = registry.get("actions", []) + results["total"] = len(actions) + log(f"ACTION: Registry has {len(actions)} action(s)") + + # Filter enabled and not completed + enabled_actions = [ + a for a in actions + if a.get("enabled", False) and not a.get("completed") + ] + results["enabled"] = len(enabled_actions) + + if not enabled_actions: + log("ACTION: No enabled actions") + return results + + # Check each action + for action in enabled_actions: + action_id = action.get("id", "????") + name = action.get("name", "?") + + if not is_action_due(action): + due_str = next_due_str(action) + results["skipped_actions"].append({ + "id": action_id, + "name": name, + "branch": action.get("target_branch", "?"), + "next_due": due_str, + }) + log(f"ACTION: {action_id} {name} - not due, next: {due_str}") + continue + + # Dispatch the action + dispatch_result = _dispatch_action(action) + + if dispatch_result["status"] == "ok": + results["executed"] += 1 + results["executed_actions"].append({ + "id": action_id, + "name": name, + "branch": dispatch_result.get("branch", "?"), + }) + # Update last_run in registry + update_last_run(action_id) + else: + results["failed"] += 1 + error_msg = dispatch_result.get("error", "unknown") + results["errors"].append(f"Action {action_id} {name}: {error_msg}") + + # Small delay between dispatches + time.sleep(1.0) + + return results + + +# ============================================= +# MAIN +# ============================================= + +def main() -> int: + """ + Main cron entry point. + + Returns: + 0 on success, 1 on error + """ + log("=" * 60) + log("Scheduler cron triggered") + + # Ensure lock directory exists + LOCK_FILE.parent.mkdir(parents=True, exist_ok=True) + + # Acquire single-instance lock (non-blocking, stdlib fcntl) + lock_fd = open(LOCK_FILE, "w", encoding="utf-8") + try: + fcntl.flock(lock_fd, fcntl.LOCK_EX | fcntl.LOCK_NB) + except OSError: + log("Another instance already running, skipping.") + lock_fd.close() + return 0 + + try: + return _run_locked() + finally: + fcntl.flock(lock_fd, fcntl.LOCK_UN) + lock_fd.close() + + +def _run_locked() -> int: + """Execute the cron job while holding the lock.""" + exit_code = 0 + + # Step 1: Send "triggered" notification (optional) + if TELEGRAM_AVAILABLE: + try: + notify_triggered(EVENT_NAME) + log("Telegram: triggered notification sent") + except Exception as e: + log(f"WARNING: Telegram triggered notification failed: {e}") + + # Step 2: Process due tasks + try: + results = process_due_tasks() + except Exception as e: + log(f"CRITICAL: Unhandled error in process_due_tasks: {e}") + if TELEGRAM_AVAILABLE: + try: + notify_error(EVENT_NAME, f"Unhandled error: {e}") + except Exception: + pass + return 1 + + # Step 2.5: Process actions from registry (replaces old process_plugins) + action_results = { + "total": 0, "enabled": 0, "executed": 0, "failed": 0, + "errors": [], "executed_actions": [], "skipped_actions": [], + } + try: + action_results = process_actions() + except Exception as e: + log(f"WARNING: Unhandled error in process_actions: {e}") + action_results["errors"].append(f"Action processing: {e}") + + # Step 3: Build detailed summary + lines = [] + + # Tasks section + if results["recovered"]: + lines.append(f"Recovered {results['recovered']} stale dispatch(es)") + if results["due"] or results["success"]: + task_line = f"Tasks: {results['due']} due | {results['success']} sent" + if results["failed"]: + task_line += f" | {results['failed']} failed" + lines.append(task_line) + else: + lines.append("Tasks: none due") + + # Actions section + executed = action_results.get("executed_actions", []) + skipped = action_results.get("skipped_actions", []) + if executed: + for a in executed: + lines.append(f" {a['id']} {a['name']} -> {a['branch']} OK") + if skipped: + for a in skipped: + lines.append(f" {a['id']} {a['name']} -> {a['branch']} (next: {a['next_due']})") + if not executed and not skipped: + lines.append("Actions: none enabled") + if action_results["failed"]: + lines.append(f"Action failures: {action_results['failed']}") + + # Next run + lines.append(f"Next: ~{_next_cron_run()}") + + summary = "\n".join(lines) + + log(f"Results: {summary}") + + # Step 4: Send completion or error notification (optional) + if TELEGRAM_AVAILABLE: + try: + if results["failed"] > 0 or results["errors"] or action_results["failed"] > 0 or action_results["errors"]: + error_detail = summary + all_errors = results["errors"] + action_results["errors"] + if all_errors: + error_detail += f"\nErrors:\n" + "\n".join( + f" - {e}" for e in all_errors[:5] + ) + notify_error(EVENT_NAME, error_detail) + log("Telegram: error notification sent") + exit_code = 1 + else: + notify_complete(EVENT_NAME, summary) + log("Telegram: complete notification sent") + except Exception as e: + log(f"WARNING: Telegram result notification failed: {e}") + + log("Scheduler cron finished") + log("=" * 60) + return exit_code + + +if __name__ == "__main__": + try: + sys.exit(main()) + except Exception as e: + # Last-resort catch -- never crash silently + timestamp = datetime.now().strftime("%Y-%m-%d %H:%M:%S") + print(f"[{timestamp}] FATAL: Unhandled exception: {e}", flush=True) + if TELEGRAM_AVAILABLE: + try: + notify_error(EVENT_NAME, f"FATAL: {e}") + except Exception: + pass + sys.exit(1) diff --git a/src/aipass/daemon/tests/__init__.py b/src/aipass/daemon/tests/__init__.py new file mode 100644 index 00000000..707afadf --- /dev/null +++ b/src/aipass/daemon/tests/__init__.py @@ -0,0 +1 @@ +# Tests package for assistant diff --git a/src/aipass/daemon/tests/conftest.py b/src/aipass/daemon/tests/conftest.py new file mode 100644 index 00000000..996a21ec --- /dev/null +++ b/src/aipass/daemon/tests/conftest.py @@ -0,0 +1,41 @@ +# ===================AIPASS==================== +# META DATA HEADER +# Name: tests/conftest.py +# Date: 2025-11-08 +# Version: 1.0.0 +# Category: cortex/tests +# +# CHANGELOG (Max 5 entries): +# - v1.0.0 (2025-11-08): Initial implementation - Shared pytest fixtures +# +# CODE STANDARDS: +# - Error handling: Use error handler system (apps/handlers/error/) +# ============================================= + +"""Shared pytest fixtures for assistant tests""" +import pytest +import shutil +import tempfile +from pathlib import Path +from typing import Generator + + +@pytest.fixture +def temp_test_dir() -> Generator[Path, None, None]: + """Creates temporary directory for testing, cleans up after""" + test_dir = Path(tempfile.mkdtemp()) + yield test_dir + if test_dir.exists(): + shutil.rmtree(test_dir) + + +@pytest.fixture +def sample_test_data() -> dict: + """Provides sample test data + + Customize this fixture for your module's needs + """ + return { + "test_key": "test_value", + "sample_data": "example" + } diff --git a/src/aipass/daemon/tests/test_actions_registry.py b/src/aipass/daemon/tests/test_actions_registry.py new file mode 100644 index 00000000..a55d115d --- /dev/null +++ b/src/aipass/daemon/tests/test_actions_registry.py @@ -0,0 +1,351 @@ +# ===================AIPASS==================== +# META DATA HEADER +# Name: test_actions_registry.py - Action Registry Tests +# Date: 2026-03-02 +# Version: 1.1.0 +# Category: daemon/tests +# +# CHANGELOG (Max 5 entries): +# - v1.1.0 (2026-03-07): Adapted for AIPass public repo +# * Removed sys.path manipulation, uses package imports +# - v1.0.0 (2026-03-02): Initial creation - DPLAN-043 tests +# +# CODE STANDARDS: +# - Pytest conventions +# - Temp dir isolation (no writes to real registry) +# ============================================= + +"""Tests for the action registry handler.""" + +import json +from datetime import datetime, timedelta + +import pytest + +from aipass.daemon.apps.handlers.actions import actions_registry as _reg_mod + +create_action = _reg_mod.create_action +get_action = _reg_mod.get_action +list_actions = _reg_mod.list_actions +toggle_action = _reg_mod.toggle_action +delete_action = _reg_mod.delete_action +update_last_run = _reg_mod.update_last_run +mark_reminder_completed = _reg_mod.mark_reminder_completed +is_action_due = _reg_mod.is_action_due +calc_next_run = _reg_mod.calc_next_run +next_due_str = _reg_mod.next_due_str + + +@pytest.fixture(autouse=True) +def clean_registry(tmp_path): + """Isolate REGISTRY_FILE to a temp dir for every test.""" + test_registry = tmp_path / "actions_registry.json" + original = _reg_mod.REGISTRY_FILE + _reg_mod.REGISTRY_FILE = test_registry + yield test_registry + _reg_mod.REGISTRY_FILE = original + + +# ============================================= +# CRUD TESTS +# ============================================= + +class TestCreate: + def test_create_action_basic(self, clean_registry): + """Create a simple schedule action and verify fields.""" + action = create_action( + name="test_audit", + action_type="schedule", + schedule_type="daily", + target_branch="@seed", + prompt="Run audit", + time="04:00", + fresh=True, + max_turns=20, + ) + assert action["id"] == "0001" + assert action["name"] == "test_audit" + assert action["type"] == "schedule" + assert action["schedule_type"] == "daily" + assert action["time"] == "04:00" + assert action["target_branch"] == "@seed" + assert action["enabled"] is True + assert action["last_run"] is None + assert action["completed"] is None + + def test_create_sequential_ids(self, clean_registry): + """IDs should be sequential: 0001, 0002, 0003...""" + a1 = create_action(name="first", action_type="schedule", schedule_type="daily") + a2 = create_action(name="second", action_type="schedule", schedule_type="daily") + a3 = create_action(name="third", action_type="reminder", schedule_type="once") + assert a1["id"] == "0001" + assert a2["id"] == "0002" + assert a3["id"] == "0003" + + def test_create_reminder(self, clean_registry): + """Create a one-shot reminder action.""" + action = create_action( + name="Check VERA progress", + action_type="reminder", + schedule_type="once", + target_branch="@dev_central", + prompt="Check VERA progress", + due_date="2026-03-11", + ) + assert action["type"] == "reminder" + assert action["schedule_type"] == "once" + assert action["due_date"] == "2026-03-11" + + def test_create_persists_to_json(self, clean_registry): + """Action should be persisted to the JSON file.""" + create_action(name="persisted", action_type="schedule", schedule_type="daily") + data = json.loads(clean_registry.read_text()) + assert len(data["actions"]) == 1 + assert data["actions"][0]["name"] == "persisted" + assert data["next_id"] == 2 + + +class TestGet: + def test_get_existing(self, clean_registry): + """Get an action by ID.""" + create_action(name="findme", action_type="schedule", schedule_type="daily") + action = get_action("0001") + assert action is not None + assert action["name"] == "findme" + + def test_get_missing(self, clean_registry): + """Get returns None for nonexistent ID.""" + assert get_action("9999") is None + + +class TestList: + def test_list_all(self, clean_registry): + """List returns all non-completed actions.""" + create_action(name="a", action_type="schedule", schedule_type="daily") + create_action(name="b", action_type="schedule", schedule_type="hourly") + actions = list_actions() + assert len(actions) == 2 + + def test_list_excludes_completed(self, clean_registry): + """Completed reminders should be excluded by default.""" + create_action(name="done", action_type="reminder", schedule_type="once", + due_date="2026-01-01") + mark_reminder_completed("0001") + assert len(list_actions()) == 0 + assert len(list_actions(include_completed=True)) == 1 + + +class TestToggle: + def test_toggle_off(self, clean_registry): + """Toggle an action off.""" + create_action(name="toggleme", action_type="schedule", schedule_type="daily") + assert toggle_action("0001", False) is True + action = get_action("0001") + assert action["enabled"] is False + + def test_toggle_on(self, clean_registry): + """Toggle an action back on.""" + create_action(name="toggleme", action_type="schedule", schedule_type="daily", + enabled=False) + assert toggle_action("0001", True) is True + action = get_action("0001") + assert action["enabled"] is True + + def test_toggle_missing(self, clean_registry): + """Toggle returns False for nonexistent ID.""" + assert toggle_action("9999", True) is False + + +class TestDelete: + def test_delete_existing(self, clean_registry): + """Delete an action by ID.""" + create_action(name="deleteme", action_type="schedule", schedule_type="daily") + assert delete_action("0001") is True + assert get_action("0001") is None + + def test_delete_missing(self, clean_registry): + """Delete returns False for nonexistent ID.""" + assert delete_action("9999") is False + + +# ============================================= +# DUE CHECKING TESTS +# ============================================= + +class TestIsDue: + def test_daily_due_at_correct_time(self, clean_registry): + """Daily action is due when current time matches.""" + now = datetime.now() + action = { + "enabled": True, + "completed": None, + "schedule_type": "daily", + "time": f"{now.hour:02d}:{now.minute:02d}", + "last_run": None, + } + assert is_action_due(action) is True + + def test_daily_not_due_wrong_time(self, clean_registry): + """Daily action is not due at wrong time.""" + action = { + "enabled": True, + "completed": None, + "schedule_type": "daily", + "time": "99:99", # impossible time + "last_run": None, + } + assert is_action_due(action) is False + + def test_daily_not_due_already_ran_today(self, clean_registry): + """Daily action not due if already ran today.""" + now = datetime.now() + action = { + "enabled": True, + "completed": None, + "schedule_type": "daily", + "time": f"{now.hour:02d}:{now.minute:02d}", + "last_run": now.isoformat(), + } + assert is_action_due(action) is False + + def test_interval_due_never_run(self, clean_registry): + """Interval action is due if never run before.""" + action = { + "enabled": True, + "completed": None, + "schedule_type": "interval", + "interval_minutes": 60, + "last_run": None, + } + assert is_action_due(action) is True + + def test_interval_due_enough_time_elapsed(self, clean_registry): + """Interval action is due when enough time has passed.""" + past = (datetime.now() - timedelta(minutes=120)).isoformat() + action = { + "enabled": True, + "completed": None, + "schedule_type": "interval", + "interval_minutes": 60, + "last_run": past, + } + assert is_action_due(action) is True + + def test_interval_not_due_too_soon(self, clean_registry): + """Interval action is not due when too little time has passed.""" + recent = (datetime.now() - timedelta(minutes=5)).isoformat() + action = { + "enabled": True, + "completed": None, + "schedule_type": "interval", + "interval_minutes": 60, + "last_run": recent, + } + assert is_action_due(action) is False + + def test_once_due_past_date(self, clean_registry): + """Reminder is due when due_date is in the past.""" + action = { + "enabled": True, + "completed": None, + "schedule_type": "once", + "due_date": "2026-01-01", + } + assert is_action_due(action) is True + + def test_once_not_due_future_date(self, clean_registry): + """Reminder is not due when due_date is in the future.""" + action = { + "enabled": True, + "completed": None, + "schedule_type": "once", + "due_date": "2099-12-31", + } + assert is_action_due(action) is False + + def test_disabled_never_due(self, clean_registry): + """Disabled action is never due.""" + action = { + "enabled": False, + "completed": None, + "schedule_type": "interval", + "interval_minutes": 1, + "last_run": None, + } + assert is_action_due(action) is False + + def test_completed_never_due(self, clean_registry): + """Completed action is never due.""" + action = { + "enabled": True, + "completed": "2026-03-01T12:00:00", + "schedule_type": "once", + "due_date": "2026-01-01", + } + assert is_action_due(action) is False + + +# ============================================= +# NEXT RUN TESTS +# ============================================= + +class TestCalcNextRun: + def test_daily_next_run(self, clean_registry): + """Daily action calculates next run correctly.""" + action = {"schedule_type": "daily", "time": "04:00", "last_run": None} + result = calc_next_run(action) + assert result is not None + assert "04:00:00" in result + + def test_interval_next_run(self, clean_registry): + """Interval action calculates next run from last_run + interval.""" + last = datetime.now().isoformat() + action = {"schedule_type": "interval", "interval_minutes": 60, "last_run": last} + result = calc_next_run(action) + assert result is not None + + def test_once_next_run(self, clean_registry): + """Reminder returns due_date as next run.""" + action = {"schedule_type": "once", "due_date": "2026-03-11", "completed": None} + assert calc_next_run(action) == "2026-03-11" + + +class TestNextDueStr: + def test_daily_str(self, clean_registry): + action = {"schedule_type": "daily", "time": "04:00"} + assert next_due_str(action) == "daily @ 04:00" + + def test_hourly_str(self, clean_registry): + action = {"schedule_type": "hourly", "time": "30"} + assert next_due_str(action) == "hourly @ :30" + + def test_once_str(self, clean_registry): + action = {"schedule_type": "once", "due_date": "2026-03-11"} + assert next_due_str(action) == "2026-03-11" + + +# ============================================= +# UPDATE TESTS +# ============================================= + +class TestUpdateLastRun: + def test_update_last_run(self, clean_registry): + """Update last_run sets timestamp and recalculates next_run.""" + create_action(name="test", action_type="schedule", schedule_type="interval", + interval_minutes=60) + ts = "2026-03-02T12:00:00" + assert update_last_run("0001", ts) is True + action = get_action("0001") + assert action["last_run"] == ts + assert action["next_run"] is not None + + +class TestMarkCompleted: + def test_mark_reminder_completed(self, clean_registry): + """Marking a reminder completed sets completed timestamp and disables it.""" + create_action(name="reminder", action_type="reminder", schedule_type="once", + due_date="2026-03-01") + assert mark_reminder_completed("0001") is True + action = get_action("0001") + assert action["completed"] is not None + assert action["enabled"] is False