Terraphim Agent Session Search - Architecture Document
Version: 1.1.0 Status: Implemented — Verified 2026-06-01 Created: 2025-12-03 Updated: 2026-06-01
Overview
This document describes the technical architecture for the Session Search and Robot Mode features in terraphim-agent. The architecture extends existing Terraphim components while introducing new modules for session management and AI-friendly interfaces.
System Context
┌─────────────────────────────────────────────────────────────────────────────┐
│ External Systems │
├─────────────┬─────────────┬─────────────┬─────────────┬─────────────────────┤
│ Claude Code │ Cursor │ Aider │ Cline │ Other Agents │
│ (JSONL) │ (SQLite) │ (Markdown) │ (JSON) │ │
└──────┬──────┴──────┬──────┴──────┬──────┴──────┬──────┴──────────┬──────────┘
│ │ │ │ │
└─────────────┴──────┬──────┴─────────────┴─────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────────────────┐
│ terraphim-agent │
│ ┌─────────────────────────────────────────────────────────────────────┐ │
│ │ Session Connectors │ │
│ │ ┌───────────┐ ┌───────────┐ ┌───────────┐ ┌───────────┐ │ │
│ │ │ Claude │ │ Cursor │ │ Aider │ │ Cline │ │ │
│ │ │ Connector │ │ Connector │ │ Connector │ │ Connector │ │ │
│ │ └─────┬─────┘ └─────┬─────┘ └─────┬─────┘ └─────┬─────┘ │ │
│ └────────┼─────────────┼─────────────┼─────────────┼──────────────────┘ │
│ └─────────────┴──────┬──────┴─────────────┘ │
│ ▼ │
│ ┌─────────────────────────────────────────────────────────────────────┐ │
│ │ Session Service │ │
│ │ ┌─────────────┐ ┌─────────────┐ ┌─────────────────────────────┐ │ │
│ │ │ Import │ │ Index │ │ Enrichment │ │ │
│ │ │ Engine │──│ (Tantivy) │──│ (Knowledge Graph) │ │ │
│ │ └─────────────┘ └─────────────┘ └─────────────────────────────┘ │ │
│ └─────────────────────────────────────────────────────────────────────┘ │
│ │ │
│ ▼ │
│ ┌─────────────────────────────────────────────────────────────────────┐ │
│ │ Command Layer │ │
│ │ ┌───────────────┐ ┌───────────────┐ ┌───────────────────────┐ │ │
│ │ │ Forgiving CLI │ │ Robot Mode │ │ Self-Documentation │ │ │
│ │ │ Parser │ │ Formatter │ │ API │ │ │
│ │ └───────────────┘ └───────────────┘ └───────────────────────┘ │ │
│ └─────────────────────────────────────────────────────────────────────┘ │
│ │ │
│ ▼ │
│ ┌─────────────────────────────────────────────────────────────────────┐ │
│ │ Existing Terraphim Core │ │
│ │ ┌─────────────┐ ┌─────────────┐ ┌─────────────────────────────┐ │ │
│ │ │ terraphim_ │ │ terraphim_ │ │ terraphim_ │ │ │
│ │ │ automata │ │ rolegraph │ │ service │ │ │
│ │ └─────────────┘ └─────────────┘ └─────────────────────────────┘ │ │
│ └─────────────────────────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────────────────────┘Module Architecture
Implementation Status
The architecture has been implemented using a feature-gated approach with terraphim-session-analyzer (CLA) integrated as a git subtree. This allows terraphim_sessions to work standalone while gaining enhanced capabilities when CLA is enabled.
New Modules
crates/
├── terraphim-session-analyzer/ # ADDED: Git subtree integration
│ ├── src/
│ │ ├── lib.rs # Core CLA exports
│ │ ├── parser.rs # Claude JSONL parsing
│ │ ├── session.rs # Session entry types
│ │ └── connectors/ # ADDED: Multi-agent connectors
│ │ ├── mod.rs # Connector trait + registry
│ │ ├── claude.rs # Claude Code connector
│ │ └── cursor.rs # Cursor SQLite connector
│
├── terraphim_agent/ # ENHANCED
│ ├── src/
│ │ ├── main.rs # --robot, --format flags
│ │ ├── repl/
│ │ │ ├── mod.rs
│ │ │ ├── commands.rs # +SessionsSubcommand
│ │ │ └── handler.rs # +handle_sessions()
│ │ ├── robot/ # Robot mode module
│ │ │ ├── mod.rs
│ │ │ ├── output.rs # JSON formatters
│ │ │ ├── schema.rs # Response schemas
│ │ │ └── docs.rs # Self-documentation
│ │ └── forgiving/ # Forgiving CLI
│ │ ├── mod.rs
│ │ ├── parser.rs # Edit-distance parser
│ │ └── suggestions.rs # Command suggestions
│
├── terraphim_sessions/ # NEW CRATE - Feature-gated
│ ├── Cargo.toml # Feature definitions
│ ├── src/
│ │ ├── lib.rs # Conditional module exports
│ │ ├── model.rs # Session, Message, ContentBlock
│ │ ├── search.rs # Hybrid BM25+KG search
│ │ ├── service.rs # SessionService facade
│ │ ├── connector/ # Connector infrastructure
│ │ │ ├── mod.rs # SessionConnector trait
│ │ │ ├── native.rs # NativeClaudeConnector (no CLA)
│ │ │ ├── aider.rs # Aider markdown connector
│ │ │ ├── cline.rs # Cline JSON connector
│ │ │ ├── codex.rs # Codex JSONL connector
│ │ │ └── opencode.rs # OpenCode JSONL connector
│ │ ├── enrichment/ # Knowledge graph enrichment
│ │ │ ├── mod.rs
│ │ │ ├── concept.rs # Concept extraction
│ │ │ └── enricher.rs # SessionEnricher
│ │ └── cla/ # #[cfg(feature = "terraphim-session-analyzer")]
│ │ ├── mod.rs # CLA integration layer
│ │ └── connector.rs # ClaClaudeConnector, ClaCursorConnectorFeature Gate Configuration
# crates/terraphim_sessions/Cargo.toml
[features]
default = []
terraphim-session-analyzer = ["dep:terraphim-session-analyzer"]
cla-full = ["terraphim-session-analyzer", "terraphim-session-analyzer/connectors"]
enrichment = ["terraphim_automata", "terraphim_rolegraph"]
full = ["cla-full", "enrichment"]| Feature | Enables | Use Case |
|---------|---------|----------|
| (none) | NativeClaudeConnector only | Minimal JSONL parsing |
| terraphim-session-analyzer | CLA core + Claude connector | Full Claude analysis |
| cla-full | CLA + Cursor SQLite | Multi-agent support |
| enrichment | Knowledge graph integration | Concept detection |
| full | Everything | Production deployment |
Component Details
1. Forgiving CLI Parser
Location: crates/terraphim_agent/src/forgiving/
Purpose: Parse commands with typo tolerance and flexibility.
/// Forgiving command parser with edit-distance correction
2. Robot Mode Output
Location: crates/terraphim_agent/src/robot/
Purpose: Structured, machine-readable output for AI agents.
/// Robot mode output configuration
/// Standard response envelope
/// Exit codes
3. Self-Documentation API
Location: crates/terraphim_agent/src/robot/docs.rs
/// Self-documentation for AI agents
4. Session Connectors
Location: crates/terraphim_sessions/src/connector/
/// Trait for session source connectors
/// Claude Code connector implementation
5. Session Index (Hybrid BM25 + Knowledge Graph)
Location: crates/terraphim_sessions/src/search.rs
Purpose: In-memory full-text search with optional KG concept boosting.
ARCHITECTURE CHANGE: Tantivy was evaluated but not adopted. The implementation uses in-memory BM25 scoring with optional Knowledge Graph boosting. See Task 2.5 in the tasks document for rationale.
/// Search sessions using BM25 scoring
;
/// Hybrid search with KG concept boosting
;Key characteristics:
- No persistent index — sessions loaded from SQLite on demand
- BM25 scoring via
terraphim_types::score::OkapiBM25Scorer - KG concept matches receive 10,000x score boost
- Performance: ~10ms for 10K sessions (well under 100ms target)
6. Knowledge Graph Enrichment
Location: crates/terraphim_sessions/src/enrichment/
use ;
use RoleGraph;
/// Enriches sessions with knowledge graph concepts
Data Flow
Import Flow
┌─────────────┐ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐
│ Source │────▶│ Connector │────▶│ Parser │────▶│ Session │
│ Files │ │ (detect) │ │ (normalize) │ │ Model │
└─────────────┘ └─────────────┘ └─────────────┘ └──────┬──────┘
│
┌─────────────┐ ┌─────────────┐ │
│ SQLite │◀────│ Enricher │◀───────────┘
│ (storage) │ │ (concepts) │
└─────────────┘ └─────────────┘Note: Sessions are stored in SQLite, not a Tantivy index. Search operates on in-memory BM25 over loaded sessions.
Search Flow
┌─────────────┐ ┌─────────────┐ ┌─────────────┐
│ Query │────▶│ Forgiving │────▶│ Query │
│ Input │ │ Parser │ │ Expansion │
└─────────────┘ └─────────────┘ └──────┬──────┘
│
┌─────────────┐ │
│ Tantivy │◀───────────┘
│ Search │
└──────┬──────┘
│
┌─────────────┐ ┌──────▼──────┐ ┌─────────────┐
│ Robot │◀────│ Result │◀────│ Concept │
│ Output │ │ Formatter │ │ Expansion │
└─────────────┘ └─────────────┘ └─────────────┘Integration with Existing Components
terraphim_automata Integration
// Use existing fuzzy search for forgiving CLI
use fuzzy_autocomplete_search_jaro_winkler;
// Use existing concept extraction for enrichment
use ;terraphim_service Integration
// Sessions integrate with existing search
terraphim_config Integration
// Session configuration in role config
Error Handling
Testing Strategy
Unit Tests
Integration Tests
async Performance Considerations
Index Performance
- Batch writes: Commit after every 1000 documents
- Reader reload: Use
reader.reload()for real-time search - Segment merging: Configure merge policy for read-heavy workload
Memory Management
- Streaming import: Process files in chunks, not all at once
- Index caching: Keep hot segments in memory
- Result pagination: Default limit of 10, max of 100
Startup Optimization
- Lazy loading: Don't load index until first search
- Background indexing: Import new sessions async
- Warm-up queries: Pre-warm common searches
Security Considerations
Data Privacy
- All data stored locally
- No network calls for session data
- File permissions respected
Secret Detection
lazy_static! Future Extensions
Phase 2+ Considerations
- Semantic Search: Add embedding support alongside BM25
- Cross-Machine Sync: Optional encrypted sync
- Session Replay: Interactive session playback
- Analytics Dashboard: TUI-based analytics view
Plugin Architecture
/// Plugin trait for custom connectors
/// Dynamic connector loading