Audit Logs
The audit_log table records every significant agent action. It provides a comprehensive audit trail for debugging, security review, shell command traceability, and usage analysis.
Schema
CREATE TABLE audit_log (
id UUID PRIMARY KEY, -- No auto-default; must be passed explicitly
timestamp TIMESTAMPTZ NOT NULL, -- Indexed; composite index with chat_id
event_type VARCHAR(100), -- Indexed
source VARCHAR(50) DEFAULT '',
action VARCHAR(200) DEFAULT '', -- e.g. skill.shell, skill.self_improve, agent.reset
input_summary TEXT DEFAULT '', -- Arguments (secrets redacted)
output_summary TEXT DEFAULT '', -- Result summary (secrets redacted)
user_id VARCHAR(50) DEFAULT '',
chat_id VARCHAR(50) DEFAULT '', -- Composite index with timestamp
latency_ms INTEGER DEFAULT 0,
error TEXT, -- Null on success
metadata_json JSONB DEFAULT '{}'
);
-- Composite index for efficient paginated queries
CREATE INDEX ix_audit_log_chat_id_timestamp ON audit_log (chat_id, timestamp);
What Gets Logged
| Action type | When | Example |
|---|---|---|
skill.shell | Every shell invocation | Command, exit code, goal context |
skill.self_improve | Every self_improve write/patch/apply | File path, gate decision, backup path |
agent.reset | Every Panic Reset | Timestamp, operator confirmation |
goal.* | Goal lifecycle events | create, complete, fail, replan |
task.* | Task execution | Scheduled task runs |
Capability-Based Logging
| Capability Level | Logged? | Notes |
|---|---|---|
| SAFE | No | calculate, datetime, system_info |
| MONITORED | No | web_search, fetch_url, browser |
| CONTROLLED | Yes | gmail, reminders, task_manager |
| RESTRICTED | Yes | shell, python_exec, http_request |
| PRIVILEGED | Yes | self_improve, broker commands |
Secret Redaction
All input and output summaries are passed through redact() from src/utils/redaction.py before writing. The shell skill has an additional layer of command-level redaction:
Global Redaction Patterns (all audit entries)
| Pattern | Example |
|---|---|
| OpenAI keys | sk-[a-zA-Z0-9]{20,} |
| Anthropic keys | sk-ant-[a-zA-Z0-9-]{20,} |
| Google keys | AIza[a-zA-Z0-9-_]{25,} |
| xAI keys | xai-[a-zA-Z0-9]{20,} |
| HuggingFace tokens | hf_[a-zA-Z0-9]{20,} |
| AWS access keys | AKIA[A-Z0-9]{12,} |
| Stripe live keys | sk_live_[a-zA-Z0-9]{24} |
| Slack tokens | xox[bpoa]-[a-zA-Z0-9-]+ |
| SendGrid keys | SG\.[a-zA-Z0-9]{22}\.[a-zA-Z0-9]{43} |
| Bearer tokens | Bearer [a-zA-Z0-9+/=]{20,} |
| Password patterns | password[=:]\S+ |
Shell-Specific Redaction (v2.6)
The shell skill applies an additional command-level redaction via _redact_command() before the global redaction:
_REDACT_RE = re.compile(
r'(sk-[A-Za-z0-9\-_]{20,}|AIza[A-Za-z0-9\-_]{30,}|'
r'xai-[A-Za-z0-9\-_]{20,}|hf_[A-Za-z0-9]{20,}|'
r'(?:password|passwd|token|secret|key)\s*[=:]\s*\S+)',
re.IGNORECASE,
)
This catches secrets embedded directly in shell commands (e.g., curl -H "Authorization: Bearer sk-...").
Automatic Retention (v2.5)
The AuditRetentionJob runs daily and automatically deletes entries older than the retention window:
AUDIT_RETENTION_DAYS=30 # default: 30 days
Uses bounded batch deletion to avoid table locks:
DELETE FROM audit_log
WHERE id IN (
SELECT id FROM audit_log
WHERE timestamp < NOW() - INTERVAL '30 days'
ORDER BY timestamp ASC
LIMIT 5000
)
Common Queries
Recent Activity
docker exec agent-postgres psql -U agent -d agent -c "
SELECT action, input_summary, output_summary, error, timestamp
FROM audit_log
ORDER BY timestamp DESC
LIMIT 20;
"
Shell Invocations (v2.6)
docker exec agent-postgres psql -U agent -d agent -c "
SELECT input_summary as command, output_summary as result, error, timestamp
FROM audit_log
WHERE action = 'skill.shell'
ORDER BY timestamp DESC
LIMIT 30;
"
Failed Shell Commands
docker exec agent-postgres psql -U agent -d agent -c "
SELECT input_summary as command, error, timestamp
FROM audit_log
WHERE action = 'skill.shell'
AND error IS NOT NULL
AND timestamp > NOW() - INTERVAL '7 days'
ORDER BY timestamp DESC;
"
Self-Improve History
docker exec agent-postgres psql -U agent -d agent -c "
SELECT input_summary as file, output_summary as result, error, timestamp
FROM audit_log
WHERE action = 'skill.self_improve'
ORDER BY timestamp DESC
LIMIT 20;
"
Agent Reset History
docker exec agent-postgres psql -U agent -d agent -c "
SELECT output_summary, timestamp
FROM audit_log
WHERE action = 'agent.reset'
ORDER BY timestamp DESC;
"
Skill Usage (Last 24h)
docker exec agent-postgres psql -U agent -d agent -c "
SELECT
action,
COUNT(*) as total_calls,
SUM(CASE WHEN error IS NULL THEN 1 ELSE 0 END) as successes,
ROUND(AVG(latency_ms)) as avg_ms
FROM audit_log
WHERE timestamp > NOW() - INTERVAL '24 hours'
GROUP BY action
ORDER BY total_calls DESC;
"
Error Analysis
docker exec agent-postgres psql -U agent -d agent -c "
SELECT action, input_summary, error, timestamp
FROM audit_log
WHERE error IS NOT NULL
AND timestamp > NOW() - INTERVAL '7 days'
ORDER BY timestamp DESC
LIMIT 30;
"
Per-Chat Activity
docker exec agent-postgres psql -U agent -d agent -c "
SELECT chat_id, COUNT(*) as calls, MAX(timestamp) as last_active
FROM audit_log
WHERE timestamp > NOW() - INTERVAL '7 days'
GROUP BY chat_id
ORDER BY calls DESC;
"
Dashboard Audit Log Page
The /audit page in the dashboard provides a paginated view of the audit log with:
- Keyset pagination (cursor-based — O(1), no OFFSET scans)
- Filter by action type and date range
- Secrets already redacted in stored records
Real-time Streaming
# Poll every 5 seconds
watch -n5 'docker exec agent-postgres psql -U agent -d agent -c "
SELECT action, error IS NOT NULL as failed, timestamp
FROM audit_log
ORDER BY timestamp DESC
LIMIT 5;
"'
Manual Cleanup
If you need to clear logs before the retention job runs:
# Delete entries older than 90 days
docker exec agent-postgres psql -U agent -d agent -c "
DELETE FROM audit_log
WHERE timestamp < NOW() - INTERVAL '90 days';
"
# Then run VACUUM ANALYZE to reclaim space
docker exec agent-postgres psql -U agent -d agent -c "VACUUM ANALYZE audit_log;"
Note: For a full cognitive reset including all audit history, use the
/resetdashboard page.