Your Agent Has 100 Tools and Still Forgets Everything

The verdict. You run an agent that talks to you daily and must remember your projects, decisions, and past failures. Do not buy a memory plugin and call it solved. Build a small memory operating system: typed knowledge, intent routing, selective ingestion, and recurring maintenance. We ran that shape with Hermes as coordinator and GBrain as substrate for three months. It grew from 178 pages / 682 chunks on 6 July to 1,144 pages / 2,099 chunks, all embedded, at publication time, and it broke in seven useful ways along the route. [Receipt: reports/gbrain-maintenance/2026-07-06-dream-cycle; live get_stats, 2026-08-21]

This is one operator’s home-lab field report, not a universal benchmark. The verified architecture is the GBrain 0.42.8.0-era stack: Docker PostgreSQL/pgvector on a workstation, accessed over HTTP MCP; Hermes is the front door and coordinator. Embeddings used local Ollama nomic-embed-text at 768 dimensions. Search synthesis, think, dream/consolidation, and fact extraction used a local vLLM Qwen route, so the memory path made 0 external LLM API calls in the 2 June verification. [Receipt: reports/hermes-gbrain-verification-2026-06, 2026-06-02]

1. Your agent does not need more memory. It needs a place to put it.

The operator here is a solo or small-team builder with an agent on Discord, Telegram, or a terminal every day. The leak is not that the agent cannot recall yesterday’s transcript. The leak is that a past decision is buried in chat, a project has no durable state, an old failure gets repeated, and the next session starts by rediscovering work that already happened.

Raw transcripts are not durable memory. They contain acknowledgements, abandoned work, duplicate conclusions, secrets you should never store, and tactical state that expires before the write finishes. Re-embedding all of it gives you a larger haystack, not an operating model.

The working model has four parts:

1. Typed substrate: pages for durable entities and reports, graph links for relationships, facts and takes for claims, and embeddings for retrieval. This separates a project page, a decision, a dated report, and a searchable chunk instead of flattening them into a chat dump.
2. Routing: a resolver chooses the memory action before the model chooses a raw tool. “What do we know about X?” maps to query; “synthesize this” maps to think; “what is hot?” maps to salience; read and write stay explicit.
3. Ingestion policy: batch session digests retain durable signal and skip noise. They are reviewable pages, not invisible per-message mutations.
4. Maintenance loop: sync, consolidation, extraction, embedding, orphan review, link repair, and doctor checks keep the substrate from becoming an attractive graveyard.

That is why “add a vector DB” is an incomplete answer. A vector index is a useful retrieval component. It is not source scope, a graph, a decision ledger, a write policy, or a way to learn that a pipeline silently stopped running.

2. The useful unit of agent memory is a typed claim, not a chat blob

In the verified June architecture, GBrain held pages, sources, graph links, search/query, reports, facts, takes, and maintenance jobs; Hermes remained the interaction layer. The active source was default, pointing to the local brain directory, not an undifferentiated transcript store. [Receipt: agents/hermes-gbrain-integration; reports/hermes-gbrain-verification-2026-06, 2026-06-02]

That distinction showed up in the live numbers. At publication time, get_stats returned 1,144 pages, 2,099 chunks, 2,099 embedded chunks, and 2,088 links. The same-day doctor reported an 85/100 health score and 86.0% inbound-link coverage, while also warning that 1,141 of 1,144 pages still had un-extracted edges. A graph is not a trophy count; it gives you an auditable place to see missing relationships. [Receipt: live get_stats and run_doctor, 2026-08-21]

The shape of the alternatives matters more than vendor logos:

Approach	Good for	Cost shape	Catch
Raw chat-log re-embedding	Fast prototype	Storage and embedding usage	Noise becomes retrieval context
DIY pgvector + embeddings	One focused corpus	Build and operator time	No typed claims, graph, or maintenance policy
GBrain + Hermes router	Durable project and decision memory	Existing workstation electricity + operator time	You own the maintenance loop
Hosted memory SaaS (Mem0-class)	Fast managed trial	Pricing to be verified at QA	Data model and retrieval behavior need inspection
Notes + chat history	Low-stakes work	Near-zero infrastructure	Recall remains manual and session-bound

The table is not a ranking. If you only need to search a small document folder, DIY embeddings may be the right smaller system. If you need an agent to remember decisions across projects and explain where a claim came from, the missing pieces are the point.

3. Routing beats an 80-tool memory buffet

The first obvious mistake was exposing the raw MCP surface. The agent saw 80-plus tools and had no reliable habit for choosing a memory operation, so it picked arbitrarily or skipped memory for direct answers and web search. This was not a model-intelligence problem. It was an interface problem. [Receipt: logs/kit-log, 2026-07-04, “Full GBrain Production Setup”]

The repair was a resolver pattern: a routing table of high-level intent to tool, a brain-first rule before external lookups, and a local GBrain skill pack. The 2 June integration record confirms that the bundled skills were installed into ~/.hermes/skills/gbrain-skillpack/; that directory exists on the active Hermes machine. High-level tools became the normal path; low-level reads, graph calls, and writes became verification or repair tools. [Receipt: agents/hermes-gbrain-integration, 2026-06-02; local directory check, 2026-08-21]

The practical routing table is deliberately boring:

Operator intent	Default action	Why
“What do we know about X?”	Query	Retrieves grounded candidate pages
“Pull this together”	Think	Multi-page synthesis, not raw chat recall
“What changed or matters?”	Salience	Surfaces recent activity patterns
“Read the record”	Get page	Returns the canonical page
“Save this decision”	Write page / fact path	Creates reviewable durable state

Do not make the model remember this as an 80-item tool catalog. Make it the agent’s operating policy. A memory layer that requires perfect tool selection on every turn will be present in your config and absent from your workflow.

4. Batch ingestion is the policy; per-message writes are the bug

The working ingestion pattern is a daily batch digest, not automatic capture of every utterance. The verified GBrain Session Digest Importer cron, job dd82b65c69ad, was created and smoke-tested on 2 June. Its policy scans recent sessions, deduplicates durable signal, excludes acknowledgements, transient task progress, secrets, and likely-short-lived facts, then writes a reviewable report page. [Receipt: reports/hermes-session-handoff-and-digest-cron; agents/hermes-gbrain-integration, 2026-06-02]

This is not theoretical hygiene. On 20 August, the digest analyzed four substantive sessions, submitted extraction for three, and deduplicated to zero new facts. That is a healthy no-op: it means the system did not manufacture novelty because a cron fired. [Receipt: logs/kit-log, 2026-08-20 22:11 IST]

The cost is also plain. Our local memory lane cost electricity on an already-owned workstation plus operator time. Buy-side memory costs would be embedding API usage plus LLM API usage for extraction, synthesis, and maintenance; prices are to be verified at QA, not guessed here. The local lane is not free. The seven incidents below are its itemized operator bill.

5. The seven ways a naive memory stack dies

These are incidents, not hypothetical caveats. Each changed the operating design.

1. The local database never starts. On 11 July, Mac PGLite failed to initialize its WASM runtime on macOS 26.x; local brain commands failed, while the remote brain intermittently returned HTTP 503 for a database connection failure. The fix was architectural: Docker PostgreSQL/pgvector on the workstation behind HTTP MCP, with the Mac as thin client. [Receipt: logs/kit-log, 2026-07-11; reports/local-vllm-and-hermes-runtime-2026-06]

2. The brain is “empty” because the source is wrong. Identity and auth can work while list_pages and search return little, because pages sit under stale source personal and the agent is scoped to default. The reusable repair was: inspect source counts, move pages, revoke the orphan client, remove the stale source while preserving storage, then sync/embed/extract/dream. [Receipt: reports/hermes-gbrain-verification-2026-06, “Critical Fix Pattern: Source Mismatch”]

3. Tool availability is not tool adoption. The 80-plus raw tools produced no habits. The resolver and brain-first policy were the repair, not another MCP tool. [Receipt: logs/kit-log, 2026-07-04; agents/hermes-gbrain-integration]

4. Auto-writing every message fills the corpus with exhaust. The digest cron deliberately filters acknowledgements, credentials, ephemeral work, and duplicates. The 20 August zero-new-facts digest is the receipt that “write less” can be correct. [Receipt: reports/hermes-session-handoff-and-digest-cron; logs/kit-log, 2026-08-20]

5. An extractor can die quietly for weeks. A stale configured model alias caused repeated Cannot connect to API failures for take proposals. Earlier maintenance also reported 1,312 legacy facts pending fence backfill. The pipeline was not trustworthy because the UI looked healthy; it became visible because maintenance reports named the failure. [Receipt: reports/gbrain-maintenance/2026-07-06-dream-cycle; reports/gbrain-maintenance/2026-08-20-2000; logs/kit-log, 2026-07-14]

6. Transport fails exactly when unattended jobs need it. MCP was unavailable inside cron runs, and larger writes were unreliable. The working fallback was CLI/SSH plus explicit read-back after every write. On 17, 18, and 22 June, remote access was unavailable, so the operator wrote local fallback reports and performed no live mutation. [Receipt: reports/hermes-session-handoff-and-digest-cron; logs/kit-log, 2026-06-17/18/22]

7. A graph can look populated while its extraction is broken. On 13 July, the system had 1,573 links and only 87 orphans, but 825 of 828 pages still had un-extracted edges because the wikilink extractor was broken. The workaround was deliberate manual linking, not pretending link count equals graph health. The same class remains visible in the live doctor warning. [Receipt: reports/gbrain-maintenance/2026-07-13-2000; live run_doctor, 2026-08-21]

The trajectory matters because it shows both scale and correction. The 6 July dream reported 178 pages, 682 chunks, and 100% embedding coverage. By 13 July, maintenance reported 828 pages, 1,385 chunks, 1,573 links, 2,192 active facts, 170 takes, and 87 orphans, with health at 55/100. A 7 July migration had surfaced 638 orphans after hundreds of untracked pages appeared; the later 87 was not magic, it was consolidation and intentional linking. [Receipt: reports/gbrain-maintenance/2026-07-06-dream-cycle; reports/gbrain-maintenance/2026-07-07-dream-cycle; reports/gbrain-maintenance/2026-07-13-2000]

By 20 August, reports recorded 1,136–1,137 pages, 100% embedding coverage, 2,078–2,090 links, and doctor 85/100. Actionable orphan counts sat between 114 and 154 depending on the scan, largely system artifacts requiring curation rather than blanket links. At publication time, the live snapshot is 1,144 pages, 2,099 chunks, 2,088 links, with doctor 85/100. [Receipt: reports/gbrain-maintenance/2026-08-20-2000; reports/gbrain-maintenance/2026-08-20-2135; live get_stats and run_doctor, 2026-08-21]

6. Skip the memory OS when the operating loop is fiction

Do not build this stack because “memory” sounds like a feature. Skip it, or shrink it first, when:

• You only need to search a static, small document set. Start with files and keyword search or a small vector index.
• You cannot name an owner for daily ingestion review and weekly maintenance. A local stack without an operator is deferred failure.
• Your agent has no trusted write boundary. Fix explicit write approval and secret exclusion before persistence.
• You have not defined what qualifies as durable signal. Start with one project, decisions, and dated reports; do not import your entire chat archive.
• You need a retrieval-quality percentage before deploying. We ran no controlled retrieval evaluation, so we do not claim a RAG accuracy improvement. That is a known follow-up, not a number to invent.

Doctor also needs interpretation. A tiny corpus can return an unhealthy score while connection, schema, sync freshness, embeddings, search mode, and dream plumbing work. Treat plumbing checks and corpus-quality checks separately. Then treat chronic warnings, such as extraction lag, as work, not as a reason to call a green connection “healthy.” [Receipt: reports/hermes-gbrain-verification-2026-06, “Doctor Interpretation”]

7. Give your memory layer 30 days to earn its ops budget

Run the same small test before you commit to a plugin, hosted service, or local stack. Record a baseline on day zero, then use the same corpus and question set on day 30.

Metric	Baseline	Day 30 pass condition
Corpus size	Pages, chunks, dated	Growth is intentional, not chat dump growth
Embedding coverage	Current percentage	100% with completion verification
Orphan trend	Count and artifact types	Down or consciously curated
Durable facts	Daily insertions	No unexplained flood or silent zero

Add N spot-check questions that the agent must answer brain-first with citations: project state, a past decision, a failure, and a dated change. Record accuracy on the same N questions at baseline and day 30; N is intentionally yours, not a fabricated universal sample size. Also keep daily digest and weekly dream/maintenance cadence visible in reports.

Decision rule: if the same spot-check set stays answered with citations for 30 days and maintenance reports show no silent extractor failure, the memory layer earns its operations budget. If either condition fails, fix the routing, source scope, ingestion policy, or maintenance loop before adding more models, embeddings, or tools.

Bottom line

Durable agent memory is an operating system, not a plugin checkbox. The three-month Hermes and GBrain run supports a narrower claim: typed substrate, resolver routing, batch ingestion, and maintenance made durable memory inspectable; raw-tool sprawl, source drift, silent extractors, flaky transport, and broken link extraction were the actual enemies.

Start smaller than you want. Give it one source, a handful of high-level routes, a reviewable daily digest, weekly maintenance, and a citation-based 30-day test. The agent memory decision is not “which vector store?” It is whether you are willing to operate the system that decides what gets remembered.

---

More on this decision, three ways to look at it:

Sources

• reports/hermes-gbrain-verification-2026-06, verified architecture, source-mismatch repair, and doctor interpretation (2026-06-02)
• agents/hermes-gbrain-integration, Hermes/GBrain operating rules, local routing, and skill-pack installation (2026-06-02)
• reports/hermes-session-handoff-and-digest-cron, batch digest policy and cron receipt (2026-06-02)
• reports/gbrain-maintenance/2026-07-06-dream-cycle, 2026-07-07-dream-cycle, and 2026-07-13-2000, dated growth, migration, health, and graph receipts
• reports/gbrain-maintenance/2026-08-20-2000 and 2026-08-20-2135, latest pre-publication maintenance receipts
• logs/kit-log, dated incident log, including 2026-06-17/18/22, 2026-07-04/06/11/14, and 2026-08-20 entries
• Live GBrain get_stats and run_doctor snapshot, publication-time counts and warnings (2026-08-21)