What Actually Went Wrong: Lessons from Building a Grounded Chatbot

The vertical slice landed in a single day (21 June 2026): embeddings, Atlas text/vector search, RAG orchestration, SSE chat, visitor UI. Everything after that was the real work — making the assistant honest, safe behind a reverse proxy, pleasant to stream, and not confused about whether to answer or navigate.

This article is the journey from that first slice through mid-July 2026: problems, symptoms, and the fixes that stuck.

Timeline at a glance

flowchart LR
  P1[Jun 21<br/>Vertical slice] --> P2[Jun 22<br/>Ops hardening]
  P2 --> P3[Jun 24-25<br/>Retrieval quality]
  P3 --> P4[Jun 25-27<br/>Security and grounding]
  P4 --> P5[Jun 27-28<br/>Gemini and compliance]
  P5 --> P6[Jun 28-Jul 6<br/>Intent routing wars]
  P6 --> P7[Jul 7-12<br/>Models and polish]

1. Prompt injection via client chat history

Problem. Early chat accepted a history array from the client. A malicious visitor could forge assistant turns that contradicted retrieved context and steer the model away from grounding.

Symptom. Grounding looked fine in happy-path tests; an attacker-controlled transcript could still talk the model into inventing CV facts.

Fix. Load history only from server-side chat sessions by session id. Ignore client-supplied turns. Later remove history from the chat request body entirely, and add a prior-turn count for edit/regenerate truncation.

Lesson. If the model must stay grounded in your retrieval, untrusted conversational context is an attack surface, not a convenience feature.

2. Confident hallucinations and false sources

Problem. Prompt text said "do not fabricate." Models still emitted confident portfolio claims, and source chips sometimes attached because a title was name-dropped — not because retrieval supported the claim.

Symptom. Pretty answers that would embarrass me in a recruiter screen-share; citations that looked authoritative and were wrong.

Fix. Treat grounding as a post-generation gate:

1. Grounded fallback — replace weak/refusal outputs with a deterministic excerpt or the fixed no-information message.
2. Grounding validation — claim-span checks against provided context; reject ungrounded confident answers.
3. Tighten source derivation so chips follow real support, not loose string matches.
4. Strip inline [Source …] markers the model still tries to emit.

flowchart TD
  Stream[LLM token stream] --> Strip[Strip inline citations]
  Strip --> Validate{Grounding validation}
  Validate -->|pass| Sources[Derive sources]
  Validate -->|fail| Fallback{Grounded fallback}
  Fallback -->|best hit excerpt| Sources
  Fallback -->|nothing usable| Refuse[Fixed no-information message]
  Refuse --> Sources
  Sources --> UI[Answer plus source chips]

Lesson. Grounding instructions are necessary and insufficient. Validate the output the same way you would validate any other untrusted generator.

3. Streaming that was not streaming (and SSR that fought the chat)

Problem. Several layered UI bugs:

• Backend "streaming" was not wired to real token streaming at first.
• Large chunks made the answer feel chunky even after streaming worked.
• Hydrating chat state from sessionStorage in the constructor caused SSR/client DOM mismatch.
• Writing sessionStorage on every token hammered storage and worsened hydration risk.
• Server and browser once used different HTML sanitizers — another hydration mismatch class.

Fix. Wire real completion streaming; split tokens on whitespace for smoother UI; defer session hydration to a browser-only after-render hook; persist session only when an exchange completes; unify DOMPurify config on SSR and browser. SSE only when Accept asks for it; the frontend consumes SSE through the same HTTP client stack as the rest of the app so interceptors and errors behave.

Lesson. Streaming is a full-stack feature. Parity between SSR and browser sanitizers is as important as the SSE framing if you hydrate HTML.

4. Rate limiting behind proxies

Problem. Chat and content APIs are public. Early rate limiting used a sync mutex in async middleware, trusted the wrong client IP behind reverse proxies, and could be bypassed with a spoofed X-Forwarded-For. Stale entries grew in memory.

Fix. An async mutex; trusted-proxy configuration; validate proxies and walk X-Forwarded-For right-to-left; a cleanup task for stale keys; a separate login rate limit after the security audit.

Lesson. For any public LLM endpoint, rate limit the real client, not the proxy — and treat forwarded headers as hostile unless the peer is in your trusted set.

5. Catalog memory tax

Problem. Catalog-in-context loaded full published documents, including embedding vectors. Roughly 395 KB of payload, mostly unused floats, on every RAG call.

Fix. A catalog query with a MongoDB projection that omits embeddings — about 23 KB of structured facts.

Lesson. "The corpus is small" is true for text. It stops being true the moment you accidentally ship 1536-dimensional vectors into the prompt path.

6. Intent routing: answer in chat vs navigate away

Problem. After chat-driven navigation shipped, the assistant often did the wrong useful thing. Analytical and ranking questions were misread as browse intents:

• "Most valuable skill" filtered a listing instead of ranking.
• Analytical questions triggered filtered project navigation.
• "Do you have…", singular vs plural category words, and "recent/latest" temporal qualifiers kept breaking focus-term extraction.

Symptom. Technically correct routes, product-wrong UX — the visitor wanted a reasoned sentence and got a search page (or the reverse).

Fix. A long series of intent refinements rather than one clever classifier: skills ranking path, analytical detection, catalog intents, singular/plural rules, temporal qualifier stripping, named experience and technology-experience navigation. That work stretched from late June into early July — weeks after the core RAG pipeline already "worked."

Lesson. For a chat-first site, routing is product logic. It outlived the core RAG pipeline by weeks and will keep evolving as visitors phrase things differently. That cost is still cheaper than an unreliable tool-calling agent for this corpus size.

7. Local/CI without live AI or Atlas

Problem. Tests cannot depend on OpenAI, Gemini, or a cloud Atlas cluster.

Fix. From day one: a mock AI provider and a mock search backend; a search-backend config switch; Atlas Local in Docker when integration tests need real $search / $vectorSearch. Live keys stay out of CI.

Lesson. Dual implementations are not optional polish for RAG projects — they are how you keep the grounding tests honest.

8. Provider and compliance churn

Problem. Chat quality, cost, and EU disclosure requirements pulled the stack in different directions.

Fix. A generic OpenAI-compatible remote client; split chat and embedding configuration; default chat to Gemini; keep embeddings on OpenAI text-embedding-3-large; lower the chat model to gemini-3.1-flash-lite with temperature 0.2; ship privacy, EU AI Act, and incident-process disclosures alongside the product.

Lesson. Treat chat model and embedding model as independent product decisions. Locking them together forces bad compromises when compliance or cost moves.

Hardening phases (synthesis)

Phase	What broke	What we installed
Ship fast	Nothing yet	Vertical slice + mocks
Ops	Async + IPs	Proxy-aware rate limits
Quality	Citations, analytics	Catalog + fallback
Audit	History injection, ungrounded claims	Server sessions + validation
Cost	Catalog embeddings	Projection
UX	Answer vs browse	Intent routing refinements
Polish	Chunks, models	Token pieces, Gemini lite

The pattern I keep returning to: ship the slice, then promote every failure mode into an explicit layer — trusted history, hybrid retrieval, catalog facts, grounding gates, deterministic routes. The chatbot got more "agentic" in the product sense by getting less autonomous in the model sense.

What I would tell myself on day one

1. Never accept client conversation history for a grounded assistant.
2. Budget engineering time for intent routing; it will outlast the first RAG demo.
3. Measure catalog payload size with embeddings included before celebrating "tiny corpus."
4. Put grounding checks after generation, not only in the prompt.
5. Keep mocks first-class so every grounding regression is a unit test, not a screenshot.

The architecture that survived is the one in the first two articles: hybrid RAG, catalog-in-context, deterministic agentic UX, and defense-in-depth grounding. The failures above are how it earned that shape.