Poppy field guide · 03
Focused chatbots
How to design a Poppy chatbot that answers inside a defined evidence boundary and knows when that boundary has been reached.
Onboarding, guided research, internal knowledge access, and repeated questions tied to an intentionally maintained board.
A polished answer implies the board contains evidence that was never supplied or is no longer current.
Job design
A chatbot needs a job description, not a personality paragraph.
Start with the recurring decision the user must make. State the audience, allowed sources, expected answer format, conditions that require clarification, and topics that must be refused or escalated. Tone matters, but it cannot compensate for an undefined operating boundary.
A strong bot often does less than the team first imagines. A product onboarding bot can explain the supplied workflow and surface the next step. It should not invent policy, promise unsupported outcomes, or become a general consultant simply because the interface accepts any question.
- Write examples of acceptable, ambiguous, unsupported, and prohibited requests.
- Define what the bot should cite or point back to when the answer matters.
- Make escalation visible instead of disguising it as another generated response.
Context contract
The board and the instruction must agree about authority.
If the instruction says to answer only from supplied material but the board mixes official evidence, brainstorming, outdated drafts, and operator opinions, the restriction is not meaningful. Content status has to be encoded in the board architecture. The chatbot should know which material is authoritative and which is context for the operator only.
Conflicting sources need an explicit response. The bot can disclose the conflict, prefer a named source class, or ask a human to resolve it. Silently choosing the most fluent interpretation makes the interface pleasant and the knowledge system unreliable.
- Separate approved knowledge from working research and superseded material.
- Describe how dates and version-specific claims should be handled.
- Require the bot to state when the available context does not support an answer.
Evaluation
Test the boundary, not only the happy path.
A demonstration question usually mirrors the supplied language and produces an impressive answer. Evaluation should include paraphrases, incomplete requests, conflicting instructions, questions just outside scope, prompts that ask the bot to ignore its rules, and cases where the correct outcome is uncertainty.
Record expected behavior before testing. Otherwise a surprising response is easily rationalized after the fact. For a customer-facing or operational bot, retain a small regression set and rerun it after the board, instructions, or upstream product behavior changes.
- Score factual support, scope discipline, useful clarification, and safe refusal separately.
- Include attempts to extract hidden instructions or unrelated private context.
- Test with a user who did not build the board and does not know the intended phrasing.
Ownership
A reusable chatbot is a maintained publication.
Someone must own source updates, instruction changes, failed-answer review, and retirement. Without that role, the chatbot keeps presenting old context with the confidence and visual permanence of a current service.
Publish the bot's purpose, limits, and a correction route near the interaction. Users should know whether they are seeing generated guidance, what body of material it represents, and how to report an unsupported answer. That transparency is part of product quality, not legal decoration.
Continue the field guide