Skip to content

fix(chatbot): handle structured/list message content from newer LLM p… - #387

Open
aarshx05 wants to merge 3 commits into
OWASP:developfrom
aarshx05:fix/chatbot-structured-content
Open

aarshx05 wants to merge 3 commits into
OWASP:developfrom
aarshx05:fix/chatbot-structured-content

Conversation

@aarshx05

Copy link
Copy Markdown

Description

The chatbot assumes an LLM reply's content is a str. Newer models (e.g. recent Anthropic Claude models) return content as a list of structured content blocks (text / thinking / tool-use / signed text). In process_user_message this value is stored in the Mongo chat history and passed to add_to_chroma_collection → Document(page_content=...), which requires a str. A list therefore raises pydantic.ValidationError; the exception is caught in /genai/ask and the raw error is returned to the user and the turn is silently never persisted (chat appears broken and RAG/history storage fails).

This bugfix adds a small message_content_to_text() helper in agent_utils.py that normalizes any provider's content to plain text (joins text blocks, drops thinking/tool-use blocks, coerces other types safely) and uses it in process_user_message before the value is stored, embedded, or returned. No behavior change for providers that already return a string (e.g. OpenAI, older models) — those pass through untouched.

Reproduction: run the chatbot with CHATBOT_LLM_PROVIDER=anthropic and a current Claude model and send a message that elicits a multi-block/signed reply. Before this change the bot returns 1 validation error for Document page_content Input should be a valid string [...]; after, it replies normally and the turn persists.

Testing

  • Added TestMessageContentToText in services/chatbot/src/chatbot/tests/test_agent_utils.py covering: plain-string passthrough, a list of signed text blocks (the crash case), skipping thinking/tool_use blocks, joining multiple and bare-string blocks, None, non-str/list coercion, and the "always returns str" contract.
  • cd services/chatbot/src && python -m pytest chatbot/tests/test_agent_utils.py → all pass.
  • Deployed locally with the Anthropic provider + a current Claude model: chat replies and history/RAG persistence now work where they previously raised the pydantic error.

Documentation

No documentation changes required — internal robustness fix, no public API/behavior change.

Checklist:

  • My changes generate no new warnings
  • I have added tests that prove my fix is effective
  • Any dependent changes have been merged
  • I have documented any changes if required in the docs (n/a)

…roviders

Signed-off-by: aarshx05 <115548002+aarshx05@users.noreply.github.com>
@aarshx05
aarshx05 force-pushed the fix/chatbot-structured-content branch from 4e3e852 to ccf55cd Compare October 30, 2026 08:26
The Newman integration tests were failing because docker compose up -d
returns immediately while services are still booting. Added --wait flag
so compose blocks until all health checks pass, plus a secondary curl
loop to verify the gateway is actually accepting connections on port
30080 before Newman runs.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011FfMVcQ5F2gnZidb2gtuLU

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants