2 / 7
LLM evaluation

Assert output format is valid JSON

Medium70 pts~25 min
  • Structured output
  • JSON mode
Practice app · Acme Support Assistant

A deterministic LLM-style support assistant with retrieval (RAG), JSON mode, safety policies and tool calls, exposed via UI and API.

BASE_URL
/api/practice
Console app
/lab/ai-testing-assert-output-format-is-valid-json

Your starter code already declares BASE_URL — call the API relative to it.

Objective

Request JSON mode and assert output_text is parseable JSON with the documented keys.

Your task

  1. 1Ask "How long do I have to request a refund?" with response_format { "type": "json_object" }.
  2. 2Parse output_text as JSON (the test must fail if parsing throws).
  3. 3Assert the object has a string "answer" containing “30 days” and an array "sources" containing "kb-refunds".

Acceptance criteria

  • POST /ai/chat returns 200
  • JSON mode is used
  • At least 3 assertions pass

LLM evaluation · AI Testing · Output validation