Overall Winners Across All Variations
Quality
GPT-5.6 Sol
Best Value
Qwen3 235B
Fastest
Qwen3 235B
The Take
- We ran it twice, no format options.
- Ask casually: Mistral and Nova open with “Certainly!”, four more fence it, only 5 of 11 are clean.
- Add “return ONLY JSON, no markdown”: 0 preambles, 9 of 11 clean.
- The preamble isn’t gone, it’s dormant. Stop policing and half revert.
- GPT-5.6 Sol, Gemini 3.1 Pro, Grok 4.3, and Qwen3 235B were clean both ways.
Variation 1
The Casual Ask
How most people actually prompt: name the keys, but drop the ONLY and the no-markdown rules. Do the old manners come back?
The Prompt
Return a valid JSON object with these keys: name (string), age (number), active (boolean)
Quality Winner:
GPT-5.6 Sol
Best Value:
Qwen3 235B
Fastest:
Qwen3 235B
GPT-5.6 Sol
Quality
{"name":"Alex","age":30,"active":true}
Clean Raw JSON
Claude Opus 5
```json
{
"name": "Alice Johnson",
"age": 32,
"active": true
}
```
Not Raw JSON
Gemini 3.1 Pro
{
"name": "Jane Doe",
"age": 28,
"active": true
}
Clean Raw JSON
Grok 4.3
{
"name": "Alice",
"age": 30,
"active": true
}
Clean Raw JSON
DeepSeek V3
```json
{
"name": "John Doe",
"age": 30,
"active": true
}
```
Not Raw JSON
Qwen3 235B
Value
{
"name": "Alice",
"age": 30,
"active": true
}
Clean Raw JSON
Llama 4 Maverick
```json
{
"name": "John Doe",
"age": 30,
"active": true
}
```
Not Raw JSON
Mistral Large 3
Here is a valid JSON object with the requested keys:
```json
{
"name": "John Doe",
"age": 30,
"active": true
}
```
You can customize the values as needed. For example:
- `name`: Any string (e.g., `"Alice"`).
- `age`: Any number (e.g., `25`).
- `active`: Either `true` or `false`.
Not Raw JSON
Nova Pro
Certainly! Below is a valid JSON object with the specified keys: `name` (string), `age` (number), and `active` (boolean).
```json
{
"name": "John Doe",
"age": 30,
"active": true
}
```
Here's a brief explanation of each key:
- `"name": "John Doe"`: The `name` key holds a string value.
- `"age": 30`: The `age` key holds a number value.
- `"active": true`: The `active` key holds a boolean value.
Not Raw JSON
Nemotron 12B
{
"name": "<THE_PERSONS_NAME>",
"age": <PERSONS_AGE>,
"active": <TRUE or FALSE>
}
Clean Raw JSON
Kimi K2.5
```json
{
"name": "Alex",
"age": 28,
"active": true
}
```
Not Raw JSON
The Take
- Drop one sentence and the habits come back.
- Mistral and Nova open with “Here is a valid JSON object…” and “Certainly!”
- Claude Opus 5, DeepSeek, Llama, and Kimi wrap theirs in a fence.
- 5 of 11 hand back a bare object. Nemotron’s is clean but uses placeholder values.
Variation 2
The Hard Ask
Now push hard: the same keys, but explicitly forbid explanations, markdown, and code fences. Maximum instruction. Do they fall into line?
The Prompt
Return ONLY a valid JSON object with these keys: name (string), age (number), active (boolean). Do not include any explanation, greeting, markdown, or code fences. Output nothing but the raw JSON object.
Quality Winner:
GPT-5.6 Sol
Best Value:
Qwen3 235B
Fastest:
Llama 4 Maverick
GPT-5.6 Sol
Quality
{"name":"Alex","age":30,"active":true}
Clean Raw JSON
Claude Opus 5
{"name": "John Doe", "age": 30, "active": true}
Clean Raw JSON
Gemini 3.1 Pro
{
"name": "John Doe",
"age": 30,
"active": true
}
Clean Raw JSON
Grok 4.3
{"name":"Alice","age":25,"active":true}
Clean Raw JSON
DeepSeek V3
{"name": "John Doe", "age": 30, "active": true}
Clean Raw JSON
Qwen3 235B
Value
{"name": "John Doe", "age": 35, "active": true}
Clean Raw JSON
Llama 4 Maverick
Speed
{"name": "John Doe", "age": 30, "active": true}
Clean Raw JSON
Mistral Large 3
```json
{
"name": "John Doe",
"age": 30,
"active": true
}
```
Not Raw JSON
Nova Pro
{
"name": "John Doe",
"age": 30,
"active": true
}
Clean Raw JSON
Nemotron 12B
```json
{
"name": "John Doe",
"age": 30,
"active": true
}
```
Not Raw JSON
Kimi K2.5
{"name": "Alex", "age": 28, "active": true}
Clean Raw JSON
The Take
- Told firmly, they behave. 9 of 11 return raw JSON, 0 preambles.
- Only Mistral Large 3 and Nemotron still fence it, and the JSON inside is valid.
- One extra sentence moves compliance from 5 to 9.
Try this comparison in AI Lab
See the full comparison, test your own prompts, and compare any models you want. No commitment on the monthly plan.
Models We Didn't Test
—
Reasoning variants (o3, R1, Grok reasoning): Excluded to keep this a fast one-shot snapshot of chat models; reasoning models add latency without changing formatting behavior.
Are you a model provider? Don't see your model here? Get in touch — we'll evaluate it for AI Lab integration.