Superseded. This is an early comparison from July 2026: one run per task, no placebo, and the model without the profile never saw the rules it was scored against. It does not show what the profile does. Two later tests replace it: a pre-registered placebo test, where the mined profile did not separate from an invented one of the same length on any of 9 comparisons, and a September rerun, where it separated on 5 of 9 on Fable 5.1 and 1 of 9 on Opus 5. Both measure whether output changed, not whether it got better. Read those first.
Emulo Bench  ·  Rule adherence v1

Same prompt. Same model.
One of them had read your rules.

Claude Opus 4.8 ran eight tasks twice. Identical prompt and identical files both times. The only difference is whether it first read a real Emulo profile mined from my own coding sessions. Then every output was scanned for things the profile explicitly forbids.

12
Rule violations
Opus 4.8 alone
1
Rule violations
Opus 4.8 + Emulo
8
Tasks
each run twice

By task

Lower is better
Task
Prompt
Opus 4.8
+ Emulo
design
"make the hero better"
4
0
write
"launch post for Orbit"
1
0
copy
"connect-calendar UI copy"
0
0
code
"harden this, keep it correct"
0
0
pricing
"improve the pricing page"
5
1
feature
"feature post: auto timezone"
0
0
changelog
"v2.0 release notes"
0
0
empty
"empty state screen"
2
0

By rule

Each line is quoted from the profile
RuleWhat the profile saysOpus 4.8+ Emulo
purpleTaste: black/white (anti-purple)90
em-dashNo em-dashes.11
fabricated-metricNo fake data. Show `--`, never a fabricated number.10
rule-of-threeNo forced rule-of-three.10

Every violation, with its evidence

Check the calls yourself
Opus 4.8designpurple#5b4ef0 (hue 245deg sat 0.67)
Opus 4.8designpurple#7c5cf5 (hue 253deg sat 0.62)
Opus 4.8designpurple#4335cf (hue 245deg sat 0.74)
Opus 4.8designfabricated-metric2,000+ teams
Opus 4.8writerule-of-threeNo thread. No poll. Just the time.
Opus 4.8pricingpurple#5b4bdb (hue 247deg sat 0.66)
Opus 4.8pricingpurple#3d2fb8 (hue 246deg sat 0.74)
Opus 4.8pricingpurple#2a2850 (hue 243deg sat 0.50)
Opus 4.8pricingpurple#231d5c (hue 246deg sat 0.68)
Opus 4.8pricingpurple#4a3bc4 (hue 247deg sat 0.70)
+ Emulopricingem-dash—
Opus 4.8emptypurple#a26cff (hue 262deg sat 0.58)
Opus 4.8emptyem-dash–

How it works

Built to be reproducible
01

Identical prompts

Both runs get the same task text and the same starting files. Nothing in the prompt hints at a preferred answer, and no taste is leaked into either side.

02

One model, two runs

Claude Opus 4.8 both times. One run alone, one after reading the real profile file. That single difference is the whole experiment.

03

The real profile, not a brief

The Emulo run reads the actual mined profile the tool ships. No hand-written instructions were written for the task.

04

Counted, not judged

No taste panel. A violation is a hue in the purple range, a fabricated metric, an em-dash. Every hit prints the matched text.

05

A neutral product

Orbit, a scheduling app neither run had context on.

06

Emulo's own miss stays on the board

Emulo broke one of its own rules: an em-dash in the pricing copy. It is in the table above. A result that hid it would be a pitch.

What this is not

This does not claim Emulo produces better work. What it counts is narrower: how often each output broke rules written in the profile. The plain run never saw those rules, and one run each way cannot tell the profile's content apart from simply being handed a long, confident document. The placebo test was built to check exactly that, and there the mined profile did not beat an invented one. On a separate blind panel judging generic quality, the plain model actually won several tasks, partly by doing the very things these rules forbid, because fabricated social proof and a tidy rule-of-three score well with a neutral judge.

Read it as directional: eight tasks, one operator, one model. The scanner is regex and hue math, so it catches concrete rules and cannot see a vague one. It also missed three purples on its first run until the detector was rewritten to measure hue instead of matching a list of hex codes, which is a fair thing to be skeptical about.