Are these hero and CTA benchmarks measured A/B lift or just observed changes?
They are observed changes, not measured lift: all 488 hero and 232 CTA figures come from detected before/after UI diffs with model-inferred rationale, drawn from 2,160 annotated experiments.[1] The impact and confidence scores are model-assigned 1-5 values usable only for relative ranking.[2] Read every stat as 'what changed and the likely reason,' never as a proven conversion result.
Hero/CTA stats are detected UI diffs with inferred rationale, not measured lift — 2,160 annotated experiments, July 2026.
Finding: detected, not measured
The experiment corpus captures before/after UI changes and an LLM's inferred rationale for each. It does not contain a measured conversion delta.[1] So a statement like 'Cycle changed its CTA from demo to free trial' is verifiable from the diff; 'and it lifted signups 12%' is not something this data can say.
Breakdown: how to read the scores
Impact and confidence are model-assigned 1-5 scores, meaningful only for relative ranking at N>=20.[2]
| Area | Avg impact (1-5) |
|---|---|
| FORM | 3.84 |
| OFFER | 3.69 |
| PRICING | 3.52 |
| HERO | 3.28 |
| CTA | 3.16 |
A 3.84 form score does not mean 'forms convert 3.84x' — it means the model rated form changes as relatively higher-impact than CTA changes in this set.
How to apply
Use these benchmarks to prioritize what to test and to see how peers frame changes — not to skip your own experiment. Keyword-incidence theme counts are non-exclusive ILIKE matches and are lower bounds. Every downstream page in this family carries the same caveat: observed change plus inferred rationale, validated by your own test.
The numbers
| Stat | Computed from |
|---|---|
| 2,160 annotated experiments (of 4,814 detected); hero 488, CTA 232 are before/after diffs with inferred rationale | hero_experiments_total / cta_experiments_total / smallSampleWarnings[detected not lift] |
| Impact/confidence are model-assigned 1-5 scores (FORM 3.84, OFFER 3.69, PRICING 3.52, HERO 3.28, CTA 3.16) | experiment_area_leaderboard |
Sources & citations
- [1] Lazyweb Research analysis of 2,160 annotated experiments (detected-experiment corpus), July 2026. Detected before/after UI diffs with LLM-inferred rationale; not measured A/B lift. ↩
- [2] Lazyweb Research analysis of 2,160 annotated experiments (detected-experiment corpus), July 2026. Impact/confidence are model-assigned 1-5 scores for relative ranking only at N>=20. ↩
Source: Lazyweb Research — proprietary analysis of real, in-market app screens. Cite as Lazyweb Research, 2026-07-07.