# Are these hero and CTA benchmarks measured A/B lift or just observed changes?

Source: Lazyweb Research
Published: 2026-07-07
Sample size: n=2160
Tags: landing-page, experiments, ux-patterns, web, mobile
HTML: https://www.lazyweb.com/research/why-hero-and-cta-tests-arent-measured-lift
Markdown: https://www.lazyweb.com/research/why-hero-and-cta-tests-arent-measured-lift.md

**Answer.** They are observed changes, not measured lift: all 488 hero and 232 CTA figures come from detected before/after UI diffs with model-inferred rationale, drawn from 2,160 annotated experiments.[1] The impact and confidence scores are model-assigned 1-5 values usable only for relative ranking.[2] Read every stat as 'what changed and the likely reason,' never as a proven conversion result.

> Hero/CTA stats are detected UI diffs with inferred rationale, not measured lift — 2,160 annotated experiments, July 2026.

## Finding: detected, not measured

The experiment corpus captures before/after UI changes and an LLM's inferred rationale for each. It does not contain a measured conversion delta.[1] So a statement like 'Cycle changed its CTA from demo to free trial' is verifiable from the diff; 'and it lifted signups 12%' is not something this data can say.

## Breakdown: how to read the scores

Impact and confidence are model-assigned 1-5 scores, meaningful only for relative ranking at N>=20.[2]

| Area | Avg impact (1-5) |
|---|---|
| FORM | 3.84 |
| OFFER | 3.69 |
| PRICING | 3.52 |
| HERO | 3.28 |
| CTA | 3.16 |

A 3.84 form score does not mean 'forms convert 3.84x' — it means the model rated form changes as relatively higher-impact than CTA changes in this set.

## How to apply

Use these benchmarks to prioritize what to test and to see how peers frame changes — not to skip your own experiment. Keyword-incidence theme counts are non-exclusive ILIKE matches and are lower bounds. Every downstream page in this family carries the same caveat: observed change plus inferred rationale, validated by your own test.

## The numbers

| Stat | Computed from |
| --- | --- |
| 2,160 annotated experiments (of 4,814 detected); hero 488, CTA 232 are before/after diffs with inferred rationale | hero_experiments_total / cta_experiments_total / smallSampleWarnings[detected not lift] |
| Impact/confidence are model-assigned 1-5 scores (FORM 3.84, OFFER 3.69, PRICING 3.52, HERO 3.28, CTA 3.16) | experiment_area_leaderboard |

## Methodology

Universe: 2,160 annotated experiments, July 2026. Method: before/after UI-diff annotation with model-inferred rationale and 1-5 impact/confidence scores. Caveat: no measured conversion lift; scores rank relatively; keyword themes are non-exclusive lower bounds.

## Sources & citations

- [1] Lazyweb Research analysis of 2,160 annotated experiments (detected-experiment corpus), July 2026. Detected before/after UI diffs with LLM-inferred rationale; not measured A/B lift.
- [2] Lazyweb Research analysis of 2,160 annotated experiments (detected-experiment corpus), July 2026. Impact/confidence are model-assigned 1-5 scores for relative ranking only at N>=20.

## Related questions

- [What are companies A/B testing on their landing-page heroes right now?](https://www.lazyweb.com/research/what-are-companies-ab-testing-on-landing-hero)
- [Which product categories have enough landing pages to benchmark against?](https://www.lazyweb.com/research/which-categories-have-enough-landing-pages-to-benchmark)
- [What % of landing pages use a product screenshot vs video vs abstract art in the hero?](https://www.lazyweb.com/research/can-i-measure-hero-media-mix-from-this-data)