# Billing toggle experiment brief

Status: proposed; outcomes unmeasured.
Toolkit: https://www.lazyweb.com/research/billing-toggle-experiment-kit

## Ownership and scope

- Product / URL / owner: [fill in]
- Feature tier, currency, monthly price and annual total: [fill in]
- Eligible population and exclusions, including tagged QA: [fill in]
- Randomization unit and persistent assignment: [fill in]
- Treatment: [savings label / monthly-equivalent annual price / default cadence]

## Observed UI differences

- Public captured reference and exact frame: [URL]
- Capture date, review date and audience context: [fill in]
- What the reference actually shows: [describe only the visible UI]
- What changed between captured variants: [fill in, or unavailable]
- Other simultaneous differences and evidence limitations: [fill in]
- Published measured outcome: [source and denominator, or unavailable]

## Proposed hypothesis

We hypothesize [outcome] because [mechanism]. This is a prediction, not a measured result.

- Control: [current presentation]
- Experiment: [one changed presentation choice]
- Prices, feature tier, eligibility, trial and checkout terms held constant: [list]
- Screenshot or implementation reference for each variant: [link]

## Price disclosure

- [ ] Selected cadence and total due are visible beside price and purchase action.
- [ ] Annual monthly equivalent is labeled as equivalent, with annual charge visible.
- [ ] Billing and renewal periods, currency, taxes and fees match checkout.
- [ ] Savings arithmetic uses the actual monthly price for the same tier.
- [ ] Trials disclose amount due now, duration, first charge date and renewal terms.
- [ ] Monthly choice remains available; keyboard and screen-reader states work.
- [ ] Narrow screens retain all disclosures; comprehension is checked.

## Measurement plan before launch

- Primary metric numerator / denominator: [completed subscriptions of chosen cadence, or all paid subscriptions] / all eligible randomized visitors.
- Existing baseline and its source/window: [fill in; do not borrow the study's UI shares]
- Assignment, exposure, checkout and completed-payment events and join keys: [fill in]
- Allocation, conversion window and late conversions: [fill in]
- Minimum detectable effect, power, sample size and stopping rule: [calculate from baseline]
- Fixed observation start/end and guardrail follow-up window: [fill in]
- Guardrails: overall paid conversion, net revenue per randomized visitor, refunds, checkout abandonment and price-comprehension errors.
- Guardrail thresholds, owner and response: [predeclare]
- Assignment imbalance, duplicate events, missing payments, QA exclusion and data-quality checks: [fill in]
- Analysis method, uncertainty interval and interaction handling: [predeclare]

## Measured outcomes after the test

- Outcome status: [unmeasured / met / not met / inconclusive / could not check]
- Actual observation and follow-up windows: [fill in]
- Eligible randomized visitors by variant and exclusions: [fill in]
- Primary numerator, denominator, rate, difference and uncertainty by variant: [fill in]
- Guardrail values, denominators and uncertainty: [fill in]
- Deviations, confounders and missing data: [fill in]
- Interpretation: [separate observed facts from hypotheses]
- Decision and smallest next step: [ship / revise / stop; owner and date]

## Source scope and limitations

July 2026 billing study:
https://www.lazyweb.com/research/monthly-vs-annual-billing-toggle-pricing
- Annual billing mentions: 172/301 pricing-area experiments (57.1%).
- Billing-toggle mentions: 102/301 experiments (33.9%).
- Default-plan preselection/emphasis mentions: 54/301 experiments (17.9%); not an annual-default-only count.
- Toggle presence: 58/255 app “All plans” screens (22.7%) across 24 companies.
Keyword/regex lower bounds over model-written summaries and vision descriptions;
mentions are not audited pages. Screens and experiments are different units.
The sample is not a representative census. Differences and inferred rationales
are not measured conversion lift.

Cal AI reference:
https://www.lazyweb.com/company/cal-ai/app-tree#frame-44
https://www.lazyweb.com/company/cal-ai/app-tree#frame-45
Historical August 2026 frames, multiple simultaneous differences; no isolated
causal effect, completed subscription or current offer is established.

Captured database experiment pairs (outside the July study denominators):

### Walmart+ · annual savings and plan choice

- Control · recorded 2024-04-10 UTC: https://zlfyzdmohcskkucuunmk.supabase.co/storage/v1/object/public/screenshots/shopping4_walmart/lazyscreenshots/2024_04_10_02-18-08_577F2BBE.png
- Experiment · recorded 2024-04-26 UTC: https://zlfyzdmohcskkucuunmk.supabase.co/storage/v1/object/public/screenshots/shopping4_walmart/compare/h_apr%2026%2C%202024%20at%208_24_pm.png_Jun%2030%2C%202024%20at%207%3A17-PM.png/apr%2026%2C%202024%20at%208_24_pm.png

Observed difference: The monthly-only trial disclosure becomes an Annual/Monthly picker. Annual is selected and shows $98/year, $8.17/month and “Save $55+/year”; Monthly remains $12.95/month.

Evidence limit: The plan picker, selected cadence, savings callout and price headline change together. The badge’s independent effect cannot be isolated. No assignment, sample size or measured outcome is available.

### IFTTT · annual total to monthly equivalent

- Control · recorded 2025-05-18 UTC: https://zlfyzdmohcskkucuunmk.supabase.co/storage/v1/object/public/screenshots/36z_ifttt/compare/%21screen/2025_05_18_17-12-39_9009CCBB.png
- Experiment · recorded 2026-01-28 UTC: https://zlfyzdmohcskkucuunmk.supabase.co/storage/v1/object/public/screenshots/uploaded_ifttt/compare/2026-01-28/1771006697818_t__-_948.PNG

Observed difference: The selected Yearly price changes from $34.99/year to $2.50/month. The footer’s annual charge changes from $34.99 to $29.99, Monthly changes from $3.99 to $4.99, and the savings badge changes from 26% to 49%.

Evidence limit: Both actual prices and the savings label change alongside the framing, so this is not a price-framing-only test. The control’s displayed trial end date precedes its recorded date; current eligibility and immediate charge cannot be established. No measured outcome is available.

### Uber One · monthly to annual selection

- Control · recorded 2023-10-06 UTC: https://zlfyzdmohcskkucuunmk.supabase.co/storage/v1/object/public/screenshots/travel1_uber/lazyscreenshots/2023_10_06_11-55-08_EC639348.png
- Experiment · recorded 2023-11-13 UTC: https://zlfyzdmohcskkucuunmk.supabase.co/storage/v1/object/public/screenshots/travel1_uber/lazyscreenshots/2023_11_12_19-31-58_0CDC129B.png

Observed difference: Monthly is the selected first card in Control; Annual is the selected first card in Experiment. The same $9.99/month and $99.99/year ($8.33/month equivalent) offers remain visible, while the billing disclosure follows the selected cadence.

Evidence limit: Card order changes with selected cadence. The stored pair documents the captured states, not verified randomized assignment or an untouched first-load default. No measured outcome is available.

These original stored pairs document UI changes, not verified randomized assignment or measured outcomes. Do not reconstruct absent screens, invent prices, or fill missing disclosures. Record an unavailable immediate charge as unavailable.

Cadence and feature tiers:
https://www.lazyweb.com/research/annual-billing-toggle-vs-plan-tiers

## Agent research prompt

Research billing presentation for [product URL] and [audience]. Use Lazyweb’s experiment database to find actual Control and Experiment screenshot pairs for savings labels, annual monthly-equivalent prices and cadence selection. Return original image URLs, recorded dates, visible differences and evidence limits. Do not generate or reconstruct screens. Separate UI observations from hypotheses and measured outcomes. Flag simultaneous price, trial, layout or tier changes and missing disclosures. Propose a test only as a hypothesis, with total due and billing period visible, a primary metric per eligible randomized visitor, guardrails and a fixed observation window. Report no conversion lift unless the source provides measured outcomes.

Connect Lazyweb: https://www.lazyweb.com/mcp-install?utm_source=lazyweb&utm_medium=research&utm_campaign=billing-toggle-kit&utm_content=brief