# A/B TEST PLAN - Gritting Service PPC landing page

## Variants

**Variant A - "Form above the fold" (control-plus)**
Split hero: value proposition left, 7-field quote card right and visible on
load at ≥ 1000 px. Primary CTA `Get a gritting quote`, secondary click-to-call.
Three inline assurance items.

**Variant B - "Call-first"**
Single-column centred hero, no form above the fold. Primary CTA is a large
click-to-call button; secondary CTA scrolls to the quote form. A four-tile
benefit strip sits immediately below the hero; the identical form appears
mid-page.

Everything below the hero is byte-identical between variants, so any difference
is attributable to the hero treatment.

## Hypotheses

- **H1 (A):** Placing the quote form in the initial viewport removes a
  navigation step and will increase **form submission rate** on desktop paid
  traffic.
- **H2 (B):** Gritting demand is urgent and weather-triggered; a phone-first
  hero will increase **total conversions (form submits + phone clicks)**,
  driven by mobile, even if form submits alone fall.

Primary metric is deliberately **total conversion rate = (form_submit +
phone_click + callback_open) / sessions**, so the two variants are compared on
business outcome rather than on the mechanism each favours.

## KPIs

| Metric | Source | Role |
|---|---|---|
| Total conversion rate | GA4 (`form_submit`+`phone_click`+`callback_open`) | **Primary** |
| Form submit rate | `form_submit` / sessions | Secondary |
| Phone click rate | `phone_click` / sessions | Secondary |
| CTA click-through to form | `cta_click` where `cta_location` contains `hero` | Diagnostic |
| Form start → submit completion | `form_start` → `form_submit` | Diagnostic |
| Micro-engagement | `faq_open`, `video_play`, `scroll_depth 75` | Guardrail |
| Bounce / engagement rate, LCP | GA4 + CrUX | Guardrail |
| Cost per conversion | Google Ads | Business |

## Sample size

Two-proportion test, 95 % confidence (α = 0.05), 80 % power, two-tailed.

| Baseline total CR | Minimum detectable effect | Sessions **per variant** |
|---|---|---|
| 3 % | +50 % rel. (→ 4.5 %) | ≈ 2,400 |
| 3 % | +25 % rel. (→ 3.75 %) | ≈ 8,700 |
| 5 % | +30 % rel. (→ 6.5 %) | ≈ 4,000 |
| 5 % | +20 % rel. (→ 6.0 %) | ≈ 8,700 |
| 8 % | +25 % rel. (→ 10 %) | ≈ 3,400 |

Formula used: `n = 2(z(1-α/2)+z(power))² · p̄(1-p̄) / (p₁-p₂)²`.
Recalculate with the real baseline once one week of data exists -
`https://www.evanmiller.org/ab-testing/sample-size.html`.

## Duration

- Run a **minimum of 2 full weeks** (covers weekday/weekend and both halves of a
  typical B2B buying week), and never stop mid-week.
- Winter search demand spikes with cold snaps. Run **at least one cold-weather
  event and one mild week** in each variant, or the result reflects weather, not
  design. Practical target: **3-4 weeks**, or until the per-variant sample above
  is reached - whichever is later.
- Do not peek-and-stop. Fix the stopping rule up front, or use a sequential
  testing method if you must monitor continuously.
- Exclude the first 48 h from analysis if tracking changes were deployed with
  the test.

## Implementation on the legacy CMS

Recommended: **two published URLs, 50/50 at the ad level** (Google Ads ad
rotation or two ad groups with equal budget), `rel=canonical` from B to A, and
`?utm_content=hero_a|hero_b` on the final URLs. No client-side flicker, no CLS
penalty, no cloaking risk.

Fallbacks and their trade-offs are documented at the bottom of
`gritting-service-variant-b.html`.

## Expected impact (planning assumptions, not guarantees)

Based on typical uplift ranges for adding an above-fold form, prominent
click-to-call and a mobile sticky action bar to a text-heavy legacy service page:

- Total conversion rate: **+20 % to +60 %** vs the current page.
- Phone clicks on mobile: largest single gain, driven by the sticky bar.
- Micro-engagement (FAQ opens, video plays): higher on both variants, useful as
  a leading indicator in week 1 when conversion volume is still thin.

Treat these as hypotheses to be measured, not as promises to the client.

## Decision rules

- Ship the winner if the primary metric is significant at 95 % **and** no
  guardrail (LCP, engagement rate, cost per conversion) has degraded materially.
- If the result is flat, ship **Variant B on mobile / Variant A on desktop** -
  segment the reported data by device before concluding, because H1 and H2
  predict opposite outcomes by device.
- Log the result, then queue the next test: form length (7 fields vs 3), and
  social-proof placement.


## Update after the pass-2 redesign

Both variants now share one design system (tokens, frost/glass surfaces, reveal
system, trust strip, estimator, timeline, plan comparison, FAQ accordion, final
band and smart sticky CTA). **The only intentional differences remain the hero
and the CTA hierarchy** - see the table at the top of
`gritting-service-variant-b.html`.

This matters for validity: because everything below the hero is now identical,
any difference in outcome is attributable to the hero treatment alone. If you
change a shared component, change it in **both** files in the same release, or
the test is confounded.

### Sticky CTA change

The sticky bar is no longer mobile-only. It now appears on desktop too, after
the hero scrolls away, and hides itself whenever the quote form or the final
band is on screen. Because this applies equally to A and B it is not a variant
difference, but it does change the mobile baseline: expect `phone_click` volume
to shift between `sticky_bar` and hero locations. Segment by `cta_location`
before comparing against pre-launch data.

### Additional metrics to report

Secondary / diagnostic only - do not use them to call the test:

- `calculator_interact` and `calculator_cta_click` engagement rate.
- `plan_compare_view` reach, and whether plan-CTA split shifts toward the
  Fixed Season Price option.
- `trust_element_click` by `trust_element`.
- `sector_click` distribution (feeds future PPC ad groups).
- `scroll_cue_click` - a high rate on Variant B would suggest the phone-first
  hero is not closing visitors and they are looking for detail.

### Guardrails

Add two, given the extra motion and JS:

- **LCP** must stay within 10 % of the pre-pass measurement on mobile.
- **Console errors: zero.** The estimator, marquee and sticky CTA all depend on
  `IntersectionObserver`; verify the graceful fallbacks on any browser in the
  P1 matrix that lacks it.