# AI-search measurement exercise, version 1

Created October 25, 2026 by GTM Vibes. Every observation, company, report value and result in this packet is fictional. The source labels in the session table illustrate familiar reporting values; they are not real referral results. No prompts were sent to a model and no campaign was run.

## Files and use

- `measurement-plan.csv`: editable worksheet. Fill the blank `your_plan` column before collecting your own data. The worked-example column is a reference, not your campaign setup.
- `panel-observations.csv`: 12 scheduled prompt trials across two dates. Read the status before interpreting a citation value.
- `session-observations.csv`: 13 synthetic session rows, including deliberate exclusion and timezone cases.
- `worked-readout.md`: calculations and interpretation. Try the questions below before reading it.

Open the CSVs in a spreadsheet app or a text editor. They contain values, not executable formulas, macros or external connections. The worked readout shows the calculations explicitly. Duplicate the plan to reuse it; preserve these original examples and credit GTM Vibes if you include them in a portfolio.

## Panel rules

The same three prompt strings, two trial slots, fictional engine/version, mode, locale and account conditions were planned for October 15 and October 22. The fictional content change is a guide clarification between those dates; there is no control group. `planned_at_utc` records the planned slot. `observed_at_utc` is blank when the slot was missed.

The target is `jobs.example`. `cited_target=1` means a completed answer contains at least one link to that domain. `0` means the completed answer contains no target link. Multiple target links still count once. A name-only mention does not count. This is the exercise's declared citation rule, not a platform impression definition.

- `complete`: the answer was inspected, so citation is 0 or 1.
- `error`: an attempted run failed, so citation is blank.
- `missing`: a planned slot was not observed, so citation is blank.

Do not fill either kind of blank with zero. Do not drop the scheduled slot when calculating coverage. Keep retries as new records rather than replacing an unfavorable or failed attempt; this packet has no retries.

Calculate each wave's completion coverage as completed slots divided by six planned slots. Its observed citation rate is cited completed answers divided by completed answers. If no answers completed, the rate is unavailable, not zero.

For the matched comparison, pair by `prompt_id` plus `trial`, under the unchanged engine/version/mode/locale/account conditions. Keep only pairs complete in both waves. Report the size and exclusions of that subset alongside its rates. This controls which slots are compared; it does not make a tiny, chosen panel representative of users or identify an effect caused by the content change.

`evidence_note` is an authored teaching annotation, not a saved model response. In a real run, keep the permitted response capture or reference, cited URL, observation timestamp and error detail. Record a model/version as `unknown` if the interface does not reveal it.

## Session and outcome rules

The fictional property timezone is **America/New_York**. Include session starts from **October 15, 2026 at 00:00 inclusive through October 23 at 00:00 exclusive** in that timezone. In this interval New York is UTC−04:00, so the UTC interval is `[2026-10-15T04:00:00Z, 2026-10-23T04:00:00Z)`.

Then require `hostname=jobs.example` and `qa_marked=0`. Keep excluded rows in the raw file and explain the exclusions. Each unique `session_id` is one fictional session. No identity join is available across the panel and session data, and they must not be joined by coincident dates.

For this exercise only, the recorded AI-source bucket requires:

1. `session_source` exactly `chatgpt.com` or `perplexity.ai`; and
2. `session_medium` exactly `referral`.

This is a small teaching allowlist, not a current complete list of AI sources. The fields are already-assigned session acquisition dimensions, not raw arrival-referrer evidence and not proof of a particular assistant answer. Keep `google / organic`, `(direct) / (none)`, newsletter and social rows separate. Do not rename unknown/direct traffic as AI.

`guide_to_jobs_events` counts clicks from a guide to a jobs hub. `apply_click_events` counts clicks on an outbound application link. For the main useful-action measure, count included sessions with **at least one** apply click and divide by included sessions. Also report the raw click-event count. Two clicks within one session are two events and one session with an action. The fields do not tell you whether someone completed an application, received an interview or was hired. These are exercise event names, not automatically collected GA4 metrics.

All rows are synthetic observed session records, with no modeled data. The fictional extract timestamp is **2026-10-25T09:00:00Z**. For real reports, record consent/measurement coverage, reporting identity, extract time, relevant filters and whether the data are preliminary. Missing or unmeasured activity must not be invented as extra sessions.

## Separate Search Console input

The fictional operator also has these site-level snapshots for October 15–22, using the standard date-range view in **Pacific Time**, extracted on October 25. The numbers are invented inputs, not sums from the session file:

| Report | Metric | Synthetic value |
|---|---|---:|
| Google Search generative AI performance | Impressions | 30 |
| Google Search overall Web performance | Impressions | 120 |
| Google Search overall Web performance | Clicks | 8 |

The GenAI report is an overlapping view within overall Web data. Do not add 30 and 120. Do not divide eight overall Web clicks by 30 GenAI impressions and label the result an AI click-through rate. This packet has no AI-specific click count. Standard Search Console days and this Analytics property's days use different boundaries.

The real GenAI report's help also warns that unavailable values displayed as `~` or `-` export as zeros. Preserve the original availability state when you record or reconcile an export; a transformed export zero alone cannot establish a measured zero. See [Google's report documentation](https://support.google.com/webmasters/answer/16984139).

## Questions to answer

1. How many scheduled trials completed in each wave? What are the citation rates among completed answers?
2. Which prompt/trial pairs are comparable in both waves? How do the matched rates differ from the all-completed rates?
3. Which session rows should be excluded, and why? Check the UTC/local date boundary explicitly.
4. How many included sessions are in the declared AI-source bucket? How many apply-click events and sessions with an apply click occur in that bucket and overall?
5. Can any supplied number establish an AI-specific click-through rate, a completed application count, total AI-influenced visits or causal lift?
6. What would you inspect or collect next before recommending more investment?