Finding the 6x gap in Amplitude's AI search attribution.

Amplitude's LLM-attributed signups didn't make sense

Claude and ChatGPT usage had gone vertical, but our attributed signup number sat stubbornly at 3% for months. Either Amplitude was the one company AI models had collectively decided to ignore, or the measurement was wrong.

Testing the links gave us the likely answer. ChatGPT app links sometimes carried UTMs and sometimes didn't. Claude stripped tracking completely. Those visits were landing in the Direct bucket instead of AI search.

The lightweight build

My engineering partner Zac Bennett and I decided to stop inferring and ask. We used Amplitude Guides & Surveys to put one question directly in the product after signup.

1. 01
   Show the survey at the data setup step, on the user's second visit and ten seconds into the session.
2. 02
   Randomize the order of nine possible discovery sources to reduce ordering bias.
3. 03
   If someone chose AI search, ask a follow-up about the model and prompt they used.
4. 04
   Start with a percentage of eligible users, watch activation, then scale only after confirming it did not move.

The result

The direct response made the attribution gap hard to ignore.

AI search tied word of mouth as our second-largest reported source and sat just behind search at 19.7%.

The important caveat

That limitation matters, but it doesn't erase the finding. Even among this qualified group, relying only on click attribution understated AI influence by roughly six times.

What we're building next

The next version moves the question directly into signup while minimizing any effect on completion rate. That should bring the measurement closer to the moment of conversion and give us a cleaner denominator.

The broader lesson is simple: attribution and influence are not the same thing. A dashboard records the touchpoint it can see. A person remembers the journey. For AI discovery, we need both datasets.