Vantage Analytics: Trial-to-paid conversion Dropped 23%

Product Management

easy40 min1 submission

Goldman Sachs

Scenario

You are the PM for Vantage Analytics's core B2B SaaS product.

Trial-to-paid conversion has fallen 23% over the past 11 days. The drop appears concentrated in Android and is most pronounced in the US.

Context that may or may not be relevant:

A redesigned onboarding flow shipped 12 days ago to 33% of users
Marketing paused a paid acquisition campaign 8 days ago
A competitor launched a free tier last month
The data team migrated the analytics pipeline 15 days ago

Your VP wants an answer by end of day and is already asking whether the redesign should be rolled back.

Supporting data

incident

metric: trial-to-paid conversion
drop pct: 23
window days: 11
concentrated region: the US
concentrated platform: Android

recent changes

onboarding rollout pct

33

onboarding redesign days ago

12

competitor free tier launched

last month

paid campaign paused days ago

8

analytics pipeline migrated days ago

15

Your task

Diagnose the drop. Provide:

Analysis — how you would structure the investigation and which hypotheses you would test in what order.
Risks — of acting too early, and of acting too late.
Recommendation — your diagnostic sequence and what you would tell the VP today.

Be specific about the data cuts you would pull.

Ready to move forward? Up next: Marlow Chemicals: One Quarter, Five DemandsNext question

How you'll be graded

100 points, 60% to pass.

data analysis25
recommendation25
problem structuring25
hypothesis generation25

Hint

Reveal suggested structure

Is it real? Check instrumentation first — the pipeline migration is the cheapest hypothesis to eliminate and the most embarrassing to miss.
Decompose the metric. For DAU: new + returning + resurrected − churned. Which component moved?
Segment. Platform, region, cohort, acquisition channel, new vs existing users.
Timeline. Does the drop align with a specific deploy, or is it gradual?
Internal vs external. Internal changes are testable and reversible; external ones are not.
Act on the evidence, with a rollback only if the evidence points there.