Case study: how an agency 3x'd campaign ROI by re-vetting creators

Agency size: 14 people. Roughly 200 creator sponsorships per quarter across 6 brand clients. Prior workflow: human-only vetting, gut feel, fast turnaround. The problem they came to us with: too many campaigns underperforming the brief, and no clean way to predict which creators were going to convert.

What they were doing before

The agency's research analysts spent about 25 minutes per creator manually scrolling the profile, checking recent posts, eyeballing engagement, and writing a one-paragraph recommendation. They'd surface roughly 40 creators per brand per quarter to a senior strategist who picked the final 8–10.

Where the workflow was leaking

Two places. First: the analysts didn't have time to deep-check bot signal — they'd glance at the comments and call it. Second: the senior strategist was making final picks based on 'this one feels good for the brand' rather than a structured score. Both decisions had blind spots that compounded.

What changed

• Analysts ran every shortlisted creator through an automated audit before writing the recommendation.
• Creators below a tier B floor were auto-excluded from the shortlist.
• Senior strategists got the structured score + sub-scores + red flags alongside the analyst's qualitative read.
• Quarterly: re-audited every creator the agency was planning to re-engage with.

The numbers, 6 months in

• Average per-campaign conversion lift: 3.1x against the prior baseline.
• Wasted spend (campaigns under 0.5x ROI): down from 23% to 9%.
• Time per creator vetting: down from 25 min to 8 min (automation surfaced what to focus on).
• Creator re-engagement quality: re-audits caught 11% of previously-tier-A creators that had drifted to tier C.

Things that didn't change

The agency didn't replace the human read. The structured score complements the analyst's qualitative take — neither one alone is sufficient. The structured score catches the kind of bot/engagement-pod signal humans miss; the human catches brand-fit nuance the scoring model doesn't capture (tone, taste, prior partnerships).

The audits didn't make our analysts faster — they made our senior strategists slower in the right ways. We started having actual conversations about why a tier-B creator might still be a yes.