Facebook A/B Test: Primer Audience vs. Facebook Native Targeting

Artifact Summary

• Conducted thorough A/B test campaign comparing Facebook’s “Advantage+” targeting vs Primer audience
• The campaign featured two identical ad sets with only the audience definition changed:

• Ad Set A: Facebook’s Advantage Plus targeting
• Ad Set B: Custom Primer audience aligned to ICP

• Despite substantially broader reach, Facebook's generalized targeting failed to deliver qualified leads
• Primer audience produced 100% conversion rate on leads and 6X+ ROI even at lower volumes
• Proves the significant value of Primer's hyper-targeted audiences beyond just social advertising
• Underscores the importance of filtering audiences to quality accounts matching the ideal customer profile

Problem

We questioned whether Facebook's native targeting could truly deliver quality leads aligned with our ideal customer profile (ICP). Recent rollouts like "Advantage+" seemed to simply apply their typical broad-based interest and demographic filtering.

Our core problem was determining if such generalized targeting could drive tangible pipeline value, or if we'd see high volumes of leads fizzle out as unqualified accounts outside our ICP.

Additionally, we aimed to validate the belief that Primer's hyper-targeted audiences, restricted to attributes of our best-fit customers, would yield superior conversion rates and downstream revenue opportunities despite likely reaching fewer people.

This test zeroed in on resolving that tension between reach and relevance. We sought to optimize the ad budget for output quality measured in qualified accounts, not simply input vanity metrics around impressions and lead counts.

Hypothesis

Going into this test, we hypothesized that Facebook's native targeting would drive a high volume of low-quality leads not truly aligned with our ICP.

We predicted their generalized "Advantage Plus" audiences would cast too wide a net, attracting impressions from non-target companies and titles that lacked real interest or budget.

Conversely, we expected Primer's highly selective custom audience, filtered to attributes of our best-fit accounts, would deliver drastically better-qualified traffic. Despite generating fewer raw leads, we believed Primer's alignment would yield exponentially better conversion rates and downstream opportunities.

Essentially, we expected a quality-over-quantity outcome - with Facebook victorious on vanity metrics like reach and submissions but Primer crushing qualified output metrics around sales pipeline influence and revenue attribution. This experiment aimed to test these predictions.

Experiment

To test our hypothesis, we structured an A/B experiment across two ad sets with the only variable being the audience definition:

Ad Set A

Facebook's Advantage Plus Audience

Targeting included broad titles like "Marketing Manager"

$300 daily budget

Ad Set B

Custom Primer Audience

Filtered to ICP attributes like 50-1,000 employees

$100 daily budget

Keeping ad creative, copy, campaigns, and other variables identical enabled direct comparison of the impact of the targeting approach. We allocated a heavier budget to Facebook's broader reach, eager to stack its volumes against our anticipation of major efficiency gains from focused Primer audiences.

This clean A/B test isolated the audience variable, allowing us to finally resolve tensions around the value of reach versus hyper-targeted qualification when evaluating output.

Results

The final results provided clear answers around the efficacy of broader reach versus tighter, qualified targeting:

Ad Set A - Facebook

• Strong awareness reach at 23,000
• 18 total form submissions
• $85 cost per lead
• 0 leads converted to sales opportunities

Ad Set B - Primer

• Lower reach at 13,000
• Just 2 form submissions
• High $1,200 cost per lead
• 100% conversion rate to 2 new sales opportunities
• 6X+ overall return on ad spend

While Facebook's wide net drove predictably higher submissions at lower acquisition costs, shockingly their overly broad targeting failed completely at delivering qualified accounts in our ICP.

Comparatively, Primer's smaller yet hyper-targeted audience returned exponentially better conversion rates, ultimately driving 2 high-value opportunities that delivered a 6X positive ROI despite much steeper cost per lead.

Key Takeaways

The results of this test revealed important lessons about audience targeting:

Judge by qualified output - Vanity metrics around impressions, clicks, and lead volumes failed to predict downstream conversion rates and pipeline value. Judging success by revenue influence is critical.

Quality over quantity - Despite fewer raw submissions, Primer's qualified audience converted at exponentially higher rates to drive real ROI. Broad targeting risks budget waste.

Refine targeting iteratively - Balance target profile precision to maximize relevance while still reaching enough accounts to drive volume and scale.

Unified approach drives efficiency - Custom audiences like Primer's, when synced across channels, improve messaging resonance and campaign performance holistically based on a common qualified foundation.

In summary, this test debunked the myth that greater reach automatically delivers better marketing results. When resources focus budget exclusively on qualified subsets most statistically likely to drive revenue, less wasteful reach easily becomes more return.