March 28, 2024
Media Buying
Advertising

How we test 3k+ videos weekly without blowing up our UA budget

6
minutes read

Context: We own 22 apps (subscription based + ad revenue) on the App Store and spend aggressively on TikTok, Applovin & more.

We produced more than 100k videos in the last 12 months and here’s what we’ve learned:

Post Header Image

Table of Contents

About the author

More than a decade in the mobile app & ad-tech industry, Alexei Chemenda is the CEO of Poolday.ai and founder of Skaalx (mobile app studio, 20+ apps). His team created and scaled 20 apps, 100k+ video ads for TikTok UA for apps in 2023 and is obsessed with creative iterations. Alexei has overseen more than $100 million in performance marketing spend with most gaming leaders in the mobile app space.

Read more

Intro

Testing UA videos implies spending money. Not all spending is equal and more importantly, not all platforms are built similarly when it comes to creative testing. At Poolday, we use a channel waterfall approach to test creatives, meaning we initiate testing on 1 channel - TikTok, and then the best creatives surfaced by TikTok would flow into Meta, followed by Applovin, Unity, and subsequently, additional channels.

It's a common misconception that successful user acquisition demands extensive, multi-platform testing, months of commitment, and substantial financial resources. However, our journey through producing and testing over 100,000 videos has led us to a streamlined strategy. Here’s how you can pinpoint high potential ads with just a couple thousand dollars and an adaptive testing philosophy: 

Why you should use Tiktok for testing?

Ever upload multiple creatives to TikTok and notice only ONE grabs most of the spend? This is exactly why we love TikTok: winner takes all. 

With over 100,000 videos produced in the past year, we required a platform capable of quickly determining the success of a creative. When we upload 16 creatives to an ad group on TikTok, the platform almost instantly zeroes in on high potential hits, with a single creative eating 90%+ of the spend in a “winner takes all” showdown. There are often false positives and false negatives, but the insights are golden: 

  1. The video can spend on TikTok (which in itself is a win)
  2. The video got something right
  3. We can try to deploy this video in the main campaigns (BAU campaigns), and either kill it or scale it. Whatever happens, it’s a learning opportunity for the next batch of videos

What budget to test with?

For an ad group with 16 videos, we typically spend $100-300/day. If you do the math on a per creative basis, it’ll be a tiny number. However since TikTok has a winner takes all approach, we end up spending 90% of that budget on 1 video which gives us initial statistical significance in the experiment. What’s ironic is that we need to produce a lot of videos for TikTok to find a winner, but then all of the other videos are thrown away after the experiment and we focus on the top performer to generate iterations for the next batch.

Learn More: Overcoming Tiktok Creative Fatigue: Strategies for Maintaining Spend and Performance High

Here is a sample of an ad group containing four creatives, to which a daily budget of $100 was assigned. It's evident that one ad monopolized the entire budget, utilizing nearly all of it at 98$. Despite each creative in the ad group having a comparable click-through rate (CTR), only one ad accounted for the total budget consumption. (To maintain the confidentiality of the app involved, all images and IDs have been obscured.)

TikTok Adgroup Budgets

How long to test for?

It depends on the goal of the campaign. Ideally, we aim to review at least 4-7 days of data, or even up to 10 days, to ensure we’re making informed decisions. If the goal is a down-funnel event like a purchase action, we target achieving 100-500 instances of the target event before calling the campaign optimized on TikTok. More on the number of required events from Facebook here.

Learn More: 9x ROAS using hook swapping: Insights from 100k+ videos tested using AI actors

What about testing other channels?

We deploy to the other channels (Meta, Applovin etc) after testing on Tik Tok. We find that in 60% of the time, top performers from TikTok perform well on Meta & Applovin. We then focus additional UA efforts on Meta to drive post-install optimization tests (to understand if our creatives back out from an ROI perspective on that channel). This is when we consider the creative a unicorn ad: it scaled on TikTok, then it proved successful on Meta using post-install KPI as a success metric.

About us:

Poolday.ai is a self-serve platform enabling performance marketers to generate videos in any language and style instantly using our AI actors. Rapidly A/B test anything alongside existing gameplay recordings to produce a wide range of video styles — from green screen to floating heads, to organic looks—comparable to what real content creators make.

Increase ROI on user acquisition and boost operational efficiency by cutting out the need to coordinate with content creators, giving marketers complete control over the script and style of the video.

About the author

More than a decade in the mobile app & ad-tech industry, Alexei Chemenda is the CEO of Poolday.ai and founder of Skaalx (mobile app studio, 20+ apps). His team created and scaled 20 apps, 100k+ video ads for TikTok UA for apps in 2023 and is obsessed with creative iterations. Alexei has overseen more than $100 million in performance marketing spend with most gaming leaders in the mobile app space.

Featured Posts

Featured Post Thumbnail Image
August 28, 2024

AI Actors - Part 3 | Critical Questions and Platform Guide for Marketing Artists

Key questions to guide your AI platform choice, ensuring your brand stays unique and impactful.
Featured Post Thumbnail Image
August 28, 2024

AI Actors- Part 2 | Overcoming Challenges, Addressing Limitations, and Embracing the Hybrid Future

Success in performance marketing hinges on balancing innovation with authenticity and creativity wit

Join Us and Let’s Explore Together

Subscribe now for exclusive content and expert insights. Be the first to stay informed!

SUBSCRIBE

Subscribe to our newsletter

Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.
Close Subscription Pop Up