Last Monday we announced a joint contest where one lucky ConversionXL winner would receive a year long subscription to GrowthHacker.tv worth $348.
To enter all you had to do was leave a comment letting us know your biggest takeaway from Peep’s interview with Bronson Taylor & tweet the link to this page.
This was an experiment to see if we could pull off a competition without a lot of effort. The results were… eye-opening
In case you missed the interview – you should watch it here:
There was a total of 58 tweets, 17 comments, and a total of 506 unique page views from the initial contest period of Monday – Friday.
Even though traffic to the contest wasn’t as high as I originally anticipated, there were quite a few things I learned from this contest.
But before we get into that…
Congratulations Matties of Winkelstraat.nl
Matties is the online marketing manager of winkelstraat.nl a Dutch high end fashion retailer.
One of my reasons for requiring a comment was to get a feel for:
- How many people were actually paying attention to the rules
- So I could pick a winner who would experience the most growth potential from the prize
Matties’ comment let me know he’d be a good pick:
My hope is that Matties can champion a data driven approach to conversion rate optimization within his company, and help them make strategic decisions based on real facts and not company politics & bureaucracy.
Matties, the first test I’d like you run on Winkelstraat is one that gets rid of the image slider on the homepage – I hear they kill conversions.
Alright, now that you know who the winner is, let’s talk about this contest.
Hypothesis & Expectations
When I heard we were doing a joint contest with GrowthHacker.tv, I could barely contain my excitement.
I really enjoy how Bronson & his team are stepping up the game on premium content. I truly believe a year of GrowthHacker.tv for the right person could radically transform their lives.
Consider that GrowthHacker.tv offers:
- In-Depth interviews with professionals from companies famous for their rapid growth
- Effective & easy to follow growth recipes
- A community of extremely talented growth professionals who regularly share their personal insights
Membership to GrowthHacker.tv normally costs $348/year, so I thought a free subscription would be very attractive for the majority of our readers.
Considering that over 10k people follow Peep on Twitter, over 3k people follow me, we have a 5 figure subscriber list, and knowing the performance of our most recent content, it seemed reasonable that we’d have the distribution to make this contest do fairly well, even on short notice and minimal effort.
I thought a free year to GrowthHacker TV would – realistically – achieve the following in the first 24 hours:
- Get 200 Tweets
- Reach 1,000 Visitors (combined traffic)
- Have between 50-100 comments
The screenshot on the right shows the share count as of Friday. As you can see, sometimes expectations don’t match up with reality.
First off, the contest did not hit it’s target of being shared 200 times – It was shared 29 times on Twitter in the first 24 hours.
Second, only 20 visitors came to this page from Twitter, which I anticipated would be the primary distribution channel.
Of those 20 visits, 6 came from me, 5 came from Nichole Elizabeth & 4 came from Peep.
That’s not that surprising, considering that my core following comes from a “content marketing” background, Nicole comes from the “Growth Hacking” community and Peep comes from the “conversion optimization” background.
It kind of caught me off guard to learn that 3/4 of the social visitors on this page came from the initial promotion links and not from people excited to take us up on the offer.
Looking at the bit.ly data, I get an even more revealing look at why the contest didn’t do what I had expected.
On Monday, there were only 6 clicks on the bit.ly links.
Now what’s interesting about these clicks is that Bitly links were only generated if you clicked the call to action in the copy of the body.
Share this post on Twitter (Click To Tweet – 1 tweet/day please)”
What most people may not have realized was that the Click To Tweet link generated very specific text for the promotion.
Because I put it in the copy under “Rules” section, I expected more people would click that to tweet about the contest.
In reality, only 2 out of the 29 people who participated that day used the link from the body copy.
— Anuj Adhiya (@AnujAdhiya) February 10, 2014
— Adam Tomat (@adamtomat) February 10, 2014
Even more interesting was that even though bit.ly reports 6 clicks, Google Analytics only registered 1 click generated from the contest link.
I can only speculate why there’s such a discrepancy in the data, but the lowest “time on site” metric from Google Analytics is 00:00:08. My guess is Google didn’t register bit.ly clicks that lasted fewer than 8 seconds.
Clarification: It’s important to realize that all links that go through Twitter are wrapped in a t.co link.
If you’re trying to figure out who sends the most Twitter traffic your way, the best thing I know how to do is find the referring t.co url in Google Analytics:
Then use Twitter search to find everyone sharing your url, hover over who shared your links, and check the bottom left hand corner of your browser to see identify the t.co link in Google Analytics.
So What Went Wrong? – A Few Hypotheses.
A contest like this hinges on your ability to tap into your core following’s friend base.
In an ideal world, this principle is pretty straight-forward. For every person who sees the offer, they refer two more friends who refer two more friends and so on.
In the real world however its a hair more complicated. If you want a concrete metric for how frequently a contest like this is shared, what you’re looking for is a viral coefficient. For real growth, it should be around 1.10 and really what most people hope for is a coefficient of between 1.2 -1.5
If you’re wondering how to calculate the viral coefficient, this thread on Quora provides excellent insight.
Pay close attention to the third paragraph, the one that starts with “The acceptance rate is #invitee signups / #invites” because this is where many contests (including ours) stumble before they even get off the start.
Without knowing first how many people you’re reaching, there’s no way to accurately measure & test effectiveness.
Hypothesis 1 – No Clear Distribution Plan
As I stated earlier, I thought that given our combined followers on Twitter, simply putting out a handful of Tweets promoting the contest would have been enough to get it kickstarted.
That’s a very naive assumption because its impossible to calculate exactly how many Twitter followers will see the message at any given time. Its even more impossible is to determine who will see it, and whether or not it will be something they really want.
Like with email marketing, I should have created segments. In this case, I would have segmented my Twitter followers using Peer Index to find the people my followers find the most influential.
With this, I could create an influencer outreach plan so other “big names” could reinforce that we were doing a high-value giveaway.
Peer Index actually created an “influencer” segment for Nokia’s 1.1M followers back in December, and discovered over 1,700 influencers with the potential reach of over 50m others.
Taking it a step further, I’d use Followerwonk’s analyze tool to know what percentage of each influencers social media following is the most active and at what time.
Even though this doesn’t give exact numbers like email, I can still calculate roughly how many followers are online when my message goes out. (E.g. I know 6.7% of my 3,216 followers or about 215 total, are on Twitter between 11 & 12 est.)
Not only does this graph this help me create a schedule for influencer’s tweets, but it gives me a more concrete idea of the sample sizes I’m working with at any given time.
Using that information, I’d ask two or more influencers in the same field with similar sized, overlapping, follower bases to Tweet out one of two messages (like in an A/B test) to determine message drives the most action. I’d then keep track of that in a spreadsheet to measure performance.
Again, this is not exact, but it is a far more controlled method of outreach and message testing through social media than posting multiple times a day and hoping something catches on.
Hypothesis 2 – We Weren’t Reaching The Right People
After the contest announcement went live on the blog, I went to the first place I could think of that an offer like this would be appealing – Growthhackers.com
Our content usually does pretty well there, and we’re friendly with many of the founding members, so it seemed like a natural fit. I emailed Morgan Brown, one of the team members, to see if it was ok that I publish something promotional. He said he’d publish it for me, I said thanks, then waited for the responses.
Looking at the comments, you’ll see that 3 out of 5 people discussing the article are already GrowthHacker.TV subscribers.
Digging a little deeper with Followerwonk’s compare tool (retroactively) I find that nearly 11% of GrowthHacker.tv’s Twitter following is made up of GrowthHacker.com’s Twitter following.
Even though neither of these metrics are 100% reflective of actual usage of both services, there is evidence emerging that a good percentage of the GrowthHacker.com community are already GrowthHacker.tv members.
If I do an advanced Google search to see how many pages have /member/ in the Growthhacker.com url, I get over 5,500 results, which at least gives me some indication of how many registered accounts are on Growthhacker.com.
Looking at the GrowthHacker.tv community, I see there are over 1,100 participants.
If you’re doing the math, that means Growthhacker.tv has roughly 19.34% of the total users of GrowthHackers.com.
On their own, those numbers are pretty meaningless.
But when put in the context of knowing there’s an overlap for both Twitter accounts & that a majority of the comments on the post on GrowthHackers.com said something to the effect of “I’m a member” that the data starts painting a picture.
Maybe people didn’t sign up because they
A.) Already had a membership or
B.) Already knew they didn’t want one
Were this a serious contest, I would have given myself more lead time for research and community outreach, learned this earlier & picked communities that would be more likely to be blown away by with GrowthHacker.tv has to offer.
Hypothesis 3 – The Original Headline Was Too Confusing
When I originally published the contest announcement article, I tried formatting the headline in a way that would be appealing to our regular readers as well as people visiting the blog page from another article.
Unfortunately, this made the article’s value proposition confusing and may have lead to hesitation.
I’m guessing this because looking at the data that shows referrals from /blog/ I see a slight uptick in traffic on the same day I changed the headline to remove “Peep Talks CRO Basics” from the headline.
Granted, it’s not a large change & it would be a huge mistake to consider this statically significant in a true A/B testing environment. Nevertheless, I know the original headline wasn’t helping anything.
Hypothesis 4 – You Didn’t Want It
Not counting the pricing experiments article and landing page optimization articles because those pages were experiencing abnormal viral growth spurts, and excluding the “/” and “/blog/” pages, because more traffic flows through those pages naturally, you see the contest post doesn’t fare too well against the other content.
The email engagement article that was published 3 days later received more than double the traffic & 4 of the other articles that are ahead of the contest announcement rank well and drive pretty reliable traffic.
As much as I hate to admit it, the data shows a very real possibility that you just didn’t want – or don’t have time for – a subscription to GrowthHacker.tv and that the offer just wasn’t a fit for you.
This might be further confirmed by the average time on site being just over 8 minutes, even though there’s an hour long video embedded on the page.
Hypothesis 5 – You Didn’t Want Competition
Alternatively, if you did want it, it’s not hard to do the math. There would only be one winner and sharing the contest would also decrease your odds of winning.
I wonder, had we put more prizes on the table, would more people have entered?
After all, there’s nothing fun about entering a contest you think you have no chance of winning.
Conclusion – For A Successful Contest, You Have To Put The Time In
Looking back, it’s easy to see all the different reasons why a promotional contest will fail when it’s based on assumptions rather than data.
It’s not enough to just have an awesome prize, you need to have data like:
- Who influences your market
- Customer surveys
- The places you’re having conversations online
- The time they spend with similar products
Beyond that, it’s important to have enough lead time to properly collect that data & find ways to use what you learn to build hype before the actual contest.
By running this experiment, I also see just how important it would have been for us to just ask more questions ahead of time. Is this something you want? Is this something you already have? Do you have time for this?
It seems so simple, but it’s so easy to forget when you’re enchanted by the possibilities.
So now I’m curious & I’d like to ask you two questions:
- If you saw the contest why didn’t you enter?
- Have you done contests like this before and seen similar results?
Looking forward to your feedback.