How to Measure Hard to Measure Marketing Channels Spark Toro

How to Measure “Hard-to-Measure” Marketing Channels - SparkToro #

Excerpt #

I’ve ranted endlessly about how marketing leaders refuse to invest in hard-to-measure marketing channels like PR, media, native social, events, many types

I’ve ranted endlessly about how marketing leaders refuse to invest in hard-to-measure marketing channels like PR, media, native social, events, many types of content, and word-of-mouth. But, never stepped up and made clear how to use metrics, imperfect though they may be, to turn impossible-to-measure into difficult, but possible. Sadly, I’ve never seen a high-quality offering to reliably point-to, either.

So, let’s talk about measuring the hard-to-measure.

From ~1996-2016, the web had twenty years of highly-trackable digital efforts. Almost every link a marketer placed on the web, every ad they bought, every search ranking they achieved, and every social network post contained easily collectable data showing where their efforts were working. Marketers (and even more so, company leadership) got addicted to this trackability. Instead of the prior century’s brand-lift style guesstimates of whether a particular TV ad, radio spot, or in-store promotion boosted sales, the Internet built a culture of “if you can’t prove a source sent converting traffic, it’s not getting investment.”

Three trends are now combining to spell doom for the world of tracking we used to know.

1. The rise of zero-click content on search, social, and content networks
2. The organic & ad tracking changes from Google, Apple, Facebook, TikTok, and others
3. Intentional obfuscation efforts by the web’s big players to turn organic traffic (i.e. marketing that doesn’t give the tech giants any ad revenue) into dark traffic, devoid of referral data, impossible to attribute in an analytics tool

Some savvy analysts compensate by building ever more complex attribution and traffic modeling, attempting to hold onto the golden age of digital measurement. Numerous platforms claim that a combination of machine-learning and user-selected attribution models can still do the job. These tool providers can assign equal weight to every interaction, weight the first and last higher than the middle ones, weight based on cost or volume, or use the provider’s patented machine learning. Some of them are pretty good… we think? We can’t know for sure.

All we are certain of is that what’s been lost cannot be restored. We’ve returned to the 20th-century’s marketing measurement systems: a model that demands less certainty, more creativity, and intuition > blind trust in analytics suites. Most businesses just don’t know it yet.

Which Marketing Channels & Tactics are Easy vs. Hard to Measure? #

This one’s simple: do you pay big tech for the data? If so, it’s easy to measure. If not, it’s almost always difficult. For example:

• Google’s paid search ads? Easy peasy.
• Facebook & Instagram ads? Lemon squeezy.
• TikTok, Reddit, Twitter, and LinkedIn ads? Pretty easy.
• Apple’s App Store, Google Play Store, Amazon e-commerce ads? Still quite easy.

Here’s the rub: “Easy to measure,” does not mean, “easy to prove Return on Ad Spend (ROAS).” In fact, I’m fairly certain that many of the big tech advertising providers prefer trackability over true performance. Big tech knows a lot about user behavior: where people go on the web before they buy, what they search for, what they browse, visit, follow, etc.

Using that data, ad platforms can predict which of their pages people who are likely to buy a Nintendo Switch or a Buffer subscription will interact with… so they show them Nintendo’s/Buffer’s ads. When the conversion happens, the grateful marketer is happy to pay exorbitant ad prices, even though that conversion might very well have occurred regardless of whether the customer saw that ad.

True measurement means measuring incrementality, a concept about which analytics advocate Avinash Kaushik has written eloquently: Attribution is Not Incrementality.

FAANG’s Advertising Slogan
(via the parable of the pizzeria)

The simplest way to think of it is this: when AirBnB, Chase, P&G, Uber, eBay, and dozens of other big advertisers cut their ad spend and/or advertising reach dramatically, their results improved! How?

Because most of those people seeing or clicking their ads were going to purchase anyway.

To measure incrementality, you have to shut off advertising. Channel by channel. Completely. For a good, long while (usually months). Only then can you truly know how many incremental sales were driven through ads. And guess what? Very few executives are willing to trade potentially lower cost of acquisition or improved measurability for a unknowable-amount of lost sales. Not when next quarter’s bonus depends on hitting the numbers!

Now let’s look at hard-to-measure tactics:

• Organic SEO? Difficult, but possible (though lacking the granularity of the pre-2013, keyword-not-provided era)
• Content marketing? Very difficult since most content doesn’t drive conversions during the same visit, and cookie-tracking past 60-90 days is no longer reliable.
• Organic social media? Immensely difficult, doubly-so if you’re doing it via native social posts (which social algorithms reward far more than traffic-driving links)
• Earning press & engaging in PR? Dastardly amount of challenge (the branding happens on someone else’s website, with virtually no metrics visibility)
• Co-Marketing with relevant partners? Hard, but made easier if the partner provides performance data. Just don’t rely on coupon codes or UTM links; those were never good measures of true impact.
• Guest contributions to publications that reach your target audience? Incredibly hard. Even if the publications give you audience numbers, there’s no way to track which ones eventually came to your website and converted (or worse, went to a store and bought your product)

Ouch. Every one of those high-potential-ROI, low cost-of-acquisition techniques is brutal to measure or report-on. No wonder 90% of marketing is just “throw more money at big tech ads!”

Hard to Measure ≠ Impossible #

Despite this infuriating labyrinth of incomplete metrics, it is still possible to measure these channels and tactics. But, it requires a return to the 20th-Century mentality of marketing measurement: brand interest and sales lift, tracked over time, with experiments in segment-able markets.

Think back to 1981. You’re running marketing for Coca-Cola, and want to figure out if a Halloween-themed marketing push for Coke (new can designs, a special kiosk in grocery stores, a mid-size run of local TV and radio ads) will lift sales and by how much. Your demographic and sales data tells you that Akron, Cleveland, and Cincinnati all have similar attributes, so you run the campaign in Cleveland, and leave Akron and Cincinnati as controls. A month after Halloween, your data comes back with a 16% lift in same-store sales in Cleveland compared to Cincinnati and Akron. Next year, you run the same promotion across every major metro in Ohio, Pennsylvania, and Illinois, leaving similar states as controls. Again, ~16% lift for the Halloween campaign in metros where you ran the promotion vs. didn’t.

Technically, you’ve proven the campaign works. But, frustratingly, you don’t know if the Halloween-Coke-can design, the in-store promotions, the radio ads, or the TV spots were responsible (and how much).

This throwback measurement process is very similar to what’s possible in digital marketing channels today. You can invest in social media marketing, PR, brand campaigns, podcast sponsorships, influencers, etc. and see potential brand and sales lift, but probably can never know if the weekly Tweetstorm was more effective than the Instagram Reels video series or the podcast ad buys.

This is a rough model of what you can build:

We know how “untrackable” these channels are, but we also know that real behaviors can be observed, even if they can’t be attributed. It’s not impossible to measure lift. It’s impossible to measure each individual visitor (which is exactly what big tech wants, because then they can sell us that tracking through a feature that does have it: ads).

Imagine it’s Q3 2023 and you decide to take the worst performing $10,000 in your performance advertising budget and put it toward hard-to-measure channels. You sponsor a few events. You double a charismatic employee’s speaking and travel budget. You ramp up zero-click content creation on LinkedIn or Instagram. That $10K goes pretty fast.

In Q1 2024, the leadership team comes to you and says, “How’d that re-investment in uh… whatever you’re calling it… go?”

You know what probably happened:

• More of your target audience heard about you from sources (events, social, niche media) they trust
• Some of those people searched Google, YouTube, and LinkedIn for you company or product names
• Some of them clicked through to your website
• Some of them followed you on a social or content network
• Some of them made purchases or signed-up for your email list
• A higher percent of visitors overall clicked-on, engaged-with, or purchased your stuff

You just can’t prove it.

Google is never gonna say “hey, this IP address that just spent $5,000 on your website―we actually saw that same IP address visit your charismatic speaker’s content on LinkedIn and download their slides from the event page.”

Not. Gonna. Happen… Sorry.

The best you can do is adopt a framework that approximates overall lift in your marketing funnel. Like this:

Top of Funnel – here, you’re measuring people that haven’t yet reached your website. They might have heard about you on an impossible-to-track channel like a podcast or YouTube channel. Maybe they saw your brand recommended in a Reddit comment or on Twitter. Maybe it was a conference or old-school word of mouth. Whatever the case, a search for your brand on Google, YouTube, or one of the social networks is very often the next step.

You want to measure this volume (imperfect though it surely is) over time to track seasonality, account for any big, obvious spikes (a big, national media mention, viral social discussion, or even the release of a popular podcast/YouTube recording that features your brand can often be spotted in this data).

New followers on social, and impressions on your social profiles (which some platforms, like Twitter and LinkedIn, provide in their analytics) are included here, because they map to similar, pre-website-visit customer journeys.

Middle of Funnel – a lot of traffic comes to your website from two sources that are almost certainly these “hard-to-measure” activities: #1 – what Google Analytics calls “Direct” and #2 – what search marketers call “Branded Keywords.”

In our company’s case (Google Analytics screenshot above) more than 60% of users and sessions come direct. And, of that 22% marked “Organic Search,” 90%+ of those searchers used branded keywords like “SparkToro,” and “Spark Toro.”

Both of those can be tracked via a combination of your analytics software and Google Search Console. You’ll have to manually mark and calculate branded keywords in the latter, but it’s relatively solvable, even on large sites, through Excel/Google Sheets.

The other mid-funnel activity that needs measuring from these activities is your audience of subscribers. I.E. the folks who you can now reach through email, text messages, or content (and ads) on your social channels. Plenty of hard-to-measure marketing is done just to boost these numbers, rather than to directly impact conversions. Social networks (and social users) tend to penalize salesy, conversion-focused content. But they reward attention-earning, entertaining content, the result of which isn’t usually a sale, it’s a follow or subscribe.

Bottom of Funnel – conversions. Email signups. Free trials. Logins. Purchases. Any event tied to a page visit or a process completion on your website should go here. You’re already tracking these, the inclusion in this process is simply to measure the correlation between off-site and hard-to-measure activities with bottom-line, eminently-measurable ones.

The Detailed Plan #

Here’s how you fit the specifics into your organization:

• Step One: Recognize that 95% of hard-to-measure marketing comes through just three visit paths in your analytics:
  • A) Direct visits (also called “type-in”)
  • B) Branded search traffic (i.e. search engine visits that contain your branded terms, the names of your product(s), people, articles, etc.)
  • C) Increased click-through and conversion rates (e.g. if your brand gets a big press mention, it will usually raise the rates of email subscriptions, sales, CTR in Google, Facebook, Twitter, LinkedIn, etc.)
• Step Two: Set up benchmark measurement of these paths, e.g.
  • Create a chart with direct, type-in, social referral, and branded search volume + traffic (sometimes these latter two can be quite different if Google is “answering” the query of your brand before someone reaches your site)
  • Set reporting on avg. click-through and conversion rates in the key parts
  • Add monthly reporting on so-called “vanity metrics” across social media and content networks (things like # of YouTube subscribers, # of Twitter followers, # of Facebook engagements, etc)
• Step Three: Make estimates of expected month-over-month benchmarks if your marketing mix and investments remained unchanged (e.g. conversion rate from free-to-paid accounts averaged 2.2% the last 6 months with <0.3% fluctuation).
• Step Four: Choose the 2-3 most relevant, straightforward, high-potential techniques in the “hard to measure” bucket, and put some serious investment into them for the next few months.
• Step Five: See what changes, how much, and when.
  • Did the conversion rate shift by more than 0.3% after you invested in a big press & PR campaign?
  • Did direct and branded search traffic rise considerably more than expected after your podcast sponsorships?
  • Did rates of email signups from the homepage increase beyond what your model predicted?

Below is a very rough example for our website ( SparkToro.com) and brand (SparkToro):

Click for full-size version
(side note: not all of our metrics are perfectly represented or perfectly accurate)

In this dashboard, I’ve put real numbers from Google Analytics, Google Search Console, Twitter Analytics, LinkedIn Analytics, our Mailchimp email list, Crowdcast webinar attendee analytics, and others to build a reporting system for how the top, middle, and bottom of our funnel relate.

What this doesn’t help me do is track any specific activity. I don’t know if Amanda’s presentation on personas at AdWorld positively impacted our numbers. I know *something* positively impacted them, and I know we’re not buying ads… But correlation does not imply causation (though it sure can be a hint).

When we try new forms of marketing, or have months of lighter or heavier activity in particular channels, this metrics dashboard shows me what happens. It sure seems like social media impressions and follows track to greater brand interest in the months that follow, and with this dashboard over 2-4 years, I could probably prove that hypothesis and intuit a rough estimate of how long it takes for an increase in Twitter or LinkedIn engagement to turn into branded search growth or free account conversion rates.

Is this a replacement for attribution tracking? Heck, no. It won’t help us attribute a single visit.

Is it a reasonable way to show how un-attributable visitors tie to hard-to-measure marketing activities? Yes. A dashboard like this, kept up-to-date, and annotated with events (and probably sales, churn, and revenue figures, too) can illuminate how well channels and tactics are working, and which ones provide observable lift.

Convincing Attribution-Metrics-Addicted Leaders #

How do you convince your organization’s executives, or (even more challenging) your client’s to adopt serendipitous marketing investments and then measure them with what have often been written-off as “vanity metrics?”

There’s three ways I’ve seen work:

1. Ask for an experimental period – three months at a minimum, hopefully six or twelve, because these efforts take time.
2. Show them the competition is doing it – a surprising number of executives are moved not by well-founded arguments and exceptional metrics but by fear of losing out to someone else. Almost every business has a competitor (or a handful) that are engaging in hard-to-measure tactics; use those examples to suggest that your brand do likewise before it’s too late.
3. Use excess bandwidth, then show the results – I hate recommending free work, but sometimes, it’s what’s required to get formal approval. In your “spare time” (I know it’s a stretch), you can make small, serendipitous investments in these channels, then show the results lift via the methods above, and perhaps get real budget and time to pursue more seriously.

None of these are ideal, but a combination of them might sway even recalcitrant purse-holders to let go of their obsession with full-funnel, provable tracking.

What’s the Right Solution for Your Business? #

I cannot tell you whether your leadership team will buy into un-attributable, impression and “vanity”-metric based lift tracking. But, I do know that data about visitors is going to get harder and harder. In the decade ahead, it wouldn’t surprise me if metrics like this are the only thing organic marketers get to keep.

One obvious thing every team should invest in is first-party data.

Slide via Dharmesh Shah, HubSpot cofounder and creator of the Wordle Unlimited Practice Game: WordPlay.com

Getting a high percent of visitors to voluntarily log-in and provide you with useful, actionable information through expressed or implied preferences and behaviors is the ideal. But even with a plethora of first-party data, you’ll never be able to track how someone actually learned about your brand, when they first interacted with you on social, what activity nudged them to search, click, visit, try, and buy.

The models I’m sharing here are imprecise. They don’t use fancy technology to predict or correlate or show interesting connections. But they really might be the only way to tell if a hard-to-measure investment pays off over time.

The Tough to Swallow Pill #

One final thing I’ll say that’s often infuriating to marketing practitioners: the absolute best way to do much of this is to give up on tracking, trust your instincts, use directional lift over time, and redirect all the hours of work the above measurement practices require into actual marketing activities.

“But how will I prove my work has value?!”

You won’t. Either core numbers (sales, MRR, signups, etc.) will rise and you’ll keep doing those things, or they won’t rise enough and you’ll have to modify your channels and tactics. Either way, you won’t waste time fetching metrics and calculating segmental attribution. That gives you more time (and more room for failure/experimentation) to invest in the marketing itself.

Is this don’t-measure approach for everyone? No. But, it really can work. SparkToro itself, and our VC-growth-rates-with-a-tiny-Chill-Work-team is proof. And, if you can’t embrace that philosophy, the rest of this post can help you build a metrics system for even the hardest-to-measure investments.

p.s. The fine folks at BrandOps are building software to measure many of the things described above, and could be worth a look for those attempting to do this in an automated way.