The True Cost of Free Analytics: Investigating & Exposing the Hidden Biases in GA4

At this point the old GA has stopped tracking and it’s fair to say there are very few people enjoying the new analytics program they didn’t ask for. No data continuity, 90% of the reports are gone and the interface is horrible. But this really isn’t the problem.

Until 2022 and 2023, the past decade in the world of web analytics had been pretty uneventful. Then we got the iOS 14 update which killed 3rd party cookies, followed by GA4 which…. killed a lot of people’s sanity. The landscape has fundamentally changed, as have the rules, but 99.9% of people don’t realize what’s going on. 

In this article I’m going to expose the truth behind GA4 and what looks like a complete abuse of monopoly power with one purpose: to over-attribute to Google’s Ad Network.

Apple & Intelligent Tracking Prevention (ITP)

Taking a step back briefly, the Apple iOS 14 update in 2022 wrecked Facebook/Meta; a rapid crash in share price followed as people realized the impact - Facebook said the move would take $10bn off their 2022 revenues. Under the guise of customer privacy protection, Apple took a chunk out of competitors and effectively killed the 3rd Party Tracking Facebook relied heavily on, and that Google benefitted from.

The move would stop app developers from giving out apps for free and relying on selling data to make their money, and create more paid apps that Apple coule take a large cut from.

This has reduced tracking accuracy and means an estimated 20% of tracking data is already lost to Ad-Blockers & Intelligent Tracking Prevention (used in Safari). This can be mitigated, but is always going to be a growing challenge for Google and Meta.

Under The Hood Of GA4: An Outrageously Biased Attribution Approach

In my opinion, the Attribution in GA4 is extremely biased. Google has a monopoly on Analytics and appears to be leveraging that position to sell more ads. It is a clever combination of features and settings that when combined, seems to leave attribution of other channels and competing ad platforms significantly lower than Google Ads. If you want a fair picture of what’s working and what’s not, I don’t think GA can be relied on any longer.

The Lookback Window is Designed to Penalize Non-Google Ads Traffic!

“Acquisition conversion events” have a lookback maximum of just 30 days. So that means ‘first user source’ can only be attributed back 30 days…. Why would this be? It is an important bit of information and very simple; the first time someone visits, you see where they came from, and store that forever against that user ID. The reason for this becomes clear when looking at the second setting.

“All other conversion events” have a maximum of 90 days. The 90-day window uses Google Signals which allows Google to see users' engagements with Google Display Network (Display ads, YouTube video ads etc).

So 30 days for all sources (eg. Meta, Organic) and 90 days for interations, however brief or unimportant, on the Google Network. Seems fair.

And now the other attribution models are gone anyway. Deemed unimportant:

Data Driven Attribution (DDA) - A Black Box

DDA is the default setting for conversion and revenue attribution, and it’s damn sneaky. It used to be for GA360 users only - a product costing upwards of 150k a year. It’s now free to everyone using GA4…. I wonder what Google is getting out of this.

There’s always a catch when something is free; you’re paying in a different way. DDA is a black box which takes a set of inputs and Google assigns value across touchpoints using algorithms. You have no way to see how these algorithms are really assigning value. Well, digging deeper, there are plenty of clues.....

The New Data Retention Limitations Make Attribution Even More Skewed

The data retention period is just 2 months by default, with 14 month maximum (and 50 in paid). There is an argument that this is needed for compliance with GDPR, but Google doesn't even store data in the EU so is already non-compliant. Why default to 2 months? This just helps compound the effects of the 30 day and 90 day lookback windows. 

Due to the more transactional searches and brand searches happening at the tail end of a conversion journey, Paid Search is more likely to feature in the end stages. This means it is more likely to get ‘last click’ attribution and also benefit from anything reducing the time window on all attribution.

“Modeled Data” Puts This Misattribution into Overdrive

We all know we are now missing more and more data due to cookies and consent. A recent survey put the number of people always accepting cookies at about 30%. This will of course change depending on if it’s a returning visitor, or customer, but this still leaves a potentially massive hole in data. GA4 takes advantage of this to further misattribute to Google properties. 

Modeled data uses information collected in the 'cookieless consent mode' for users who have chosen not to give consent for tracking and Google Signals data for users logged in to Google.

You might think it would simply be an extrapolation of what we know to fill what we don’t - and so affect all traffic sources equally when it comes to attribution. However Google Signals data is taken from people who are logged in to Google or YouTube. So, Google is extrapolating from a pool of users who are more likely to have data attribution to a google property. Google Signals is required to use 'modeled data'.

Impartial attribution modeling is vitally important but in my opinion GA4 is completely skewed to Google - DM me if you think I have something wrong here. If you only ever use Google Ads, and are never going to use another channel, and don’t care about proper attribution to your email or Organic Search / Social either, then it’s ok. However, we doubt that most businesses fit this.

How Much is Free Analytics Costing You?

This is a very individual question. If your data was off by 30%, by 50%..... if the insights you were getting were leading you in the opposite direction to reality, how much budget could that waste? Or how much growth would be sacrificed? It’s important to understand that GA4 is not just a disappointment in terms of UX, visuals and report variety, it is actually very questionable as a source of decision making data full stop.

It’s clear you can’t trust Google to mark their own homework but by all means, keep GA4. It’s free and there’s nothing wrong with having multiple sources of data, per se. Just be aware of the limitations and faults in it. If you are spending significant amounts on marketing and driving people online, you should be looking for premium solutions for your data and web analytics anyway. 

If you rely on accurate, impartial data-driven decision making then even if the data in GA4 was fully reliable it is still missing an awful lot. For example, if you want to measure ROI accurately, is it set up to take into account SKU level profit margin data? Is it taking into account the full marketing costs that eventually drove the sale? (not with a 30 day attribution cut off period!)

GA4 is fine to quickly check some top level traffic numbers, or for a level of attribution for Google Network activities. But to really understand performance, you need more complex solutions. It is time to invest in other tools and diversify your analytics toolkit.