How Ads.txt is Being Exploited by "Baddies" for Fun and Profit

Ads.txt is a great idea and a much needed standard. The exploits below are observed in the wild currently; bad guys are further scaling their ad fraud and better covering their tracks. What follows is a description of how abuses of ads.txt are carried out. The intent is to enable others to look for tell-tale signs like abnormal numbers of sites using identical ads.txt files, and decide if they still want to buy from the suppliers/sellers who are doing shady things.

"Note that this is NOT the fault of the ads.txt standard itself -- it's humans behaving badly and committing fraud deliberately."

The Original Good Idea of Ads.txt

Ads.txt is a simple text file that publishers add to their root domains -- for example, espn.com adds the file as espn.com/ads.txt. The file contains a simple list of exchanges and the sellerIDs that correspond to each exchange. The idea is for buyers to have a way to verify which exchanges are allowed to sell a particular publishers' ad inventory and what the sellerIDs are. But the vast majority of the placement reports that buyers get have no sellerIDs included, just the domain and the quantity of impressions served on that domain. This means the buyer cannot do the verification to find sellerIDs that do not match the domain, and thus find domain spoofing fraud.

Recommendation: Buy from sites that have ads.txt (versus not); but don't just assume that there is not fraud or fake traffic inflating impression quantities.

Domain Spoofing Fraud Continues, If You're Not Looking

Domain spoofing occurs when a fake site pretends to be a legitimate site by specifying the domain of a legitimate publisher in the bid request. But the fraudster puts their OWN sellerID in the bid request, not the sellerID of the domain they were spoofing, because the bad guy wants to get paid. So if a placement report had both sellerID and domain and quantity, it would be very easy to spot the mismatch - the bad guy's sellerID is not one that should go with the legit domain. But most placement reports don't have sellerIDs included and most buyers don't bother to do this detailed reconciliation, since no one asked them to. Alan Reed of adstxt.com points out that "ad networks can do the verification in real-time, but many don't."

Buyers assume that they do and that is how fraud continues to get away with it. Even the most elementary domain spoofing continues unabated while everyone thinks ads.txt has solved ad fraud, including the Association of National Advertisers -- "War On Ad Fraud Is Succeeding" and "Ads.txt ... has worked to reduce desktop spoofing. The rates for desktop ad fraud were the lowest in the history of the report."

Recommendation: Always insist on placement reports that have three things -- domain, sellerID, and quantity -- not just domain and quantity; always reconcile sellerIDs seen in your placement reports with sellerIDs listed in a domain's ads.txt file. Don't assume the exchange, SSP, or someone else did that for you. They didn't.

Starting with Fake or Blank Ads.txt Files

In the early days of rolling out ads.txt, several exchanges got PR for themselves by saying they would not allow anyone that doesn't have an ads.txt file to continue to sell through the exchange. What do you think the bad guys did? They all went out and added fake or blank ads.txt files to their domains so their moneymaking wouldn't even skip a beat. This was possible because those exchanges only checked for the EXISTENCE of an ads.txt file on the domains, not the CONTENTS of the ads.txt file.

This misinformation put everyone at ease for another few months, happily thinking the exchanges were doing their part to help "stamp out fraud" while the fraud was happily continuing in broad daylight.

Then people started to realize it may not actually be stamping out fraud. Turns out, they were right ... ads.txt was not reducing fraud at all. ‘It is a sticking plaster’: Ads.txt is not stamping out fraud - Digiday, Feb 2019. And it may have been used to better hide fraud -- fake sites selling ad inventory by hiding behind authorized resellers. Scammers Target Ad Industry’s Initiative to Thwart Fraud - WSJ, Feb 2019.

Note that reseller companies are not necessarily fraudulent companies. They are simply arbitraging an opportunity. Due to the complexity that remains in the supply chain, it is straightforward and easy for actual fraudsters to mix loads of bot traffic in with real traffic - and everyone is happy because more volume means more profits. No one said the traffic was actual humans looking at webpages and causing ads to load, right?

Recommendation: Don't assume the exchange or SSP is checking for more than the existence of the ads.txt file rather than using its contents to verify inventory is sold by the right sellers. Do the other stuff mentioned above to check for yourself.

Ads.txt Files with "Errors" <-- those were not f**ng "errors"

Ads.txt files do not have "errors" due to simple mistakes or oversight. These are done deliberately and usually for nefarious reasons.

Here's an example. Breitbart.com works with complicit "supply side platforms" (SSP) to sell ad inventory using the same sellerID as other mainstream sites, that also sell through that SSP. This conveniently launders those impressions because a brietbart-specific sellerID is not seen by the exchanges making payment to the SSP.

Last but not least, the following form of fraud is even larger in scale.

Recommendation: Use adstxt.com to check domains you may be suspicious about and see how many other sites they share an ads.txt file with; or how many other sites use the same sellerIDs. It may not be fraud, but it also COULD be fraud. Check it for yourself.

Ads.txt Syndication - a.k.a Renting Complete Ads.txt files

This latest "zero-day" exploit of ads.txt finds bad guys renting entire ads.txt files from established sellers. This is how it happens. Fake sites are not able to easily get new sellerIDs, because exchanges are being more strict about who they let in. So how do they start selling fraudulent inventory? Right, by renting someone else's ads.txt file, in its entirety. They cut a deal where the owner of the vetted ads.txt file allows them to run inventory through them. They even take payment from the exchanges, and then make under-the-table payments to these sub-sites that couldn't get their own sellerIDs.

This way, the sub-sites never even show up on payment flows because the exchanges never paid to sellerIDs of strange sites. All the exchanges see is an increase in volume coming through suppliers they have been working with for years. How convenient. P.S. Exactly the same phenomenon is going on with mobile apps and app-ads.txt will also not stop that form of fraud.

"Ads.txt was never intended to address the issue of traffic quality and sellers that monetize counterfeit traffic have found a way to still be fully compliant with the Ads.txt initiative." Ads.txt was designed to bring transparency to who was authorized to sell the inventory; but does not determine whether the inventory was fake or not. And it doesn't solve the ad fraud coming from sites that sell entirely fake ad inventory. The following is an example of someone promoting ads.txt syndication - they give you the ads.txt file to put on your site.

Leveraging data from adstxt.com, we can see large networks of sites that use the exact same ads.txt -- the hash is a simplified representation of identical ads.txt files. Note the top item has more than 15,000 domains with the same ads.txt file.

The following are some of the 15,000 sites that have the same ads.txt file. I am not saying these domains are fraudulent. But you should check them yourself and ask yourself whether you want to keep buying from them and have your ads shown on them. I recommend you take a quick look at Alexa data for each domain using the following syntax: https://www.alexa.com/siteinfo/0x1gab.com You should also use Alexa's Audience Overlap Tool too find other sites that have high audience overlap. Sites that have very high or 100% audience overlap might be using the same botnet to create traffic.

Recommendation: Look for abnormal changes in traffic/impression volume. Check Alexa stats on the domains you are suspicious of. When you see abrupt changes, something's wrong (a site doesn't magically get a far larger human audience overnight; it's more likely they just bought all that traffic from bots disguised as humans).

How Significant is this Exploit?

Taking a group of sites that syndicate the same ads.txt file -- and taking only the top domains that have more than 100 million potential impressions per month. Using the round number of 54 billion potential impressions from this network of sites, and assuming a $5 CPM, the potential dollar amount at stake is $270 million per month.

Recommendation: blacklist domains and reduce spending on fraudulent/suspicious sites; see if your business outcomes change. If your outcomes do not change, then whether it was fraud or not, it is not worth investing more in.

In conclusion, have a look for yourself (using tools like adstxt.com) and decide whether you want to continue buying ads on these sites. And be sure to NOT assume that ads.txt has solved ad fraud. The protocol itself does not solve the bad behaviors of humans. And the loopholes mentioned above should be closed by more aggressive action by the exchanges and SSPs: A member of the ads.txt working group commented in this way:

Much of the blame here is on the exchanges/SSPs for allowing

1. Vague networks of sites to use the same account/sellerID.
2. Reuse of one seller accounts name (domain, bundle) by other accounts who are controlled by unknown entities
3. Syndication abuse in various forms
4. Not cross-checking the financial flows based upon the ids+domain tuples in the ads.txt files.

Exchanges and SSPs need to do a better job of locking this down. Otherwise, they are leaving large loopholes open for bad guys to exploit.

About the Author:  “I advise advertisers and publishers on the technical aspects of fighting digital ad fraud and improving the effectiveness and transparency of digital advertising. I help audit their campaigns and show them detailed data so they can verify for themselves what is fraud and what is not fraud.” 

Follow me here on LinkedIn (click) and on Twitter @acfou (click)

Further reading: http://www.slideshare.net/augustinefou/presentations