What is Referrer Spam?
With smaller websites especially, you may have noticed the huge swarm of annoying referral spam that has been plaguing our Google Analytics in recent months. In fact, if you’ve just launched a new website and aren’t yet pushing for much traffic, if you sort your traffic by Bounce Rate, it might look a little like this:
Gross, right? An eighth of the website’s traffic is from a single spammy website. Just sitting there, screwing up my analytics. Of course, you could exclude them right there and then as you’re browsing. But they’ll keep popping up, infecting your data with more fake traffic… You need to get rid of them once and for all.
Those figures above seem small, but they add up and can skew your website data massively depending on the incoming traffic of your website. Giving you the wrong figures for things such as conversation rates, bounce rates, and a whole load of other metrics in addition to just pageviews and visits.
How To Filter Referrer Spam in Google Analytics Using REGEX
There are two main types of this Google Analytics spam: the annoying web crawlers which you can block from accessing your website (and therefore your analytics) using .htaccess, and ghost referrals. Ghost referrals don’t actually visit your website in the first place, so blocking them in your .htaccess will do nothing.
This solution will only prevent referral spam from appearing in your Google Analytics. Annoying, attention-seeking web crawlers will still be able to access your website but their visit won’t be recorded in Analytics, unless you block them in your .htaccess too.
1) Create a Master View – Don’t create new filters on your main All Website Data view!
Google highly recommends that webmasters always maintain one view that collects all website data, with no filters to alter the data. (In fact, this even comes up in the Google Analytics Certification exam that you should always keep one unfiltered view!) This is so that you are always able to have access to all incoming traffic and data, should you need it.
Creating a new “Master View” (by which I mean the view that you will most likely be checking everyday) is easy. It’s as easy as clicking on Create new view, where you will be able to name your view. And that’s all there is to it. Once you’ve named your view, it’s right there for you to do with it what you please. By default, it will just be unfiltered. Filters do not work retroactively, so I recommend creating the filter as soon as you have your view, before any spam gets in.
2) Create your Fake Referral Spam filter
In the account’s Admin, once you have selected your new View, you can click on Filters and + New Filter. Where you will see the Add Filter To View page:
You can name your Filter whatever you want to help you remember what the filter actually does. You want to set your Filter Type to Custom, so that you can Exclude by Campaign Source in the Filter Field.
And then in the Filter Pattern, you can type out or paste in the REGEX pattern to block your spam. The one in this example will block all of the main ones that pop up, so feel free to copy and paste this:
You can add to or replace the REGEX above with your most common spam referrers. Make sure that every dot (.) has a backslash (\) in front of it as you literally do mean a . and not “any character” as a dot would usually signify in REGEX. The backslash “escapes” the following character
If you frequently get hit by any other fake referral websites, you should add them in to the pattern or create a new filter to add them in. You may need more than one filter to filter all these websites as there is a character limit of 255 for filter patterns.
3) Edit your Fake Referral Spam Filter for any new spam sites
New websites will in time pop up, and it’s important to keep on top of this. You can add to or edit your filters whenever you want, or simply create a new one.
Spam Referrers to Filter:
There are too many to list… but these guys seem to pop up the most!
As you can see, these referrer spam domain names often tend to disguise themselves as popular websites (e.g. hulfingtonpost.com and theguardlan.com) making you think that one of the top names in the media industry has linked to you somewhere, and encouraging you to click through to see where you were linked from. Or they make you think as if you’ve won some sort of website award so you want to see what award you’ve won (e.g. bestwebsitesawards.com). But most of the time, the domain name tries to offer you something for “free” or something that is the “best” (such as SEO, traffic, social sharing buttons, or some other tool or service).
There’s a huge blacklist available to help you block known spammy HTTP referrers in your .htaccess too. Of which you can select which ones to filter in your analytics or whatever. For obvious reasons, some of the URLs on that list are a bit… NSFW perhaps. Also, be warned that you should NOT, under any circumstances, just copy and paste the whole thing into your website’s .htaccess. But don’t forget: this will only affect the referrer spambots that actually visit the website. Not the ghost referral bots.