Introduction
It’s been a long time since I took care of this blog’s “Analytics” ( In the blacksmith’s house, a wooden knife). And I noticed that would be cool having the info about the Anchor Text the sites referring to my sites are using to link me.
So I’m sharing the solution I built today in order to capture which Anchor Text was on the referring URLs and sending the info back to Google Tag Manager and from there we’ll be able send an event to APP+WEB or to any other place we want 🙂
How it works
Execution Flow Chart
The flow chart on the right side, shows how the executions flow works. We’ll have 2 main pieces:
– One GTM CUSTOM HTML Tag
– One PHP File
The first one will the responsible of doing the main logic and doing a XMLRequest call to the second one that will take care of reading the current visitor referrer page and scrape it in order to try to find the current Anchor Text that the user clicked.
We’re using extense logic to void any kind of false positives/duplicate hits. For example when an user goes back into a mobile phone or swipes. We don’t want to consider these “page reloads” as landings despite they may still hold a valid referrer info.
SERVER SIDE CODE
PHP Snippet Code
First we need to upload the following php snippet to any server supporting PHP 7.x ( because of the use of arrays literals ).
This code can be highly improved for example for adding a timeout in the case the page is not reachable. If someone asks I may adding more sanity check for the script.
// David Vallejo (@thyngster)
// 2020-04-14
// Needs PHP7.X
if(!isset($_GET["url"])){
die("missing url parameter");
}
$links = [];
if(isset($_SERVER["HTTP_REFERER"])){
$url = $_GET["url"];
$referrer_link_html_content = file_get_contents($url);
$current_domain = str_replace("www.","", parse_url($_SERVER["HTTP_REFERER"], PHP_URL_HOST));
$doc = new DOMDocument();
$doc->loadHTML($referrer_link_html_content);
$rows = $doc->getElementsByTagName('a');
foreach ($rows as $row)
{
if($row instanceof DOMElement){
preg_match_all('/'.$current_domain.'/i', $row->getAttribute('href'), $matches, PREG_OFFSET_CAPTURE);
if(count($matches[0]) > 0){
$links[] = [
"url" => $row->getAttribute('href'),
"anchor_text" => $row->textContent
];
}
}
}
}
header('Content-type: application/json; charset=UTF-8');
header("Access-Control-Allow-Origin: *");
echo json_encode($links, JSON_PRETTY_PRINT | JSON_UNESCAPED_UNICODE | JSON_UNESCAPED_SLASHES);
exit;Python Snippet code
I know this code is not the best one since I’m not a python coder, but it can give an overall idea about how to run this based on the Python.
should be used like:
python anchor.py REFFERER_LINK LINKTOSEARCH# use: python anchor.py REFFERER LINKTOSEARCH
#!/usr/bin/env python
import json
import urllib2
import requests
import sys
from bs4 import BeautifulSoup
from urlparse import urlparse
links = []
if len(sys.argv) > 1:
url = sys.argv[1]
else:
print("URL argument is missing")
sys.exit()
if len(sys.argv) > 2:
referrer = sys.argv[2]
else:
print("REFERRER argument is missing")
sys.exit()
headers = {'User-Agent': 'Mozilla/5.0'}
response = requests.get(url, headers = headers)
soup = BeautifulSoup(response.text, "html.parser")
for ahref in soup.select('a[href*="'+urlparse(referrer).netloc.replace("www.", "")+'"]'):
links.append({
"url": ahref.attrs["href"],
"anchor_text": ahref.text
})
print json.dumps(links, sort_keys=True,indent=4, separators=(',', ': '))GTM Custom HTML Code
NOTE Remember that the following code needs to be added to GTM wrapped between
<script></script>tags!
Also remember that we need to update the endPointUrl value to the domain where we’ve uploaded the PHP script
(function(){
try{
var endPointUrl = 'https://domain.com/getLinkInfo.php';
// We don't want this to run on page reloads or navigations. Just on Real Landings
if (window.performance && window.performance.navigation && window.performance.navigation.type === 0) {
var referrer = document.referrer;
var current_url = document.location.href;
var grab_hostname_from_url = function(url) {
var h;
var a = document.createElement("a");
a.href = url;
h = a.hostname.replace('www.', '');
return h;
}
// Only continue if the current referrer is set to a valid URL
if (referrer.match(/^(?:http(s)?:\/\/)?[\w.-]+(?:\.[\w\.-]+)+[\w\-\._~:/?#[\]@!\$&'\(\)\*\+,;=.]+$/)) {
// current referrer domain != current_domain
console.log(grab_hostname_from_url(grab_hostname_from_url(referrer).indexOf(grab_hostname_from_url(current_url)) === -1))
if (grab_hostname_from_url(referrer).indexOf(grab_hostname_from_url(current_url)) === -1) {
fetch(endPointUrl+ '?url=' + referrer).then(function(response) {
return response.json();
}).then(function(json) {
json.forEach(function(link) {
if (current_url.indexOf(link.url)>-1) {
//if (current_url===link.url.indexOf) {
window.dataLayer.push({
event: 'incoming-link',
linked_url: link.url,
landing_url: document.location.href,
referring_url: referrer,
anchor_text: link.linkText
});
}
})
});
}
}
}
}catch(e){}
})();Now we’re only one step away of having this working, we’ll need to setup a firing trigger for our tag, this ideally should be the All Pages trigger to get it fired asap.
Reported Data Info
| dataLayer Key | dataLayer Value |
|---|---|
event | incoming-link |
linked_url | Current Link in the Referral Page |
landing_url | Current URL |
referring_url | Full Referrer Info |
anchor_text | The Anchor Text on the referrer page linking to your site |
Caveats
Please note that this solution relies on the current document.referrer, so don’t expect it to work for all referrals since some of them may be stripping the full referrer info, like Google SERPS do, or even some browser may end stripping the referrer details down to origin for privacy reason.
Also it may happens that the referring URL is linking to us in more than 1 place, on this case the scraping endpoint will return all the links and anchors texts matching. From that point of, it’s up to you how you report it in Google Analytics or any too 😀
In any case this should work for most of the common referrals traffic.
