Blog

  • The Definitive Approach for preventing duplicate transactions on Google Analytics – Using a Universal CustomTask

    It’s been a long time since I wrote my post about how to prevent duplicate transactions on Google Analytics. At that point, theย customTaskย wasn’t a thing on theย Google Analytics JSย library, and the approach consisted of writing a cookie on each transaction and then work with some blocking triggers.

    It’s a working solution for sure, but based on all the feedback I had over the years, it was not easy to understand for people. Things got worse even with the Enhanced Ecommerce since there’s no specific hit type to block ( remember that on EEC, any hit is used as a transport for the Ecommerce data ).

    That’s why I’m releasing a completely new approach to prevent duplicate transactions on Google Analytics. It’s based on the customTask functionality and it will work out of the box independently on how you have set up your Enhanced Ecommerce Tracking, sound good yes?

    If you wonder how are we going to achieve this, take a look at the following flow chart

    Basically, we’ll check the current hit payload to find out if it has any transaction-related data, and, only, in that case, we’ll be removed the e-commerce related data from the hit, If that transaction has been already tracked on the current browser ( we’ll be using a cookie to keep track of recorded transactions, just as we used to do in our old solution )

    To have this working the only thing we need to do it to create a new Variable in Google Tag Manager with the following for our “duplicate transactions blocking customTask” .

    *Note that I tried t add as many comments as I could in the customTask code, so please take some time to understand how it works! ๐Ÿ™‚

    function() {
      return function(customTaskModel) {
        var originalSendHitTask = customTaskModel.get('sendHitTask');
        // Helper Function to grab the rootDomain
        // Will help on seeting the cookie to the highest domain level
        var getRootDomain = function() {
          var domain = document.location.host;
          var rootDomain = null;
          if (domain.substring(0, 4) == "www.") {
            domain = domain.substring(4, domain.length);
          }
          var domParts = domain.split('.');
          for (var i = 1; i <= domParts.length; i++) {
            document.cookie = "testcookie=1; path=/; domain=" + domParts.slice(i * -1).join('.');
            if (document.cookie.indexOf("testcookie") != -1) {
              var rootDomain = domParts.slice(i * -1).join('.');
              document.cookie = "testcookie=1; expires=Thu, 01 Jan 1970 00:00:01 GMT; path=/; domain=" + domParts.slice(i * -1).join('.');
              break;
            }
          }
          return rootDomain;
        };
        // The custom Task
        customTaskModel.set('sendHitTask', function(model) {
          try {
            // Let's grab the hit payload
            var rawHitPayload = model.get('hitPayload');
            // We're converting the payload string into a key=>value object
            var hitPayload = (rawHitPayload).replace(/(^\?)/, '').split("&").map(function(n) {
              return n = n.split("="),
                this[n[0]] = n[1],
                this
            }.bind({}))[0];
    
            // Let's check if this hit contains a transaction info
            // if the hit contains a &pa parameter and the value equals to "purchase" this hits contains a transaction info        
            if ((hitPayload && hitPayload.pa && hitPayload.pa === "purchase")) {
              // Let's grab our the previous transactions saved in our cookie ( if any )  
              var transactionIds = document.cookie.replace(/(?:(?:^|.*;\s*)__transaction_ids\s*\=\s*([^;]*).*$)|^.*$/, "$1") ? document.cookie.replace(/(?:(?:^|.*;\s*)__transaction_ids\s*\=\s*([^;]*).*$)|^.*$/, "$1").split('|') : [];
              // if the current transaction ID is already logged into our cookie, let's perform the magic
              if (transactionIds.length > 0 && transactionIds.indexOf(hitPayload.ti) > -1) {            
                // EEC hit keys magic regex. The following regex will match all the payload keys that are related to the ecommerce
                var eecRelatedKeys = /^(pa|ti|ta|tr|ts|tt|tcc|pr(\d+)[a-z]{2}((\d+)|))$/;
                // Now we'll loop through all the payload keys and we'll remove the ones that are related to the ecommerce
                for (var key in hitPayload) {
                  if (key.match(eecRelatedKeys)) {
                    delete(hitPayload[key]);
                  }
                }
                // not let's update the payload into the hit model! :)
                model.set('hitPayload', Object.keys(hitPayload).map(function(key) {
                    return key + '=' + hitPayload[key];
                }).join('&'), true);            
              } else {
                // IF the execution arrived to this point. It means that this is a NEW transaction
                // Then, we'll do nothing to the payload but instead we'll be adding the current transaction ID to our cookie
                transactionIds = [hitPayload.ti].concat(transactionIds);
                var _expireDate = new Date();
                // This cookie will expire in 2 years
                _expireDate.setMonth(_expireDate.getMonth() + 24);
                document.cookie = "__transaction_ids=" + transactionIds.join('|') + ";expires=" + _expireDate + ";domain=" + getRootDomain() + ";path=/";            
              }
    
            }
            // Send the hit
            originalSendHitTask(model);
          } catch (err) {
            // In case the above fails, we want to send the hit in any case!!!
            originalSendHitTask(model);
          }
        });
      };
    }

    We’re done. From now all this customTask will be taking care of detecting transactions traveling on the hits, writing it to a cookie and removing the transaction data from the hit if needed!

    • You don’t need a blocking trigger
    • You don’t need an extra condition on your firing trigger
    • You don’t need a variable for checking for the value of the cookie
    • It’s doesn’t matter how you’ve set up your e-commerce tracking, the customTask will work despite your current approach ( sending it with the default pageview, or an event, or if using the dataLayer data or based on a variable that builds up the e-commerce data for GTM ).ย 
    • You won’t need to block your default pageview on the confirmation page to have the ecommerce working without duplicates.

    It will just simply work!

    Of course, you may want to block some other tags from firing since the customTask will write all the data into a cookie, it would be accessible for you to use it at your need. Just grab the “__transaction_ids” cookie value, and search for your already recorded transactions

  • Test API . Unit testing for Google Tag Manager Custom Templates

    Google Tag Manager team has just added, in a stealth mode, a new Test API for the Custom Templates. This Test API will allow us to define some Unit Tests in order to be able to automate the testing of our template before publishing them.

    You will find a new tab within the templates editor named “test” where you will be able to run some code before the tests start, and then add a set of unit test to run each time you want to test your library.

    For those who are not much into the programming according to the Wikipedia a Unit Test is:

    Inย computer programming,ย unit testingย is aย software testingย method by which individual units ofย source code, sets of one or more computer program modules together with associated control data, usage procedures, and operating procedures, are tested to determine whether they are fit for use.[1]

    https://en.wikipedia.org/wiki/Unit_testing

    In essence, they’re automated tests that make sure that a certain part of program execution, a unit, is working as expected. For example, if we were writing some unit tests for a page, we would be writing some Unit tests for example for checking if the contact form works, another to check if the add to card works, another one for testing if the search engine works, or if the current pages contain some specific elements.

    According to Google Tag Manager support pages, each test can provide sample input values, mock function calls, and assert code behaviors. ( More details GTM Tests ) .

    Unit Tests are also exported/imported, so this would help people that need to tweak any template to have their new code tested if the original author provided all the needed unit tests for testing the template.

    Setting up the Unit Testing may look like a tedious and time-wasting task the first time you have to deal with it, but with the time you’ll widely recover all the time invested ( NOT WASTED ) while setting up the tests.

    This new Test API doesn’t need to be initialized as the other ones since it’s run within its own section of the editor and its looks it’s already enabled there and it provided 5 new API’s we can use to defined and run our unit tests:

    assertApi Returns a matcher object that can be used to fluently make assertions about the given API.
    assertThat Theย assertThatย API is the one in charge to check if some specific value/variable matches the one we are expecting. GTM used Google’sย Truthย library as a model for this. Having one asset failing doesn’t affect other test cases.
    fail This will force the test to fail and return a giving message if any is provided.
    mock Theย mockย API will allow us to override/replace the current sandbox API’s behavior
    runCode Calling this method will run the code for the current template,

    I’m trying to show some simple examples of how these APIs work with examples. If you want to dig deeper I suggest you going to the official developer site for the Test API for a further read: https://developers.google.com/tag-manager/templates/api#runcode

    assertAPI

    assertApi(apiName)

    This API will return a matcher object that we will able to use to make asserts against the given API.

    For example, each tag execution must contain one call to gtmOnSuccess. Would make no sense having 2 calls to success within a single tag execution that could mean that we’re firing it at some point when it’s may not be true. Let’s write an API assertion to check that gtmOnSuccess is being called just once.

    This is our test current custom tag code

    // dummmmyyyy template
    data.gtmOnSuccess();
    const log = require('logToConsole');
    log('data =', data);
    
    data.gtmOnSuccess();

    There’re not many details about the currently supported subjects for this assertion tests. I’ll try to keep the following table updated as long as I find/test them out.

    Available subjects list

    wasCalled(number of calls)
    wasNotCalled()

    assertThat

    This API will allow making the assertions against our variables or subjects. It has been designed based on the Truth library by Google, which is a library for running assertion tests for Java and Android. You can find more info in the official URL: https://truth.dev/

    I’m not sure if all the subjects available on truth and that been currently modeled into the Test API For GTM, but you can find a list of officially provided one in the project GitHub repository, in case you’re curious about it: https://github.com/google/truth/tree/master/core/src/main/java/com/google/common/truth

    assertThat(actual, opt_message)

    For this let’s imagine that we have a variable that we expect to be returning “SIMO” as a string, ok?. For this test, we’ll write a small variable that returns “DAVID” instead of our expected value in order to test an assertion.

    // assetThat API Test
    // No Needed Libraries
    // return "SIMO"
    return "DAVID";

    Now, we’ll write a simple test that run an assert to test that the value returned by the variable actually is “DAVID”

    There’re not many details about the currently supported matched subjects for this assertion tests. I’ll try to keep the following table updated as long as I find/test them out.

    Available subjects list

    isEqualTo(expected)
    isNotEqualTo(expected)

    fail

    fail(opt_message)

    mock

    This API will allow us to replace or override the current Sandbox API’s functionality.
    Let use the following Custom Variable Code

    mock(apiName, returnValue);
    // Mock API Test
    // Add Needed Libraries
    const log = require('logToConsole');
    const encodeUri = require('encodeUri');
    
    const testValue = encodeUri("http://www.google.es");
    log('testValue=', testValue);
    
    return testValue;

    If we run variable we would expect to have the testValue to be the URL encoded, but we’ve mocked the encodeUri API in our test to return a fixed value:

    runCode

    This API actually takes care of running our template code. It optionally accepts a data object as a parameter and it will return a value if we’re testing a Variable or undefined in any other case.

    runCode(data)
    runCode({
      measurementID: 'G-XXXXXXXX' 
    });

    Passing an object data will allow us to set some values for the current tag/variable without the need to manually fill the data into the template fields.

    In the example above the data.measurementID variable will hold the value ‘G-XXXXXXX’ even w didn’t populate our field.

    Wrapping up

    Google Tag Manager is giving us the perfect tool ( as a start ) for being able to have some fail-proof containers and code. AFAIK it seems the library is supporting some basic matchers, but I expect them to be expanded in the future ( or documented! ).

  • UDO Debugger – Easiest way to work and QA Tealium implementations

    Some years ago, I published my “Tealium Debugger” extension for Chrome which after a nice email my Inbox got, ended being named “UDO Debugger” ( no hard feelings at all ).

    You may click no the next CTA if you want to install the extension in your browser =)

    That version published circa Apr 2017, was just showing up the current app state on the page loads, the current account, profile, and environment details. There was no support for anything else. Sadly I switched works and I was not able to work with Tealium anymore, despite my requests to get some testing account access to improve the extension.

    The good news is that I’m back on some works with Tealium which means it’s a nice time to update the extension. I’m pleased to say that after a lot of hours of works, I’ve completely refactored the extension, and I’ve added lots of cool features that I hope makes the analysts and developers in the need to work with Tealium a bit more easier ๐Ÿ™‚

    Let me put a small list of the current features on the debugger:

    • Tealium Presence Detection and profile, account, environment reporting.
    • NEW. App State, Utag.link, utag.view Reporting
    • NEW. The current fired tags are reported.
    • NEW. Clipboard Support.
    • NEW. Environments Switcher
    • NEW. Debug Enabler / Reporter

    Tealium Presence Detection

    The extension will automatically report to you if Tealium has been loaded on the current page.

    Along with this info, you’ll be able to see the current enabled environment, profile and account name. Also, you will be able to see the current version loaded ( useful for easily seeing when the last publication is live ).

    App State, Utag.link, utag.view Reporting

    In the previous versions, only the App State was being reported. ( when I refer to app state it’s the original utag_data value.

    On the new version, all the UDO model is shown on the page load, and for making the debugging easier all the data layer keys that are internal to Tealium. As you may know, Tealium collects some details by default data that can be used, for cookies, meta tags, audienceStream audiences, queryString parameters, etc ), are grouped into the _internal key, also we’re doing the same for the keys that are related to the eCommerce.

    This way you could just focus on the Universal Data Object keys that are really up to you.

    And yes now the UTAG.LINK and UTAG.VIEW calls are being reported. So each time your site does a call you’ll be notified.

    And the coolest thing is that you’ll be able to see if the current link/view was scoped for a certain tag or if they were pushed along with any callback, even being able to see a fully prettified and formated output of the callback code.

    The current fired tags are reported

    This new UDO Debugger version can to show you the current tags being fired by the site via Tealium. And they will appear in the report automatically as they fire :).

    And … did you ever meet any implementation where the tag names were named so badly, something like “marketing pixel” , or even not matching the current pixel, like firing a Google Ads Pixel and having a name like “Conversion Pixel FB” ?, I’m sure you did. Now on UDO Debugger, you’ll be able to click on the tags ID’s and you’ll get a full report on what that pixel is. Like the real template name, the description, the available configurable fields for that pixel on Tealium and … you will be able to see the current tag code, in a prettified and formated way without needing to go to Tealium and editing the template! ( say yay! )

    Clipboard Support

    Any app state/utag.link/utag.view is eligible to be copied into your clipboard with a mouse click. And they will be copied on a nicely formatted way, easy to report to your IT department or for writing your documentation ๐Ÿ™‚

    Environments Switcher

    Within the main window of the extension, you’ll be able to switch to DEV / QA / PROD environments. This setting while last while the browser session is open.

    If you ever worked with the native solution to switch environments on Tealium it has 2 big issues in my opinion:

    1. utag.sync is not affected. So it will not be redirected.
    2. If you use the switching cookie, instead of doing a redirect, it will load a second utag.js on top of the first one, ending on some odd behavior under certain circumstances. For example, let’s say that your “prod” environment has an extension that sets a global variable window.thyngster, then you say let’s load our “dev” environment where that variable is not set. You may see that variable being set despite you’re not expecting it, making you go crazy!

    We’re have solved this 2 issues, the newly added environment switcher added to UDO Debugger will perform 307 redirects for both utag.js and utag.sync.js files requests so you can be able to do some proper QA and testing.

    Debug Enabler / Reporter

    As I’m sure you already know you can have Tealium pushing all the debug logs to the console ( via the calls to utag.DB in the templates ). For getting this done, you’ll only need to create a cookie named ‘utagdb’ with a true value this way:

    document.cookie="utagdb=true";

    Still, a lot of stuff is pushed to the console, which makes difficult to follow what going on, since it’ll be messed with other messages, request failed or scripts errors.

    This version allows you to enable the debug switch just from the interface, and it will then show you in a nice table. you’ll even be able to check previous pages logs. Which will make the utag.debug diggest a lot easier.

  • Google Tag Manager Custom Template to grab the current Root domain Name

    It happened to me a lot of times that while working with cookies that for whatever the reason I didn’t remembered to add the domain name to the settings, or that I was not able to properly define the root domain.

    If you ever have had the need to set a cookie to the main root domain for your current domain, to make a cookie available for all the place, that depending on your current domain it may not be just enough to split the current hostname, because we need to think on the TLDs ( top-level domains ), SDLs ( second-level domain ), ccTLD ( country code top-level domain ), having all these variables you may end not knowing where the root domain is, for example is we’re talking about a co.uk domain , we could check things by the numbers of characters on each level, but that would be also not as relialable as it would look at a first time.

    Actually ,afaik, the only reliable way to check this is relying on a proper lookup table or database ( http://www.iana.org/domains/root/db ).

    Good thing on JavaScript is that we try grab the current hostname and start setting a cookie till it get’s actually set.

    For example let’s say that we have my.super.duper.domain.co.uk , we would be doing the following:

    Good thing is that now you can have this value available without needing to deal with any code, just import the following Custom Variable to your containers ๐Ÿ™‚

    Available on GTM Gallery or via GitHub: https://github.com/david-vallejo-com/gtm-template-root-domain-variable/

  • #proTip – Proactively Cookies Auditing with Google Tag Manager and Google Analytics

    Since GDPR was announced and more even with the rise of Safari’s ITP and the other upcoming protection feature by the other major browsers, the concers about users privacy was grown which is really nice.

    But despite browsers taking some actions to prevent the abuses, it’s difficult to follow up the current cookies being used in our sites because we may end finding new ones that has been added by the developers, or some new pixel added by someone into our TMS.

    For helping on this task, some weeks I launched Cookies.Report which is a online tool for checking the cookies generated by a site. It has some special features that will make the report more useful than just looking the cookies tab in our browser:

    • Report it’s run within a fully clean browser state. This is we won’t be targetting personal cookies that we may have because we’re logged somewhere else, or old cookies that we may have in place since time immemorial
    • The report will split the first and third parties cookies, so you can easily identify them
    • The report will tell you which cookies are set within a TMS ( currently Google Tag Manager and Tealium are supported ), or which ones are set in some other way ( hardcoded tags )
    • You will be able to export the cookies in xls, csv or pdf to share it with anyone else

    Proactively Tracking Cookies

    This is going to be a easy code, we’ll just to use the following snippet to pass a list of current cookies names into the user browsers into a variable and then we could pass it as a custom dimension in our pageview/event hits!.

    function(){
        document.cookie.split(';').map(function(c) {
            return c.split('=')[0].replace(/^[\s\uFEFF\xA0]+|[\s\uFEFF\xA0]+$/g, '');
        }).join("|");    
    }

    There’s a point to have in mind, if it’s the first time the user gets into the page it may happen that our pageview fires before our other javascripts are loaded and therefore we may be missing that data just on the landing pages.

    You could use a non-interactianal event that fires on DomReady of Window.load if this is a issue for you.

    Having the code needed to get the cookies list, we may think in some other more explicit workaround that may save hits. For example: We may generate a hash of the current cookies list, and just send an event each time that it changes, this way we’ll only be sending the data once each time a new cookie is added.

    After we start receiving the data, we will be able to do/know a lot of interesting things.

    • When a new cookies was added to the site
    • Which are the most set cookies
    • In which pages are the cookies being set
    • Setting an alert if dimension hold some forbidden cookie name

    Hope someone find this useful in some way. Happy to listen to any comment you may have about this approach ๐Ÿ™‚

  • Setting GA Cookies Server Side with PHP – ITP Workaround

    Yeah, I know, this has been done before by others, but since I had this piece of code around, I decided to publish it.

    It’s not going to be usual, method since we’ll be relying on sending a duplicate hit to our tracking subdomain. That way we’ll be able to have our stuff tracked server-side in a secondary dimension while those hits will take care of setting the cookie expiring time on the main. isn’t it cool?

    What we’ll need

    1. A subdomain for our domain, for example collect.domain.com ( it’s better if we don’t make it that obvious )
    2. A customTask added in our tags, in order to send a hit copy to our subdomain.
    3. We’ll need some PHP code that will receive the hit details ( payload, cookie name, etc ), will have the job of writting the cookie back and also will be taking care of the CORS headers.

    Working Flow

    2. Let’s Start: Setting up our customTask

    The first thing we need to do is to create a custom Variable with our customTask code. You can find the code below.


    Copy the code

    function() {
        var gaProxyEndpoint = 'https://sub.domain.com/collect/';
        return function(customTaskModel) {
            customTaskModel.set('cookieUpdate', false);
            var originalSendHitTask = customTaskModel.get('sendHitTask');
            customTaskModel.set('sendHitTask', function(model) {
                originalSendHitTask(model);
                try {
                    var gaCookieName = model.get('cookieName');
                    var base64encodedPayload = btoa(model.get('hitPayload')+'&ckn='+gaCookieName);             
                    var xhr = new XMLHttpRequest;
                    xhr.withCredentials = true;
                    xhr.open("POST", gaProxyEndpoint, false);
                    xhr.send(base64encodedPayload);
                } catch (err) {
                    originalSendHitTask(model);
                }
            });
        }
        ;
    }

    After we have the variable, we’ll need to tell the tags to use it a customTask, if you’re already using a customTask you’ll need to sort it out yourself mergin this piece of code with your current code.

    As you may see on the following screenshot, there’s an extra field set. It’s name “cookieUpdate”, and we’ll need that set to false because otherways analytics.js will keep updating the cookie expire time after we did the same, therefore the cookie time will be reset to 7 days on Safari.

    3. Let’s finish: Setting up the PHP files

    We’ll need to upload the php files to our subdomain

    Files: https://github.com/david-vallejo-com/ga-proxy-itp

    We are all set now!

    After we publish everything, for each hit send via GTM, a request will be also be sent to our endpoint, which will take care of everything so cookies are written from the server.

    There’re some tweaks you may want to add to the customTask, for example you may want to check if it’s a Safari browser, and just run the custom task if the check is true. Just give a try to “Detect Safari JavaScript” search on Google ๐Ÿ™‚

    Extra

    As you may have read, we could also set the PHP file to send a copy of the hits. For enabling this feature, just open the index.php and uncomment the following line

    // private $propertyId = 'UA-XXXXXXX-UU';

    The good thing about this is that the payload goes encoded using BASE64, so it won’t be detectable by adblockers at all. So you may end finding some better sessions info on the secondary property?.

    ok, I know, most adblockers will likely block the analytics.js file so, none of this will make a difference, but, even if it’s not a good idea you may plan to server the file locally with a different name, do you follow me? ๐Ÿ™‚

    If someone gives it a try just let me know!

  • #tip – Grabbing current dataLayer variable name within GTM

    This is going to be a quick #tip post. and it’s coming from a post on Measure Slack.

    Let’s learn how to grab the current dataLayer variable name within GTM. We’ll need to create a variable with the following code:

    function() {
        return Object.values(window.google_tag_manager).find(function(e) {
            if (e.dataLayer) return true;
        }).dataLayer.name;
    }

    Now our variable should hold the dataLayer name ๐Ÿ™‚

  • Send APP+WEB Hits to any remote endpoint

    Google released APP+WEB Properties some weeks ago now, and one of the most missed features looking back to Universal Analytics, is the chance to work with a customTask ( Hi Simo!!! ).

    In this post I’ll cover how to send a copy of the hits to anywhere we want, in case we want to hold a copy of them, or processing them in some other place (like snowplow)

    First, we need to take in mind that APP+WEB hits, may contain +1 hits into their payload, therefore we’ll splitting the hits in single hits before sending them to our endpoint. If you want to learn more about how APP+WEB Protocol works you can take a look to this post

    APP+WEB API makes use of navigator.sendBeacon to send the hits, so we’ll writting a proxy pattern to listing to sendBeacon calls in order to be able to grab the payloads and sending them back to our own endpoint. In the middle of this process we’ll be splitting the hits in single hits.

    On this specific example we’re sending the data via POST, but it can be easily be send as a QS payload. Just feel free to modify the code at your needs ).

    In order to have this working, we’ll need create a new TAG in Google Tag Manager ( or just add this code into your site in some other way ) with the following code. ( feel free to modify the endpoint variable at the end. Let’s call it: “TOOL – APP + WEB Remote Logger”

    <script>
    "use strict";
    (function() {
        try {
            var proxied = window.navigator.sendBeacon;
            window.navigator.sendBeacon = function() {
                var parse = function getQueryParameters(str) {
                    return (str || document.location.search).replace(/(^\?)/, "").split("&").map(function(n) {
                        return (n = n.split("=")),
                        (this[n[0]] = n[1]),
                        this;
                    }
                    .bind({}))[0];
                };
                // Check if it's a APP+WEB hit
                if (arguments && arguments[0].match(/google-analytics\.com.*v\=2\&/)) {
    
                    var payloads = [];
                    var qs = arguments[0].split("?")[1];
                    // APP+WEB Can hold more than 1 hit on the same payload, we'll send separete this, let's split them
                    if (arguments[1] && !qs.en) {
                        arguments[1].split("\n").forEach(function(e) {
                            var tmp = Object.assign(parse(qs), parse(e));
                            payloads.push(tmp);
                        });
                    } else {
                        payloads.push(qs);
                    }
                    var data = {
                        records: payloads
                    };
                    var endpoint = "https://myownendpoint/g/collect";
                    fetch(endpoint, {
                        method: "POST",
                        body: JSON.stringify(data)
                    });
                }
    
                return proxied.apply(this, arguments);
            }
            ;
        } catch (e) {}
    }
    )();
    </script>

    After we have that tag in place we’ll need to use the tag secuencien with our pageview tag ( this needs to run just one, so don’t add it to other hits ).

    And now we’ll only need to publish our container, and a copy of the hits will be sent to the endpoint we’ve defined on the code! Easy!

  • Release: Client-Side dataLayer validation Library

    If you’re a web analyst, you SHOULD be worried about data quality, we want to have data, but we need this data to be correct, there’s no point of having tons of corrupted data that we won’t be able to sue and for this we need to monitor our implementations regularly to be sure our data is always being reported properly.

    Since We all are web analyst I’m pretty sure we are relying a flawless dataLayer ( are we? ๐Ÿ™‚ ). But I’m sure that at least once in the past our precious data holding variable ( aka “datalayer” ) ended messed up for whatever reason ( a refactor task that messed up some keys, some JS code that overrided a value ).

    It’s true there are out there plenty of solutions to monitor our implementations setups like Observe Point , Hub Scan , which do a real great job, but that maybe have too many features that we won’t be use at all.

    On this post we’ll talking about build a dataLayer model monitoring tool, ( yep, Simo and I already published an automatated testing tool some time ago: https://www.simoahava.com/analytics/automated-tests-for-google-tag-managers-datalayer/ ) , but that implies having some IT infraestructure, the checks are going to be done each day, it will only be tested under just one JS engine.

    So this time my proposal is doing the monitoring client-side. This will allow us to have a full sampling from almost any browser/connection/location on how the dataLayer is being reported for users!.

    The advantages are:

    • We’ll get alerts in real time
    • No need for any extra servers, infrastructure ( we’ll be able to use GA )
    • We’ll have data for all kind of browsers and users
    • Easy setup

    For this we’ll be using a little validation library that I wrote some years ago, that will be take care of validatin the current dataLayer model against a predefined dataLayer Schema rules.

    Let’s start adding a tag with the library on GTM, we’ll just need to copy and paste the following snippet, there’s not need to change any on it

    /* Code Moved to GitHub*/
    Get the snippet from the following URL: 
    https://raw.githubusercontent.com/david-vallejo-com/cs-datalayer-monitor/master/build/bundle.min.js

    Next thing to do is defining our dataLayer validation Schema, which is defined this way:

    var mySchema = { "page": {
                           "author": "required",
                   },
    		       "user": {
    				        "id": "integer",
    				        "name": "contains:David"
       		      }
    };

    We’ll need to define each key we want to monitor, and the validation rule we want to apply.

    Right now the following rules are available:

    • present ( this checks if a key is present, despite it’s set as null, undefined, empty string )
    • required ( this requires the key to have a value )
    • contains ( case sensitive )
    • notcontains ( case sensitive )
    • regex ( case insensitive )
    • integer
    • float
    • string
    • object

    If the post gets enough attention I’m planning on adding support for “contains”, “regex”, comparisions ( <=> ), and chained checks.

    Then we’ll need to call the validation library:

    var validationResults = window._validateDataLayer(myDatalayer , mySchema ,false);

    The dataLayer validation library will return an array of errors found in the datalayer if any are found. From that on we could for example fire an event to GA, or notifing any other tool we want.

    I’m attaching a small snippets that take care of grabbing the current Google Tag Manager dataLayer model and then it pushes the data to the dataLayer, from that point we can do whatever we want with the info.

    // This goes after the library code above.
    <script>
    /* 
    [REPLACE WITH THE CODE FROM GITHUB]
    */
    
    var myDatalayer = { user :{ id: 1, name:'david' }};
    var mySchema = { "page": {
                           "author": "required",
                   },
    		       "user": {
    				        "id": "integer",
    				        "name": "contains:David"
       		      }
    };
    var validationResults = window._validateDataLayer(myDatalayer , mySchema ,false);
    
    if(validationResults && validationResults.length>0){     
            dataLayer.push({
                event: 'dataLayer-validation-errors',
                errors: validationResults
            });
    }
    </script>

    You may be thinking that this may led on hundreds of pushed being send to GTM, but the library has a throttling cookie, which will just send 1 event per browser session, that way we’ll be only notified once per user/day.

    This should work for any other TMS rather than GTM, but for GTM we need to have an extra thing in mind, and it’s WHEN we fire the validation tool.
    We need to defined at which execution/event time we want to check the data. I suggest to fire the validation on Window.Load event ( gtm.load ) , since in almost most time the dataLayer model will be totally loaded, but if our setup is relying on All Pages (gtm.js) and we expect the data to be available on that event ( or any other custom one ), we’ll be able to set the triggers for our monitoring tag for that same event! )

    The new version includes a dataLayer grabing help, so you would only need to do the following.

    <script>
    /* 
    [REPLACE WITH THE CODE FROM GITHUB]
    */
    
    var mySchema = { "page": {"author": "required"},
    	       "user": {
                           "id": "integer",
    		       "name": "contains:David"
       	       }
    };
    var validationResults = window._validateDataLayer('gtm', mySchema ,false);
    if(validationResults && validationResults.length>0){     
            dataLayer.push({
                event: 'dataLayer-validation-errors',
                errors: validationResults
            });
    }
    </script>

    Let me know your thoughs and improvement ideas!

    Official Library Repository: https://github.com/david-vallejo-com/cs-datalayer-monitor

  • Technical Guide to Global Site Tag ( gtag.js )

    Global Site Tag was introduced by Google back in October 2017. As the name suggests it was launched in order to unify all the different tracking libraries that we can use for Google Products, Adwords, Google Analytics, DoubleClick, etc.

    In my opinion the launch of APP+WEB Properties has set an inflexion point about the gtag.js use, since it’s now when it is really coming into the scene, and it’s just for a simple reason: before APP+WEB it was just a duplicate library not offering anything extra from the legacy libraries, meaning that there wasn’t a real reason to move on into Global Site Tag tracking. But now Google launched the first product’s JS library 100% based on GTAG.js, it makes the perfect moment to explain how it works ๐Ÿ™‚

    What’s the Global Site Tag?

    It’s an API and a tagging platform for all Google Products ( Google Analytics, Google Ads, Google Marketing Platform, APP+WEB ), which will help us to have an standarized implementation that will be able to work with any products.

    It’s based on a list of predefined events ( “purchase”, “page_view”, “add_to_cart” , etc ),

    This means that we’ll just need to setup a “purchase” event and all the transactional data could be sent to Adwords Conversion Pixel, to Google Analytics or to other any supported tool.

    For this task Google has pre-defined a list of events, each of them supported a predefined parameters, and GTAG.js will take care of mapping this event data to any of the supported products by Global Site Tag at our convenience.

    How do I Add GTAG to my site

    It will be as simple as adding the following snippet to our site, preferible before closing the </head> tag.

    <script async src = "https://www.googletagmanager.com/gtag/js?id={{TRACK_ID}}"></script>
    <script>
      window.dataLayer = window.dataLayer || [];
      function gtag() {
        dataLayer.push(arguments);
      }
    gtag('js', new Date());
    gtag('config', '{{TRACK_ID}}');
    </script>

    Of course we gonna need to update the {{TRACK_ID}} with the proper value.

    How does GTAG know where to send the data?

    GTAG.js will now where it needs to fire the data because of the {{TRACK_ID}} value. Each of the Google Products “TrackID”s start with a prefix will tells google server with code to provide.

    For example if we’re setting up a Google Analytics tracking, the {{TRACK_ID}} will start with a “UA-” prefix.

    I’m attaching a table with the current possible values

    PrefixTool NameTrack ID Name
    AW-AdWordsConversion ID
    DC-Double ClickAdvertiser ID
    UA-Universal AnalyticsProperty ID
    G-APP+WEBMeasurement ID

    Then we’ll have the ‘config’ events where we’ll sending the initializacion and extra parameters.

    For example if we want to enable the cross-tracking for our Universal Analytics setups we’ll be using the following

    gtag('config', 'UA-XXXXXXX-Y', {
      'linker': {
        'domains': ['domain.com']
      }
    });

    Then if we want to setup a Universal Analytics pageview tracking + Adwords Remarketing ( both firing for all pages ) >

        window.dataLayer = window.dataLayer || [];
        function gtag(){dataLayer.push(arguments);}
        gtag('js', new Date());
    
        gtag('config', 'UA-XXXXXXX-Y');
        gtag('config', 'AW-1231231231');

    GTAG will know that we want to send the information to Adwords and Google Analytics and will take the stuff from here and fire everything for us. For example if we send a purchase event, it will send a transaction to Google Analytics and a conversion to Adwords

    How does it internally works

    GTAG.js is backed up using Google Tag Manager, maybe it’s not an exact definition of what GTAG.js is, but I really feel that we could mostly think on it as a “predefined” container that supports all Google Products tags and that takes care of mapping the data to all of them in an automatic way. ( NOTE: this may be a way to simplistic way to define it, but hope you get the idea )

    Since it relies on GTM code, and henceit has a dataLayer ( which will be shared with GTM if the site is using it also ), and that’s why you find sites not using Google Tag Manager and still having a dataLayer variable and even some of the default events like the gtm.dom and gtm.load.

    Let’s take a look to the window.gtag function:

    function gtag() {
     // Takes all Arguments and push them back to the dataLayer
     dataLayer.push(arguments);
    }

    Yep, it’s just a proxy, any parameters you send to gtag are pushed back to the dataLayer as an Arguments variable.,

    Arguments Type Object is an Array like variable that is only avaiable within the function internal scope and that holds a list of all parameters being pass to the function.

    Despite all these technical definitions, and just get the point all the data passed to gtag() function it’s forwarded to the dataLayer as an Array

    Let’s check the following flow to understand how it works

    GTAG.JS WORKING FLOW

    Did you know that you can tell GTAG.js to use an specific dataLayer name instead of the default one? , just ad a “l” (it’s a lowerCase “L”) parameter to you loading script

    <script async src = "https://www.googletagmanager.com/gtag/js?id={{TRACK_ID}}?dl=myOwnDataLayerVariableName"></script>

    Which events are currently supported by GTAG.js

    I tried to build a list with all the events I was able to ( as 15th Auh 2019 ), but please take in mind that this list may be missing some of them, and that Google is likely going to add new ones in the future. Refer to official documentation for more curated data.

    EventCategoryEEC DataNon-InteractionalParameters
    add_payment_infoecommerceNoNo
    add_to_cartecommerceYesNovalue
    currency
    items
    add_to_wishlistecommerceNoNovalue
    currency
    items
    begin_checkoutecommerceYesNovalue
    currency
    items
    coupon
    checkout_progressecommerceYesNovalue
    currency
    items
    coupon
    checkout_step
    checkout_option
    exceptionerrorsNoNodescription
    fatal
    purchaseecommerceYesNotransaction_id
    value
    currency
    tax
    shipping
    items
    coupon
    refundecommerceYesNotransaction_id
    value
    currency
    tax
    shipping
    items
    remove_from_cartecommerceYesNovalue
    currency
    items
    set_checkout_optionecommerceYesNocheckout_step
    checkout_option
    generate_leadengagementNoNovalue
    currency
    transaction_id
    loginengagementNoNomethod
    searchengagementNoNosearch_term
    select_contentengagementYesNoitems
    promotions
    content_type
    content_id
    shareengagementNoNomethod
    content_type
    content_id
    sign_upengagementNoNomethod
    view_itemengagementYesYesitems
    view_item_listengagementYesYesitems
    view_promotionengagementYesYespromotions
    view_search_resultsengagementNoYessearch_term
    page_viewenhanced measurementNoNopage_title
    page_location
    page_path
    screen_viewenhanced measurementNoNoscreen_name
    app_name
    app_id
    app_version
    app_installer_id
    scrollenhanced measurementNoNopercent_scrolled
    clickenhanced measurementNoNo
    timing_completetimingNoYesname
    value
    event_category
    event_label

    Hope this post helps you to lose some fear to the GTAG.js and helps you understand how face your upcoming APP+WEB implementations with some more background about how things work ๐Ÿ™‚