Universal Analytics Migration Library - Custom task
When Google Announced Google Analytics 4, I started to work on this library, with the main aim to be used as a customTask.
Now the current deadline is 1 year-ish for all free Universal Analytics users and 3 extra months ( thanks? ) for paid users. In my very humble opinion, this is very short notice for such a big work. In my experience, a big implementation will take at least 6 months on average, for defining everything, write up the specs, have it team implement it, the GTM ( or any other TMS ) setup and then doing the QAing a dozen times until finally all the data is coming as expected and with the proper values.
The current help Google is offering in this situation is allowing us to turn on a switch on the interface that will autogenerate some GA4 events and pageviews from the Universal Analytics hits. If you wonder how they're doing this, they're adding a listener for the window.ga('send') calls.
At a first look, it may look like good help, but I really feel that's all against how Google Analytics 4 is supposed to be implemented. I really see a lot of issues coming from this:
- No data model will be implemented on
GA4, if you have messy events tracking setup, you're new data onGA4will be at least as messy as your events were on Universal Analytics - It won't pass the custom
dimensions/metrics/content grouping, this is likely going to be insufficient for anyone having at least 1 dimension. - Timing, Exceptions, Legacy Transaction Events, and Social hits, won't be migrated.
Another issue with the automatic migration tool Google offers is that an Event Action seems to be converted to an Event Name. That's a terrible thing, most of the clients I worked on, will end up having thousands of different events. ( I wonder if this is the reason for having a "non-written" no limit for the events coming from the website ).
In any case, if you're migrating your setup to Google Analytics 4, there's a no better chance to fix whatever is not right with your setup. One of the most important implementation steps in GA4 is the data model definition, that's gonna define your current data quality and it's gonna limit you in the future ( event parameter limits, user properties limits, etc ). Don't take the lazy path and regret it later.
Universal Analytics Payload Parser
Now, the main post point, I've released a Universal Analytics Payload Parser library. This library is able of taking a Univeral Analytics Hit and converting them into some properly modeled dataLayer pushes.
This library can be used as a customTask in Universal Analytics and it will push the hit contained data into a Google Tag Manager (dataLayer.push), Tealium IQ (utag.link) or Adobe Launch (_satellite.track) pushes.
Instead of having someone send some predefined data into your GA4 property, this tool will push the data to your TMS so you can send the data to GA4 in the format, you want or pass the data to ANY other tool you want if at some point you want to migrate to any other analytics solution ( Snowplow, Matomo, Amplitude, Adobe Analytics, etc ), or even to any internal data solution you have built in your company.
Features
This library accepts a Universal Analytics Payload and parses it to return a formated and standardized object that can be used to pass the current info to any Tag Management System. At the time of writing this post, the library supports the output for Google Tag Manager, Tealium IQ, or Adobe Launch, adding a callback of support for sending the data to any external source should be pretty straightforward.
This library will take all the UA hit types, and will create an object with all data contained on that hit.
- It generates individual pushes for each UA hit.
- It'll attach all event-related implicit parameters ( like the page title or page URL for the pageview event, or the event
category,action,labelfor the events ). Check the table below for more details. - It will also map all the custom dimensions, metrics, and content groupings related to the current hit, and will allow you to define friendly names for them ( it: have the event to hold "client_id" rather than "cd1"
- This library will also extract/parse all the e-commerce data from the payloads and generate a
standalone Ecommerce Eventwith the data, also including the custom dimensions/metrics for the products.
Supported Hit Types / Parameters
Next, you can find the list of supported hit types and all the data that will be shipped with the generated object push.
| Hit Type | Status | Default Parameters |
|---|---|---|
| pageview | Supported | page_title,page_location, page_path |
| event | Supported | category, action, label, value, nonInt |
| timing | Supported | timing_category, timing_value, timing_time, timing_label |
| social | Supported | social_action, social_network, social_target |
| exception | Supported | exception_description, exception_is_fatal |
| transaction | Supported | transaction_id, transaction_affiliation, transaction_revenue, transaction_shipping, transaction_tax, currency |
| item | Supported | transaction_id, item_id, item_name, item_price, item_quantity, item_variation. currency |
Ecommerce
As we mentioned before the Universal Analytics Payload Parser the library will also take care of parsing the Enhanced Ecommerce data contained within the hits and will generate the following standalone Ecommerce Pushes.
| Event | Parameters |
|---|---|
| view_item_list | [item_id, item_name, quantity, item_brand, item_variant, item_category, item_category2, item_category3, item_category4, item_category5, price, index,item_list_name] |
| select_item | [item_id, item_name, quantity, item_brand, item_variant, item_category, item_category2, item_category3, item_category4, item_category5, price, index, item_list_name] |
| view_item | [item_id, item_name, quantity, item_brand, item_variant, item_category, item_category2, item_category3, item_category4, item_category5, price] |
| add_to_cart | [item_id, item_name, quantity, item_brand, item_variant, item_category, item_category2, item_category3, item_category4, item_category5, price, quantity] |
| view_cart | [item_id, item_name, quantity, item_brand, item_variant, item_category, item_category2, item_category3, item_category4, item_category5, price, quantity] |
| being_checkout | [item_id, item_name, quantity, item_brand, item_variant, item_category, item_category2, item_category3, item_category4, item_category5, price, quantity] |
| remove_from_cart | [item_id, item_name, quantity, item_brand, item_variant, item_category, item_category2, item_category3, item_category4, item_category5, price, quantity] |
| purchase | [item_id, item_name, quantity, item_brand, item_variant, item_category, item_category2, item_category3, item_category4, item_category5, price, quantity], transaction_id, value, tax, shipping, affiliation, coupon |
| refund | [item_id, item_name, quantity, item_brand, item_variant, item_category, item_category2, item_category3, item_category4, item_category5, price, quantity], transaction_id |
| view_promotion | [promotion_id, promotion_name, creative_name, creative_slot] |
| select_promotion | [promotion_id, promotion_name, creative_name, creative_slot] |
Setup / Configuration
The library expects a configuration variable, like this one.
var parserConfig = {
tms: 'log',
eventsName: 'ga_event',
ecommerceEventsEnabled: true,
skipTransportEvent: false,
mapping: {
keys: {
cd1: 'client_id',
cd3: 'logged_in',
cd7: 'is_in_stock',
cd19: 'referrer',
cd20: 'full_url',
// Metrics
cm201: 'votes',
// content Grouping
cg1: 'content_group_1',
cg3: 'page_type'
},
events: {
'checkout': {
'1' : 'view_cart',
'2' : 'begin_checkout'
}
}
}
}| Option | |
|---|---|
| tms | gtm, tealiumiq, launch, log |
| eventsName | The event name to be used for the events. |
| ecommerceEventsEnabled | Whether or not to parse the e-commerce data and push it |
| skipTransportEvent | If we set this to true, the current hit/event will be skipped and only the EEC event will be generated |
| mapping.keys | Our dimensions, metrics, and content_grouping definition, this will convert the “cd1” payload key into something more readable like “client_id” in the pushes. |
| mapping.events.checkout | GA4 not longer accepts checkout_steps, this will let you define which current Checkout Steps should be considered the new “begin_checkout” and “view_cart” events |
Examples
pageview hit ( no dimensions )
v=1&_v=j96&a=609235607&t=pageview&_s=1&dl=https%3A%2F%2Fwww.thyngster.com%2F&ul=es-es&de=UTF-8&dt=Web%20Analyst%20and%20Sr.%20Implementation%20Consultant%20-%20David%20Vallejo&sd=24-bit&sr=1920x1080&vp=432x1009&je=0&fl=Service%20Provider%7CNetwork%20Domain&_u=QACAAAABAAAAAC~&jid=2095545038&gjid=1011316425&cid=353185870.1650295583&tid=UA-286304-9&_gid=1709590183.1650826121&_r=1>m=2wg4k09LNT&cd1=353185870.1650295583&z=637815381
Pageview Hit Payload
Generated Object Push
{
"event": "page_view",
"eventData": {
"page_title": "Web Analyst and Sr. Implementation Consultant - David Vallejo",
"page_location": "https://www.thyngster.com/"
}
}pageview hit ( with dimensions, metrics, content_groupings )
v=1&_v=j96&a=609235607&t=pageview&_s=1&dl=https%3A%2F%2Fwww.thyngster.com%2F&ul=es-es&de=UTF-8&dt=Web%20Analyst%20and%20Sr.%20Implementation%20Consultant%20-%20David%20Vallejo&sd=24-bit&sr=1920x1080&vp=432x1009&je=0&&cd1=123123123123.2321231232132&cd2=homepage&cg1=homepage&cm3=1&fl=Service%20Provider%7CNetwork%20Domain&_u=QACAAAABAAAAAC~&jid=2095545038&gjid=1011316425&cid=353185870.1650295583&tid=UA-286304-9&_gid=1709590183.1650826121&_r=1>m=2wg4k09LNT&cd1=353185870.1650295583&z=637815381
Generated Object Push
{
"event": "page_view",
"eventData": {
"client_id": "353185870.1650295583",
"dimension_2": "homepage",
"content_group_1": "homepage",
"metric_3": "1",
"page_title": "Web Analyst and Sr. Implementation Consultant - David Vallejo",
"page_location": "https://www.thyngster.com/"
}
}event hit ( with dimensions, metrics, content_groupings and add_to_cart e-commerce details )
v=1&_v=j96&a=136223288&t=event&cu=USD&_s=5&dl=https%3A%2F%2Fenhancedecommerce.appspot.com%2Fitem%2Fb55da&ul=es-es&de=UTF-8&dt=Product%20View&sd=24-bit&sr=1920x1080&vp=683x992&je=0&ec=ecommerce&ea=add_to_cart&ev=16&_u=SCCAAUALAAAAAC~&jid=1374636135&gjid=489822857&cid=625473581.1650821355&tid=UA-41425441-17&_gid=335165655.1650821355&_r=1>m=2ou4k0&pa=add&pr1id=b55da&pr1nm=Flexigen%20T-Shirt&pr1pr=16.00&pr1qt=1&pr1br=Flexigen&pr1ca=T-Shirts&pr1va=red&z=494832007
Generated Object Pushes
{
"event": "ga_event",
"eventData": {
"category": "ecommerce",
"action": "add_to_cart",
"value": "16",
"nonInt": true
}
}{
"event": "add_to_cart",
"items": [
{
"item_id": "b55da",
"item_name": "Flexigen T-Shirt",
"price": "16.00",
"quantity": "1",
"item_brand": "Flexigen",
"item_category": "T-Shirts",
"item_variant": "red"
}
]
}event - including purchase info
v=1&_v=j96&a=1751633849&t=event&cu=USD&_s=13&dl=https%3A%2F%2Fenhancedecommerce.appspot.com%2Fcheckout&ul=es-es&de=UTF-8&dt=Checkout&sd=24-bit&sr=1920x1080&vp=967x1009&je=0&ec=ecommerce&ea=purchase&ev=115&_u=SCCAAUALAAAAAC~&jid=&gjid=&cid=625473581.1650821355&tid=UA-41425441-17&_gid=335165655.1650821355>m=2ou4k0&pa=purchase&pr1id=b55da&pr1nm=Flexigen%20T-Shirt&pr1pr=16.00&pr1qt=3&pr1br=Flexigen&pr1ca=T-Shirts&pr1va=red&pr2id=8835a&pr2nm=Isoternia%20T-Shirt&pr2pr=57.00&pr2qt=1&pr2br=Isoternia&pr2ca=T-Shirts&pr2va=red&pr2ps=1&tr=115&tt=5.00&ts=5.00&z=1549501619
Generated Object Pushes
{
"event": "purchase",
"items": [
{
"item_id": "b55da",
"item_name": "Flexigen T-Shirt",
"price": "16.00",
"quantity": "3",
"item_brand": "Flexigen",
"item_category": "T-Shirts",
"item_variant": "red"
},
{
"item_id": "8835a",
"item_name": "Isoternia T-Shirt",
"price": "57.00",
"quantity": "1",
"item_brand": "Isoternia",
"item_category": "T-Shirts",
"item_variant": "red",
"index": "1"
}
],
"value": "115",
"tax": "5.00",
"shipping": "5.00"
}hit - impressions
v=1&_v=j96&a=1653659752&t=event&ni=1&_s=2&dl=https%3A%2F%2Fenhancedecommerce.appspot.com%2F&dr=https%3A%2F%2Fwww.google.com%2F&ul=es-es&de=UTF-8&dt=Home&sd=24-bit&sr=2560x1440&vp=570x1321&je=0&ec=engagement&ea=view_item_list&_u=SCCAAUAL~&jid=&gjid=&cid=625473581.1650821355&tid=UA-41425441-17&_gid=335165655.1650821355>m=2ou4k0&il1nm=homepage&il1pi1id=9bdd2&il1pi1nm=Compton%20T-Shirt&il1pi1pr=44.00&il1pi1br=Compton&il1pi1ca=T-Shirts&il1pi2id=f6be8&il1pi2nm=Comverges%20T-Shirt&il1pi2pr=33.00&il1pi2br=Comverges&il1pi2ca=T-Shirts&il1pi2ps=1&il1pi3id=b55da&il1pi3nm=Flexigen%20T-Shirt&il1pi3pr=16.00&il1pi3br=Flexigen&il1pi3ca=T-Shirts&il1pi3ps=2&il1pi4id=bc823&il1pi4nm=Fuelworks%20T-Shirt&il1pi4pr=92.00&il1pi4br=Fuelworks&il1pi4ca=T-Shirts&il1pi4ps=3&il1pi5id=035f0&il1pi5nm=Futuris%20T-Shirt&il1pi5pr=55.00&il1pi5br=Futuris&il1pi5ca=T-Shirts&il1pi5ps=4&il1pi6id=8835a&il1pi6nm=Isoternia%20T-Shirt&il1pi6pr=57.00&il1pi6br=Isoternia&il1pi6ca=T-Shirts&il1pi6ps=5&il1pi7id=57b9d&il1pi7nm=Kiosk%20T-Shirt&il1pi7pr=55.00&il1pi7br=Kiosk&il1pi7ca=T-Shirts&il1pi7ps=6&il1pi8id=dc646&il1pi8nm=Lunchpod%20T-Shirt&il1pi8pr=90.00&il1pi8br=Lunchpod&il1pi8ca=T-Shirts&il1pi8ps=7&il1pi9id=7w9e0&il1pi9nm=Masons%20T-Shirt&il1pi9pr=31.00&il1pi9br=Masons&il1pi9ca=T-Shirts&il1pi9ps=8&il1pi10id=239b5&il1pi10nm=Pigzart%20T-Shirt&il1pi10pr=82.00&il1pi10br=Pigzart&il1pi10ca=T-Shirts&il1pi10ps=9&il1pi11id=6d9b0&il1pi11nm=Poyo%20T-Shirt&il1pi11pr=62.00&il1pi11br=Poyo&il1pi11ca=T-Shirts&il1pi11ps=10&il1pi12id=6c3b0&il1pi12nm=Zappix%20T-Shirt&il1pi12pr=99.00&il1pi12br=Zappix&il1pi12ca=T-Shirts&il1pi12ps=11&il2nm=shirts%20you%20may%20like&il2pi1id=6c3b0&il2pi1nm=Zappix%20T-Shirt&il2pi1pr=99.00&il2pi1br=Zappix&il2pi1ca=T-Shirts&il2pi2id=8835a&il2pi2nm=Isoternia%20T-Shirt&il2pi2pr=57.00&il2pi2br=Isoternia&il2pi2ca=T-Shirts&il2pi2ps=1&il2pi3id=035f0&il2pi3nm=Futuris%20T-Shirt&il2pi3pr=55.00&il2pi3br=Futuris&il2pi3ca=T-Shirts&il2pi3ps=2&il2pi4id=239b5&il2pi4nm=Pigzart%20T-Shirt&il2pi4pr=82.00&il2pi4br=Pigzart&il2pi4ca=T-Shirts&il2pi4ps=3&z=819126443
Generated Object Pushes
{
"event": "ga_event",
"eventData": {
"category": "engagement",
"action": "view_item_list",
"nonInt": false
}
}{
"event": "view_item_list",
"items": [
{
"item_list_name": "shirts you may like",
"item_id": "6c3b0",
"item_name": "Zappix T-Shirt",
"price": "99.00",
"item_brand": "Zappix",
"item_category": "T-Shirts"
},
{
"item_list_name": "shirts you may like",
"item_id": "8835a",
"item_name": "Isoternia T-Shirt",
"price": "57.00",
"item_brand": "Isoternia",
"item_category": "T-Shirts",
"index": "1"
},
{
"item_list_name": "shirts you may like",
"item_id": "035f0",
"item_name": "Futuris T-Shirt",
"price": "55.00",
"item_brand": "Futuris",
"item_category": "T-Shirts",
"index": "2"
},
{
"item_list_name": "shirts you may like",
"item_id": "239b5",
"item_name": "Pigzart T-Shirt",
"price": "82.00",
"item_brand": "Pigzart",
"item_category": "T-Shirts",
"index": "3"
},
{
"item_list_name": "homepage",
"item_id": "035f0",
"item_name": "Futuris T-Shirt",
"price": "55.00",
"item_brand": "Futuris",
"item_category": "T-Shirts",
"index": "4"
},
{
"item_list_name": "homepage",
"item_id": "8835a",
"item_name": "Isoternia T-Shirt",
"price": "57.00",
"item_brand": "Isoternia",
"item_category": "T-Shirts",
"index": "5"
},
{
"item_list_name": "homepage",
"item_id": "57b9d",
"item_name": "Kiosk T-Shirt",
"price": "55.00",
"item_brand": "Kiosk",
"item_category": "T-Shirts",
"index": "6"
},
{
"item_list_name": "homepage",
"item_id": "dc646",
"item_name": "Lunchpod T-Shirt",
"price": "90.00",
"item_brand": "Lunchpod",
"item_category": "T-Shirts",
"index": "7"
},
{
"item_list_name": "homepage",
"item_id": "7w9e0",
"item_name": "Masons T-Shirt",
"price": "31.00",
"item_brand": "Masons",
"item_category": "T-Shirts",
"index": "8"
},
{
"item_list_name": "homepage",
"item_id": "239b5",
"item_name": "Pigzart T-Shirt",
"price": "82.00",
"item_brand": "Pigzart",
"item_category": "T-Shirts",
"index": "9"
},
{
"item_list_name": "homepage",
"item_id": "6d9b0",
"item_name": "Poyo T-Shirt",
"price": "62.00",
"item_brand": "Poyo",
"item_category": "T-Shirts",
"index": "10"
},
{
"item_list_name": "homepage",
"item_id": "6c3b0",
"item_name": "Zappix T-Shirt",
"price": "99.00",
"item_brand": "Zappix",
"item_category": "T-Shirts",
"index": "11"
}
]
}CustomTask Setup
As usual, we will need to set up a customTask variable for our tags. I'm attaching an example for Google Tag Manager
function() {
return function(customTaskModel) {
var originalSendHitTask = customTaskModel.get('sendHitTask');
customTaskModel.set('sendHitTask', function(model) {
try {
var parserConfig = {
tms: 'gtm',
eventsName: 'ga_event',
ecommerceEventsEnabled: true,
skipTransportEvent: false,
mapping: {
keys: {
cd1: 'client_id',
cd3: 'logged_in',
cd7: 'is_in_stock',
cd19: 'referrer',
cd20: 'full_url',
// Metrics
cm201: 'votes',
// content Grouping
cg1: 'content_group_1',
cg3: 'page_type'
},
events: {
'checkout': {
'1': 'view_cart',
'2': 'begin_checkout'
}
}
}
};
[[ADD CODE FROM GITHUB]]
uaPayloadParser(parserConfig, model.get('hitPayload'));
originalSendHitTask(model);
} catch (e) {
originalSendHitTask(model);
}
});
}
}In order to have updated content, please replace the [[ADD CODE FROM GITHUB]] with the code from the build/ folder in the GitHub project.
https://github.com/thyngster/universal-analytics-payload-parser/blob/main/build/uaPayloadParser.min.js
Recap
So, that's all, as you can see rather than sending data to Google Analytics 4, this library allows sending the data to the dataLayer ( or anywhere else ) instead so you can work with the data in any way you want, you could define different event names, you could sanitize the values, you can block/skip the events you don't need.
The best point is that you'll have full control over how the data ends in your Google Analytics 4 account, and this will work even if you're using GTAG, or even if you have a hardcoded implementation, just adding the custom task in your main snippet should allow you to have the data anywhere else.
GITHUB PROJECT LINK: https://github.com/thyngster/universal-analytics-payload-parser