Technical overview of new APP+WEB Properties

Some days ago, Google announced a new upcoming property type for Google Analytics, they called them "APP + WEB". And yes, the name says it all, finally we're going to be able to measure our apps and websites at the same time. This has been for years one of the biggest handicaps of Google Analytics, and not only that, when Firebase came into the scene, we had ended having two platforms to measure apps, double the SDK's and despite having all this tools we were not able to measure users accross platforms properly.

But Google finally took the right step and they're going to merge everything within the same platform, and also they are moving away from a "session" based tracking model to a "events" based tracking model. Yes, you've heard it right, sessions as we do know them now will be gone ( don't worry we'll have sessions anyway, continue reading ..)

This post won't be covering the basic details about this new properties details, how the reports work since that's all is covered by other authors in the blogs: New App + Web Properties in Google Analytics by Krista Seiden or A new way to unify app and website measurement in Google Analytics on Google Blog , or a google all-in introduction by Simo Ahava HERE .

The star feature of the new properties is that they offer a way to unify our measurement strategy across different devices in one single place. Yay! ,
On this post we'll be learning how the new properties work, from a technical/implementation point of view and which are the most important changes compared to the current Google Analytics tracking.

The main change on the tracking side and that from now on ALL is an "event" . The new platform is based on GA4F ( Google Analytics for Firebase ) , so we need to define everything ourselves.

We need to start sending events and then based on these we'll build the reports. In any case Google is including a default web tracking model within the APP+WEB properties which will allow us to have the most web relevant events in place and will be also offering some in-built parameters reporting ( same way we previously had dimensions and metrics based on events, returning visitors, etc ).

Events

event: {
        param1,
        param2,
        param3,
        paramN,
        user_parameters: {
         	         userParam1,
                         userParam2,
                         userParamN
        }
 }

In the Schema above we can see the kind of data we can send along with an event to a APP+WEB property. Note that there's a reserver parameter key called "user_parameters" that will hold and object with a list of the parameters related to the user. ( By the way we'll need to define these in the UI first ).
As you may noticied, there's no longer any custom dimensions or metrics, so there are not cdX or cdN keys on our hits. Now we have Event Parameters and User Parameters instead.

Measurement Protocol v2

Along with the new APP+WEB properties, Google released a new protocol ( it's more a v1 protocol version improvement rather than a new version ).

If you usually work on debugging Google Analytics implementations you'll be recognize most of th e parameters that you'll see in the hits

As an example, check the image below for how a simple page_view event hit looks like. I'm pretty sure you'll feel familiar with most of keys.

New Parameters List

I tried to document the most import new keys that are part of the new Measurement Protocol. Here is a list of them:

en	Event Name
_fv	First Visit
_ss	Session Start
sid	Session ID
seg	Session Engaged
sct	Session Count
_p	Cache Buster?
_et	Event Time ( time since start for this event )
ep.[a-z]	Event Paramter ( String )
epn.[a-z]	Event Parameter ( Number )
up.[a-z]	User Parameter ( String )
upn.[a-z]	User Parameter ( Number )

DISCLAIMER: APP+WEB is currently in beta and it's likely adding new parameters in the upcoming months to cover all the current Google Analytics functionality / data points.

Transport details

Actually in Universal Analytics, the hits may use 3 different transport types:

- new Image() ( used for GET hits )
- XMLHttpRequest ( used for POST hits )
- sendBeacon ( Optional transport )

It looks like the new library just uses sendBeacon for sending the hits, which looks like a good choice looking to the current functionality adoption by browsers:

We also can find a new endpoint: Now the hits will be send to the following URL ( note the "g" )

https://www.google-analytics.com/g/collect

So, you said that we'll have to look at the Request Payload in order to get the current hits data, is this correct?

Not at all, what we said is that now hits are sent using the POST method (sendBeacon uses POST method to work). But still we can POST data to an URL that have a querystring.

This means that on the new measure protocol the data is sent to the endpoint using and hybrid approach, a mix of a QueryString and Request Payload are used for sending the data. Some data is sent in the Query String and some details "may" be sent in the Request Payload ( POST Data ). In overall common hit details like the clientId, the language, the screen resolution ( all this data will be shared by all the events ) will be sent using the Query String, and the events details will be send in the Request Payload. Anyway we'll see that this won't be this way in all cases, since there will be differences on how the hit payload it send depending if the hit has one event, or more than just one.

Continue reading to learn how to identify the different hit formats you'll find.

Multi Event Hits

In the newly released protocol, we can send multiple events within one single hit request ( beware this is not the same as the previous "batches" in Universal Analytics ).

This is a nice feature because it will help us reducing the number of requests that our site will be loading.

But, wait a second...

In the previous table you showed there was only one slot key for the event name, how is it supposed to work if we can send more than 1 event?.

We already learnt that the new protocol uses a mix of QueryString and Request Payload in order to send the data to the endpoint. And we also briefly mentioned that depending on the number of events travelling within the hit the behaviour could be differents.

Let's take a deeper look to how it works:

So, events details may go into the Query String or within the Post Data, depending on the events count in the hit. This helps on reducing the number of request send to Google Analytics servers and also helps reducing the payload size, since the details shared by the events are only send once. Double Win!.

Events Queing

This is not directly related to the measurement protocol itself, but it's one of the first things that I saw when I started playing around with APP+WEB and it made me think I was doing something wrong.

You may have noticed that when you push an event using the official GTAG API the data its not flowing to the endpoint as soon as you push it, There's a small delay between the time you push the event and the hit shows up in the network tab. DON'T WORRY, this is normal, the new GTAG library, queues the events and dispatch them altogether each 5 seconds ( +/- ), remember that the new protocol allowed to send multiple events on each hit?, this take advantage of that feature.

Debugging

With all the previous details, you should be able to understand and read most of the details from the hit requests, but you can make your life easier using some extension that will parse the requests details properly and show you the details in a more friendly way.

If you want you can take a look to my GTM/GA Debug Extension which actually support GA4F ( Google Analytics for Firebase ) hits.

Cookies and Sessions

The APP+WEB properties use not 1 but 2 cookies in order to perform the tracking on a website. The first one is an old friend of all us:

"_ga" cookie, nothing changed on it, it will be taking care of holding the current user ClientID.

"_ga_XXXXXX" cookies We'll be find a new cookie for each of the Measurement ID's we're using on our site. Meaning that if we have a dual or triple tracking setup we'll have the same amount of cookies, since they keep they own namespace and cookie.

Let's review all the technical details about this new cookie

Cookie Name

_ga_MEASUREMENT_ID

Cookie Domain and Path

The cookie is set to the nearest root domain name and it uses '/' path by default

Cookie Duration

2 years

Cookie USe and Format

Do you remember back in Urchin or legacy analytics (ga.js) days when the session data was calculated via user cookies?, Then we switched to Universal Analytics and everything was being calculated server-side on Google side.

Now with the new APP+WEB properties we're back to the roots ( almost ) since it uses a similar approach as the first Gooogle Analytics versions ( remember the UTMA and UTMB cookies? ), but with one small difference, the visits attribution will be still calculated on Google side ( no UTMZ cookie ). So that's the main use for this cookie, keeping track of total sessions and current session state.

Check the following image for a full cookie definition ( I have not found the use of the last cookie value yet )

WEB MODEL

Google is offering a pre-built reports and events model for websites tracking. So we don't need to start from scratch. Therefore we'll be able to send page_views, and APP+WEB will take care of collections some extra data for these events without us needing to define any extra parameters ( like the page_titles, sources, mediums, browser details, geographical details ,etc )

If you take a look to any APP+WEB Property you'll notice that you have at least 4 different events, despite you may be only sending a "page_view" event:

I'm sure that you may be thinking, What kind of magic is this?, I didn't setup any 'user_engagement' , 'session_start' or "first_visit" events on my setup, just the "page_view" one.

The same way that APP+WEB has some in-built parameters for the "page_view" events like the source, medium, title, browser, etc . Google will use generate some auto-events based on some parameters from the hits.

user_engagement	From hits including the key "*seg*" in the payload.
session_start	From hits including the key "*_ss*" in the payload.
first_visit	From hits including the key "*_fv*" in the payload.

HEY WAIT! Remember that almost everyone around is mentioning that sessions won't be longer available on GA, that's not a 100% true statement, it's true that we're moving to an event-tracked model instead of a sessions-based mode, but that doesn't necesarilly mean that we won't have sessions count at all. Now we'll have some events to calculate our old sessions metrics along with some other ones. Based on the the current auto-events on App+Web, we'll have:

user_engagement	Can be used to calculate the bounce rate
session_start	Can be used to calculate the total sessions
first_visit	Can be used to calculate new VS returning visitors

Enhanced Measurement

The "page_view" is the core event for web tracking, this didn't change on this new version, but APP+WEB offers something called "Enhanced Measurement" which will help us setting some tracking in our sites without even needing to do anything else but turning on a checkbox.

Youtube Video Tracking
Outbound clicks ( Click Listeners )
File Downloads ( Click Listeners )
Scroll Tracking ( 90% event )
Pageviews ( on page load or history changes )
Site Search Events