Tracking downloads in Sitecore Experience Analytics

This blog is generally aimed at developers whereas the contents of this post could be categorized as a topic for marketers, but I’ve decided to include it as its likely something a Sitecore dev could get asked about and its also quite useful to know about.

Out the box Sitecore’s Experience Analytics comes with a set of pre-configured reports to give insights into the sites visitors. Previously I blogged about Populating the internal search report in Sitecore which unless done will probably lead to someone asking why it’s blank. Another report which initially won’t show any data is downloads.

AnalyticsBehaviourTab

Located under behavior there is actually now two reports relating to downloads. The Assets report and the Downloads report.

Assets, in Sitecores words – “Describes your marketing assets or content used to attract contacts to your website and increase their engagement with your organization.”

Downloads – “Describes your specific assets, their download activity, and their value.”

These reports are populated by assigning a download event to a piece of media and organizing it as a marketing asset.

Adding a download event to an item

Sitecore doesn’t know which items you think are important to track as downloads so content editors need to mark them manually.

  1. Go to the Media Library

    Media Library

  2. Select the item you want to track as a download

    Select Item

  3. From the ribbon select the Analyze tab and click Attributes

    Click Attributes

  4. Select download form the list of events

    Select Download Attribute

  5. Remember to publish your changes.

Categorizing an Asset

By assigning a marketing asset type to your items they can be grouped for analysis. Such as grouping downloads into categories like white paper and product brochure.

  1. Before you assign a marketing asset to an item you will first need to create your assets. On the Sitecore Launchpad, open the Marketing Control Panel.

    Marketing Control Panel

  2. Go to Assets which is located under Taxonomies.

    Asset

  3. Create your set of Asset Groups and Assets within each. In this example I’ve created a group called Content and Assets called Instruction Manual and Product Brochure.

    Assets

  4. Navigate to the item you want to assign the asset to and select the relevant asset in the marketing asset field.

    Marketing Asset on Item

  5. Publish all your changes.

The result

Now you’ve set a download event and a marketing asset, the assets and downloads reports will start populating.

Going Further

The concept of having to tag every download as a download may seem a little tedious and also prone to being missed in the future. If all your downloads are PDF’s and you want to track all PDF’s as a download, one way to make life easier is to update the standard values on the PDF template item (/sitecore/templates/System/Media/Unversioned/Pdf) so that it always has the attribute of download. The content authors will still need to do the marketing asset categorization, but at least this gives them 1 less thing to do.

Advertisements

Populating the internal search report in Sitecore

Out the box Sitecore ships with a number of reports pre-configured. Some of these will show data without you doing anything. e.g. The pages report will automatically start showing the top entry and exit pages as a page view is something Sitecore can track.

Other’s like the internal search report will just show a message of no data to display, which can be confusing/frustrating for your users. Particularly when they’ve just spent money on a license fee to get great analytics data only to see a blank report.

Internal Search Report

The reason it doesn’t show any information is relatively straight forward. Sitecore doesn’t know how your site search is going to work and therefore it can’t do the data capture part of the process. That part of the process however is actually quite simple to do.

Sitecore has a set of page events that can be registered in the analytics tracker. Some of these like Page Visited will be handled by Sitecore. In this instance the one we are interested in is Search and will we have to register it manually.

Search Event

To register the search event use some code like this (note, there is a constant that references the item id of the search event). The query parameter should be populated with the search term the user entered.

using Sitecore.Analytics;
using Sitecore.Analytics.Data;
using Sitecore.Data.Items;
using Sitecore.Diagnostics;
using SitecoreItemIds;

namespace SitecoreServices
{
    public class SiteSearch
    {
        public static void TrackSiteSearch(Item pageEventItem, string query)
        {
            Assert.ArgumentNotNull(pageEventItem, nameof(pageEventItem));
            Assert.IsNotNull(pageEventItem, $"Cannot find page event: {pageEventItem}");

            if (Tracker.IsActive)
            {
                var pageEventData = new PageEventData("Search", ContentItemIds.Search)
                {
                    ItemId = pageEventItem.ID.ToGuid(),
                    Data = query,
                    DataKey = query,
                    Text = query
                };
                var interaction = Tracker.Current.Session.Interaction;
                if (interaction != null)
                {
                    interaction.CurrentPage.Register(pageEventData);
                }
            }
        }
    }
}

Now after triggering the code to be called a few times, your internal search report should start to be populated like this.

Internal Search Report Populated

Sitecore xDB – How to get count of interactions

If you need to rebuild your Path Analyzer maps, Sitecore have a knowledge base article on how to do it here – https://kb.sitecore.net/articles/766858

In the FAQ section it gives some info on how long you should expect it to take

Depending on the processing power of your server, your XDB configuration, this could take from minutes to hours. Most of the time is spent on bringing over and de-serializing XDB interaction data on the Processing Server.

For the reference, it takes approximately 2 hours on XDB with 7M interactions, running on a single server.

What it doesn’t tell you though is how you find out how many interactions your site has to process. The data is stored in analytics collection in your Mongo DB, so go there an then run…

db.getCollection('Interactions').find({}).count()

Sitecore: Setting up Mongo with Authentication

One of the new features in Sitecore 7.5 was the xDB, a new analytics database that rather than using SQL Server was Mongo based. If your main job is as a Sitecore dev, like me this may be the first time you’ve ever had to use Mongo, or a least the first time you’ve ever had to commercially use it.

Installing Mongo DB is a relatively painless experience, Mongo provide a decent enough guide to installing on Windows. Half way through you may be wondering why its running in a console window, but eventually you get to instructions for setting it up as a service.

However there are a couple of gotchas:

  1. Sitecore doesn’t work with the latest version of Mongo. At time of writing the latest version of Sitecore is 8.0 rev. 150427 and Mongo is 3.0. Sitecore however only supports version 2.6.x of Mongo though.
  2. None of the installation instructions include any details on authentication. Which while this is fine for your local dev machine is not so great for a live site.

So here’s my guide for getting up and running with Mongo DB and Sitecore.

Installing Mongo

First off get Mongo DB installed as a service on your machine. I’m not going to include instructions here for this bit, just follow the guide on the Mongo DB website.

One thing to note though, the Windows installer for Mongo 2.6 will by default install Mongo to C:\Program Files\MongoDB 2.6 Standard\. However the instructions for setting up Mongo include example commands expecting it to be installed in C:\mongodb\

Setting Up Authentication

Once you’ve got Mongo up and running you’ll need to create a user.

  1. Open a command prompt and connect to MongoDB
    C:\Program Files\MongoDB 2.6 Standard\bin\mongo.exe
    
  2. Switch to the admin db by doing
    use admin
    
  3. Create the user using
    db.createUser({user: "admin_mongo",pwd: "your password",roles: [ { role: "userAdminAnyDatabase", db:"admin" }, { role: "root", db:"admin" } ]  })
    
  4. To verify the user has been created you can use the following commands
    db.auth("admin_mongo", "your password")
    db.getUsers()
    

Mongo Create Admin User

Next update your config file to require authentication by adding auth=true. You will also need to restart the service for the change to take effect.

logpath=c:\data\log\mongod.log
dbpath=c:\data\db
auth=true

Sitecore uses 4 db’s in Mongo; Analytics, tracking_live, tracking_history and tracking_contact. We need to create a user in each of them.

Open a mongo shell again, switch to the admin db and connect with your admin login.

Now create each of the users as follows:

use analytics
db.createUser({user: "mongo_user",pwd: "your password",roles: [ { role: "readWrite", db:"analytics" } ]  })

use tracking_live
db.createUser({user: "mongo_user",pwd: "your password",roles: [ { role: "readWrite", db:"tracking_live" } ]  })

use tracking_history
db.createUser({user: "mongo_user",pwd: "your password",roles: [ { role: "readWrite", db:"tracking_history" } ]  })

use tracking_contact
db.createUser({user: "mongo_user",pwd: "your password",roles: [ { role: "readWrite", db:"tracking_contact" } ]  })

Sitecore Connection Strings

All that’s left now is to update the connection strings in Sitecore.

The default connection strings Sitecore give you look like this:

  
  
  
  

The format for a connection string with authentication is

mongodb://[username:password@]host1[:port1]/database

Your connection string should now look something like

  
  
  
  

Sitecore: How to change the personalisation trigger page count

One of Sitecores features is the ability to assign profile cards or profile key scores to pages. As visitors browse the website the profile key scores from each page are added to the visitors profile. Visitors are then assigned a pattern card using N-Dimonsional Euclidean Distance to calculate the closest matching pattern card to the visitors profile.

However before a pattern card is assigned to a visitor they must visit a minimum number of pages. The default for this is set to 3.

Reasons for changing this could be that you actually want to be more certain of a users profile before triggering personalisation this could be to avoid visitors landing pages on the site heavily dictating which persona they initially become profiled as, which in turn could influence personalisation that keeps them in that persona.

Or it could be because you want to lower the number of pages for a more immediate effect.

Whatever the reason to change the default from 3 you need to add the following setting to your config:

 <setting name="Analytics.Patterns.MinimalProfileScoreCount" value="3"/>

Sitecore: Extend profile matching over multiple visits

In Sitecore, to gain a better understanding of our visitors interests we have the ability to define Profile Keys and Cards to tag our content with. As our visitors navigate through the site, this data is used by Sitecore to build a profile of the visitor. A pre-defined Pattern Card that most resembles the visitors profile is then assigned to the visitor which can be used as the basis of selecting the content that should be displayed on a page for that visitor.

However what this doesn’t do is carry the visitors profile over multiple sessions. Each time a visitor comes back to the site within a new session, the visitors profile key values are reset back to zero.

So what’s Sitecore actually doing?

Before working out how to carry this information between visits, lets look at how a profile is actually being created.

If we look in the Profiles table within the Analytics database we can see the profile data that’s been recorded for a visitors visit.

Sitecore profile data

The Pattern Values column contains the current profile key scores for each key the visitor has a score for. e.g.

background=40;scope=50

If the visitor was to visit a page which has scope score of 5 and background score of 10 these values would be added to the visitors current key scores. e.g.

background=50;scope=55

When a pattern card is assigned, the card with the closest shape of keys is chosen. e.g. If the visitor has a high value for background and low value for scope they will be assigned a pattern card with similar proportional key values.

How do we extend this over multiple visits?

So the easiest way to carry the visit information from one visit to the next would be to simply copy the profile key values from the last session to the next. The code for this would look similar to the following:

var currentVisitIndex = Tracker.CurrentVisit.VisitorVisitIndex;
 
if (currentVisitIndex <= 1 || !Tracker.CurrentVisit.Profiles.Any())
{
    return;
}

var previousProfiles = Tracker.Visitor.GetVisit(currentVisitIndex - 1, VisitLoadOptions.All).Profiles;

foreach (var profile in previousProfiles)
{
    var currentProfile = Tracker.CurrentVisit.GetOrCreateProfile(profile.ProfileName);

    currentProfile.BeginEdit();

    foreach (var ProfileKey in profile.Values)
    {
        currentProfile.Score(ProfileKey.Key, ProfileKey.Value);
    }
    currentProfile.UpdatePattern();

    currentProfile.EndEdit();
}

Now the visitors profile is how it was when they left and crucially we can use this data to personalize the sites homepage for the visitor.

So why shouldn’t we do this?

As simple as this is, it comes with one potentially massive downside. If we go back to the way the profile values are built up they key values are essentially just being accumulated. Each time the visitor visits an item with a background score of 10, the visitors background profile key score in increased by 10.

Our visitors are humans going through different stages of there life, with constantly changing jobs and interests. There’s nothing to ever reduce a profile keys score other than the fact everything is normally zeroed on each visit. By copying the data from the last visit on the start of the next this would never happen and the profile key’s will continue to count up forever. The key value obtained from an item viewed 2 months ago would counted as just as important as the value from another key viewed on an item today.

So if you were running a travel site and a visitor looked at summer holidays for 3 weeks they will have a profile highly weighted towards summer holidays. If they then started to look at winter holidays we wouldn’t want them to have to look at winter holidays for 3 weeks just to have an even likeness of summer and winter.

Overcoming this issue isn’t so simple and largely depends on your business needs. If your visitors interests could change each week then you need something that will degrade the old visit data values quickly. Whereas if your trying to differentiate between people that are in a 2 week vs 6 month buying pattern, you need to retain that data a lot longer.

Some things we can do when copying the data from the visitors previous profile though could include:

  • Halving the profile scores, or reducing by a different factor. This would reduce the importance of values obtained on previous visits. So if a visitor received a 10 on the first visit, it would be worth 5 on the second, 2.5 on the third etc
  • Look at the date of the last visit. Is it to old to be relevant still or can we use the age to determine what factor we should reduce the scores by
  • Look at a combination of multiple last visits to establish what the recent scores were

All these ideas though need to be used on conjunction with what your trying to profile. If it’s age then you know people are going to get older. If it’s an interest that will change frequently then you know the data needs to degrade quickly, but if it’s male/female then that doesn’t necesserally need to degrade at all.

Pragmatically add request tracking for an item in Sitecore

Sitecore’s Engagement Analytic’s engine automatically tracks all page requests. When you assign profile cards to items this also triggers data about a persons interests to start being built up against their visit record, which can then be used to create a personalized site experience.

However what if you want items that are not pages to also contribute to a users profile?

This scenario came about with a recent site we took over that the client wanted to add personalisation too. The site (for unknown reasons) had been built with one product page which looked at a querystring parameter to determine the product information to be displayed (the querystring format was hidden from end users through the use of a URL rewrite). Product data was being stored as Sitecore items allowing them to have profile cards assigned, but as the item was never visited the profile cards values were never applied to the users profile.

After a bit of searching through the Sitecore.Analytics.dll I stumbled across the TrackingFieldProcessor class. This class contains a process function that takes an item parameter an in turn triggers all the functionality for processing campaigns, profiles and events related to the item.

To use it your code would look like this:

Sitecore.Data.Database db = Sitecore.Configuration.Factory.GetDatabase("web");
(new TrackingFieldProcessor()).Process(db.GetItem(new ID("395BDEF7-16CB-4C94-B9B6-A6EAC148401F")));

This will cause the profile key values to be updated but in the visitor history it still looks like the visitor was only looking at the one page. To change those values we can do this:

VisitorDataSet.PagesRow rawUrl = Tracker.CurrentVisit.GetOrCreateCurrentPage();
rawUrl.Url = "new url value";
rawUrl.UrlText = "new url value";

Note: I was unable to find any documentation on these functions, or any official way of doing this. Use at your own risk!