A simple registration attribution tracking solution for Laravel (UTM Parameters and Referrers)

Overview

🐾 Footprints for (UTM and Referrer Tracking)

Footprints for Laravel (UTM and Referrer Tracking)

Latest Version on Packagist Software License Build Status Total Downloads

Footprints is a simple registration attribution tracking solution for Laravel 7+

β€œI know I waste half of my advertising dollars...I just wish I knew which half.” ~ Henry Procter.

By tracking where user signups (or any other kind of registrations) originate from you can ensure that your marketing efforts are more focused.

Footprints makes it easy to look back and see what lead to a user signing up.

Install

Via Composer

$ composer require kyranb/footprints

Publish the config and migration files:

php artisan vendor:publish --provider="Kyranb\Footprints\FootprintsServiceProvider"

Add the \Kyranb\Footprints\Middleware\CaptureAttributionDataMiddleware::class either to a group of routes that should be tracked or as a global middleware in App\Http\Kernel.php (after the EncryptCookie middleware!) like so:

    /**
     * The application's global HTTP middleware stack.
     *
     * These middleware are run during every request to your application.
     *
     * @var array
     */
    protected $middleware = [
        \Illuminate\Foundation\Http\Middleware\CheckForMaintenanceMode::class,
        \App\Http\Middleware\EncryptCookies::class,
        \Kyranb\Footprints\Middleware\CaptureAttributionDataMiddleware::class, // <-- Added
    ];

Add tracking to the model where registration should be tracked (usually the Eloquent model \App\Models\User) by implementing the TrackableInterface and using the TrackRegistrationAttribution trait like so:

namespace App\Models;

use Illuminate\Auth\Authenticatable;
use Illuminate\Database\Eloquent\Model;
use Kyranb\Footprints\TrackableInterface;
use Kyranb\Footprints\TrackRegistrationAttribution;

class User extends Model implements TrackableInterface // <-- Added
{
    use Authenticatable;
    use TrackRegistrationAttribution; // <-- Added 

    /**
     * The database table used by the model.
     *
     * @var string
     */
    protected $table = 'users';

}

Configuring

Go over the configuration file, most notably the model you wish to track:

connection name (optional - if you need a separated tracking database):

'connection_name' => 'mytrackingdbconnection'

model name:

'model' => 'App\Models\User'

authentication guard:

'guard' => 'web'

the column name:

'model_column_name' => 'user_id'

and attribution duration (in seconds)

'attribution_duration' => 2628000

also you can define some route what you don't want to track:

'landing_page_blacklist' => ['genealabs/laravel-caffeine/drip', 'admin']

if you want to use on multiple subdomain with a wildcard cookie, you can set your custom domain name:

'cookie_domain' => .yourdomain.com

this boolean will allow you to write the tracking data to the db in your queue (optional):

'async' => true

tracking in cases where cookies are disabled can be achieved by disabling the setting:

'uniqueness' => false

Usage

How does Footprints work?

Footprints tracks the UTM parameters and HTTP refererers from all requests to your application that are sent by un-authenticated users. Not sure what UTM parameters are? Wikipedia has you covered:

UTM parameters (UTM) is a shortcut for Urchin Traffic Monitor. This text tags allow users to track and analyze traffic sources in analytical tools (f.e. Google Analytics). By adding UTM parameters to URLs, you can identify the source and campaigns that send traffic to your website. When a user clicks a referral link / ad or banner, these parameters are sent to Google Analytics (or other analytical tool), so you can see the effectiveness of each campaign in your reports

Here is example of UTM parameters in a URL: www.wikipedia.org/?utm_source=domain.com&utm_medium=banner&utm_campaign=winter15&utm_content=blue_ad&utm_term=headline_v1

There are 5 dimensions of UTM parameters:
  • utm_source = name of the source (usually the domain of source website)

  • utm_medium = name of the medium; type of traffic (f.e. cpc = paid search, organic = organic search; referral = link from another website etc.)

  • utm_campaign = name of the campaign, f.e. name of the campaign in Google AdWords, date of your e-mail campaign, etc.

  • utm_content = to distinguish different parts of one campaign; f.e. name of AdGroup in Google AdWords (with auto-tagging you will see the headline of - your ads in this dimension)

  • utm_term = to distinguish different parts of one content; f.e.keyword in Google AdWords

And how is it logged?
  • CaptureAttributionDataMiddleware: Only routes using this middleware can be tracked
  • TrackingFilter: Used to determine whether or not a request should be logged
  • TrackingLogger: Doest the actual logging of requests to an Eloquent Visit model
  • Footprinter: Does the "linking" of requests using cookies or if configured falls back to using ip and the User-agent header
  • TrackRegistrationAttributes: Is used on the Eloquent model that we wish to track registration of (usually the User model)

For a more technical explanation of the flow, please consult the section [Tracking process in details](#Tracking process in details) below.

What data is tracked for each visit?

The default configuration tracks the most relevant information

  • landing_page
  • referrer_url
  • referrer_domain
  • utm_source
  • utm_campaign
  • utm_medium
  • utm_term
  • utm_content
  • created_at (date of visit)

But the package also makes it easy to the users ip address or basically any information available from the request object.

Get all of a user's visits before registering.
$user = User::find(1);
$user->visits;
Get the attribution data of a user's initial visit before registering.
$user = User::find(1);
$user->initialAttributionData();
Get the attribution data of a user's final visit before registering.
$user = User::find(1);
$user->finalAttributionData();
Events

The TrackingLogger emits an event RegistrationTracked once a registration has been processed while it is possible to listen for any visits tracked by simply listening for the Eloquent Events on the Visit model.

Tracking process in details

First off the CaptureAttributionDataMiddleware can be registered globally or on a selected list of routes.

Whenever an incoming request passes through the CaptureAttributionDataMiddleware middleware then it checks whether or not the request should be tracked using the class TrackingFilter (can be changed to any class implementing the TrackingFilterInterface) and if the request should be logged TrackingLogger will do so (can be changed to any class implementing TrackingLoggerInterface).

The TrackingLogger is responsible for logging relevant information about the request as a Vist record. The most important parameter is the request's "footprint" which is the entity that should be the same for multiple requests performed by the same user and hence this is what is used to link different requests.

Calculating the footprint is done with a request macro which in turn uses a Footprinter singleton (can be changed to any class implementing FootprinterInterface). It will look for the presence of a footprints cookie (configurable) and use that if it exists. If the cookie does not exist then it will create it so that it can be tracked on subsequent requests. It might be desireable for some to implement a custom logic for this but note that it is important that the calculation is a pure function meaning that calling this method multiple times with the same request as input should always yield the same result.

At some point the user signs up (or any trackable model is created) which fires the job AssignPreviousVisits. This job calculates the footprint of the request and looks for any existing logged Visit records and link those to the new user.

Keeping the footprints table light

Prune the table

Without pruning, the visits table can accumulate records very quickly. To mitigate this, you should schedule the footprints:prune Artisan command to run daily:

$schedule->command('footprints:prune')->daily();

By default, all entries unassigned to a user older than the duration you set on the config file with attribution_duration . You may use the days option when calling the command to determine how long to retain Footprint data. For example, the following command will delete all records created over 10 days ago:

$schedule->command('footprints:prune --days=10')->daily();

=======

Disable robots tracking

Before disabling robots tracking, you will need to install jaybizzle/crawler-detect. To do so : composer require jaybizzle/crawler-detect

Your table can get pretty big fast, mostly because of robots (Google, Bing, etc.). To disable robots tracking, change your footprints.php file on config folder accordingly :

'disable_robots_tracking' => true 

Upgrading

2.x => 3.x

Version 3.x of this package contains a few breaking changes that must be addressed if upgrading from earlier versions.

  • Rename the cookie_token column to footprint, in the table configured in config('footprints.table_name')
  • Add field ip' as a nullable string to the configured footprints table
  • Implement TrackableInterface on any models where the tracking should be tracked (usually the Eloquent model User)
  • (optional | recommended) Publish the updated configuration file: php artisan vendor:publish --provider="Kyranb\Footprints\FootprintsServiceProvider" --tag=config --force
  • If any modifications have been made to TrackRegistrationAttribution please consult the updated version to ensure proper compatability

Change log

Please see the commit history for more information what has changed recently.

Testing

Haven't got round to this yet - PR's welcome ;)

$ composer test

Contributing

If you run into any issues, have suggestions or would like to expand this packages functionality, please open an issue or a pull request :)

Thanks

Thanks to ZenCast, some of the best Podcast Hosting around.

License

The MIT License (MIT). Please see License File for more information.

Comments
  • Class Kyranb\Footprints\Middleware\CaptureAttributionDataMiddleware does not exist

    Class Kyranb\Footprints\Middleware\CaptureAttributionDataMiddleware does not exist

    L.S.,

    I got the following error: ReflectionException in Container.php line 734: Class Kyranb\Footprints\Middleware\CaptureAttributionDataMiddleware does not exist in Container.php line 734 at ReflectionClass->__construct('Kyranb\Footprints\Middleware\CaptureAttributionDataMiddleware') in Container.php line 734 at Container->build('Kyranb\Footprints\Middleware\CaptureAttributionDataMiddleware', array()) in Container.php line 629 at Container->make('Kyranb\Footprints\Middleware\CaptureAttributionDataMiddleware', array()) in Application.php line 697 at Application->make('Kyranb\Footprints\Middleware\CaptureAttributionDataMiddleware') in Pipeline.php line 126 at Pipeline->Illuminate\Pipeline\{closure}(object(Request)) at call_user_func(object(Closure), object(Request)) in Pipeline.php line 32 at Pipeline->Illuminate\Routing\{closure}(object(Request)) in VerifyCsrfToken.php line 64 at VerifyCsrfToken->handle(object(Request), object(Closure)) at call_user_func_array(array(object(VerifyCsrfToken), 'handle'), array(object(Request), object(Closure))) in Pipeline.php line 136 at Pipeline->Illuminate\Pipeline\{closure}(object(Request)) at call_user_func(object(Closure), object(Request)) in Pipeline.php line 32 at Pipeline->Illuminate\Routing\{closure}(object(Request)) at call_user_func(object(Closure), object(Request)) in Pipeline.php line 102 at Pipeline->then(object(Closure)) in Router.php line 726 at Router->runRouteWithinStack(object(Route), object(Request)) in Router.php line 699 at Router->dispatchToRoute(object(Request)) in Router.php line 675 at Router->dispatch(object(Request)) in Kernel.php line 246 at Kernel->Illuminate\Foundation\Http\{closure}(object(Request)) at call_user_func(object(Closure), object(Request)) in Pipeline.php line 52 at Pipeline->Illuminate\Routing\{closure}(object(Request)) in InputTrimmer.php line 20 at InputTrimmer->handle(object(Request), object(Closure)) at call_user_func_array(array(object(InputTrimmer), 'handle'), array(object(Request), object(Closure))) in Pipeline.php line 136 at Pipeline->Illuminate\Pipeline\{closure}(object(Request)) at call_user_func(object(Closure), object(Request)) in Pipeline.php line 32 image

    One way to solve it is using php artisan optimize --force

    Do you have an idea about what could go wrong?

    ` protected $middleware = [ \Kyranb\Footprints\FootprintsServiceProvider::class, ];

    protected $routeMiddleware = [ ... 'capture.attribution' => \Kyranb\Footprints\Middleware\CaptureAttributionDataMiddleware::class, ... ]; `

    opened by django23 7
  • Releases/v3

    Releases/v3

    • [x] Fixes minor bug about footprint being a pure function (introduced in #41 dammit!)
    • [x] Now only supporting PHP7.4+ and Laravel7+
    • [x] Delete unused Facade
    • [x] Implement return type casting and typed properties
    • [x] Implement Github actions (stolen 100% from https://github.com/spatie/laravel-http-logger πŸ™ )
    • [x] Update Laravel version in package image @kyranb ?
    opened by bilfeldt 5
  • Fix request flow and fingerprint

    Fix request flow and fingerprint

    This PR implements modifications to the middleware and now does the tracking pre-controller (issue #40) while it also introduces a new request macro $request->footprint() which uses a new Fingerprinter class.

    Missing:

    • ~Update README.md as the interfaces have changed~ (was not needed)
    • [x] Find a way to make the footprint() macro configurable (perhaps a new Footprinting class which implements a FootprintingInterface?)

    @kyranb This is inspired from you suggestion on a fingerprint method. I think for this to be really useful then it should be possible to change the logic used for generating the footprint. Do you have any suggestions how we can do that? Making this configurable would be a good way for SPAs to do some customisation like providing the footprint as a query parameter from the SPA.

    opened by bilfeldt 5
  • user_id is always blank

    user_id is always blank

    I'm noticing that Visits "user_id" is never being set in my system. My registration is unconventional (occurs when another resource is created), and I'm wondering if this might be the cause.

    Are we listening for an event on User created in this package?

    Any help is greatly appreciated!

    opened by iredmedia 5
  • Manually Creating a User Fails

    Manually Creating a User Fails

    Hi,

    I think there might be an issue when manually creating a User through Eloquent (e.g. User::create($data)). In TrackRegistrationAttribution.php there is this boot method:

     public static function bootTrackRegistrationAttribution()
        {
            // Add an observer that upon registration will automatically sync up prior visits.
            static::created(function (Model $model) {
                $model->trackRegistration();
            });
        }
    

    Which calls this:

        /**
         * Assign earlier visits using current request.
         */
        public function trackRegistration(Request $request): void
        {
            $job = new AssignPreviousVisits($request->footprint(), $this);
    
            if (config('footprints.async') == true) {
                dispatch($job);
            } else {
                $job->handle();
            }
        }
    

    But there is no Request so it fails with error:

    Too few arguments to function App\Models\User::trackRegistration(), 0 passed

    opened by spinalwiz 3
  • Refactor logging jobs

    Refactor logging jobs

    ~DRAFT PR: until #37 and #38 are merged.~

    This PR refactors the logging jobs by passing them any class that implements the TrackableInterface instead of just the id of the eloquent model.

    This allows for much greater flexibility and would allow for tracking multiple models (closes #21) while it also makes it possible to make modifications like matching based on ip and headers (essentially closing #19).

    Note this is a breaking change and notes about upgrading has been included in README.md

    opened by bilfeldt 3
  • Logging user IP and much more

    Logging user IP and much more

    @kyranb would you be willing to consider a PR implementing implementing what is described below?

    Description

    Currently there is a concept of "Custom Parameters" (defined in the configuration: footprints.custom_parameters) which should make it possible to log extra stuff that might be relevant. However we are limited to request inputs since this is implemented like so:

    // https://github.com/kyranb/Footprints/blob/master/src/Middleware/CaptureAttributionDataMiddleware.php
        /**
         * @return array
         */
        protected function getCustomParameter()
        {
            $arr = [];
    
            if (config('footprints.custom_parameters')) {
                foreach (config('footprints.custom_parameters') as $parameter) {
                    $arr[$parameter] = $this->request->input($parameter); // <--- Only inputs
                }
            }
    
            return $arr;
        }
    

    Cons

    Now we all know that cookies are not always the best way to track users. In many applications where it was not possible to track a cookie then I assume that it would be sufficient to match previous visits based on the IP address of the user (for many applications you will never see multiple users from the same IP address) or slightly better would be a combination of IP address and the User-agent provided in the headers. Now none of these approaches are possible using the package right now.

    Suggestion

    These suggestions would each constitute a breaking change, so I am proposing that all of them are implemented in the same go:

    Adding IP

    I suggest adding the IP of the request ($request->ip()) in CaptureAttributionDataMiddleware since this is simple and could be used for matching in many cases. This would also need to be added to the Visit model and to the migration.

    Note that I am not proposing to save the headers as described as an option in the initial description.

    Introducing CaptureHandler

    I suggest modifying the CaptureAttributionDataMiddleware so that everything except the handle method is moved to a separate class called CaptureHandler which implements a new interface called CaptureHandlerInterface.

    The point being that a developer can easily swap out the Handler for another concrete implementation and hence save the request in any way or shape they find useful. This would also solve #21.

    Moving authenticated user

    Since we are already introducing breaking changes then I suggest moving the TrackVisit.php#L26:

        public function handle()
        {
            $user = [];
            $user[config('footprints.column_name')] = Auth::user() ? Auth::user()->id : null; // <-- Moving to CaptureHandler enable modifications
    
            $visit = Visit::create(array_merge([
            ....
    

    to the new CaptureHandler (previously CaptureAttributionDataMiddleware.php#L106:

    which would partially solve #30

    opened by bilfeldt 3
  • TrackVisit using authenticated user id

    TrackVisit using authenticated user id

    The TrackUser function is using the authenticated user id even if we are using a different model:

    $user[config('footprints.column_name')] = Auth::user() ? Auth::user()->id : null;

    opened by liran-co 3
  • Case-sensitive PSR-4 autoload directory names

    Case-sensitive PSR-4 autoload directory names

    Fix https://github.com/kyranb/Footprints/issues/28

    "The subdirectory name MUST match the case of the sub-namespace names."

    https://www.php-fig.org/psr/psr-4/

    opened by hughgrigg 3
  • Multiple models

    Multiple models

    It would be nice if we could track multiple models instead of just one. Using a morphables, you could have model_type and model_id columns. This is useful when we have registration for several different models.

    opened by liran-co 3
  • Laravel 5.4 error

    Laravel 5.4 error

    Hey,

    I have just upgraded to Laravel 5.4 and seem to be getting an issue. The error is Closure object cannot have properties. The issue seems to be related to asyncTrackVisit in CaptureAttributionDataMiddleware.

    Anyone else seen this or knows how to fix it?

    opened by jbardnz 3
  • Error in cookie

    Error in cookie

    Hi guys. I think I found a problem with the Footprinter.php file.

    on the line 21 we have this function

    public function footprint(Request $request): string
    {
            $this->request = $request;
    
            if ($request->hasCookie(config('footprints.cookie_name'))) {
                return $request->cookie(config('footprints.cookie_name'));
            }
    
            // This will add the cookie to the response
            Cookie::queue(
                config('footprints.cookie_name'),
                $footprint = substr($this->fingerprint(), 0, 255),
                config('footprints.attribution_duration'),
                null,
                config('footprints.cookie_domain')
            );
    
            return $footprint;
    }
    

    the problem is in the line 32 because laravel make a cookie encryptation and then the first request returns the value stored in the variable $footprint

    but the second request returns the cookie value encrypted

    my solution was to add the cookie name in the EncryptCookies middleware and so we prevent laravel from changing the value.

    opened by winkelco 0
Releases(3.1.5)
Owner
Kyran
Kyran
Simple and fast issue tracking system

Treenga β€” Simple and fast issue tracking system Treenga is simple. Team functions as the main umbrella under which all issues sit. It shares everythin

null 38 Dec 24, 2022
Well secured MySQL Database Login and Registration with an seperate dashboard for consumers and admins.

WebApplicationPHP Well secured MySQL Database Login and Registration with an seperate dashboard for consumers and admins. Functions Well secured MySQL

z3ntl3 root 1 Jan 21, 2022
Complete Login and Registration system using HTML, CSS, JAVASCRIPT, BOOTSTRAP, PHP and MYSQL

Complete Login and Registration system using HTML, CSS, JAVASCRIPT, BOOTSTRAP, PHP and MYSQL

JehanKandy 11 Jul 13, 2022
School Management System Ver 1.0 with login and registration and admin panel

School Management System Ver 1.0 with login and registration and admin panel

JehanKandy 10 Jul 13, 2022
Complete Login and Registration System with HTML CSS Bootstrap PHP and MYSQL

Complete-Login-and-Registration-System Complete Login and Registration System with HTML CSS Bootstrap PHP and MYSQL for .sql file run xampp server ope

JehanKandy 10 Jul 13, 2022
A web application for a school, facilitating the registration of students. Built using HTML/CSS, PHP and Oracle.

Student Registration System A web application for a school, facilitating the registration of students. Built using HTML/CSS, PHP and Oracle. Included

Sana Khan 4 Oct 2, 2021
Savsoft Quiz v6.0 - An open source and free solution to conduct online quiz or exams.

Savsoft Quiz v6.0 is an Opern Source and Free php based web application (script) to create and manage online quiz, test, exam on your website or serve

null 22 Dec 10, 2022
LiveZilla - a help desk software that offers a help desk solution for small companies to large businesses

LiveZilla includes a live chat software with multi-website support, visitor monitoring and a help desk system that allows you to not only integrate emails that you receive from customers but also messages from Twitter and Facebook in your ticket system.

Maher Amara 9 Nov 10, 2022
A simple wrapper for PHP Intervention Library to provide a more simple interface and convenient way to convert images to webp

This package is a simple wrapper for PHP Intervention Library to provide a more simple interface and convenient way to convert images to webp - next generation format - extension, and resize them to render only needed sizes.

eyad hamza 18 Jun 28, 2022
Simple-podcast-generator - πŸ‘‰ A very simple way to host your podcast files and generate the RSS Podcast feed πŸŽ™

Podcast RSS Feed Generator A very simple way to host your podcast files and generate the RSS Podcast feed ?? ?? Contents Summary Requirements Installa

β™š PH⑦ de Soriaβ„’β™› 11 Dec 2, 2022
Simple-cache - PHP FIG Simple Cache PSR

PHP FIG Simple Cache PSR This repository holds all interfaces related to PSR-16. Note that this is not a cache implementation of its own. It is merely

PHP-FIG 8k Jan 3, 2023
Created simple login system and chat type website using mysql database along with php and html , css and javascript.

Created simple login system and chat type website using mysql database along with php and html , css and javascript.

null 1 Jan 6, 2022
A simple helpdesk tickets system for Laravel 5.1+ which integrates smoothly with Laravel default users and auth system

A simple helpdesk tickets system for Laravel 5.1+ which integrates smoothly with Laravel default users and auth system, demo is available at: http://ticketit.kordy.info/tickets

Ahmed Kordy 857 Dec 30, 2022
TinyFileManager is web based file manager and it is a simple, fast and small file manager with a single file, multi-language ready web application

TinyFileManager is web based file manager and it is a simple, fast and small file manager with a single file, multi-language ready web application for storing, uploading, editing and managing files and folders online via web browser. The Application runs on PHP 5.5+, It allows the creation of multiple users and each user can have its own directory and a build-in support for managing text files with cloud9 IDE and it supports syntax highlighting for over 150+ languages and over 35+ themes.

Prasath Mani 3.5k Jan 7, 2023
Bolt is a simple CMS written in PHP. It is based on Silex and Symfony components, uses Twig and either SQLite, MySQL or PostgreSQL.

⚠️ Note - Not the latest version This is the repository for Bolt 3. Please know that Bolt 5 has been released. If you are starting a new project, plea

Bolt 4.1k Dec 27, 2022
Modern, simple and fresh looking glass based on Bootstrap 5 and PHP 7

Modern, simple and fresh looking glass based on Bootstrap 5 and PHP 7. A looking glass is a network utility which is made user-friendly for everyone to use. It allows you to execute network related commands within a remote network, usually that of an ISP.

Hybula 77 Jan 1, 2023
Koel is a simple web-based personal audio streaming service written in Vue and Laravel

Koel (also stylized as koel, with a lowercase k) is a simple web-based personal audio streaming service written in Vue on the client side and Laravel on the server side. Targeting web developers, Koel embraces some of the more modern web technologies – CSS grid, audio, and drag-and-drop API to name a few – to do its job.

Koel 14.3k Jan 4, 2023
A simple and useful blog coded with laravel & php.

Blog with Setup Directions npm install composer install Set database infos in the .env php artisan migrate:fresh --seed php artisan serve npm run hot

Mustafa Tofur 1 Oct 11, 2022
Simple Laravel Invoice Generator Sling β€” open-source web application that helps you create invoices and track income.

Simple Laravel Invoice Generator Sling β€” open-source web application that helps you create invoices and track income. Table of Contents About

Ray Icemont 4 Nov 22, 2022