Laravel Model Repository

This package provides an abstraction layer for easily implementing industry-standard caching strategies with Eloquent models.

• Laravel Model Repository
  • Main Advantages
    • Simplify caching strategies and buy time
    • Save cache storage and money
  • Installation
    • Laravel version Compatibility
    • Lumen version Compatibility
    • Install the package via Composer
  • Creating a Repository for a Model
  • Use with Singleton Pattern
  • Eloquent like methods
  • Making Eloquent Queries
  • Caching methods overview
  • Implementing Caching Strategies
    • Read-Aside
    • Read-Through
    • Write-Through
    • Write-Back
  • Pretty Queries
  • Cache Invalidation Techniques
    • Saving cache storage
    • Keeping cache consistency
  • Exception handling
    • Database Exceptions
    • Cache Store Exceptions
  • Repository Events
  • Some things I wish somebody told me before
  • Bibliography

Main Advantages

Simplify caching strategies and buy time

Implementing high availability and concurrency caching strategies could be a complex and time consuming task without the appropiate abstraction layer.

Laravel Model Repository simplifies caching strategies using human-readable chained methods for your existing Eloquent models :)

Save cache storage and money

Current available methods for caching Laravel models store the entire PHP object in cache. That consumes a lot of extra storage and results in slower response times, therefore having a more expensive infrastructure.

Laravel Model Repository stores only the business specific data of your model in order to recreate exactly the same instance later (after data being loaded from cache). Saving more than 50% of cache storage and significantly reducing response times from the cache server.

Installation

Make sure you have properly configured a cache connection and driver in your Laravel/Lumen project. You can find cache configuration instructions for Laravel at https://laravel.com/docs/7.x/cache and for Lumen at https://lumen.laravel.com/docs/6.x/cache

Laravel version Compatibility

Lumen version Compatibility

Install the package via Composer

$ composer require krazydanny/laravel-repository

Creating a Repository for a Model

In order to simplify caching strategies we will encapsulate model access within a model repository.

Two parameters can be passed to the constructor. The first parameter (required) is the model's full class name. The second parameter (optional) is the prefix to be used in cache to store model data.

namespace App\Repositories;

use App\User;
use KrazyDanny\Laravel\Repository\BaseRepository;

class UserRepository extends BaseRepository {

	public function __construct ( ) {

		parent::__construct(
			User::class, // Model's full class name
			'Users' // OPTIONAL the name of the cache prefix. The short class name will be used by default. In this case would be 'User'
		);
	}
}

Use with Singleton Pattern

As a good practice to improve performance and keep your code simple is strongly recommended to use repositories along with the singleton pattern, avoiding the need for creating separate instances for the same repository at different project levels.

First register the singleton call in a service provider:

namespace App\Providers;

use App\Repositories\UserRepository;
use Illuminate\Support\ServiceProvider;

class AppServiceProvider extends ServiceProvider {

    public function register ( ) {

        $this->app->singleton( 
           UserRepository::class, 
            function () {
                return (new UserRepository);
            }
        );
    }

    # other service provider methods here
}

Add a line like this on every file you call the repository in order to keep code clean and pretty ;)

use App\Repositories\UserRepository;

Then access the same repository instance anywhere in your project :)

$userRepository = app( UserRepository::class );

You can also typehint it as a parameter in controllers, event listeners, middleware or any other service class and laravel will automatically inject the repository instance

namespace App\Http\Controllers;

use Illuminate\Http\Request;
use App\Repositories\UserRepository;

class UserController extends Controller
{
	public function myMethod( UserRepository $userRepository, $id){
		// you can now use the repository to work with cached models
		$user = $userRepository->get( $id );
	}
}

Eloquent like methods

Calling Eloquent-like methods directly from our repository gives us the advantage of combining them with caching strategies. First, let's see how we call them. It's pretty straightforward :)

create()

Create a new model:

$user = app( UserRepository::class )->create([
	'firstname' => 'Krazy',
	'lastname'  => 'Danny',
	'email'	    => '[email protected]',
	'active'    => true,
]);

$user_id = $user->getKey();

get()

Get a specific model by ID:

$user = app( UserRepository::class )->get( $user_id );

save()

Update a specific model:

$user->active = false;

app( UserRepository::class )->save( $user );

delete()

Delete a specific model:

app( UserRepository::class )->delete( $user );

Making Eloquent Queries

Unlike get() or save(), query methods work a little different. They receive as parameter the desired query builder instance (Illuminate\Database\Eloquent\Builder) in order to execute the query.

This will allow us to combine queries with caching strategies, as we will cover forward on this document. For now let's focus on the query methods only. For example:

find()

To find all models under a certain criteria:

$q = User::where( 'active', true );

$userCollection = app( UserRepository::class )->find( $q );

first()

To get the first model instance under a certain criteria:

$q = User::where( 'active', true );

$user = app( UserRepository::class )->first( $q );

count()

To count all model instances under a certain criteria:

$q = User::where( 'active', true );

$userCount = app( UserRepository::class )->count( $q );

Caching methods overview

remember() & during()

Calling remember() before any query method like find(), first() or count() stores the query result in cache for a given time. Always followed by the during() method, which defines the duration of the results in cache (TTL/Time-To-Live in seconds)

VERY IMPORTANT: For Laravel/Lumen v5.7 and earlier versions TTL param passed to during() are minutes instead of seconds. This library follows Laravel standards so check what unit of time your version uses for the Cache facade.

$q = User::where( 'active', true );

app( UserRepository::class )->remember()->during( 3600 )->find( $q );

Also a model instance could be passed as parameter in order to store that specific model in cache.

app( UserRepository::class )->remember( $user )->during( 3600 );

according()

The according() method does almost the same as during() but with a difference, it reads the time-to-live in seconds from a given model's attribute:

app( ReservationsRepository::class )->remember( $reservation )->according( 'expiresIn' );

This is useful if different instances of the same class have/need different or dynamic time-to-live values.

rememberForever()

Calling rememberForever() before any query method like find(), first() or count() stores the query result in cache without an expiration time.

$q = User::where( 'active', true );

app( UserRepository::class )->rememberForever()->find( $q );

Also a model instance could be passed as parameter in order to store that specific model in cache without expiration.

app( UserRepository::class )->rememberForever( $user );

fromCache()

Calling fromCache() before any query method like find(), first() or count() will try to retrieve the results from cache ONLY.

$q = User::where( 'active', true );

app( UserRepository::class )->fromCache()->find( $q );

Also a model instance could be passed as parameter in order to retrieve that specific model from cache ONLY.

app( UserRepository::class )->fromCache( $user );

forget()

This method removes one or many models (or queries) from cache. It's very useful when you have updated models in the database and need to invalidate cached model data or related query results (for example: to have real-time updated cache).

The first parameter must be an instance of the model, a specific model ID (primary key) or a query builder instance (Illuminate\Database\Eloquent\Builder).

Forget query results:

$query = User::where( 'active', true );

app( UserRepository::class )->forget( $query );

Forget a specific model using the object:

app( UserRepository::class )->forget( $userModelInstance );

Forget a specific model by id:

app( UserRepository::class )->forget( $user_id );

The second parameter (optional) could be an array to queue forget() operations in order to be done in a single request to the cache server.

When passed the forget() method appends to the array (by reference) the removal operations instead of sending them instantly to the cache server.

It's useful when you need to expire many cached queries or models of the same repository. You can do it in one request optimizing response times for your cache server, therefore your app :)

For example:

$user->active = false;
$user->save();

$forgets = [];

#removes user model from cache
app( UserRepository::class )->forget( $user, $forgets );

#removes query that finds active users
$query = User::where( 'active', true );
app( UserRepository::class )->forget( $query, $forgets );

#requests all queued removals to the cache server
app( UserRepository::class )->forget( $forgets );

Implementing Caching Strategies

Read-Aside Cache

How it works?

1. The app first looks the desired model or query in the cache. If the data was found in cache, we’ve cache hit. The model or query results are read and returned to the client without database workload at all.
2. If model or query results were not found in cache we have a cache miss, then data is retrieved from database.
3. Model or query results retrived from database are stored in cache in order to have a successful cache hit next time.

Use cases

Works best for heavy read workload scenarios and general purpose.

Pros

Provides balance between lowering database read workload and cache storage use.

Cons

In some cases, to keep cache up to date in real-time, you may need to implement cache invalidation using the forget() method.

Usage

When detecting you want a model or query to be remembered in cache for a certain period of time, Laravel Model Repository will automatically first try to retrieve it from cache. Otherwise will automatically retrieve it from database and store it in cache for the next time :)

Read-Aside a specific model by ID:

$user = app( UserRepository::class )->remember()->during( 3600 )->get( $user_id );

Read-Aside query results:

$q = User::where( 'active', true );

$userCollection = app( UserRepository::class )->remember()->during( 3600 )->find( $q );

$userCount = app( UserRepository::class )->remember()->during( 3600 )->count( $q );

$firstUser = app( UserRepository::class )->remember()->during( 3600 )->first( $q );

Read-Through Cache

How it works?

1. The app first looks the desired model or query in the cache. If the data was found in cache, we’ve cache hit. The model or query results are read and returned to the client without database workload at all.
2. If model or query results were not found in cache we have a cache miss, then data is retrieved from database ONLY THIS TIME in order to be always available from cache.

Use cases

Works best for heavy read workload scenarios where the same model or query is requested constantly.

Pros

Keeps database read workload at minimum because always retrieves data from cache.

Cons

If you want cache to be updated you must combine with Write-Through strategy (incrementing writes latency and workload in some cases) or implementing cache invalidation using the forget() method.

Usage

When detecting you want a model or query to be remembered in cache forever, Laravel Model Repository will automatically first try to retrieve it from cache. Otherwise will automatically retrieve it from database and store it without expiration, so it will be always available form cache :)

Read-Through a specific model by ID:

$user = app( UserRepository::class )->rememberForever()->get( $user_id );

Read-Through query results:

$q = User::where( 'active', true );

$userCollection = app( UserRepository::class )->rememberForever()->find( $q );

$userCount = app( UserRepository::class )->rememberForever()->count( $q );

$firstUser = app( UserRepository::class )->rememberForever()->first( $q );

Write-Through Cache

How it works?

Models are always stored in cache and database.

Use cases

Used in scenarios where consistency is a priority or needs to be granted.

Pros

No cache invalidation techniques required. No need for using forget() method.

Cons

Could introduce write latency in some scenarios because data is always written in cache and database.

Usage

When detecting you want a model to be remembered in cache, Laravel Model Repository will automatically store it in cache and database (inserting or updating depending on the case).

Write-Through without expiration time:

# create a new user in cache and database
$user = app( UserRepository::class )->rememberForever()->create([
	'firstname' => 'Krazy',
	'lastname'  => 'Danny',
	'email'	    => '[email protected]',
	'active'    => true,
]);

# update an existing user in cache and database
$user->active = false;

app( UserRepository::class )->rememberForever()->save( $user );

Write-Through with expiration time (TTL):

# create a new user in cache and database
$user = app( UserRepository::class )->remember()->during( 3600 )->create([
	'firstname' => 'Krazy',
	'lastname'  => 'Danny',
	'email'	    => '[email protected]',
	'active'    => true,
]);

# update an existing user in cache and database
$user->active = false;

app( UserRepository::class )->remember()->during( 3600 )->save( $user );

Write-Back Cache

How it works?

Models are stored only in cache until they are massively persisted in database.

Use cases

Used in heavy write load scenarios and database-cache consistency is not a priority.

Pros

Very performant and resilient to database failures and downtimes

Cons

In some cache failure scenarios data may be permanently lost.

Usage

IMPORTANT!! THIS STRATEGY IS AVAILABLE FOR REDIS CACHE STORES ONLY (at the moment)

With the buffer() or index() method Laravel Model Repository will store data in cache untill you call the persist() method which will iterate many (batch) of cached models at once, allowing us to persist them the way our project needs through a callback function.

First write models in cache:

Using buffer()

Stores models in cache in a way only accesible within the persist() method callback. Useful for optimizing performance and storage when you don't need to access them until they are persisted in database.

$model = app( TransactionsRepository::class )->buffer( new Transactions( $data ) );

Using index()

Stores models in a way that they are available to be loaded from cache by get() method too. Useful when models need to be accesible before they are persisted.

$model = app( TransactionsRepository::class )->index( new Transactions( $data ) );

Then massively persist models in database:

Using persist()

The persist() method could be called later in a separate job or scheduled task, allowing us to manage how often we need to persist models into the database depending on our project's traffic and infrastructure.

app( TransactionsRepository::class )->persist( 

    // the first param is a callback which returns true if models were persisted successfully, false otherwise
    function( $collection ) {
        
        foreach ( $collection as $model ) {

            // do database library custom and optimized logic here

            // for example: you could use bulk inserts and transactions in order to improve both performance and consistency
        }        

        if ( $result )
            return true; // if true remove model ids from persist() queue
        
        return false; // if false keeps model ids in persist() queue and tries again next time persist() method is called
    },

    // the second param (optional) is an array with one or many of the following available options
    [
        'written_since' => 0, // process only models written since ths specified timestamp in seconds
        'written_until' => \time(), // process only models written until the given timestamp in seconds
        'object_limit'  => 500, // the object limit to be processed at the same time (to prevent memory overflows)
        'clean_cache'   => true, // if true and callback returns true, marks models as persisted
        'method'        => 'buffer' // buffer | index
    ] 
);

The method parameter:

It has two possible values.

• buffer (default)

Performs persist() only for those models stored in cache with the buffer() method;

• index

Performs persist() only for those models stored in cache with the index() method;

Pretty Queries

You can create human readable queries that represent your business logic in an intuititve way and ensures query criteria consistency encapsulating it's code.

For example:

namespace App\Repositories;

use App\User;
use KrazyDanny\Laravel\Repository\BaseRepository;

class UserRepository extends BaseRepository {

	public function __construct ( ) {

		parent::__construct(
			User::class, // Model's class name
			'Users' // the name of the cache prefix
		);
	}

	public function findByState ( string $state ) {

		return $this->find(
			User::where([
				'state'      => $state,
				'deleted_at' => null,
			])
		);
	}

}

Then call a pretty query :)

$activeUsers = app( UserRepository::class )->findByState( 'active' );

$activeUsers = app( UserRepository::class )->remember()->during( 3600 )->findByState( 'active' );

$activeUsers = app( UserRepository::class )->rememberForever()->findByState( 'active' );

Cache invalidation techniques

In some cases we will need to remove models or queries from cache even if we've set an expiration time for them.

Saving cache storage

To save storage we need data to be removed from cache, so we'll use the forget() method. Remember?

For specific models:

app( UserRepository::class )->forget( $user );

For queries:

$user->active = false;
$user->save();

$query = User::where( 'active', true );
app( UserRepository::class )->forget( $query );

On events

Now let's say we want to invalidate some specific queries when creating or updating a model. We could do something like this:

namespace App\Repositories;

use App\User;
use KrazyDanny\Laravel\Repository\BaseRepository;

class UserRepository extends BaseRepository {

	public function __construct ( ) {

		parent::__construct(
			User::class, // Model's class name
			'Users' // the name of the cache prefix
		);
	}

	// then call this to invalidate active users cache and any other queries or models cache you need.
	public function forgetOnUserSave ( User $user ) {

		// let's use a queue to make only one request with all operations to the cache server
		$invalidations = [];

		// invalidates that specific user model cache
		$this->forget( $user, $invalidations );

		// invalidates the active users query cache
		$this->forget(
			User::where([
				'state'      => 'active',
				'deleted_at' => null,
			]),
			$invalidations
		);

		// makes request to the server and invalidates all cache entries at once

		$this->forget( $invalidations );
	}

}

Then, in the user observer...

namespace App\Observers;

use App\User;
use App\Repositories\UserRepository;

class UserObserver {   

    public function saved ( User $model ) {

    	app( UserRepository::class )->forgetOnUserSave( $model );
    }

    # here other observer methods
}

For real-time scenarios

To keep real-time cache consistency we want model data to be updated in the cache instead of being removed.

For specific models:

We will simply use remember(), during() and rememberForever() methods:

app( UserRepository::class )->rememberForever( $user );
// or
app( UserRepository::class )->remember( $user )->during( 3600 );

For queries:

We would keep using forget() method as always, otherwise it would be expensive anyway getting the query from the cache, updating it somehow and then overwriting cache again.

On events

Let's assume we want to update model A in cache when model B is updated.

We could do something like this in the user observer:

namespace App\Observers;

use App\UserSettings;
use App\Repositories\UserRepository;

class UserSettingsObserver {   

    public function saved ( UserSettings $model ) {

    	app( UserRepository::class )->remember( $model )->during( 3600 );
    }

    # here other observer methods
}

Repository Events

We can also observe the following repository-level events.

• afterGet
• afterFirst
• afterFind
• afterCount

On each call

$callback = function ( $cacheHit, $result ) {

	if ( $cacheHit ) {
		// do something when the query hits the cache
	}
	else {
		// do something else when the query hits the database
		// this is not for storing the model in cache, remember the repository did it for you.
	}
}

app( UserRepository::class )->observe( $callback )->rememberForever()->get( $user_id );

On every call

$callback = function ( $cacheHit, $result ) {

	if ( $cacheHit ) {
		// do something when the query hits the cache
	}
	else {
		// do something else when the query hits the database
		// this is not for storing the model in cache, remember the repository did it for you.
	}
}

app( UserRepository::class )->observeAlways( 'afterGet', $callback);

app( UserRepository::class )->rememberForever()->get( $user_A_id );

app( UserRepository::class )->rememberForever()->get( $user_B_id );

Some use cases...

• Monitoring usage of our caching strategy in production environments.
• Have a special treatment for models or query results loaded from cache than those retrieved from database.

Exceptions handling

Cache Exceptions

app( UserRepository::class )->handleCacheExceptions(function( $e ){
	// here we can do something like log the exception silently
})

Database Exceptions

app( UserRepository::class )->handleDatabaseExceptions(function( $e ){
	// here we can do something like log the exception silently
})

The silently() method

When called before any method, that operation will not throw database nor cache exceptions. Unless we've thrown them inside handleDatabaseExceptions() or handleCacheStoreExceptions() methods.

For example:

app( UserRepository::class )->silently()->rememberForever()->get( $user_id );

Some things I wish somebody told me before

"Be shapeless, like water my friend" (Bruce Lee)

There's no unique, best or does-it-all-right caching technique.

Every caching strategy has it's own advantages and disadvantages. Is up to you making a good analysis of what you project needs and it's priorities.

Even in the same project you may use different caching strategies for different models. For example: Is not the same caching millons of transaction logs everyday than registering a few new users in your app.

Also this library is designed to be implemented on the go. This means you can progressively apply caching techniques on specific calls.

Lets say we currently have the following line in many places of our project:

$model = SomeModel::create( $data );

Now assume we want to implement write-back strategy for that model only in some critical places of our project and see how it goes. Then we should only replace those specifice calls with:

$model = app( SomeModelRepository::class )->buffer( new SomeModel( $data ) );

And leave those calls we want out of the caching strategy alone, they are not affected at all. Besides some things doesn't really need to be cached.

Be like water my friend... ;)

Bibliography

Here are some articles which talk in depth about caching strategies:

• https://bluzelle.com/blog/things-you-should-know-about-database-caching
• https://zubialevich.blogspot.com/2018/08/caching-strategies.html
• https://codeahoy.com/2017/08/11/caching-strategies-and-how-to-choose-the-right-one/
• https://docs.aws.amazon.com/AmazonElastiCache/latest/mem-ug/BestPractices.html