A Redis based, fully automated and scalable database cache layer for Laravel

Overview

Lada Cache

A Redis based, fully automated and scalable database cache layer for Laravel

Build Status Code Climate Total Downloads Latest Stable Version Latest Unstable Version License

Contributors wanted! Have a look at the open issues and send me an email if you are interested in a quick introduction via Hangouts.

Table of Contents

For further information on how this library works and how to debug it please have a look at the Wiki.

Features

  • Automatically caches all database queries
  • Intelligent cache invalidation with high granularity
  • Works with existing code, no changes required after setup
  • Possibility to cache only specific models or exclude some models
  • Makes use of Laravel Redis (supports clustering)

Version Compatibility

Laravel PHP Lada Cache
5.1-5.6 5.6.4+ 2.x
5.7-5.8 7.1+ 3.x
6.x 7.2+ 4.x
7.x 7.2+ 5.x
8.x 7.3+ 5.x

Performance

The performance gain achieved by using Lada Cache varies between 5% and 95%. It heavily depends on the quantity and complexity of your queries. The more (redundant) queries per request your application fires and the more complex they are, the bigger the performance gain will be. Another important factor to consider is the amount of data returned by your queries, if a query returns 500MB of data, Lada Cache won't make it faster at all. Based on experience, the performance gain in a typical Laravel web application is around 10-30%.

Other than the performance gain, an essential reason to use Lada Cache is the reduced the load on the database servers. Depending on your infrastructure, this may result in reasonable lower cost and introduce new possibilities to scale up your application.

Why?

A lot of web applications make heavy use of the database. Especially using an ORM like Eloquent, queries repeat often and are not always very efficient. One of the most common solutions for this problem is caching the database queries.

Most RDBMS provide internal cache layers (for example Mysql Query Cache). Unfortunately, these caching systems have some very serious limitations:

  • They do not cache queries over multiple tables (e.g. if the queries are using joins)
  • The invalidation granularity is very low (if a single row changes, the entire table gets removed from the cache)
  • They are not distributed, if you have multiple database servers the cache will be created on all of them
  • They are not scalable

Laravel, on the other hand, provides the possibility to cache particular queries manually. The problem is that it doesn't invalidate the cached queries automatically, you'll need to let them expire after a certain time or invalidate them manually on all places where the affected data might be changed.

This library provides a solution for all of the mentioned problems. Install, scale up and lean back.

Why only Redis?

As you may have discovered while looking at the source code, this library is built directly on top of Laravel Redis instead of Laravel Cache, which would make more sense from a general point of view. However, there are several important reasons behind this decision:

  • Storage must be in-memory (wouldn't make much sense otherwise)
  • Storage must be easily scalable
  • Storage must support tags (Laravel Cache does support tags, but the implementation is very bad and slow)

If you still want to use another storage backend, please feel free to contribute.

Requirements

Installation

Lada Cache can be installed via Composer by requiring the spiritix/lada-cache package in your project's composer.json. Or simply run this command:

composer require spiritix/lada-cache

The Lada Cache service provider will automatically be installed using Package Discovery.

Finally, all your models must include the Spiritix\LadaCache\Database\LadaCacheTrait trait. It's a good practice to create a base model class which includes the trait and then gets extended by all your models.

class Car extends \Illuminate\Database\Eloquent\Model {

    use \Spiritix\LadaCache\Database\LadaCacheTrait;
    
    // ...
}

Don't try to only have specific models including the Lada Cache trait, it will result in unexpected behavior. In the configuration, you will find the possibility to include or exclude specific models.

Configuration

Use the following command to publish the lada-cache.phpconfig file to your configuration folder:

php artisan vendor:publish 

Console commands

You may truncate the cache by running the following command:

php artisan lada-cache:flush

If you want to temporarily disable the cache (for example before running migrations), use these commands:

php artisan lada-cache:disable
php artisan lada-cache:enable

Known issues and limitations

  • Doesn't work with raw SQL queries. This would require an SQL parser to be implemented which is quite hard and very inefficient. As long as you are only using raw queries for reading data, it just won't get cached. Serious issues will only occur if you use raw queries for writing data (which you shouldn't be doing anyway).
  • Doesn't work with multiple connections if done like DB::connection('foo'). Instead, specify the protected $connection = 'foo'; property in the relevant models.
  • The cache must be truncated manually after migrations are executed.
  • Pessimistic locking (sharedLock, lockForUpdate) requires usage of raw SQL queries.

Contributing

Contributions in any form are welcome. Please consider the following guidelines before submitting pull requests:

  • Coding standard - It's PSR-2
  • Add tests! - Your PR won't be accepted if it doesn't have tests.
  • Create feature branches - I won't pull from your master branch.

License

Lada Cache is free software distributed under the terms of the MIT license.

Comments
  • Problem with caching

    Problem with caching

    I have a problem. At some point of time the cache of the following request ceases to be updated for the unknown reasons.

    $newProducts = $product->catalogItem()
    ->whereNotNull('available')
    ->whereNull('preorder')
    ->orderBy('release_date', 'desc')
    ->limit($limitProductsOnTabs)
    ->get();
    
    public function scopeCatalogItem($query)
    {
    return $query->select('products.id', 'image', 'name', 'rrp', 'wmr', 'activation_id', 'slug', 'has_discount', getNameCurr())->with('activation')->with('genres');
    }
    

    But the page of product is successfully cached. Why?

    opened by roxik 19
  • Cache not clearing

    Cache not clearing

    Hey,

    Great package, thanks for releasing and maintaining this.

    I currently have a problem that I just noticed, it was fine until a few hours ago.

    The cache on my website doesn't get cleared when new "items" get added to the database. It doesn't happen with every model, just with a particular one.

    It's quite odd to describe and it works just fine on my local env.

    Here's a URL in case you want to see it: https://insegreto.com/it/fresh

    In that page you'll find a list of posts. New posts are not visible until I manually clear the cache but you can see new comments right away.

    I know the info is not much, but maybe you have already an idea of what's happening? Or maybe you can recommend a way to debug this?

    --- EDIT More info. I'm using Envoyer to manage my deployments and after re-deploying (same code) everything is working as expected. I would expect though to have the same issue again in a few hours. Will keep posted.

    --- EDIT #2 So.. after 6 hours, here we go again. Same issue. This time, re-deploying didn't fix the issue. I had to run php artisan lada-cache:flush.

    opened by nicgutierrez 18
  • Lada breaks Laravel Telescope

    Lada breaks Laravel Telescope

    So Lada creates a QueryBuilder here: https://github.com/spiritix/lada-cache/blob/master/src/Spiritix/LadaCache/Database/Connection.php#L31

    But doesn't pass the fifth required argument, a model. Could this be an optional model? I guess telescope somehow uses this function and breaks because it tries to create a new instance of QueryBuilder without the Model.

    bug help wanted 
    opened by genesiscz 11
  • Caching relations

    Caching relations

    I have a model (table) companies

    id (index key), companyname, address, phone

    I also have a model (table) categories

    id (index key), category

    this relation is many to many, companies may have many categories and one category may have many companies. For that reason I created a table companies_categories

    id_company (index key), id_category (index key)

    My question is if lada cache with the current logic will cache everything included the relation with row invalidation? As you see I have assigned both tables with primary key "id" but the relation table has no "id" and I am not sure how this will work.

    opened by george-kar 9
  • updateExistingPivot method doesn't invalidate data

    updateExistingPivot method doesn't invalidate data

    Just wondering if there is a way to bypass the cache. For example, while doing a cron job and wanting direct query result, not a cached version. Or, do I simply need to flush the cache before doing the cron job?

    Thanks

    bug help wanted 
    opened by rbruhn 8
  • phpredis extension support

    phpredis extension support

    I've tried to use phpredis extension with lada-cache instead of Predis, but there is a problem where sadd parameters will not be properly passed to phpredis and on redis-cli monitor i see this

    "SADD" "lada:tags:database:receptes:table:user:row:1" "Array" instead of "SADD" "lada:tags:database:receptes:table:user:row:1" "lada:90e60daab231709fb94aa8e729d54e51"

    opened by Swanty 8
  • Support laravel 8

    Support laravel 8

    Not sure if you want to make old laravel versions incompatible since no changes in the actual package were made but I see that was done for the previous version so:

    • Updated dependencies
    • Updated factories
    • Removed unsupported php from travis
    • Migrated phpunit configuration
    • Updated readme to reflect that phpredis is now default in laravel
    opened by miraries 7
  • Cache not refreshed for relations when using updateOrCreate()

    Cache not refreshed for relations when using updateOrCreate()

    I am getting old data after i updated some details. These are my files:

    UserInfo.php

    <?php
    
    namespace App;
    
    use Carbon\Carbon;
    use Illuminate\Database\Eloquent\Model;
    
    class UserInfo extends Model
    {
        protected $guarded = [];
    
        use \Spiritix\LadaCache\Database\LadaCacheTrait;
    
    }
    

    User.php

    <?php
    
    namespace App;
    
    use Illuminate\Foundation\Auth\User as Authenticatable;
    
    class User extends Authenticatable
    {
        use \Spiritix\LadaCache\Database\LadaCacheTrait;
    
        protected $guarded = [];
    
        protected $with = ['user_info'];
    
        public function user_info()
        {
            return $this->hasOne(UserInfo::class);
        }
    }
    
    $user = User::find(1);
    echo $user->user_info->first_name // outputs 'test1';
    

    then i tried to update his first_name like this:

    UserService.php

    $user_info = $user->user_info()->updateOrCreate(['user_id' => 1], [
    	'first_name' => 'test2'
    ]);
    

    then it results in this:

    $user = User::find(1);
    echo $user->user_info->first_name // outputs 'test1';
    

    what is the issue here?

    opened by hen3b182 7
  • Expiration time for queries

    Expiration time for queries

    Of course this is not very pretty code, but what do you think of the idea?

    It can be used like this: Article::cacheExpire(300)->where('publish_at', '<', \DB::raw('now()'))->limit(10)->get(); After 5 minutes (300 seconds) it will fetch from db again

    opened by Swanty 7
  • Possibility to only cache specific models

    Possibility to only cache specific models

    Not working at the moment.

    It would result in all models getting cached but not invalidated properly. This is because the class who writes values in the cache doesn't know which models are extending the cache model.

    Also I am not sure what would happen if you join a model that is cached from a model which is not cached.

    enhancement 
    opened by spiritix 7
  • Doesn't work with spatie / laravel-permission

    Doesn't work with spatie / laravel-permission

    I am using default setup of spatie / laravel-permission package and lada-cache packages. Everything works well except user<->roles relation.

    class User extends Authenticatable { use HasRoles; use LadaCacheTrait;

    // ...
    

    }

    If i try to make request with 'roles' relation i'm getting an exception.

    \App\User::whereHas('roles')->get();

    TypeError Argument 1 passed to Spiritix\LadaCache\Reflector::getTablesFromWhere() must be an instance of Spiritix\LadaCache\Database\QueryBuilder, instance of Illuminate\Database\Query\Builder given, called in /home/belgiets/projects/laravel7/vendor/spiritix/lada-cache/src/Spiritix/LadaCache/Reflector.php on line 145

    Reproduced on clear installation of:

    • "laravel/framework": "^7.24"
    • "spatie/laravel-permission": "^3.16"
    • "spiritix/lada-cache": "^5.0"
    bug enhancement help wanted 
    opened by Belgiets 6
  • Table names when using aliases

    Table names when using aliases

    When using tables with aliases IE: SELECT * FROM users AS user lada-cache returns the table as users AS user; The table name should actually be modified to use the real database table so that the tags are correctly wrote and cleared. I've recently started having issues with specific queries and noticed this bug.

    I have fixed it locally by adding a preg_replace. Here are my changes in the Reflector class:

        /**
         * Returns all affected tables, including joined ones.
         *
         * @return array
         */
        public function getTables()
        {
            // Get main table
            $tables = [];
            if (is_string($this->queryBuilder->from)) {
                $tables[] = trim(preg_replace('#[\s]+(AS[\s]+)[\w\.]+#i', '', $this->queryBuilder->from));
            }
    
            // Add possible join tables
            $joins = $this->queryBuilder->joins ?: [];
            foreach ($joins as $join) {
                if (!in_array($join->table, $tables) && is_string($join->table)) {
                    $tables[] = trim(preg_replace('#[\s]+(AS[\s]+)[\w\.]+#i', '', $join->table));
                }
            }
    
            $this->getTablesFromWhere($this->queryBuilder, $tables);
    
            return $tables;
        }
    
        /**
         * Get Table Names From Where Exists, Not Exists (whereHas/whereDoesnthave builder syntax)
         *
         * @param QueryBuilder $queryBuilder
         */
        private function getTablesFromWhere(QueryBuilder $queryBuilder, &$tables) {
            if (!isset($queryBuilder->wheres)) {
                return;
            }
            $wheres = $queryBuilder->wheres ?: [];
            foreach ($wheres as $where) {
                if ($where['type'] == 'Exists' || $where['type'] == 'NotExists') {
                    $table = trim(preg_replace('#[\s]+(AS[\s]+)[\w\.]+#i', '', $where['query']->from));
                    if (!in_array($table, $tables) && is_string($table)) {
                        $tables[] = $table;
                    }
    
                    // Add possible join tables
                    $joins = $where['query']->joins ?: [];
                    foreach ($joins as $join) {
    
                        if (!in_array($join->table, $tables) && is_string($join->table)) {
                            $tables[] = trim(preg_replace('#[\s]+(AS[\s]+)[\w\.]+#i', '', $join->table));
                        }
                    }
                }
                if (isset($where['query'])) {
                    $this->getTablesFromWhere($where['query'], $tables);
                }
            }
        }
    
        /**
         * Returns all affected rows as a multidimensional array, split up by table.
         *
         * @return array
         */
        public function getRows()
        {
            $rows = [];
            $wheres = $this->queryBuilder->wheres ?: [];
    
            foreach ($wheres as $where) {
    
                // Skip unsupported clauses
                if (!isset($where['column'])) {
                    continue;
                }
    
                // If it doesn't contain the table name assume it's the "FROM" table
                if (strpos($where['column'], '.') === false) {
                    $where['column'] = implode('.', [trim(preg_replace('#[\s]+(AS[\s]+)[\w\.]+#i', '', $this->queryBuilder->from)), $where['column']]);
                }
    
                list($table, $column) = $this->splitTableAndColumn($where['column']);
    
                // Make sure that the where clause applies for the primary key column
                if ($column !== $this->queryBuilder->model->getKeyName()) {
                    continue;
                }
    
                // Initialize a set for the current table
                if (!isset($rows[$table])) {
                    $rows[$table] = [];
                }
    
                // Add the rows to the current table set
                if ($where['type'] === 'Basic') {
                    if ($where['operator'] === '=' && is_numeric($where['value'])) {
                        $rows[$table][] = $where['value'];
                    }
                }
                else if ($where['type'] === 'In') {
                    $rows[$table] += $where['values'];
                }
            }
    
            return $rows;
        }
    

    This is for the Laravel 5.x branch and may need to be tweaked for 6.x. I have not upgraded to 6 yet, so I'm not able to put a PR as of this moment.

    help wanted feature 
    opened by KnightAR 2
  • Some queries bypass Lada Cache

    Some queries bypass Lada Cache

    After some comprehensive testing, I figured out, that since Laravel 6 there are some queries that bypass Lada Cache. This doesn't cause unexpected behavior but reduces the effect of the cache. If used with Debug Bar, the "Queries" tab must not show any queries after the second page load. All queries which are still shown bypass the cache.

    bug help wanted 
    opened by spiritix 3
  • Disable lada-cache for certains queries

    Disable lada-cache for certains queries

    lada-cache on Laravel 6 is giving me some problem and sometimes the queries are not invalidated and subsequent queries to the database return inconsistent data.

    I would therefore like to understand if it is possible to disable lada-cache for certain queries to the database, here is an example:

    MyModel::withoutLadaCache()->updateOrCreate(...);
    

    What do you think?

    help wanted feature 
    opened by f-liva 7
  • Unable to select which REDIS configuration is used

    Unable to select which REDIS configuration is used

    Note that since Laravel 5.7

    php artisan cache:clear will not clear the default REDIS connection that lada-cache is configured to use (and doesn't appear to have any way to change)

    This means that currently lada-cache is using the 'database' configuration of REDIS in Laravel (called 'default') (for things like sessions) and not the 'cache' configuration (called 'cache') .

    The only work around I can see at the moment, is to change the redis cache configuration in Laravel (/app/config/cache.php) to have it's connection set to the 'default' and not 'cache'.

    This of course undoes the reason it was changed in the first place, doing php artisan cache:clear will also delete anything placed in the REDIS 'database' configuration (i.e. sessions and the like)

    enhancement help wanted 
    opened by SlyDave 4
  • Memory exhausted using Scout

    Memory exhausted using Scout

    Hello

    Running the command php artisan scout:import App\Model it results in a fatal error thrown by lada cache:

    Allowed memory size of 268435456 bytes exhausted (tried to allocate 20480 bytes) in /home/runcloud/webapps/laravel-vivicity/vendor/spiritix/lada-cache/src/Spiritix/LadaCache/Encoder.php on line 43

    help wanted feature 
    opened by f-liva 1
Releases(v5.2)
Owner
Matt
Matt
Laravel 5 - Repositories to abstract the database layer

Laravel 5 Repositories Laravel 5 Repositories is used to abstract the data layer, making our application more flexible to maintain. See versions: 1.0.

Anderson Andrade 4k Jan 6, 2023
Spot v2.x DataMapper built on top of Doctrine's Database Abstraction Layer

Spot DataMapper ORM v2.0 Spot v2.x is built on the Doctrine DBAL, and targets PHP 5.4+. The aim of Spot is to be a lightweight DataMapper alternative

Spot ORM 602 Dec 27, 2022
Doctrine Database Abstraction Layer

Doctrine DBAL 4.0-dev 3.0 2.13 N/A N/A Powerful database abstraction layer with many features for database schema introspection, schema management and

Doctrine 8.9k Dec 28, 2022
Database Abstraction Layer, Schema Introspection, Schema Generation, Query Builders

Cycle DBAL Secure, multiple SQL dialects (MySQL, PostgreSQL, SQLite, SQLServer), schema introspection, schema declaration, smart identifier wrappers,

Cycle ORM 30 Oct 18, 2022
Orm is a simple database abstraction layer that supports postgresql.

Orm What is it Orm is a simple database abstraction layer that supports postgresql. Welcome to join us or star us for encouragement. Requires php 8.1

null 2 Sep 28, 2022
Pure PHP NoSQL database with no dependency. Flat file, JSON based document database.

Please give it a Star if you like the project ?? ❤️ SleekDB - A NoSQL Database made using PHP Full documentation: https://sleekdb.github.io/ SleekDB i

Kazi Mehedi Hasan 745 Jan 7, 2023
Eloquent Filter is a package for filter data of models by the query strings. Easy to use and fully dynamic.

Eloquent Filter Eloquent Filter adds custom filters to your Eloquent Models in Laravel. It's easy to use and fully dynamic. Table of Content Introduct

Mehdi Fathi 327 Dec 28, 2022
:gem: Simple MySQLi Abstraction Layer + Doctrine/DBAL support

?? Simple MySQLi Class This is a simple MySQL Abstraction Layer compatible with PHP 7+ that provides a simple and secure interaction with your databas

Lars Moelleken 40 Sep 5, 2022
A Redis bundle for Symfony supporting Predis and PhpRedis

RedisBundle About This bundle integrates Predis and PhpRedis into your Symfony 3.4+ application, providing a fast and convenient interface to Redis. U

Henrik Westphal 1k Dec 22, 2022
This is raw connection between redis server and django python app

Django_Redis This repository contains the code for this blogpost. Running the Application Clone the repository git clone https://github.com/xxl4tomxu9

Tom Xu 1 Sep 15, 2022
SleekwareDB is a NoSQL database storage service. A database storage service that can be used for various platforms and is easy to integrate.

SleekwareDB is a NoSQL database storage service. A database storage service that can be used for various platforms and is easy to integrate. NoSQL API

SleekwareDB 12 Dec 11, 2022
Поддержка очередей Redis (и на RabbitMq, и на Filesystem, и через DBAL) в Битриксе

Модуль для Битрикса, организующий работу с очередями через Redis (и не только) Поддерживаемый транспорт: Redis RabbitMq Filesystem DBAL Установка comp

Fedy 4 Aug 20, 2021
Async Redis client implementation, built on top of ReactPHP.

clue/reactphp-redis Async Redis client implementation, built on top of ReactPHP. Redis is an open source, advanced, in-memory key-value database. It o

Christian Lück 240 Dec 20, 2022
a distributed-redis-lock implementation for hyperf2.*

hyperf-redis-lock English | 中文 an easy redis-based distributed-lock implementation for hyperf 2.*。 This extension features distributed-lock includes b

lysice 11 Nov 8, 2022
Feather - a highly performant SQLite Cache Driver for Kirby 3

?? Kirby3 SQLite Cache-Driver Feather - a highly performant SQLite Cache Driver for Kirby 3 Commerical Usage Support open source! This plugin is free

Bruno Meilick 1 Dec 15, 2021
Laravel Code Generator based on MySQL Database

Laravel Code Generator Do you have a well structed database and you want to make a Laravel Application on top of it. By using this tools you can gener

Tuhin Bepari 311 Dec 28, 2022
phpSleekDBAdmin - a web-based SleekDB database admin tool written in PHP

phpSleekDBAdmin is a web-based SleekDB database admin tool written in PHP. Following in the spirit of the flat-file system used by SleekDB, phpSleekDBAdmin consists of a single source file, phpsleekdbadmin.php. The interface and user experience is comparable to that of phpLiteAdmin and phpMyAdmin.

GalAnonym 8 Oct 26, 2022
Connect and work with MySQL/MariaDB database through MySQLi in PHP. This is an introductory project, If you need a simple and straightforward example that takes you straight to the point, you can check out these examples.

First MySQLi PHP Connect and work with MySQL/MariaDB database through MySQLi in PHP. The above exercises are designed for students. This is an introdu

Max Base 4 Feb 22, 2022
Clean up your Magento database by removing orphaned, unused and wrongly added attribute, attribute values and settings (for M2).

Magento 2 EAV Cleaner Console Command Purpose of this project is to check for different flaws that can occur due to EAV and provide cleanup functions.

FireGento e. V. - Hackathons 41 Dec 14, 2022