Integrate your PHP application with your HTTP caching proxy

Overview

FOSHttpCache

CI Scrutinizer Quality Score Code Coverage Latest Stable Version Documentation Status

Introduction

This library integrates your PHP applications with HTTP caching proxies such as Varnish. Use this library to send invalidation requests from your application to the caching proxy and to test your caching and invalidation code against a Varnish setup.

It does this by abstracting some caching concepts and attempting to make sure these can be supported across Varnish, Nginx and Symfony HttpCache.

If you use Symfony, have a look at the FOSHttpCacheBundle. The bundle provides the invalidator as a service, along with a number of Symfony-specific features to help with caching and caching proxies.

Features

Documentation

For more information, see the documentation.

License

This library is released under the MIT license. See the included LICENSE file for more information.

Comments
  • Add support for Varnish xkey

    Add support for Varnish xkey

    xkey promises to have vastly better performance than invalidation by BAN for two reasons:

    • It keeps a hash table of the tags around so it can efficiently do invalidation directly instead of filling up a list of regex bans on request data slowing things down
    • It supports soft purge making it able to take into account grace time

    It can be installed on any supported platform by Varnish 4.1+ as long as people have src files for Varnish, and is also available on some Linux distros as package. And besides that, it can be used with Varnish Plus.

    Further reading:

    • Doc: https://github.com/varnish/varnish-modules/blob/master/docs/vmod_xkey.rst
    • Wikipedia: https://wikitech.wikimedia.org/wiki/XKey

    Doc draft

    Configuring xkey:

    [
        'tags_header' => 'xkey-softpurge',// Or 'xkey-purge' if you can not use grace, see VCL
        'tag_mode' => 'purgekeys',
    ]
    

    Todo:

    • [x] Compile xkey on travis
      • [x] ~~Figure out why the (added) Varnish 5.2 testing job fails~~ (move 5.2 support to other pr)
    • [x] Add unit test coverage
    • [x] Add functional test coverage
    opened by andrerom 47
  • Symfony HTTP Cache / Tag Invalidation

    Symfony HTTP Cache / Tag Invalidation

    Currently this library contains several classes related to extending the Symfony HTTP Cache by way of an EventDispatchingKernel trait.

    There is a general need to implement tag invalidation with the Symfony HTTP cache, and this has been done here.

    Most of the logic in that package is concerned with tags and @magnusnordlander is proposing to create a new package which is dedicated to tag storage. Therefore the amount code required for adding the tagging functionality to the HTTP cache is not very large.

    We could therefore reasonably add the tagging functionality in this package, however I am not so sure that it belongs here -

    • I believe all of the functionality we add to the Symfony HTTP Cache can be used without the infrastructure of the FOSHttpCache.
    • I think this bundle should integrate with third-party caches and not provide solutions itself.

    I therefore propose to create a new package containing to the Symfony HTTP cache and ultimately remove the SymfonyCache namespace from this package.

    wdyt?

    enhancement Symfony HttpCache 
    opened by dantleech 40
  • Add abstracted tag handling

    Add abstracted tag handling

    Aims to allow support for tagging also on Symfony HTTP Cache and potentially other caches by limiting the feature to focus on only tags and not random headers.

    Closes #234

    • [x] explain changes in CHANGELOG.md
    • [x] Update documentation
    opened by andrerom 38
  • Out of band tag handling

    Out of band tag handling

    the current tag handling is all about sending tags with the response and sending tag specific requests to the caching proxy. there is another concept for tagging: storing tag => urls ourselves (database, sqlite, redis, whatever) and send normal refresh / ban requests on invalidation. (see also the symfony con talk by @magnusnordlander)

    how can we model this set up? should we add an isInline check to TagsInterface and a saveTags($requestUrl) in ResponseTagger? maybe rather have an InlineTagsInterface for the get header name and get header value methods? or should we try to hide this away and magically save tags in ResponseTagger on getTagsHeaderValue?

    the TagsInterface::invalidateTags looks fine to me. but the clients (Varnish/Nginx/Symfony) should maybe use composition for the functionality rather than implement the method inline, so you can plug either one that calls ban() on the client or the one that calls purge / refresh based on its stored tag to url map.

    opened by dbu 35
  • Decouple from HTTP client library

    Decouple from HTTP client library

    Fix #53.

    Goals:

    • Completely decouple our caching proxy clients from any HTTP client implementation.
    • Remove dependency on Guzzle (or any other) HTTP client.
    • Depend only on PSR-7 and the generic php-http/adapter instead.

    Todo:

    • [x] update docs to reflect changes
    • [x] fix missing host test
    • [x] Switch RTD default to stable, so users won't be confused by our updated (2.0) docs
    v2 
    opened by ddeboer 33
  • Symfony taggable cache

    Symfony taggable cache

    replace #373, fix #286 squashed all commits and did some tweaks to the documentation.

    TODO

    • [x] Once Symfony 3.4 is out: remove the @beta in composer.json

    => Bundle support: https://github.com/FriendsOfSymfony/FOSHttpCacheBundle/issues/403

    opened by dbu 30
  • Fix Host header for proxy client requests

    Fix Host header for proxy client requests

    When using ban requests with Varnish proxy client, requests are queued and sent with flush(). Queued requests are registered without host, making Guzzle create an empty Host header.

    When flushing, Guzzle request is re-created starting with the original one (so without host), with all registered headers. The problem is that the empty Host header is also copied, and thus not re-created by Guzzle with the server URL. In the backend, cURL sees it and takes it into account, overriding passed URL silently.

    This causes requests with empty host being fired by Guzzle, which is kind of annoying :smiley:.

    This patch ensures Host header is removed before sending the real request.

    opened by lolautruche 30
  • Users get sent to /_fos_user_context_hash upon login (ezpublish application)

    Users get sent to /_fos_user_context_hash upon login (ezpublish application)

    Not sure if it is a bug in FOSHttpCache, eZPublish or custom code. The observed behaviour (no varnish, http cache on, ezp 5.4.5):

    • user accesses the site
    • gets redirected to /login
    • fills in the form, posts to /login_check
    • gets redirected to /_fos_user_context_hash

    Since I fixed this by setting:

    security: firewalls: ezpublish_front: form_login: always_use_default_target_path: true

    I suspect that what goes on here is that the internal request made by Sf to retrieve the user-hash gets somehow 'remember' by the session as the last url visited by the end user just before logging in.

    Any tips on fixing this in a more flexible way? Or is this really a bug?

    opened by gggeek 29
  • User Context - Cookies

    User Context - Cookies

    Hey,

    Just raise this issue as after looking more in varnish, it seems that @dbu was right about cookies, and we can have an another way that simply remove it in user context configuration.

    In fact, varnish has a default vcl which is compiled in its core, available here: (for varnish 3) https://www.varnish-cache.org/trac/browser/bin/varnishd/default.vcl?rev=3.0

    So we basically have 2 methods for dealing with cookies in user-context, and i'm wondering what method do you think it's best to show / test :

    • Keep remove / insert cookies when needed
    • Force the passage to lookup in our conditions
    sub vcl_recv {
        // Handle the original request: send a HEAD request to retrieve the user hash
        if (req.restarts == 0 && req.http.cookie && (req.request == "GET" || req.request == "HEAD")) {
            set req.http.x-original-url     = req.url;
            set req.http.x-original-request = req.request;
    
            # Retrieve a unique session id from the cookie
            set req.http.x-session-id = req.http.cookie;
    
            set req.url     = "/user_context_head.php";
            set req.request = "HEAD";
    
            # Force varnish to lookup in this case even with cookies
            return (lookup);
        }
    
        // After the HEAD request, reset the request to the original one, which
        // will be restarted in vcl_deliver.
        if (req.restarts > 0 && req.http.X-Original-Method) {
            set req.request = req.http.x-original-method;
            set req.url     = req.http.x-original-url;
    
            unset req.http.req.http.x-original-method;
            unset req.http.req.http.x-original-url;
    
            # Force varnish to lookup in this case even with cookies
            return (lookup);
        }
    }
    

    I think it's better than remove / insert cookies which is more a hack.

    question 
    opened by joelwurtz 28
  • LiteSpeed support

    LiteSpeed support

    WIP for Open Source Litespeed (OLS) and possibly the commercial Litespeed Web Server (LSWS).

    Cache tagging is only implemented in OLS but not at all in LSWS. The changes worked with OLS but the latest OLS version seems to have done BC breaks / regressions with cache tagging.

    The original author does not continue on this - if you need this, you will need to wrap up this merge request or pay someone to do it...


    This is a possible implementation for LSWS (https://www.litespeedtech.com/products/litespeed-web-server) support.

    needs new contributor 
    opened by Toflar 26
  • Add Fastly support

    Add Fastly support

    Closes #403

    Changes:

    • Use batch tag purging to reduce risk of hitting API limits
    • Support PurgeCapable
    • Drop multi services, does not seems to be a good fit for Purge, & unsure what the use case is
    • Added unit testing

    Follow Up topics:

    • Add support in Bundle
    • Abstract Reverse proxy TTL to use Surrogate Control on Fastly and optionally others #455

    Context of this:

    • eZ Platform (v3) is being worked on by @ViniTou to remove several of our HttpCache abstractions and move to FosHttpCache 2.x, which means we among other things we'd like Fastly support on FosHttpCache level instead ;)
    opened by andrerom 24
  • Ability to have several context hashes

    Ability to have several context hashes

    Recent use case popped up that we have been thinking about some years now is to add support for several context hashes. In order to allow our end users to deal with personalized responses when there is a use case to still allow it to be cached.

    Context

    We use X-User-Context-Hash for user permissions, and if some of our end users extend that to add additional things to the hash, lets say profile choices or other personalized user info you'll end up with:

    • A lot of cache variants for ALL responses, consuming lots of additional memory
    • Very low cache hit ratio

    Solution

    Support for several kind of hashes, each with own Vary header, so only responses that truly need to vary on e.g. X-User-Profile-Hash will actually do so (typically in addition to varying X-User-Context-Hash).

    VCL would probably need to be adapted for this, unless we have a convention we can match in VCL and in PHP (SymfonyCache).

    opened by andrerom 5
  • Switch to Symfony HttpClient?

    Switch to Symfony HttpClient?

    Would it make sense to switch to Symfony HttpClient to reduce setup time with the lib/bundle?

    Technically library would only need to rely on symfony/http-client-contracts OR just PSR-18 as proposed in #443 if that covers all our needs. And furthermore rely on symfony/http-client as dev dependency, while bundle maybe relying and wiring that up by default.

    opened by andrerom 4
  • Use Surrogate-Control across all backends instead of custom X-Reverse-Proxy-TTL

    Use Surrogate-Control across all backends instead of custom X-Reverse-Proxy-TTL

    As discussed in the original Fastly PR in order to generalize TTL handling across backends like Varnish, Fastly, Symfony, it could be an idea to:

    1. abstract it on client side?
    2. to be able to emit Surrogate-Control for fastly
    3. ~If feasible look into also emit Surrogate-Control on Varnish instead of X-Reverse-Proxy-TTL, VCL could still handle for X-Reverse-Proxy-TTL for BC.~

    https://www.w3.org/TR/edge-arch/

    opened by andrerom 10
  • Fix risky tests

    Fix risky tests

    1. FOS\HttpCache\Tests\Unit\CacheInvalidatorTest::testMethodException This test did not perform any assertions

    2. FOS\HttpCache\Tests\Unit\ProxyClient\HttpDispatcherTest::testBanWithoutBaseUri This test did not perform any assertions

    3. FOS\HttpCache\Tests\Unit\Test\Proxy\SymfonyProxyTest::testStart This test did not perform any assertions

    opened by Toflar 0
  • [WIP] Add Varnishadm CLI client

    [WIP] Add Varnishadm CLI client

    Fix #61.

    1. The idea is that using this client involves less custom VCL. However, for the ban lurker to work, some custom VCL is still necessary to copy e.g. the URL header to X-Url.
    2. As far as I know, purge cannot be done through varnishadm, so we should either not implement PurgeInterface or fake the purge by actually doing a ban when purge() is called.
    3. Same goes for refresh. We could fake it by doing a ban and then manually getting a fresh response from the application.
    opened by ddeboer 7
Releases(2.14.1)
  • 2.14.1(Jul 7, 2022)

  • 2.14.0(Jul 6, 2022)

    Symfony Compatibility

    • Removed the internal BaseEvent class and extend our events from the Symfony contracts class directly.
    • Made the CacheInvalidation interface compatible with Symfony 6. Symfony 6 does a BC break for HttpKernelInterface::fetch - if you implement the method in your application, you need to adjust when upgrading to Symfony 6.
    Source code(tar.gz)
    Source code(zip)
  • 2.12.1(Mar 16, 2022)

    Symfony Compatibility

    • Do not extend the wrong Event class when installed with legacy Symfony but the Symfony contracts happen to be installed
    Source code(tar.gz)
    Source code(zip)
  • 2.13.0(Jan 25, 2022)

  • 2.12.0(Jan 12, 2022)

    Cloudflare

    • Added Cloudflare ProxyClient Adapter with ClearCapable, PurgeCapable and TagCapable. This allows to use FOSHttpCache to invalidate caches on Cloudflare. See the "Proxy Client" section of the documentation for how to configure the Cloudflare client.

    Varnish Cache

    • Added a fos_user_context_hash method to be called in vcl_hash when using the user context hash mechanism. This can avoid performance problems Varnish can run into when the hash Varys on the basic authentication or session cookie. If you use the user context, read the updated documentation and call fos_user_context_hash in your vcl_hash function.

    Noop Client

    • Implemented ClearCapable.
    Source code(tar.gz)
    Source code(zip)
  • 2.11.0(Sep 30, 2021)

    Symfony HttpCache

    • Added flag on CustomTtlListener to allow keeping the custom TTL header on the response

    Fastly client

    • Marked the @internal constants in FOS\HttpCache\ProxyClient\Fastly as private
    Source code(tar.gz)
    Source code(zip)
  • 2.10.1(Jun 28, 2021)

  • 2.10.0(Dec 2, 2020)

    General

    • Raised minimal PHP version to 7.2
    • Added support for PHP 8.0

    Symfony HttpCache

    • Added a neutral "Bad Request" body to user hash mismatch response to have something searchable in the code when debugging.
    Source code(tar.gz)
    Source code(zip)
  • 1.4.6(Jul 16, 2020)

    Symfony HttpCache

    • Added a neutral "Bad Request" body to user hash mismatch response to have something searchable in the code when debugging.
    Source code(tar.gz)
    Source code(zip)
  • 2.9.2(May 19, 2020)

  • 2.9.1(May 7, 2020)

  • 2.9.0(Feb 5, 2020)

    General

    • Added support for the fastly caching service
    • Raised minimal PHP version to 7.1
    • Test helper classes upgraded to support PHPUnit 7/8

    Symfony HttpCache

    • Fixed issue with PurgeTagsListener and Symfony 5
    • Fixed clearing the cache completely together with toflar psr6 store did not work
    Source code(tar.gz)
    Source code(zip)
  • 2.8.0(Nov 27, 2019)

  • 2.7.0(May 14, 2019)

  • 2.6.0(Apr 29, 2019)

    • Added: ClearCapable to clear the whole cache in one efficient call. Currently supported only by the Symfony HttpCache.
    • Allow installing with Httplug 2.0 / PSR-18.
    Source code(tar.gz)
    Source code(zip)
  • 2.5.4(Oct 31, 2018)

    Symfony HttpCache

    • Fixed: Avoid regression of 2.5.3: If there are no messages to be dispatched, do not throw an exception if the HttpCache is not set.
    Source code(tar.gz)
    Source code(zip)
  • 2.5.3(Oct 31, 2018)

    Symfony HttpCache

    • Fixed: Handle HttpCache not available in KernelDispatcher and fix return type annotations - if HttpCache is not set, it can't be returned.
    Source code(tar.gz)
    Source code(zip)
  • 2.5.2(Oct 3, 2018)

    Varnish

    • Fixed: Remove the xkey header in vcl_deliver if we are not in debug mode
    • Do not cleanup the Vary header and keep the user context hash if we are in debug mode

    Cache Tagging

    • Fixed: Clear the ResponseTagger after we tagged a response. Usually PHP uses a new instance for every request. But for example the hash lookup when using Symfony HttpCache does two requests in the same PHP process.
    Source code(tar.gz)
    Source code(zip)
  • 2.5.1(Sep 27, 2018)

  • 2.5.0(Sep 10, 2018)

    Cache Tagging

    • Added: MaxHeaderValueLengthFormatter to allow splitting cache tag headers into multiple headers.

    Symfony HttpCache

    • Have cache invalidator check for presence of Psr6Store for a better guess whether the cache really is TagCapable or not.
    Source code(tar.gz)
    Source code(zip)
  • 2.4.0(Jul 30, 2018)

    Symfony

    • Added: CleanupCacheTagsListener to remove the cache tags header from the final response that is sent to the client. Add this listener to your cache kernel.

    Tagging

    • Improved: The ResponseTagger does now remove duplicate tags.
    Source code(tar.gz)
    Source code(zip)
  • 2.3.1(Jul 6, 2018)

    Varnish

    • Fixed: Do not preg_quote tags when using xkey. Quoting is only used for BAN requests that expect a regular expression. This bug only affected you if you use xkey and used characters in your tags that are changed by preg_quote.
    Source code(tar.gz)
    Source code(zip)
  • 2.3.0(Apr 24, 2018)

    ProxyClient

    • The HttpProxyClient now accepts an instance of the new Dispatcher interface instead of the concrete HttpDispatcher, allowing for more flexibility.

    Symfony

    • Introduced a new KernelDispatcher for the Symfony proxy that calls the application kernel directly instead of executing a full HTTP request.
    Source code(tar.gz)
    Source code(zip)
  • 2.2.1(Mar 27, 2018)

  • 2.2.0(Mar 23, 2018)

    Varnish

    • Added support for the more efficient xkey cache tag system. BAN remains the default cache tagging system, but if you can install the varnish modules in your system, it is recommended to update to xkey.
    • No inline C is needed for the custom TTL header with Varnish 4 or better. use std.duration() instead.

    Symfony user context

    • You can now also specify which headers are used for authentication to detect anonymous requests. By default, the headers are the previously hardcoded Authorization, HTTP_AUTHORIZATION and PHP_AUTH_USER.
    Source code(tar.gz)
    Source code(zip)
  • 1.4.5(Mar 17, 2018)

    • Symfony user context: You can now also specify which headers are used for authentication to detect anonymous requests. By default, the headers are the previously hardcoded Authorization, HTTP_AUTHORIZATION and PHP_AUTH_USER.
    Source code(tar.gz)
    Source code(zip)
  • 2.1.3(Mar 15, 2018)

  • 2.1.2(Feb 7, 2018)

  • 2.1.1(Feb 1, 2018)

  • 2.1.0(Dec 19, 2017)

    • Support Symfony 4.

    Testing

    • Upgraded phpunit to 5.7 / 6. If you use anything from the FOS\HttpCache\Test namespace you need to update your project to use PHPUnit 6 (or 5.7, if you are using PHP 5.6).

    Symfony HttpCache

    • Cache tagging supprt for Symfony HttpCache

      Added a PurgeTagsListener for tag based invalidation with the Symfony HttpCache reverse caching proxy. This requires the newly created Toflar Psr6Store built on PSR-6 cache and supporting pruning expired cache entries.

    • Using Request::isMethodCacheable rather than Request::isMethodSafe to correctly handle OPTIONS and TRACE requests.

    Source code(tar.gz)
    Source code(zip)
A simple HTTP server behaving as proxy between webhooks and Appwrite Functions.

A simple HTTP server behaving as proxy between webhooks and Appwrite Functions, allowing for instance Stripe payments integration into Appwrite.

Matej Bačo 19 Sep 14, 2022
Integrate reCAPTCHA using async HTTP/2, making your app fast with a few lines.

ReCaptcha Integrate reCAPTCHA using async HTTP/2, making your app fast with a few lines. use Illuminate\Support\Facades\Route; Route::post('login', f

Laragear 11 Aug 23, 2022
Repman - PHP Repository Manager: packagist proxy and host for private packages

Repman - PHP Repository Manager Repman is a PHP repository manager. Main features: free and open source works as a proxy for packagist.org (speeds up

Repman 423 Sep 25, 2022
Proxy Judge coded In PHP

Simple proxy judge created in PHP What is a Proxy Judge ? A ProxyJuge is usually a PHP script that returns a subset of the environment variables of th

CHINO TECH TOOLS 3 Sep 6, 2021
Small Library to Serve Images in PHP in a Better Way (Resize, Compress) with Caching Support

A library for serving images and optimizing, changing their sizes, this library uses the caching feature, and in addition, it is very fast and small in size. In addition to these features, this library also optimizes images.

Developix 8 Aug 15, 2022
Proxy based Redis cluster solution supporting pipeline and scaling dynamically

Codis is a proxy based high performance Redis cluster solution written in Go. It is production-ready and widely used at wandoujia.com and many compani

null 12.7k Sep 21, 2022
Bitcoin Faucet integrated with banlist and VPN/Proxy Shield

Bitcoin Faucet integrated with banlist and VPN/Proxy Shield. It uses the service of Google reCaptcha (v2; box) and IPHub. Any claims will be saved in the account balance and can be withdrawn to ExpressCrypto, FaucetPay or directly using Block.io

null 1 Mar 25, 2022
Michael Pratt 304 Sep 20, 2022
CPAY is a sdk that encapsulates the Orange Money api with an intuitive syntax allowing you to integrate the Orange Money payment into your PHP project.

CPAY CHOCO PAY is a sdk that encapsulates the Orange Money api with an intuitive syntax allowing you to integrate the Orange Money payment into your P

faso-dev 1 Dec 23, 2021
Integrate Paytm Payment gateway to php website.

paytm-integration-kit-project-for-PHP Integrate Paytm Payment gateway to php website. Copy PaytmKit folder in document root of your server (like /var/

Er. Dipankar Mohanta 1 Feb 11, 2022
A PHP library to integrate with eWAY's Rapid Payment API.

A PHP library to integrate with eWAY's Rapid Payment API.

null 0 Jul 15, 2022
Simple HTTP smoke testing for your Symfony application

Shopsys HTTP Smoke Testing This package enables you to do simple HTTP smoke testing of your Symfony application. Basically, it generates a HTTP reques

Shopsys 65 Feb 3, 2022
A first party module to integrate Elastic App Search in Magento 2.

A first-party Magento integration for building excellent, relevant search experiences with Elastic App Search. ⚠️ This is a beta version of the client

elastic 25 Apr 22, 2022
The Laravel eCommerce ABA Payment Gateway module allows the admin to integrate the ABA payment gateway to the online store.

Introduction Bagisto ABA Payment Gateway. Requirements: Bagisto: v1.3.2. Installation with composer: Run the following command composer require bagist

Bagisto 3 May 31, 2022
With the help of the Laravel eCommerce CashU Payment Gateway, the admin can integrate the CashU payment method in the Bagisto store.

Introduction Bagisto CashU Payment add-on allow customers to pay for others using CashU payment gateway. Requirements: Bagisto: v1.3.2 Installation wi

Bagisto 2 Aug 22, 2022
A Magento 2 module to integrate Relay.

Magento Relay Installation First, install Relay as a PHP extension for your CLI and FPM environments. Next, install the Magento module: composer requi

CacheWerk 3 Jul 11, 2022
Plugin allowing to integrate marketing automation made by ActiveCampaign in Sylius.

The SyliusActiveCampaignPlugin takes care of creating and updating contacts, connections, orders, and abandoned carts on ActiveCampaign.

Webgriffe® 3 Apr 1, 2022
🏆 Learn You PHP! - An introduction to PHP's core features: i/o, http, arrays, exceptions and so on.

Learn You PHP! The very first PHP School workshop. A revolutionary new way to learn PHP Bring your imagination to life in an open learning eco-system

PHP School 312 Sep 22, 2022
Minimalist PHP frame for Core-Library, for Developing PHP application that gives you the full control of your application.

LazyPHP lightweight Pre-Made Frame for Core-library Install Run the below command in your terminal $ composer create-project ryzen/lazyphp my-first-pr

Ry-Zen 7 Aug 21, 2022