xkey promises to have vastly better performance than invalidation by BAN for two reasons:

• It keeps a hash table of the tags around so it can efficiently do invalidation directly instead of filling up a list of regex bans on request data slowing things down
• It supports soft purge making it able to take into account grace time

It can be installed on any supported platform by Varnish 4.1+ as long as people have src files for Varnish, and is also available on some Linux distros as package. And besides that, it can be used with Varnish Plus.

Further reading:

• Doc: https://github.com/varnish/varnish-modules/blob/master/docs/vmod_xkey.rst
• Wikipedia: https://wikitech.wikimedia.org/wiki/XKey

Doc draft

Configuring xkey:

[
    'tags_header' => 'xkey-softpurge',// Or 'xkey-purge' if you can not use grace, see VCL
    'tag_mode' => 'purgekeys',
]

Todo:

• [x] Compile xkey on travis
  • [x] ~~Figure out why the (added) Varnish 5.2 testing job fails~~ (move 5.2 support to other pr)
• [x] Add unit test coverage
• [x] Add functional test coverage

Currently this library contains several classes related to extending the Symfony HTTP Cache by way of an EventDispatchingKernel trait.

There is a general need to implement tag invalidation with the Symfony HTTP cache, and this has been done here.

Most of the logic in that package is concerned with tags and @magnusnordlander is proposing to create a new package which is dedicated to tag storage. Therefore the amount code required for adding the tagging functionality to the HTTP cache is not very large.

We could therefore reasonably add the tagging functionality in this package, however I am not so sure that it belongs here -

• I believe all of the functionality we add to the Symfony HTTP Cache can be used without the infrastructure of the FOSHttpCache.
• I think this bundle should integrate with third-party caches and not provide solutions itself.

I therefore propose to create a new package containing to the Symfony HTTP cache and ultimately remove the SymfonyCache namespace from this package.

wdyt?

Aims to allow support for tagging also on Symfony HTTP Cache and potentially other caches by limiting the feature to focus on only tags and not random headers.

Closes #234

• [x] explain changes in CHANGELOG.md
• [x] Update documentation

the current tag handling is all about sending tags with the response and sending tag specific requests to the caching proxy. there is another concept for tagging: storing tag => urls ourselves (database, sqlite, redis, whatever) and send normal refresh / ban requests on invalidation. (see also the symfony con talk by @magnusnordlander)

how can we model this set up? should we add an isInline check to TagsInterface and a saveTags($requestUrl) in ResponseTagger? maybe rather have an InlineTagsInterface for the get header name and get header value methods? or should we try to hide this away and magically save tags in ResponseTagger on getTagsHeaderValue?

the TagsInterface::invalidateTags looks fine to me. but the clients (Varnish/Nginx/Symfony) should maybe use composition for the functionality rather than implement the method inline, so you can plug either one that calls ban() on the client or the one that calls purge / refresh based on its stored tag to url map.

Fix #53.

Goals:

• Completely decouple our caching proxy clients from any HTTP client implementation.
• Remove dependency on Guzzle (or any other) HTTP client.
• Depend only on PSR-7 and the generic php-http/adapter instead.

Todo:

• [x] update docs to reflect changes
• [x] fix missing host test
• [x] Switch RTD default to stable, so users won't be confused by our updated (2.0) docs

replace #373, fix #286 squashed all commits and did some tweaks to the documentation.

TODO

• [x] Once Symfony 3.4 is out: remove the @beta in composer.json

=> Bundle support: https://github.com/FriendsOfSymfony/FOSHttpCacheBundle/issues/403

When using ban requests with Varnish proxy client, requests are queued and sent with flush(). Queued requests are registered without host, making Guzzle create an empty Host header.

When flushing, Guzzle request is re-created starting with the original one (so without host), with all registered headers. The problem is that the empty Host header is also copied, and thus not re-created by Guzzle with the server URL. In the backend, cURL sees it and takes it into account, overriding passed URL silently.

This causes requests with empty host being fired by Guzzle, which is kind of annoying :smiley:.

This patch ensures Host header is removed before sending the real request.

Not sure if it is a bug in FOSHttpCache, eZPublish or custom code. The observed behaviour (no varnish, http cache on, ezp 5.4.5):

• user accesses the site
• gets redirected to /login
• fills in the form, posts to /login_check
• gets redirected to /_fos_user_context_hash

Since I fixed this by setting:

security: firewalls: ezpublish_front: form_login: always_use_default_target_path: true

I suspect that what goes on here is that the internal request made by Sf to retrieve the user-hash gets somehow 'remember' by the session as the last url visited by the end user just before logging in.

Any tips on fixing this in a more flexible way? Or is this really a bug?

Hey,

Just raise this issue as after looking more in varnish, it seems that @dbu was right about cookies, and we can have an another way that simply remove it in user context configuration.

In fact, varnish has a default vcl which is compiled in its core, available here: (for varnish 3) https://www.varnish-cache.org/trac/browser/bin/varnishd/default.vcl?rev=3.0

So we basically have 2 methods for dealing with cookies in user-context, and i'm wondering what method do you think it's best to show / test :

• Keep remove / insert cookies when needed
• Force the passage to lookup in our conditions

sub vcl_recv {
    // Handle the original request: send a HEAD request to retrieve the user hash
    if (req.restarts == 0 && req.http.cookie && (req.request == "GET" || req.request == "HEAD")) {
        set req.http.x-original-url     = req.url;
        set req.http.x-original-request = req.request;

        # Retrieve a unique session id from the cookie
        set req.http.x-session-id = req.http.cookie;

        set req.url     = "/user_context_head.php";
        set req.request = "HEAD";

        # Force varnish to lookup in this case even with cookies
        return (lookup);
    }

    // After the HEAD request, reset the request to the original one, which
    // will be restarted in vcl_deliver.
    if (req.restarts > 0 && req.http.X-Original-Method) {
        set req.request = req.http.x-original-method;
        set req.url     = req.http.x-original-url;

        unset req.http.req.http.x-original-method;
        unset req.http.req.http.x-original-url;

        # Force varnish to lookup in this case even with cookies
        return (lookup);
    }
}

I think it's better than remove / insert cookies which is more a hack.

WIP for Open Source Litespeed (OLS) and possibly the commercial Litespeed Web Server (LSWS).

Cache tagging is only implemented in OLS but not at all in LSWS. The changes worked with OLS but the latest OLS version seems to have done BC breaks / regressions with cache tagging.

The original author does not continue on this - if you need this, you will need to wrap up this merge request or pay someone to do it...

This is a possible implementation for LSWS (https://www.litespeedtech.com/products/litespeed-web-server) support.

Closes #403

Changes:

• Use batch tag purging to reduce risk of hitting API limits
• Support PurgeCapable
• Drop multi services, does not seems to be a good fit for Purge, & unsure what the use case is
• Added unit testing

Follow Up topics:

• Add support in Bundle
• Abstract Reverse proxy TTL to use Surrogate Control on Fastly and optionally others #455

Context of this:

• eZ Platform (v3) is being worked on by @ViniTou to remove several of our HttpCache abstractions and move to FosHttpCache 2.x, which means we among other things we'd like Fastly support on FosHttpCache level instead ;)

Recent use case popped up that we have been thinking about some years now is to add support for several context hashes. In order to allow our end users to deal with personalized responses when there is a use case to still allow it to be cached.

Context

We use X-User-Context-Hash for user permissions, and if some of our end users extend that to add additional things to the hash, lets say profile choices or other personalized user info you'll end up with:

• A lot of cache variants for ALL responses, consuming lots of additional memory
• Very low cache hit ratio

Solution

Support for several kind of hashes, each with own Vary header, so only responses that truly need to vary on e.g. X-User-Profile-Hash will actually do so (typically in addition to varying X-User-Context-Hash).

VCL would probably need to be adapted for this, unless we have a convention we can match in VCL and in PHP (SymfonyCache).

Would it make sense to switch to Symfony HttpClient to reduce setup time with the lib/bundle?

Technically library would only need to rely on symfony/http-client-contracts OR just PSR-18 as proposed in #443 if that covers all our needs. And furthermore rely on symfony/http-client as dev dependency, while bundle maybe relying and wiring that up by default.

As discussed in the original Fastly PR in order to generalize TTL handling across backends like Varnish, Fastly, Symfony, it could be an idea to:

1. abstract it on client side?
2. to be able to emit Surrogate-Control for fastly
3. ~If feasible look into also emit Surrogate-Control on Varnish instead of X-Reverse-Proxy-TTL, VCL could still handle for X-Reverse-Proxy-TTL for BC.~

https://www.w3.org/TR/edge-arch/

1. FOS\HttpCache\Tests\Unit\CacheInvalidatorTest::testMethodException This test did not perform any assertions

2. FOS\HttpCache\Tests\Unit\ProxyClient\HttpDispatcherTest::testBanWithoutBaseUri This test did not perform any assertions

3. FOS\HttpCache\Tests\Unit\Test\Proxy\SymfonyProxyTest::testStart This test did not perform any assertions

Fix #61.

1. The idea is that using this client involves less custom VCL. However, for the ban lurker to work, some custom VCL is still necessary to copy e.g. the URL header to X-Url.
2. As far as I know, purge cannot be done through varnishadm, so we should either not implement PurgeInterface or fake the purge by actually doing a ban when purge() is called.
3. Same goes for refresh. We could fake it by doing a ban and then manually getting a fresh response from the application.