I get the following error when I try to write an uploaded file my Amazon S3 bucket

Could not write the "2014/04/24/53596551ee0ec.m4a" key content." at .../vendor/knplabs/gaufrette/src/Gaufrette/Filesystem.php line 113

Currently, the AzureBlobStorage adapter requires a single container name which will be filesystem root, it might try to create it.

If I'm using a single storage account to hold only one type of asset (say, only images), I'd like to use one container per image (which will hold the original and all the variants), much like for example Azure Media Services does. This also seems to be a recommended way of doing things.

What could be done here is add an additional flag / mode which would take the storage key and use the part until the first slash, create a container (if it doesn't already exist) and use the rest of the path as key. Ie.

write('my/image/path.jpeg') // "my" = container, "image/path.jpeg" = key

Some sort of container default creation options should also be provided (for example, access mode), but I think it shouldn't be all that hard.

I get the following error when using Gaufrette in a Symfony 2.4 environment when i do a cac:clear:

Cannot use phpseclib\Net\SFTP as SFTP because the name is already in use in ../knplabs/gaufrette/src/Gaufrette/Adapter/PhpseclibSftp.php on line 6

I'm using the following versions:

knplabs/gaufrette | 0.2.1 knplabs/knp-gaufrette-bundle | 0.2.0

as proposed in #118 . Filesystem now checks if adapter has listKeys method (no need to implement it for all adapters, imho, since filesystem/ftp traversing is not subject to optimization), gridfs implements listKeys via MongoRegex.

I also added a typehint to FunctionTest $filesystem property.

@l3l0 , please check it this PR complies to your thoughts about Gaufrette. If it's ok, I'll propapbly implement it also for doctrine via LIKE (is also should be applicable to S3 / Rackspace, but I've no posibillity to test that).

Hi again

My colleague @mtarvainen has now finished an initial version of MogileFS adapter. Would you be interested in pulling that in?

I know tests are still missing but we're going to make them 'soon'. Anyway, I was told that at least the basic functionality should be ok.

Stacktrace (most recent call first):

  File "/home/foobar/vendor/knplabs/gaufrette/src/Gaufrette/StreamWrapper.php", line 72, in handleError
    return @stream_wrapper_unregister($scheme);
  File "/home/foobar/vendor/knplabs/gaufrette/src/Gaufrette/StreamWrapper.php", line 72, in streamWrapperUnregister
    return @stream_wrapper_unregister($scheme);
  File "/home/foobar/vendor/knplabs/gaufrette/src/Gaufrette/StreamWrapper.php", line 48, in register
    static::streamWrapperUnregister($scheme);
  File "/home/foobar/vendor/knplabs/knp-gaufrette-bundle/Knp/Bundle/GaufretteBundle/KnpGaufretteBundle.php", line 25, in boot
    StreamWrapper::register($this->container->getParameter('knp_gaufrette.stream_wrapper.protocol'));
  File "/home/foobar/app/bootstrap.php.cache", line 2190, in boot
    $bundle->boot();
  File "/home/foobar/app/AppKernel.php", line 185, in boot
    parent::boot();
  File "/home/foobar/vendor/symfony/symfony/src/Symfony/Bundle/FrameworkBundle/HttpCache/HttpCache.php", line 56, in forward
    $this->getKernel()->boot();
  File "/home/foobar/vendor/symfony/symfony/src/Symfony/Component/HttpKernel/HttpCache/HttpCache.php", line 430, in fetch
    $response = $this->forward($subRequest, $catch);
  File "/home/foobar/vendor/symfony/symfony/src/Symfony/Component/HttpKernel/HttpCache/HttpCache.php", line 330, in lookup
    return $this->fetch($request, $catch);
  File "/home/foobar/vendor/symfony/symfony/src/Symfony/Component/HttpKernel/HttpCache/HttpCache.php", line 194, in handle
    $response = $this->lookup($request, $catch);
  File "/home/foobar/web/app.php", line 13, in null
    $response = $kernel->handle($request);

Ok, here's a new one. I think I managed to do this with a lot less BCBs, maybe even none.

Here's what I did:

• Added File folder that contains storage specific file objects. Some adapters may not need them so they can use File class directly but some adapters definitely benefit from customized File object.
• Removed metadata related code from adapters as it's a feature of a file (not BCB because it didn't work before)
• Added get function into adapters for retrieving the file object, altered logic in Filesystem::get() to support this.
• Changed the meaning of createFile. Now it's purely a factory method for creating a new file object.
• Added writeFile function into adapters for writing files
• Added metadata as optional parameter for write function. (I think this really must be here to enable any kind of metadata support for old write function. It does not break BC because it's optional.)
• Moved all interfaces to root folder from Adapter folder. (This is not mandatory, but I really think it's nice to reserve Adapter folder only for adapters, not interfaces)
• Changed MetadataSupporter to contain method isMetadataKeyAllowed($metaKey);

This is how it can be used now:

//Object-oriented write
$f = $filesystem->createFile($key);
$f->setMetadataItem('metaKey1', 'metaValue1');
$f->setMetadataItem('metaKey2', 'metaValue2');
$f->setContent($fileContentBytes);
$f->setName('myfile.jpg'); //human-readable name
$f = $filesystem->writeFile($f, [$overwrite]);

//Object-oriented get, returns file object
$g = $filesystem->get($key);

//Procedural write
$filesystem->write($key, $content, [$overwrite], [$metadata]);

//Procedural read (does not support metadata)
$filesystem->read($key)

//Delete
$filesystem->delete($key);

And finally, as before, I only implemented GridFS and Local as a proof of concept. More work has to be done before merging in any case...

This is a modification of AwsS3 adapter to allow multipart uploads when files are bigger than a certain size.

Size limits / part size are passed as options to the adapter and set to defaults, respectively 5Go and 5Mo. 5Go being the maximum size AWS S3 accepts in a single upload, and 5Mo the minimum part size.

Limitations : won't work with string content, you have to pass a resource handle otherwise write() will return false. We may use InMemoryStream to bypass this limitation ?

PHP7.1 fails on docker image, not sure why. May need some help on this.

That is happens because Gaufrette\Util\Size::fromContent uses mb_strlen and InMemoryBuffer stream uses not multi-byte string manipulations so the numbers are different in case of binary data.

Changing mb_strlen to strlen fixed issue but I didn't test it with other adapters.

There is a serious bug in cache adapter - if source has failed to write or to rename a file (eg. S3 invalid key in my case) following happens:

• native aws exceptions is suppressed with catch return false
• return false by source silently ignored
• cache has been written
• Filesystem reports write/rename was successful
• FAIL, file does not exist

Gaufrette\Adapter\Cache::rename()

Gaufrette\Adapter\Cache::write()

So in my application the user uploads images and Im using Gaufrette to store it either locally in my staging environment or on S3 in production. i was thinking that there must be a way to access the direct URL for a file. whether it's local, S3, Dropbox ... etc. I know this won't be supported in all adapters but it'd be great if its exists in adapters that can support it.

See https://github.com/phpspec/phpspec/pull/1435 for a similar update and for the reason why ci isn't successfully yet, as phpspec does't support 8.2 yet.

Added class FlysystemV2V3 - can be used on both versions.

• Much of time spent to fix analizers.
• Psalm upgraded to 4.30.0 to use new plugin system
• CI flow divided to test and analyse different versions of flysystem

Inspired on #650

Hi,

I use Gaufrette with GoogleCloudStorage adapter in production and local filesystem in development and I observe differences on the returned type of getContent() method of File object.

getContent calls Filesystem->read() which calls adaptater read() method.

Returned type of File->getContent() should be a string. But, with GoogleCloudStorage adapter, the read() method return a StreamInterface ($response->getBody()) when Google_Http_Request does not exist (view here).

I suppose this line should be fixed with $response->getBody()->getContents()

Thx in advance.

Hello,

Problem: There is no easy way to add tags to S3 objects during upload :worried:

Solution: I facilitated tag adding by transforming an array of tags into a query string (format supported by AWS). :smile:

Ex:

• Input: ['public' => 'yes', 'private' => 'no']
• Output: public=yes&private=no

Can you tag the person who is in charge of AWS S3 ? I can't find him (his account no longer exist). :exclamation:

@Nyholm

Have a great day!