In order to use the single table pattern of DynamoDB it would be nice to customize the separator to fit different use cases. This PR adds that possibility by allowing to define that option.

When waiting on multiple Results over Result::wait() we randomly got following error:

Fatal error: Uncaught Error: Generator passed to yield from was aborted without proper return and is unable to continue in /var/www/html/vendor/async-aws/core/src/Stream/IterableStream.php:51
Stack trace:
#0 /var/www/html/vendor/symfony/http-client/HttpClientTrait.php(310): AsyncAws\Core\Stream\IterableStream->getIterator()
#1 [internal function]: Symfony\Component\HttpClient\CurlHttpClient::Symfony\Component\HttpClient\{closure}(Object(AsyncAws\Core\Stream\IterableStream))
#2 /var/www/html/vendor/symfony/http-client/HttpClientTrait.php(292): Generator->valid()
#3 /var/www/html/vendor/symfony/http-client/CurlHttpClient.php(462): Symfony\Component\HttpClient\CurlHttpClient::Symfony\Component\HttpClient\{closure}(65536)
#4 /var/www/html/vendor/symfony/http-client/CurlHttpClient.php(247): Symfony\Component\HttpClient\CurlHttpClient::readRequestBody(65536, Object(Closure), '', false)
#5 [internal function]: Symfony\Component\HttpClient\CurlHttpClient::Symfony\Component\HttpClient\{closure}(Resource id #8604, NULL, 65536)
#6 / in /var/www/html/vendor/async-aws/core/src/Stream/IterableStream.php on line 51

The results that are being waited on are a list of PutObjectOutput.

We aren't able to reproduce the error reliably and don't know what may trigger that error.

Supporting Laravel would be very beneficial. Bref and Vapor (serverless) would be great use cases for lowering the memory footprint. As Laravel has a driver based approach, it could be a separate composer package that exposes the async-aws based drivers.

I had a look at the Framework, and these were the places I found relevant

SQS:

• Connector: https://github.com/laravel/framework/blob/master/src/Illuminate/Queue/Connectors/SqsConnector.php
• Job: https://github.com/laravel/framework/blob/master/src/Illuminate/Queue/Jobs/SqsJob.php
• Queue: https://github.com/laravel/framework/blob/master/src/Illuminate/Queue/SqsQueue.php

SES:

• https://github.com/laravel/framework/blob/master/src/Illuminate/Mail/Transport/SesTransport.php

Dynamo:

• https://github.com/laravel/framework/blob/master/src/Illuminate/Cache/DynamoDbStore.php

S3:

So Laravel supports Flysystem out of the box, we only need a asyncs3 driver

• https://github.com/laravel/framework/blob/master/src/Illuminate/Filesystem/FilesystemManager.php#L203-L213

Hi great library, i am trying to add rekognition service to async-aws, i encounter a problem with the method SearchFacesByImage the problem is the next:

`In RestJsonSerializer.php line 158:

Type float is not yet implemented `

I am unable to track the issue, i am a newbie in OSS but want to get this done, Thanks for the help i will start adding the test in the meanwhile

PutObject is limited to 5GB. Bigger objects (up to 5TB) require using a multipart upload. The official SDK has a upload() method, which handles this transparently (by default, it will switch to multipart much earlier than the 5GB threshold, probably because they found it more efficient or more reliable to upload in chunks). Does it make sense to provide the same kind of feature in this SDK ?

I'm running a Symfony app in EKS, and I'm using SQS with Messenger + this lib.

I'm having small clusters of failures at long intervals with 403 responses from the SQS APIs:

HTTP 403 returned for "https://sqs.eu-central-1.amazonaws.com/".

Code:    AccessDenied
Message: Access to the resource https://sqs.eu-central-1.amazonaws.com/ is denied.
Type:    Sender
Detail:

I've dig into the logs, and I've found this warning:

php.WARNING: Warning: file_get_contents(/var/run/secrets/eks.amazonaws.com/serviceaccount/token): failed to open stream: No such file or directory {"exception":"[object] (ErrorException(code: 0): Warning: file_get_contents(/var/run/secrets/eks.amazonaws.com/serviceaccount/token): failed to open stream: No such file or directory at /var/www/my-project/vendor/async-aws/core/src/Credentials/WebIdentityProvider.php:92)"} []

So I've traced down the issue to something that happened on the official SDK too: https://github.com/aws/aws-sdk-php/issues/2014

Basically, it's a combination of:

• tokens are short lived
• EKS replaces tokens on-the-fly
• PHP's realpath cache gets in the way...

... so you get brief intervals in which you try to read the old, now missing token.

In the offical SDK, the issue was mitigated by clearing the cache in a precise and ordered way of the following paths:

/var/run/secrets/eks.amazonaws.com/serviceaccount/..data/token
/var/run/secrets/eks.amazonaws.com/serviceaccount/..data
/var/run/secrets/eks.amazonaws.com/serviceaccount/token

in this PR: https://github.com/aws/aws-sdk-php/pull/2043

Can we apply the same logic here too?

Since the session handler was already done and is working great I thought I try something different. 😉

I implemented the rds data service. I would like to use it in my dbal-rds-data library... in fact I already tried it and it seems to work fine in a symfony-doctrine-orm project.

A few notes:

• I really disliked the idea of having every single value wrapped in an object. With the rds data api it isn't unlikely to have hundreds of thousands of values. And it's also a bit more difficult to read the values with value objects... and also the code generator had problems with a recursive schema so I decided to adjust the generator to allow to exclude types from being converted to value objects which can be defined in manifest.json. These values will just be passed though.
• the BatchExecuteStatementRequest had array of arrays of objects in the schema for parameterSets. I had to implement that in the generator and I haven't checked if all metadata for it is correct yet... and it might not be perfectly thought though so... yeah, would be nice if someone could take a look that knows the generator better. If not than I would just remove that method from the current api since it isn't really necessary and I'm not using that method.
• I had an unexpected string instead of an int in ExecuteStatementResponseTest.php:72 with numberOfRecordsUpdated. I haven't yet searched why yet.
• I haven't implemented the ExecuteSql api call since it is deprecated.
• I haven't looked at the integration test yet
• The generator did not populate the endpoints. I also haven't yet looked why and just manually did that.

So since there are more points than I expected, I'm now in the awkward position of having an unfinished surprise feature and having to ask if you actually want to have it.

So: am I on the right track? Is this an api you would merge? I would try to address some of the points I had this weekend then.

NB The generator command was unable to generate DescribeStream:

In StructureShape.php line 29:
                                                          
  [InvalidArgumentException]                              
  The member "StreamDescription.Shards" does not exists.  
                                                          

Exception trace:
  at /data/src/CodeGenerator/src/Definition/StructureShape.php:29
 AsyncAws\CodeGenerator\Definition\StructureShape->getMember() at /data/src/CodeGenerator/src/Generator/PaginationGenerator.php:105
 AsyncAws\CodeGenerator\Generator\PaginationGenerator->generateOutputPagination() at /data/src/CodeGenerator/src/Generator/PaginationGenerator.php:74
 AsyncAws\CodeGenerator\Generator\PaginationGenerator->generate() at /data/src/CodeGenerator/src/Generator/OperationGenerator.php:122
 AsyncAws\CodeGenerator\Generator\OperationGenerator->generate() at /data/src/CodeGenerator/src/Command/GenerateCommand.php:269
 AsyncAws\CodeGenerator\Command\GenerateCommand->generateService() at /data/src/CodeGenerator/src/Command/GenerateCommand.php:147
 AsyncAws\CodeGenerator\Command\GenerateCommand->generateServicesSequential() at /data/src/CodeGenerator/src/Command/GenerateCommand.php:84
 AsyncAws\CodeGenerator\Command\GenerateCommand->execute() at /data/vendor/symfony/console/Command/Command.php:256
 Symfony\Component\Console\Command\Command->run() at /data/vendor/symfony/console/Application.php:971
 Symfony\Component\Console\Application->doRunCommand() at /data/vendor/symfony/console/Application.php:290
 Symfony\Component\Console\Application->doRun() at /data/vendor/symfony/console/Application.php:166
 Symfony\Component\Console\Application->run() at /data/src/CodeGenerator/generate.php:20
 require() at /data/generate:32

Hi, I start using this package instead of the official AWS SDK over a week ago, and since then I have been having connection issues, while new sending messages to SQS.

I run a serverless setup: "SQS => Lambda trigger" and execute Symfony Messenger with bref/symfony-messenger During the handling of the messages, new messages are pushed to SQS. Very occasionally this fails, but I dont understand why. The only difference I can think of with the AWS SDK is that it implements an retry schema.

Does anyone else experience these occasional failures? Some of the errors messages I extracted:

• fopen(https://52.95.120.176/): failed to open stream: Connection timed out
• fopen(https://52.95.112.152/): failed to open stream: HTTP request failed!
• fopen(): SSL: Handshake timed out
• Server returned nothing (no headers, no data) for "https://sqs.eu-west-1.amazonaws.com/".

So I was considering, would it be possible/wise/useful to create a Retry middleware of some sort, that would handle these failures, and retry if appropriate? The HttpClient should have some support for middlewares according to https://github.com/symfony/symfony/pull/36779 The retry mechanism could mimimic eg https://github.com/aws/aws-sdk-php/blob/master/src/RetryMiddleware.php

I'm trying to use a different endpoint for my local DynamoDB docker http://dynamo:8000 but the endpoint for the local region is hardcoded to http://localhost:8000.

Also, it should work with any credentials, but I'm getting:

Request must contain either a valid (registered) AWS access key ID or X.509 certificate.

I just get this error message:

In AsyncResponse.php line 438:
                                                                  
  A chunk passthru cannot yield more than one "isFirst()" chunk.  

It means that our Response opens the buffer twice, ie the first chunk is seen twice.

I think it is related to https://github.com/symfony/symfony/pull/43537

Can we get a tagged release of async-aws/iam 1.4.0 please?

Also, the 1.3.0 tag appears to point to the wrong commit. It currently points to 074ecc2a, which doesn't include any of the changes made in 4568b205. Is it possible / pain-free to re-tag, or will 1.3.1 need to be added instead? At the moment I'm requiring dev-master as a workaround

Thanks :)

Hi. I'm using async-aws with localstack to mock requests and I have a problem with putObject. Here you can find my PHP and JavaScript code to check if it works -> https://gist.github.com/dudekm/807f94dcc37a51adf43ec9e8b3652238

PHP + ASYNC-AWS + localstack - doens't work I can list buckets and files - everything works perfectly. Logs from localstack: localstack_demo | 2022-11-08T06:54:51.842 INFO --- [ asgi_gw_0] localstack.services.infra : Starting mock S3 service on http port 4566 ... localstack_demo | 2022-11-08T06:58:24.996 INFO --- [ asgi_gw_0] localstack.request.aws : AWS s3.ListBuckets => 200 localstack_demo | 2022-11-08T06:58:25.008 INFO --- [ asgi_gw_0] localstack.request.aws : AWS s3.ListObjectsV2 => 200

But when I want to use putObject, it's not possible because of "Could not contact remote server" and in localstack I can't see any request related to putObject.

JavaScript + AWS SDK + localstack - works I decided to test if JavaScript aws-sdk will work. The upload works perfectly.

localstack_demo | 2022-11-08T06:58:22.476 INFO --- [ asgi_gw_0] localstack.request.aws : AWS s3.PutObject => 200

For sure it's not a problem with localstack (for JS sdk everything is fine). So, looks like async-aws has a bug or my config for async-aws is not correct. But, is it possible that config is not correct only for putObject???

I am trying to copy a list of ~50 files from one bucket on one region to a bucket on the region I'm using, and I'm running into issues where random large-ish files (~270MB) are failing to copy. Sometimes they will copy, but most times they won't. I get no errors.

// ...

// $config is data imported from a yaml file
$cred = $config['credentials'];
$bucket = $cred['bucket'];
$destination = 'my-bucket';

$s3 = new S3Client([
    'accessKeyId' => $cred['access_key_id'],
    'accessKeySecret' => $cred['access_key_secret'],
    'region' => $config['local_region']
]);
$extS3 = new S3Client([
    'accessKeyId' => $cred['access_key_id'],
    'accessKeySecret' => $cred['access_key_secret'],
    'region' => $cred['region'] ?? $config['local_region']
]);

$result = $extS3->listObjectsV2([
    'Bucket' => $bucket
]);
$generator = $result->getIterator();

foreach ($generator as $g) {
    $key = $g->getKey();
    
    $s3->copyObject([
        'Key' => $key,
        'Bucket' => $destination,
        'CopySource' => "$bucket/$key",
        'region' => $config['local_region']
    ]);
}

For most files this works just fine, but occasionally it just silently fails to transfer files. If I use Amazon's official API it works just fine, but I'd rather use this one if I can. I've tried using ->resolve() with a high timeout, but it doesn't seem to make any difference. Is there something else I should be doing?

This is a continuation of https://github.com/async-aws/aws/issues/756

What changed since then? https://twitter.com/nicolasgrekas/status/1583454980769730560

https://github.com/FriendsOfPHP/well-known-implementations has been created which matches this use case. We can have both worlds:

• This package doesn't have to require symfony/http-client
• If consumer doesn't have http-client, it will be installed

PHP Fatal error: Uncaught TypeError: AsyncAws\Sqs\Result\ReceiveMessageResult::getMessages(): Return value must be of type array, null returned in /srv/app/vendor/async-aws/sqs/src/Result/ReceiveMessageResult.php:28