Hello!

we recently switched on our EC2 instances from the v2 to v3 (most recent version) of the PHP SDK. We use SWF, DynamoDB and S3. Since we switched to the v3, we are facing cURL errors that appear completely randomly it seems:

The application has thrown an exception! Aws\Swf\Exception\SwfException Error executing "PollForActivityTask" on "https://swf.us-east-1.amazonaws.com"; AWS HTTP error: cURL error 56: SSL read: error:00000000:lib(0):func(0):reason(0), errno 104 (see http://curl.haxx.se/libcurl/c/libcurl-errors.html)

Sometimes the exception appears twice per hour, sometimes once in three hours... there's no scheme.

We already switched back to an older AMI and updated cURL from 7.35.0 (there is a problem with chunked upload in the 7.35) to 7.36.0 but nothing helps. We are polling SWF for activity tasks with long polling enabled, so there shouldn't be too many requests.

I googled this of course before opening an issue, but the only topics I found either date from 2012-2013 or were related to a broken load balancer on AWS. And that would probably be too much of a coincidence in our case...

Does anyone know about this problem..? I can't even say if it's a problem in the SDK or something else. So I'm grateful for every hint you can give me!

Thanks a lot! Pascal

Hi!

Our company only uses the S3 and SQS libraries, however, in order to have these installed via Composer, the entire AWS package needs to be downloaded, inflating Composer's autoload files. Would it perhaps be possible to split the AWS SDK into separately installable components (e.g. composer require aws/s3-php-dk) which would then have some requirements of their own (e.g. the s3 package would require aws/common-php)?

My coworker and I downloaded 2.8.2, then 2.7.0, and 2.6.16. None of the phars for these versions worked, and spat out this error:

[Thu Apr 30 15:50:21.909040 2015] [:error] [pid 2621] [client ] PHP Warning: require(phar://aws.phar/aws-autoloader.php): failed to open stream: phar error: invalid url or non-existent phar "phar://aws.phar/aws-autoloader.php" in /usr/local/lib/php/aws.phar on line 3, referer: https:// [Thu Apr 30 15:50:21.909127 2015] [:error] [pid 2621] [client ] PHP Fatal error: require(): Failed opening required 'phar://aws.phar/aws-autoloader.php' (include_path='.:/usr/share/php:/usr/share/pear:/usr/local/lib/php:/usr/local/lib/php/db:/var/www') in /usr/local/lib/php/aws.phar on line 3, referer: https://

We then went up from 2.6 to 2.6.9, 2.6.12, 2.16.14, and 2.6.15, at that point realizing that 2.6.15 is the most up-to-date version of the .phar we can use. We are considering the use of Composer in the future (which we are actually already using), but wanted to report our issues with these phar versions.

Is this related to a problem?

I want to use USER_SRP_AUTH for InitiateAuth in my application to log the user's device info into Cognito. But I need to pass "SRP_A" as AuthParameters in the request. And we don't have any method in SDK to generate "SRP_A" in the current SDK.

Feature description

It is better if we have the method to generate "SRP_A" in SDK so we can generate it easily.

Confirm by changing [ ] to [x] below to ensure that it's a bug:

• [X] I've gone though Developer Guide and API reference
• [X] I've checked AWS Forums and StackOverflow for answers
• [X] I've searched for previous similar issues and didn't find any solution

Describe the bug When doing MultiPartCopy of large files (50GB+), failing parts can result in corrupted files at the destination. The files at the destination will be significantly larger than the original file (sometimes close to double the size).

While I've seen several transfers succeed despite failures, this does not always appear to be the case. From my perspective, I'm informed of a large file being corrupted in a downstream service and in every one of those cases, checking the logs, the MultiPartCopy had some kind of error that it had to recover from. That said, there are other instances of error recoveries where the outputs were fine/not corrupted.

A couple of the error types that have triggered this are:

• AWS HTTP error: cURL error 52: Empty reply from server
• AWS HTTP error: Server error: (500 response code along with "We encountered an internal error. Please try again.")

` 2020-07-02T02:42:45.056396-07:00: event=multipartUploadFailed level=warning attempts=0 error="An exception occurred while uploading parts to a multipart upload. The following parts had errors:

• Part 2318: Error executing "UploadPartCopy" on "https://"; AWS HTTP error: Server error: PUT https://<removed> resulted in a 500 Internal Server Error response: InternalErrorWe encountered an internal error. Please try again.removed (truncated...) InternalError (server): We encountered an internal error. Please try again. - InternalErrorWe encountered an internal error. Please try again.removedremoved
• Part 6606: Error executing "UploadPartCopy" on "https://"; AWS HTTP error: Server error: PUT https://<removed> resulted in a 500 Internal Server Error response: InternalErrorWe encountered an internal error. Please try again.2M9HER (truncated...) --

2020-07-02T07:30:04.365831-07:00: event=multipartUploadFailed level=warning attempts=0 error="An exception occurred while uploading parts to a multipart upload. The following parts had errors:

• Part 49: Error executing "UploadPartCopy" on "https://"; AWS HTTP error: cURL error 52: Empty reply from server (see https://curl.haxx.se/libcurl/c/libcurl-errors.html) " `

The code I'm using is similar to the sample code provided that has a try/catch within a do/while in order to recover from these kinds of failures. My only real change is adding a maximum number of attempts to retry.

Note that, according to my log, there is never more than one retry needed/attempted -- the transfer behaves as if it was successful and it's not until further downstream in our workflow that it's determined that the file transfer resulted in a corrupted file. My workaround for this will be to do a file size check after a successful copy and confirm there's a match, and restart the entire copy if not. This is obviously not ideal because some of these files will be 300-500GB and will waste several hours.

`

    $this->s3 = new S3Client([
        'version' => 'latest',
        'region'  => $_ENV['AWS_REGION'],     // this is us-east-1 for this instance
        'retries'  => 3,
        'credentials' => new Credentials(<parameters removed>)
    ]);

    // NOTE -- this is not used by the copier, but I'm including it in case the context matters
    $this->s3->registerStreamWrapper();

    $partNumber = 0;
    $percentComplete = 0;
    $partSize = 536870912; // 512 MiB
    if (isset($_ENV['AWS_S3_MULTIPART_PARTSIZE']) === true && intval($_ENV['AWS_S3_MULTIPART_PARTSIZE']) > 536870912) {
        $partSize = intval($_ENV['AWS_S3_MULTIPART_PARTSIZE']);
    }

    $copier = new MultipartCopy(
        $this->s3,
        $sourceBucket.'/'.$sourceKey,
        [
            'bucket' => $destinationBucket,
            'key' => $destinationKey,
            'part_size' => $partSize,
            'before_upload' => function () use ($bytesToCopy, $partSize, &$partNumber, &$asset) {
                $percentComplete = intval(100.0 * round((float) ($partNumber * $partSize) / (float) $bytesToCopy, 2));

                $this->logger->log([
                    'event' => 'debug_moveS3Object',
                    'level' => 'debug',
                    'assetId' => $asset->getId(),
                    'bytesToCopy' => $bytesToCopy,
                    'percentComplete' => $percentComplete
                ]);

                $asset->setCopyProgress(intval($percentComplete));
                $this->entityManager->flush();

                $partNumber++;
            }
        ]
    );

    $attempts = 0;
    $errorMsg = null;
    $result = null;
    $maxAttempts = 3;
    if (isset($_ENV['AWS_S3_MULTIPART_MAXATTEMPTS']) === true && intval($_ENV['AWS_S3_MULTIPART_MAXATTEMPTS']) > 0) {
        $maxAttempts = intval($_ENV['AWS_S3_MULTIPART_MAXATTEMPTS']);
    }

    do {
        try {
            $result = $copier->copy();
        } catch (MultipartUploadException $e) {
            $copier = new MultipartCopy(
                $this->s3,
                $sourceBucket.'/'.$sourceKey,
                [
                    'state' => $e->getState(),
                ]
            );

            $errorMsg = $e->getMessage();

            $this->logger->log([
                'event' => 'multipartUploadFailed',
                'level' => 'warning',
                'attempts' => $attempts,
                'error' => $errorMsg
            ]);

            sleep(10 + ($attempts * 10));
        }
    } while (isset($result) !== true && $attempts <= $maxAttempts);

`

Version of AWS SDK for PHP? Example: v3.140.2

Version of PHP (php -v)? 7.3.19-1~deb10u1

To Reproduce (observed behavior) See code snippet above. The issue happens intermittently and requires AWS to fail on at least one part of the multi-part upload. However, even this doesn't guarantee a corrupted file as several "failed part" instances have recovered successfully.

Expected behavior The destination file should be identical to the source file.

Screenshots If applicable, add screenshots to help explain your problem.

Additional context I'm using this library within a Symfony Command worker. The worker waits for an event informing it that a file is ready for copying and, once triggered, performs the copy (along with some other unrelated tasks like getting an MD5 of the source prior to the copy).

I'm using a part size of 1 GB.

It's worth noting that we run 3 of these workers at the same time on the same instance to allow up to 3 transfers to occur simultaneously. I wouldn't think there would be a multi-threading issue with this function, but it seems like a possibility given how intermittent the problem is. Also, in the latest instance of this bug showing up, there were indeed multiple copies happening at the same time. I don't have the data readily available as to whether this was the case in previous occurrences, though.

I have a long execute php script (Executes for months)

I find that we have a memory leakage in aws-sdk-php , When I use GetObjectAsync method of S3 client , and after minutes I run the garbage collector , these objects are in memory

GuzzleHttp\Psr7\Request
GuzzleHttp\Psr7\Stream
GuzzleHttp\Psr7\Uri
Aws\RetryMiddleware
Aws\S3\PutObjectUrlMiddleware
Aws\S3\PermanentRedirectMiddleware
Aws\Command
Aws\HandlerList

I am using the defaultProvider to sign an elastic search request but a recent update to the SDK or guzzle has caused it to to start failing.

I have isolated the problem to the chaining going on in the default provider, it seems that if the last item in the chain fails the whole request fails with the following error.

The promise was rejected with reason: Invoking the wait callback did not resolve the promise

I am using the role based credentials (instanceProfile) but if that option appears last as it does with the default provider the request fails, if I reverse the last two options the promise resolves

This Fails

$provider = CredentialProvider::chain(CredentialProvider::env(), CredentialProvider::ini(), CredentialProvider::instanceProfile(), CredentialProvider::ecsCredentials());
call_user_func($provider)->wait();

This returns a credentials object

$provider = CredentialProvider::chain(CredentialProvider::env(), CredentialProvider::ini(), CredentialProvider::ecsCredentials(), CredentialProvider::instanceProfile());
call_user_func($provider)->wait();

Has anyone encountered this or knows why it's happening?

I have the following code

$client = S3Client::factory(array('region' => 'eu-west-1','key' => 'xxx','secret' => 'xxx',));

$command = $client->getCommand('PutObject', array(
    'Bucket' => 'myBucket',
    'Key' => 'testing/signed_'.time(),
    'ContentType' => 'image/jpeg',
    'Body' => 'dump' //it's mandatory, it's not needed here
));
$signedUrl = $command->createPresignedUrl('+5 minutes');
$signedUrl .= '&Content-Type=image%2Fjpeg';
echo("\n\nThe URL is: ". $signedUrl . "\n");
echo("Now run from console for upload:\ncurl -v -H \"Content-Type: image/jpeg\" -T /tmp/temp.jpg '" . $signedUrl . "'");

When trying to use curl command I'm getting and error SignatureDoesNotMatch with message The request signature we calculated does not match the signature you provided. Check your key and signing method.

The similar code in aws-sdk for Javascript is working, so I thing there is an issue in aws-sdk-php.

Please help me. It's urgent :+1:

I've expanded on the excellent work done by @27pchrisl in PR #2332. I rebased his branch onto the latest version of master, upgraded the new tests to be able to use phpunit 9 and added a new CI-job that lints all code in src to make sure it is compatible with all supported versions of php (5.5-8.1) using phpcompatibility.

This should add support for php 8.0 and php 8.1 to the project and make sure no code will trigger bugs because of changes in php.

This should probably fix #2378, #2367

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

I current use this environment

aws/aws-sdk-php 3.186.4 AWS SDK for PHP - Use Amazon Web Services in your PH... aws/aws-sdk-php-laravel 3.6.0 A simple Laravel 5/6/7/8 service provider for includ laravel/framework v8.53.0 The Laravel Framework. laravel/nova 3.27.0 A wonderful administration interface for Laravel.

If I execute composer update for upgrade SDK version 3.187.0 receive this. The problem is DynamoDB thatt i use for driver Cache and Session in Laravel. If change Driver from DYNAMODB to FIle.. no reveive error, but if i use dynamodb exception in all pages my application..

If downgrade return all OK

Confirm by changing [ ] to [x] below to ensure that it's a bug:

• [x] I've gone though Developer Guide and API reference
• [x] I've checked AWS Forums and StackOverflow for answers
• [x] I've searched for previous similar issues and didn't find any solution

Describe the bug When the endpoint option is set and path style endpoints are set to true, the S3EndpointMiddleware produces a Uri with both the scheme and the host in the host property of the Uri. This causes issues when the Uri get's to Guzzle as Guzzle replaces the blank scheme with 'http', duplicating 'http' in the final Uri.

Version of AWS SDK for PHP? v3.173.0

Version of PHP (php -v)? na

To Reproduce (observed behavior) We encountered this bug while using the sdk configured via the Symfony aws bundle, with the options for path style endpoints set to true and custom endpoints for localstack.

aws:
  S3:
    use_path_style_endpoint: true
    endpoint: http://my-endpoint:123

Expected behavior Here is a failing unit test showing what I thought the expected behavior should be.

    public function testApplyingEndpointWhenEndpointOptionAndPathStyleAreSet()
    {
        $nextHandler = function ($command, Request $request){
            $uri = $request->getUri();
            $this->assertEquals($expectedHost = 'my-endpoint', $uri->getHost());
            $this->assertEquals($expectedScheme = 'http', $uri->getScheme());
        };
        $command = new Command('CreateBucket', ['Bucket' => 'abc']);
        $middleware = new S3EndpointMiddleware( $nextHandler, 'us-west-2', [
            'endpoint' => 'http://my-endpoint:123',
            'path_style' => true
        ]);
        $requestUri = new Uri('http://my-endpoint:123');
        $request = new Request('PUT',$requestUri);
        $middleware($command, $request);
    }

Additional context I'm not the most familiar with the implications of path style endpoints. I've read a bit about the plan for them to be deprecated. So sorry in advance if there is something I am missing and this is not an issue.

Description of changes:

Handles cases when there is an empty request path, but a non-empty query. This is a non-issue with standard signing, but resolves an issue with calculating SigV4A signatures and reflects legacy endpoint resolution behavior prior to S3 model processing.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Describe the bug

On recent RHEL/RockyLinux/Alma 9 the default system security policies disable SHA-1.

This cause CloudFrontClient to silently fail signature generation in getSignedUrl when using openssl_sign function and generate an URL with empty signature.

Expected Behavior

An exception or a log that mention the issue.

Current Behavior

Generate the URL but with empty signature

Reproduction Steps

Run getSignedUrl on RHEL/Alma/RockyLinux with default crypto policies or another system without SHA-1 available in OpenSSL.

Possible Solution

Throw an exception when feature not available See #2590

Additional Information/Context

No response

SDK version used

3.252.1

Environment details (Version of PHP (php -v)? OS name and version, etc.)

PHP 8.1.8 (cli) (built: Jul 5 2022 21:55:55) (NTS gcc x86_64) - Rocky Linux release 9.1 (Blue Onyx)

Describe the feature

The EKSClient currently does not offer an getToken() method. The STSClient also does not offer a method to presign a url.

Use Case

We try to access an eks kubernetes cluster from a lambda. Ideally, I should add the lambda's role to the clusters aws-auth configmap and use the EKSClient to generate a token, just like I would be able to do on the cli.

On the CLI I can do aws eks get-token --cluster-name test to get a token. There is to my knowledge no way to do this via the sdk for php.

Proposed Solution

Implement the getToken() method for the EKSClient.

Other Information

No response

Acknowledgements

• [ ] I may be able to implement this feature request
• [ ] This feature might incur a breaking change

SDK version used

3.249.0

Environment details (Version of PHP (php -v)? OS name and version, etc.)

PHP 8.1.13 (cli)

Description of changes: Resolves PHP 8.2 deprecation error when init'ing a HashingStream by declaring the stream property rather than relying on @property

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Describe the bug

using direct CLI, this sample would work:

DBENDPOINT="my-rds.cn1tffyvlovu.us-east-2.rds.amazonaws.com" DBPORT=5432 DBREGION="us-east-2" DBUSER="iam_username" DBNAME="pg_db"; PGPASSWORD=$(aws rds generate-db-auth-token --region "$DBREGION" --hostname "$DBENDPOINT" --port "$DBPORT" --username "$DBUSER") PGSSLMODE=require psql -h "$DBENDPOINT" -U "$DBUSER" -d "$DBNAME" -p "$DBPORT"

taking the output of "echo $PGPASSWORD" and using that as the password in pgAdmin will also allow me to connect.

however, when using the php SDK, the token generated does not seem to work.

Expected Behavior

the token generated from

$provider = CredentialProvider::env(); 

  $RdsAuthGenerator = new Aws\Rds\AuthTokenGenerator($provider);

  $token = $RdsAuthGenerator->createToken($endpoint, $region, $username, $lifetime = 15);

would successfully authenticate pg connection when used as the password for pg_connect()

Current Behavior

passing the token generated to pg_connect always results in "Warning: pg_connect(): Unable to connect to PostgreSQL server: FATAL: PAM authentication failed for user "iam_dbuser"" when token is issued via Aws\Rds\AuthTokenGenerator 's createToken() method.

Reproduction Steps

use Aws\RDS\AuthTokenGenerator;
use Aws\Credentials\CredentialProvider;
use GuzzleHttp\Promise;
use GuzzleHttp\Promise\RejectedPromise;

$provider = CredentialProvider::env();
$RdsAuthGenerator = new Aws\Rds\AuthTokenGenerator($provider);
$token = $RdsAuthGenerator->createToken($endpoint, $region, $username, $lifetime = 15);

Possible Solution

No response

Additional Information/Context

No response

SDK version used

3.252.5

Environment details (Version of PHP (php -v)? OS name and version, etc.)

PHP 8.0.24 (cli) (built: Sep 29 2022 22:26:26) ( NTS ) - Ubuntu 18.04

Proposed solution to below issue

Issue #, if available: https://github.com/aws/aws-sdk-php/issues/2591

Description of changes: prefixing profile name with profile passed into sso function

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.