Two initiatives are working towards goals that will require Goutte in Drupal 8's composer.json. One is adding Behat to Drupal 8 The other is re-architecting Drupals' WebTestBase on top of Mink. Drupal 8 currently uses Guzzle 4. We'd rather not rely on Guzzle 3 and Guzzle 4 at the same time. This updates Goutte to use Guzzle 4. It will require a new 2.0 branch as it bumps php to 5.4. I'm happy to help maintain this version.

Hello,

I'm trying to get data from here: https://wyobiz.wy.gov/Business/FilingSearch.aspx

I'm trying to check if business name is free or not. But this website is ASP.NET Web Forms and has this:

            '__VIEWSTATE' => '',
            '__VIEWSTATEGENERATOR' => '9E6EC73D',
            '__EVENTVALIDATION' => '',

Is it possible to submit this form somehow and get returned data?

Thank you.

My code:

        $crawler = $client->request('GET', 'https://wyobiz.wy.gov/Business/FilingSearch.aspx');
        $form = $crawler->selectButton('Search')->form();
        $formValues = $form->getValues();
        $crawler = $client->submit($form, array(
            '__VIEWSTATE' => $formValues['__VIEWSTATE'],
            '__VIEWSTATEGENERATOR' => $formValues['__VIEWSTATEGENERATOR'],
            '__EVENTVALIDATION' => $formValues['__EVENTVALIDATION'],
            'ctl00$MainContent$myScriptManager' => 'MainContent_myScriptManager',
            'ctl00$MainContent$txtFilingName' => 'Google',
            'ctl00$MainContent$searchOpt' => 'chkSearchStartWith',
            'ctl00$MainContent$txtFilingID' => null,
        ));
        echo $crawler->html();

Error:

The current node list is empty.

I'm trying to do BDD testing on an upload method. I'm using Behat with Mink in a symfony2 project. I'm trying to use the Guzzle client to send a form with file.

$url = $this->minkParameters["base_url"] . '/' . ltrim($url, '/'); 
$file = new \Symfony\Component\HttpFoundation\File\UploadedFile($path, "video");
$fields = json_encode($table->getColumnsHash()[0]); //array("user" => "test")
$this->client->request("POST", $url, array('content-type' => 'multipart/form-data'), array($file), array(), $fields);

I receive this error:

Call to undefined method GuzzleHttp\Stream\Stream::addFile()

Where am i wrong? Thank you

Currently curl options are hardcoded in Client.php, but sometimes more options have to be set (increase timeout or max. redirects).

Code from Client.php:

 $guzzleRequest->getCurlOptions()
        ->set(CURLOPT_FOLLOWLOCATION, false)
        ->set(CURLOPT_MAXREDIRS, 0)
        ->set(CURLOPT_TIMEOUT, 30);

When using the standard CookieJar the cookies only last the duration of the PHP script. When the PHP script terminates, the cookies will have been lost. To maintain the cookies across script executions, I updated Goutte to support FileCookieJar and SessionCookieJar.

Goutte is one of the few Drupal 9.1 dependencies left that does not yet support PHP 8. It would be great to start running CI with PHP 8 to see what needs to be changed.

In order to scrape data that change dynamically according to GET parameters in a website, I need to create a GET request with parameters such as:

//a=a&b=b
$params = array(
            'a' => 'a',
            'b' => 'b',
);

$crawler = $client->request('GET', 'http://www.example.com/', $params);

Then the complete URL should be formed by the URI (http://www.example.com/) plus the Request parameters (a=a&b=b) so that the request is sent to the correct URL.

Hi,

Is there any way using Goutte to remove nodes? I'm building a parser but I need to remove header and footer elements, along with others.

I've briefly looked at the source code for Symfony Crawler and I found the reduce function but I'm unsure on how to use it.

Hi, I'm trying to get source of images which are in a div by id="demo". Here's what I did:

$crawler->filter('#demo img')->each(function ($node) {
    $src = $node->attr('src');
    echo $src . '<br>';
});

It doesn't work But if you try getting any other attribute like image height or image class , it works fine. It only doesn't work when you pass src in attr().

Demonstrates bug in #146 Basically Guzzle 4.0 doesn't support boolean header values. So for special case of Https, we convert that to 'on'. Not sure if anything is needed at all.

I'm trying to install the latest Goutte version via Composer and I keep geting an error that the "Goutte\Client" can't be found. I noticed that there was a recent change that pulled the code out of the "src/" directory. When Composer makes the autoloader, it still tries to include this in the namespacing:

'Goutte' => $vendorDir . '/fabpot/goutte/src/',

If I manually remove it, I get this exception: https://gist.github.com/2704894

Seems like something's broken....not sure what, though.

is a way to run a js function with goutte?

<input id="ingresar" onclick="submitForm()" name="ingresar" class="btn btn-success" style="width:390px;-webkit-border-radius:0 !important;-moz-border-radius:0 !important;border-radius:0 !important;text-align:center;font-size:18px;font-weight:bold;color:#fff;padding:20px 0;cursor:pointer;background-image:none;" value="Ingresar">

This html change and now i cant submit and when y fill the form i can´t send it.

I have the next problema with this code, the last 5 month work with no problems.

$form = $crawler->selectButton('Ingresar')->form();
$form->setValues(['rut' => 'admin', 'password' => 'Pascual4900']);
$crawler = $client->submit($form);

This is the html from the page

<form id="login-form" method="POST" action="/sessions/authenticate">
	<div class="login-container">
		<div class="input" style="cursor:text;" onclick="jQuery('input', this).focus();">
			<h5 style="margin:0">Usuario</h5>
			<input name="rut" type="text" style="font-size:18px;border:0;-webkit-box-shadow:0 0 0;-moz-box-shadow:0 0 0;box-shadow:0 0 0 inset;padding:10px 0 0 0;" placeholder="Escriba su usuario...">
		</div>
		<div class="input" style="cursor:text;" onclick="jQuery('input', this).focus();">
			<h5 style="margin:0">Clave</h5>
			<input name="password" type="password" style="font-size:18px;border:0;-webkit-box-shadow:0 0 0;-moz-box-shadow:0 0 0;box-shadow:0px 0px 0px inset;padding:10px 0 0 0;" placeholder="Escriba su clave...">
		</div>
		<input id="ingresar" onclick="submitForm()" name="ingresar" class="btn btn-success" style="width:390px;-webkit-border-radius:0 !important;-moz-border-radius:0 !important;border-radius:0 !important;text-align:center;font-size:18px;font-weight:bold;color:#fff;padding:20px 0;cursor:pointer;background-image:none;" value="Ingresar">
	</div>
</form>

Thiis return this error

InvalidArgumentException The current node list is empty.

Hello, I don't have any problem running goutte on my local host, but when I transfer files on a shared host, I encounter this error. Thank you for helping me soon.

Fatal error: Uncaught Error: Call to undefined function Symfony\Component\HttpClient\Internal\curl_multi_init() in /home2/farmasco/domains/almas-net.com/public_html/test/test/vendor/symfony/http-client/Internal/CurlClientState.php:43 Stack trace: #0 /home2/farmasco/domains/almas-net.com/public_html/test/test/vendor/symfony/http-client/CurlHttpClient.php(74): Symfony\Component\HttpClient\Internal\CurlClientState->__construct(6, 50) #1 /home2/farmasco/domains/almas-net.com/public_html/test/test/vendor/symfony/http-client/HttpClient.php(54): Symfony\Component\HttpClient\CurlHttpClient->__construct(Array, 6, 50) #2 /home2/farmasco/domains/almas-net.com/public_html/test/test/vendor/symfony/browser-kit/HttpBrowser.php(37): Symfony\Component\HttpClient\HttpClient::create() #3 /home2/farmasco/domains/almas-net.com/public_html/test/test/index.php(16): Symfony\Component\BrowserKit\HttpBrowser->__construct() #4 /home2/farmasco/domains/almas-net.com/public_html/test/test/index.php(27): Start->card() #5 /home2/farmasco/domains/almas-net.com/public_html/test/test/index.php(98): Start->titel() #6 {main} thrown in /home2/farmasco/domains/almas-net.com/public_html/test/test/vendor/symfony/http-client/Internal/CurlClientState.php on line 43

It seems when the numbers 2020 or 24 are in the url passed to request, it will not return any data, as I believe the url encoding is breaking the url.

after googling for hours, I have tried to pass
$crawler->getQuery()->setEncodingType(false); $crawler->setEncodingType(false);

with no luck

An example of a non working URL is: https://www.ebay.com/sch/i.html?_from=R40&_nkw=2019-20+248+prizm+zion+williamson+ruby+wave&_in_kw=1&_ex_kw=&_sacat=0&LH_Sold=1&_udlo=&_udhi=&_samilow=&_samihi=&_sadis=15&_sargn=-1%26saslc%3D1&_salic=1&_sop=12&_dmd=1&_ipg=60&LH_Complete=1&_fosrp=1&LH_Sold=1

and an example of a working URL is: https://www.ebay.com/sch/i.html?_from=R40&_nkw=2019-20+248+prizm+zion+williamson&_in_kw=1&_ex_kw=&_sacat=0&LH_Sold=1&_udlo=&_udhi=&_samilow=&_samihi=&_sadis=15&_sargn=-1%26saslc%3D1&_salic=1&_sop=12&_dmd=1&_ipg=60&LH_Complete=1&_fosrp=1&LH_Sold=1

i cannot figure for the life of me how to fix this, I have tried urlecoding, decoding, a million different versions of the url, and the only thing different between the 2 url's is the keyword, _nkw param.

Thanks in advance

This is the Error i get while creating a new instance of the Client : It works locally, but not in production. i use php 8.1 locally & 8.0.19 in prod so it should not be a syntax probleme. when i checked prod's laravel logs it says this : [2022-06-22 14:22:19] local.ERROR: syntax error, unexpected token ")" {"userId":2,"exception":"[object] (ParseError(code: 0): syntax error, unexpected token ")" at /home/749128.cloudwaysapps.com/maxxhthfnk/public_html/vendor/symfony/http-client/CurlHttpClient.php:68)