A browser testing and web crawling library for PHP and Symfony

Overview

Panther

A browser testing and web scraping library for PHP and Symfony

CI SymfonyInsight

Panther is a convenient standalone library to scrape websites and to run end-to-end tests using real browsers.

Panther is super powerful. It leverages the W3C's WebDriver protocol to drive native web browsers such as Google Chrome and Firefox.

Panther is very easy to use, because it implements Symfony's popular BrowserKit and DomCrawler APIs, and contains all the features you need to test your apps. It will sound familiar if you have ever created a functional test for a Symfony app: as the API is exactly the same! Keep in mind that Panther can be used in every PHP project, as it is a standalone library.

Panther automatically finds your local installation of Chrome or Firefox and launches them, so you don't need to install anything else on your computer, a Selenium server is not needed!

In test mode, Panther automatically starts your application using the PHP built-in web-server. You can focus on writing your tests or web-scraping scenario and Panther will take care of everything else.

Features

Unlike testing and web scraping libraries you're used to, Panther:

  • executes the JavaScript code contained in webpages
  • supports everything that Chrome (or Firefox) implements
  • allows taking screenshots
  • can wait for asynchronously loaded elements to show up
  • lets you run your own JS code or XPath queries in the context of the loaded page
  • supports custom Selenium server installations
  • supports remote browser testing services including SauceLabs and BrowserStack

Documentation

Installing Panther

Use Composer to install Panther in your project. You may want to use the --dev flag if you want to use Panther for testing only and not for web scraping in a production environment:

composer req symfony/panther

composer req --dev symfony/panther

Installing ChromeDriver and geckodriver

Panther uses the WebDriver protocol to control the browser used to crawl websites.

On all systems, you can use dbrekelmans/browser-driver-installer to install ChromeDriver and geckodriver locally:

composer require --dev dbrekelmans/bdi
vendor/bin/bdi detect drivers

Panther will detect and use automatically drivers stored in the drivers/ directory.

Alternatively, you can use the package manager of your operating system to install them.

On Ubuntu, run:

apt-get install chromium-chromedriver firefox-geckodriver

On Mac, using Homebrew:

brew install chromedriver geckodriver

On Windows, using chocolatey:

choco install chromedriver selenium-gecko-driver

Finally, you can download manually ChromeDriver (for Chromium or Chrome) and GeckoDriver (for Firefox) and put them anywhere in your PATH or in the drivers/ directory of your project.

Registering the PHPUnit Extension

If you intend to use Panther to test your application, we strongly recommend registering the Panther PHPUnit extension. While not strictly mandatory, this extension dramatically improves the testing experience by boosting the performance and allowing to use the interactive debugging mode.

When using the extension in conjunction with the PANTHER_ERROR_SCREENSHOT_DIR environment variable, tests using the Panther client that fail or error (after the client is created) will automatically get a screenshot taken to help debugging.

To register the Panther extension, add the following lines to phpunit.xml.dist:

<!-- phpunit.xml.dist -->
<extensions>
    <extension class="Symfony\Component\Panther\ServerExtension" />
</extensions>

Without the extension, the web server used by Panther to serve the application under test is started on demand and stopped when tearDownAfterClass() is called. On the other hand, when the extension is registered, the web server will be stopped only after the very last test.

Basic Usage

<?php

use Symfony\Component\Panther\Client;

require __DIR__.'/vendor/autoload.php'; // Composer's autoloader

$client = Client::createChromeClient();
// Or, if you care about the open web and prefer to use Firefox
$client = Client::createFirefoxClient();

$client->request('GET', 'https://api-platform.com'); // Yes, this website is 100% written in JavaScript
$client->clickLink('Get started');

// Wait for an element to be present in the DOM (even if hidden)
$crawler = $client->waitFor('#installing-the-framework');
// Alternatively, wait for an element to be visible
$crawler = $client->waitForVisibility('#installing-the-framework');

echo $crawler->filter('#installing-the-framework')->text();
$client->takeScreenshot('screen.png'); // Yeah, screenshot!

Testing Usage

The PantherTestCase class allows you to easily write E2E tests. It automatically starts your app using the built-in PHP web server and let you crawl it using Panther. To provide all the testing tools you're used to, it extends PHPUnit's TestCase.

If you are testing a Symfony application, PantherTestCase automatically extends the WebTestCase class. It means you can easily create functional tests, which can directly execute the kernel of your application and access all your existing services. In this case, you can use all crawler test assertions provided by Symfony with Panther.

<?php

namespace App\Tests;

use Symfony\Component\Panther\PantherTestCase;

class E2eTest extends PantherTestCase
{
    public function testMyApp(): void
    {
        $client = static::createPantherClient(); // Your app is automatically started using the built-in web server
        $client->request('GET', '/mypage');

        // Use any PHPUnit assertion, including the ones provided by Symfony
        $this->assertPageTitleContains('My Title');
        $this->assertSelectorTextContains('#main', 'My body');
        
        // Or the one provided by Panther
        $this->assertSelectorIsEnabled('.search');
        $this->assertSelectorIsDisabled('[type="submit"]');
        $this->assertSelectorIsVisible('.errors');
        $this->assertSelectorIsNotVisible('.loading');
        $this->assertSelectorAttributeContains('.price', 'data-old-price', '42');
        $this->assertSelectorAttributeNotContains('.price', 'data-old-price', '36');

        // Use waitForX methods to wait until some asynchronous process finish
        $client->waitFor('.popin'); // wait for element to be attached to the DOM
        $client->waitForStaleness('.popin'); // wait for element to be removed from the DOM
        $client->waitForVisibility('.loader'); // wait for element of the DOM to become visible
        $client->waitForInvisibility('.loader'); // wait for element of the DOM to become hidden
        $client->waitForElementToContain('.total', '25 €'); // wait for text to be inserted in the element content
        $client->waitForElementToNotContain('.promotion', '5%'); // wait for text to be removed from the element content
        $client->waitForEnabled('[type="submit"]'); // wait for the button to become enabled 
        $client->waitForDisabled('[type="submit"]'); // wait for  the button to become disabled 
        $client->waitForAttributeToContain('.price', 'data-old-price', '25 €'); // wait for the attribute to contain content
        $client->waitForAttributeToNotContain('.price', 'data-old-price', '25 €'); // wait for the attribute to not contain content
        
        // Let's predict the future
        $this->assertSelectorWillExist('.popin'); // element will be attached to the DOM
        $this->assertSelectorWillNotExist('.popin'); // element will be removed from the DOM
        $this->assertSelectorWillBeVisible('.loader'); // element will be visible
        $this->assertSelectorWillNotBeVisible('.loader'); // element will be visible
        $this->assertSelectorWillContain('.total', '€25'); // text will be inserted in the element content
        $this->assertSelectorWillNotContain('.promotion', '5%'); // text will be removed from the element content
        $this->assertSelectorWillBeEnabled('[type="submit"]'); // button will be enabled 
        $this->assertSelectorWillBeDisabled('[type="submit"]'); // button will be disabled 
        $this->assertSelectorAttributeWillContain('.price', 'data-old-price', '€25'); // attribute will contain content
        $this->assertSelectorAttributeWillNotContain('.price', 'data-old-price', '€25'); // attribute will not contain content
    }
}

To run this test:

bin/phpunit tests/E2eTest.php

A Polymorphic Feline

Panther also gives you instant access to other BrowserKit-based implementations of Client and Crawler. Unlike Panther's native client, these alternative clients don't support JavaScript, CSS and screenshot capturing, but they are super-fast!

Two alternative clients are available:

  • The first directly manipulates the Symfony kernel provided by WebTestCase. It is the fastest client available, but it is only available for Symfony apps.
  • The second leverages Symfony's HttpBrowser. It is an intermediate between Symfony's kernel and Panther's test clients. HttpBrowser sends real HTTP requests using Symfony's HttpClient component. It is fast and is able to browse any webpage, not only the ones of the application under test. However, HttpBrowser doesn't support JavaScript and other advanced features because it is entirely written in PHP. This one is available even for non-Symfony apps!

The fun part is that the 3 clients implement the exact same API, so you can switch from one to another just by calling the appropriate factory method, resulting in a good trade-off for every single test case (Do I need JavaScript? Do I need to authenticate with an external SSO server? Do I want to access the kernel of the current request? ... etc).

Here is how to retrieve instances of these clients:

<?php

namespace App\Tests;

use Symfony\Component\Panther\PantherTestCase;
use Symfony\Component\Panther\Client;

class E2eTest extends PantherTestCase
{
    public function testMyApp(): void
    {
        $symfonyClient = static::createClient(); // A cute kitty: Symfony's functional test tool
        $httpBrowserClient = static::createHttpBrowserClient(); // An agile lynx: HttpBrowser
        $pantherClient = static::createPantherClient(); // A majestic Panther
        $firefoxClient = static::createPantherClient(['browser' => static::FIREFOX]); // A splendid Firefox
        // Both HttpBrowser and Panther benefits from the built-in HTTP server

        $customChromeClient = Client::createChromeClient(null, null, [], 'https://example.com'); // Create a custom Chrome client
        $customFirefoxClient = Client::createFirefoxClient(null, null, [], 'https://example.com'); // Create a custom Firefox client
        $customSeleniumClient = Client::createSeleniumClient('http://127.0.0.1:4444/wd/hub', null, 'https://example.com'); // Create a custom Selenium client
        // When initializing a custom client, the integrated web server IS NOT started automatically.
        // Use PantherTestCase::startWebServer() or WebServerManager if you want to start it manually.

        // enjoy the same API for the 3 felines
        // $*client->request('GET', '...')

        $kernel = static::createKernel(); // If you are testing a Symfony app, you also have access to the kernel

        // ...
    }
}

Creating Isolated Browsers to Test Apps Using Mercure or WebSocket

Panther provides a convenient way to test applications with real-time capabilities which use Mercure, WebSocket and similar technologies.

PantherTestCase::createAdditionalPantherClient() creates additional, isolated browsers which can interact with each other. For instance, this can be useful to test a chat application having several users connected simultaneously:

<?php

use Symfony\Component\Panther\PantherTestCase;

class ChatTest extends PantherTestCase
{
    public function testChat(): void
    {
        $client1 = self::createPantherClient();
        $client1->request('GET', '/chat'); 
 
        // Connect a 2nd user using an isolated browser and say hi!
        $client2 = self::createAdditionalPantherClient();
        $client2->request('GET', '/chat');
        $client2->submitForm('Post message', ['message' => 'Hi folks 👋😻']);

        // Wait for the message to be received by the first client
        $client1->waitFor('.message');

        // Symfony Assertions are always executed in the **primary** browser
        $this->assertSelectorTextContains('.message', 'Hi folks 👋😻');
    }
}

Accessing Browser Console Logs

If needed, you can use Panther to access the content of the console:

<?php

use Symfony\Component\Panther\PantherTestCase;

class ConsoleTest extends PantherTestCase
{
    public function testConsole(): void
    {
        $client = self::createPantherClient(
            [],
            [],
            [
                'capabilities' => [
                    'goog:loggingPrefs' => [
                        'browser' => 'ALL', // calls to console.* methods
                        'performance' => 'ALL', // performance data
                    ],
                ],
            ]
        );

        $client->request('GET', '/');
        $consoleLogs = $client->getWebDriver()->manage()->getLog('browser'); // console logs 
        $performanceLogs = $client->getWebDriver()->manage()->getLog('performance'); // performance logs
    }
}

Passing Arguments to ChromeDriver

If needed, you can configure the arguments to pass to the chromedriver binary:

<?php

use Symfony\Component\Panther\PantherTestCase;

class MyTest extends PantherTestCase
{
    public function testLogging(): void
    {
        $client = self::createPantherClient(
            [],
            [],
            [
                'chromedriver_arguments' => [
                    '--log-path=myfile.log',
                    '--log-level=DEBUG'
                ],
            ]
        );

        $client->request('GET', '/');
    }
}

Checking the State of the WebDriver Connection

Use the Client::ping() method to check if the WebDriver connection is still active (useful for long-running tasks).

Additional Documentation

Since Panther implements the API of popular libraries, it already has an extensive documentation:

Environment Variables

The following environment variables can be set to change some Panther's behaviour:

  • PANTHER_NO_HEADLESS: to disable the browser's headless mode (will display the testing window, useful to debug)
  • PANTHER_WEB_SERVER_DIR: to change the project's document root (default to ./public/, relative paths must start by ./)
  • PANTHER_WEB_SERVER_PORT: to change the web server's port (default to 9080)
  • PANTHER_WEB_SERVER_ROUTER: to use a web server router script which is run at the start of each HTTP request
  • PANTHER_EXTERNAL_BASE_URI: to use an external web server (the PHP built-in web server will not be started)
  • PANTHER_APP_ENV: to override the APP_ENV variable passed to the web server running the PHP app
  • PANTHER_ERROR_SCREENSHOT_DIR: to set a base directory for your failure/error screenshots (e.g. ./var/error-screenshots)

Changing the Hostname and Port of the Built-in Web Server

If you want to change the host and/or the port used by the built-in web server, pass the hostname and port to the $options parameter of the createPantherClient() method:

// ...

$client = self::createPantherClient([
    'hostname' => 'example.com', // Defaults to 127.0.0.1
    'port' => 8080, // Defaults to 9080
]);

Chrome-specific Environment Variables

  • PANTHER_NO_SANDBOX: to disable Chrome's sandboxing (unsafe, but allows to use Panther in containers)
  • PANTHER_CHROME_ARGUMENTS: to customize Chrome arguments. You need to set PANTHER_NO_HEADLESS to fully customize.
  • PANTHER_CHROME_BINARY: to use another google-chrome binary

Firefox-specific Environment Variables

  • PANTHER_FIREFOX_ARGUMENTS: to customize Firefox arguments. You need to set PANTHER_NO_HEADLESS to fully customize.
  • PANTHER_FIREFOX_BINARY: to use another firefox binary

Accessing To Hidden Text

According to the spec, WebDriver implementations return only the displayed text by default. When you filter on a head tag (like title), the method text() returns an empty string. Use the method html() to get the complete contents of the tag, including the tag itself.

Interactive Mode

Panther can make a pause in your tests suites after a failure. It is a break time really appreciated for investigating the problem through the web browser. For enabling this mode, you need the --debug PHPUnit option without the headless mode:

$ PANTHER_NO_HEADLESS=1 bin/phpunit --debug

Test 'App\AdminTest::testLogin' started
Error: something is wrong.

Press enter to continue...

To use the interactive mode, the PHPUnit extension must be registered.

Using an External Web Server

Sometimes, it's convenient to reuse an existing web server configuration instead of starting the built-in PHP one. To do so, set the external_base_uri option:

<?php

namespace App\Tests;

use Symfony\Component\Panther\PantherTestCase;

class E2eTest extends PantherTestCase
{
    public function testMyApp(): void
    {
        $pantherClient = static::createPantherClient(['external_base_uri' => 'https://localhost']);
        // the PHP integrated web server will not be started
    }
}

Having a Multi-domain Application

It happens that your PHP/Symfony application might serve several different domain names.

As Panther saves the Client in memory between tests to improve performances, you will have to run your tests in separate processes if you write several tests using Panther for different domain names.

To do so, you can use the native @runInSeparateProcess PHPUnit annotation.

Note: it is really convenient to use the external_base_uri option and start your own webserver in the background, because Panther will not have to start and stop your server on each test. Symfony CLI can be a quick and easy way to do so.

Here is an example using the external_base_uri option to determine the domain name used by the Client:

<?php

namespace App\Tests;

use Symfony\Component\Panther\PantherTestCase;

class FirstDomainTest extends PantherTestCase
{
    /**
     * @runInSeparateProcess
     */
    public function testMyApp(): void
    {
        $pantherClient = static::createPantherClient([
            'external_base_uri' => 'http://mydomain.localhost:8000',
        ]);
        
        // Your tests
    }
}
<?php

namespace App\Tests;

use Symfony\Component\Panther\PantherTestCase;

class SecondDomainTest extends PantherTestCase
{
    /**
     * @runInSeparateProcess
     */
    public function testMyApp(): void
    {
        $pantherClient = static::createPantherClient([
            'external_base_uri' => 'http://anotherdomain.localhost:8000',
        ]);
        
        // Your tests
    }
}

Using a Proxy

To use a proxy server, set the following environment variable: PANTHER_CHROME_ARGUMENTS='--proxy-server=socks://127.0.0.1:9050'

Accepting Self-signed SSL Certificates

To force Chrome to accept invalid and self-signed certificates, set the following environment variable: PANTHER_CHROME_ARGUMENTS='--ignore-certificate-errors' This option is insecure, use it only for testing in development environments, never in production (e.g. for web crawlers).

Docker Integration

Here is a minimal Docker image that can run Panther with both Chrome and Firefox:

FROM php:alpine

# Chromium and ChromeDriver
ENV PANTHER_NO_SANDBOX 1
# Not mandatory, but recommended
ENV PANTHER_CHROME_ARGUMENTS='--disable-dev-shm-usage'
RUN apk add --no-cache chromium chromium-chromedriver

# Firefox and GeckoDriver (optional)
ARG GECKODRIVER_VERSION=0.28.0
RUN apk add --no-cache firefox libzip-dev; \
    docker-php-ext-install zip
RUN wget -q https://github.com/mozilla/geckodriver/releases/download/v$GECKODRIVER_VERSION/geckodriver-v$GECKODRIVER_VERSION-linux64.tar.gz; \
    tar -zxf geckodriver-v$GECKODRIVER_VERSION-linux64.tar.gz -C /usr/bin; \
    rm geckodriver-v$GECKODRIVER_VERSION-linux64.tar.gz

Build it with docker build . -t myproject Run it with docker run -it -v "$PWD":/srv/myproject -w /srv/myproject myproject bin/phpunit

GitHub Actions Integration

Panther works out of the box with GitHub Actions. Here is a minimal .github/workflows/panther.yml file to run Panther tests:

name: Run Panther tests

on: [ push, pull_request ]

jobs:
  tests:

    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v2

      - name: Install dependencies
        run: composer install -q --no-ansi --no-interaction --no-scripts --no-progress --prefer-dist

      - name: Run test suite
        run: bin/phpunit

Travis CI Integration

Panther will work out of the box with Travis CI if you add the Chrome addon. Here is a minimal .travis.yml file to run Panther tests:

language: php
addons:
  # If you don't use Chrome, or Firefox, remove the corresponding line
  chrome: stable
  firefox: latest

php:
  - 8.0

script:
  - bin/phpunit

Gitlab CI Integration

Here is a minimal .gitlab-ci.yml file to run Panther tests with Gitlab CI:

image: ubuntu

before_script:
  - apt-get update
  - apt-get install software-properties-common -y
  - ln -sf /usr/share/zoneinfo/Asia/Tokyo /etc/localtime
  - apt-get install curl wget php php-cli php7.4 php7.4-common php7.4-curl php7.4-intl php7.4-xml php7.4-opcache php7.4-mbstring php7.4-zip libfontconfig1 fontconfig libxrender-dev libfreetype6 libxrender1 zlib1g-dev xvfb chromium-chromedriver firefox-geckodriver -y -qq
  - export PANTHER_NO_SANDBOX=1
  - export PANTHER_WEB_SERVER_PORT=9080
  - php -r "copy('https://getcomposer.org/installer', 'composer-setup.php');"
  - php composer-setup.php --install-dir=/usr/local/bin --filename=composer
  - php -r "unlink('composer-setup.php');"
  - composer install

test:
  script:
    - bin/phpunit

AppVeyor Integration

Panther will work out of the box with AppVeyor as long as Google Chrome is installed. Here is a minimal appveyor.yml file to run Panther tests:

build: false
platform: x86
clone_folder: c:\projects\myproject

cache:
  - '%LOCALAPPDATA%\Composer\files'

install:
  - ps: Set-Service wuauserv -StartupType Manual
  - cinst -y php composer googlechrome chromedriver firfox selenium-gecko-driver
  - refreshenv
  - cd c:\tools\php80
  - copy php.ini-production php.ini /Y
  - echo date.timezone="UTC" >> php.ini
  - echo extension_dir=ext >> php.ini
  - echo extension=php_openssl.dll >> php.ini
  - echo extension=php_mbstring.dll >> php.ini
  - echo extension=php_curl.dll >> php.ini
  - echo memory_limit=3G >> php.ini
  - cd %APPVEYOR_BUILD_FOLDER%
  - composer install --no-interaction --no-progress

test_script:
  - cd %APPVEYOR_BUILD_FOLDER%
  - php bin\phpunit

Usage with Other Testing Tools

If you want to use Panther with other testing tools like LiipFunctionalTestBundle or if you just need to use a different base class, Panther has got you covered. It provides you with the Symfony\Component\Panther\PantherTestCaseTrait and you can use it to enhance your existing test-infrastructure with some Panther awesomeness:

<?php

namespace App\Tests\Controller;

use Liip\FunctionalTestBundle\Test\WebTestCase;
use Symfony\Component\Panther\PantherTestCaseTrait;

class DefaultControllerTest extends WebTestCase
{
    use PantherTestCaseTrait; // this is the magic. Panther is now available.

    public function testWithFixtures(): void
    {
        $this->loadFixtures([]); // load your fixtures
        $client = self::createPantherClient(); // create your panther client

        $client->request('GET', '/');
    }
}

Limitations

The following features are not currently supported:

  • Crawling XML documents (only HTML is supported)
  • Updating existing documents (browsers are mostly used to consume data, not to create webpages)
  • Setting form values using the multidimensional PHP array syntax
  • Methods returning an instance of \DOMElement (because this library uses WebDriverElement internally)
  • Selecting invalid choices in select

Pull Requests are welcome to fill the remaining gaps!

Save the Panthers

Many of the wild cat species are highly threatened. If you like this software, help save the (real) panthers by donating to the Panthera organization.

Credits

Created by Kévin Dunglas. Sponsored by Les-Tilleuls.coop.

Panther is built on top of PHP WebDriver and several other FOSS libraries. It has been inspired by Nightwatch.js, a WebDriver-based testing tool for JavaScript.

Comments
  • Operation timed out after 30001 milliseconds with 0 bytes received

    Operation timed out after 30001 milliseconds with 0 bytes received

    Hello,

    I have a lot of functional tests and when I launch phpunit, an errors occurs:

    Operation timed out after 30001 milliseconds with 0 bytes received.

    When I launch test one by one or I comment some tests, everything is ok.

    For details, all tests are executed into a docker container. I don't have change the panther configuration.

    Thanks for you help.

    opened by qferr 30
  • Missing execution bit from chromedriver preventing its execution

    Missing execution bit from chromedriver preventing its execution

    Hi, I'm trying to use Panthère in Symfony on Debian, but I have this error when launching my tests : RuntimeException: sh: 1: exec: /srv/app/vendor/symfony/panthere/src/ProcessManager/../../chromedriver-bin/chromedriver_linux64: Permission denied

    Indeed, here's the permission status : image

    Is there a way to install chromedriver without setting permissions by hand ?

    good first issue 
    opened by XavierVallot 23
  • Impossible to get the HttpFoundation objects from the Client

    Impossible to get the HttpFoundation objects from the Client

    | Q | A | ---------------- | ----- | Bug report? | yes | Feature request? | no | BC Break report? | yes

    When using PanthereTestCase, and when making a request, we can't get access to the Symfony\Component\HttpFoundation\Response object that should normally be sent to the Kernel. Instead, we only get access to the BrowserKit one.

    Steps to reproduce:

    <?php
    
    use Panthere\PanthereTestCase;
    
    class MyPanthereTest extends PanthereTestCase
    {
        public function testMyApp()
        {
            $client = static::createPanthereClient();
            $crawler = $client->request('GET', static::$baseUri.'/any-page');
    
            static::assertSame(200, $client->getResponse()->getStatusCode());
        }
    }
    

    PHPUnit's output:

    Error : Call to undefined method Symfony\Component\BrowserKit\Response::getStatusCode()
    

    I consider this as a BC break, because the $client->request() method SHOULD return an instance of the Symfony\Component\BrowserKit\Client class, for which getResponse() returns an instance of Symfony\Component\HttpFoundation\Response.

    This is the same for the Request via $client->getRequest() by the way.

    The issue comes from these lines: https://github.com/dunglas/panthere/blob/b3f0601e105010d365360f238ca1b572a4e5ec82/src/Client.php#L238-L241

    IMO, the Request and Response object should be either blocked from the PanthereClient via an exception (easy pick), or created & adapted based on the WebDriver results (harder).

    WDYT?

    opened by Pierstoval 15
  • Add support for Symfony 6

    Add support for Symfony 6

    It is required to run with dev dependencies, otherwise it wont be installed. I didn't found a dev dependencies build so I added one (if it is not correct this way, I can revert it or change to another one)

    opened by jordisala1991 14
  • Could not start chrome (or it crashed) after 30 seconds.

    Could not start chrome (or it crashed) after 30 seconds.

    Hey! Seems like duplicated but I cant solve the issue. Starting with firefox driver also causes Could not start firefox (or it crashed) after 30 seconds. Also tries to increase $timeout to 60 secs inside WebServerReadinessProbeTrait.php, still no luck.

    Server has enough resources 4 cores 16 gb ram. Composer dependency"symfony/panther": "^0.7.1"

    Any ideas?

    opened by phenoengine 13
  • Increase a lot (x4) boot sequence

    Increase a lot (x4) boot sequence

    1. Use symfony/http-client to get the status of webdriver process
    2. Do not wait an extra 1 s

    The major point is the first one. I paste here the commit details:

    With a simple PHPUnit setup and a 'regular' web page I got the following numbers:

    Before:

    Time: 13.39 seconds, Memory: 6.00MB

    After:

    Time: 3.27 seconds, Memory: 6.00MB

    I Profiler the page with blackfire. And I noticed file_get_contents() was very slow.

    I first updated the binary (see previous commit) but without success.

    Then I tried to reproduce manually what happend. The chromedrive boot really fast, and is directly available on HTTP. So there is something wrong with file_get_contents(). I used tcpdump to debug it, and the chrome driver do well its job. The issue is in PHP :(.

    Since we have a nice HTTP client Symfony, let's use it !

    Here is the tcpdump of the dial between php and the driver

    1:66, ack 1, win 342, options [nop,nop,TS val 3026213891 ecr 3026213891],
    length 65 E..u..@.@.............%+...sO......V.i.....
    .`\..`\.GET /status HTTP/1.1 Host: 127.0.0.1:9515 Connection: close
    
    16:05:44.854645 IP 127.0.0.1.9515 > 127.0.0.1.59044: Flags [P.], seq 1:237,
    ack 66, win 342, options [nop,nop,TS val 3026213891 ecr 3026213891], length
    236 E.. .x@.@..]........%+..O..........V.......
    .`\..`\.HTTP/1.1 200 OK Content-Length:133 Content-Type:application/json;
    charset=utf-8 Connection:close
    
    {"sessionId":"","status":0,"value":{"build":{"version":"alpha"},"os":{"arch":"x86_64","name":"Linux","version":"4.15.0-46-generic"}}}
    16:05:44.854651 IP 127.0.0.1.59044 > 127.0.0.1.9515: Flags [.], ack 237,
    win 350, options [nop,nop,TS val 3026213891 ecr 3026213891], length 0
    E..4..@.@.............%+....O......^.(.....
    .`\..`\.
    
    [< HANG A LOT HERE >]
    
    16:05:54.864184 IP 127.0.0.1.59044 > 127.0.0.1.9515: Flags [F.], seq 66,
    ack 237, win 350, options [nop,nop,TS val 3026223901 ecr 3026213891],
    length 0 E..4..@.@.............%+....O......^.(.....
    .`...`\.
    

    EDIT

    I initially thought it will increase only the very boot sequence, but actually it speeds up eveything.

    With 3 tests:

    Before:

    Time: 42.06 seconds, Memory: 6.00MB After: Time: 8.96 seconds, Memory: 6.00MB

    opened by lyrixx 13
  • [QUESTION] Chrome Driver - Docker image

    [QUESTION] Chrome Driver - Docker image

    Hi everyone,

    Small question about the usage of Panther into a PHP Alpine image, I've tried to install Chromedriver and chromium but here's the error that I get:

    #!/usr/bin/env php
    PHPUnit 6.5.13 by Sebastian Bergmann and contributors.
    
    Error:         No code coverage driver is available
    
    Testing E2E
    E                                                                   1 / 1 (100%)
    
    Time: 8.08 seconds, Memory: 40.25MB
    
    There was 1 error:
    
    1) E2E\Core\HomeE2ETest::testStatusCode
    RuntimeException: sh: exec: line 1: /var/www/marketReminder/vendor/symfony/panther/src/ProcessManager/../../chromedriver-bin/chromedriver_linux64: not found
    
    
    /var/www/marketReminder/vendor/symfony/panther/src/ProcessManager/WebServerReadinessProbeTrait.php:50
    /var/www/marketReminder/vendor/symfony/panther/src/ProcessManager/ChromeManager.php:49
    /var/www/marketReminder/vendor/symfony/panther/src/Client.php:80
    /var/www/marketReminder/vendor/symfony/panther/src/Client.php:272
    /var/www/marketReminder/vendor/symfony/panther/src/Client.php:186
    /var/www/marketReminder/E2E/Core/HomeE2ETest.php:26
    
    ERRORS!
    Tests: 1, Assertions: 0, Errors: 1.
    
    Remaining deprecation notices (1)
    
      1x: Doctrine\Common\ClassLoader is deprecated.
        1x in HomeE2ETest::testStatusCode from E2E\Core
    
    make: *** [panther] Error 2
    

    For information, here's the Dockerfile:

    # Development build
    FROM php:fpm-alpine as base
    
    ARG WORKFOLDER
    
    ENV COMPOSER_ALLOW_SUPERUSER 1
    ENV PANTHER_NO_SANDBOX 1
    ENV PANTHER_WEB_SERVER_PORT 9800
    ENV WORKPATH ${WORKFOLDER}
    
    RUN apk add --no-cache --virtual .build-deps $PHPIZE_DEPS icu-dev postgresql-dev gnupg graphviz make autoconf git zlib-dev curl chromium chromium-chromedriver go rabbitmq-c rabbitmq-c-dev \
        && docker-php-ext-configure pgsql --with-pgsql=/usr/local/pgsql \
        && docker-php-ext-install zip intl pdo_pgsql pdo_mysql opcache json pdo_pgsql pgsql mysqli \
        && pecl install apcu redis grpc protobuf amqp \
        && docker-php-ext-enable apcu mysqli redis grpc protobuf amqp
    
    COPY docker/php/conf/php.ini /usr/local/etc/php/php.ini
    
    # Composer
    COPY --from=composer:latest /usr/bin/composer /usr/bin/composer
    
    # Blackfire (Docker approach) & Blackfire Player
    RUN version=$(php -r "echo PHP_MAJOR_VERSION.PHP_MINOR_VERSION;") \
        && curl -A "Docker" -o /tmp/blackfire-probe.tar.gz -D - -L -s https://blackfire.io/api/v1/releases/probe/php/alpine/amd64/$version \
        && tar zxpf /tmp/blackfire-probe.tar.gz -C /tmp \
        && mv /tmp/blackfire-*.so $(php -r "echo ini_get('extension_dir');")/blackfire.so \
        && printf "extension=blackfire.so\nblackfire.agent_socket=tcp://blackfire:8707\n" > $PHP_INI_DIR/conf.d/blackfire.ini \
        && mkdir -p /tmp/blackfire \
        && curl -A "Docker" -L https://blackfire.io/api/v1/releases/client/linux_static/amd64 | tar zxp -C /tmp/blackfire \
        && mv /tmp/blackfire/blackfire /usr/bin/blackfire \
        && rm -Rf /tmp/blackfire \
        && curl -OLsS http://get.blackfire.io/blackfire-player.phar \
        && chmod +x blackfire-player.phar \
        && mv blackfire-player.phar /usr/local/bin/blackfire-player
    
    # PHP-CS-FIXER & Deptrac
    RUN wget http://cs.sensiolabs.org/download/php-cs-fixer-v2.phar -O php-cs-fixer \
        && chmod a+x php-cs-fixer \
        && mv php-cs-fixer /usr/local/bin/php-cs-fixer \
        && curl -LS http://get.sensiolabs.de/deptrac.phar -o deptrac.phar \
        && chmod +x deptrac.phar \
        && mv deptrac.phar /usr/local/bin/deptrac
    
    RUN mkdir -p ${WORKPATH} \
        && chown -R www-data /tmp/ \
        && mkdir -p \
           ${WORKPATH}/var/cache \
           ${WORKPATH}/var/logs \
           ${WORKPATH}/var/sessions \
        && chown -R www-data ${WORKPATH}/var
    
    WORKDIR ${WORKPATH}
    
    COPY --chown=www-data:www-data . ./
    
    # Production build
    FROM base as production
    
    COPY --from=gcr.io/marketreminder-206206/node:latest ${WORKPATH}/public/build ./public/build
    
    COPY docker/php/conf/production/php.ini /usr/local/etc/php/php.ini
    
    RUN rm -rf /usr/local/bin/deptrac \
        && rm -rf /usr/local/bin/php-cs-fixer
    

    I don't really see where the problem is coming from and i've search to see if someone has the same problem but nothing about this type of error, does anyone has an idea ?

    PS: I've tried the installation recommanded by the documentation but the driver doesn't seem to be found

    Thanks for the help :)

    opened by Guikingone 12
  • Replace $_SERVER by $_ENV

    Replace $_SERVER by $_ENV

    PHPUnit internally uses $_ENV for the env vars. This, it is possible to edit env vars through the phpunit.xml.dist:

    <phpunit>
        <php>
            <env name="PANTHER_WEB_SERVER_ROUTER" value="router.php"/>
        </php>
    </phpunit>
    

    cf https://phpunit.readthedocs.io/en/7.3/configuration.html?highlight=env#setting-php-ini-settings-constants-and-global-variables

    opened by maidmaid 12
  • cannot test assert for series of pages

    cannot test assert for series of pages

    I'm trying to click on a link after logging in to dashboard, which will eventually open up another page inside dashboard and search for assert text contains. These are all normal http request/response page reload and not ajax. Is there a solution to test multiple urls which are not ajaxified?

    ``<?php

    namespace App\Tests\Controller;

    use Symfony\Component\Panther\PantherTestCase;

    class SecurityControllerTest extends PantherTestCase { public function testCustomerRoleAfterLogin() { $client = static::createPantherClient();

        $client->request('GET', '/');
    
        $client->submitForm('Log In', [
            'username' => '[email protected]',
            'password' => 'asd123'
        ]);
    
        $client->request('GET', '/dashboard/index');
        $client->waitFor('#main-menu-sidebar-wrapper .settings');
        $this->assertSelectorTextContains('#main-menu-sidebar-wrapper .settings', 'Settings');
        $client->clickLink('#main-menu-sidebar-wrapper .settings');
    
        $client->request('GET', '/dashboard/settings');
        $client->waitFor('#editCustomerSettings span');
        $this->assertSelectorTextContains('#editCustomerSettings span', 'Customer Settings');
    
    }
    

    } ``

    opened by adityarai 11
  • ServerTrait: Don't check isWebServerStarted on pause

    ServerTrait: Don't check isWebServerStarted on pause

    For the long story, check #427 .

    After some investigation, I've found out that we aren't really testing weather a browser window stays open after a test failure or error (this might be untestable, tbf):

    (From the issue posts)


    I've found the test that is supposed to check weather the browser will be paused on test failure. The problem with the test itself is that it skips calling $this->pause(sprintf('Failure: %s', $message));:

    public function testPauseOnFailure(string $method, string $expected): void
    {
        $extension = new ServerExtension();
        $extension->testing = true;
        ...
    

    In the beginning of the test, the flag $extension->testing is set. Later, in pause() we see this:

    if (!$this->testing) {
        fgets(\STDIN);
    }
    

    And this is why a test wouldn't cover the issue reported here. So I added some echo lines throughout the code to debug a little bit and I've found out that in my particular case, when I reach the test failure, PantherTestCase::isWebServerStarted() is false, so the pause is never done:

    (with my debug lines)

    private function pause($message): void
    {
        echo PantherTestCase::isWebServerStarted() ? "Web server started\n" : "Web server hasn't started!!\n";
    
        if (PantherTestCase::isWebServerStarted()
            && \in_array('--debug', $_SERVER['argv'], true)
            && $_SERVER['PANTHER_NO_HEADLESS'] ?? false
        ) {
            echo "$message\n\nPress enter to continue...";
            if (!$this->testing) {
                fgets(\STDIN);
            }
        }
    }
    

    And the output is:

    Testing /home/shadowc/web-local/decoramemas/tests/e2e
    Test 'App\Tests\e2e\AdminBasicTest::testBasicAdminPageLoads' started
    Web server hasn't started!!
    Test 'App\Tests\e2e\AdminBasicTest::testBasicAdminPageLoads' ended
    

    Thus the pause isn't performed.


    By just removing that check in pause I was able to make the browser window stick after a test error or failure.

    opened by shadowc 9
  • Adjust the size of the window/screenshot

    Adjust the size of the window/screenshot

    I tried sth like:

    <?php
    
    declare(strict_types=1);
    
    namespace App\Tests\Acceptance\Util;
    
    use Facebook\WebDriver\WebDriverDimension;
    use Symfony\Component\Panther\Client as PantherClient;
    use Symfony\Component\Panther\PantherTestCase;
    
    class ScreenshotTestCase extends PantherTestCase
    {
        protected static function createPantherClient(array $options = [], array $kernelOptions = [], array $managerOptions = []): PantherClient
        {
            $client = self::createPantherClient();
    
            $size = new WebDriverDimension(1400, 4000);
            $client->manage()->window()->maximize()->setSize($size);
    
            return $client;
        }
    }
    

    But the size of the screenshot doesn't change 🤔

    opened by OskarStark 9
  • clickLink not working with special characters.

    clickLink not working with special characters.

    Hi, seems this wont work with special characters like »

    This is the code

    <?php
    include('../vendor/autoload.php');
    
    use PHPUnit\Framework\TestCase;
    use Symfony\Component\BrowserKit\HttpBrowser;
    use Symfony\Component\Panther\Client;
    
    class PantherTest
    {
    	public function __construct() 
    	{
    	}
    
    	function test()
    	{
    		try
    		{
    			$options = [
                '--headless',
                '--no-sandbox',
                '--disable-gpu'
                ];
    
    			$client = Client::createChromeClient('D:\xampp\htdocs\drivers\chromedriver.exe', null, $options, null);
    			$client->request('GET', 'https://www.lauritz.com/da/auktioner/?IST=0&SelectedTab=12');
    
    			$crawler = $client->waitFor('#FooterArticleControl_BackToTopText');
    			$client->takeScreenshot('screen.png'); // Yeah, screenshot!
    
    			// remove cookie popup
    			try {
    				$link = $client->clickLink('Tillad alle cookies');
    			}
    			catch(InvalidArgumentException $e){}
    
    			$client->clickLink('»');
    
    			$crawler = $client->waitFor('#FooterArticleControl_BackToTopText');
    			$client->takeScreenshot('screen1.png'); // Yeah, screenshot!
    
    			print "* success <br>";	
    		}
    		catch(Exception $e)
    		{
    			print "* Error " . var_dump($e) . "<br>";
    		}
    	}
    }
    
    $testclient = new PantherTest();
    $testclient->test();
    ?>
    
    

    The error is:

    object(Facebook\WebDriver\Exception\ElementClickInterceptedException)#18 (8) { ["message":protected]=> string(311) "element click intercepted: Element ... is not clickable at point (345, 602). Other element would receive the click: ...

    So it seems to select the menu ... instead of choosen », and the ... isnt active on first page..

    menu

    opened by iBuaSpaz 0
  • Documentation missing remote browser integration instructions

    Documentation missing remote browser integration instructions

    The doc say:

    Even if Chrome is the default choice, Panther can control any browser supporting the WebDriver protocol. It also supports remote browser testing services such as Selenium Grid (open source), SauceLabs and Browserstack.

    But, there is nowhere in the docs that explains how to use these remote browsers.

    How do you set up a remote browser, instead of Chrome. ie. using BrowserStack?

    opened by bscholl-parcelytics 0
  • Allow for a WebServer process to be started in a specific environment

    Allow for a WebServer process to be started in a specific environment

    I am using symfony 6.1 with the latest version of PHPUnit + Panther and I was trying to setup a test which checks if the RECAPTCHA is setup correctly. So what I needed was an environment in which the RECAPTCHA is enabled, but didn't want to modify the default 'panther' or 'test' environments to have the RECAPTCHA enabled for obvious reasons. So what I did was create a new 'panther_with_recaptcha' environment, which it's own config/packages/panther_with_recaptcha/ directory in which I've enabled RECAPTCHA.

    All good so far, until I tried to start the panther client with that environment. I though I could use the following:

            $this->client = static::createPantherClient(
                [
                    'env' => [
                        'APP_ENV' => 'panther_with_recaptcha'
                    ],
                ]
            );
    

    unfortunetally this didn't work, so I went digging and it turns out it's because the WebServerManager has the following in it's constructor:

            if (isset($_SERVER['PANTHER_APP_ENV'])) {
                if (null === $env) {
                    $env = [];
                }
                $env['APP_ENV'] = $_SERVER['PANTHER_APP_ENV'];
            }
    

    Thus, even if I've set the environment variable, it's not actually used. Is this the intended behavior?

    The workaround I've used is the following:

        public function testRecaptcha() {
            $old_panther_app_env = $_SERVER["PANTHER_APP_ENV"];
    
            // NOTE:
            //
            // Stopping the webserver, because
            // if we have multiple panther test cases running
            // once the webserver get's started it persists
            // throughout the whole test suite in order to improve performance.
            //
            // If we don't stop the server, then a previous panther test
            // would've started the server in the 'panther' environment
            // which would mean that this test will fail because
            // recaptcha is not enabled for the basic 'panther' environment
            $this->stopWebServer();
    
            // NOTE:
            //
            // This is currently the only way to force
            // the panther webserver to be started
            // in the 'panther_with_recaptcha' environment
            // which has recaptcha enabled in it.
            $_SERVER["PANTHER_APP_ENV"] = "panther_with_recaptcha";
    
            // NOTE:
            // Test has to be run using a webdriver
            // because recaptcha is implemented with javascript
            $this->client = static::createPantherClient();
           
           .....
            
            // NOTE:
            // Stopping the webserver, because
            // if we have multiple panther test cases running
            // once the webserver get's started it persists
            // throughout the whole test suite in order to improve performance.
            //
            // If we don't stop the server, then the next panther test
            // will also be running in the 'panther_with_recaptcha' env
            // which is not what we want.
            $this->stopWebServer();
            $_SERVER["PANTHER_APP_ENV"] = $old_panther_app_env;
    }
    

    But this seems like a bit of a hack to me.

    Would it make sense to change the WebServerManager constructor if statement to:

            if (null === $env) {
                $env = [];
            }
    
            if (isset($_SERVER['PANTHER_APP_ENV'])) {
                $env['APP_ENV'] = $env['APP_ENV'] ?? $_SERVER['PANTHER_APP_ENV'];
            }
    

    Or something along those lines?

    opened by shadydealer 0
  • Add loginUser() helper method to Panther client

    Add loginUser() helper method to Panther client

    Please see here for discussion context

    When doing E2E testing, it is common to need to login a user to perform certain actions. We often don't care about the login process for 99% of our tests, but we must still perform the steps each time. This can add unneccesary time and complexity onto the tests that could be avoided if only we had a way to simulate logins.

    It would be nice to have a helper function named loginUser() similar to the Application Test client. However, as we are doing E2E testing, we know that there is an added complexity of the environment settings of the alternate webserver (specifically how it handles session data). So, to solve this, we can pass SessionStorageInterface and firewallContext as arguments (but with sensisble defaults so they could be omitted in most cases).

    Example of calling code would be:

    $panther->logInUser(
        $user,
        new MockFileSessionStorage('/app/var/cache/panther/sessions'),
        'main'
    );
    

    Or simple (for most cases)

    $panther->logInUser($user);
    

    Have tested this approach (in userland code) and is working well for me. Was suggested in the discussion that I raise a PR for integration to Panther library code so here it is for your consideration. Please let me know if you need anything more from me.

    Richard.

    opened by plotbox-io 0
  • google account sign in button click not working

    google account sign in button click not working

    I'm working on a test script using PHP and Panther which requires me to first login to my Google account to then access my app. I've tried a whole bunch of different ways of getting the script to click on the Next button during sign-in:

    3C938906-2D12-432E-BD4A-ACDC4A4CE703

    The button html for the Next button is the following:

    8D926C11-5547-413E-B7EE-8A43B2788F9E

    This is my script with Panther:

    `use Symfony/Component/Panther/Client; require DIR.'vendor/autoload.php';

    $client = Client::createFirefoxClient; $client->request('GET' 'https://accounts.google.com/ServiceLogin?service=mail&passive=true&[email protected]'); $client->executeScript('document.querySelector('VfPpkd-LgbsSe').click(); $client->quite();`

    The script executes without any error but in no_headless the button never gets clicked and I'm not able to proceed. Any thoughts and ideas on what I'm doing wrong here?

    Thanks for the help in advance.

    opened by Kronenu 1
Releases(v2.0.1)
  • v2.0.1(Dec 2, 2021)

    What's Changed

    • Fix accessing PantherTestCaseTrait::$webServerDir before initialization by @kbond in https://github.com/symfony/panther/pull/523

    Full Changelog: https://github.com/symfony/panther/compare/v2.0.0...v2.0.1

    Source code(tar.gz)
    Source code(zip)
  • v2.0.0(Nov 30, 2021)

    What's Changed

    • Add support for Symfony 6 by @jordisala1991 in https://github.com/symfony/panther/pull/509
    • Allow deprecation-contracts 3 by @derrabus in https://github.com/symfony/panther/pull/519
    • Remove polyfill by @derrabus in https://github.com/symfony/panther/pull/522
    • Remove support for Symfony 4.4, add more type declarations for v2 by @chalasr in https://github.com/symfony/panther/pull/517

    New Contributors

    • @jordisala1991 made their first contribution in https://github.com/symfony/panther/pull/509
    • @weaverryan made their first contribution in https://github.com/symfony/panther/pull/513

    Full Changelog: https://github.com/symfony/panther/compare/v1.1.2...v2.0.0

    Source code(tar.gz)
    Source code(zip)
  • v1.1.2(Nov 30, 2021)

    What's Changed

    • Fix Form::offsetGet() return type by @chalasr in https://github.com/symfony/panther/pull/500
    • Allow deprecation-contracts 3 by @derrabus in https://github.com/symfony/panther/pull/519

    Full Changelog: https://github.com/symfony/panther/compare/v1.1.1...v1.1.2

    Source code(tar.gz)
    Source code(zip)
  • v1.1.1(Jul 14, 2021)

  • v1.1.0(Jul 13, 2021)

    • Add a PANTHER_DEVTOOLS environment variable to disable the dev tools
    • Add a PANTHER_ERROR_SCREENSHOT_ATTACH environment variable to attach screenshots to PHPUnit reports in the JUnit format
    • Add a chromedriver_arguments option to pass custom arguments to Chromedriver
    • Add an env option to pass custom environment variables to the built-in web server from PantherTestCase
    • Add the possibility to pass options to ChromeManager
    • Automatically find the Chromedriver binary installed by lanfest/binary-chromedriver
    • Symfony 5.3 compatibility
    • Fix assertions that were not working with clients other than PantherClient
    • Fix the ability to keep the window of the browser open when a test fail by using the --debug option
    • Fix the ServerExtension when registerClient() is called multiple times
    • Fix undefined constant errors when using PantherTestCaseTrait directly
    Source code(tar.gz)
    Source code(zip)
  • v1.0.1(Feb 4, 2021)

  • v1.0.0(Feb 3, 2021)

    • Add Client::waitForEnabled(), Client::waitForDisabled(), Client::waitForAttributeToContain() and Client::waitForAttributeToNotContain() methods
    • Add PantherTestCase::assertSelectorAttributeContains(), PantherTestCase::assertSelectorAttributeNotContains(), PantherTestCase::assertSelectorWillExist(), PantherTestCase::assertSelectorWillNotExist(), PantherTestCase::assertSelectorWillBeVisible(), PantherTestCase::assertSelectorWillNotBeVisible(), PantherTestCase::assertSelectorWillContain(), PantherTestCase::assertSelectorWillNotContain(), PantherTestCase::assertSelectorWillBeEnabled(), PantherTestCase::assertSelectorWillBeDisabled, PantherTestCase::assertSelectorAttributeWillContain(), and PantherTestCase::assertSelectorAttributeWillNotContain() assertions
    • Automatically take a screenshot when a test fail and if the PANTHER_ERROR_SCREENSHOT_DIR environment variable is set
    • Add missing return types
    • Breaking Change: Remove the deprecated PHPUnit listener, use the PHPUnit extension instead
    • Breaking Change: Remove deprecated support for Goutte, use HttpBrowser instead
    • Breaking Change: Remove deprecated support for PANTHER_CHROME_DRIVER_BINARY and PANTHER_GECKO_DRIVER_BINARY environment variables, add the binaries in your PATH instead
    • Don't allow unserializing classes with a destructor
    Source code(tar.gz)
    Source code(zip)
  • v0.9.0(Dec 29, 2020)

    • Breaking Change: ChromeDriver and geckodriver binaries are not included in the archive anymore and must be installed separately, refer to the documentation
    • PHP 8 compatibility
    • Add Client::waitForStaleness() method to wait for an element to be removed from the DOM
    • Add Client::waitForInvisibility() method to wait for an element to be invisible
    • Add Client::waitForElementToContain() method to wait for an element containing the given parameter
    • Add Client::waitForElementToNotContain() method to wait for an element to not contain the given parameter
    • Add PantherTestCase::assertSelectorIsVisible(), PantherTestCase::assertSelectorIsNotVisible(), PantherTestCase::assertSelectorIsEnabled() and PantherTestCase::assertSelectorIsDisabled() assertions
    • Fix baseUri not taken into account when using Symfony HttpBrowser
    Source code(tar.gz)
    Source code(zip)
  • v0.8.0(Oct 3, 2020)

    • Upgrade ChromeDriver to version 85.0.4183.87
    • Upgrade geckodriver to version 0.27.0
    • Add a Client::waitForVisibility() method to wait for an element to appear
    • Allow passing options to the browser manager from PantherTestCase::createPantherClient()
    • Add a Client::ping() method to check if the WebDriver connection is still active
    • Fix setting a new value to an input field when there is an existing value
    • Improve the error message when the web server crashes
    • Throw an explanative LogicException when driver is not started yet
    • Prevent timeouts caused by the integrated web server
    • Fix the value of cookie secure flags
    • Throw an exception when getting history (unsupported feature)
    • Add docs to use Panther with GitHub Actions
    • Various bug fixes and documentation improvements
    Source code(tar.gz)
    Source code(zip)
  • v0.7.1(Feb 24, 2020)

  • v0.7.0(Feb 20, 2020)

    • Add built-in support for Firefox (using GeckoDriver)
    • Add support for Symfony HttpBrowser
    • Deprecate Goutte support (use HttpBrowser instead)
    • Allow to configure RemoteWebDriver timeouts to when using Selenium
    • Allow to pass custom environment variables to the built-in web server
    • Fix some compatibility issues with PHP WebDriver 1.8
    • Upgrade ChromeDriver to version 80.0.3987.106
    • Prevent access to fixture files even if the web server is misconfigured
    Source code(tar.gz)
    Source code(zip)
  • v0.6.1(Jan 20, 2020)

    • Upgrade ChromeDriver to version 79.0.3945.36
    • Allow to pass custom timeouts as options of ChromeManager (connection_timeout_in_ms and request_timeout_in_ms)
    Source code(tar.gz)
    Source code(zip)
  • v0.6.0(Oct 28, 2019)

    • Add compatibility with Symfony 5
    • Allow to use Client::waitFor() to wait for invisible elements
    • Add support to pass XPath expressions as parameters of Client::waitFor()
    • Fix Crawler::attr() signature (it can return null)
    • Deprecate ServerListener (use ServerExtension instead)
    • Upgrade ChromeDriver to version 78.0.3904.70
    • New logo
    • Various docs fixes and improvements

    Source code(tar.gz)
    Source code(zip)
  • v0.5.2(Aug 27, 2019)

  • v0.5.1(Aug 23, 2019)

    • Allow to override the APP_ENV environment variable passed to the web server by setting PANTHER_APP_ENV
    • Fix using assertions with a client created through PantherTestCase::createClient()
    • Don't call PantherTestCase::getClient() if this method isn't static
    • Fix remaining deprecations
    Source code(tar.gz)
    Source code(zip)
  • v0.5.0(Aug 22, 2019)

    • Add support for Crawler test assertions
    • Add the PantherTestCase::createAdditionalPantherClient() to retrieve additional isolated browsers, useful to test applications using Mercure or WebSocket
    • Improved support for non-standard web server directories
    • Allow the integrated web server to start even if the homepage doesn't return a 200 HTTP status code
    • Increase default timeouts from 5 seconds to 30 seconds
    • Improve error messages
    • Add compatibility with Symfony 4.3
    • Upgrade ChromeDriver to version 76.0.3809.68
    • Various quality improvements
    Source code(tar.gz)
    Source code(zip)
  • v0.4.1(Jun 20, 2019)

  • v0.4.0(Jun 4, 2019)

    • Speed up the boot sequence
    • Add basic support for file uploads
    • Add a readinessPath option to use a custom path for server readiness detection
    • Fix the behavior of ChoiceFormField::getValue() to be consistent with other BrowserKit implementations
    • Ensure to clean the previous content of field when using TextareaFormField::setValue() and InputFormField::setValue()
    Source code(tar.gz)
    Source code(zip)
  • v0.3.0(Feb 27, 2019)

    • Add a new API to manipulate the mouse
    • Keep the browser window open on fail, when running in non-headless mode
    • Automatically open Chrome DevTools when running in non-headless mode
    • PHPUnit 8 compatibility
    • Add a PHPUnit extension to keep alive the webserver and the client between tests
    • Change the default port of the web server to 9080 to prevent a conflit with Xdebug
    • Allow to use an external web server instead of the built-in one for testing
    • Allow to use a custom router script
    • Allow to use a custom Chrome binary
    Source code(tar.gz)
    Source code(zip)
  • v0.2.0(Sep 26, 2018)

    • Compatibility with PHP 7.1 (the minimal version was 7.2 before)
    • Added a PHPUnit listener (ServerListener) to keep the webserver active during the entire test suite
    • The kernel is now always booted when in a Symfony test, and it's possible to configure it
    • Added a new Client::refreshCrawler() method to refresh the crawler when WebDriver elements are stale
    • Possibility to pass custom arguments to ChromeDriver
    • Added JS execution capabilities to the Client class
    • Improved documentation
    • Improved CI
    Source code(tar.gz)
    Source code(zip)
  • v0.1.0(Jul 17, 2018)

Owner
Symfony
Symfony
Crawlzone is a fast asynchronous internet crawling framework aiming to provide open source web scraping and testing solution.

Crawlzone is a fast asynchronous internet crawling framework aiming to provide open source web scraping and testing solution. It can be used for a wide range of purposes, from extracting and indexing structured data to monitoring and automated testing. Available for PHP 7.3, 7.4, 8.0.

null 68 Dec 27, 2022
Libraries and scripts for crawling the TYPO3 page tree. Used for re-caching, re-indexing, publishing applications etc.

Libraries and scripts for crawling the TYPO3 page tree. Used for re-caching, re-indexing, publishing applications etc.

AOE 0 Sep 14, 2021
A small example of crawling another website and extracting the required information from it to save the website wherever we need it

A small example of crawling another website and extracting the required information from it to save the website wherever we need it Description This s

Mohammad Qasemi 9 Sep 24, 2022
Symfony bundle for Roach PHP

roach-php-bundle Symfony bundle for Roach PHP. Roach is a complete web scraping toolkit for PHP. It is a shameless clone heavily inspired by the popul

Pisarev Alexey 7 Sep 28, 2022
Library for Rapid (Web) Crawler and Scraper Development

Library for Rapid (Web) Crawler and Scraper Development This package provides kind of a framework and a lot of ready to use, so-called steps, that you

crwlr.software 60 Nov 30, 2022
PHP Scraper - an highly opinionated web-interface for PHP

PHP Scraper An opinionated & limited way to scrape the web using PHP. The main goal is to get stuff done instead of getting distracted with xPath sele

Peter Thaleikis 327 Dec 30, 2022
A configurable and extensible PHP web spider

Note on backwards compatibility break: since v0.5.0, Symfony EventDispatcher v3 is no longer supported and PHP Spider requires v4 or v5. If you are st

Matthijs van den Bos 1.3k Dec 28, 2022
Goutte, a simple PHP Web Scraper

Goutte, a simple PHP Web Scraper Goutte is a screen scraping and web crawling library for PHP. Goutte provides a nice API to crawl websites and extrac

null 9.1k Jan 1, 2023
Goutte, a simple PHP Web Scraper

Goutte, a simple PHP Web Scraper Goutte is a screen scraping and web crawling library for PHP. Goutte provides a nice API to crawl websites and extrac

null 9.1k Jan 4, 2023
Roach is a complete web scraping toolkit for PHP

?? Roach A complete web scraping toolkit for PHP About Roach is a complete web scraping toolkit for PHP. It is heavily inspired (read: a shameless clo

Roach PHP 1.1k Jan 3, 2023
This script scrapes the HTML from different web pages to get the information from the video and you can use it in your own video player.

XVideos PornHub RedTube API This script scrapes the HTML from different web pages to get the information from the video and you can use it in your own

null 57 Dec 16, 2022
A program to scrape online web-content (APIs, RSS Feeds, or Websites) and notify if search term was hit.

s3n Search-Scan-Save-Notify A program to scrape online web-content (APIs, RSS Feeds, or Websites) and notify if search term was hit. It is based on PH

Aamer 11 Nov 8, 2022
Get info from any web service or page

Embed PHP library to get information from any web page (using oembed, opengraph, twitter-cards, scrapping the html, etc). It's compatible with any web

Oscar Otero 1.9k Jan 1, 2023
The most integrated web scraper package for Laravel.

Laravel Scavenger The most integrated web scraper package for Laravel. Top Features Scavenger provides the following features and more out-the-box. Ea

Reliq Arts 134 Jan 4, 2023
Property page web scrapper

Property page web scrapper This tool was built to expermiment with extracting features for property pages on websites like booking.com and Airbnb. Thi

Vaugen Wake 2 Feb 24, 2022
PHP library to Scrape website into entity easily

Scraper Scraper can handle multiple request type and transform them into object in order to create some API. Installation composer require rem42/scrap

null 5 Dec 18, 2021
PHP scraper for ZEE5 Live Streaming URL's Using The Channel ID and Direct Play Anywhere

It can scrape ZEE5 Live Streaming URL's Using The Channel ID and Direct Play Anywhere

null 1 Mar 24, 2022
🕷 CrawlerDetect is a PHP class for detecting bots/crawlers/spiders via the user agent

crawlerdetect.io About CrawlerDetect CrawlerDetect is a PHP class for detecting bots/crawlers/spiders via the user agent and http_from header. Current

Mark Beech 1.7k Dec 30, 2022
:spider: The progressive PHP crawler framework! 优雅的渐进式PHP采集框架。

QueryList QueryList is a simple, elegant, extensible PHP Web Scraper (crawler/spider) ,based on phpQuery. API Documentation 中文文档 Features Have the sam

Jaeger(黄杰) 2.5k Dec 27, 2022