The best php curl library.

Related tags

HTTP php-curl
Overview

中文文档

About

Implemented by using php-curl internal io event with high performance,high universality,high extensibility which especially suitable for massive tasks and complex logic cases.

Demand

PHP: >=5.3

Install

composer require ares333/php-curl

Features

  1. Extremely low cpu and memory consumption with high performance(download 3000 html pages per second,download images with 1000Mbps on server with 1Gbps interface).
  2. All curl options are exposed directly which enables high universality and high extensibility.
  3. Api is very simple.
  4. Support process disruption and resume from last running state.
  5. Support dynamic tasks.
  6. Support transparent file cache.
  7. Support retry failed tasks automatically.
  8. Support global config,task config,callback config on same format and priority is from low to high.
  9. All configs can be changed on the fly and take effect immediately.

Work Flow

Curl::add() add tasks to task pool.Curl::start() start the event loop and block.Events(onSuccess,onFail,onInfo,onTask...) will be triggered and callbacks will be called on the fly.The loop finished when all tasks finished.

Tutorial

basic

use Ares333\Curl\Curl;
$curl = new Curl();
$curl->add(
    array(
        'opt' => array(
            CURLOPT_URL => 'http://baidu.com',
            CURLOPT_RETURNTRANSFER => true
        ),
        'args' => 'This is user argument'
    ),
    function ($r, $args) {
        echo "Request success for " . $r['info']['url'] . "\n";
        echo "\nHeader info:\n";
        print_r($r['info']);
        echo "\nRaw header:\n";
        print_r($r['header']);
        echo "\nArgs:\n";
        print_r($args);
        echo "\n\nBody size:\n";
        echo strlen($r['body']) . ' bytes';
        echo "\n";
    });
$curl->start();

file download

use Ares333\Curl\Curl;
$curl = new Curl();
$url = 'https://www.baidu.com/img/bd_logo1.png';
$file = __DIR__ . '/download.png';
// $fp is closed automatically on download finished.
$fp = fopen($file, 'w');
$curl->add(
    array(
        'opt' => array(
            CURLOPT_URL => $url,
            CURLOPT_FILE => $fp,
            CURLOPT_HEADER => false
        ),
        'args' => array(
            'file' => $file
        )
    ),
    function ($r, $args) {
        if($r['info']['http_code']==200) {
            echo "download finished successfully, file=$args[file]\n";
        }else{
            echo "download failed\n";
        }
    })->start();

task callback

Task can be added in task callback. See more details in Curl::$onTask.

use Ares333\Curl\Toolkit;
use Ares333\Curl\Curl;
$toolkit = new Toolkit();
$toolkit->setCurl();
$curl = $toolkit->getCurl();
$curl->maxThread = 1;
$curl->onTask = function ($curl) {
    static $i = 0;
    if ($i >= 50) {
        return;
    }
    $url = 'http://www.baidu.com';
    /** @var Curl $curl */
    $curl->add(
        array(
            'opt' => array(
                CURLOPT_URL => $url . '?wd=' . $i ++
            )
        ));
};
$curl->start();

running info

use Ares333\Curl\Toolkit;
use Ares333\Curl\Curl;
$curl = new Curl();
$toolkit = new Toolkit();
$curl->onInfo = array(
    $toolkit,
    'onInfo'
);
$curl->maxThread = 2;
$url = 'http://www.baidu.com';
for ($i = 0; $i < 100; $i ++) {
    $curl->add(
        array(
            'opt' => array(
                CURLOPT_URL => $url . '?wd=' . $i
            )
        ));
}
$curl->start();

Run in cli and will output with following format:

SPD    DWN  FNH  CACHE  RUN  ACTIVE  POOL  QUEUE  TASK  FAIL  
457KB  3MB  24   0      3    3       73    0      100   0

'onInfo' callback will receive all information.The default callback only show part of it.

SPD:Download speed
DWN:Bytes downloaded
FNH:Task count which has finished
CACHE:Cache count which were used 
RUN:Task running count
ACTIVE:Task count which has IO activities
POOL:Task count in task pool
QUEUE:Task count which has finished and waiting for onSuccess callback to process
TASK:Task count has been added to the task pool
FAIL:Task count which has failed after retry.

transparent cache

use Ares333\Curl\Toolkit;
use Ares333\Curl\Curl;
$curl = new Curl();
$toolkit = new Toolkit();
$curl->onInfo = array(
    $toolkit,
    'onInfo'
);
$curl->maxThread = 2;
$curl->cache['enable'] = true;
$curl->cache['dir'] = __DIR__ . '/output/cache';
if (! is_dir($curl->cache['dir'])) {
    mkdir($curl->cache['dir'], 0755, true);
}
$url = 'http://www.baidu.com';
for ($i = 0; $i < 20; $i ++) {
    $curl->add(
        array(
            'opt' => array(
                CURLOPT_URL => $url . '?wd=' . $i
            )
        ));
}
$curl->start();

Run the script second time and will output:

SPD  DWN  FNH  CACHE  RUN  ACTIVE  POOL  QUEUE  TASK  FAIL  
0KB  0MB  20   20     0    0       0     0      20    0

The result indicate that all tasks is using cache and there is no network activity.

dynamic tasks

use Ares333\Curl\Curl;
$curl = new Curl();
$url = 'http://baidu.com';
$curl->add(array(
    'opt' => array(
        CURLOPT_URL => $url
    )
), 'cb1');
echo "add $url\n";
$curl->start();

function cb1($r)
{
    echo "finish " . $r['info']['url'] . "\n";
    $url = 'http://bing.com';
    $r['curl']->add(
        array(
            'opt' => array(
                CURLOPT_URL => $url
            )
        ), 'cb2');
    echo "add $url\n";
}

function cb2($r)
{
    echo "finish " . $r['info']['url'] . "\n";
}

Output is as below:

add http://baidu.com
finish https://www.baidu.com/
add http://bing.com
finish http://cn.bing.com/

Finished url has '/' suffix because curl has processed the 3xx redirect automatically(Curl::$opt[CURLOPT_FOLLOWLOCATION]=true). Curl::onTask should be used to deal with massive tasks.

Curl (src/Curl.php Core functionality)

public $maxThread = 10

Max work parallels which can be change on the fly.

public $maxTry = 3

Max retry before onFail event is triggered.

public $opt = array ()

Global CURLOPT_* which can be overwritten by task config.

public $cache = array(
    'enable' => false,
    'compress' => 0, //0-9,6 is a good choice if you want use compress.
    'dir' => null, //Cache dir which must exists.
    'expire' => 86400,
    'verifyPost' => false //If http post will be part of cache id.
);

Global cache config.Cache id is mapped from url.The config can be overwritten by task config and onSuccess callback return value with same format.

public $taskPoolType = 'queue'

stack or queue.

public $onTask = null

Will be triggered when work parallels is less than Curl::$maxThread and task pool is empty.The callback parameter is current Curl instance.

public $onInfo = null

Running state callback is triggered on IO events with max frequency 1/s.The parameters are as below:

  1. $info array with two keys 'all' and 'running'.Key 'running' contains response header(curl_getinfo()) for each running task.Key 'all' contains global information with keys as below:
    • $info['all']['downloadSpeed'] Download speed.
    • $info['all']['bodySize'] Body sized downloaded.
    • $info['all']['headerSize'] Header size downloaded.
    • $info['all']['activeNum'] Task has IO activity.
    • $info['all']['queueNum'] Tasks waiting for onSuccess.
    • $info['all']['finishNum'] Tasks has finished.
    • $info['all']['cacheNum'] Cache hits.
    • $info['all']['failNum'] Failed tasks number after retry.
    • $info['all']['taskNum'] Task number in the task pool.
    • $info['all']['taskRunningNum'] Running task number.
    • $info['all']['taskPoolNum'] Task pool number.
    • $info['all']['taskFailNum'] Retrying task number.
  2. Current Curl instance.
  3. Is last call or not.
public $onEvent = null

Triggered on IO events.The callback parameter is current Curl instance.

public $onFail = null

Global callback for failed task which can be overwritten by task 'onTask'.The callback receive two parameters.

  1. array with keys as below:
  • errorCode CURLE_* constants.
  • errorMsg Error message.
  • info Response header.
  • curl Current Curl instance.
  1. $item['args'] value from Curl::add().
public function add(array $item, $onSuccess = null, $onFail = null, $ahead = null)

Add one task to the pool.

  • $item
    1. $item['opt']=array() CURLOPT_* for current task.
    2. $item['args'] Parameters for callbacks.
    3. $item['cache']=array() Cache config for current task.
  • $onSuccess Triggered on task finish.
    • Callback has two Parameters:
      1. $result Array with keys as below:
        • $result['info'] Response header.
        • $result['curl'] Current Curl instance.
        • $result['body'] Response body.Not exist in download task.
        • $result['header'] Raw response header.Exists when CURLOPT_HEADER was enabled.
        • $result['cacheFile'] Exists when cache is used.
      2. Value from $item['args']
    • Values can be returned.Must be array if exist.Array keys is as below:
      • cache Same format with Curl::$cache.This is the last chance to control caching.
  • $onFail Overwrite Curl::$onFail。
  • $ahead Add to high priority poll or not.

Return: current Curl instance.

public function start()

Start the event loop and block.

public function stop()

Stop the event loop and return unprocessed tasks.

public function parseResponse($response)

Parse http header and body from response.

public function getCacheFile($url, $post = null)

Generate relative cache path.

Toolkit (src/Toolkit.php Necessary tools)

function setCurl($curl = null)

Default Curl instance is used if $curl is null.

The default instance will initialize Curl::$opt,Curl::onInfo,Curl::onFail. Curl::$opt initial values are as follows:

array(
    CURLINFO_HEADER_OUT => true,
    CURLOPT_HEADER => true,
    CURLOPT_CONNECTTIMEOUT => 10,
    CURLOPT_TIMEOUT => 30,
    CURLOPT_AUTOREFERER => true,
    CURLOPT_USERAGENT => 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/59.0.3071.115 Safari/537.36',
    CURLOPT_RETURNTRANSFER => true,
    CURLOPT_FOLLOWLOCATION => true,
    CURLOPT_SSL_VERIFYHOST => false,
    CURLOPT_SSL_VERIFYPEER => false,
    CURLOPT_MAXREDIRS => 5
)
function onFail($error, $args)

Default fail callback.See Curl::$onFail for details.

function onInfo($info)

Default info callback.See Curl::onInfo for details.

The method can be triggered manually with a string parameter which will be added to output buffer.The purpose is to avoid interference of shell control characters.

function htmlEncode($html, $in = null, $out = 'UTF-8', $mode = 'auto')

Powerful html encoding transformer which can get current encoding automatically and replace html encoding value in <head></head>. Parameters:

  • $html Html string.
  • $in Current encoding.It's best to specify one.
  • $out Target encoding.
  • $mode auto|iconv|mb_convert_encoding.

Return: New encoded html.

function isUrl($url)

Full url or not.Return bool.

function formatUrl($url)

Replace space,process scheme and hosts and remove anchor etc.

function buildUrl(array $parse)

Inverse function for parse_url().

function uri2url($uri, $urlCurrent)

Transform uri to full url for currentPage.$urlCurrent should be redirected after 3xx.

function url2uri($url, $urlCurrent)

Transform full url to uri for currentPage.$urlCurrent should be redirected after 3xx.

function url2dir($url)

Transform full url to dir.$urlCurrent should be redirected after 3xx.

function url2absolute($url, $urlCurrent)

Combine a base URL and a relative URL to produce a new absolute URL.

function urlRemoveDotSegments($path)

Filter out "." and ".." segments from a URL's path and return the result.

function getCurl()

Return current Curl instance.

Comments
  •  Undefined index: scheme

    Undefined index: scheme

    你好,出现大量的scheme错误,是什么原因?怎么解决 E_NOTICE: ErrorException: Undefined index: scheme in /Users/Arist/Git/php-curl/src/Toolkit.php:302 Stack trace:

    function urlFormater($url)
    {
        if (! $this->isUrl($url)) {
            return;
        }
        $url = trim($url);
        $url = str_replace(' ', '+', $url);
        $parse = parse_url($url);
        strtolower($parse['scheme']);
        strtolower($parse['host']);
        return $this->buildUrl($parse);
    }
    
    opened by aristoler 5
  • 实时速度等信息无法显示

    实时速度等信息无法显示

    设置 $curl->onInfo = array( $toolkit, 'onInfo' ); 后报错。

    开始下载... E_WARNING: ErrorException: call_user_func() expects parameter 1 to be a valid callback, first array member is not a valid class name or object in D:\WWW\1spider\mcurl\vendor\ares333\php-curl\src\Curl.php:415 Stack trace: #0 [internal function]: ErrorHandler::{closure}(2, 'call_user_func(...', 'D:\WWW\1spider\...', 415, Array) #1 D:\WWW\1spider\mcurl\vendor\ares333\php-curl\src\Curl.php(415): call_user_func(Array, Array, Object(Ares333\Curl\Curl)) #2 D:\WWW\1spider\mcurl\vendor\ares333\php-curl\src\Curl.php(288): Ares333\Curl\Curl->onInfo() #3 D:\WWW\1spider\shokdown.php(46): Ares333\Curl\Curl->start() #4 {main}

    程序能正常执行,但会不停报上面的错误。信息也不能显示。

    opened by googles8 4
  • The args parameter is set to unavailable.

    The args parameter is set to unavailable.

    The args parameter is set, but it is not received by the backend. This is the code: Ares333\Curl\Curl; $curl = new Curl(); $curl->add( array( 'opt' => array( CURLOPT_URL => 'http://127.0.0.1:8080/cs/request.php', CURLOPT_RETURNTRANSFER => true ), 'args' => array( "word"=>"你好" ) ), function ($r, $args) { echo "Request success for " . $r['info']['url'] . "
    "; echo "
    Header info:
    "; print_r($r['info']); echo "
    Raw header:
    "; print_r($r['header']); echo "
    Args:
    "; print_r($args); echo "

    Body size:
    "; echo strlen($r['body']) . ' bytes'; echo "
    "; echo "
    Body:
    "; echo "

    ".htmlspecialchars($r['body'])."
    "; // print_r($r); }); $curl->start();

    This is the return: GET:Array ( )

    opened by 123xuwu 3
  • 下载后所有的中文都是Unicode

    下载后所有的中文都是Unicode

    网站:http://www.bjszxx.cn

    部分源码:

    <meta http-equiv="Content-Type" content="text/html;charset=UTF-8">
        <meta charset="UTF-8">
        <!--TODO LSY &#28155;&#21152;&#35270;&#21475;&#26174;&#31034;-->
        <meta name="renderer" content="webkit">
        <meta name="viewport" content="width=device-width,initial-scale=1.0,maximum-scale=1.0,minimum-scale=1.0,user-scalable=no">
        <meta http-equiv="Cache-Control" content="no-transform">
        <meta http-equiv="Cache-Control" content="no-siteapp">
       <title>&#19977;&#21482;&#23567;&#29066;&#20799;&#31461;&#25668;&#24433; - &#19987;&#19994;&#21271;&#20140;&#20799;&#31461;&#25668;&#24433;&#26426;&#26500;</title>
       <meta name="description" content="&#19977;&#21482;&#23567;&#29066;&#20799;&#31461;&#25668;&#24433;,&#26159;&#20840;&#22269;&#30693;&#21517;&#30340;&#19987;&#19994;&#21271;&#20140;&#20799;&#31461;&#25668;&#24433;&#26426;&#26500;&#12290;&#20026;&#23458;&#25143;&#25552;&#20379;&#20799;&#31461;&#25668;&#24433;&#22242;&#36141;,&#20799;&#31461;&#25668;&#24433;&#20316;&#21697;&#27983;&#35272;,&#39044;&#32422;&#22871;&#31995;,&#20146;&#23376;&#25668;&#24433;&#31561;&#26381;&#21153;&#12290;">
        <link rel="stylesheet" href="template/default/css/main.css">
    
    opened by luoei 3
  • 图片下载零字节

    图片下载零字节

    php环境:7.1

    Warning: file_get_contents(): Filename cannot be empty in /xxx/php-curlmulti-master/src/Core.php on line 462

    Warning: fopen(/xxx/php-curlmulti-master/demo/base/static/xxx/indoors/../uploads/allimg/161208/1-16120Q61432.jpg): failed to open stream: Too many open files in /xxx/php-curlmulti-master/src/AutoClone.php on line 224

    opened by luoei 2
  • 克隆网站到2G,一直没结束

    克隆网站到2G,一直没结束

    网站如下: http://www.jujiaobaby.com PHP版本:7.2 OSX 版本:10.13.1 部分错误日志: E_WARNING: ErrorException: file_put_contents(/Users/luoei/Downloads/php-curl-master-1/demo/output/clone/http_www.jujiaobaby.com/index.php?s=%2FHome%2Fwz%2Fxiang%2Fid%2F9.html): failed to open stream: Too many open files in /Users/luoei/Downloads/php-curl-master-1/src/HttpClone.php:306 Stack trace: #0 [internal function]: ErrorHandler::{closure}(2, 'file_put_conten...', '/Users/luoei/Do...', 306, Array) #1 /Users/luoei/Downloads/php-curl-master-1/src/HttpClone.php(306): file_put_contents('/Users/luoei/Do...', 'onProcess(Array, Array) #3 [internal function]: HttpCloneDemo->onProcess(Array, Array) #4 /Users/luoei/Downloads/php-curl-master-1/src/Curl.php(492): call_user_func(Array, Array, Array) #5 /Users/luoei/Downloads/php-curl-master-1/src/Curl.php(331): Ares333\Curl\Curl->onProcess(Array, Array) #6 /Users/luoei/Downloads/php-curl-master-1/src/HttpClone.php(124): Ares333\Curl\Curl->start() #7 /Users/luoei/Downloads/php-curl-master-1/demo/www.jujiaobaby.com.php(75): Ares333\Curl\HttpClone->start() #8 {main} E_WARNING: ErrorException: file_put_contents(/Users/luoei/Downloads/php-curl-master-1/demo/output/cache/a19/2f3/70d843e37b8152072481cbc4e3): failed to open stream: Too many open files in /Users/luoei/Downloads/php-curl-master-1/src/Curl.php:562 Stack trace: #0 [internal function]: ErrorHandler::{closure}(2, 'file_put_conten...', '/Users/luoei/Do...', 562, Array) #1 /Users/luoei/Downloads/php-curl-master-1/src/Curl.php(562): file_put_contents('/Users/luoei/Do...', 'x\x9C\xED]}s\xD3V\xD6\xFF\ef\xF8\x0Ew...', 2) #2 /Users/luoei/Downloads/php-curl-master-1/src/Curl.php(499): Ares333\Curl\Curl->cache(Array, Array) #3 /Users/luoei/Downloads/php-curl-master-1/src/Curl.php(331): Ares333\Curl\Curl->onProcess(Array, Array) #4 /Users/luoei/Downloads/php-curl-master-1/src/HttpClone.php(124): Ares333\Curl\Curl->start() #5 /Users/luoei/Downloads/php-curl-master-1/demo/www.jujiaobaby.com.php(75): Ares333\Curl\HttpClone->start() #6 {main}

    E_WARNING: ErrorException: file_put_contents(/Users/luoei/Downloads/php-curl-master-1/demo/output/cache/a19/2f3/70d843e37b8152072481cbc4e3): failed to open stream: Too many open files in /Users/luoei/Downloads/php-curl-master-1/src/Curl.php:562 Stack trace: #0 [internal function]: ErrorHandler::{closure}(2, 'file_put_conten...', '/Users/luoei/Do...', 562, Array) #1 /Users/luoei/Downloads/php-curl-master-1/src/Curl.php(562): file_put_contents('/Users/luoei/Do...', 'x\x9C\xED]}s\xD3V\xD6\xFF\ef\xF8\x0Ew...', 2) #2 /Users/luoei/Downloads/php-curl-master-1/src/Curl.php(499): Ares333\Curl\Curl->cache(Array, Array) #3 /Users/luoei/Downloads/php-curl-master-1/src/Curl.php(331): Ares333\Curl\Curl->onProcess(Array, Array) #4 /Users/luoei/Downloads/php-curl-master-1/src/HttpClone.php(124): Ares333\Curl\Curl->start() #5 /Users/luoei/Downloads/php-curl-master-1/demo/www.jujiaobaby.com.php(75): Ares333\Curl\HttpClone->start() #6 {main} E_WARNING: ErrorException: file_put_contents(/Users/luoei/Downloads/php-curl-master-1/demo/output/clone/http_www.jujiaobaby.com/index.php?s=%2FHome%2Fwz%2Fxiang%2Fid%2F8.html): failed to open stream: Too many open files in /Users/luoei/Downloads/php-curl-master-1/src/HttpClone.php:306 Stack trace: #0 [internal function]: ErrorHandler::{closure}(2, 'file_put_conten...', '/Users/luoei/Do...', 306, Array) #1 /Users/luoei/Downloads/php-curl-master-1/src/HttpClone.php(306): file_put_contents('/Users/luoei/Do...', 'onProcess(Array, Array) #3 [internal function]: HttpCloneDemo->onProcess(Array, Array) #4 /Users/luoei/Downloads/php-curl-master-1/src/Curl.php(492): call_user_func(Array, Array, Array) #5 /Users/luoei/Downloads/php-curl-master-1/src/Curl.php(331): Ares333\Curl\Curl->onProcess(Array, Array) #6 /Users/luoei/Downloads/php-curl-master-1/src/HttpClone.php(124): Ares333\Curl\Curl->start() #7 /Users/luoei/Downloads/php-curl-master-1/demo/www.jujiaobaby.com.php(75): Ares333\Curl\HttpClone->start() #8 {main}

    E_WARNING: ErrorException: file_put_contents(/Users/luoei/Downloads/php-curl-master-1/demo/output/clone/http_www.jujiaobaby.com/index.php?s=%2FHome%2Fwz%2Fxiang%2Fid%2F8.html): failed to open stream: Too many open files in /Users/luoei/Downloads/php-curl-master-1/src/HttpClone.php:306 Stack trace: #0 [internal function]: ErrorHandler::{closure}(2, 'file_put_conten...', '/Users/luoei/Do...', 306, Array) #1 /Users/luoei/Downloads/php-curl-master-1/src/HttpClone.php(306): file_put_contents('/Users/luoei/Do...', 'onProcess(Array, Array) #3 [internal function]: HttpCloneDemo->onProcess(Array, Array) #4 /Users/luoei/Downloads/php-curl-master-1/src/Curl.php(492): call_user_func(Array, Array, Array) #5 /Users/luoei/Downloads/php-curl-master-1/src/Curl.php(331): Ares333\Curl\Curl->onProcess(Array, Array) #6 /Users/luoei/Downloads/php-curl-master-1/src/HttpClone.php(124): Ares333\Curl\Curl->start() #7 /Users/luoei/Downloads/php-curl-master-1/demo/www.jujiaobaby.com.php(75): Ares333\Curl\HttpClone->start() #8 {main} E_WARNING: ErrorException: file_put_contents(/Users/luoei/Downloads/php-curl-master-1/demo/output/cache/6d9/28c/80a14279cfdbc8cc5276c5f405): failed to open stream: Too many open files in /Users/luoei/Downloads/php-curl-master-1/src/Curl.php:562 Stack trace: #0 [internal function]: ErrorHandler::{closure}(2, 'file_put_conten...', '/Users/luoei/Do...', 562, Array) #1 /Users/luoei/Downloads/php-curl-master-1/src/Curl.php(562): file_put_contents('/Users/luoei/Do...', 'x\x9C\xED]\xF9S\eG\xF6\xFF\xD9\xAE\xF2\xFF0...', 2) #2 /Users/luoei/Downloads/php-curl-master-1/src/Curl.php(499): Ares333\Curl\Curl->cache(Array, Array) #3 /Users/luoei/Downloads/php-curl-master-1/src/Curl.php(331): Ares333\Curl\Curl->onProcess(Array, Array) #4 /Users/luoei/Downloads/php-curl-master-1/src/HttpClone.php(124): Ares333\Curl\Curl->start() #5 /Users/luoei/Downloads/php-curl-master-1/demo/www.jujiaobaby.com.php(75): Ares333\Curl\HttpClone->start() #6 {main}

    E_WARNING: ErrorException: file_put_contents(/Users/luoei/Downloads/php-curl-master-1/demo/output/cache/6d9/28c/80a14279cfdbc8cc5276c5f405): failed to open stream: Too many open files in /Users/luoei/Downloads/php-curl-master-1/src/Curl.php:562 Stack trace: #0 [internal function]: ErrorHandler::{closure}(2, 'file_put_conten...', '/Users/luoei/Do...', 562, Array) #1 /Users/luoei/Downloads/php-curl-master-1/src/Curl.php(562): file_put_contents('/Users/luoei/Do...', 'x\x9C\xED]\xF9S\eG\xF6\xFF\xD9\xAE\xF2\xFF0...', 2) #2 /Users/luoei/Downloads/php-curl-master-1/src/Curl.php(499): Ares333\Curl\Curl->cache(Array, Array) #3 /Users/luoei/Downloads/php-curl-master-1/src/Curl.php(331): Ares333\Curl\Curl->onProcess(Array, Array) #4 /Users/luoei/Downloads/php-curl-master-1/src/HttpClone.php(124): Ares333\Curl\Curl->start() #5 /Users/luoei/Downloads/php-curl-master-1/demo/www.jujiaobaby.com.php(75): Ares333\Curl\HttpClone->start() #6 {main} E_WARNING: ErrorException: file_put_contents(/Users/luoei/Downloads/php-curl-master-1/demo/output/clone/http_www.jujiaobaby.com/index.php?s=%2FHome%2FJujiao%2Fabout%2Fid%2F8.html): failed to open stream: Too many open files in /Users/luoei/Downloads/php-curl-master-1/src/HttpClone.php:306 Stack trace: #0 [internal function]: ErrorHandler::{closure}(2, 'file_put_conten...', '/Users/luoei/Do...', 306, Array) #1 /Users/luoei/Downloads/php-curl-master-1/src/HttpClone.php(306): file_put_contents('/Users/luoei/Do...', 'onProcess(Array, Array) #3 [internal function]: HttpCloneDemo->onProcess(Array, Array) #4 /Users/luoei/Downloads/php-curl-master-1/src/Curl.php(492): call_user_func(Array, Array, Array) #5 /Users/luoei/Downloads/php-curl-master-1/src/Curl.php(331): Ares333\Curl\Curl->onProcess(Array, Array) #6 /Users/luoei/Downloads/php-curl-master-1/src/HttpClone.php(124): Ares333\Curl\Curl->start() #7 /Users/luoei/Downloads/php-curl-master-1/demo/www.jujiaobaby.com.php(75): Ares333\Curl\HttpClone->start() #8 {main}

    E_WARNING: ErrorException: file_put_contents(/Users/luoei/Downloads/php-curl-master-1/demo/output/clone/http_www.jujiaobaby.com/index.php?s=%2FHome%2FJujiao%2Fabout%2Fid%2F8.html): failed to open stream: Too many open files in /Users/luoei/Downloads/php-curl-master-1/src/HttpClone.php:306 Stack trace: #0 [internal function]: ErrorHandler::{closure}(2, 'file_put_conten...', '/Users/luoei/Do...', 306, Array) #1 /Users/luoei/Downloads/php-curl-master-1/src/HttpClone.php(306): file_put_contents('/Users/luoei/Do...', 'onProcess(Array, Array) #3 [internal function]: HttpCloneDemo->onProcess(Array, Array) #4 /Users/luoei/Downloads/php-curl-master-1/src/Curl.php(492): call_user_func(Array, Array, Array) #5 /Users/luoei/Downloads/php-curl-master-1/src/Curl.php(331): Ares333\Curl\Curl->onProcess(Array, Array) #6 /Users/luoei/Downloads/php-curl-master-1/src/HttpClone.php(124): Ares333\Curl\Curl->start() #7 /Users/luoei/Downloads/php-curl-master-1/demo/www.jujiaobaby.com.php(75): Ares333\Curl\HttpClone->start() #8 {main} E_WARNING: ErrorException: file_put_contents(/Users/luoei/Downloads/php-curl-master-1/demo/output/cache/199/21e/86909667f5f14b89bf1291a251): failed to open stream: Too many open files in /Users/luoei/Downloads/php-curl-master-1/src/Curl.php:562 Stack trace: #0 [internal function]: ErrorHandler::{closure}(2, 'file_put_conten...', '/Users/luoei/Do...', 562, Array) #1 /Users/luoei/Downloads/php-curl-master-1/src/Curl.php(562): file_put_contents('/Users/luoei/Do...', 'x\x9C\xBD:ks\xDB\xC6\xB5\x9F\xA5\x19\xFD\x87\r...', 2) #2 /Users/luoei/Downloads/php-curl-master-1/src/Curl.php(499): Ares333\Curl\Curl->cache(Array, Array) #3 /Users/luoei/Downloads/php-curl-master-1/src/Curl.php(331): Ares333\Curl\Curl->onProcess(Array, Array) #4 /Users/luoei/Downloads/php-curl-master-1/src/HttpClone.php(124): Ares333\Curl\Curl->start() #5 /Users/luoei/Downloads/php-curl-master-1/demo/www.jujiaobaby.com.php(75): Ares333\Curl\HttpClone->start() #6 {main}

    E_WARNING: ErrorException: file_put_contents(/Users/luoei/Downloads/php-curl-master-1/demo/output/cache/199/21e/86909667f5f14b89bf1291a251): failed to open stream: Too many open files in /Users/luoei/Downloads/php-curl-master-1/src/Curl.php:562 Stack trace: #0 [internal function]: ErrorHandler::{closure}(2, 'file_put_conten...', '/Users/luoei/Do...', 562, Array) #1 /Users/luoei/Downloads/php-curl-master-1/src/Curl.php(562): file_put_contents('/Users/luoei/Do...', 'x\x9C\xBD:ks\xDB\xC6\xB5\x9F\xA5\x19\xFD\x87\r...', 2) #2 /Users/luoei/Downloads/php-curl-master-1/src/Curl.php(499): Ares333\Curl\Curl->cache(Array, Array) #3 /Users/luoei/Downloads/php-curl-master-1/src/Curl.php(331): Ares333\Curl\Curl->onProcess(Array, Array) #4 /Users/luoei/Downloads/php-curl-master-1/src/HttpClone.php(124): Ares333\Curl\Curl->start() #5 /Users/luoei/Downloads/php-curl-master-1/demo/www.jujiaobaby.com.php(75): Ares333\Curl\HttpClone->start() #6 {main} E_WARNING: ErrorException: file_put_contents(/Users/luoei/Downloads/php-curl-master-1/demo/output/clone/http_www.jujiaobaby.com/index.php?s=%2FHome%2FZuopin%2Fworks.html): failed to open stream: Too many open files in /Users/luoei/Downloads/php-curl-master-1/src/HttpClone.php:306 Stack trace: #0 [internal function]: ErrorHandler::{closure}(2, 'file_put_conten...', '/Users/luoei/Do...', 306, Array) #1 /Users/luoei/Downloads/php-curl-master-1/src/HttpClone.php(306): file_put_contents('/Users/luoei/Do...', 'onProcess(Array, Array) #3 [internal function]: HttpCloneDemo->onProcess(Array, Array) #4 /Users/luoei/Downloads/php-curl-master-1/src/Curl.php(492): call_user_func(Array, Array, Array) #5 /Users/luoei/Downloads/php-curl-master-1/src/Curl.php(331): Ares333\Curl\Curl->onProcess(Array, Array) #6 /Users/luoei/Downloads/php-curl-master-1/src/HttpClone.php(124): Ares333\Curl\Curl->start() #7 /Users/luoei/Downloads/php-curl-master-1/demo/www.jujiaobaby.com.php(75): Ares333\Curl\HttpClone->start() #8 {main}

    E_WARNING: ErrorException: file_put_contents(/Users/luoei/Downloads/php-curl-master-1/demo/output/clone/http_www.jujiaobaby.com/index.php?s=%2FHome%2FZuopin%2Fworks.html): failed to open stream: Too many open files in /Users/luoei/Downloads/php-curl-master-1/src/HttpClone.php:306 Stack trace: #0 [internal function]: ErrorHandler::{closure}(2, 'file_put_conten...', '/Users/luoei/Do...', 306, Array) #1 /Users/luoei/Downloads/php-curl-master-1/src/HttpClone.php(306): file_put_contents('/Users/luoei/Do...', 'onProcess(Array, Array) #3 [internal function]: HttpCloneDemo->onProcess(Array, Array) #4 /Users/luoei/Downloads/php-curl-master-1/src/Curl.php(492): call_user_func(Array, Array, Array) #5 /Users/luoei/Downloads/php-curl-master-1/src/Curl.php(331): Ares333\Curl\Curl->onProcess(Array, Array) #6 /Users/luoei/Downloads/php-curl-master-1/src/HttpClone.php(124): Ares333\Curl\Curl->start() #7 /Users/luoei/Downloads/php-curl-master-1/demo/www.jujiaobaby.com.php(75): Ares333\Curl\HttpClone->start() #8 {main} E_WARNING: ErrorException: file_put_contents(/Users/luoei/Downloads/php-curl-master-1/demo/output/cache/760/767/6be4aa0b78703abfb6f67785d8): failed to open stream: Too many open files in /Users/luoei/Downloads/php-curl-master-1/src/Curl.php:562 Stack trace: #0 [internal function]: ErrorHandler::{closure}(2, 'file_put_conten...', '/Users/luoei/Do...', 562, Array) #1 /Users/luoei/Downloads/php-curl-master-1/src/Curl.php(562): file_put_contents('/Users/luoei/Do...', 'x\x9C\xED\}s\xDB\xC6\x99\xFF[\x9A\xD1w\xD8...', 2) #2 /Users/luoei/Downloads/php-curl-master-1/src/Curl.php(499): Ares333\Curl\Curl->cache(Array, Array) #3 /Users/luoei/Downloads/php-curl-master-1/src/Curl.php(331): Ares333\Curl\Curl->onProcess(Array, Array) #4 /Users/luoei/Downloads/php-curl-master-1/src/HttpClone.php(124): Ares333\Curl\Curl->start() #5 /Users/luoei/Downloads/php-curl-master-1/demo/www.jujiaobaby.com.php(75): Ares333\Curl\HttpClone->start() #6 {main}

    E_WARNING: ErrorException: file_put_contents(/Users/luoei/Downloads/php-curl-master-1/demo/output/cache/760/767/6be4aa0b78703abfb6f67785d8): failed to open stream: Too many open files in /Users/luoei/Downloads/php-curl-master-1/src/Curl.php:562 Stack trace: #0 [internal function]: ErrorHandler::{closure}(2, 'file_put_conten...', '/Users/luoei/Do...', 562, Array) #1 /Users/luoei/Downloads/php-curl-master-1/src/Curl.php(562): file_put_contents('/Users/luoei/Do...', 'x\x9C\xED\}s\xDB\xC6\x99\xFF[\x9A\xD1w\xD8...', 2) #2 /Users/luoei/Downloads/php-curl-master-1/src/Curl.php(499): Ares333\Curl\Curl->cache(Array, Array) #3 /Users/luoei/Downloads/php-curl-master-1/src/Curl.php(331): Ares333\Curl\Curl->onProcess(Array, Array) #4 /Users/luoei/Downloads/php-curl-master-1/src/HttpClone.php(124): Ares333\Curl\Curl->start() #5 /Users/luoei/Downloads/php-curl-master-1/demo/www.jujiaobaby.com.php(75): Ares333\Curl\HttpClone->start() #6 {main}

    opened by luoei 1
  • 如何防止出现 403 问题

    如何防止出现 403 问题

    首先感谢 Ares 的 CurlMulti🙏🏻🙏🏻 问题:在采集中遇到了403,于是想在程序中 sleep,但是不知道应该写在哪??还有最大连接数是不是也要设置小一点??

    if ($i < $j) {
        $curl->add(array(
            'url' => "xxx",
            'args' => array (
                'i' => $i,
            ),
        ), 'getSinglePage');
    
        $i ++;
    
        sleep(1);
    
        if ($i == $j) {
            $curl->cbTask = null;
        }
    }
    
    opened by 885783558 1
  • Core.php Line:559

    Core.php Line:559

                if (true == $config ['enable']) {
                    $expire = $config ['expire'];
                } else {
                    $expire = $config ['expire'];
                }
    
    opened by wenpeng 1
Releases(v4.6.1)
Owner
Ares
Ares
The best php curl library.

中文文档 About Implemented by using php-curl internal io event with high performance,high universality,high extensibility which especially suitable for ma

Ares 431 Dec 12, 2022
Custom PHP curl library for the Laravel 5 framework - developed by Ixudra

ixudra/curl Custom PHP cURL library for the Laravel 4 or 5 framework - developed by Ixudra. The package provides an easy interface for sending cURL re

Jan Oris 556 Jan 6, 2023
This library provides an object-oriented wrapper of the PHP cURL extension

PHP Curl Class This library provides an object-oriented wrapper of the PHP cURL extension. If you have questions or problems with installation or usag

PHP MOD 321 Dec 30, 2022
PHP Curl ile letgo api kütüphanesi oluşturuldu. php ile letgo giriş yap.

Kendi LETGO API ile işlemler gerçekleştirelim. // email işlemleri $server = 'imap.gmail.com'; $user = '[email protected]'; $pass = 'password'; $port = 9

Görkem Bayraktar 2 Nov 3, 2022
A Chainable, REST Friendly, PHP HTTP Client. A sane alternative to cURL.

Httpful Httpful is a simple Http Client library for PHP 7.2+. There is an emphasis of readability, simplicity, and flexibility – basically provide the

Nate Good 1.7k Dec 21, 2022
PHP Curl Class makes it easy to send HTTP requests and integrate with web APIs

PHP Curl Class: HTTP requests made easy PHP Curl Class makes it easy to send HTTP requests and integrate with web APIs. Installation Requirements Quic

null 3.1k Jan 5, 2023
PHP cURL for feed Instagram Graph API

php-curl-instagram-graph PHP cURL for feed Instagram Graph API Script made based on the new (2020) Instagram API that requires authorization token gen

null 12 Apr 13, 2022
Simple PHP curl wrapper class

php-curl The smallest possible OOP wrapper for PHP's curl capabilities. Note that this is not meant as a high-level abstraction. You should still know

Andreas Lutro 243 Dec 5, 2022
Simple HTTP cURL client for PHP 7.1+ based on PSR-18

Simple HTTP cURL client for PHP 7.1+ based on PSR-18 Installation composer require sunrise/http-client-curl QuickStart composer require sunrise/http-f

Sunrise // PHP 15 Sep 5, 2022
Plug & Play [CURL + Composer Optional], Proxy as a Service, Multi-tenant, Multi-Threaded, with Cache & Article Spinner

?? .yxorP The SAAS(y), Multitenancy & Augmenting Web Proxy Guzzler is a 100% SAAS(y) plug-and-play (PHP CURL+Composer are Optional) solution that leverages SAAS architecture to provide multi-tenancy, multiple threads, caching, and an article spinner service.

4D/ҵ.com Dashboards 12 Nov 17, 2022
Online tool to convert `curl` requests to Laravel `Http` requests

curl Converter Another bit of automation from Shift to convert curl requests to Laravel Http requests. This project is a WIP. You may follow along wit

Laravel Shift 66 Dec 17, 2022
Requests for PHP is a humble HTTP request library. It simplifies how you interact with other sites and takes away all your worries.

Requests for PHP Requests is a HTTP library written in PHP, for human beings. It is roughly based on the API from the excellent Requests Python librar

null 3.5k Dec 31, 2022
Requests for PHP is a humble HTTP request library. It simplifies how you interact with other sites and takes away all your worries.

Requests for PHP Requests is a HTTP library written in PHP, for human beings. It is roughly based on the API from the excellent Requests Python librar

null 3.5k Dec 31, 2022
Unirest in PHP: Simplified, lightweight HTTP client library.

Unirest for PHP Unirest is a set of lightweight HTTP libraries available in multiple languages, built and maintained by Mashape, who also maintain the

Kong 1.3k Dec 28, 2022
PHP Secure Communications Library

phpseclib - PHP Secure Communications Library Supporting phpseclib Become a backer or sponsor on Patreon One-time donation via PayPal or crypto-curren

null 4.9k Jan 3, 2023
A modern PHP library that allows you to make resilient calls to external services

Resiliency, an implementation for resilient and modern PHP applications Main principles This library is compatible with PHP 7.4+. Installation compose

We love open source softwares 77 Dec 12, 2022
Requests - a HTTP library written in PHP, for human beings

Requests is a HTTP library written in PHP, for human beings. It is roughly based on the API from the excellent Requests Python library. Requests is ISC Licensed (similar to the new BSD license) and has no dependencies, except for PHP 5.6+.

WordPress 3.5k Jan 6, 2023
Declarative HTTP Clients using Guzzle HTTP Library and PHP 8 Attributes

Waffler How to install? $ composer require waffler/waffler This package requires PHP 8 or above. How to test? $ composer phpunit Quick start For our e

Waffler 3 Aug 26, 2022
Supercharge your app or SDK with a testing library specifically for Guzzle

Full Documentation at guzzler.dev Supercharge your app or SDK with a testing library specifically for Guzzle. Guzzler covers the process of setting up

null 275 Oct 30, 2022