PHP Coroutine HTTP client - Swoole Humanization Library

swlib

Last update: Dec 17, 2022

Related tags

Frameworks php http async ajax http-client requests axios swoole psr7-http-client saber coroutine psr7

Overview

Saber

简介

HTTP军刀(呆毛王), Swoole人性化组件库之PHP高性能HTTP客户端, 基于Swoole原生协程, 支持多种风格操作, 底层提供高性能解决方案, 让开发者专注于功能开发, 从传统同步阻塞且配置繁琐的Curl中解放.

English Document

基于Swoole协程Client开发
人性化使用风格, ajax.js/axios.js/requests.py用户福音, 同时支持PSR风格操作
浏览器级别完备的Cookie管理机制, 完美适配爬虫/API代理应用
请求/响应/异常拦截器
多请求并发, 并发重定向优化
连接池, 自动化复用长连接
通道池(Chan): 最大连接数限制+无阻塞
HTTPS连接, CA证书自动化支持
HTTP/Socks5 Proxy支持
WebSocket连接支持
毫秒级超时定时器
自动化编码请求/解析响应数据
响应报文自动编码转换
异步超大文件上传/下载, 断点重传
自动重试机制
单次并发数控制
多模式/超细粒度异常处理机制
(=)浏览器级别缓存机制
(=)随机UA生成器

安装

最好的安装方法是通过 Composer 包管理器 :

composer require swlib/saber

依赖

PHP71 or later
Swoole 2.1.2 or later
Swoole 4 is the best

协程调度

Swoole底层实现协程调度, 业务层无需感知, 开发者可以无感知的用同步的代码编写方式达到异步IO的效果和超高性能，避免了传统异步回调所带来的离散的代码逻辑和陷入多层回调中导致代码无法维护.

需要在onRequet, onReceive, onConnect等事件回调函数中使用, 或是使用go关键字包裹 (swoole.use_shortname默认开启).

go(function () {
    echo SaberGM::get('http://httpbin.org/get');
})

例子

静态方法

数据自动打包: 传入的data会自动转换成content-type所指定的类型格式

默认为x-www-form-urlencoded, 也支持json等其它格式

SaberGM := Saber Global Manager, 如果觉得类名有点长, 可以使用class_alias自己取别名, 推荐服务中使用生成实例的方式使用, 而把SaberGM作为快捷方式.

SaberGM::get('http://httpbin.org/get');
SaberGM::delete('http://httpbin.org/delete');
SaberGM::post('http://httpbin.org/post', ['foo' => 'bar']);
SaberGM::put('http://httpbin.org/put', ['foo' => 'bar']);
SaberGM::patch('http://httpbin.org/patch', ['foo' => 'bar']);

生成实例

适用API代理服务

$saber = Saber::create([
    'base_uri' => 'http://httpbin.org',
    'headers' => [
        'Accept-Language' => 'en,zh-CN;q=0.9,zh;q=0.8',
        'Content-Type' => ContentType::JSON,
        'DNT' => '1',
        'User-Agent' => null
    ]
]);
echo $saber->get('/get');
echo $saber->delete('/delete');
echo $saber->post('/post', ['foo' => 'bar']);
echo $saber->patch('/patch', ['foo' => 'bar']);
echo $saber->put('/put', ['foo' => 'bar']);

生成会话

Session会自动保存cookie信息, 其实现是浏览器级别完备的

$session = Saber::session([
    'base_uri' => 'http://httpbin.org',
    'redirect' => 0
]);
$session->get('/cookies/set?foo=bar&k=v&apple=banana');
$session->get('/cookies/delete?k');
echo $session->get('/cookies')->body;

并发请求

注意: 此处使用了并发重定向优化方案, 多个重定向总是依旧并发的而不会退化为队列的单个请求

success_num} ok, {$responses->error_num} error ]:\n" ."consuming-time: {$responses->time}s\n"; // multi-requests [ 3 ok, 0 error ]: // consuming-time: 0.79090881347656s">

$responses = SaberGM::requests([
    ['uri' => 'http://github.com/'],
    ['uri' => 'http://github.com/'],
    ['uri' => 'https://github.com/']
]);
echo "multi-requests [ {$responses->success_num} ok, {$responses->error_num} error ]:\n" ."consuming-time: {$responses->time}s\n";

// multi-requests [ 3 ok, 0 error ]:
// consuming-time: 0.79090881347656s

// 别名机制可以省略参数书写参数名
$saber = Saber::create(['base_uri' => 'http://httpbin.org']);
echo $saber->requests([
    ['get','/get'],
    ['post','/post'],
    ['patch','/patch'],
    ['put','/put'],
    ['delete','/delete']
]);

数据解析

目前支持json,xml,html,url-query四种格式的数据快速解析

[$json, $xml, $html] = SaberGM::list([
    'uri' => [
        'http://httpbin.org/get',
        'http://www.w3school.com.cn/example/xmle/note.xml',
        'http://httpbin.org/html'
    ]
]);
var_dump($json->getParsedJsonArray());
var_dump($json->getParsedJsonObject());
var_dump($xml->getParsedXmlArray());
var_dump($xml->getParsedXmlObject(true));
var_dump($html->getParsedDomObject()->getElementsByTagName('h1')->item(0)->textContent);

网络代理

支持HTTP和SOCKS5代理

$uri = 'http://myip.ipip.net/';
echo SaberGM::get($uri, ['proxy' => 'http://127.0.0.1:1087'])->body;
echo SaberGM::get($uri, ['proxy' => 'socks5://127.0.0.1:1086'])->body;

文件上传

底层自动协程调度, 可支持异步发送超大文件, 断点续传

同时上传三个文件(三种参数风格string| array |object)

$file1 = __DIR__ . '/black.png';
$file2 = [
    'path' => __DIR__ . '/black.png',
    'name' => 'white.png',
    'type' => ContentType::MAP['png'],
    'offset' => null, //re-upload from break
    'size' => null //upload a part of the file
];
$file3 = new SwUploadFile(
    __DIR__ . '/black.png',
    'white.png',
    ContentType::MAP['png']
);

echo SaberGM::post('http://httpbin.org/post', null, [
        'files' => [
            'image1' => $file1,
            'image2' => $file2,
            'image3' => $file3
        ]
    ]
);

超大文件下载

Download收到数据后会直接异步写入到磁盘, 而不是在内存中对HttpBody进行拼接. 因此download仅使用小量内存, 就可以完成超大文件的下载. 且支持断点续传, 通过设置offset参数来进行断点下载.

异步下载Saber壁纸

$download_dir = '/tmp/saber.jpg';
$response = SaberGM::download(
    'https://ws1.sinaimg.cn/large/006DQdzWly1fsr8jt2botj31hc0wxqfs.jpg',
    $download_dir
);
if ($response->success) {
    exec('open ' . $download_dir);
}

自动重试

在爬虫项目中, 请求失败自动重试是非常常见的需求, 比如会话过期后重新登录.

而Saber内置了此功能, 并可使用拦截器来强化它.

如未设置retry_time而设置了retry拦截器, 则retry_time会置为1, 如retry拦截器的回调方法返回了false, 无论retry_time是多少, 都会在返回false时终止重试.

withBasicAuth('foo', 'bar'); //发现失败后添加验证信息 if ('i don not want to retry again') { return false; // shutdown } } ] ); echo $res;">

$uri = 'http://eu.httpbin.org/basic-auth/foo/bar';
$res = SaberGM::get(
    $uri, [
        'exception_report' => 0,
        'retry_time' => 3,
        'retry' => function (Saber\Request $request) {
            echo "retry...\n";
            $request->withBasicAuth('foo', 'bar'); //发现失败后添加验证信息
            if ('i don not want to retry again') {
                return false; // shutdown
            }
        }
    ]
);
echo $res;

缓存机制

有时候HTTP资源并不会总是变更, 我们可以学习浏览器缓存不会变动的资源, 来加快请求效率, 由Saber自动化地完成且不必自己维护缓存逻辑(CURD或文件读写), 协程的调度使得其不论如何都不会阻塞服务器, Saber没有使用中间件机制因为它和Swoole是强相关的, 但是缓存可以使用 内存/文件/数据库 等多种方式, 所以虽然它尚未实现, 但它将会列入Saber的后续路线图中.

PSR风格

$bufferStream = new BufferStream();
$bufferStream->write(json_encode(['foo' => 'bar']));
$response = SaberGM::psr()
    ->withMethod('POST')
    ->withUri(new Uri('http://httpbin.org/post?foo=bar'))
    ->withQueryParams(['foo' => 'option is higher-level than uri'])
    ->withHeader('content-type', ContentType::JSON)
    ->withBody($bufferStream)
    ->exec()->recv();
echo $response->getBody();

WebSocket

可以通过websocketFrame数据帧的__toString方法直接打印返回数据字符串

push("hello"); co::sleep(1); }">

$websocket = SaberGM::websocket('ws://127.0.0.1:9999');
while (true) {
    echo $websocket->recv(1) . "\n";
    $websocket->push("hello");
    co::sleep(1);
}

极限压力测试

测试机器为最低配MacBookPro, 请求服务器为本地echo服务器

0.9秒完成6666个请求, 成功率100%.

time}s\n"; echo "success: $res->success_num, error: $res->error_num"; }); // on MacOS // use 0.91531705856323s // success: 6666, error: 0">

co::set(['max_coroutine' => 8191]);
go(function () {
    $requests = [];
    for ($i = 6666; $i--;) {
        $requests[] = ['uri' => 'http://127.0.0.1'];
    }
    $res = SaberGM::requests($requests);
    echo "use {$res->time}s\n";
    echo "success: $res->success_num, error: $res->error_num";
});
// on MacOS
// use 0.91531705856323s
// success: 6666, error: 0

列式请求集

在实际项目中, 经常会存在使用URL列表来配置请求的情况, 因此提供了list方法来方便使用:

echo SaberGM::list([
    'uri' => [
        'https://www.qq.com/',
        'https://www.baidu.com/',
        'https://www.swoole.com/',
        'http://httpbin.org/'
    ]
]);

单次并发控制

在实际爬虫项目中, 我们往往要限制单次并发请求数量以防被服务器防火墙屏蔽, 而一个max_co参数就可以轻松地解决这个问题, max_co会将请求根据上限量分批将请求压入队列并执行收包.

1])->time."\n";">

// max_co is the max number of concurrency request once, it's very useful to prevent server-waf limit.
$requests = array_fill(0, 10, ['uri' => 'https://www.qq.com/']);
echo SaberGM::requests($requests, ['max_co' => 5])->time."\n";
echo SaberGM::requests($requests, ['max_co' => 1])->time."\n";

高性能无极限协程连接池

在常驻内存的服务器中使用时, 一定要手动开启连接池选项:

$swoole = Saber::create([
    'base_uri' => 'https://www.swoole.com/',
    'use_pool' => true
]);

在通过该实例使用时, 就会启用连接池特性, 即底层与www.swoole.com网站的连接客户端将会用一个全局连接池存取, 避免了每次使用创建/连接的开销.

无限连接池

在参数为true时, 该网站的连接池容量是无限的, 一般情况下没有问题, 且无限容量的连接池性能更好.

定容连接池

但如果你使用其作为爬虫代理服务, 遭遇大量请求时, 连接池中的客户端数量就会不可控制地快速上升, 甚至超出你所请求的源网站的最大允许连接数, 这时候你就需要将use_pool设置为一个理想数值(int), 此时, 底层会使用Channel作为连接池, 在连接池创建的客户端超出数量且不够取用时, 挂起需要取用客户端的协程, 并等待正在使用客户端的协程归还客户端, 协程等待和切换几乎没有多大的性能消耗, 是一种非常先进的解决方式.

动态变容

需要注意的是, 连接池是绑定服务器IP+端口的, 即如果你有多个实例面向的是同一个服务器IP+端口, 他们之间使用的连接池也是同一个.

所以你在重复创建服务器IP+端口的实例时, 新创建的实例指定的use_pool是允许覆盖之前数值的, 即连接池底层是自动变容的, 容量增加时底层会重新创建新的连接池并转移客户端, 容量减少时也会销毁在连接池内的多余的客户端.

注意事项

注册你所希望的配置

除了一定要记得配备连接池以外, 异常处理的方式也需要注意是符合你的编程习惯的, Saber默认的异常处理是最主流且严谨的抛出异常, 但Saber也支持静默地使用错误码和状态位, 可能更符合很多人的口味.

SaberGM::exceptionReport(0); // 关闭抛出异常报告, 在业务代码之前注册即可全局生效
$saber->exceptionReport(0);  //也可以单独设置某个实例

同理, 你所希望的配置都可以在业务代码之前如onWorkerStart甚至是swoole_server启动之前预先配置.

SaberGM::default([
    'exception_report' => 0
    'use_pool' => true
]);

像这样配置你所期望的选项可以让你获得更好的使用体验!

注意在一次性脚本中释放连接池

go(function(){
    // your code with pool...
    saber_pool_release(); // and this script will exit
});

如果你在一次性脚本中使用的连接池, 由于协程客户端是存在池中的, 引用计数为1无法释放, 就会导致swoole一直处于事件循环中, 脚本就无法退出, 你需要手动调用saber_pool_release或saber_exit或swoole_event_exit来正常退出, 也可以使用exit强制退出当前脚本(不要在server中使用exit).

配置参数表

|符号分割多种可选值

key	type	introduction	example	remark
protocol_version	string	HTTP协议版本	1.1	HTTP2还在规划中
base_uri	string	基础路径	`http://httpbin.org`	将会与uri按照rfc3986合并
uri	string	资源标识符	`http://httpbin.org/get` \| `/get` \| `get`	可以使用绝对路径和相对路径
uri_query	string\|array	请求信息	`['foo' => 'bar']`	非字符串会自动转换
method	string	请求方法	`get` \| `post` \| `head` \| `patch` \| `put` \| `delete`	底层自动转换为大写
headers	array	请求报头	`['DNT' => '1']` \| `['accept' => ['text/html'], ['application/xml']]`	字段名不区分大小写, 但会保留设定时的原始大小写规则, 底层每个字段值会根据PSR-7自动分割为数组
cookies	`array`\|`string`		`['foo '=> 'bar']` \| `'foo=bar; foz=baz'`	底层自动转化为Cookies对象, 并设置其domain为当前的uri, 具有浏览器级别的完备属性.
useragent	string	用户代理	`curl-1.0`	默认为macos平台的chrome
referer	string	来源地址	`https://www.google.com`	默认为空
redirect	int	最大重定向次数	5	默认为3, 为0时不重定向.
keep_alive	bool	是否保持连接	`true` \| `false`	默认为true, 重定向时会自动复用连接
content_type	string	发送的内容编码类型	`text/plain` \| `Swlib\Http\ContentType::JSON`	默认为application/x-www-form-urlencoded
data	`array` \| `string`	发送的数据	`'foo=bar&dog=cat'` \| `['foo' => 'bar']`	会根据content_type自动编码数据
before	`callable` \| `array`	请求前拦截器	`function(Request $request){}`	具体参考拦截器一节
after	`callable` \| `array`	响应后拦截器	`function(Response $response){}`	具体参考拦截器一节
before_redirect	`callable` \| `array`	重定向后拦截器	`function(Request $request, Response $response){}`	具体参考拦截器一节
timeout	float	超时时间	0.5	默认5s, 支持毫秒级超时
bind_address	string	绑定地址	192.168.1.1 或 eth0	默认不设置
bind_port	int	绑定端口	80	默认不设置
proxy	string	代理	`http://127.0.0.1:1087` \| `socks5://127.0.0.1:1087`	支持http和socks5
ssl	int	是否开启ssl连接	`0=关闭` `1=开启` `2=自动`	默认自动
cafile	string	ca文件	`__DIR__ . '/cacert.pem'`	默认自带
ssl_verify_peer	bool	验证服务器端证书	`false` \| `true`	默认关闭
ssl_allow_self_signed	bool	允许自签名证书	`true` \| `false`	默认允许
ssl_cert_file	string	cert 证书	`__DIR__ . '/ssl.cert'`	默认不设置
ssl_key_file	string	key 私钥	`__DIR__ . '/ssl.key'`	默认不设置
iconv	array	指定编码转换	`['gbk', 'utf-8']`	共三个参数为`from,to,use_mb`, 默认自动识别
exception_report	int	异常报告级别	HttpExceptionMask::E_ALL	默认汇报所有异常
exception_handle	callable\|array	异常自定义处理函数	`function(Exception $e){}`	函数返回true时可忽略错误
retry	callable	自动重试拦截器	`function(Request $request, Response $response){}`	位于发生错误后及重试之前
retry_time	int	自动重试次数		默认不重试
use_pool	bool\|int	连接池	`true`	`false`
pool_key	callable\|array	连接池的key	`function(Request $request):string { return $key; }`	默认为请求地址的`host:port`

配置参数别名

为了使用方便与容错, 配置项的键值具有别名机制, 建议尽量使用本名:

key	alias
method	0
uri	`1` \| `url`
data	`2` \| `body`
base_uri	base_url
after	callback
content_type	`content-type` \| `contentType`
cookies	cookie
headers	header
redirect	follow
useragent	`ua` \| `user-agent`
exception_report	`error_report` \| `report`
before_retry	retry
referer	`ref` \| `referrer`

拦截器

拦截器是Saber的一个非常强大的特性, 它可以让你非常方便地处理各种事情, 比如打印dev日志:

function (Saber\Response $response) { if ($response->success) { echo "log: success!\n"; } else { echo "log: failed\n"; } echo "use {$response->time}s"; } ]); // log: request http://twosee.cn/ now... // log: success! // use 0.52036285400391s">

SaberGM::get('http://twosee.cn/', [
    'before' => function (Saber\Request $request) {
        $uri = $request->getUri();
        echo "log: request $uri now...\n";
    },
    'after' => function (Saber\Response $response) {
        if ($response->success) {
            echo "log: success!\n";
        } else {
            echo "log: failed\n";
        }
        echo "use {$response->time}s";
    }
]);
// log: request http://twosee.cn/ now...
// log: success!
// use 0.52036285400391s

甚至连异常自定义处理函数,会话都是通过拦截器来实现的.

拦截器可以有多个, 会依照注册顺序执行, 并且你可以为拦截器命名, 只需要使用数组包裹并指定key值, 如果你要删除这个拦截器, 给它覆盖一个null值即可.

[
    'after' => [
        'interceptor_new' => function(){},
        'interceptor_old' => null
    ]
]

拦截器可以使用四种方式注册(4种PHP回调函数):

callable: function(){}
string: 'function_name'
string: 'ClassName::method_name'
array: [$object, 'method_name']

Cookies

Cookie的实现是浏览器级别完备的, 它具体参考了Chrome浏览器的实现, 并遵循其相关规则.

属性

Cookies是一堆Cookie的集合, 而每个Cookie具有以下属性:

name, value, expires, path, session, secure, httponly, hostonly

任意格式互转

并且Cookies类支持多种格式互转, 如

foo=bar; foz=baz; apple=banana
Set-Cookie: logged_in=no; domain=.github.com; path=/; expires=Tue, 06 Apr 2038 00:00:00 -0000; secure; HttpOnly
['foo'=>'bar', 'foz'=>'baz']

等格式转到Cookie类, 或是Cookie类到该几种格式的序列化.

域名路径和过期时限校验

Cookie也支持域名和时限校验, 不会丢失任何信息, 如domain是github.comcookie, 不会出现在help.github.com, 除非domain不是hostonly的(.github.com通配).

如果是session-cookie(没有过期时间,浏览器关闭则过期的), expires属性会设置为当前时间, 你可以通过拦截器来对其设置具体的时间.

持久化存储

通过读取Cookies的raw属性, 可以轻松地将其持久化到数据库中, 非常适合登录类爬虫应用.

更多详情具体请参考Swlib/Http库文档和例子.

异常机制

Saber遵循将业务与错误分离的守则, 当请求任意环节失败时, 默认都将会抛出异常.

强大的是, Saber的异常处理也是多样化的, 且和PHP的原生的异常处理一样完善.

异常的命名空间位于Swlib\Http\Exception

Exception	Intro	scene
RequestException	请求失败	请求配置错误
ConnectException	连接失败	如无网络连接, DNS查询失败, 超时等, errno的值等于Linux errno。可使用swoole_strerror将错误码转为错误信息。
TooManyRedirectsException	重定向次数超限	重定向的次数超过了设定的限制, 抛出的异常将会打印重定向追踪信息
ClientException	客户端异常	服务器返回了4xx错误码
ServerException	服务器异常	服务器返回了5xx错误码
BadResponseException	未知的获取响应失败	服务器无响应或返回了无法识别的错误码

除一般异常方法外, 所有HTTP异常类还拥有以下方法 :

Method	Intro
getRequest	获取请求实例
hasResponse	是否获得响应
getResponse	获取响应实例
getResponseBodySummary	获取响应主体的摘要内容

捕获例子

try {
    echo SaberGM::get('http://httpbin.org/redirect/10');
} catch (TooManyRedirectsException $e) {
    var_dump($e->getCode());
    var_dump($e->getMessage());
    var_dump($e->hasResponse());
    echo $e->getRedirectsTrace();
}
// int(302)
// string(28) "Too many redirects occurred!"
// bool(true)
#0 http://httpbin.org/redirect/10
#1 http://httpbin.org/relative-redirect/9
#2 http://httpbin.org/relative-redirect/8

异常报告级别控制

同时, Saber亦支持以温和的方式来对待异常, 以免使用者陷入在不稳定的网络环境下, 必须在每一步都使用try包裹代码的恐慌中:

设定errorReport级别, 它是全局生效的, 对已创建的实例不会生效.

// 启用所有异常但忽略重定向次数过多异常
SaberGM::exceptionReport(
    HttpExceptionMask::E_ALL ^ HttpExceptionMask::E_REDIRECT
);

掩码表

下面的值（数值或者符号）用于建立一个二进制位掩码，来制定要报告的错误信息。可以使用按位运算符来组合这些值或者屏蔽某些类型的错误。标志位与掩码

Mask	Value	Intro
E_NONE	0	忽略所有异常
E_REQUEST	1	对应RequestException
E_CONNECT	2	对应RequestException
E_REDIRECT	4	对应RequestException
E_BAD_RESPONSE	8	对应BadRException
E_CLIENT	16	对应ClientException
E_SERVER	32	对应ServerException
E_ALL	63	所有异常

异常自定义处理函数

本函数可以用你自己定义的方式来处理HTTP请求中产生的错误, 可以更加随心所欲地定义你想要捕获/忽略的异常.

注意: 除非函数返回 TRUE (或其它真值)，否则异常会继续抛出而不是被自定义函数捕获.

SaberGM::exceptionHandle(function (\Exception $e) {
    echo get_class($e) . " is caught!";
    return true;
});
SaberGM::get('http://httpbin.org/redirect/10');
//output: Swlib\Http\Exception\TooManyRedirectsException is caught!

Road Map

File Upload ✔	WebSocket ✔	AutoParser ✔	AutoRetry ✔	BigFile Download ✔	Cache	ClientPool	Random UA
4 (High-priority)	3	2	1	.5	.5	.5	.175

Why not Http2 ?

As the main HTTP/2 benefit is that it allows multiplexing many requests within a single connection, thus [almost] removing the limit on number of simultaneous requests - and there is no such limit when talking to your own backends. Moreover, things may even become worse when using HTTP/2 to backends, due to single TCP connection being used instead of multiple ones, so Http2 Will not be a priority. (#ref)

IDE Helper

将本项目源文件加入到IDE的 Include Path 中.

(使用composer安装,则可以包含整个vendor文件夹, PHPStorm会自动包含)

良好的注释书写使得Saber完美支持IDE自动提示, 只要在对象后书写箭头符号即可查看所有对象方法名称, 名称都十分通俗易懂, 大量方法都遵循PSR规范或是参考Guzzle项目(感谢)而实现.

对于底层Swoole相关类的IDE提示则需要引入eaglewu的swoole-ide-helper(composer在dev环境下会默认安装), 但是该项目为手动维护, 不太完整, 也可以使用swoft-ide-helper或:

Swoole官方的ide-helper.

重中之重

欢迎提交issue和PR.

附录

Saber API

由于无法在魔术方法中使用协程(__call, __callStatic), 源码中的方法都是手动定义.

为了使用方便，已为所有支持的请求方法提供了别名。

Swlib\SaberGM

public static function psr(array $options = []): Swlib\Saber\Request
public static function wait(): Swlib\Saber
public static function request(array $options = [])
public static function get(string $uri, array $options = [])
public static function delete(string $uri, array $options = [])
public static function head(string $uri, array $options = [])
public static function options(string $uri, array $options = [])
public static function post(string $uri, $data = null, array $options = [])
public static function put(string $uri, $data = null, array $options = [])
public static function patch(string $uri, $data = null, array $options = [])
public static function download(string $uri, string $dir, int $offset, array $options = [])
public static function requests(array $requests, array $default_options = []): Swlib\Saber\ResponseMap
public static function list(array $options, array $default_options = []): Swlib\Saber\ResponseMap
public static function websocket(string $uri)
public static function default(?array $options = null): array
public static function exceptionReport(?int $level = null): int
public static function exceptionHandle(callable $handle): void

Swlib\Saber

public static function create(array $options = []): self
public static function session(array $options = []): self
public static function websocket(string $uri): WebSocket
public function request(array $options)
public function get(string $uri, array $options = [])
public function delete(string $uri, array $options = [])
public function head(string $uri, array $options = [])
public function options(string $uri, array $options = [])
public function post(string $uri, $data = null, array $options = [])
public function put(string $uri, $data = null, array $options = [])
public function patch(string $uri, $data = null, array $options = [])
public function download(string $uri, string $dir, int $offset, array $options = [])
public function requests(array $requests, array $default_options = []): ResponseMap
public function list(array $options, array $default_options = []): ResponseMap
public function upgrade(?string $path = null): WebSocket
public function psr(array $options = []): Request
public function wait(): self
public function exceptionReport(?int $level = null): int
public function exceptionHandle(callable $handle): void
public static function getAliasMap(): array
public function setOptions(array $options = [], ?Swlib\Saber\Request $request = null): self
public static function getDefaultOptions(): array
public static function setDefaultOptions(array $options = [])

Swlib\Saber\Request

public function getExceptionReport(): int
public function setExceptionReport(int $level): self
public function isWaiting(): bool
public function getPool()
public function withPool($bool_or_max_size): self
public function tryToRevertClientToPool(bool $connect_failed = false)
public function getSSL(): int
public function withSSL(int $mode = 2): self
public function getCAFile(): string
public function withCAFile(string $ca_file = __DIR__ . '/cacert.pem'): self
public function getSSLCertFile(): string
public function withSSLCertFile(string $cert_file): self
public function getSSLKeyFile(): string
public function withSSLKeyFile(string $key_file): self
public function withSSLVerifyPeer(bool $verify_peer = false, ?string $ssl_host_name = ''): self
public function withSSLAllowSelfSigned(bool $allow = true): self
public function getSSLConf()
public function getKeepAlive()
public function withKeepAlive(bool $enable): self
public function withBasicAuth(?string $username = null, ?string $password = null): self
public function withXHR(bool $enable = true)
public function getProxy(): array
public function withProxy(string $host, int $port): self
public function withSocks5(string $host, int $port, ?string $username, ?string $password): self
public function withoutProxy(): self
public function getBindAddress(): ?string
public function withBindAddress(string $address): self
public function getBindPort(): ?int
public function withBindPort(int $port): self
public function getTimeout(): float
public function withTimeout(float $timeout): self
public function getRedirect(): int
public function getName()
public function withName($name): self
public function withRedirect(int $time): self
public function isInQueue(): bool
public function withInQueue(bool $enable): self
public function getRetryTime(): int
public function withRetryTime(int $time): self
public function withAutoIconv(bool $enable): self
public function withExpectCharset(string $source = 'auto', string $target = 'utf-8', bool $use_mb = false): self
public function withDownloadDir(string $dir): self
public function withDownloadOffset(int $offset): self
public function resetClient($client)
public function exec()
public function recv()
public function getRequestTarget(): string
public function withRequestTarget($requestTarget): self
public function getMethod(): string
public function withMethod($method): self
public function getUri(): Psr\Http\Message\UriInterface
public function withUri(?Psr\Http\Message\UriInterface $uri, $preserveHost = false): self
public function getCookieParams(): array
public function getCookieParam(string $name): string
public function withCookieParam(string $name, ?string $value): self
public function withCookieParams(array $cookies): self
public function getQueryParam(string $name): string
public function getQueryParams(): array
public function withQueryParam(string $name, ?string $value): self
public function withQueryParams(array $query): self
public function getParsedBody(?string $name = null)
public function withParsedBody($data): self
public function getUploadedFile(string $name): Psr\Http\Message\UploadedFileInterface
public function getUploadedFiles(): array
public function withUploadedFile(string $name, ?Psr\Http\Message\UploadedFileInterface $uploadedFile): self
public function withoutUploadedFile(string $name): self
public function withUploadedFiles(array $uploadedFiles): self
public function __toString()
public function getProtocolVersion(): string
public function withProtocolVersion($version): self
public function hasHeader($name): bool
public function getHeader($name): array
public function getHeaderLine($name): string
public function getHeaders(bool $implode = false, bool $ucwords = false): array
public function getHeadersString(bool $ucwords = true): string
public function withHeader($raw_name, $value): self
public function withHeaders(array $headers): self
public function withAddedHeaders(array $headers): self
public function withAddedHeader($raw_name, $value): self
public function withoutHeader($name): self
public function getBody(): Psr\Http\Message\StreamInterface
public function withBody(?Psr\Http\Message\StreamInterface $body): self
public function getCookies()
public function setCookie(array $options): self
public function unsetCookie(string $name, string $path = '', string $domain = ''): self
public function withInterceptor(string $name, array $interceptor)
public function withAddedInterceptor(string $name, array $functions): self
public function withoutInterceptor(string $name): self
public function callInterceptor(string $name, $arguments)
public function getSpecialMark(string $name = 'default')
public function withSpecialMark($mark, string $name = 'default'): self

Swlib\Saber\Response

public function isSuccess(): bool
public function getUri(): Psr\Http\Message\UriInterface
public function getTime(): float
public function getRedirectHeaders(): array
public function getStatusCode()
public function withStatus($code, $reasonPhrase = '')
public function getReasonPhrase()
public function __toString()
public function getProtocolVersion(): string
public function withProtocolVersion($version): self
public function hasHeader($name): bool
public function getHeader($name): array
public function getHeaderLine($name): string
public function getHeaders(bool $implode = false, bool $ucwords = false): array
public function getHeadersString(bool $ucwords = true): string
public function withHeader($raw_name, $value): self
public function withHeaders(array $headers): self
public function withAddedHeaders(array $headers): self
public function withAddedHeader($raw_name, $value): self
public function withoutHeader($name): self
public function getBody(): Psr\Http\Message\StreamInterface
public function withBody(?Psr\Http\Message\StreamInterface $body): self
public function getCookies()
public function setCookie(array $options): self
public function unsetCookie(string $name, string $path = '', string $domain = ''): self
public function getSpecialMark(string $name = 'default')
public function withSpecialMark($mark, string $name = 'default'): self
public function getParsedJsonArray(bool $reParse = false): array
public function getParsedJsonObject(bool $reParse = false): object
public function getParsedQueryArray(bool $reParse = false): array
public function getParsedXmlArray(bool $reParse = false): array
public function getParsedXmlObject(bool $reParse = false): SimpleXMLElement
public function getParsedDomObject(bool $reParse = false): DOMDocument
public function getDataRegexMatch(string $regex, $group = null, int $fill_size)
public function getDataRegexMatches(string $regex, int $flag): array
public function isExistInData(string $needle, int $offset)

Swlib\Saber\RequestQueue

public function enqueue($request)
public function getMaxConcurrency(): int
public function withMaxConcurrency(int $num = -1): self
public function recv(): Swlib\Saber\ResponseMap
public function withInterceptor(string $name, array $interceptor)
public function withAddedInterceptor(string $name, array $functions): self
public function withoutInterceptor(string $name): self
public function callInterceptor(string $name, $arguments)

Swlib\Saber\ResponseMap

public $time = 0.0;
public $status_map = [];
public $success_map = [];
public $success_num = 0;
public $error_num = 0;
public function offsetSet($index, $response)
public function __toString()

Swlib\Saber\WebSocket

public function withMock(bool $ssl): self
public function recv(float $timeout = -1)
public function push(string $data, int $opcode = 1, bool $finish = true): bool
public function close(): bool

Swlib\Saber\WebSocketFrame

public $finish = true;
public $opcode = null;
public $data = null;
public function getOpcodeDefinition()
public function getOpcode()
public function getData()
public function __toString()

Comments

echo $res->body 是乱码，什么情况？

Swlib\Http\SwooleBuffer 转字符串，就是乱码

如果响应的字节比较少，就不是乱码，比如get('http://myip.ipip.net/') 如果响应的字节多，就是乱码，比如get('http://www.myip.cn/')

不知道是不是 swoole 的问题，文档里也有人反映 https://wiki.swoole.com/wiki/page/582.html

环境 CentOS 7.3 + docker(alpine 3.8) php 7.1.23，装了zlib swoole 4.3.1 swlib/saber v1.0.6

731724ac8bad:~# php --ri swoole

swoole

Swoole => enabled Author => Swoole Team [email protected] Version => 4.3.1 Built => Mar 20 2019 12:52:44 coroutine => enabled epoll => enabled eventfd => enabled signalfd => enabled spinlock => enabled rwlock => enabled sockets => enabled openssl => LibreSSL 2.7.4 http2 => enabled mutex_timedlock => enabled pthread_barrier => enabled mysqlnd => enabled async_redis => enabled coroutine_postgresql => enabled

Directive => Local Value => Master Value swoole.enable_coroutine => On => On swoole.display_errors => On => On swoole.use_shortname => On => On swoole.unixsock_buffer_size => 8388608 => 8388608 731724ac8bad:~#
question

opened by fanybook 13

能否考虑支持下 IPv6 HTTP 请求

我这里测试似乎是不支持 IPv6 HTTP 请求的，直接报错，测试的域名是 ipv6.hu60.cn

root@Chuwen:/opt/www/wwwroot# php run/1.php
PHP Fatal error:  Uncaught Swlib\Http\Exception\ConnectException: HTTP -1 Unknown: Connect timeout! the server is not listening on the port or the network is missing! in /opt/www/wwwroot/vendor/swlib/saber/src/Request.php:704
Stack trace:
#0 /opt/www/wwwroot/vendor/swlib/saber/src/RequestQueue.php(119): Swlib\Saber\Request->recv()
#1 /opt/www/wwwroot/vendor/swlib/saber/src/Saber.php(279): Swlib\Saber\RequestQueue->recv()
#2 /opt/www/wwwroot/vendor/swlib/saber/src/SaberGM.php(90): Swlib\Saber->requests()
#3 /opt/www/wwwroot/run/1.php(18): Swlib\SaberGM::requests()
#4 {main}
  thrown in /opt/www/wwwroot/vendor/swlib/saber/src/Request.php on line 704

Fatal error: Uncaught Swlib\Http\Exception\ConnectException: HTTP -1 Unknown: Connect timeout! the server is not listening on the port or the network is missing! in /opt/www/wwwroot/vendor/swlib/saber/src/Request.php:704
Stack trace:
#0 /opt/www/wwwroot/vendor/swlib/saber/src/RequestQueue.php(119): Swlib\Saber\Request->recv()
#1 /opt/www/wwwroot/vendor/swlib/saber/src/Saber.php(279): Swlib\Saber\RequestQueue->recv()
#2 /opt/www/wwwroot/vendor/swlib/saber/src/SaberGM.php(90): Swlib\Saber->requests()
#3 /opt/www/wwwroot/run/1.php(18): Swlib\SaberGM::requests()
#4 {main}
  thrown in /opt/www/wwwroot/vendor/swlib/saber/src/Request.php on line 704

测试代码：

<?php
include_once dirname(__DIR__) . "/vendor/autoload.php";

use Swlib\SaberGM;

co::set(['max_coroutine' => 8191]);

go(function () {
    $requests = [];
    for ($i = 2; $i--;) {
        $requests[] = ['uri' => 'https://ipv6.hu60.cn'];
    }

    $res = SaberGM::requests($requests);

    foreach ($res->getArrayCopy() as $v){
        $time = sprintf("%.2f", $v->time*1000);
        echo "状态码：{$v->statusCode}，耗时：{$time}\n";
    }

    echo "use {$res->time}s\n";
    echo "success: $res->success_num, error: $res->error_num";
});

opened by PrintNow 12

$Uncaught Swlib\Http\Exception\ConnectException:Connect timeout! the server is not listening on the port or the network is missing!$

Uncaught Swlib\Http\Exception\ConnectException:Connect timeout! the server is not listening on the port or the network is missing!

代码是这样的: SaberGM::psr([ 'uri' => 'https://httpbin.org/ip', 'ssl_verify_peer' => true, 'ssl_host_name' => 'httpbin.org', 'proxy'=>'http://192.162.133.40:47163', ])->withCAFile()->getProxy();

如果加了代理会报上面的错误，去掉代理正常访问。用guzzleHttp试过代理ip正常可用

opened by JasonBike 8
开启use_pool为true时，第二次请求当body为空时，body未重置，导致请求把上次请求的body带到下次请求的body

操作步骤： 1、设置use_pool为true 2、第一次发送POST请求，并带上body 3、第二次发送GET请求，没有body

结果：第二次GET请求的Body为第一次POST请求的body，实际上第二次GET请求并为带上任何Body参数

定位错误文件路径：swlib/saber/src/Request.php:627 /** 设置请求主体 */ $body = (string) ($this->getBody() ?? ''); if ($body !== '') { $this->client->setData($body); }
bug

opened by tomorrowsky 7

请求一直乱码

第一次使用，不知道什么原因，一直乱码。

   //乱码
    $uri = "https://wenda.swoole.com";
    $content = (string)SaberGM::get($uri)->getBody();
    echo $content;

    //正常
    $cli = new \Swoole\Coroutine\Http\Client('wenda.swoole.com', 443,true);
    $cli->get('/');
    echo $cli->body;
    $cli->close();

question

opened by scfff 7

$【Fix】\Swlib\Saber\Request::SSL_AUTO 判断HTTPS的方法不正确$
【Fix】\Swlib\Saber\Request::SSL_AUTO 判断HTTPS的方法不正确
https://github.com/swlib/saber/blob/1188d0a67d18430d5c1a11f8dcdc135852fc1e31/src/Request.php#L126

不应该用端口来判断是否是https还是http，

一来，443端口也可以运行http程序，https也可以运行在非443端口，AUTO应该且必须根据scheme来决定，

二来，有些web服务器在443端口同时提供http和https ，但是要求某些api必须通过特定的scheme调用，这会会导致saber的一些判断出现问题。

curl --request POST --url 'http://eco.taobao.com/router/rest?app_key=123456&method=taobao.top.secret.get&v=2.0' curl --request POST --url 'https://eco.taobao.com/router/rest?app_key=123456&method=taobao.top.secret.get&v=2.0'
opened by ihipop 6

$Swoole\Buffer::__construct(): handle 10060447 exceed 10000000 in /vendor/swlib/http/src/SwooleBuffer.php on line 19$

Swoole\Buffer::__construct(): handle 10060447 exceed 10000000 in /vendor/swlib/http/src/SwooleBuffer.php on line 19

请求了次数多了以后就出现这样的问题，已经开启连接池

文档上没说明到底多大的限制 https://wiki.swoole.com/wiki/page/246.html

hp --ri swoole

swoole

Swoole => enabled
Author => Swoole Team <[email protected]>
Version => 4.3.1
Built => Apr 12 2019 18:22:24
coroutine => enabled
epoll => enabled
eventfd => enabled
signalfd => enabled
cpu_affinity => enabled
spinlock => enabled
rwlock => enabled
sockets => enabled
openssl => OpenSSL 1.1.0h  27 Mar 2018
http2 => enabled
pcre => enabled
zlib => enabled
mutex_timedlock => enabled
pthread_barrier => enabled
futex => enabled
mysqlnd => enabled
async_redis => enabled
coroutine_postgresql => enabled

Directive => Local Value => Master Value
swoole.enable_coroutine => On => On
swoole.display_errors => On => On
swoole.use_shortname => On => On
swoole.unixsock_buffer_size => 8388608 => 8388608

fixed

opened by ihipop 6

按照 redeme 中的连接 WebSocket 报错

(base) MBP:blive chen$ php artisan test
[2019-07-03 19:49:48 @20747.0]	ERROR	(PHP Fatal Error: 10001):
Swoole\Coroutine\Http\Client::upgrade: API must be called in the coroutine
Stack trace:
#0  Swoole\Coroutine\Http\Client->upgrade() called at [/Users/chen/wwwroot/blive/vendor/swlib/saber/src/WebSocket.php:40]
#1  Swlib\Saber\WebSocket->__construct() called at [/Users/chen/wwwroot/blive/vendor/swlib/saber/src/Saber.php:308]
#2  Swlib\Saber->upgrade() called at [/Users/chen/wwwroot/blive/vendor/swlib/saber/src/Saber.php:134]
#3  Swlib\Saber::websocket() called at [/Users/chen/wwwroot/blive/vendor/swlib/saber/src/SaberGM.php:101]
#4  Swlib\SaberGM::websocket() called at [/Users/chen/wwwroot/blive/app/Console/Commands/test.php:43]
#5  App\Console\Commands\test->handle()
#6  call_user_func_array() called at [/Users/chen/wwwroot/blive/vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php:32]
#7  Illuminate\Container\BoundMethod::Illuminate\Container\{closure}() called at [/Users/chen/wwwroot/blive/vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php:90]
#8  Illuminate\Container\BoundMethod::callBoundMethod() called at [/Users/chen/wwwroot/blive/vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php:34]
#9  Illuminate\Container\BoundMethod::call() called at [/Users/chen/wwwroot/blive/vendor/laravel/framework/src/Illuminate/Container/Container.php:576]
#10 Illuminate\Container\Container->call() called at [/Users/chen/wwwroot/blive/vendor/laravel/framework/src/Illuminate/Console/Command.php:183]
#11 Illuminate\Console\Command->execute() called at [/Users/chen/wwwroot/blive/vendor/symfony/console/Command/Command.php:255]
#12 Symfony\Component\Console\Command\Command->run() called at [/Users/chen/wwwroot/blive/vendor/laravel/framework/src/Illuminate/Console/Command.php:170]
#13 Illuminate\Console\Command->run() called at [/Users/chen/wwwroot/blive/vendor/symfony/console/Application.php:921]
#14 Symfony\Component\Console\Application->doRunCommand() called at [/Users/chen/wwwroot/blive/vendor/symfony/console/Application.php:273]
#15 Symfony\Component\Console\Application->doRun() called at [/Users/chen/wwwroot/blive/vendor/symfony/console/Application.php:149]
#16 Symfony\Component\Console\Application->run() called at [/Users/chen/wwwroot/blive/vendor/laravel/framework/src/Illuminate/Console/Application.php:90]
#17 Illuminate\Console\Application->run() called at [/Users/chen/wwwroot/blive/vendor/laravel/framework/src/Illuminate/Foundation/Console/Kernel.php:133]
#18 Illuminate\Foundation\Console\Kernel->handle() called at [/Users/chen/wwwroot/blive/artisan:37]
(base) MBP:blive chen$

opened by StringKe 5

BUG: add ssl_host_name setting for https urls and parsing HTTP AUTH credentials from URI on Request instance creating.

During production use i've found an issue with connection to https://https://uds.hasene.online. Request failed with warning:

WARNING swSSL_connect: SSL_connect(fd=9) failed. Error: (null)[1|0].

After investigation fix https://github.com/swoole/swoole-src/commit/77876aad4b423fd2538152d86ce7f63fd075fe71 i've added 'ssl_host_name' with host name for SSL request.

Also added parsing HTTP AUTH credentials from URI on Request instance creating.

PHP version: 7.3.3

Swoole:

Swoole => enabled Author => Swoole Team [email protected] Version => 4.3.1 Built => Mar 27 2019 15:38:16 coroutine => enabled kqueue => enabled rwlock => enabled openssl => OpenSSL 1.0.2r 26 Feb 2019 http2 => enabled pcre => enabled zlib => enabled brotli => enabled async_redis => enabled
in progress

opened by alebedev80 5

大神代码出现 Segmentation fault 不知如何定位

go(function () use ($host, $port, $poolSize, $queueKey) {
    $pool = new DotCloud\RedisPool($host, $port, $poolSize);
    while (true) {
        $redis = $pool->get();
        if ($redis->lLen($queueKey) > 0) {
            $data = $redis->rPop($queueKey);
            if (empty($data)) {
                $pool->put($redis);
                continue;
            }
            // ......
            // redis 里取出数据[ ['uri'=> ''], ['uri'=> ''] ]，使用requests并发请求
            $responses = Swlib\SaberGM::requests($urls);
            // ......
        }
        $pool->put($redis);
    }
});

# php --ri swoole

swoole

swoole support => enabled
Version => 4.2.9
Author => Swoole Group[email: [email protected]]
coroutine => enabled
debug => enabled
trace_log => enabled
epoll => enabled
eventfd => enabled
signalfd => enabled
cpu_affinity => enabled
spinlock => enabled
rwlock => enabled
sockets => enabled
openssl => OpenSSL 1.0.2k-fips  26 Jan 2017
http2 => 1.31.1
pcre => enabled
zlib => enabled
mutex_timedlock => enabled
pthread_barrier => enabled
futex => enabled
mysqlnd => enabled
async_redis => enabled

Directive => Local Value => Master Value
swoole.enable_coroutine => On => On
swoole.aio_thread_num => 2 => 2
swoole.display_errors => On => On
swoole.use_namespace => On => On
swoole.use_shortname => On => On
swoole.fast_serialize => Off => Off
swoole.unixsock_buffer_size => 8388608 => 8388608

# cat /var/log/messages | grep swoole
Dec  3 11:06:01 izwz99ilp8qh025s8ak1y9z kernel: php[12589]: segfault at 8 ip 00007fdd0544409b sp 00007fdd04f9fed0 error 4 in swoole.so[7fdd053c9000+132000]
Dec  3 13:40:32 izwz99ilp8qh025s8ak1y9z kernel: php[16214]: segfault at 8 ip 00007fd42a84409b sp 00007fd42a39fed0 error 4 in swoole.so[7fd42a7c9000+132000]

# uname -a
Linux izwz99ilp8qh025s8ak1y9z 3.10.0-693.2.2.el7.x86_64 #1 SMP Tue Sep 12 22:26:13 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux

# php -v
PHP 7.1.24 (cli) (built: Nov  7 2018 18:08:20) ( NTS )
Copyright (c) 1997-2018 The PHP Group
Zend Engine v3.1.0, Copyright (c) 1998-2018 Zend Technologies
    with Zend OPcache v7.1.24, Copyright (c) 1999-2018, by Zend Technologies

# gcc -v
Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/usr/libexec/gcc/x86_64-redhat-linux/4.8.5/lto-wrapper
Target: x86_64-redhat-linux
Configured with: ../configure --prefix=/usr --mandir=/usr/share/man --infodir=/usr/share/info --with-bugurl=http://bugzilla.redhat.com/bugzilla --enable-bootstrap --enable-shared --enable-threads=posix --enable-checking=release --with-system-zlib --enable-__cxa_atexit --disable-libunwind-exceptions --enable-gnu-unique-object --enable-linker-build-id --with-linker-hash-style=gnu --enable-languages=c,c++,objc,obj-c++,java,fortran,ada,go,lto --enable-plugin --enable-initfini-array --disable-libgcj --with-isl=/builddir/build/BUILD/gcc-4.8.5-20150702/obj-x86_64-redhat-linux/isl-install --with-cloog=/builddir/build/BUILD/gcc-4.8.5-20150702/obj-x86_64-redhat-linux/cloog-install --enable-gnu-indirect-function --with-tune=generic --with-arch_32=x86-64 --build=x86_64-redhat-linux
Thread model: posix
gcc version 4.8.5 20150623 (Red Hat 4.8.5-28) (GCC)

timedout

opened by springleng 5

尝试使用中遇见的问题汇总

1, 只能在go(function(){})写代码吗？代码如下：

public function index(Request $request) {
    dd(SaberGM::get('http://httpbin.org/get'));
}

报错信息如下：

You can only use coroutine client in `go` function or some Event callback functions. Please check https://wiki.swoole.com/wiki/page/696.html

2, 只能通过php-cli使用这个项目吗？代码如下：

public function index(Request $request) {
    dd(go(function () {
        SaberGM::get('http://httpbin.org/get');
    }));
}

报错信息如下：

Symfony \ Component \ Debug \ Exception \ FatalErrorException (E_UNKNOWN)
go(): async-io must be used in PHP CLI mode

3, 如果不用CLI该如何使用？

opened by amorZhu 4

获取请求内容乱码

…… $url = "https://apis.map.qq.com/ws/geocoder/v1/?location={$lat},{$lng}&key={$key}&get_poi={$get_poi}"; $response = SaberGM::get($url); $response->getBoay()->getContents();乱码

PHP 8.1.2 (cli)

[PHP Modules]

bcmath calendar Core ctype curl date dom exif FFI fileinfo filter ftp gd gettext hash iconv igbinary intl json libxml mbstring mysqli mysqlnd openssl pcntl pcre PDO xlswriter xml xmlreader xmlwriter xsl Zend OPcache zip zlib

swoole

Swoole => enabled Author => Swoole Team [email protected] Version => 4.8.9 Built => Jun 9 2022 09:54:58 coroutine => enabled with boost asm context epoll => enabled eventfd => enabled signalfd => enabled cpu_affinity => enabled spinlock => enabled rwlock => enabled openssl => OpenSSL 3.0.2 15 Mar 2022 dtls => enabled http2 => enabled json => enabled mutex_timedlock => enabled pthread_barrier => enabled futex => enabled async_redis => enabled

Directive => Local Value => Master Value swoole.enable_coroutine => On => On swoole.enable_library => On => On swoole.enable_preemptive_scheduler => Off => Off swoole.display_errors => On => On swoole.use_shortname => Off => Off swoole.unixsock_buffer_size => 8388608 => 8388608

opened by FATZNG 3
如何禁止url自动转码?

最近打算使用saber来作为twitter api 的客户端,发现saber会自动对url进行转码.导致api请求失败.

经过测试发现: 使用 SaberGM::get("http://127.0.0.1/r.php?max_results=10&tweet.fields=lang,created_at,entities&expansions=attachments.media_keys&media.fields=type,preview_image_url,url");

服务端接收到经过转码的QUERY_STRING: [QUERY_STRING] => max_results=10&tweet_fields=lang%2Ccreated_at%2Centities&expansions=attachments.media_keys&media_fields=type%2Cpreview_image_url%2Curl

而用curl 发送请求: curl "http://127.0.0.1/r.php?max_results=10&tweet.fields=lang,created_at,entities&expansions=attachments.media_keys&media.fields=type,preview_image_url,url" 服务端能接收到正确的QUERY_STRING: [QUERY_STRING] => max_results=10&tweet.fields=lang,created_at,entities&expansions=attachments.media_keys&media.fields=type,preview_image_url,url

请问如何禁止saber对url自动转码?

opened by Ben-Huang 2

You can't enqueue a waiting request when using the max concurrency control!

设置了retry_time和max_co的时候，触发重试会报错"You can't enqueue a waiting request when using the max concurrency control!". 最小化代码：

<?php
go(function() {
    $idList = [
        '1',
        '2',
        '3',
        '4',
        '5',
    ];
    foreach ($idList as $id) {
        $urls[] = [
          'uri' => 'https://domain/path/ . $id,
        ];
    }
    
    try {
        $responses = SaberGM::requests($urls, [
            'exception_report' => 0,
            'retry_time'       => 3,
            'max_co'           => 2
        ]);
    } catch (Exception $e) {
        echo $e->getMessage(); // You can't enqueue a waiting request when using the max concurrency control!
    }
});

opened by axpwx 0

Cookie parseHeader方法解析cookie需要增加容错

$kvs = explode(';', $header); // 需要去除";"后的空格，因为有些服务设置cookie没有在分号后追加空格

$cookie['value'] = str_replace(['''', '""'], '', trim($nv[1])); // 去除首位空格

$kv = explode('=', trim($kv)); // 同上

opened by ppanphper 1

关于Http客户端download时请求头Range

问题描述

在使用http客户端下载文件时, 对端返回的http status code 为 419. 测试后发现下载文件的请求头中包含了 Range 但是为空. 示例:

Host: 192.168.8.52
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8
Accept-Encoding: gzip
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.132 Safari/537.36
Content-Type: application/x-www-form-urlencoded
Range:

这并不符合: Range: bytes=start-end 的格式。因此服务端会判定为异常并返回code 419. 于是试图换个思路，通过传递headers来复写请求头中的header。当我传递Range: bytes=1-后, 发现请求头中的Range消失了....，文件可以正常的下载

虽然目前解决了业务中的问题，但是感觉这应该属于不合理的地方。应该修改withDownloadOffset方法当offset = 0时请求头不包含Range

或者有没有更合适的方法来解决这个问题。请大佬们帮忙看看。

版本信息

composer info | grep saber
swlib/saber                        v1.x-dev 7b97f01   Swoole coroutine HTTP client

PHP 8.0.10 (cli) (built: Aug 27 2021 01:57:09) ( NTS )
Copyright (c) The PHP Group
Zend Engine v4.0.10, Copyright (c) Zend Technologies

swoole

Swoole => enabled
Author => Swoole Team <[email protected]>
Version => 4.6.7
Built => Sep  6 2021 02:26:14
coroutine => enabled with boost asm context
epoll => enabled
eventfd => enabled
signalfd => enabled
spinlock => enabled
rwlock => enabled
openssl => OpenSSL 1.1.1l  24 Aug 2021
dtls => enabled
http2 => enabled
json => enabled
curl-native => enabled
pcre => enabled
zlib => 1.2.11
mutex_timedlock => enabled
pthread_barrier => enabled
async_redis => enabled

示例代码及结果

<?php

use Swlib\Saber;
use Swlib\SaberGM;
use Swlib\Http\Exception\RequestException;

require 'base.php';

\Swoole\Coroutine\run(function () {
    $url = 'http://192.168.8.52/group1/M00/00/00/wKgINGFEMNSAH4vNAAAsk_heX2w512.jpg?token=9ad2493d8d0c3ab2eab7d419d277992d&ts=1631953656';

    $client = Saber::create();

    // 请求头中存在Range, 但是Range为空
    $request = $client->download($url, './test', 0, [
        'timeout' => 60,
        'psr' => true
    ]);
    echo $request->getHeadersString() . PHP_EOL;
    echo '-----------' . PHP_EOL;

    // 请求头中存在Range, Range为  Range: bytes=5-
    $request = $client->download($url, './test', 5, [
        'timeout' => 60,
        'psr' => true
    ]);
    echo $request->getHeadersString() . PHP_EOL;
    echo '-----------' . PHP_EOL;
    
    // 请求头中不存在Range
    $request = $client->download($url, './test', 0, [
        'timeout' => 60,
        'psr' => true,
        'headers' =>[
            'Range' => 'bytes=1-',
            'Test' => 'bytes=1-',
        ]
    ]);
    echo $request->getHeadersString() . PHP_EOL;
    echo '-----------' . PHP_EOL;

    // 请求头中存在Range, Range为  Range: bytes=5- 
    $request = $client->download($url, './test', 5, [
        'timeout' => 60,
        'psr' => true,
        'headers' =>[
            'Range' => 'bytes=1-',
            'Test' => 'bytes=1-',
        ]
    ]);
    echo $request->getHeadersString() . PHP_EOL;
    echo '-----------' . PHP_EOL;
    
});

Host: 192.168.8.52
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8
Accept-Encoding: gzip
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.132 Safari/537.36
Content-Type: application/x-www-form-urlencoded
Range:
-----------
Host: 192.168.8.52
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8
Accept-Encoding: gzip
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.132 Safari/537.36
Content-Type: application/x-www-form-urlencoded
Range: bytes=5-
-----------
Host: 192.168.8.52
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8
Accept-Encoding: gzip
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.132 Safari/537.36
Content-Type: application/x-www-form-urlencoded
Test: bytes=1-
-----------
Host: 192.168.8.52
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8
Accept-Encoding: gzip
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.132 Safari/537.36
Content-Type: application/x-www-form-urlencoded
Test: bytes=1-
Range: bytes=5-
-----------

bug

opened by Dmcz 0

redirect后cookies失效

正确响应的情况

$ch = curl_init();
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_URL, 'https://fees.uspto.gov/mntfee-services/v1/maintenancefee/details?patentNumber=8690232&applicationNumber=13869358');
curl_setopt($ch, CURLOPT_COOKIEJAR, getcwd() . "/cookies.txt");
curl_setopt($ch, CURLOPT_COOKIEFILE, getcwd() . "/cookies.txt");
$result = curl_exec($ch);
dump($result);

未能正确响应

$response = SaberGM::get('https://fees.uspto.gov/mntfee-services/v1/maintenancefee/details?patentNumber=8690232&applicationNumber=13869358');
dump($response->getStatusCode());
dump($response->getBody()->getContents());

经过2次跳转后，正常结果应该是json文本。

opened by blesswarrior 0

Releases(v1.0.18)

v1.0.18(Apr 20, 2021)
Bind Support (#119)

Support ssl_cert_file/ssl_key_file (#123)

Source code(tar.gz)
Source code(zip)
v1.0.17(Jan 21, 2021)
Fixed add data at the same time when uploading files (#103)

Added get httpcode when error code is empty (#106)

Fixed non-redirect failure (#113)

Source code(tar.gz)
Source code(zip)
v1.0.16(Aug 15, 2020)
update composer.json (#101)

Source code(tar.gz)
Source code(zip)
v1.0.15(May 10, 2020)

Source code(tar.gz)
Source code(zip)
v1.0.14(May 8, 2020)

Fix getProtocolVersion error
Source code(tar.gz)
Source code(zip)
v1.0.13(May 7, 2020)

Fix bound error
Source code(tar.gz)
Source code(zip)
v1.0.12(May 7, 2020)

Optimization for redirect response
Source code(tar.gz)
Source code(zip)
v1.0.11(May 7, 2020)

Some bug fixes
Source code(tar.gz)
Source code(zip)
v1.0.10(Sep 26, 2019)
Fix empty body related to transOptionsToRequest()

Source code(tar.gz)
Source code(zip)
v1.0.9(Sep 25, 2019)
Fix empty Referer header

Fix HTTP message generated by __toString()

Source code(tar.gz)
Source code(zip)
v1.0.7(Jul 23, 2019)
Fix Swoole\Buffer deprecated

Source code(tar.gz)
Source code(zip)
v1.0.6(Nov 28, 2018)
Fixed retry handler and add some tests (#19 via 5cd1134)

Source code(tar.gz)
Source code(zip)
v1.0.5(Nov 13, 2018)
Improve redirect test

Fixed cross-site redirect BUG (#16)

Source code(tar.gz)
Source code(zip)
v1.0.4(Nov 2, 2018)
Adapt the URI which is like localhost, automatically complete the protocol part of this kind of URI

Source code(tar.gz)
Source code(zip)
v1.0.3(Nov 2, 2018)
Improve Swoole compatibility.

Source code(tar.gz)
Source code(zip)
v1.0.2(Oct 3, 2018)
PHPUNIT支持运行单个测试

修复手动设置Host标头无法覆盖的问题

修复错误的错误码判断

Source code(tar.gz)
Source code(zip)
v1.0.1(Sep 25, 2018)
单元测试适配Swoole 4

修复use_pool为数字时, 使用base_uri报错的问题

Source code(tar.gz)
Source code(zip)
v1.0.0-beta.1(Apr 6, 2018)

修复已知BUG.
Source code(tar.gz)
Source code(zip)
v1.0.0-beta(Apr 6, 2018)
基于Swoole协程Client开发

人性化使用风格, ajax.js/axios.js/requests.py用户福音, 同时支持PSR风格操作

浏览器级别完备的Cookie管理机制, 完美适配爬虫/API代理应用

请求/响应/异常拦截器

多请求并发, 并发重定向优化, 自动化复用长连接

响应报文自动编码转换

HTTPS连接, CA证书自动化支持

HTTP/Socks5 Proxy支持

重定向控制, 自动化长连接复用

自动化编码请求/解析响应数据

毫秒超时定时器

Source code(tar.gz)
Source code(zip)
v1.0.0(Aug 17, 2018)

First stable release.
Source code(tar.gz)
Source code(zip)
v1.0.0-rc.2(Aug 10, 2018)
Improve some codes.

Update dependencies.

Source code(tar.gz)
Source code(zip)
v1.0.0-rc.1(Jul 23, 2018)
Saber[v1.0.0-rc.1]

建议使用Swoole4最新版驱动以获得最强性能和最大稳定性

基于Swoole协程Client开发

人性化使用风格, ajax.js/axios.js/requests.py用户福音, 同时支持PSR风格操作

浏览器级别完备的Cookie管理机制, 完美适配爬虫/API代理应用

请求/响应/异常拦截器

多请求并发, 并发重定向优化

连接池, 自动化复用长连接

通道池(Chan): 最大连接数限制+无阻塞

HTTPS连接, CA证书自动化支持

HTTP/Socks5 Proxy支持

WebSocket连接支持

毫秒级超时定时器

自动化编码请求/解析响应数据

响应报文自动编码转换

异步超大文件上传/下载, 断点重传

自动重试机制

单次并发数控制

多模式/超细粒度异常处理机制

Source code(tar.gz)
Source code(zip)