DoraRPC is an RPC For the PHP MicroService by The Swoole

Overview

Dora RPC

Build Status Latest Stable Version Latest Unstable Version License

简介(Introduction)

Dora RPC 是一款基础于Swoole定长包头通讯协议的最精简的RPC, 用于复杂项目前后端分离,分离后项目都通过API工作可更好的跟踪、升级、维护及管理。

问题提交: Issue

For complex projects separation, the project can be better maintained by the API project management.

  • Dora RPC is an Basic Swoole Fixed Header TCP Proctol tiny RPC
  • Now support an simple PHP version
  • If you find something wrong,please submit an issue
  • add the http protocol and KeepAlive for the other program language

设计思路(Design)

功能支持(Function)

  • 支持单API调用,多API并发调用
  • 支持同步调用,异步任务下发不等待结果,异步任务下发统一拿回结果
  • 其他相关知识请参考Swoole扩展
  • 客户端长链接,请求完毕后仍旧保留,减少握手消耗
  • guid收发一致性检测,避免发送和接收数据不一致
  • 基于redis制作的服务发现
  • Single API RPC \ Multi API Concurrent RPC
  • Asynchronous,synchronization no need result, synchronization get result by manual
  • Please visit Swoole official for further infomation
  • keep the connection of client after the request finishe
  • check the guid when the send<->recive
  • service discovery.
  • base on Redis. Service discovery for High available

请安装依赖(depend)

  • Swoole 1.8.x+
  • PHP 5.4+
  • zlib for compress packet

##Installation

composer require "xcl3721/dora-rpc"

文件功能简介(File)

dora-rpc/src/Client.php

  • 使用最简单的方式实现的客户端,通过这个框架可以轻松实现PHP的伪多线程,通过分布式加快接口响应速度及高可用
  • an simple client,it's easy adn simply to implement the multi fake thread,you can speed up you API by this distribute RPC

dora-rpc/src/BackEndServer.php

  • API服务端
  • 目前需要继承才能使用,继承后请实现dowork,这个函数是实际处理任务的函数参数为提交参数
  • 做这个只是为了减少大家启用RPC的开发时间
  • 开启服务发现功能,服务端在启动的时候,如果指定redis配置则会自动将当前服务器信息注册到Redis上
  • 返回结果是一个数组 分两部分,第一层是通讯状态code,第二层是处理状态 code
  • a powerful API server
  • you must extends the swserver and implement dowork function
  • it's use for decrease the dev cycle
  • when you setup the redis config the server will register this server to the redis for service discovery
  • the result will be a two-level arrayfirst is communicate state 'code field' ,second is dowork state

dora-rpc/src/Monitor.php

  • 服务发现客户端,通过扫描Redis获取到所有可用后端服务列表,并生成配置到指定路径
  • an discovery controller client that:scan all the redis and get the list of available service and general config file to special path

dora-rpc/src/groupclient.php (combined to client.php)

  • 服务发现monitor进程产生的配置可以用这个客户端直接引用,请求时可以指定使用哪个组的服务
  • an client for service discovery (monitor general the config from redis) that you can use the config directly

使用方法(Example)

任务下发模式介绍(task deploy mode)

  • 0 sync wait result 同步下发任务阻塞等待结果返回
  • 1 async no need result 下发异步任务,下发成功返回下发成功提示,不等待任务处理结果
  • 2 async get result by getAsyncData function 下发异步任务,下发成功返回下发成功提示,可以在后续调用getAsyncData 获取所有下发的异步结果

TCP客户端(TCP Client)

$config = include("client.conf.php");
//define the mode
$mode = array("type" => 1, "group" => "group1");

$maxrequest = 0;

//new obj
$obj = new \DoraRPC\Client($config);

//change connect mode
$obj->changeMode($mode);

for ($i = 0; $i < 10000; $i++) {
    //echo $i . PHP_EOL;

    //single
    $time = microtime(true);

    //single && sync
    $ret = $obj->singleAPI("/module_a/abc" . $i, array("mark" => 234, "foo" => $i), \DoraRPC\DoraConst::SW_MODE_WAITRESULT, 1);
    var_dump("single sync", $ret);

    //single call && async
    $ret = $obj->singleAPI("/module_b/abc" . $i, array("yes" => 21321, "foo" => $i), \DoraRPC\DoraConst::SW_MODE_NORESULT, 1);
    var_dump("single async", $ret);

    //single call && async
    $ret = $obj->singleAPI("/module_c/abd" . $i, array("yes" => 233, "foo" => $i), \DoraRPC\DoraConst::SW_MODE_ASYNCRESULT, 1);
    var_dump("single async result", $ret);

    //multi

    //multi && sync
    $data = array(
        "oak" => array("name" => "/module_c/dd" . $i, "param" => array("uid" => "ff")),
        "cd" => array("name" => "/module_f/ef" . $i, "param" => array("pathid" => "fds")),
    );
    $ret = $obj->multiAPI($data, \DoraRPC\DoraConst::SW_MODE_WAITRESULT, 1);
    var_dump("multi sync", $ret);

    //multi && async
    $data = array(
        "oak" => array("name" => "/module_d/oakdf" . $i, "param" => array("dsaf" => "32111321")),
        "cd" => array("name" => "/module_e/oakdfff" . $i, "param" => array("codo" => "f11ds")),
    );
    $ret = $obj->multiAPI($data, \DoraRPC\DoraConst::SW_MODE_NORESULT, 1);
    var_dump("multi async", $ret);

    //multi && async
    $data = array(
        "oak" => array("name" => "/module_a/oakdf" . $i, "param" => array("dsaf" => "11")),
        "cd" => array("name" => "/module_b/oakdfff" . $i, "param" => array("codo" => "f11ds")),
    );
    $ret = $obj->multiAPI($data, \DoraRPC\DoraConst::SW_MODE_ASYNCRESULT, 1);
    var_dump("multi async result", $ret);

    //get all the async result
    $data = $obj->getAsyncData();
    var_dump("allresult", $data);
    
    //compare each request
    $time = bcsub(microtime(true), $time, 5);
    if ($time > $maxrequest) {
        $maxrequest = $time;
    }
    echo $i . " cost:" . $time . PHP_EOL;
}
echo "max:" . $maxrequest . PHP_EOL;

HTTP客户端(Http Client)

http protocol for the other language use performance is common.suggest used tcp client

for ($i = 0; $i < 10000; $i++) {
    $time = microtime(true);

    //mutil call sync wait result
    $data = array(
        "guid" => md5(mt_rand(1000000, 9999999) . mt_rand(1000000, 9999999) . microtime(true)),

        "api" => array(
            "oak" => array("name" => "/module_d/oakdf", "param" => array("dsaf" => "32111321")),
            "cd" => array("name" => "/module_e/oakdfff", "param" => array("codo" => "f11ds")),
        )
    ,
    );

    $data_string = "params=" . urlencode(json_encode($data));

    $ch = curl_init('http://127.0.0.1:9566/api/multisync');
    curl_setopt($ch, CURLOPT_CUSTOMREQUEST, "POST");
    curl_setopt($ch, CURLOPT_POSTFIELDS, $data_string);
    curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
    curl_setopt($ch, CURLOPT_HTTPHEADER, array(
            'Connection: Keep-Alive',
            'Keep-Alive: 300',
        )
    );

    $result = curl_exec($ch);
    var_dump(json_decode($result, true));


    //multi call no wait result
    $data = array(
        "guid" => md5(mt_rand(1000000, 9999999) . mt_rand(1000000, 9999999) . microtime(true)),

        "api" => array(
            "oak" => array("name" => "/module_d/oakdf", "param" => array("dsaf" => "32111321")),
            "cd" => array("name" => "/module_e/oakdfff", "param" => array("codo" => "f11ds")),
        )
    ,
    );

    $data_string = "params=" . urlencode(json_encode($data));

    $ch = curl_init('http://127.0.0.1:9566/api/multinoresult');
    curl_setopt($ch, CURLOPT_CUSTOMREQUEST, "POST");
    curl_setopt($ch, CURLOPT_POSTFIELDS, $data_string);
    curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
    curl_setopt($ch, CURLOPT_HTTPHEADER, array(
            'Connection: Keep-Alive',
            'Keep-Alive: 300',
        )
    );

    $result = curl_exec($ch);
    var_dump(json_decode($result, true));


    $time = bcsub(microtime(true), $time, 5);
    if ($time > $maxrequest) {
        $maxrequest = $time;
    }
    echo $i . " cost:" . $time . PHP_EOL;
    //var_dump($ret);
}
echo "max:" . $maxrequest . PHP_EOL;

服务端(Server)

class Server extends DoraRPCServer {

    //all of this config for optimize performance
    //以下配置为优化服务性能用,请实际压测调试
    protected  $externalConfig = array(

        //to improve the accept performance ,suggest the number of cpu X 2
        //如果想提高请求接收能力,更改这个,推荐cpu个数x2
        'reactor_num' => 32,

        //packet decode process,change by condition
        //包处理进程,根据情况调整数量
        'worker_num' => 40,

        //the number of task logical process progcessor run you business code
        //实际业务处理进程,根据需要进行调整
        'task_worker_num' => 20,
    );

    function initServer($server){
        //the callback of the server init 附加服务初始化
        //such as swoole atomic table or buffer 可以放置swoole的计数器,table等
    }
    function doWork($param){
        //process you logical 业务实际处理代码仍这里
        //return the result 使用return返回处理结果
        return array("hehe"=>"ohyes");
    }

    function initTask($server, $worker_id){
        //require_once() 你要加载的处理方法函数等 what's you want load (such as framework init)
    }
}

$res = new Server();

###客户端监控器(Client Local Monitor)

include "src/Doraconst.php";
include "src/Packet.php";
include "src/Monitor.php";


//redis for service discovery register
//when you on product env please prepare more redis to registe service for high available
$redisconfig = array(
    array(//first reporter
        "ip" => "127.0.0.1",
        "port" => "6379",
    ),
    array(//next reporter
        "ip" => "127.0.0.1",
        "port" => "6379",
    ),
);

//ok start server
$res = new \DoraRPC\Monitor("0.0.0.0", 9569, $redisconfig, "./client.conf.php");
//this server will auto get the node server list from redis and general the client config on special path

以上代码测试方法

include以上两个文件,使用命令行启动即可(客户端支持在apache nginx fpm内执行,服务端只支持命令行启动)

  • php democlient.php
  • php demoserver.php

错误码及含义(Error Code)

  • 0 Success work
  • 100001 async task success
  • 100002 unknow task type
  • 100003 you must fill the api parameter on you request
  • 100005 Signed check error
  • 100006 Pack decode type wrong
  • 100007 socket error the recive packet length is wrong
  • 100008 the return guid wrong may be the socket trasfer wrong data
  • 100009 the recive wrong or timeout
  • 100010 there is no server can connect
  • 100011 unknow cmd of controlle
  • 100012 Get Async Result Fail: Client Closed.
  • 100099 unknow communicate mode have been set
  • 100100 guid wront please retry..

性能(Performance)

  • Mac I7 Intel 2.2Mhz
  • Vagrant with Vm 1 Core
  • 1G Memory
  • with example code (loop forever)

测试结果Result

  • Network Cost:0.002~0.004/sec Per Request
  • CPU 10~25% 以上还有很大优化空间 There is still a lot of optimization space

Optimize performance性能优化

vim demoserver.php
to see $externalConfig var
and swoole offcial document

如果想优化性能请参考以上文件的$externalConfig配置

Server Config Optimize

License授权

Apache

QQ Group

QQ Group:346840633

Issues
  • 关于rpc的一个疑问

    关于rpc的一个疑问

    如果rpc是阻塞的,那么在等待rpc返回的这段时间里php进程就没有别的事可做就进程饥饿,就很难支持大qps了。。。就算使用的异步的httpclient如果有一个wait的过程那还是有进程饥饿,dora用的是swoole1。x版本吧,这个问题是如果解决的呢?

    opened by gouchaoer 3
  • 客户端共用同一个swoole_client的冲突问题

    客户端共用同一个swoole_client的冲突问题

    如果在同一个程序中创建两个客户端,分别连接不同的服务器,比如:

    第一个客户端连接 127.0.0.1:3000, 第二个客户端连接 127.0.0.1:3001,因为共用同一个 swoole_client,会造成冲突,无法使用。

    opened by andot 2
  • 数据安全问题

    数据安全问题

    包头将会做改进,留在这里做记录 将会对包信息配置指定是否签名验证

    opened by archdesigner 1
  • 发现一个Bug

    发现一个Bug

    BackEndServer.php line:350 括号闭合的位置错了:count($requestInfo["api"] == 0) if (!is_array($requestInfo["api"]) && count($requestInfo["api"] == 0)) { ...

    opened by beyern 1
  • 新的异步方式将会支持

    新的异步方式将会支持

    在客户端程序启动的时候,下发一批要异步处理的任务,并获取到他们对应的token 当应用服务器获取到任务后执行完毕,将结果以token+ip+端口放到redis内

    客户端在处理完所有事宜后,去redis用token拿到本次所有调用结果

    opened by archdesigner 1
  • 修改格式

    修改格式

    opened by hehexianshi 1
  • 增加 travis

    增加 travis

    我就能干点这个 别介意

    opened by hehexianshi 1
  • ip自动情况下,有可能出现group发生变化(运行了其它dora server),而monitor机制仍然正常的情况

    ip自动情况下,有可能出现group发生变化(运行了其它dora server),而monitor机制仍然正常的情况

    虽然这种情况可能比较特殊,不过我使用docker确确实实遇到了。

    描述: 由于redis中dora.serverlist中没有用redis ttl,依赖的是server report。如果server ip是自动获取的,比如这种情况会发生问题:Server1原ip 172.20.0.12,其group为groupA。服务重启(或重建),Server1的ip变为172.20.0.20,其group为groupA;原ip 172.20.0.12可能成为了Server2,其group为groupB。这个时候,172.20.0.12这个ip的时间戳仍会更新,client的monitor会继续生成groupA中含有172.20.0.12。

    opened by ddonng 0
  • Update README.md

    Update README.md

    fix some English grammer stuff

    opened by ShiFengAway 0
  • fix doc markdown style

    fix doc markdown style

    fix doc markdown style

    opened by wujunze 0
  • swoole升级到2.0.6后报错跑步起来

    swoole升级到2.0.6后报错跑步起来

    Notice: Undefined property: APIServer::$server in /mnt/winshare/test/myswoole/Dora-RPC-master/src/BackEndServer.php on line 249

    Call Stack: 0.0059 235664 1. {main}() /mnt/winshare/test/myswoole/Dora-RPC-master/test/demoserver.php:0 0.0239 531832 2. DoraRPC\BackEndServer->start() /mnt/winshare/test/myswoole/Dora-RPC-master/test/demoserver.php:85 0.0244 540032 3. Swoole\Http\Server->start() /mnt/winshare/test/myswoole/Dora-RPC-master/src/BackEndServer.php:208 14.5473 554160 4. DoraRPC\BackEndServer::onRequest(class Swoole\Http\Request, class Swoole\Http\Response) /mnt/winshare/test/myswoole/Dora-RPC-master/src/BackEndServer.php:0

    Fatal error: Call to a member function task() on null in /mnt/winshare/test/myswoole/Dora-RPC-master/src/BackEndServer.php on line 249

    Call Stack: 0.0059 235664 1. {main}() /mnt/winshare/test/myswoole/Dora-RPC-master/test/demoserver.php:0 0.0239 531832 2. DoraRPC\BackEndServer->start() /mnt/winshare/test/myswoole/Dora-RPC-master/test/demoserver.php:85 0.0244 540032 3. Swoole\Http\Server->start() /mnt/winshare/test/myswoole/Dora-RPC-master/src/BackEndServer.php:208 14.5473 554160 4. DoraRPC\BackEndServer::onRequest(class Swoole\Http\Request, class Swoole\Http\Response) /mnt/winshare/test/myswoole/Dora-RPC-master/src/BackEndServer.php:0

    希望兼容高版本的swoole。

    opened by jxy918 3
  • 建议增加命令行管理服务功能

    建议增加命令行管理服务功能

    比如:

    php server.php start -d -h 127.0.0.1 -p 9567 -w 4 -r 20 -t 20
    -d, --deamon 启用守护进程模式
    -h, --host [<value>] 指定监听地址
    -p, --port  [<value>] 指定监听端口
    --help 显示帮助界面
    -w, --worker  [<value>] 设置Worker进程的数量
    -r, --thread [<value>]  设Reactor进程的数量
    -t, --takser [<value>]  设置Task进程的数量
    
    php server.php reload 重新加载
    php server.php stop 停止服务
    

    同时建议master进程名中显示启动参数

    opened by maxincai 3
  • mac test 启动server报错

    mac test 启动server报错

    Found Report Config... Start Report Process [2016-06-19 10:29:35 @1645.0] ERROR swSocket_set_buffer_size(:313): setsockopt(4, SOL_SOCKET, SO_SNDBUF, 0) failed. Error: Invalid argument[22]. [2016-06-19 10:29:35 @1645.0] ERROR swSocket_set_buffer_size(:313): setsockopt(5, SOL_SOCKET, SO_SNDBUF, 0) failed. Error: Invalid argument[22]

    opened by michealzh 1
  • 提几点建议

    提几点建议

    代码写的很工整,注释也比较清楚,赞一个!这里提几点建议,仅供参考~ 1、虽然很容易看懂,也有例子,但最好再详细介绍一下,比如协议设定,使用方法。 2、服务端需要在初始化的时候载入类库,例子中没有说明。 3、最好采用配置文件的方式,而不是每次去修改类。 4、服务端可以引入控制层,可以启动服务,也可以关闭服务,便于管理。 5、从抽象的角度,应该可以适配更多的协议。 6、可以参考psr-4规范,引入命名空间,自动载入。

    opened by nosun 3
Releases(0.4.15_beta)
Owner
Chang Long Xu
fight goal
Chang Long Xu
Multi-process coroutine edition Swoole spider !! Learn about Swoole's network programming and the use of its related APIs

swoole_spider php bin/spider // Just do it !! Cache use Swoole\Table; use App\Table\Cache; $table = new Table(1<<20); // capacity size $table->column

null 3 Apr 22, 2021
A easy way to install your basic yii projetc, we have encrypt database password in phpfile, my class with alot funtions to help you encrypt and decrypt and our swoole server install just run ./yii swoole/start and be happy!

Yii 2 Basic Project Template with swoole and Modules Yii 2 Basic Project Template is a skeleton Yii 2 application best for rapidly creating small proj

null 2 Nov 25, 2021
swoole,easyswoole,swoole framework

EasySwoole - A High Performance Swoole Framework EasySwoole is a distributed, persistent memory PHP framework based on the Swoole extension. It was cr

null 4.4k Jan 14, 2022
🚀 PHP Microservice Full Coroutine Framework

PHP microservice coroutine framework 中文说明 Introduction Swoft is a PHP microservices coroutine framework based on the Swoole extension. Like Go, Swoft

Swoft Cloud 5.4k Jan 18, 2022
Files Course Laravel Microservice E-mail

Curso Laravel Microservices com RabbitMQ (micro e-mail) Saiba Mais Sobre o Curso Requisitos Este micro e-mail depende do microservice 01, portanto, pr

EspecializaTi 9 Oct 21, 2021
Light, concurrent RPC framework for PHP & C

Yar - Yet Another RPC framework for PHP Light, concurrent RPC framework for PHP(see also: Yar C framework, Yar Java framework) Requirement PHP 7.0+ (m

Xinchen Hui 1.4k Jan 11, 2022
PHP Kafka client is used in PHP-FPM and Swoole. PHP Kafka client supports 50 APIs, which might be one that supports the most message types ever.

longlang/phpkafka Introduction English | 简体中文 PHP Kafka client is used in PHP-FPM and Swoole. The communication protocol is based on the JSON file in

Swoole Project 196 Jan 13, 2022
一个极简高性能php框架,支持[swoole | php-fpm ]环境

One - 一个极简高性能php框架,支持[swoole | php-fpm ]环境 快 - 即使在php-fpm下也能1ms内响应请求 简单 - 让你重点关心用one做什么,而不是怎么用one 灵活 - 各个组件松耦合,可以灵活搭配使用,使用方法保持一致 原生sql可以和模型关系with搭配使用,

vic 845 Jan 9, 2022
球球大作战(PHP+Swoole)

球球大作战 在线多人H5游戏(H5客户端, 服务端) W A S D 控制 技术栈: PHP7.4+ Swoole4.6(多进程, WebSocket, 共享内存) SpriteJS v3(2D Canvas渲染) 演示Demo 安装: 环境要求:Linux,PHP7.4+(启用Swoole拓展)

Vacant 13 Jan 8, 2022
🚀 Developing Rocketseat's Next Level Week (NLW#05) Application using PHP/Swoole + Hyperf

Inmana PHP ?? Developing Rocketseat 's Next Level Week (NLW#05) Application using Swoole + Hyperf. This is the app of the Elixir track. I know PHP/Swo

Leo Cavalcante 20 Nov 21, 2021
Simple live support server with PHP Swoole Websocket and Telegram API

Telgraf Simple live support server with PHP Swoole Websocket and Telegram API. Usage Server Setup Clone repository with following command. git clone h

Adem Ali Durmuş 6 Oct 4, 2021
Redis watcher for PHP-Casbin in Swoole.

Redis watcher for PHP-Casbin in Swoole Redis watcher for PHP-Casbin in Swoole , Casbin is a powerful and efficient open-source access control library.

PHP-Casbin 1 Nov 22, 2021
Socks5 proxy server written in Swoole PHP

php-socks This is a Socks5 proxy server implementation built with PHP & Swoole. To start the proxy server, clone this repo, run composer install to in

Nazmul Alam 1 Nov 25, 2021
基于 swoole 的多进程队列系统,低延时(最低毫秒级)、低资源占用, 支持一键化协程、超时控制、失败重试。可与 laravel thinkphp 等框架配合使用

multi-process-queue 基于swoole的多进程队列系统,manage进程管理子进程,master进程监听队列分发任务,worker进程执行任务, 多进程、低延时(最低毫秒级)、低资源占用。可与 laravel thinkphp 等框架配合使用 版本要求: php>=7.1 swoo

yuntian 19 Jan 14, 2022
swoole worker

SwooleWorker SwooleWorker is a distributed long connection development framework based on Swoole4. 【Github】 【HomePage】 Manual 【ENGLISH】 【简体中文】 Usage s

null 90 Jan 9, 2022
Library for Open Swoole extension

Open Swoole Library This library works with Open Swoole since release version v4.7.1. WIP Table of Contents How to Contribute Code Requirements Develo

Open Swoole 1 Dec 19, 2021
Hyperf instant messaging program based on swoole framework

Hyperf instant messaging program based on swoole framework

null 16 Jan 6, 2022
LaravelS is an out-of-the-box adapter between Swoole and Laravel/Lumen.

?? LaravelS is an out-of-the-box adapter between Swoole and Laravel/Lumen.

Biao Xie 3.5k Jan 15, 2022
Hprose asynchronous client & standalone server based on swoole

Hprose for Swoole Introduction Hprose is a High Performance Remote Object Service Engine. It is a modern, lightweight, cross-language, cross-platform,

Hprose 185 Nov 15, 2021