硬件环境

腾讯云 8核cpu，16G内存，100G SSD本地硬盘，

现象描述

最初写入速度 200w/分钟，几分钟后维持在平均180w/分钟，但写入数据量达到80G后，写入速度下降到50w/分钟

问题

请问这个是合理的下降还是有什么配置可以优化?

感谢360宋同学的回复 单独压测pika slotmigrate 要为 no 因为现在打开codis之后，其实你的所有kv都是存在set数据结构下，这样每一次操作其实在引擎下面会转化成很多自操作，相比关闭之后会慢不少

写入方式为 golang 创建 50个redis客户端，然后50个协程调用test函数

func test(client *redis.Client){
	for true{
		pipe:=client.Pipeline();
		u1 := uuid.NewV4().String()
		key0 := u1+":user:0"
		pipe.Incr(key0);
		pipe.Incr(u1+":user:1");
		pipe.Incr(u1+":user:2");
		pipe.Incr(u1+":user:3");
		pipe.Incr(u1+":user:4");
		//检查是否处理过
		pipe.Exists(u1)
		pipe.Set(u1+":orderId","1",time.Hour*24);
		pipe.LPush(u1+":user:4",
			"{'openId':'6BdpVU','yyyappid':'97107fa23','action':'minus'," +
				"'spend':1000," +
				"'note':'abc'," +
				"'logTime':1503383236932,'userVc':542939}");
		_ ,err:= pipe.Exec();
		if(nil!=err){
			fmt.Println(err)
		}
	}
}

下面是pika的配置信息

# Pika port
port : 9221
# Thread Number
thread-num : 7
# Sync Thread Number
sync-thread-num : 6
# Item count of sync thread queue
sync-buffer-size : 10
# Pika log path
log-path : ./log_1/
# Pika glog level: only INFO and ERROR
loglevel : INFO
# Pika db path
db-path : ./db_1/
# Pika write-buffer-size
write-buffer-size : 268435456
# Pika timeout
timeout : 60
# Requirepass
requirepass : 
# Masterauth
masterauth : 
# Userpass
userpass : 
# User Blacklist
userblacklist : 
# Dump Prefix
dump-prefix : 
# daemonize  [yes | no]
#daemonize : yes
# slotmigrate  [yes | no]
slotmigrate : yes
# Dump Path
dump-path : ./dump_1/
# Expire-dump-days
dump-expire : 0
# pidfile Path
pidfile : ./pika.pid
# Max Connection
maxclients : 20000
# the per file size of sst to compact, defalut is 2M
target-file-size-base : 20971520
# Expire-logs-days
expire-logs-days : 7
# Expire-logs-nums
expire-logs-nums : 10
# Root-connection-num
root-connection-num : 2
# Slowlog-log-slower-than
slowlog-log-slower-than : 10000
# slave-read-only(yes/no, 1/0)
slave-read-only : false
# Pika db sync path
db-sync-path : ./dbsync_1/
# db sync speed(MB) max is set to 125MB, min is set to 0, and if below 0 or above 125, the value will be adjust to 125
db-sync-speed : 125
# network interface
# network-interface : eth1
# replication
# slaveof : master-ip:master-port
#
# CronTask, format: start-end/ratio, like 02-04/60, pika will check to schedule compaction between 2 to 4 o'clock everyday
#                   if the freesize/disksize > 60%. NOTICE:if compact-interval is set, compact-cron will be mask and disable.
# compact-cron :
#
# Compact-interval, format: interval/ratio, like 6/60, pika will check to schedule compaction every 6 hours,
#                           if the freesize/disksize > 60%. NOTICE:compact-interval is prior than compact-cron;
# compact-interval :

###################
## Critical Settings
###################
# binlog file size: default is 100M,  limited in [1K, 2G]
binlog-file-size : 104857600
# Compression
compression : snappy
# max-background-flushes: default is 1, limited in [1, 4]
max-background-flushes : 1
# max-background-compactions: default is 1, limited in [1, 4]
max-background-compactions : 2
# max-cache-files default is 5000
max-cache-files : 5000
# max_bytes_for_level_multiplier: default is 10, you can change it to 5
max-bytes-for-level-multiplier : 10

pika_version:2.3.1 info 部分log I0228 11:26:12.409108 13469 pika_trysync_thread.cc:239] Should connect master I0228 11:26:12.412156 13469 pika_trysync_thread.cc:254] Finish to start rsync, path:./dbsync/ I0228 11:26:12.412297 13469 pika_trysync_thread.cc:257] Connect to master ip:10.0.74.107port: 9221 I0228 11:26:12.412344 13469 pika_trysync_thread.cc:68] *2 $4 auth ....

warning log Log line format: [IWEF]mmdd hh:mm:ss.uuuuuu threadid file:line] msg W0228 11:26:12.413130 13469 pika_trysync_thread.cc:93] Reply from master after trysync: OK W0228 11:26:12.413245 13469 pika_trysync_thread.cc:125] something wrong with sync, come in SyncError stage W0228 11:26:12.413254 13469 pika_server.cc:672] Sync error, set repl_state to PIKA_REPL_ERROR

Hi, do you have some C/C++ api for connection to pika db? i have this issue: https://github.com/fastogt/fastonosql/issues/53 . What you think about integration pika driver into FastoNoSQL? as result you will have gui for db.

1、master端需要密码认证时，在slave端使用slaveof命令，无法连接到master。 请问如何在master有密码的情况下，slave端能成功连接？ 2、在slave端使用slaveof命令时，偶尔会出现以下错误，无法连接到master： I0811 17:52:15.360584 31455 pika_admin.cc:118] Trysync, Slave ip_port: xxx.xxx.xxx.xxx:9221 filenum: 0 pro_offset: 0 I0811 17:52:15.361500 31455 pika_admin.cc:130] Trysync, dont FindSlave, so AddBinlogSender rsync: read error: Connection reset by peer (104) rsync error: error in rsync protocol data stream (code 12) at io.c(759) [sender=3.0.6] W0811 17:52:15.366693 31487 pika_server.cc:577] rsync send file failed! From: ./dump/20160811/list/000025.sst, To: list/000025.sst, At: xxx.xxx.xxx.xxx:12221, Error: 3072 rsync: read error: Connection reset by peer (104) rsync error: error in rsync protocol data stream (code 12) at io.c(759) [sender=3.0.6] rsync: read error: Connection reset by peer (104) rsync error: error in rsync protocol data stream (code 12) at io.c(759) [sender=3.0.6] rsync: read error: Connection reset by peer (104) rsync error: error in rsync protocol data stream (code 12) at io.c(759) [sender=3.0.6] rsync: read error: Connection reset by peer (104) rsync error: error in rsync protocol data stream (code 12) at io.c(759) [sender=3.0.6] rsync: read error: Connection reset by peer (104) rsync error: error in rsync protocol data stream (code 12) at io.c(759) [sender=3.0.6]

出现的概率很小，但是出现以后，即使执行重启服务、清库等操作，主从也无法连接成功。

env: pika 3.4.0 Oct 12 master branch centos 7.8

master log pika.INFO

I1124 15:38:02.599153 13796 pika_server.cc:1186] Partition: db0 RSync Send Files Success
I1124 15:38:02.987164 23083 pika_repl_server_conn.cc:108] Receive Trysync, Slave ip: xx.xx.xx.245, Slave port:9221, Partition: db0, filenum: 108128, pro_offset: 16589588
I1124 15:38:02.987440 23083 pika_repl_server_conn.cc:192] Partition: db0 TrySync Success, Session: 224
W1124 15:41:49.478492 23082 pika_slave_node.cc:32] Ack offset Start: filenum: 108128 offset: 104856077 term: 0 index: 42319875412 binglog size: 0 acked: 0End: filenum: 108128 offset: 104862926 term: 0 index: 660307 binglog size: 0 acked: 0 not found in binlog controller window.
window status 
      Size: 53
      Begin_item: filenum: 108128 offset: 104856077 term: 0 index: 42319875412 binglog size: 94 acked: 0
      End_item: filenum: 108129 offset: 5198 term: 0 index: 42319875464 binglog size: 118 acked: 0
W1124 15:41:49.478677 23082 pika_rm.cc:203] (db0:0)Corruption: UpdateAckedInfo failed
W1124 15:41:49.478691 23082 pika_repl_server_conn.cc:483] Update binlog ack failed db0 0 Corruption: UpdateAckedInfo failed
I1124 15:41:49.478791 23081 pika_repl_server_thread.cc:29] ServerThread Close Slave Conn, fd: 1864, ip_port: xx.xx.xx.245:41037
I1124 15:41:49.478897 23081 pika_server.cc:740] Delete Slave Success, ip_port: xx.xx.xx.245:9221
I1124 15:41:49.478926 23081 pika_rm.cc:90] Remove Slave Node, Partition: (db0:0), ip_port: xx.xx.xx.245:9221
I1124 15:41:54.239161 23137 pika_stable_log.cc:151] ./log/log_db0/ Success purge 1 binlog file
I1124 15:41:59.032140 23083 pika_repl_server_conn.cc:42] Receive MetaSync, Slave ip: xx.xx.xx.245, Slave port:9221
I1124 15:41:59.032279 23083 pika_server.cc:843] Add New Slave, xx.xx.xx.245:9221
I1124 15:41:59.172390 23084 pika_repl_server_conn.cc:108] Receive Trysync, Slave ip: xx.xx.xx.245, Slave port:9221, Partition: db0, filenum: 108128, pro_offset: 104862926
I1124 15:41:59.172577 23084 pika_rm.cc:79] Add Slave Node, partition: (db0:0), ip_port: xx.xx.xx.245:9221
I1124 15:41:59.172596 23084 pika_repl_server_conn.cc:181] Partition: db0 TrySync Success, Session: 225
W1124 15:41:59.214401 23083 pika_repl_server_conn.cc:466] Activate Binlog Sync failed partition=db0_0,ip_port=xx.xx.xx.245:9221,session id=0 Corruption: Init binlog file reader failedCorruption: partition=db0_0,ip_port=xx.xx.xx.245:9221,session id=225  binlog reader init failed
I1124 15:41:59.214589 23081 pika_repl_server_thread.cc:29] ServerThread Close Slave Conn, fd: 816, ip_port: xx.xx.xx.245:41109

master side log, pika.WARNING

W1124 15:41:49.478492 23082 pika_slave_node.cc:32] Ack offset Start: filenum: 108128 offset: 104856077 term: 0 index: 42319875412 binglog size: 0 acked: 0End: filenum: 108128 offset: 104862926 term: 0 index: 660307 binglog size: 0 acked: 0 not found in binlog controller window.
window status 
      Size: 53
      Begin_item: filenum: 108128 offset: 104856077 term: 0 index: 42319875412 binglog size: 94 acked: 0
      End_item: filenum: 108129 offset: 5198 term: 0 index: 42319875464 binglog size: 118 acked: 0
W1124 15:41:49.478677 23082 pika_rm.cc:203] (db0:0)Corruption: UpdateAckedInfo failed
W1124 15:41:49.478691 23082 pika_repl_server_conn.cc:483] Update binlog ack failed db0 0 Corruption: UpdateAckedInfo failed
W1124 15:41:59.214401 23083 pika_repl_server_conn.cc:466] Activate Binlog Sync failed partition=db0_0,ip_port=xx.xx.xx.245:9221,session id=0 Corruption: Init binlog file reader failedCorruption: partition=db0_0,ip_port=xx.xx.xx.245:9221,session id=225  binlog reader init failed
W1124 15:42:09.223461 23082 pika_repl_server_conn.cc:466] Activate Binlog Sync failed partition=db0_0,ip_port=xx.xx.xx.245:9221,session id=0 Corruption: Init binlog file reader failedCorruption: partition=db0_0,ip_port=xx.xx.xx.245:9221,session id=226  binlog reader init failed
E1124 15:42:11.377209 23102 pika_client_conn.cc:160] ip_port: xx.xx.xx.248:41503, table: db0, command: "INFO", command_size: 4, arguments: 1, start_time(s): 1637739731, duration(us): 10893, do_duration_(us): 0
W1124 15:42:19.234411 23083 pika_repl_server_conn.cc:466] Activate Binlog Sync failed partition=db0_0,ip_port=xx.xx.xx.245:9221,session id=0 Corruption: Init binlog file reader failedCorruption: partition=db0_0,ip_port=xx.xx.xx.245:9221,session id=227  binlog reader init failed
W1124 15:42:29.245422 23082 pika_repl_server_conn.cc:466] Activate Binlog Sync failed partition=db0_0,ip_port=xx.xx.xx.245:9221,session id=0 Corruption: Init binlog file reader failedCorruption: partition=db0_0,ip_port=xx.xx.xx.245:9221,session id=228  binlog reader init failed
E1124 15:42:35.230178 23105 pika_client_conn.cc:160] ip_port: xx.xx.xx.248:36931, table: db0, command: "info", command_size: 4, arguments: 1, start_time(s): 1637739755, duration(us): 10758, do_duration_(us): 0
W1124 15:42:39.256443 23083 pika_repl_server_conn.cc:466] Activate Binlog Sync failed partition=db0_0,ip_port=xx.xx.xx.245:9221,session id=0 Corruption: Init binlog file reader failedCorruption: partition=db0_0,ip_port=xx.xx.xx.245:9221,session id=229  binlog reader init failed
W1124 15:42:49.267314 23084 pika_repl_server_conn.cc:466] Activate Binlog Sync failed partition=db0_0,ip_port=xx.xx.xx.245:9221,session id=0 Corruption: Init binlog file reader failedCorruption: partition=db0_0,ip_port=xx.xx.xx.245:9221,session id=230  binlog reader init failed
E1124 15:42:51.380913 23105 pika_client_conn.cc:160] ip_port: 10.10.59.236:38280, table: db0, command: "INFO", command_size: 4, arguments: 1, start_time(s): 1637739771, duration(us): 10969, do_duration_(us): 0
W1124 15:42:59.279383 23082 pika_repl_server_conn.cc:466] Activate Binlog Sync failed partition=db0_0,ip_port=xx.xx.xx.245:9221,session id=0 Corruption: Init binlog file reader failedCorruption: partition=db0_0,ip_port=xx.xx.xx.245:9221,session id=231  binlog reader init failed
E1124 15:43:06.643637 23112 pika_client_conn.cc:160] ip_port: xx.xx.xx.248:36965, table: db0, command: "info", command_size: 4, arguments: 1, start_time(s): 1637739786, duration(us): 10818, do_duration_(us): 0
W1124 15:43:09.189357 23082 pika_repl_server_conn.cc:466] Activate Binlog Sync failed partition=db0_0,ip_port=xx.xx.xx.245:9221,session id=0 Corruption: Init binlog file reader failedCorruption: partition=db0_0,ip_port=xx.xx.xx.245:9221,session id=232  binlog reader init failed
W1124 15:43:09.218698 23136 pika_rm.cc:780] send binlog to xx.xx.xx.245:9221 failed, NotFound: The xx.xx.xx.245:9221 fd cannot be found
W1124 15:43:09.219070 23136 pika_rm.cc:780] send binlog to xx.xx.xx.245:9221 failed, NotFound: The xx.xx.xx.245:9221 fd cannot be found

slave side log, pika.INFO

Log file created at: 2021/11/24 15:36:01
Running on machine: pika-03
Log line format: [IWEF]mmdd hh:mm:ss.uuuuuu threadid file:line] msg
I1124 15:36:01.708359  6538 pika.cc:193] Server at: ../conf/pika.conf
I1124 15:36:01.708884  6538 pika_server.cc:176] Using Networker Interface: eth0
I1124 15:36:01.709555  6538 pika_server.cc:219] host: xx.xx.xx.245 port: 9221
I1124 15:36:01.709585  6538 pika_server.cc:95] Worker queue limit is 1350
I1124 15:36:01.710130  6538 pika_binlog.cc:94] Binlog: Manifest file not exist, we create a new one.
I1124 15:36:02.767280  6538 pika_partition.cc:87] db0 DB Success
I1124 15:36:02.774467  6538 pika_server.cc:273] Pika Server going to start
I1124 15:36:02.774777  6668 pika_repl_client.cc:146] Try Send Meta Sync Request to Master (xx.xx.xx.248:9221)
I1124 15:36:02.775851  6556 pika_server.cc:618] Mark try connect finish
I1124 15:36:02.775871  6556 pika_repl_client_conn.cc:146] Finish to handle meta sync response
I1124 15:36:02.916692  6557 pika_repl_client_conn.cc:261] Partition: db0 Need To Try DBSync
I1124 15:36:03.016842  6559 pika_repl_client_conn.cc:182] Partition: db0 Need Wait To Sync
I1124 15:38:02.628144  6668 pika_partition.cc:236] Partition: db0 Information from dbsync info,  master_ip: xx.xx.xx.248, master_port: 9221, filenum: 108128, offset: 16589588, term: 0, index: 0
I1124 15:38:02.628228  6668 pika_partition.cc:293] Partition: db0, Prepare change db from: ./db/db0_bak
I1124 15:38:02.875358  6668 pika_partition.cc:314] Partition: db0, Change db success
I1124 15:38:02.987746  6560 pika_repl_client_conn.cc:258] Partition: db0 TrySync Ok
I1124 15:41:49.478904  6555 pika_repl_client_thread.cc:21] ReplClient Close conn, fd=124, ip_port=xx.xx.xx.248:11221
W1124 15:41:49.478972  6555 pika_repl_client_thread.cc:31] Master conn disconnect : xx.xx.xx.248:11221 try reconnect
I1124 15:41:59.031689  6668 pika_repl_client.cc:146] Try Send Meta Sync Request to Master (xx.xx.xx.248:9221)
I1124 15:41:59.032572  6561 pika_server.cc:618] Mark try connect finish
I1124 15:41:59.032599  6561 pika_repl_client_conn.cc:146] Finish to handle meta sync response
I1124 15:41:59.172834  6562 pika_repl_client_conn.cc:258] Partition: db0 TrySync Ok
I1124 15:41:59.214607  6555 pika_repl_client_thread.cc:21] ReplClient Close conn, fd=688, ip_port=xx.xx.xx.248:11221
W1124 15:41:59.214651  6555 pika_repl_client_thread.cc:31] Master conn disconnect : xx.xx.xx.248:11221 try reconnect

slave side pika.WARNING

Log file created at: 2021/11/24 15:38:02
Running on machine: pika-03
Log line format: [IWEF]mmdd hh:mm:ss.uuuuuu threadid file:line] msg
W1124 15:41:49.478972  6555 pika_repl_client_thread.cc:31] Master conn disconnect : xx.xx.xx.248:11221 try reconnect
W1124 15:41:59.214651  6555 pika_repl_client_thread.cc:31] Master conn disconnect : xx.xx.xx.248:11221 try reconnect
W1124 15:42:09.223732  6555 pika_repl_client_thread.cc:31] Master conn disconnect : xx.xx.xx.248:11221 try reconnect
W1124 15:42:19.234695  6555 pika_repl_client_thread.cc:31] Master conn disconnect : xx.xx.xx.248:11221 try reconnect
W1124 15:42:29.245780  6555 pika_repl_client_thread.cc:31] Master conn disconnect : xx.xx.xx.248:11221 try reconnect
W1124 15:42:39.256779  6555 pika_repl_client_thread.cc:31] Master conn disconnect : xx.xx.xx.248:11221 try reconnect

我这里能稳定重现这个问题，请问有谁遇到过相似的问题吗？

从git上clone master版本，根据安装提示，安装了必要的依赖包，在make的时候报以下错误，提示文件或目录不存在 相关log如下： make[1]: Leaving directory /soft/pika_soft/pika/third/slash/slash' make[1]: Entering directory/soft/pika_soft/pika/third/pink/pink' In file included from src/pb_conn.cc:6:0: ../pink/include/pb_conn.h:12:37: fatal error: google/protobuf/message.h: No such file or directory #include "google/protobuf/message.h" ^ compilation terminated. GEN src/build_version.cc make[1]: Leaving directory /soft/pika_soft/pika/third/pink/pink' make[1]: Entering directory/soft/pika_soft/pika/third/pink/pink' In file included from src/pb_conn.cc:6:0: ../pink/include/pb_conn.h:12:37: fatal error: google/protobuf/message.h: No such file or directory #include "google/protobuf/message.h" ^ compilation terminated. GEN src/build_version.cc CXX src/build_version.o CXX src/server_thread.o CXX src/redis_conn.o CXX src/thread_pool.o CXX src/client_thread.o CXX src/pink_cli.o CXX src/pink_pubsub.o CXX src/dispatch_thread.o CXX src/server_socket.o CXX src/pink_thread.o CXX src/worker_thread.o CXX src/redis_cli.o CXX src/holy_thread.o CXX src/pink_epoll.o CXX src/pb_cli.o src/pb_cli.cc:9:37: fatal error: google/protobuf/message.h: No such file or directory #include <google/protobuf/message.h> ^ compilation terminated. make[1]: *** [src/pb_cli.o] Error 1 make[1]: Leaving directory `/soft/pika_soft/pika/third/pink/pink' make: *** [/soft/pika_soft/pika/third/pink/pink/lib/libpink.a] Error 2

系统环境如下：

cat /etc/redhat-release

CentOS Linux release 7.6.1810 (Core)

rpm -qa | grep snappy-devel

snappy-devel-1.1.0-3.el7.x86_64

rpm -qa | grep glog-devel

glog-devel-0.3.3-8.el7.x86_64

rpm -qa | grep gcc-c++

gcc-c++-4.8.5-36.el7_6.2.x86_64

gcc --version

gcc (GCC) 4.8.5 20150623 (Red Hat 4.8.5-36) Copyright (C) 2015 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

版本：3.4.1-beta，gcc版本4.8.5 pika使用的编译版本，直接解压改配置文件，然后启动后报错 pika.FATAL日志打印如下： Log file created at: 2022/06/27 17:38:07 Running on machine: 10-195-19-73 Log line format: [IWEF]mmdd hh:mm:ss.uuuuuu threadid file:line] msg F0627 17:38:07.466754 20708 pika_server.cc:229] Start Rsync Error: bind port 10220 failed, Listen on this port to receive Master FullSync Data pika.INFO打印如下 Log file created at: 2022/06/27 17:38:07 Running on machine: 10-195-19-73 Log line format: [IWEF]mmdd hh:mm:ss.uuuuuu threadid file:line] msg I0627 17:38:07.458413 20708 pika.cc:193] Server at: conf/pika.conf I0627 17:38:07.459499 20708 pika_server.cc:176] Using Networker Interface: eth0 I0627 17:38:07.459726 20708 pika_server.cc:219] host: 10.195.19.73 port: 9220 I0627 17:38:07.459754 20708 pika_server.cc:95] Worker queue limit is 20100 I0627 17:38:07.460306 20708 pika_binlog.cc:94] Binlog: Manifest file not exist, we create a new one.

请问这个是什么原因导致的？

目前pika使用每天会挂掉一次，机器是8c, 60g的，基本上每天都会因为out of memory被kill掉，下面是调用info命令的信息： `

Server

pika_version:2.2.5 pika_git_sha:e48ddac06cd53a501209d21d269b7d934e5b2ff4 pika_build_compile_date: Aug 22 2017 os:Linux 4.1.39-27.28.amzn1.x86_64 x86_64 arch_bits:64 process_id:17034 tcp_port:6379 thread_num:4 sync_thread_num:4 uptime_in_seconds:6418 uptime_in_days:1 config_file:/usr/local/pika/conf/pika.6379.conf

Data

db_size:6492113654 db_size_human:6191M compression:snappy used_memory:171024208 used_memory_human:163M db_memtable_usage:73656944 db_tablereader_usage:97367264

Log

log_size:20910169590 log_size_human:19941M safety_purge:write2file1135 expire_logs_days:7 expire_logs_nums:200 binlog_offset:1145 15763128

Clients

connected_clients:601

Stats

total_connections_received:3894 instantaneous_ops_per_sec:6 total_commands_processed:4096337 is_bgsaving:No, , 0 is_slots_reloading:No, , 0 is_scaning_keyspace:No is_compact:No compact_cron:

Replication(MASTER)

role:master connected_slaves:0

Keyspace

Time:1970-01-01 08:00:00

kv keys:0 hash keys:0 list keys:0 zset keys:0 set keys:0 ` 目前我们链接数基本稳定在600个，之前是2000个链接，但发现600个链接数支撑时间反而没有2000个链接支撑的时间更长，我想问下，pika的缓存策略是什么样的，什么条件下会触发内存释放，我们数据一共才30G，内存60G，现在基本上每天都会爆掉，之前是30G的内存，基本上也是每天爆一次，是否存在内存泄漏。

问题描述: 使用python客户端pipeline入库时报错,但数据却入进去了.

python程序: import redis r=redis.StrictRedis(host='10.40.0.70',port='9221') pr=redis.pipeline() pr.set('t1',1) pr.set('t2',2) pr.execute()

Traceback (most recent call last): File "q5.py", line 7, in pr.execute() File "/usr/lib/python2.7/site-packages/redis/client.py", line 2879, in execute return execute(conn, stack, raise_on_error) File "/usr/lib/python2.7/site-packages/redis/client.py", line 2772, in execute_transaction response = self.parse_response(connection, '') File "/usr/lib/python2.7/site-packages/redis/client.py", line 2838, in parse_response self, connection, command_name, **options) File "/usr/lib/python2.7/site-packages/redis/client.py", line 680, in parse_response response = connection.read_response() File "/usr/lib/python2.7/site-packages/redis/connection.py", line 629, in read_response raise response redis.exceptions.ResponseError: Err unknown or unsupported command 'exec'

查看数据库，数据却入进去了. 10.40.0.70:9221> keys *

1. "t1"
2. "t2"

补充说明： redis-py版本:2.10.5 测试了将 r=redis.StrictRedis(host='10.40.0.70',port='9221') 的ip 端口改成本地redis的ip 端口就不会报错,因此推断客户端没有问题,想请问是不是pika不支持python客户端的批量？

之前使用redis时，内存已经30G+，所以准备改用pika。但是对数据库进行操作时，有时会用keys命令，使用keys命令时，得到如下错误：

_File "/usr/lib/python2.6/site-packages/redis/client.py", line 568, in keys

return self.execute_command('KEYS', pattern)

File "/usr/lib/python2.6/site-packages/redis/client.py", line 330, in execute_command *_options File "/usr/lib/python2.6/site-packages/redis/client.py", line 312, in _execute_command return self.parse_response(command_name, *_options) File "/usr/lib/python2.6/site-packages/redis/client.py", line 390, in parse_response response = self._parse_response(command_name, catch_errors) File "/usr/lib/python2.6/site-packages/redis/client.py", line 349, in parse_response raise ResponseError(response) redis.exceptions.ResponseError: buf is too large

已经试过修改源码中REDIS_MAX_MESSAGE的值，可是依旧报相同的错误。 也试过用scan命令进行扫库，但是效率比redis的scan差很多，不符合使用的要求

/usr/bin/ld: /home/gaoyixian/package/pika/src/pika.o: in function std::string::operator=(std::string&&)': /usr/include/c++/4.8/bits/basic_string.h:583: undefined reference tofLS::FLAGS_log_dir' collect2: error: ld returned 1 exit status make: *** [Makefile:215: pika] Error 1

这个是编译报错，

环境准备按官方的来的

Debian (Ubuntu) 安装必要的lib $ sudo apt-get install libgflags-dev libsnappy-dev $ sudo apt-get install libprotobuf-dev protobuf-compiler （这个用的自己编译的3.4.1，目录在/usr/protobuf） $ sudo apt install libgoogle-glog-dev 安装gcc4.8或者以上 $ sudo apt-get install gcc-4.8 $ sudo apt-get install g++-4.8 如果机器gcc版本低于gcc4.8，需要切换到gcc4.8或者以上，下面为切换到4.8的方法 $ sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-4.8 50 $ sudo update-alternatives --config gcc

Hi，我们在对 Pika 进行压力测试，测试过程中发现 Pika 出现概率性挂掉的情况，通过检查 core 发现调用到 malloc_consolidate 时收到了 SIGABRT 信号导致，求问产生的原因以及如何解决 ～ pika 版本: 3.0.16

core 信息

[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Core was generated by `/home/pika -c /home/pika2.conf'.
Program terminated with signal SIGABRT, Aborted.
#0  0x00007f36380f7e97 in __GI_raise (sig=<optimized out>) at ../sysdeps/unix/sysv/linux/raise.c:51
51	../sysdeps/unix/sysv/linux/raise.c: No such file or directory.
[Current thread is 1 (Thread 0x7f361dffb700 (LWP 3310))]
(gdb) bt
#0  0x00007f36380f7e97 in __GI_raise (sig=<optimized out>) at ../sysdeps/unix/sysv/linux/raise.c:51
#1  0x00007f36380f9801 in __GI_abort () at abort.c:81
#2  0x00007f3638142897 in __libc_message (action=do_abort, fmt=<optimized out>) at ../sysdeps/posix/libc_fatal.c:113
#3  0x00007f363814990a in malloc_consolidate (av=0x2) at malloc.c:4475
#4  0x000000000000002c in ?? ()
#5  0x00007f35e8b57df0 in ?? ()
#6  0xcf8fe759bac38200 in ?? ()
#7  0x00007f35b401f470 in ?? ()
#8  0x00007f35e8b57e80 in ?? ()
#9  0x000000000000471c in ?? ()
#10 0x00007f35e8b57e90 in ?? ()
#11 0x0000000000000000 in ?? ()

配置信息 :

调整了线程数以及write buffer size ，其他按照默认配置使用，无主从复制

# Pika port
port : 9221
# Thread Number
thread-num : 16
# Thread Pool Size
thread-pool-size : 16
# Sync Thread Number
sync-thread-num : 6
# Item count of sync thread queue
sync-buffer-size : 10
# Pika log path
log-path : ./log/
# Pika glog level: only INFO and ERROR
loglevel : info
# Pika db path
db-path : ./db/
# Pika write-buffer-size
write-buffer-size : 2684354560
# Pika timeout
timeout : 60
# Requirepass
requirepass :
# Masterauth
masterauth :
# Userpass
userpass :
# User Blacklist
userblacklist :
# Dump Prefix
dump-prefix :
# daemonize  [yes | no]
#daemonize : yes
# slotmigrate  [yes | no]
#slotmigrate : no
# Dump Path
dump-path : ./dump/

服务器信息：

Architecture:        x86_64
CPU op-mode(s):      32-bit, 64-bit
Byte Order:          Little Endian
CPU(s):              48
On-line CPU(s) list: 0-47
Thread(s) per core:  2
Core(s) per socket:  12
Socket(s):           2
NUMA node(s):        2
Vendor ID:           GenuineIntel
CPU family:          6
Model:               63
Model name:          Intel(R) Xeon(R) CPU E5-2680 v3 @ 2.50GHz
Stepping:            2
CPU MHz:             2494.423
BogoMIPS:            4988.84
Virtualization:      VT-x
L1d cache:           32K
L1i cache:           32K
L2 cache:            256K
L3 cache:            30720K
NUMA node0 CPU(s):   0-11,24-35
NUMA node1 CPU(s):   12-23,36-47
Flags:               fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt aes xsave avx f16c rdrand lahf_lm abm cpuid_fault epb invpcid_single pti intel_ppin tpr_shadow vnmi flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid cqm xsaveopt cqm_llc cqm_occup_llc dtherm arat pln pts

goal：

• merge pink 、slash、blackwidow into pika repository
• update rocksdb.
• support cmake

env: ubuntu 22.04.1 LTS g++ version: g++ 11.3.0 rocksdb version : v7.7.3