High performance distributed database for mysql

Related tags

Database heisenberg
Overview

heisenberg

High performance distributed database for mysql, define shardings by velocity&groovy scripts, can be expanded nodes flexible...

有一些问题汇总先搞到QA.md了,实在没时间

强大好用的mysql分库分表中间件,改编自cobar, 结合了cobar和TDDL的优势,让其分片策略变为分库表策略,节约了大量连接

其优点: 分库分表与应用脱离,分库表如同使用单库表一样 减少db 连接数压力 热重启配置 可水平扩容 遵守Mysql原生协议 读写分离 无语言限制,mysqlclient,c,java等都可以使用 Heisenberg服务器通过管理命令可以查看,如连接数,线程池,结点等,并可以调整 采用velocity的分库分表脚本进行自定义分库表,相当的灵活

邮箱:[email protected][email protected]

1.0 正式发版

1.0.3.2 主要解决连接上带有脏数据问题 修复其它小bug

1.0.4 增加了后端连接再次回收利用过程 日志保存时间问题

1.0.5 2016.5.3 增加NIO后端能力,暂废弃

1.0.6 2016.10.31 1.增加连接复用问题 2.提高多并发连接利用效率

1.0.7 2017.1.10 1.修复killChannel的问题 2.修复读取数据递归问题

1.0.8.1 2018.8.2

1.使用expression完美支持分片替换问题
2.支持insert批量以及select in的分片替换,以及支持嵌套问题
mysql> explain insert into `test` (id,name) values (1,'brucexx'),('2','brucexx');
No connection. Trying to reconnect...
Connection id:    1
Current database: *** NONE ***

+------------+----------------------------------------------------------+
| DATA_NODE  | SQL                                                      |
+------------+----------------------------------------------------------+
| local_node | INSERT INTO `test_01` (id, name) VALUES (1, 'brucexx')   |
| local_node | INSERT INTO `test_02` (id, name) VALUES ('2', 'brucexx') |
+------------+----------------------------------------------------------+
2 rows in set (0.31 sec)
mysql> explain select * from test where id in ('1',2,3,4,5,6);
+------------+----------------------------------------------+
| DATA_NODE  | SQL                                          |
+------------+----------------------------------------------+
| local_node | SELECT * FROM `test_00` WHERE id IN (4)      |
| local_node | SELECT * FROM `test_01` WHERE id IN ('1', 5) |
| local_node | SELECT * FROM `test_02` WHERE id IN (2, 6)   |
| local_node | SELECT * FROM `test_03` WHERE id IN (3)      |
+------------+----------------------------------------------+
4 rows in set (0.08 sec)


mysql> explain select * from (select * from test union select * from test where id =1) as t where t.id in ('1',2,3,4,5,6);
+------------+--------------------------------------------------------------------------------------------------------------------+
| DATA_NODE  | SQL                                                                                                                |
+------------+--------------------------------------------------------------------------------------------------------------------+
| local_node | SELECT * FROM ((SELECT * FROM `test_00`) UNION (SELECT * FROM `test_00` WHERE id = 1)) AS T WHERE t.id IN (4)      |
| local_node | SELECT * FROM ((SELECT * FROM `test_01`) UNION (SELECT * FROM `test_01` WHERE id = 1)) AS T WHERE t.id IN ('1', 5) |
| local_node | SELECT * FROM ((SELECT * FROM `test_02`) UNION (SELECT * FROM `test_02` WHERE id = 1)) AS T WHERE t.id IN (2, 6)   |
| local_node | SELECT * FROM ((SELECT * FROM `test_03`) UNION (SELECT * FROM `test_03` WHERE id = 1)) AS T WHERE t.id IN (3)      |
+------------+--------------------------------------------------------------------------------------------------------------------+
4 rows in set (0.02 sec)

 mysql> explain select t.name from (select * from test ) as t left join test on t.id=test.id where t.id in ('1',2,3,4,5,6);
+------------+----------------------------------------------------------------------------------------------------------------+
| DATA_NODE  | SQL                                                                                                            |
+------------+----------------------------------------------------------------------------------------------------------------+
| local_node | SELECT t.name FROM (SELECT * FROM `test_00`) AS T LEFT JOIN `test_00` ON t.id = test.id WHERE t.id IN (4)      |
| local_node | SELECT t.name FROM (SELECT * FROM `test_01`) AS T LEFT JOIN `test_01` ON t.id = test.id WHERE t.id IN ('1', 5) |
| local_node | SELECT t.name FROM (SELECT * FROM `test_02`) AS T LEFT JOIN `test_02` ON t.id = test.id WHERE t.id IN (2, 6)   |
| local_node | SELECT t.name FROM (SELECT * FROM `test_03`) AS T LEFT JOIN `test_03` ON t.id = test.id WHERE t.id IN (3)      |
+------------+----------------------------------------------------------------------------------------------------------------+
4 rows in set (0.01 sec)
 
mysql> explain select t.name from (select * from test ) as t  left join test  on t.id = test.id where t.id =1 ;
+------------+--------------------------------------------------------------------------------------------------------+
| DATA_NODE  | SQL                                                                                                    |
+------------+--------------------------------------------------------------------------------------------------------+
| local_node | SELECT t.name FROM (SELECT * FROM `test_01`) AS T LEFT JOIN `test_01` ON t.id = test.id WHERE t.id = 1 |
+------------+--------------------------------------------------------------------------------------------------------+
1 row in set (0.01 sec)
Issues
  • 两个字段拆分有问题

    两个字段拆分有问题

    按照文档配置两个字段拆分,测试的时候发现,拆分id加引号和不加引号的结果不一样,如果加引号,按照第一个拆分字段拆分,如果不加引号按照第二个拆分字段拆分。两个拆分字段均为int类型 mysql> explain insert into trans_tb(trans_id,uid) values(788,12809); +-------------+------------------------------------------------------------+ | DATA_NODE | SQL | +-------------+------------------------------------------------------------+ | transDN[80] | insert into trans_tb_80_9(trans_id,uid) values(788,12809) | +-------------+------------------------------------------------------------+ 1 row in set (0.02 sec)

    mysql> explain insert into trans_tb(trans_id,uid) values('788','12809'); +-------------+----------------------------------------------------------------+ | DATA_NODE | SQL | +-------------+----------------------------------------------------------------+ | transDN[78] | insert into trans_tb_78_8(trans_id,uid) values('788','12809') | +-------------+----------------------------------------------------------------+ 1 row in set (0.00 sec)

    opened by dingding2015 3
  • 拆分规则是否可以自定义?

    拆分规则是否可以自定义?

    看了rule.xml,拆分规则是默认拆分了100个分表,是否支持自定义拆分规则啊?

    opened by dingding2015 1
  • 怎么使用?

    怎么使用?

    我自己想学习下,我想找一个外部独立系统使用此项目为例,我怎么才能使用呢,有没有打算写一个demo,让有和我一样迷惑的人也学习下(网上很少有这个项目的介绍),谢谢。

    opened by copyshop 1
  • 1.0.1发布

    1.0.1发布

    1.分库分表 2.读写分离

    opened by brucexx 0
  • 支持自动分库分表吗?

    支持自动分库分表吗?

    自定义路由规则,当存储的量达到上限后,会自动创建库和表,还是需要人为的去创建库和相应的表并更改配置

    opened by xingfengshi 1
  • limit 用法

    limit 用法

    为什么分库分表的数据查询,不带分表键查询 192张表,limit 1会返回192条记录;limit 10 返回 1920条记录

    opened by jianyun 0
  • 启动报错

    启动报错

    common.log

    opened by xiaoyuMo 1
  • 文档在哪里啊?

    文档在哪里啊?

    只有源码中的那个几年前的文档吗?没别的资料了吗

    opened by wang2650 0
  • 我们在测试你的产品

    我们在测试你的产品

    请留个国内联系方式,gmail 不太畅通。

    opened by yanqingluo 1
  • insert语句不写明具体列名无法路由

    insert语句不写明具体列名无法路由

    如:insert into tab values(1,'xiaoming'); 这条语句会将数据插入到所有的表中。

    opened by imkodo 1
  •  执行SQL的限制

    执行SQL的限制

    当SQL为:select * from 'table'; 时 table 名字带引号,程序就报错。

    ERROR 1064 (HY000): You have an error in your SQL syntax; Error occurs around this fragment: {from 'table'}. Error cause: unexpected token for tableFactor: LITERAL_CHARS. lexer state: [email protected]{curIndex=25, ch=, token=LITERAL_CHARS, sqlLeft=, sql=select * from 'table'} 当SQL为:insert into table('id','name') values(1,haha); 字段名字带引号也报错。

    当SQL为:select * from table 此时正常 当SQL为:insert into table(id,name) values(1,‘haha’); 此时正常 heisenberg是不是限制的太多了。。。

    opened by dingding2015 1
  • 关于与Cobar的区别

    关于与Cobar的区别

    hi brucexx 从代码结构看,貌似是基于Cobar改造而来的,能否list一下主要的新增feature?

    opened by 0xmalloc 3
  • 是不是应该把baidu的repo去掉啊

    是不是应该把baidu的repo去掉啊

    开源了,就去掉baidu repo的依赖吧,默认好像无法下载依赖的jar

    opened by xuhengfei 1
Releases(1.0.8.1)
  • 1.0.8.1(Aug 3, 2018)

    1.使用expression完美支持分片替换问题 2.支持insert批量以及select in的分片替换,以及支持嵌套问题 mysql> explain insert into test (id,name) values (1,'brucexx'),('2','brucexx'); No connection. Trying to reconnect... Connection id: 1 Current database: *** NONE ***

    +------------+----------------------------------------------------------+ | DATA_NODE | SQL | +------------+----------------------------------------------------------+ | local_node | INSERT INTO test_01 (id, name) VALUES (1, 'brucexx') | | local_node | INSERT INTO test_02 (id, name) VALUES ('2', 'brucexx') | +------------+----------------------------------------------------------+ 2 rows in set (0.31 sec) mysql> explain select * from test where id in ('1',2,3,4,5,6); +------------+----------------------------------------------+ | DATA_NODE | SQL | +------------+----------------------------------------------+ | local_node | SELECT * FROM test_00 WHERE id IN (4) | | local_node | SELECT * FROM test_01 WHERE id IN ('1', 5) | | local_node | SELECT * FROM test_02 WHERE id IN (2, 6) | | local_node | SELECT * FROM test_03 WHERE id IN (3) | +------------+----------------------------------------------+ 4 rows in set (0.08 sec)

    mysql> explain select * from (select * from test union select * from test where id =1) as t where t.id in ('1',2,3,4,5,6); +------------+--------------------------------------------------------------------------------------------------------------------+ | DATA_NODE | SQL | +------------+--------------------------------------------------------------------------------------------------------------------+ | local_node | SELECT * FROM ((SELECT * FROM test_00) UNION (SELECT * FROM test_00 WHERE id = 1)) AS T WHERE t.id IN (4) | | local_node | SELECT * FROM ((SELECT * FROM test_01) UNION (SELECT * FROM test_01 WHERE id = 1)) AS T WHERE t.id IN ('1', 5) | | local_node | SELECT * FROM ((SELECT * FROM test_02) UNION (SELECT * FROM test_02 WHERE id = 1)) AS T WHERE t.id IN (2, 6) | | local_node | SELECT * FROM ((SELECT * FROM test_03) UNION (SELECT * FROM test_03 WHERE id = 1)) AS T WHERE t.id IN (3) | +------------+--------------------------------------------------------------------------------------------------------------------+ 4 rows in set (0.02 sec)

    mysql> explain select t.name from (select * from test ) as t left join test on t.id=test.id where t.id in ('1',2,3,4,5,6); +------------+----------------------------------------------------------------------------------------------------------------+ | DATA_NODE | SQL | +------------+----------------------------------------------------------------------------------------------------------------+ | local_node | SELECT t.name FROM (SELECT * FROM test_00) AS T LEFT JOIN test_00 ON t.id = test.id WHERE t.id IN (4) | | local_node | SELECT t.name FROM (SELECT * FROM test_01) AS T LEFT JOIN test_01 ON t.id = test.id WHERE t.id IN ('1', 5) | | local_node | SELECT t.name FROM (SELECT * FROM test_02) AS T LEFT JOIN test_02 ON t.id = test.id WHERE t.id IN (2, 6) | | local_node | SELECT t.name FROM (SELECT * FROM test_03) AS T LEFT JOIN test_03 ON t.id = test.id WHERE t.id IN (3) | +------------+----------------------------------------------------------------------------------------------------------------+ 4 rows in set (0.01 sec)

    mysql> explain select t.name from (select * from test ) as t left join test on t.id = test.id where t.id =1 ; +------------+--------------------------------------------------------------------------------------------------------+ | DATA_NODE | SQL | +------------+--------------------------------------------------------------------------------------------------------+ | local_node | SELECT t.name FROM (SELECT * FROM test_01) AS T LEFT JOIN test_01 ON t.id = test.id WHERE t.id = 1 | +------------+--------------------------------------------------------------------------------------------------------+ 1 row in set (0.01 sec)

    Source code(tar.gz)
    Source code(zip)
  • 1.0.8(Jul 23, 2018)

  • 1.0.7(Jan 10, 2017)

  • 1.0.6(Nov 10, 2016)

  • 1.0.5(Oct 31, 2016)

  • 1.0.3.2(Apr 21, 2016)

  • 1.0.0(Apr 12, 2014)

    功能log list

    分库分表功能,自定义velocity脚本分库表功能

    增加读写分离功能

    修复多连接事务中断BUG

    修改读写分离后dbIndex默认为1的情况

    增加日志路径,配置路径,是否调试部署脚本可配置化

    增加perf,sql,route的分析日志功能,大于300ms就到sql-perf.log中去

    优化log4j.xml日志配置

    Source code(tar.gz)
    Source code(zip)
Owner
brucexx
brucexx
a php client for distributed transaction framework dtm. 分布式事务管理器dtm的php客户端

a client for distributed transaction manager dtm dtmcli 是分布式事务管理器dtm的客户端sdk dtm分布式事务管理服务 DTM是一款跨语言的开源分布式事务管理器,优雅的解决了幂等、空补偿、悬挂等分布式事务难题。提供了简单易用、高性能、易水平扩

null 20 Jan 8, 2022
a distributed-redis-lock implementation for hyperf2.*

hyperf-redis-lock English | 中文 an easy redis-based distributed-lock implementation for hyperf 2.*。 This extension features distributed-lock includes b

lysice 4 Dec 10, 2021
A mysql database backup package for laravel

Laravel Database Backup Package This package will take backup your mysql database automatically via cron job. Installing laravel-backup The recommende

Mahedi Hasan Durjoy 20 Jun 23, 2021
Laravel Code Generator based on MySQL Database

Laravel Code Generator Do you have a well structed database and you want to make a Laravel Application on top of it. By using this tools you can gener

Tuhin Bepari 302 Jan 6, 2022
Connect and work with MySQL/MariaDB database through MySQLi in PHP. This is an introductory project, If you need a simple and straightforward example that takes you straight to the point, you can check out these examples.

First MySQLi PHP Connect and work with MySQL/MariaDB database through MySQLi in PHP. The above exercises are designed for students. This is an introdu

Max Base 3 Nov 26, 2021
Small script for importing the KvK (Dutch Chamber of Commerce) Open Data Set (CSV file) to a MySQL database.

KvK-CSV-2-SQL Small script for importing the KvK (Dutch Chamber of Commerce) Open Data Set (CSV file) to a MySQL database. Table of content KvK-CSV-2-

BASTIAAN 3 Nov 1, 2021
Pure PHP NoSQL database with no dependency. Flat file, JSON based document database.

Please give it a Star if you like the project ?? ❤️ SleekDB - A NoSQL Database made using PHP Full documentation: https://sleekdb.github.io/ SleekDB i

Kazi Mehedi Hasan 587 Jan 17, 2022
ATK Data - Data Access Framework for high-latency databases (Cloud SQL/NoSQL).

ATK Data - Data Model Abstraction for Agile Toolkit Agile Toolkit is a Low Code framework written in PHP. Agile UI implement server side rendering eng

Agile Toolkit 247 Jan 9, 2022
Independent query builders for MySQL, PostgreSQL, SQLite, and Microsoft SQL Server.

Aura.SqlQuery Provides query builders for MySQL, Postgres, SQLite, and Microsoft SQL Server. These builders are independent of any particular database

Aura for PHP 395 Dec 29, 2021
A web interface for MySQL and MariaDB

phpMyAdmin A web interface for MySQL and MariaDB. https://www.phpmyadmin.net/ Code status Download You can get the newest release at https://www.phpmy

phpMyAdmin 5.8k Jan 18, 2022
A validating SQL lexer and parser with a focus on MySQL dialect.

SQL Parser A validating SQL lexer and parser with a focus on MySQL dialect. Code status Installation Please use Composer to install: composer require

phpMyAdmin 302 Jan 11, 2022
PHP version of mysqldump cli that comes with MySQL

MySQLDump - PHP Requirements | Installing | Getting started | API | Settings | PDO Settings | TODO | License | Credits This is a php version of mysqld

diego torres 1k Jan 11, 2022
MySQL Spatial Data Extension integration with Laravel.

Laravel MySQL Spatial extension Laravel package to easily work with MySQL Spatial Data Types and MySQL Spatial Functions. Please check the documentati

Joseph Estefane 690 Jan 12, 2022
Symfony 5.2 + api platform project with ELK stack + elastic FileBeats for the log management. All running in 7 docker containers: nginx, php 8, mysql, elastic search, logstash, kibana, fileBeats.

Symfony with ELK and Elastic FileBeats Stack Prerequisites: Make sure that docker and docker-compose are installed in your machine and available for y

null 9 Dec 24, 2021
MySQL Load Data Infile Support For Laravel

Laravel Load File ?? A package to help with loading files into MySQL tables. This uses MySQL's LOAD DATA statement to load text files quickly into you

Ellis Green 46 Jan 18, 2022
A simple PHP and MySQL based internet forum that displays the messages in classical threaded view (tree structure)

my little forum my little forum is a simple PHP and MySQL based internet forum that displays the messages in classical threaded view (tree structure).

Mark Hoschek 76 Dec 19, 2021
phpMyFAQ - Open Source FAQ web application for PHP and MySQL, PostgreSQL and other databases

phpMyFAQ 3.1 What is phpMyFAQ? phpMyFAQ is a multilingual, completely database-driven FAQ-system. It supports various databases to store all data, PHP

Thorsten Rinne 522 Jan 10, 2022
A simple and extensible fixture loader for PHP 7.3+, supporting SQLite and MySQL

Flowder Flowder is a (really) simple fixture loader for PHP 7.3+, supporting SQLite and MySQL. Using Flowder in PHP 7.2 or below? Try version 1 instea

Joe Haines 6 Jan 17, 2021
A Symfony application for managing and automating regular backups of MySQL databases.

DbSaver DbSaver is an application written by Bastien LOUGHIN allowing you to make automatic daily backups (and manual backups) for your MySQL database

Bastien 19 Dec 15, 2021