Dev Dives: Streamline document processing with UiPath Studio Web
My sql fabric ha and sharding solutions
1. www.vmcd.org
MySQL Fabric 为 Oracle 接手 MySQL 之后推出的一颗重磅炸弹。 它以集群形式管理 MySQL 提供 HA 功能的同时可以提供数据分片的功能
(Sharding)。其中 HA 模式依赖 MySQL5.6+的 GTID 功能,数据分片可以使用多种方式如 Range,Hash 等。随着互联网的发展 MySQL 在 RDBMS
中的地位也越来越高.以目前 Oracle 的态度来说,MySQL 在将来将是一颗重要的棋子,在内部 Oracle 也在积极推动部分的 Cluster 朝 MySQL cluster
的转移。包括最新的 12c RAC Clusterware 套件也将会支持 MySQL。Fabric 的推出表明了 Oracle 推进 MySQL 向成熟化企业级应用发展的决心。
Fabric 总体架构
2. www.vmcd.org
MySQL Fabric 需要安装的组件:
●MySQL Fabric: Prerequisites
●MySQL Servers (version 5.6.10 or later)
●Backing store database server
●Application databaseservers
●Python 2.6 or 2.7
●No support for 3.x yet
●MySQL Utilities 1.4
●Available at https://dev.MySQL.com/downloads/tools/utilities
●MySQL-connect-python-1.2.2-1.e16.noarch
$MySQL --socket=/data/MySQL3334/data/MySQL.sock
Welcome to the MySQL monitor. Commands end with ; or g.
Your MySQL connection id is 57
Server version: 5.6.16-log MySQL Community Server (GPL)
Copyright (c) 2000, 2014, Oracle and/or its affiliates. All rights reserved.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
Type 'help;' or 'h' for help. Type 'c' to clear the current input statement.
MySQL>
3. www.vmcd.org
我们的环境为:
MySQL Fabric + DB : 10.0.128.115:3334 MySQL5.6.16
MySQL server : 10.0.128.116-117:3334 10.0.128.254:3334 MySQL5.6.16
先创建 Fabric 用户注意 这个用户所在的 DB 为 localhost 即 Fabric manage 所在主机的 DB:
MySQL> show grants for 'Fabric'@'127.0.0.1';
+---------------------------------------------------------------------------------------------------------------+
| Grants for Fabric@127.0.0.1 |
+---------------------------------------------------------------------------------------------------------------+
| GRANT USAGE ON *.* TO 'Fabric'@'127.0.0.1' IDENTIFIED BY PASSWORD '*14E65567ABDB5135D0CFD9A70B3032C179A49EE7' |
| GRANT ALL PRIVILEGES ON `Fabric`.* TO 'Fabric'@'127.0.0.1' |
+---------------------------------------------------------------------------------------------------------------+
2 rows in set (0.00 sec)
MySQL> show grants for 'Fabric'@'localhost';
+---------------------------------------------------------------------------------------------------------------+
| Grants for Fabric@localhost |
+---------------------------------------------------------------------------------------------------------------+
| GRANT USAGE ON *.* TO 'Fabric'@'localhost' IDENTIFIED BY PASSWORD '*14E65567ABDB5135D0CFD9A70B3032C179A49EE7' |
| GRANT ALL PRIVILEGES ON `Fabric`.* TO 'Fabric'@'localhost' |
+---------------------------------------------------------------------------------------------------------------+
2 rows in set (0.00 sec)
6. www.vmcd.org
threads = 5
user = admin
address = 10.0.128.115:32274
password = admin
[executor]
executors = 5
[Sharding]
MySQLdump_program = /usr/bin/MySQLdump
MySQLclient_program = /usr/bin/MySQL
直接初始化 Fabric 注意 执行命令会后将在 Fabric 节点 MySQL database 建立相关的 table:
MySQLFabric manage setup --param=storage.user=Fabric --param=storage.password=secret
MySQL> use Fabric;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A
9. www.vmcd.org
Fabric 的全部命令 (后面利用 add group 等命令可以添加激活 Fabric 集群)
#MySQLFabric help commands
statistics node Retrieve statistics on the Fabric node.
statistics group Retrieve statistics on Procedures.
statistics procedure Retrieve statistics on Procedures.
group activate Activate failure detector for server(s) in a group.
group description Update group's description.
group deactivate Deactivate failure detector for server(s) in a group.
group create Create a group.
group remove Remove a server from a group.
group add Add a server into group.
group health Check if any server within a group has failed and report health
information.
group lookup_servers Return information on existing server(s) in a group.
group destroy Remove a group.
group demote Demote the current master if there is one.
group promote Promote a server into master.
group lookup_groups Return information on existing group(s).
dump Fabric_nodes Return a list of Fabric servers.
dump shard_index Return information about the index for all mappings matching any of
the patterns provided.
dump Sharding_information Return all the Sharding information about the tables passed as
patterns.
dump servers Return information about servers.
10. www.vmcd.org
dump shard_tables Return information about all tables belonging to mappings matching any
of the provided patterns.
dump shard_maps Return information about all shard mappings matching any of the
provided patterns.
manage teardown Teardown Fabric Storage System.
manage stop Stop the Fabric server.
manage setup Setup Fabric Storage System.
manage ping Check whether Fabric server is running or not.
manage start Start the Fabric server.
manage logging_level Set logging level.
server set_weight Set a server's weight.
server clone Clone the objects of a given server into a destination server.
server lookup_uuid Return server's uuid.
server set_mode Set a server's mode.
server set_status Set a server's status.
role list List roles and associated permissions
user roles Change roles for a Fabric user * protocol: Protocol of the user (for
example 'xmlrpc') * roles: Comma separated
list of roles, IDs or names (see `role
list`)
user usercommand Base class for all user commands
user list List users and their roles
user add Add a new Fabric user.
user password Change password of a Fabric user.
user delete Delete a Fabric user.
threat report_error Report a server error.
11. www.vmcd.org
threat report_failure Report with certantity that a server has failed or is unreachable.
Sharding list_definitions Lists all the shard mapping definitions.
Sharding remove_definition Remove the shard mapping definition represented by the Shard Mapping
ID.
Sharding move_shard Move the shard represented by the shard_id to the destination group.
Sharding disable_shard Disable a shard.
Sharding remove_table Remove the shard mapping represented by the Shard Mapping object.
Sharding split_shard Split the shard represented by the shard_id into the destination
group.
Sharding create_definition Define a shard mapping.
Sharding add_shard Add a shard.
Sharding add_table Add a table to a shard mapping.
Sharding lookup_table Fetch the shard specification mapping for the given table
Sharding enable_shard Enable a shard.
Sharding remove_shard Remove a Shard.
Sharding list_tables Returns all the shard mappings of a particular Sharding_type.
Sharding prune_shard Given the table name prune the tables according to the defined
Sharding specification for the table.
Sharding lookup_servers Lookup a shard based on the give Sharding key.
event trigger Trigger an event.
event wait_for_procedures Wait until procedures, which are identified through their uuid in a
list and separated by comma, finish their
execution.
12. www.vmcd.org
Fabric HA 测试:
直接添加 GROUP:
MySQLFabric group create pajk_group
在 MySQL server 中 (data node) 加入 Fabric 用户:
Grant all privileges on *.* to ‘Fabric’@’%’ identified by ‘Fabric’;
在 group 中添加 node:
MySQLFabric group add pajk_group 10.0.128.254:3334
MySQLFabric group add pajk_group 10.0.128.116:3334
MySQLFabric group add pajk_group 10.0.128.117:3334
激活 GROUP 选举出 PRIAMRY 节点
MySQLFabric group promote pajk_group --slave_id=10.0.128.116:3334
MySQLFabric group activate pajk_group
13. www.vmcd.org
查看节点选举情况:
#MySQLFabric group lookup_servers pajk_group
Command :
{ success = True
return = [{'status': 'SECONDARY', 'server_uuid': 'c05867d3-f2cd-11e3-b224-5254009c6d0c', 'mode': 'READ_ONLY', 'weight': 1.0, 'address':
'10.0.128.254:3334'}, {'status': 'PRIMARY', 'server_uuid': 'e2ca75e1-d65f-11e3-b8c2-525400d3350c', 'mode': 'READ_WRITE', 'weight': 1.0, 'address':
'10.0.128.116:3334'}, {'status': 'SECONDARY', 'server_uuid': 'e90d3e07-f2cd-11e3-b225-525400991d01', 'mode': 'READ_ONLY', 'weight': 1.0, 'address':
'10.0.128.117:3334'}]
activities =
}
可以看到 已经成功建立了复制集合 注意需要使用 GTID
Kill 掉主节点 PRIMARY
[WARNING] 1403000877.240037 - FailureDetector(pajk_group) - Server (e2ca75e1-d65f-11e3-b8c2-525400d3350c) in group (pajk_group) is unreachable.
[WARNING] 1403000879.259873 - FailureDetector(pajk_group) - Server (e2ca75e1-d65f-11e3-b8c2-525400d3350c) in group (pajk_group) is unreachable.
[WARNING] 1403000881.279679 - FailureDetector(pajk_group) - Server (e2ca75e1-d65f-11e3-b8c2-525400d3350c) in group (pajk_group) is unreachable.
[WARNING] 1403000881.458935 - Executor-2 - Reported issue (FAULTY) for server (e2ca75e1-d65f-11e3-b8c2-525400d3350c).
[INFO] 1403000881.463634 - Executor-2 - Master (e2ca75e1-d65f-11e3-b8c2-525400d3350c) in group (pajk_group) has been lost.
[WARNING] 1403000882.248376 - Executor-2 - Error (<server(uuid=e90d3e07-f2cd-11e3-b225-525400991d01, address=10.0.128.117:3334, mode=READ_ONLY,
status=SECONDARY>) trying to process transactions in the relay log for candidate (Error waiting for slave to catch up. Binary log (MySQL-bin.000018, 2050964).).
[INFO] 1403000882.990106 - Executor-2 - Master has changed from e2ca75e1-d65f-11e3-b8c2-525400d3350c to e90d3e07-f2cd-11e3-b225-525400991d01.