[Mysql] mysql-MGR集群配置


前提须知:mgr限制条件

1
2
3
4
5
仅支持InnoDB表,并且每张表一定要有一个主键,用于做write set的冲突检测;必须打开GTID特性,二进制日志格式必须设置为ROW,用于选主与write set
COMMIT可能会导致失败,类似于快照事务隔离级别的失败场景
目前一个MGR集群最多支持9个节点
不支持外键于save point特性,无法做全局间的约束检测与部分部分回滚
二进制日志不支持binlog event checksum

0、修改每个节点hosts文件,如下面所示(三个节点均需修改)

[root@dax-mysql-master log]# cat /etc/hosts

1
2
3
4
5
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
10.0.7.53 dax-mysql-master
10.0.7.50 dax-mysql-slave
10.0.7.51 dax-mysql-mha

1、修改三个节点的my.cnf配置文件,在[mysqld]后添加如下信息,local_address这一行配置为自己的ip地址,其他不变

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
[mysqld]
master_info_repository = "TABLE"
relay_log_info_repository = "TABLE"
transaction_write_set_extraction = "XXHASH64"
slave_parallel_workers = 2
slave_preserve_commit_order = 1
slave_parallel_type = "LOGICAL_CLOCK"
binlog_checksum = NONE
#group_replication_allow_local_disjoint_gtids_join = ON
#group_replication_auto_increment_increment = 1
loose-group_replication_group_name = "867fee38-746b-11e8-b006-000d3a800ed3"
loose-group_replication_start_on_boot = off
loose-group_replication_local_address = "dax-mysql-mha:24901"
loose-group_replication_group_seeds = "dax-mysql-master:24901,dax-mysql-slave:24901,dax-mysql-mha:24901"
loose-group_replication_bootstrap_group = off

集群参数详解:

1
2
3
4
5
6
7
#group_replication_auto_increment_increment = 1 单主模式下可以设置为1(每个节点都要设置,不设置默认仍然为7)
#loose-用于上述group_replication变量 的前缀 指示服务器在服务器启动时尚未加载组复制插件时继续启动。
#配置会 transaction_write_set_extraction 指示服务器为每个事务处理它必须收集写入集并使用XXHASH64散列算法将其编码为 散列。
#配置会 group_replication_group_name 告诉插件它正在加入或创建的组名为“aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa”。
#值 group_replication_group_name 必须是有效的UUID。在二进制日志中为组复制事件设置GTID时,此UUID在内部使用。使用SELECT UUID();生成一个UUID。
#配置 group_replication_start_on_boot 指示插件在服务器启动时不自动启动操作。这在设置组复制时很重要,因为它可以确保您可以在手动启动#插件之前配置服务器。成员配置完成后,您可以将其设置 group_replication_start_on_boot 为开启,以便在服务器引导时自动启动组复制。
#配置 group_replication_local_address 告诉插件使用网络地址127.0.0.1和端口24901与组中的其他成员进行内部通信。

添加完参数之后重启数据库(三个节点均需重启)

2、主节点1(dax-mysql-master)执行:

1
2
3
4
5
6
7
8
9
10
11
mysql> 
SET SQL_LOG_BIN=0;
CREATE USER repl@'%' IDENTIFIED BY '12345678';
GRANT REPLICATION SLAVE ON *.* TO repl@'%';
FLUSH PRIVILEGES;
SET SQL_LOG_BIN=1;
CHANGE MASTER TO MASTER_USER='repl', MASTER_PASSWORD='12345678' FOR CHANNEL 'group_replication_recovery';
#安装组复制插件
INSTALL PLUGIN group_replication SONAME 'group_replication.so';
#查看插件是否安装成功
SHOW PLUGINS;

插件安装完成之后,将配置文件注释的那两行#去掉,重启数据库,并启动组复制进程:

1
2
3
4
5
6
#启动组复制
SET GLOBAL group_replication_bootstrap_group=ON;
START GROUP_REPLICATION;
SET GLOBAL group_replication_bootstrap_group=OFF;
#查看组成员
SELECT * FROM performance_schema.replication_group_members;
2.1、创建数据,可在添加其他节点之前加,或者之后加(测试在节点之前加,添加其他节点后,会自动同步)(该步骤可选择性执行)
1
2
3
4
5
6
7
8
mysql>
CREATE DATABASE test1;
USE test;
CREATE TABLE t4 (c1 INT PRIMARY KEY, c2 TEXT NOT NULL);
INSERT INTO t4 VALUES (1, 'Luis');
检查表t1和二进制日志的内容。
SELECT * FROM t4;
SHOW BINLOG EVENTS;

3、从节点2(dax-mysql-slave)执行:

1
2
3
4
5
6
7
8
SET SQL_LOG_BIN=0;
CREATE USER repl@'%' IDENTIFIED BY '12345678';
GRANT REPLICATION SLAVE ON *.* TO repl@'%';
FLUSH PRIVILEGES;
SET SQL_LOG_BIN=1;
CHANGE MASTER TO MASTER_USER='repl', MASTER_PASSWORD='12345678' FOR CHANNEL 'group_replication_recovery';
INSTALL PLUGIN group_replication SONAME 'group_replication.so';
show plugins;

将配置文件注释的那两行#去掉,重启数据库

1
mysql>START GROUP_REPLICATION;

4、从节点3(dax-mysql-mha)执行:

1
2
3
4
5
6
7
8
SET SQL_LOG_BIN=0;
CREATE USER repl@'%' IDENTIFIED BY '12345678';
GRANT REPLICATION SLAVE ON *.* TO repl@'%';
FLUSH PRIVILEGES;
SET SQL_LOG_BIN=1;
CHANGE MASTER TO MASTER_USER='repl', MASTER_PASSWORD='12345678' FOR CHANNEL 'group_replication_recovery';
INSTALL PLUGIN group_replication SONAME 'group_replication.so';
show plugins;

将配置文件注释的那两行#去掉,重启数据库

1
mysql>START GROUP_REPLICATION;

5、安装mysql-shell(只在主节点执行便可)

下载mysql-shell

1
wget https://dev.mysql.com/get/Downloads/MySQL-Shell/mysql-shell-8.0.11-linux-glibc2.12-x86-64bit.tar.gz

MySQL Shell用来配置服务器以供InnoDB集群使用的供应脚本需要访问Python版本2.7。对于沙箱部署,在用于部署的单台机器上需要Python,生产部署需要在每个服务器实例上使用Python。

1
2
tar -zxvf mysql-shell-8.0.11-linux-glibc2.12-x86-64bit.tar.gz
mv mysql-shell-8.0.11-linux-glibc2.12-x86-64bit mysql-shell

主节点1(dax-mysql-master),连接主节点,授权repl用户集群管理权限

1
2
3
4
GRANT ALL PRIVILEGES ON mysql_innodb_cluster_metadata.* TO repl@'%' identified by '12345678' WITH GRANT OPTION;
GRANT RELOAD, SHUTDOWN, PROCESS, FILE, SUPER, REPLICATION SLAVE, REPLICATION CLIENT, CREATE USER ON *.* TO repl@'%' identified by '12345678' WITH GRANT OPTION;
GRANT SELECT ON *.* TO repl@'%' identified by '12345678' WITH GRANT OPTION;
flush privileges;

通过mysql-shell连接到主节点1配置集群:

1
2
3
4
5
6
7
8
/data/soft/mysql-shell/bin/mysqlsh --uri repl@dax-mysql-master:3306
<输入用户名
mysql-shell>dba.verbose=2
创建集群
mysql-shell>var cluster = dba.createCluster('prodCluster', {adoptFromGR: true});
查看集群状态
mysql-shell>cluster = dba.getCluster("prodCluster")
mysql-shell>cluster.status()

6、在应用端安装mysql-router

1
2
3
4
5
6
sudo bash
mkdir -p /data/soft
cd /data/soft
wget https://dev.mysql.com/get/Downloads/MySQL-Router/mysql-router-8.0.11-linux-glibc2.12-x86-64bit.tar.gz
tar -zxvf mysql-router-8.0.11-linux-glibc2.12-x86-64bit.tar.gz
mv mysql-router-8.0.11-linux-glibc2.12-x86-64bit mysql-router

配置mysqlrouter

1
/data/soft/mysql-router/bin/mysqlrouter --bootstrap repl@10.5.0.6:3306 --directory /data/soft/myrouter --user=root --conf-use-sockets --force

在/data/soft/myrouter目录下生成关于mysqlrouter的配置文件,用于记录用于连接到mgr集群
其中mysqlrouter.conf配置文件内容如下:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
[metadata_cache:prodCluster]
router_id=7
bootstrap_server_addresses=mysql://dax-mysql-slave:3306,mysql://dax-mysql-master
:3306,mysql://dax-mysql-mha:3306
user=mysql_router7_umuf7ikos8fp
metadata_cluster=prodCluster
ttl=5

[routing:prodCluster_default_rw]
bind_address=0.0.0.0
bind_port=6446
socket=/tmp/myrouter/mysql.sock
destinations=metadata-cache://prodCluster/default?role=PRIMARY
routing_strategy=round-robin
protocol=classic

[routing:prodCluster_default_ro]
bind_address=0.0.0.0
bind_port=6447
socket=/tmp/myrouter/mysqlro.sock
destinations=metadata-cache://prodCluster/default?role=SECONDARY
routing_strategy=round-robin
protocol=classic

[routing:prodCluster_default_x_rw]
bind_address=0.0.0.0
bind_port=64460
socket=/tmp/myrouter/mysqlx.sock
destinations=metadata-cache://prodCluster/default?role=PRIMARY
routing_strategy=round-robin
protocol=x

[routing:prodCluster_default_x_ro]
bind_address=0.0.0.0
bind_port=64470
socket=/tmp/myrouter/mysqlxro.sock
destinations=metadata-cache://prodCluster/default?role=SECONDARY
routing_strategy=round-robin
protocol=x

启动mysqlrouter命令:

1
2
/data/soft/myrouter/start.sh
netstat -tunlp|grep mysql

加入开机启动

1
2
3
4
echo "
rm -rf /data/soft/myrouter/mysqlrouter.pid
/data/soft/myrouter/start.sh " >> /etc/rc.local
cat /etc/rc.local

授权脚本执行权限

1
2
chmod +x /etc/rc.d/rc.local
ll /etc/rc.d/rc.local

####7、测试连接库是否有问题,并查看连接的库是主还是从
连接读写库

1
mysql -u root -h 127.0.0.1 -P 6446 -p

连接只读库

1
mysql -u root -h 127.0.0.1 -P 6447 -p

8、查看集群状态

查看组成员

1
SELECT * FROM performance_schema.replication_group_members;

查看主节点

1
SHOW STATUS LIKE 'group_replication_primary_member';

或者使用如下方法

1
2
set global show_compatibility_56 = on;
select MEMBER_HOST from performance_schema.replication_group_members where MEMBER_ID=(select VARIABLE_VALUE from information_schema.global_status where variable_name='GROUP_REPLICATION_PRIMARY_MEMBER');

点击阅读

[Mysql] 一台服务器安装多个mysql数据库程序


1、已安装第一个mysql程序的相关信息:

1
2
3
端口:3306
目录:/data/mysql/
配置文件:/etc/my.cnf

2、新建用于第二个mysql程序的目录:

1
mkdir -p /data/twomysql

将安装包解压到twomysql目录(下载地址https://dev.mysql.com/get/Downloads/MySQL-5.7/mysql-5.7.22-linux-glibc2.12-x86_64.tar.gz)

1
2
3
tar -zxvf mysql-5.7.22-linux-glibc2.12-x86_64.tar.gz -C /data/twomysql
cd /data/twomysql
mv mysql-5.7.22-linux-glibc2.12-x86_64 twomysql

创建相关数据、日志、配置文件目录:

1
2
3
4
5
6
7
8
9
10
mkdir -p /data/twomysql/twomysql/log
touch /data/twomysql/twomysql/log/mysqld.log
chown -R mysql.mysql /data/twomysql/twomysql/log
mkdir -p /data/twomysql/twomysql/data
chown -R mysql.mysql /data/twomysql/twomysql/data
mkdir -p /data/twomysql/twomysql/tmp
chown -R mysql.mysql /data/twomysql/twomysql/tmp
mkdir -p /data/twomysql/twomysql/etc
chown -R mysql.mysql /data/twomysql/twomysql/etc
cd /data/twomysql/twomysql/etc

配置文件信息如下,端口设为3307,目录设置跟上面创建的目录匹配:
vim my.cnf

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
[client]
port = 3307
socket = /data/twomysql/twomysql/tmp/mysql.sock
[mysql]
prompt="\u@db \R:\m:\s [\d]> "
no-auto-rehash
[mysqld]
user = mysql
port = 3307
basedir = /data/twomysql/twomysql
datadir = /data/twomysql/twomysql/data
socket = /data/twomysql/twomysql/tmp/mysql.sock
pid-file = /data/twomysql/twomysql/tmp/mysql.pid
character-set-server = utf8mb4
skip_name_resolve = 1
open_files_limit = 65535
back_log = 1024
max_connections = 2000
max_connect_errors = 100
net_read_timeout = 30
net_write_timeout = 60
slave_net_timeout = 90
table_open_cache = 1024
table_definition_cache = 1024
table_open_cache_instances = 64
thread_stack = 512K
external-locking = FALSE
max_allowed_packet = 32M
sort_buffer_size = 4M
join_buffer_size = 4M
thread_cache_size = 768
query_cache_size = 0
query_cache_type = 0
interactive_timeout = 600
wait_timeout = 600
tmp_table_size = 32M
max_heap_table_size = 32M
slow_query_log = 1
slow_query_log_file = /data/twomysql/twomysql/log/slow.log
log-error = /data/twomysql/twomysql/log/error.log
long_query_time = 0.1
server-id = 3306101
log-bin = /data/twomysql/twomysql/log/mysql-binlog
sync_binlog = 1
binlog_cache_size = 4M
max_binlog_cache_size = 1G
max_binlog_size = 1G
expire_logs_days = 7
gtid_mode = on
enforce_gtid_consistency = 1
log_slave_updates
binlog_format = row
relay_log_recovery = 1
relay-log = relay-log
relay-log-purge = 1
key_buffer_size = 32M
read_buffer_size = 8M
read_rnd_buffer_size = 4M
bulk_insert_buffer_size = 64M
lock_wait_timeout = 60
explicit_defaults_for_timestamp = 1
innodb_thread_concurrency = 0
innodb_sync_spin_loops = 100
innodb_spin_wait_delay = 30
transaction_isolation = REPEATABLE-READ
innodb_buffer_pool_size = 2048M
innodb_buffer_pool_instances = 8
innodb_buffer_pool_load_at_startup = 1
innodb_buffer_pool_dump_at_shutdown = 1
innodb_data_file_path = ibdata1:1G:autoextend
innodb_flush_log_at_trx_commit = 1
innodb_log_buffer_size = 32M
innodb_log_file_size = 1G
innodb_log_files_in_group = 3
innodb_max_undo_log_size = 4G
innodb_io_capacity = 4000
innodb_io_capacity_max = 8000
innodb_flush_neighbors = 0
innodb_write_io_threads = 8
innodb_read_io_threads = 8
innodb_purge_threads = 4
innodb_page_cleaners = 4
innodb_open_files = 65535
innodb_max_dirty_pages_pct = 50
innodb_flush_method = O_DIRECT
innodb_lru_scan_depth = 4000
innodb_checksum_algorithm = crc32
innodb_lock_wait_timeout = 30
innodb_rollback_on_timeout = 1
innodb_print_all_deadlocks = 1
innodb_file_per_table = 1
innodb_online_alter_log_max_size = 4G
internal_tmp_disk_storage_engine = InnoDB
innodb_stats_on_metadata = 0
innodb_status_file = 1
innodb_status_output = 0
innodb_status_output_locks = 0
performance_schema = 1
performance_schema_instrument = '%=on'
lower_case_table_names = 1
explicit_defaults_for_timestamp = off
sql_mode = STRICT_TRANS_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION
[mysqldump]
quick
max_allowed_packet = 32M
[mysql.server]
basedir=/data/twomysql/twomysql

3、初始化数据库

1
2
3
sudo mkdir /data/twomysql/twomysql/mysql-files
sudo chown mysql:mysql /data/twomysql/twomysql/mysql-files
sudo chmod 750 /data/twomysql/twomysql/mysql-files

设置无密码登录,在后面在重新创建密码:

1
2
/data/twomysql/twomysql/bin/mysqld  --defaults-file=/data/twomysql/twomysql/etc/my.cnf --initialize-insecure --user=mysql 
sudo /data/twomysql/twomysql/bin/mysql_ssl_rsa_setup

4、初始化完成之后,启动数据库:

1
/data/twomysql/twomysql/bin/mysqld_safe --defaults-file=/data/twomysql/twomysql/etc/my.cnf &


5、修改用户密码

1
/data/twomysql/twomysql/bin/mysqladmin -u root -P 3307  -S /data/twomysql/twomysql/tmp/mysql.sock password 123456

6、登录mysql,需要输入密码:

1
/data/twomysql/twomysql/bin/mysql --socket=/data/twomysql/twomysql/tmp/mysql.sock --port=3307 -p

点击阅读

[Zabbix] zabbix监控hp-switch


1、通过console口、telnet、ssh连接到switch的命令行,登录具有管理员权限的用户。使用system-view进入特权模式

1
# system-view

2、配置snmp服务,设置版本、联系人信息、设备位置、团体名

1
2
3
4
5
# snmp-agent
# snmp-agent sys-info version v2c
# snmp-agent sys-info contact Zamasu <zamasu@dbsuper.com>
# snmp-agent sys-info location Universe10 - IT Room
# snmp-agent community read GokuBlack

3、保存配置信息:

1
#save

4、保存完成之后hp交换机snmp服务已经配置完成,下面测试zabbix服务器和hp交换机之间的snmp团体名。在zabbix安装snmp测试snmp服务:

1
2
# yum install snmp(apt-get install snmp)
# snmpwalk -v2c -c GokuBlack 192.168.0.200

192.168.0.200为hp交换机的地址
下面为snmpwalk输出结果:

1
2
3
4
5
6
7
SNMPv2-MIB::sysDescr.0 = STRING: HPE V1910-48G Switch Software Version 5.20, Release 1519P03
Copyright(c) 2010-2017 Hewlett Packard Enterprise Development, L.P.
SNMPv2-MIB::sysObjectID.0 = OID: SNMPv2-SMI::enterprises.25506.11.1.85
DISMAN-EVENT-MIB::sysUpTimeInstance = Timeticks: (197804302) 22 days, 21:27:23.02
SNMPv2-MIB::sysContact.0 = STRING: Zamasu <zamasu@dbsuper.com>
SNMPv2-MIB::sysName.0 = STRING: TECH-SW01
SNMPv2-MIB::sysLocation.0 = STRING: Universe10 - IT Room

5、配置zabbix仪表板:

创建主机


下面配置snmp团体名用于zabbix连接hp交换机

根据对应的交换机型号,去zabbix官网下载对应得模板:
http://www.zabbix.org/wiki/Zabbix_Templates,点击[跳转](http://www.zabbix.org/wiki/Zabbix_Templates)

点击阅读

[Oracle] oracle 定期删除trace文件


oracle 定期删除trace文件

cat del_trace.sh

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
#!/bin/bash
#oracle 11gR2 rdbms

#声明oracle环境变量
export ORACLE_BASE=/u01/app/oracle
export ORACLE_HOME=$ORACLE_BASE/product/11.2.0.4/dbhome_1
#声明主机名
export HOST_NAME=`hostname`
#声明日期,用于删除几天前的文件
export DAYS=7

#一台服务器装有多个实例,可使用循环进行删除
for dbname in instancea instanceb
do
export DB_NAME=$dbname
export DB_UNIQUE_NAME=${DB_NAME}
export ORACLE_SID=${DB_NAME}
#当前实例为rac模式节点1,启用该变量
#export ORACLE_SID=${DB_NAME}1
#当前实例为rac模式节点2,启用该变量
#export ORACLE_SID=${DB_NAME}2

#指定存放日志的路径
DIAGNOSTIC_DEST=${ORACLE_BASE}

#如果audit_trail_dest的一个默认值不生效,会使用下面第二个默认值:
AUDIT_FILE_DEST=${ORACLE_BASE}/admin/${DB_NAME}/adump

#删除audit_trail_dest第一个默认值下的审计日志,并删除7天前的日志
find ${ORACLE_HOME}/rdbms/audit -name "${ORACLE_SID}_*.aud" -mtime +${DAYS} -exec rm -f {} \;
#如果audit_trail_dest的一个默认值不生效,查找第二个默认值下的审计日志,并删除7天前的日志:
find ${AUDIT_FILE_DEST} -name "${ORACLE_SID}_*.aud" -mtime +${DAYS} -exec rm -f {} \;
#删除指定路径alert下的文件
find ${DIAGNOSTIC_DEST}/diag/rdbms/${DB_UNIQUE_NAME}/${ORACLE_SID}/alert -name "log_[0-9]*.xml" -mtime +${DAYS} -exec rm -f {} \;
#删除指定路径trace下的文件
find ${DIAGNOSTIC_DEST}/diag/rdbms/${DB_UNIQUE_NAME}/${ORACLE_SID}/trace -name "${ORACLE_SID}_*.tr[c|m]" -mtime +${DAYS} -exec rm -f {} \;
find ${DIAGNOSTIC_DEST}/diag/rdbms/${DB_UNIQUE_NAME}/${ORACLE_SID}/trace -name "cdmp_*" -mtime +${DAYS} -exec rm -rf {} \;
done

将该脚本,添加到cron下每日定期删除

点击阅读

[Oracle] oracle-audit_file_dest和audit_trail参数详解


1、AUDIT_FILE_DEST参数:

当AUDIT_TRAIL初始化参数被设置为os, xml, or xml,extended,AUDIT_FILE_DEST指定操作系统目录,用于记录跟踪审计。
AUDIT_FILE_DEST的第一个默认值为:

1
ORACLE_BASE/admin/ORACLE_SID/adump

如果第一个值指定的目录不存在或者该值不可用,会启用第二个默认值:

1
ORACLE_HOME/rdbms/audit

这两个默认值只适用于unix系统,其他平台系统默认值不同。
在多租户cdb模式下,默认值会被追加pdb的guid,用于存储属于pdb的审计记录。
例如,pdb的guid为03E1F908EE04252CE053B280E80AAAA3,第一个默认目录是:

1
ORACLE_BASE/admin/ORACLE_SID/adump/03E1F908EE04252CE053B280E80AAAA3

关于pdb的guid可以通过V$CONTAINERS视图参看。

如果AUDIT_TRAIL参数设置为xml, or xml,extended,审计记录会以xml的格式存储。如果AUDIT_SYS_OPERATIONS参数启用,用于审计sys用户的记录,审计信息也会强制写入这个位置。
在多租户容器数据库,audit_file_dest的值设置范围是cdb。虽然审计记录是每个pdb提供的,但是每个pdb不能修改该初始化参数。

2、audit_trail参数

value值详解:
none:禁用数据库审计
os:开启数据库审计,并将所有审计记录定向到操作系统的审计跟踪。
db:开启数据库审计,并将所有审计记录定向到数据库审计跟踪(sys.aud$)
db, extended:开启数据库审计,并将所有审计记录定向到数据库审计跟踪(sys.aud$),另外,sys.aud$表中类型为clob的两列(sqlbind和sqltext会被写入数据)
xml:开启数据库审计,并将所有审计记录以xml格式写入操作系统文件
xml,extended:开启数据库审计,并打印所有的审计记录列,包括SqlText和SqlBind字段

点击阅读

[Linux] linux区分当前服务器是物理机还是虚拟机方法


1、查看系统产品名称:

dmidecode -s system-product-name
物理机:

Azure微软云服务器

aws亚马逊云服务器

阿里云服务器

使用vmware建立的虚拟机

使用kvm建立的虚拟机

2、查看scsi供应商:

cat /proc/scsi/scsi|grep Vendor
物理机:

Azure微软云服务器

aws亚马逊云服务器(看不到任何信息)

阿里云服务器

使用vmware建立的虚拟机

使用kvm建立的虚拟机

3、查看内核缓冲信息:

dmesg | grep -i virtual
物理机:

Azure微软云服务器

aws亚马逊云服务器

阿里云服务器

使用vmware建立的虚拟机

使用kvm建立的虚拟机

点击阅读

[Mysql] mysql忘记root密码,修改密码方法


1、修改my.cnf文件,在[mysqld]下添加’skip-grant-tables’参数:

cat /usr/local/mysql/etc/my.cnf

1
2
3
4
5
6
7
8
[mysqld]
!include /usr/local/mysql/etc/mysqld.cnf
port = 3306
basedir = /usr/local/mysql/
socket = /usr/local/mysql/tmp/mysql.sock
pid-file = /usr/local/mysql/var/mysql.pid
datadir = /usr/local/mysql/var/
skip-grant-tables

2、重启mysql进程:

1
/etc/init.d/mysql restart

3、进入mysql控制台,修改mysql密码:

1
2
3
4
5
6
7
8
mysql> USE mysql ; 
Database changed
mysql> UPDATE user SET Password = password ( '12345678' ) WHERE User = 'root' ;
Query OK, 1 row affected (0.06 sec)
Rows matched: 1 Changed: 1 Warnings: 0
mysql> flush privileges ;
Query OK, 0 rows affected (0.00 sec)
mysql> quit

4、注释掉my.cnf配置文件下的skip-grant-tables这行参数,重新启动mysql:

1
/etc/init.d/mysql restart

5、测试密码是否修改成功,能否正常登陆控制台:

1
mysql -u root -p12345678

点击阅读

[Mysql] ERROR 1805 (HY000): Column count of mysql.user is wrong. Expected 45, found 43. The table is probably corrupted


1、从mysql5.6 msyqldump全备出来数据,导入到mysql5.7的库,导入导出均正常,在5.7的库新增用户时,提示以下错误:

1
2
root@db 09:21:  [(none)]> grant select,update,delete,insert on *.* to 'clevergo'@'%' identified by '123123';
ERROR 1805 (HY000): Column count of mysql.user is wrong. Expected 45, found 43. The table is probably corrupted

2、造成该问题是因为把5.6的库数据,全部导入5.7造成的,因此需要使用mysql_upgrade将库升级到最新版本:

/data/mysql/bin/mysql_upgrade -uroot -p123456

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
mysql_upgrade: [Warning] Using a password on the command line interface can be insecure.
Checking if update is needed.
Checking server version.
Running queries to upgrade MySQL server.
Checking system database.
mysql.columns_priv OK
mysql.db OK
mysql.engine_cost OK
mysql.event OK
mysql.func OK
mysql.general_log OK
mysql.gtid_executed OK
mysql.help_category OK
mysql.help_keyword OK
mysql.help_relation OK
mysql.help_topic OK
mysql.innodb_index_stats OK
mysql.innodb_table_stats OK
mysql.ndb_binlog_index OK
mysql.plugin OK
mysql.proc OK
mysql.procs_priv OK
mysql.proxies_priv OK
mysql.server_cost OK
mysql.servers OK
mysql.slave_master_info OK
mysql.slave_relay_log_info OK
mysql.slave_worker_info OK
mysql.slow_log OK
mysql.tables_priv OK
mysql.time_zone OK
mysql.time_zone_leap_second OK
mysql.time_zone_name OK
mysql.time_zone_transition OK
mysql.time_zone_transition_type OK
mysql.user OK
The sys schema is already up to date (version 1.5.1).
Found 0 sys functions, but expected 22. Re-installing the sys schema.
Upgrading the sys schema.
Checking databases.
t.t_test OK
Upgrade process completed successfully.
Checking if update is needed.

3、进入mysql,再次执行创建用户:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
root@db 09:26:  [(none)]> grant select,update,delete,insert on *.* to 'clevergo'@'%' identified by '123123';
Query OK, 0 rows affected, 1 warning (0.01 sec)

root@db 09:26: [(none)]> flush privileges;
Query OK, 0 rows affected (0.01 sec)

root@db 09:26: [(none)]> select host,user from mysql.user;
+-----------+----------------+
| host | user |
+-----------+----------------+
| % | clevergo |
| localhost | mysql.session |
| localhost | mysql.sys |
| localhost | root |
+-----------+----------------+
6 rows in set (0.00 sec)

root@db 09:26: [(none)]> exit
Bye

执行成功。

点击阅读

[Mysql] 使用mysqldiff和mysqldbcompare检查数据一致性


1、官网下载mysql-utilities工具

1
wget https://cdn.mysql.com/archives/mysql-utilities/mysql-utilities-1.6.5.tar.gz

2、解压mysql-utilities工具:

1
2
3
4
5
6
7
tar -zxvf mysql-utilities-1.6.5.tar.gz
cd mysql-utilities-1.6.5/
cd scripts/
[root@dax-mysql-slave scripts]# ls
mysqlauditadmin.py mysqlbinlogpurge.py mysqldbcopy.py mysqldiff.py mysqlfrm.py mysqlmetagrep.py mysqlrpladmin.py mysqlrplshow.py mysqlserverinfo.py mysqluserclone.py
mysqlauditgrep.py mysqlbinlogrotate.py mysqldbexport.py mysqldiskusage.py mysqlgrants.py mysqlprocgrep.py mysqlrplcheck.py mysqlrplsync.py mysqlslavetrx.py
mysqlbinlogmove.py mysqldbcompare.py mysqldbimport.py mysqlfailover.py mysqlindexcheck.py mysqlreplicate.py mysqlrplms.py mysqlserverclone.py mysqluc.py

在scripts目录下存在mysqldiff和mysqldbcompare用于比对数据的脚本

3、使用mysqldbcompare需要安装connector-python依赖关系:

官网下载

1
wget https://cdn.mysql.com/archives/mysql-connector-python-2.2/mysql-connector-python-2.2.3-0.1.el7.x86_64.rpm

安装

1
rpm -ivh mysql-connector-python-2.2.3-0.1.el7.x86_64.rpm

4、安装完成之后测试mysqldbcompare能否正常使用:

./mysqldbcompare.py

1
2
3
4
Traceback (most recent call last):
File "./mysqldbcompare.py", line 28, in <module>
from mysql.utilities.common.tools import check_python_version
ImportError: No module named utilities.common.tools

将python2.7下的模块,软连接到lib64目录下:

1
ln -s /usr/lib/python2.7/site-packages/mysql/utilities /usr/lib64/python2.7/site-packages/mysql/utilities

重新测试mysqldbcompare能否正常使用:
./mysqldbcompare.py

1
2
3
Usage: mysqldbcompare --server1=user:pass@host:port:socket --server2=user:pass@host:port:socket db1:db2

mysqldbcompare: error: You must specify at least one database to compare or use the --all option to compare all databases.

提示需要指定对表的库:
./mysqldbcompare.py –server1=root:12345678@10.0.7.50:3306:/data/mysql/tmp/mysql.sock –server2=root:12345678@10.0.7.51:3306:/data/mysql/tmp/mysql.sock test:test
或者
./mysqldbcompare.py –server1=root:12345678@10.0.7.50:3306 –server2=root:12345678@10.0.7.51:3306 test:test
执行结果:

1
2
3
4
5
6
# WARNING: Using a password on the command line interface can be insecure.
# server1 on 10.0.7.50: ... connected.
# server2 on 10.0.7.51: ... connected.
# Checking databases test on server1 and test on server2
#
ERROR: The list of objects differs among database test and test.

5、准备测试数据:

server1:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
root@db 03:40:  [test1]> select * from test;
+----+
| ID |
+----+
| 1 |
| 2 |
| 3 |
| 4 |
| 5 |
| 6 |
+----+
6 rows in set (0.00 sec)

root@db 03:40: [test1]> show create table test;
+-------+-------------------------------------------------------------------------------------------------------------+
| Table | Create Table |
+-------+-------------------------------------------------------------------------------------------------------------+
| test | CREATE TABLE `test` (
`ID` int(11) NOT NULL,
PRIMARY KEY (`ID`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 |
+-------+-------------------------------------------------------------------------------------------------------------+
1 row in set (0.00 sec)

server2:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
root@db 03:37:  [test1]> select * from test;
+----+
| ID |
+----+
| 1 |
| 2 |
| 3 |
| 4 |
| 5 |
+----+

root@db 06:20: [test1]> show create table test
-> ;
+-------+-------------------------------------------------------------------------------------------------------------+
| Table | Create Table |
+-------+-------------------------------------------------------------------------------------------------------------+
| test | CREATE TABLE `test` (
`ID` int(11) NOT NULL,
PRIMARY KEY (`ID`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 |
+-------+-------------------------------------------------------------------------------------------------------------+
1 row in set (0.00 sec)

6、mysqldiff用于检测表结构的差异,如果某个对比表表结构相同,数据不同,mysqldiff并不会检测出来:

./mysqldiff.py –server1=root:12345678@10.0.7.50:3306 –server2=root:12345678@10.0.7.51:3306 test1:test1

1
2
3
4
5
6
7
# WARNING: Using a password on the command line interface can be insecure.
# server1 on 10.0.7.50: ... connected.
# server2 on 10.0.7.51: ... connected.
# Comparing `test1` to `test1` [PASS]
# Comparing `test1`.`aa` to `test1`.`aa` [PASS]
# Comparing `test1`.`test` to `test1`.`test` [PASS]
# Success. All objects are the same.

单独对比某一张表:
./mysqldiff.py –server1=root:12345678@10.0.7.50:3306 –server2=root:12345678@10.0.7.51:3306 test1.test:test1.test

1
2
3
4
5
# WARNING: Using a password on the command line interface can be insecure.
# server1 on 10.0.7.50: ... connected.
# server2 on 10.0.7.51: ... connected.
# Comparing test1.test to test1.test [PASS]
# Success. All objects are the same.

7、结构一致,数据内容不一致,使用mysqldbcompare(用于检测数据库字符集,表结构,表数据等)检测:

./mysqldbcompare.py –server1=root:12345678@10.0.7.50:3306 –server2=root:12345678@10.0.7.51:3306 test1 –changes-for=server1 –difftype=sql

1
2
3
4
5
6
7
8
9
10
11
12
# WARNING: Using a password on the command line interface can be insecure.
# server1 on 10.0.7.50: ... connected.
# server2 on 10.0.7.51: ... connected.
# Checking databases test1 on server1 and test1 on server2
#
# Defn Row Data
# Type Object Name Diff Count Check
# -------------------------------------------------------------------------
# TABLE aa pass pass -
# - Compare table checksum pass
# TABLE test pass FAIL ERROR: Row counts are not the same among `test1`.`test` and `test1`.`test`.
#

提示test1库下面的test表数据不一致。

8、创建测试数据,表结构不同,数据内容相同:

server1:

1
2
3
4
5
6
7
8
9
10
11
12
create table test1 (id int(10) primary key);
insert into test1 values (1),(2),(3),(4),(5);
select * from test1;
+----+
| id |
+----+
| 1 |
| 2 |
| 3 |
| 4 |
| 5 |
+----+

server2:

1
2
3
4
5
6
7
8
9
10
11
12
create table test1 (id bigint(20));
insert into test1 values (1),(2),(3),(4),(5);
root@db 06:28: [test1]> select * from test1;
+------+
| id |
+------+
| 1 |
| 2 |
| 3 |
| 4 |
| 5 |
+------+

9、使用mysqldiff检测:

./mysqldiff.py –server1=root:12345678@10.0.7.50:3306 –server2=root:12345678@10.0.7.51:3306 test1.test1:test1.test1

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
# WARNING: Using a password on the command line interface can be insecure.
# server1 on 10.0.7.50: ... connected.
# server2 on 10.0.7.51: ... connected.
# Comparing test1.test1 to test1.test1 [FAIL]
# Object definitions differ. (--changes-for=server1)
#

--- test1.test1
+++ test1.test1
@@ -1,4 +1,3 @@
CREATE TABLE `test1` (
- `id` int(10) NOT NULL,
- PRIMARY KEY (`id`)
+ `id` bigint(20) DEFAULT NULL
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4
# Compare failed. One or more differences found.

提示表结构存在差异。

10、使用mysqldbcompare检测:

./mysqldbcompare.py –server1=root:12345678@10.0.7.50:3306 –server2=root:12345678@10.0.7.51:3306 test1 –changes-for=server1 –difftype=sql

1
2
3
4
5
6
7
8
9
10
11
12
# WARNING: Using a password on the command line interface can be insecure.
# server1 on 10.0.7.50: ... connected.
# server2 on 10.0.7.51: ... connected.
# Checking databases test1 on server1 and test1 on server2
#
# Defn Row Data
# Type Object Name Diff Count Check
# -------------------------------------------------------------------------
# TABLE aa pass pass -
# - Compare table checksum pass
# TABLE test pass FAIL ERROR: Row counts are not the same among `test1`.`test` and `test1`.`test`.
#

当检测到异常之后会直接退出,不进行下面的比较,因此未检测到test1表的内容,加上-t(-t, –run-all-tests do not abort when a diff test fails)参数,运行测试模式,遇到异常之后仍然执行下面的比较:
./mysqldbcompare.py –server1=root:12345678@10.0.7.50:3306 –server2=root:12345678@10.0.7.51:3306 test1 –changes-for=server1 –difftype=sql -t

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
# WARNING: Using a password on the command line interface can be insecure.
# server1 on 10.0.7.50: ... connected.
# server2 on 10.0.7.51: ... connected.
# Checking databases test1 on server1 and test1 on server2
#
# Defn Row Data
# Type Object Name Diff Count Check
# -------------------------------------------------------------------------
# TABLE aa pass pass -
# - Compare table checksum pass
# TABLE test pass FAIL -
# - Compare table checksum FAIL
# - Find row differences FAIL
#
# Row counts are not the same among `test1`.`test` and `test1`.`test`.
#
# Transformation for --changes-for=server1:
#

DELETE FROM `test1`.`test` WHERE `ID` = '6';


# TABLE test1 FAIL pass -
# - Compare table checksum FAIL
# - Find row differences SKIP
#
# Transformation for --changes-for=server1:
#

ALTER TABLE `test1`.`test1`
DROP PRIMARY KEY,
CHANGE COLUMN id id bigint(20) NULL;

# The table test1 does not have an usable Index or primary key.


# Database consistency check failed.
#
# ...done

11、创建两个test库,server1比server2实例多一个gtid_test14的表:

server1:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
show tables;
+----------------+
| Tables_in_test |
+----------------+
| articles |
| gtid_test10 |
| gtid_test11 |
| gtid_test12 |
| gtid_test13 |
| gtid_test14 |
| gtid_test15 |
| gtid_test2 |
| gtid_test3 |
| gtid_test4 |
| gtid_test5 |
| gtid_test6 |
| gtid_test7 |
| gtid_test8 |
| gtid_test9 |
| ratings |
+----------------+
16 rows in set (0.00 sec)

server2:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
show tables;
+----------------+
| Tables_in_test |
+----------------+
| articles |
| gtid_test10 |
| gtid_test11 |
| gtid_test12 |
| gtid_test13 |
| gtid_test15 |
| gtid_test2 |
| gtid_test3 |
| gtid_test4 |
| gtid_test5 |
| gtid_test6 |
| gtid_test7 |
| gtid_test8 |
| gtid_test9 |
| ratings |
+----------------+
15 rows in set (0.00 sec)

12、使用mysqldiff测试:

./mysqldiff.py –server1=root:12345678@10.0.7.50:3306 –server2=root:12345678@10.0.7.51:3306 test:test

1
2
3
4
5
6
# WARNING: Using a password on the command line interface can be insecure.
# server1 on 10.0.7.50: ... connected.
# server2 on 10.0.7.51: ... connected.
# WARNING: Objects in server1.test but not in server2.test:
# TABLE: gtid_test14
# Compare failed. One or more differences found.

提示server2.test库下不存在gtid_test14表。

13、使用mysqldbcompare测试:

./mysqldbcompare.py –server1=root:12345678@10.0.7.50:3306 –server2=root:12345678@10.0.7.51:3306 test

1
2
3
4
5
6
# WARNING: Using a password on the command line interface can be insecure.
# server1 on 10.0.7.50: ... connected.
# server2 on 10.0.7.51: ... connected.
# Checking databases test on server1 and test on server2
#
ERROR: The list of objects differs among database test and test.

只会提示两个实例的test库有差异,并没有列出详细信息。在对比数据时可先使用mysqldiff工具检测两个server端的表结构是否一致,如果没有差异,再使用mysqldbcompare工具去检测表数据是否一致。

点击阅读

[Linux] 使用certbot为域名生成免费证书(apache版)


1、下载certbot

1
2
3
cd /data/soft
wget https://dl.eff.org/certbot-auto
chmod a+x certbot-auto

2、生成证书

/data/soft/certbot-auto –apache certonly

1
2
3
4
5
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Could not choose appropriate plugin: The apache plugin is not working; there may be problems with your existing configuration.
The error was: NoInstallationError('Cannot find Apache executable apachectl',)
The apache plugin is not working; there may be problems with your existing configuration.
The error was: NoInstallationError('Cannot find Apache executable apachectl',)

3、上面报错提示找不到执行路径,需要指定apache的路径

sudo env PATH=$PATH:/usr/local/apache2/bin ./certbot-auto –apache certonly

1
2
3
4
5
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Could not choose appropriate plugin: The apache plugin is not working; there may be problems with your existing configuration.
The error was: NoInstallationError('Could not find configuration root',)
The apache plugin is not working; there may be problems with your existing configuration.
The error was: NoInstallationError('Could not find configuration root',)

4、上面报错提示找不到配置目录,需要指定–apache-server-root

sudo env PATH=$PATH:/usr/local/apache2/bin ./certbot-auto –apache –apache-server-root /usr/local/apache2

1
2
3
4
5
6
7
8
9
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator apache, Installer apache
No names were found in your configuration files. Please enter in your domain
name(s) (comma and/or space separated) (Enter 'c' to cancel): www.test.com
Obtaining a new certificate
Performing the following challenges:
http-01 challenge for www.test.com
Cleaning up challenges
Unable to find a virtual host listening on port 80 which is currently needed for Certbot to prove to the CA that you control your domain. Please add a virtual host for port 80.

5、使用certbot申请申请域名免费证书,默认会访问80端口,如果80端口不存在,会报以上错误,修改httpd.conf配置文件,添加上80端口,并重启apache

1
2
3
4
5
6
7
8
Listen 80

<VirtualHost *:80>
ServerAdmin test@test.example.com
ServerName www.test.com
ServerAlias test
DocumentRoot /var/www/html
</VirtualHost>

6、重新生成证书,成功之后会在/etc/letsencrypt/live/ebank.cbibank.com目录下生成四个文件.pem文件和一个README文件

1
cert.pem  chain.pem  fullchain.pem  privkey.pem  README

7、修改conf/httpd.conf文件

1
2
#Include conf/extra/httpd-ssl.conf
#LoadModule ssl_module modules/mod_ssl.so

将这两行的#去掉

8、配置conf/extra/httpd-ssl.conf文件,修改对应的域名和证书路径:

1
2
3
4
5
6
7
8
<VirtualHost *:443>
DocumentRoot "/var/www/html"
ServerName ebank.cbibank.com
SSLEngine on
SSLCertificateFile /etc/letsencrypt/live/ebank.cbibank.com/cert.pem
SSLCertificateKeyFile /etc/letsencrypt/live/ebank.cbibank.com/privkey.pem
SSLCertificateChainFile /etc/letsencrypt/live/ebank.cbibank.com/chain.pem
</VirtualHost>

9、修改完成后重启apache:

1
/usr/local/apache2/bin/apachectl restart

重启过程报错,无法关闭apache提示以下错误:

1
httpd: Syntax error on line 434 of /usr/local/apache2/conf/httpd.conf: Cannot load /usr/local/apache2/modules/mod_ssl.so into server: /usr/local/apache2/modules/mod_ssl.so: cannot open shared object file: No such file or directory

在/usr/lib64/下面没有httpd的模块,yum安装mod_ssl:

1
yum install mod_ssl

安装完成之后在/usr/lib64/httpd/modules/下面会有mod_ssl.so

1
2
/usr/lib64/httpd/modules/mod_ssl.so
ln -s /usr/lib64/httpd/modules/mod_ssl.so /usr/local/apache2/modules/mod_ssl.so

再次尝试重启apache,报错:

1
httpd: Syntax error on line 434 of /usr/local/apache2/conf/httpd.conf: Cannot load /usr/local/apache2/modules/mod_ssl.so into server: /usr/local/apache2/modules/mod_ssl.so:undefined symbol: ap_global_mutex_create

google了一下,有说yum安装的mod_ssl与apache的安装版本不兼容的问题,因此尝试使用对应版本的tar包将模块文件拷过去:
拷贝modules目录下的ssl目录和loggers的内容到/usr/local/apache2/modules/ssl目录下、拷贝include目录下的内容到/usr/local/apache2/modules/ssl目录下,拷贝完之后,在/usr/local/apache2/modules/ssl目录下执行以下命令:

1
/usr/local/apache2/bin/apxs -a -i -c mod_ssl.c

执行完成之后再次重启apache,依旧报错:

1
httpd: Syntax error on line 434 of /usr/local/apache2/conf/httpd.conf: Cannot load /usr/local/apache2/modules/mod_ssl.so into server: /usr/local/apache2/modules/mod_ssl.so: undefined symbol: ssl_cmd_SSLPassPhraseDialog

需要指定openssl路径,执行以下命令:

1
/usr/local/apache2/bin/apxs -a -i -c -L/usr/lib/openssl/engines/lib -c *.c -lcrypto -lssl -ldl

再次重启apache

1
httpd: Syntax error on line 434 of /usr/local/apache2/conf/httpd.conf: Cannot load /usr/local/apache2/modules/mod_ssl.so into server: /usr/local/apache2/modules/mod_ssl.so:undefined symbol: ap_global_mutex_create

重启apache依旧报错undefinedsymbol:ap_global_mutex_create,没找到任何解决办法,最后只能添加-enable-ssl参数,重新编译安装apache。

点击阅读

Proudly powered by Hexo and Theme by Lap
本站访客数人次
© 2020 zeven0707's blog