• Welcome to the world's largest Chinese hacker forum

    Welcome to the world's largest Chinese hacker forum, our forum registration is open! You can now register for technical communication with us, this is a free and open to the world of the BBS, we founded the purpose for the study of network security, please don't release business of black/grey, or on the BBS posts, to seek help hacker if violations, we will permanently frozen your IP and account, thank you for your cooperation. Hacker attack and defense cracking or network Security

    business please click here: Creation Security  From CNHACKTEAM

Recommended Posts

整理一下以前的笔记:

项目背景:

需要使用麦克斯韦进行对关系型数据库的业务库数据采集到卡夫卡

版本选择:当时v1.29.1开始支持哈

参考文档:https://max wells-守护进程。io/快速启动/

下载地址:https://github。com/zendesk/Maxwell/releases/tag/v 1。29 .一

解压:tar-zxvf Maxwell-1 . 29 . 1 . tar . gz-C/opt/soft

关系型数据库权限:

mysql GRANT ALL on maxwell .*以麦克斯韦' @'% '标识的麦克斯韦1223’;

*上的关系型数据库授权选择、复制客户端、复制从属服务器。*到'麦克斯韦' @ ' % ';

mysql同花顺特权;

重启mysql:服务服务器重启

麦克斯韦启动:

# 过滤数据库只采集dw _测试

#要提前创建卡夫卡_话题dw _测试

/opt/soft/Maxwell-1。26 .3/bin/Maxwell-user=' Maxwell '-password=' Maxwell 1223 '-host=' 127。0 .0 .1 ' \

-制片人=卡夫卡-卡夫卡。自举。服务器=CDH 01:9092,cdh02:9092,cdh03:9092 \

-Kafka _ topic=dw _ test-filter '排除: * .*,包括: dw _测试.*'

卡夫卡监控:卡夫卡-鹰http://cdh02:8048/

卡夫卡消费:

卡夫卡-控制台-消费者。嘘\

-引导服务器cdh01:9092,cdh02:9092,cdh03:9093 \

-主题dw _测试\

从头开始

关系型数据库建表:

创建表“测试01”(

` id ` int(11)NOT NULL自动增量注释键',

` host ' varchar(45)默认空注释ip ',

` port ' int(11)默认空注释'进程id ',

主键(` id `)

)ENGINE=InnoDB DEFAULT CHARSET=utf8;

插入数据测试:

插入测试01(主机,端口)值(' 127.0.0.1 ',99)

插入测试01(主机,端口)值(' 127.0.0.1 ',133)

卡夫卡消费者消费:

{ '数据库:'dw_test ','表:'test01 ','类型: '插入',' ts':1651319606,' xid':65972 '提交' :真'数据:{'id':1,'主机' : ' 127 .0 .1,'端口' :99}}

{ '数据库:'dw_test ','表:'test01 ','类型: '插入',' ts':1651319687,' xid':67733 '提交' :真'数据:{'id':2,'主机' : ' 127 .0 .1,'端口' :133}

Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now