maria db install 및 connect 연동

maria db install

sudo apt update
sudo apt install mariadb-server
//////////////////////////////////
sudo systemctl start mariadb
sudo mysql_secure_installation
//////////////////////////////////
sudo mysql -u root -p

---

// test용 topic 생성
kafka-topics --bootstrap-server localhost:9092 --create --topic test-topic
// test 해봄
kafka-console-producer --bootstrap-server localhost:9092 --topic test-topic
kafka-console-consumer --bootstrap-server localhost:9092 --topic test-topic

---

https://www.confluent.io/hub/confluentinc/kafka-connect-jdbc

JDBC Connector (Source and Sink)

 

1. 위에서 다운로드 받는다.

2. 압축을 푼다.

3. kafka설치 경로인 ~/confluent/etc/kafka에 mv한다. (connect-standalone.properties 파일이 있는...)

4. 그리고 connect-standalone.properties를 수정

5. /home/jeongjihong/confluent/etc/kafka에 connect-mariadb-sink.properties파일을 작성

name=mariadb-sink-connector
connector.class=io.confluent.connect.jdbc.JdbcSinkConnector
tasks.max=1
topics=test-topic
connection.url=jdbc:mariadb://127.0.0.1:3306/kafka_data
connection.user=root
connection.password=11234
auto.create=false
auto.evolve=true
insert.mode=insert
table.name.format=messages
pk.mode=none

6. connect실행

connect-standalone /home/jeongjihong/confluent/etc/kafka/connect-standalone.properties /home/jeongjihong/confluent/etc/kafka/connect-mariadb-sink.properties