Message Queue/Kafka

[CDC] kafka Connect, Debezium, JDBC Sink Connector

Confluent 사의 JDBC Sink Connector 통해 구축 Source DB : MySQL Source Connector: kafka Connect (Source Connector, Debezium) Target DB : Mysql, Oracle Sink Connector : kafka Connect (JDBC Sink Connector) 1. Docker Container 주키퍼, 카프카 docker-compose.yml version: "3" services: zookeeper: container_name: zookeeper image: wurstmeister/zookeeper ports: - "2181:2181" kafka: container_name: kafka image: wurstm..

Kafka Streams (Streaming Data Processing Programing)

build.gradle kafka-streams 추가 dependencies { implementation 'org.apache.kafka:kafka-streams' } Kafka Streams strams_log 토픽 → stream() 소스프로세서 → to() 싱크프로세서 → stream_log_copy 토픽 @SpringBootApplication public class KafkaStreamsApplication { private static String APPLICATION_NAME = "streams-filter-application"; private static String BOOTSTRAP_SERVERS = "localhost:9092"; private static String STREAM_..