More stories

  • Real time CDC from Oracle to PostgreSQL using Debezium, Docker Compose with Grafana

    This Proof of Concept demonstrates Debezium’s ability to enable real-time data synchronization between Oracle and PostgreSQL databases through CDC. In addition to replicating data, the solution supports conditional data filtering. A secondary flow is also configured to support analytics and monitoring, feeding live stats into Grafana dashboards. Simple diagram- Configuration Steps Following the different files, […] More

  • Real time processing/streaming using MSK Kafka, Confluent schema registry and Spark streaming in Glue ETL for Avro records compatibility

    Background In the era of big data, businesses are increasingly relying on real-time data processing to gain insights, make informed decisions, and enhance customer experiences. The ability to process and analyze data as it flows in is crucial for staying competitive in today’s data-driven landscape. In this article, apart from exploring a robust and scalable […] More