Checkout the slides

Abstract:

In the digital age, data flows incessantly, and for many applications, the ability to harness this continuous stream of information in real-time is the key to gaining valuable insights and staying competitive. To address this challenge, our presentation will delve into the captivating world of real-time data processing, shining a spotlight on the formidable combination of Go (Golang) and PostgreSQL. These two powerful technologies come together to create a cutting-edge data streaming platform, enabling organizations to capture, process, and distribute data changes instantaneously.

In this session, you will embark on a journey that will empower you to set up a robust real-time data streaming application. We will place special emphasis on the concepts of Change Data Capture (CDC) and pub/sub mechanisms, showcasing how Go and PostgreSQL can be seamlessly integrated to manage the flow of data efficiently and with precision. The result is a solution that facilitates real-time analytics, monitoring, and the immediate availability of actionable insights.

Our talk will provide hands-on guidance, ensuring that you have the practical knowledge needed to take on such a project. We will cover every step of the process, from setting up your development environment to simulating data sources. You will learn how to design a high-performance PostgreSQL schema that is optimized for real-time data. We will also guide you through the implementation of Change Data Capture using triggers and Go routines, offering a comprehensive view of the behind-the-scenes magic that enables real-time data processing.

But the journey doesn't end there. In addition to discussing the nuts and bolts of implementing real-time data streaming, we will also cover the creation of a user-friendly web-based dashboard for data visualization and monitoring. This dashboard will serve as a vital tool for users who want to interact with the data in a meaningful and intuitive way.

Join us in exploring the extraordinary synergy between Go and PostgreSQL. This powerful combination opens doors to a diverse range of real-time use cases. Whether your interests lie in the world of IoT, social media analysis, financial data monitoring, or you simply want to expand your knowledge of data streaming technologies, our presentation promises to offer valuable insights and practical expertise.

We will share best practices, illuminate challenges we've encountered along the way, and present strategies for scaling your solution to meet increasing demands. Security and authentication, crucial aspects of any real-time data system, will also be covered extensively. In conclusion, this talk is not just an opportunity to learn but also an invitation to embark on an exciting journey into the world of real-time data with Go and PostgreSQL.