Presented by:

64061c19a85c648aa72c9be20e34ee35

Hemendra Sharma

from Squadstack

Platform engineer who loves building and optimizing systems to keep things running smoothly

No video of the event yet, sorry!

This proposal outlines how to use PostgreSQL as a message broker, addressing memory issues encountered with AmazonMQ and observability limitations in Redis. The system design incorporates a robust database schema, efficient worker processes, and additional features for task scheduling and management. Key components include:

Database Schema: Utilizes tables for queues, tasks, task instances, and workers, with a visibility timeout mechanism to prevent duplicate processing. Worker Design: Implements concurrent task processing using "SELECT FOR UPDATE SKIP LOCKED" queries, complemented by a sidecar process for monitoring and task recovery. Metrics: Focuses on tracking total messages, worker count, consumption rate, and ingestion rate for comprehensive system observability. Additional Features: Incorporates a cron task scheduler for periodic tasks and supports ETA-based task execution.

This architecture aims to provide a scalable, observable, and reliable message brokering solution using PostgreSQL's robust features and transactional capabilities.

Date:
Duration:
45 min
Room:
Conference:
PGConf India, 2025
Language:
Track:
Application Developer
Difficulty:
Medium