Scouttlo
All ideas/devtools/A SaaS platform that monitors, analyzes, and automatically alerts on anomalous Kafka topic lag, providing recommendations to resolve bottlenecks and optimize consumption.
GitHubB2BData infradevtools

A SaaS platform that monitors, analyzes, and automatically alerts on anomalous Kafka topic lag, providing recommendations to resolve bottlenecks and optimize consumption.

Scouted Mar 16, 2026

7.0/ 10
Overall score

Turn this signal into an edge

We help you build it, validate it, and get there first.

From detected pain to an actionable plan: who pays, which MVP to launch first, how to validate it with real users, and what to measure before spending months.

Expanded analysis

See why this idea is worth it

Unlock the full write-up: what the opportunity really means, what problem exists today, how this idea attacks the pain, and the key concepts you need to know to build it.

We'll only use your email to send you the digest. Unsubscribe any time.

Score breakdown

Urgency8.0
Market size7.0
Feasibility8.0
Competition5.0
The pain

The lag in processing messages on a Kafka topic keeps growing despite active consumers, impacting system efficiency.

Who'd pay

Development and operations teams managing Kafka infrastructures in companies using messaging systems for real-time data processing.

Signal that triggered it

"The lag value of one topic has been growing monotonically for the last 4 weeks, despite always having a consumer connected."

Original post

Kafka topic lag keeps growing monotonically (snuba-generic-metrics-distributions-commit-log)

Published: Mar 16, 2026

Repository: getsentry/self-hosted Author: daishan Self-Hosted Version 26.1.0 CPU Architecture x86_64 Docker Version 29.3.0, build 5927d80 Docker Compose Version v5.1.0 Machine Specification meets Sentry minimum requirements Installation Type: upgrade from 25.11, first install 25.6 Steps to Reproduce: check all kafka topics with docker compose exec kafka kafka-consumer-groups --bootstrap-server localhost:9092 --describe --all-groups Expected Result: outside load spikes, lag value of all Kafka topics should go down and stay close to zero Actual Result: lag value of one topic has been growing monotonically for 4 weeks despite always having a consumer connected, started about 10 days after upgrade to 26.1.0 Current status shows lag of 392719 on topic snuba-generic-metrics-distributions-commit-log

Your daily digest

Liked this one? Get 5 like it every morning.

SaaS opportunities scored by AI on urgency, market size, feasibility and competition. Curated from Reddit, HackerNews and more.

Free. No spam. Unsubscribe any time.