Production memory leaks are uncomfortable because they mix technical uncertainty with real user impact. You rarely get a clean error message. You get rising memory, slow services, alerts, and a system that becomes less trustworthy over time.
The most useful lesson I learned is simple: do not debug from vibes. Measure first, form a small hypothesis, change one thing, and watch the result.
Start with the shape of the failure
Before looking for the exact line of code, I want to understand the pattern. Does memory grow after every request? Only during a scheduled job? Only for one tenant or payload type? Does it drop after garbage collection, or does it keep climbing?
That shape tells you where to look. A cache that never expires feels different from an event listener leak. A heavy query feels different from a retained object graph. A bad retry loop feels different from a normal traffic spike.
Observability is part of the fix
It is tempting to treat dashboards as secondary work. In practice, good dashboards shorten the distance between suspicion and confidence. When I worked on backend stability, adding the right dashboards and alerts made the system easier to reason about and easier to support.
- Track memory trends, not only current memory.
- Separate service-level symptoms from endpoint-level patterns.
- Make alerts actionable enough that people know what to inspect.
- Keep the dashboard readable for someone joining the incident late.
Small fixes beat heroic rewrites
The best production fixes are often boring: remove accidental retention, limit payload size, close a stream, clean a timer, add pagination, tune a query, or make a cache honest about its lifetime.
Boring is good. It means the change can be reviewed, tested, deployed, and measured without turning one production issue into three new ones.
The human part matters too
When systems are unstable, communication becomes part of engineering. Clear status updates, honest uncertainty, and calm prioritization help the team make better decisions.
A memory leak is a technical bug, but fixing it well is a product responsibility. Users do not care which object stayed in memory. They care that the product is reliable.