How would you know your crypto gateway stopped watching the chain?

Every crypto payment gateway rests on one process: something that reads the chain and turns a confirmed transfer into a credit. It is the least glamorous component and the only one that has to be right, because when it stops, nothing tells you. There is no failed request, no 500, no alert. Deposits simply stop arriving.

And “no deposits in the last twenty minutes” is a perfectly normal thing to see at 4am on a quiet night. That ambiguity is the whole problem: the healthy state and the broken state produce identical evidence.

The ways a watcher dies quietly

It rarely crashes outright — a crashed process gets restarted by the orchestrator and someone notices. The dangerous failures are the ones where the process is alive and doing nothing useful.

The most common is rate limiting. Chain APIs like TronGrid have no way to ask about many addresses at once, so watching a hundred pending deposits means a hundred requests per polling cycle. Poll every fifteen seconds and you are issuing several requests a second against a key the whole platform shares. Cross the provider’s ceiling and every read starts returning 429. If those errors are caught and skipped — and they usually are, because a single address failing should not break the loop — the watcher spins forever, credits nothing, and logs politely to a file nobody is reading.

The others are quieter still: a database connection pool exhausted so every credit write fails, a tick that throws before its first request and gets swallowed by an outer catch, or a deploy where an environment variable is missing and the process starts but the poller never does.

Reduce the requests before you add the alarm

The rate limit case is worth fixing at the source, because being throttled is not an outage you can monitor your way out of. Two changes do most of the work.

The first is to stop polling every address equally. A player who is going to pay almost always pays within the first few minutes; an address that has been open for fifty minutes is far less likely to see anything in the next fifteen seconds. Coinflux checks deposits under five minutes old on every cycle, older ones every minute, and anything past half an hour every three minutes. In a measured test that was eleven requests for a fresh deposit against one for a forty-five-minute-old one across the same window — the same coverage where it matters, a fraction of the budget.

The second is a single shared ceiling. Coinflux has two jobs reading the chain, the watcher and the reconciler, and without a common budget the watcher’s burst gets both of them throttled. Every read now passes one token bucket set deliberately below the provider’s limit. Waiting a few hundred milliseconds for a token is invisible. Being throttled is not.

Then make silence measurable

You cannot alert on “no deposits”, because that is often correct. You can alert on “we have not successfully read the chain in ten minutes while deposits are waiting to be paid”, which is never correct.

That is the distinction worth building around. Coinflux records when each background job last completed a successful pass, how many consecutive failures it has seen, and what the last error was, and exposes it on a health endpoint any uptime monitor can poll. If the chain goes unread past the threshold while pending deposits exist, an alarm is raised — once an hour, not once a minute, because an alert that fires sixty times overnight teaches people to ignore it.

The qualifier matters as much as the threshold. A watcher with nothing to watch is not an incident, and paging on it is how you train yourself to dismiss the page that eventually is real.

What to ask

Ask your provider one question: how would I find out that your chain watcher stopped, before my players tell me? A status page for the API does not answer it — the API stays up and cheerfully accepts new deposit requests while nothing behind it is being credited. What you want is a health signal that covers the part doing the work, and a number attached to it you can point a monitor at.