meteor

Why Meteor publications fall back to polling

The single subtle query operator that turns a `change_stream` publication into a `polling` one — and how to spot it before users do.

2 min readUptimeClarity Team

Meteor's LiveQuery has three drivers — change_stream, oplog, and polling — and the engine quietly chooses one per publication, at runtime. The chosen driver determines, more than anything else, how your app scales under load.

This is the silent-killer category. Polling is 5–10× the Mongo load of the other two drivers. And nothing in Meteor's default output tells you which one each pub is using.

The driver ladder, in one paragraph

For each subscription, LiveQuery tries change_stream first. If the query isn't change-stream-safe, it tries oplog. If the query isn't oplog-safe either (or oplog access isn't configured), it falls back to polling — which means re-running the query every ~10 seconds. Forever. For each subscriber.

What disqualifies a query

A query gets bumped down a rung if it uses any of:

  • $where (server-side JavaScript predicates).
  • $text / full-text search.
  • Mongo.Cursor transforms with side effects.
  • Some geo operators.
  • Sorts on fields without indexes (oplog driver only).

The most common cause we see in real apps is a single $where clause added to a query that used to be just an _id lookup. The pub used to be on change streams. Now it's on polling — quietly — and the dashboards don't show it.

How to find pubs on polling

Open UptimeClarity's Reactive view and sort by Driver = polling. Look at the DDP message rate column: any pub on polling pushing > 0 messages per minute is a candidate to fix.

If you're running a different APM, you can detect it manually with this snippet (server console, dev only):

import { Meteor } from 'meteor/meteor';
 
const counts = { change_stream: 0, oplog: 0, polling: 0 };
const orig = Meteor.connection?._stream?.on;
// …a homegrown count-by-driver harness, see the Meteor docs for refs.

It's the kind of thing you write once, never trust, and then go back to your APM's view.

The fix

Three options, in increasing pain:

  1. 1

    Restructure the query to remove the disqualifying operator. A $where that compares two fields can usually be replaced with a $expr, which is oplog-safe.

  2. 2

    Split the publication. Filter what you can in Mongo and apply the JavaScript predicate in Meteor.publish itself, where it doesn't taint LiveQuery's analysis of the cursor.

  3. 3

    Cache the answer in a denormalised field that's index-friendly and oplog-safe.

The wins are usually 5–10× in throughput and a similar drop in Mongo CPU.