Partial Subscriptions: The Batch Tail Is Where Resubscribes Fail

A watchlist of 340 symbols goes down with the venue at 02:00 and resubscribes at 02:11. Every log line reads connected. Every batch reports that it was sent. Three days later, reconciliation shows two symbols — and always the same two — with no data from 02:00 onward. They are the last two codes of the final subscription batch. Nothing errored, nothing alerted, and the client will keep reporting healthy until someone compares the list of confirmed subscriptions against the list that was requested.

This failure has a specific shape, and it is not the shape of a disconnect. The connection is fine, the resubscribe was sent, and most symbols are flowing. What failed is one batch, at the tail, with no event attached to it.

The mechanics of reconnecting — detecting the drop, backing off, recovering state — are covered in WebSocket reconnect best practices. The duplicates a reconnect produces, and the state traps that cause them, are covered in three subscription state traps. What neither covers is the failure between those two: a resubscribe that succeeds in some batches and silently loses the rest. That gap is this article.

The thesis in one line: a resubscribe is confirmed one batch at a time, and counting confirmations proves only that something came back — comparing sets is the only thing that proves everything did.

Why subscriptions arrive in batches

WebSocket subscription messages are capped per plan: the pricing tiers list a per-connection symbol cap, and a batch that exceeds it comes back as an error frame — WS_CONN_LIMIT is the code the stream returns when you are past the connection or symbol limit for your tier. That cap is not an implementation detail; it is the reason the failure mode exists.

A watchlist of a few hundred symbols cannot be sent as one message, whatever tier you are on. The client must split it, send the pieces, and collect confirmations. Each split point is a place where one piece can be accepted while the next is not — dropped by a rate limiter, rejected for capacity, lost in a frame that never got parsed as error. The probability of losing at least one batch rises with the number of batches, which means the failure gets more likely exactly as the watchlist gets bigger.

And the loss is not symmetric across the watchlist. The first batch is usually fine: it is fresh in the send loop, and any retry logic around the loop re-fires it. The tail is different. It is the last thing written, the first thing assumed, and the piece nobody checks — because by the time the loop finishes, the code has moved on to the recovery path, and the recovery path assumes subscriptions are already in place.

Three shapes of a partial resubscribe

The three shapes need different handling, and conflating them is expensive in both directions. Treating a rejection as a partial and retrying in a loop burns connection quota against a failure that will reproduce identically. Treating a partial as silent sends you hunting through storage for a gap that has not happened yet, because the symbols were never subscribed at all.

The first shape at least produces a frame. The second produces nothing — which is why it survives three days of healthy-looking logs, and why the rest of this article is about it.

Compare sets, not counts

The intuitive check is counting: "I got eleven confirmations, I sent twelve." A count is almost the right idea, and almost is what makes it dangerous — it cannot tell which codes are missing, it breaks when a venue confirms a code twice, and it degrades into a threshold ("most of them arrived") the moment someone wants the alert to stop firing.

The check that survives those pressures is set comparison:

import json
import os
import time

import websocket  # pip install websocket-client

WS_URL = "wss://api.tiqex.io/api/v1/market/stream"
API_KEY = os.environ["TIQEX_API_KEY"]          # server-side only; never in git
BATCH_SIZE = 50                                 # per-message symbol cap is plan-capped; read yours from config


def chunk(codes: list[str], size: int) -> list[list[str]]:
    return [codes[i:i + size] for i in range(0, len(codes), size)]


def resubscribe(ws, market: str, codes: list[str], timeout_s: float = 5.0) -> dict:
    """Send batches, then wait until every requested code is confirmed."""
    expected = set(codes)
    confirmed: set[str] = set()
    for batch in chunk(codes, BATCH_SIZE):
        ws.send(json.dumps({"action": "subscribe", "market": market,
                            "symbols": batch}))

    deadline = time.monotonic() + timeout_s
    while time.monotonic() < deadline and confirmed != expected:
        frame = json.loads(ws.recv())
        kind = frame.get("type")
        if kind == "subscribed":
            confirmed |= set(frame.get("symbols", []))  # confirm ack field names against the stream docs for your tier
        elif kind == "error":
            # WS_CONN_LIMIT or an entitlement code reproduces identically on retry
            return {"state": "rejected", "code": frame.get("code"),
                    "msg": frame.get("msg"), "confirmed": sorted(confirmed)}
        # tick frames arriving mid-subscribe are fine; other types are not data

    missing = expected - confirmed
    if missing:
        # Do not declare success on a partial confirmation
        return {"state": "partial", "missing": sorted(missing)}
    return {"state": "ok", "confirmed": len(confirmed)}

Three properties make this safe.

It compares sets rather than counting confirmations. Ten confirmations for twelve requested codes is a partial success, and a count-based check will accept it. The set version also survives duplicates for free: a venue that confirms a code twice still leaves the set unchanged.

It treats an error frame as terminal for the whole resubscribe, not as something to retry inside the loop. A rejection caused by WS_CONN_LIMIT or an entitlement code will reproduce identically on the next attempt, and every blind retry consumes connection quota that a real consumer will need.

It tolerates tick frames arriving mid-subscribe. Streams do not pause while you are confirming; a strict frame-order assumption here is a source of intermittent failure that shows up only under load — the worst possible time to meet it first.

Why the missing symbol sits at a batch boundary

The opening incident is diagnostic once you look at positions. The two dead symbols were the last two codes of the final batch, and that placement is not a coincidence.

If symbols failed at random, the cause would be upstream — the venue, the feed, the market. If they fail in a contiguous run at the end of the subscription list, the cause is in the batch loop: the last chunk was sent and its confirmation never arrived, or it was never sent at all. Contiguity is the fingerprint.

That observation turns reconciliation from a report into a diagnostic:

import json
import urllib.request


def locate_lost_symbols(missing: list[str], codes: list[str],
                        size: int) -> list[dict]:
    """Map each missing code to its position in the batch plan."""
    plan = {code: (i // size, i % size) for i, code in enumerate(codes)}
    out = []
    for code in missing:
        if code in plan:
            out.append({"code": code, "batch": plan[code][0],
                        "position": plan[code][1]})
    contiguous = (
        len({d["batch"] for d in out}) == 1
        and sorted(d["position"] for d in out)
        == list(range(len(out)))
    )
    note = ("contiguous run at a batch tail: failure is in the send loop"
            if contiguous else "scattered: investigate upstream or symbol-level issues")
    return out + [{"diagnosis": note}]


if __name__ == "__main__":
    import argparse

    p = argparse.ArgumentParser(description="Verify resubscribe coverage end to end")
    p.add_argument("--market", default="crypto")
    p.add_argument("--symbols", nargs="+", required=True,
                   help="e.g. BTCUSDT ETHUSDT ... (crypto codes, comma-free)")
    args = p.parse_args()

    ws = websocket.create_connection(WS_URL, header=[f"X-API-Key: {API_KEY}"])
    result = resubscribe(ws, args.market, args.symbols)
    print(result)

    if result["state"] == "partial":
        # Cross-check the hole against REST before deciding how to recover
        req = urllib.request.Request(
            "https://api.tiqex.io/api/v1/market/crypto/quote"
            f"?symbols={','.join(result['missing'])}",
            headers={"X-API-Key": API_KEY})
        with urllib.request.urlopen(req, timeout=10) as resp:
            snapshot = json.load(resp)
        print("REST envelope:", snapshot.get("succ"), snapshot.get("code"))

        report = locate_lost_symbols(result["missing"], args.symbols, BATCH_SIZE)
        print(report[-1]["diagnosis"])

Scattered misses and tail-contiguous misses have different causes and different fixes. The script above sorts them in one pass, before anyone starts reading code. Symbols missing entirely from the plan are a third case — they were never in the watchlist, and the discrepancy is in configuration, not transport.

When the hole is already past retrieval

A partial subscription discovered quickly is a resubscribe away from fixed. One discovered three days later has a second problem stacked on top: the historical window may already be outside what the REST endpoints return — history depth is plan-capped, and the tiers differ.

That changes the recovery goal. You can resubscribe and the live stream resumes, but the missing history is not recoverable by retrying anything — and retrying is exactly what most recovery loops do. The correct move is to record the window as unrecoverable, in storage, with boundaries, so the hole is auditable rather than invisible. The mechanics of backfill windows and their limits are covered in the recorder article; what matters here is the decision rule: a partial subscription older than the backfill window is a data-quality incident to be recorded, not a transport error to be retried.

This is also why partial confirmations should be loud in your own logs. A partial result with a sorted list of missing codes is one grep away from the incident report. The same information buried in a success path is what turns a two-symbol outage into a three-day one.

Prove it: a five-minute drill

A resubscribe check that has never caught a lost batch is a hypothesis. Manufacturing the failure takes minutes and converts it into a test:

1. Build a watchlist larger than your tier's per-message symbol cap, so the batch loop actually runs.
2. Inject the failure: drop the final batch before it is sent — delete the last chunk in a copy of the send loop, or have the loop skip it on a flag.
3. Run resubscribe and assert it returns partial, with the missing codes being exactly the injected ones.
4. Run locate_lost_symbols and assert the diagnosis reads as tail-contiguous.
5. Resubscribe only the missing codes, and assert the confirmed set now equals the expected set.

Step three is the one that catches implementation bugs: a count-based check returns ok for the same input. Step five closes the loop on the recovery path itself — a partial fix that resubscribes the wrong subset leaves a hole that is now harder to see, because the check already passed once.

Run the drill on a schedule. The recovery path is code, and code rots: a tier change, a rewritten watchlist loader, or a platform-side change to the symbol cap can each turn a passing test into one that passes vacuously.

FAQ

The client reconnected but some symbols get no ticks. Is that a disconnect?

Not at the transport layer. First compare the confirmed subscription set against the requested one — a partial resubscribe produces exactly this symptom with no error anywhere. If the sets match, the failure is downstream of subscription state, and a per-symbol gap check is the next step.

Where should the set comparison live — in the client or in monitoring?

Both, doing different jobs. In the client, inline, as the condition for declaring recovery complete — that is what stops the hole from opening. In monitoring, as an independent check that reads the confirmed set from storage, because a client-side check that itself fails silently is the original failure wearing a new coat.

The per-message symbol cap is a platform detail. Should I hard-code it?

No. Read it from configuration with a default, and treat the rejection codes as the authority — the cap is plan-capped, so the number is a property of your current tier, not of the platform. If the limit changes, code with the constant baked in fails in a confusing way — chunks of a different size than assumed — while a configured limit plus set comparison fails loudly and correctly, telling you exactly which codes never confirmed.

A rejection and a partial both mean some symbols are dark. Are they the same incident?

No, and the distinction matters for the on-call decision. A rejection is a capacity or entitlement problem: it reproduces identically, so the fix is configuration, not retry. A partial is a delivery problem: resubscribing only the missing codes fixes it. Sending a rejection through a partial-recovery loop produces quota pressure on top of the original fault.

How long can a partial subscription go unnoticed?

As long as your health signal tolerates it. If health means "socket open and process alive", indefinitely — that is the opening incident. If health includes subscription_confirm_ratio — confirmed codes divided by requested codes after every reconnect — the window is one reconnect. The ratio is one line to compute, and it is the difference between the failure you choose to see and the one reconciliation finds for you.

Next step

Compare your confirmed subscription set against your requested one after the next reconnect. If they match, this article cost you nothing. If they do not, you have the list of missing codes — and you have had it for less time than you think.

A free key is available on signup, so the check can be built against delayed data before any tier decision is made. No credit card is required.

Get a free API key and start building →

Endpoint behaviour, frame types (subscribed / tick / error), error codes such as WS_CONN_LIMIT, and plan entitlements cited here come from the official tiqex.io documentation — the errors page (/docs/errors), the authentication page (/docs/auth), the developer hub (/developers), and the WebSocket live quotes guide (/guides/websocket-live-quotes) — plus the pricing page, all accessed 2026-10-20. Exact ack frame field names and per-tier symbol caps should be confirmed against the stream docs and console for your tier before production use. Field and parameter availability varies by market and tier; confirm the actual response structure before relying on any check shown here. Figures illustrate engineering practice only and constitute no investment advice or forecast of future prices.