Single-market data processing rarely produces time-related bugs, because all the data shares one timezone and one calendar by construction. The moment you integrate several markets — US equities, China A-shares, Hong Kong, Japan, Europe — time alignment becomes a hotbed of bugs. And these bugs do not throw exceptions. They silently produce wrong results.
This article breaks down the three traps in cross-market time alignment, plus a set of rules that actually works.
Trap one: the timezone semantics of timestamps
Market data APIs return two time fields with different semantics. Mixing them shifts your data outright:
timestampMs— a millisecond epoch number in REST responses. The number itself carries no timezone, but it denotes an absolute instant (UTC-referenced).at— an RFC3339 string in streaming tick frames, with an explicit offset. Also an absolute instant.
The trouble starts at the localisation step. If your code uses "current local time" or formats according to the server's timezone, results diverge across environments — the same data lands differently in a UTC server and a UTC+8 server.
There is exactly one correct rule: store in UTC, convert to local time only at the presentation layer. Never introduce local timezones into the middle of a data pipeline.
Trap two: a trading calendar is not a business calendar
This is the most underestimated point. Trading calendars differ from civil calendars in several ways:
| Difference | Example | Consequence |
|---|---|---|
| Different weekends | Some Middle Eastern markets close Friday and Saturday | Assuming Mon–Fri misses or invents trading days |
| Independent holidays | Each country's holidays are unrelated | Using one country's holiday table builds a wrong timeline |
| Daylight saving shifts | US and European opens move by an hour in local time | Fixed local-time alignment is off by an hour twice a year |
| Half days | Early closes before holidays | Missing bars get misclassified as data loss |
Daylight saving is the sneakiest. The US open is fixed at 9:30 New York time, but converted to your local timezone it shifts when DST changes. If your system hardcodes something like "start pulling US data at 21:30 local", it silently breaks twice a year.
Trap three: halts and half days masquerading as missing data
When validating data quality across markets, the most common misjudgement is treating legitimate no-trade periods as missing data.
A typical case: a Hong Kong stock is halted for the day and the candle endpoint returns nothing for that date. If your validation rule is "trading days must have data", you get a permanent alert. If your rule is "fill gaps forward", you poison downstream calculations with fabricated history.
The correct approach models no-trade explicitly as a state rather than an absence:
- Distinguish market closed, halted, and missing — and handle each differently.
- Never forward-fill prices during a halt. Filling creates fake continuous positions in backtests.
- Persist explicit flags such as
unavailableinstead of representing them as nulls.
Market data responses include an unavailable boolean precisely to keep "not available" separate from "empty value". Validation logic should read it first.
A workable alignment scheme
Turning the principles into executable rules:
- Store everything in UTC. Convert before insert and suffix column names with
_utcto prevent misuse. - Use exchange calendars, not civil calendars. Bind each market to its own calendar covering holidays, half days, and DST rules.
- Generate one timeline per market. Forcing every market onto a shared timeline manufactures false gaps and defeats your validation rules.
- Model no-trade explicitly. Store and alert on closed, halted, and missing as three separate states.
- Localise only at presentation. Convert to the user's timezone at render time and nowhere else.
- Handle DST by rule, not by constant. Rely on the timezone database rather than hardcoded offsets.
Python: aligning time across markets
from datetime import datetime, timezone
from zoneinfo import ZoneInfo
# Exchange timezones by IANA name, so DST is handled automatically
EXCHANGE_TZ = {
"us": "America/New_York", # US equities
"hk": "Asia/Hong_Kong", # Hong Kong
"cn": "Asia/Shanghai", # China A-shares
"jp": "Asia/Tokyo", # Japan
"uk": "Europe/London", # UK
"de": "Europe/Berlin", # Germany
}
def to_utc(ts_ms: int) -> datetime:
"""REST timestampMs is an absolute instant; convert straight to UTC."""
return datetime.fromtimestamp(ts_ms / 1000, tz=timezone.utc)
def parse_rfc3339(value: str) -> datetime:
"""The streaming tick 'at' field is RFC3339; Python 3.11+ parses it directly."""
return datetime.fromisoformat(value).astimezone(timezone.utc)
def local_session_time(market: str, moment_utc: datetime) -> str:
"""Convert a UTC instant into exchange-local time for session checks."""
tz = ZoneInfo(EXCHANGE_TZ[market])
return moment_utc.astimezone(tz).strftime("%Y-%m-%d %H:%M:%S %Z")
def classify(quote: dict) -> str:
"""Separate closed / halted / missing / ok instead of calling everything missing."""
if quote.get("unavailable"):
return "unavailable" # halted or unavailable, explicit flag
if quote.get("last") is None:
return "missing" # genuinely absent, worth alerting
return "ok"
# Example: one UTC instant expressed in three exchange timezones
moment = to_utc(1789779600000)
for m in ("us", "hk", "jp"):
print(f"{m:3} {local_session_time(m, moment)}")
# us 2026-09-18 21:00:00 EDT
# hk 2026-09-19 09:00:00 HKT
# jp 2026-09-19 10:00:00 JST
The key detail is using IANA names such as America/New_York rather than fixed offsets. zoneinfo applies DST rules automatically, so a check like "US open at 9:30 Eastern" stays correct all year.
Data quality checklist
- Timestamp monotonicity. Timestamps per symbol should strictly increase; a regression points at the source or your merge logic.
- Trading day coverage. Validate day counts against the exchange calendar, not the civil calendar.
- Unavailable ratio. Track the share of
unavailableresponses; a spike usually means halts or a feed problem. - Timezone consistency. Spot-check stored timestamps to confirm no local time leaked in.
- DST boundaries. Sample a day either side of each transition to confirm session times convert correctly.
FAQ
Why can't I use Monday–Friday to determine trading days?
Because some markets observe different weekends and every market has its own holidays. Civil-calendar logic produces constant misjudgements; you need exchange calendars.
Should I forward-fill prices during a halt?
No. Forward filling creates fake continuous positions and zero-volatility stretches in backtests, inflating strategy performance. Mark halts explicitly as no-trade instead.
What breaks concretely at a DST transition?
Most often scheduled jobs drift: a hardcoded local pull time shifts by an hour, causing missing data at the open or duplicate pulls. IANA timezone names avoid this entirely.
Should multi-market data share one timeline or use one per market?
One per market. Each market has its own calendar and sessions; forcing them onto a shared timeline creates false gaps and invalidates your validation rules.
How do I distinguish a halt from missing data?
Read the unavailable field first — it is an explicit flag designed to keep "not available" distinct from "empty value". Only treat an empty value as missing, and alert on it, when that flag is absent.
Should I also store the exchange's local time?
Not for computation. A UTC timestamp plus a market identifier is sufficient — the exchange timezone is derivable from the market code. Storing a second local timestamp only creates an opportunity for the two to disagree, and once they do, it is hard to know which to trust.
Next steps
Cross-market alignment comes down to two things: UTC in storage, exchange calendars in logic. Verify calendars with the trading calendar tool, and see the multi-market integration guide for the wider design. Session times for specific markets are on pages such as Hong Kong stocks and UK stocks; the time field definitions are in the multi-market quote docs.