[Draft] feat(market-data): UK/Irish equity market support (.L/.IL) - #1206
Draft
cgycorey wants to merge 12 commits into
Draft
[Draft] feat(market-data): UK/Irish equity market support (.L/.IL)#1206cgycorey wants to merge 12 commits into
cgycorey wants to merge 12 commits into
Conversation
UK equities (LSE .L, ISE .IL) previously fell through _SOURCE_PATTERNS to the tushare default, were classified as a_share, and routed down the China fallback chain — ending in _unresolved after several seconds of network attempts, indistinguishable from a dead ticker. Add UK as a first-class market with parity to Canada/US across the routing surface: - market_data.py: _SOURCE_PATTERNS maps .L/.IL -> yahoo - _market_hooks.py: _MARKET_PATTERNS -> uk_equity, currency GBP, _detect_submarket -> uk - registry.py: FALLBACK_CHAINS uk_equity: [yahoo, yfinance, local] - runner.py: _MARKET_TO_SOURCE uk_equity -> yahoo; GlobalEquity set literals include uk_equity (prevents silent CryptoEngine routing) - composite.py: dispatch uk_equity -> GlobalEquityEngine(market=uk) Verified: VOD.L / SHEL.L / HSBA.L fetch 15 real LSE bars each under source=auto (served by yfinance after Yahoo fallback); engine dispatch, currency (GBP), and submarket (uk) all resolve. 109 tests pass.
Add comprehensive UK market coverage and close a volume-unit parity gap found while testing: - test_uk_equity_support.py: 17 tests across source detection, market classification, submarket, GBP currency, fallback chain, and backtest engine routing (GlobalEquityEngine market=uk, never CryptoEngine). - test_uk_backtest_smoke.py: end-to-end engine run on in-memory LSE bars, mirroring the India/Korea/Vietnam smoke convention; verifies fills, whole-share sizing, and the zero-commission generic equity path. - yahoo/yfinance loaders: declare volume_units["uk_equity"] = "shares" (was null/undeclared, so UK volume would read as unknown unit and break the provenance contract from HKUDS#1062/HKUDS#1065). 165 related tests pass; 21 new UK tests pass.
UK Main Market equities carry Stamp Duty Reserve Tax: 0.5% of consideration charged to the buyer, rounded to the nearest penny with an exact half-penny rounding UP (FA86/S99(13)). The generic GlobalEquity path charged zero, so a UK backtest ignored a real exchange-mandated cost. Add market='uk' support to GlobalEquityEngine: - uk_stamp_tax config key (default 0.005); uk_slippage defaults to slippage_us - calc_commission charges SDRT on purchase fills only (direction > 0, including buying to cover a short); sells pay nothing — unlike HK's bilateral stamp tax - Decimal ROUND_HALF_UP for the penny rounding: float round() is banker's and miscounts exact half-pennies (1.005 -> 1.00), which the HMRC manual requires to round up (13.455 -> 13.46) - _direction param renamed to direction (was reserved; UK uses it) Exemptions (UCITS ETFs, eligible AIM shares, gilts, new issues) are documented as the caller's concern — the engine applies the statutory Main Market rate. Verified against the HMRC manual's worked examples: 13.4547 -> 13.45, 13.455 -> 13.46. 172 related tests pass.
test_all_expected_markets_present asserted the exact FALLBACK_CHAINS key set, so adding uk_equity (PR HKUDS#1206) failed CI on Python 3.11 and 3.14. Add uk_equity to the expected set and a uk chain-composition test mirroring the canada/vietnam ones.
cgycorey
marked this pull request as ready for review
August 24, 2026 16:17
cgycorey
marked this pull request as draft
August 24, 2026 16:17
…loader Yahoo serves .L (always) and .IL (GBp-quoted) equities in pence: VOD.L was ~117p -> the engine priced a £1.17 share at £117, a 100x notional error on positions and SDRT. Normalize OHLC ÷100 when the source quotes GBp, at the loader boundary so every consumer (MCP tool, backtests, indicators) sees GBP values matching code_currency/engine currency: - yahoo_client.get_chart now returns (rows, currency) from chart meta - yahoo_loader: gate .L/.IL (they were silently skipped -> yfinance always served UK), scale when meta says GBp, declare price_units - yfinance_loader: scale .L/.IL via shared suffix rule (no meta there) - base: scale_pence_to_currency + is_gbp_pence_symbol helpers; bump loader cache version v4->v5 so pre-normalization pence parquet files are never served again (HKUDS#1062 precedent) - market_data provenance: currency_conversion reports 'GBp->GBP (÷100)' for UK from the serving loader's price_units; 'none' elsewhere Verified live: VOD.L close now 1.1715 GBP (was 117.15), HSBA.L 15.18 GBP (was 1514.6); notional and SDRT follow. 245 affected tests pass. Known follow-up: Yahoo-meta EUR-quoted .IL names (e.g. Irish corporates) pass through unmodified on the yahoo path, but the yfinance fallback's suffix rule would scale them; those symbols fetch no data today.
Journals carrying VOD.L/HSBA.L/DCC.IL trades were inferred as 'other' (no .L/.IL branch), degrading shadow analysis to NaN price features and 'Other' rule labels. Add .L/.IL -> 'uk' inference, map 'uk' to uk_equity in the shadow extractor's registry key map, and label it 'UK equity' in rule translation (HKUDS#1206).
…ahoo
UK had no fundamentals surface: .L/.IL symbols were rejected by the
statements tool (suffix error), while get_fundamentals stays US-only by
contract (SEC XBRL PIT panels; Yahoo has no filed-date PIT history, so
that boundary stays honest and loud). Add the feasible parity:
- _classify_market: .L/.IL -> uk
- _fetch_yahoo_statement: income/balance/cashflow from Yahoo quoteSummary
statement histories (crumb-gated client already exists); financialData +
defaultKeyStatistics for indicators; {raw, fmt, longFmt} flattened to
plain numbers; newest-first; capped periods
- Tool description/params + suffix error mention .L/.IL
- 4 UK tests (routing, flattening, indicators, failure envelope); live
verified VOD.L income 4 periods (netIncome -4.17B), balance 4, indicators 1
Yahoo histories are annual-only; a quarter request returns the annual
periods (documented degradation, matches Eastmoney's graceful fallback).
Adversarial review found the UK branch silently returned ANNUAL periods
for period='quarter' (an explicit 'del period'), whose docstring claimed
this 'matches Eastmoney' — false: Eastmoney fetches actual quarter
reports. Yahoo quoteSummary carries incomeStatementHistoryQuarterly /
balanceSheetHistoryQuarterly / cashflowStatementHistoryQuarterly for
LSE/ISE tickers, verified live for VOD.L (4 quarterly periods; Q3'25
revenue 10.45B). Map (statement, cadence) -> module + key so a quarter
request returns real quarters; quarterly endDate is {raw, fmt} and
flattens to int like every other value.
Adversarial round-trip review exposed a real bug: base.py's close callsite passes the POSITION side (before.direction), so closing a long (direction=1) was charged 0.5% SDRT on the SALE, and covering a short (direction=-1) skipped the charge on an actual PURCHASE. The unit tests only ever fed ideal trade-side values, so the round-trip path was never exercised. Trade side is direction on opens (1=buy-long, -1=short) and -direction on closes (1=close-long is a sale, -1=cover-short is a buy). Charge SDRT only when trade_is_buy, i.e. direction>0 if is_open else direction<0. HK (bilateral, sign-independent), Canada (rate * notional) and US (zero) are unaffected. Added close-path assertions to the smoke test.
get_stock_profile sends the ticker verbatim to Yahoo, so VOD.L already fetched its full profile — but _market_for classified everything non-HK as 'us', so a UK profile was reported as market='us'. Extend the classifier (.L/.IL -> uk), update the tool description and the parameter help to advertise the supported suffixes. Verified live: VOD.L profile sections all render; label now 'uk'.
…ll ref Review caught the two user-facing doc surfaces still claiming US/HK-only while the code fixes were already in: the MCP wrapper's get_stock_profile docstring (published to MCP clients) and the yfinance skill reference (tool_get_stock_profile.md — loaded into agent context: description, ticker help, market enum us|hk|uk, market-label rule, example list). Both now advertise .L/.IL; 7 profile tests pass.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
References
Draft — UK equity support
Draft while maintainers confirm scope on #1205. Full parity: routing, classification, settlement currency, price units, engine dispatch, statutory SDRT, plus the journal/shadow and financial-statements downstream consumers.
What this adds
UK equities (LSE
.L, Irish ISE.IL) currently fall through_SOURCE_PATTERNSto thetusharedefault, get classifieda_share, and die on the China fallback chain with a misleading_unresolved(several seconds of network attempts, indistinguishable from a dead ticker). This makes UK a first-class market with full parity to Canada/US:_SOURCE_PATTERNS:.L/.IL→yahoo;FALLBACK_CHAINS:uk_equity: [yahoo, yfinance, local]_MARKET_PATTERNS→uk_equity;_detect_submarket→ukcode_currency→GBP(wasUNKNOWN:uk_equity)GlobalEquityEngine(market="uk"); never silent CryptoEngine fallthrough (2x set literals in runner, +_MARKET_TO_SOURCE)Decimal ROUND_HALF_UP; config keyuk_stamp_tax; trade-side detection is is_open-aware (opens: direction>0=buy; closes: -direction — closing a long is a sale, covering a short is a purchase, matching base.py's position-side close path); validated against HMRC manual worked examples (13.4547->13.45, 13.455->13.46)uk_equity: shares(was null/undeclared, breaking the fix(data): declare per-market volume units in market-data provenance #1065/[Bug] A-share volume unit inconsistency across fallback chain (lots vs shares, 100x) and missing unit metadata #1062 provenance contract)currency == "GBp"on the Yahoo path, suffix rule on the yfinance fallback) so every consumer sees GBP values matchingcode_currency; provenance declarescurrency_conversion: "GBp->GBP (/100)"; loader cache version bumped v4->v5 (pre-normalization parquet never served again, [Bug] A-share volume unit inconsistency across fallback chain (lots vs shares, 100x) and missing unit metadata #1062 precedent)_infer_market_from_symbol:.L/.IL→"uk"(was"other", silently degrading shadow-analysis price features to NaN); shadow_MARKET_KEY_MAP/labels mapuk→uk_equity,"UK equity"Verified
VOD.Lclose now GBP 1.1715 (was 117.15),HSBA.LGBP 15.184 (was 1514.6) — yahoo direct (fallback_used: false; the.L/.ILgate in the yahoo loader was closing out UK before, so yfinance always served)Known scope notes
Follow-ups (not included)
MARKET_SPECShas no London session; missing market fails loud, never silent)