Skip to content

Gate changes with Alpaca and Tradier paper tests - #1127

Open
mpelteshki wants to merge 14 commits into
devfrom
test/paper-broker-deploy-gate
Open

Gate changes with Alpaca and Tradier paper tests#1127
mpelteshki wants to merge 14 commits into
devfrom
test/paper-broker-deploy-gate

Conversation

@mpelteshki

@mpelteshki mpelteshki commented Jul 13, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • require one primary CI gate for real Alpaca and Tradier paper-account boundaries on pull requests targeting dev or main
  • verify account, position, and order reads; submit/read/cancel; and a one-iteration public Strategy lifecycle for both brokers
  • retain the earlier Alpaca strategy coverage for real quotes, daily bars, SPY call/put chains, and valid option-contract resolution
  • serialize the shared paper accounts and make the aggregate LintAndTest result depend on the live gate

Safety

  • dedicated paper or sandbox credentials only
  • one-share nonmarketable AAPL limit orders
  • cancellation in normal and cleanup paths
  • broker streams and Alpaca background order threads disabled
  • no new secrets, IAM permissions, or security scopes
  • missing credentials or incomplete real broker behavior fail closed

Consolidation

This supersedes #1104. Its useful Alpaca strategy data and option-chain checks are folded into this PR. The duplicate workflow, duplicate deterministic agent/MCP runs, alternate news-credential plumbing, unrelated IBKR formatting, and reverted performance churn are intentionally omitted. Deterministic agent, MCP, permission, provider-key, and built-in news tests remain in the normal unit shards.

Validation

  • 4 live-gate tests collected locally
  • Ruff F/I checks and formatting pass for the consolidated test
  • 53 relevant Alpaca and options tests passed locally; 2 credential-dependent tests skipped; 5 apitests deselected
  • the exact pushed commit must pass the real Alpaca and Tradier paper-account CI job plus all ordinary required checks

Summary by CodeRabbit

  • Tests
    • Added a live “paper broker CI gate” for Alpaca and Tradier paper accounts, including order submit/read and cancellation verification.
    • Added end-to-end strategy run checks to confirm orders transition to cancelled reliably.
  • Chores
    • Expanded CI to run on pull requests targeting both main and dev, with concurrency-controlled live broker validation and stricter workflow success gating.
  • Bug Fixes
    • Improved market-open fallback logic to use timezone-aware timestamps.
  • Documentation
    • Documented paper broker CI gate requirements and the required Alpaca/Tradier CI environment variables.
  • Backtests
    • Refreshed the BackdoorButterfly0DTESmartLimit backtest baseline results.

@coderabbitai

coderabbitai Bot commented Jul 13, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 2babd9e4-d608-4a12-a996-7f88af251490

📥 Commits

Reviewing files that changed from the base of the PR and between 21a685e and 460e724.

📒 Files selected for processing (1)
  • tests/test_live_broker_gate.py
🚧 Files skipped from review as they are similar to previous changes (1)
  • tests/test_live_broker_gate.py
📜 Recent review details
⏰ Context from checks skipped due to timeout. (2)
  • GitHub Check: build
  • GitHub Check: Lint

📝 Walkthrough

Walkthrough

Adds a paper-broker integration gate for Alpaca and Tradier, wires it into pull-request CI, documents credentials and execution, updates timezone-aware market-open handling and lazy imports, and refreshes one backtest baseline.

Changes

Paper broker CI gate

Layer / File(s) Summary
Paper broker integration checks
tests/test_live_broker_gate.py
Adds Alpaca and Tradier paper account, order cancellation, and real strategy lifecycle checks with cleanup and cancellation polling.
CI workflow enforcement
.github/workflows/cicd.yaml
Runs the gate for main and dev pull requests, upgrades actions, disables checkout credentials, serializes gate runs, and makes the aggregate job require success.
Paper broker gate documentation
docs/PAPER_BROKER_CI.md, docsrc/environment_variables.rst
Documents required credentials, tested operations, safety constraints, cleanup behavior, and local execution.

Runtime compatibility updates

Layer / File(s) Summary
Alpaca timezone behavior
lumibot/brokers/alpaca.py, tests/test_alpaca.py
Uses a timezone-aware current datetime for market-open checks and updates fallback tests accordingly.
Lazy dependency loading
lumibot/data_sources/interactive_brokers_rest_data.py, lumibot/credentials.py
Defers JSON, Decimal, and gateway imports and clarifies lazy export cleanup comments.
Acceptance baseline refresh
tests/backtest/acceptance_backtests_baselines.json
Updates the BackdoorButterfly0DTESmartLimit baseline identifier, note, and performance metrics.

Estimated code review effort: 4 (Complex) | ~45 minutes

Sequence Diagram(s)

sequenceDiagram
  participant GitHubActions
  participant LiveBrokerGate
  participant AlpacaTradier
  participant Trader
  GitHubActions->>LiveBrokerGate: run broker gate
  LiveBrokerGate->>AlpacaTradier: read accounts, positions, and orders
  LiveBrokerGate->>AlpacaTradier: submit and cancel paper order
  LiveBrokerGate->>Trader: run one strategy iteration
  Trader->>AlpacaTradier: submit strategy order
  AlpacaTradier-->>LiveBrokerGate: report cancelled status
Loading

Suggested reviewers: grzesir

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 75.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly matches the main change: adding a required Alpaca and Tradier paper-test CI gate.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch test/paper-broker-deploy-gate

Warning

There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure.

🔧 Pylint (4.0.6)
tests/test_live_broker_gate.py

************* Module pylintrc
pylintrc:1:0: F0011: error while parsing the configuration: File contains no section headers.
file: 'pylintrc', line: 1
'known-third-party=lumibot' (config-parse-error)
[
{
"type": "convention",
"module": "tests.test_live_broker_gate",
"obj": "",
"line": 70,
"column": 0,
"endLine": null,
"endColumn": null,
"path": "tests/test_live_broker_gate.py",
"symbol": "line-too-long",
"message": "Line too long (109/100)",
"message-id": "C0301"
},
{
"type": "convention",
"module": "tests.test_live_broker_gate",
"obj": "",
"line": 90,
"column": 0,
"endLine": null,
"endColumn": null,
"path": "tests/test_live_broker_gate.py",
"symbol": "line-too-long",
"message": "Line too long (103/100)",
"message-id": "C0301"
},
{
"type": "convention",
"module": "tests.test_liv

... [truncated 16144 characters] ...

te",
"obj": "test_alpaca_paper_strategy_run_submits_and_cancels",
"line": 266,
"column": 0,
"endLine": 266,
"endColumn": 54,
"path": "tests/test_live_broker_gate.py",
"symbol": "missing-function-docstring",
"message": "Missing function or method docstring",
"message-id": "C0116"
},
{
"type": "convention",
"module": "tests.test_live_broker_gate",
"obj": "test_tradier_paper_strategy_run_submits_and_cancels",
"line": 288,
"column": 0,
"endLine": 288,
"endColumn": 55,
"path": "tests/test_live_broker_gate.py",
"symbol": "missing-function-docstring",
"message": "Missing function or method docstring",
"message-id": "C0116"
}
]


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
.github/workflows/cicd.yaml (1)

57-57: 🔒 Security & Privacy | 🔵 Trivial | ⚡ Quick win

Set persist-credentials: false on the remaining checkout steps. The live-broker-gate checkout already opts out, but the lint, unit-tests, and backtest-tests checkouts persist the GITHUB_TOKEN into the workspace git config unnecessarily (none of these jobs push). zizmor flags this as artipacked.

  • .github/workflows/cicd.yaml#L57-L57: add with: { persist-credentials: false } to the lint job checkout.
  • .github/workflows/cicd.yaml#L102-L102: add with: { persist-credentials: false } to the unit-tests job checkout.
  • .github/workflows/cicd.yaml#L192-L192: add with: { persist-credentials: false } to the backtest-tests job checkout.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.github/workflows/cicd.yaml at line 57, Update the checkout steps in the
lint, unit-tests, and backtest-tests jobs at .github/workflows/cicd.yaml lines
57-57, 102-102, and 192-192 to set persist-credentials: false under each
action's with configuration; apply the same change at all three sites.

Source: Linters/SAST tools

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In @.github/workflows/cicd.yaml:
- Line 57: Update the checkout steps in the lint, unit-tests, and backtest-tests
jobs at .github/workflows/cicd.yaml lines 57-57, 102-102, and 192-192 to set
persist-credentials: false under each action's with configuration; apply the
same change at all three sites.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 7b6d6120-91ed-424b-9352-6b1b5c6840b6

📥 Commits

Reviewing files that changed from the base of the PR and between b618afa and bf22bf0.

📒 Files selected for processing (2)
  • .github/workflows/cicd.yaml
  • tests/test_live_broker_gate.py

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
lumibot/credentials.py (1)

461-475: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Sync docs for the new IBKR REST env vars.

RUNNING_ON_SERVER, IB_GATEWAY_PORT, IB_GATEWAY_INSTANCE_ID, IB_USE_PAPER_ACCOUNT, IBEAM_DOCKER_TAG, IB_AUTH_TIMEOUT, IB_AUTH_POLL_INTERVAL, IB_REQUEST_TIMEOUT, and IB_VERIFY_SSL are newly wired into INTERACTIVE_BROKERS_REST_CONFIG. Please confirm public/engineering docs for Interactive Brokers REST setup mention these. Also note RUNNING_ON_SERVER breaks the IB_-prefix convention used by its siblings — worth double-checking that's intentional.

As per coding guidelines, "If an environment variable is introduced or changed, make sure both engineering and public documentation stay synchronized with the new behavior."

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@lumibot/credentials.py` around lines 461 - 475, Update the engineering and
public Interactive Brokers REST setup documentation to describe all environment
variables wired through INTERACTIVE_BROKERS_REST_CONFIG, including their purpose
and defaults where applicable. Also verify whether RUNNING_ON_SERVER
intentionally lacks the IB_ prefix; if not, align the configuration and
documentation with the established naming convention.

Source: Coding guidelines

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@lumibot/credentials.py`:
- Around line 461-475: Update the engineering and public Interactive Brokers
REST setup documentation to describe all environment variables wired through
INTERACTIVE_BROKERS_REST_CONFIG, including their purpose and defaults where
applicable. Also verify whether RUNNING_ON_SERVER intentionally lacks the IB_
prefix; if not, align the configuration and documentation with the established
naming convention.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: d2a4c85d-0631-4319-801b-e372c66f642b

📥 Commits

Reviewing files that changed from the base of the PR and between bf22bf0 and d84db4f.

📒 Files selected for processing (6)
  • .github/workflows/cicd.yaml
  • lumibot/brokers/alpaca.py
  • lumibot/credentials.py
  • lumibot/data_sources/interactive_brokers_rest_data.py
  • tests/backtest/acceptance_backtests_baselines.json
  • tests/test_alpaca.py
🚧 Files skipped from review as they are similar to previous changes (1)
  • .github/workflows/cicd.yaml
📜 Review details
⏰ Context from checks skipped due to timeout. (1)
  • GitHub Check: Lint
🧰 Additional context used
📓 Path-based instructions (6)
**/*.{py,md,rst,txt,yml,yaml,json,ini,env}

📄 CodeRabbit inference engine (CLAUDE.md)

Never fabricate, synthesize, forward-fill, interpolate, or default-fill missing market data in backtests; return empty / explicit absence instead, and remove any code that returns fake bars as real data.

Files:

  • lumibot/credentials.py
  • lumibot/brokers/alpaca.py
  • tests/test_alpaca.py
  • tests/backtest/acceptance_backtests_baselines.json
  • lumibot/data_sources/interactive_brokers_rest_data.py
**/*.{py,md,rst}

📄 CodeRabbit inference engine (CLAUDE.md)

If an environment variable is introduced or changed, make sure both engineering and public documentation stay synchronized with the new behavior.

Files:

  • lumibot/credentials.py
  • lumibot/brokers/alpaca.py
  • tests/test_alpaca.py
  • lumibot/data_sources/interactive_brokers_rest_data.py
**

⚙️ CodeRabbit configuration file

**: Review every pull request as if LumiBot is a public open-source trading framework.
Prioritize real bugs, security/privacy issues, user-facing regressions, and release risk.

Always check changed lines for:

  • hardcoded credentials, API keys, tokens, account emails, private URLs, private hostnames, local credential paths, or personal filesystem paths;
  • accidental leakage of BotSpot, Lumiwealth, customer, broker, paid-vendor, CI, or maintainer-only operational details into public code/docs/tests;
  • code that logs, prints, persists, screenshots, or commits secrets or customer/broker data;
  • test fixtures or docs that look fake but could be copied into real usage as credentials or private endpoints;
  • changes that weaken authentication, authorization, data-source safety, broker/order safety, or CI/release gates.

If a finding depends on repository context, explain the specific source file and invariant instead of giving generic advice.

Files:

  • lumibot/credentials.py
  • lumibot/brokers/alpaca.py
  • tests/test_alpaca.py
  • tests/backtest/acceptance_backtests_baselines.json
  • lumibot/data_sources/interactive_brokers_rest_data.py
lumibot/**

⚙️ CodeRabbit configuration file

lumibot/**: Review runtime/library changes for trading safety, broker/account isolation, data
correctness, secret handling, and public/private boundary violations. Flag hardcoded
BotSpot-specific behavior unless it is clearly provider-generic and appropriate for
open-source LumiBot.

Files:

  • lumibot/credentials.py
  • lumibot/brokers/alpaca.py
  • lumibot/data_sources/interactive_brokers_rest_data.py
tests/**/*.py

📄 CodeRabbit inference engine (.github/copilot-instructions.md)

tests/**/*.py: Add unit tests for any new functionality
Ensure high level of test coverage using pytest with coverage reporting
Ensure all tests are well-documented and follow best practices

tests/**/*.py: Treat any test whose earliest commit date is before 2025-06-01 as LEGACY. For LEGACY tests, fix the code, not the test. Only change a LEGACY test when you can clearly justify that the old expectation was incorrect or behavior was intentionally changed for correctness, and document it in the test file.
Treat any test whose earliest commit date is before 2025-01-01 as FROZEN LEGACY (effectively 'do not change'). Only change a FROZEN LEGACY test in exceptional cases and include a clear write-up in the test and PR.
If you change any expected values or assertions in a test, add a short note near the change explaining why (what changed, and why the new expectation is correct). Prefer making the test more robust (less brittle) over updating magic numbers.

Files:

  • tests/test_alpaca.py
tests/**

📄 CodeRabbit inference engine (AGENTS.md)

For tests under tests/, older tests have higher authority than newer ones; when they conflict, fix the code first unless the newer test clearly represents new functionality.

Files:

  • tests/test_alpaca.py
  • tests/backtest/acceptance_backtests_baselines.json

⚙️ CodeRabbit configuration file

tests/**: Tests are public. Flag hardcoded real-looking credentials, account emails, personal
paths, private endpoints, and fixtures that could accidentally exercise production or
paid-vendor resources without explicit opt-in.

Files:

  • tests/test_alpaca.py
  • tests/backtest/acceptance_backtests_baselines.json
🔇 Additional comments (9)
tests/backtest/acceptance_backtests_baselines.json (1)

127-141: LGTM!

lumibot/brokers/alpaca.py (1)

580-582: LGTM!

tests/test_alpaca.py (2)

12-12: LGTM!

Also applies to: 105-105


117-140: 🩺 Stability & Availability

No issue: the calendar fallback uses lumibot.brokers.broker.datetime, not the patched Alpaca module.
_is_market_open_from_initialized_calendar() is defined in lumibot.brokers.broker and calls datetime.now(timezone.utc), so the SimpleNamespace(datetime=FixedDatetime) patch on lumibot.brokers.alpaca.datetime does not trigger the tz is not None assert.

			> Likely an incorrect or invalid review comment.
lumibot/data_sources/interactive_brokers_rest_data.py (3)

13-20: LGTM!


58-91: LGTM!


115-131: 🩺 Stability & Availability

No module-level dependency here The repo doesn’t import or monkeypatch InteractiveBrokersRESTData.IBeamGateway, ExternalIbkrGateway, DEFAULT_IBEAM_HOST_PORT, or DEFAULT_IBEAM_TAG, so moving them into __init__ doesn’t break current callers.

			> Likely an incorrect or invalid review comment.
lumibot/credentials.py (2)

65-78: LGTM!


1191-1195: 🔒 Security & Privacy

Drop this comment. The cleanup also removes BROKER and DATA_SOURCE, so there’s no stale cached export left behind.

			> Likely an incorrect or invalid review comment.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant