Market Analysis with AI Helpers

Status: Initial public snapshot. See docs/ai-trail/ for how this slice was built with AI coding assistance.

A Python application for retail-investor market data aggregation: a multi-source ETL pipeline backed by PostgreSQL, exposed through a FastAPI surface, with architectural decisions recorded as ADRs.

This repository is a public excerpt of a larger private codebase. It ships the architectural and API layers; the UI layer and several enrichment sources are deferred to a follow-up workstream — see Public scope below.

---

Why this exists

A demonstration of architectural and engineering choices that go into a well-typed, layered, observability-friendly data pipeline:

• Strict layering: Extractor → Service → Repository → API. No shortcuts. (ADR-0004)
• PostgreSQL-only: No SQLite fallback; local and production run the same engine. (ADR-0002)
• Decimal, not float: Money never touches IEEE-754. (ADR-0001)
• Pydantic at boundaries: Typed contracts at every layer crossing.
• Pipeline error visibility: No silent swallow-and-continue. (ADR-0010)

Decision records live in decisions/.

How this slice was built — plan, session, PR, final code — is recorded in docs/ai-trail/.

Built with AI assistance

This repository was built primarily through terminal chat with AI coding assistants — using a private workflow that paired a long-running orchestrating session with focused executor sessions for individual workstreams.

The architectural-decisions trail in decisions/ is the most direct evidence of what was decided and why. The commit-by-commit story of how the decisions evolved — sprint plan, one session brief, the resulting commits, and the final code — lives in docs/ai-trail/.

Tech stack

• Python 3.13
• FastAPI + Pydantic v2
• PostgreSQL 16 (Cloud SQL-compatible)
• SQLAlchemy 2.x async + Alembic migrations
• yfinance (public market data) + SEC EDGAR (free filings)
• Docker / docker-compose for local

All data sources used are public or free-tier — see docs/data-sources.md for the source-by-source setup walkthrough (free-tier registration steps for every named provider; ~30 minutes total to set up the broader source set).

Repo layout

api/                 FastAPI app + route modules
alembic/             SQLAlchemy migrations (single init migration, 5 tables)
data/universe.yaml   Demo-basket ticker definitions (illustrative — not a portfolio)
db/                  SQLAlchemy ORM models + repositories
decisions/           Architectural Decision Records
docs/screenshots/    Inline images referenced from this README
extractors/          External data source clients (yfinance, SEC EDGAR, ...)
models/              Pydantic domain models
services/            Business logic — pipelines, repositories, ticker resolution
scripts/             Utilities — seed, backfill, ai_export package
ui/                  Streamlit pages — sidebar nav to Company Deep-Dive

Dockerfile           App container build (FastAPI + uvicorn)
docker-compose.yml   Local dev orchestration (Postgres + app)
alembic.ini          Alembic config
init_db.py           Schema-create + companies-seed entrypoint (run once on first setup)
requirements.txt     Pinned runtime deps (pip-tools-frozen)
requirements-dev.txt Pinned dev/test deps
.env.example         Environment-variable template (copy to .env)

Public scope

This is a snapshot, not the full project. Specifically:

• The API and decision records are present in their architectural shape.
• The full architectural arc is present: extractor → service → repository → API → UI, end-to-end runnable per the Setup walkthrough below.
• Static output samples from the multi-source data slice live under examples/ai_analysis/ — illustrative deep dives produced by feeding the slice's data tables to chat assistants (Claude, Gemini, Mistral Le Chat). The helper runtime (alerts pipeline, prompt routing, live LLM integration) remains deferred — the public slice does not require an LLM key to run.
• Tests are not shipped in this snapshot. The private codebase has ~700 tests across the same surface; carving the right subset (without coupling to private-only fixtures) is a separate workstream.
• The 5 tickers in data/universe.yaml (JNJ, MSFT, AAPL, SAP.DE, 7203.T) are a demo basket — not the maintainer's portfolio. They were chosen to span US, EU, and JP markets so the walkthrough exercises multi-currency handling.

Setup

Five minutes from clone to running API. No paid API keys required — yfinance is unauthenticated.

Prerequisites: Docker, Python 3.13.

# 1. Clone + enter
git clone https://github.com/onkovstefan-dot/Market-analysis-with-AI-helpers.git
cd Market-analysis-with-AI-helpers

# 2. Environment
cp .env.example .env
python -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt

# 3. Database
docker compose up -d db
export DATABASE_URL=postgresql://invest_user:invest_pass@localhost:5432/investment_db
alembic upgrade head     # creates the 5-table schema
python init_db.py        # seeds 5 demo-basket companies

# 4. Backfill 5y of price + dividend data via yfinance (~3–5 min)
python -m scripts.backfill --source yfinance \
  --ticker JNJ --ticker MSFT --ticker AAPL --ticker SAP.DE --ticker 7203.T \
  --period 5y

# 5. Boot the API
python -m uvicorn api.main:app --host 0.0.0.0 --port 8000

# 6. Verify (in a second terminal)
curl -s localhost:8000/health/freshness | jq
# Browser: http://localhost:8000/docs

# 7. Boot the UI (Streamlit, in a third terminal)
streamlit run ui/app.py
# Browser: http://localhost:8501
# Sidebar → Company Deep-Dive → pick a ticker

API examples

Pre-captured JSON responses live under examples/api/ — handy if you want to see the response shapes without booting the slice. See examples/api/README.md for the file map and regenerate command.

AI deep-dive examples

Pre-captured markdown deep dives live under examples/ai_analysis/ — five demo-basket tickers × three chat models, each grounded in the same multi-source data tables the public slice produces. See examples/ai_analysis/README.md for the file map, capture date, and what was redacted.

The slice exercises multi-currency handling: USD tickers (JNJ, MSFT, AAPL), an EUR ticker (SAP.DE), and a JPY ticker (7203.T) all render through the same FastAPI-backed UI. The dividend column reflects the source currency; no FX normalization happens in the slice.

The company_fundamentals table will show "status": "empty" in the freshness report — the public slice doesn't ship a fundamentals fetcher; only daily_prices and dividend_events are populated by the yfinance backfill.

License & disclaimer

• License: MIT — see LICENSE.
• Not financial advice. See DISCLAIMER.md.