Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
68 commits
Select commit Hold shift + click to select a range
82ae672
Add object_classes filter, validate README examples. Bump to 0.7.3
caseydreier Jul 5, 2026
3749e9f
Phase 0: add refactor safety net (characterization + golden master)
caseydreier Jul 25, 2026
420a593
Phase 1: consolidate award-type knowledge into one registry
caseydreier Jul 25, 2026
198f771
Phase 2a: collapse filter-method bodies onto _with_filter
caseydreier Jul 25, 2026
0d57abb
Phase 2b: give _clone one implementation and a construction seam
caseydreier Jul 25, 2026
586092d
Phase 2c: separate raw from effective counts, share the count mechanisms
caseydreier Jul 25, 2026
dd732bc
Phase 2d: make count() a template method, and stop caching it
caseydreier Jul 25, 2026
fa7fb75
Phase 2e: extract AwardScopedQuery and SortableQuery mixins
caseydreier Jul 25, 2026
6d8657c
Phase 2f: parameterize the HTTP verb, consolidate toptier validation
caseydreier Jul 25, 2026
7ea1303
Phase 2g: extract FilterTreeQuery for the two filter-tree queries
caseydreier Jul 25, 2026
ff0992a
Phase 3a: extract ProcurementAward, drop redundant Loan overrides
caseydreier Jul 25, 2026
64f30c9
Phase 3b: dedup Award agency accessors and Agency obligation properties
caseydreier Jul 25, 2026
07b2981
Phase 3c: drop Agency's write-only subtier_data parameter
caseydreier Jul 25, 2026
d94095a
Phase 3d: route hand-rolled int coercions through to_int
caseydreier Jul 25, 2026
41b050c
Phase 3e: fix Transaction equality and FederalAccount.count
caseydreier Jul 25, 2026
613f3cd
Phase 3f: extract SpendingMixin, collapse the spending reprs
caseydreier Jul 25, 2026
00a5469
Phase 3h: document the Award __class__ swap and the mixin's MRO hazard
caseydreier Jul 25, 2026
9c0d97f
Phase 3g: extract award-identifier parsing, derive the agency flat keys
caseydreier Jul 25, 2026
53166e9
Name the spending fields by concern, not by mechanism
caseydreier Jul 25, 2026
ca55307
Fix AttributeError on IDV child-award agency accessors
caseydreier Jul 25, 2026
3b75d78
Phase 4: return None for absent money and string values
caseydreier Jul 25, 2026
72aba29
Phase 4 follow-up: close the gap my own sweep missed
caseydreier Jul 25, 2026
9f57f74
Phase 5: consolidate recipient-ID normalization, fixing a live defect
caseydreier Jul 25, 2026
dc40444
Phase 5: move TAS agencies onto the filter-tree pattern, dedup the fi…
caseydreier Jul 25, 2026
ff5a7f8
Phase 5 follow-up: avoid the R level by membership, and normalize one…
caseydreier Jul 25, 2026
53d5c0c
Expose Recipient.recipient_level instead of a level parameter
caseydreier Jul 25, 2026
4c9beb1
Phase 6: retire the smart_sentence_case wrapper, rename off the agenc…
caseydreier Jul 25, 2026
a7729b3
Phase 6: index the special cases instead of scanning them, 7x faster …
caseydreier Jul 25, 2026
3c8e578
Phase 6: fail loudly on corrupt casing data, and three smaller fixes
caseydreier Jul 26, 2026
e40a05f
Warn instead of raising when the casing data is unreadable
caseydreier Jul 26, 2026
6681f10
Stop handing callers a shared mutable list, and drop a redundant request
caseydreier Jul 26, 2026
7f05853
Index the sentence-case lookup too, and warn on colliding casing data
caseydreier Jul 26, 2026
bbde1a9
Correct the filter-tree contracts that described themselves wrongly
caseydreier Jul 26, 2026
909cdf5
Allowlist the recipient aggregates the golden masters compared exactly
caseydreier Jul 27, 2026
456e360
Phase 6: split formatter.py into dates, numbers and textcase
caseydreier Jul 27, 2026
c0fdc27
Return a query from client.tas.agencies, matching its sibling levels
caseydreier Jul 27, 2026
f634c06
Update comment with N+1 warning
caseydreier Jul 27, 2026
7c5df1d
Fetch each TAS filter-tree level once per model
caseydreier Jul 27, 2026
19eefdd
Parse transaction date bounds once, and run the documented examples
caseydreier Jul 27, 2026
d540570
Skip the transaction date predicate when no bound is set, and fix 10 …
caseydreier Jul 27, 2026
ca6df27
Stop all() spending a request on a count it discards
caseydreier Jul 27, 2026
24dc5dd
Make count, truthiness and limits agree with what a query yields
caseydreier Jul 27, 2026
967015c
Reattach every nested model that holds a client, not just the lazy ones
caseydreier Jul 28, 2026
a0fd1d9
Read the API's own date shape without walking the format list
caseydreier Jul 28, 2026
5968ab9
Narrow a datetime to a date at the second of the two places that must
caseydreier Jul 28, 2026
bb39910
Answer an unusable date the way each parser's contract says
caseydreier Jul 28, 2026
4538d66
Hold since() and until() to the same date rules as time_period()
caseydreier Jul 28, 2026
6c1cff5
Stop a value that is not a date from flushing the shared strptime cache
caseydreier Jul 28, 2026
c5c4be9
Table the date accept and reject cases, as the sibling tests already do
caseydreier Jul 28, 2026
c45d298
Wire up the dead sort validator, and settle on one message per check
caseydreier Jul 28, 2026
a8e86a4
Guard the resource and query classes the surface snapshot was missing
caseydreier Jul 28, 2026
385ae0f
Record why _REATTACH_INVALIDATES stays a list rather than a descriptor
caseydreier Jul 28, 2026
d2a286f
Let the recipient and period models read their own search-result keys
caseydreier Jul 28, 2026
e44172a
Apply the batch review: close the gaps the eight commits left
caseydreier Jul 28, 2026
e128d56
Record the one shape the shorter format list made slower
caseydreier Jul 28, 2026
bf1f367
Stop filter aggregation from mutating shared filter lists
caseydreier Jul 28, 2026
77785f3
Make client-side indexing read the limited collection
caseydreier Jul 28, 2026
2be8df3
Make count() honor the caller's bounds on every builder
caseydreier Jul 28, 2026
72424b7
Let a failed lazy load surface instead of latching silent Nones
caseydreier Jul 28, 2026
fc44330
Restore the two 0.7.3 surfaces the refactor dropped without an alias
caseydreier Jul 28, 2026
9a293dd
Make every test able to fail for the behavior it claims to guard
caseydreier Jul 28, 2026
d509c23
Make the API-surface snapshot interpreter-independent and signature-a…
caseydreier Jul 28, 2026
14d7f67
Guard the golden master's volatile fields and the search projection
caseydreier Jul 28, 2026
9483aa4
Ship 0.8.0: complete the CHANGELOG and harden the release metadata
caseydreier Jul 28, 2026
e007b3f
Make request-count assertions hold on every supported Python
caseydreier Jul 28, 2026
bae6308
Add continuous integration
caseydreier Jul 28, 2026
6d9883b
Lead the release notes with what breaks
caseydreier Jul 28, 2026
5000a6b
Update CHANGELOG.md
caseydreier Jul 28, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 30 additions & 0 deletions .github/workflows/integration.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
name: integration

# Hits the live USAspending API (about 50 requests), so this runs on a weekly
# schedule and on demand rather than gating every push. The golden-master
# suite inside it is the only check against live data shapes. Note GitHub
# disables scheduled workflows after 60 days without repository activity.

on:
schedule:
- cron: "0 14 * * 1"
workflow_dispatch:

permissions:
contents: read

concurrency:
group: integration
cancel-in-progress: false

jobs:
live:
runs-on: ubuntu-latest
timeout-minutes: 15
steps:
- uses: actions/checkout@v4
- uses: astral-sh/setup-uv@v9.0.0
- name: Sync environment
run: uv sync --locked
- name: Run integration suite against the live API
run: uv run --no-sync pytest -m integration -q
71 changes: 71 additions & 0 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
name: test

on:
push:
branches: [main]
pull_request:
workflow_dispatch:

permissions:
contents: read

concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: ${{ github.event_name == 'pull_request' }}

jobs:
unit:
runs-on: ubuntu-latest
timeout-minutes: 10
strategy:
fail-fast: false
matrix:
python-version: ["3.9", "3.10", "3.11", "3.12", "3.13", "3.14"]
steps:
- uses: actions/checkout@v4
- uses: astral-sh/setup-uv@v9.0.0
with:
# Also keys the wheel cache per interpreter; without it every
# matrix leg shares the key of whichever leg saved first.
python-version: ${{ matrix.python-version }}
- name: Sync environment
run: uv sync --locked
- name: Run unit suite
run: uv run --no-sync pytest -q

checks:
runs-on: ubuntu-latest
timeout-minutes: 10
steps:
- uses: actions/checkout@v4
- uses: astral-sh/setup-uv@v9.0.0
- name: Sync environment
run: uv sync --locked
- name: Ruff lint
run: uv run --no-sync ruff check src/ tests/
- name: Ruff format
run: uv run --no-sync ruff format --check src/ tests/
- name: Coverage floor
run: uv run --no-sync pytest -q --cov=src/usaspending --cov-fail-under=80

package:
runs-on: ubuntu-latest
timeout-minutes: 10
steps:
- uses: actions/checkout@v4
- uses: astral-sh/setup-uv@v9.0.0
- name: Build sdist and wheel
run: uv build
- name: Smoke-test the built wheel
# A behavioral assertion rather than a bare import: the sdist include
# list is hand-written, and a dropped package-data file (such as
# special_cases.yaml) degrades output silently instead of failing.
run: |
uv venv /tmp/smoke
uv pip install --python /tmp/smoke/bin/python dist/*.whl
/tmp/smoke/bin/python -c "
import usaspending
from usaspending.utils.textcase import titlecase_name
got = titlecase_name('NASA GODDARD SPACE FLIGHT CENTER, L.L.C.')
assert got == 'NASA Goddard Space Flight Center, LLC', got
"
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -4,3 +4,6 @@ __pycache__
.*_cache
repomix*
dist/
.claude/
/~/
.coverage
404 changes: 404 additions & 0 deletions CHANGELOG.md

Large diffs are not rendered by default.

31 changes: 23 additions & 8 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -114,6 +114,11 @@ tests/
- Run integration tests: `uv run pytest -m integration`
- Lint: `uv run ruff check src/ tests/`
- Format: `uv run ruff format src/ tests/`
- Format check (what CI runs): `uv run ruff format --check src/ tests/`
- Coverage gate (what CI runs): `uv run pytest --cov=src/usaspending --cov-fail-under=80`
- CI: `.github/workflows/test.yml` runs the unit suite on Python 3.9-3.14 plus the
checks above on every push to `main` and every PR; `integration.yml` runs the
live-API suite weekly and on demand
- No Justfile in this project

## Implementation Patterns
Expand All @@ -132,15 +137,25 @@ tests/

### Query Builders

- Hierarchy: `BaseQuery[T]` -> `QueryBuilder[T]` -> `SearchQueryBuilder[T]`; also `ClientSideQueryBuilder[T]` (extends `BaseQuery`)
- `QueryBuilder[T]` handles paginated API queries; `ClientSideQueryBuilder[T]` handles client-side filtering
- Implement abstract methods:
- `_endpoint()`: API endpoint
- Hierarchy: `BaseQuery[T]` -> `QueryBuilder[T]` -> `SearchQueryBuilder[T]`; also
`ClientSideQueryBuilder[T]` (extends `BaseQuery`) -> `FilterTreeQuery[T]`
- `QueryBuilder[T]` handles paginated API queries; `ClientSideQueryBuilder[T]` handles
client-side filtering; `FilterTreeQuery[T]` fetches one unpaginated filter-tree level
once and filters it in memory
- A paginated builder implements:
- `_endpoint`: API endpoint (property)
- `_build_payload()`: Request payload
- `_transform_result()`: Result transformation
- `_clone()`: Return cloned instance
- `__iter__()`: Iterable interface
- All filter methods return cloned instances
- `_compute_raw_count()`: How this endpoint reports its count. Use one of
`_count_via_endpoint`, `_count_via_page_metadata` or `_count_via_paging`
- `_new_instance()`: Only when `__init__` takes more than a client
- `_http_method`: Only for GET endpoints (payload is sent as query parameters)
- A filter-tree query implements `_scope()`, `_build_model()` and `_new_instance()`
- `_clone()` is inherited; override it only to carry subclass state, always via
`super()._clone()`. `count()` is inherited and should not be overridden
- Shared slices of behavior live in `queries/mixins.py` (`AwardScopedQuery`,
`SortableQuery`); compose them ahead of the builder base
- All filter methods return cloned instances, via `_with_filter()` where possible

### Caching

Expand Down Expand Up @@ -218,7 +233,7 @@ tests/
### Quality Gates

- 100% public API documented
- Type hints pass mypy
- Type hints pass mypy (aspirational; not yet enforced, no CI job runs mypy)
- Tests pass with >80% coverage
- No TODO/FIXME in code

Expand Down
85 changes: 67 additions & 18 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -51,11 +51,14 @@ with USASpendingClient() as client:
```python
with USASpendingClient() as client:
award = client.awards.find_by_award_id("80GSFC18C0008")
award.recipient.location.full_address # -> 105 Jessup Hall, Iowa City, IA, 52242, United States
award.subawards.count() # -> 100
award.subawards[2].recipient.place_of_performance.district # -> AL-03
award.recipient.name # -> "The University of Iowa"
award.recipient.location.formatted_address # -> "105 Jessup Hall\nIowa City, IA, 52242\nUnited States"
award.subawards.count() # -> 150
award.subawards[2].recipient.name # -> "Bison Aerospace, Inc"
```

Sample outputs throughout this document were validated against live USAspending data in July 2026. Counts, amounts, and dates will drift as agencies report new spending.

#### Searching for Awards

You can query awards data using the `search()` method on the `client.awards` object:
Expand All @@ -76,7 +79,17 @@ awards_query = client.awards.search() \
```

This returns a query object that can be further refined or executed to return results.
The methods `.all()`, `.first()`, `.count()` will trigger a query to the API, as will iterating over the query object.
The methods `.all()`, `.first()`, `.count()` will trigger a query to the API, as will iterating over the query object and testing it with `if query:` (which reads a single row). Query objects also support the standard Python list interface, so `len(awards_query)` fires the corresponding count endpoint:

```python
len(client.awards.search()
.agency("National Aeronautics and Space Administration")
.grants()
.keywords("Mars")
.fiscal_year(2024)) # -> 404
```

Additional filters follow the same snake_case pattern. For example, `.object_classes("10", "252")` filters awards by object class codes (per Office of Management and Budget Circular A-11; pass codes, not names).

### Example: Searching for National Aeronautics and Space Administration Contracts to SpaceX in 2023

Expand All @@ -101,16 +114,16 @@ with USASpendingClient() as client:
top_spacex_award = awards_query.first()

# Returned value is an Award object with all properties mapped
# and properly typed.
top_spacex_award.total_obligation # -> Decimal('3029850123.69')
# and properly typed. The top result is the Human Landing System contract.
top_spacex_award.total_obligation # -> Decimal('3047851482.99')
top_spacex_award.category # -> "contract"
top_spacex_award.description # -> "The Commercial Crew Program (CCP) contract ...."
top_spacex_award.description # -> "Work required for the design, development, manufacture, test, launch..."

# Helper methods provide easy access to common fields without having to account for
# inconsistent naming or nested structures in the raw API response
top_spacex_award.award_identifier # -> "80GSFC18C0008"
top_spacex_award.start_date # -> datetime.date(2016, 12, 30)
top_spacex_award.end_date # -> datetime.date(2023,12,31)
top_spacex_award.award_identifier # -> "80MSFC20C0034"
top_spacex_award.start_date # -> datetime.date(2020, 5, 13)
top_spacex_award.end_date # -> datetime.date(2027, 12, 6)

# The resulting object provides a normalized interface to the full Award record,
# and provides access to related data via chained associations
Expand All @@ -121,11 +134,40 @@ with USASpendingClient() as client:

# Award Transactions
last_transaction = top_spacex_award.transactions.order_by("action_date", "desc").first()
last_transaction.action_date # -> datetime.date(2025, 10, 08)
last_transaction.action_date # -> datetime.date(2026, 7, 1)
last_transaction.action_type_description # -> "SUPPLEMENTAL AGREEMENT FOR WORK WITHIN SCOPE"

```

### Example: Top National Aeronautics and Space Administration Contractors in Fiscal Year 2024

Aggregation endpoints are available through `client.spending`. The same fluent filters
build a category rollup instead of an award list, and iterating the query handles
pagination automatically:

```python
with USASpendingClient() as client:
top_recipients = (
client.spending.search()
.by_recipient()
.agency("National Aeronautics and Space Administration")
.contracts()
.fiscal_year(2024)
.limit(5)
)

for spending in top_recipients:
print(f"{spending.name}: ${spending.amount:,.0f}")

# California Institute of Technology: $2,175,598,668
# Space Exploration Technologies Corp.: $1,589,359,578
# The Boeing Company: $1,123,735,127
# Lockheed Martin Corp: $570,859,903
# Blue Origin Washington, LLC: $502,942,297
```

Spending rollups can also be grouped geographically with `.by_district()` or `.by_state()`.

### Downloading Bulk Data

The `client.downloads` resource queues server-side download jobs and returns a `DownloadJob`
Expand Down Expand Up @@ -153,6 +195,8 @@ job = client.downloads.search(query, spending_level=["awards"], file_format="csv
status = job.wait_for_completion()
```

The `/api/v2/download/search/` endpoint does not accept every search filter. In particular, `object_classes` is ignored by the endpoint, so the library emits a `UserWarning` when a download query includes it.

## Configuration

### Session Management and Lazy-Loading
Expand All @@ -168,7 +212,8 @@ with USASpendingClient() as client:
awards = client.awards.search().agency("National Aeronautics and Space Administration").all()
for award in awards:
# Access lazy-loaded properties inside the context
print(f"{award.recipient.name}: ${award.total_obligation:,.2f}")
# Money getters return None when the award reports no figure
print(f"{award.recipient.name}: ${award.total_obligation or 0:,.2f}")
print(f"Subawards: {award.subaward_count}")
# Session automatically closed here

Expand Down Expand Up @@ -199,14 +244,18 @@ with USASpendingClient() as client:

# Reattach to a new session to access related properties
with USASpendingClient() as new_client:
award.reattach(new_client)
print(f"Subawards: {award.subawards.count()}") # Works!

# Recursive reattach for nested objects
# recursive=True is normally what you want: it also rebinds models already
# loaded from this one, such as award.recipient
award.reattach(new_client, recursive=True)
print(f"Recipient: {award.recipient.name}") # Recipient also reattached
print(f"Subawards: {award.subawards.count()}")
print(f"Recipient: {award.recipient.name}")
```

Without `recursive=True` only the award itself is rebound. That is enough for
properties it loads afresh, but a nested model already loaded, such as
`award.recipient`, keeps pointing at the old session and raises
`DetachedInstanceError` once that session is gone.

### Performance Considerations

#### Lazy Loading and N+1 Queries
Expand Down Expand Up @@ -350,7 +399,7 @@ usaspending_config.configure(
# Set HTTP request parameters (default: max_retries=3, timeout=30)
# Set number of retries for failed requests (default: 3)
max_retries=5,
# Set delay between retries in seconds (default: 1.0)
# Set delay between retries in seconds (default: 10.0)
retry_delay=10.0,
# Set exponential backoff factor for retries (default: 2.0)
retry_backoff=2.0,
Expand Down
Loading
Loading