Skip to content

Add UK and India national ID formats - #1461

Open
AtomicGlance wants to merge 5 commits into
databrickslabs:mainfrom
AtomicGlance:agent/extend-national-id-patterns
Open

Add UK and India national ID formats#1461
AtomicGlance wants to merge 5 commits into
databrickslabs:mainfrom
AtomicGlance:agent/extend-national-id-patterns

Conversation

@AtomicGlance

@AtomicGlance AtomicGlance commented Aug 12, 2026

Copy link
Copy Markdown
Contributor

Summary

Extend is_valid_national_id with format checks for:

  • UK National Insurance numbers (GB), including HMRC-unallocated prefix exclusions.
  • Indian Permanent Account Numbers (IN), with the fourth character restricted to supported holder-type codes.

The existing US behavior remains unchanged. Added Spark integration coverage for valid, invalid, and null PAN/NINO values, plus updated the quality-check reference.

Closes #1413

Validation

  • .venv\Scripts\pytest.exe tests/unit/test_row_checks.py -q — 71 passed
  • .venv\Scripts\ruff.exe check src/databricks/labs/dqx/check_funcs.py tests/unit/test_row_checks.py tests/integration/test_row_checks.py — passed
  • .venv\Scripts\black.exe --check src/databricks/labs/dqx/check_funcs.py tests/unit/test_row_checks.py tests/integration/test_row_checks.py — passed
  • git diff --check — passed
  • The targeted Spark integration test requires configured Databricks workspace credentials; the PR workflows are awaiting maintainer approval.

@github-actions

Copy link
Copy Markdown
Contributor

All commits in PR should be signed ('git commit -S ...'). See https://docs.github.com/en/authentication/managing-commit-signature-verification/signing-commits

@AtomicGlance
AtomicGlance marked this pull request as ready for review August 12, 2026 07:14
@AtomicGlance
AtomicGlance requested a review from a team as a code owner August 12, 2026 07:14
@AtomicGlance
AtomicGlance requested review from nehamilak-db and removed request for a team August 12, 2026 07:14
@AtomicGlance
AtomicGlance force-pushed the agent/extend-national-id-patterns branch from 5b960fc to 43b6208 Compare August 12, 2026 07:22
@AtomicGlance
AtomicGlance force-pushed the agent/extend-national-id-patterns branch from 43b6208 to 1b40120 Compare August 12, 2026 09:52

@ghanse ghanse left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good overall! Left a few small suggestions.

Comment thread src/databricks/labs/dqx/check_funcs.py Outdated
Comment thread tests/unit/test_row_checks.py
@ghanse ghanse added under-review This PR is currently being reviewed by one of DQX maintainers. needs-changes Changes required after review labels Aug 12, 2026
@AtomicGlance
AtomicGlance requested a review from ghanse August 12, 2026 19:58
@AtomicGlance

Copy link
Copy Markdown
Contributor Author

@ghanse @nehamilak-db I’ve addressed the requested PAN/NINO changes in 02512c4, including the integration coverage, and the signed-commit check is passing. Could you please re-review when convenient? The required CI workflows are currently waiting for maintainer approval.

@ghanse ghanse left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left a few comments on the tests. Otherwise looks good!

Comment thread tests/integration/test_row_checks.py Outdated
Comment thread tests/integration/test_row_checks.py Outdated
Comment thread tests/integration/test_row_checks.py Outdated
@AtomicGlance

Copy link
Copy Markdown
Contributor Author

Addressed the latest test feedback in 2459815: the integration test is now parameterized across US, GB, and IN, with empty-string, trailing-newline, and surrounding-space cases for NINO and PAN. Ruff, Black, diff checks, and all 71 row-check unit tests pass. The targeted integration test still requires DATABRICKS_HOST locally, so the approved PR workflow will provide the Spark run. All three review conversations are resolved. @ghanse, could you take another look when convenient?

@AtomicGlance
AtomicGlance requested a review from ghanse August 14, 2026 08:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

needs-changes Changes required after review under-review This PR is currently being reviewed by one of DQX maintainers.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[FEATURE]: Extend is_valid_national_id to support UK and India

2 participants