Repository navigation
docs(roadmap): refresh Unicode spec; test: close orphaned search pool - #117
Merged
Merged
Conversation
…robe Mark Phase 1, truncation, and ICU collation done. Correct the \w-is-ASCII claim (the real bug was hyphens, fixed in #115). Record that the shipped title index sends accented Latin and non-Latin titles to 0-9. New finding, tested on postgres:18-alpine: with the deployed --locale=C, ctype is C and pg_trgm extracts no trigrams from Cyrillic/CJK, so the planned Phase 2 fallback would be a no-op. Adding --lc-ctype=C.UTF-8 fixes it and keeps ICU ordering; the builtin provider fixes it but loses linguistic order. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MjiSEYcZ2q8F7bke96VGHT
Each reload orphaned the previous PostgresSearch pool; GC then finalized it on one of its own worker threads, raising 'cannot join current thread' in ConnectionPool.__del__. Test suite now runs with zero warnings. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MjiSEYcZ2q8F7bke96VGHT
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two independent commits.
1. Roadmap: Unicode spec refreshed against the code and a live database probe
roadmap/05-unicode-foreign-language-support.mdhad fallen behind the code:'simple'FTS (migration 008), CJK-aware truncation, ICU collation, Alpine ILIKE/LOWER folding (verified).\w+only matches ASCII. That is false in Python 3. The real operator bug was hyphenated usernames, fixed in fix(search): hyphenated authors and uppercase operators #115.letter_bucket()sends every title not starting with ASCII a–z to0-9. That includes accented Latin (Éclair,über), not only CJK/Cyrillic. A cheap first fix is diacritic folding via NFKD.New finding: Phase 2 (pg_trgm) is blocked by the deployed ctype
Tested on
postgres:18-alpine(PG 18.6) with the exactPOSTGRES_INITDB_ARGSfromdocker-compose.yml:show_trgm('Привет')show_trgm('日本語')ORDER BYicu … --locale=C){}{}a,A,b,B,e,é,Z--lc-ctype=C.UTF-8a,A,b,B,e,é,Zbuiltin C.UTF-8A,B,Z,a,b,e,é✗--locale=Cleaves ctype atC, so pg_trgm ignores all non-ASCII characters. The planned CJK fallback would silently do nothing. The spec now records--lc-ctype=C.UTF-8as a Phase 2 prerequisite. The compose file is not changed in this PR; it only affects newly initialized clusters.2. Test: close the search-engine pool before reloading
search_servertests/test_dynamic_pages.pyreloadssearch_serverper test, which orphaned the previousPostgresSearchpool. GC then finalized it on one of its own worker threads (RuntimeError: cannot join current threadinConnectionPool.__del__). The fixtures now close it before each reload and on teardown.Result: full suite 989 passed, 0 warnings (3 consecutive runs). It was 639 warnings two PRs ago. ruff clean.
🤖 Generated with Claude Code
https://claude.ai/code/session_01MjiSEYcZ2q8F7bke96VGHT