docs(destination-pgvector): correct connector documentation - #84386
Draft
devin-ai-integration[bot] wants to merge 1 commit into
Draft
docs(destination-pgvector): correct connector documentation#84386devin-ai-integration[bot] wants to merge 1 commit into
devin-ai-integration[bot] wants to merge 1 commit into
Conversation
…tation Co-Authored-By: bot_apk <apk@cognition.ai>
Contributor
Author
🤖 Devin AI EngineerI'll be helping with this pull request! Here's what you should know: ✅ I will automatically:
Note: I can only respond to comments from users who have write access to this repository. ⚙️ Control Options:
|
devin-ai-integration
Bot
added
area/documentation
Contributor
👋 Greetings, Airbyte Team Member!Here are some helpful tips and reminders for your convenience. 💡 Show Tips and TricksPR Slash CommandsAirbyte Maintainers (that's you!) can execute the following slash commands on your PR:
📚 Show Repo GuidanceHelpful Resources
|
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
| embedding method you choose. | ||
|
|
||
| #### Configure Network Access | ||
| #### Configure network access |
Contributor
There was a problem hiding this comment.
[markdownlint] reported by reviewdog 🐶
MD001/heading-increment Heading levels should only increment by one level at a time [Expected: h3; Actual: h4]
| #### **Permissions** | ||
|
|
||
| You need a Postgres user with the following permissions: | ||
| #### Permissions |
Contributor
There was a problem hiding this comment.
[markdownlint] reported by reviewdog 🐶
MD001/heading-increment Heading levels should only increment by one level at a time [Expected: h3; Actual: h4]
Contributor
|
Deploy preview for airbyte-docs ready!
Deployed with vercel-action |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Triggering Context
Run triggered by: Commit
b355a806613b2f4179e9ce42a6b603e00c1c42cemerged tomaster(CDK bump to remediate CVE-2025-68664 in the langchain dependency, version 0.1.11 → 0.1.12).Relevant context: #84364
Confidence impact: The trigger is a small, already-merged dependency bump with unambiguous scope, so the triggering context itself is clear (5/5); the documentation work it surfaced is broader than the trigger.
Documentation Confidence Assessment
Overall Confidence: 3/5
pgvector_processor.py,config.py, and the sharedvector_db_basedCDK, all of which I read end to end.metadata.yamllists only 2 external doc URLs.ql: 300/sl: 200.Weighted composite: 0.15(4) + 0.15(4) + 0.25(2) + 0.10(4) + 0.15(4) + 0.10(5) + 0.10(4) = 3.6 → 3. No hard cap applied. Per the confidence gate,
auto-mergeis not applied at 3.Adjustments based on code comprehension
airbyte_cdk.destinations.vector_db_basedimplementation; only the Postgres write path is connector-specific.integration_tests/spec.jsonor the shared CDK constants.document_idandchunk_idare produced, and what deduplication does; I have not run a live sync.What I Verified vs. What I Inferred
document_id,chunk_id,metadata(JSON),document_content,embedding(Vector(embedding_dimensions)) inpgvector_processor.py.chunk_idis a random UUID integer, anddocument_idisStream_{stream}_Key_{primary key}or a random UUID integer when the record has no primary key.supports_merge_insert = False: deduplication deletes all rows for the affecteddocument_idvalues and reinserts the current chunks.LowerCaseNormalizer(lower case, non-alphanumerics to underscores, leading digit prefixed), and the connector creates the schema if it doesn't exist.config.py:port5432,default_schemapublic, single schema, password is a secret._ab_streamis always added to metadata;_ab_record_idis added only for append+dedup streams that have a primary key (document_processor.py).text_fieldsormetadata_fieldsmeans all record fields are used (_extract_relevant_fields).omit_raw_text;document_contentis always written.OPEN_AI_VECTOR_SIZE = 1536,COHERE_VECTOR_SIZE = 1024in the shared CDK; OpenAI-compatible dimensions are user-configured.destination.py: overwrite, append, append+dedup.api.openai.com,api.cohere.ai,${embedding.api_base}) frommetadata.yaml.CREATE EXTENSION vectorand provides HNSW and IVFFlat indexes, which the connector does not create; Postgres folds unquoted identifiers to lower case and truncates identifiers at 63 bytes; OpenAI publishes embedding rate limits that bound sync throughput.Areas of Concern
jsoncolumn with no such limit. It also promised "Coming soon: Hugging Face'se5-base-v2", which does not exist in the code.2026-08-13, PR 84364) matches the merge date of the triggering commit, and all recent link texts match their URLs.What
The PGVector destination doc described behavior this connector doesn't have. It documented metadata type and size limits from a different vector destination, listed an embedding model that was never implemented, omitted the required pgvector extension step from the prerequisites, and pasted the generic Postgres identifier rules instead of what the connector actually does to stream names. This PR corrects those statements and documents the table layout, deduplication behavior, and embedding options as they exist in the code.
The trigger was the CDK bump in #84364, which needed no doc change of its own beyond the changelog row it already added.
How
Corrections
jsoncolumn, so nested objects and arrays survive.e5-base-v2".[Step 1](#step-1-optional-create-a-dedicated-read-only-user)anchor, and the setup steps that were interleaved with the naming-conventions section and numbered1..8then7.Default Schemais one schema, not a comma-separated search path.Additions
CREATE EXTENSION IF NOT EXISTS vectoris now part of the Postgres setup rather than an aside, along with the privilege it needs.document_idandchunk_idare produced and why records without a primary key can't be traced back.embeddingcolumn, with a pointer to pgvector's index docs._ab_record_idis only present for append+dedup streams with a primary key. The old text implied it was always there.Removals
#### Target Databasesection that only said to pick a database.The changelog is unchanged.
Review guide
docs/integrations/destinations/pgvector.md— the metadata data type section and the naming conventions section are the two places where the old text was wrong rather than incomplete.OPEN_AI_VECTOR_SIZEandCOHERE_VECTOR_SIZEin the shared CDK.User Impact
Docs only. No connector behavior changes.
Can this PR be safely reverted and rolled back?
Link to Devin session: https://app.devin.ai/sessions/f141ee2f661844ef8995637a4705d44d