Skip to content

docs(destination-sftp-json): fix output file naming, add setup and troubleshooting - #84391

Draft
devin-ai-integration[bot] wants to merge 2 commits into
masterfrom
docs/auto/destination-sftp-json
Draft

docs(destination-sftp-json): fix output file naming, add setup and troubleshooting#84391
devin-ai-integration[bot] wants to merge 2 commits into
masterfrom
docs/auto/destination-sftp-json

Conversation

@devin-ai-integration

@devin-ai-integration devin-ai-integration Bot commented Aug 13, 2026

Copy link
Copy Markdown
Contributor

Triggering Context

Run triggered by: Merged connector commit on master389b32dff7b70dca7fbd7f17e42250b3f72fdf00, "fix(destination-sftp-json): escape SFTP URI components" (destination-sftp-json 0.2.16).

Relevant context: #78111

Confidence impact: The trigger is a small, already-merged bug fix with a clear diff and unit test, so the scope of what needed documenting was unambiguous (Triggering Context 5/5).

Documentation Confidence Assessment

Overall Confidence: 3/5

Dimension Score Rationale
Code Comprehension of the Documented Surface 4/5 Small Python CDK connector; every documented behavior comes from spec.json, client.py, destination.py, and the unit tests, which I read in full.
API Documentation Quality 3/5 The "API" is the SFTP protocol itself; the connector's only external doc link is the IETF SFTP draft, which says nothing about server-specific permissions or auth policies.
Change Scope & Risk 3/5 Roughly 60 lines changed: one factual correction plus new setup, output, and troubleshooting sections.
Existing Doc Maturity 2/5 The prose portion of the previous doc was under 40 lines and contained a config field that doesn't exist.
Connector Sensitivity 5/5 Community connector, ql 100 / sl 100, alpha release stage.
Triggering Context 5/5 Triggered by a small merged bug-fix PR with an accompanying unit test.
Inference Ratio 4/5 Config, file naming, sync-mode, and escaping behavior are code-verified; the troubleshooting causes are reasoned from the code paths rather than reproduced against a live server.

Adjustments based on code comprehension

Baseline for Python CDK is 3.

  • Declarative surface area: upward — the setup table is derived field-by-field from destination_sftp_json/spec.json, including which fields are required and the port default of 22.
  • Test coverage of the documented claim: upwardunit_tests/unit_test.py::test_get_uri_escapes_reserved_characters asserts that a password containing # round-trips through smart_open.ssh.parse_uri, which is the claim in the new troubleshooting entry.
  • Independent corroboration: upward — file naming appears in both SftpClient._get_path and test_get_path.
  • Calibrated self-assessment: downward — I did not run the connector against a live SFTP server, so I cannot state exactly which error message each failure mode produces. The troubleshooting section therefore describes causes and fixes, not error strings.

What I Verified vs. What I Inferred

  • Verified from code:
    • There is no filename config field. The only fields are host, port, username, password, and destination_path (spec.json).
    • Output paths are {destination_path}/airbyte_json_{stream}.jsonl (SftpClient._get_path), so the file name derives from the stream name only.
    • Records are written as bare source data with no _airbyte_* metadata columns (SftpClient.write serializes record.data).
    • Overwrite mode deletes the stream's file before writing (DestinationSftpJson.write); append modes open with mode a+.
    • check writes then deletes a file under destination_path, so the account needs write and delete permission there (DestinationSftpJson.check).
    • Nothing in the connector creates directories, so destination_path must already exist.
    • Authentication is password-based; the spec has no private key field, and paramiko.SSHClient.connect is called with look_for_keys=False.
    • 0.2.16 percent-encodes username, password, host, and path before building the SFTP URI (SftpClient._get_uri, #78111).
    • Supported sync modes match supported_destination_sync_modes and supportsIncremental in spec.json.
  • Verified from API docs: nothing specific. SFTP is a protocol rather than a documented vendor API, and the connector's single externalDocumentationUrls entry is the IETF draft.
  • Inferred:
    • That a failing connection check with valid credentials is most often a missing or non-writable destination_path. This is the code's behavior, but I did not reproduce it.
    • That partial-sync records remaining in append mode matter to users enough to call out. The behavior itself is code-verified; the advice to deduplicate downstream is editorial.

Areas of Concern

  • The old doc's filename example has been there since the connector was added in 2022. If some deployment path does surface a filename field that I couldn't find in the spec, my correction would be wrong — worth a second pair of eyes.
  • The changelog date for 0.2.16 was 2026-05-15; I changed it to 2026-08-13, the date #78111 merged to master.

What

Fixes and expands the destination-sftp-json documentation. The main correctness problem: the "Getting Started" section told users to configure a filename field and predicted output at {destination_path}/{filename}.jsonl. No such field exists in the connector's spec, and the connector names files after the stream. Anyone following the old example would look for a file that never appears.

The doc also had no configuration reference, no troubleshooting, and nothing about how overwrite versus append behaves — and the 0.2.16 changelog entry carried the date the version was bumped rather than the date it merged.

How

Corrections

  • Removed the filename field and its example. Replaced with the actual naming scheme, {destination_path}/airbyte_json_{stream_name}.jsonl, and a worked example.
  • Changed the 0.2.16 changelog date from 2026-05-15 to 2026-08-13 (the merge date of fix(destination-sftp-json): escape SFTP URI components #78111) and made the entry describe the user-visible effect rather than only the internal change.

Additions

  • Prerequisites: password authentication only, and destination_path must already exist.
  • Setup: a table of the five config fields with required-ness and the port default, plus a note that the connection check needs write and delete permission.
  • Output files: naming, one JSON object per line, no _airbyte_* metadata columns, and the namespace collision that follows from naming files after streams alone.
  • Sync mode behavior: what overwrite deletes and what append leaves behind.
  • Troubleshooting: password-auth-only servers, missing or read-only destination directories, and the URI-reserved-character fix in 0.2.16.

Removals and restructuring

  • Dropped the nested "Overview → Sync Overview → Output schema" heading stack in favor of flat sections that match how a reader looks things up.
  • Folded the standalone "Namespace support" section into "Output files", where the consequence is concrete.

Review guide

  • docs/integrations/destinations/sftp-json.md lines 5–23 — prerequisites and the config table, checked against destination_sftp_json/spec.json.
  • Lines 25–37 — file naming, checked against SftpClient._get_path.
  • Lines 49 and 55–71 — sync mode behavior and troubleshooting, the most inference-heavy prose in the PR.
  • Line 74 area — the 0.2.16 changelog date correction.

User Impact

Users configuring this destination stop looking for a filename field that doesn't exist, and can find the output path, required permissions, and the password-special-character fix without reading the source.

Can this PR be safely reverted and rolled back?

  • YES 💚
  • NO ❌

Note: I am an AI assistant (Devin) and have proposed these documentation updates based on a review of the connector source code and its unit tests. Reviewers may merge, modify, or close this PR as they see fit.

Requested automatically by the autodoc workflow (GitHub CI service account) after commit 389b32d merged to master.


Devin session

devin-ai-integration Bot and others added 2 commits August 13, 2026 23:21
…leshooting

Co-Authored-By: bot_apk <apk@cognition.ai>
Co-Authored-By: bot_apk <apk@cognition.ai>
@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment, CI, and merge conflict monitoring

@github-actions

Copy link
Copy Markdown
Contributor

👋 Greetings, Airbyte Team Member!

Here are some helpful tips and reminders for your convenience.

💡 Show Tips and Tricks

PR Slash Commands

Airbyte Maintainers (that's you!) can execute the following slash commands on your PR:

  • 🛠️ Quick Fixes
    • /format-fix - Fixes most formatting issues.
    • /bump-version - Bumps connector versions, scraping changelog description from the PR title.
      • Bump types: patch (default), minor, major, major_rc, rc, promote.
      • The rc type is a smart default: applies minor_rc if stable, or bumps the RC number if already RC.
      • The promote type strips the RC suffix to finalize a release.
      • Example: /bump-version type=rc or /bump-version type=minor
    • /bump-progressive-rollout-version - Alias for /bump-version type=rc. Bumps with an RC suffix and enables progressive rollout.
  • ❇️ AI Testing and Review (internal link: AI-SDLC Docs):
    • /ai-prove-fix - Runs prerelease readiness checks, including testing against customer connections.
    • /ai-canary-prerelease - Rolls out prerelease to 5-10 connections for canary testing.
    • /ai-review - AI-powered PR review for connector safety and quality gates.
  • 📝 AI Documentation:
    • /ai-docs-review - AI-powered documentation review for PRs with connector changes.
    • /ai-create-docs-pr - Creates a documentation PR for connector changes, stacked on the current PR.
  • 🚀 Connector Releases:
    • /publish-connectors-prerelease - Publishes pre-release connector builds (tagged as {version}-preview.{git-sha}) for all modified connectors in the PR.
    • /enable-autopilot-rollouts - Enables autopilot progressive rollouts for the modified connector(s) in the PR, remediating "autopilot rollouts not enabled for {connector-name}" auto-merge blockers. Sets defaultRolloutMode: autopilot and enableProgressiveRollout: true, preserving any existing autopilotConfig.
      • Optional args: connector=<CONNECTOR_NAME> (defaults to the modified connectors in the PR), strategy=fast|slow|default (defaults to fast).
      • Example: /enable-autopilot-rollouts or /enable-autopilot-rollouts connector=source-faker strategy=slow
  • ☕️ JVM connectors:
    • /update-connector-cdk-version connector=<CONNECTOR_NAME> - Updates the specified connector to the latest CDK version.
      Example: /update-connector-cdk-version connector=destination-bigquery
  • 🐍 Python connectors:
    • /poe connector source-example lock - Run the Poe lock task on the source-example connector, committing the results back to the branch.
    • /poe source example lock - Alias for /poe connector source-example lock.
    • /poe source example use-cdk-branch my/branch - Pin the source-example CDK reference to the branch name specified.
    • /poe source example use-cdk-latest - Update the source-example CDK dependency to the latest available version.
  • ⚙️ Admin commands:
    • /force-merge reason="<REASON>" - Force merges the PR using admin privileges, bypassing CI checks. Requires a reason.
      Example: /force-merge reason="CI is flaky, tests pass locally"
📚 Show Repo Guidance

Helpful Resources

📝 Edit this welcome message.

@github-actions

Copy link
Copy Markdown
Contributor

Deploy preview for airbyte-docs ready!

Project:airbyte-docs
Status: ✅  Deploy successful!
Preview URL:https://airbyte-docs-p6vdw1jd5-airbyte-growth.vercel.app
Latest Commit:09a4b29

Deployed with vercel-action

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/documentation Improvements or additions to documentation team/documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

0 participants