Skip to content

docs(frontend): document local FunASR transcription - #13667

Closed
LauraGPT wants to merge 5 commits into
Significant-Gravitas:devfrom
LauraGPT:codex/funasr-transcription-docs
Closed

docs(frontend): document local FunASR transcription#13667
LauraGPT wants to merge 5 commits into
Significant-Gravitas:devfrom
LauraGPT:codex/funasr-transcription-docs

Conversation

@LauraGPT

@LauraGPT LauraGPT commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

Why / What / How

Why: #13500 added custom OpenAI-compatible transcription endpoints, but the new environment variables were not discoverable in the default configuration or self-hosting docs. Users who want private AutoPilot voice input therefore had no documented path to FunASR/SenseVoice.

What: This PR exposes the three optional TRANSCRIPTION_* settings in the frontend environment template and adds a copy-paste FunASR setup to the existing self-hosted AutoPilot guide.

How: The guide starts the official funasr-server OpenAI-compatible endpoint, configures the server-side frontend route, distinguishes Docker Desktop, native Linux, and non-Docker addresses, documents container recreation, and calls out authentication/network safety. Empty settings preserve the existing OpenAI whisper-1 fallback.

Follow-up to #13500 and #13347.

Changes 🏗️

  • Add TRANSCRIPTION_API_BASE_URL, TRANSCRIPTION_MODEL, and TRANSCRIPTION_API_KEY to frontend/.env.default as optional empty settings.
  • Document CPU and CUDA FunASR/SenseVoice startup commands and a direct transcription smoke test.
  • Document Docker networking, frontend container recreation, optional bearer authentication, and the unchanged OpenAI fallback.

Current dev refresh

  • Merged exact dev@c29ca86f885ae092fd8c2972088864e2d5b41ff2 into signed+DCO head 7b623d4fac8d1514f0bb1d4f340fc69f478d7409; the only conflict was the adjacent .env.default block, resolved by retaining both the optional transcription variables and upstream's brain-dump feature flag guidance.
  • The PR remains a focused three-file diff (+81/-2) with all review threads resolved.
  • Validation uses the repository-required Node 24.18.0 and pinned pnpm 10.20.0.

Checklist 📋

For code changes:

  • I have clearly listed my changes in the PR description
  • I have made a test plan
  • I have tested my changes according to the test plan:
    • pnpm lint (ESLint and Prettier checks pass)
    • pnpm generate:api (no tracked changes)
    • pnpm types
    • pnpm exec vitest run src/app/api/transcribe/__tests__/route.test.ts (9 passed)
    • pnpm exec vitest run --coverage --maxWorkers=50% (458 files; 4,805 passed, 2 skipped)
    • git diff --check
    • Verify both FunASR documentation links return HTTP 200

For configuration changes:

  • .env.default is updated or already compatible with my changes
  • docker-compose.yml is updated or already compatible with my changes
  • I have included a list of my configuration changes in the PR description (under Changes)

The existing frontend env_file order already loads .env.default and then the optional .env override, so no Compose change is required.

@LauraGPT
LauraGPT requested a review from a team as a code owner July 23, 2026 21:15
@LauraGPT
LauraGPT requested review from Bentlybro and ntindle and removed request for a team July 23, 2026 21:15
@github-project-automation github-project-automation Bot moved this to 🆕 Needs initial review in AutoGPT development kanban Jul 23, 2026
@CLAassistant

CLAassistant commented Jul 23, 2026

Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

@github-actions github-actions Bot added documentation Improvements or additions to documentation platform/frontend AutoGPT Platform - Front end labels Jul 23, 2026
@coderabbitai

coderabbitai Bot commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review

Walkthrough

The frontend environment template adds optional transcription settings, the route test validates the sensevoice model, and documentation covers separate voice transcription configuration, self-hosted FunASR setup, environment propagation, network considerations, and OpenAI fallback behavior.

Changes

Voice transcription configuration

Layer / File(s) Summary
Transcription configuration and validation
autogpt_platform/frontend/.env.default, autogpt_platform/frontend/src/app/api/transcribe/__tests__/route.test.ts
Adds optional transcription variables and updates the route test to use sensevoice.
FunASR setup and fallback guide
docs/platform/copilot-local-llm.md
Documents FunASR installation, endpoint verification, environment configuration, frontend container recreation, network exposure, authentication, and OpenAI fallback behavior.

Estimated code review effort: 1 (Trivial) | ~5 minutes

Possibly related PRs

Suggested reviewers: ntindle, bentlybro

Poem

A rabbit checks the voice config bright,
FunASR starts with settings right.
Sensevoice keeps the test in tune,
Whisper waits beneath the moon.

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Title check ✅ Passed The title clearly summarizes the primary change: documenting local FunASR transcription configuration in the frontend.
Description check ✅ Passed The description directly explains the transcription configuration, documentation, environment changes, testing, and fallback behavior.
✨ Finishing Touches 💡 1
🛠️ Fix failing CI checks 💡
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@LauraGPT
LauraGPT force-pushed the codex/funasr-transcription-docs branch from d4157fc to 459b7c9 Compare July 23, 2026 21:19

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@docs/platform/copilot-local-llm.md`:
- Around line 145-148: Update the documentation around the OpenAI fallback to
reflect route precedence: when TRANSCRIPTION_API_BASE_URL is unset, the route
uses OPENAI_API_BASE_URL, while OPENAI_API_KEY is applied only for the default
OpenAI endpoint. Clarify that a custom OPENAI_API_BASE_URL still receives
whisper-1 requests without bearer authentication, or adjust the documented
fallback behavior accordingly.
- Line 98: Update the documented FunASR installation command to install funasr
together with the server dependencies fastapi, uvicorn, and python-multipart, so
fresh environments can run funasr-server and support multipart uploads.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: aaf50735-fd7d-4036-ac32-127b0b9de122

📥 Commits

Reviewing files that changed from the base of the PR and between 0b5fcf1 and d4157fc.

📒 Files selected for processing (2)
  • autogpt_platform/frontend/.env.default
  • docs/platform/copilot-local-llm.md
📜 Review details
⏰ Context from checks skipped due to timeout. (7)
  • GitHub Check: check API types
  • GitHub Check: integration_test
  • GitHub Check: lint
  • GitHub Check: Analyze (python)
  • GitHub Check: Analyze (typescript)
  • GitHub Check: Check PR Status
  • GitHub Check: end-to-end tests
⚠️ CI failures not shown inline (1)

Commit Status: Vercel: Vercel

Conclusion: failure

Authorization required to deploy.
🧰 Additional context used
📓 Path-based instructions (2)
autogpt_platform/frontend/.env*

📄 CodeRabbit inference engine (autogpt_platform/AGENTS.md)

Frontend environment configuration: frontend/.env.default provides defaults (tracked in git), frontend/.env provides user overrides (gitignored)

Files:

  • autogpt_platform/frontend/.env.default
autogpt_platform/**/.env*

📄 CodeRabbit inference engine (autogpt_platform/AGENTS.md)

Platform environment configuration: .env.default provides Supabase/shared defaults (tracked in git), .env provides user overrides (gitignored)

Files:

  • autogpt_platform/frontend/.env.default
🪛 dotenv-linter (4.0.0)
autogpt_platform/frontend/.env.default

[warning] 50-50: [UnorderedKey] The TRANSCRIPTION_API_KEY key should go before the TRANSCRIPTION_MODEL key

(UnorderedKey)

🔇 Additional comments (2)
autogpt_platform/frontend/.env.default (1)

46-50: LGTM!

docs/platform/copilot-local-llm.md (1)

8-9: LGTM!

Comment thread docs/platform/copilot-local-llm.md Outdated
Comment thread docs/platform/copilot-local-llm.md Outdated
Comment thread docs/platform/copilot-local-llm.md Outdated

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
autogpt_platform/frontend/.env.default (1)

48-50: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Reorder the transcription keys to satisfy dotenv-linter.

Place TRANSCRIPTION_API_KEY before TRANSCRIPTION_MODEL.

Proposed fix
 TRANSCRIPTION_API_BASE_URL=
-TRANSCRIPTION_MODEL=
 TRANSCRIPTION_API_KEY=
+TRANSCRIPTION_MODEL=
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@autogpt_platform/frontend/.env.default` around lines 48 - 50, Reorder the
transcription environment variables so TRANSCRIPTION_API_KEY appears before
TRANSCRIPTION_MODEL, while keeping TRANSCRIPTION_API_BASE_URL and the existing
values unchanged.

Source: Linters/SAST tools

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@autogpt_platform/frontend/.env.default`:
- Around line 48-50: Reorder the transcription environment variables so
TRANSCRIPTION_API_KEY appears before TRANSCRIPTION_MODEL, while keeping
TRANSCRIPTION_API_BASE_URL and the existing values unchanged.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 4513ceba-c9da-4a1b-a49c-efea56d7fd86

📥 Commits

Reviewing files that changed from the base of the PR and between d4157fc and 459b7c9.

📒 Files selected for processing (2)
  • autogpt_platform/frontend/.env.default
  • docs/platform/copilot-local-llm.md
🚧 Files skipped from review as they are similar to previous changes (1)
  • docs/platform/copilot-local-llm.md
📜 Review details
⏰ Context from checks skipped due to timeout. (8)
  • GitHub Check: check API types
  • GitHub Check: lint
  • GitHub Check: integration_test
  • GitHub Check: end-to-end tests
  • GitHub Check: Seer Code Review
  • GitHub Check: Analyze (python)
  • GitHub Check: Analyze (typescript)
  • GitHub Check: Check PR Status
⚠️ CI failures not shown inline (1)

Commit Status: Vercel: Vercel

Conclusion: failure

Authorization required to deploy.
🧰 Additional context used
📓 Path-based instructions (2)
autogpt_platform/frontend/.env*

📄 CodeRabbit inference engine (autogpt_platform/AGENTS.md)

Frontend environment configuration: frontend/.env.default provides defaults (tracked in git), frontend/.env provides user overrides (gitignored)

Files:

  • autogpt_platform/frontend/.env.default
autogpt_platform/**/.env*

📄 CodeRabbit inference engine (autogpt_platform/AGENTS.md)

Platform environment configuration: .env.default provides Supabase/shared defaults (tracked in git), .env provides user overrides (gitignored)

Files:

  • autogpt_platform/frontend/.env.default
🪛 dotenv-linter (4.0.0)
autogpt_platform/frontend/.env.default

[warning] 50-50: [UnorderedKey] The TRANSCRIPTION_API_KEY key should go before the TRANSCRIPTION_MODEL key

(UnorderedKey)

@codecov

codecov Bot commented Jul 23, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 77.94%. Comparing base (c29ca86) to head (7b623d4).

Additional details and impacted files
@@            Coverage Diff             @@
##              dev   #13667      +/-   ##
==========================================
- Coverage   77.95%   77.94%   -0.02%     
==========================================
  Files        2932     2932              
  Lines      219804   219803       -1     
  Branches    20847    20785      -62     
==========================================
- Hits       171352   171320      -32     
- Misses      43851    43941      +90     
+ Partials     4601     4542      -59     
Flag Coverage Δ
platform-frontend 52.76% <ø> (-0.05%) ⬇️
platform-frontend-e2e 29.67% <ø> (-0.49%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components Coverage Δ
Platform Backend 83.91% <ø> (+<0.01%) ⬆️
Platform Frontend 55.80% <ø> (-0.10%) ⬇️
AutoGPT Libs ∅ <ø> (∅)
Classic AutoGPT 28.43% <ø> (ø)
🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Comment thread autogpt_platform/frontend/.env.default Outdated
Comment thread docs/platform/copilot-local-llm.md
@github-project-automation github-project-automation Bot moved this from 🆕 Needs initial review to 🚧 Needs work in AutoGPT development kanban Jul 24, 2026
@LauraGPT

LauraGPT commented Jul 26, 2026

Copy link
Copy Markdown
Contributor Author

@Pwuts, the branch is current again and the two requested changes remain present with all review threads resolved.

  • exact base: dev@c29ca86f885ae092fd8c2972088864e2d5b41ff2
  • signed+DCO head: 7b623d4fac8d1514f0bb1d4f340fc69f478d7409
  • scope: 3 files, +81/-2, 0 commits behind
  • focused route tests: 9/9 passed
  • complete frontend suite: 458 files, 4,805 passed, 2 skipped
  • pnpm types, Next lint, generated-API no-diff, changed-scope formatting, both documentation links, and git diff --check passed
  • official current-head matrix: 34 successful, 1 expected skip, 1 neutral; both amd64 and arm64 single-container build, smoke, and scan jobs passed, with zero repository-owned failures or pending checks

The only merge conflict was the adjacent .env.default block; the resolution retains both the optional TRANSCRIPTION_* variables and upstream's new brain-dump flag guidance. The remaining Vercel authorization and CLA statuses are account-level gates rather than source failures. Could you re-review the refreshed head when convenient?

Signed-off-by: zhifu gao <18321252+LauraGPT@users.noreply.github.com>
@github-actions github-actions Bot added the conflicts Automatically applied to PRs with merge conflicts label Aug 5, 2026
@github-actions

github-actions Bot commented Aug 5, 2026

Copy link
Copy Markdown
Contributor

This pull request has conflicts with the base branch, please resolve those so we can evaluate the pull request.

@github-actions github-actions Bot added the cla: pending CLA not yet signed by all contributors label Aug 6, 2026
@github-actions

github-actions Bot commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

👋 Friendly reminder: This PR is waiting on a signed CLA.

All contributors need to sign our Contributor License Agreement before we can merge this PR.

➡️ Sign the CLA here

Why do we need a CLA?

The CLA protects both you and the project by clarifying the terms under which your contribution is made. It's a one-time process — once signed, it covers all your future contributions.

Common issues
  • Email mismatch: Make sure your Git commit email matches your GitHub account email
  • Stale branch: Sync your branch with the current dev branch and push the updated branch normally
  • Multiple authors: All commit authors need to sign, not just the PR author

If you have questions, just ask! 🙂

Signed-off-by: zhifu gao <18321252+LauraGPT@users.noreply.github.com>
@github-actions

Copy link
Copy Markdown
Contributor

Conflicts have been resolved! 🎉 A maintainer will review the pull request shortly.

@github-actions github-actions Bot removed the conflicts Automatically applied to PRs with merge conflicts label Aug 11, 2026
@coderabbitai

coderabbitai Bot commented Aug 11, 2026

Copy link
Copy Markdown
Contributor

Note

GitHub couldn't provide a complete incremental comparison for this pull request, so CodeRabbit is performing a full review instead. This review may take a little longer.

@github-actions

Copy link
Copy Markdown
Contributor

⚠️ This PR will be automatically closed in 7 days if the CLA is not signed.

We haven't received a signed CLA from all contributors yet. Please sign it to keep this PR open:

➡️ Sign the CLA here

If you're unable to sign or have questions, please let us know — we're happy to help!

@github-actions

Copy link
Copy Markdown
Contributor

👋 Closing this PR due to unsigned CLA after 30 days.

Thank you for your contribution! If you'd still like to contribute:

  1. Sign the CLA
  2. Re-open this PR or create a new one

Maintainers: To reopen and exempt from future auto-close, add the cla: override label before reopening. Without it, the PR will not be re-closed automatically (a reopened PR is treated as a maintainer decision).

We appreciate your interest in AutoGPT and hope to see you back! 🚀

@github-actions github-actions Bot closed this Aug 23, 2026
@github-project-automation github-project-automation Bot moved this from 🚧 Needs work to ✅ Done in AutoGPT development kanban Aug 23, 2026
@github-project-automation github-project-automation Bot moved this to Done in Frontend Aug 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cla: pending CLA not yet signed by all contributors documentation Improvements or additions to documentation platform/frontend AutoGPT Platform - Front end size/m

Projects

Status: ✅ Done
Status: Done

Development

Successfully merging this pull request may close these issues.

3 participants