workflow: refactor InferenceClient endpoints logic by hannahblair · Pull Request #13683 · gradio-app/gradio

hannahblair · 2026-07-29T09:27:47Z

The model-node schema table in workflow.py was hardcoded, so every time huggingface_hub added a new task method, we would have to manually add a new entry mirroring the signature.

Changes in this PR:

Replaces the hardcoded dict with runtime introspection. Iterate dir(InferenceClient), run inspect.signature() on each method, derive port shapes from param names + annotations. Any model on the Hub now works with no code change here.
Vision tasks (image-text-to-text, visual-question-answering, document-question-answering, image-to-text) all route through chat_completion instead of the InferenceClient methods. This extends Workflow UX improvements, and fix gradio skills add clobbering skills via symlinked dirs #13666's fix (which only covered image-text-to-text) to the other three vision tags
Added a "+ Add param" button on model nodes, users type a name, pick a type, get a port. It becomes an extra kwarg via extra_body in InferenceClient. Ports are marked custom: true in the workflow JSON so they survive schema refreshes and endpoint switches, and render at the top of the node with the required params for visiblity

Note that there's still a lot of other data to hardcode as a result, because introspection doesn't get everything we need. Here's Claude's explanation of each datatype:

	What introspection can't answer
`_PORT_TYPE_BY_PARAM`	HF types `image: str \| Path \| bytes` — that annotation doesn't say "this string is an image URL." Python typing can't express modality.
`_MEDIA_TOKEN_TO_TYPE`	Substring safety net for the above — if HF renames `image` to `img`, we still detect it.
`_OUTPUT_SUFFIX_TO_TYPE`	Return types are `bytes` or `Image` — introspection can't tell "audio bytes" from "image bytes". Method-name suffix is the reliable signal.
`_MEDIA_EXT`	Which file extension to save binary output as. Not derivable from any Python type.
`_JSON_OUTPUT_METHODS`	Return types are `list[ClassificationOutputElement]` etc. — we'd need semantic knowledge to know these serialize as structured JSON vs plain text.
`_OUTPUT_LABELS`	"Detections" vs "Segments" vs "Labels" — subjective UX distinction, not in method metadata.
`_SKIP_PARAMS`	HF doesn't mark params as "user-facing" vs "internal knob". Someone has to curate.
`_NON_TASK_METHODS`	`InferenceClient` has helpers like `.close()`, `.post()`, `.health_check()` that aren't inference tasks. HF doesn't tag them.
`_PIPELINE_TAG_ALIASES`	HF's pipeline_tag → provider capability mapping doesn't exist as an API (confirmed by HF). Vision tags all route through `chat_completion` because no provider serves the task-specific endpoints.

Replace the two hardcoded task tables (`_INFERENCE_ENDPOINT_SCHEMAS`, `_PIPELINE_TAG_TO_ENDPOINT`) with runtime introspection of the `InferenceClient` methods. Adds ~50 lines of discovery, removes ~230 of per-task schemas — net −54 lines — and picks up 3 endpoints that were missing from the hardcoded table (`audio_to_audio`, `table_question_answering`, `zero_shot_image_classification`). Endpoint schemas now derive from method signatures: - port `id` = param name - port `label` = titleized param name - port `type` = name lookup (`image`/`audio`/`video`) or annotation (`int`/`float` → number, `bool` → boolean, else text). Unwraps `Optional[X]` so numeric knobs with a default aren't misclassified as text - port `required` = whether the param has a default - output `type` = derived from method name (`text_to_image` → image, `*_classification` → json, else text) Pipeline-tag → endpoint resolution uses `pipeline_tag.replace("-", "_")` with a small alias table for tags that don't match that convention (`text2text-generation`, `conversational`, `text-to-audio`, and the four vision tags that all route to `chat_completion`). `chat_completion` stays as a synthetic endpoint (hand-crafted `image + prompt → text` shape) merged in after auto-discovery, since it doesn't correspond 1:1 to any InferenceClient method signature. The streaming dispatch, `max_tokens=16384`, `/gradio_api/file=` handling and reasoning-aware error messages added in #13666 are preserved. Also broadens vision routing beyond `image-text-to-text` to also cover `visual-question-answering`, `document-question-answering`, and `image-to-text` — none of these are served as task-specific endpoints by any Inference Provider, so all four now route through `chat_completion` for consistency. Zero-day behavior: when huggingface_hub ships a new task method, adds a param, or changes a param's default, the schema updates automatically with no code change here. New pipeline_tags that don't match any method fall through to the raw inference API path (server-side dispatch). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

gradio-pr-bot · 2026-07-29T09:28:21Z

🪼 branch checks and previews

•	Name	Status	URL
	Spaces	ready!	Spaces preview
	Website	ready!	Website preview
	Storybook	ready!	Storybook preview
🦄	Changes	detected!	Details

Install Gradio from this PR

pip install https://huggingface.co/buckets/gradio/pypi-previews/resolve/f3f0e2ad768a115f0f9a1975263a1a23288a9e0f/gradio-6.22.0-py3-none-any.whl

Install Gradio Python Client from this PR

pip install "gradio-client @ git+https://github.com/gradio-app/gradio@f3f0e2ad768a115f0f9a1975263a1a23288a9e0f#subdirectory=client/python"

Import Gradio JS Client from this PR via CDN

import { Client } from "https://huggingface.co/buckets/gradio/npm-previews/resolve/f3f0e2ad768a115f0f9a1975263a1a23288a9e0f/browser.js";

Resolves conflicts in: - workflow-store.ts: wrap switch_endpoint return in reconcileComponentRoles (main's new role-reconciliation logic) while keeping custom-port preservation. Also wrap remove_custom_port for consistency. - WorkflowNodeSF.svelte: merge add_custom_port/remove_custom_port imports with main's new setNodeSize import. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Copilot

Pull request overview

This PR refactors workflow model-endpoint handling by auto-discovering huggingface_hub.InferenceClient task methods (instead of maintaining a static schema), and extends the workflow canvas to support user-added “custom param” input ports that are forwarded to the backend as kwargs / extra_body.

Changes:

Backend: introspect InferenceClient signatures to generate endpoint schemas at runtime, and route custom-port extras into extra_body when supported.
Frontend: add UI + store actions for adding/removing custom input ports, and serialize model calls as dict args when custom ports are present.
Tests: add schema “drift” coverage to detect breaking changes across huggingface_hub upgrades.

Reviewed changes

Copilot reviewed 10 out of 10 changed files in this pull request and generated 7 comments.

Show a summary per file

File	Description
`gradio/workflow.py`	Replaces static endpoint schemas with runtime introspection; adds tag→endpoint resolution and custom-kwarg handling.
`test/test_workflow.py`	Adds endpoint schema stability test and extends coverage for new backend behavior.
`js/workflowcanvas/workflow/workflow-executor.ts`	Sends dict args for model calls when custom ports exist.
`js/workflowcanvas/workflow/workflow-store.ts`	Preserves custom ports across schema refresh/endpoint switch; adds add/remove custom port actions.
`js/workflowcanvas/workflow/WorkflowNodeSF.svelte`	Adds “+ Add param” UI, custom-port ordering/visibility, and removal controls.
`js/workflowcanvas/workflow/workflow-types.ts`	Extends `Port` with a `custom?: boolean` flag.
`js/workflowcanvas/workflow/workflow-store.test.ts`	Adds tests for custom port add/remove and preservation across refresh/switch.
`js/workflowcanvas/workflow/model-api.ts`	Updates pipeline-tag routing to align more vision-language tags to chat completion.
`gradio/workflow_port_types.py`	Adds a new (currently unused) port-type mapping module.
`.changeset/auto-endpoints-discover.md`	Adds a manual changeset entry for the workflow refactor.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

gradio-pr-bot · 2026-07-31T16:28:51Z

🦄 change detected

This Pull Request includes changes to the following packages.

Package	Version
`@gradio/workflowcanvas`	`minor`
`gradio`	`minor`

workflow: refactor InferenceClient endpoints logic

Something isn't right?

Maintainers can change the version label to modify the version bump.
If the bot has failed to detect any changes, or if this pull request needs to update multiple packages to different versions or requires a more comprehensive changelog entry, maintainers can update the changelog file directly.

…o into workflow-auto-discovery

abidlabs · 2026-07-31T22:16:14Z

Amazing, I'll give this a spin @hannahblair! Perhaps we can add one or two more Workflow demos in the demo folder that'll make it easier to test this & also guide users/llms on this?

abidlabs · 2026-08-04T00:51:02Z

+    "automatic_speech_recognition": "Transcript",
+}
+
+_SKIP_PARAMS: frozenset[str] = frozenset(


This seems fragile (if new parameters are added, this will not work anymore), feels like there might be a better way?

abidlabs · 2026-08-04T00:51:09Z

+    }
+)
+
+_NON_TASK_METHODS: frozenset[str] = frozenset(


Same for this

abidlabs · 2026-08-04T00:53:18Z

I'm not sure if this is a good idea @hannahblair, it seems like this is more fragile than before -- any changes to InferenceClient or its methods could break this. I actually think hardcoding for all of the different tasks is probably more robust since parameter name are unlikely to change for those tasks. Perhaps I'm missing something?

hannahblair · 2026-08-05T16:23:40Z

ok good to know, i'll defer to your judgement! two things worth flagging for the record: new tasks would still need to be added manually. and introspection derives required flags from the signatures - I'm not certain the hardcoded schemas are fully correct right now. the main risk the other way is hf_hub renaming a param and silently breaking things, but i imagine that doesn't happen often. that said, the introspection may buy less than it costs

i've opened a smaller PR with just the task improvements on top of the existing hardcoded schemas #13710

hannahblair changed the title ~~Auto-discover InferenceClient endpoints in gr.Workflow~~ workflow: refactor InferentClient endpoints logic Jul 29, 2026

hannahblair mentioned this pull request Jul 29, 2026

Let users add custom params to model nodes #13684

Closed

hannahblair changed the title ~~workflow: refactor InferentClient endpoints logic~~ workflow: refactor InferenceClient endpoints logic Jul 29, 2026

hannahblair marked this pull request as draft July 29, 2026 16:46

hannahblair and others added 3 commits July 31, 2026 15:50

refactor

00ff19a

format

9fc6659

hannahblair requested a review from Copilot July 31, 2026 15:33

Copilot started reviewing on behalf of hannahblair July 31, 2026 15:34 View session

Copilot AI reviewed Jul 31, 2026

View reviewed changes

hannahblair and others added 3 commits July 31, 2026 16:43

lint

298843f

address comments

7bc16dc

add changeset

437ac41

hannahblair added 4 commits July 31, 2026 17:40

format

4095841

Merge branch 'workflow-auto-discovery' of github.com:gradio-app/gradi…

82365d5

…o into workflow-auto-discovery

revert

9d59006

fix test

cdaaf47

hannahblair assigned abidlabs Jul 31, 2026

hannahblair marked this pull request as ready for review July 31, 2026 22:08

hannahblair commented Jul 31, 2026

View reviewed changes

Comment thread demo/workflow/workflow.json Outdated

hannahblair added 2 commits August 1, 2026 00:19

add vision model chat workflow

67138f8

Merge branch 'main' into workflow-auto-discovery

f3f0e2a

abidlabs reviewed Aug 4, 2026

View reviewed changes

Comment thread gradio/workflow.py

}

)

_NON_TASK_METHODS: frozenset[str] = frozenset(

abidlabs Aug 4, 2026

Copy link
Copy Markdown

Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same for this

hannahblair closed this Aug 5, 2026

Navigation Menu

Uh oh!

workflow: refactor InferenceClient endpoints logic - #13683

workflow: refactor InferenceClient endpoints logic#13683
hannahblair wants to merge 13 commits into
mainfrom
workflow-auto-discovery

hannahblair commented Jul 29, 2026 •

edited

Loading

Uh oh!

gradio-pr-bot commented Jul 29, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

gradio-pr-bot commented Jul 31, 2026

Something isn't right?

Uh oh!

abidlabs commented Jul 31, 2026

Uh oh!

Uh oh!

abidlabs Aug 4, 2026

Uh oh!

abidlabs Aug 4, 2026

Uh oh!

abidlabs commented Aug 4, 2026

Uh oh!

hannahblair commented Aug 5, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Uh oh!

Conversation

hannahblair commented Jul 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

gradio-pr-bot commented Jul 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🪼 branch checks and previews

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

gradio-pr-bot commented Jul 31, 2026

🦄 change detected

This Pull Request includes changes to the following packages.

Something isn't right?

Uh oh!

abidlabs commented Jul 31, 2026

Uh oh!

Uh oh!

abidlabs Aug 4, 2026

Choose a reason for hiding this comment

Uh oh!

abidlabs Aug 4, 2026

Choose a reason for hiding this comment

Uh oh!

abidlabs commented Aug 4, 2026

Uh oh!

hannahblair commented Aug 5, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

hannahblair commented Jul 29, 2026 •

edited

Loading

gradio-pr-bot commented Jul 29, 2026 •

edited

Loading