Running against pre-installed hugging face models

[AhmedHawater2003]

Hello,

First of all, thanks for the great tool!

Is it possible to run the tool against a pre-installed model that I have installed from hugging face (e.g. gemma-3-4B-it-qat-GGUF)? Because from my understanding, the plugins are installed against specific models and these models need to be installed separately.

Just one more note for additional context: I am using Windows along with WSL2 and I have installed the gemma-3-4B-it-qat-GGUF model from LM studio.

Thanks in advance!

[Muguett-DBY]

Yes — if the GGUF is already available in LM Studio, you can have LM Studio serve it and point LLM at its OpenAI-compatible API. You don't need a second model download or a model-specific LLM plugin for this route.

1. Start the server in LM Studio's Developer tab and make the model available there. From the environment where you run llm, check the server and copy the model's id from the response:

   curl -fsS http://127.0.0.1:1234/v1/models

2. Run dirname "$(llm logs path)" to find LLM's data directory. Create or edit extra-openai-models.yaml there; if it already has entries, add this to the existing YAML list:

   - model_id: gemma-local
     model_name: "PASTE-THE-MODEL-ID-FROM-V1-MODELS"
     api_base: "http://127.0.0.1:1234/v1"

3. Check that LLM registered the model, then send a prompt:

   llm models --options -m gemma-local
   llm -m gemma-local "Hello!"

The address above assumes LLM can reach LM Studio on 127.0.0.1:1234. If LM Studio runs on Windows and LLM runs inside WSL2, that works with mirrored networking; with the default NAT setup you may need the Windows host IP instead, and the server must accept that connection. Microsoft's WSL networking guide explains both modes. If you enable network access in LM Studio, check its authentication and firewall settings too, since that can expose the server beyond localhost.

Sources: LM Studio's server guide, its OpenAI-compatible endpoints, and LLM's custom model configuration. I checked the current documentation and LLM source, but haven't run this against your LM Studio/WSL setup; the exact model ID and reachable address depend on your configuration.