KNIME Documentation

Prompt a local model via GPT4All ​

Run large language models locally in KNIME using GPT4All, without relying on external APIs or internet connectivity.

When this is useful ​

Prompt a local model when you want to:

  • Run LLMs fully offline
  • Avoid sending data to external providers
  • Work with privacy-sensitive or regulated data
  • Eliminate usage-based API costs

If you want to use hosted models (e.g., OpenAI, Anthropic, Gemini), see Prompt a model instead.

Prompt a local model via GPT4All step by step ​

Prerequisites

Before you start, make sure that you have:

1. Install GPT4All ​

Download and install GPT4All on your machine.

GPT4All runs entirely on your local hardware and does not require an internet connection after installation.

2. Download a model using GPT4All ​

Use the GPT4All application to download a compatible local model.

  1. Download and install GPT4All from https://gpt4all.io/index.html or on your machine using pip:

    bash
    pip install gpt4all

  2. Open the GPT4All application and download a model from the built-in model list

  3. Note the file location where the model is stored

The model file remains local and can be reused across workflows.

3. Connect to the local model in KNIME ​

Use the Local GPT4All LLM Connector node to load the model.

In the node configuration:

  • Point to the model file downloaded by GPT4All
  • Select the hardware backend:
    • cpu (default)
    • gpu (if available)
    • amd, nvidia, or intel for vendor-specific GPUs

Selecting a GPU can improve performance for larger models, but is optional.

No authentication required

Local GPT4All models run entirely on your machine. No API keys or Authenticator nodes are needed.

4. Select the local model ​

After configuring the connector, use the corresponding LLM Selector node to select the local model.

5. Send a prompt ​

Create your prompt as usual using a text column or an expression.

Then use the LLM Prompter node to send the prompt to the local model.

Each input row is processed independently.

Local model behavior

Local models often require more explicit prompting and may produce shorter or less consistent outputs than large hosted models.

Result ​

The model generates a response entirely on your local machine.

You can download and explore an example workflow on KNIME Community Hub: Prompt a local model via GPT4All.

Next steps ​