How GenAI Gateway works

The GenAI Gateway routes requests from Knime workflows to large language models (LLMs) and embedding models hosted by external providers, such as OpenAI or Azure OpenAI. Knime does not host these models, the gateway only manages the connection to the provider you configure.

Registering a model with the GenAI Gateway has several advantages:

• Register a model once and use it across workflows and users.
• Centralize governance over which models are available and who can access them.
• Share access at the team level instead of managing individual credentials.

How models are used

Models created on the GenAI gateway are accessible in workflows using the Knime LLM Selector and the Knime Embedding Model Selector. The Knime AI Model Lister node displays all available models the authenticated user has access to.

All nodes use the KNIME Authenticator to authenticate with the current user or with another user.

After authenticating and selecting a model from the available ones, you can connect it to downstream nodes such as the LLM Prompter or the Agent Chat Widget nodes.

Model scope

Models can be created as:

• Team GenAI models: created by team admins and tied to a specific team.
• Global GenAI models: created by global admins and not tied to any team.

Model access is further split into two levels:

• Can use: use the model in a workflow.
• Can edit: use the model, and also edit or delete it.

By default:

• Team members have only use access to team models.
• Team admins have edit access to models belonging to their team.
• Global admins have edit access to all global models.

Manage model access

Only team and global admins can create models and manage access. Model access is different for team and global models.

Team models

• can be shared with individual members of a team
• can be shared with another team, but not with individual members of another team.

Global models

• can be shared with teams, not with individual users.