KNIME Documentation

Build a Retrieval-augmented Question Answering Workflow ​

In this tutorial, you build a complete retrieval-augmented generation (RAG) workflow in KNIME.

You will index a small set of product FAQs, retrieve relevant entries for a user question, and use them to generate grounded answers with a language model.

By the end of this tutorial, you will have a working RAG pipeline that you can adapt to your own data.

What you will build ​

You will create a workflow that:

  1. Loads a file containing product FAQs
  2. Selects and authenticates an embedding model
  3. Generates embeddings
  4. Stores embeddings in a vector store
  5. Retrieves relevant document chunks for a user question
  6. Injects retrieved context into a prompt
  7. Generates a grounded answer

Overview of the completed RAG workflow

Prerequisites ​

Before you start, make sure you have:

Step 1: Load and prepare the documents ​

  1. Download the sample dataset of product FAQs:
    product_FAQs.csv

  2. Use a CSV Reader node to load the file into your workflow.

Step 2: Authenticate and select an embedding model ​

To generate embeddings, authenticate with your LLM provider:

  1. Insert a Credentials Configuration node.
  2. Enter your API key.
  3. Connect it to the corresponding Authenticator node.

If the authenticator node traffic light turns green, the connection is successful.

Next, select an embedding model:

  1. Add an Embedding Model Selector node.
  2. Choose an embedding-capable model (for example, text-embedding-3-small).

Step 3: Generate embeddings ​

Convert each text chunk into an embedding vector.

  1. Connect the chunked text table to the Text Embedder node.
  2. Select the column containing the text.
  3. Execute the node.

The output table now contains a new column with embedding vectors. Each row contains a separate embedding vector.

Step 4: Create a vector store ​

Store the embeddings so they can be retrieved later.

  1. Insert a FAISS Vector Store Creator or Chroma Vector Store Creator node.
  2. Connect the table containing embeddings.
  3. Configure the node to use the embedding column.
  4. Use a Model Writer node to save the vector store.

Create and persist the vector store

Step 5: Ask a question and retrieve relevant documents ​

Now retrieve relevant content for a user question.

  1. Create a table containing a user question using the Table Creator node.
    The query can be any string; in this tutorial, a single-row table is sufficient.

  2. Load the vector store using a Model Reader node.

  3. Connect a Vector Store Retriever node to:

    • the table containing the user question, and
    • the loaded vector store.

  4. Configure the retriever to return the top k most similar document chunks.

The output is a table containing the document chunks that are most relevant to the user’s question.

Retrieve relevant documents from the vector store

Step 6: Build a prompt with retrieved context ​

Combine the retrieved text and the user question into a single prompt.

  1. Use an Expression node to create a new column called Prompt.
  2. In the prompt, clearly separate context from the question.

Example prompt structure:

text
Use the following context to answer the question.

Context:
{{retrieved_documents}}

Question:
{{user_question}}

This structure makes it explicit to the model which information it should use.

Inject retrieved context into the prompt

Step 7: Generate the answer ​

Send the prompt to the language model.

  1. Select a model using an LLM Selector node.

  2. Connect the model input port of the LLM Prompter to the LLM Selector.

  3. Connect the data input port to the table containing the Prompt column.

  4. In the LLM Prompter configuration, select Prompt as the input column.

  5. Execute the node.

The model produces an answer grounded in the retrieved documents.

Result ​

You now have a complete retrieval-augmented generation workflow.

Each answer is generated using:

  • the user’s question
  • relevant document chunks retrieved from the vector store

This grounding reduces hallucinations and improves reliability.

➡️ View the complete workflow on KNIME Hub:
Open the finished RAG workflow.

Next Steps ​

If you want to keep learning about AI and data workflows in KNIME, take one of our free courses and earn a microcredential badge to showcase your skills: