Introduction

KNIME Server is an enterprise-grade solution for advanced analytics workloads such as sharing workflows, and executing workflows.

KNIME Server consists of a server side component and a client interface.

The server side component is based on a Tomcat application server, and uses a KNIME Executor to execute the workflows.

KNIME Server provides KNIME software users with a shared remote workflow repository. Workflows can be uploaded to and stored on the Server or retrieved from it to a local KNIME workspace. This is done via the client interface which is integrated in KNIME Analytics Platform. For an overview on how to use the KNIME Analytics Platform as a client interface with KNIME Server please refer to the KNIME Server User Guide.

Workflows on the Server can be executed as jobs either as needed or as part of a one time or periodically scheduled event. Additionally, workflows may be executed through a web browser using the KNIME WebPortal or via RESTful web service interfaces which allow control from 3rd party applications.

This document aims to give a quick overview of the steps needed to perform the installation of KNIME Server, and a short description of the options that can be changed at install time. After installation, you can refer to the KNIME Server Administration Guide for detailed information about the Server architecture, configuration options and general administration tasks.

Please be aware that to make use of KNIME Server you will need a license. If you have not received a license file, please contact KNIME (contact@knime.com), or get in touch with your local KNIME representative.

Installation planning

KNIME Server variants

KNIME Server is available in the following variants. All single node instances can be installed in the same way with respect to the operating system. KNIME Server Large also has the possibility to use multiple KNIME Executors in a distributed system.

  • Single node

    • KNIME Server Small

    • KNIME Server Medium

    • KNIME Server Large

  • Distributed KNIME Executors

    • KNIME Server Large

Beginning with KNIME Server 4.11, we use an integrated message broker for communication between application server and Executor, based on Apache Qpid (https://qpid.apache.org/). This can be used for running single node KNIME Server installations. Apache Qpid is bundled as part of all KNIME Server installations, so no additional setup is necessary.

If you want to set up distributed KNIME Executors for KNIME Server Large, please follow all steps in this document related to the main installation, and then follow the additional steps outlined here. This explains how to setup RabbitMQ (which is used as message broker instead of Qpid), as well as how to add additional Executors.

Supported operating systems

KNIME Server can be installed on machines running either Windows or Linux. The following operating systems are officially supported:

  • Windows Server 2016, 2019, and 2022

  • Ubuntu 20.04 LTS and 22.04 LTS and derivatives

  • RHEL/CentOS/Rocky Linux 7, 8 or 9

    • Some features of KNIME Analytics Platform such as image generation will not work on RHEL/CentOS 7 because the operating system libraries are too old. We recommend using at least version 8.

KNIME Server may also run on other operating systems, however, we neither test nor support them through the KNIME Server standard support package.

Network requirements

  • KNIME Server (Small, Medium, or Large) on a single node:

    • Port 8080, 8443 must be accessible from all KNIME Analytics Platform clients requiring access to the Server

    • Port 8080, 8443 must be accessible for all browsers requiring WebPortal access

  • KNIME Server Large with distributed KNIME Executors

    • Port 8080, 8443 must be accessible from all KNIME Analytics Platform clients requiring access to the Server

    • Port 8080, 8443 must be accessible for all browsers requiring WebPortal access

    • Port 5672, must be accessible on both the KNIME Server and KNIME Executors in order for them to communicate with each other.

Hardware requirements

  • 64-bit operating system with at least 32GB RAM. If workflows are executed on the same machine, the minimum number of cores depends on the licensed execution cores and according to your license. Additionally, we recommend to reserve 2-4 cores for the application server (Tomcat) which result into a minimum configuration of 6-8 cores for a single KNIME Server including execution. Please contact KNIME for sizing help.

  • KNIME Server stores all workflows and related configuration information in the KNIME Server workflow repository. The folder, that will be created during the installation process, should exist on a local drive and have enough free space to store the workflows and jobs. We recommend a minimum of 250 GB, although this may need to be increased considerably depending on the intended use case.

Java requirements

For fresh installations of KNIME Server, use Java Development Kit (JDK) 11 or 17. We officially support Adoptium distributions of JDK.

Please notice that JDK 17 is supported, but in case you are using OIDC, you should not go beyond JDK 11.

Software requirements

Assuming that:

  • You want to perform an installation where the Tomcat application server and KNIME Executor live on the same machine

  • You have internet access on the target machine where you want to perform the installation

You can install the Executor during the Server installation process. You can then skip the next section and move to the installation steps sections for Windows or Linux operating systems.

In case the above assumptions do not apply you can download the Executor before starting the Server installation process and point to the Executor installation folder when requested.

Install KNIME Executor

You can install KNIME Executor downloading its executable from here.

executor

On Windows:

  • The self-extracting archive simply creates a folder containing the KNIME Executor installation files. You don’t need any software to manage archiving.

  • The zip archive can be downloaded, saved, and extracted in your preferred location. We recommend to use an archive utility such as 7-zip for extraction, as the Windows built-in zip file utility may have trouble with some of the long filenames.

On Linux:

  • Extract the downloaded tarball to a folder of your choice:

    tar -zxf knime-full-latest53-linux.gtk.x86_64.tar.gz -C /<executor-folder>/

Make sure that users other than the installation owner either have no write permissions to the installation folder at all or that they have full write permission to at least the "configuration" folder. Otherwise you may run into startup issues. We strongly recommend revoking all write permissions from everybody but the installation owner.

You can download and install the extensions to the Executor adding Local Update Sites as zip files if you have limited internet access on the target machine. You can download the KNIME update sites as zip files at the links listed here.

Please note also that a standard KNIME Analytics Platform build will not be sufficient to be used as Executor. In case you want to use a pre-existing KNIME Analytics Platform installation as an Executor you will need to install the KNIME Executor connector extension, available from the pre-registered KNIME Analytics Platform update site. To install it run the KNIME Analytics Platform instance that is intended to be the Executor and navigate to File → Install KNIME Extensions…​ and select KNIME Executor connector under KNIME Server Extensions.

The versions of the KNIME Server Executor extensions must match the server’s version (e.g. "4.17").

Also you may want to install the KNIME Report Designer extension to the KNIME Analytics Platform build to be used as KNIME Executor.

Finally, please also be aware that you will need a different KNIME Analytics Platform installation to use as client interface.

In case you will need to install additional extensions to the Executor please refer to the KNIME Executor administration section of the KNIME Server Administration Guide.

Client interface

A KNIME Analytics Platform is needed to use it as client interface.

KNIME Analytics Platform is available to install for operating system Windows, Linux, or macOS (independent from the Server OS).

Installation steps

KNIME Server single node installation on Windows

We will install Java, download the KNIME Server installer, install KNIME Server alongside with KNIME Executor, and perform a small test to validate the KNIME Server. This includes options to configure automatic startup and shutdown processes for KNIME, email configuration, and uploading of a KNIME Server License.

The first step is to download and install the latest Temurin OpenJDK 11 (LTS) from the Adoptium Website.

02 windows openjdk
Please check that JDK is added to $JAVA_HOME during the installation process. If it is not the case please manually add the path to Adoptium to $JAVA_HOME in the environment variables list under System Variables.

During the installation process take note of the path to the JDK installation as you will need to point to it later during the installation process. It usually is C:\Program Files\Eclipse Adoptium\jdk-<version> by default.

Do not install the Server as Administrator or root user! Follow the installation instructions step by step. Don’t skip anything unless it is marked as optional.
In case you are installing KNIME Server on a machine which already has RabbitMQ installed, please stop the RabbitMQ service first before starting the installation.

  1. Download and launch the KNIME Server installer. The installer is called knime-server-installer-<server-version.x>.jar (where <server-version.x> is the current release number, e.g. 4.13.0) and it can be downloaded from here.

    img dl server installer

  2. Click Next and you will go through a total number of 18 steps.

    In Step 2 of 18 you are asked to point to the JDK installation folder. In case you skipped the installation you are pointed to the the Adoptium Website.

    img 2 18

  3. In Step 3 of 18 you can insert the location to which you want to install the Server. By default the C:\knime_server folder is suggested. If the folder you choose still does not exist on your machine you will be warned that the folder will be created.

    img 3 18

  4. In the next step (Step 4 of 18) you are asked to choose the KNIME Executor that will be used to run workflows. Assuming that you want to perform an installation where the Tomcat application server and the KNIME Executor live on the same machine and that you have internet access on the target machine you can choose option Install KNIME Executor.

    img 4 18

    In the next step you are asked for the folder where to install the KNIME Executor. By default the C:\knime_server\knime_executor folder is suggested. Then you can choose the update sites from which the installer will add all bundles available in the listed update sites. By default the KNIME Analytics Platform and the KNIME Community Extensions (Trusted) update sites are suggested.

    If instead you do not have access to internet connection on the target machine you can download the KNIME Executor before starting this installation as specified in the Prerequisites section. In this case choose the first option Select existing KNIME Executor installation. Then you can point the Server installer to the Executor installation folder in the next step.

    Selecting option 1 or 2 will automatically add additional settings to the Executor’s knime.ini file that are needed to establish a connection to the application server, and that can be used to manage the Executor’s preference setting.

    Changes to the knime.ini file will not be made if you point the installer to an existing workflow repository in the next step. This is done to not contradict user-defined settings in knime-server.config.

    In case you want to perform a distributed installation please follow the steps in the Distributed KNIME Executors installation section.

  5. The installer will ask you to enter the path to the workflow repository. It is recommended that the workflow repository is stored independently from the Server, in a location that is frequently copied into a backup. In case the folder exists, the installer will not modify the folder, assuming that you pointed it to the workflow repository of a previous installation.

  6. OPTIONAL: It is easier to upload the license from the KNIME WebPortal, but you may provide already here the path where the license is located as part of the installation.

  7. OPTIONAL: Default ports are 8080, and 8443. These ports can be changed in the range: 1024-49151.

  8. In Step 11 of 18 you can choose:

    • The mount ID, which is a common name for the Server used to describe the KNIME Server to the clients that connect to it. For simple installations, the default name, or a simple name e.g. mycompany-knime is usually appropriate. For an environment with multiple KNIME Servers, mountpoints such as knime-server-prod or knime-server-dev would allow users to distinguish between development and production environments. The mount ID can also be changed at a later time.

    • The Context Root (CONTEXT) is used to define the KNIME WebPortal URL. All WebPortal functionality will be available at:

      https://HOSTNAME:PORT/CONTEXT/webportal/
      Please be aware that with the release of KNIME WebPortal version 4.11+ it is necessary to always specify a Context Root.

  9. OPTIONAL: Change the knimeadmin password. The default username is knimeadmin, and the password is set to the hostname of the machine. It is strongly recommended that you change this password, and you can do so at a later time via the administration portal.

    If using LDAP authentication, changes are required to the server.xml file in <installation-path>/apache-tomcat*/conf. Please refer to the KNIME Server Advanced Setup Guide for full information.

  10. OPTIONAL: Configure email. We advise you to set up the email resources at a later time as explained in the KNIME Server Administration Guide.

  11. Now the Server is installed and launched. A new Tomcat command prompt window appears. You can ignore any warnings that might be shown in this window. At this point of the process, closing this window will shutdown KNIME Server. Click Next and a page will display information about the installation, such as the URLs required to connect.

    img 16 18

  12. OPTIONAL: Generate an automatic installation script. Once successfully executed, you can generate an xml file, called auto-install.xml, and describes all parameters that were set during the installation process. In case you want to perform an installation using the same parameters you can simply run:

    java -jar knime-server-installer-4.x.y.jar <path-to>/auto-install.xml

  13. Go to the following address from a browser that has network access to port 8443, or to the PORT you set up if different from the default one:

    https://HOSTNAME:PORT/CONTEXT/webportal/

    Enter the credentials for the admin user and connect to the Server WebPortal interface. Here you will be able to upload the KNIME Server license file.

    img upload license file
    In case you do not have a license, please contact KNIME (contact@knime.com), or get in touch with your local KNIME representative.

  14. Now you have to start up KNIME Executor. You can use the batch script in the Executor folder:

    <executor-directory>/start-executor.bat
    Please make sure to start the KNIME Executor after starting KNIME Server. Otherwise, the Executor will not be able to acquire the core tokens it needs to operate.
In case the installer did not add the path to Qpid to the Executor’s knime.ini, please add the following line to the bottom of your knime.ini file:
-Dcom.knime.enterprise.executor.msgq=amqp://knime:20knime16@localhost/

Starting and stopping KNIME Server under Windows

KNIME Server can be started and stopped using scripts in the <apache-tomcat>/bin folder.

For Windows use:

<tomcat-folder>/bin/shutdown.bat
<tomcat-folder>/bin/startup.bat

Then start the Executor with:

<executor directory>/start-executor.bat

After starting you should be able to access KNIME Server from both KNIME Analytics Platform and KNIME WebPortal. More details can be found in the KNIME Server User Guide and KNIME WebPortal User Guide.

You can access via web browser at this address:

https://HOSTNAME:PORT/CONTEXT/webportal/

To test that your KNIME Server installation and the relative KNIME Executor are up and running, access to KNIME WebPortal and run a test workflow. For example you might try running the workflow Examples/REST/Predict Results Using REST API.

You can mount the Server access point in the KNIME Analytics Platform using the following address:

https://HOSTNAME:PORT/

If you encounter problems please have a look at the log files (see section Log Files in the KNIME Server Administration Guide for details).

KNIME Server single node installation on Linux

We will install Java, create a non-root user, set-up the appropriate directory structure, download, install, and perform a small test to validate the KNIME Server. This includes options to configure automatic startup and shutdown processes for KNIME, email configuration, and uploading of a KNIME Server License.

Do not install the Server as Administrator or root user! Follow the installation instructions step by step. Don’t skip anything unless it is marked as optional.
In case you are installing KNIME Server on a machine which already has RabbitMQ installed, please stop the RabbitMQ service first before starting the installation.

  1. Create a new, non-root user to install KNIME Server and KNIME Executor and a new directory for the installation to live. The folder is owned by the user just created. Here we create a directory in /opt/ and a user named knime that owns it. In a terminal run the command:

    sudo useradd -d /opt/knime -m -s /bin/bash knime

    We recommend that both the Tomcat server and the KNIME Executor are installed by the same user. This user does not need administration privileges. The recommended user name is knime. No other user requires write access to any files created during the installation.

    This user must perform the entire KNIME Server Executor installation including unpacking of KNIME Analytics Platform and installation of all desired additional features. If additional features are added later with a user that cannot write to the installation folder, the installation may appear to work normally, but causes problems some time later which can be difficult to track down.

  2. Install Java 11.

    Support for Java 11 has been added in KNIME Server 4.12. For fresh installations, we recommend to use Java 11. 
    sudo apt-get install openjdk-11-jdk

  3. Install Chromium

    sudo apt-get install libgtk-3-0

  4. Switch accounts to the new knime user in step 1 and start working in the new /opt/knime/ directory

    sudo su - knime

  5. Download the KNIME Server installer to the /opt/knime directory. The installer is called knime-server-installer-<server-version.x>.jar (where <server-version.x> is the current release number, e.g. 4.12.2) and it can be downloaded from here.

    img dl server installer

  6. Run the KNIME Server Installer

    java -jar <knime server installer filename> -console

    Additional installer command-line flags

    The installer reads out system properties specified on command-line, which are usually not required in a standard installation but can be useful for automated (update) scripts.

    • -Dknime.server.executor.memory=<value> (<value> is for instance 4G or 2500M)
      The value specified is added to the KNIME Server Executor’s knime.ini file and specifies how much memory a KNIME Server Executor is allowed to use at most. If not given, a default is used, which is based on the system’s available main memory.

    • -Dknime.server.nostart=true
      If specified, KNIME Server is not started after the installation.

  7. First the installer will prompt you to enter the path to the system’s JDK that you installed in step 2.

  8. Then the installer will prompt you to enter a new directory that will be created and will host the KNIME Server. You can select any directory, but it is recommended you enter:

    /opt/knime/<server-version>

  9. The installer will prompt you to select the KNIME Executor that will be used to run workflows. Assuming that you want to perform an installation where the Tomcat application server and the KNIME Executor live on the same machine and that you have internet access on the target machine you can choose option 1.

    If instead you do not have access to internet connection on the target machine you can download the KNIME Executor before starting this installation as specified in the Prerequisites section. In this case choose option 0 and point the installer to the Executor installation folder in the next step.

    In case you want to perform a distributed installation please follow the steps in the Distributed KNIME Executors installation section.

  10. The installer will prompt you to enter the path to the workflow repository. It is recommended that the workflow repository is stored independently from the Server, in a location that is frequently copied into a backup. In case the folder exists, the installer will not modify the folder, assuming that you pointed it to the workflow repository of a previous installation.

  11. OPTIONAL: It is easier to upload the license from the KNIME WebPortal, but you may provide already here the path where the license is located as part of the installation.

  12. OPTIONAL: Default ports are 8080, and 8443. These ports can be changed in the range: 1024-49151.

  13. Now you can specify:

    • The mount ID, which is a common name for the Server used to describe the KNIME Server to the clients that connect to it. For simple installations, the default name, or a simple name e.g. mycompany-knime is usually appropriate. For an environment with multiple KNIME Servers, mountpoints such as knime-server-prod or knime-server-dev would allow users to distinguish between development and production environments. The mount ID can also be changed at a later time.

    • The Context Root (CONTEXT) is used to define the KNIME WebPortal URL. All WebPortal functionality will be available at:

      https://HOSTNAME:PORT/CONTEXT/webportal/
      Please be aware that with the release of KNIME WebPortal version 4.11+ it is necessary to always specify a Context Root.

  14. OPTIONAL: Change the knimeadmin password. The default username is knimeadmin, and the password is set to the hostname of the machine. It is strongly recommended that you change this password, and you can do so at a later time via the administration portal.

    If using LDAP authentication, changes are required to the server.xml file in <installation-path>/apache-tomcat*/conf. Please refer to the KNIME Server Advanced Setup Guide for full information.

  15. OPTIONAL: Configure email. We advise you to set up the email resources at a later time as explained in the KNIME Server Administration Guide.

  16. OPTIONAL: Generate an automatic installation script. Once successfully executed, you can generate an xml file, called auto-install.xml, and describes all parameters that were set during the installation process. In case you want to perform an installation using the same parameters you can simply run:

    java -jar knime-server-installer-4.x.y.jar <path-to>/auto-install.xml

  17. Now the Server automatically starts. Go to the following address from a browser that has network access to port 8443, or to the PORT you set up if different from the default one:

    https://HOSTNAME:PORT/CONTEXT/webportal/

    Enter the credentials for the admin user and connect to the Server WebPortal interface. Here you will be able to upload the KNIME Server license file.

    img upload license file
    In case you do not have a license, please visit our website and contact KNIME.

  18. Now you have to start up KNIME Executor. You can use the batch script in the Executor folder:

    <executor-directory>/start-executor.sh
    Please make sure to start the KNIME Executor after starting KNIME Server. Otherwise, the Executor will not be able to acquire the core tokens it needs to operate.
In case the installer did not add the path to Qpid to the Executor’s knime.ini, please add the following line to the bottom of your knime.ini file: -Dcom.knime.enterprise.executor.msgq=amqp://knime:20knime16@localhost/

Starting and stopping KNIME Server under Linux

KNIME Server can be started and stopped using scripts in the <apache-tomcat>/bin folder.

For Linux restart with:

<apache-tomcat>/bin/shutdown.sh
<apache-tomcat>/bin/startup.sh

Then start the Executor with:

<executor-directory>/start-executor.sh

Please make sure to adjust the permissions of the start-executor.sh file to also allow execution by the user running the Executor, e.g.

chmod 744 start-executor.sh

In case the executor startup script is not present, you can start the Executor from command line by running

./knime -nosplash -consolelog -application com.knime.enterprise.slave.KNIME_REMOTE_APPLICATION

After starting you should be able to access KNIME Server from both KNIME Analytics Platform and KNIME WebPortal. More details can be found in the KNIME Server User Guide and KNIME WebPortal User Guide.

You can access via web browser at this address:

https://HOSTNAME:PORT/CONTEXT/webportal/

You can mount the Server access point in the KNIME Analytics Platform using the following address:

https://HOSTNAME:PORT/CONTEXT/

If you encounter problems please have a look at the log files (see section Log Files in the KNIME Server Administration Guide for details).

In some cases it may take quite some time (up to several minutes) until the server responds to requests on Linux systems. This is usually caused by insufficient entropy for the random number generator used by Tomcat. A description on how to remedy this issue is included in the KNIME Server Administration Guide

Automatically starting KNIME Server

In order to start KNIME Server after a reboot, you need to define a service that automatically starts Tomcat and brings it down at a machine shutdown.

Note that the services described below allow setting basic configuration such as the Java installation to use and the memory available to KNIME Server. This is also the recommended way to set these values. However, the KNIME Server Installer also sets these values directly in the Apache Tomcat installation using <apache-tomcat>/bin/setenv.sh or <apache-tomcat>\bin\setenv.bat (so that they are set when KNIME Server is started manually). The values defined in this file will always override the settings from the system services. Therefore we recommend deleting this file or at least comment out the settings inside it.

Creating a Windows service

If you want to start and stop KNIME Server automatically under Windows, you need to create a Windows service.

Open a command line window as administrator, navigate to <apache-tomcat>\bin and run the following command:

service.bat install

This will create a Windows service called "Apache Tomcat 9.0 Tomcat9" that can be started and stopped by going to Control Panel > Administrative Tools > Services.

Here you can right click on the Apache Tomcat service and choose Start or Stop from the context menu that opens.

Select Properties to change the settings of the service. In the windows that opens go to LogOn tab to make sure to configure the service to run as Local System Account, and not as Local Service Account.

In order to uninstall the service, open a command line window as administrator, navigate to <apache-tomcat>\bin and run the following command:

service.bat uninstall

If you want to know more details, have a look at the Tomcat documentation which describes this process in detail.

Running Executors as services on Windows

It is also possible to run Executors as services that are automatically started during system startup (and stopped during shut down). This is the recommended method to use when not running on a docker deployment.

For increased security, we recommend to run the Executor service as a different user than the Tomcat service.

On Windows Executors can be run as Windows services by using NSSM (Non-Sucking Service Manager). The following steps assume that you have a KNIME Analytics Platform installation that contains the KNIME Executor connector extension as described in KNIME Executor installation section.

  1. Edit

    <knime_executor>/install-executor-as-service.bat

    and adjust the variables at the top of the file to your needs.

  2. Run this batch file as administrator. This will install the service.

  3. Open the Windows Services application, look for the KNIME Executor service in the list and start it.

  4. If you want to remove the executor service again, run the following as administrator:

    <knime_executor>/remove-executor-as-service.bat

Note that if you move the KNIME Executor installation you first have to remove the service before moving the installation and then re-create it.

If you remove the KNIME Executor service this way, it does not immediately remove it, but instead marks it for deletion. You must close any Task Manager or Service console windows to allow the operating system to perform cleanup and finish the service deletion before you can re-create it.

Creating a Linux service

This section assumes that you are running a supported Linux distribution with systemd such as Ubuntu >=16.04, RHEL >=7.x, or derivatives.

  1. Copy the content of the whole folder to the root of your file system.

    sudo cp -r <knime_server>/install-data/linux-runlevel-templates/systemd/. /

    The folder includes the systemd service description for KNIME Server, a start script, and a systemd override file that allows configuration of the service (such as file system location or the userid under which the server should run).

  2. Run

    sudo systemctl daemon-reload

  3. To create a Linux service for KNIME Server, run:

    sudo systemctl edit knime-server.service

    Adjust the settings in the editor that opens, and save the changes. Make sure that the user specified in this file exists on the system. Otherwise startup will fail.

    If your custom configuration contains any non-alphanumeric characters (including @) you have to encode them using systemd-escape <string> before pasting them into the file.

  4. Enable the service with

    sudo systemctl enable knime-server.service

Running Executors as services on Linux

It is also possible to run Executors as services that are automatically started during system startup (and stopped during shut down). This is the recommended method to use when not running on a docker deployment.

For increased security, we recommend to run the Executor service as a different user than the Tomcat service.

  1. Enable the Xvfb service which is required for creating images in KNIME nodes. If Xvfb is not available it can crash the executor.

    sudo systemctl enable Xvfb.service
sudo systemctl start Xvfb.service

  2. Run:

    sudo systemctl edit knime-executor.service

    Adjust the settings in the editor that opens, and save the changes. Here you need to update the path to point towards the Executor. Make sure that the user specified in this file exists on the system. Otherwise startup will fail unless your version of systemd supports DynamicUser. In this case a temporary user account will be created.

    If your custom configuration contains any non-alphanumeric characters (including @) you have to encode them using systemd-escape <string> before pasting them into the file.

  3. Enable the service with the command:

    sudo systemctl enable knime-executor.service

  4. If you changed the User make sure to also change it in the Xvfb service accordingly by running

    sudo systemctl edit Xvfb.service

    and supplying the following contents

    [Service]
User=<same user as for the knime-executor.service>

KNIME Server settings

With KNIME Server version 4.12 an administration portal has been added. It is reachable when signing in to the KNIME WebPortal as an administrator.

The administration portal gives you an overview on the status of your KNIME Server, allows for an easy upload of a new license file, and also to configure your KNIME Server options through a browser-based user interface.

To have more information about Server settings and how to change them please refer to the KNIME Server Administration Guide.

KNIME Server authentication

In order to manage user authentication to KNIME Server it is possible to use any of the authentication methods available to Tomcat. By default the KNIME Server installer configures a database (H2) based authentication method. However, for enterprise applications, use of LDAP authentication is recommended, and user/group management is handled in Active Directory/LDAP itself. Also a token-based authentication is supported by KNIME Server.

Examples for each of the authentication methods are contained within the default installation. Configuration of all three authentication methods are described briefly in the User authentication section of the KNIME Server Administration Guide. In all cases the Tomcat documentation should be considered the authoritative information source.

Distributed KNIME Executors

As part of a highly available architecture, KNIME Server 4.17 allows you to distribute execution of workflows over several Executors that can sit on separate hardware resources. This allows KNIME Server to scale workflow execution with increasing load because it is no longer bound to a single computer.

If you’re planning to use the distributed KNIME Executors in production environments please get in touch with us directly for more information.

Installation, configuration, and operation is very similar to the single Executor setup. The server communicates with the Executors via a message queueing system (and HTTP(S)). We use RabbitMQ for this purpose, and it’s recommended, although not required, to install that on a separate machine as part of a highly available architecture.

09a distributed exec architecture

To enable KNIME Executors you will first need to install KNIME Server, shutdown the Server and proceed with the instructions available in the KNIME Server Administration Guide.

Installation steps

Enabling KNIME Executors consists of the following steps:

  • In case you haven’t installed KNIME Server already, please follow the steps in the above dedicated section

  • Shut down the server if it has been started by the installer

  • Install RabbitMQ following the instructions below

  • Adjust configuration files for the server and Executor following the instructions below

  • Start the server and one or more Executors.

Installing RabbitMQ

The server talks to the Executors via a message queueing system called RabbitMQ. This is a standalone service that needs to be installed in addition to KNIME Server and the executors. You can install it on the same computer as KNIME Server or on any other computer directly reachable by both KNIME Server and the Executors.

KNIME Server requires RabbitMQ 3.6+ which can be installed according to the Get Started documentation on their web page.

Make sure RabbitMQ is running, then perform either of the following steps depending on the availability of a GUI access to the machine running the service.

  • In case the machine has GUI access:

    • Enable the RabbitMQ management plug-in by following the online documentation

    • Log into the RabbitMQ Management which is available at http://localhost:15672/ (with user guest and password guest if this is a standard installation). The management console can only be accessed from the host on which RabbitMQ is installed.

    • Got to the Admin tab and add a new user, e.g. knime.

    • Also in the Admin tab add a new virtual host (select the virtual hosts section on the right), e.g. using the hostname on which KNIME Server is running or simply knime-server.

    • Click on the newly created virtual host, go to the Permissions section and set permission for the new knime user (all to ".*" which is the default).

  • In case the machine does not have GUI access:

    • Run the following three commands:

      rabbitmqctl add_user knime 20knime16
rabbitmqctl add_vhost knime-server
rabbitmqctl set_permissions --vhost knime-server knime ".*" ".*" ".*"

When RabbitMQ is running on a Windows machine, the following additional steps are recommended:

  • Install sysinternals tool: In order to avoid repeating error messages in the log files, e.g. [error] <> Could not find handle.exe, please install from sysinternals, please install the sysinternals free tool provided by Microsoft, and add the directory to the path. This will allow RabbitMQ to display all of its stats on the management portal.

  • Change RabbitMQ base directory: RabbitMQ by default saves the data and logs under the folder C:\<user>\AppData. In case you do not want to allow the software to write to C:\ you can change the base directory as explained nicely, for example, in this guide.

Connecting Server and KNIME Executors

KNIME Server and the KNIME Executors now need to be configured to connect to the message queue.

For KNIME Server you must specify the address of RabbitMQ instead of the path to the local Executor installation in the knime-server.config. I.e. comment out the com.knime.server.executor.knime_exe option (with a hash sign) and add the option com.knime.enterprise.executor.msgq. The latter takes a URL to the RabbitMQ virtual host: amqp://<user>:<password>@<rabbit-mq-host>/<virtual host>, e.g.

com.knime.enterprise.executor.msgq=amqp://<username>:<password>@rabbitmq-host/knime-server

Note that any special characters in the password must be URL encoded.

The same URL must also be provided to the Executor as system property via the knime.ini:

-Dcom.knime.enterprise.executor.msgq=amqp://<username>:<password>@rabbitmq-host/knime-server

Alternatively you can provide the message queue address as an environment variable:

KNIME_EXECUTOR_MSGQ=amqp://<username>:<password>@rabbitmq-host/knime-server
In case RabbitMQ High Available Queues are used, simply add additional <rabbitmq-host>:<port> separated by commas to the initial amqp address: 
-Dcom.knime.enterprise.executor.msgq=amqp://<username>:<password>@rabbitmq-host/knime-server,amqp://<rabbitmq-host2>:<port2>,amqp://<rabbitmq-host3>:<port3>
In order to use RabbitMQ, you need to explicitly deactivate the embedded Qpid message broker by setting com.knime.enterprise.executor.embedded-broker=false in knime-server.config. Qpid does not support more than one KNIME Executor, and it doesn’t support Executors running on separate hosts.

While commands between the server and KNIME Executors are exchanged via the message queue, actual data (e.g. workflows to be loaded) are exchanged via HTTP(S). Therefore, the KNIME Executors must know where to reach the server. The server tries to auto-detect its own address however in certain cases this address is not reachable by the KNIME Executors or — in case of https connections — the hostname doesn’t match the certificate’s hostname. In such cases you have to specify the correct public address in the knime-server.config with the option com.knime.server.canonical-address, e.g.

com.knime.server.canonical-address=https://knime-server:8443/

You don’t have to specify the context path as this is reliably auto-detected. Now you can start the server.

The KNIME Executors must be started manually, the server does not start them. In order to start an Executor (on any machine) launch the KNIME application (that has been created by the installer) with the following arguments:

./knime -nosplash -consolelog -application com.knime.enterprise.slave.KNIME_REMOTE_APPLICATION

You can also add these arguments at the top of the knime.ini if the installation is only used as an Executor. You can start as many KNIME Executors as you like and they can run on different hosts. They will all connect to RabbitMQ (you can see them in the RabbitMQ Management in the Connections tab).

When you start the Executor in a shell, a very simple command line interface is available to control the Executor. Enter help at the Executor> prompt to get a list of available commands.

On Windows a separate window is opened for the Executor process. In case there is a problem during startup (e.g. the Executor cannot acquire core tokens from the server) then this window closes immediately. In this case you can add -noexit to the command above to keep it open and look at the log output or open at the log file which by default is <user home>/knime-workspace/.metadata/knime/knime.log unless you provided a different workspace location with -data.

You may find it helpful for an Executor to use customization profiles provided by the KNIME Server. In this case consult the Customizations section of the KNIME Server Administration Guide. For example editing the startup command for the Executor will apply the executor profile.

./knime -nosplash -consolelog -profileLocation http://knime-server:8080/knime/rest/v4/profiles/contents -profileList
executor com.knime.enterprise.slave.KNIME_REMOTE_APPLICATION

Running KNIME Executors as services

It’s also possible to run KNIME Executors as services that are automatically started during system startup (and stopped during shut down). This is the recommended method to use when not running on a docker deployment.

Linux with systemd

Running KNIME Executors as services is only supported on Linux distributions that use systemd (e.g. Ubuntu >= 16.04, RHEL 7.x and derivates). The following steps assume that you have a KNIME Executor installed that contains the KNIME Executor connector extension as described in the section KNIME Executor installation.

  1. Copy the whole folder

    <knime_executor>/systemd/

    to the root of your file system. The folder includes the systemd service description for knime-executor and an override file that allows configuration of the service (such as file system location or the userid under which the Executor should run).

  2. Run

    systemctl daemon-reload

  3. Run

    systemctl edit knime-executor.service

    Adjust the settings in the editor that will open, and save the changes. Make sure that the User specified in this file exists on the system. Otherwise startup will fail unless your version of systemd supports DynamicUser. In this case a temporary user account will be created.

  4. Enable the service with

    systemctl enable knime-executor.service
Windows

On Windows KNIME Executors can be run as Windows services by using NSSM (Non-Sucking Service Manager). The following steps assume that you have a KNIME Analytics Platform installation that contains the KNIME Executor connector extension as described in the Install KNIME Executor section.

  1. Edit

    <knime_executor>/install-executor-as-service.bat

    and adjust the variables at the top of the file to your needs. Please make sure to change the entry for KNIME_EXECUTOR_MSGQ in line 4 to point to your RabbitMQ instance.

  2. Run this batch file as administrator. This will install the service.

  3. Open the Windows Services application, look for the KNIME Executor service in the list and start it.

  4. If you want to remove the Executor service again, run the following as administrator:

    <knime_executor>/remove-executor-as-service.bat

Note that if you move the KNIME Executor installation you first have to remove the service before moving the installation and then re-create it.