Introduction
This guide outlines how to deploy and configure the KNIME Edge Control Plane.
The KNIME Edge Control Plane is implemented as a collection of KNIME Workflows running on a KNIME Server providing both the API endpoints for the KNIME Edge Cluster and the user interface for interacting with KNIME Edge clusters via the KNIME WebPortal.
These KNIME Edge Control Plane workflows can be deployed in a single operation via the KNIME Edge Initialization Workflow hosted on the KNIME Community Hub.
Installation Planning
KNIME Server Large
A KNIME Server Large installation runs a set of Workflows which implement the central Control Plane for one or more KNIME Edge clusters; for information on obtaining and installing KNIME Server, see the Server Installation Guide .
Supported version(s): KNIME Server Large 4.11.0+ |
KNIME Community Hub
A KNIME Community Hub account is required for accessing the KNIME Edge Initialization Workflow; register at https://hub.knime.com/ .
KNIME Analytics Platform
KNIME Analytics Platform (AP) is required to upload KNIME Edge Workflows to the KNIME Server installation for configuring the Control Plane; KNIME AP can be downloaded from https://www.knime.com/downloads .
PostgreSQL Database
To maintain the state of all KNIME Edge clusters via KNIME Server, a PostgreSQL database is needed so all workflows (regardless of whether they handle API requests or KNIME WebPortal user interfaces) can leverage it to maintain state regardless of the number of Executors set up in the Server installation.
Supported version(s): The minimum version for PostgreSQL tested is 11.8 or higher. |
SQL files containing the needed schemas for the PostgreSQL instance are bundled together with the workflows and can be found in the KNIME Edge space under the /<Edge-Root-Path>/config/SQLFiles
directory after configuring the KNIME Server.
The initialization workflow sets up the needed schemas by executing the SQL files and configures the PostgreSQL Database Connection nodes in all KNIME Edge Control Plane workflows.
Configuring KNIME Server
KNIME Edge initialization workflow
Configure KNIME Server to act as the Control Plane for KNIME Edge clusters by using a local installation of KNIME Analytics Platform to either drag-and-drop or download and import the KNIME Edge Initialization Workflow from KNIME Community Hub.
Uploading the initialization workflow to KNIME Server
Using the KNIME Explorer view in Analytics Platform, connect to a KNIME Server and deploy or copy/paste the KNIME Edge Initialization workflow to any location where a user will have execute access to run the workflow.
Executing the initialization workflow
Log into the KNIME Webportal via a browser, then navigate to where the Initialization workflow was saved.
On the first page, there is a prompt for the "Edge Root Path". This the root workflow group that will be created and all needed workflows will be deployed to.
Note that you can specify a location that does not exist (in the case of installing a new set of workflows), or you can enter the root location to an existing KNIME Edge deployment and the currently specified parameters will be fetched and used in the next configuration step.
If "Use current KNIME Server login details for requests" is deselected, the next page will prompt for KNIME Server basic auth credentials.
Running the Initialization workflow on an existing KNIME Edge root directory can be used to update the version of workflows deployed there, as well as apply new schema updates to the PostgreSQL database.
After the KNIME Edge root path is entered, click "Next" and the subsequent view will prompt the user for additional configuration options.
The Edge Version option is used to select a specific bundled version of KNIME Edge workflows to the KNIME Server. See the Version and Update documentation to determine what versions of KNIME Edge clusters are compatible with specific KNIME Server workflow versions.
The list of available KNIME Edge Versions is dynamically populated when the Initialization workflow is run, so it will always show an up-to-date list of available workflow bundles.
The remaining fields are used to enter the connection parameters for the required PostgreSQL database.
Once the KNIME Edge version and database connection parameters are entered, click "Next" and the final Initialization view will be shown, with options to the user on how to configure the database.
Checking the Update Database Schema to latest version option will sequentially apply all schema updates to the database to make sure it is in the required state needed for the pre-selected KNIME Edge version used. When upgrading to a newer version of KNIME Edge workflows, keep this box checked to ensure all updates are applied. Data will not be lost when Updating Database Schemas.
Checking the Reset Database Schema Before Updating option will completely destroy all data and recreate the schema, then recreate the needed schema as long as the update database schema box is also selected.
Only use the Reset Database Schema Before Updating option if the intention is to completely reset a KNIME Edge Configuration on KNIME Server. Data will be lost during this process! |
Confirming the KNIME Edge Control Plane is configured properly
After the Initialization workflow has executed successfully you will now see a collection of workflows created inside the "Edge Root Path" previously specified.
These workflows are divided into three types:
-
API workflows mainly used for communication directly with the KNIME Edge Cluster.
-
Configuration workflows used for installation and maintenance of the KNIME Edge Server and database.
-
Application workflows that can be run by users to view and modify the state of KNIME Edge clusters and inference deployments.
Inside the "apps" workflow group you can execute the View KNIME Edge Clusters or View Deployments workflows to see an overview of the state of connected KNIME Edge clusters. As long as these workflows execute successfully then you should be confident that the KNIME Edge Control Plane is correctly configured.
OAuth Considerations
Users with KNIME Servers configured to authenticate via OAuth need to be aware of the following considerations.
Communication between the KNIME Edge Control Plane and a KNIME Edge cluster occurs via calls to the KNIME Server REST API. In addition, the Initialization workflow and several of the "Application" workflows make requests to the KNIME Server REST API to set up scheduled jobs and query certain information.
Currently these requests are unable to be made by a user authenticated via OAuth. To ensure these workflows run successfully the Initialization workflow as well as the KNIME Server user credentials configured within the KNIME Edge cluster needs to be run by a "service account" user that authenticates via a non OAuth manner.
Information on setting up one of these user accounts can be found in the KNIME Server documentation.