Introduction

KNIME Business Hub is a customer-managed Hub instance. Once you have a license for it and proceed with installation you will have access to Hub resources and will be able to customize specific features, as well as give access to these resources to your employees, organize them into Teams and give them the ability to manage specific resources.

This guide is intended to be an overview of the migration process of your KNIME Server to a KNIME Business Hub instance.

You are provided with a user guide and a migration workflow.

The user guide will cover:

  • The main feature differences between KNIME Server and KNIME Hub

  • How to use the provided migration workflow to migrate your KNIME Server to KNIME Hub, which prerequisites you will need to satisfy and a step by step user guide through the different pages of the workflow.

To install a KNIME Business Hub please refer to the KNIME Business Hub Installation Guide. To administrate a KNIME Business Hub instance please refer instead to the KNIME Business Hub Administration Guide.

Migration process overview

Nearly no code made it from KNIME Server to KNIME Business Hub. KNIME Business Hub, in fact, is built from the ground up on scalable cloud-native architecture, while still providing most of the functionality from KNIME Server.

KNIME Business Hub includes robust collaboration capabilities through introduction of spaces. This allows organizations to share their internal collections of solutions. Users can easily browse through workflows, making it easier to learn more, upskill, and onboard users with or without a license.

KNIME Business Hub also improves the experience of automating execution and deploying data apps and services. Users have a single intuitive frontend where they can share solutions in spaces and set schedules, and ultimately deploy and monitor their models.

However, while KNIME Business Hub provides most of the functionality from KNIME Server a one-to-one mapping of features between KNIME Server and KNIME Hub is not always possible.

We will not cover in this guide the installation process and the migration of users, but we want to give an overview on the main features.

Migration of the KNIME Server setup

The setup of your KNIME Server is not covered by the migration workflow. The following setups will need to be migrated separately.

  • The configurations of your Server setup such as job timeouts, and so on, should now be addressed via the configuration of the available execution contexts. The execution context creation can now be done by the Team administrators. Follow this guide to know more about the execution context creation.

  • The configuration of the Executor, e.g. the KNIME Analytics Platform version used for execution, the installed extensions and so on, are now bundled into an Executor docker image that can be configured by the KNIME Business Hub admin, as explained here. Each execution contexts can then have assigned the desired Executor image when created by the Team administrator.

Migration of existing Users and Groups

The migration of existing Users and Groups is not covered by the migration workflow. The following setups will need to be migrated separately.

  • If your KNIME Server installation uses the H2 database you will need to migrate the users manually.

  • LDAP instead is simply also supported by KNIME Business Hub, so you can set it up manually.

An interesting feature that is introduced with KNIME Business Hub is the usage of application passwords that will provide the users with the ability to provide authentication when using the KNIME Hub REST API, for example when executing deployed REST services. You can find more information on how to create and use application passwords in the Application passwords section of the KNIME Business Hub User Guide.

Teams

A team is a group of users on KNIME Hub that work together on shared projects. A Team will own specific Hub resources, to which the Team members will have access.

When users are migrated to the Hub one or more teams (depending on the type of license or your KNIME Business Hub) are created and users are assigned to a team and they are assigned a role:

When migrating by using the migration workflow you will migrate items from KNIME Server to KNIME Hub, under a specific team. For this reason, you will need to access the KNIME Business Hub using a user that has assigned team administrator role for the specific team.

Migration of workflows, files and workflow groups

This part is handled by the "KNIME Server to KNIME Hub Migration" workflow. Some new concepts have been introduced with KNIME Business Hub that are fundamental in this step of the migration.

Spaces and permissions

The basic concept that is the core of the KNIME Hub operations is the space. A space is a collection of items used to organize components, workflows and files belonging to a project.

Spaces are created for organizing a team’s work around a specific project or domain. Spaces are owned by a team and members of the teams have access to the spaces by default. The permissions can then be defined for each group or user.

The roles and permissions assigned to the team members or to the single users can then be assigned by the team admins.

During the migration process we recommend to transfer folders on the Server into a space on the Hub. Using the migration workflow described in the following sections you will be able to select one or multiple folders from the Server and to move their content (workflows, components and data files) to one or more spaces.

Spaces also will have a default execution context assigned which is the condition necessary for the workflows contained in the space to be executed. During the migration process via the migration workflow you are able to assign an already created execution context to the created spaces. Please note that if you want to migrate the schedule deployments of workflows as explained in the section Deployments, you will need to have assigned an execution context to the space.

Versioning

On KNIME Business Hub the concept of a version is introduced. A version is a snapshot in time of an item on Business Hub.

An item version history is available to the team members as a list of all the versions that have been created of a specific item.

When using the migration workflow snapshots can be transferred to the Hub as workflows. They will be located in a separate backup folder in the appropriate space and contain the date and comment of the snapshot in their name for later identification.

Deployments

On KNIME Server, when a workflow was uploaded to the Server it was automatically also possible to execute the workflow as a data app via the KNIME WebPortal, to schedule the execution of the workflows via the KNIME Analytics Platform and to use the workflow as a service to be called either by another workflow or by an external tool.

With KNIME Business Hub, the upload of a workflow to a space on the Hub needs to be followed by the creation of a version and then the creation of the desired deployment.

There are different types of deployments on KNIME Business Hub:

  • Data app

  • Schedule

  • Service

Since it is not possible to detect a priori whether a workflow is deployed as a data app or is executed as a service on KNIME Server, the migration process will need a manual step where the versions are created and the workflows are deployed as data apps or services.

Scheduled workflows instead can be migrated together with their scheduled jobs. Migrating scheduled executions from Server to Hub means that the same workflow will be scheduled on the Hub with the same configuration as on the Server.

By default, those schedules are only created and configured but not enabled. It is recommended to check that a workflow still executes successfully on the Hub before enabling it manually on the Hub. However, if desired, you can also select the option "Enable active schedules on Hub" when performing the migration via the migration workflow. If this option is selected schedules that were enabled on the Server will immediately be enabled on the Hub as well.

Note that “Call Workflow” and “Save Report” actions as well as configured credentials cannot be migrated to the Hub. They require manual configuration.

Migration workflow overview

The “KNIME Server to KNIME Hub Migration” workflow helps you to migrate the following content from your KNIME Server to a team on the KNIME Business Hub:

  • Files (workflows, components, data files) and folders

    • They will be transferred unmodified while keeping the folder structure.

  • Workflow schedules

    • They will be configured the same way as on KNIME Server. “Call Workflow” and “Save Report” actions as well as configured credentials cannot be migrated and require manual configuration afterward.

  • Workflow snapshots

    • In Business Hub, the concept of workflow snapshots is now improved. During the migration process, it is possible to copy the snapshots over to Hub as workflows. They are copied to a separate backup folder in the appropriate space and contain the date and comment of the snapshot in their name for later identification.

The workflow needs to be uploaded to the Server and executed in the WebPortal. It will interactively guide the user through different migration steps.

The workflow is expected to be run by a user that is the administrator of the Hub team where the content should be migrated to. The user also needs to have permission to access all files on KINME Server that should be migrated.

Requirements, prerequisites, and limitations

Requirements

Technical

  1. KNIME Business Hub:

    1. KNIME Business Hub version 1.5 or higher.

      1. For migration toward KNIME Business Hub version 1.4 or lower please follow the documentation here.

    2. The Server network where the migration workflow will be executed needs to have access to the Hub instance, especially to the api subdomain https://api.<base-url>.com/.

  2. KNIME Server:

    1. Version 4.14 or higher.

    2. KNIME Executor version 4.5 or higher.

    3. The following extensions need to be installed on the Executor:

    4. The KNIME Server installation needs to allow other authentications than OAuth. If not, a service account needs to be created in KNIME Server that can be authenticated via Basic Authentication.

User

  1. Users need to exist with the following role and permissions:

    1. On KNIME Server: a user with permission to access all the content on the Server that should be migrated to the Hub.

    2. On KNIME Business Hub: a user that is an admin of the Hub team you are migrating the content to.

Prerequisites

  1. You should be familiar with new concepts on the Hub, see the KNIME Business Hub User Guide.

  2. You need to create an application password on the Business Hub that will be used by the migration workflow.

  3. If you want to migrate schedules, an execution context is required in the Hub team you are migrating the content to.

Limitations

There are a few things that cannot be migrated that you should be aware of. If those things affect you depends on the current Server setup.

  1. Workflows will be transferred to the Hub unmodified. Generally, a workflow will still run and do on the Hub the same as on Server. However, if there are file paths or shared components used within a workflow, such paths or links might be broken afterward. It is recommended to check after migration manually if such workflows still work and, if not, re-configure them. Affected workflows contain, e.g.

    1. A reader or writer node (e.g. Excel Reader, CSV Writer).

    2. A node that calls another workflow (e.g. Call Workflow Service).

    3. A shared/linked component.

  2. The access to external systems (e.g. databases, file systems) could be different on the new Hub installation compared to the Server. If a workflow relies on such a connection, make sure to check that the connection can still be established. If not, you might need a reconfiguration to be set up by an IT administrator. Affected workflows contain, e.g.

    1. A database node.

    2. A node that connects to an external system (e.g. Snowflake Connector, Amazon S3 Connector, SSH Connector).

  3. Scheduled executions on the Server could be configured with certain actions that take place after successful or failed execution. In case there were "Call Workflow" or "Save Report" actions configured, those will require reconfiguration on the Hub as workflow and file paths cannot be mapped from Server to Hub. The migration workflow informs you about such cases.

  4. Scheduled executions on the Server could be configured with credentials that are used by the workflow. For security reasons, credentials cannot be migrated to the Hub and need to be reconfigured. The migration workflow informs you about such cases.

  5. The permissions concept changed on the Hub. While on Server, permissions could be set on the file level, on Hub they are set on the space level. If you have fine-granular permissions configured on Server, make sure to give proper space access to other users after migration.

  6. The migration workflow does not perform any kind of user migration.

Application Guide

You can download the workflow from this KNIME Community Hub space.

Upload the “KNIME Server to KNIME Hub Migration” workflow to the Server and run it in the WebPortal.

The workflow guides you interactively through different steps that support migration from KNIME Server to KNIME Hub, including copying files and folders from Server to Hub spaces and configuring schedules.

After migration, it is recommended to double-check that the content has been fully transferred to the Hub. Also, workflows might not successfully execute anymore and require manual adjustment, e.g. of file paths, or component links.

Configure Server and Hub Connections

On the first page, it’s necessary to establish connections with both the Server and the Hub. Usually, the connection to the Server can be configured automatically as the workflow is running on the Server. However, the connection to the Hub requires manual configuration. To do this, you’ll need to provide the location of the Hub and the application password of the Hub user.

configure server hub connection

After establishing a connection to the Hub, you can select a Hub team to migrate the content to. You can only select a team of which the Hub user is an administrator.

Multiple or Single Space Migration (Selection)

On the second page, you can select the files and folders on the Server that you want to migrate to the Hub. You can perform two types of migrations:

Migrate to a single space

With this migration method, the files and folders from the Server that you select are migrated to a single space on the Hub. This space can be either one that already exists or a new one.

single space migration

Migrate to multiple spaces

With this migration method, you can select several folders on the Server to migrate to the Hub. The content of each selected folder will be transferred to a space on the Hub that has the same name as the folder. If a space does not already exist, it will be created automatically.

multiple spaces migration

Clicking the “Analyze” button is optional and gives insights into the selected folders and files, such as the number of workflows and scheduled executions.

Create space

The initial step of the migration process involves creating the necessary spaces on the Hub. The resulting outcome of this step is shown on this page.

create space

Copy files/folders to Hub

In the following step, content (including workflows, components, and data files) is first downloaded from the Server and afterward is uploaded to the Hub. The folder structure is retained. The outcome of this step can be viewed on this page.

copy files folders to hub

Backup Snapshots

This page is only displayed if the selected folders contain workflow snapshots. Here you can choose whether you want to migrate them to the Hub.

Snapshot migration means that they are transferred to the Hub as workflows. They will be located in a separate backup folder in the appropriate space and contain the date and comment of the snapshot in their name for later identification.

backup snapshots

The following page will show the outcome of the snapshot migration if it was enabled.

backup snapshots results

Execution Context Selection

On this page, you have the option to choose an execution context to assign as the default execution context for the previously created spaces. An execution context is necessary to create deployments on the Hub, so selecting one is mandatory if you plan to migrate scheduled executions from the Server to the Hub. In case there are no available execution contexts, you can create one (see documentation) and then click the "Refresh" button. If no execution context is selected, the migration of scheduled executions will be skipped.

execution context selection

The following page will show the outcome of the execution context assignment if one has been selected.

execution context selection results

Select Schedules to Migrate

This page is only shown if an execution context has been selected in the previous step. It displays scheduled executions configured on the Server and allows you to select the ones that should be migrated to the Hub.

Migrating scheduled executions from Server to Hub means that the same workflow will be scheduled on the Hub with the same configuration as on the Server. By default, those schedules are only created and configured but not enabled. It is recommended to check that a workflow still executes successfully on the Hub before enabling it manually on the Hub. However, if desired, in this page you can also select the option "Enable active schedules on Hub". If this option is selected schedules that were enabled on the Server will immediately be enabled on the Hub as well.

Note that “Call Workflow” and “Save Report” actions as well as configured credentials cannot be migrated to the Hub. They require manual configuration.

select schedules to migrate

The following page shows the outcome of the migration of the schedules.

select schedules to migrate results

Migration Summary

The last page allows you to review the logs written during the whole migration process. The logs are also stored in the “Hub Migration Logs” folder next to the migration workflow on the Server. In addition, the page allows you to directly download the file.

migration summary

Wrap up

After the migration workflow is finished, we recommend manually checking if the migration was done as expected.

  1. Were all spaces created as expected?

  2. Do the spaces contain all the folders, workflows, and data files as expected?

  3. If you selected to backup workflow snapshots on the Hub: Were snapshot backup folders created in the corresponding spaces containing all the workflow snapshots?

  4. If you selected to migrate scheduled executions: Have corresponding schedule deployments been created on the Hub?

It is important to understand that workflows have been copied unmodified to the Hub. If a workflow requires certain connections to other systems or relies on files being present in certain locations, it might need manual reconfiguration on the Hub. Check if all workflows, especially ones being deployed on a schedule, do still execute successfully. See also the Limitations section which describes possibly required manual follow-up actions.