KNIME Server to KNIME Hub Migration Guide

User management on KNIME Business Hub is handled via keycloak and differs dramatically from the User management on KNIME Server.

An interesting feature that is introduced with KNIME Business Hub is the usage of application passwords that will provide the users with the ability to provide authentication when using the KNIME Hub REST API, for example when executing deployed REST services. You can find more information on how to create and use application passwords in the Application passwords section of the KNIME Business Hub User Guide. If you use the migration workflow, you will use the application password of a team member (see below) from a team you want to migrate content to in order to authenticate.

This is a new concept on KNIME Hub, as a successor of KNIME Server Groups. A team is a group of users on KNIME Hub that work together on shared projects. A Team will own specific Hub resources, to which only Team members will have access.

When users are migrated to the Hub one or more teams are created (depending on the type of license or your KNIME Business Hub) and users are assigned to a team. Additionally, each user has a role assigned, which can be the following:

When migrating using the migration workflow you will migrate items from KNIME Server to KNIME Hub to a specific team. For this reason, you will need to access the KNIME Business Hub as a user that has the Member or Administrator role in the specific team.

One core concept of the KNIME Hub is the concept of “spaces”. A space is a collection of items used to organize components, workflows and files. Spaces are intended to organize a team’s work around a specific project or domain and they are owned by a team. Members and admins of the teams have access to the spaces by default, but permissions can be defined for each role or user, i.e. space access permissions can be restricted for certain users in the team.

The roles and permissions assigned to the team members or to the single users can then be assigned by the team admins.

During the migration process we recommend to transfer top level folders on the Server into a space on the Hub. Using the migration workflow described in the following sections you will be able to select one or multiple folders from the Server and to move their content (workflows, snapshots, schedules, components, data files, …​) to one or more spaces.

The snapshot feature of KNIME Server relates to item versions on KNIME Business Hub. A version is a snapshot in time of an item on Business Hub and is associated with some metadata (title, description, creator, creation timestamp, …​).

An item version history is available to the team members as a list of all the versions that have been created of a specific item.

When using the migration workflow, KNIME Server snapshots can be transferred to the Hub as item versions. Migrated snapshots will bear the title “VX snapshot from KNIME Server”, where the X will be the number of the snapshot (low number: old snapshot; high number: recent snapshot). The item as it resided on KNIME Server will always be versioned by the migration workflow and bear the title “VY from KNIME Server”, Y being the number of KNIME Server Snapshots plus one. The metadata of the KNIME Server Snapshot is given in the Version Description on the KNIME Hub Item Version.

On KNIME Server, when a workflow is uploaded to the Server it is automatically possible to execute the workflow as a data app via the KNIME WebPortal, to schedule the execution of the workflows via the KNIME Analytics Platform, and to use the workflow as a service to be called either by another workflow or by an external tool.

While it is still possible to do ad hoc execution of workflows, which is intended for one-time usage e.g. during development, an extra step is introduced on KNIME Hub to separate the workflow from its intended usage: Deployments. With KNIME Business Hub, the upload of a workflow to a space on the Hub needs to be followed by the creation of a version and then the creation of the desired deployment. There are different types of deployments on KNIME Business Hub:

Trigger deployments are entirely new, and allow to trigger a workflow when a file was modified, for example, or a new version of a workflow was created.

We provide examples for each deployment, which are explained here.

Since it is not possible to detect a priori whether a workflow is intended as a data app or is executed as a service on KNIME Server, the migration process will need a manual step where the workflows are deployed as data apps or services.

Scheduled workflows instead can be migrated together with their scheduled jobs. Migrating scheduled executions from Server to Hub means that the same workflow will be scheduled on the Hub with the same configuration as on the Server.

By default, those schedules are only created and configured but not enabled. It is recommended to check that a workflow still executes successfully on the Hub before enabling it manually on the Hub. If desired, however, you can change this setting in the migration workflow.

Note that “Call Workflow” and “Save Report” actions as well as configured credentials cannot be migrated to the Hub. They require manual configuration, and the migration workflow will give further instructions if this case arises.

Please note that if you want to migrate the schedule deployments of workflows the team you want to migrate to needs to have access to an execution context.

Workflow execution on KNIME Business Hub does still happen on Executors like on the KNIME Server, only that they are wrapped in so-called Execution Contexts. You configure them with similar settings as on the Server (job timeouts, swapping, …​). Creation of execution contexts is handled by the Team administrators or the Global admin, and they can be made available for all teams. Follow this guide on how to set up an execution context.

The configuration of the Executor, e.g. the KNIME Analytics Platform version used for execution, the installed extensions and so on, are now bundled into an Executor docker image that can be configured by the KNIME Business Hub Admin, as explained here. Each execution context has the desired executor image assigned when created.

Dependent on your Server setup, there are a few things that cannot be migrated that you should be aware of.

A list of known incompatibilities is preloaded into the tool via Table Creator nodes.

This list categorizes incompatibilities into three main groups:

The list is flexible and customizable. New incompatibilities can be easily added to the tables if more issues are identified during usage. This allows you to extend and adapt the tool as new cases arise.

On the first page, you establish connections with both the Server and the Hub.

Usually, the connection to the Server can be configured automatically as the workflow is running on the Server. If not, try the “Connect to Server” button at least once, or switch to “Manual authentication” and enter your credentials. Alternatively, if the Server is configured to do OAuth, you can authenticate via JWT if you select the “Token-based authentication”. Double check the KNIME Server URL and then enter the contents of the file that you download from <knime-server-url>/knime/rest/v4/auth/jwt.

The connection to the Hub requires manual configuration. To do this, you’ll need to provide the URL of the Hub and the application password of the Hub user, who is a team member in the destination team. The process of obtaining an application password is described in the KNIME Hub User Guide.

On the second page, you select the destination team and space(s) and the actual data on the KNIME Server you want to migrate to the KNIME Hub.

In case there were any errors during the migration, they will show up first. Please try to make sense of the errors, and act accordingly if possible.

Similar to the Overview prior to the migration, you will then find an overview of the items that were actually migrated. You can tick the “Migrate to more teams?” box to start the migration all over again with the same connection settings when clicking the “Next” button.

If there were Shared Metanode in the data-selection, a warning is issued, with the possibility to delete them right away, since they are a feature not supported on KNIME Hub. We recommend changing shared metanodes to shared components.

Below the overview, you will find the migrated items in a table, from which you can access the items directly via link.

Finally, the new schedule deployments are linked, and any potential issues that might need attention accordingly. Possible Issues are:

If you didn’t choose to migrate to another team, a click on the “Next” button leads you to the last page. It shows a migration log that was created during the whole migration process. The logs are available for download and are also stored in the “KNIME Server to KNIME Hub Migration Logs” folder next to the migration workflow on the Server.

This workflow is designed to be run locally or controlled via the Job Viewer.

There are two main workflow parameters on the first level that need to be defined.

First, you need to choose which mountpoint should be targeted by the analysis. The mountpoint connector allows you to connect to a Server or a Local space. If you have already migrated all the content using the migration workflow, you can connect to the List of Nodes metanode and run the analysis in the same manner.

In the yellow circle below, the hardcoded incompatibilities are listed in a table that can be easily filled out and modified further. These are grouped by topic. The table also specifies the corrective action to facilitate an estimate of the amount of work to be done as well as provide immediate feedback.

The workflow reads the workflow XML file and extracts a list of nodes and configurations. It then checks for and finds matches with the hardcoded lists of incompatibilities, providing a report at the end: