Create a New Python based KNIME Extension

This is a quickstart guide that will walk you through the essential steps of writing and running your first Python Node Extension containing a single node. We will use tutorial_extension as the basis. The steps of the tutorial requiring modification of the Python code in my_extension.py have corresponding comments in the file, for convenience.

For an extensive overview of the full API, please refer to the Defining a KNIME Node in Python: Full API section, as well as our Read the Docs page.

As you develop your Python extension, you are able to run and debug it locally by setting the knime.python.extension.config system property in your KNIME Analytics Platform’s knime.ini to point to the config.yml, or in the launch configuration’s VM arguments in Eclipse. See the Registering Python extensions during development and Customizing the Python executable sections at the end of this guide for more information.

In order to share your Python extension with others, please refer to the Bundling your Python Extension Nodes section.

The input and output ports of a node can be configured by decorating the node class with @knext.input_table, @knext.input_binary, and respectively @knext.output_table and @knext.output_binary. Additionally, an output port producing a view can be added with the @knext.output_view decorator.

These port decorators have the following properties:

The @knext.input_table and @knext.output_table decorators configure the port to consume and respectively produce a KNIME table.

If you want to receive or send other data, e.g. a trained machine learning model, use @knext.input_binary and @knext.output_binary. This decorator has an additional argument, id, used to identify the type of data going along this port connection. Only ports with equal id can be connected, and it is good practice to use your domain in reverse to prevent id clashes with other node extensions. The data is expected to have type bytes.

The port configuration determines the expected signature of the configure and execute methods:

Here is an example with two input ports and one output port.

Alternatively, you can populate the input_ports and output_ports attributes of your node class (on class or instance level) for more fine grained control.

Each node in your Python node extension is assigned a category via the category parameter of the @knext.node decorator, which dictates where the node will be located in the node repository of KNIME Analytics Platform. Without an explicit category, the node will be placed in the root of the node repository, thus you should always specify a category for each node.

In order to define a custom category for the nodes of your extension, you can use the knext.category helper function. If autocompletion is enabled in your IDE, you should be able to see the list of the expected parameters of the function, together with their detailed description.

If you are a community developer, you should use the Community Nodes category as the parent category of your Python node extensions. This is done by specifying the path="/community" parameter of the knext.category function:

Note that it is possible to further split your custom category into subcategories. This is useful if, for instance, nodes of your extension can be grouped based on their functionality. By first defining a parent category for the node extension, you can then use it as the path parameter when defining the subcategories:

In order to add parameterization to your node’s functionality, we can define and customize its configuration dialog. The user-configurable parameters that will be displayed there, and whose values can be accessed inside the execute method of the node via self.param_name, are set up using a list of parameter types that have been predefined:

All of the above have arguments label and description, which are displayed in the node description in KNIME Analytics Platform, as well as in the configuration dialog itself.

Parameters are defined in the form of class attributes inside the node class definition (similar to Python descriptors):

While each parameter type listed above has default type validation, they also support custom validation via a property-like decorator notation. By wrapping a function that receives a tentative parameter value, and raises an exception should some condition be violated, with the @some_param.validator decorator, you are able to add an additional layer of validation to the parameter some_param. This should be done below the definition of the parameter for which you are adding a validator, and above the configure and execute methods:

It is also possible to define groups of parameters, which are displayed as separate sections in the configuration dialog UI. By using the @knext.parameter_group decorator with a dataclass-like class definition, you are able to encapsulate parameters and, optionally, their validators into a separate entity outside of the node class definition, keeping your code clean and maintainable. A parameter group is linked to a node just like an individual parameter would be:

Another benefit of defining parameter groups is the ability to provide group validation. As opposed to only being able to validate a single value when attaching a validator to a parameter, group validators have access to the values of all parameters contained in the group, allowing for more complex validation routines.

We provide two ways of defining a group validator, with the values argument being a dictionary of parameter_name : parameter_value mappings:

Intuitively, parameter groups can be nested inside other parameter groups, and their parameter values accessed during the parent group’s validation:

You can use the @knext.output_view(name="", description="") decorator to specify that a node returns a view. In that case, the execute method should return a tuple of port outputs and the view (of type knime_views.NodeView).

Currently, a Python node is only able to access flow variables that have been propagated to it via a table/binary input port, as opposed to a dedicated flow variable port.

You can access the flow variables available to the node in both the configure and execute methods, via the config_context.flow_variables and exec_context.flow_variables attributes respectively. The flow variables are provided as a dictionary of flow_variable_name : flow_variable_value mappings, and support the following types:

By mutating the flow_variables dictionary, you can access, modify, and delete existing flow variables, as well as create new ones to be propagated to downstream nodes.

To ensure that the users you have shared your extension with are able to utilise its functionality fully and error-free, we bundle the source files together with the required packages using conda as the bundling channel.

The knime.yml file (refer to the Python Node Extension Setup section for an example of this configuration file) contains the information required to bundle your extension, including:

These YAML files can be automatically generated by activating the desired environment, and running one of the following commands, which will result in configuration files of various strictness:

Note that, in addition to packages installed with conda, you are also able to install packages from PyPI via pip. When done from within your activated conda environment, such packages are also automatically included in the YAML configuration file generated with the above commands.

Once you have finished implementing your Python extension, you can bundle it, together with the appropriate conda environment, into a local update site. This allows other users to install your extension in the KNIME Analytics Platform.

Follow the steps of extension setup. Once you have prepared the YAML configuration file for the environment used by your extension, and have set up the knime.yml file, you can proceed to generating the local update site.

We provide a special conda package, knime-extension-bundling, which contains the necessary tools to automatically build your extension. Run the following commands in your terminal (Linux/macOS) or Anaconda Prompt (Windows). They will setup a conda environment, which gives the tools to bundle extensions. Then the extension will be bundled.

The generated repository can now be shared with and installed by other users.

Once you have finished implementing your Python extension, you can share it, together with the appropriate conda environment, to KNIME Hub.

You can use the logging Python module to send warnings and errors to the KNIME Analytics Platform console. By going to File → Preferences → KNIME → KNIME GUI, you can choose the Console View Log Level. Each consecutive level includes the previous levels (i.e. DEBUG will also allow message from INFO, WARN, and ERROR to come through in the console, whereas WARN will only allow WARN and ERROR levels of messages).

In your Python script, you can initiate the logger, and use it to send out messages to the KNIME Analytics Platform console as follows:

In order to allow for a smooth user experience, the Analytics Platform caches the gateways used for non-execution tasks (such as the spec propagation or settings validation) of the last used Python extensions. This cache can be configured via two system properties:

The debug_mode: true propertly of config.yml discussed before effectively disables caching for individual extensions. By default, all extensions use caching.