Skip to content

Add-on feature

Sovereign Execution is a paid add-on available for the Business and Enterprise plans. For all available add-ons, see Add-ons.

Use a permanent Python environment ​

Use a permanent environment when you want workflows to run with a Python environment that you control. You install the environment once on your sovereign executor, and workflow builders select it with the Python Environment Reader (Labs) node. To create an environment for each run instead, which also works on managed environments, see Set up Python for an environment.

INFO

Permanent environments work only on sovereign executors, as you need access to the executor installation folder. Environments that Knime manages don't support them.

Install the environment ​

To install the environment:

  1. Go to your executor installation folder.
  2. Open the bundling folder, next to the configuration, dropins, and features folders. This folder exists only if the Knime Python Integration is installed.
  3. In the bundling folder, create a folder for your environment, for example my-python-env. Workflow builders select the environment by this folder name.
  4. In this folder, create the Pixi environment. It must contain knime-python-base and the packages your workflow builders need. knime-python-base provides pandas, pyarrow, py4j, and the other packages that Knime Python nodes require.

The result is a complete Pixi project. The node uses only the default Pixi environment:

text
knime/
└── bundling/
    └── my-python-env/
        ├── .pixi/
        │   └── envs/
        │       └── default/
        ├── pixi.lock
        └── pixi.toml

Make the environment read-only ​

Knime deletes writable environments in bundling/

Every time the executor restarts, Knime deletes the folders in bundling/ that don't belong to an installed extension. To keep your environment, make it read-only and owned by root. You can't turn this cleanup off.

To make the environment read-only:

  1. Import pandas once in the environment. Python writes cache files on the first import, which isn't possible after the next step.
  2. Change the permissions of the environment folder:
    • On Linux, make root the owner and remove write access for all users.
    • On Windows, give administrators and SYSTEM full control, and give users read and execute access only.
  3. Restart the executor.

After each restart, the log shows Failed to delete unused Conda environment directory once for your environment. This error is expected and means that the protection works.

Use the environment in a workflow ​

To use the environment in a workflow:

  1. Add the Python Environment Reader (Labs) node to the workflow.
  2. In the node, keep Input Mode on Bundled Environment and select my-python-env from the drop-down menu.
  3. On the Python node, for example the Python Script node, add the Python environment input port from the node's + menu.
  4. Connect the Python Environment Reader (Labs) node to this port.

Update the environment ​

To change the packages in a permanent environment, repeat the installation steps with the updated pixi.toml and pixi.lock files. Running workflows keep using the files that are already there.

Prevent workflow builders from creating their own environments ​

To prevent workflow builders from creating their own environments, blacklist the Python Environment Provider node in the sovereign environment. Its node factory ID is org.knime.pixi.nodes.PythonEnvironmentProviderNodeFactory. For the steps, see Blacklist nodes.

Workflow builders can still point the Python Environment Reader (Labs) node to a pixi.toml file. This doesn't let them create an environment, because the node never installs or updates anything and needs a complete Pixi project next to the file.