Azure Blob Storage Setup Guide

This guide provides step-by-step instructions for provisioning and configuring the Azure Blob Storage resources required to back persistent data for a KNIME Business Hub installation.

Note

By default, this guide uses the naming convention $CLUSTER_NAME-<container-name> (e.g., myhub-catalog-service). You may use a different naming scheme if desired by modifying the values in your ArgoCD or Replicated configuration.

Prerequisites

Before starting the setup process, verify you have the following:

Required Tools

• An existing Azure Blob Storage account.
• Azure CLI installed and configured with appropriate permissions, or access to the storage account via the Azure Portal.
• Administrative access to create Azure Blob Storage resources under the storage account.

Existing Infrastructure

• A Kubernetes cluster (EKS recommended) with networking infrastructure (VPC, subnets, security groups) set up based on installation guides (either KURL or existing infrastructure).
• Your KNIME Business Hub domain name (e.g., hub.yourcompany.com).

Security Requirements

• All containers are private by default.
• All data is encrypted both in transit and at rest.

Setup Instructions

1. Configure Environment Variables

Define the following variables in your terminal to streamline the resource creation process:

# Basic Configuration
export CLUSTER_NAME="your-eks-cluster-name"
export HUB_ORIGIN="hub.yourcompany.com"

# Azure Blob Storage
export STORAGE_ACCOUNT="your-storage-account-name"

IMPORTANT

Replace the placeholder values above with your actual infrastructure details.

2. Create Azure Blob Storage Containers

KNIME Business Hub requires four private containers. You can create these via the Azure Portal (Data storage > Containers) or by running the following CLI commands:

# Catalog Service (Workflows and metadata)
az storage container create --name $CLUSTER_NAME-catalog-service --public-access off --account-name $STORAGE_ACCOUNT

# Avatars (User profile pictures)
az storage container create --name $CLUSTER_NAME-avatars --public-access off --account-name $STORAGE_ACCOUNT

# Execution Jobs (Job data and results)
az storage container create --name $CLUSTER_NAME-execution-jobs --public-access off --account-name $STORAGE_ACCOUNT

# Customization Profiles (User settings)
az storage container create --name $CLUSTER_NAME-customization-profiles --public-access off --account-name $STORAGE_ACCOUNT

3. Configure Cross-Origin Resource Sharing (CORS)

You must allow your KNIME Business Hub domain to perform PUT requests against the storage account.

az storage cors add --services b --methods PUT --origins http://$HUB_ORIGIN https://$HUB_ORIGIN --allowed-headers '*' --max-age 3000 --account-name $STORAGE_ACCOUNT

Container Reference & Architecture

For reference, the following table describes the role and default naming convention of the storage buckets created above. All containers are required to be within the same Azure Storage Account.

Container Suffix	Purpose	Access Level
-catalog-service	Stores workflow files and metadata	Private
-avatars	Stores user-uploaded profile images	Private
-execution-jobs	Stores ephemeral job data and results	Private
-customization-profiles	Stores executor and user profiles	Private

Custom Naming

While we recommend the $CLUSTER_NAME-<suffix> convention, you can customize these names during the Hub installation in the ArgoCD or Replicated configuration.

Hub Connection (KOTSADM)

Once the infrastructure is provisioned, connect the services via the KOTSADM UI (typically https://<your-kotsadm-domain>:8800):

Enable Advanced Settings

1. Navigate to the Config tab.
2. Check the View Advanced Settings checkbox.
3. Locate the Advanced: External S3 compatible storage or Azure Blob Storage section.

Configure External Storage

1. Select Azure as the External Storage Type.
2. Provide the Storage API URL (e.g., <your-storage-account-name>.blob.core.windows.net).
3. Enter the specific container names created in the previous steps for Catalog, Avatars, Execution, and Customization.

Configure Credentials

Important

KNIME Business Hub currently only supports Access Keys for authentication.

To retrieve your keys, navigate to Security + networking > Access keys in the Azure Portal, or run:

az storage account keys list -n $STORAGE_ACCOUNT

Option A: UI Configuration

Enter the credentials directly in the Advanced - Credentials section:

• Access Key ID: Your storage account name.
• Secret Access Key: Your storage account access key.

Option B: Manual Kubernetes Secrets

Select Manually provide S3 or Azure Blob Storage credentials as secrets and apply a secret template for each container following this structure:

apiVersion: v1
kind: Secret
metadata:
  name: <container-name>-minio-credentials
  namespace: <hub-namespace>
type: Opaque
data:
  accessKeyId: <base64-storage-account-name>
  secretAccessKey: <base64-storage-account-key>

Deployment

1. Save Configuration: Click "Save Configuration" at the bottom of the page.
2. Preflight Checks: Wait for the checks to finish and verify the results.
3. Deploy: Proceed with deploying the application if preflights succeeded.

Cleanup

If you need to remove the Azure Blob Storage resources created in this guide:

1. Delete Containers:

az storage container delete --name $CLUSTER_NAME-catalog-service --account-name $STORAGE_ACCOUNT;
az storage container delete --name $CLUSTER_NAME-avatars --account-name $STORAGE_ACCOUNT;
az storage container delete --name $CLUSTER_NAME-execution-jobs --account-name $STORAGE_ACCOUNT;
az storage container delete --name $CLUSTER_NAME-customization-profiles --account-name $STORAGE_ACCOUNT

2. Restore CORS Settings: Remove the rule in the Portal under Settings > Resource sharing (CORS), or clear all rules via CLI:

az storage cors clear --services b --account-name $STORAGE_ACCOUNT

Caution

Clearing via CLI removes all rules, which may affect other services using the same storage account.