Quick Links

View 
SPDX-License-Identifier: Apache-2.0
Copyright (c) 2019 Intel Corporation

Userspace CNI

Overview

Userspace CNI is a Container Network Interface (CNI) Kubernetes* plugin that was designed to simplify the process of deployment of DPDK-based applications in Kubernetes pods. The plugin uses Kubernetes and Multus CNI CRDs to provide pods with a virtual DPDK-enabled Ethernet port. In this document, you can find details about how to install Smart Edge Open with Userspace CNI support and how to use its main features.

Setup Userspace CNI

Smart Edge Open for Network Edge has been integrated with Userspace CNI to allow users to easily run DPDK- based applications inside Kubernetes pods. To install Smart Edge Open Network Edge with Userspace CNI support, add the value userspace to variable kubernetes_cnis in inventory/default/group_vars/all/10-open.yml and set value of the variable kubeovn_dpdk in inventory/default/group_vars/all/10-open.yml to true:

# inventory/default/group_vars/all/10-open.yml
kubernetes_cnis:
- kubeovn
- userspace

# inventory/default/group_vars/all/10-open.yml
kubeovn_dpdk: true

HugePages configuration

DPDK apps require that a specific number of HugePages are enabled. By default, the Ansible* scripts will enable 1024 of 2M HugePages on a system and then start OVS-DPDK with 1Gb of those HugePages. To change this setting to reflect your specific requirements, set the Ansible variables as defined in the example below. This example enables 4 of 1GB HugePages and appends 1 GB to OVS-DPDK, leaving 3 pages for DPDK applications that will be running in the pods.

# inventory/default/group_vars/controller_group/10-open.yml
hugepage_size: "1G"
hugepage_amount: "4"
default_grub_params: "default_hugepagesz= hugepagesz= hugepages= intel_iommu=on iommu=pt"

# inventory/default/group_vars/edgenode_group/10-open.yml
hugepage_size: "1G"
hugepage_amount: "4"
default_grub_params: "default_hugepagesz= hugepagesz= hugepages= intel_iommu=on iommu=pt"

# inventory/default/group_vars/all/10-open.yml
# Hugepage size to be used with DPDK: 2Mi or 1Gi
kubeovn_dpdk_hugepage_size: "1Gi"
# Overall amount of hugepages available to DPDK
kubeovn_dpdk_hugepages: "1Gi"

Pod deployment

To deploy a pod with a DPDK interface, create a pod with hugepages mounted to /dev/hugepages, host directory /var/run/openvswitch/ (with mandatory trailing slash character) mounted into the pod with the volume name shared-dir (the name shared-dir is mandatory), and userspace-openness network annotation. Find an example pod definition with two DPDK ports below:

apiVersion: v1
kind: Pod
metadata:
  name: userspace-example
  annotations:
    k8s.v1.cni.cncf.io/networks: userspace-openness, userspace-openness
spec:
  containers:
  - name: userspace-example
    image: image-name
    imagePullPolicy: Never
    securityContext:
      privileged: true
    volumeMounts:
    - mountPath: /ovs
      name: shared-dir
    - mountPath: /dev/hugepages
      name: hugepages
    resources:
      requests:
        memory: 1Gi
      limits:
        hugepages-1Gi: 2Gi
    command: ["sleep", "infinity"]
  volumes:
  - name: shared-dir
    hostPath:
      path: /var/run/openvswitch/
  - name: hugepages
    emptyDir:
      medium: HugePages

Virtual interface usage

Socket files for virtual interfaces generated by Userspace CNI are created on the host machine in /var/run/openvswitch directory. This directory has to be mounted into your pod by volume with obligatory name shared-dir (in our example pod definition /var/run/openvswitch is mounted to pod as /ovs). You can then use the sockets available from your mount-point directory in your DPDK-enabled application deployed inside a pod. Find more examples in Userspace CNI’s documentation.

NOTE: When 2MB HugePages are enabled for Userspace CNI there is a limitation to how many memory regions can be used by Virtio - in an event where number of memory regions is exceeded by DPDK application, a flag ‘–single-file-segments’ needs to be added to the applications starting command line.

On this page: