Affected Releases

Hybrid Cloud Management 2017.XX Releases

Hybrid Cloud Management 2018.XX Releases

Impact

Docker-runc through 1.0-rc6, as used in Docker before 18.09.2 and other products, allows attackers to overwrite the host runc binary (and consequently obtain host root access) by leveraging the ability to execute a command as root within one of these types of containers:

1. A new container with an attacker-controlled image, i.e. a non-MicroFocus image that a customer would have to download from the attacker and run it with the suite containers in the CDF K8s cluster, or
2. An existing container that is running in the CDF K8s cluster, to which the attacker previously had write access to manipulate the container maliciously, that is then attached to with docker exec by a regular user of the cluster.

Hybrid Cloud Management Suite is based on Micro Focus Container Deployment Foundation and included an impacted Docker version before its 2019.02 release. It is not a supported or recommended option to run non-MicroFocus images in a CDF deployment, and downloading of Micro Focus images from Docker Hub can be secured by validating the code signing signatures on the images,

so exploitability to the listed vulnerability (1) has been reduced. Exploitability of (2) depends on how well gated the particular CDF deployment is with regards to intruders to the cluster.

Mitigation Actions

• Option 1 (Recommended): Upgrade to Hybrid Cloud Management Suite 2019.02 Release.
• Option 2: If running Hybrid Cloud Management Suite 2018.11
  • Upgrade OS to RHEL 7.6
  • Apply CDF patch. Please follow patch documentation

Patch Readme

This patch package provides a patch_runc.sh script to replace some specific docker-runc binaries in certain CDF releases.
It will replace the docker-runc binaries under $K8S_HOME/bin/ and docker-runc binaries in the $K8S_HOME/zip/ITOM_Suite_Foundation_Node.tar.gz package.
----------------------------------------------------------------------------------------------------------
Patch Fixes
This patch contains fixes below
Number       Impacted Version        Severity     Description        
----------------------------------------------------------------------------------------------------------
QCCR8B33550      2018.11               High        Fix the runc security issue which allows attackers to
                                                  overwrite the host runc binary. See patch CVE-2019-5736 for details.
----------------------------------------------------------------------------------------------------------
Patch Installation

Perquisites:
Before you install the patch, make sure all your worker nodes and master nodes have at least 4 GB free disk space under the filesystem where $K8S_HOME locates.

Installation steps:
Perform the following steps one by one to install the patch.
1.    Run the following command on one of the master nodes. This step is to cordon all nodes to avoid the pods shifting continuously during patching.
    # kubectl get nodes -o custom-columns=NAME:.metadata.name --no-headers|xargs -i -t kubectl cordon {}
2.    Perform the following steps on all master nodes and worker nodes one after another. You must wait for the following steps to finish running on one node before you perform them on another node.The steps MUST NOT be run in parallel on multiple nodes
    2.1    Run the following commands to stop Kubernetes.  
        # cd $K8S_HOME/bin
        # ./kube-stop.sh
    2.2    After you have stopped Kubernetes, run the following command under the directory where patch_runc.sh locates:
        # ./patch_runc.sh
    2.3    Run the following commands to restart Kubernetes.
        # cd $K8S_HOME/bin
        # ./kube-start.sh
    2.4    Run the following command on the current node to uncordon the node. Replace <current-node> with the IPv4 address or hostname of the current node.
        # kubectl uncordon <current-node>
        Note:
        You can run the command “kubectl get nodes” to list all the nodes in cluster and find the current node through IPv4 address or hostname.
        Run the command "hostname -f" or "hostname -i" to find the IPv4 address or hostname of the current node.
3.    Run the following command to check the patching result.               
    # $K8S_HOME/bin/docker-runc -v
    If you get the "654fccfbaf89aac424b877dd1d914cee39262706a" as the commit value, you have applied the patch successfully.

Rolling Back
When you encounter some unexpected errors while patching, follow the steps below to roll back the patch.

1.    Run the following command on one of the master nodes. This step cordons all nodes to avoid the pods shifting continuously during patching.
    # kubectl get nodes -o custom-columns=NAME:.metadata.name --no-headers|xargs -i -t kubectl cordon {}
2.    Perform the following steps on all the master nodes and worker nodes one after another. You must wait for the commands to finish running on each node before you run it on another node.
    2.1    Run the following commands to stop kubernetes:   
        # cd $K8S_HOME/bin
        # ./kube-stop.sh
    2.2    Get the backup copy of docker-runc binary. The backup copy of docker-runc is placed under $K8S_HOME/bin/docker-runc-${commit_id}.
       Replace {commit_id} with the hashed docker-runc version.
       For example, “/opt/kubernetes/bin/docker-runc-09c8266bf2fcf9519a651b04ae54c967b9ab86ec”.
    2.3    Run the following commands to restore docker-runc under $K8S_HOME/bin.
        # cd $K8S_HOME/bin
        # cp -f docker-runc-${commit_id}  docker-runc
    2.4    Run the following command to restore docker-runc under $K8S_HOME/zip on the node where the patch was applied.
        # cd $K8S_HOME/zip
        # gunzip ITOM_Suite_Foundation_Node.tar.gz
        # tar -xf ITOM_Suite_Foundation_Node.tar
        # cp -f $K8S_HOME/bin/docker-runc ITOM_Suite_Foundation_Node/bin/
        # tar -czf ITOM_Suite_Foundation_Node.tar.gz ITOM_Suite_Foundation_Node
        # rm -fr  ITOM_Suite_Foundation_Node.tar
        # rm -fr  ITOM_Suite_Foundation_Node
    2.5    Run the following commands to start Kubernetes.
        # cd $K8S_HOME/bin
        # ./kube-start.sh
    2.6    Run the following command on the current node to uncordon the node.
        Replace the <current_node> with the current node IPv4 address or hostname.
        # kubectl uncordon <current-node>
         Note:
        You can run the command “kubectl get nodes” to list all the nodes in cluster and find the current node through IPv4 address or hostname.
        Run the command "hostname -f" or "hostname -i" to find the IPv4 address or hostname of the current node.