Kubernetes is an open-source system that automates deployment, scaling, and management of applications run in containers. The updated guidance refreshes the two agencies’ first Cybersecurity Technical Report regarding Kubernetes hardening guidance from August 2021. CISA says the update contains additional details and explanations based on feedback from industry, including more detailed info on logging and threat detection in addition to other clarifications. SEE: What is cloud computing? Everything you need to know about the cloud explained Some of the updates are subtle but important for those who protect Kubernetes clusters. NSA and CISA do not list what the changes are in the updated guidance, but the initial recommendations weren’t met with universal approval. For example, NCC Group noted that advice about Kubernetes authentication was “largely incorrect when it states that Kubernetes does not provide an authentication method by default”, whereas most customer implementations NCC Group had reviewed “support both token and certification authentication, both of which are supported natively.” NCC Group advised against both for production loads because Kubernetes does not support certificate revocation, which can be a problem if an attacker has gained access to a certificate issued to privileged accounts. The updated guidance now says that “several user authentication mechanisms are supported but not enabled by default.” Otherwise, key points of the original document appear to be unchanged. It looks at hardening within the context of typical Kubernetes cluster designs that include the control plane, worker nodes (for running containerized apps for the cluster), and pods for containers that are hosted upon these nodes. These clusters are often hosted in the cloud and across multiple clouds in AWS, Azure, Google and elsewhere. The agencies maintain that Kubernetes is commonly targeted for data theft, computational power theft, or denial of service. Historically, flaws in Kubernetes and various dependencies as well as misconfigurations have been used to deploy crypto miners on victim’s infrastructure. It also maintains that Kubernetes is exposed to significant supply chain risks because clusters often have software and hardware dependences built by third-party developers. For example, security analysts last year warned of attacks against Kubernetes clusters via misconfigured Argo Workflows container workflow engines for K8s clusters. Besides supply chain risks, other key actors in the agencies’ threat model include malicious outsiders and insider threats. These help define its hardening recommendations. For example, there is a common cloud case where workloads that aren’t managed by a given Kubernetes cluster share the same physical network. In that instance, a workload may have access to the kubelet and to control-plane components, such as the API server. So, the agencies recommend network-level isolation. The agencies provide advice on how to ensure strict workload isolation between pods running on the same node in a cluster, given that Kubernetes doesn’t by default guarantee this separation. Announcing the updated guidance, the NSA says: “Primary actions include the scanning of containers and pods for vulnerabilities or misconfigurations, running containers and pods with the least privileges possible, and using network separation, firewalls, strong authentication, and log auditing.” The agencies also recommend periodic reviews of Kubernetes settings and vulnerability scans to ensure appropriate risks are accounted for and security patches are applied. SEE: There’s a critical shortage of women in cybersecurity, and we need to do something about it But patching is not easy in the context of Kubernetes. CISA regularly publishes alerts about new Kubernetes-related vulnerabilities. In February, for example, it warned of a critical (severity score 8.8 out of 10) privilege escalation flaw, CVE-2022-23652, which affected the capsule-proxy reverse proxy for Capsule Operator. But as NCC Group points out: “patching everything is hard”, partly because of the pressure to avoid downtime, but also because vulnerabilities span Kubernetes, Containerd, runc, the Linux kernel, and more. “This is something that Kubernetes can help with, as the whole concept of orchestration is intended to keep services running even as nodes go on and offline. Despite this, we still regularly see customers running nodes that haven’t had patches applied in several months, or even years. (As a tip, server uptime isn’t a badge of honour as much as it used to be; it’s more likely indicative that you’re running an outdated kernel),” NCC Group noted.
