Skip to main content

New K8s Workload Created With Host Access

This alert occurs when Lacework detects a workload was created with access to host resources.

Why this alert is important​

Generally, most application workloads need limited access to host resources to run successfully as a root process (uid 0) without access to host information. Workloads with access to the host can bypass the containerization and isolation provided by Kubernetes. They can potentially access and manipulate sensitive resources on the host, such as system files, network configurations, and other containers running on the same host. This can lead to unauthorized data access, data leakage, or compromise of the entire host.

Investigation​

Follow these steps to investigate the alert:

  1. Examine the Kubernetes cluster configuration, including ClusterRoleBinding and ClusterRole, to identify any roles or permissions that grant access to the host. Look for any misconfigurations or unauthorized role bindings.
  2. Review the Kubernetes audit logs to identify any relevant events or activities related to creating the workload with host access. Look for log entries that indicate the creation of privileged workloads, modifications to RBAC settings, or suspicious activity that could suggest unauthorized access to the host.
  3. Analyze the specifications of the created workloads, particularly the Pod manifests, to identify any privileged settings or volumes mounted from the host. Look for indicators such as hostPath volumes, privileged containers, or capabilities that grant broad host access.
  4. Monitor the network traffic within the Kubernetes cluster to detect any unusual communication patterns or traffic originating from the privileged workload. Look for connections to the host IP or suspicious network activities that could indicate unauthorized access.
  5. Dive into the logs generated by the container runtime to identify any abnormal behavior or evidence of unauthorized access to the host. Look for logs related to container escapes, direct interactions with host resources, or unauthorized modifications to host-level configurations.
  6. Perform runtime monitoring, threat detection, and vulnerability scanning to identify potential security risks and anomalies within the Kubernetes environment, including workloads with host access.
  7. If there is suspicion of a security incident or compromise, conduct a forensic analysis of the affected host and workloads. Collect and analyze system logs, file system artifacts, and other relevant data to determine the extent of the access, identify potential attacker activities, and establish a timeline of events.

Resolution​

Follow these steps to resolve the alert:

  1. Immediately disable or remove the unauthorized workload from the Kubernetes cluster. This helps prevent unauthorized activities and limits the potential impact on the host and other workloads.
  2. Perform a thorough analysis of the host system to identify any unauthorized modifications, malicious artifacts, or potential backdoors. This may involve conducting a forensic investigation or utilizing security tools to identify and remediate any vulnerabilities or unauthorized changes on the host.
  3. Evaluate the RBAC (Role-Based Access Control) configuration and permissions within the Kubernetes cluster. Identify any misconfigurations or unauthorized privileges that allowed the workload to access the host. Update the RBAC settings to ensure only authorized workloads have appropriate access levels, and revoke any unnecessary privileges.
  4. Ensure that the host system, container runtime, and Kubernetes components are up-to-date with the latest security patches and updates. This helps protect against known vulnerabilities and strengthens the overall security posture of the cluster.
  5. Apply security best practices to harden the host system and Kubernetes environment. This includes configuring appropriate network policies, securing communication channels, enabling audit logging, implementing container isolation, and enforcing strict security controls to minimize the attack surface.
  6. Perform regular security audits and assessments of the Kubernetes environment to identify any security gaps or potential vulnerabilities. This can include vulnerability scanning, penetration testing, or engaging third-party security experts to assess the overall security posture and provide recommendations for improvement.
  7. Implement robust monitoring and logging mechanisms to detect future unauthorized access attempts or suspicious activities within the Kubernetes cluster. Use tools and technologies such as intrusion detection systems, log analysis solutions, and real-time monitoring to proactively identify and respond to security incidents.