Migrate From Storage-Based to Pub/Sub-Based GCP Audit Log Integration - Manual Configuration

This topic describes how you can use the Lacework Console to manually migrate your GCP Storage-based audit log integration to a Pub/Sub-based audit log integration for audit log monitoring.

Lacework recommends this migration procedure because it ensures audit log monitoring coverage for your GCP organization or project during the migration.

The migration involves the following four steps. All the steps are required to ensure that there is no gap in delivery of audit log data from GCP to the Lacework platform during the migration.

1. Collect details of existing Storage-based audit log integration
2. Create Pub/Sub topic and subscription
3. Create Pub/Sub-based audit log integration
4. Mark existing Storage-based audit log integration for migration

note

You can also use Terraform to migrate your Storage-based integration to a Pub/Sub-based integration. For more information, see Migrate From Storage-Based to Pub/Sub-Based GCP Audit Log Integration Using Terraform.

Important

If you do not want audit log monitoring coverage for your GCP organization or project during the migration, you can skip this migration procedure and perform the following three steps. Note that this can result in a brief gap in delivery of audit log data from GCP to the Lacework platform.

1. Create a Pub/Sub-based audit log integration using one of the following methods:

   • Guided integration method described in Create an Audit Log (PubSub)+Configuration Integration.

     Caution: The guided integration method will overwrite the Terraform files in the ~/lacework/gcp directory. If this directory contains the Terraform files for an existing GCP configuration or audit log integration, all the resources created for that integration in GCP will be deleted. Hence, Lacework recommends that you delete the main.tf and terraform.tfstate files in the ~/lacework/gcp directory before you use the guided integration method.

   • Manual integration method described in Pub/Sub-based GCP Audit Log Integration - Manual Configuration.

2. In the Lacework Console, go to Settings > Integrations > Cloud Accounts and delete your Storage-based audit log integration.
3. (Optional) Delete the sink and storage bucket for the Storage-based audit log integration. For more information, see Delete the Sink and Storage Bucket for the Storage-based Integration.

Prerequisites

• Ensure that you have the required privileges in Google Cloud. For more information, see Required Roles for GCP Configuration and Audit Log Integrations.
• Google Cloud Console - Administrator access to Google Cloud Console.
• Lacework Console - Org admin or Account admin access to the Lacework Console.
• Lacework CLI

Collect Details of Existing Storage-Based Audit Log Integration

You can reuse the project and service account that you created for the Storage-based audit log integration for the Pub/Sub-based audit log integration. Do the following to collect the project and service account details from the Lacework Console.

1. In the Lacework Console, go to Settings > Integrations > Cloud Accounts.

2. Select the row for the Storage-based integration. A Storage-based integration has the provider as GCP and type as Audit Log (Storage).

   The Cloud Account page displays the integration details.

   • The Account field displays the ID of the project in which you configured the resources for the Storage-based integration.

   • The Client Email field displays the email ID of the service account you created for the Storage-based integration.

     The service account email ID is in the format: my-service-account@my-project-name.iam.gserviceaccount.com.

   • The ID field displays the ID of the integration.

3. Copy the project ID, service account email ID, and integration ID for use in the procedures below.

Create Pub/Sub Topic and Subscription

In this procedure, you will create a Pub/Sub topic and subscription to record audit log events.

1. Create a Pub/Sub topic in the project you identified in the Collect Details of Existing Storage-Based Audit Log Integration procedure. Follow the steps in Create a topic.

2. Create a subscription for the Pub/Sub topic. Follow the steps in Add a subscription.

3. Create a log sink, select the Pub/Sub topic as the sink destination, and add inclusion and exclusion filters.

   1. Do one of the following:

      • For a project-level Pub/Sub audit log integration, create a log sink, select the Pub/Sub topic as the sink destination, and add inclusion and exclusion filters using the instructions in Create a sink.
      • For an organization-level Pub/Sub audit log integration, create an aggregated log sink, select the Pub/Sub topic as the sink destination, and add inclusion and exclusion filters using the instructions in Create an aggregated sink.

   2. Add the following inclusion filter to the log sink:

      (protoPayload.@type=type.googleapis.com/google.cloud.audit.AuditLog)

   3. Add the following exclusion filters to the log sink:

      (protoPayload.serviceName="k8s.io") AND (protoPayload.serviceName="login.googleapis.com") AND (protoPayload.methodName="storage.objects")

4. Grant the roles/pubsub.publisher role to the sink's writer identity using the instructions in Set destination permissions.

5. Grant the following roles to the service account you identified in the Collect Details of Existing Storage-Based Audit Log Integration procedure.

   • roles/monitoring.viewer role on the project you identified in the Collect Details of Existing Storage-Based Audit Log Integration procedure to the service account.
   • roles/pubsub.subscriber role on the Pub/Sub subscription to the service account.

Create Pub/Sub-based Audit Log Integration

Create the Pub/Sub-based audit log integration using the instructions in Create the GCP Audit Log Integration on the Lacework Console.

Mark Existing Storage-Based Audit Log Integration for Migration

After you create the Pub/Sub-based audit log integration, you must mark the existing Storage-based audit log integration for migration. When you mark a Storage-based integration for migration, the Lacework Platform ensures that all the audit log messages in the storage bucket for the integration are ingested, and then safely deletes the integration.

1. Run the following Lacework CLI command:

   lacework cloud-account migrate IntegrationID

   Where IntegrationID is the integration ID you identified in the Collect Details of Existing Storage-Based Audit Log Integration procedure.

Delete the Sink and Storage Bucket for the Storage-based Integration (Optional)

To reduce your GCP storage costs, you can delete the log sink and storage bucket for the Storage-based integration.

1. In the Lacework Console, go to Settings > Integrations > Cloud Accounts.
2. Ensure that the Storage-based audit log integration that you marked for migration is not displayed on the Cloud accounts page. It can take up to five hours for an integration that is marked for migration to be deleted.
3. To delete a sink, see the instructions in Manage Sinks.
4. To delete a storage bucket, see the instructions in Delete a Bucket.