AWS Hands-On | S3 Security | S3 Access Logs
📌 Notice
This is a hands-on tutorial accompanying the blog post:
“Pass the AWS Certified Solutions Architect Associate Certification SAA-C03-(Episode 12: S3 Security)”
🔹 Focus: Practical implementation of Amazon S3.
🔹 For theoretical concepts and exam-style questions, please refer to the main blog post (link to parent blog).
Here’s a step-by-step guide to practicing S3 Access Logs, based on the provided content.
Objective: Understand how to enable S3 Server Access Logging to record detailed information about requests made to your S3 bucket and store these logs in a separate logging bucket.
Prerequisites:
✔ An AWS account with access to Amazon S3.
✔ An existing S3 bucket that you want to enable logging for (this will be your “Source Bucket”).
✔ A sample file (e.g., coffee.jpg) to upload to the source bucket.
Step 1: Create the Dedicated Logging Bucket
- Navigate to S3: Go to the Amazon S3 console.
- Create Bucket: Click “Create bucket”.
- Configure Bucket Details (Logging Bucket):
- Bucket name:
s3-access-logs-mino-v2(or a unique name of your choice). This bucket will store the logs. - AWS Region: Choose the same region as your “Source Bucket” (the one you will be logging). If they are in different regions, logging won’t work directly via the console.
- Leave other settings as default.
4. Create Bucket: Click “Create bucket”.
Important: This logging bucket should ideally not have server access logging enabled on itself, to prevent recursive logging and potentially infinite loops.
Step 2: Enable Server Access Logging on Your Source Bucket
- Navigate to Your Source Bucket: In the S3 console, select the bucket you want to enable logging for (e.g., one you created in a previous exercise).
- Go to “Properties” Tab: Click on the “Properties” tab for your selected source bucket.
- Edit Server Access Logging: Scroll down to “Server access logging” and click “Edit”.
- Enable Logging:
- Select “Enable”.
- Target bucket:
- Click “Browse S3” (or type the bucket name directly).
- Select the logging bucket you created in Step 1 (e.g.,
s3-access-logs-mino-v2). - Click “Choose path”.
- Target prefix (optional): Leave this blank for now. (You could add a prefix like
logs/to store logs in a subfolder). - Log object key format: Keep the default option.
5. Save Changes: Click “Save changes”.
Observation: S3 will automatically update the bucket policy of your logging bucket to allow the S3 logging service to write logs to it.
Step 3: Verify the Logging Bucket’s Permissions (Optional but Recommended)
- Navigate to the Logging Bucket: Go to your
s3-access-logs-mino-v2bucket. - Go to “Permissions” Tab: Click on the “Permissions” tab.
- Review Bucket Policy: Scroll down to “Bucket policy” and click “Edit” (you don’t need to change anything, just view).
- Observation: You will see a policy statement similar to this (ARNs will vary):
{
"Version": "2012-10-17",
"Id": "S3-Console-Auto-Gen-Policy-1748785057436",
"Statement": [
{
"Sid": "S3PolicyStmt-DO-NOT-MODIFY-1748785055804",
"Effect": "Allow",
"Principal": {
"Service": "logging.s3.amazonaws.com"
},
"Action": "s3:PutObject",
"Resource": "arn:aws:s3:::s3-access-logs-mino-v2/*",
"Condition": {
"StringEquals": {
"aws:SourceAccount": "050752610240"
}
}
}
]
}
Explanation: This policy explicitly grants the AWS S3 logging service (
logging.s3.amazonaws.com) permission to put objects (logs) into your logging bucket.
Step 4: Generate Activity in the Source Bucket
- Navigate to the Source Bucket: Go back to the bucket where you enabled logging.
- Perform S3 Operations:
- Browse Objects: Click on objects, open them, download them (if possible).
- Upload a File: Click “Upload”, “Add files”, and upload
coffee.jpg(or any file). - Delete a File: Delete an existing file.
- Navigate: Click around the “Objects” tab.
Explanation: Every action you perform (GET, PUT, DELETE, LIST operations, etc.) against this source bucket will generate an entry in the access logs.
Step 5: Monitor the Logging Bucket for Log Files
- Navigate to the Logging Bucket: Go to your
s3-access-logs-mino-v2bucket. - Refresh Objects: Click the refresh icon in the “Objects” tab repeatedly over the next few minutes to a few hours.
Observation: It takes time for S3 to process and deliver the logs. You might not see them immediately. Eventually, you will start seeing new objects appearing in this bucket, typically named with a format like
[bucket-name].[datetime]-[hash].
3. Inspect a Log File:
- Once log files appear, click on one of them (you may need to download it).
- Observation: The content will be a plain text file with space-separated values, representing different fields of the S3 access log format. It contains information like:
- Bucket owner and name
- Timestamp of the request
- Remote IP address
- Requester (e.g., IAM user ARN)
- Operation (e.g.,
REST.GET.OBJECT,REST.PUT.OBJECT) - Key (the object being accessed)
- HTTP status code (e.g., 200, 404)
- Error code (if any)
- Bytes sent/received
- Referer, User-Agent, and more.
- Challenge: The raw log format can be difficult to read and analyze directly. Typically, these logs are further processed by other services (e.g., Athena, Glue, or third-party log analysis tools) for easier querying and insights.
Conclusion:
You have successfully demonstrated S3 Server Access Logging. You’ve learned how to:
- Create a dedicated bucket to store access logs.
- Enable server access logging on a source bucket, specifying the destination logging bucket.
- Understand that S3 automatically updates the logging bucket’s policy to grant necessary write permissions.
- Generate activity to produce logs.
- Observe log files appearing in the logging bucket (with a delay).
- Briefly examine the content of a raw S3 access log file.
This logging capability is crucial for security auditing, troubleshooting, and understanding usage patterns for your S3 resources.
