Day 67: AWS S3 Bucket Creation and Management

Day 67: AWS S3 Bucket Creation and Management

What is AWS S3 Bucket?

  • Amazon Simple Storage Service (Amazon S3) is an object storage service offering industry-leading scalability, data availability, security, and performance. Customers of all sizes and industries can store and protect any amount of data for virtually any use case, such as data lakes, cloud-native applications, and mobile apps.

  • With cost-effective storage classes and easy-to-use management features, you can optimize costs, organize data, and configure fine-tuned access controls to meet specific business, organizational, and compliance requirements.

product-page-diagram_Amazon-S3_HIW@2x ee85671fe5c9ccc2ee5c5352a769d7b03d7c0f16

What is Object Storage?

  • Object storage is a data storage architecture that manages data as objects, as opposed to other storage architectures like file systems which manage data as a file hierarchy and block storage which manages data as blocks within sectors and tracks.

  • Each object typically includes the data itself, a variable amount of metadata, and a globally unique identifier. Object storage can be implemented at multiple levels, including the device level (object storage device), the system level, and the interface level.

  • In each case, object storage seeks to enable capabilities not addressed by other storage architectures, like interfaces that can be directly programmable by the application, a namespace that can span multiple instances of physical hardware, and data management functions like data replication and data distribution at object-level granularity.

Task-01

Create an S3 bucket using Terraform.

  • Step-01: First create a "main.tf" with proper Provider and Region details and then create an S3 bucket resource with the name devopsbucket123xyz
provider "aws" {
  access_key = ""
  secret_key = ""
  region     = "ap-south-1"
}
resource "aws_s3_bucket" "devops_name_bucket_1" {
  bucket = "devopsbucket123xyz"
}

Screenshot from 2023-10-09 23-01-25

  • In this Terraform Code:

  • The provider block specifies the AWS provider configuration. It includes your AWS access key, secret key, and the AWS region you want to use. Make sure to keep your access key and secret key secure and do not share them publicly.

  • The resource block defines an AWS S3 bucket resource named "devops_name_bucket_1" and sets the desired bucket name to "devopsbucket123xyz".

  • Step-02: Initialize Terraform

terraform init

Screenshot from 2023-10-09 23-01-32

  • Step-03: Now apply the Terraform Code
terraform apply

Screenshot from 2023-10-09 23-02-14

  • Step-04: Now go to AWS Console and check the S3 bucket.

Screenshot from 2023-10-09 23-06-14


Configure the bucket to allow public read access.

  • Step-01: Now create a new file named "public_access.tf" and add the following code to it.
resource "aws_s3_bucket_public_access_block" "example" {
  bucket = aws_s3_bucket.devops_name_bucket_1.id

  block_public_acls       = false
  block_public_policy     = false
  ignore_public_acls      = false
  restrict_public_buckets = false
}

resource "aws_s3_bucket_acl" "bucket_acl" {
  bucket = aws_s3_bucket.devops_name_bucket_1.id
  acl    = "public-read"
}

Screenshot from 2023-10-09 23-12-14

  • In this Terraform Code:

  • The aws_s3_bucket_public_access_block resource creates a configuration for controlling public access to the "devops_name_bucket_1" S3 bucket. You have specified that public ACLs, public policies, and public bucket access should not be blocked.

  • The aws_s3_bucket_acl resource defines an Access Control List (ACL) for the "devops_name_bucket_1" S3 bucket, setting it to "public-read." This configuration allows public read access to objects in the bucket.

  • Step-02: Now go to bucket > permissions > edit object permissions and make ACLs enabled.

Screenshot from 2023-10-09 23-34-09

  • Step-03: Now apply the Terraform Code
terraform apply

Screenshot from 2023-10-09 23-35-54

  • Step-04: Now go to AWS Console and check the S3 bucket, as it should be public now.

Screenshot from 2023-10-09 23-37-48


Create an S3 bucket policy that allows read-only access to a specific IAM user or role.

  • Step-01: Now create a new file named "IAM_access.tf" and add the following code to it.
resource "aws_s3_bucket_policy" "bucket_policy" {
  bucket = aws_s3_bucket.devops_name_bucket_1.id
  policy = data.aws_iam_policy_document.allow_read_only_access.json
}

data "aws_iam_policy_document" "allow_read_only_access" {
  statement {
    principals {
      type        = "AWS"
      identifiers = ["024977362083"]
    }

    actions = [
      "s3:GetObject",
      "s3:ListBucket",
    ]

    resources = [
      aws_s3_bucket.devops_name_bucket_1.arn,
      "${aws_s3_bucket.devops_name_bucket_1.arn}/*",
    ]
  }
}

Screenshot from 2023-10-10 00-14-02

  • In this code we also need to add the AWS Account ID in the identifiers section.

Screenshot from 2023-10-10 00-02-56

  • In this Terraform Code:

  • The aws_s3_bucket_policy resource attaches a policy to the "devops_name_bucket_1" S3 bucket. It uses the policy defined in the data block.

  • The data "aws_iam_policy_document" block defines an IAM policy document named "allow_read_only_access." This policy allows read-only access to the specified AWS account ID for the S3 bucket and its objects.

  • Step-02: Now apply the Terraform Code

terraform apply

Screenshot from 2023-10-10 00-14-28

  • Step-03: Now go to AWS Console and check the S3 bucket policy

Screenshot from 2023-10-10 00-16-03


Enable versioning on the S3 bucket.

  • Step-01: Now create a new file named "versioning.tf" and add the following code to it.
resource "aws_s3_bucket" "my_bucket_versioning" {
  bucket = aws_s3_bucket.devops_name_bucket_1.id

  versioning {
    enabled = true
  }
}
  • In this Terraform Code:

  • We create a resource of type aws_s3_bucket named "my_bucket_versioning."

  • We specify the name of the existing S3 bucket "devops_name_bucket_1" in the bucket attribute.

  • We enable versioning for the existing bucket using the versioning block.

  • Step 02: Check the AWS S3 bucket versioning settings.

image


Happy Terraforming:)

Did you find this article valuable?

Support DevOps by becoming a sponsor. Any amount is appreciated!