Anypoint Connector for Amazon S3 (Amazon S3 Connector) provides connectivity to the Amazon S3 API, enabling you to interface with Amazon S3 to store objects, download and use data with other AWS services, and build applications that require internet storage.

Instant access to the Amazon S3 API enables seamless integrations between Amazon S3 and other databases, CMS applications such as Drupal, and CRM applications such as Salesforce.

Use Amazon S3 to store and retrieve any amount of data at any time, from anywhere on the web. You can accomplish these tasks by using the simple and intuitive web interface of the AWS Management Console.

AWS SDK for Java provides a Java API for AWS infrastructure services. The Amazon S3 connector is built using the SDK for Java. For more info Mule Training

Prerequisites

Before creating an app, you must have access to the Amazon S3 target resource, Amazon Web Services, and Anypoint Platform. You must also understand how to create a Mule app using Design Center or Anypoint Studio, and have AWS Identity and Access Management (IAM) credentials.

Amazon S3 Design Center Configuration - Mule 4

Design Center enables you to create apps visually. To use Design Center, work with your Anypoint Platform administrator to ensure that you have a Design environment.

To create an app in Design Center:

Configure the input source (trigger) for your app.
Add the connector as a component to process the input for the app.

Configure the Input Source Trigger

To configure a trigger:

In Design Center, click Create.
Click Create new application.
Specify a value for Project name, and click Create.
Exit from Let’s get started by clicking Go straight to canvas.
Click on the Trigger card.
Configure the trigger.
You can use the following items as a trigger:
- The connector’s On Deleted Object operation to initiate access to your app when an Amazon S3 object is deleted.
- The connector’s On New Object operation to initiate access to your app when an Amazon S3 object is created.
- HTTP Connector to initiate access to your app when the HTTP Listener accepts a request from a browser or application, such as Postman or CURL.
- scheduler to initiate access your app at a specific time.

If you use an Amazon S3 connector’s operation as a trigger, enter the name of the Amazon S3 bucket associated with the operation in the Bucket field on the General tab. Learn practical skills from Mulesoft Certification

Configure the Target Component

Click “+” next to the trigger card.
In Select a component, search for and select the connector name.
Select an operation for the connector.
Enter the required values in the General tab.
If you are:
- Using the default Amazon S3 storage, leave the default entries for the Proxy and the Advanced tabs.
- Connecting to a different storage than the default AWS S3, specify its URL in the the Advanced tab’s S3 Compatible Storage URL field.
If needed, enter values for other tabs.
Specify access information to the connector resource, as described below.
Click Test to test the connection.
Validating a connection with Test Connection requires that you have permission in AWS IAM to the action s3:ListAllMyBuckets. If you don’t have this permission, the test fails. However, you can still use the connector and the operations to which you have access.
Access to operations on Amazon S3 is further controlled through policies. It is not always possible to validate your credentials before the exact operation for which you have access completes. This can vary based on the bucket name and other parameters. For example, the test connection can fail if your credentials have a restricted policy.

Amazon S3 Additional Configuration Information - Mule 4

Create Object Operation

In the Create Object operation, set the Content Length to a value greater than zero. If the Content Length is set to zero (0), the Create Object operation creates a zero-byte object.

Use an AWS KMS Master Key

To encrypt objects that you want to store in S3 buckets using customer-managed master keys, specify a Customer Master Key ID in the KMS Master Key field in the Create Object configuration.

Prerequisites for Source Operations

Amazon S3 source operations use Amazon SQS for notification of when a new object is created or deleted in an S3 bucket.

There are two scenarios for the On New Object and On Deleted Object Amazon S3 source operations:

The queue is already created and specified
The queue is created by the source. Learn advanced skills from Mulesoft Online Training

Queue Is Already Created

To correctly use a source for a queue that already exists, the user must have these actions enabled for the Amazon SQS queue:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": [
                "sqs:DeleteMessage",
                "sqs:GetQueueUrl",
                "sqs:ReceiveMessage",
                "sqs:GetQueueAttributes"
            ],
            "Resource": "arn:aws:sqs:REGION:ACCOUNT_ID:QUEUE_NAME"
        }
    ]
}

Additionally, to enable the queue to receive messages when a new object is created or deleted in an S3 bucket, create a policy for the queue in this format:

{
  "Version": "2012-10-17",
  "Id": "arn:aws:sqs:REGION:ACCOUNT_ID:QUEUE_NAME/SQSDefaultPolicy",
  "Statement": [
    {
      "Sid": "Sid1593770403887",
      "Effect": "Allow",
      "Principal": "*",
      "Action": "SQS:SendMessage",
      "Resource": "arn:aws:sqs:REGION:ACCOUNT_ID:QUEUE_NAME",
      "Condition": {
        "ArnLike": {
          "aws:SourceArn": "arn:aws:s3:::BUCKET_NAME"
        }
      }
    }
  ]
}

Queue Is Created by Source

If the queue is created by the source operation, the queue name has the prefix MULE-S3-TRIGGER.

When the queue is created by the source, the user must have permission to both create a queue and set the queue attributes:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": [
                "sqs:DeleteMessage",
                "sqs:ReceiveMessage",
                "sqs:GetQueueAttributes",
                "sqs:CreateQueue",
                "sqs:SetQueueAttributes"
                "sqs:GetQueueUrl"
            ],
            "Resource": "arn:aws:sqs:REGION:ACCOUNT_ID:MULE-S3-TRIGGER-*"
        }
    ]
}

When the queue is created by the source operation, an SQS policy is added to the queue when the S3 object is created or deleted in the bucket in the following format: Learn new version techniques from Mule 4 Training

{
  "Version": "2012-10-17",
  "Id": "example-ID",
  "Statement": [
    {
      "Sid": "example-statement-ID",
      "Effect": "Allow",
      "Principal": {
        "AWS": "*"
    },
      "Action": "SQS:SendMessage",
      "Resource":QUEUE_ARN,
      "Condition": {
        "ArnLike": {
          "aws:SourceArn": "BUCKET_ARN"
        }
      }
    }
  ]
}

S3 Policy Bucket

For the S3 policy, use this format for both cases:

{
    "Version": "2012-10-17",
    "Id": "Policy1593761427184",
    "Statement": [
        {
            "Sid": "Stmt1593760119344",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::ACCOUNT_ID:user/test"
            },
            "Action": "SPECIFIC_ACTION",
            "Resource": "arn:aws:s3:::BUCKET_NAME/*"
        },
        {
            "Sid": "Stmt1593760259223",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::ACCOUNT_ID:user/test"
            },
            "Action": ["s3:PutBucketNotification", "s3:GetBucketNotification"],
            "Resource": "arn:aws:s3:::BUCKET_NAME"
        }
    ]
}

In this example, the SPECIFIC_ACTION is s3:PutObject or s3:DeleteObject, depending on the specified source operation.

To get in-depth knowledge, enroll for a live free demo on Mulesoft Training

Search This Blog

tecprogramming

Amazon S3 Connector - Mule 4