In this guide, We will learn how to Configure S3 buckets and Lambda function to track any changes that occurs in S3 buckets such as uploading files , removing files etc.
During such actions in the S3 buckets , It will trigger the Lambda function and the response will be logged in Cloudwatch Logs.
Services Involved
S3 Bucket
Cloudwatch
Lambda
Steps Involved
Create S3 bucket.
Create Role and add permissions for the S3 buckets to work with Lambda functions.
Create and configure Lambda function with S3 bucket as Triggers.
Once the user upload the file to S3 bucket , It will trigger the lambda function (as it has necessary execute function) , The lambda will then send the response to the Cloudwatch logs where the users can track the changes such as uploading files.
Creating S3 Buckets
I have explained in detail in my previous article on how to create and manage S3 buckets , Refer this article .
Also, Check the more article related to S3.
Setup S3 Bucket Cross region replication
Cross domain resource sharing in S3 bucket
Automate backup of files from Windows to S3 bucket
To create S3 buckets using AWS Console , Login to S3 Console
Choose Create bucket
Provide a unique name for the bucket and choose the AWS region where the bucket should be created.
and click Create bucket
Now that we have created S3 bucket.
Creating IAM Role
To create IAM Role , Login to IAM Console
Choose Roles and then click Create role
Under AWS Service , Choose Lambda
And click Next: permissions , here we have to grant full access permissions such as S3 , Lambda and Cloudwatch.
And then click Next:Tags
Provide a name for the role and then click Create role
Now we have created IAM role and attached the necessary permissions with it.
Create & Configure Lambda
To create Lambda function , Login to Lambda Console
Click Create function , Choose Author from scratch
For Basic information , Provide a name for the function
Choose Runtime as Node.js 12.x
For permissions , Under Choose or create an execution role , Select use an existing role
and click Create function .
Now that we have created a Lambda function.
Lets go ahead and configure the lambda function such as adding S3 buckets as triggers.
Under Function code
Replace the existing code with the code attached below.
exports.handler = function(event, context, callback) {
console.log("Incoming Event: ", event);
const bucket = event.Records[0].s3.bucket.name;
const filename = decodeURIComponent(event.Records[0].s3.object.key.replace(/\+/g, ' '));
const message = `File is uploaded in - ${bucket} -> ${filename}`;
console.log(message);
callback(null, message);
};
and then click Save .
Under Designer , Click + Add trigger
For trigger configuratio n , Select a trigger as S3
Under Bucket , Select the bucket you have created.
Under Event type , Choose All object create events
Check Enable trigge r and then click Add.
Now you will able to see the S3 triggers If you choose S3 under Designer.
Testing Lambda Trigger
Now We have to upload a file to the S3 bucket , To check , whether it triggers the lambda function and it outputs the response to Cloudwatch Logs.
Go to S3 console , Select the bucket you have created.
Choose Upload , Click Add files , It will take you to the file manager , Select the file and then click Upload .
To verify the same , Go to Cloudwatch Console , In the left navigation pane , Under Logs , Select Log groups
And search for /aws/lambda/functionname
Select the Log group and then select the Log stream
You will see the response as shown below.
INFO File is uploaded in - bucketname -> filename
This shows that the Whenever we upload a file to the s3 bucket , It triggers the lambda function and output the response in Cloudwatch console.
Conclusion
We have configured S3 buckets and Lambda function to track S3 bucket changes such as uploading files and log the response in the Cloudwatch log.
Hope you found it helpful.Thanks for reading this article.
Please do check out my other publications.