Move clean
An Add-On implements the move action. With this Add-On, you can ensure that files are scanned before users can download them. By default, the Add-On does not move unscannable Files. You can change this via the MoveUnscannableFiles
configuration parameter.
Setup
The ReportCleanFiles configuration parameter of the bucketAV stack must be set to
true
.
Install Add-On (requires a running bucketAV installation)
- Set the Stack name to
bucketav-move-clean
. - Set the BucketAVStackName parameter to the stack name of bucketAV (if you followed the docs, the name is
bucketav
). - Set the TargetBucketName parameter to an existing S3 bucket where clean files are stored.
- Select I acknowledge that AWS CloudFormation might create IAM resources.
- Click on the Create stack button to save.
Install Add-On (requires a running bucketAV installation)
- Set the Stack name to
bucketav-move-clean
. - Set the BucketAVStackName parameter to the stack name of bucketAV (if you followed the docs, the name is
bucketav
). - Set the TargetBucketName parameter to an existing R2 bucket where clean files are stored.
- Select I acknowledge that AWS CloudFormation might create IAM resources.
- Click on the Create stack button to save.
Multi-account setup
If you have a multi-account setup, you might want to scan buckets created in additional AWS accounts.
IAM role-based access requires version >= 2.9.0 of this Add-On. To update to the latest version, follow the update instructions.
CloudFormation snippet
# [...]
Resources:
# [...]
MoveClean:
Type: 'AWS::CloudFormation::Stack'
Properties:
Parameters:
BucketAVStackName: 'bucketav' # if you followed the docs, the name is bucketav
TargetBucketName: 'YOUR_BUCKET_NAME_FOR_CLEAN_FILES' # TODO replace bucket name placeholder
TemplateURL: 'https://bucketav-add-ons.s3.eu-west-1.amazonaws.com/move-clean/v2.14.0/bucketav-add-on-move-clean.yaml'
# [...]
Resources:
# [...]
MoveClean:
Type: 'AWS::CloudFormation::Stack'
Properties:
Parameters:
BucketAVStackName: 'bucketav' # if you followed the docs, the name is bucketav
TargetBucketName: 'YOUR_BUCKET_NAME_FOR_CLEAN_FILES' # TODO replace bucket name placeholder
TemplateURL: 'https://bucketav-add-ons.s3.eu-west-1.amazonaws.com/move-clean/cloudflare/v2.14.0/bucketav-add-on-move-clean-cloudflare.yaml'
Terraform snippet
resource "aws_cloudformation_stack" "bucketav_add_on_move_clean" {
name = "bucketav-move-clean"
template_url = "https://bucketav-add-ons.s3.eu-west-1.amazonaws.com/move-clean/v2.14.0/bucketav-add-on-move-clean.yaml"
capabilities = ["CAPABILITY_IAM"]
parameters = {
BucketAVStackName = "bucketav" # if you followed the docs, the name is bucketav
TargetBucketName = "YOUR_BUCKET_NAME_FOR_CLEAN_FILES" # TODO replace bucket name placeholder
}
}
resource "aws_cloudformation_stack" "bucketav_add_on_move_clean" {
name = "bucketav-move-clean"
template_url = "https://bucketav-add-ons.s3.eu-west-1.amazonaws.com/move-clean/cloudflare/v2.14.0/bucketav-add-on-move-clean-cloudflare.yaml"
capabilities = ["CAPABILITY_IAM"]
parameters = {
BucketAVStackName = "bucketav" # if you followed the docs, the name is bucketav
TargetBucketName = "YOUR_BUCKET_NAME_FOR_CLEAN_FILES" # TODO replace bucket name placeholder
}
}
Update
- To update this Add-On to version v2.14.0, go to the AWS CloudFormation Management Console.
- Double-check the region at the top right.
- Search for
bucketav-move-clean
, otherwise search for the name you specified. - Select the stack and click on Update.
- Select Replace current template and set the Amazon S3 URL to
https://bucketav-add-ons.s3.eu-west-1.amazonaws.com/move-clean/v2.14.0/bucketav-add-on-move-clean.yaml
Copy - Click on Next.
- Scroll to the bottom of the page and click on Next.
- Scroll to the bottom of the page and click on Next.
- Scroll to the bottom of the page, enable I acknowledge that AWS CloudFormation might create IAM resources, and click on Update stack.
- While the update runs, the stack status is UPDATE_IN_PROGRES. Reload the table from time to time and …
- … wait until the CloudFormation stack status switches to UPDATE_COMPLETE.
- To update this Add-On to version v2.14.0, go to the AWS CloudFormation Management Console.
- Double-check the region at the top right.
- Search for
bucketav-move-clean
, otherwise search for the name you specified. - Select the stack and click on Update.
- Select Replace current template and set the Amazon S3 URL to
https://bucketav-add-ons.s3.eu-west-1.amazonaws.com/move-clean/cloudflare/v2.14.0/bucketav-add-on-move-clean-cloudflare.yaml
Copy - Click on Next.
- Scroll to the bottom of the page and click on Next.
- Scroll to the bottom of the page and click on Next.
- Scroll to the bottom of the page, enable I acknowledge that AWS CloudFormation might create IAM resources, and click on Update stack.
- While the update runs, the stack status is UPDATE_IN_PROGRES. Reload the table from time to time and …
- … wait until the CloudFormation stack status switches to UPDATE_COMPLETE.
Architecture
The following AWS services are used:
- SNS Subscription to connect to the Findings Topic.
- Lambda Function to move the clean files into the target bucket.
- CloudWatch Alarms to monitor the used AWS services.
- CloudWatch Logs to store logs.
Limitations
- S3 object ACLs are not preserved. Instead, we set the ACL bucket-owner-full-control.
Release Notes
Subscribe to our Atom feed or newsletter to stay up-to-date! We also publish a machine-readable JSON file.
v2.14.0
Changes:
- Reserved Concurrent Execution for Lambda functions
Release date:2024-11-19
Template: https://bucketav-add-ons.s3.eu-west-1.amazonaws.com/move-clean/v2.14.0/bucketav-add-on-move-clean.yaml
v2.13.1
Changes:
- Bug fixes
Release date:2024-10-02
Template: https://bucketav-add-ons.s3.eu-west-1.amazonaws.com/move-clean/v2.13.1/bucketav-add-on-move-clean.yaml
v2.13.0
Changes:
- Add CloudFormation output LambdaDeadLetterQueueName
Release date:2024-09-24
Template: https://bucketav-add-ons.s3.eu-west-1.amazonaws.com/move-clean/v2.13.0/bucketav-add-on-move-clean.yaml
v2.12.0
Changes:
- Add Lambda DLQs for async invoked functions
- Bug fixes
Release date:2024-07-09
Template: https://bucketav-add-ons.s3.eu-west-1.amazonaws.com/move-clean/v2.12.0/bucketav-add-on-move-clean.yaml
v2.11.0
Changes:
- Support for files up to 5 TB in size
Release date:2024-04-05
Template: https://bucketav-add-ons.s3.eu-west-1.amazonaws.com/move-clean/v2.11.0/bucketav-add-on-move-clean.yaml
v2.10.1
Changes:
- Multi-account with S3 bucket policies was broken
Release date:2024-04-05
Template: https://bucketav-add-ons.s3.eu-west-1.amazonaws.com/move-clean/v2.10.1/bucketav-add-on-move-clean.yaml
v2.10.0
Changes:
- Adding copy mode
Release date:2024-03-20
Template: https://bucketav-add-ons.s3.eu-west-1.amazonaws.com/move-clean/v2.10.0/bucketav-add-on-move-clean.yaml
v2.9.0
Changes:
- Multi-account with IAM roles
Release date:2024-03-15
Template: https://bucketav-add-ons.s3.eu-west-1.amazonaws.com/move-clean/v2.9.0/bucketav-add-on-move-clean.yaml
v2.8.0
Changes:
- Update Lambda runtime to Node.js 20
Release date:2024-02-14
Template: https://bucketav-add-ons.s3.eu-west-1.amazonaws.com/move-clean/v2.8.0/bucketav-add-on-move-clean.yaml
v2.7.0
Changes:
- Add Service Discovery
Release date:2023-12-07
Template: https://bucketav-add-ons.s3.eu-west-1.amazonaws.com/move-clean/v2.7.0/bucketav-add-on-move-clean.yaml
v2.6.0
Changes:
- Update Lambda runtime to Node.js 18
- Performance improvements
Release date:2023-08-31
Template: https://bucketav-add-ons.s3.eu-west-1.amazonaws.com/move-clean/v2.6.0/bucketav-add-on-move-clean.yaml
v2.5.0
Changes:
- Update Lambda runtime to Node.js 16
Release date:2022-06-08
Template: https://bucketav-add-ons.s3.eu-west-1.amazonaws.com/move-clean/v2.5.0/bucketav-add-on-move-clean.yaml
v2.4.0
Changes:
- Adding support for customer-managed KMS keys
Release date:2022-04-14
Template: https://bucketav-add-ons.s3.eu-west-1.amazonaws.com/move-clean/v2.4.0/bucketav-add-on-move-clean.yaml
v2.3.2
Changes:
- Bug fixes
Release date:2022-02-28
Template: https://bucketav-add-ons.s3.eu-west-1.amazonaws.com/move-clean/v2.3.2/bucketav-add-on-move-clean.yaml
v2.3.1
Changes:
- Bug fixes
Release date:2022-02-16
Template: https://bucketav-add-ons.s3.eu-west-1.amazonaws.com/move-clean/v2.3.1/bucketav-add-on-move-clean.yaml
v2.3.0
Changes:
- Adding experssion to ignore objects
Release date:2022-02-16
Template: https://bucketav-add-ons.s3.eu-west-1.amazonaws.com/move-clean/v2.3.0/bucketav-add-on-move-clean.yaml
v2.2.2
Changes:
- Improve Lambda config
Release date:2021-12-22
Template: https://bucketav-add-ons.s3.eu-west-1.amazonaws.com/move-clean/v2.2.2/bucketav-add-on-move-clean.yaml
v2.2.1
Changes:
- Improve S3 policy
Release date:2021-12-20
Template: https://bucketav-add-ons.s3.eu-west-1.amazonaws.com/move-clean/v2.2.1/bucketav-add-on-move-clean.yaml
v2.2.0
Changes:
- Support for MoveUnscannableFiles
Release date:2021-10-19
Template: https://bucketav-add-ons.s3.eu-west-1.amazonaws.com/move-clean/v2.2.0/bucketav-add-on-move-clean.yaml
v2.1.0
Changes:
- Ignore empty folders (do not move folders created with AWS UI)
Release date:2021-06-23
Template: https://bucketav-add-ons.s3.eu-west-1.amazonaws.com/move-clean/v2.1.0/bucketav-add-on-move-clean.yaml
v2.0.0
Changes:
- Initial release
Release date:2021-06-06
Template: https://bucketav-add-ons.s3.eu-west-1.amazonaws.com/move-clean/v2.0.0/bucketav-add-on-move-clean.yaml
Subscribe to our Atom feed or newsletter to stay up-to-date! We also publish a machine-readable JSON file.
v2.14.0
Changes:
- Reserved Concurrent Execution for Lambda functions
Release date:2024-11-19
Template: https://bucketav-add-ons.s3.eu-west-1.amazonaws.com/move-clean/cloudflare/v2.14.0/bucketav-add-on-move-clean-cloudflare.yaml
v2.13.1
Changes:
- Bug fixes
Release date:2024-10-02
Template: https://bucketav-add-ons.s3.eu-west-1.amazonaws.com/move-clean/cloudflare/v2.13.1/bucketav-add-on-move-clean-cloudflare.yaml
v2.13.0
Changes:
- Add CloudFormation output LambdaDeadLetterQueueName
Release date:2024-09-24
Template: https://bucketav-add-ons.s3.eu-west-1.amazonaws.com/move-clean/cloudflare/v2.13.0/bucketav-add-on-move-clean-cloudflare.yaml
v2.12.0
Changes:
- Add Lambda DLQs for async invoked functions
- Bug fixes
Release date:2024-07-09
Template: https://bucketav-add-ons.s3.eu-west-1.amazonaws.com/move-clean/cloudflare/v2.12.0/bucketav-add-on-move-clean-cloudflare.yaml