Skip to main content

S3 Storage

Object Storage for Files and Static Assets

Amazon S3 stores documents, exports, checkpoints, and static assets.

Buckets

BucketPurposeAccess
amply-storage-prodApplication files (documents, exports)Private
amply-public-dataPublic checkpoints, ledger exportsPublic read
amply-frontend-prodReact frontend static filesPublic (via CloudFront)
amply-widgets-prodWidget bundlePublic (via CloudFront)
amply-backups-prodDatabase backups, archivesPrivate

Bucket Configurations

amply-storage-prod (Private)

Application file storage:

BucketName: amply-storage-prod
Region: eu-central-1

# Versioning for recovery
VersioningConfiguration:
Status: Enabled

# Encryption
BucketEncryption:
ServerSideEncryptionConfiguration:
- ServerSideEncryptionByDefault:
SSEAlgorithm: aws:kms
KMSMasterKeyID: alias/amply-s3-key

# Block public access
PublicAccessBlockConfiguration:
BlockPublicAcls: true
BlockPublicPolicy: true
IgnorePublicAcls: true
RestrictPublicBuckets: true

# Lifecycle rules
LifecycleConfiguration:
Rules:
# Move old exports to cheaper storage
- Id: archive-exports
Status: Enabled
Filter:
Prefix: exports/
Transitions:
- Days: 90
StorageClass: STANDARD_IA
- Days: 365
StorageClass: GLACIER

# Delete old temporary files
- Id: cleanup-temp
Status: Enabled
Filter:
Prefix: temp/
Expiration:
Days: 7

Folder structure:

amply-storage-prod/
├── documents/
│ └── org_{id}/
│ ├── verification/ # Verification documents
│ └── reports/ # Generated reports
├── exports/
│ └── org_{id}/
│ └── ledger-{date}.json
├── temp/
│ └── upload-{id}/ # Temporary uploads
└── avatars/
└── user_{id}.jpg

amply-public-data (Public)

Public verification data:

BucketName: amply-public-data
Region: eu-central-1

# Versioning for immutability
VersioningConfiguration:
Status: Enabled

# Public read access
PublicAccessBlockConfiguration:
BlockPublicAcls: false
BlockPublicPolicy: false
IgnorePublicAcls: false
RestrictPublicBuckets: false

# Bucket policy for public read
BucketPolicy:
Statement:
- Effect: Allow
Principal: "*"
Action: s3:GetObject
Resource: arn:aws:s3:::amply-public-data/*

Folder structure:

amply-public-data/
├── checkpoints/
│ ├── chk_2025_01_01.json
│ ├── chk_2025_01_02.json
│ └── ...
├── exports/
│ └── weekly/
│ └── ledger-2025-w01.json.gz
└── keys/
└── amply-checkpoint-key-2025.pub

amply-frontend-prod (Static Hosting)

React frontend:

BucketName: amply-frontend-prod
Region: eu-central-1

# Static website hosting
WebsiteConfiguration:
IndexDocument: index.html
ErrorDocument: index.html # SPA routing

# CORS for API calls
CorsConfiguration:
CorsRules:
- AllowedHeaders: ["*"]
AllowedMethods: [GET, HEAD]
AllowedOrigins: ["https://amply-impact.org"]
MaxAgeSeconds: 3600

amply-widgets-prod (Widget CDN)

Widget JavaScript bundle:

BucketName: amply-widgets-prod
Region: eu-central-1

# CORS for embedding
CorsConfiguration:
CorsRules:
- AllowedHeaders: ["*"]
AllowedMethods: [GET, HEAD]
AllowedOrigins: ["*"] # Widgets embedded anywhere
MaxAgeSeconds: 86400

Application Integration

Upload Files

import boto3
from botocore.config import Config

s3 = boto3.client(
's3',
region_name='eu-central-1',
config=Config(signature_version='s3v4')
)

async def upload_document(
org_id: str,
file_content: bytes,
filename: str,
content_type: str
) -> str:
"""Upload document to S3."""
key = f"documents/org_{org_id}/verification/{filename}"

s3.put_object(
Bucket='amply-storage-prod',
Key=key,
Body=file_content,
ContentType=content_type,
ServerSideEncryption='aws:kms'
)

return key

Generate Presigned URLs

async def get_download_url(key: str, expires_in: int = 3600) -> str:
"""Generate presigned URL for download."""
url = s3.generate_presigned_url(
'get_object',
Params={
'Bucket': 'amply-storage-prod',
'Key': key
},
ExpiresIn=expires_in
)
return url

async def get_upload_url(key: str, content_type: str) -> str:
"""Generate presigned URL for upload."""
url = s3.generate_presigned_url(
'put_object',
Params={
'Bucket': 'amply-storage-prod',
'Key': key,
'ContentType': content_type
},
ExpiresIn=3600
)
return url

Publish Checkpoint

async def publish_checkpoint(checkpoint_data: dict) -> str:
"""Publish checkpoint to public bucket."""
key = f"checkpoints/{checkpoint_data['checkpoint_id']}.json"

s3.put_object(
Bucket='amply-public-data',
Key=key,
Body=json.dumps(checkpoint_data, indent=2),
ContentType='application/json',
CacheControl='public, max-age=31536000' # Immutable
)

return f"https://amply-public-data.s3.eu-central-1.amazonaws.com/{key}"

Frontend Deployment

# Build React app
npm run build

# Sync to S3
aws s3 sync build/ s3://amply-frontend-prod/ \
--delete \
--cache-control "public, max-age=31536000" \
--exclude "index.html"

# Upload index.html with no-cache
aws s3 cp build/index.html s3://amply-frontend-prod/index.html \
--cache-control "no-cache, no-store, must-revalidate"

# Invalidate CloudFront
aws cloudfront create-invalidation \
--distribution-id XXXXXX \
--paths "/*"

Security

Bucket Policies

Private buckets:

{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Deny",
"Principal": "*",
"Action": "s3:*",
"Resource": [
"arn:aws:s3:::amply-storage-prod",
"arn:aws:s3:::amply-storage-prod/*"
],
"Condition": {
"Bool": { "aws:SecureTransport": "false" }
}
}
]
}

IAM Policies

Application access:

{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:GetObject",
"s3:PutObject",
"s3:DeleteObject"
],
"Resource": "arn:aws:s3:::amply-storage-prod/*"
},
{
"Effect": "Allow",
"Action": [
"s3:PutObject"
],
"Resource": "arn:aws:s3:::amply-public-data/checkpoints/*"
}
]
}

Monitoring

CloudWatch Metrics

  • NumberOfObjects
  • BucketSizeBytes
  • AllRequests
  • 4xxErrors
  • 5xxErrors

Alarms

- AlarmName: s3-amply-storage-errors
MetricName: 5xxErrors
Namespace: AWS/S3
Dimensions:
- Name: BucketName
Value: amply-storage-prod
Threshold: 10
Period: 300

Cost Optimisation

  1. Lifecycle policies: Move old data to cheaper tiers
  2. Intelligent-Tiering: For unpredictable access patterns
  3. Delete old versions: After retention period
  4. Compression: Gzip for exports and large files

Related: