Amazon S3¶
This backend implements the Django File Storage API for Amazon Web Services’s (AWS) Simple Storage Service (S3).
Installation¶
The backend is based on the boto3 library which must be installed; the minimum required version is 1.4.4 although
we always recommend the most recent. Either add it to your requirements or use the optional s3
extra e.g:
pip install django-storages[s3]
Configuration & Settings¶
Django 4.2 changed the way file storage objects are configured. In particular, it made it easier to independently configure
storage backends and add additional ones. To configure multiple storage objects pre Django 4.2 required subclassing the backend
because the settings were global, now you pass them under the key OPTIONS
. For example, to save media files to S3 on Django
>= 4.2 you’d define:
STORAGES = {
"default": {
"BACKEND": "storages.backends.s3.S3Storage",
"OPTIONS": {
...your_options_here
},
},
}
On Django < 4.2 you’d instead define:
DEFAULT_FILE_STORAGE = "storages.backends.s3.S3Storage"
To put static files on S3 via collectstatic
on Django >= 4.2 you’d include the staticfiles
key (at the same level as
default
above inside of the STORAGES
dictionary while on Django < 4.2 you’d instead define:
STATICFILES_STORAGE = "storages.backends.s3.S3Storage"
The settings documented in the following sections include both the key for OPTIONS
(and subclassing) as
well as the global value. Given the significant improvements provided by the new API, migration is strongly encouraged.
Authentication Settings¶
There are several different methods for specifying the AWS credentials used to create the S3 client. In the order that S3Storage
searches for them:
session_profile
orAWS_S3_SESSION_PROFILE
access_key
orAWS_S3_ACCESS_KEY_ID
orAWS_S3_SECRET_ACCESS_KEY
secret_key
orAWS_ACCESS_KEY_ID
orAWS_SECRET_ACCESS_KEY
- The environment variables AWS_S3_ACCESS_KEY_ID and AWS_S3_SECRET_ACCESS_KEY
- The environment variables AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY
- Use Boto3’s default session
Settings¶
bucket_name
or AWS_STORAGE_BUCKET_NAME
Required
The name of the S3 bucket that will host the files.
object_parameters
or AWS_S3_OBJECT_PARAMETERS
Default:
{}
Use this to set parameters on all objects. To set these on a per-object basis, subclass the backend and override
S3Storage.get_object_parameters
.To view a full list of possible parameters (there are many) see the Boto3 docs for uploading files; an incomplete list includes:
CacheControl
,SSEKMSKeyId
,StorageClass
,Tagging
andMetadata
.
default_acl
or AWS_DEFAULT_ACL
Default:
None
- the file will beprivate
per Amazon’s defaultUse this to set an ACL on your file such as
public-read
. If not set the file will beprivate
per Amazon’s default. If theACL
parameter is set inobject_parameters
, then this setting is ignored.Options such as
public-read
andprivate
come from the list of canned ACLs.
querystring_auth
or AWS_QUERYSTRING_AUTH
Default:
True
Setting
AWS_QUERYSTRING_AUTH
toFalse
to remove query parameter authentication from generated URLs. This can be useful if your S3 buckets are public.
max_memory_size
or AWS_S3_MAX_MEMORY_SIZE
Default:
0
i.e do not roll overThe maximum amount of memory (in bytes) a file can take up before being rolled over into a temporary file on disk.
querystring_expire
or AWS_QUERYSTRING_EXPIRE
Default:
3600
The number of seconds that a generated URL is valid for.
url_protocol
or AWS_S3_URL_PROTOCOL
Default:
https:
The protocol to use when constructing a custom domain,
custom_domain
must beTrue
for this to have any effect.
file_overwrite
or AWS_S3_FILE_OVERWRITE
Default:
True
By default files with the same name will overwrite each other. Set this to
False
to have extra characters appended.
location
or AWS_LOCATION
Default:
''
A path prefix that will be prepended to all uploads.
gzip
or AWS_IS_GZIPPED
Default:
False
Whether or not to enable gzipping of content types specified by
gzip_content_types
.
gzip_content_types
or GZIP_CONTENT_TYPES
Default:
(text/css,text/javascript,application/javascript,application/x-javascript,image/svg+xml)
The list of content types to be gzipped when
gzip
isTrue
.
region_name
or AWS_S3_REGION_NAME
Default:
None
Name of the AWS S3 region to use (eg. eu-west-1)
use_ssl
or AWS_S3_USE_SSL
Default:
True
Whether or not to use SSL when connecting to S3, this is passed to the boto3 session resource constructor.
verify
or AWS_S3_VERIFY
Default:
None
Whether or not to verify the connection to S3. Can be set to False to not verify certificates or a path to a CA cert bundle.
endpoint_url
or AWS_S3_ENDPOINT_URL
Default:
None
Custom S3 URL to use when connecting to S3, including scheme. Overrides
region_name
anduse_ssl
. To avoidAuthorizationQueryParametersError
errors,region_name
should also be set.
addressing_style
or AWS_S3_ADDRESSING_STYLE
Default:
None
Possible values
virtual
andpath
.
proxies
or AWS_S3_PROXIES
Default:
None
Dictionary of proxy servers to use by protocol or endpoint, e.g.:
{'http': 'foo.bar:3128', 'http://hostname': 'foo.bar:4012'}.
transfer_config
or AWS_S3_TRANSFER_CONFIG
Default:
None
Set this to customize the transfer config options such as disabling threads for
gevent
compatibility; See the Boto3 docs for TransferConfig for more info.
custom_domain
or AWS_S3_CUSTOM_DOMAIN
Default:
None
Set this to specify a custom domain for constructed URLs.
Note
You’ll have to configure CloudFront to use the bucket as an origin for this to work.
Warning
Django’s STATIC_URL must end in a slash and this must not. It is best to set this variable independently of STATIC_URL.
signature_version
or AWS_S3_SIGNATURE_VERSION
Default:
None
As of
boto3
version 1.13.21 the default signature version used for generating presigned urls is stillv2
. To be able to access your s3 objects in all regions through presigned urls, explicitly set this tos3v4
.Set this to use an alternate version such as
s3
. Note that only certain regions support the legacys3
(also known asv2
) version. You can check to see if your region is one of them in the S3 region list.Warning
The signature versions are not backwards compatible so be careful about url endpoints if making this change for legacy projects.
CloudFront Signed URLs¶
If you want to generate signed Cloudfront URLs, you can do so by following these steps:
- Generate a CloudFront Key Pair as specified in the AWS docs.
- Add
cloudfront_key
andcloudfront_key_id
as above with the generated settings - Install one of cryptography or rsa
- Set both
cloudfront_key_id/AWS_CLOUDFRONT_KEY_ID
andcloudfront_key/AWS_CLOUDFRONT_KEY
django-storages will now generate signed cloudfront urls.
IAM Policy¶
The IAM policy definition needed for the most common use case is:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "VisualEditor0",
"Effect": "Allow",
"Action": [
"s3:PutObject",
"s3:GetObjectAcl",
"s3:GetObject",
"s3:ListBucket",
"s3:DeleteObject",
"s3:PutObjectAcl"
],
"Principal": {
"AWS": "arn:aws:iam::example-AWS-account-ID:user/example-user-name"
},
"Resource": [
"arn:aws:s3:::example-bucket-name/*",
"arn:aws:s3:::example-bucket-name"
]
}
]
}
For more information about Principal, please refer to AWS JSON Policy Elements