Questions tagged [amazon-s3]
Amazon S3 (simple storage service) is an online object storage service from Amazon Web Services. QUESTIONS MUST BE ABOUT PROGRAMMING. Questions about general S3 support, functionality, configuration, etc. are OFF-TOPIC.
amazon-s3
19,976
questions with no upvoted or accepted answers
15
votes
1
answer
3k
views
Glue Dynamic Frame is way slower than regular Spark
In the image below we have the same glue job run with three different configurations in terms of how we write to S3:
We used a dynamic frame to write to S3
We used a pure spark frame to write to S3
...
- 6,748
14
votes
3
answers
2k
views
Nginx Proxy Pass S3 specific html file
I there, before I had my files on my localhost and I was able to serve them just fine. Now, I moved my static files to S3, but I want to serve the index.html starter file with a custom domain. From ...
- 5,453
13
votes
1
answer
3k
views
Best strategy to upload files with unknown size to S3
I have a server-side application that runs through a large number of image URLs and uploads the images from these URLs to S3.
The files are served over HTTP. I download them using InputStream I get ...
- 1,412
13
votes
1
answer
2k
views
recreate versions - carrierwave-fog-aws
I'm trying to recreate the images that I have uploaded using the following in my model...
Post.all.each do |ym|
ym.avatar.cache_stored_file!
ym.avatar.retrieve_from_cache!(ym.avatar....
- 5,340
12
votes
0
answers
3k
views
SKIP/OFFSET/ScanRange of rows in AWS S3 Select
I wonder if anyone figured out how to skip rows in S3 Select?
SELECT S.* FROM s3object S SKIP 100 LIMIT 200
--or
SELECT * from s3object s LIMIT 5, 10
--or
SELECT * from s3object s limit 5 OFFSET 10
...
- 2,325
12
votes
2
answers
8k
views
Origin header missing from cross origin request to S3 on Chrome
I am trying to load an asset (a font) from an s3 bucket. Fonts on Firefox and IE need to have Access-Control-Allow-Origin headers returned in the response. Unfortunately it seems that Chrome is not ...
- 7,426
12
votes
2
answers
2k
views
Cognito: Federated Identity Id and User Attributes
Story:
I have a Cognito User Pool with Users.
This User Pool is an authentication provider in a Federated Identity Pool.
I have an S3 bucket where users are limited to uploading to a private path ...
- 850
11
votes
1
answer
3k
views
Can I use the Cognito User Pool Group in S3 Bucket policies?
So I just read half the internet and I think I have a pretty good grab on everything that is possible. I still have one issue though for my use-case.
My requirements: I have users that are managed ...
- 8,651
11
votes
1
answer
13k
views
CloudFront redirect all requests with path prefix
I have a static web site which makes requests to an API server. I'm hosting this static pages with S3 and I'd like to use CloudFront to redirect the api calls to the api server. Api calls can be ...
- 3,780
10
votes
0
answers
566
views
S3DistCp filename after merging files
I am having an issue with the command s3-dist-cp on Amazon EMR. What I want to achieve is being able to define the name of the file when merging all the little files in my S3 folder. Example:
s3://...
- 1,994
10
votes
0
answers
2k
views
Does boto3 attempt to retry multipart uploads by default?
I am using boto3 1.4.4 do handle uploads of large files (usually hundreds of megabytes to several gigabytes) to S3 using S3.Client.upload_file.
The boto3 docs claim that botocore handles retries for ...
- 717
10
votes
0
answers
2k
views
Data integrity check during upload to S3 with server side encryption
Data integrity check is something that the AWS Java SDK claims that it provides by default where either the client can calculate the object checksum on its own and add it as a header “Headers....
- 4,488
10
votes
0
answers
5k
views
HTML2Canvas with CORS in S3 and CloudFront
I have a problem and I'm totally desperate... :(
I'm using HTML2Canvas to create a screenshot of a div with many contents.
I have read many similar questions: Question 1 and Question2, and I have ...
- 14.5k
10
votes
1
answer
9k
views
Can I get the Content-Type from an S3 object stream without making an explicit call to headObject?
I am streaming files from S3 and through my API to the client in order to enforce access control rules. In doing so, I need to set the content-type header appropriately. Does anyone know of a way to ...
- 497
9
votes
2
answers
3k
views
How to force delete all versions of objects in S3 bucket and then eventually delete the entire bucket using aws-sdk-go?
I have an S3 bucket with versioning enabled. The bucket has few files which have versions. I have written a sample golang program which can do the following:
GetBucketVersioning - It is able to get ...
- 2,193