All Questions
842
questions
0
votes
0
answers
13
views
Different shaped responses for generate_presigned_post
My application's deployed across 2 regions, and calling generate_presigned_post from the two regions returns different response shapes. The first returns:
{
"url": "https://app-1-...
- 7,939
1
vote
0
answers
44
views
botocore >= 1.28.0 slower in multithread application
The official Boto3 docs recommends creating a new resource per thread: https://boto3.amazonaws.com/v1/documentation/api/latest/guide/resources.html#multithreading-or-multiprocessing-with-resources
...
- 1,067
1
vote
1
answer
1k
views
S3:HeadObject returns 403 on MinIO instance proxied by Cloudflare
When sending HeadObject request to MinIO instance proxied by Cloudflare, 403 is returned instead of 404 (for non-existent files). GET, PUT and DELETE operations can be performed with no issue.
For ...
- 746
-1
votes
1
answer
90
views
Grant s3 user access in bucket level policy
I have minio with bucket named "bucket" and IAM user named "user1"
I'll try to grant access to this bucket with Bucket Level Policy
client = boto3.client('s3', endpoint_url='...
- 71
0
votes
0
answers
177
views
Error when using pandas .to_csv function to upload to S3. "Anonymous access is forbidden for this operation"
I'm getting and error "Anonymous access is forbidden for this operation" when trying to use the df.to_csv function.
df.to_csv("s3://testdatateam1/test6.csv",
...
- 13
1
vote
0
answers
419
views
Multithreading in AWS boto pagination Python not working
I tried multithreading with paginator but instead of running on multiple thread, it is running on a single thread.
response: <class 'botocore.paginate.PageIterator'>
page.get('Contents') return ...
- 110
0
votes
1
answer
858
views
S3 tagging and access control policies not working for limiting the tags keyset on an object
Trying to restrict tags to only a given set of keys that can be attached to the objects. Using bucket level policies to define this condition. However, the logic is not working. Bucket policy (https://...
- 1,880
0
votes
1
answer
1k
views
trying to use boto copy to s3 unless file exists
in my code below,
fn2 is the local file and "my_bucket_object.key" is a list of files in my s3 bucket.
I am looking at my local files, taking the latest one by creation date and then looking ...
- 99
1
vote
0
answers
121
views
Return filename of specific pattern which was last modified from s3 bucket
I am writing a python script where I have to fetch just the filename (Eg. 'abc.csv') from a directory in a S3 Bucket which was last modified. I have managed to fetch the filename from the bucket but ...
- 11
0
votes
1
answer
690
views
Cannot upload the files into AWS S3 bucket with ACL pubic-read using python
I am trying to upload the file into s3 bucket but I am getting access denied error when I am trying to upload the files with ACL public-read
It is working fine with ACL private
class S3:
def ...
- 277
2
votes
0
answers
336
views
S3 boto - list objects in parallel
I want to quickly (in parallel?) list directory/prefix with 100K+ objects.
The pagination works (see below) but I'm unable to get NextKeyMarker until I download the last page - it breaks the idea of ...
- 1,539
0
votes
1
answer
816
views
Python : XML file downloaded from S3 full of string escaping characters
I have a number of XML files that I have added to S3 (localstack sever). I can view these files through Cyberduck and they are valid xml files. However, when I download the objects, the XML data is ...
- 6,933
-1
votes
1
answer
4k
views
How I can upload file to folder in bucket in S3 using Python?
I have local S3 and want to upload some files into folders in mu bucket.
For example:
my_bucket
- folder1
- folder2
I have to upload file file1.json to folder1 and file1.json to folder1
I try do ...
- 31
1
vote
0
answers
84
views
Can I read a trained linear model from s3, without reconstructing a local copy?
In order to run a dask pipeline on a coiled cluster that uses a previously trained linear model in each task, I believe I need to read the model directly from S3. Reading the model within a task did ...
- 11
0
votes
1
answer
516
views
Differentiate between website endpoint from REST API endpoint for AWS S3
I have an input provided by a user, that would be used as the endpoint url for bucket operations for an S3 bucket.
Is there a way to differentiate if the url is a REST API endpoint or a website ...
- 399