Questions tagged [amazon-s3]
Amazon S3 (simple storage service) is an online object storage service from Amazon Web Services. QUESTIONS MUST BE ABOUT PROGRAMMING. Questions about general S3 support, functionality, configuration, etc. are OFF-TOPIC.
amazon-s3
51,030
questions
0
votes
1
answer
21
views
Reading Parquets From S3 With Apache Spark Slows Down At Later Stages
I have millions of parquets files on s3 with directory structure as code/day=xx/hour=/*.parquets.
At max under hour folder we have 2000 parquest file with average size of 100kb.
I am not able to ...
- 1
1
vote
0
answers
26
views
Not able to Access S3 image via Cloudfront
I have created S3 bucket with public access with below mentioned Bucket policy and CORS. My bucket image is accessible in my web app.
But When I create cloudfront distribution for this bucket with ...
- 11
-3
votes
0
answers
13
views
hls.js not resolving segment file paths correctly for files hosted on S3 and accessed via cloudfront [closed]
I've a HLS video upload on an S3 bucket with following folder structure
https://bucket-name.us-west-1.amazonaws.com/folder/path/to/video/output.m3u8
The contents of output.m3u8 file as following:
#...
- 1
0
votes
0
answers
11
views
Amazon Managed Service for Apache Flink keeping S3 as data source
Is it possible to read files from S3 whenever a new file is added from AWS Managed Apache Flink and I need to process the list of new files data one by one, the final result needs to be added to ...
- 495
0
votes
0
answers
16
views
How do I ingest data from singstat into an S3 bucket using AWS Lambda?
this is the code that i have been using:
import json
import urllib.request
import boto3
import os
from datetime import datetime
def lambda_handler(event, context):
# Define the SingStat API
...
0
votes
0
answers
48
views
Spark EOF Error (Parquet Read from S3)- Spark to Pandas conversion
I am reading close to 1 million rows stored in S3 as parquet files into a dataframe (900 MB size data in a bucket). Filtering the dataframe based on values and then later converting to a Pandas ...
- 132
0
votes
0
answers
8
views
How to Efficiently Upload Files in Parallel in Next.js/Node.js Using Async and S3?
I'm working on a Next.js project where I need to upload multiple files to AWS S3 in parallel. I'm using the @aws-sdk/client-s3 for uploading the files. Despite trying different approaches, my uploads ...
0
votes
2
answers
31
views
Too many "Authorized committer" errors after upgrading to Pyspark==3.5.1
The problem
I have recently upgraded my apps to run on Spark3.5.1+YARN3.3.6, and observing frequent failures saying "Authorized committer". The apps run PySpark and I observe the error ...
- 2,202
0
votes
0
answers
16
views
Amazon S3 bucket requirements for create spot-datafeed-subscription
I want to create a spot-datafeed-subscription with the documentation on Track your Spot Instance costs by subscribing to a data feed - Amazon Elastic Compute Cloud.
I am kind of lost in how I have to ...
0
votes
0
answers
11
views
To upload the Google Drive images to AWS S3 using Laravel import functionality
I'm trying to upload the Google Drive images to AWS S3 using Laravel import functionality. I have more than 2000 products in it. Each product has 2 mandatory images, in which I'm facing one issue. The ...
0
votes
1
answer
42
views
S3 Intelligent - Tiering with snowflake managed iceberg tables
Trying to save on huge rarely used data storage costs, with snowflake managed iceberg tables.
Would you apply the economical s3 intelligent-Tiering storage,
to the iceberg’s external volume on s3,
...
- 13
-3
votes
0
answers
23
views
What is the server-side modified date of a file in the AWS S3 cloud? [closed]
Files in the AWS S3 cloud have two different dates/time listed in their properties, the "Server-side modified" and the "Client-side modified". I would like to know what is the ...
- 1
0
votes
0
answers
6
views
I have react-image-crop problems with IOS
It works perfectly in Android and desktop browsers, but I have issues with Iphone and Safari Mac
I am uploading an image cropping with react-image-crop and uploading to S3 AWS
Do you guys know any ...
1
vote
0
answers
32
views
PHP: Does move_uploaded_file() actually work with the S3 Stream Wrapper?
The Amazon S3 stream wrapper "enables you to store and retrieve data from Amazon S3 using built-in PHP functions, such as file_get_contents, fopen, copy, rename, unlink, mkdir, and rmdir. ... ...
- 3,245
1
vote
0
answers
21
views
S3 server simulation in FastAPI, Python
I want to implement S3 server using FastAPI and Python. This is my code for signature validation:
async def verify_signature(request: Request):
try:
authorization_header = request.headers.get(&...
- 21