How to inspect/read an .h5 file stored remotely in a AWS bucket?

Hello, everyone

I have decided to read/copy files straight from their online repository to avoid download the files at first. Given this is my first attempt at this, this's been my first interaction with aws.s3 .

First, just to make sure I could run something simple, I checked if the bucket existed. I did so with bucket_exists defining both the bucket and the region. The bucket does exist.

However, the file I want to inspect is an .h5 file. To work with it, I got the rhdf5 library from BiocManager. Then, to inspect the one file, I did the following:

     FUN = rhdf5::H5Fopen, 
     bucket = "s3://arpa-e-perform/ERCOT/",
     region = "us-west-2",
     object = "s3://arpa-e-perform/ERCOT/2018/Solar/Actuals/BA_level/BA_solar_actuals_2018.h5")

Unfortunately, it didn't work. The message and the error message I got follow:

List of 6
$ Code     : chr "PermanentRedirect"
$ Message  : chr "The bucket you are attempting to access must be addressed using the specified endpoint. Please send all future "| truncated
$ Endpoint : chr ""
$ Bucket   : chr "arpa-e-perform"
$ RequestId: chr "BGEZ97HJH10KAPRE"
$ HostId   : chr "pxKXcYNLchSYTwEaPLDoFRo11qkWontw+kWAtb8ZqTTEYwTptAkSgl8dbJoI8a2URXIxDCOE7/g="
- attr(*, "headers")=List of 7
..$ x-amz-bucket-region: chr "us-west-2"
..$ x-amz-request-id   : chr "BGEZ97HJH10KAPRE"
..$ x-amz-id-2         : chr "pxKXcYNLchSYTwEaPLDoFRo11qkWontw+kWAtb8ZqTTEYwTptAkSgl8dbJoI8a2URXIxDCOE7/g="
..$ content-type       : chr "application/xml"
..$ transfer-encoding  : chr "chunked"
..$ date               : chr "Mon, 06 Jun 2022 17:35:35 GMT"
..$ server             : chr "AmazonS3"
..- attr(*, "class")= chr [1:2] "insensitive" "list"
- attr(*, "class")= chr "aws_error"
Error in parse_aws_s3_response(r, Sig, verbose = verbose) :
Moved Permanently (HTTP 301).

It's been a day since my first interaction with aws.s3 and I'm still going through the manual/forums, so all help will be appreciated. Thank you.

This topic was automatically closed 21 days after the last reply. New replies are no longer allowed.

If you have a query related to it or one of the replies, start a new topic and refer back with a link.