How to upload ndjson to aws s3?

I need to save a r object to Amazon s3. I can do this already with the aws.s3 package e.g:

s3save(mtcars, bucket = "s3://ourco-emr/", object = "tables/adhoc.db/mtcars/2020/03/06/mtcars")

But I need mtars to be in ndjson format.

I can create a file in ndjson format like so:

predictions_file <- file("mtcars.json")
jsonlite::stream_out(mtcars, predictions_file)

This write a file to my directory called mtcars.json and it's in the right format.

The documentation for aws.s3::s3save says:

s3save {aws.s3}	R Documentation
Save/load R object(s) to/from S3

s3save(..., object, bucket, envir = parent.frame(), opts = NULL)

s3save_image(object, bucket, opts = NULL)

s3load(object, bucket, envir = parent.frame(), ...)
For s3save, one or more R objects to be saved via save and uploaded to S3. For s3load, see opts.

For s3save, a character string of the name of the object you want to save to. For s3load, a character string of the name of the object you want to load from S3.

Character string with the name of the bucket, or an object of class “s3_bucket”.

For s3save, an R environment to save objects from; for s3load, the environment to load objects into. Default is the parent.frame() from which the function is called.

Additional arguments passed to s3HTTP.

For s3save, a logical, invisibly. For s3load, NULL invisibly.

The ... argument is the data I want to save to aws s3. The problem is I cannot get the correctly formatted contents of mtcars.json as an object in memory to send to s3.

How could I transfer the file, mtcars.json to aws s3 with the s3save() function?

I tried this with no success:

              FUN = jsonlite::stream_out,
              con = predictions_file,
              object = "tables/adhoc.db/mtcars/2020/03/06")


Error in if (verbose) message("opening ", is(con), " output connection.") :
argument is not interpretable as logical

This topic was automatically closed 21 days after the last reply. New replies are no longer allowed.