Hi!
I want so save an arrow schema as .rds file and later load it back into R.
As soon as I define one field as string() I get the following error message (other arrow types work as expected):
library(arrow)
library(readr)
test_schema <- list(test = string())
write_rds(test_schema, "test.rds")
read_rds("test.rds")
#> $test
#> Utf8
#> Error: Invalid <Utf8>, external pointer to null
Created on 2022-11-28 with reprex v2.0.2
Hi @duringju211 !
What's happening here is that the string() object in the test_schema list contains a pointer to a C++ object, and it can't be serialised.
As a short-term workaround, you could save the code for the schema, and then read it back into R and evaluate it to recreate the schema. I don't love this solution though, as I've found in the past that most times a solution involves calling eval it's not the right one, so I'll see if I can find you a better alternative!
library(arrow)
library(readr)
test_schema <- schema(test = string())
write_rds(test_schema$code(), "test.rds")
input_schema_call <- read_rds("test.rds")
input_schema <- eval(input_schema_call)
input_schema
#> Schema
#> test: string
Nic