pin_write hangs when writing to board_databricks

General
1

I am trying to use pins to write to my company's Databricks Unity Catalog (UC) Volumes. I am using this documentation as a reference.

Environment:

  • Local Windows 10 Enterprise machine
  • R 4.4.1 RC
  • RStudio 2024.04.2+764
  • pins 1.4

I am able to write the iris dataset to my UC volume using board_databricks and pin_write. I can view the expected "data.txt" and "iris.csv" files in UC.

However, when I try to read a 16MB CSV file from disk into R and then use pin_write, it starts the process ("Creating new version...") and then hangs. It creates the folder structure and files ("data.txt" and "test_file.csv") in the UC volume, but the process never completes. I have let it run for an hour without any success. When I interrupt the R process and download the file from UC, it is only 64K. I have tried multiple files, and it always shows that the file size is 64K after interrupting it (no matter how long I let the process run). This makes me think it could be a buffer size issue with curl.

Does anyone have any advice on how to troubleshoot this issue?

2

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.

If you have a query related to it or one of the replies, start a new topic and refer back with a link.