Unexpected Plumber API issues with Reticulate and RStudio Connect

[Wolfpack2]

I've encountered some odd behavior in RStudio Connect while deploying a Plumber API with Reticulate.

I have a piece of equipment that performs a CURL POST command to upload a file to my Plumber API. The API copies the file locally and runs a Python script against it using Reticulate. In the Plumber API, I am building the Python Virtual Env in the code which I suspect is the root of the issue. These are the types of functions I am calling in Plumber:

1. virtualenv_create
2. virtualenv_install
3. py_run_string
4. py_run_file

The code executes successfully when it is called only once or twice. The problem occurs when I have a lot of files I am trying to send. The files are successfully sent to the API and copied but the Python script fails. This is the type of error I am seeing in the logs:

<Rcpp::exception in py_run_file_impl(file, local, convert): Unable to open file 'script.py' (does it exist?)
An error occurred: render-python-code-error
Possible causes: There is a bug in the Python code being deployed.

I'm wondering if somehow RStudio Connect is choking on multiple calls the virtual environment that is created. Anyone have any ideas on what's going on here or how to debug this?

[edavidaja]

One potential issue here: Connect builds an isolated environment when your content is deployed, so I would avoid using virtualenv_create() and virtualenv_install() in your deployed API; if you enumerate the needed python packages in a requirements.txt file, remove those lines from your code, and redploy you may have a better experience.

[Wolfpack2]

Thanks for the quick response. My impression of requirements.txt is it's used when deploying Python-specific applications like Dash, Streamlit, Bokeh, etc. If I include a requirements.txt in a Plumber API, will it construct the virtual env for me? How would I access it from the code?

[edavidaja]

Here's an example Plumber API that's serving a spaCy model: https://github.com/sol-eng/python-examples/tree/main/sentiment-analysis

the requirements.txt tells Connect what is needed to rebuild a python environment, whether that environment is serving interactive applications or just functions.

[Wolfpack2]

Does this mean that including a requirements.txt file will automatically generate a Python environment available to Plumber via Reticulate? That would be very handy.