Error reading uploaded file and saving to new location

[Erin_Gill]

Hello,
I've created a python shiny express app where I'm trying to allow the user to upload a file, then save it to a new location so that multiple functions can be performed on it. To accomplish this, I've done the following:

# imports
from shiny.express import input, render, ui # interactivity
from shiny import reactive, render # reactivity (i.e. calculations)
from shiny.types import FileInfo
from pathlib import Path

# create an upload box
ui.input_file("file1", accept=['.csv'], multiple = False,)

# when the user uploads a file, open it and save it to a new location
@reactive.calc
def parsed_file():
    if not input.file1():
        return
    try:
        # try to open the file and read contents
        with open(input.file1()[0]["datapath"], "r") as f:
            content = f.read()
        # this is the new location where the file contents will be saved
        file_path = Path(__file__).parent / "./data/results/user_input.csv"
        # first create the file
        file_path.touch()
        # then open it and write the contents from the uploaded file
        with open(file_path, "w") as f2:
            f2.write(content)
        # return the file contents 
        return content
    except:
        # if this doesn't work, tell me
        return "Error"

When I run the code locally, everything works as expected, but when I deploy to shinyapps.io, "Error" is returned.

I have tried to edit my code to make sure that this directory is in the $PATH:

from pathlib import Path
import os
path = Path(__file__).parent / "./data/results/"
os.environ['PATH'] += os.pathsep + str(path)

I'm not sure what else to try. I seem to be doing things the same way as this example Shiny examples.

[pieterjanvc]

Hello,

I think what might be happening here is that the data/results folders don't exist remotely when you first start the app, and thus you can't move a file there. Try to explicitly create the empty folder first before moving files to it.

Although this doesn't seem an issue in this case, also remember that on remote Shiny app data can only be read/written into the root or sub-folders of the app, and nothing outside of that.

Hope this helps,
PJ

[Erin_Gill]

Hi PJ,
thanks for your reply. I've made sure that the directory /data/results exists before the app is deployed. I've also added error messages that are returned from the function that reads the file and writes it to the new location. Each step of the function appends new text to the error message. The code now looks like this:

# imports
from shiny.express import input, render, ui # interactivity
from shiny import reactive, render # reactivity (i.e. calculations)
from shiny.types import FileInfo
from pathlib import Path
import os
import time
import subprocess

# create an upload box
ui.input_file("file1", accept=['.csv'], multiple = False,)

# when the user uploads a file, open it and save it to a new location
@reactive.calc
    def parsed_file():
        err_msg = "Error"
        if not input.file1():
            return
        try:
            # read the file
            with open(input.file1()[0]["datapath"], "r") as f:
                content = f.read()
                err_msg += "read"
                timestr = time.strftime("%m%d-%H%M%S")
                # create file path
                file_path = Path(__file__).parent / f"./data/results/{timestr}.fasta" #need unique name
                err_msg += "path"
                # change ownership of the directory "/data/results" to shiny:shiny
                os.system("sudo chown -R shiny:shiny ./data/results/")
                err_msg += "own"
                # change permissions for the directory to 777
                subprocess.run(['chmod', '0777', "./data/results/"])
                err_msg += "mod"
                # make node for file in directory
                fh1 = os.open (f"./data/results/{timestr}.fasta", os.O_CREAT, 0o777)
                os.close (fh1)
                err_msg += "touch"
                # open new file
                with open(file_path, "w") as f2:
                    err_msg += "open"
                    # write contents to new file
                    f2.write(content)
                    err_msg += "write"
                # if it works, return file contents and file path
                return content, file_path
        except:
                # if it doesn't work, return error message
                return err_msg, err_msg

On the server, everything works up to (and including) modifying directory permissions. Creating the file node doesn't work.

The directory structure of my app (when it is pushed to the server) is as follows:

app
|
-css
|    |-styles.css
|
-data
|    |-reference_files
|    |    |-reference_file1.csv
|    |
|    |-results
|    |    |- (empty)
|    |
|    |-datafile1.csv
|    |-datafile2.csv
|
-tests
|    |-test_pytest.py
|
|-__init__.py
|-app.py
|-functions.py

Any thoughts?

[pieterjanvc]

Hello,

Few thoughts:

• First, I'd be surprised if a sudo command is going to work on the server as you will generally not have these types of permissions. That said, you should be owner and have full control over the files and subfolders within your app folder
• Second, although you have the results folder locally, when pushing empty folders are ignored I think (similar to how it's handled in Git). I would test to create the actual results folder in your code if it's not present then proceed with creating the file
• Finally, can you share the actual error message shown in the logs (you can view them on shinyapps.io or wherever you host)

Let me know if you make any progress
PJ

[Erin_Gill]

Thanks so much! The error was that the empty folder wasn't getting pushed to the server. Everything is working as expected now.