Python 2.7 error after successfully deploying to shinyapps.io using reticulate virtual_env()

Posit Professional & Hosted Products shinyapps.io
1

Hi,

I recently discovered shinyapps.io and loved it. I am having trouble using reticulate and python correctly.

I have written an app that uses reticulate and a small custom python3 module.
The app runs successfully locally, and after a bit of effort I was also able to get a successfully completed message in the Rstudio console after hitting "publish" (to deploy the app in my account on shinyapps.io).

But if I go online to my app I get the following error:

00
002156×854 77.2 KB

I followed the instructions on this post.

I am fairly certain this error is raised when Python2 is trying to use the code (which is meant to work with Python 3).

This is the setup part of my app.R code (that works successfully locally)

library(shiny)
library(shinythemes)
library(scales)
library(reticulate)

virtualenv_create(envname = "web-app-env", python= 'python3')
virtualenv_install('web-app-env', c('pandas', 'numpy'))
reticulate::use_virtualenv('web-app-env', required = TRUE)
source_python(file='./codebase/price_f.py')

# Define UI for application that draws a histogram
ui <- fluidPage( theme = shinytheme('flatly'),

Any idea why it tries to work with Python2 even though I explicitly give it the 'python3' option?

Any help would be appreciated!

2

Try running this line locally and then deploy again and see if it works.

devtools::install_github("rstudio/rsconnect", ref='737cd48', force = TRUE)

You may have to download devtools if you have not already. I had a similar problem and this fixed it for me. For some reason it seems that the current version of rsconnect does not work very well with python3 in reticulate. Hope that helps!

3

@esulli,

Thank you for the suggestion - unfortunately this didn't fix it for me.
Just to confirm that I followed your suggestion correctly: I opened Rstudio and ran your command in the terminal which installed successfully. Then hit "publish" again.

Thoughts?

Thanks again

4

Hi @Konstantin — thanks for the update. Unfortunately I'm not exactly sure what the problem is, but here is what I would try next:

  • Run the command above in the console and make sure that when you get a prompt that says These packages have more recent versions available. Which would you like to update?, you just click enter so that it doesn't update. I don't think this is the problem, but worth a try.
  • Try making a new folder with only your application (app.R) and python file price_f.py in it (and anything else that is directly needed in the app).
    • Then change your code to source_python('price_f.py').
  • If you don't have a line that says py_run_file('price_f.py'), add it right after the source_python('price_f.py') line.

Try those things and then re-deploy and see if it works. The questions I have if that doesn't work are:

  • What is going on in line 26 (classes.py)? It seems that is where the error is.
  • See what the logs say (if you go to your dashboard there will be a tab where you can see the logs) and let us know what error you are getting there.

Good luck!

5

Hi @esulli,

Thanks for the reply.
I took your suggestions and created a new directory containing only app.R and one python script that contains everything I need price_f.py . I also edited the app.R code according to your suggestions and the setup part is the following:

library(shiny)
library(shinythemes)
library(scales)
library(reticulate)

virtualenv_create(envname = "web-app-env", python= 'python3')
virtualenv_install('web-app-env', c('pandas', 'numpy'))
use_virtualenv('web-app-env', required = TRUE)
source_python(file='price_f.py')
py_run_file('price_f.py') # I think not needed

After publishing I got a similar error and then looked at the log (see below).
I think the error is caused by loading python2 instead of 3.

2019-04-11T18:21:11.170237+00:00 shinyapps[857967]: Collecting python-dateutil>=2.5.0 (from pandas)
2019-04-11T18:21:11.207555+00:00 shinyapps[857967]: Collecting pytz>=2011k (from pandas)
2019-04-11T18:21:11.364055+00:00 shinyapps[857967]:   Downloading https://files.pythonhosted.org/packages/3d/73/fe30c2daaaa0713420d0382b16fbb761409f532c56bdcc514bf7b6262bb6/pytz-2019.1-py2.py3-none-any.whl (510kB)
2019-04-11T18:21:11.444431+00:00 shinyapps[857967]: Collecting six>=1.5 (from python-dateutil>=2.5.0->pandas)
2019-04-11T18:21:11.476329+00:00 shinyapps[857967]: Installing collected packages: six, python-dateutil, numpy, pytz, pandas
2019-04-11T18:21:11.462367+00:00 shinyapps[857967]:   Downloading https://files.pythonhosted.org/packages/73/fb/00a976f728d0d1fecfe898238ce23f502a721c0ac0ecfedb80e0d88c64e9/six-1.12.0-py2.py3-none-any.whl
2019-04-11T18:21:15.159824+00:00 shinyapps[857967]: Successfully installed numpy-1.16.2 pandas-0.24.2 python-dateutil-2.8.0 pytz-2019.1 six-1.12.0
2019-04-11T18:21:15.417645+00:00 shinyapps[857967]: Error in value[[3L]](cond) : 
2019-04-11T18:21:15.417646+00:00 shinyapps[857967]:   SyntaxError: invalid syntax (<string>, line 24)
2019-04-11T18:21:15.417648+00:00 shinyapps[857967]: Calls: local ... tryCatch -> tryCatchList -> tryCatchOne -> <Anonymous>
2019-04-11T18:21:15.417649+00:00 shinyapps[857967]: Execution halted
2019-04-11T18:34:48.373630+00:00 shinyapps[857967]: Server version: 1.7.2-7
2019-04-11T18:34:48.373687+00:00 shinyapps[857967]: jsonlite version: 1.6
2019-04-11T18:34:48.373688+00:00 shinyapps[857967]: htmltools version: 0.3.6
2019-04-11T18:34:48.373682+00:00 shinyapps[857967]: R version: 3.4.3
2019-04-11T18:34:48.373684+00:00 shinyapps[857967]: shiny version: 1.2.0
2019-04-11T18:34:48.373685+00:00 shinyapps[857967]: httpuv version: 1.4.5.1
2019-04-11T18:34:48.373686+00:00 shinyapps[857967]: knitr version: NA
2019-04-11T18:34:48.373632+00:00 shinyapps[857967]: LANG: en_US.UTF-8
2019-04-11T18:34:48.373686+00:00 shinyapps[857967]: rmarkdown version: NA
2019-04-11T18:34:48.373688+00:00 shinyapps[857967]: RJSONIO version: NA
2019-04-11T18:34:49.423669+00:00 shinyapps[857967]: Did you forget to set the RETICULATE_PYTHON environment variable in your .Rprofile before publishing?
2019-04-11T18:34:49.423650+00:00 shinyapps[857967]: warning: using reticulate but python was not specified; will use python at /usr/bin/python 
2019-04-11T18:34:49.423842+00:00 shinyapps[857967]: Using pandoc at /opt/connect/ext/pandoc2
2019-04-11T18:34:49.508886+00:00 shinyapps[857967]: Using jsonlite for JSON processing
2019-04-11T18:34:49.571616+00:00 shinyapps[857967]: 
2019-04-11T18:34:49.571619+00:00 shinyapps[857967]: Starting R with process ID: '19'
2019-04-11T18:34:49.761391+00:00 shinyapps[857967]: Creating virtual environment 'web-app-env' ...
2019-04-11T18:34:49.761394+00:00 shinyapps[857967]: Using python: python3
2019-04-11T18:34:50.095004+00:00 shinyapps[857967]: New python executable in /home/shiny/.virtualenvs/web-app-env/bin/python2
2019-04-11T18:34:50.095006+00:00 shinyapps[857967]: Also creating executable in /home/shiny/.virtualenvs/web-app-env/bin/python
2019-04-11T18:34:53.021852+00:00 shinyapps[857967]: Installing setuptools, pkg_resources, pip, wheel...done.
2019-04-11T18:34:53.027847+00:00 shinyapps[857967]: Running virtualenv with interpreter /usr/bin/python2
2019-04-11T18:34:53.036703+00:00 shinyapps[857967]: Using virtual environment 'web-app-env' ...
2019-04-11T18:34:53.829041+00:00 shinyapps[857967]: DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7.
2019-04-11T18:34:53.929021+00:00 shinyapps[857967]: Collecting pandas
2019-04-11T18:34:54.170788+00:00 shinyapps[857967]:   Downloading https://files.pythonhosted.org/packages/db/83/7d4008ffc2988066ff37f6a0bb6d7b60822367dcb36ba5e39aa7801fda54/pandas-0.24.2-cp27-cp27mu-manylinux1_x86_64.whl (10.1MB)
2019-04-11T18:34:54.837614+00:00 shinyapps[857967]: Collecting numpy
2019-04-11T18:34:55.136547+00:00 shinyapps[857967]:   Downloading https://files.pythonhosted.org/packages/c4/33/8ec8dcdb4ede5d453047bbdbd01916dbaccdb63e98bba60989718f5f0876/numpy-1.16.2-cp27-cp27mu-manylinux1_x86_64.whl (17.0MB)
2019-04-11T18:34:56.188383+00:00 shinyapps[857967]: Collecting python-dateutil>=2.5.0 (from pandas)
2019-04-11T18:34:56.215386+00:00 shinyapps[857967]:   Downloading https://files.pythonhosted.org/packages/41/17/c62faccbfbd163c7f57f3844689e3a78bae1f403648a6afb1d0866d87fbb/python_dateutil-2.8.0-py2.py3-none-any.whl (226kB)
2019-04-11T18:34:56.229229+00:00 shinyapps[857967]: Collecting pytz>=2011k (from pandas)
2019-04-11T18:34:56.496965+00:00 shinyapps[857967]:   Downloading https://files.pythonhosted.org/packages/3d/73/fe30c2daaaa0713420d0382b16fbb761409f532c56bdcc514bf7b6262bb6/pytz-2019.1-py2.py3-none-any.whl (510kB)
2019-04-11T18:34:56.599479+00:00 shinyapps[857967]: Collecting six>=1.5 (from python-dateutil>=2.5.0->pandas)
2019-04-11T18:34:56.620450+00:00 shinyapps[857967]:   Downloading https://files.pythonhosted.org/packages/73/fb/00a976f728d0d1fecfe898238ce23f502a721c0ac0ecfedb80e0d88c64e9/six-1.12.0-py2.py3-none-any.whl
2019-04-11T18:34:56.636426+00:00 shinyapps[857967]: Installing collected packages: six, python-dateutil, numpy, pytz, pandas
2019-04-11T18:35:00.975228+00:00 shinyapps[857967]: Successfully installed numpy-1.16.2 pandas-0.24.2 python-dateutil-2.8.0 pytz-2019.1 six-1.12.0
2019-04-11T18:35:01.368140+00:00 shinyapps[857967]: Error in value[[3L]](cond) : 
2019-04-11T18:35:01.368142+00:00 shinyapps[857967]:   SyntaxError: invalid syntax (<string>, line 24)
2019-04-11T18:35:01.368145+00:00 shinyapps[857967]: Execution halted
2019-04-11T18:35:01.368144+00:00 shinyapps[857967]: Calls: local ... tryCatch -> tryCatchList -> tryCatchOne -> <Anonymous>

I think the wrong python load happens in lines

2019-04-11T18:34:49.761394+00:00 shinyapps[857967]: Using python: python3
2019-04-11T18:34:50.095004+00:00 shinyapps[857967]: New python executable in /home/shiny/.virtualenvs/web-app-env/bin/python2

Line 24 of the code is a call to pandas from a class method.

self.dates = pd.date_range(**kwds, normalize=True)

I guess this could be a bug but it works fine locally. I am trying to test it with python 2 locally but I haven't figured out how to do that yet...

Thanks for your help so far.

6

Hi! I'm sorry to hear those things didn't work. Another thought that I just had could be that it is not recognizing pd as pandas (so you could try pandas.date_range instead of pd.date_range). It seems unlikely given that it's working locally, but worth a try.

Regarding the logs, a more experienced user may be able to help interpret them more, but interestingly I have an app that successfully deployed on shinyapps.io and requires Python3 for which the logs also imply that I am using Python2 (like yours). I am trying to figure out what the differences could be and will let you know if I come up with anything. Good luck!

7

Hi @Konstantin! Have you figured out a way to resolve this issue? I had almost the same problem (see the topic I created here), and it seems that the server keeps switching back to python2.

8

Unfortunately I wasn't able to resolve it. I would be interested in a solution.
Until then I will be writing my code in R

1 Like