Error in Installing Keras on RStudio Cloud

Posit Professional & Hosted Products Posit Cloud
1

Today I change my plan from free to instructor in RStudio Cloud and I try to deploy my ML working space on it. But it doesn't work:

library(reticulate)
library(keras)
install_keras()
Collecting package metadata (current_repodata.json): ...working... done
Solving environment: ...working... done

# All requested packages already installed.
...
Installing collected packages: tensorflow, pyyaml, keras, tensorflow-hub, Pillow
ERROR: Could not install packages due to an EnvironmentError: [Errno 28] No space left on device
Error: Error installing package(s): 'tensorflow==2.2.0', 'keras', 'tensorflow-hub', 'h5py', 'pyyaml==3.12', 'requests', 'Pillow', 'scipy'
2

You are not alone: Installation of tensorflow terminated on RStudio Cloud

3

This actually appears to be a different error than the one experienced when running out of memory.

Have you installed anything else into the project before this?

4

At first I thought it might be the problem of the beta version of RStudio Cloud, but for now I upgrade my plan it still exists.

I don't have too much experience of working with Linux system, but it works fine on my Windows desktop.

5

No, it is a brand new project

4 CPU and 8G RAM.(which may be the top configuration of RStudio Cloud now.)

6

Hello

I think i have a workaround that should work for you. By default, these libraries use miniconda, but if you use virtaulenv instead, things should install. For example:

library(reticulate)
library(keras)
virtualenv_create("myenv")
install_keras(method="virtualenv", envname="myenv")

Let me know if that works for you.

thanks
sam

7

Thank you for your kind help.
it seems success but I still can't use keras normally.
Here I listed my output log

> install_keras(method="virtualenv", envname="myenv")
Using virtual environment 'myenv' ...
Collecting tensorflow==2.2.0
  Using cached tensorflow-2.2.0-cp36-cp36m-manylinux2010_x86_64.whl (516.2 MB)
Collecting keras
  Using cached Keras-2.4.3-py2.py3-none-any.whl (36 kB)
Collecting tensorflow-hub
  Using cached tensorflow_hub-0.8.0-py2.py3-none-any.whl (101 kB)
Requirement already up-to-date: h5py in /home/rstudio-user/.local/share/r-miniconda/envs/r-reticulate/lib/python3.6/site-packages (2.10.0)
Processing /home/rstudio-user/.cache/pip/wheels/40/06/c3/a48dc77c7d8f60b64eaf0fffd5ee5ab8abce5d13893b73109b/PyYAML-3.12-cp36-cp36m-linux_x86_64.whl
Requirement already up-to-date: requests in /home/rstudio-user/.local/share/r-miniconda/envs/r-reticulate/lib/python3.6/site-packages (2.24.0)
Collecting Pillow
  Using cached Pillow-7.2.0-cp36-cp36m-manylinux1_x86_64.whl (2.2 MB)
Requirement already up-to-date: scipy in /home/rstudio-user/.local/share/r-miniconda/envs/r-reticulate/lib/python3.6/site-packages (1.5.2)
Requirement already satisfied, skipping upgrade: tensorflow-estimator<2.3.0,>=2.2.0 in /home/rstudio-user/.local/share/r-miniconda/envs/r-reticulate/lib/python3.6/site-packages (from tensorflow==2.2.0) (2.2.0)
Requirement already satisfied, skipping upgrade: wrapt>=1.11.1 in /home/rstudio-user/.local/share/r-miniconda/envs/r-reticulate/lib/python3.6/site-packages (from tensorflow==2.2.0) (1.12.1)
Requirement already satisfied, skipping upgrade: keras-preprocessing>=1.1.0 in /home/rstudio-user/.local/share/r-miniconda/envs/r-reticulate/lib/python3.6/site-packages (from tensorflow==2.2.0) (1.1.2)
Requirement already satisfied, skipping upgrade: opt-einsum>=2.3.2 in /home/rstudio-user/.local/share/r-miniconda/envs/r-reticulate/lib/python3.6/site-packages (from tensorflow==2.2.0) (3.3.0)
Requirement already satisfied, skipping upgrade: protobuf>=3.8.0 in /home/rstudio-user/.local/share/r-miniconda/envs/r-reticulate/lib/python3.6/site-packages (from tensorflow==2.2.0) (3.12.4)
Requirement already satisfied, skipping upgrade: google-pasta>=0.1.8 in /home/rstudio-user/.local/share/r-miniconda/envs/r-reticulate/lib/python3.6/site-packages (from tensorflow==2.2.0) (0.2.0)
Requirement already satisfied, skipping upgrade: termcolor>=1.1.0 in /home/rstudio-user/.local/share/r-miniconda/envs/r-reticulate/lib/python3.6/site-packages (from tensorflow==2.2.0) (1.1.0)
Requirement already satisfied, skipping upgrade: gast==0.3.3 in /home/rstudio-user/.local/share/r-miniconda/envs/r-reticulate/lib/python3.6/site-packages (from tensorflow==2.2.0) (0.3.3)
Requirement already satisfied, skipping upgrade: absl-py>=0.7.0 in /home/rstudio-user/.local/share/r-miniconda/envs/r-reticulate/lib/python3.6/site-packages (from tensorflow==2.2.0) (0.9.0)
Requirement already satisfied, skipping upgrade: tensorboard<2.3.0,>=2.2.0 in /home/rstudio-user/.local/share/r-miniconda/envs/r-reticulate/lib/python3.6/site-packages (from tensorflow==2.2.0) (2.2.2)
Requirement already satisfied, skipping upgrade: wheel>=0.26; python_version >= "3" in ./myenv/lib/python3.6/site-packages (from tensorflow==2.2.0) (0.35.0)
Requirement already satisfied, skipping upgrade: numpy<2.0,>=1.16.0 in /home/rstudio-user/.local/share/r-miniconda/envs/r-reticulate/lib/python3.6/site-packages (from tensorflow==2.2.0) (1.19.1)
Requirement already satisfied, skipping upgrade: grpcio>=1.8.6 in /home/rstudio-user/.local/share/r-miniconda/envs/r-reticulate/lib/python3.6/site-packages (from tensorflow==2.2.0) (1.30.0)
Requirement already satisfied, skipping upgrade: astunparse==1.6.3 in /home/rstudio-user/.local/share/r-miniconda/envs/r-reticulate/lib/python3.6/site-packages (from tensorflow==2.2.0) (1.6.3)
Requirement already satisfied, skipping upgrade: six>=1.12.0 in /home/rstudio-user/.local/share/r-miniconda/envs/r-reticulate/lib/python3.6/site-packages (from tensorflow==2.2.0) (1.15.0)
Requirement already satisfied, skipping upgrade: chardet<4,>=3.0.2 in /home/rstudio-user/.local/share/r-miniconda/envs/r-reticulate/lib/python3.6/site-packages (from requests) (3.0.4)
Requirement already satisfied, skipping upgrade: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /home/rstudio-user/.local/share/r-miniconda/envs/r-reticulate/lib/python3.6/site-packages (from requests) (1.25.10)
Requirement already satisfied, skipping upgrade: certifi>=2017.4.17 in /home/rstudio-user/.local/share/r-miniconda/envs/r-reticulate/lib/python3.6/site-packages (from requests) (2020.6.20)
Requirement already satisfied, skipping upgrade: idna<3,>=2.5 in /home/rstudio-user/.local/share/r-miniconda/envs/r-reticulate/lib/python3.6/site-packages (from requests) (2.10)
Requirement already satisfied, skipping upgrade: setuptools in ./myenv/lib/python3.6/site-packages (from protobuf>=3.8.0->tensorflow==2.2.0) (49.6.0)
Requirement already satisfied, skipping upgrade: google-auth-oauthlib<0.5,>=0.4.1 in /home/rstudio-user/.local/share/r-miniconda/envs/r-reticulate/lib/python3.6/site-packages (from tensorboard<2.3.0,>=2.2.0->tensorflow==2.2.0) (0.4.1)
Requirement already satisfied, skipping upgrade: google-auth<2,>=1.6.3 in /home/rstudio-user/.local/share/r-miniconda/envs/r-reticulate/lib/python3.6/site-packages (from tensorboard<2.3.0,>=2.2.0->tensorflow==2.2.0) (1.20.0)
Requirement already satisfied, skipping upgrade: tensorboard-plugin-wit>=1.6.0 in /home/rstudio-user/.local/share/r-miniconda/envs/r-reticulate/lib/python3.6/site-packages (from tensorboard<2.3.0,>=2.2.0->tensorflow==2.2.0) (1.7.0)
Requirement already satisfied, skipping upgrade: werkzeug>=0.11.15 in /home/rstudio-user/.local/share/r-miniconda/envs/r-reticulate/lib/python3.6/site-packages (from tensorboard<2.3.0,>=2.2.0->tensorflow==2.2.0) (1.0.1)
Requirement already satisfied, skipping upgrade: markdown>=2.6.8 in /home/rstudio-user/.local/share/r-miniconda/envs/r-reticulate/lib/python3.6/site-packages (from tensorboard<2.3.0,>=2.2.0->tensorflow==2.2.0) (3.2.2)
Requirement already satisfied, skipping upgrade: packaging>=20.2 in ./myenv/lib/python3.6/site-packages (from wheel>=0.26; python_version >= "3"->tensorflow==2.2.0) (20.4)
Requirement already satisfied, skipping upgrade: requests-oauthlib>=0.7.0 in /home/rstudio-user/.local/share/r-miniconda/envs/r-reticulate/lib/python3.6/site-packages (from google-auth-oauthlib<0.5,>=0.4.1->tensorboard<2.3.0,>=2.2.0->tensorflow==2.2.0) (1.3.0)
Requirement already satisfied, skipping upgrade: pyasn1-modules>=0.2.1 in /home/rstudio-user/.local/share/r-miniconda/envs/r-reticulate/lib/python3.6/site-packages (from google-auth<2,>=1.6.3->tensorboard<2.3.0,>=2.2.0->tensorflow==2.2.0) (0.2.8)
Requirement already satisfied, skipping upgrade: rsa<5,>=3.1.4; python_version >= "3.5" in /home/rstudio-user/.local/share/r-miniconda/envs/r-reticulate/lib/python3.6/site-packages (from google-auth<2,>=1.6.3->tensorboard<2.3.0,>=2.2.0->tensorflow==2.2.0) (4.6)
Requirement already satisfied, skipping upgrade: cachetools<5.0,>=2.0.0 in /home/rstudio-user/.local/share/r-miniconda/envs/r-reticulate/lib/python3.6/site-packages (from google-auth<2,>=1.6.3->tensorboard<2.3.0,>=2.2.0->tensorflow==2.2.0) (4.1.1)
Requirement already satisfied, skipping upgrade: importlib-metadata; python_version < "3.8" in /home/rstudio-user/.local/share/r-miniconda/envs/r-reticulate/lib/python3.6/site-packages (from markdown>=2.6.8->tensorboard<2.3.0,>=2.2.0->tensorflow==2.2.0) (1.7.0)
Requirement already satisfied, skipping upgrade: pyparsing>=2.0.2 in ./myenv/lib/python3.6/site-packages (from packaging>=20.2->wheel>=0.26; python_version >= "3"->tensorflow==2.2.0) (2.4.7)
Requirement already satisfied, skipping upgrade: oauthlib>=3.0.0 in /home/rstudio-user/.local/share/r-miniconda/envs/r-reticulate/lib/python3.6/site-packages (from requests-oauthlib>=0.7.0->google-auth-oauthlib<0.5,>=0.4.1->tensorboard<2.3.0,>=2.2.0->tensorflow==2.2.0) (3.1.0)
Requirement already satisfied, skipping upgrade: pyasn1<0.5.0,>=0.4.6 in /home/rstudio-user/.local/share/r-miniconda/envs/r-reticulate/lib/python3.6/site-packages (from pyasn1-modules>=0.2.1->google-auth<2,>=1.6.3->tensorboard<2.3.0,>=2.2.0->tensorflow==2.2.0) (0.4.8)
Requirement already satisfied, skipping upgrade: zipp>=0.5 in /home/rstudio-user/.local/share/r-miniconda/envs/r-reticulate/lib/python3.6/site-packages (from importlib-metadata; python_version < "3.8"->markdown>=2.6.8->tensorboard<2.3.0,>=2.2.0->tensorflow==2.2.0) (3.1.0)
Installing collected packages: tensorflow, pyyaml, keras, tensorflow-hub, Pillow
ERROR: After October 2020 you may experience errors when installing or updating packages. This is because pip will change the way that it resolves dependency conflicts.

We recommend you use --use-feature=2020-resolver to test your packages with the new resolver before it becomes the default.

tensorflow 2.2.0 requires scipy==1.4.1; python_version >= "3", but you'll have scipy 1.5.2 which is incompatible.
Successfully installed Pillow-7.2.0 keras-2.4.3 pyyaml-3.12 tensorflow-2.2.0 tensorflow-hub-0.8.0

Installation complete.


Restarting R session...
> library(reticulate)
> library(keras)
> mnist <- dataset_mnist()
Error: Installation of TensorFlow not found.

Python environments searched for 'tensorflow' package:
 /home/rstudio-user/.local/share/r-miniconda/envs/r-reticulate/bin/python3.6

You can install TensorFlow using the install_tensorflow() function.

The Install_Keras should install tensorflow at the same time it runs, I don't know what to do now.
I try to use tensorflow package , the same error appears.

8

Hi Samp, I have the same problem -- TF not install. Try the workaround but still no luck. Error message: Killed
Error: Error installing package(s): 'tensorflow==2.2.0', 'keras', 'tensorflow-hub', 'h5py', 'pyyaml==3.12', 'requests', 'Pillow'

9

I fix this problem by using the code posted above, but new problems occurred and the installation failed.

10

Hi hamdany, you will need more than 1GB of RAM available if you want to use this library. If you continue to have problems, can you please create a new topic?

Ziqian_xia, I'm not sure what the issue is. I will have to find someone more familiar with tensorflow.

thanks
sam

11

It looks like it is still looking in miniconda. I think my workaround was missing a command, can you try this:

use_virtualenv("myenv")
12

I tried to add this code but it still doesn't work.
Below is my code log:

> library(reticulate)
> library(keras)
> virtualenv_create("myenv")
virtualenv: myenv
> use_virtualenv("myenv")
> install_keras(method="virtualenv", envname="myenv")
Using virtual environment 'myenv' ...
Requirement ....already satisfied

Installation complete.


Restarting R session...

Error saving session (creating directory): No space left on device
Error saving session (rversion): No such file or directory
Error in file(file, "wb") : cannot open the connection
In addition: Warning message:
In file(file, "wb") :
  cannot open file '/cloud/project/.Rproj.user/D4524F12/ctx/ctx-8787/libpaths': No such file or directory
Error saving session (libpaths): R code execution error
Error in file(file, "wb") : cannot open the connection
Error saving session (options): R code execution error
Error saving session (history): No such file or directory
Error saving session (console_actions): No such file or directory
Error in file(file, "wb") : cannot open the connection
In addition: Warning message:
In file(file, "wb") :
  cannot open file '/cloud/project/.Rproj.user/D4524F12/ctx/ctx-8787/environmentTmp': No such file or directory
Error saving session (search_path): R code execution error
WARNING: Forcing suspend of process in spite of all session data not being fully saved.
13

It looks like you have run out of disk space. Was this with a brand new project? I created a new project with 2GB RAM and ran the same commands you ran and had no errors. I also have just under 1GB of free disk space on the project volume. Is it possible something else was added to the project?

sam

14

Hi samp, thanks for your patience.
I created a new project with 2GB RAM and 1.5 CPU, and the code works, but I still can't use keras normally:
Here I listed my log :

library(reticulate)
library(keras)
virtualenv_create("myenv")
use_virtualenv("myenv")
install_keras(method="virtualenv", envname="myenv")

So far it works fine:

> library(reticulate)
> library(keras)
> virtualenv_create("myenv")
virtualenv: myenv
> use_virtualenv("myenv")
> install_keras(method="virtualenv", envname="myenv")
Using virtual environment 'myenv' ...
Collecting tensorflow==2.2.0
  Downloading tensorflow-2.2.0-cp35-cp35m-manylinux2010_x86_64.whl (516.2 MB)
Collecting keras
  Downloading Keras-2.4.3-py2.py3-none-any.whl (36 kB)
Collecting tensorflow-hub
  Downloading tensorflow_hub-0.9.0-py2.py3-none-any.whl (103 kB)
Collecting h5py
  Downloading h5py-2.10.0-cp35-cp35m-manylinux1_x86_64.whl (2.8 MB)
Collecting pyyaml==3.12
  Downloading PyYAML-3.12.tar.gz (253 kB)
Collecting requests
  Downloading requests-2.24.0-py2.py3-none-any.whl (61 kB)
Collecting Pillow
  Downloading Pillow-7.2.0-cp35-cp35m-manylinux1_x86_64.whl (2.2 MB)
Collecting scipy
  Downloading scipy-1.4.1-cp35-cp35m-manylinux1_x86_64.whl (26.0 MB)
Collecting termcolor>=1.1.0
  Downloading termcolor-1.1.0.tar.gz (3.9 kB)
Collecting keras-preprocessing>=1.1.0
  Downloading Keras_Preprocessing-1.1.2-py2.py3-none-any.whl (42 kB)
Collecting protobuf>=3.8.0
  Downloading protobuf-3.13.0-cp35-cp35m-manylinux1_x86_64.whl (1.3 MB)
Collecting numpy<2.0,>=1.16.0
  Downloading numpy-1.18.5-cp35-cp35m-manylinux1_x86_64.whl (19.9 MB)
Collecting grpcio>=1.8.6
  Downloading grpcio-1.31.0-cp35-cp35m-manylinux2014_x86_64.whl (3.4 MB)
Collecting tensorflow-estimator<2.3.0,>=2.2.0
  Downloading tensorflow_estimator-2.2.0-py2.py3-none-any.whl (454 kB)
Collecting gast==0.3.3
  Downloading gast-0.3.3-py2.py3-none-any.whl (9.7 kB)
Collecting wrapt>=1.11.1
  Downloading wrapt-1.12.1.tar.gz (27 kB)
Requirement already satisfied, skipping upgrade: wheel>=0.26; python_version >= "3" in ./myenv/lib/python3.5/site-packages (from tensorflow==2.2.0) (0.35.1)
Collecting astunparse==1.6.3
  Downloading astunparse-1.6.3-py2.py3-none-any.whl (12 kB)
Collecting absl-py>=0.7.0
  Downloading absl_py-0.10.0-py3-none-any.whl (127 kB)
Collecting google-pasta>=0.1.8
  Downloading google_pasta-0.2.0-py3-none-any.whl (57 kB)
Collecting opt-einsum>=2.3.2
  Downloading opt_einsum-3.3.0-py3-none-any.whl (65 kB)
Collecting tensorboard<2.3.0,>=2.2.0
  Downloading tensorboard-2.2.2-py3-none-any.whl (3.0 MB)
Collecting six>=1.12.0
  Downloading six-1.15.0-py2.py3-none-any.whl (10 kB)
Collecting urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1
  Downloading urllib3-1.25.10-py2.py3-none-any.whl (127 kB)
Collecting idna<3,>=2.5
  Downloading idna-2.10-py2.py3-none-any.whl (58 kB)
Collecting chardet<4,>=3.0.2
  Downloading chardet-3.0.4-py2.py3-none-any.whl (133 kB)
Collecting certifi>=2017.4.17
  Downloading certifi-2020.6.20-py2.py3-none-any.whl (156 kB)
Requirement already satisfied, skipping upgrade: setuptools in ./myenv/lib/python3.5/site-packages (from protobuf>=3.8.0->tensorflow==2.2.0) (49.6.0)
Collecting markdown>=2.6.8
  Downloading Markdown-3.2.2-py3-none-any.whl (88 kB)
Collecting google-auth-oauthlib<0.5,>=0.4.1
  Downloading google_auth_oauthlib-0.4.1-py2.py3-none-any.whl (18 kB)
Collecting tensorboard-plugin-wit>=1.6.0
  Downloading tensorboard_plugin_wit-1.7.0-py3-none-any.whl (779 kB)
Collecting google-auth<2,>=1.6.3
  Downloading google_auth-1.20.1-py2.py3-none-any.whl (91 kB)
Collecting werkzeug>=0.11.15
  Downloading Werkzeug-1.0.1-py2.py3-none-any.whl (298 kB)
Collecting importlib-metadata; python_version < "3.8"
  Downloading importlib_metadata-1.7.0-py2.py3-none-any.whl (31 kB)
Collecting requests-oauthlib>=0.7.0
  Downloading requests_oauthlib-1.3.0-py2.py3-none-any.whl (23 kB)
Collecting cachetools<5.0,>=2.0.0
  Downloading cachetools-4.1.1-py3-none-any.whl (10 kB)
Collecting pyasn1-modules>=0.2.1
  Downloading pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Collecting rsa<5,>=3.1.4; python_version >= "3.5"
  Downloading rsa-4.6-py3-none-any.whl (47 kB)
Collecting zipp>=0.5
  Downloading zipp-1.2.0-py2.py3-none-any.whl (4.8 kB)
Collecting oauthlib>=3.0.0
  Downloading oauthlib-3.1.0-py2.py3-none-any.whl (147 kB)
Collecting pyasn1<0.5.0,>=0.4.6
  Downloading pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Building wheels for collected packages: pyyaml, termcolor, wrapt
  Building wheel for pyyaml (setup.py): started
  Building wheel for pyyaml (setup.py): finished with status 'done'
  Created wheel for pyyaml: filename=PyYAML-3.12-cp35-cp35m-linux_x86_64.whl size=43059 sha256=1c439b3a2d71b37d7bb6775588e70163526597a49589c62add2713078c7056aa
  Stored in directory: /home/rstudio-user/.cache/pip/wheels/36/e8/a2/931a0f5060f69996e1d78a0b0ae235c042fee2a767177393bc
  Building wheel for termcolor (setup.py): started
  Building wheel for termcolor (setup.py): finished with status 'done'
  Created wheel for termcolor: filename=termcolor-1.1.0-py3-none-any.whl size=4830 sha256=08122a9079037de5d14ab14f065c042b53ce7832cfb61f0bfb064a879a243654
  Stored in directory: /home/rstudio-user/.cache/pip/wheels/91/0e/11/1f1321dce76e9c542907008e4a94ff79f8bf525a3fa32b09f3
  Building wheel for wrapt (setup.py): started
  Building wheel for wrapt (setup.py): finished with status 'done'
  Created wheel for wrapt: filename=wrapt-1.12.1-cp35-cp35m-linux_x86_64.whl size=64234 sha256=499296631f7eb949b93ac721f8aad515c2f658a6da9df4b2dfa05c98e877ac82
  Stored in directory: /home/rstudio-user/.cache/pip/wheels/b8/2d/88/2fb764a7404287819251978fbfe4cacf9b4d8a1d41fce5dd97
Successfully built pyyaml termcolor wrapt
Installing collected packages: termcolor, numpy, six, keras-preprocessing, protobuf, scipy, h5py, grpcio, tensorflow-estimator, gast, wrapt, astunparse, absl-py, google-pasta, opt-einsum, zipp, importlib-metadata, markdown, urllib3, idna, chardet, certifi, requests, oauthlib, requests-oauthlib, cachetools, pyasn1, pyasn1-modules, rsa, google-auth, google-auth-oauthlib, tensorboard-plugin-wit, werkzeug, tensorboard, tensorflow, pyyaml, keras, tensorflow-hub, Pillow
  Attempting uninstall: six
    Found existing installation: six 1.10.0
    Not uninstalling six at /usr/lib/python3/dist-packages, outside environment /cloud/project/myenv
    Can't uninstall 'six'. No files were found to uninstall.
  Attempting uninstall: chardet
    Found existing installation: chardet 2.3.0
    Not uninstalling chardet at /usr/lib/python3/dist-packages, outside environment /cloud/project/myenv
    Can't uninstall 'chardet'. No files were found to uninstall.
Successfully installed Pillow-7.2.0 absl-py-0.10.0 astunparse-1.6.3 cachetools-4.1.1 certifi-2020.6.20 chardet-3.0.4 gast-0.3.3 google-auth-1.20.1 google-auth-oauthlib-0.4.1 google-pasta-0.2.0 grpcio-1.31.0 h5py-2.10.0 idna-2.10 importlib-metadata-1.7.0 keras-2.4.3 keras-preprocessing-1.1.2 markdown-3.2.2 numpy-1.18.5 oauthlib-3.1.0 opt-einsum-3.3.0 protobuf-3.13.0 pyasn1-0.4.8 pyasn1-modules-0.2.8 pyyaml-3.12 requests-2.24.0 requests-oauthlib-1.3.0 rsa-4.6 scipy-1.4.1 six-1.15.0 tensorboard-2.2.2 tensorboard-plugin-wit-1.7.0 tensorflow-2.2.0 tensorflow-estimator-2.2.0 tensorflow-hub-0.9.0 termcolor-1.1.0 urllib3-1.25.10 werkzeug-1.0.1 wrapt-1.12.1 zipp-1.2.0

Installation complete.


Restarting R session...

But when cames to use Keras

> library(keras)
> mnist <- dataset_mnist()
No non-system installation of Python could be found.
Would you like to download and install Miniconda?
Miniconda is an open source environment management system for Python.
See https://docs.conda.io/en/latest/miniconda.html for more details.

Would you like to install Miniconda? [Y/n]:y

After install Miniconda

* Downloading 'https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh' ...
trying URL 'https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh'
Content type 'application/x-sh' length 93052469 bytes (88.7 MB)
==================================================
downloaded 88.7 MB

* Installing Miniconda -- please wait a moment ...
PREFIX=/home/rstudio-user/.local/share/r-miniconda
Unpacking payload ...
Collecting package metadata (current_repodata.json): ...working... done                                        
Solving environment: ...working... done

## Package Plan ##

  environment location: /home/rstudio-user/.local/share/r-miniconda

  added / updated specs:
    - _libgcc_mutex==0.1=main
    - ca-certificates==2020.1.1=0
    - certifi==2020.4.5.1=py38_0
    - cffi==1.14.0=py38he30daa8_1
    - chardet==3.0.4=py38_1003
    - conda-package-handling==1.6.1=py38h7b6447c_0
    - conda==4.8.3=py38_0
    - cryptography==2.9.2=py38h1ba5d50_0
    - idna==2.9=py_1
    - ld_impl_linux-64==2.33.1=h53a641e_7
    - libedit==3.1.20181209=hc058e9b_0
    - libffi==3.3=he6710b0_1
    - libgcc-ng==9.1.0=hdf63c60_0
    - libstdcxx-ng==9.1.0=hdf63c60_0
    - ncurses==6.2=he6710b0_1
    - openssl==1.1.1g=h7b6447c_0
    - pip==20.0.2=py38_3
    - pycosat==0.6.3=py38h7b6447c_1
    - pycparser==2.20=py_0
    - pyopenssl==19.1.0=py38_0
    - pysocks==1.7.1=py38_0
    - python==3.8.3=hcff3b4d_0
    - readline==8.0=h7b6447c_0
    - requests==2.23.0=py38_0
    - ruamel_yaml==0.15.87=py38h7b6447c_0
    - setuptools==46.4.0=py38_0
    - six==1.14.0=py38_0
    - sqlite==3.31.1=h62c20be_1
    - tk==8.6.8=hbc83047_0
    - tqdm==4.46.0=py_0
    - urllib3==1.25.8=py38_0
    - wheel==0.34.2=py38_0
    - xz==5.2.5=h7b6447c_0
    - yaml==0.1.7=had09818_2
    - zlib==1.2.11=h7b6447c_3


The following NEW packages will be INSTALLED:

  _libgcc_mutex      pkgs/main/linux-64::_libgcc_mutex-0.1-main
  ca-certificates    pkgs/main/linux-64::ca-certificates-2020.1.1-0
  certifi            pkgs/main/linux-64::certifi-2020.4.5.1-py38_0
  cffi               pkgs/main/linux-64::cffi-1.14.0-py38he30daa8_1
  chardet            pkgs/main/linux-64::chardet-3.0.4-py38_1003
  conda              pkgs/main/linux-64::conda-4.8.3-py38_0
  conda-package-han~ pkgs/main/linux-64::conda-package-handling-1.6.1-py38h7b6447c_0
  cryptography       pkgs/main/linux-64::cryptography-2.9.2-py38h1ba5d50_0
  idna               pkgs/main/noarch::idna-2.9-py_1
  ld_impl_linux-64   pkgs/main/linux-64::ld_impl_linux-64-2.33.1-h53a641e_7
  libedit            pkgs/main/linux-64::libedit-3.1.20181209-hc058e9b_0
  libffi             pkgs/main/linux-64::libffi-3.3-he6710b0_1
  libgcc-ng          pkgs/main/linux-64::libgcc-ng-9.1.0-hdf63c60_0
  libstdcxx-ng       pkgs/main/linux-64::libstdcxx-ng-9.1.0-hdf63c60_0
  ncurses            pkgs/main/linux-64::ncurses-6.2-he6710b0_1
  openssl            pkgs/main/linux-64::openssl-1.1.1g-h7b6447c_0
  pip                pkgs/main/linux-64::pip-20.0.2-py38_3
  pycosat            pkgs/main/linux-64::pycosat-0.6.3-py38h7b6447c_1
  pycparser          pkgs/main/noarch::pycparser-2.20-py_0
  pyopenssl          pkgs/main/linux-64::pyopenssl-19.1.0-py38_0
  pysocks            pkgs/main/linux-64::pysocks-1.7.1-py38_0
  python             pkgs/main/linux-64::python-3.8.3-hcff3b4d_0
  readline           pkgs/main/linux-64::readline-8.0-h7b6447c_0
  requests           pkgs/main/linux-64::requests-2.23.0-py38_0
  ruamel_yaml        pkgs/main/linux-64::ruamel_yaml-0.15.87-py38h7b6447c_0
  setuptools         pkgs/main/linux-64::setuptools-46.4.0-py38_0
  six                pkgs/main/linux-64::six-1.14.0-py38_0
  sqlite             pkgs/main/linux-64::sqlite-3.31.1-h62c20be_1
  tk                 pkgs/main/linux-64::tk-8.6.8-hbc83047_0
  tqdm               pkgs/main/noarch::tqdm-4.46.0-py_0
  urllib3            pkgs/main/linux-64::urllib3-1.25.8-py38_0
  wheel              pkgs/main/linux-64::wheel-0.34.2-py38_0
  xz                 pkgs/main/linux-64::xz-5.2.5-h7b6447c_0
  yaml               pkgs/main/linux-64::yaml-0.1.7-had09818_2
  zlib               pkgs/main/linux-64::zlib-1.2.11-h7b6447c_3


Preparing transaction: ...working... done
Executing transaction: ...working... done
installation finished.
Collecting package metadata (current_repodata.json): ...working... done
Solving environment: ...working... done

## Package Plan ##

  environment location: /home/rstudio-user/.local/share/r-miniconda

  added / updated specs:
    - conda


The following packages will be downloaded:

    package                    |            build
    ---------------------------|-----------------
    brotlipy-0.7.0             |py38h7b6447c_1000         322 KB
    ca-certificates-2020.6.24  |                0         125 KB
    certifi-2020.6.20          |           py38_0         156 KB
    cffi-1.14.1                |   py38he30daa8_0         228 KB
    conda-4.8.4                |           py38_0         2.8 MB
    idna-2.10                  |             py_0          50 KB
    libedit-3.1.20191231       |       h14c3975_1         116 KB
    libffi-3.3                 |       he6710b0_2          50 KB
    pip-20.2.2                 |           py38_0         1.7 MB
    pycparser-2.20             |             py_2          94 KB
    pyopenssl-19.1.0           |             py_1          48 KB
    requests-2.24.0            |             py_0          56 KB
    ruamel_yaml-0.15.87        |   py38h7b6447c_1         249 KB
    setuptools-49.6.0          |           py38_0         748 KB
    six-1.15.0                 |             py_0          13 KB
    sqlite-3.33.0              |       h62c20be_0         1.1 MB
    tk-8.6.10                  |       hbc83047_0         3.0 MB
    tqdm-4.48.2                |             py_0          63 KB
    urllib3-1.25.10            |             py_0          98 KB
    yaml-0.2.5                 |       h7b6447c_0          75 KB
    ------------------------------------------------------------
                                           Total:        11.1 MB

The following NEW packages will be INSTALLED:

  brotlipy           pkgs/main/linux-64::brotlipy-0.7.0-py38h7b6447c_1000

The following packages will be UPDATED:

  ca-certificates                                2020.1.1-0 --> 2020.6.24-0
  certifi                                 2020.4.5.1-py38_0 --> 2020.6.20-py38_0
  cffi                                1.14.0-py38he30daa8_1 --> 1.14.1-py38he30daa8_0
  conda                                        4.8.3-py38_0 --> 4.8.4-py38_0
  idna                                             2.9-py_1 --> 2.10-py_0
  libedit                           3.1.20181209-hc058e9b_0 --> 3.1.20191231-h14c3975_1
  libffi                                     3.3-he6710b0_1 --> 3.3-he6710b0_2
  pip                                         20.0.2-py38_3 --> 20.2.2-py38_0
  pycparser                                       2.20-py_0 --> 2.20-py_2
  pyopenssl          pkgs/main/linux-64::pyopenssl-19.1.0-~ --> pkgs/main/noarch::pyopenssl-19.1.0-py_1
  requests           pkgs/main/linux-64::requests-2.23.0-p~ --> pkgs/main/noarch::requests-2.24.0-py_0
  ruamel_yaml                        0.15.87-py38h7b6447c_0 --> 0.15.87-py38h7b6447c_1
  setuptools                                  46.4.0-py38_0 --> 49.6.0-py38_0
  six                 pkgs/main/linux-64::six-1.14.0-py38_0 --> pkgs/main/noarch::six-1.15.0-py_0
  sqlite                                  3.31.1-h62c20be_1 --> 3.33.0-h62c20be_0
  tk                                       8.6.8-hbc83047_0 --> 8.6.10-hbc83047_0
  tqdm                                          4.46.0-py_0 --> 4.48.2-py_0
  urllib3            pkgs/main/linux-64::urllib3-1.25.8-py~ --> pkgs/main/noarch::urllib3-1.25.10-py_0
  yaml                                     0.1.7-had09818_2 --> 0.2.5-h7b6447c_0



Downloading and Extracting Packages
cffi-1.14.1          | 228 KB    | ########## | 100% 
ca-certificates-2020 | 125 KB    | ########## | 100% 
libffi-3.3           | 50 KB     | ########## | 100% 
six-1.15.0           | 13 KB     | ########## | 100% 
tk-8.6.10            | 3.0 MB    | ########## | 100% 
certifi-2020.6.20    | 156 KB    | ########## | 100% 
ruamel_yaml-0.15.87  | 249 KB    | ########## | 100% 
tqdm-4.48.2          | 63 KB     | ########## | 100% 
requests-2.24.0      | 56 KB     | ########## | 100% 
brotlipy-0.7.0       | 322 KB    | ########## | 100% 
pyopenssl-19.1.0     | 48 KB     | ########## | 100% 
libedit-3.1.20191231 | 116 KB    | ########## | 100% 
idna-2.10            | 50 KB     | ########## | 100% 
setuptools-49.6.0    | 748 KB    | ########## | 100% 
conda-4.8.4          | 2.8 MB    | ########## | 100% 
sqlite-3.33.0        | 1.1 MB    | ########## | 100% 
pip-20.2.2           | 1.7 MB    | ########## | 100% 
urllib3-1.25.10      | 98 KB     | ########## | 100% 
yaml-0.2.5           | 75 KB     | ########## | 100% 
pycparser-2.20       | 94 KB     | ########## | 100% 
Preparing transaction: ...working... done
Verifying transaction: ...working... done
Executing transaction: ...working... done
Collecting package metadata (current_repodata.json): ...working... done
Solving environment: ...working... done

## Package Plan ##

  environment location: /home/rstudio-user/.local/share/r-miniconda/envs/r-reticulate

  added / updated specs:
    - numpy
    - python=3.6


The following packages will be downloaded:

    package                    |            build
    ---------------------------|-----------------
    _libgcc_mutex-0.1          |      conda_forge           3 KB  conda-forge
    _openmp_mutex-4.5          |            1_gnu          22 KB  conda-forge
    ca-certificates-2020.6.20  |       hecda079_0         145 KB  conda-forge
    certifi-2020.6.20          |   py36h9f0ad1d_0         151 KB  conda-forge
    ld_impl_linux-64-2.34      |       hc38a660_9         612 KB  conda-forge
    libblas-3.8.0              |      17_openblas          11 KB  conda-forge
    libcblas-3.8.0             |      17_openblas          11 KB  conda-forge
    libffi-3.2.1               |    he1b5a44_1007          47 KB  conda-forge
    libgcc-ng-9.3.0            |      h24d8f2e_15         7.8 MB  conda-forge
    libgfortran-ng-7.5.0       |      hdf63c60_15         1.3 MB  conda-forge
    libgomp-9.3.0              |      h24d8f2e_15         378 KB  conda-forge
    liblapack-3.8.0            |      17_openblas          11 KB  conda-forge
    libopenblas-0.3.10         |pthreads_hb3c22a3_4         7.8 MB  conda-forge
    libstdcxx-ng-9.3.0         |      hdf63c60_15         4.0 MB  conda-forge
    ncurses-6.2                |       he1b5a44_1         993 KB  conda-forge
    numpy-1.19.1               |   py36h3849536_2         5.2 MB  conda-forge
    openssl-1.1.1g             |       h516909a_1         2.1 MB  conda-forge
    pip-20.2.2                 |             py_0         1.1 MB  conda-forge
    python-3.6.11              |h4d41432_2_cpython        34.2 MB  conda-forge
    python_abi-3.6             |          1_cp36m           4 KB  conda-forge
    readline-8.0               |       he28a2e2_2         281 KB  conda-forge
    setuptools-49.6.0          |   py36h9f0ad1d_0         935 KB  conda-forge
    sqlite-3.33.0              |       h4cf870e_0         1.4 MB  conda-forge
    tk-8.6.10                  |       hed695b0_0         3.2 MB  conda-forge
    wheel-0.35.1               |     pyh9f0ad1d_0          29 KB  conda-forge
    xz-5.2.5                   |       h516909a_1         343 KB  conda-forge
    zlib-1.2.11                |    h516909a_1007         106 KB  conda-forge
    ------------------------------------------------------------
                                           Total:        72.1 MB

The following NEW packages will be INSTALLED:

  _libgcc_mutex      conda-forge/linux-64::_libgcc_mutex-0.1-conda_forge
  _openmp_mutex      conda-forge/linux-64::_openmp_mutex-4.5-1_gnu
  ca-certificates    conda-forge/linux-64::ca-certificates-2020.6.20-hecda079_0
  certifi            conda-forge/linux-64::certifi-2020.6.20-py36h9f0ad1d_0
  ld_impl_linux-64   conda-forge/linux-64::ld_impl_linux-64-2.34-hc38a660_9
  libblas            conda-forge/linux-64::libblas-3.8.0-17_openblas
  libcblas           conda-forge/linux-64::libcblas-3.8.0-17_openblas
  libffi             conda-forge/linux-64::libffi-3.2.1-he1b5a44_1007
  libgcc-ng          conda-forge/linux-64::libgcc-ng-9.3.0-h24d8f2e_15
  libgfortran-ng     conda-forge/linux-64::libgfortran-ng-7.5.0-hdf63c60_15
  libgomp            conda-forge/linux-64::libgomp-9.3.0-h24d8f2e_15
  liblapack          conda-forge/linux-64::liblapack-3.8.0-17_openblas
  libopenblas        conda-forge/linux-64::libopenblas-0.3.10-pthreads_hb3c22a3_4
  libstdcxx-ng       conda-forge/linux-64::libstdcxx-ng-9.3.0-hdf63c60_15
  ncurses            conda-forge/linux-64::ncurses-6.2-he1b5a44_1
  numpy              conda-forge/linux-64::numpy-1.19.1-py36h3849536_2
  openssl            conda-forge/linux-64::openssl-1.1.1g-h516909a_1
  pip                conda-forge/noarch::pip-20.2.2-py_0
  python             conda-forge/linux-64::python-3.6.11-h4d41432_2_cpython
  python_abi         conda-forge/linux-64::python_abi-3.6-1_cp36m
  readline           conda-forge/linux-64::readline-8.0-he28a2e2_2
  setuptools         conda-forge/linux-64::setuptools-49.6.0-py36h9f0ad1d_0
  sqlite             conda-forge/linux-64::sqlite-3.33.0-h4cf870e_0
  tk                 conda-forge/linux-64::tk-8.6.10-hed695b0_0
  wheel              conda-forge/noarch::wheel-0.35.1-pyh9f0ad1d_0
  xz                 conda-forge/linux-64::xz-5.2.5-h516909a_1
  zlib               conda-forge/linux-64::zlib-1.2.11-h516909a_1007



Downloading and Extracting Packages
liblapack-3.8.0      | 11 KB     | ########## | 100% 
setuptools-49.6.0    | 935 KB    | ########## | 100% 
sqlite-3.33.0        | 1.4 MB    | ########## | 100% 
_libgcc_mutex-0.1    | 3 KB      | ########## | 100% 
libstdcxx-ng-9.3.0   | 4.0 MB    | ########## | 100% 
ca-certificates-2020 | 145 KB    | ########## | 100% 
openssl-1.1.1g       | 2.1 MB    | ########## | 100% 
certifi-2020.6.20    | 151 KB    | ########## | 100% 
zlib-1.2.11          | 106 KB    | ########## | 100% 
libblas-3.8.0        | 11 KB     | ########## | 100% 
python_abi-3.6       | 4 KB      | ########## | 100% 
libffi-3.2.1         | 47 KB     | ########## | 100% 
ld_impl_linux-64-2.3 | 612 KB    | ########## | 100% 
xz-5.2.5             | 343 KB    | ########## | 100% 
libgfortran-ng-7.5.0 | 1.3 MB    | ########## | 100% 
libopenblas-0.3.10   | 7.8 MB    | ########## | 100% 
readline-8.0         | 281 KB    | ########## | 100% 
_openmp_mutex-4.5    | 22 KB     | ########## | 100% 
pip-20.2.2           | 1.1 MB    | ########## | 100% 
ncurses-6.2          | 993 KB    | ########## | 100% 
libcblas-3.8.0       | 11 KB     | ########## | 100% 
tk-8.6.10            | 3.2 MB    | ########## | 100% 
numpy-1.19.1         | 5.2 MB    | ########## | 100% 
wheel-0.35.1         | 29 KB     | ########## | 100% 
python-3.6.11        | 34.2 MB   | ########## | 100% 
libgcc-ng-9.3.0      | 7.8 MB    | ########## | 100% 
libgomp-9.3.0        | 378 KB    | ########## | 100% 
Preparing transaction: ...working... done
Verifying transaction: ...working... done
Executing transaction: ...working... done
#
# To activate this environment, use
#
#     $ conda activate r-reticulate
#
# To deactivate an active environment, use
#
#     $ conda deactivate

* Miniconda has been successfully installed at '/home/rstudio-user/.local/share/r-miniconda'.
Error: Installation of TensorFlow not found.

Python environments searched for 'tensorflow' package:
 /home/rstudio-user/.local/share/r-miniconda/envs/r-reticulate/bin/python3.6

You can install TensorFlow using the install_tensorflow() function.
> mnist <- dataset_mnist()
Error: Python module tensorflow.keras was not found.

Detected Python configuration:

python:         /home/rstudio-user/.local/share/r-miniconda/envs/r-reticulate/bin/python
libpython:      /home/rstudio-user/.local/share/r-miniconda/envs/r-reticulate/lib/libpython3.6m.so
pythonhome:     /home/rstudio-user/.local/share/r-miniconda/envs/r-reticulate:/home/rstudio-user/.local/share/r-miniconda/envs/r-reticulate
version:        3.6.11 | packaged by conda-forge | (default, Aug  5 2020, 20:09:42)  [GCC 7.5.0]
numpy:          /home/rstudio-user/.local/share/r-miniconda/envs/r-reticulate/lib/python3.6/site-packages/numpy
numpy_version:  1.19.1

An errors occur : Python module tensorflow.keras was not found. the same as before

15

Hi Ziqian_Xia,

I believe I was able to get this to work with the following steps: Starting with a new project using 2GB memory and 1.5 CPU, I used these steps from Sam's workaround:

library(reticulate)
library(keras)
virtualenv_create("myenv")
use_virtualenv("myenv")
install_keras(method="virtualenv", envname="myenv")

Now after you see the message "Restarting R session..." but before you try to use Keras, run use_virtualenv("myenv") a second time. I believe the virtualenv was deactivated when the R session was restarted.

After I ran these steps I was able to do:

library(keras)
mnist <- dataset_mnist()
x_train <- mnist$train$x
y_train <- mnist$train$y
x_test <- mnist$test$x
etc

I was not prompted to install miniconda and I can use the mnist dataset.

I hope that works for you.

Best,
Ming

Edit: here is my full console history starting from the new project

install.packages('reticulate')
install.packages('keras')
library(reticulate)
library(keras)
virtualenv_create("myenv")
use_virtualenv("myenv")
install_keras(method="virtualenv", envname="myenv")
use_virtualenv("myenv")
library(keras)
mnist <- dataset_mnist()
x_train <- mnist$train$x
16

Hi mingb , thanks for your kind help I opened a new project and this code works great!

17

I would also strongly recommend using:

use_virtualenv("myenv", required = TRUE)

as RStudio may ignore a use_virtualenv() request if the Python session has already been initialized, or if the requested environment doesn't satisfy other declared constraints (most common being NumPy not being available).

18

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.