Using tensorflow with GPU on Windows

Machine Learning and Modeling
, ,
1

I am a biologist and not a programmer/software developer. I read the book "Deep Learning with R (second Edition)" and try to develop a keras model.
I bought a new fancy computer with a NVIDIA RTX 4070 and Windows 11. I have tried for days to find an "easy" solution to run in R keras and tensorflow on my GPU. Apparently tensorflow > 2.10 is not supporting GPU on native Windows? Apparently there is no (easy) solution to this. I installed CUDA and Cudnn, also Anaconda and VisualStudio and C++ compiler.

I am aware, that you have to check for the correct versions of CUDA and cudnn (which I did for the newest tensorflow version before I realised that this doesn't support Windows anymore, however that seems unnecessary anyway when the newest tensorflow versions don't support Windows?

I was able work tensorflow on GPU when using python, but not in my RStudio environment.

I tried following tutorials:
Tensorflow Install Guide 'tensorflow/install/gpu_plugins'

Using RStudio Server in Windows WSL2 – Posit Support

libraries - How to solve library dependancy for Rstudio in Ubuntu 22.04 - Ask Ubuntu 'askubuntu/questions/1404233'

DirectML Plugin for TensorFlow 2 | Microsoft Learn

Most of them are not suitable for RStudio or just don't work, as links are outdated. Furthermore, it seems that RStudio Server is not provided for Ubuntu, I get for example these errors:

'libraries - How to solve library dependancy for Rstudio in Ubuntu 22.04 - Ask Ubuntu' : 'askubuntu/questions/1404233

'python - ImportError: libGL.so.1: cannot open shared object file: No such file or directory - Stack Overflow' : 'stackoverflow/questions/55313610

It seems to me, the easiest solution would be to install an older version of tensorflow in RStudio. However, the oldest version is 2.12 or something, the install_tensorflow() function wouldn't let me install anything older than that.

Google Search, Stackoverflow and youtube, no results. There are some similar questions on StackOverflow and here on PositCommunity, but they seemed not to be answered or relate to older versions e.g. Windows 10, tensorflow 1.xx etc.

Machine learning is on the rise, and Windows is the most used OS worldwide. How can it be, that there is no support at all? Overall, I think RStudio is doing great for their IDE and linking python and R. But for this special case, it seems redicilously complicated.

So my question(s): How do I set up Tensorflow and Keras for Windows and RStudio so it uses the GPU? Any tutorial (for dummies)? And why is it so overlyc complicated/not supported?

Many thanks for all your answers, maybe some other non-developers may find this helpful, and there might be more support for the future?

2

If you want to use a GPU on Windows with TensorFlow, WSL is the way to do it. (The official builds of TensorFlow no longer provide GPU support on Windows).

This unfortunately makes the learning curve a little bit steeper, since you effectively have to learn how to work in a Linux environment, in addition to everything related to deep learning.

You can continue to use the RStudio IDE while working in WSL through this guide: https://support.posit.co/hc/en-us/articles/360049776974-Using-RStudio-Server-in-Windows-WSL2

Once you have R (and optionally, RStudio) installed in the WSL Linux image, then installing tensorflow should be a simple tensorflow::install_tensorflow() call in R.

Note that beginning with TF Version 2.14, installation of CUDA is substantially simplified. For tensorflow to use the GPU, all that needs to be preinstalled to use is a GPU driver (most likely already provided as part of the Ubuntu image). Everything else (CUDA, cuDNN, etc.) is provided by install_tensorflow().

If you run into specific issues, please post an issue at GitHub - rstudio/tensorflow: TensorFlow for R.

3

It finally worked for me.
Here is what I did:
I deleted the Windows Subsystem for Linux and Ubuntu completely, downloaded Ubuntu 22 and the newest susbsystem update.
Then I used

wsl --shutdown
wslconfig /unregister Ubuntu
wsl --install --distribution Ubuntu

Following outdated tutorials might have added wrong entries into the directory [=linux - Is it impossible to install R 4.0 on Ubuntu 18.04.4 LTS because r-base-core requires a libc6 version >= 2.29? - Stack Overflow stackoverflow questions/61586021/is-it-impossible-to-install-r-4-0-on-ubuntu-18-04-4-lts-because-r-base-core-requ

Afther that, I followed
RStudio Server - Posit
whereby one has to be carefull to click on
Ubuntu Packages For R - Brief Instructions (rstudio.com)

So indeed, the problem here was the uncertainty and not existing knowledge of Linux, which hindered the success.
Now, I run into more problems, I can't install any packages, it says for example

Warning in install.packages :
  installation of package ‘rlang’ had non-zero exit status
ERROR: dependency ‘sys’ is not available for package ‘askpass’
* removing ‘/home/tobias/R/x86_64-pc-linux-gnu-library/4.3/askpass’
Warning in install.packages :
  installation of package ‘askpass’ had non-zero exit status
ERROR: dependencies ‘Rcpp’, ‘rlang’ are not available for package ‘later’
* removing ‘/home/tobias/R/x86_64-pc-linux-gnu-library/4.3/later’

But I can ask that in a seperate topic if thats more appropriate to the moderator?
Many thanks

4

Ok, here are the specific problems I am running in at the moment, following two different tutorials:

Trying Using RStudio Server in Windows WSL2 – Posit Support

sudo apt install -y r-base r-base-core r-recommended r-base-dev gdebi-core build-essential libcurl4-gnutls-dev libxml2-dev libssl-dev
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
gdebi-core is already the newest version (0.9.5.7+nmu6).
libssl-dev is already the newest version (3.0.2-0ubuntu1.12).
libssl-dev set to manually installed.
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:

The following packages have unmet dependencies:
 r-base-core : Depends: libicu66 (>= 66.1-1~) but it is not installable
E: Unable to correct problems, you have held broken packages.

Or

wget https://rstudio.org/download/latest/stable/server/bionic/rstudio-server-latest-amd64.deb
udo gdebi rstudio-server-latest-amd64.deb--2023-11-01 16:34:13--  https://rstudio.org/download/latest/stable/server/bionic/rstudio-server-latest-amd64.deb
Resolving rstudio.org (rstudio.org)... 45.33.2.142
Connecting to rstudio.org (rstudio.org)|45.33.2.142|:443... connected.
HTTP request sent, awaiting response... 404 Not Found
2023-11-01 16:34:14 ERROR 404: Not Found.

Trying:
RStudio Server - Posit

wget https://download2.rstudio.org/server/focal/amd64/rstudio-server-2023.09.1-494-amd64.deb
--2023-11-01 16:35:57--  https://download2.rstudio.org/server/focal/amd64/rstudio-server-2023.09.1-494-amd64.deb
Resolving download2.rstudio.org (download2.rstudio.org)... 108.138.26.72, 108.138.26.56, 108.138.26.113, ...
Connecting to download2.rstudio.org (download2.rstudio.org)|108.138.26.72|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 101518324 (97M) [application/octet-stream]
rstudio-server-2023.09.1-494-amd64.deb.3: Permission denied

Cannot write to ‘rstudio-server-2023.09.1-494-amd64.deb.3’ (Success).

Despite downloading the RStudio Server Debian package, I faced persistent issues related to unmet dependencies, particularly with the 'libssl' package. Here is a summary of the problem:

  1. Initially, I attempted to install RStudio Server using the dpkg command:

    sudo dpkg -i rstudio-server-2023.09.1-494-amd64.deb

    However, the installation process failed due to unmet dependencies for the 'libssl' package.

  2. I tried to resolve the issue by installing the 'libssl' package using the following command:

    sudo apt-get install libssl1.1

    Unfortunately, the 'libssl1.1' package was not available in the current repositories, leading to further complications.

  3. I attempted to find the required 'libssl' package version (e.g., 'libssl1.0.2') through the package manager, but it was also not available:

    sudo apt-get install libssl1.0.2

    This further hindered the installation process, leaving me unable to proceed.

I would appreciate any guidance or advice from the community on how to resolve these dependency issues and successfully install RStudio Server on my system. If anyone has encountered similar problems or has insights into alternative solutions, please feel free to share your expertise.

Thank you in advance for your assistance.

5

It is difficult to diagnose from afar what is going on. The fact that apt install r-base-core fails is unrelated to anything with TensorFlow or GPUs, this is a more general Linux issue (see my earlier comment about the steeper learning curve... knowing a little about how to manage a Linux machine is unfortunately the price of entry for this type of work.)

The failure from apt about "broken packages" is often very frustrating and difficult to get out of. This most often happens from installing conflicting packages (that is, previous apt commands have left you in a broken state). It is certainly not what should happen when following the instructions on a fresh machine.

A few incantations of apt purge <offending-package> is typically the most accessible way to get back to a working state. Occasionally reaching for dpkg -r <pkg> or even dpkg -P <pkg> (run dpkg -l to see package names). If you're on WSL and you don't have much invested in the current Linux image, sometimes deleting the entire WSL linux image and starting "fresh" might be easiest.

This indicates a permissions issue - likely the rstudio-server.deb file was first created with sudo, and now as a standard user, you can no longer modify (overwrite) it. chmod, chown, or simply sudo rm, are some commands that you can use to get back to a working state.

6

This was the solution to the other problem:
Installing packages as normal didn't work. e.g. over tools -> install packages
or

install.packages("dplyr")

But

sudo apt-get install r-cran-dplyr

does the job. Alll these little things and tricks were not mentioned during the normal tutorials on how to install RStudio server etc.
Thanks

7

This does work with RStudio Server, so I would suggest that you have a problem with your installation.

You could check e.g. install.packages("dplyr") from within R (i.e. without running RStudio Server) to see whether that is successful.

8

This topic was automatically closed 21 days after the last reply. New replies are no longer allowed.

If you have a query related to it or one of the replies, start a new topic and refer back with a link.