install.packages() does not install dependencies on Ubuntu Linux. Very bad problem.

Dear all,

I am running RStudio under Ubuntu Linux version 20.04.

I have the longstanding problem that the install.packages() command will not install or update dependencies. This means I have to spend hours, sometimes six or seven hours, installing packages and all of their dependencies, and all of their dependencies' dependencies, and so on.

I've tried using the depdencies = TRUE flag, and it is ignored. I've tried using also using the lib = library_path flag and that also makes no difference.

I'm wondering if the problem is that I have multiple libraries in .libPaths(). I read somewhere that this can prevent dependencies from being installed, because R doesn't know where you want to put them. Could this be the problem?

I don't want multiple libraries in .libPaths(). I'd like to just have one.

Does anyone have any advice? I've had this problem for years, and it's un-google'able.

I'm most interested in specific code (in R or in the terminal window) that can get the job done and fix my problem, rather than general suggestions. My coding and unix shell skills are good but not as expert as many of you.

Thanks so much for your assistance!

Sincerely,
J. B.

I have not seen this problem here before and unless someone has run across it, it will be hard to make specific suggestions without more information. Having multiple libraries on .libPaths() is normal, so I can't see how that would cause this problem.

I suggest you go to the CRAN list of packages (CRAN Packages By Name) and pick one you don't have and that has at least one dependency that you don't have. The abc package looked like a decent candidate, but you may see something better. Start a new R session and run

sessionInfo()
.libPaths()
install.packages("Package_you_picked")

and then post the output of all that here. Please put a line with three back ticks just before and after the pasted output so it is formatted as code.

Dear FC, please below. I had to replace links with xxxx:// because I'm a new user and not allowed to post links.

> sessionInfo()
R version 4.3.1 (2023-06-16)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 20.04.4 LTS

Matrix products: default
BLAS:   /usr/lib/x86_64-linux-gnu/blas/libblas.so.3.9.0 
LAPACK: /usr/lib/x86_64-linux-gnu/lapack/liblapack.so.3.9.0

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C               LC_TIME=en_US.UTF-8       
 [4] LC_COLLATE=en_US.UTF-8     LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8   
 [7] LC_PAPER=en_US.UTF-8       LC_NAME=C                  LC_ADDRESS=C              
[10] LC_TELEPHONE=C             LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       

time zone: America/Chicago
tzcode source: system (glibc)

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

loaded via a namespace (and not attached):
[1] compiler_4.3.1 tools_4.3.1   
> .libPaths()
[1] "/home/sample/R/x86_64-pc-linux-gnu-library/4.3" "/usr/local/lib/R/site-library"                 
[3] "/usr/lib/R/site-library"                        "/usr/lib/R/library"                            
> install.packages("tidyverse", dependencies = TRUE)
Installing package into ‘/home/sample/R/x86_64-pc-linux-gnu-library/4.3’
(as ‘lib’ is unspecified)
also installing the dependencies ‘tidyselect’, ‘gargle’, ‘timechange’, ‘vroom’, ‘broom’, ‘conflicted’, ‘dbplyr’, ‘dplyr’, ‘dtplyr’, ‘forcats’, ‘googledrive’, ‘googlesheets4’, ‘haven’, ‘hms’, ‘lubridate’, ‘modelr’, ‘readr’, ‘readxl’, ‘reprex’, ‘rstudioapi’, ‘tidyr’, ‘covr’, ‘mockr’, ‘rmarkdown’

trying URL 'xxxx://cloud.r-project.org/src/contrib/tidyselect_1.2.0.tar.gz'
Content type 'application/x-gzip' length 101498 bytes (99 KB)
==================================================
downloaded 99 KB

trying URL 'xxxx://cloud.r-project.org/src/contrib/gargle_1.5.2.tar.gz'
Content type 'application/x-gzip' length 627560 bytes (612 KB)
==================================================
downloaded 612 KB

trying URL 'xxxx://cloud.r-project.org/src/contrib/timechange_0.2.0.tar.gz'
Content type 'application/x-gzip' length 102806 bytes (100 KB)
==================================================
downloaded 100 KB

trying URL 'xxxx://cloud.r-project.org/src/contrib/vroom_1.6.4.tar.gz'
Content type 'application/x-gzip' length 750198 bytes (732 KB)
==================================================
downloaded 732 KB

trying URL 'xxxx://cloud.r-project.org/src/contrib/broom_1.0.5.tar.gz'
Content type 'application/x-gzip' length 641849 bytes (626 KB)
==================================================
downloaded 626 KB

trying URL 'xxxx://cloud.r-project.org/src/contrib/conflicted_1.2.0.tar.gz'
Content type 'application/x-gzip' length 17071 bytes (16 KB)
==================================================
downloaded 16 KB

trying URL 'xxxx://cloud.r-project.org/src/contrib/dbplyr_2.3.4.tar.gz'
Content type 'application/x-gzip' length 723021 bytes (706 KB)
==================================================
downloaded 706 KB

trying URL 'xxxx://cloud.r-project.org/src/contrib/dplyr_1.1.3.tar.gz'
Content type 'application/x-gzip' length 1083635 bytes (1.0 MB)
==================================================
downloaded 1.0 MB

trying URL 'xxxx://cloud.r-project.org/src/contrib/dtplyr_1.3.1.tar.gz'
Content type 'application/x-gzip' length 150971 bytes (147 KB)
==================================================
downloaded 147 KB

trying URL 'xxxx://cloud.r-project.org/src/contrib/forcats_1.0.0.tar.gz'
Content type 'application/x-gzip' length 294183 bytes (287 KB)
==================================================
downloaded 287 KB

trying URL 'xxxx://cloud.r-project.org/src/contrib/googledrive_2.1.1.tar.gz'
Content type 'application/x-gzip' length 1609266 bytes (1.5 MB)
==================================================
downloaded 1.5 MB

trying URL 'xxxx://cloud.r-project.org/src/contrib/googlesheets4_1.1.1.tar.gz'
Content type 'application/x-gzip' length 232584 bytes (227 KB)
==================================================
downloaded 227 KB

trying URL 'xxxx://cloud.r-project.org/src/contrib/haven_2.5.3.tar.gz'
Content type 'application/x-gzip' length 313303 bytes (305 KB)
==================================================
downloaded 305 KB

trying URL 'xxxx://cloud.r-project.org/src/contrib/hms_1.1.3.tar.gz'
Content type 'application/x-gzip' length 43379 bytes (42 KB)
==================================================
downloaded 42 KB

trying URL 'xxxx://cloud.r-project.org/src/contrib/lubridate_1.9.3.tar.gz'
Content type 'application/x-gzip' length 428043 bytes (418 KB)
==================================================
downloaded 418 KB

trying URL 'xxxx://cloud.r-project.org/src/contrib/modelr_0.1.11.tar.gz'
Content type 'application/x-gzip' length 121480 bytes (118 KB)
==================================================
downloaded 118 KB

trying URL 'xxxx://cloud.r-project.org/src/contrib/readr_2.1.4.tar.gz'
Content type 'application/x-gzip' length 298083 bytes (291 KB)
==================================================
downloaded 291 KB

trying URL 'xxxx://cloud.r-project.org/src/contrib/readxl_1.4.3.tar.gz'
Content type 'application/x-gzip' length 2093488 bytes (2.0 MB)
==================================================
downloaded 2.0 MB

trying URL 'xxxx://cloud.r-project.org/src/contrib/reprex_2.0.2.tar.gz'
Content type 'application/x-gzip' length 1088076 bytes (1.0 MB)
==================================================
downloaded 1.0 MB

trying URL 'xxxx://cloud.r-project.org/src/contrib/rstudioapi_0.15.0.tar.gz'
Content type 'application/x-gzip' length 115689 bytes (112 KB)
==================================================
downloaded 112 KB

trying URL 'xxxx://cloud.r-project.org/src/contrib/tidyr_1.3.0.tar.gz'
Content type 'application/x-gzip' length 823546 bytes (804 KB)
==================================================
downloaded 804 KB

trying URL 'xxxx://cloud.r-project.org/src/contrib/covr_3.6.3.tar.gz'
Content type 'application/x-gzip' length 163301 bytes (159 KB)
==================================================
downloaded 159 KB

trying URL 'xxxx://cloud.r-project.org/src/contrib/mockr_0.2.1.tar.gz'
Content type 'application/x-gzip' length 20956 bytes (20 KB)
==================================================
downloaded 20 KB

trying URL 'xxxx://cloud.r-project.org/src/contrib/rmarkdown_2.25.tar.gz'
Content type 'application/x-gzip' length 2188934 bytes (2.1 MB)
==================================================
downloaded 2.1 MB

trying URL 'xxxx://cloud.r-project.org/src/contrib/tidyverse_2.0.0.tar.gz'
Content type 'application/x-gzip' length 704618 bytes (688 KB)
==================================================
downloaded 688 KB

* installing *source* package ‘tidyselect’ ...
** package ‘tidyselect’ successfully unpacked and MD5 sums checked
** using staged installation
** R
** inst
** byte-compile and prepare package for lazy loading
Error: package ‘glue’ was installed before R 4.0.0: please re-install it
Execution halted
ERROR: lazy loading failed for package ‘tidyselect’
* removing ‘/home/sample/R/x86_64-pc-linux-gnu-library/4.3/tidyselect’
Warning in install.packages :
  installation of package ‘tidyselect’ had non-zero exit status
* installing *source* package ‘gargle’ ...
** package ‘gargle’ successfully unpacked and MD5 sums checked
** using staged installation
** R
** inst
** byte-compile and prepare package for lazy loading
Error: package ‘fs’ was installed before R 4.0.0: please re-install it
Execution halted
ERROR: lazy loading failed for package ‘gargle’
* removing ‘/home/sample/R/x86_64-pc-linux-gnu-library/4.3/gargle’
Warning in install.packages :
  installation of package ‘gargle’ had non-zero exit status
* installing *source* package ‘timechange’ ...
** package ‘timechange’ successfully unpacked and MD5 sums checked
** using staged installation
** libs
using C compiler: ‘gcc (Ubuntu 9.4.0-1ubuntu1~20.04.2) 9.4.0’
using C++ compiler: ‘g++ (Ubuntu 9.4.0-1ubuntu1~20.04.2) 9.4.0’
using C++11
g++ -std=gnu++11 -I"/usr/share/R/include" -DNDEBUG -I. -I./cctz/src/ -I'/usr/local/lib/R/site-library/cpp11/include'     -fpic  -g -O2 -fdebug-prefix-map=/build/r-base-aeZVAy/r-base-4.3.1=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2  -c common.cpp -o common.o
g++ -std=gnu++11 -I"/usr/share/R/include" -DNDEBUG -I. -I./cctz/src/ -I'/usr/local/lib/R/site-library/cpp11/include'     -fpic  -g -O2 -fdebug-prefix-map=/build/r-base-aeZVAy/r-base-4.3.1=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2  -c cpp11.cpp -o cpp11.o
g++ -std=gnu++11 -I"/usr/share/R/include" -DNDEBUG -I. -I./cctz/src/ -I'/usr/local/lib/R/site-library/cpp11/include'     -fpic  -g -O2 -fdebug-prefix-map=/build/r-base-aeZVAy/r-base-4.3.1=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2  -c get.cpp -o get.o
make: *** [/usr/lib/R/etc/Makeconf:200: get.o] Interrupt
ERROR: compilation failed for package ‘timechange’
* removing ‘/home/sample/R/x86_64-pc-linux-gnu-library/4.3/timechange’

I'm no expert on this but it looks like some of your packages are up to date but were installed under an older version of R and must be reinstalled. What version of glue do you have? The latest is 1.6.2 and it was released 2022-02-24. If you have upgraded from R 3.x to R 4.x since then, you need to reinstall glue. I think R 4.0.0 came out in April 2020. Did you stay on R3.x for a while?

The remove.packages() function needs to know what library to find the package in. My glue is in /usr/local/lib/R/site-library and I'll be curious to know if yours is in the same place.

FJCC,

I think that's right, but the problem is it's not just one package. When I go to install glue, it has dependencies that also are outdated, and I get the same errors that those need to be reinstalled. In the end, I have to manually reinstall around 100 packages one-by-one. It's a horrible mess. Why can't R automatically reinstall those dependencies? Shouldn't dependencies = TRUE take care of it recurrsively?

If it's an issue of R not knowing what library to install glue, then that's why I was trying to delete the extra paths. I just want one place where R looks for everything.

I confess to being especially frustrated, because R does not experience these problems on Mac or PC. When I upgraded to R 4.0 on my Mac, I simply re-installed tidyverse, and R automatically reinstalled any dependencies that were out-of-date. These problems are only experienced in the Linux environment.

Thanks so much for your help!

Best,
J.B.

It's time to root out all traces of the existing R installation and its associated libraries.

to_reinstall <- installed.packages()
write.csv(to_reinstall,"SOMEWHERE SAFE")

After reinstalling R with apt remove then apt install

to_reinstall <- read.csv("SOMEWHERE SAFE")
install.packages(to_reinstall, ask = FALSE)

This will take a while. Do it from the R console, not RStudio (or you'll get the silly "one or more opened ... want new session" prompt that is actually only for the benefit on Windows users).

Alternatively, see the r2u package, which will give you binaries. Very quickly. ALL of them. With system library dependencies as well. All by the guy who has been lead on all things related to Debian on R for over 20 years. Read carefully the instructions at GitHub - eddelbuettel/r2u: CRAN as Ubuntu Binaries and skip the optional step 5. It should pick-up everything in your to_reinstall except for some oddball and stuff not on CRAN. Then you can compare the installed.packages() with the saved list.

I got into your pickle because for a long time apt update lagged badly and I took up installing from source.

1 Like

Thank you very much @technocrat. It works!!! This is The Way.

I adapted your instructions a bit to make it work for my setup. Notes:

Please note: I had to replace https:// links with xxxx:// because I am a new user and therefore not allowed to post links. When you use these instructions, replace xxxx:// with https://

  1. First I did this in R (or Rstudio):
to_reinstall <- installed.packages()
write.csv(to_reinstall,"SOMEWHERE SAFE")
  1. Then I ran this in R (or Rstudio) to find all of the library paths:
.libPaths()
  1. Then I opened a terminal window and deleted each of the libraries:
sudo rm -r PATH-TO-LIBRARY-FOLDER
  1. Then, also from the terminal, I deleted R (and Rstudio for good measure):
sudo apt-get remove r-base-core
sudo apt-get remove r-base
sudo apt-get autoremove
sudo apt-get remove rstudio
  1. Then I re-booted the machine
  2. Then I re-installed R and Rstudio:
    xxxx://youtu.be/eE1zh0YVMeM?si=kRfHJRoLywAkg1hR
  3. Then I installed the r2u package according to the instructions, more or less:
    xxxx://github.com/eddelbuettel/r2u
    Note that the main page has instructions for Ubuntu 22.04. If you want instructions for Ubuntu 20.04, see:
    xxxx://github.com/eddelbuettel/r2u/blob/master/inst/scripts/add_cranapt_focal.sh
  4. Some modifications to those instructions were needed:
    a) Anywhere you see "apt", you must change it to "sudo apt". And I do mean everywhere. Or else you get a fatal error and the whole thing stops. For instance:
apt update -qq && apt install --yes --no-install-recommends wget \
    ca-certificates gnupg

must be:

sudo apt update -qq && sudo apt install --yes --no-install-recommends wget \
    ca-certificates gnupg

Notice that "sudo" is added in two different places to the same line of script! One addition is in the middle of the script. Don't miss those or the whole thing will fail!
You may want to just do a find-and-replace to identify all the instances of "apt" to replace with "sudo apt" so that you don't inadvertently miss any.
b) Any time you see "echo" you must replace it with "sudo bash -c 'echo PUT THE REST OF IT IN HERE'"
for instance:

echo "deb [arch=amd64] xxxx://r2u.stat.illinois.edu/ubuntu jammy main" \
     > /etc/apt/sources.list.d/cranapt.list

becomes:

sudo bash -c 'echo "deb [arch=amd64] xxxx://r2u.stat.illinois.edu/ubuntu jammy main" \
     > /etc/apt/sources.list.d/cranapt.list'

Note the open ' and closed ' symbols that were added.
c) Do not skip the optional step 5. I found this step to be crucial. Specifically, install the bspm package.
9) Then open R from the terminal (not via Rstudio!). You do this by simply opening a terminal window and typing:

R
  1. Set the R working directory to the directory where you saved your list of installed packages in step #1, above.
  2. Enable the bspm package, so that you can install R packages using the binaries rather than source. In R, type:
bspm::enable()
  1. Now, also in R, re-install all those packages you used to have. Note: I had to adapt @technocrat's code slightly at this step, because a column index was needed. Also, I used dependencies = TRUE as a belt-and-suspenders type of thing to make sure R would catch the dependencies and not freak out.
install.packages(to_reinstall[,1], dependencies = TRUE, ask = FALSE)
  1. Voila! Your life will be much improved. No more installing 100 or more dependent packages, manually and recursively!
1 Like

This topic was automatically closed 42 days after the last reply. New replies are no longer allowed.

If you have a query related to it or one of the replies, start a new topic and refer back with a link.