Are packages installed with `conda` located in the same place as the ones installed through RStudio?

raul_guarini_riva · September 4, 2023, 2:11pm

My data science projects typically involve both Python and R and multiple co-authors. We tend to use conda environments to keep track of package versioning. Whenever I'm using R, my workflow is the following:

Activate the respective conda environment for that given project;
Launch that respective project file ("open Project.Rproj") through the terminal
Start working right away on RStudio

RStudio is smart enough to understand it should look for the R version installed in that particular conda environment and not the installed installed on the base environment. That's great.

Whenever I need to install a new package, I use the terminal and the conda-forge repository as my default choice. But sometimes conda won't find the package I need to install and I am forced to install stuff through "install.packages()" through RStudio's console. I have two questions about that:

Are packages installed in the same place?
If the answer for the first question is "yes", is conda able to understand that RStudio installed a specific package in that environment? Is conda able to "see" it?

Thanks in advance!

Gabor · September 4, 2023, 2:39pm

Why don't you try it, and then you can tell us?
You can use find.package() to see where a package is installed, and .libPaths() to see the configured package libraries.

Gabor · September 6, 2023, 4:06pm

I am sorry if my answer came across as rude, that's was not my intention. My sentiment was that it seems that you already have this system set up, so for you it is easy to try things.

Whereas, in my experience not too many people on this forum use RStudio with conda, so for us to answer these questions properly, we would first need to install conda, then RStudio, ideally using the same OS, and then experiment. This is a substantial investment of time and effort. We also don't know what conda (mamba, miniconda?) version you are using, on what OS, how you create and activate projects, etc.

Anyway, for your first question, to check if packages are installed at the same place, call find.package() for a package that you installed with conda, and then for another one that you installed with install.packages(). My guess is that the answer is yes, but it is better to actually try:

dirname(find.package("<first-package>"))
dirname(find.package("<second-package>"))

For your second question, I am not sure what exactly you mean by "seeing" it. If you mean that listing your installed conda packages would include the R package installed with install.packages(), my guess is that the answer is no, and conda won't see it. But again, since you already have the system set up, you can also try and see.

Gabor · September 6, 2023, 6:42pm

OK, I installed conda, and while I didn't install RStudio, that most probably does not matter. RStudio just uses R from the PATH, at least on Linux.

So the answers are 1. yes and 2 no:

> find.package("jsonlite")
[1] "/root/anaconda3/lib/R/library/jsonlite"
> find.package("pkgconfig")
Error in find.package("pkgconfig") :
  there is no package called 'pkgconfig'
> install.packages("pkgconfig")
[...]
> find.package("pkgconfig")
[1] "/root/anaconda3/lib/R/library/pkgconfig"

(base) root@df83022deab1:~# conda list | grep jsonlite
r-jsonlite                1.5                    mro350_0
(base) root@df83022deab1:~# conda list | grep pkgconfig
(base) root@df83022deab1:~#

I guess if you want conda to see the package installed with insall.packages(), you need to create a conda package for it, as in conda-build.

raul_guarini_riva · September 6, 2023, 7:57pm

Hi @Gabor. Your answer didn't come as rude at all! And the commands you showed me were really helpful!

I did the following:

Install RStudio and R using homebrew (I'm on Mac)
Install conda;
Create a new virtual environment with conda and install R on it using conda itself. For the sake of this example I will use a virtual environment called disagreement, which in fact is a real project of mine. How to do that? Using the following sequence of commands on the terminal:

conda create -n disagreement -y
conda activate disagreement
conda install -c conda-forge r-base r-essentials

Install packages on it using conda through the terminal. Let's use as an example the package here:

conda install -c conda-forge r-here

Now let's say we want to open RStudio and make it use the R version and packages installed on the disagreement environment. This project has an .RProj file associated to it. I usually start my workflow invoking

open Disagreement.RProj

After RStudio loads, in the R console, I get the following:

> R.home()
[1] "/Users/raul/miniconda3/envs/disagreement/lib/R"
> find.package("here")
[1] "/Users/raul/miniconda3/envs/disagreement/lib/R/library/here"

Now, it's also true that not all R packages can be installed through conda. One example is httpgd. Although there is a conda instruction (R Httpgd :: Anaconda.org) on how to install it, I couldn't make it work. But I could of course install it through the R console on RStudio. My way of installing it was:

install.packages("httpgd")

And after that I get:

> find.package("httpgd")
[1] "/Users/raul/miniconda3/envs/disagreement/lib/R/library/httpgd"

Notice that this package won't be (and shouldn't be) available to the base R installation - that's the principle of using conda environments after all. But if I list all packages intalled in the disagreement environment, we won't see httpgd listed there:

conda list

# packages in environment at /Users/raul/miniconda3/envs/disagreement:
#
# Name                    Version                   Build  Channel
_r-mutex                  1.0.1               anacondar_1    conda-forge
backports                 1.0                pyhd8ed1ab_3    conda-forge
backports.functools_lru_cache 1.6.5              pyhd8ed1ab_0    conda-forge
bwidget                   1.9.14               hce30654_1    conda-forge
bzip2                     1.0.8                h3422bc3_4    conda-forge
c-ares                    1.19.1               hb547adb_0    conda-forge
ca-certificates           2023.7.22            hf0a4a13_0    conda-forge
cairo                     1.16.0            h1e71087_1016    conda-forge
cctools_osx-arm64         973.0.1             h62378fb_14    conda-forge
cffi                      1.15.1          py311hae827db_3    conda-forge
clang                     16.0.6               he79909e_1    conda-forge
clang-16                  16.0.6          default_h07b2d96_1    conda-forge
clang_osx-arm64           16.0.6               hc421ffc_1    conda-forge
clangxx                   16.0.6          default_h298e243_1    conda-forge
clangxx_osx-arm64         16.0.6               hcd7bac0_1    conda-forge
compiler-rt               16.0.6               hf8d1dfb_0    conda-forge
compiler-rt_osx-arm64     16.0.6               hf8d1dfb_0    conda-forge
curl                      8.2.1                hc52a3a8_0    conda-forge
expat                     2.5.0                hb7217d7_1    conda-forge
font-ttf-dejavu-sans-mono 2.37                 hab24e00_0    conda-forge
font-ttf-inconsolata      3.000                h77eed37_0    conda-forge
font-ttf-source-code-pro  2.038                h77eed37_0    conda-forge
font-ttf-ubuntu           0.83                 hab24e00_0    conda-forge
fontconfig                2.14.2               h82840c6_0    conda-forge
fonts-conda-ecosystem     1                             0    conda-forge
fonts-conda-forge         1                             0    conda-forge
freetype                  2.12.1               hd633e50_1    conda-forge
fribidi                   1.0.10               h27ca646_0    conda-forge
gettext                   0.21.1               h0186832_0    conda-forge
gfortran_impl_osx-arm64   12.3.0               hbbb9e1e_1    conda-forge
gfortran_osx-arm64        12.3.0               h57527a5_1    conda-forge
gmp                       6.2.1                h9f76cd9_0    conda-forge
graphite2                 1.3.13            h9f76cd9_1001    conda-forge
gsl                       2.7                  h6e638da_0    conda-forge
harfbuzz                  7.3.0                h46e5fef_0    conda-forge
icu                       72.1                 he12128b_0    conda-forge
isl                       0.25                 h9a09cb3_0    conda-forge
krb5                      1.21.2               h92f50d5_0    conda-forge
ld64_osx-arm64            609                 ha4bd21c_14    conda-forge
lerc                      4.0.0                h9a09cb3_0    conda-forge
libblas                   3.9.0           17_osxarm64_openblas    conda-forge
libcblas                  3.9.0           17_osxarm64_openblas    conda-forge
libclang-cpp16            16.0.6          default_h07b2d96_1    conda-forge
libcurl                   8.2.1                hc52a3a8_0    conda-forge
libcxx                    16.0.6               h4653b0c_0    conda-forge
libdeflate                1.18                 h1a8c8d9_0    conda-forge
libedit                   3.1.20191231         hc8eb9b7_2    conda-forge
libev                     4.33                 h642e427_1    conda-forge
libexpat                  2.5.0                hb7217d7_1    conda-forge
libffi                    3.4.2                h3422bc3_5    conda-forge
libgfortran               5.0.0           12_3_0_hd922786_1    conda-forge
libgfortran-devel_osx-arm64 12.3.0               hc62be1c_1    conda-forge
libgfortran5              12.3.0               ha3a6a3e_1    conda-forge
libglib                   2.76.4               h24e9cb9_0    conda-forge
libiconv                  1.17                 he4db4b2_0    conda-forge
libjpeg-turbo             2.1.5.1              h1a8c8d9_0    conda-forge
liblapack                 3.9.0           17_osxarm64_openblas    conda-forge
libllvm16                 16.0.6               he79909e_2    conda-forge
libnghttp2                1.52.0               hae82a92_0    conda-forge
libopenblas               0.3.23          openmp_hc731615_0    conda-forge
libpng                    1.6.39               h76d750c_0    conda-forge
libsqlite                 3.43.0               hb31c410_0    conda-forge
libssh2                   1.11.0               h7a5bd25_0    conda-forge
libtiff                   4.5.1                h23a1a89_1    conda-forge
libwebp-base              1.3.1                hb547adb_0    conda-forge
libxml2                   2.11.5               he3bdae6_0    conda-forge
libzlib                   1.2.13               h53f4e23_5    conda-forge
llvm-openmp               16.0.6               h1c12783_0    conda-forge
llvm-tools                16.0.6               he79909e_2    conda-forge
make                      4.3                  he57ea6c_1    conda-forge
mpc                       1.3.1                h91ba8db_0    conda-forge
mpfr                      4.2.0                he09a6ba_0    conda-forge
ncurses                   6.4                  h7ea286d_0    conda-forge
openssl                   3.1.2                h53f4e23_0    conda-forge
pandoc                    3.1.3                hce30654_0    conda-forge
pango                     1.50.14              h9f7e0c6_1    conda-forge
pcre2                     10.40                hb34f9b4_0    conda-forge
pip                       23.2.1             pyhd8ed1ab_0    conda-forge
pixman                    0.40.0               h27ca646_0    conda-forge
prompt-toolkit            3.0.39             pyha770c72_0    conda-forge
prompt_toolkit            3.0.39               hd8ed1ab_0    conda-forge
pycparser                 2.21               pyhd8ed1ab_0    conda-forge
pygments                  2.16.1             pyhd8ed1ab_0    conda-forge
python                    3.11.5          h47c9636_0_cpython    conda-forge
python_abi                3.11                    3_cp311    conda-forge
r-askpass                 1.1               r42h21dc0da_4    conda-forge
r-assertthat              0.2.1             r42hc72bb7e_4    conda-forge
r-backports               1.4.1             r42h21dc0da_2    conda-forge
r-base                    4.2.3                h8babed3_6    conda-forge
r-base64enc               0.1_3           r42h21dc0da_1006    conda-forge
r-bayestestr              0.13.1            r42hc72bb7e_1    conda-forge
r-bh                      1.81.0_1          r42hc72bb7e_1    conda-forge
r-bit                     4.0.5             r42h21dc0da_1    conda-forge
r-bit64                   4.0.5             r42h21dc0da_2    conda-forge
r-blob                    1.2.4             r42hc72bb7e_1    conda-forge
r-boot                    1.3_28.1          r42hc72bb7e_1    conda-forge
r-broom                   1.0.5             r42hc72bb7e_1    conda-forge
r-bslib                   0.5.1             r42hc72bb7e_0    conda-forge
r-cachem                  1.0.8             r42h21dc0da_1    conda-forge
r-callr                   3.7.3             r42hc72bb7e_1    conda-forge
r-cellranger              1.1.0           r42hc72bb7e_1006    conda-forge
r-checkmate               2.2.0             r42h21dc0da_1    conda-forge
r-class                   7.3_22            r42h21dc0da_1    conda-forge
r-cli                     3.6.1             r42he62749d_1    conda-forge
r-clipr                   0.8.0             r42hc72bb7e_2    conda-forge
r-colorspace              2.1_0             r42h21dc0da_1    conda-forge
r-commonmark              1.9.0             r42h21dc0da_1    conda-forge
r-conflicted              1.2.0             r42h785f33e_1    conda-forge
r-corrplot                0.92              r42hc72bb7e_2    conda-forge
r-cpp11                   0.4.6             r42hc72bb7e_0    conda-forge
r-crayon                  1.5.2             r42hc72bb7e_2    conda-forge
r-curl                    5.0.2             r42h867807f_0    conda-forge
r-data.table              1.14.8            r42h5bdffde_2    conda-forge
r-datawizard              0.8.0             r42hc72bb7e_0    conda-forge
r-dbi                     1.1.3             r42hc72bb7e_2    conda-forge
r-dbplyr                  2.3.3             r42hc72bb7e_0    conda-forge
r-desctools               0.99.49           r42hc2f3928_1    conda-forge
r-digest                  0.6.33            r42he62749d_0    conda-forge
r-dplyr                   1.1.2             r42he62749d_1    conda-forge
r-dtplyr                  1.3.1             r42hc72bb7e_1    conda-forge
r-e1071                   1.7_13            r42he62749d_1    conda-forge
r-ellipsis                0.3.2             r42h21dc0da_2    conda-forge
r-evaluate                0.21              r42hc72bb7e_1    conda-forge
r-exact                   3.2               r42hc72bb7e_2    conda-forge
r-expm                    0.999_7           r42h79f127a_1    conda-forge
r-fansi                   1.0.4             r42h21dc0da_1    conda-forge
r-farver                  2.1.1             r42he62749d_2    conda-forge
r-fastmap                 1.1.1             r42he62749d_1    conda-forge
r-fontawesome             0.5.2             r42hc72bb7e_0    conda-forge
r-forcats                 1.0.0             r42hc72bb7e_1    conda-forge
r-fs                      1.6.3             r42he62749d_0    conda-forge
r-gargle                  1.5.2             r42h785f33e_0    conda-forge
r-generics                0.1.3             r42hc72bb7e_2    conda-forge
r-ggplot2                 3.4.3             r42hc72bb7e_0    conda-forge
r-ggsci                   3.0.0             r42hc72bb7e_1    conda-forge
r-gld                     2.6.6             r42h21dc0da_1    conda-forge
r-glue                    1.6.2             r42h21dc0da_2    conda-forge
r-googledrive             2.1.1             r42hc72bb7e_1    conda-forge
r-googlesheets4           1.1.1             r42h785f33e_1    conda-forge
r-gtable                  0.3.4             r42hc72bb7e_0    conda-forge
r-haven                   2.5.3             r42he62749d_0    conda-forge
r-here                    1.0.1             r42hc72bb7e_2    conda-forge
r-highr                   0.10              r42hc72bb7e_1    conda-forge
r-hms                     1.1.3             r42hc72bb7e_1    conda-forge
r-htmltools               0.5.6             r42he62749d_0    conda-forge
r-httr                    1.4.7             r42hc72bb7e_0    conda-forge
r-ids                     1.0.1             r42hc72bb7e_3    conda-forge
r-insight                 0.19.3            r42hc72bb7e_0    conda-forge
r-isoband                 0.2.7             r42he62749d_2    conda-forge
r-jquerylib               0.1.4             r42hc72bb7e_2    conda-forge
r-jsonlite                1.8.7             r42h21dc0da_0    conda-forge
r-kableextra              1.3.4             r42hc72bb7e_2    conda-forge
r-knitr                   1.43              r42hc72bb7e_1    conda-forge
r-labeling                0.4.3             r42hc72bb7e_0    conda-forge
r-lattice                 0.21_8            r42h21dc0da_1    conda-forge
r-lifecycle               1.0.3             r42hc72bb7e_2    conda-forge
r-lmom                    3.0               r42h81fe24e_0    conda-forge
r-lubridate               1.9.2             r42h21dc0da_2    conda-forge
r-magrittr                2.0.3             r42h21dc0da_2    conda-forge
r-markdown                1.8               r42hc72bb7e_0    conda-forge
r-mass                    7.3_60            r42h21dc0da_1    conda-forge
r-matrix                  1.6_1             r42hdce2eb6_0    conda-forge
r-memoise                 2.0.1             r42hc72bb7e_2    conda-forge
r-mgcv                    1.9_0             r42h8420647_0    conda-forge
r-mime                    0.12              r42h21dc0da_2    conda-forge
r-modelr                  0.1.11            r42hc72bb7e_1    conda-forge
r-modelsummary            1.4.2             r42hc72bb7e_0    conda-forge
r-munsell                 0.5.0           r42hc72bb7e_1006    conda-forge
r-mvtnorm                 1.2_3             r42h79f127a_0    conda-forge
r-nlme                    3.1_163           r42h81fe24e_0    conda-forge
r-openssl                 2.0.6             r42h138cb08_1    conda-forge
r-parameters              0.21.1            r42h785f33e_1    conda-forge
r-performance             0.10.4            r42hc72bb7e_1    conda-forge
r-pillar                  1.9.0             r42hc72bb7e_1    conda-forge
r-pkgconfig               2.0.3             r42hc72bb7e_3    conda-forge
r-prettyunits             1.1.1             r42hc72bb7e_3    conda-forge
r-processx                3.8.2             r42h21dc0da_0    conda-forge
r-progress                1.2.2             r42hc72bb7e_4    conda-forge
r-proxy                   0.4_27            r42h21dc0da_2    conda-forge
r-ps                      1.7.5             r42h21dc0da_1    conda-forge
r-purrr                   1.0.2             r42h21dc0da_0    conda-forge
r-r6                      2.5.1             r42hc72bb7e_2    conda-forge
r-ragg                    1.2.5             r42he383b5e_2    conda-forge
r-rappdirs                0.3.3             r42h21dc0da_2    conda-forge
r-rcolorbrewer            1.1_3             r42h785f33e_2    conda-forge
r-rcpp                    1.0.11            r42he62749d_0    conda-forge
r-readr                   2.1.4             r42he62749d_1    conda-forge
r-readxl                  1.4.3             r42h9c2710d_0    conda-forge
r-rematch                 2.0.0             r42hc72bb7e_0    conda-forge
r-rematch2                2.1.2             r42hc72bb7e_3    conda-forge
r-reprex                  2.0.2             r42hc72bb7e_2    conda-forge
r-rlang                   1.1.1             r42he62749d_1    conda-forge
r-rmarkdown               2.24              r42hc72bb7e_0    conda-forge
r-rootsolve               1.8.2.3           r42h81fe24e_2    conda-forge
r-rprojroot               2.0.3             r42hc72bb7e_1    conda-forge
r-rstudioapi              0.15.0            r42hc72bb7e_0    conda-forge
r-rvest                   1.0.3             r42hc72bb7e_2    conda-forge
r-sandwich                3.0_2             r42hc72bb7e_2    conda-forge
r-sass                    0.4.7             r42he62749d_0    conda-forge
r-scales                  1.2.1             r42hc72bb7e_2    conda-forge
r-selectr                 0.4_2             r42hc72bb7e_3    conda-forge
r-stringi                 1.7.12            r42hb8c85c0_2    conda-forge
r-stringr                 1.5.0             r42h785f33e_1    conda-forge
r-svglite                 2.1.1             r42hfa92ce8_1    conda-forge
r-sys                     3.4.2             r42h21dc0da_1    conda-forge
r-systemfonts             1.0.4             r42h4f0b353_2    conda-forge
r-tables                  0.9.17            r42hc72bb7e_1    conda-forge
r-textshaping             0.3.6             r42h614c5d1_6    conda-forge
r-tibble                  3.2.1             r42h21dc0da_2    conda-forge
r-tidyr                   1.3.0             r42he62749d_1    conda-forge
r-tidyselect              1.2.0             r42h011812f_1    conda-forge
r-tidyverse               2.0.0             r42h785f33e_1    conda-forge
r-timechange              0.2.0             r42he62749d_1    conda-forge
r-tinytex                 0.46              r42hc72bb7e_0    conda-forge
r-tzdb                    0.4.0             r42he62749d_1    conda-forge
r-utf8                    1.2.3             r42h21dc0da_1    conda-forge
r-uuid                    1.1_1             r42h21dc0da_0    conda-forge
r-vctrs                   0.6.3             r42he62749d_0    conda-forge
r-viridislite             0.4.2             r42hc72bb7e_1    conda-forge
r-vroom                   1.6.3             r42he62749d_1    conda-forge
r-webshot                 0.5.5             r42hc72bb7e_0    conda-forge
r-withr                   2.5.0             r42hc72bb7e_2    conda-forge
r-xfun                    0.40              r42he62749d_0    conda-forge
r-xml2                    1.3.5             r42h77ef413_0    conda-forge
r-yaml                    2.3.7             r42h21dc0da_1    conda-forge
r-zoo                     1.8_12            r42h21dc0da_1    conda-forge
radian                    0.6.7              pyhd8ed1ab_0    conda-forge
rchitect                  0.4.2           py311heffc1b2_0    conda-forge
readline                  8.2                  h92ec313_1    conda-forge
setuptools                68.1.2             pyhd8ed1ab_0    conda-forge
sigtool                   0.1.3                h44b9a77_0    conda-forge
six                       1.16.0             pyh6c4a22f_0    conda-forge
tapi                      1100.0.11            he4954df_0    conda-forge
tk                        8.6.12               he1e0b03_0    conda-forge
tktable                   2.10                 h4161312_3    conda-forge
tzdata                    2023c                h71feb2d_0    conda-forge
wcwidth                   0.2.6              pyhd8ed1ab_0    conda-forge
wheel                     0.41.2             pyhd8ed1ab_0    conda-forge
xz                        5.2.6                h57fd34a_0    conda-forge
zlib                      1.2.13               h53f4e23_5    conda-forge
zstd                      1.5.5                h4f39d0f_0    conda-forge

(Notice that the R packages are listed as r-something)

Hence, I conclude that conda isn't aware of packages installed through the R console but RStudio correctly picks up the packages installed by conda.

Gabor · September 6, 2023, 8:02pm

Yeah, that makes sense.

Btw. I think you don't need to install R from homebrew at all, I am fairly sure that you don't need that. RStudio will only use R from conda, if you start RStudio while conda is active and conda's R is on the PATH.

And then it is just simpler to have a single (conda) R installation, to avoid a mix up in the future.

raul_guarini_riva · September 6, 2023, 8:18pm

I agree this is the best policy if one wishes to control environments using conda.

system · September 27, 2023, 8:18pm

This topic was automatically closed 21 days after the last reply. New replies are no longer allowed.

If you have a query related to it or one of the replies, start a new topic and refer back with a link.