Issue with renv.lock file

General
1

Hi!

I'm absolutely new to renv, and started to use today for the first time. Based on the Introduction to renv vignette, it seemed to be a viable alternative to conda by creating isolated virtul environments, but it's doesn't seem to be doing what I expected it to do.

Setup

I created a new RStudio project, and chose to use renv with it.

Inital contents of renv.lock 
{
  "R": {
    "Version": "4.1.0",
    "Repositories": [
      {
        "Name": "CRAN",
        "URL": "https://cran.rstudio.com"
      }
    ]
  },
  "Packages": {
    "renv": {
      "Package": "renv",
      "Version": "0.13.2",
      "Source": "Repository",
      "Repository": "CRAN",
      "Hash": "079cb1f03ff972b30401ed05623cbe92"
    }
  }
}

Then I installed the fpp3 package, and I was able to load it successfully.

> library(fpp3)
-- Attaching packages ------------------------------------------------ fpp3 0.4.0 --
v tibble      3.1.2      v tsibble     1.0.1 
v dplyr       1.0.6      v tsibbledata 0.3.0 
v tidyr       1.1.3      v feasts      0.2.2 
v lubridate   1.7.10     v fable       0.3.1 
v ggplot2     3.3.3      
-- Conflicts ----------------------------------------------------- fpp3_conflicts --
x lubridate::date()    masks base::date()
x dplyr::filter()      masks stats::filter()
x tsibble::intersect() masks base::intersect()
x tsibble::interval()  masks lubridate::interval()
x dplyr::lag()         masks stats::lag()
x tsibble::setdiff()   masks base::setdiff()
x tsibble::union()     masks base::union()

Problem

I wanted a snapshot of this state, so tried for that, but renv didn't allow it.

> renv::snapshot()
* The lockfile is already up to date.

And the renv.lock file contents remained unchanged.

I considered the possibility that something is captured "somehow", and uninstalled fpp3. Then if I called to restore it, obviously it did not, saying it's already up to date.

> .libPaths()
[1] "C:/Users/anirb/Desktop/fpp3/renv/library/R-4.1/x86_64-w64-mingw32"
[2] "C:/Users/anirb/AppData/Local/Temp/Rtmpao4McL/renv-system-library" 
> utils::remove.packages("fpp3")
Removing package from ‘C:/Users/anirb/Desktop/fpp3/renv/library/R-4.1/x86_64-w64-mingw32’
(as ‘lib’ is unspecified)
> library(fpp3)
Error in library(fpp3) : there is no package called ‘fpp3’
> renv::restore()
* The library is already synchronized with the lockfile.

Question

Is this behaviour expected? If so, can someone please explain what is the point of using renv if I can't take a snapshot when I choose to do it? If not, what can I do to fix this?

Session Info 
> sessionInfo()
R version 4.1.0 (2021-05-18)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 19043)

Matrix products: default

locale:
[1] LC_COLLATE=English_India.1252  LC_CTYPE=English_India.1252   
[3] LC_MONETARY=English_India.1252 LC_NUMERIC=C                  
[5] LC_TIME=English_India.1252    

attached base packages:
[1] stats     graphics  grDevices datasets  utils     methods   base     

loaded via a namespace (and not attached):
[1] compiler_4.1.0 tools_4.1.0    renv_0.13.2

Thanks.

2

Did you install the 'fpp3' package within the Rstudio project? You may need to install the package within the project and then you could use renv::snapshot(type='all') to capture alll packages in the ren.lock file and later manually check if entery fpp3 included or not in the ren.lock file

3

Yes, I installed it within the project. If you see my .libPaths() output before, you'll see that the package did get installed in a separate folder (under renv/library within my project directory).

This will lead to entries corresponding to fpp3 and all it's dependencies (and theirs and so on) in the lockfile, if I understand correctly. Do I really need that? For my project, I expected an entry for fpp3 is sufficient, as if renv is capable of installing the current version of it, all the dependencies should be satisfied as well.

I may be wrong in this expectation. Please correct me if that is the case. My idea cam from python structures of requirement.txt, where only what I need is mentioned and not everything that comes from pip freeze.

Seeing documentation, it seems implicit option may fulfill this expectation, if I have few files with library calls in my project. Let me know if that's right. I'd have loved if it was possible to add specific packages which I choose to add, but I don't know whether that option is there or not.

Edit 1 (in reply to kevin):

I tried type = "explicit", but I'm afraid I need a little more guidance. I don't have much experience on package management in R, sorry.

I created the following DESCRIPTION file, with just one line:

Imports: fpp3

And the rest of the setup remained as before, ie. installed just fpp3 in a fresh new project, and as of now there are no scripts yet.

At this step, running renv::snapshot(type = "explicit") and renv::snapshot(type = "implicit") leads to identical results, and renv::snapshot(type = "all") contains a few standard packages with priority recommended.

Output 
> renv::snapshot(type = "explicit")
The following package(s) will be updated in the lockfile:

# CRAN ===============================
- BH               [* -> 1.75.0-0]
- MASS             [* -> 7.3-53]
- Matrix           [* -> 1.2-18]
- R6               [* -> 2.5.0]
- RColorBrewer     [* -> 1.1-2]
- Rcpp             [* -> 1.0.6]
- anytime          [* -> 0.3.9]
- cli              [* -> 2.5.0]
- colorspace       [* -> 2.0-1]
- cpp11            [* -> 0.2.7]
- crayon           [* -> 1.4.1]
- digest           [* -> 0.6.27]
- distributional   [* -> 0.2.2]
- dplyr            [* -> 1.0.6]
- ellipsis         [* -> 0.3.2]
- fable            [* -> 0.3.1]
- fabletools       [* -> 0.3.1]
- fansi            [* -> 0.5.0]
- farver           [* -> 2.1.0]
- feasts           [* -> 0.2.2]
- fpp3             [* -> 0.4.0]
- generics         [* -> 0.1.0]
- ggplot2          [* -> 3.3.3]
- glue             [* -> 1.4.2]
- gtable           [* -> 0.3.0]
- isoband          [* -> 0.2.4]
- labeling         [* -> 0.4.2]
- lattice          [* -> 0.20-41]
- lifecycle        [* -> 1.0.0]
- lubridate        [* -> 1.7.10]
- magrittr         [* -> 2.0.1]
- mgcv             [* -> 1.8-33]
- munsell          [* -> 0.5.0]
- nlme             [* -> 3.1-149]
- numDeriv         [* -> 2016.8-1.1]
- pillar           [* -> 1.6.1]
- pkgconfig        [* -> 2.0.3]
- progressr        [* -> 0.7.0]
- purrr            [* -> 0.3.4]
- rlang            [* -> 0.4.11]
- rstudioapi       [* -> 0.13]
- scales           [* -> 1.1.1]
- slider           [* -> 0.2.1]
- tibble           [* -> 3.1.2]
- tidyr            [* -> 1.1.3]
- tidyselect       [* -> 1.1.1]
- tsibble          [* -> 1.0.1]
- tsibbledata      [* -> 0.3.0]
- urca             [* -> 1.3-0]
- utf8             [* -> 1.2.1]
- vctrs            [* -> 0.3.8]
- viridisLite      [* -> 0.4.0]
- warp             [* -> 0.2.0]
- withr            [* -> 2.4.2]

Do you want to proceed? [y/N]:

So, my question is why does it list all these packages and not just fpp3? Even fpp3 package's DESCRIPTION file do not have all of these listed.

Imports: cli (>= 1.0.0), crayon (>= 1.3.4), dplyr (>= 0.7.4), fable (>=
        0.3.0), fabletools (>= 0.3.0), feasts (>= 0.1.7), ggplot2 (>=
        3.1.1), lubridate (>= 1.7.4), magrittr (>= 1.5), purrr (>=
        0.2.4), rstudioapi (>= 0.7), tibble (>= 1.4.2), tidyr (>=
        0.8.3), tsibble (>= 0.9.3), tsibbledata (>= 0.2.0), urca (>=
        1.3-0)

I'll probably stick with implicit option only as ifendo suggested as I'll have scripts under this project soon, but just wanted to understand why it's happening.

Edit 2 (In reply to nirgrahamuk and kevinushey):

I understand dependencies and their dependencies and so on, but my point was if renv is able to store the current version of a package, it should be enough to store only that package details so that it'll automatically resolve all the dependencies to create a working virtual environment.

Admittedly, this is very much influenced from python where no one lists all libraries in requirements, just the only ones which are actually used and the rest gets solved automatically by conda or pip. So, listing of every packages under all type options are very surprising to me.

But of course these are different languages and neither everything is same nor I can expect. I'm sorry for that and I'll use the existing implementation. Thank you.

4

Yes, I think your understanding is correct, I mentioned this to just make sure fpp3 entry actually got included. And from your original description, it looked like this fpp3 somehow was not included in the ren.lock. I usually used this one as this is the simplest method to check what packages included.

So for this one, not quite sure if it works if you include the library(fpp3) in one of your file and then call renv::snapshot() again.

5

Your first question, I think, is answered in the FAQ: Frequently asked questions • renv

I'd have loved if it was possible to add specific packages which I choose to add, but I don't know whether that option is there or not.

You can configure your project to use so-called "explicit" snapshots. In such a case, renv will read the dependencies as declared in the project's DESCRIPTION file (see R Packages (2e) - 9  DESCRIPTION for more details; although these are generally used for R packages they can also be used for regular projects as well). I believe this is explained in the snapshot documentation: Record current state of the project library in the lockfile — snapshot • renv

7

for example there's distributional , which while not a direct dependency of fpp3, is a dependency of fable. I would expect them all to be 'justified' along similar lines.

8

This should explain why:

> tools::package_dependencies("fpp3", recursive = TRUE)
$fpp3
 [1] "cli"            "crayon"         "dplyr"          "fable"         
 [5] "fabletools"     "feasts"         "ggplot2"        "lubridate"     
 [9] "magrittr"       "purrr"          "rstudioapi"     "tibble"        
[13] "tidyr"          "tsibble"        "tsibbledata"    "urca"          
[17] "glue"           "utils"          "grDevices"      "methods"       
[21] "ellipsis"       "generics"       "lifecycle"      "R6"            
[25] "rlang"          "tidyselect"     "vctrs"          "pillar"        
[29] "Rcpp"           "stats"          "distributional" "progressr"     
[33] "scales"         "grid"           "slider"         "digest"        
[37] "gtable"         "isoband"        "MASS"           "mgcv"          
[41] "withr"          "fansi"          "pkgconfig"      "cpp11"         
[45] "anytime"        "nlme"           "graphics"       "BH"            
[49] "numDeriv"       "farver"         "Matrix"         "splines"       
[53] "lattice"        "utf8"           "labeling"       "munsell"       
[57] "RColorBrewer"   "viridisLite"    "warp"           "colorspace"
10

renv could do that, but then you may run into issues where an update in one or more dependencies of the package you're using changes the behaviour (e.g. breaks) code in your project. Without all that information, renv would be unable to restore your project with the exact versions that were installed at the time the lockfile was generated, which is important for many scenarios.

Admittedly, this is very much influenced from python where no one lists all libraries in requirements, just the only ones which are actually used and the rest gets solved automatically by conda or pip . So, listing of every packages under all type options are very surprising to me.

I'm not sure if that's true. Users might maintain a list of required packages + constraints in their project in some way, but usually an lockfile will capture the full set of dependencies. For example, poetry uses a pyproject.toml file for declared dependencies, and poetry.lock for capturing all dependencies (and their current version at the time the lockfile was generated).

In that case, calling poetry add <package> updates the pyproject.toml file with the requested package, and then also creates poetry.lock capturing each dependent package + version.

In that sense, renv.lock is similar to poetry.lock, while the definition of "required" packages depends on the snapshot type. By default, it is implicit based on the packages that appear to be used in the project, but that behavior can be configured.

11

This topic was automatically closed 21 days after the last reply. New replies are no longer allowed.

If you have a query related to it or one of the replies, start a new topic and refer back with a link.