Migrating from MRAN to PPPM: thousands of win.binary packages missing. What to do?

Hi all,

I know this is similar to these posts:
missing binary package "fst" for Windows (R4.1 and 4.2) - R Admins / Package Management - RStudio Community

No binaries found in packagemanager when using M1 Mac (arm64) - R Admins / Package Management - RStudio Community

As Windows users, we are currently evaluating how to migrate from MRAN to Posit Public Package Manager. We noticed that thousands of packages are missing in PPPM (for the same snapshot date) which endangers our project reproducibility in the future.

Short result of MRAN vs Posit PPM

Here's the the short result. Sources coverage is pretty decent, but there are lots of binaries missing.

SNAPSHOT_DATE TYPE NAME_MATCH VERSION_MATCH VERSION_MISMATCH MISSING_POSIT MISSING_MRAN
2021-12-24 source 18621 18618 1 2 0
2021-12-24 win.binary 18675 15206 102 3325 42
2023-01-25 source 19075 19075 0 0 0
2023-01-25 win.binary 19129 14893 100 4102 34

Question

  • Will the missing binary packages be added at some time?
  • Are you (Posit) striving to jump in for the retiring MRAN?

Thanks a lot for your answer!

source code and reprex

library(tidyverse)

snapshot_dates <- c("2023-01-25", "2021-12-24")

url_stubs_tbl <- tribble(~PM_NAME, ~URL_STUB,
                         "MRAN", "https://mran.microsoft.com/snapshot/",
                         "POSIT", "https://packagemanager.posit.co/cran/")
pkg_types <- c("win.binary", "source")


overview_tbl <- expand.grid(SNAPSHOT_DATE = snapshot_dates, 
                            TYPE = pkg_types,
                            PM_NAME = c("MRAN", "POSIT"),
                            stringsAsFactors = FALSE) |>
    as_tibble() |>
    left_join(url_stubs_tbl, by = "PM_NAME") |>
    mutate(URL = str_c(URL_STUB, SNAPSHOT_DATE))

overview_tbl <- overview_tbl |>
    mutate(AVAILABLE_PKGS = map2(URL, TYPE, ~available.packages(repos = .x, type = .y) |> as_tibble()))


# function to compare to available pkgs tables
compare_avail_pkgs <- function(tbl_1, tbl_2) {
    matching <- select(tbl_1, Package, Version_left = Version) |>
        full_join(select(tbl_2, Package, Version_right = Version),
                  by = "Package") |>
        mutate(NAME_MATCH = TRUE,
               VERSION_MATCH = Version_left == Version_right)
    
    matching <- matching |>
        mutate(MISSING_RIGHT = is.na(Version_right),
               MISSING_LEFT = is.na(Version_left))
    
    matching |>
        rename_with(str_to_upper)
}


wide_overview_tbl <- overview_tbl |> 
    select(SNAPSHOT_DATE, PM_NAME, TYPE, AVAILABLE_PKGS) |>
    pivot_wider(names_from = c(PM_NAME), values_from = AVAILABLE_PKGS) |>
    mutate(COMPARE_RESULT = map2(MRAN, POSIT, compare_avail_pkgs)) |>
    select(SNAPSHOT_DATE, TYPE, COMPARE_RESULT) |>
    unnest(COMPARE_RESULT) |>
    rename_with(~str_replace(.x, "LEFT", "MRAN")) |>
    rename_with(~str_replace(.x, "RIGHT", "POSIT"))
    
wide_overview_tbl |>
    group_by(SNAPSHOT_DATE, TYPE) |>
    mutate(VERSION_MISMATCH = !VERSION_MATCH) |>
    summarise(across(c(NAME_MATCH, VERSION_MATCH, VERSION_MISMATCH, 
                       MISSING_POSIT, MISSING_MRAN), sum, na.rm = TRUE))
#> `summarise()` has grouped output by 'SNAPSHOT_DATE'. You can override using the
#> `.groups` argument.
#> # A tibble: 4 × 7
#> # Groups:   SNAPSHOT_DATE [2]
#>   SNAPSHOT_DATE TYPE       NAME_MATCH VERSION_MATCH VERSION_MI…¹ MISSI…² MISSI…³
#>   <chr>         <chr>           <int>         <int>        <int>   <int>   <int>
#> 1 2021-12-24    source          18621         18618            1       2       0
#> 2 2021-12-24    win.binary      18675         15206          102    3325      42
#> 3 2023-01-25    source          19075         19075            0       0       0
#> 4 2023-01-25    win.binary      19129         14893          100    4102      34
#> # … with abbreviated variable names ¹​VERSION_MISMATCH, ²​MISSING_POSIT,
#> #   ³​MISSING_MRAN

Created on 2023-01-27 with reprex v2.0.2

1 Like

Hi @smichal,

Developer on the Posit Package Manager product here. To answer your questions:

Will the missing binary packages be added at some time?

Yes, we're actively reviewing the data you've posted here. We strive to build 90%+ (99% ideally) of the binary packages for Windows that CRAN provides and should have an update soon.

Are you (Posit) striving to jump in for the retiring MRAN?

While the news of MRAN shutting down is recent, we hope Public Posit Package Manager can act as a replacement for MRAN for the community. We're reviewing any missing functionality that MRAN provides and hoping to either replicate it or have a path forward for most cases by the shutdown date.

I hope that helps,

Tyler

2 Likes

Hi Tyler,

thanks for your swift reply. A solution to that is much appreciated :+1:

Hey @smichal, the recent large dip in Windows binary coverage should be resolved now. The current available Windows binary counts for R 4.2, 4.1, and source are:

> nrow(available.packages(contriburl = "https://packagemanager.posit.co/cran/latest/bin/windows/contrib/4.2", filters = list()))
[1] 17458

> nrow(available.packages(contriburl = "https://packagemanager.posit.co/cran/latest/bin/windows/contrib/4.1", filters = list()))
[1] 18207

> nrow(available.packages(contriburl = "https://packagemanager.posit.co/cran/latest/src/contrib", filters = list()))
[1] 19181

R 4.2 is above 90% but still a little low, but we're always working on improving coverage, especially for very highly used packages.

There are also a few other reasons why Package Manager's Windows binary coverage is lower than CRAN's:

  • We don't currently support Bioconductor binary packages, so any CRAN packages that have Bioconductor dependencies won't be available as binary.
  • Available binary package versions always match the latest source package versions. If a new package version fails to build, Package Manager won't include the previous working version in the binary PACKAGES like CRAN does. This may be changed in the future, but you can still install previous versions of binary packages using an older snapshot URL.
  • Package Manager maintains binary package builds for the latest and 4 previous R versions (R 3.5 through 4.2 currently), so as latest packages drop support for older versions of R over time, the binary coverage will drop for older versions of R.
1 Like