Subset of Windows binaries are invalid zip files

Posit Professional & Hosted Products Package Management
1

Hello,

Thanks to everyone who helps maintain the Posit Public Package Manager.

I've come across some Windows binaries that are not valid zip files. Here's one case:

$ curl -O -fSsL https://packagemanager.rstudio.com/cran/2023-01-25/bin/windows/contrib/4.1/rstan_2.21.8.zip

$ file rstan_2.21.8.zip
rstan_2.21.8.zip: gzip compressed data, original size modulo 2^32 5011968

$ unzip rstan_2.21.8.zip
Archive:  rstan_2.21.8.zip
  End-of-central-directory signature not found.  Either this file is not
  a zipfile, or it constitutes one disk of a multi-part archive.  In the
  latter case the central directory and zipfile comment will be found on
  the last disk(s) of this archive.
unzip:  cannot find zipfile directory in one of rstan_2.21.8.zip or
        rstan_2.21.8.zip.zip, and cannot find rstan_2.21.8.zip.ZIP, period.

The zip is actually a tar.gz:

$ tar --list -f rstan_2.21.8.zip | head -3
rstan/
rstan/NAMESPACE
rstan/tools/

Note that the packages with these invalid zips are not listed in PACKAGES (tools::write_PACKAGES() will silent skip over the invalid zips). The consequence, then, is just that any package with an invalid zips will not have a binary available.

It seems there must be a problem in the build procedure for Windows binaries. I'm not sure why it only affects a subset of packages, or how extensive these invalid zips are. However, if I randomly sample 10 packages in the 2023-01-25 repo that are not listed as having an Windows binary, each of them has an invalid zip:

library(tidyverse)

rspm_base_url <- "https://packagemanager.rstudio.com/cran/2023-01-25"

get_pkgs <- function(type) {
  url <- contrib.url(rspm_base_url, type)
  available.packages(contriburl = url) %>%
    as_tibble() %>%
    select(package = Package, version = Version)
}

rspm <- left_join(get_pkgs("source"),
                  get_pkgs("win.binary"),
                  by = "package",
                  suffix = c("_source", "_win"))

win_url <- contrib.url(rspm_base_url, "win.binary")
rspm %>%
  filter(is.na(version_win)) %>%
  sample_n(10)  %>%
  transmute(url = paste0(win_url, "/",
                         package, "_", version_source, ".zip")) %>%
  arrange() %>%
  deframe()
#>  [1] "https://packagemanager.rstudio.com/cran/2023-01-25/bin/windows/contrib/4.1/TailRank_3.2.2.zip"
#>  [2] "https://packagemanager.rstudio.com/cran/2023-01-25/bin/windows/contrib/4.1/Rsymphony_0.1-33.zip"
#>  [3] "https://packagemanager.rstudio.com/cran/2023-01-25/bin/windows/contrib/4.1/pencal_1.3.2.zip"
#>  [4] "https://packagemanager.rstudio.com/cran/2023-01-25/bin/windows/contrib/4.1/segmenTier_0.1.2.zip"
#>  [5] "https://packagemanager.rstudio.com/cran/2023-01-25/bin/windows/contrib/4.1/RcmdrPlugin.DCCV_0.1-1.zip"
#>  [6] "https://packagemanager.rstudio.com/cran/2023-01-25/bin/windows/contrib/4.1/RcmdrPlugin.BiclustGUI_1.1.3.1.zip"
#>  [7] "https://packagemanager.rstudio.com/cran/2023-01-25/bin/windows/contrib/4.1/simPop_2.1.2.zip"
#>  [8] "https://packagemanager.rstudio.com/cran/2023-01-25/bin/windows/contrib/4.1/PALMO_0.1.2.zip"
#>  [9] "https://packagemanager.rstudio.com/cran/2023-01-25/bin/windows/contrib/4.1/mssm_0.1.6.zip"
#> [10] "https://packagemanager.rstudio.com/cran/2023-01-25/bin/windows/contrib/4.1/mHMMbayes_0.2.0.zip"

Example of one:

$ file mHMMbayes_0.2.0.zip
mHMMbayes_0.2.0.zip: gzip compressed data, original size modulo 2^32 5012480
1 Like
2

Hi, thanks for the report. There's a decent chunk of Windows binary packages missing right now, and we're working to resolve that (also see Migrating from MRAN to PPPM: thousands of win.binary packages missing. What to do? - #2 by tylfin).

Those listed packages aren't available as binary and are actually source packages (.tar.gz), so I think this is a bug - Package Manager should really be returning 404 responses on those URLs. We'll look into getting this fixed.

In the meantime, this shouldn't affect the usual package installation in R. Those missing binaries aren't included in the binary PACKAGES listing, so R should only be downloading the source packages in .tar.gz format, e.g., from https://packagemanager.rstudio.com/cran/latest/src/contrib/rstan_2.21.8.tar.gz, and never attempt to use those invalid .zip binary URLs.

3

Those listed packages aren't available as binary and are actually source packages (.tar.gz), so I think this is a bug - Package Manager should really be returning 404 responses on those URLs. We'll look into getting this fixed.

Thanks.

In the meantime, this shouldn't affect the usual package installation in R.

Correct. As I mentioned in my original post, these are not listed as available binaries, so installation isn't affected. My post was just to flag the issue of the .tar.gz files being exposed as the Windows binary zips.

Thanks again.

1 Like
4

This should be fixed in the next release of Package Manager. (and also, rstan should be available as binary now for R 4.0/4.1). Thanks!

1 Like