Control Over TinyTeX Package Install Timeout?

Hello,

I am supporting some devs working with containerized RStudio and TinyTeX. In our image build, we generate a simple set of prototypical documents to flesh out the LaTeX package dependencies via automatic installation with TinyTeX to support what we'll need in a production container. We also use Jenkins CI/CD for building, testing, and pushing our images.

Anyway, in practice, we've been finding that these automatic package installs fail quite frequently without raising a significant error, which kicks the can of pipeline failure down the road to our testing. What I'm hoping to be pointed to are some TinyTeX settings related to downloading packages, e.g. the maximum number of download attempts and the timeout for each attempt, ideally. On that note, are there any mirrors that we can add for better reliability with package downloading, and if so, how might I do that?

Although the TinyTeX docs say that "Everything is automatic for R Markdown users, and you don’t need to manually search for or install missing LaTeX packages," we're using it correctly by starting with tinytex:::install_prebuilt() in our R and still having to deal with packages that fail to install automatically in the lines that directly follow, ostensibly due to sporadically timing out. We're on AWS with a good connection by design, and what fails to download is also not consistent by package or any similar metric I can see. Last of all, our builds do pass when all of the packages happen to install properly.

I'd sincerely appreciate any input from the community here, and I hope that the granularity I'm looking for is just not particularly documented rather than non-existent. That said, the TinyTeX dev requires a post on this forum or StackOverflow plus 24 hours to raise an issue on their GitHub repo, so I will either be asking them the same thing this time tomorrow or that any key findings provided here be documented.

Thanks.

Here's something from a tinytex release note last year.

Note that the prebuilt binaries are provided only for your convenience. I do not recommend that you use them routinely. Normally you should install TinyTeX with tinytex::install_tinytex() . However, in certain cases, you may not be able to run tinytex::install_tinytex() . For example, github.com may be blocked in your region, or the latest version of TeX Live is broken (e.g., #210 #213). Then you can use tinytex:::install_prebuilt() as a temporary workaround, which will give you a working version of TeX Live. This version also includes additional LaTeX packages, and you may request that it includes even more packages by editing tools/pkgs-yihui.txt and send a pull request.

The AppVeyor channel seems to be unreliable. I'm not sure, though, if this applies only to WIN.

install_prebuilt isn't imported, but you can see where the source code might hang silently. As a guess, something may be interfering with cache invalidation, a notoriously difficult task.

One of the moderators is deeply involved in TinyTex; I'll flag this for his attention.

Thanks @technocrat for the inputs!

Here are some precisions.

This is not more true. See release note of version 0.26 Release tinytex 0.26 · rstudio/tinytex · GitHub

This issue does not me the AppVeyor channel is broke. It was just an issue with GHA to install TinyTeX binary using r-lib/action/setup-tinytex that is using the install-bin-windows.bat. From what I understand of the above, it should not be related.

Regarging TinyTeX installation

With the last version of tinytex (the R package), we improved the way user can install TinyTeX (the TeX distribution). There is now a specific repo for the binaries

Binaries for OSx, Linux and Windows are available for download as a github release. There 3 binaries for each OS based on how many TeX packages are preinstall. Those binaries can be installed using different methods like using R (for R user) or not (for non R user mainly).

Now you can directly use tinytex::install_tinytex(). It will install the recommended binary for R Markdown which is TinyTeX-1. You can select the version - it will default to the latest. You need to use install_prebuilt() to install one of the other package for example (TinyTeX-0 or TinyTeX) e.g tinytex:::install_prebuilt('TinyTeX')

You need the last version of tinytex from CRAN for this.

Regarding TeX Packages

The binaries comes with a pre-installed set of TeX packages. The list comes from our test suites so that R Markdown and other related product can compile with their default template.

For missing packages, tinytex will parse the log to find the missing package, and try to install them using tinytex::tlmgr_install(). It is not the R package that is doing the download of the missing packages. The R function is just a wrapper around the command line tlmgr.

If this is not working well for you, it may be cause by the CTAN repository mirror that you are using. This is not a TinyTeX feature or tinytex configuration. This is relatated to TeX Live. You can see the repository used with tlmgr repository list (Doc about the command: tlmgr - the native TeX Live Manager)
Maybe changing the url to a better mirror could help, but in theory the best should be chosen but tlmgr.

Also, as you are mentionning CI, sometimes finding and installing all missing packages can be time consuming and the runner can not like this time spent and may timeout. (it happened on Travis for example).

Know that you can install a version of TinyTex with more packages (called TinyTeX) by following one of the installation method in GitHub - rstudio/tinytex-releases: Windows/macOS/Linux binaries and installation methods of TinyTeX

Know that with the R function you can also pass a vector of TeX package to install. If you have the list, this would be faster as it would not have to find the missing packages. We do that in our CI sometimes to speed things up.

If you can share more about the error you get and the issue you have (like error or log messages), we can look into this further.

Hope it helps

1 Like

Thanks for the details, and I realize I was light on describing the actual error. We haven't touched repositories, so this is the mirror we have by default.

List of repositories (with tags if set):
        https://mirrors.concertpass.com/tex-archive/systems/texlive/tlnet (main)

In the course of our builds, what we've been seeing is

/root/.TinyTeX/bin/x86_64-linux/tlmgr: TLPDB::from_file could not initialize from: https://mirror.las.iastate.edu/tex-archive/systems/texlive/tlnet/tlpkg/texlive.tlpdb

What I've added to our Dockerfile is tlmgr option repository ctan, and that's proving a lot more reliable with no more failing builds due to timing out. I think this may be resolved, so thanks again.

2 Likes

Glad it works now !

I think this is mirror issue. I get that sometimes to in CI.
So finding and using a reliable ctan mirror is important.

1 Like

This topic was automatically closed 21 days after the last reply. New replies are no longer allowed.

If you have a query related to it or one of the replies, start a new topic and refer back with a link.