How to pass “warnings” for Travis-CI if R testthat build fails?

Package development
1

I currently writing tests for an R package using testthat, the standard library for writing tests in R.

All the tests are currently in a single script all_tests.R within the testthat subdirectory. When I run the entire script all_tests.R or each individual function, all code runs without any errors. If there were any "warnings", I used suppressWarnings().

When I build and check the package manually with the commands

R CMD build package_name
R CMD check package_name.tar.gz ## the tar.gz file is the output of R CMD build

there are no errors. The 00check.log shows nothing, as far as I can see.

However, the build from Travis-CI consistently fails. Here is an example:

...
 testthat results ================================================================
  OK: 521 SKIPPED: 0 FAILED: 9
  1. Error: function1 (@all_tests.R#194) 
  2. Error: function2 (@all_tests.R#247) 
  3. Error: function3 (@all_tests.R#269) 
  4. Error: function4 (@all_tests.R#385) 
  5. Error: function5 (@all_tests.R#353) 
  6. Error: function6 (@all_tests.R#716) 
  7. Error: function7 (@all_tests.R#744) 
  8. Error: function8 (@all_tests.R#787) 
  9. Error: function9 (@all_tests.R#832)

In order to get more information to try to resolve this issue, I added this line in .travis.yml:

script:
- R CMD build .
- R CMD check *tar.gz
- cat /home/travis/build/repo/package_name/package_name.Rcheck/00check.log

In the Travis-CI Job log, I see

The command "cat /home/travis/build/repo/package_name/package_name.Rcheck/00check.log " exited with 0

but that is not very helpful.

(1) How could I resolve this issue? Naturally, I could comment out each of these tests, but I don't understand why these are failing.

(2) What is the correct way to properly view why function1, function2, etc. are failing?

2

There are a few essential parts of using testthat that you seem to be missing:

  • Each file must start with test
  • Each file should start with a context
  • Tests are constructed using expectations, like expect_equal(1 + 1, 2) within each test formed by test_that. This enables you to isolate a failing test with an informative diagnostic. See http://r-pkgs.had.co.nz/tests.html

Having said that, I can't see any link between these things and the error you're facing. Can you share the package?

3

Agree with @hughparsonage. You'll need to link to the source on GitHub or similar for people to offer any advice.

You can also poke around the testing setup for other packages that use testthat and travis. It's standard for the tidyverse packages, for example. Try to follow the general conventions you see there. You might answer your own question or, if not, people are more willing and able to help if you're using testthat and r-travis as designed.

4

It's very strange. Running devtools::test() within the top level of the package subdirectory passes with no warnings, but there are these build errors in Travis which continue to plague me.

You’ll need to link to the source on GitHub or similar for people to offer any advice.

Is it possible to privately message the URL to users, and then provide a summary on this forum how to resolve this issue for future readers (once we figure it out)? I'm not sure what the community.rstudio.com standards are---I prefer to only privately share this project, if possible.

5

I could certainly DM you with the URL and afterwards provide a summary on this forum how to resolve this issue for future readers (once we figure it out).

6

If your operating system is not Ubuntu (specifically trusty), it is likely that the travis results are revealing something about your package or tests that is OS-specific. Or, in general, something about setting up dependencies or paths. You should get more detail on the test failures in your log file, which is why people are suggesting you take a hard look and make sure you aren't using testthat or r-travis in a peculiar way.

7

I'd be happy to take a quick look. If it's a private repo, please add me as a contributor (my GitHub handle is HughParsonage); otherwise, I'll take a look at the repo privately.

Can I ask why you want it private?

8

You should get more detail on the test failures in your log file

It's not clear to me how to view the entire contents of the log file. The cat command detailed in the original post doesn't appear to work.

Perhaps this is a novice error: how does one properly view the 00check.log to check whether these errors are somehow OS specific?

9

Hi there

Are direct messages possible on this forum?

It's actually a public repo---I should get permission from collaborators before posting it here though.

10

Don't know.

Perhaps this is a novice error: how does one properly view the 00check.log to check whether these errors are somehow OS specific?

One could add to the .travis.yml file

after_failure:
- Rscript -e 'readLines("/home/travis/build/repo/package_name/package_name.Rcheck/00check.log")'

But it's up to you to provide us with a way to view your package.

1 Like
11

An update: using this as a guide, Home · craigcitro/r-travis Wiki · GitHub, I included

after_failure:
 - ./travis-tool.sh dump_logs

in order to investigate the log file. It appears that each of errors is due to

unused argument (force = TRUE)

unfortunately, it doesn't appear that suppressWarnings() takes care of this issue.

12

An update: using this as a guide, craigcitro/r-travis/wiki, I included

after_failure:
 - ./travis-tool.sh dump_logs

in order to investigate the log file. It appears that each of errors is due to

unused argument (force = TRUE)

unfortunately, it doesn't appear that suppressWarnings() takes care of this issue.

13

@jennybryan
Latest update:

All testthat issues have been resolved.

R CMD check package_name.tar.gz results in no ERRORS

However, there is still an error relating to LaTeX that is causing the Travis-CI to build:

[* checking tests ...
  Running ‘testthat.R’
 OK
* checking PDF version of manual ... WARNING
LaTeX errors when creating PDF version.
This typically indicates Rd problems.
* checking PDF version of manual without hyperrefs or index ... ERROR
Re-running with no redirection of stdout/stderr.
Hmm ... looks like a package
Error in texi2dvi(file = file, pdf = TRUE, clean = clean, quiet = quiet,  : 
  pdflatex is not available
Error in texi2dvi(file = file, pdf = TRUE, clean = clean, quiet = quiet,  : 
  pdflatex is not available
Error in running tools::texi2pdf()
You may want to clean up by 'rm -rf /tmp/RtmpR5975i/Rd2pdf413d6976434f'
* DONE
Status: 1 ERROR, 5 WARNINGs, 2 NOTEs
See
  ‘/home/travis/build/package/package.Rcheck/00check.log’
for details.
The command "R CMD check *tar.gz" exited with 1.

This looks like it may be tricky to fix:

14

I would strongly recommend using the built in support for R in travis (https://docs.travis-ci.com/user/languages/r/) rather than using the legacy r-travis script, which is no longer being maintained.

15

@jimhester

Thank you! The issue is now solved, and this question closed!

The previous .travis.yml config file was based on the github link at /craigcitro/r-travis/wiki

which uses

language: c

I added this to the config file:

language: r

before_install:
  - tlmgr install index

Then, the Travis-CI build will pass.