I'd second the use of a CI tool like travis-CI. I haven't any experience with Canvas, but travis has good integration with GitHub and there are lots of resources/tutorials/templates and the like to make getting started relatively painless.
It might also be worth taking a look at the testthat package (GitHub - r-lib/testthat: An R π¦ to make testing π), which although usually used for package development could perhaps be used for your needs here. I had a similar(?) question recently and this was recommended to me: