Create private helper functions in R packages

[shamindras]

Hi All,

I'm currently co-developing an open source package on github, with the eventual goal
of pushing to CRAN. We have written a number of functions, all documented with
roxygen2 and all with the @export tag, so that they are populated in the NAMESPACE.

For example a function would be pkgname::foo(), and a helper function called by foo
would be pkgname::foo_helper(). The pkgname::foo_helper() is purely internal i.e.
its only purpose is to help with some computation in pkgname::foo(), and is not the
focus of our package, but pkgname::foo() is.

In this example we would like to convert the pkgname::foo_helper() functions
to "private", as defined by the following criteria:

1. The pkgname::foo_helper() should by default not appear in the pkgdown>References section, only pkgname::foo() should appear
2. The pkgname::foo_helper() function should be fully visible on github i.e. not hidden
3. The pkgname::foo_helper() functions should be accessible to the R user by running
   pkgname::foo_helper(), if they want to play with it in R directly.

What is the best way to make pkgname::foo_helper() "private" in the above sense.

Also, if we wanted to hide pkgname::foo_helper() in R (so user is not distracted by this helper),
what would we change to make them private in this sense?

I assumed that adding a "." at the start of the function name achieves this, but felt there would be
more to it. I was concerned about the implications of removing the @export tag in roxygen,
especially if other internal package functions relied on these helper functions.

Any help on this is appreciated.

Note: This question is now cross-posted here to get more visibility.

[mattwarkentin]

Hi @shamindras,

Your package functions have full access to ALL of the internal functions in your package, even if they are not exported. Exporting is only necessary to make a function available in your packages NAMESPACE (i.e. make it user-facing).

To use your example, if you have an exported function pkg::foo() and it uses some internal function foo_helper(), you do NOT need to export foo_helper() if it only gets used internally by functions in your package. If you do not export it, it will not appear in the references, yet it will remain "visible" on GitHub in the sense that it is not a dot-file or some other special file that would be ignored/hidden.

However, if you do not @export, the function should not really be used/access by users - they CAN technically still access non-exported functions via the triple-colon pkg:::foo_helper(), but this is not recommended.

I think you will have CRAN check issues if you try exporting a function but don't want to document it to avoid it appearing in some listings. You may consider documenting the helper functions inside of the documentation for the main function. For example, foo_helper() could be documented inside the documentation for foo:

#' Foo
#' @export
foo <- function() {
  ...my sweet code...
}

#' @rdname foo
#' @export
foo_helper <- function() {
  ...more code...
}

In this case, the function documentation will be foo.Rd, and is searchable by ?foo, which will document both foo and foo_helper(). Hope this is helpful.

[shamindras]

Hi @mattwarkentin

Thanks for your very helpful response. I just had a few quick follow up questions.
Currently we already have detailed roxygen2 documentation for all our helpers
they are all currently exported using @export.

As such I understand we just need to keep the existing roxygen2
documentation for all helper functions , but remove the @export tag for them.
This will ensure that they don't show up in R help, and also that the 2 colon syntax i.e. pkgname::foo_helper() will not work. They will still be accessible using
pkgname:::foo_helper() as I understand. The purpose of fully documenting these
helpers (after removing @export) will help pass CRAN check issues, even though
they are not exported.

Is my understanding of your answer correct?

Also if we remove @export from pkgname::foo_helper(), then what does
having #' @rdname foo do?

Thanks

[mattwarkentin]

As such I understand we just need to keep the existing roxygen2
documentation for all helper functions , but remove the @export tag for them.

If you no longer export a function, then the roxygen tags are ignored by roxygen2. You can keep them if you've already documented the function, but no documentation will be produced and thus won't show up in the help document. Documenting non-exported functions can still be helpful for you, the author of the package, and also for anyone who digs around the source code on GitHub, for example. But it is not strictly necessary.

This will ensure that they don't show up in R help, and also that the 2 colon syntax i.e. pkgname::foo_helper() will not work. They will still be accessible using
pkgname:::foo_helper() as I understand.

Yes, the two-colon syntax does not work to access a non-exported function. The three-colon syntax will always have access to ALL functions/data/etc. in a package, regardless of whether it is exported or not.

The purpose of fully documenting these
helpers (after removing @export ) will help pass CRAN check issues, even though
they are not exported.

Not quite. Internal functions do not need documentation to pass CRAN checks. But exported functions DO need to have documentation. If you no longer export a function, it doesn't matter whether it is documented or not.

Also if we remove @export from pkgname::foo_helper() , then what does
having #' @rdname foo do?

I am pretty sure @rdname does nothing if you do not export the function.

[shamindras]

Thanks so much @mattwarkentin. This is extremely helpful.

I'll mark your solution as accepted.

[jlacko]

One more thing for you to consider is marking the helper function as internal - via #' @keywords internal in your roxygen slug. It will remove the function from the documentation index (so it will not appear as "missing").

I am not 100% certain what is the handling of not-exported but not-internal functions, but it does not hurt to have it in your documentation - even if it is not a documentation intended for package users, but for package developers (including, above all, the future you).

[shamindras]

Hi @jlacko,

Many thanks for your help. Based on a modification of @mattwarkentin's post above, would you then recommend doing something like this?

#' @rdname foo
#' @keywords internal
foo_helper <- function() {
  ...more code...
}

Note that here the #' @export is removed and replaced with #' @keywords internal
per your suggestion.

As I understand from this change, foo_helper() can only be called from R using pkgname:::foo_helper()
i.e. using three colons, and will not appear in help i.e. help(foo_helper) should not return
any matches. Or would a help file still be created for foo_helper()? I believe
there is also the @noRd option, but it is not clear what the difference is
between this and #' keywords internal.

If your @keywords internal is added/removed what would that mean from the
user's point of view in R?

Thanks again. I'm new to these documentation technicalities, so this is very helpful.

[jlacko]

This is almost exactly my approach (except for the @rdname which I omit, as I prefer the one function - one R file approach for simplicity sake).

For your inspiration about a possible implementation approach: this is a helper function that I use in my {RCzechia} package, https://github.com/jlacko/RCzechia/blob/master/R/downloader.R - it is used by a number of "public" functions to serve an object stored as rds file on the internet. On CRAN since early 2018.

[shamindras]

Thanks @jlacko.

I tried the following:

install.packages("RCzechia")
RCzechia:::downloader # Works as expected i.e. displays internal (non-exported) function source
RCzechia::downloader # Works as expected i.e. gives an error that it is not an exported object
?RCzechia:::downloader # Works as expected i.e. displays the helpfile for the function

So I guess using #' keywords internal allowed the help file to be shown for the
downloader() function? I suppose if you had put @noRd instead, it would have
not shown the helpfile at all compared to the current status?

Finally, what would adding #' @rdname foo in this example actually do here (you can
replace foo with a more relevant function name here for your package)?

Sorry for these final follow up questions, some of these more refined
roxygen2 options are quite new to me.

[jlacko]

The internal means it is not listed on the help pages of the package, i.e. help(package = RCzechia) has no entry for the downloader function, as it is not publicly available.

It still has some content when accessed directly and ?RCzechia:::downloader returns a (skeleton of a) help page.

Of course it can be, despite being private, accessed via triple colon, which is strongly discouraged.

The use case for @rdname is when you need to spread out documentation of an object over multiple R files. I yet have to feel the need to do so and so far I have avoided using the function. I might change my mind if & when I start using complex S3 objects with sophisticated method calls, but until then...

[shamindras]

Thanks @jlacko - appreciate your helpful clarifications.

[dgabbe]

If you don't mind 2 other suggestions. The first is build a companion website using pkgdown. You can host it in the same GitHub repo. I find it very helpful to have a web reference for a package in addition to the built-in help after the package is installed.

Second is creating a package level help page which describes any concepts or idea that may be helpful to using the package. In your R/ directory, add a file named project-package.R where project is the name of your package. The contents of the file is all Roxygen comments and ends with:

#' @docType package
#' @name wdprompt-package
#' @keywords internal
"_PACKAGE"
NULL

Here's an example, minus publishing it on CRAN:

Project: https://github.com/dgabbe/wdprompt
Pkgdown site: https://blog.frame38.com/wdprompt/reference/wdprompt-package.html