This might be crazy... if you have lots of Plumber APIs, you could also think about logging via another RESTful API endpoint You implement a single logging service (either via Plumber or something else) and log via calls to that logging service.
Alternatively, you could create an R package to encapsulate the sort of thing @andrie is talking about, so that it is easy to add the behavior to any app or API that you are publishing to Connect.
EDIT: One other thought is to implement the logging via a filter
endpoint as the plumber docs illustrate. Again, this is only in addition to what @andrie has already mentioned:
#* Log some information about the incoming request
#* @filter logger
function(req){
cat(as.character(Sys.time()), "-",
req$REQUEST_METHOD, req$PATH_INFO, "-",
req$HTTP_USER_AGENT, "@", req$REMOTE_ADDR, "\n")
plumber::forward()
}