tidyverse gargle workflow on Google Compute Engine (GCE)

[jnnakhara]

Hello Everyone,
I recently updated to gargle release 1.4.0 and encountered a problem that started for me with updates committed in release 1.2.1. After a number of tests I was able to see that my issue is specific to running requests on a virtual machine spun up with google cloud platform. Updates were made such that gargle would detect if it was being run on GCE and then the authentication would seem to force using service account credentials rather than the oauth2 user credentials.

In order to address the 1.2.1 issue I was able to run the following command before I initialized other tidyverse packages. Using googlesheets4 as a specific use case I would do the following steps to initialize

# Run this function to fix the hierarchy issue
gargle::cred_funs_add(credentials_user_oauth2 = credentials_user_oauth2)

# Reference oauth2 user credentials
googlesheets4::gs4_auth_configure(path = "path/to/my/credentials_google-oauth-client.json") 
googlesheets4::gs4_auth(email = "CLIENT_EMAIL")

With the 1.4.0 release the first line fix I used to have now returns an error

Error in `gargle::cred_funs_add()`:
! This name already appears in the credential function registry:
✖ credentials_user_oauth2
Run `rlang::last_trace()` to see where the error occurred.

And whenever I attempt to access/edit something in googlesheets4 (or googledrive as well) I receive permission error (example below) which are the same as when I was being forced into the service account credentials.

Error in `gargle::response_process()`:
! Client error: (403) PERMISSION_DENIED
• Client does not have sufficient permission. This can happen because the OAuth token does not have the right scopes, the client doesn't have
  permission, or the API has not been enabled for the client project.
• Request had insufficient authentication scopes.

So just to clarify, the permission error isn't a surprise - I know the service account does not have access to the target files. My issue is trying to avoid being forced into using the service account on a GCE instance. Again, I was able to avoid it by previously running a gargle::cred_funs_add() command that is no longer working now

Thanks very much in advance for any help here

[jnnakhara]

Adding more information here with options(gargle_verbosity = "debug") output

With gargle 1.2.1 and no "fix"

#	Authorize googlesheets4
googlesheets4::gs4_auth_configure(path = "path/to/my/credentials_google-oauth-client.json")
googlesheets4::gs4_auth(email = "USER_EMAIL")

Result:

trying `token_fetch()`
trying `credentials_service_account()`
Error caught by `token_fetch()`:
Argument 'txt' must be a JSON string, URL or file.
trying `credentials_external_account()`
aws.ec2metadata not installed; can't detect whether running on EC2 instance
trying `credentials_app_default()`
trying `credentials_gce()`
Use a local file ('.httr-oauth'), to cache OAuth access credentials between R sessions?

1: Yes
2: No

So previously it would always get stuck at the above prompt

With 1.2.1 and "fix"

#	Start with cred_funs_add() update
gargle::cred_funs_add(credentials_user_oauth2 = credentials_user_oauth2)
#	Authorize googlesheets4
googlesheets4::gs4_auth_configure(path = "path/to/my/credentials_google-oauth-client.json")
googlesheets4::gs4_auth(email = "USER_EMAIL")

Result:

trying `token_fetch()`
trying `credentials_user_oauth2()`
Gargle2.0 initialize
adding "userinfo.email" scope
loading token from the cache
matching token found in the cache

With the cred_funs_add() update it successfully pulls through the existing token from cache as I believe it was forced to try credentials_user_oauth2 BEFORE getting stuck at credentials_gce

With 1.4.0 (but unable to apply cred_funs_add())

#	Authorize googlesheets4
googlesheets4::gs4_auth_configure(path = "path/to/my/credentials_google-oauth-client.json")
googlesheets4::gs4_auth(email = "USER_EMAIL")

Result:

trying `token_fetch()`
Trying `credentials_byo_oauth()` ...
Error caught by `token_fetch()`:
inherits(token, "Token2.0") is not TRUE
trying `credentials_service_account()`
Error caught by `token_fetch()`:
Argument 'txt' must be a JSON string, URL or file.
trying `credentials_external_account()`
aws.ec2metadata not installed; can't detect whether running on EC2 instance
trying `credentials_app_default()`
Trying `credentials_gce()` ...
! This requested scope is not among the scopes for the "default" service account:
✖ https://www.googleapis.com/auth/spreadsheets
ℹ If there are problems downstream, this might be the root cause.
GceToken initialize
GceToken init_credentials
! This requested scope is not among the scopes for the access token returned by the metadata server:
✖ https://www.googleapis.com/auth/spreadsheets
ℹ If there are problems downstream, this might be the root cause.
! Updating token scopes to reflect its actual scopes:
• https://www.googleapis.com/auth/cloud-platform
GCE service account email: 123456789-compute@developer.gserviceaccount.com
GCE service account name: "default"
GCE access token scopes: "...cloud-platform"

So rather than the pre-fix 1.2.1 issue with getting stuck at the prompt the authorization finishes but again is landing on the service account even though I specified gs4_auth_configure() with the oauth2 user token

[jnnakhara]

Last note just confirming I solved this problem. As noted at various points the cred_funs_add() function was the first thing that I noticed had changed and should have checked the code on 1.4.0 right there (gargle/cred_funs.R at main · r-lib/gargle · GitHub). Changes are documented and indicate various ways to update the previous "fix" I used to avoid credentials_gce

Noting that I'm using the most direct command to prioritize the credentials_user_oauth2 method gargle::cred_funs_set(list(credentials_user_oauth2 = credentials_user_oauth2))

Authentication is now working as expected