I have a workflow (in WDL) pipeline setup on a remote HPC cluster, it executes successfully with 2 commands as follows.
# cd to the project folder
cd folder/project_folder/with_cromwell
# invoke the .sh script, it requires the library name as input
sh ./execute_from_shiny.sh input_library_name
Wanted to create a R/Shiny app so that, users can log-in to the remote server via R/shiny UI and execute the pipeline, track progress and view the outputs back in R/Shiny. I can successfully SSH via R/Shiny ( using ID and Password) and execute the basic bash commands. (like cd, ls -lh, etc)
However, certain commands like module load python and other HPC environment-specific commands do not work. It seems R/Shiny is not invoking the right bash environment, but no clue how to ensure that the necessary environment or configurations are properly loaded within the SSH session.
When I tried the same commands after SSH-ing from Mac OS terminal or, setting up a basic python flask app, was able login with user credentials and execute the pipeline successfully without the issue with HPC environment specific commands like module load or source activate. Which I assume python is able to load the appropriate remote env and r could not.
Here is the sample app. I'm stuck at execution of pipeline step, not sure what challenges are with using passkey instead of login & pswd, track progress and viewing the outputs on Shiny UI would be. Any suggestion on how to approach this app are helpful.
library(shiny)
library(ssh)
#> Linking to libssh v0.9.5
# UI
ui <- fluidPage(
textOutput("output"),
actionButton("connect", "Connect and Run")
)
# Server
server <- function(input, output, session) {
observeEvent(input$connect, {
# Hardcoded values
# TO DO: Setup ssh key
server_address <- "portal.univ.edu"
username<- "login_name"
password <- "login_pswd"
command <- c("cd to/project/directory","module load python")
output_file <- "path/to/bash_console/command_output.txt" # Output file path on the remote server
tryCatch({
# Establish an SSH connection
con <- ssh_connect(paste(username,server_address,sep = "@"),passwd = password)
# Construct a command to redirect output to a file
command_with_redirect <- paste0(command, " > ", output_file)
# Execute the command with output redirection
ssh::ssh_exec_wait(con, command_with_redirect)
ssh_exec_internal(con, command_with_redirect)
# Read the content of the output file
output_text <- ssh::ssh_exec_internal(con, paste("cat", output_file))
# Display the output
output$output <- renderText({
paste("Command Output:\n", rawToChar(output_text$stdout),sep = "n")
})
# Close the SSH connection
ssh_disconnect(con)
}, error = function(e) {
output$output <- renderText({
paste("Error: ", e$message)
})
})
})
}
shinyApp(ui, server)
<sup>Created on 2024-01-10 with [reprex v2.0.2](https://reprex.tidyverse.org)</sup>
<details style="margin-bottom:10px;">
<summary>
Session info
sessionInfo()
#> R version 4.2.3 (2023-03-15)
#> Platform: x86_64-apple-darwin17.0 (64-bit)
#> Running under: macOS Big Sur ... 10.16
#>
#> Matrix products: default
#> BLAS: /Library/Frameworks/R.framework/Versions/4.2/Resources/lib/libRblas.0.dylib
#> LAPACK: /Library/Frameworks/R.framework/Versions/4.2/Resources/lib/libRlapack.dylib
#>
#> locale:
#> [1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8
#>
#> attached base packages:
#> [1] stats graphics grDevices utils datasets methods base
#>
#> other attached packages:
#> [1] ssh_0.8.3 shiny_1.7.4
#>
#> loaded via a namespace (and not attached):
#> [1] Rcpp_1.0.10 rstudioapi_0.14 knitr_1.42 magrittr_2.0.3
#> [5] xtable_1.8-4 R6_2.5.1 rlang_1.1.0 fastmap_1.1.1
#> [9] sys_3.4.1 tools_4.2.3 xfun_0.37 cli_3.6.1
#> [13] jquerylib_0.1.4 withr_2.5.0 htmltools_0.5.4 askpass_1.1
#> [17] ellipsis_0.3.2 openssl_2.0.6 yaml_2.3.7 digest_0.6.31
#> [21] lifecycle_1.0.3 later_1.3.1 sass_0.4.5 credentials_1.3.2
#> [25] fs_1.6.1 promises_1.2.0.1 cachem_1.0.8 glue_1.6.2
#> [29] evaluate_0.20 mime_0.12 rmarkdown_2.20 reprex_2.0.2
#> [33] bslib_0.4.2 compiler_4.2.3 jsonlite_1.8.4 httpuv_1.6.11