Image detection using YOLOv8 in Python using shiny

[Leprechault]

Hi Everyone!!

I'd like to create a Shiny app using R and Python a cause of the Yolov8 model was developed in Python. But, try to use my app calling some *py codes (setup.py,image_classification.py) in my app directory and it doesn't work, despite the advances of the use o Python and R in Posit to harmonize this two languages.

In my example I try:

library(shiny)
library(shinydashboard)
library(rsconnect)
library(tidyverse)
library(reticulate)
library(purrr)
library(stringr)

# Read setup.py with Yolov8 in Python 
#setup.py file content: ---------------------
# # install yolov8
# from ultralytics import YOLO
#-------------------------------------------
header1<-"# install yolov8",
write.table(header1,file="setup.py",row.names = FALSE,quote=FALSE,col.names=FALSE)
header2<-"from ultralytics import YOLO"
write.table(header2,file="setup.py",append=TRUE,row.names = FALSE,quote=FALSE,col.names=FALSE)



# Create conda env if not exist
if(!("yolodetec_py1" %in% conda_list()$name)){
  # conda_create("yolodetec_py1", python_version = "3.7")
  use_condaenv("yolodetec_py1", required = TRUE)
  # Set up python libraries for object detection
  source_python("setup.py")
}

# Open the training YOLOv8 *pt image_classification.py
# image_classification.py file content: ----
#
# Import my trained model 
# model = YOLO (r"https://github.com/ultralytics/assets/releases/download/v0.0.0/yolov8n.pt") 
# Load detection model 
detection_model = model.predict()
#-------------------------------------------
header1<-"#Import my trained model",
write.table(header1,file="image_classification.py",row.names = FALSE,quote=FALSE,col.names=FALSE)
header2<-"model = YOLO (r"https://github.com/ultralytics/assets/releases/download/v0.0.0/yolov8n.pt")"
write.table(header2,file="image_classification.py",append=TRUE,row.names = FALSE,quote=FALSE,col.names=FALSE)


# Load model and prediction functions
source_python("image_classification.py")


# Load the model
model <-reticulate::model # from imagem_classification.py

# Define the UI
ui <- fluidPage(
  # App title ----
  titlePanel("Hello YOLOv8!"),
  # Sidebar layout with input and output definitions ----
  sidebarLayout(
    # Sidebar panel for inputs ----
    sidebarPanel(
      # Input: File upload
      fileInput("image_path", label = "Input a JPEG image")
    ),
    # Main panel for displaying outputs ----
    mainPanel(
      # Output: Histogram ----
      textOutput(outputId = "prediction"),
      plotOutput(outputId = "image")
    )
  )
)

# Define server logic required to draw a histogram ----
server <- function(input, output) {
  
  image <- reactive({
    req(input$image_path)
    jpeg::readJPEG(input$image_path$datapath)
  })
  
  output$prediction <- renderText({
    
    img <- image() %>% 
      array_reshape(., dim = c(1, dim(.), 1))
    
    paste0("The predicted class is ", detection_model(img)) # from imagem_classification.py
  })
  
  output$image <- renderPlot({
    plot(as.raster(image()))
  })
  
}

shinyApp(ui, server)

Please any help with it?

Thanks in advance,

Alexandre

[gordon]

You might have an easier time with Shiny for Python if that works for your use case. Reticulate can sometimes add some complexity to python environment setup, and if your app is mostly a wrapper for the python library, keeping everything in Python can simplify things.