Creating a plumber API that generates video mp4

[howardbaek]

Hello, I'm trying to create a plumber API that accepts as input a Google Slides URL, then runs through a few functions and eventually generates a video mp4 file. Here is the reproducible code example:

Package Installation:

# Install packages from GitHub
remotes::install_github("jhudsl/ari", "ariExtra-immigration")
# need code from ariExtra-immigration branch, not the main branch
remotes::install_github("fhdsl/gsplyr")
remotes::install_github("fhdsl/ptplyr")

Plumber Code:

# Packages ----
library(plumber)

#* @apiTitle ari Plumber API
#* @apiDescription A plumber API that generates automated videos from Google Slides or PowerPoint slides.

#* Generate Automated Video from Google Slides
#* @param link Google Slides URL
#* @param service Text to Speech Service
#* @param model_name  Deep Learning model for Text-to-Speech Conversion
#* @param vocoder_name Voice coder used for speech coding and transmission
#* @serializer contentType list(type="video/mp4")
#* @get /generate_gs
function(link, service, model_name, vocoder_name){
  # Dpwnload Google Slides as PowerPoint
  pptx_path <- gsplyr::download(link, type = "pptx")
  # Extract speaker notes from the PowerPoint file
  pptx_notes_vector <- ptplyr::extract_notes(pptx_path)
  
  # Dpwnload Google Slides as PDF
  pdf_path <- gsplyr::download(link, type = "pdf")
  # Convert PDF file of slides into individual PNG images
  image_path <- ptplyr::convert_pdf_png(pdf_path)
  
  # Generate video from images and speaker notes
  res <- ari::ari_spin(images = image_path, 
                       paragraphs = pptx_notes_vector,
                       output = "ari-video.mp4",
                       tts_engine_args = 
                         list(
                           service = service,
                           model_name = model_name,
                           vocoder_name = vocoder_name))
  # Path to video output
  outfile <- attr(res, "outfile")
  # ???
  readBin(outfile, "raw", n = file.info(outfile)$size)
}

I got the idea of writing the last line from this plumber doc: Rendering Output • plumber. I think this is useful for rendering PDFs, but doesn't work for mp4 files.

I was wondering how to render mp4 video files in the last line, and if this entire approach is wrong, what the right approach to generating mp4 files may be.

Thank you

[meztez]

Try creating a tmp file for your output like the example does. Otherwise I think this should work, no?

[howardbaek]

Hi @meztez , I appreciate the response. I had used the Swagger UI that pops up when you run the API in RStudio and saw this as my response:

Since the Response body contained a bunch of gibberish, I thought I had done something wrong. But I copied-pasted the Request URL into my browser, and it worked!

I had one followup: Do you know why readBin(tmp_video, "raw", n = file.info(tmp_video)$size) is able to render a mp4 video? The example renders a PDF with that line of code, so was surprised to see readBin able to render a mp4.

[meztez]

It's reading the binary (the 1 and 0s) directly from disk and returning that.

The "gibberish" is just the browser attempt at representing the binary data as text and failing because, it is, in fact, not text.

If you want to get a file download instead of returning the video directly to be rendered inside the browser, you can do something like this.

...
  # Path to video output
  outfile <- attr(res, "outfile")
  # ???
  plumber::as_attachment(readBin(outfile, "raw", n = file.info(outfile)$size), "video.mp4")
}

[howardbaek]

Awesome, thank you for your help!