How can I dynamically render individual frames of a video for every iteration of a for loop in Py Shiny?

shiny
1

Hi, I just tried out Shiny for python yesterday and I'm wondering how can I render an individual video frame (basically an image) as it is being outputted by a for loop? I'm trying to make a web interface for the supervision library and I want the processed/annotated frames to be shown to the user while it's being processed but I can't figure out how to do it. In streamlit it's very easy I since I can simply render the frame to a st.empty() container, is there a similar way to do it in shiny?

I kind of managed to do it but only by using a button to get the next frame however that is not the behavior I want. I would like the frames to continuously render dynamically without any additional user input. I'm trying to use a reactive value to change the "src" of the image but the for loop seems to be blocking me from accessing the new reactive value while its looping hence I only get the last frame .

Rendering an tag does not help either since a new one gets created for each iteration of the loop.

This code below is the code with the 'Next frame' button. Please help me how to modify it so that the frames gets rendered after a successful video upload and not only after a button press.

import supervision as sv

from shiny.express import input, render, ui
from shiny import reactive

from cv2 import imwrite

TEMP_IMG_FILENAME = "temp_img.jpg"

ui.input_file("vid_input", "Upload")

img_obj = reactive.value({})

@reactive.calc  
@reactive.event(input.vid_input)
def get_frames():
    vid_path = input.vid_input()[0]['datapath']
    for frame in sv.get_video_frames_generator(source_path=vid_path): # Returns every frame of a video
        yield frame

ui.input_task_button("next_frame_button", "Next frame")

@reactive.event(input.next_frame_button)
def next_frame():
    frame = next(get_frames()) # Get next frame from the generator
    imwrite(TEMP_IMG_FILENAME, frame) # Write image to a file using opencv
    img_obj.set({"src" : TEMP_IMG_FILENAME, "width": 500}) # Set new value for the img_ojb reactive value
    return img_obj.get() 

@render.image
def show_frames():
    return next_frame()
2

Hi @krvspacetime, welcome to the forum!

While it isn't a direct replacement due to the differences between the Shiny and Streamlit execution models, I think you can achieve a similar functionality using reactive.invalidate_later:

This function is time-based instead of event-based like Streamlit, and you can change it at whatever interval you choose.

Best,
Randy

1 Like
3

Thank you, it kind of works but still not the behavior I want. Since I'm processing the frame using some computer vision model and then writing the image to a file so I can pass it as "src" to the image in the image output, putting a constant time to re-execute the code is not ideal since a lot of times its either re-executing the code before the whole process is done or take much longer than I need to if I increase the time. Also, is there a way to render an image as a PIL.Image or ndarray directly without writing it to a file first?

Here's a much simpler example to the behavior I want. I want to render the text 0,1,2,3,4 sequentially but I'm unable to since the text only renders after the loop is done. Is invalidate_later the only solution to this? Thank you again in advance.

import time

from shiny.express import render
from shiny import reactive


val = reactive.value()

@reactive.effect
def _():
    for i in range(5):
        time.sleep(1)
        val.set(i)

@render.text
def print():
    return f"Number: {val.get()}"
1 Like