How to incorporate Autogen multiagent chat in Shiny

shiny
,
1

Hi,

Shiny with Python has a chat component where you can stream chat responses from OpenAI GPT models.

Is there any example of incorporating a multiagent chat using the Autogen framework in stream mode into a Shiny application?

Thanks

2

Hi @ayuksel, welcome to the community!

We have been playing with generative AI internally, but I don't think anyone has specifically built this example. What use case are you hoping to solve with this?

Best,
Randy

3

Thank you @randyzwitch for the reply.

I believe having a chat component in Shiny significantly expanded the number of use cases and enabled more dynamic and personalized interactions.

I want to build two types of interactivity in Shiny:

  1. My first goal is leveraging multiagent chatbots to answer questions by combining structured and unstructured inputs. The structured inputs would be captured through input widgets, such as sliders, dropdown menus, and other user input forms, while the unstructured data would come from user-generated chat messages. So whenever the user writes something in chat, I will append this message with some structured input taken from input widgets (e.g. generating additional context for LLm using structured input) and then send this combined message to Autogen chatbot.

  2. Based on the chatbot response I want to alter certain outputs (tables, plots) without the need for a user action. LLMs and thus chatbots can return structured outputs, such as JSON. Assume that the chatbot sends a JSON response. This may trigger an event where I add a new output plot in Shiny. I can even change the values of input widgets thanks to the reactivity functionality of Shiny or using whole UIs as components.

I would appreciate it a lot if you could put a minimal example for this use case. For this, we first need to deploy a multiagent Autogen chatbot, and then communicate with it inside the Shiny. I would suggest using HuggingFace spaces, where you can both deploy a chatbot as well as Shiny. I am sure such an example will get a lot of user attention since there is no other example of combining multiagent chatbots with structured inputs and outputs on the web.

Best