Let's use this thread to share our AI projects. Post a brief overview of what you've built, the technologies involved, and any interesting insights or challenges. Looking forward to seeing your work!
I'm somewhat obsessed with using LLMs to interact with information, especially RAG. I started with a simple "Framingham Recyclebot" - answering questions about a city recycling program. It wasn't RAG because the information was small enough I could just stuff everything into each prompt. That app was originally written in Python and Chainlit but I re-did it in R & Shiny once tools like ellmer and shinychat emerged.
I've created a couple of other simple chatbots using RAG and the new ragnar package (I am SO HAPPY there's an R package to do RAG!!). Quarto Tips searches my own collection of Quarto tips (this one also lets the LLM use its own knowledge since my tips are limited), the GitHub repo is at https://github.com/smach/QuartoTips.
For fun I also made a chatbot for my retirement blog, Next Chapter Chatbot.
Now I'm working on some public service hobby RAG projects that need more than semantic matching, because they need to take into account things like dates. For example, a local group just posted an archive of 25+ years of their newsletter PDFs, which include a lot of articles about local history. I wanted people to be able to ask things like "What happened at last year's annual meeting?" but that doesn't work with semantic search alone, so I wanted to include filtering for newsletter year.
However, "Tell me about the city during the Civil War" has a time component but shouldn't filter for year, since there are no newsletters from the 1860s! So the rule is basically "do regular RAG if no time is mentioned, first filter by year if a time is mentioned before RAG BUT ONLY if the time is year 1998 or after". I think I have that working (the opening screen is conventional text search, there's a nav tab for the chatbot).
Now I'm working on a public service hobby project to chat with city government meeting agendas and minutes. I want to filter for date AND board AND whether something is an agenda or minutes. I'm working on the rules, which are more complex. I think the key is making sure users have an easy way to see what documents were used in generating the response and clicking to see the actual text.
I'm also playing with adding chatbots to events calendar listings, which I think is a great interface for events listings. I've run a local events calendar for years, but it's not always easy to find events of interest if there are too many listings.
Did I mention that I'm somewhat obsessed with using LLMs and RAG as a great way to interact with information?
I love that I'm able to do all this in R. I was playing around with LangChain and LlamaIndex in Python, but I don't know Python nearly as well as R so everything took way longer for me to do, plus I tended to get caught up in a lot of sysadmin and package version issues that are a lot easier for me to manage in R.
1 Like
Wow, there is so much great stuff in here. Glad the R packages are working out so well for you. With several real applications under your belt, is there something that stands out that you wish ellmer and chatlas did?
One intersting challenge we have had with RAGs is it really is easy to end up with too much unrelated content. LLMs actually get confused by this. Not sure if you have seen that happen yet.
1 Like
Thanks! I hope to be talking about a few of those apps at my posit::conf lightning talk 
Funny you should ask about this, I just posted an issue/question on the ragnar GitHub asking if there's an easy/obvious way to pre-filter chunks in the data store via metadata fields like Date or Board. ragnar lets you add additional fields to the store, but I haven't figured out how to pre-filter chunks based on those fields. Probably because my knowledge of SQL and DuckDB is pretty limited.
The use case is, if I ask something like "What happened at last year's annual meeting?", I want to first be able to filter by Year==2024 and then do my semantic search for annual meeting. Otherwise with a data store of info that goes back many years, I'll end up with a bunch of chunks about annual meetings in other years. I think this kind of pre-filtering for domain-specific use cases really helps get relevant info.
I've been working on improving my "chat with Framingham meeting agendas and minutes" hobby app. I've got metadata fields in my store for Date, Board, and Type (Agenda or Minutes). I'd like to prefilter:
- If a query has a time component, have the LLM calculate start and end dates (for phrases like "last month", "next week", "last year", etc.) and add a filter for that.
- If a query mentions a specific board or committee, add a filter for that. (Less critical since the generic RAG retrieval does better with these kinds of queries, but still useful.)
- If a query asks about things that happened in the past, filter for Type == "Minutes". If it's asking about what will happen in the future, Type == "Agenda".
So, if someone asks "Did the Getchell Trail get CPA funding last year?", there would be filters for a date range of last year and a Type of Minutes. If they asked "What's happening at the next City Council meeting?" there would be filters for date range starting today (maybe no end date, maybe an end date in the next 30 days), Board == "City Council", and Type=="Agenda". If they asked "What's the status of the Nobscot intersection project?" there might not be any filter and it would just do a regular semantic search and chunk retrieval.
You don't need this kind of filtering for a RAG app that's just querying something like a code base or package documentation. But it would be an enormous increase in relevancy and usefulness for apps where something can be semantically relevant but irrelevant for non-language-related reasons, such as time.
I was on the content/evaluation (NOT coding!) team at work when we were developing the Smart Answers RAG app with an external vendor. I got a good look at how complex and difficult it can be to answer what appears to be a simple question in a RAG app!
I'm just getting started with Ellmer and btw. I'm really digging Simon Couch's approach, which he explains in this episode of the R for the Rest of Us podcast.
He points out that state is really important in Data Science workflows (e.g., what variables you currently have defined in your environment), and that's not necessarily the case with other software engineering patterns. So, these tools are an important bridge to give the LLM context about your state with minimal effort.
There's a bit of a setup cost (e.g., you have to get an API key and set it in your environment). But, the payoff is that it is flexible to different LLM providers so swapping models is plug and play.
Also just really enjoying all the docs from Posit. This article in the Ellmer package docs gives a great illustration for how to think about interacting with an LLM.