Hi Demetri,
I understand that is a difficult tradeoff. However, at the moment, the only way to use Posit AI in RStudio is via our subscription service. Some more info: https://docs.posit.co/posit-ai/user/faq/#can-i-use-posit-assistant-with-my-own-api-key-local-models-or-a-custom-cloud-provider
I have been using (and absolutely enjoying) the beta. I know that there is now an Early Release and that beta users can continue with their old accounts until the end of the month. I tried downloading the latest Daily Build but obtain the following error upon opening this version of RStudio.
Error in .rs.downloadFile(url = "https://www.rstudio.org/links/posit-assistant-manifest", :
cannot open URL 'https://www.rstudio.org/links/posit-assistant-manifest'
In addition:
Warning message:
In .rs.downloadFile(url = "https://www.rstudio.org/links/posit-assistant-manifest", :
URL 'https://www.rstudio.org/links/posit-assistant-manifest': Timeout of 60 seconds was reached
I presume this has nothing to do with my Posit AI account but is an issue with the latest daily build. Any fixes? I'd appreciate any help!
The Posit AI assistant is working now. I presume this was a network issue.
So, I joined the beta a few weeks ago and had an incredibly useful first experience with the Posit AI assistant. I was able to use it to build a crosswalk between two Bolivian datasets of communities: one used by the Census / INE to gather demographic data and another used by the military geographers / IGM to spatially locate communities. The agent navigated the overall problem and responded to multiple lines of feedback in uncovering the patterns in the matching process, which ended up having eight stages and generating correspondences for all but 6 out of 19,418 communities.
The Quarto document skill produced a very impressive walkthrough of that analysis which is available here: INE–IGM Community Crosswalk: Methods and Results
What's still not working for me is the code completion aspect, which produces unhelpful blue and white triangles amid working code and too often suggests things in the middle of a line. I prefer CoPilot and have enabled it instead.
What remains deeply unreliable is the analytical text generated by the models, which frequently does not correspond to the actual data analysis. For a striking example, see this graphic, where the analysis skips over the 4th and 5th items to present the 6th as the "fourth most common." Similar results pop up randomly the analytical text generated on a regular basis, resulting in text that contradicts the clear results of the code.
Hi Carwil, thanks for the feedback. Relating to Code Completion, I spoke with the team and they believe they know what you are referring to. They also think it might have been resolved in recent RStudio releases.
Can you let me know which daily build you were on? Here is the current recommendation: https://www.rstudio.org/links/posit-ai-rstudio-download
I have upgraded to later daily version and had my first useful blue triangle edit. I'll circle back as I see if it still gets stuck when the suggestions aren't useful.
The latest Posit AI build seems to uniformly conclude that its alterations to graphs and out put are "Perfect", "much better" etc. with no actual way of checking this. This behavior is disconcerting.
Hi, thanks for finally bringing AI into RStudio. It is a really great initiative and I love the idea. I was initially using "MyOwnRobs", which requires an API key. It works really well when it does, but most of the time it kept running into errors while consuming API credits. Learning about Posit AI was really exciting, and I signed up and subscribed immediately. However, the experience has been quite disappointing so far.
I already have a Claude Pro Plan subscription, which has been mostly adequate for my work, including Claude and Cowork use. So seeing that Posit AI was a Claude-based subscription, I believed I was technically doubling my Claude usage, but that is not the case. The Posit AI subscription depletes very quickly with no daily or weekly session renewals. It does not make sense that the same $20 Claude subscription depletes so fast for a niche use case, while I get far more usage in Claude itself for the same amount.
Claude is currently running a 2x usage limit on all plans, so why do we not get that with the Posit AI subscription? When I first saw the message "Your Posit AI credits have been depleted" after only a few prompts, I thought it was going to reset after a few hours or at worst the next day, but no. My $20 "monthly subscription" was gone after those few prompts, while the same subscription through Claude is still running, refreshing daily, with 2x promotional usage on top. I feel robbed!
Hello - sorry to hear that you haven't had a good experience so far. Depending on the model used and type of work you do, usage credits are consumed at different rates. The Early Release provides access to models we believe work best with the new tooling. Some of the reasons are outlined in the docs.
That said, this is a new service and we are evaluating additional pricing approaches, so thank you very much for the feedback.
Thanks, Alex! However, I will not renew my subscription on the current usage/pricing as it is not worth it. Better I stay subscribed to Claude Pro and do 'copy and paste' to R studio. I will keep and eye out and come back to resubscribe when usage or pricing is revised.
Thanks for the feedback, @CarwilB! It looks like this is a screenshot of a Quarto document, not a Posit Assistant conversation, right? If Posit Assistant made a mistake like this in conversation, I'd be pretty surprised (though not totally shocked, these are LLMs after all!), because it would be composing that analysis text immediately after running that table code and looking at the result. So that would be a pure hallucination.
Instead, what I suspect is happening here is that the agent has to create the entire .qmd from beginning to end in one shot, so it's writing the code chunk and the analysis text without having the benefit of seeing the table output.
I would still be somewhat surprised that it's making this mistake if the conversation you had with Posit Assistant before generating the .qmd, contained basically this exact table. I know it's been 17 days, but do you have any idea if that's true? Or did you ask it to create this Quarto document without having done this analysis in-conversation (the same conversation!) first?
Regardless, thank you for pointing out this issue; in any case, Posit Assistant should be self-aware of this failure mode and either 1) refuse to make up analysis text on output it hasn't seen, or 2) render the report iteratively as it goes, or at least 3) review the report after it's been rendered and clean up the text after the fact.
@awsamu, in answer to your question:
Claude is currently running a 2x usage limit on all plans, so why do we not get that with the Posit AI subscription?
That promotional pricing is only for Claude's own plans--it's not offered to third parties like us.
In general, Claude Pro pricing is impossible for us to match. Third parties like us don't have access to the pricing model of the Claude Pro plan, instead we pay API key rates ($/Mtoken), at their standard pricing or close to it. We also don't have billions of dollars of VC money to burn through; we're a conservatively run business, as we think is appropriate for a company that primarily makes open source software that we intend to continue developing and supporting for many more decades.
For now, that means that anyone who's used to Claude subscription pricing is going to feel like we're expensive, despite the fact that we're not charging enough to cover our costs yet (when you factor in the free trials).
We still think it was important for us to offer this service, as there are a lot of users who spend a LOT of time in RStudio and who consider their time to be worth a lot, and will easily get their money's worth out of every token they pay for through Posit AI. (Or the work they do consumes a reasonable amount of tokens, or works really well with Haiku, or they simply don't mind spending a few tens of dollars a month to accelerate their work.)
In the future, we're hopeful that the trend of dramatically falling inference costs will continue, to the point that we won't think much about inference cost anymore, in much the same way (most?) people don't think much about how many bytes they're transferring as they're casually surfing the web.
I've spent around seven hours using Claude to set up a new project, importing data from a set of spreadsheets and shapefiles. I've made half a dozen scatterplots and sixteen maps, and I just got a credit warning. 80% of my monthly credit has been exhausted.
This is out of line with what I expect from three years of use with ChatGPT and a year with Jetbrains AI subscription integral with their IDEs.
I can see that yesterday Claude checked in on a long-running spatial join process (which worked fine but was poorly designed) periodically more than a hundred times over the course of 30 mins with "getConsoleContent" which every time returned "Error getting console content: Request runtime/getConsoleContent".
I wonder if that behaviour burned a large proportion of my credits. It's not something I asked for. In fact I specifically said to it when launching the command "As this table has a lot more rows, please wait a sufficient amount of time and don't send redundant commands."
Does Posit have an explanation of how credits are consumed? How to optimise their use? Is it possible to have a greater use without "auto-recharge"? I'm going to have to give this another $20 at least but I don't want it charging me $200 by silently auto-renewing while I try to get some work done.
