I have a project for an e-commerce company that involves downloading data via various API's and storing the data in a database. After that, people in the company are going to need to view and edit the data (think of setting prices and running sales), and other partners are going to need to access the data via our own API.
Just to see how all the parts work together, I'm using R and duckdb to figure out how all the API calls work, how to transform the data, and how to parse it out into various relational database tables.
Now I'm wondering if I can just use this setup once we go live, or will I have to either switch to AWS (or the equivalent)? That is, can the R/Shiny environment handle the security, authentication, authorization, data interaction (nothing too fancy, just filter, sort and edit) and outside API access that the project requires?
If anyone has done this in the past, please let me know. Or if I need to use "real" database tools to finish the project, I should know that now so I can start investing my research time elsewhere.
While DuckDB does support ACID, in their concurrency sections DuckDB notes that it's not intended for the use case of having multiple processes writing to the database unless you are going to implement the logic itself. Since that logic can be complex, and since you are expecting multiple team members to be editing data, I would choose an alternative database such as Postgres which has provides the locking/multiple process transactions functionality.
Thanks! I understand those issues, but at the moment they're not that big a problem -- most users will have read-only access to the data, with one or two people on the top being able to edit.
I'm most concerned about allowing our outside vendors to have some kind of API access to the data without compromising security. Is this something that plumber can address?
Hi, I have a very similar setup for some apps I've developed for clients. I use Postgres hosted on Digital Ocean for a backend DB. Authentication/access management for the shiny app is managed by Auth0 (Auth0.com). I also have a Digital Ocean Linux droplet to host a Plumber API which connects to the same DB and an Auth0 API to manage the authentication for that. For performance I have them all hosted in the US east region as that's where my shinyapps are hosted.