how to load & render part of data when user click on certain page

Hi warm community, I am facing an out-of-memory issue in R, when I tried to load multiple tables and render using DT in shiny (only have limited RAM..). I also posted this in stackoverflow, you may reply from there

I am wondering if it is possible to only provide table structure (eg, no of rows & column names) to DT, and pre-load first N rows data to display in the app, then load another N rows when user click another page (pagination enabled). I found DT has a dataTableAjax function that return a Ajax URL and can be queried by DT (not sure how it done)

The original datatables JS library has an similar feature (if I not mistaken), as in

For example,

sample_table <- data.frame(a = rnorm(1e7), b = rnorm(1e7), c = rnorm(1e7))


# write large data on disk
write_fst(sample_table, "sample_table.fst")

# how to load data on disk on-demand using Ajax?
  ui = fluidPage(
    title = 'Server-side processing of DataTables',
  server = function(input, output, session) {
    # create a widget using an Ajax URL created above
    tbl_ajax_url <- reactiveVal({
        read_fst("sample_table.fst", from = 1, to = 100, = TRUE), 
        outputId = 'tbl')
    observeEvent(input$tbl_rows_current, {
      rows <- input$tbl_rows_current
        # random access like fst, only load required data when user click the page
        read_fst("sample_table.fst", from = min(rows), to = max(rows), 
        = TRUE), 
        outputId = 'tbl'))

    output$tbl = DT::renderDataTable({
        a = numeric(), b = numeric(), c = numeric(),
        check.names = FALSE), rownames = FALSE, options = list(
          ajax = list(
            serverSide = TRUE, processing = TRUE,
            # not sure how to do this part, where url only return part of data
            url = tbl_ajax_url()

If you have any other suggestions, please let me know as well. My primary objective is to prevent loading all tables in R at once, instead only load partially on-demand.

PS: I am not familiar with any HTML, CSS & JS, please be patient and provide as many details as possible, thanks in advance!

I have figured out a solution to this, please refer to this stackoverflow answer

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.

If you have a query related to it or one of the replies, start a new topic and refer back with a link.