I am using shiny to create a machine learning application. This application will find or detect the fraud cases and shows the fraud data with datatable. I trying to create this as a near real time application and this application reads files that i am creating as a stream using sparklyr "stream_generate()". This application is reading the files as a realtime application ,but I want to merge or keep those all files application reads in one dataframe and use for analysis or visualization.
There will not be user interaction at datainput section or machine learning section. User will only interact with the result of the application .
So , how can i merge or save those input files in single file?
Code:
# Define server logic
# Server will work on the data reading,data passing to model and passing data to ui section for visualization
## calling real time file reading
# source("server/00-datacreator-srv.R", local = TRUE)
data <- read_csv("/home/diwash/projeckt/credit_card_feaud/credit-card-fraud-detector/data/data.csv")
IsThereNewFile <- function() { # cheap function whose values over time will be tested for equality;
# inequality indicates that the underlying value has changed and needs to be
# invalidated and re-read using valueFunc
filenames <- list.files(pattern = "*.csv", full.names = TRUE)
length(filenames)
}
ReadAllData <- function() { # A function that calculates the underlying value
filenames <- list.files(pattern = "*.csv", full.names = TRUE)
read_csv(filenames[length(filenames)])
}
shinyServer(server <- function(input, output, session) {
values <- reactiveValues()
values$data <- read_csv("/home/diwash/projeckt/credit_card_feaud/credit-card-fraud-detector/data/data.csv")
## read the files in 3 second difference
sampled_data <- reactivePoll(3000, session, IsThereNewFile, ReadAllData)
real_data <- reactive({values$data <- rbind(values$data, sampled_data())
})
data <- reactive({assign(values$data ,real_data())})
observe({
print(values$data)
})
source("server/01-dashboard-srv.R", local = TRUE)
source("server/02-alert-srv.R", local = TRUE)
})