average noise values in data

[Plux]

Hello, I have a long list of values. There is a bit of noise in (eg very small variations) in the data I've recorded. I was wondering if there's a way to remove this by, for example calculating the mean of the parts of my data where there is noise
You can see in the plot below, there is some small variations in the second part (x > 3000) of the my data. Is there a way I can just average these parts of my data, without affecting the other bits? So I'd only need to average data where there are small variations. Basically I'd need a low pass filter on this data.
Here you can get the sample csv file. Just import it and then plot with

plot(string, type="l")

Thanks!!

[nirgrahamuk]

@Plux

I can't seem to be able to upload a csv file

you can put a csv on your google drive, or on a github and link to it.

[Plux]

thanks, done. sorry, thought it wasn't nice to post external links

[nirgrahamuk]

This should get you started

library(tidyverse)
library(slider)  # for moving average
library(cowplot) # for plot_grid
library(signal)  # for signal processing (butterworth)
string <- read_csv("R/string.csv") 

loe_mod <- loess(formula = x~t, data=string,span = .01)
string$y_loess <- predict(loe_mod,string)


string$y_ave5 <- slide_dbl(.x = string,
      .f = ~mean(pull(.x,"x"),na.rm = T),.before=2,.after=2)

bf <- butter(2,.01,type = "low")

string$y_bf <- signal::filter(bf,string$x)

plotthem <- function(data,zoom=TRUE){
  if(zoom)
    mydata <- data %>% dplyr::filter(between(t,2500,4000))
 else 
   mydata <- data
g1 <- ggplot(data=mydata,
       mapping = aes(x=t,y=x)) +
  geom_line(alpha=.2)  +
  geom_line(mapping=aes(y=y_loess),color="darkred")


g2<- ggplot(data=mydata,
       mapping = aes(x=t,y=x)) +
  geom_line(alpha=.2)  +
  geom_line(mapping=aes(y=y_ave5),color="darkblue") 


g3 <- ggplot(data=mydata,
       mapping = aes(x=t,y=x)) +
  geom_line(alpha=.2)  +
  geom_line(mapping=aes(y=y_bf),color="darkgreen")
  
plot_grid(g1,g2,g3,align="h",labels=c("loess","ave5","sigfilt"),ncol=3,nrow=1)
}

p1<-plotthem(string)
p2<-plotthem(string,FALSE)

plot_grid(p1,p2,nrow=2)

[Plux]

thanks that's perfect!