I keep getting the error "x missing values and NaN not allowed if na.rm is false"

[Renren]

Hi, I am trying to filter data based on the 80th percentile, but I keep getting the following error:

AGE_M <- Full_data %>%
select(AGE2, SEX, SF36PCS_HLEQ1) %>%
filter(SEX == 1, !is.na(AGE2), !is.na(quantile(SF36PCS_HLEQ1, probs = c(0.8)))) %>%
summarise(AGE_mean_M = mean(AGE2, na.rm = TRUE)) %>%
print()

Error: problem with filter() input 3. x missing values and NaNs not allowed if na.rm is FALSE.

Any suggestions would be much appreciated,
Thanks

[nirgrahamuk]

Add na.rm=TRUE as an additional parameter to quantile()

[Renren]

Hi, thanks for your suggestion. Indeed, this stops the error form occurring, but for some reason the 'probs = c(0.8)' input does not seem to be applied, as the same output is produced for the summarise command, despite changing the c() value. Is there any way of rectifying this? Thanks again!

[nirgrahamuk]

What is your intent for that quantile to do ?
You have it set up so if its result is not NA that rows will be kept, I guess it's a low bar.

[Renren]

Hi, I would like to keep rows that fall within the 80th percentile of the SF36PCS_HLEQ1 variable whilst discarding NA values so that the mean function will work. Thanks

[nirgrahamuk]

I don't have your data, but using the iris dataset, the equivalent approach might look something like :

library(tidyverse)

# make an example dataset with some pesky NA values sprinkled in 
spoiliris <- iris
spoiliris[1,1] <- NA
spoiliris[10,2] <- NA
spoiliris[101,3] <- NA

in step by step

# determine what the 80 percentile of sepal.length variable is 
(q80 <- quantile(spoiliris$Sepal.Length,probs = 0.8,na.rm=TRUE))

#filter on the value
(spiris_within_q80 <- filter(spoiliris,
       Sepal.Length <= q80))

#summarise the result
summarise(spiris_within_q80,
          Petal.Width_mean = mean(Petal.Width,na.rm=TRUE))

in a more condensed form

filter(spoiliris,
       Sepal.Length <= quantile(Sepal.Length,probs = 0.8,na.rm=TRUE)) %>% 
summarise(  Petal.Width_mean = 
              mean(Petal.Width,na.rm=TRUE))