Select columns with values less than 1 from a big matrix

I have a big matrix with 705184 by 670 dimensions. It has columns with values less than 1 and there are also columns with values greater than 1. I would like to create a subset that has columns with values less than 1 from this big matrix. I would also like to get rid of columns with NAs. I searched for a solution online but couldn't find anything satisfactory. Below is the reprex of the code.

Thanks in advance.

library(bigmemory)
library(reprex)

Phenome <- ("D:/My Data/Academic/Research/PhD/2020/Univeristy of Bayreuth/Biomes of Southern Africa/Protected Areas/Big Matrix")
setwd(Phenome)

#Load big matrix
statsMODIS<-attach.big.matrix("D:/My Data/Academic/Research/PhD/2020/Univeristy of Bayreuth/Biomes of Southern Africa/Protected Areas/Big Matrix/statsMODIS.desc")

dim(statsMODIS)
#> [1] 705184 670

#head(statsMODIS, n = 6)

#class(statsMODIS)

#describe(statsMODIS)

Here is one way to do that.
Please note that I could not use your reprex because I do not have access to the data you load with attach.big.matrix. There is no need to use your actual data in a reprex. A small example matrix is enough.

MAT <- matrix(c(0.1,0.2,0.3,0.4,1,2,3,4,0,0.4,0.5,0.6,4,5,6,7,0.2,0.3,NA,0.3),nrow=4)
MAT
#>      [,1] [,2] [,3] [,4] [,5]
#> [1,]  0.1    1  0.0    4  0.2
#> [2,]  0.2    2  0.4    5  0.3
#> [3,]  0.3    3  0.5    6   NA
#> [4,]  0.4    4  0.6    7  0.3
MyFunc <- function(VEC) max(VEC, na.rm  = TRUE) >= 1 | any(is.na(VEC))
BadCol <- apply(X = MAT, MARGIN = 2,  MyFunc)
CleanMAT <- MAT[,!BadCol]
CleanMAT
#>      [,1] [,2]
#> [1,]  0.1  0.0
#> [2,]  0.2  0.4
#> [3,]  0.3  0.5
#> [4,]  0.4  0.6

Created on 2020-10-31 by the reprex package (v0.3.0)

This worked, thanks!

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.

If you have a query related to it or one of the replies, start a new topic and refer back with a link.