This is a companion discussion topic for the original entry at https://www.rstudio.com/blog/sparklyr-0-5
We’re happy to announce that version 0.5 of the sparklyr package is now available on CRAN. The new version comes with many improvements over the first release, including:
Extended dplyr support by implementing: do() and n_distinct().
New functions including sdf_quantile(), ft_tokenizer() and ft_regex_tokenizer().
Improved compatibility, sparklyr now respects the value of the ‘na.action’ R option and dim(), nrow() and ncol().
Experimental support for Livy to enable clients, including RStudio, to connect remotely to Apache Spark.