How do I resolve the following error

General
1

Hi, Please assist in how i can resolve the following error: when writing a single pipeline using the %>% operator to create and store a Spark dataframe:

Code chunk:
df_London <- London_file_1 %>%
group_by(start_year,start_month,start_day) %>%
count(start_station) %>%
select(station_id =start_station,year = start_year,month = start_month,day = start_day,demand=n) %>%
left_join(London_file_1, by =c("day" = "day","month" ="month","year" ="year")) %>%
left_join(London_file_4, by =c("station_id"))

image
image962×451 22.2 KB

Error: java.lang.IllegalArgumentException: Object not found 96
at sparklyr.StreamHandler.handleMethodCall(stream.scala:115)
at sparklyr.StreamHandler.read(stream.scala:62)
at sparklyr.BackendHandler.$anonfun$channelRead0$1(handler.scala:60)
at scala.util.control.Breaks.breakable(Breaks.scala:42)
at sparklyr.BackendHandler.channelRead0(handler.scala:41)
at sparklyr.BackendHandler.channelRead0(handler.scala:14)
at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)

2

This topic was automatically closed 21 days after the last reply. New replies are no longer allowed.

If you have a query related to it or one of the replies, start a new topic and refer back with a link.