I often ssh to a remote R interactive session through RStudio terminal. When I load large documents on the remote machine, e.g. with readr::read_tsv_chunked I am seeing a spike in local CPU usage with RStudio, which sometimes causes the local machine to become unresponsive. Any idea what is going on or how to avoid this, other than disconnect from tmux while the file is loading?
when you SSH in general thats akin to directly programming the other machine; you should only be using that machines resources; on the local side SSH shouldnt be using any more resources than to monitor the remote machine to give you feedback/results. Is it possible the chunk progress reports are coming through at such a high rate that your SSH is spiking to accomodate ? I wouldnt have thought such a scenario could arise. Is there any possibility of you misattributing the spike in useage , how reliably does it occur ?
1 Like
Hi thanks for your response @nirgrahamuk. It definitely occurs reliably, and I should have said it also occurs with regular read_tsv, but it gets extreme for larger files which I generally load and filter with load_tsv_chunked. I have been thinking it could be the progress bar, but i can't figure out for the life of me why this should be so resource intensive.
Yes, in fact, setting options(readr.show_progress = FALSE) solved the issue.
Seems like the progress bar could be rewritten to update less often for larger files.
Admittedly, mine may be a rare use case and it can be disabled easily enough.
2 Likes
This topic was automatically closed 21 days after the last reply. New replies are no longer allowed.
If you have a query related to it or one of the replies, start a new topic and refer back with a link.