Displaying more than 1000 lines in the R Studio console?

[AJHatswell]

Hello,

So after 6 years of moving from Stata, I've finally found a question someone else doesn't seem to have a satisfactory answer for on google / stackexchange / here / Quora or any number of other sites I commonly consult.

Quite simply, I would like to be able to scroll back through more than 1000 lines in the console.

This should be possible you would think based on Tools>Global Options>Display>Limit length of lines... but this doesn't do anything other than go down in values. You cannot go up past 1000.

I have done quite a bit of searching, as it seems like a simple request, and it does seem to have come up repeatedly in different places since 2010 (and apparently stems back to coding decisions in 2008?). For example:

2010: Apparently then options(max.print=10000) would work - it does not now
stackoverflow how-to-increase-the-limit-for-max-print-in-r

2014: no real resolution
stackoverflow scrolling-back-through-the-rstudio-console

2016: No answer
stackoverflow avoid-string-printed-to-console-getting-truncated-in-rstudio

2017: A guide from an IT department setting it in global settings
link removed

2017: A suggestion to use 'rstudioapi::writeRStudioPreference("console_max_lines", 300)', which no longer works (this was version 3.6.2

More than 1000 lines output in R Studio console

Hi there, I have adjusted the printed output with options(max.print=100000) and I got the whole data_frame() as output in the console, but then I can only scroll back 1000 lines in R Studio console. Is there an hint to disable this limitation? Thanks, Michael

2019: No answer, and a workaround using sink (which isn't really a substitute)
stack overflow rstudio-increase-console-line-limit

2019: Someone apparently making it editable, but sadly I don't understand the raw code (I use R, I'm not a programmer), to understand what they did, and how I can access it

Make console max line limit configurable

2020: Answer of no, as the user tried to use the solution from 3.6.2 which no longer worked
rstudio community can-you-see-more-than-1000-lines-output-in-r-studio-console/80370

2020: Also unresolved
stackoverflow rstudio-expand-the-number-of-lines-the-console-shows

From the final link ,the author states:
"So I've done the above. Gone to preferences, set the console to display 10000 lines, I also put in my code options(max.print=10000). I still only get the bottom of my code outputs. My code take somewhere between 5 and 50 minutes to run depending on the specifics, but I still want to see all my code in the console. Heaven forbid there's an error somewhere, I have no clue where it is. I have over 2000 outputs that I need, and doing it piece meal takes too much time."

In case of interest, I'm similar. The work I do is complex, and at the moment involves simulating a dataset, then analysing it using a particular statistical method (I'm currently testing 3 different methods). I need to be able to follow each through as I see what they are doing along the way. I also output diagnostic values so if a method is doing something incorrectly I can see at which stage it happens. At present I'm just not able to go back. I've managed previously, but now have spent the last 2 hours searching for an answer, because it is bothering me so much.

It seems to be a not infrequent ask, so there must be a way to do this that I haven't found? I get that it might be a limiter on performance, but even with that health warning, there must be a way to show more? Even from when since this first came up (I found a topic in 2010) I had a good PC; an AMD64 X2 I think. I then upgraded to an i5, then again, and now again - the hardware has come on so much (from what was already a top end model). Surely now more than 1000 lines of output should be managable by even and entry level laptop - never mind a number crunching desktop?

Anyway thanks for reading, and also thankyou for the questions answered over the years - I've lost track of how many times I've been here and not had to register as someone had already answered the same thing

[AJHatswell]

Note, as a new user I'm unable to post more than 2 links, so I've taken out the least important ones, but left enough so you can easily find them, if interested

[nirgrahamuk]

I think the thing to do is understand the the console is for working interactively and not for logging long running processes. Its fairly trivial, and more robust to set up logging, from base::sink approach, to more sophistated logging packages, and get the log for your large project with thousands of lines of code saved out into a timestamped file.

[technocrat]

I suggest approaching this as a mental ergonomics problem.

Even were the console to display thousands of lines, visually scanning them is not ideal. @nirgrahamuk suggests some workflows that could be considered. But we don't always find ourselves in the position of designing one and some classes and functions may be workflow-resistant.

What I'd suggest in dealing with an output stream of this size is a simple use of the subset operators when dealing with long data frame or matrices.

Assume that mtcars is a long data frame and you want to focus on the first three variables for the second through fourth rows or the second, fifth and seventh rows for the fourth, sixth and ninth variable

mtcars[2:4,1:3]
#>                 mpg cyl disp
#> Mazda RX4 Wag  21.0   6  160
#> Datsun 710     22.8   4  108
#> Hornet 4 Drive 21.4   6  258
mtcars[c(2,5,7),c(4,6,9)]
#>                    hp    wt am
#> Mazda RX4 Wag     110 2.875  1
#> Hornet Sportabout 175 3.440  0
#> Duster 360        245 3.570  0

^{Created on 2021-01-12 by the reprex package (v0.3.0.9001)}

The trick is to think [row,col] (like Visicalc and Supercalc of old and their descendants). Another technique if you have an idea of where your area of interests lies is to use head and tail

library(magrittr)
head(mtcars,5) %>% tail(.,2)
#>                    mpg cyl disp  hp drat    wt  qsec vs am gear carb
#> Hornet 4 Drive    21.4   6  258 110 3.08 3.215 19.44  1  0    3    1
#> Hornet Sportabout 18.7   8  360 175 3.15 3.440 17.02  0  0    3    2

``` r
# create an ID column
library(tibble)
rownames_to_column(mtcars, var = "ID") -> mtcars

# make a function for reuse

isolate_one <- function(x,y) mtcars[which(x["ID"] == y),]
isolate_one(mtcars,"Duster 360")
#>           ID  mpg cyl disp  hp drat   wt  qsec vs am gear carb
#> 7 Duster 360 14.3   8  360 245 3.21 3.57 15.84  0  0    3    4

[AJHatswell]

Thanks for the responses. When using data, I do tend to use head(), as it let's me see the structure, and if I want to look at more, I just use View. Neither of these however seems to address the fundamental issue of the rather arbitrary 1000 line limit?

As an example of the type of work we do, we published this paper a couple of months ago, which includes the R code behind the paper. The code is 800 lines, but being able to walk through and see the summaries pop up isn't an unreasonable thing to want to do

What is the Impact of the Analysis Method Used for Health State Utility...

This particular paper is a simulation study where we first simulate data for individuals (over a lifetime), and then sample from that as would be seen with a clinical trial, before testing a few approaches for analysing that data, to see which gets closest to the truth (which we know from the simualted data).

Is there no way to force R Studio to do it? When I first started on R I was using an old macbook pro, now with an i5 8th gen, and an M2 SSD, surely it can handle at least 2000 lines? Being able to scroll through should be possible; with the nice colour coding of commands, outputs, errors (and warnings), which really helps - not flicking between programs? Adulterating the code to figure out sink and trying to use notepad with a txt output is a really poor substitute for what doesn't seem (to the non-programmer) like a wild request?

[nirgrahamuk]

@AJHatswell

As an example of the type of work we do, we published this paper a couple of months ago, which includes the R code behind the paper. The code is 800 lines, but being able to walk through and see the summaries pop up isn't an unreasonable thing to want to do

Have you considered R Notebook (R Markdown) ?

[AJHatswell]

I've come across Rmd files, but it's on my list of things to learn in the future (along with parallel processing, what the jobs pane does, and R Project files). How would it help? Not being familiar with how they work, I don't see how a report format would help? And it would also be another way of, for lack of a better word, hacking an output (to have to switch to), rather than getting R to do something directly? If it will work as a stopgap I guess that would suffice, though would be far from ideal.

[martin.R]

As nirgrahamuk suggests, the work method you describe of running chunks of code and displaying the associated output is what R Notebooks are designed for.

[benjaminhlina]

Base function str() which stands for structure allows for a better and more comprehensive view of the data than head() and tail(). Dplyr has it's own function that does the same thing but lines things up more evenly making it easier to read that function is glimpse(). Both of these functions show the data as well as the data's class e.g. numeric, character ect. I highly suggest using either of these functions to view data as well as previous suggested workflow methods. I use both str() and glimpse() multiple times within a script.

Additionally as suggested Rmarkdown would help facilitate this as well as code collapsing using commented lines of four # or = or -. Workflow suggestion would also be helpful in this relm as well which have been suggested.