Is c function (combine) limited to ~4.8KB?

General
,
1

The c() works fine until the vector size is around 4.8KB
When using larger memory size I get "+" and when entering ")" the vector doesn't include all the values.

Is this error? known limitation?

As a workaround, I cut it to several vectors and then combine the vectors c(vec1,vec2,vec3)
Is there a better way?

https://www.rdocumentation.org/packages/base/versions/3.6.2/topics/c

1 Like
2

I see similar behavior where manually entering data with c() truncated the input. Curiously, the vector ended up being 2042 elements when I tried the process in RStudio and it was 2041 elements directly in R. Maybe it is significant that 2^11 = 2048.
What are you doing that requires using c() like that?

3

Hi FJCC,

I think it is the memory, not the number of elements.
When I replace 1 long (like 1.12345678) numbers by 2 short elements like (1,1) it worked.

I try to load vector without using an external file. Do you know a better way?

4

When I have had to make a long vector, I either had a file available or the vector followed some pattern that could be made with the rep() or seq() functions. If you must manually enter it, using c() is the only method I know.
To be clear, a vector can occupy a lot of memory. The command

MyVec <- rep(1.3456789, 100000)

makes a 780 KB vector. There appears to be some limitation to the c() function.

1 Like
5

This is a limitation of R. From the official manual An Introduction to R:

Command lines entered at the console are limited to about 4095 bytes (not characters).

1 Like
6

Thanks Nwerth!

I see that one byte is not one character as you wrote. http://adv-r.had.co.nz/memory.html
Do you have any idea how many characters will okay always?

I assume I should count all the characters, so: c(12345,67890) will be 14 characters.

7

The number of bytes in a character isn't always the same. I'm not an expert in this, but here's my mental model:

  • Characters are stored as a sequence of bytes, and most modern software follows the Unicode Standard to figure out how to translate bytes into characters.
  • ASCII characters (English alphabet and punctuation, most Greek letters, some accented letters, and some math symbols) are one byte.
  • Everything else (Cyrillic, Chinese, emojis) takes multiple bytes.

Can I ask why you're trying to run huge commands? I'm curious.

8

Hi Nwerth,

In R It isn't a s simple as that ...
I'm using only c(), . and numbers, I test and the memory depend only on the number of characters.
I tried repeated numbers and it didn't really matter.

I can't really find the logic...

Characters, Bytes
1-7, 112
8-15, 120
16-31,136
32-47,152
48-63,168
64-127,231
123-?,240

I used:

library(pryr)
object_size()

example:
> x="1234567890"
> object_size(x)
120 B

I know that R is mainly build to load files, but sometimes using just the online is easier

9

Apparently the limit used to be lower(1k) and was changed to the 4k we have at present.
Here is the source code for it.
So in theory you could compile your own version of R, with your own choice of line limit...

1 Like
10

Thanks Nirgrahamuk,

I understand the memory limit, now the question is maximum simple characters that still meets the memory limit.

Thanks

11

This function might be useful to do your investigations with
https://www.rdocumentation.org/packages/base/versions/3.6.2/topics/nchar

12

The size of a character vector in memory is not the sum of the bytes of the individual characters it contains. A character vector is a structure in the background C code and has a bunch of data bundled together: values, length, attributes, etc.

The console commands are just strings of characters, so they don't come with the overhead cost of R objects. If you want to know how many bytes are in a string, use the stringi package:

library(stringi)
stri_numbytes("1234567890")
# [1] 10
nchar("一二三四五六七八九十")
# [1] 10
stri_numbytes("一二三四五六七八九十")
# [1] 30

If by "simple" you mean ASCII, it's 4095.

13

I had a similar issue pasting long json strings into the console (i.e. ending in "+"). This discussion Does Console impose an upper limit on the length of strings cleared it up for me. Sourcing from a script doesn't have the same limit.

1 Like