For loop not updating variable

[stikpet]

I have a for loop that doesn't seem to update the dataframe (all_perms). Hard-coding two loops works just fine. I can't understand the mistake.

Here is my example code:

#preparation
data <- c("MARRIED", "DIVORCED", "MARRIED", "SEPARATED", "DIVORCED", "NEVER MARRIED", "DIVORCED", "DIVORCED", "NEVER MARRIED", "MARRIED", "MARRIED", "MARRIED", "SEPARATED", "DIVORCED", "NEVER MARRIED", "NEVER MARRIED", "DIVORCED", "DIVORCED", "MARRIED")
observed <- c(table(data))
n <- sum(observed)
k <- length(observed)
expProb =rep(1/k, k)
pObs = dmultinom(sort(observed, decreasing=TRUE), size=n, expProb)

counts <- seq(0, n, by = 1)
kCounts <- matrix(,nrow=n+1, ncol=k)
for (i in 1:k){
  kCounts[,i] <- counts
}

all_perm <- merge(kCounts[,1], as.data.frame(kCounts[,2]),all=TRUE)
all_perm <- all_perm[rowSums(all_perm) <= n,]

#WITHOUT FOR LOOP
#column three and four
all_perm <- merge(all_perm, as.data.frame(kCounts[,3]),all=TRUE)
all_perm <- all_perm[rowSums(all_perm) <= n,]
dim(all_perm)
#shows correct 3 columns
all_perm <- merge(all_perm, as.data.frame(kCounts[,4]),all=TRUE)
all_perm <- all_perm[rowSums(all_perm) <= n,]
dim(all_perm)
#shows correct 4 columns

#THE FOR LOOP THAT DOESN'T WORK
#reset first two columns
all_perm <- merge(kCounts[,1], as.data.frame(kCounts[,2]),all=TRUE)
all_perm <- all_perm[rowSums(all_perm) <= n,]

#the for loop
for (i in 3:k){
  all_perm <- merge(all_perm, as.data.frame(kCounts[,i]),all=TRUE)
  all_perm <- all_perm[rowSums(all_perm) <= n,]
  print(dim(all_perm))
}
#first iteration good (column 3), fourth not being added.

I tried using a while loop instead, but also that didn't work. I really don't get why the for loop is not working any help would be appreciated.

[technocrat]

The overarching reason is in the class of problems that Inigo Montoya with respect to Vizzini's frequent use of the word inconceivable to express surprise or to cast doubt on some future course of events' predicted outcome. In users trained in imperative/procedural languages it seems an immutable property of the universe that a variable springs into being immediately upon its assignment of a value.

In a procedural language (at least as it usually presents to the user) such as R that is only partially true. Because lazy evaluation and environment scoping.

What happens in Vegas stays in Vegas —The Hospitality Commission, or whatever they call it.

Let f(x) = y where x is the cash and credit brought to the table and y is whatever amount greater than x or, more likely y<<x. Think of the environment in which f operates as its local environment. But just as the punter stays in Vegas before going home, the states that x stay in a f until it returns some value of y to become part of the global environment.

Within f objects in the global environment can be changed, as done with

@stikpet

for (i in 1:k){
  kCounts[,i] <- counts
}

kCounts exists in the global environment, as does its successive subsets. This

@stikpet

for (i in 3:k){
  all_perm <- merge(all_perm, as.data.frame(kCounts[,i]),all=TRUE)
  all_perm <- all_perm[rowSums(all_perm) <= n,]
  print(dim(all_perm))
}

creates a local version , within for of all_perm and in every iteration along kCount creates a new version. In R it is often difficult to perceive the intent when presented procedurally.

Can you show the structure of the object you are trying to create with a reprex. See the F AQ. Then we can work on how to express this in a vectorized fashion.

On the other hand, if you are wed to C/C++, there's a package to import functions into namespace.

[nirgrahamuk]

I think your issue comes about due to lack of care over the column names created, for my money "kCounts[, 2]" is not a great name, and besides that; in the for loop the same column name is re-used "kCounts[, i]"and therefore overwritten rather than side by side. Add additional code to name columns to ensure their uniqueness

for (i in 3:k){
  set_to_merge <- data.frame(dummyname=kCounts[,i])
  colnames(set_to_merge) <- paste0("k_",i)
  all_perm <- merge(all_perm, set_to_merge,all=TRUE)
  all_perm <- all_perm[rowSums(all_perm) <= n,]
  print(dim(all_perm))
}

[stikpet]

Thanks for this extensive answer and explanation on why the for loop doesn't work. I suspected that it had to deal with something like global/local, or something like how the vall_perm get stored.

The code is for just a theoretical exercise for myself (and others interested) to perform a an exact multionimal test. I know there is already an R library with a function that can do this, but always like to be able to do the math myself

The code from nirgrahamuk fixes the issue. I know the code can probably be optimized, used something else than a for loop, but that actually fixes for me the issue. Still thanks for your detailed background info, since I also do like to understand 'why' if my code doesn't do what I suspect it to do.

Here is the reprex:

#preparation
data <- c("MARRIED", "DIVORCED", "MARRIED", "SEPARATED", "DIVORCED", "NEVER MARRIED", "DIVORCED", "DIVORCED", "NEVER MARRIED", "MARRIED", "MARRIED", "MARRIED", "SEPARATED", "DIVORCED", "NEVER MARRIED", "NEVER MARRIED", "DIVORCED", "DIVORCED", "MARRIED")
observed <- c(table(data))
n <- sum(observed)
k <- length(observed)
expProb =rep(1/k, k)
pObs = dmultinom(sort(observed, decreasing=TRUE), size=n, expProb)

counts <- seq(0, n, by = 1)
kCounts <- matrix(,nrow=n+1, ncol=k)
for (i in 1:k){
  kCounts[,i] <- counts
}

all_perm <- merge(kCounts[,1], as.data.frame(kCounts[,2]),all=TRUE)
all_perm <- all_perm[rowSums(all_perm) <= n,]

for (i in 3:k){
  set_to_merge <- data.frame(dummyname=kCounts[,i])
  colnames(set_to_merge) <- paste0("k_",i)
  all_perm <- merge(all_perm, set_to_merge,all=TRUE)
  all_perm <- all_perm[rowSums(all_perm) <= n,]
  print(dim(all_perm))
}
#> [1] 1540    3
#> [1] 8855    4


#WITHOUT FOR LOOP
#column three and four
all_perm <- merge(all_perm, as.data.frame(kCounts[,3]),all=TRUE)
all_perm <- all_perm[rowSums(all_perm) <= n,]
dim(all_perm)
#> [1] 42504     5
#shows correct 3 columns
all_perm <- merge(all_perm, as.data.frame(kCounts[,4]),all=TRUE)
all_perm <- all_perm[rowSums(all_perm) <= n,]
dim(all_perm)
#> [1] 177100      6
#shows correct 4 columns

#THE FOR LOOP THAT DOESN'T WORK
#reset first two columns
all_perm <- merge(kCounts[,1], as.data.frame(kCounts[,2]),all=TRUE)
all_perm <- all_perm[rowSums(all_perm) <= n,]

for (i in 3:k){
  all_perm <- merge(all_perm, as.data.frame(kCounts[,i]),all=TRUE)
  all_perm <- all_perm[rowSums(all_perm) <= n,]
  print(dim(all_perm))
}
#> [1] 1540    3
#> [1] 1540    3
#first iteration good (column 3), fourth not being added.

^{Created on 2023-01-09 with reprex v2.0.2}

[stikpet]

Thanks for this. It solved the issue.

For others, note that this indeed solved my issue, but my code is far from optimized