Error message when trying to loop through each file

[BellAM]

I am trying to create a data set from raw data. Below I have put the error code and the code I have used so far. I have checked and 'thisCond' matches with the column in the csv file. What have I done wrong?

Error in select():
! Can't select columns that don't exist.
Column thisCond doesn't exist.
Run rlang::last_trace() to see where the error occurred.

rlang::last_trace()
<error/vctrs_error_subscript_oob>
Error in select():
! Can't select columns that don't exist.
Column thisCond doesn't exist.

---

Backtrace:
▆

1. ├─temp_data %>% ...
2. ├─dplyr::select(...)
3. └─dplyr:::select.data.frame(., thisCond, corrAns, key_resp_4.keys, key_resp_4.rt, thisWord)
   Run rlang::last_trace(drop = FALSE) to see 17 hidden frames.

rlang::last_trace(drop = FALSE)
<error/vctrs_error_subscript_oob>
Error in select():
! Can't select columns that don't exist.
Column thisCond doesn't exist.

---

Backtrace:
▆

1. ├─temp_data %>% ...
2. ├─dplyr::select(...)
3. ├─dplyr:::select.data.frame(., thisCond, corrAns, key_resp_4.keys, key_resp_4.rt, thisWord)
4. │ └─tidyselect::eval_select(expr(c(...)), data = .data, error_call = error_call)
5. │ └─tidyselect:::eval_select_impl(...)
6. │ ├─tidyselect:::with_subscript_errors(...)
7. │ │ └─base::withCallingHandlers(...)
8. │ └─tidyselect:::vars_select_eval(...)
9. │ └─tidyselect:::walk_data_tree(expr, data_mask, context_mask)
10. │ └─tidyselect:::eval_c(expr, data_mask, context_mask)
11. │ └─tidyselect:::reduce_sels(node, data_mask, context_mask, init = init)
12. │ └─tidyselect:::walk_data_tree(new, data_mask, context_mask)
13. │ └─tidyselect:::as_indices_sel_impl(...)
14. │ └─tidyselect:::as_indices_impl(...)
15. │ └─tidyselect:::chr_as_locations(x, vars, call = call, arg = arg)
16. │ └─vctrs::vec_as_location(...)
17. └─vctrs (local) <fn>()
18. └─vctrs:::stop_subscript_oob(...)

19. └─vctrs:::stop_subscript(...)
    

20.   └─rlang::abort(...)
    

Here is the code I have used so far:

# Loop through each file
for (file in file_list) {
  
  # Read the .csv file
  temp_data <- read.csv(file, header = TRUE)  ## Let's use the posder labels from the results files
  demo_data <- temp_data  ## We will use this for demographics
  
  # Select the desired columns (e.g., "Column1" and "Column2")
  selected_data <- temp_data %>%
    select(thisCond, corrAns, key_resp_4.keys, key_resp_4.rt, thisWord)  # Replace with the names of the columns you want
  selected_data <- selected_data[13:132, ]  # Replace with the row numbers that you want
  
  # Add a new column with the filename (excluding path and extension) as it adds the participant ID
  selected_data$SourceFile <- basename(file)
  selected_data$SourceFile <- gsub("\\.csv$", "", selected_data$SourceFile)  # Remove the .csv extension
  
  # Select demographic info and save it to selected_demo keeping only row2 (should be same for all group experiments)
  selected_demo <- demo_data %>%
    select(background_survey.question4, background_survey.question6, 
           background_survey.question1, background_survey.question2)
  selected_demo <- selected_demo[2, ]
  
  # Append to the combined dataframe
  combined_data <- rbind(combined_data, selected_data)  ## You add to data from files already analyzed
  combined_demo <- rbind(combined_demo, selected_demo)  ## This is a loop through each file
}

[prubin]

If you insert print(names(temp_data)) right after the call to read.csv, does it print the name "thisCond" each time through the loop?

[BellAM]

Hi, thanks for your reply. Turns out the problem was because there were two empty csv files. They've been removed now, and I've re-run it and thank goodness there is no error message this time. R must have noticed that the two files had no headings in compared with the other correct files and it threw a wobbly!