Uploading CSV from Google Storage with nulls

General
1

Hi there,

I'm trying to load a CSV file from google cloud storage to BigQuery. Problem is that the file has nulls for fields that are integers.

I'm using the following

bq_table_load(bq_Temp, bucket_uri, source_format ="CSV", fields = fields_spec, quiet=FALSE)

but the process is erroring out with a bunch of errors like "Could not parse Field as INT64 'with message 'Unable to parse'

Seems BigQuery can't handle nulls unless the field is string. But I'd love to avoid having to re-parse the file when it's in BigQuery. I've tried the autodetect function but the field names come out labelled as 'String1', 'String2', etc. Which is no good either.

Using BigRQuery and BigQueryR

2

After lots of trial and error I finally resolved this by including nskip=1. The error was actually that BigQuery was attempting to parse the header row of the CSV as the data. Which obviously didn't work on numeric columns.

bq_table_load(bq_Temp,bucket_uri, source_format ="CSV", fields = fields_spec, nskip = 1, quiet=FALSE)

1 Like
3

This topic was automatically closed 21 days after the last reply. New replies are no longer allowed.

If you have a query related to it or one of the replies, start a new topic and refer back with a link.