在Python语言中使用google.cloud.bigquery写入bigQuery时出现缺少必需参数的错误
在Python语言中使用google.cloud.bigquery写入bigQuery时出现缺少必需参数的错误
提问于 2018-08-21 14:02:45
我正在使用Python2.7中的以下代码片段将新行分隔的JSON加载到bigQuery：
from google.cloud import bigquery
from apiclient.discovery import build
from oauth2client.service_account import ServiceAccountCredentials
bigquery_client = bigquery.Client()
dataset = bigquery_client.dataset('testGAData')
table_ref = dataset.table('gaData')
table = bigquery.Table(table_ref)
with open('gaData.json', 'rb') as source_file:
job_config = bigquery.LoadJobConfig()
job_config.source_format = 'NEWLINE_DELIMITED_JSON'
job = bigquery_client.load_table_from_file(
source_file, table, job_config=job_config)它返回以下错误：
File "/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/google/cloud/bigquery/client.py", line 897, in load_table_from_file
raise exceptions.from_http_response(exc.response)
google.api_core.exceptions.BadRequest: 400 POST https://www.googleapis.com/upload/bigquery/v2/projects/test-project-for-experiments/jobs?uploadType=resumable: Required parameter is missing为什么我会得到这个错误？我该如何解决这个问题呢？其他人有没有遇到过类似的问题？提前谢谢。编辑:添加了最后一段，包含了python导入并更正了缩进。
回答 1
Stack Overflow用户
回答已采纳
发布于 2018-08-23 08:36:05
使用初始代码观察到的问题
- 您缺少表的架构。您可以使用
job_config.autodetect = True或job_config.schema = [bigquery.SchemaField("FIELD NAME", "FIELD TYPE")]. - From文档，您应该为JSON文件源设置
job_config.source_format = `bigquery.SourceFormat.NEWLINE_DELIMITED_JSON` - 您应该在JSON中将
table_ref变量作为参数传递，而不是table变量
文档的Link
工作代码
下面的代码适用于我。我使用的是python 3和google-cloud-bigquery v1.5
from google.cloud import bigquery
client = bigquery.Client()
dataset_id, table_id = "TEST_DATASET", "TEST_TABLE"
data_ref = client.dataset(dataset_id)
table_ref = data_ref.table(table_id)
file_path = "path/to/test.json"
job_config = bigquery.LoadJobConfig()
job_config.source_format = bigquery.SourceFormat.NEWLINE_DELIMITED_JSON
#job_config.autodetect = True
job_config.schema = [bigquery.SchemaField("Name", "STRING"), bigquery.SchemaField("Age", "INTEGER")]
with open(file_path, 'rb') as source_file:
job = client.load_table_from_file(source_file, table_ref, location='US', job_config=job_config)
job.result()
print('Loaded {} rows into {}:{}.'.format(job.output_rows, dataset_id, table_id))输出
>> Loaded 2 rows into TEST_DATASET:TEST_TABLE.
页面原文内容由Stack Overflow提供。腾讯云小微IT领域专用引擎提供翻译支持
原文链接：
https://stackoverflow.com/questions/51942492
复制相关文章
相似问题