blocks|key|1452412|text|从CSV+RFC4180页面：“如果双引号用于括住字段，那么在字段中出现的双引号必须用另一个双引号来转义。”|type|unstyled|depth|inlineStyleRanges|entityRanges|offset|length|data|1452413|你可能想这么做：|1452414|This+%7C+is+%7C+test+only+%7C+to+check+quotes
second+%7C+line+%7C+"with+quotes"+%7C+no+text
third+line+%7C+with+%7C+"+""start+quote""+and+"+%7C+a+word+after+quotes|code-block|syntax|javascript|1452415|更多关于我们的CSV输入格式这里。|1452416|entityMap|0|LINK|mutability|MUTABLE|url|https://www.rfc-editor.org/rfc/rfc4180|1|https://developers.google.com/bigquery/docs/developers_guide#csvformat^0|1|D|0|0|0|0|E|2|1|0^^$0|@$1|2|3|4|5|6|7|W|8|@]|9|@$A|X|B|Y|1|Z]]|C|$]]|$1|D|3|E|5|6|7|10|8|@]|9|@]|C|$]]|$1|F|3|G|5|H|7|11|8|@]|9|@]|C|$I|J]]|$1|K|3|L|5|6|7|12|8|@]|9|@$A|13|B|14|1|15]]|C|$]]|$1|M|3|-4|5|6|7|16|8|@]|9|@]|C|$]]]|N|$O|$5|P|Q|R|C|$S|T]]|U|$5|P|Q|R|C|$S|V]]]]

From the <a href="https://www.rfc-editor.org/rfc/rfc4180" rel="nofollow noreferrer">CSV RFC4180 page</a>: &quot;If double-quotes are used to enclose fields, then a double-quote appearing inside a field must be escaped by preceding it with another double quote.&quot;
You probably want to do this:
<pre><code>This | is | test only | to check quotes
second | line | &quot;with quotes&quot; | no text
third line | with | &quot; &quot;&quot;start quote&quot;&quot; and &quot; | a word after quotes
</code></pre>
More about our CSV input format <a href="https://developers.google.com/bigquery/docs/developers_guide#csvformat" rel="nofollow noreferrer">here</a>.

blocks|key|1928836|text|您也可以在上传数据时使用其他标志。我使用了bq工具和下面的标志|type|unstyled|depth|inlineStyleRanges|entityRanges|data|1928837|bq+load+-F+,+--source_format+CSV+--skip_leading_rows+1+--max_bad_records+1+--format+csv+-E+UTF-8+yourdatset+gs://datalocation.|code-block|syntax|javascript|1928838|entityMap^0|0|0^^$0|@$1|2|3|4|5|6|7|I|8|@]|9|@]|A|$]]|$1|B|3|C|5|D|7|J|8|@]|9|@]|A|$E|F]]|$1|G|3|-4|5|6|7|K|8|@]|9|@]|A|$]]]|H|$]]

You can use the other flags also while uploading the data. I used the bq tool with following flags

<pre><code>bq load -F , --source_format CSV --skip_leading_rows 1 --max_bad_records 1 --format csv -E UTF-8 yourdatset gs://datalocation.
</code></pre>

blocks|key|1928846|text|API+V2+https://cloud.google.com/bigquery/docs/reference/v2/jobs#configuration.load.quote|type|unordered-list-item|depth|inlineStyleRanges|entityRanges|offset|length|data|1928847|bq命令-引号:引号字符用于括住记录。默认值为“。若要表示根本没有引号字符，请使用空字符串。|1928848|unstyled|entityMap|0|LINK|mutability|MUTABLE|url|https://cloud.google.com/bigquery/docs/reference/v2/jobs#configuration.load.quote^0|7|29|0|0|0^^$0|@$1|2|3|4|5|6|7|O|8|@]|9|@$A|P|B|Q|1|R]]|C|$]]|$1|D|3|E|5|6|7|S|8|@]|9|@]|C|$]]|$1|F|3|-4|5|G|7|T|8|@]|9|@]|C|$]]]|H|$I|$5|J|K|L|C|$M|N]]]]

<ul>
<li>API V2
<a href="https://cloud.google.com/bigquery/docs/reference/v2/jobs#configuration.load.quote" rel="nofollow">https://cloud.google.com/bigquery/docs/reference/v2/jobs#configuration.load.quote</a></li>
<li>bq command
--quote: Quote character to use to enclose records. Default is ". To indicate no quote character at all, use an empty string.</li>
</ul>

blocks|key|1452479|text|使用--quote非常有效。|type|unstyled|depth|inlineStyleRanges|offset|length|style|CODE|entityRanges|data|1452480|bq+load+
--source_format+CSV+--quote+""+
--field_delimiter+\t+
--max_bad_records+10+
-E+UTF-8+++
destination+table
Source+files+|code-block|syntax|javascript|1452481|entityMap^0|2|7|0|0^^$0|@$1|2|3|4|5|6|7|M|8|@$9|N|A|O|B|C]]|D|@]|E|$]]|$1|F|3|G|5|H|7|P|8|@]|D|@]|E|$I|J]]|$1|K|3|-4|5|6|7|Q|8|@]|D|@]|E|$]]]|L|$]]

Using <code>--quote</code> worked perfectly.

<pre><code>bq load 
--source_format CSV --quote "" 
--field_delimiter \t 
--max_bad_records 10 
-E UTF-8 
destination table
Source files 
</code></pre>

blocks|key|956548|text|每次都尝试用bq+shell加载。|type|unstyled|depth|inlineStyleRanges|offset|length|style|CODE|entityRanges|data|956549|我必须加载1100列。当尝试使用带有所有错误选项的控制台时，它抛出了许多错误。忽略控制台中的错误意味着丢失记录。|956550|因此，使用shell尝试并成功地加载了所有记录。|956551|尝试以下几点：|956552|bq+load+--source_format+CSV+--quote+""+--field_delimiter+\t+--allow_jagged_rows+--ignore_unknown_values+--allow_quoted_newlines+--max_bad_records+10+-E+UTF-8+{dataset_name}.{table_name}+gs://{google_cloud_storage_location}/*+{col_1}:{data_type1},{col_2}:{data_type2},+....|code-block|syntax|javascript|956553|参考文献：|956554|https://cloud.google.com/bigquery/docs/loading-data-cloud-storage-csv#bigquery_load_table_gcs_csv-cli

https://cloud.google.com/bigquery/docs/loading-data-cloud-storage-csv#csv-options|956555|entityMap^0|6|2|0|0|0|0|0|0|0^^$0|@$1|2|3|4|5|6|7|W|8|@$9|X|A|Y|B|C]]|D|@]|E|$]]|$1|F|3|G|5|6|7|Z|8|@]|D|@]|E|$]]|$1|H|3|I|5|6|7|10|8|@]|D|@]|E|$]]|$1|J|3|K|5|6|7|11|8|@]|D|@]|E|$]]|$1|L|3|M|5|N|7|12|8|@]|D|@]|E|$O|P]]|$1|Q|3|R|5|6|7|13|8|@]|D|@]|E|$]]|$1|S|3|T|5|N|7|14|8|@]|D|@]|E|$O|P]]|$1|U|3|-4|5|6|7|15|8|@]|D|@]|E|$]]]|V|$]]

Try loading every time with <code>bq</code> shell.

I had to load 1100 columns. While trying with the console with all the error options, it threw lot many errors. Ignoring the errors in the console means loosing records. 

Hence tried with the shell and succeeded loading all the records. 

Try the following:

<pre><code>bq load --source_format CSV --quote "" --field_delimiter \t --allow_jagged_rows --ignore_unknown_values --allow_quoted_newlines --max_bad_records 10 -E UTF-8 {dataset_name}.{table_name} gs://{google_cloud_storage_location}/* {col_1}:{data_type1},{col_2}:{data_type2}, ....
</code></pre>

References:

<pre><code>https://cloud.google.com/bigquery/docs/loading-data-cloud-storage-csv#bigquery_load_table_gcs_csv-cli

https://cloud.google.com/bigquery/docs/loading-data-cloud-storage-csv#csv-options
</code></pre>

blocks|key|956562|text|将此作为另一种选择：|type|unstyled|depth|inlineStyleRanges|entityRanges|data|956563|将MySQL备份文件加载到Cloud实例中。|unordered-list-item|956564|直接从BigQuery中读取MySQL中的数据。|956565|更长的如何-到：|956566|https://medium.com/google-cloud/loading-mysql-backup-files-into-bigquery-straight-from-cloud-sql-d40a98281229|offset|length|956567|entityMap|0|LINK|mutability|MUTABLE|url^0|0|0|0|0|0|31|0|0^^$0|@$1|2|3|4|5|6|7|T|8|@]|9|@]|A|$]]|$1|B|3|C|5|D|7|U|8|@]|9|@]|A|$]]|$1|E|3|F|5|D|7|V|8|@]|9|@]|A|$]]|$1|G|3|H|5|6|7|W|8|@]|9|@]|A|$]]|$1|I|3|J|5|D|7|X|8|@]|9|@$K|Y|L|Z|1|10]]|A|$]]|$1|M|3|-4|5|6|7|11|8|@]|9|@]|A|$]]]|N|$O|$5|P|Q|R|A|$S|J]]]]

Try this as an alternative:

<ul>
<li>Load the MySQL backup files into a Cloud SQL instance.</li>
<li>Read the data in BigQuery straight out of MySQL.</li>
</ul>

Longer how-to: 

<ul>
<li><a href="https://medium.com/google-cloud/loading-mysql-backup-files-into-bigquery-straight-from-cloud-sql-d40a98281229" rel="nofollow noreferrer">https://medium.com/google-cloud/loading-mysql-backup-files-into-bigquery-straight-from-cloud-sql-d40a98281229</a></li>
</ul>

In the example given below, the last line is not uploaded. I get an error:

<pre><code>Data between close double quote (") and field separator: 
</code></pre>

This looks like a bug since all the data between pipe symbol should be treated as a single field.

Schema: one:string,two:string,three:string,four:string

Upload file:

<pre><code>This | is | test only | to check quotes
second | line | "with quotes" | no text
third line | with | "start quote" and | a word after quotes
</code></pre>

The first and second line above is processed. But not the third.

<hr>

Update:

Can some please explain why does the following work except the third line?

<pre><code>This | is | test only | to check quotes
second | line | "with quotes" | no text
third line | with | "start quote" and | a word after quotes
forth line | enclosed | {"GPRS","MCC_DETECTED":false,"MNC_DETECTED":false} | how does this work?
fifth line | with | {"start quote"} and | a word after quotes
</code></pre>

There can be some fancy explanation to this. From the end user perspective this is absurd.

Data between quotes and field separator

翻译质量差，导致语言生硬或混乱。

没有提供实际的解决方法或示例。

解答不清晰，无法理解或解决问题。

页面排版不美观，阅读体验差。

文章

问答

视频

学习中心

腾讯云实验室

直播

竞赛

腾讯云代码分析专区

腾讯iOA零信任安全管理系统专区

腾讯云架构师技术同盟交流圈

腾讯云数据库专区

腾讯云顾问专区

腾讯云原生专区

腾讯混元专区

腾讯云TCE专区

腾讯云Lighthouse专区

腾讯云HAI专区

腾讯云Edgeone专区

腾讯云存储专区

腾讯云智能专区

腾讯轻联专区 

腾讯云开发专区

TAPD专区

腾讯轻量云游戏服专区

腾讯云最具价值专家

腾讯云架构师技术同盟

腾讯云创作之星

腾讯云开发者先锋

腾讯云代码助手

云原生构建

TAPD 敏捷项目管理

Cloud Studio

SDK中心

API中心

命令行工具

涵盖代码开发、场景应用、自动测试全流程，助你从零构建专属AI助手

一站式MCP教程库，解锁AI应用新玩法

在下面给出的示例中，最后一行未上载。我收到一个错误：Data between close double quote (") and field separator: 这看起来像一个bug，因为管道符号之间的所有数据都应该作为一个字段来处理。模式：1:string，2:string，3:string，4:string上传...

问引号和字段分隔符之间的数据
EN

Stack Overflow用户

社区

活动

圈层

关于

腾讯云开发者

热门产品

热门推荐

更多推荐

问引号和字段分隔符之间的数据EN