blocks|key|4272|text|如果您有配置单元设置，则可以使用hdfs/s3中的配置单元加载命令直接放置本地数据集。|type|unstyled|depth|inlineStyleRanges|entityRanges|data|4273|在编写加载命令时，您需要使用"Local“关键字。|4274|hiveload命令的语法|4275|LOAD+DATA+[LOCAL]+INPATH+'filepath'+[OVERWRITE]+INTO+TABLE+tablename+[PARTITION+(partcol1=val1,+partcol2=val2+...)]|code-block|syntax|javascript|4276|有关更多详细信息，请参阅以下链接。https://cwiki.apache.org/confluence/display/Hive/LanguageManual%2520DML#LanguageManualDML-Loadingfilesintotables|offset|length|4277|entityMap|0|LINK|mutability|MUTABLE|url|https://cwiki.apache.org/confluence/display/Hive/LanguageManual%2520DML#LanguageManualDML-Loadingfilesintotables^0|0|0|0|0|H|32|0|0^^$0|@$1|2|3|4|5|6|7|W|8|@]|9|@]|A|$]]|$1|B|3|C|5|6|7|X|8|@]|9|@]|A|$]]|$1|D|3|E|5|6|7|Y|8|@]|9|@]|A|$]]|$1|F|3|G|5|H|7|Z|8|@]|9|@]|A|$I|J]]|$1|K|3|L|5|6|7|10|8|@]|9|@$M|11|N|12|1|13]]|A|$]]|$1|O|3|-4|5|6|7|14|8|@]|9|@]|A|$]]]|P|$Q|$5|R|S|T|A|$U|V]]]]

if you have a hive setup you can put the local dataset directly using Hive load command in hdfs/s3. 

You will need to use "Local" keyword when writing your load command.

Syntax for hiveload command

<pre><code>LOAD DATA [LOCAL] INPATH 'filepath' [OVERWRITE] INTO TABLE tablename [PARTITION (partcol1=val1, partcol2=val2 ...)]
</code></pre>

Refer below link for more detailed information.
<a href="https://cwiki.apache.org/confluence/display/Hive/LanguageManual%20DML#LanguageManualDML-Loadingfilesintotables">https://cwiki.apache.org/confluence/display/Hive/LanguageManual%20DML#LanguageManualDML-Loadingfilesintotables</a>

blocks|key|4337|text|让我为您完成以下简单的步骤：|type|unstyled|depth|inlineStyleRanges|entityRanges|data|4338|步骤：|4339|首先，使用csv文件中的字段名称在hive上创建一个表。例如，假设您的csv文件包含三个字段(id、name、salary)，并且您希望在配置单元中创建一个名为"staff“的表。使用下面的代码在配置单元中创建表。|4340|hive>+CREATE+TABLE+Staff+(id+int,+name+string,+salary+double)+row+format+delimited+fields+terminated+by+',';|code-block|syntax|javascript|4341|其次，现在您的表已在hive中创建，让我们将csv文件中的数据加载到hive上的"staff“表中。|4342|hive>++LOAD+DATA+LOCAL+INPATH+'/home/yourcsvfile.csv'+OVERWRITE+INTO+TABLE+Staff;|4343|最后，在hive上显示"Staff“表的内容，以检查数据是否已成功加载|4344|hive>+SELECT+*+FROM+Staff;|4345|谢谢。|4346|entityMap^0|0|0|0|0|0|0|0|0|0^^$0|@$1|2|3|4|5|6|7|W|8|@]|9|@]|A|$]]|$1|B|3|C|5|6|7|X|8|@]|9|@]|A|$]]|$1|D|3|E|5|6|7|Y|8|@]|9|@]|A|$]]|$1|F|3|G|5|H|7|Z|8|@]|9|@]|A|$I|J]]|$1|K|3|L|5|6|7|10|8|@]|9|@]|A|$]]|$1|M|3|N|5|H|7|11|8|@]|9|@]|A|$I|J]]|$1|O|3|P|5|6|7|12|8|@]|9|@]|A|$]]|$1|Q|3|R|5|H|7|13|8|@]|9|@]|A|$I|J]]|$1|S|3|T|5|6|7|14|8|@]|9|@]|A|$]]|$1|U|3|-4|5|6|7|15|8|@]|9|@]|A|$]]]|V|$]]

Let me work you through the following simple steps:

Steps:

First, create a table on hive using the field names in your csv file. Lets say for example, your csv file contains three fields (id, name, salary) and you want to create a table in hive called "staff". Use the below code to create the table in hive.

<pre><code>hive&gt; CREATE TABLE Staff (id int, name string, salary double) row format delimited fields terminated by ',';
</code></pre>

Second, now that your table is created in hive, let us load the data in your csv file to the "staff" table on hive.

<pre><code>hive&gt; LOAD DATA LOCAL INPATH '/home/yourcsvfile.csv' OVERWRITE INTO TABLE Staff;
</code></pre>

Lastly, display the contents of your "Staff" table on hive to check if the data were successfully loaded

<pre><code>hive&gt; SELECT * FROM Staff;
</code></pre>

Thanks.

blocks|key|221129|text|对于csv文件格式，数据将采用以下格式|type|unstyled|depth|inlineStyleRanges|entityRanges|data|221130|"column1",+"column2","column3","column4"|code-block|syntax|javascript|221131|如果我们使用以'，‘结尾的字段，那么每一列都会得到如下所示的值。|221132|"column1"++++"column2"+++++"column3"+++++"column4"|221133|此外，如果任何列值都以comma作为值，那么它将根本不起作用。|offset|length|style|CODE|221134|因此，创建表的正确方法是使用OpenCSVSerde|221135|create+table+tableName+(column1+datatype,+column2+datatype+,+column3+datatype+,+column4+datatype)
ROW+FORMAT+SERDE+
'org.apache.hadoop.hive.serde2.OpenCSVSerde'+
STORED+AS+TEXTFILE+;|221136|entityMap^0|0|0|0|0|B|5|0|0|0^^$0|@$1|2|3|4|5|6|7|W|8|@]|9|@]|A|$]]|$1|B|3|C|5|D|7|X|8|@]|9|@]|A|$E|F]]|$1|G|3|H|5|6|7|Y|8|@]|9|@]|A|$]]|$1|I|3|J|5|D|7|Z|8|@]|9|@]|A|$E|F]]|$1|K|3|L|5|6|7|10|8|@$M|11|N|12|O|P]]|9|@]|A|$]]|$1|Q|3|R|5|6|7|13|8|@]|9|@]|A|$]]|$1|S|3|T|5|D|7|14|8|@]|9|@]|A|$E|F]]|$1|U|3|-4|5|6|7|15|8|@]|9|@]|A|$]]]|V|$]]

For csv file formate data will be in below format 

<pre><code>"column1", "column2","column3","column4"
</code></pre>

And if we will use field terminated by ',' then each column will get values like below. 

<pre><code>"column1" "column2" "column3" "column4"
</code></pre>

also if any of the column value has <code>comma</code> as value then it will not work at all .

So the correct way to create a table would be by using OpenCSVSerde

<pre><code>create table tableName (column1 datatype, column2 datatype , column3 datatype , column4 datatype)
ROW FORMAT SERDE 
'org.apache.hadoop.hive.serde2.OpenCSVSerde' 
STORED AS TEXTFILE ;
</code></pre>

As this is coming from a newbie...

I had Hadoop and Hive set up for me, so I can run Hive queries on my computer accessing data on AWS cluster.
Can I run Hive queries with .csv data stored on my computer, like I did with MS SQL Server? 

How do I load .csv data into Hive then? What does it have to do with Hadoop and which mode I should run that one?

What settings I should care about so that if I did something wrong I can always go back and run queries on Amazon without compromising what was set up for me earlier?

Hadoop/Hive : Loading data from .csv on a local machine

SQL Server

Hadoop 

Hive

因为这是一个新手说的。我已经为我设置了Hadoop和Hive，因此我可以在我的计算机上运行Hive查询，以访问AWS集群上的数据。我是否可以像使用MS SQL Server一样，使用存储在计算机上的.csv数据运行配置单元查询？那么如何将.csv数据加载到配置单元中呢？它与Hadoop有什么关系?我应该运行哪种模式？我...

问Hadoop/Hive :在本地机器上从.csv加载数据
EN

社区

活动

资源

关于

腾讯云开发者

热门产品

热门推荐

更多推荐

问Hadoop/Hive :在本地机器上从.csv加载数据EN

社区

活动

资源

关于

腾讯云开发者

热门产品

热门推荐

更多推荐

问Hadoop/Hive :在本地机器上从.csv加载数据
EN