HIVE入门_1

RUNNING HIVE

$ $HADOOP_HOME/bin/hadoop fs -mkdir       /tmp
$ $HADOOP_HOME/bin/hadoop fs -mkdir       /user/hive/warehouse
$ $HADOOP_HOME/bin/hadoop fs -chmod g+w   /tmp
$ $HADOOP_HOME/bin/hadoop fs -chmod g+w   /user/hive/warehouse
$ export HIVE_HOME=<hive-install-dir>

#Running Hive CLI
$ $HIVE_HOME/bin/hive

#Running HiveServer2 and Beeline
$ $HIVE_HOME/bin/hiveserver2
$ $HIVE_HOME/bin/beeline -u jdbc:hive2://$HS2_HOST:$HS2_PORT

$ $HIVE_HOME/bin/beeline -u jdbc:hive2://

DDL

Data Definition Language

#Creating Hive Tables
hive> CREATE TABLE pokes (foo INT, bar STRING);
hive> CREATE TABLE invites (foo INT, bar STRING) PARTITIONED BY (ds STRING);

#Browsing through Tables
hive> SHOW TABLES;
hive> SHOW TABLES '.*s';  # follows Java regular expressions.
hive> DESCRIBE invites;

#Altering and Dropping Tables
hive> ALTER TABLE events RENAME TO 3koobecaf;
hive> ALTER TABLE pokes ADD COLUMNS (new_col INT);
hive> ALTER TABLE invites ADD COLUMNS (new_col2 INT COMMENT 'a comment');
hive> ALTER TABLE invites REPLACE COLUMNS (foo INT, bar STRING, baz INT COMMENT 'baz replaces new_col2');

hive> ALTER TABLE invites REPLACE COLUMNS (foo INT COMMENT 'only keep the first column');

hive> DROP TABLE pokes;

Metadata Store

Metadata is in an embedded Derby database whose disk storage location is determined by the Hive configuration variable named javax.jdo.option.ConnectionURL. By default this location is ./metastore_db (see conf/hive-default.xml).

DML

Data Manipulation Language

#loading
hive> LOAD DATA LOCAL INPATH './examples/files/kv1.txt' OVERWRITE INTO TABLE pokes;

hive> LOAD DATA LOCAL INPATH './examples/files/kv2.txt' OVERWRITE INTO TABLE invites PARTITION (ds='2008-08-15');
hive> LOAD DATA LOCAL INPATH './examples/files/kv3.txt' OVERWRITE INTO TABLE invites PARTITION (ds='2008-08-08');

#The above command will load data from an HDFS file/directory to the table.
#Note that loading data from HDFS will result in moving the file/directory. As a result, the operation is almost instantaneous.
hive> LOAD DATA INPATH '/user/myname/kv2.txt' OVERWRITE INTO TABLE invites PARTITION (ds='2008-08-15');

SQL

Examples are shown: Some example queries are shown below. They are available in build/dist/examples/queries. More are available in the Hive sources at ql/src/test/queries/positive.

#SELECTS and FILTERS
hive> SELECT a.foo FROM invites a WHERE a.ds='2008-08-15';
hive> INSERT OVERWRITE DIRECTORY '/tmp/hdfs_out' SELECT a.* FROM invites a WHERE a.ds='2008-08-15';
hive> INSERT OVERWRITE LOCAL DIRECTORY '/tmp/local_out' SELECT a.* FROM pokes a;

hive> INSERT OVERWRITE TABLE events SELECT a.* FROM profiles a;
hive> INSERT OVERWRITE TABLE events SELECT a.* FROM profiles a WHERE a.key < 100;
hive> INSERT OVERWRITE LOCAL DIRECTORY '/tmp/reg_3' SELECT a.* FROM events a;
hive> INSERT OVERWRITE DIRECTORY '/tmp/reg_4' select a.invites, a.pokes FROM profiles a;
hive> INSERT OVERWRITE DIRECTORY '/tmp/reg_5' SELECT COUNT(*) FROM invites a WHERE a.ds='2008-08-15';
hive> INSERT OVERWRITE DIRECTORY '/tmp/reg_5' SELECT a.foo, a.bar FROM invites a;
hive> INSERT OVERWRITE LOCAL DIRECTORY '/tmp/sum' SELECT SUM(a.pc) FROM pc1 a;

#GROUP BY
hive> FROM invites a INSERT OVERWRITE TABLE events SELECT a.bar, count(*) WHERE a.foo > 0 GROUP BY a.bar;
hive> INSERT OVERWRITE TABLE events SELECT a.bar, count(*) FROM invites a WHERE a.foo > 0 GROUP BY a.bar;

#JOIN
hive> FROM pokes t1 JOIN invites t2 ON (t1.bar = t2.bar) INSERT OVERWRITE TABLE events SELECT t1.bar, t1.foo, t2.foo;

#MULTITABLE INSERT
FROM src
INSERT OVERWRITE TABLE dest1 SELECT src.* WHERE src.key < 100
INSERT OVERWRITE TABLE dest2 SELECT src.key, src.value WHERE src.key >= 100 and src.key < 200
INSERT OVERWRITE TABLE dest3 PARTITION(ds='2008-04-08', hr='12') SELECT src.key WHERE src.key >= 200 and src.key < 300
INSERT OVERWRITE LOCAL DIRECTORY '/tmp/dest4.out' SELECT src.value WHERE src.key >= 300;

#STREAMING
hive> FROM invites a INSERT OVERWRITE TABLE events SELECT TRANSFORM(a.foo, a.bar) AS (oof, rab) USING '/bin/cat' WHERE a.ds > '2008-08-09';

Simple Example Use Cases

参考资料

  1. get started

本文参与腾讯云自媒体分享计划,欢迎正在阅读的你也加入,一起分享。

发表于

我来说两句

0 条评论
登录 后参与评论

相关文章

来自专栏跟着阿笨一起玩NET

winfrom如何做一个语法着色控件

本文转载:http://www.cnblogs.com/hexin0614/archive/2012/01/17/2324224.html

391
来自专栏跟着阿笨一起玩NET

SQL Server 2008中的Pivot和UnPivot

今天给新成员讲解PIVOT 和 UNPIVOT示例,顺便整理了一下其用法。这是自SQL Server 2005起提供的新功能。

432
来自专栏Python研发

MySql[二]

视图就是一个虚拟表(非真实存在),其本质是[根据SQL语句获取动态的数据集,并为其命名],用户使用时只需要使用[名称]即可获取结果集并可以将其当做代表来使用·

1114
来自专栏杨建荣的学习笔记

sqlplus连接数的测试

最近测试库上会有很多的连接的问题,很多都是jdbc连接的问题,在此基础上我测试了一下sqlplus连接的情况, 根据sqlplus的命令,有conn,disc,...

3469
来自专栏Android知识点总结

SpringBoot-14-MyBatis预热篇,MySQL小结

新建一个测试表:将sword_insert_test中test_name赋值为sword表中id大于5的name

1011
来自专栏抠抠空间

MySQL 如何删除有外键约束的表数据

今天删除数据库中数据,提示因为设置了foreign key,无法修改删除 可以通过设置FOREIGN_KEY_CHECKS变量来避免这种情况。

683
来自专栏Netkiller

新闻数据库分表案例

文章节选自《Netkiller Architect 手札》 6.3. 新闻数据库分表案例 这里我通过一个新闻网站为例,解决分表的问题 避免开发中经常拼接表,我采...

3546
来自专栏数据分析

Database、Table的所有约束

列出Database或Table的所有约束 很多时候我们想使用像 INSERT、UPDATE、DELETE 这样的DML命令。有时候因为某个表被设置约束,导致我...

2654
来自专栏代码小睿

Medoo个人修改版

  Medoo是一款轻量级的php数据库操作类,下面不会介绍Medoo的使用方法,想学习Medoo请前往官网自学:http://medoo.in/   在接触M...

2037
来自专栏web编程技术分享

第三节 - mysql 建表

3325

扫码关注云+社区