前往小程序,Get更优阅读体验!
立即前往
首页
学习
活动
专区
工具
TVP
发布
社区首页 >专栏 >hadoop hive 与 Oracle 互相导入数据

hadoop hive 与 Oracle 互相导入数据

作者头像
用户2836074
发布2018-08-15 09:49:16
8650
发布2018-08-15 09:49:16
举报
文章被收录于专栏:清风清风
代码语言:javascript
复制
#1.在Oracle上建立要处理的表
create table SOURCE_TABLE_NAME as 
SELECT t.*,rownum as row_num FROM SOURCE_TABLE_NAME_O t ;
alter table SOURCE_TABLE_NAME
  add constraint SOURCE_TABLE_NAME_P primary key (ROW_NUM);
#2.在Oracle上建立处理结果表
DEST_TABLE_NAME
#3.oracle导入到hadoop
nohup \
sqoop import \
--hive-import \
--connect jdbc:oracle:thin:@192.168.1.14:1521:oradb1 \
--username USER1 \
--password "密码" \
--verbose -m 32 \
--table SOURCE_TABLE_NAME \
--hive-table db_hive.SOURCE_TABLE_NAME \
--fields-terminated-by '\t' \
--lines-terminated-by '\n' &
#4.计算
hive -e "drop table db_hive.DEST_TABLE_NAME ;
create table db_hive.DEST_TABLE_NAME row format delimited fields terminated by '\t' STORED AS TEXTFILE as \
select \
max(Dn)  as  Dn, \
EutranCellTdd_uk , \
max(EutranCellTdd_name)  as  EutranCellTdd_name, \
max(GsmRelation) as GsmRelation, \
adj_uk, \
max(adj_name) as adj_name, \
max(EnbFunction_uk)  as  EnbFunction_uk, \
max(EnbFunction_name)  as  EnbFunction_name, \
max(ManagedElement_uk)  as  ManagedElement_uk, \
max(ManagedElement_name)  as  ManagedElement_name, \
max(omc_uk)  as  omc_uk, \
max(omc_name)  as  omc_name, \
sum(HO_ToGsmAttOutPerRelation)         as HO_ToGsmAttOutPerRelation, \
sum(HO_ToGsmSuccOutPrepPerRelation)    as HO_ToGsmSuccOutPrepPerRelation, \
sum(HO_ToGsmSuccOutPerRelation)        as HO_ToGsmSuccOutPerRelation, \
max(VENDOR_UK)  as  VENDOR_UK, \
max(VENDOR_NAME)  as  VENDOR_NAME, \
max(city_name) as city_name \
from db_hive.SOURCE_TABLE_NAME \
group by EutranCellTdd_uk, adj_uk ;"
#5.hadoop导出oracle(需要清空目标表DEST_TABLE_NAME)
sqoop export \
--connect jdbc:oracle:thin:@192.168.1.14:1521:oradb1 \
--username USER1 \
--password "密码" \
--table DEST_TABLE_NAME \
--export-dir /user/hive/warehouse/db_hive.db/DEST_TABLE_NAME \
--input-fields-terminated-by '\t' \
--input-lines-terminated-by '\n' \
--null-string '\\N' \
--null-non-string '\\N'
本文参与 腾讯云自媒体同步曝光计划,分享自作者个人站点/博客。
原始发表:2016/12/16 ,如有侵权请联系 cloudcommunity@tencent.com 删除

本文分享自 作者个人站点/博客 前往查看

如有侵权,请联系 cloudcommunity@tencent.com 删除。

本文参与 腾讯云自媒体同步曝光计划  ,欢迎热爱写作的你一起参与!

评论
登录后参与评论
0 条评论
热度
最新
推荐阅读
领券
问题归档专栏文章快讯文章归档关键词归档开发者手册归档开发者手册 Section 归档