sqoop 常见错误以及处理方式

Oracle: Connection Reset Errors 错误代码

11/05/26 16:23:47 INFO mapred.JobClient: Task Id : attempt_201105261333_0002_m_000002_0, Status : FAILED java.lang.RuntimeException: java.lang.RuntimeException: java.sql.SQLRecoverableException: IO Error: Connection reset at com.cloudera.sqoop.mapreduce.db.DBInputFormat.setConf(DBInputFormat.java:164) at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:62) at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:605) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:322) at org.apache.hadoop.mapred.Child$4.run(Child.java:268) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1115) at org.apache.hadoop.mapred.Child.main(Child.java:262) Caused by: java.lang.RuntimeException: java.sql.SQLRecoverableException: IO Error: Connection reset at com.cloudera.sqoop.mapreduce.db.DBInputFormat.getConnection(DBInputFormat.java:190) at com.cloudera.sqoop.mapreduce.db.DBInputFormat.setConf(DBInputFormat.java:159) ... 9 more

这个问题的产生原因是由于缺少一个生成随机数的设备 找到 $JAVA_HOME/jre/lib/security目录下的java.security文件 修改securerandom.source=file:/dev/../dev/urandom Oracle: Case-Sensitive Catalog Query Errors 下面是错误代码

1/09/21 17:18:49 INFO manager.OracleManager: Time zone has been set to GMT 11/09/21 17:18:49 DEBUG manager.SqlManager: Using fetchSize for next query: 1000 11/09/21 17:18:49 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM addlabel_pris t WHERE 1=0 11/09/21 17:18:49 DEBUG manager.OracleManager$ConnCache: Caching released connection for jdbc:oracle:thin: 11/09/21 17:18:49 ERROR sqoop.Sqoop: Got exception running Sqoop: java.lang.NullPointerException java.lang.NullPointerException at com.cloudera.sqoop.hive.TableDefWriter.getCreateTableStmt(TableDefWriter.java:148) at com.cloudera.sqoop.hive.HiveImport.importTable(HiveImport.java:187) at com.cloudera.sqoop.tool.ImportTool.importTable(ImportTool.java:362) at com.cloudera.sqoop.tool.ImportTool.run(ImportTool.java:423) at com.cloudera.sqoop.Sqoop.run(Sqoop.java:144) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65) at com.cloudera.sqoop.Sqoop.runSqoop(Sqoop.java:180) at com.cloudera.sqoop.Sqoop.runTool(Sqoop.java:219) at com.cloudera.sqoop.Sqoop.runTool(Sqoop.java:228) at com.cloudera.sqoop.Sqoop.main(Sqoop.java:237)

解决方式:用户名和表名全用大写 Oracle: ORA-00933 error (SQL command not properly ended) 如果没有指定的话,sqoop会默认使用oracle.jdbc.OracleDriver Omit选项 --driver oracle.jdbc.driver.OracleDriver 然后重新提交 MySQL: Import of TINYINT(1) from MySQL behaves strangely jdbc会把tinyint(1)认为是java.sql.Types.BIT,然后sqoop就会转为Boolean了,悲剧吧 解决方法:在连接上加上一句话tinyInt1isBit=false jdbc:mysql://localhost/test?tinyInt1isBit=false 另一种解决方式是 hive使用 --map-column-hive foo=tinyint 非hive使用--map-column-java foo=integer

本文参与腾讯云自媒体分享计划,欢迎正在阅读的你也加入,一起分享。

发表于

我来说两句

0 条评论
登录 后参与评论

相关文章

来自专栏竹清助手

安装MariaDB 无法登录解惑

Debian 9 installed mariadb-server , the phpmyadmin can't login and terminal too,...

1473
来自专栏Golang语言社区

Knapsack problem algorithms for my real-life carry-on knapsack

I'm a nomad and live out of one carry-on bag. This means that the total weight o...

1392
来自专栏木宛城主

Fix SharePoint 2013 Site in Read only mode after an interrupted backup

Problem When I was backing up SharePoint Site Collection Automatically with Powe...

2368
来自专栏算法修养

POJ-2184 Cow Exhibition(01背包变形)

Cow Exhibition Time Limit: 1000MS Memory Limit: 65536K Total Submission...

29310
来自专栏Linyb极客之路

你该怎么做?当你发现springboot扫描controller没有生效

加上@SpringBootApplication为啥就能扫描到,其实我也不知道。那就看下这个注解到底是个啥样的奇葩,它的注解构造了解一下

3.7K3
来自专栏闻道于事

商城项目整理(三)JDBC增删改查

商品表的增加,修改,删除,订单表的增加,确认,用户表的查看,日志表的增加,查看 商品表建表语句: 1 create table TEST.GOODS_TABL...

6685
来自专栏数据结构与算法

BZOJ1053: [HAOI2007]反素数ant(爆搜)

  对于任何正整数x,其约数的个数记作g(x)。例如g(1)=1、g(6)=4。如果某个正整数x满足:g(x)>g(i) 0<i<x

952
来自专栏西二旗一哥

Clang - Use Xcode to make first Clang plugin

2245
来自专栏Pulsar-V

OpenCV编译参数一览

全部参数一览 //Path to a program. ANT_EXECUTABLE:FILEPATH=D:/apache-ant-1.10.1/bin/ant...

7796
来自专栏一个会写诗的程序员的博客

Caused by: android.os.NetworkOnMainThreadException

Caused by: android.os.NetworkOnMainThreadException at android.os.StrictMode$And...

1242

扫码关注云+社区

领取腾讯云代金券