我正在编写一个mapReduce作业来读取和处理Avrofile。输入文件为Avro,输出格式为Avro
当我执行Mapreduce作业时,我在reducer阶段得到以下异常。当reducer抛出IOException时,我无法在reducer中捕获和消除它。色调中的错误堆栈跟踪看起来
java.io.IOException: Invalid int encoding
at org.apache.avro.io.DirectBinaryDecoder.readInt(DirectBinaryDecoder.java:113)
at org.apache.avro.io.ValidatingDecoder.readInt(ValidatingDecoder.java:83)
at org.apache.avro.reflect.ReflectDatumReader.readInt(ReflectDatumReader.java:166)
at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:156)
at org.apache.avro.generic.GenericDatumReader.readRecord(GenericDatumReader.java:177)
at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:148)
at org.apache.avro.generic.GenericDatumReader.readArray(GenericDatumReader.java:206)
at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:150)
at org.apache.avro.generic.GenericDatumReader.readRecord(GenericDatumReader.java:177)
at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:148)
at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:139)
at org.apache.avro.hadoop.io.AvroDeserializer.deserialize(AvroDeserializer.
通过谷歌搜索,我注意到有一个apache JIRA票据(https://issues.apache.org/jira/browse/AVRO-882)。没有更新。
我使用的是AVRO-1.7.5,下面是依赖项maven
<dependency>
<groupId>org.apache.avro</groupId>
<artifactId>avro</artifactId>
<version>1.7.5</version>
</dependency>
如有任何帮助,我将不胜感激。谢谢
https://stackoverflow.com/questions/21762960
复制相似问题