Hadoop源码分析:Writable接口和WritableComparable接口

1、 Writable接口

由于Hadoop的MapReduce和HDFS都有通信的需求,需要对通信的对象进行序列化。Hadoop并没有采用Java的序列化,而是引入了它自己的系统。 org.apache.hadoop.io中定义了大量的可序列化对象,他们都实现了Writable接口。

1.1 Writable接口源码

package org.apache.hadoop.io;

import java.io.DataOutput;
import java.io.DataInput;
import java.io.IOException;
import org.apache.hadoop.classification.InterfaceAudience;
import org.apache.hadoop.classification.InterfaceStability;


@InterfaceAudience.Public
@InterfaceStability.Stable
public interface Writable {
  /** 
   * Serialize the fields of this object to <code>out</code>.
   * 
   */
  void write(DataOutput out) throws IOException;

  /** 
   * Deserialize the fields of this object from <code>in</code>.  
   * 
   */
  void readFields(DataInput in) throws IOException;
}

1.2 自定义Writable实现类

实现了Writable接口的一个典型例子如下,该代码实际上是在Writable接口源码的注释中。

public class MyWritable implements Writable {
       // Some data     
       private int counter;
        private long timestamp;

        public void write(DataOutput out) throws IOException {
         out.writeInt(counter);
         out.writeLong(timestamp);
       }

       public void readFields(DataInput in) throws IOException {
        counter = in.readInt();
         timestamp = in.readLong();
       }

      public static MyWritable read(DataInput in) throws IOException {
        MyWritable w = new MyWritable();
        w.readFields(in);
        return w;
       }
    }

2 WritableComparable接口

WritableComparable是Writable接口和java.lang.Comparable的一个子接口。

2.1 WritableComparable接口源码

package org.apache.hadoop.io;
import org.apache.hadoop.classification.InterfaceAudience;
import org.apache.hadoop.classification.InterfaceStability;

@InterfaceAudience.Public
@InterfaceStability.Stable
public interface WritableComparable<T> extends Writable, Comparable<T> {
}

2.2 WritableComparable一个典型实现类

 public class MyWritableComparable implements WritableComparable<MyWritableComparable> {
       // Some data
        private int counter;
        private long timestamp;

        public void write(DataOutput out) throws IOException {
          out.writeInt(counter);
          out.writeLong(timestamp);
        }

        public void readFields(DataInput in) throws IOException {
          counter = in.readInt();
          timestamp = in.readLong();
        }

       public int compareTo(MyWritableComparable o) {
          int thisValue = this.value;
          int thatValue = o.value;
          return (thisValue &lt; thatValue ? -1 : (thisValue==thatValue ? 0 : 1));
        }

      public int hashCode() {
         final int prime = 31;
         int result = 1;
         result = prime * result + counter;
         result = prime * result + (int) (timestamp ^ (timestamp &gt;&gt;&gt; 32));
         return result
       }
     }

2.3 IntWritable

IntWritable实现了WritableComparable。

package org.apache.hadoop.io;
import java.io.DataInput;
import java.io.DataOutput;
import java.io.IOException;
import org.apache.hadoop.classification.InterfaceAudience;
import org.apache.hadoop.classification.InterfaceStability;


@InterfaceAudience.Public
@InterfaceStability.Stable
public class IntWritable implements WritableComparable<IntWritable> {
  private int value;

  public IntWritable() {}

  public IntWritable(int value) { set(value); }

  /** Set the value of this IntWritable. */
  public void set(int value) { this.value = value; }

  /** Return the value of this IntWritable. */
  public int get() { return value; }

  @Override
  public void readFields(DataInput in) throws IOException {
    value = in.readInt();
  }

  @Override
  public void write(DataOutput out) throws IOException {
    out.writeInt(value);
  }

  /** Returns true iff <code>o</code> is a IntWritable with the same value. */
  @Override
  public boolean equals(Object o) {
    if (!(o instanceof IntWritable))
      return false;
    IntWritable other = (IntWritable)o;
    return this.value == other.value;
  }

  @Override
  public int hashCode() {
    return value;
  }

  /** Compares two IntWritables. */
  @Override
  public int compareTo(IntWritable o) {
    int thisValue = this.value;
    int thatValue = o.value;
    return (thisValue<thatValue ? -1 : (thisValue==thatValue ? 0 : 1));
  }

  @Override
  public String toString() {
    return Integer.toString(value);
  }

  /** A Comparator optimized for IntWritable. */ 
  public static class Comparator extends WritableComparator {
    public Comparator() {
      super(IntWritable.class);
    }

    @Override
    public int compare(byte[] b1, int s1, int l1,
                       byte[] b2, int s2, int l2) {
      int thisValue = readInt(b1, s1);
      int thatValue = readInt(b2, s2);
      return (thisValue<thatValue ? -1 : (thisValue==thatValue ? 0 : 1));
    }
  }

  static {                                        // register this comparator
    WritableComparator.define(IntWritable.class, new Comparator());
  }
}

本文参与腾讯云自媒体分享计划,欢迎正在阅读的你也加入,一起分享。

发表于

我来说两句

0 条评论
登录 后参与评论

相关文章

来自专栏向治洪

RecyclerView 实现横向滚动效果

我相信很久以前,大家在谈横向图片轮播是时候,优先会选择具有HorizontalScrollView效果和ViewPager来做,不过自从Google大会之后,系...

2956
来自专栏蜉蝣禅修之道

Android之本地文件夹实时检测与上传服务实现

1773
来自专栏向治洪

仿微信聊天表情发送

如题,这是公司项目的一个功能模块,先上个效果图: ? 其次大致说说原理: 1,首先判断输入的字符,是否包含表情的文字,比如   ?  这个表情对应的文件名为 e...

3487
来自专栏Android工程师的修仙之旅

自己动手写Android插件化框架

本文旨在通过两个实例直观的说明插件的实现原理以加深对插件内开发的理解,因此不会深入探讨背景和原理,代码也尽量专注于核心逻辑。

3212
来自专栏开发之途

Android 解析RecyclerView(1)——带点击事件监听的通用Adapter

1193
来自专栏Vamei实验室

安卓第十夜 亚当的诞生

上一讲介绍了用WebView来抓取一个网页内容。这一讲我将介绍如何在安卓内部直接进行HTTP通信。 《亚当的诞生》,西斯廷礼拜堂的吊顶画,米开朗基罗之作。当时的...

1989
来自专栏吴小龙同學

Android zxing

二维码一般在项目都会用到,今天分享下我开发二维码的一些心得。 zxing介绍 github地址:https://github.com/zxing/zxin...

3608
来自专栏王磊的博客

react-native绑定优酷SDK-附效果图和源码

ReactNative绑定优酷SDK需要用到两部分知识: 优酷本身的sdk绑定; RN与原生界面的交互; 效果: ? RN版本:0.49.3 代码更新日期:20...

3476
来自专栏Java学习网

Java中为图片添加水印效果的方法——实例代码

直接写出代码: import java.awt.AlphaComposite; import java.awt.Color; import java.awt....

5397
来自专栏知识分享

android之WIFI小车编程详述

有了前几篇wifi模块eps8266的使用,单片机设置eps8266程序,android TCP客户端,现在就做一个wifi小车 先上图 ? ? 小车是四个轮子...

3579

扫码关注云+社区