Java NIO通道概览与文件通道【源码笔记】

瓜农老梁

发布于 2020-02-18 16:54:56

2.4K0

发布于 2020-02-18 16:54:56

小结：由图可以看出直接继承Channel接口的接口由5个分别为：AsynchronousChannel、NetworkChannel、ReadableByteChannel、WritableByteChannel、InterruptibleChannel。其他接口和类都从这5个接口派生。两个字节操作接口ReadableByteChannel、WritableByteChannel，即：通道只能在字节缓冲区上操作。

二、文件通道使用

1.文件通道类图

2.文件通道示例

以示例方式串下文件通道的基本操作，示例内容为：将字符串写入文件，再读出来打印。

File file =new File("/Users/yongliang/mytest/channletst.tmp");

RandomAccessFile randomAccessFile =new RandomAccessFile(file,"rw");
FileChannel fileChannel = randomAccessFile.getChannel();// @1
ByteBuffer buffer = ByteBuffer.allocate(20);
String str ="It's a good day!";

buffer.put(str.getBytes("UTF-8"));
buffer.flip();
while(buffer.hasRemaining()) {
    fileChannel.write(buffer);// @2
}

fileChannel.force(false);// @3

System.out.println("FileChannel current position="+fileChannel.position());

fileChannel.position(0);// @4
ByteBuffer buffer2 = ByteBuffer.allocate(16);
int data = fileChannel.read(buffer2);// @5

System.out.println("length=" + data);
System.out.println("content=" +new String(buffer2.array()));

fileChannel.close();// @6

@1 开启文件通道

@2 将ByteBuffer数据写入FileChannel

@3 强制刷盘

@4 FileChannel重置到开始位置

@5 从FileChannel中读取数据到ByteBuffer

@6 关闭FileChannel

小结：梳理了FileChannle的继承关系以及通过一个示例说明FileChannle的基本操作。

三、文件通道开启源码

1.通道开启示例

RandomAccessFile randomAccessFile = new RandomAccessFile(file,"rw");
FileChannel fileChannel = randomAccessFile.getChannel();

2.RandomAccessFile创建源码

代码位置：java.io.RandomAccessFile

public RandomAccessFile(File file, String mode)
    throws FileNotFoundException {
    String name = (file != null ? file.getPath() : null);
    int imode = -1;
    if (mode.equals("r"))
        imode = O_RDONLY;
    else if (mode.startsWith("rw")) {
        imode = O_RDWR;
        rw = true;
        if (mode.length() > 2) {
            if (mode.equals("rws"))
                imode |= O_SYNC;
            else if (mode.equals("rwd"))
                imode |= O_DSYNC;
            else
                imode = -1;
        }
    } // @1

    // ...

    fd = new FileDescriptor(); // @2
    fd.attach(this);
    path = name;
    open(name, imode); // @3
}

@1 r只读模式；rw读写模式；rws模式保证数据同步写入磁盘；rwd模式保证数据和元数据同步写入磁盘。

@2 初始化文件描述符，此时初始值为-1

@3 open调用本地方法打开文件

小结：RandomAccessFile通过Native的open方法打开一个文件。

3.文件打开源码

调用链条

1.open(name,imode);
2.open0(Stringname,intmode)
3.Java_java_io_RandomAccessFile_open0
4.fileOpen
5.handleOpen

代码位置：jdk/src/solaris/native/java/io/io_util_md.c

Java_java_io_RandomAccessFile_open0(JNIEnv*env,
jobjectthis, jstring path, jint mode)
{
intflags = 0;
if(mode & java_io_RandomAccessFile_O_RDONLY)
flags =O_RDONLY;
elseif (mode & java_io_RandomAccessFile_O_RDWR) {
flags =O_RDWR | O_CREAT;
if(mode & java_io_RandomAccessFile_O_SYNC)
flags|= O_SYNC;
elseif (mode & java_io_RandomAccessFile_O_DSYNC)
flags|= O_DSYNC;
}
fileOpen(env,this, path, raf_fd, flags);
}

代码位置：jdk/src/solaris/native/java/io/io_util_md.c

fileOpen(JNIEnv *env, jobjectthis, jstring path, jfieldID fid,int flags)
{
WITH_PLATFORM_STRING(env, path, ps) {
    FD fd;

#if defined(__linux__) || defined(_ALLBSD_SOURCE)
/* Remove trailing slashes, since the kernel won't */
char *p = (char *)ps +strlen(ps) -1;
while ((p > ps) && (*p =='/'))
        *p-- ='\0';
#endif
    fd = handleOpen(ps, flags,0666);
if (fd !=-1) {
        SET_FD(this, fd, fid);
    }else {
        throwFileNotFoundException(env, path);
    }
} END_PLATFORM_STRING(env, ps);
}

代码位置：jdk/src/solaris/native/java/io/io_util_md.c

handleOpen(constchar *path,int oflag,int mode) {
FD fd;
RESTARTABLE(open64(path, oflag, mode), fd);// @1
if (fd !=-1) {
structstat64buf64;
int result;
    RESTARTABLE(fstat64(fd, &buf64), result);
if (result !=-1) {
if (S_ISDIR(buf64.st_mode)) {
            close(fd);
            errno = EISDIR;
            fd =-1;
        }
    }else {
        close(fd);
        fd =-1;
    }
}
return fd;

#define RESTARTABLE(_cmd, _result) do { \
do { \
        _result = _cmd; \// @2
    }while((_result ==-1) && (errno == EINTR)); \
}while(0)

@1 通过open调用链条跟踪，最后调用open64函数来打开一个文件，并返回文件描述符

@2 将文件描述符赋值

open64函数说明

The open64()function,similartotheopen()function,opensafileandreturnsanumbercalledafiledescriptor.
open64()differsfromopen()inthatitautomaticallyopensthefilewiththeO_LARGEFILEflagset.

小结：RandomAccessFile的创建，即通过open64()函数打开一个文件返回文件描述符。

4.开启通道源码

publicfinal FileChannel getChannel() {
    synchronized (this) {
if (channel ==null) {
            channel = FileChannelImpl.open(fd, path,true, rw,this);
        }
return channel;
    }
}

public static FileChannelopen(FileDescriptor var0, String var1, boolean var2, boolean var3, Object var4) {
return new FileChannelImpl(var0, var1, var2, var3,false, var4);
}

private FileChannelImpl(FileDescriptor var1, String var2, boolean var3, boolean var4, boolean var5, Object var6) {
this.fd = var1;
this.readable = var3;
this.writable = var4;
this.append = var5;
this.parent = var6;
this.path = var2;
this.nd = new FileDispatcherImpl(var5);
}

小结：开启通道即创建FileChannelImpl实例。

四、ByteBuffer写入通道源码

给予以上fileChannel.write(buffer)示例分析。

代码位置：sun.nio.ch.FileChannelImpl

public int write(ByteBuffer var1) throws IOException {
// ...
    synchronized(this.positionLock) {
// ...
try {
this.begin();// @1
// ...
do {
 var3 = IOUtil.write(this.fd, var1, -1L,this.nd); // @2
}while(var3 == -3 &&this.isOpen());
int var5 = IOStatus.normalize(var3);
return var5;
}finally {
}

@1/@3 可中断I/O操作，另文分析

@2 执行IOUtil.write操作

代码位置：jdk/src/solaris/native/sun/nio/fs/UnixNativeDispatcher.c

Java_sun_nio_fs_UnixNativeDispatcher_write(JNIEnv* env, jclass this, jint fd,
    jlong address, jint nbytes)
{
    ssize_t n;
    void* bufp = jlong_to_ptr(address);
    RESTARTABLE(write((int)fd, bufp, (size_t)nbytes), n); // @1
    if (n == -1) {
        throwUnixException(env, errno);
    }
    return (jint)n;
}

@1 本地函数执行write方法

pwrite函数说明

write函数说明

小结：将ByteBuffer写入FileChannel，底层通过pwrite()和write()将字节写入到文件。

五、强制刷盘源码

以下源码给予fileChannel.force()进行跟踪展开。

代码位置：sun.nio.ch.FileDispatcher

public void force(boolean var1) throws IOException {
     // ...
     var2 = this.nd.force(this.fd, var1); // @1
     // ...

}

int force(FileDescriptor var1, boolean var2) throws IOException {
   return force0(var1, var2);
}

static native int force0(FileDescriptor var0, boolean var1) throws IOException; // @2

@1 调用FileDispatcherImpl#force强制刷盘

@2 调用native方法force0

代码位置：jdk/src/solaris/native/sun/nio/ch/FileDispatcherImpl.c

Java_sun_nio_ch_FileDispatcherImpl_force0(JNIEnv *env, jobject this,
                                          jobject fdo, jboolean md)
{
    jint fd = fdval(env, fdo);
    int result = 0;

    if (md == JNI_FALSE) {
        result = fdatasync(fd); // @1
    } else {
#ifdef _AIX
        int getfl = fcntl(fd, F_GETFL);
        if (getfl >= 0 && (getfl & O_ACCMODE) == O_RDONLY) {
            return 0;
        }
#endif
        result = fsync(fd); // @2
    }
    return handle(env, result, "Force failed");
}

@1 如果fileChannel.force(false)执行fdatasync()函数

@2 如果fileChannel.force(true)执行fsync()函数

fdatasync()函数说明

fdatasync() is similar to fsync(), but does not flush modified metadata unless that metadata is needed in order to allow a subsequent data retrieval to be correctly handled.

fsync()函数说明

The fsync() function transfers all data for the file indicated by the open file descriptor file_descriptor to the storage device associated with file_descriptor. fsync() does not return until the transfer is complete, or until an error is detected.

小结：FileChannel.force(false)调用Native函数fdatasync()同步刷盘，不写入元数据；FileChannel.force(true)调用Native函数fsync()同步刷盘，同时写入元数据信息；元数据包括修改人、修改时间等信息。

六、通道重置位点源码

给予fileChannel.position(0)进行源码追踪。

代码位置：sun.nio.ch.FileChannelImpl

public FileChannel position(long var1) throws IOException {
        this.ensureOpen();
        // ...
        do {
            var4 = this.nd.seek(this.fd, var1);
        } while(var4 == -3L && this.isOpen());
        // ...
    }
}

long seek(FileDescriptor var1, long var2) throws IOException {
    return seek0(var1, var2);
}

static native long seek0(FileDescriptor var0, long var1) throws IOException; // @1

@1 调用Native lseek()函数重置位点。

lseek()函数说明

The lseek() function changes the current file offset to a new position in the file. The new position is the given byte offset from the position specified by whence. After you have used lseek() to seek to a new location, the next I/O operation on the file begins at that location.

小结：由以上源码可以看出，fileChannel.position(0)通过Native方法seek0来实现位点重置，底层为lseek()函数重置文件位点。

七、读取数据到ByteBuffer源码

给予fileChannel.read(buffer2)进行源码跟踪

1.JDK源码跟踪

代码位置：sun.nio.ch.FileChannelImpl

public int read(ByteBuffer var1) throws IOException {
    // ...
    if (this.isOpen()) {
        do {
            var3 = IOUtil.read(this.fd, var1, -1L, this.nd);
        } while(var3 == -3 && this.isOpen());

        int var12 = IOStatus.normalize(var3);
        return var12;
    }
    // ...
}

static int read(FileDescriptor var0, ByteBuffer var1, long var2, NativeDispatcher var4) throws IOException {
    // ...
    try {
        int var6 = readIntoNativeBuffer(var0, var5, var2, var4);
        var5.flip();
        if (var6 > 0) {
            var1.put(var5);
        }

    } finally {
        Util.offerFirstTemporaryDirectBuffer(var5);
    }
    // ...
}

private static int readIntoNativeBuffer(FileDescriptor var0, ByteBuffer var1, long var2, NativeDispatcher var4) throws IOException {
    if (var2 != -1L) {
        var9 = var4.pread(var0, ((DirectBuffer)var1).address() + (long)var5, var7, var2);
    } else {
        var9 = var4.read(var0, ((DirectBuffer)var1).address() + (long)var5, var7);
    }

    if (var9 > 0) {
        var1.position(var5 + var9);
    }

    return var9;
    }
    }

小结：从JDK源码fileChannel.read最终调用Native的pread和read来读取。

2.Native源码跟踪

Java_sun_nio_ch_FileDispatcherImpl_read0(JNIEnv *env, jclass clazz,
                             jobject fdo, jlong address, jint len)
{
    jint fd = fdval(env, fdo);
    void *buf = (void *)jlong_to_ptr(address);

    return convertReturnVal(env, read(fd, buf, len), JNI_TRUE);
}

JNIEXPORT jint JNICALL
Java_sun_nio_ch_FileDispatcherImpl_pread0(JNIEnv *env, jclass clazz, jobject fdo,
                            jlong address, jint len, jlong offset)
{
    jint fd = fdval(env, fdo);
    void *buf = (void *)jlong_to_ptr(address);

    return convertReturnVal(env, pread64(fd, buf, len, offset), JNI_TRUE);
}

read()函数说明

read() attempts to read up to count bytes from file descriptor fd
into the buffer starting at buf.

pread64()函数说明

小结：分别调用了Native的read()函数和pread64()函数，都是从文件描述符读取数据到ByteBuffer中，pread64()支持大文件读取。

八、通道关闭源码

给予fileChannel.close()进行追踪

1.JDK源码追踪

代码位置：java.nio.channels.spi.AbstractInterruptibleChannel

public final void close() throws IOException {
    synchronized (closeLock) {
        if (!open)
            return;
        open = false;
        implCloseChannel();
    }
}

protected void implCloseChannel() throws IOException {
    if (this.fileLockTable != null) {
        Iterator var1 = this.fileLockTable.removeAll().iterator();

        while(var1.hasNext()) {
            FileLock var2 = (FileLock)var1.next();
            synchronized(var2) {
                if (var2.isValid()) {
                    this.nd.release(this.fd, var2.position(), var2.size());
                    ((FileLockImpl)var2).invalidate();
                }
            }
        }
    }

    this.threads.signalAndWait();
    if (this.parent != null) {
        ((Closeable)this.parent).close();
    } else {
        this.nd.close(this.fd);
    }

}

void release(FileDescriptor var1, long var2, long var4) throws IOException {
    release0(var1, var2, var4);
}

小结：通道关闭调用最终调用Native方法release0。

2.Native方法源码跟踪

Java_sun_nio_ch_FileDispatcherImpl_release0(JNIEnv *env, jobject this,
                                         jobject fdo, jlong pos, jlong size)
{
    jint fd = fdval(env, fdo);
    jint lockResult = 0;
    struct flock64 fl;
    int cmd = F_SETLK64;

    fl.l_whence = SEEK_SET;
    if (size == (jlong)java_lang_Long_MAX_VALUE) {
        fl.l_len = (off64_t)0;
    } else {
        fl.l_len = (off64_t)size;
    }
    fl.l_start = (off64_t)pos;
    fl.l_type = F_UNLCK;
    lockResult = fcntl(fd, cmd, &fl);
    if (lockResult< 0) {
        JNU_ThrowIOExceptionWithLastError(env, "Release failed");
    }
}

fcntl()函数说明

The fcntl() function performs various actions on open descriptors, such as obtaining or changing the attributes of a file or socket descriptor.

F_SETLK64

Sets or clears a file segment lock for a large file. You must specify a third argument of type struct flock64

小结：FileChannel的关闭操作，通过调用fcntl()函数参数为F_SETLK64，清理该文件上的锁。

九、文件截取源码

给予fileChannel.truncate(10)来进行源码跟踪。

public FileChannel truncate(long var1){
     var4 = this.nd.truncate(this.fd, var1);
}

static native int truncate0(FileDescriptor var0, long var1) throws IOException;

Java_sun_nio_ch_FileDispatcherImpl_truncate0(JNIEnv *env, jobject this,
                                             jobject fdo, jlong size)
{
    return handle(env,
                  ftruncate64(fdval(env, fdo), size),
                  "Truncation failed");
}

ftruncate64函数说明

The truncate() and ftruncate() functions cause the regular file named by path or referenced by fd to be truncated to a size of precisely length bytes.
If the file previously was larger than this size, the extra data is lost. If the file previously was shorter, it is extended, and the extended part reads as null bytes ('\0').
The file offset is not changed.
If the size changed, then the st_ctime and st_mtime fields (respectively, time of last status change and time of last modification; see stat(2)) for the file are updated, and the set-user-ID and set-group-ID permission bits may be cleared.
With ftruncate(), the file must be open for writing; with truncate(), the file must be writable.

小结：文件的截取通过Native函数ftruncate64来实现，从文件开始位置截取指定的长度。

十、参考资料

参考书籍：《Java NIO》第三章

文章总结：本文梳理了通道接口继承关系，以文件通道FileChannel的示例入手，跟踪每个操作的Native方法，以及给出这些Native方法的调用源码和说明。

十一、系列文章

系统层面I/O【原理笔记】

Java NIO缓存区基本操作【源码笔记】

Java NIO字节缓存区【源码笔记】

「瓜农老梁学习同行」

本文参与腾讯云自媒体分享计划，分享自微信公众号。

原始发表：2020-01-13，如有侵权请联系 cloudcommunity@tencent.com 删除

access