刨解OkHttp之缓存机制

时间一晃而过,今天想给大家带来OkHttp的zuihou最后一篇文章,主要讲一下OkHttp的缓存机制。OkHttp的责任链中有一个拦截器就是专门应对OkHttp的缓存的,那就是CacheInterceptor拦截器。

CacheInterceptor

其对应的方法如下,我们就从这个方法讲起:

public Response intercept(Chain chain) throws IOException {
    
    //假如有缓存,会得到拿到缓存,否则为null
    Response cacheCandidate = cache != null
        ? cache.get(chain.request())
        : null;

    long now = System.currentTimeMillis();

    //获取缓存策略
    CacheStrategy strategy = new CacheStrategy.Factory(now, chain.request(), cacheCandidate).get();

    //缓存策略请求
    Request networkRequest = strategy.networkRequest;
    //缓存策略响应
    Response cacheResponse = strategy.cacheResponse;

    //缓存非空判断
    if (cache != null) {
      cache.trackResponse(strategy);
    }

    //本地缓存不为空并且缓存策略响应为空
    if (cacheCandidate != null && cacheResponse == null) {
      closeQuietly(cacheCandidate.body()); // The cache candidate wasn't applicable. Close it.
    }

    //缓存策略请求和缓存策略响应为空,禁止使用网络直接返回
    // If we're forbidden from using the network and the cache is insufficient, fail.
    if (networkRequest == null && cacheResponse == null) {
      return new Response.Builder()
          .request(chain.request())
          .protocol(Protocol.HTTP_1_1)
          .code(504)
          .message("Unsatisfiable Request (only-if-cached)")
          .body(Util.EMPTY_RESPONSE)
          .sentRequestAtMillis(-1L)
          .receivedResponseAtMillis(System.currentTimeMillis())
          .build();
    }

    //缓存策略请求为空,即缓存有效则直接使用缓存不使用网络
    // If we don't need the network, we're done.
    if (networkRequest == null) {
      return cacheResponse.newBuilder()
          .cacheResponse(stripBody(cacheResponse))
          .build();
    }

    //缓存无效,则执行下一个拦截器以获取请求
    Response networkResponse = null;
    try {
      networkResponse = chain.proceed(networkRequest);
    } finally {
      // If we're crashing on I/O or otherwise, don't leak the cache body.
      if (networkResponse == null && cacheCandidate != null) {
        closeQuietly(cacheCandidate.body());
      }
    }

    //假如本地也有缓存,则根据条件选择使用哪个响应
    // If we have a cache response too, then we're doing a conditional get.
    if (cacheResponse != null) {
      if (networkResponse.code() == HTTP_NOT_MODIFIED) {
        Response response = cacheResponse.newBuilder()
            .headers(combine(cacheResponse.headers(), networkResponse.headers()))
            .sentRequestAtMillis(networkResponse.sentRequestAtMillis())
            .receivedResponseAtMillis(networkResponse.receivedResponseAtMillis())
            .cacheResponse(stripBody(cacheResponse))
            .networkResponse(stripBody(networkResponse))
            .build();
        networkResponse.body().close();

        // Update the cache after combining headers but before stripping the
        // Content-Encoding header (as performed by initContentStream()).
        cache.trackConditionalCacheHit();
        //更新缓存
        cache.update(cacheResponse, response);
        return response;
      } else {
        closeQuietly(cacheResponse.body());
      }
    }

    //没有缓存,则直接使用网络响应
    Response response = networkResponse.newBuilder()
        .cacheResponse(stripBody(cacheResponse))
        .networkResponse(stripBody(networkResponse))
        .build();

    if (cache != null) {
      //缓存到本地
      if (HttpHeaders.hasBody(response) && CacheStrategy.isCacheable(response, networkRequest)) {
        // Offer this request to the cache.
        CacheRequest cacheRequest = cache.put(response);
        return cacheWritingResponse(cacheRequest, response);
      }

      if (HttpMethod.invalidatesCache(networkRequest.method())) {
        try {
          cache.remove(networkRequest);
        } catch (IOException ignored) {
          // The cache cannot be written.
        }
      }
    }

    return response;
  }

这就是整个缓存拦截器的主要方法,首先会从cache去拿缓存,没有则返回null,然后通过CacheStrategy来获取缓存策略,CacheStrategy根据之前缓存的结果与当前将要发送Request的header进行策略,并得出是否进行请求的结果。由于篇幅关系,这一块不细讲因为涉及网络协议,最终他的得出的规则如下如:

image.png

因为我把注释流程都写在代码了,大家可以看上面方法代码理解,其整体缓存流程如下:

  1. 如果有缓存,则取出缓存否则为null
  2. 根据CacheStrategy拿到它的缓存策略请求和响应
  3. 缓存策略请求和缓存策略响应为空,禁止使用网络直接返回
  4. 缓存策略请求为空,即缓存有效则直接使用缓存不使用网络
  5. 缓存无效,则执行下一个拦截器以获取请求
  6. 假如本地也有缓存,则根据条件选择使用哪个响应,更新缓存
  7. 没有缓存,则直接使用网络响应
  8. 添加缓存

到这里我们可以看到,缓存的“增删改查”都是cache(Cache)类来进行操作的。下面让我们来看一下这个类吧。

Cache

Cache的“增删改查”其实都是基于DiskLruCache,下面我们会继续讲,先来看一下“增删改查”的各个方法吧

  • 添加缓存
CacheRequest put(Response response) {
    String requestMethod = response.request().method();
    //如果请求是"POST","PUT","PATCH","PROPPATCH","REPORT"则移除这些缓存  
    if (HttpMethod.invalidatesCache(response.request().method())) {
      try {
        remove(response.request());
      } catch (IOException ignored) {
      }
      return null;
    }
    //仅支持GET的请求缓存,其他请求不缓存
    if (!requestMethod.equals("GET")) {
       return null;
    }
    //判断请求中的http数据包中headers是否有符号"*"的通配符,有则不缓存  
    if (HttpHeaders.hasVaryAll(response)) {
      return null;
    }
    //把response构建成一个Entry对象
    Entry entry = new Entry(response);
    DiskLruCache.Editor editor = null;
    try {
      //生成DiskLruCache.Editor对象
      editor = cache.edit(key(response.request().url()));
      if (editor == null) {
        return null;
      }
      //对缓存进行写入
      entry.writeTo(editor);
      //构建一个CacheRequestImpl类,包含Ok.io的Sink对象
      return new CacheRequestImpl(editor);
    } catch (IOException e) {
      abortQuietly(editor);
      return null;
    }
  }
  • 得到缓存
Response get(Request request) {
    //获取url转换过来的key
    String key = key(request.url());
    DiskLruCache.Snapshot snapshot;
    Entry entry;
    try {
         //根据key获取对应的snapshot 
         snapshot = cache.get(key);
         if (snapshot == null) {
             return null;
         }
    } catch (IOException e) {
      return null;
    }
    try {
     //创建一个Entry对象,并由snapshot.getSource()获取Sink
      entry = new Entry(snapshot.getSource(ENTRY_METADATA));
    } catch (IOException e) {
      Util.closeQuietly(snapshot);
      return null;
    }
    //通过entry和response生成respson,通过Okio.buffer获取请求体,然后封装各种请求信息
    Response response = entry.response(snapshot);
    if (!entry.matches(request, response)) {
      //对request和Response进行比配检查,成功则返回该Response。
      Util.closeQuietly(response.body());
      return null;
    }
    return response;
  }
  • 更新缓存
void update(Response cached, Response network) {
    //用Respon构建一个Entry
    Entry entry = new Entry(network);
    //从缓存中获取DiskLruCache.Snapshot
    DiskLruCache.Snapshot snapshot = ((CacheResponseBody) cached.body()).snapshot;
    DiskLruCache.Editor editor = null;
    try {
      //获取DiskLruCache.Snapshot.edit对象
      editor = snapshot.edit(); // Returns null if snapshot is not current.
      if (editor != null) {
        //将entry写入editor中
        entry.writeTo(editor);
        editor.commit();
      }
    } catch (IOException e) {
      abortQuietly(editor);
    }
  }
  • 删除缓存
void remove(Request request) throws IOException {
    //通过url转化成的key去删除缓存
    cache.remove(key(request.url()));
  }

Cache的"增删改查"大体通过注释代码的方式给出,Cache还有一个更重要的缓存处理类就是DiskLruCache。

DiskLruCache

不仔细看还以为这个类和JakeWharton写的DiskLruCache:[https://link.jianshu.com/t=https://github.com/JakeWharton/DiskLruCache(https://link.jianshu.com/t=https://github.com/JakeWharton/DiskLruCache)是一样的,其实主体架构差不多,只不过OkHttp的DiskLruCache结合了Ok.io,用Ok.io处理数据文件的储存. 我们可以看到上面的DiskLruCache有shang三个内部类,分别是Entry,Snapshot,Editor。

Entry

final String key;

    /** Lengths of this entry's files. */
    final long[] lengths;
    final File[] cleanFiles;
    final File[] dirtyFiles;

    /** True if this entry has ever been published. */
    boolean readable;

    /** The ongoing edit or null if this entry is not being edited. */
    Editor currentEditor;

    /** The sequence number of the most recently committed edit to this entry. */
    long sequenceNumber;

    Entry(String key) {
      this.key = key;

      lengths = new long[valueCount];
      cleanFiles = new File[valueCount];
      dirtyFiles = new File[valueCount];

      // The names are repetitive so re-use the same builder to avoid allocations.
      StringBuilder fileBuilder = new StringBuilder(key).append('.');
      int truncateTo = fileBuilder.length();
      for (int i = 0; i < valueCount; i++) {
        fileBuilder.append(i);
        cleanFiles[i] = new File(directory, fileBuilder.toString());
        fileBuilder.append(".tmp");
        dirtyFiles[i] = new File(directory, fileBuilder.toString());
        fileBuilder.setLength(truncateTo);
      }
    }
    
    //省略
    ......

实际上只是用于存储缓存数据的实体类,一个url对应一个实体,在Entry还有Snapshot对象,代码如下:

Snapshot snapshot() {
      if (!Thread.holdsLock(DiskLruCache.this)) throw new AssertionError();

      Source[] sources = new Source[valueCount];
      long[] lengths = this.lengths.clone(); // Defensive copy since these can be zeroed out.
      try {
        for (int i = 0; i < valueCount; i++) {
          sources[i] = fileSystem.source(cleanFiles[i]);
        }
        return new Snapshot(key, sequenceNumber, sources, lengths);
      } catch (FileNotFoundException e) {
        // A file must have been deleted manually!
        for (int i = 0; i < valueCount; i++) {
          if (sources[i] != null) {
            Util.closeQuietly(sources[i]);
          } else {
            break;
          }
        }
        // Since the entry is no longer valid, remove it so the metadata is accurate (i.e. the cache
        // size.)
        try {
          removeEntry(this);
        } catch (IOException ignored) {
        }
        return null;
      }
    }

即一个Entry对应着一个Snapshot对象,在看一下Snapshot的内部代码:

public final class Snapshot implements Closeable {
    private final String key;
    private final long sequenceNumber;
    private final Source[] sources;
    private final long[] lengths;

    Snapshot(String key, long sequenceNumber, Source[] sources, long[] lengths) {
      this.key = key;
      this.sequenceNumber = sequenceNumber;
      this.sources = sources;
      this.lengths = lengths;
    }

    public String key() {
      return key;
    }

    /**
     * Returns an editor for this snapshot's entry, or null if either the entry has changed since
     * this snapshot was created or if another edit is in progress.
     */
    public @Nullable Editor edit() throws IOException {
      return DiskLruCache.this.edit(key, sequenceNumber);
    }

    /** Returns the unbuffered stream with the value for {@code index}. */
    public Source getSource(int index) {
      return sources[index];
    }

    /** Returns the byte length of the value for {@code index}. */
    public long getLength(int index) {
      return lengths[index];
    }

    public void close() {
      for (Source in : sources) {
        Util.closeQuietly(in);
      }
    }
  }

初始化的Snapshot仅仅只是存储了一些变量而已。

Editor

在Editor的初始化中要传入Editor,其实Editor就是编辑entry的类。源码如下:

public final class Editor {
    final Entry entry;
    final boolean[] written;
    private boolean done;

    Editor(Entry entry) {
      this.entry = entry;
      this.written = (entry.readable) ? null : new boolean[valueCount];
    }
  
    void detach() {
      if (entry.currentEditor == this) {
        for (int i = 0; i < valueCount; i++) {
          try {
            fileSystem.delete(entry.dirtyFiles[i]);
          } catch (IOException e) {
            // This file is potentially leaked. Not much we can do about that.
          }
        }
        entry.currentEditor = null;
      }
    }

    //返回指定index的cleanFile的读入流
    public Source newSource(int index) {
      synchronized (DiskLruCache.this) {
        if (done) {
          throw new IllegalStateException();
        }
        if (!entry.readable || entry.currentEditor != this) {
          return null;
        }
        try {
          return fileSystem.source(entry.cleanFiles[index]);
        } catch (FileNotFoundException e) {
          return null;
        }
      }
    }
    
    //向指定index的dirtyFiles文件写入数据
    public Sink newSink(int index) {
      synchronized (DiskLruCache.this) {
        if (done) {
          throw new IllegalStateException();
        }
        if (entry.currentEditor != this) {
          return Okio.blackhole();
        }
        if (!entry.readable) {
          written[index] = true;
        }
        File dirtyFile = entry.dirtyFiles[index];
        Sink sink;
        try {
          sink = fileSystem.sink(dirtyFile);
        } catch (FileNotFoundException e) {
          return Okio.blackhole();
        }
        return new FaultHidingSink(sink) {
          @Override protected void onException(IOException e) {
            synchronized (DiskLruCache.this) {
              detach();
            }
          }
        };
      }
    }

    //这里执行的工作是提交数据,并释放锁,最后通知DiskLruCache刷新相关数据
    public void commit() throws IOException {
      synchronized (DiskLruCache.this) {
        if (done) {
          throw new IllegalStateException();
        }
        if (entry.currentEditor == this) {
          completeEdit(this, true);
        }
        done = true;
      }
    }

    //终止编辑,并释放锁
    public void abort() throws IOException {
      synchronized (DiskLruCache.this) {
        if (done) {
          throw new IllegalStateException();
        }
        if (entry.currentEditor == this) {
          completeEdit(this, false);
        }
        done = true;
      }
    }

    //除非正在编辑,否则终止
    public void abortUnlessCommitted() {
      synchronized (DiskLruCache.this) {
        if (!done && entry.currentEditor == this) {
          try {
            completeEdit(this, false);
          } catch (IOException ignored) {
          }
        }
      }
    }
  }

各个方法对应作用如下:

  • Source newSource(int index):返回指定index的cleanFile的读入流
  • Sink newSink(int index):向指定index的dirtyFiles文件写入数据
  • commit():这里执行的工作是提交数据,并释放锁,最后通知DiskLruCache刷新相关数据
  • abort():终止编辑,并释放锁
  • abortUnlessCommitted():除非正在编辑,否则终止

剩下关键来了,还记得上面我们讲Cache添加有一行代码entry.writeTo(editor);,里面操作如下:

 public void writeTo(DiskLruCache.Editor editor) throws IOException {
      BufferedSink sink = Okio.buffer(editor.newSink(ENTRY_METADATA));

      sink.writeUtf8(url)
          .writeByte('\n');
      sink.writeUtf8(requestMethod)
          .writeByte('\n');
      sink.writeDecimalLong(varyHeaders.size())
          .writeByte('\n');
      for (int i = 0, size = varyHeaders.size(); i < size; i++) {
        sink.writeUtf8(varyHeaders.name(i))
            .writeUtf8(": ")
            .writeUtf8(varyHeaders.value(i))
            .writeByte('\n');
      }

      sink.writeUtf8(new StatusLine(protocol, code, message).toString())
          .writeByte('\n');
      sink.writeDecimalLong(responseHeaders.size() + 2)
          .writeByte('\n');
      for (int i = 0, size = responseHeaders.size(); i < size; i++) {
        sink.writeUtf8(responseHeaders.name(i))
            .writeUtf8(": ")
            .writeUtf8(responseHeaders.value(i))
            .writeByte('\n');
      }
      sink.writeUtf8(SENT_MILLIS)
          .writeUtf8(": ")
          .writeDecimalLong(sentRequestMillis)
          .writeByte('\n');
      sink.writeUtf8(RECEIVED_MILLIS)
          .writeUtf8(": ")
          .writeDecimalLong(receivedResponseMillis)
          .writeByte('\n');

      if (isHttps()) {
        sink.writeByte('\n');
        sink.writeUtf8(handshake.cipherSuite().javaName())
            .writeByte('\n');
        writeCertList(sink, handshake.peerCertificates());
        writeCertList(sink, handshake.localCertificates());
        sink.writeUtf8(handshake.tlsVersion().javaName()).writeByte('\n');
      }
      sink.close();
    }

上面的都是Ok.io的操作了,不懂OK.io的可以去看一下相关知识。BufferedSink sink = Okio.buffer(editor.newSink(ENTRY_METADATA));editor.newSink拿到ok.io版的OutputStream(Sink)生成Ok.io的输入类,剩下的就是把数据用ok.io写入文件,然后关闭输出类。

同理我看们可以一下上面Cache获取缓存的代码 Response response = entry.response(snapshot);,在response方法里又有一个方法:CacheResponseBody()就是获取缓存的方法,代码如下:

 CacheResponseBody(final DiskLruCache.Snapshot snapshot,String contentType, String contentLength) {
      this.snapshot = snapshot;
      this.contentType = contentType;
      this.contentLength = contentLength;

      Source source = snapshot.getSource(ENTRY_BODY);
      bodySource = Okio.buffer(new ForwardingSource(source) {
        @Override public void close() throws IOException {
          snapshot.close();
          super.close();
        }
      });
    }

new ForwardingSource(source)相当于传入ok.io版的InputStream(Source)生成Ok.io的读取类,剩下的都是读取缓存数据然后生成Response.

而上面Cache的Update()方法,其写入过程也和上面的添加是一样的,不同的只不过先构造成一个就得Entry然后再把新的缓存写上去更新而已,因为涉及我重要的Ok.io是一样的,所以不细讲。

剩下就是删除了,在Cache的delete方法里,在removeEntry就是执行删除操作,代码如下:

 boolean removeEntry(Entry entry) throws IOException {
  
    //省略

    journalWriter.writeUtf8(REMOVE).writeByte(' ').writeUtf8(entry.key).writeByte('\n');
    lruEntries.remove(entry.key);

    //省略
    return true;
  }

上面这两句代码就是删除的关键, journalWriter.writeUtf8表示在DiskLruCache的本地缓存清单列表里删除,lruEntries.remove表示在缓存内存里删除。

到此增删给查的流程基本结束,其实DiskLruCache还有很多可以讲,但是我的重心是OKhttp的缓存底层是用Ok.io,为此在这里点到为止。

内容有点多,如有错误请多多指出

本文参与腾讯云自媒体分享计划,欢迎正在阅读的你也加入,一起分享。

发表于

我来说两句

0 条评论
登录 后参与评论

相关文章

来自专栏Java帮帮-微信公众号-技术文章全总结

Java多线程详解5【面试+工作】

Java多线程详解【面试+工作】 Java线程:新特征-信号量 Java的信号量实际上是一个功能完毕的计数器,对控制一定资源的消费与回收有着很重要的意义,信号量...

41610
来自专栏Jack的Android之旅

刨解OkHttp之访问连接

因为OkHttp能讲的东西太多了,上一篇文章只是讲到了他的设计架构即责任链模式和异步多线程网络访问,这对于OkHttp只是冰山一角,对于一个网络请求框架,最重要...

1361
来自专栏破晓之歌

Django框架下admin.py的中文修改 原

#所以更改setttings.py 下 LANGUAGE_CODE = 'zh-Hans' 

1072
来自专栏扎心了老铁

springboot使用zookeeper(curator)实现注册发现与负载均衡

最简单的实现服务高可用的方法就是集群化,也就是分布式部署,但是分布式部署会带来一些问题。比如: 1、各个实例之间的协同(锁) 2、负载均衡 3、热删除 这里通过...

2.6K10
来自专栏肖蕾的博客

OKHttp3(支持Retrofit)的网络数据缓存Interceptor拦截器

3343
来自专栏hotqin888的专栏

engineercms利用pdf.js制作连续看图功能

版权声明:本文为博主原创文章,未经博主允许不得转载。 https://blog.csdn.net/hotqin888/article/det...

1691
来自专栏Java开发者杂谈

Netty(1):第一个netty程序

为什么选择Netty   netty是业界最流行的NIO框架之一,它的健壮型,功能,性能,可定制性和可扩展性都是首屈一指的,Hadoop的RPC框架Avro就使...

4017
来自专栏Android开发实战

谷歌官方Android应用架构库——LiveData

LiveData 是一个数据持有者类,它持有一个值并允许观察该值。不同于普通的可观察者,LiveData 遵守应用程序组件的生命周期,以便 Observer 可...

1113
来自专栏黑泽君的专栏

day52_BOS项目_04

第一步:导入pinyin4j-2.5.0.jar包,拷贝PinYin4jUtils.java工具类至utils包中 第二步:测试类代码如下:

892
来自专栏Java 源码分析

SpringBoot 笔记 ( 五 ):缓存

3954

扫码关注云+社区

领取腾讯云代金券