文档章节

OkHttp离线缓存实现

叶大侠
 叶大侠
发布于 2016/12/11 23:29
字数 1512
阅读 947
收藏 36

场景应用:

OkHttp内部已经支持了标准的Http协议缓存策略,如Last-Modified, Etag, Cache-Control等方式,看起来已经是非常够用了,但是我们还想让缓存在这样的场景下得到使用: 当客户端设备网络中断或者服务端出现了错误,但是本地存在缓存副本的时候,我们依然想取用这部分缓存。

OkHttp保存缓存的时机:

okhttp3.internal.http.HttpEngine:
private void maybeCache() throws IOException {
    InternalCache responseCache = Internal.instance.internalCache(client);
    if (responseCache == null) return;

    // Should we cache this response for this request?
    if (!CacheStrategy.isCacheable(userResponse, networkRequest)) {
      if (HttpMethod.invalidatesCache(networkRequest.method())) {
        try {
          responseCache.remove(networkRequest);
        } catch (IOException ignored) {
          // The cache cannot be written.
        }
      }
      return;
    }

    // Offer this request to the cache.
    storeRequest = responseCache.put(stripBody(userResponse));
}

这里我们可以看到OkHttp是否缓存请求结果其中的一个条件:CacheStrategy.isCacheable(), 看看这里面的代码:

okhttp3.internal.http.CacheStrategy:
public static boolean isCacheable(Response response, Request request) {
    // Always go to network for uncacheable response codes (RFC 7231 section 6.1),
    // This implementation doesn't support caching partial content.
    switch (response.code()) {
      case HTTP_OK:
      case HTTP_NOT_AUTHORITATIVE:
      case HTTP_NO_CONTENT:
      case HTTP_MULT_CHOICE:
      case HTTP_MOVED_PERM:
      case HTTP_NOT_FOUND:
      case HTTP_BAD_METHOD:
      case HTTP_GONE:
      case HTTP_REQ_TOO_LONG:
      case HTTP_NOT_IMPLEMENTED:
      case StatusLine.HTTP_PERM_REDIRECT:
        // These codes can be cached unless headers forbid it.
        break;

      case HTTP_MOVED_TEMP:
      case StatusLine.HTTP_TEMP_REDIRECT:
        // These codes can only be cached with the right response headers.
        // http://tools.ietf.org/html/rfc7234#section-3
        // s-maxage is not checked because OkHttp is a private cache that should ignore s-maxage.
        if (response.header("Expires") != null
            || response.cacheControl().maxAgeSeconds() != -1
            || response.cacheControl().isPublic()
            || response.cacheControl().isPrivate()) {
          break;
        }
        // Fall-through.

      default:
        // All other codes cannot be cached.
        return false;
    }

    // A 'no-store' directive on request or response prevents the response from being cached.
    return !response.cacheControl().noStore() && !request.cacheControl().noStore();
}

我们接着回到maybeCache()方法往下看,最后一句responseCache.put(stripBody(userResponse))将会调用下面的方法:

okhttp3.Cache:
private CacheRequest put(Response response) throws IOException {
    String requestMethod = response.request().method();

    if (HttpMethod.invalidatesCache(response.request().method())) {
      try {
        remove(response.request());
      } catch (IOException ignored) {
        // The cache cannot be written.
      }
      return null;
    }
    if (!requestMethod.equals("GET")) {
      // Don't cache non-GET responses. We're technically allowed to cache
      // HEAD requests and some POST requests, but the complexity of doing
      // so is high and the benefit is low.
      return null;
    }

    if (OkHeaders.hasVaryAll(response)) {
      return null;
    }

    Entry entry = new Entry(response);
    DiskLruCache.Editor editor = null;
    try {
      editor = cache.edit(urlToKey(response.request()));
      if (editor == null) {
        return null;
      }
      //缓存写入文件:
      entry.writeTo(editor);
      return new CacheRequestImpl(editor);
    } catch (IOException e) {
      abortQuietly(editor);
      return null;
    }
}

总的来说就是只要是Get请求,同时请求和响应头部CacheControl没有设置no store就会把请求结果缓存了。

OkHttp保存缓存的格式:

url所对应的缓存文件名规则,不难看出是url的md5值。

Util.md5Hex(request.url().toString())

保存的代码很明显就在entry.writeTo里面啦。

okhttp3.Cache.Entry:
public void writeTo(DiskLruCache.Editor editor) throws IOException {
  //.0结尾的文件
  BufferedSink sink = Okio.buffer(editor.newSink(ENTRY_METADATA));

  sink.writeUtf8(url);
  sink.writeByte('\n');
  sink.writeUtf8(requestMethod);
  sink.writeByte('\n');
  sink.writeDecimalLong(varyHeaders.size());
  sink.writeByte('\n');
  for (int i = 0, size = varyHeaders.size(); i < size; i++) {
	sink.writeUtf8(varyHeaders.name(i));
	sink.writeUtf8(": ");
	sink.writeUtf8(varyHeaders.value(i));
	sink.writeByte('\n');
  }

  sink.writeUtf8(new StatusLine(protocol, code, message).toString());
  sink.writeByte('\n');
  sink.writeDecimalLong(responseHeaders.size());
  sink.writeByte('\n');
  for (int i = 0, size = responseHeaders.size(); i < size; i++) {
	sink.writeUtf8(responseHeaders.name(i));
	sink.writeUtf8(": ");
	sink.writeUtf8(responseHeaders.value(i));
	sink.writeByte('\n');
  }

  if (isHttps()) {
	sink.writeByte('\n');
	sink.writeUtf8(handshake.cipherSuite().javaName());
	sink.writeByte('\n');
	writeCertList(sink, handshake.peerCertificates());
	writeCertList(sink, handshake.localCertificates());
	// The handshake’s TLS version is null on HttpsURLConnection and on older cached responses.
	if (handshake.tlsVersion() != null) {
	  sink.writeUtf8(handshake.tlsVersion().javaName());
	  sink.writeByte('\n');
	}
  }
  sink.close();
}

我们打开相应的缓存目录看一下,果然其中包含了以url的md5值作为文件名的一系列文件,还有一个journal文件,这个文件和DiskLruCache缓存有关,我们这里不展开分析。我们看一下.0和.1文件的内容吧,其实.0里面的东西从上面的代码已经可以知道就是和请求和响应头部的一些信息。

http://www.infoq.com/cn/articles/etags
GET
2
Accept-Encoding: gzip
User-Agent: okhttp/3.2.0
HTTP/1.1 200 OK
15
Date: Sun, 11 Dec 2016 14:50:36 GMT
Server: Apache
Sniply-Options: BLOCK
Set-Cookie: JSESSIONID=7FFBDE344A66525EE343DCCC6DD23692; Path=/
Last-Modified: Sun, 11 Dec 2016 14:50:36 GMT
Vary: Accept-Encoding,User-Agent
Access-Control-Allow-Credentials: true
Accept-Ranges: none
Access-Control-Allow-Origin: http://www.infoq.com
Content-Encoding: gzip
Connection: close
Transfer-Encoding: chunked
Content-Type: text/html;charset=utf-8
OkHttp-Sent-Millis: 1481467841870
OkHttp-Received-Millis: 1481467844412

md5(url).1文件很容易猜到就是response本身的内容了。读者可以自行验证,另外如果response是gzip的格式,那么这里缓存的直接就是gzip后的内容了。

OkHttp离线缓存实现:

通过以上的分析,我们的思路就很清晰了,其实就是一开始就判断是否连接网络,如果是否就直接返回缓存的内容,根据response的头部是否是Content-Encoding: gzip来确定是否需要解压后返回;如果请求服务端失败,那么也返回缓存的内容。

通过分析OkHttp的InterceptorChain实现:

okhttp3.RealCall.ApplicationInterceptorChain:

public Response proceed(Request request) throws IOException {
      // If there's another interceptor in the chain, call that.
      if (index < client.interceptors().size()) {
        Interceptor.Chain chain = new ApplicationInterceptorChain(index + 1, request, forWebSocket);
        Interceptor interceptor = client.interceptors().get(index);
        Response interceptedResponse = interceptor.intercept(chain);

        if (interceptedResponse == null) {
          throw new NullPointerException("application interceptor " + interceptor
              + " returned null");
        }

        return interceptedResponse;
      }

      // No more interceptors. Do HTTP.
      return getResponse(request, forWebSocket);
}

不难发现只要通过一个Interceptor返回response即可中断http后续的请求动作,如果服务端错误,response将会返回相应的错误码,总结最后的实现代码如下:

final class CacheControlInterceptor implements Interceptor {

    private final String mCacheDir;

    private final ILog mLog;

    CacheControlInterceptor(ILog log, String cacheDir){
        this.mCacheDir = cacheDir;
        this.mLog = log;
    }

    @Override
    public Response intercept(Chain chain) throws IOException {
        Request request = chain.request();
        boolean networkAvail = NetUtils.isNetworkAvailable();
        if (!networkAvail){
            Response cacheResponse = getCacheResponse(request);
            if(cacheResponse != null){
                return cacheResponse;
            }
        }
        Response networkResponse = chain.proceed(request);
        if(networkAvail && !networkResponse.isSuccessful()){
            Response cacheResponse = getCacheResponse(request);
            if(cacheResponse != null){
                return cacheResponse;
            }
        }
        return networkResponse;
    }

    private Response getCacheResponse(Request request) throws FileNotFoundException{

        if(!"get".equalsIgnoreCase(request.method())|| mCacheDir == null){
            return null;
        }

        String urlMd5 = Util.md5Hex(request.url().url().toString());
        File headerCacheFile = new File(new File(mCacheDir), urlMd5 + ".0");
        Response.Builder cacheResponseBuilder = null;
        if (headerCacheFile.exists()) {
            cacheResponseBuilder = new Response.Builder();
            cacheResponseBuilder.request(request);
            readCacheResponseHeaders(headerCacheFile,cacheResponseBuilder);
        }

        if(cacheResponseBuilder != null){
            Response cacheResponse = cacheResponseBuilder.build();
            File bodyCacheFile = new File(new File(mCacheDir), urlMd5 + ".1");
            Source cacheSource;
            if(!"gzip".equalsIgnoreCase(cacheResponse.header("Content-Encoding"))){
                cacheSource = Okio.source(bodyCacheFile);
            }else{
                cacheSource = new GzipSource(Okio.source(bodyCacheFile));
            }
            RealResponseBody responseBody = new RealResponseBody(cacheResponse.headers(),Okio.buffer(cacheSource));
            return cacheResponse.newBuilder().body(responseBody).build();
        }

        return null;
    }

    private void readCacheResponseHeaders(File headerCacheFile,Response.Builder responseBuilder) {
        BufferedReader reader = null;
        try {
            reader = new BufferedReader(new InputStreamReader(new FileInputStream(headerCacheFile)));
            String lineText;
            int code = 0;
            boolean isHeaderBegin = false;
            while((lineText = reader.readLine()) != null){

                if(isHeaderBegin){
                    int sepIndex = lineText.indexOf(":");
                    String headerName = lineText.substring(0,sepIndex);
                    String headerVal = lineText.substring(sepIndex + 1).trim();
                    responseBuilder.addHeader(headerName,headerVal);
                }

                if(code != 0 && NumberUtils.parseInt(lineText,0) != 0){
                    isHeaderBegin = true;
                    continue;
                }

                if(lineText.startsWith("HTTP/1.1")){
                    code = Integer.valueOf(lineText.substring(lineText.indexOf(" ")).trim());
                    responseBuilder.protocol(Protocol.HTTP_1_1);
                    responseBuilder.code(code);
                }
            }
        } catch (IOException ex) {
            ex.printStackTrace();
        } finally {
            IOUtils.closeSilently(reader);
        }
    }
}

参考资料

  1. http缓存: https://developers.google.cn/web/fundamentals/performance/optimizing-content-efficiency/http-caching?hl=zh-cn
  2. tomcat中对客户端的缓存机制: http://blog.csdn.net/liweisnake/article/details/8524179
  3. Okhttp Interceptors: https://github.com/square/okhttp/wiki/Interceptors

© 著作权归作者所有

共有 人打赏支持
叶大侠

叶大侠

粉丝 57
博文 44
码字总数 67312
作品 5
广州
程序员
加载中

评论(2)

OSC_vlkPzm
OSC_vlkPzm
不明觉厉
金贞花
金贞花
不错的文
Android Okhttp缓存:精细化每一个Request的CacheControl缓存控制策略(二)

Android Okhttp缓存:精细化每一个Request的CacheControl缓存控制策略(二) 之前我写的附录文章1,只是简单的使用缺省的方法实现Okhttp的缓存。现在使用CacheControl,精细化到每一个Reque...

开开心心过
2017/10/24
0
0
OkHttp 文档翻译之 Calls

Calls Http client 的工作是接收请求和生成响应。这个在原理上很简答的问题,在实现时变得很棘手。 Requests 每个 Http request 包含一个 URL,一个方法(例如 :GET or Post),和 Headers ...

黑泥卡
08/21
0
0
Android面试有迹可循(一)OkHttp3.9拦截器原理与区别

接上回 传送门 上回我们讲到,OkHttp的请求过程中有个非常重要的东西-“拦截器”,而且拦截器又分为interceptors和networkInterceptors两种,那它们具体有何区别呢?又要怎么来使用?现在来一...

SillyMonkey
05/19
0
0
feign和okhttp的结合

背景 使用feign可以很方便的调用各种http接口 http请求神器之Feign 那么feign是如何做到的呢? 分析 本质上默认场景feign仍然是使用httpClient进行调用的。 通过声明式的RequestMapping等注解...

Mr_Qi
07/11
0
0
Android Okhttp缓存:Cache,创建OkHttpClient实现(一)

Android Okhttp缓存Cache,创建OkHttpClient实现 Android Okhttp使用缓存通过Cache实现。在创建OkHttpClient实现。构造Cache需要传递一个缓存文件目录已经缓存的大小尺寸。 下面演示一个简单...

开开心心过
2017/10/24
0
0

没有更多内容

加载失败,请刷新页面

加载更多

CentOS7防火墙firewalld操作

firewalld Linux上新用的防火墙软件,跟iptables差不多的工具。 firewall-cmd 是 firewalld 的字符界面管理工具,firewalld是CentOS7的一大特性,最大的好处有两个:支持动态更新,不用重启服...

dingdayu
今天
1
0
关于组件化的最初步

一个工程可能会有多个版本,有国际版、国内版、还有针对各种不同的渠道化的打包版本、这个属于我们日常经常见到的打包差异化版本需求。 而对于工程的开发,比如以前的公司,分成了有三大块业...

DannyCoder
今天
2
0
Spring的Resttemplate发送带header的post请求

private HttpHeaders getJsonHeader() { HttpHeaders headers = new HttpHeaders(); MediaType type = MediaType.parseMediaType("application/json; charset=UTF-8"); ......

qiang123
昨天
3
0
Spring Cloud Gateway 之 Only one connection receive subscriber allowed

都说Spring Cloud Gateway好,我也来试试,可是配置了总是报下面这个错误: java.lang.IllegalStateException: Only one connection receive subscriber allowed. 困扰了我几天的问题,原来...

ThinkGem
昨天
27
0
学习设计模式——观察者模式

1. 认识观察者模式 1. 定义:定义对象之间一种一对多的依赖关系,当一个对象状态发生变化时,依赖该对象的其他对象都会得到通知并进行相应的变化。 2. 组织结构: Subject:目标对象类,会被...

江左煤郎
昨天
4
0

没有更多内容

加载失败,请刷新页面

加载更多

返回顶部
顶部