[MyBatis] 缓存模块实现原理剖析

架构探险之道

发布于 2020-05-26 10:13:22

7640

发布于 2020-05-26 10:13:22

文章被收录于专栏：架构探险之道架构探险之道

[MyBatis] 缓存模块实现原理剖析

简介

本文主要介绍 MyBatis 缓存模块，介绍其实现原理和配置方式，并分析了下一级缓存和二级缓存的特点和使用差异。

设计模式装饰器模式一级缓存源码分析小结二级缓存小结获取更多

手机用户请横屏获取最佳阅读体验，REFERENCES中是本文参考的链接，如需要链接和更多资源，可以扫码加入『知识星球』（文末）获取长期知识分享服务。

MyBatis 缓存模块实现了以下功能：

MyBatis 缓存的实现是基于 Map 的，从缓存里面读写数据是缓存模块的核心基础功能；
除核心功能之外，有很多额外的附加功能，如：防止缓存击穿，添加缓存清空策略（FIFO、 LRU）、序列化功能、日志能力、定时清空能力等；
附加功能可以以任意的组合附加到核心基础功能之上；基于 Map 核心缓存能力，将阻塞、清空策略、序列化、日志等等能力以任意组合的方式优雅的增强；

这是 MyBatis 缓存模块实现最大的难题，用动态代理或者继承的方式扩展多种附加能力的传统方式存在以下问题：这些方式是静态的，用户不能控制增加行为的方式和时机；

另外，新功能的存在多种组合，使用继承可能导致大量子类存在。综上，MyBtis 缓存模块采用了装饰器模式实现了缓存模块。

设计模式

装饰器模式

装饰器模式是一种用于代替继承的技术，无需通过继承增加子类就能扩展对象的新功能。使用对象的关联关系代替继承关系，更加灵活，同时避免类型体系的快速膨胀。装饰器 UML 类图如下：

组件（Component）：组件接口定义了全部组件类和装饰器实现的行为；
组件实现类（ConcreteComponent）：实现 Component 接口，组件实现类就是被装饰器装饰的原始对象，新功能或者附加功能都是通过装饰器添加到该类的对象上的；
装饰器抽象类（Decorator）：实现 Component 接口的抽象类，在其中封装了一个 Component 对象，也就是被装饰的对象；
具体装饰器类（ConcreteDecorator）：该实现类要向被装饰的对象添加某些功能；

装饰器相对于继承，装饰器模式灵活性更强，扩展性更强：

灵活性：装饰器模式将功能切分成一个个独立的装饰器，在运行期可以根据需要动态的添加功能，甚至对添加的新功能进行自由的组合；
扩展性：当有新功能要添加的时候，只需要添加新的装饰器实现类，然后通过组合方式添加这个新装饰器，无需修改已有代码，符合开闭原则；

装饰器模式使用举例：

IO 中输入流和输出流的设计

BufferedReader bufferedReader = new BufferedReader(new InputStreamReader(new FileInputStream("c://a.txt")));

对网络爬虫的自定义增强，可增强的功能包括：多线程能力、缓存、自动生成报表、黑白名单、random 触发等

装饰器模式

MyBatis 缓存模块是一个经典的使用装饰器实现的模块，类图如下：

Cache：Cache 接口是缓存模块的核心接口，定义了缓存的基本操作；
PerpetualCache：在缓存模块中扮演 ConcreteComponent 角色，使用 HashMap 来实现 cache 的相关操作；
BlockingCache：阻塞版本的缓存装饰器，保证只有一个线程到数据库去查找指定的 key 对应的数据；

BlockingCache 是阻塞版本的缓存装饰器，这个装饰器通过 ConcurrentHashMap 对锁的粒度进行了控制，提高加锁后系统代码运行的效率（注：缓存雪崩的问题可以使用细粒度锁的方式提升锁性能），源码分析见：org.apache.ibatis.cache.decorators.BlockingCache；

除了 BlockingCache 之外，缓存模块还有其他的装饰器如：

LoggingCache：日志能力的缓存；
ScheduledCache：定时清空的缓存；
BlockingCache：阻塞式缓存；
SerializedCache：序列化能力的缓存；
SynchronizedCache：进行同步控制的缓存；

分类

基本缓存
淘汰算法缓存
装饰器缓存

一级缓存

本地缓存，会话层面进行缓存，默认是开启的，不需要任何配置。

源码分析

//BaseExecutor#query
public <E> List<E> query(MappedStatement ms, Object parameter, RowBounds rowBounds, ResultHandler resultHandler, CacheKey key, BoundSql boundSql) throws SQLException {
    ErrorContext.instance().resource(ms.getResource()).activity("executing a query").object(ms.getId());
    if (closed) {
      throw new ExecutorException("Executor was closed.");
    }
    if (queryStack == 0 && ms.isFlushCacheRequired()) {
      clearLocalCache();
    }
    List<E> list;
    try {
      queryStack++;
      list = resultHandler == null ? (List<E>) localCache.getObject(key) : null;
      if (list != null) {
        handleLocallyCachedOutputParameters(ms, key, parameter, boundSql);
      } else {
        list = queryFromDatabase(ms, parameter, rowBounds, resultHandler, key, boundSql);
      }
    } finally {
      queryStack--;
    }
    if (queryStack == 0) {
      for (DeferredLoad deferredLoad : deferredLoads) {
        deferredLoad.load();
      }
      // issue #601
      deferredLoads.clear();
      if (configuration.getLocalCacheScope() == LocalCacheScope.STATEMENT) {
        // issue #482
        clearLocalCache();
      }
    }
    return list;
  }

localCacheScope 设置为STATEMENT时关闭一级缓存。每次执行完会清空掉当前会话的缓存。

查询数据

为了验证一级缓存，我们先关闭掉全局的二级缓存配置：

 <settings>
        <!-- 打印查询语句 -->
        <setting name="logImpl" value="STDOUT_LOGGING" />

        <!-- 控制全局缓存（二级缓存），默认 true-->
        <setting name="cacheEnabled" value="false"/>

        <setting name="localCacheScope" value="SESSION"/>
    </settings>

 @Test
    void testSelect() throws IOException {
        SqlSession session = sqlSessionFactory.openSession();
        try {
            MemberMapper mapper = session.getMapper(MemberMapper.class);
            Member blog = mapper.selectByPrimaryKey(116);
            Member blog2 = mapper.selectByPrimaryKey(116);
            System.out.println(blog);
        } finally {
            session.close();
        }
    }

日志打印

Opening JDBC Connection
Sun May 17 17:26:19 CST 2020 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
Created connection 104716441.
Setting autocommit to false on JDBC Connection [com.mysql.jdbc.JDBC4Connection@63dd899]
==>  Preparing: select id, name, age, addr, status from tb_member where id = ? 
==> Parameters: 116(Integer)
<==    Columns: id, name, age, addr, status
<==        Row: 116, methodA1, 18, Guangzhou, -1
<==      Total: 1
Member(id=116, name=methodA1, age=18, addr=Guangzhou, status=-1)
Resetting autocommit to true on JDBC Connection [com.mysql.jdbc.JDBC4Connection@63dd899]
Closing JDBC Connection [com.mysql.jdbc.JDBC4Connection@63dd899]
Returned connection 104716441 to pool.

Disconnected from the target VM, address: '127.0.0.1:59256', transport: 'socket'

Process finished with exit code 0

很明显第二次查询没有触发数据的查询操作。那么这个多会话共享的么？

 @Test
    void testSelect() throws IOException {
        SqlSession session = sqlSessionFactory.openSession(); 
        SqlSession session2 = sqlSessionFactory.openSession(); 
        try {
            MemberMapper mapper = session.getMapper(MemberMapper.class);
            MemberMapper mapper2 = session2.getMapper(MemberMapper.class);
            Member blog = mapper.selectByPrimaryKey(116);
            Member blog2 = mapper2.selectByPrimaryKey(116);
            System.out.println(blog);
        } finally {
            session.close();
        }
    }

日志打印了两次查询，可以得出不同session之间时不能共享一级缓存的。

那么一级缓存何时放入？何时获取？何时清空？一级缓存的Key又是如何构成的呢？带着问题我们去看下源码。

何时放入

//BaseExecutor
private <E> List<E> queryFromDatabase(MappedStatement ms, Object parameter, RowBounds rowBounds, ResultHandler resultHandler, CacheKey key, BoundSql boundSql) throws SQLException {
    List<E> list;
    localCache.putObject(key, EXECUTION_PLACEHOLDER);
    try {
      list = doQuery(ms, parameter, rowBounds, resultHandler, boundSql);
    } finally {
      localCache.removeObject(key);
    }
    localCache.putObject(key, list);
    if (ms.getStatementType() == StatementType.CALLABLE) {
      localOutputParameterCache.putObject(key, parameter);
    }
    return list;
  }

何时获取

//BaseExecutor#query
@SuppressWarnings("unchecked")
@Override
public <E> List<E> query(MappedStatement ms, Object parameter, RowBounds rowBounds, ResultHandler resultHandler, CacheKey key, BoundSql boundSql) throws SQLException {
  ErrorContext.instance().resource(ms.getResource()).activity("executing a query").object(ms.getId());
  if (closed) {
    throw new ExecutorException("Executor was closed.");
  }
  if (queryStack == 0 && ms.isFlushCacheRequired()) {
      // flushCache="true"时，即使是查询，也清空一级缓存
      clearLocalCache();
    }
  List<E> list;
  try {
    // 防止递归查询重复处理缓存
    queryStack++;
    //获取一级缓存
    list = resultHandler == null ? (List<E>) localCache.getObject(key) : null;
    if (list != null) {
      handleLocallyCachedOutputParameters(ms, key, parameter, boundSql);
    } else {
       // 真正的查询流程
      list = queryFromDatabase(ms, parameter, rowBounds, resultHandler, key, boundSql);
    }
  } finally {
    queryStack--;
  }
  if (queryStack == 0) {
    for (DeferredLoad deferredLoad : deferredLoads) {
      deferredLoad.load();
    }
    // issue #601
    deferredLoads.clear();
    if (configuration.getLocalCacheScope() == LocalCacheScope.STATEMENT) {
      // issue #482
      clearLocalCache();
    }
  }
  return list;
}

构造缓存key

//BaseExecutor#query
@Override
public <E> List<E> query(MappedStatement ms, Object parameter, RowBounds rowBounds, ResultHandler resultHandler) throws SQLException {
  BoundSql boundSql = ms.getBoundSql(parameter);
  CacheKey key = createCacheKey(ms, parameter, rowBounds, boundSql);
  return query(ms, parameter, rowBounds, resultHandler, key, boundSql);
}
//创建缓存KEY
@Override
  public CacheKey createCacheKey(MappedStatement ms, Object parameterObject, RowBounds rowBounds, BoundSql boundSql) {
    if (closed) {
      throw new ExecutorException("Executor was closed.");
    }
    CacheKey cacheKey = new CacheKey();
    cacheKey.update(ms.getId());
    cacheKey.update(rowBounds.getOffset());
    cacheKey.update(rowBounds.getLimit());
    cacheKey.update(boundSql.getSql());
    List<ParameterMapping> parameterMappings = boundSql.getParameterMappings();
    TypeHandlerRegistry typeHandlerRegistry = ms.getConfiguration().getTypeHandlerRegistry();
    // mimic DefaultParameterHandler logic
    for (ParameterMapping parameterMapping : parameterMappings) {
      if (parameterMapping.getMode() != ParameterMode.OUT) {
        Object value;
        String propertyName = parameterMapping.getProperty();
        if (boundSql.hasAdditionalParameter(propertyName)) {
          value = boundSql.getAdditionalParameter(propertyName);
        } else if (parameterObject == null) {
          value = null;
        } else if (typeHandlerRegistry.hasTypeHandler(parameterObject.getClass())) {
          value = parameterObject;
        } else {
          MetaObject metaObject = configuration.newMetaObject(parameterObject);
          value = metaObject.getValue(propertyName);
        }
        cacheKey.update(value);
      }
    }
    if (configuration.getEnvironment() != null) {
      // issue #176
      cacheKey.update(configuration.getEnvironment().getId());
    }
    return cacheKey;
  }
//CacheKey#update
 public void update(Object object) {
    int baseHashCode = object == null ? 1 : ArrayUtil.hashCode(object);

    count++;
    checksum += baseHashCode;
    baseHashCode *= count;

    hashcode = multiplier * hashcode + baseHashCode;

    updateList.add(object);
  }

何时清空

同一个会话中，update(包含delete)会导致一级缓存被清空
select 时，如果配置了 flushCache=true，也会清空
事务回滚时。
事务提交时。

//DefaultSqlSession#delete
@Override
  public int delete(String statement) {
    return update(statement, null);
  }

一级缓存跨会话数据过时

@Test
void testSelect() throws IOException {
    SqlSession session = sqlSessionFactory.openSession();
    SqlSession session2 = sqlSessionFactory.openSession();
    try {
        MemberMapper mapper = session.getMapper(MemberMapper.class);

        MemberMapper mapper2 = session2.getMapper(MemberMapper.class);
        Member blog = mapper.selectByPrimaryKey(106);

        System.out.println(">>>mapper1:"+blog.toString());

        blog.setName("修改xxx");
        mapper.updateByPrimaryKey(blog);

        System.out.println(">>>mapper1 again:"+mapper2.selectByPrimaryKey(106));
    } finally {
        session.close();
    }
}//输出日志

Created connection 166694583.
Setting autocommit to false on JDBC Connection [com.mysql.jdbc.JDBC4Connection@9ef8eb7]
==>  Preparing: select id, name, age, addr, status from tb_member where id = ? 
==> Parameters: 106(Integer)
<==    Columns: id, name, age, addr, status
<==        Row: 106, 修改4, 16, Beijing, 0
<==      Total: 1
>>>mapper1:Member(id=106, name=修改4, age=16, addr=Beijing, status=0)
==>  Preparing: update tb_member set name = ?, age = ?, addr = ?, status = ? where id = ? 
==> Parameters: 修改xxx(String), 16(Integer), Beijing(String), 0(Integer), 106(Integer)
<==    Updates: 1
Opening JDBC Connection
Mon May 18 23:51:46 CST 2020 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
Created connection 806813022.
Setting autocommit to false on JDBC Connection [com.mysql.jdbc.JDBC4Connection@3016fd5e]
==>  Preparing: select id, name, age, addr, status from tb_member where id = ? 
==> Parameters: 106(Integer)
<==    Columns: id, name, age, addr, status
<==        Row: 106, 修改4, 16, Beijing, 0
<==      Total: 1
>>>mapper1 again:Member(id=106, name=修改4, age=16, addr=Beijing, status=0)
Rolling back JDBC Connection [com.mysql.jdbc.JDBC4Connection@9ef8eb7]
Resetting autocommit to true on JDBC Connection [com.mysql.jdbc.JDBC4Connection@9ef8eb7]
Closing JDBC Connection [com.mysql.jdbc.JDBC4Connection@9ef8eb7]
Returned connection 166694583 to pool.

可以看到mapper1两次查询的数据时没变化的，不会因为另外一个会话修改了数据而获取到最新的数据。

小结

一级缓存工作范围是一个会话，不支持跨会话，不同会话，会出现读取过时数据的问题。
更新和删除时会清空，查询时会放入缓存，但是也可以通过配置在查询时清空缓存。
配置为<setting name="localCacheScope" value="SESSION"/>

二级缓存

二级缓存解决的时一级缓存跨会话共享的问题，范围是namespace级别的，可以被多个SqlSession共享，生命周期和应用同步。

开启二级缓存

<!-- 控制全局缓存（二级缓存），默认 true-->
<setting name="cacheEnabled" value="true"/>

总开关默认开启，但是每个Mapper的二级缓存开关默认是关闭的。如果要开启，则需要配置<cache/>标签。

//CachingExecutor#query 
@Override
  public <E> List<E> query(MappedStatement ms, Object parameterObject, RowBounds rowBounds, ResultHandler resultHandler, CacheKey key, BoundSql boundSql)
      throws SQLException {
    Cache cache = ms.getCache();
    //判断是否配置了缓存的标签
    if (cache != null) {
      flushCacheIfRequired(ms);
      if (ms.isUseCache() && resultHandler == null) {
        ensureNoOutParams(ms, boundSql);
        @SuppressWarnings("unchecked")
        List<E> list = (List<E>) tcm.getObject(cache, key);
        if (list == null) {
          list = delegate.query(ms, parameterObject, rowBounds, resultHandler, key, boundSql);
          tcm.putObject(cache, key, list); // issue #578 and #116
        }
        return list;
      }
    }
    return delegate.query(ms, parameterObject, rowBounds, resultHandler, key, boundSql);
  }

验证二级缓存

/**
 * 通过 SqlSession.getMapper(XXXMapper.class)  接口方式
 *
 * @throws IOException
 */
@Test
void secondCache() throws IOException {
    SqlSession session = sqlSessionFactory.openSession();
    SqlSession session2 = sqlSessionFactory.openSession();
    try {
        MemberMapper mapper = session.getMapper(MemberMapper.class);

        MemberMapper mapper2 = session2.getMapper(MemberMapper.class);
        Member blog = mapper.selectByPrimaryKey(106);

        session.commit();

        System.out.println("跨会话查询："+mapper2.selectByPrimaryKey(106));;
    } finally {
        session.close();
    }
}

Opening JDBC Connection
Mon May 18 23:47:27 CST 2020 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
Created connection 516875052.
Setting autocommit to false on JDBC Connection [com.mysql.jdbc.JDBC4Connection@1ecee32c]
==>  Preparing: select id, name, age, addr, status from tb_member where id = ? 
==> Parameters: 106(Integer)
<==    Columns: id, name, age, addr, status
<==        Row: 106, 修改4, 16, Beijing, 0
<==      Total: 1
Cache Hit Ratio [com.yido.example.dao.mapper.MemberMapper]: 0.5
跨会话查询：Member(id=106, name=修改4, age=16, addr=Beijing, status=0)
Resetting autocommit to true on JDBC Connection [com.mysql.jdbc.JDBC4Connection@1ecee32c]
Closing JDBC Connection [com.mysql.jdbc.JDBC4Connection@1ecee32c]
Returned connection 516875052 to pool.

可以看到只去数据库查询了一次。

跨会话数据共享

/**
 * 通过 SqlSession.getMapper(XXXMapper.class)  接口方式
 * 跨会话数据共享
 * @throws IOException
 */
@Test
void secondCache() throws IOException {
    SqlSession session = sqlSessionFactory.openSession();
    SqlSession session2 = sqlSessionFactory.openSession();
    try {
        MemberMapper mapper = session.getMapper(MemberMapper.class);

        MemberMapper mapper2 = session2.getMapper(MemberMapper.class);
        Member blog = mapper2.selectByPrimaryKey(106);
        blog.setName("修改--11111");
        mapper2.updateByPrimaryKey(blog);
        session2.commit();
        System.out.println("跨会话查询："+mapper.selectByPrimaryKey(106));;
    } finally {
        session.close();
    }
}

Cache Hit Ratio [com.yido.example.dao.mapper.MemberMapper]: 0.0
Opening JDBC Connection
Mon May 18 23:56:35 CST 2020 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
Created connection 516875052.
Setting autocommit to false on JDBC Connection [com.mysql.jdbc.JDBC4Connection@1ecee32c]
==>  Preparing: select id, name, age, addr, status from tb_member where id = ? 
==> Parameters: 106(Integer)
<==    Columns: id, name, age, addr, status
<==        Row: 106, 修改4, 16, Beijing, 0
<==      Total: 1
==>  Preparing: update tb_member set name = ?, age = ?, addr = ?, status = ? where id = ? 
==> Parameters: 修改--11111(String), 16(Integer), Beijing(String), 0(Integer), 106(Integer)
<==    Updates: 1
Committing JDBC Connection [com.mysql.jdbc.JDBC4Connection@1ecee32c]
Cache Hit Ratio [com.yido.example.dao.mapper.MemberMapper]: 0.0
Opening JDBC Connection
Mon May 18 23:56:35 CST 2020 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
Created connection 743648472.
Setting autocommit to false on JDBC Connection [com.mysql.jdbc.JDBC4Connection@2c532cd8]
==>  Preparing: select id, name, age, addr, status from tb_member where id = ? 
==> Parameters: 106(Integer)
<==    Columns: id, name, age, addr, status
<==        Row: 106, 修改--11111, 16, Beijing, 0
<==      Total: 1
跨会话查询：Member(id=106, name=修改--11111, age=16, addr=Beijing, status=0)

可以观察到，跨会话的时候，查询到的数据时最新修改的数据。此处需要注意的式，操作完，需要执行commit才能刷新缓存。

显示关闭缓存

@Override
public <E> List<E> query(MappedStatement ms, Object parameterObject, RowBounds rowBounds, ResultHandler resultHandler, CacheKey key, BoundSql boundSql)
    throws SQLException {
  Cache cache = ms.getCache();
  //判断是否配置了缓存的标签
  if (cache != null) {
    flushCacheIfRequired(ms);
    //如果配置为false，则每次查询时不使用缓存
    if (ms.isUseCache() && resultHandler == null) {
      ensureNoOutParams(ms, boundSql);
      @SuppressWarnings("unchecked")
      List<E> list = (List<E>) tcm.getObject(cache, key);
      if (list == null) {
        list = delegate.query(ms, parameterObject, rowBounds, resultHandler, key, boundSql);
        tcm.putObject(cache, key, list); // issue #578 and #116
      }
      return list;
    }
  }
  return delegate.query(ms, parameterObject, rowBounds, resultHandler, key, boundSql);
}

//CachingExecutor#flushCacheIfRequired
 private void flushCacheIfRequired(MappedStatement ms) {
    Cache cache = ms.getCache();
   //如果配置为true，则每次查询时清空
    if (cache != null && ms.isFlushCacheRequired()) {
      tcm.clear(cache);
    }
  }