聊聊hikari连接池的idleTimeout及minimumIdle属性

本文主要研究一个hikari连接池的idleTimeout及minimumIdle属性

idleTimeout

默认是600000毫秒,即10分钟。如果idleTimeout+1秒>maxLifetime 且 maxLifetime>0,则会被重置为0;如果idleTimeout!=0且小于10秒,则会被重置为10秒。如果idleTimeout=0则表示空闲的连接在连接池中永远不被移除。 只有当minimumIdle小于maximumPoolSize时,这个参数才生效,当空闲连接数超过minimumIdle,而且空闲时间超过idleTimeout,则会被移除。

minimumIdle

控制连接池空闲连接的最小数量,当连接池空闲连接少于minimumIdle,而且总共连接数不大于maximumPoolSize时,HikariCP会尽力补充新的连接。为了性能考虑,不建议设置此值,而是让HikariCP把连接池当做固定大小的处理,默认minimumIdle与maximumPoolSize一样。 当minIdle<0或者minIdle>maxPoolSize,则被重置为maxPoolSize,该值默认为10。

HikariPool.HouseKeeper

HikariCP-2.7.6-sources.jar!/com/zaxxer/hikari/pool/HikariPool.java

    private final long HOUSEKEEPING_PERIOD_MS = Long.getLong("com.zaxxer.hikari.housekeeping.periodMs", SECONDS.toMillis(30));

    this.houseKeeperTask = houseKeepingExecutorService.scheduleWithFixedDelay(new HouseKeeper(), 100L, HOUSEKEEPING_PERIOD_MS, MILLISECONDS);

   /**
    * The house keeping task to retire and maintain minimum idle connections.
    */
   private final class HouseKeeper implements Runnable
   {
      private volatile long previous = plusMillis(currentTime(), -HOUSEKEEPING_PERIOD_MS);

      @Override
      public void run()
      {
         try {
            // refresh timeouts in case they changed via MBean
            connectionTimeout = config.getConnectionTimeout();
            validationTimeout = config.getValidationTimeout();
            leakTaskFactory.updateLeakDetectionThreshold(config.getLeakDetectionThreshold());

            final long idleTimeout = config.getIdleTimeout();
            final long now = currentTime();

            // Detect retrograde time, allowing +128ms as per NTP spec.
            if (plusMillis(now, 128) < plusMillis(previous, HOUSEKEEPING_PERIOD_MS)) {
               LOGGER.warn("{} - Retrograde clock change detected (housekeeper delta={}), soft-evicting connections from pool.",
                           poolName, elapsedDisplayString(previous, now));
               previous = now;
               softEvictConnections();
               return;
            }
            else if (now > plusMillis(previous, (3 * HOUSEKEEPING_PERIOD_MS) / 2)) {
               // No point evicting for forward clock motion, this merely accelerates connection retirement anyway
               LOGGER.warn("{} - Thread starvation or clock leap detected (housekeeper delta={}).", poolName, elapsedDisplayString(previous, now));
            }

            previous = now;

            String afterPrefix = "Pool ";
            if (idleTimeout > 0L && config.getMinimumIdle() < config.getMaximumPoolSize()) {
               logPoolState("Before cleanup ");
               afterPrefix = "After cleanup  ";

               final List<PoolEntry> notInUse = connectionBag.values(STATE_NOT_IN_USE);
               int toRemove = notInUse.size() - config.getMinimumIdle();
               for (PoolEntry entry : notInUse) {
                  if (toRemove > 0 && elapsedMillis(entry.lastAccessed, now) > idleTimeout && connectionBag.reserve(entry)) {
                     closeConnection(entry, "(connection has passed idleTimeout)");
                     toRemove--;
                  }
               }
            }

            logPoolState(afterPrefix);

            fillPool(); // Try to maintain minimum connections
         }
         catch (Exception e) {
            LOGGER.error("Unexpected exception in housekeeping task", e);
         }
      }
   }

这个HouseKeeper是一个定时任务,在HikariPool构造器里头初始化,默认的是初始化后100毫秒执行,之后每执行完一次之后隔HOUSEKEEPING_PERIOD_MS(30秒)时间执行。 这个定时任务的作用就是根据idleTimeout的值,移除掉空闲超时的连接。 首先检测时钟是否倒退,如果倒退了则立即对过期的连接进行标记evict;之后当idleTimeout>0且配置的minimumIdle<maximumPoolSize时才开始处理超时的空闲连接。 取出状态是STATE_NOT_IN_USE的连接数,如果大于minimumIdle,则遍历STATE_NOT_IN_USE的连接的连接,将空闲超时达到idleTimeout的连接从connectionBag移除掉,若移除成功则关闭该连接,然后toRemove—。 在空闲连接移除之后,再调用fillPool,尝试补充空间连接数到minimumIdle值

HikariPool.fillPool

HikariCP-2.7.6-sources.jar!/com/zaxxer/hikari/pool/HikariPool.java

         private final PoolEntryCreator POOL_ENTRY_CREATOR = new PoolEntryCreator(null /*logging prefix*/);
      private final PoolEntryCreator POST_FILL_POOL_ENTRY_CREATOR = new PoolEntryCreator("After adding ");
      LinkedBlockingQueue<Runnable> addConnectionQueue = new LinkedBlockingQueue<>(config.getMaximumPoolSize());
      this.addConnectionQueue = unmodifiableCollection(addConnectionQueue);
      this.addConnectionExecutor = createThreadPoolExecutor(addConnectionQueue, poolName + " connection adder", threadFactory, new ThreadPoolExecutor.DiscardPolicy());

   /**
    * Fill pool up from current idle connections (as they are perceived at the point of execution) to minimumIdle connections.
    */
   private synchronized void fillPool()
   {
      final int connectionsToAdd = Math.min(config.getMaximumPoolSize() - getTotalConnections(), config.getMinimumIdle() - getIdleConnections())
                                   - addConnectionQueue.size();
      for (int i = 0; i < connectionsToAdd; i++) {
         addConnectionExecutor.submit((i < connectionsToAdd - 1) ? POOL_ENTRY_CREATOR : POST_FILL_POOL_ENTRY_CREATOR);
      }
   }

PoolEntryCreator

   /**
    * Creating and adding poolEntries (connections) to the pool.
    */
   private final class PoolEntryCreator implements Callable<Boolean>
   {
      private final String loggingPrefix;

      PoolEntryCreator(String loggingPrefix)
      {
         this.loggingPrefix = loggingPrefix;
      }

      @Override
      public Boolean call() throws Exception
      {
         long sleepBackoff = 250L;
         while (poolState == POOL_NORMAL && shouldCreateAnotherConnection()) {
            final PoolEntry poolEntry = createPoolEntry();
            if (poolEntry != null) {
               connectionBag.add(poolEntry);
               LOGGER.debug("{} - Added connection {}", poolName, poolEntry.connection);
               if (loggingPrefix != null) {
                  logPoolState(loggingPrefix);
               }
               return Boolean.TRUE;
            }

            // failed to get connection from db, sleep and retry
            quietlySleep(sleepBackoff);
            sleepBackoff = Math.min(SECONDS.toMillis(10), Math.min(connectionTimeout, (long) (sleepBackoff * 1.5)));
         }
         // Pool is suspended or shutdown or at max size
         return Boolean.FALSE;
      }

      /**
       * We only create connections if we need another idle connection or have threads still waiting
       * for a new connection.  Otherwise we bail out of the request to create.
       *
       * @return true if we should create a connection, false if the need has disappeared
       */
      private boolean shouldCreateAnotherConnection() {
         return getTotalConnections() < config.getMaximumPoolSize() &&
            (connectionBag.getWaitingThreadCount() > 0 || getIdleConnections() < config.getMinimumIdle());
      }
   }

shouldCreateAnotherConnection方法决定了是否需要添加新的连接

createPoolEntry

   /**
    * Creating new poolEntry.  If maxLifetime is configured, create a future End-of-life task with 2.5% variance from
    * the maxLifetime time to ensure there is no massive die-off of Connections in the pool.
    */
   private PoolEntry createPoolEntry()
   {
      try {
         final PoolEntry poolEntry = newPoolEntry();

         final long maxLifetime = config.getMaxLifetime();
         if (maxLifetime > 0) {
            // variance up to 2.5% of the maxlifetime
            final long variance = maxLifetime > 10_000 ? ThreadLocalRandom.current().nextLong( maxLifetime / 40 ) : 0;
            final long lifetime = maxLifetime - variance;
            poolEntry.setFutureEol(houseKeepingExecutorService.schedule(
               () -> {
                  if (softEvictConnection(poolEntry, "(connection has passed maxLifetime)", false /* not owner */)) {
                     addBagItem(connectionBag.getWaitingThreadCount());
                  }
               },
               lifetime, MILLISECONDS));
         }

         return poolEntry;
      }
      catch (Exception e) {
         if (poolState == POOL_NORMAL) { // we check POOL_NORMAL to avoid a flood of messages if shutdown() is running concurrently
            LOGGER.debug("{} - Cannot acquire connection from data source", poolName, (e instanceof ConnectionSetupException ? e.getCause() : e));
         }
         return null;
      }
   }

createPoolEntry方法创建一个poolEntry,同时给它的lifetime过期设定了一个延时任务。

小结

  • HouseKeeper是一个定时任务,在HikariPool构造器里头初始化,默认的是初始化后100毫秒执行,之后每执行完一次之后隔HOUSEKEEPING_PERIOD_MS(30秒)时间执行。
  • 如果发现时钟倒退,则立即标记evict连接,然后退出;否则都会执行fillPool,来试图维持空闲连接到minimumIdle的数值
  • 当idleTimeout>0且配置的minimumIdle<maximumPoolSize时,会移除超过idleTimeout的空闲连接,否则不做操作,继续往下执行fillPool
  • 当minIdle<0或者minIdle>maxPoolSize,则minIdle被重置为maxPoolSize,该值默认为10,官方建议设置为一致,当做固定大小的连接池处理提高性能

idleTimeout有点类似tomcat jdbc pool里头的min-evictable-idle-time-millis参数。不同的是tomcat jdbc pool的连接泄露检测以及空闲连接清除的工作都放在一个名为PoolCleaner的timerTask中处理,该任务的执行间隔为timeBetweenEvictionRunsMillis,默认为5秒;而hikari的连接泄露是每次getConnection的时候单独触发一个延时任务来处理,而空闲连接的清除则是使用HouseKeeper定时任务来处理,其运行间隔由com.zaxxer.hikari.housekeeping.periodMs环境变量控制,默认为30秒。

doc

  • configuration-knobs-baby

原文发布于微信公众号 - 码匠的流水账(geek_luandun)

原文发表时间:2018-02-05

本文参与腾讯云自媒体分享计划,欢迎正在阅读的你也加入,一起分享。

发表于

我来说两句

0 条评论
登录 后参与评论

相关文章

来自专栏Spring相关

oauth2.0实现sso单点登录的方式和相关代码

百科:SSO英文全称Single Sign On,单点登录。SSO是在多个应用系统中,用户只需要登录一次就可以访问所有相互信任的应用系统。它包括可以将这次主要的...

65120
来自专栏坚毅的PHP

Hbase 源码分析之 Get 流程及rpc原理

分析版本为hbase 0.94 附上趋势团队画的图: rpc角色表: HBase通信信道 HBase的通信接口 客户端 服务端 HBase Cl...

55040
来自专栏李蔚蓬的专栏

1.Android系统源代码目录与系统目录

想要看完整个Android的源代码,需要懂C、懂脚本、懂Java、软硬兼通。所以一般情况下,我们了解源代码的框架结构,出了问题知道从哪里着手解决就可以了。这就好...

79320
来自专栏IT进修之路

原 JAVA的那些事儿

22670
来自专栏james大数据架构

MVC中局部视图的使用

加载部分视图 $("#result").load("/home/message",function(){ //加载完之后隐藏进度条 });  public Ac...

21270
来自专栏Java编程技术

创建线程以及线程池时候要指定与业务相关的名字,以便于追溯问题

日常开发中当一个应用中需要创建多个线程或者线程池时候最好给每个线程或者线程池根据业务类型设置具体的名字,以便在出现问题时候方便进行定位,下面就通过实例来说明不设...

9610
来自专栏大内老A

[WCF的Binding模型]之三:信道监听器(Channel Listener)

信道管理器是信道的创建者,一般来说信道栈的中每个信道对应着一个信道管理器。基于不同的消息处理的功能,将我们需要将相应的信道按照一定的顺序能组织起来构成一个信道栈...

21050
来自专栏移动开发的那些事儿

BlockCanary源码解析

如上代码中的loop()方法是Looper中的,我们的目的是监测主线程的卡顿问题,因为UI更新界面都是在主线程中进行的,所以在主线程中做耗时操作可能会造成界面卡...

16420
来自专栏Java面试通关手册

面试必备之深入理解自旋锁

分享一个我自己总结的Java学习的系统知识点以及面试问题,目前已经开源,会一直完善下去,欢迎建议和指导欢迎Star: https://github.com/Sn...

65730
来自专栏Android 研究

OKHttp源码解析(八)--中阶之连接与请求前奏

在http请求中,对于请求速度提升和降低延迟,keepalive在网络连接发挥着重大作用。

37820

扫码关注云+社区

领取腾讯云代金券