文章/答案/技术大牛

发布

社区首页 >专栏 >内存分配统计视图 | 全方位认识 sys 系统库

内存分配统计视图 | 全方位认识 sys 系统库

原创

沃趣科技

修改于 2018-09-05 08:01:08

1.8K01

代码可运行

文章被收录于专栏：沃趣科技沃趣科技

运行总次数：1

代码可运行

在上一篇《按 file 分组统计视图 | 全方位认识 sys 系统库》中，我们介绍了sys 系统库中按 file 分组统计的视图，本期的内容将为大家介绍内存事件和innodb buffer pool内存分配的统计视图。下面请跟随我们一起开始 sys 系统库的系统学习之旅吧。

PS：

对innodb buffer pool的统计视图对数据库的性能影响较大（可能会导致性能陡降），它主要是提供给专业DBA人员做问题分析排查使用，一般情况下不要随意使用
对innodb buffer pool的统计视图数据来源于information_schema系统库，考虑到大家可能有MySQL 5.7之前的版本中使用需求，所以本文中特意列出了对innodb buffer pool的统计视图的select语句文本

01.innodb_buffer_stats_by_schema,x$innodb_buffer_stats_by_schema

按照schema分组的 InnoDB buffer pool统计信息，默认按照分配的buffer size大小降序排序--allocated字段。数据来源：information_schema.innodb_buffer_page

视图select语句文本如下：

# 不带x$前缀的视图select语句文本
SELECT IF(LOCATE('.', ibp.table_name) = 0, 'InnoDB System', REPLACE(SUBSTRING_INDEX(ibp.table_name, '.', 1), '`', '')) AS object_schema,
  sys.format_bytes(SUM(IF(ibp.compressed_size = 0, 16384, compressed_size))) AS allocated,
  sys.format_bytes(SUM(ibp.data_size)) AS data,
  COUNT(ibp.page_number) AS pages,
  COUNT(IF(ibp.is_hashed = 'YES', 1, NULL)) AS pages_hashed,
  COUNT(IF(ibp.is_old = 'YES', 1, NULL)) AS pages_old,
  ROUND(SUM(ibp.number_records)/COUNT(DISTINCT ibp.index_name)) AS rows_cached
FROM information_schema.innodb_buffer_page ibp
WHERE table_name IS NOT NULL
GROUP BY object_schema
ORDER BY SUM(IF(ibp.compressed_size = 0, 16384, compressed_size)) DESC;

# 带x$前缀的视图select语句文本
SELECT IF(LOCATE('.', ibp.table_name) = 0, 'InnoDB System', REPLACE(SUBSTRING_INDEX(ibp.table_name, '.', 1), '`', '')) AS object_schema,
  SUM(IF(ibp.compressed_size = 0, 16384, compressed_size)) AS allocated,
  SUM(ibp.data_size) AS data,
  COUNT(ibp.page_number) AS pages,
  COUNT(IF(ibp.is_hashed = 'YES', 1, NULL)) AS pages_hashed,
  COUNT(IF(ibp.is_old = 'YES', 1, NULL)) AS pages_old,
  ROUND(IFNULL(SUM(ibp.number_records)/NULLIF(COUNT(DISTINCT ibp.index_name), 0), 0)) AS rows_cached
FROM information_schema.innodb_buffer_page ibp
WHERE table_name IS NOT NULL
GROUP BY object_schema
ORDER BY SUM(IF(ibp.compressed_size = 0, 16384, compressed_size)) DESC;

下面我们看看使用该视图查询返回的结果。

# 不带x$前缀的视图
admin@localhost : sys 06:15:41> select * from innodb_buffer_stats_by_schema;
+---------------+------------+-----------+-------+--------------+-----------+-------------+
| object_schema | allocated  | data      | pages | pages_hashed | pages_old | rows_cached |
+---------------+------------+-----------+-------+--------------+-----------+-------------+
| InnoDB System | 23.73 MiB  | 21.76 MiB |  1519 |            0 |        24 |      21474 |
| mysql        | 240.00 KiB | 14.57 KiB |    15 |            0 |        15 |        179 |
| xiaoboluo    | 128.00 KiB | 38.93 KiB |    8 |            0 |        5 |        982 |
| sys          | 16.00 KiB  | 354 bytes |    1 |            0 |        1 |          6 |
| 小萝卜        | 16.00 KiB  | 135 bytes |    1 |            0 |        1 |          3 |
+---------------+------------+-----------+-------+--------------+-----------+-------------+
5 rows in set (0.43 sec)

# 带x$前缀的视图
admin@localhost : sys 06:15:54> select * from x$innodb_buffer_stats_by_schema;
+---------------+-----------+----------+-------+--------------+-----------+-------------+
| object_schema | allocated | data    | pages | pages_hashed | pages_old | rows_cached |
+---------------+-----------+----------+-------+--------------+-----------+-------------+
| InnoDB System |  24887296 | 22809628 |  1519 |            0 |        24 |      21498 |
| mysql        |    245760 |    14917 |    15 |            0 |        15 |        179 |
| xiaoboluo    |    131072 |    39865 |    8 |            0 |        5 |        982 |
| sys          |    16384 |      354 |    1 |            0 |        1 |          6 |
| 小萝卜        |    16384 |      135 |    1 |            0 |        1 |          3 |
+---------------+-----------+----------+-------+--------------+-----------+-------------+
5 rows in set (0.42 sec)

视图字段含义如下：

object_schema：schema级别对象的名称，如果该表属于Innodb存储引擎，则该字段显示为InnoDB System，如果是其他引擎，则该字段显示为每个schema name.
allocated：当前已分配给schema的总内存字节数
data：当前已分配给schema的数据部分使用的内存字节总数
pages：当前已分配给schema内存总页数
pages_hashed：当前已分配给schema的自适应hash索引页总数
pages_old：当前已分配给schema的旧页总数（位于LRU列表中的旧块子列表中的页数）
rows_cached：buffer pool中为schema缓冲的总数据行数

02.innodb_buffer_stats_by_table,x$innodb_buffer_stats_by_table

按照schema和表分组的 InnoDB buffer pool 统计信息，与sys.innodb_buffer_stats_by_schema视图类似，但是本视图是按照schema name和table name分组。数据来源：information_schema.innodb_buffer_page

视图select语句文本如下：

# 不带x$前缀的视图select语句文本
SELECT IF(LOCATE('.', ibp.table_name) = 0, 'InnoDB System', REPLACE(SUBSTRING_INDEX(ibp.table_name, '.', 1), '`', '')) AS object_schema,
  REPLACE(SUBSTRING_INDEX(ibp.table_name, '.', -1), '`', '') AS object_name,
  sys.format_bytes(SUM(IF(ibp.compressed_size = 0, 16384, compressed_size))) AS allocated,
  sys.format_bytes(SUM(ibp.data_size)) AS data,
  COUNT(ibp.page_number) AS pages,
  COUNT(IF(ibp.is_hashed = 'YES', 1, NULL)) AS pages_hashed,
  COUNT(IF(ibp.is_old = 'YES', 1, NULL)) AS pages_old,
  ROUND(SUM(ibp.number_records)/COUNT(DISTINCT ibp.index_name)) AS rows_cached
FROM information_schema.innodb_buffer_page ibp
WHERE table_name IS NOT NULL
GROUP BY object_schema, object_name
ORDER BY SUM(IF(ibp.compressed_size = 0, 16384, compressed_size)) DESC;

# 带x$前缀的视图select语句文本
SELECT IF(LOCATE('.', ibp.table_name) = 0, 'InnoDB System', REPLACE(SUBSTRING_INDEX(ibp.table_name, '.', 1), '`', '')) AS object_schema,
  REPLACE(SUBSTRING_INDEX(ibp.table_name, '.', -1), '`', '') AS object_name,
  SUM(IF(ibp.compressed_size = 0, 16384, compressed_size)) AS allocated,
  SUM(ibp.data_size) AS data,
  COUNT(ibp.page_number) AS pages,
  COUNT(IF(ibp.is_hashed = 'YES', 1, NULL)) AS pages_hashed,
  COUNT(IF(ibp.is_old = 'YES', 1, NULL)) AS pages_old,
  ROUND(IFNULL(SUM(ibp.number_records)/NULLIF(COUNT(DISTINCT ibp.index_name), 0), 0)) AS rows_cached
FROM information_schema.innodb_buffer_page ibp
WHERE table_name IS NOT NULL
GROUP BY object_schema, object_name
ORDER BY SUM(IF(ibp.compressed_size = 0, 16384, compressed_size)) DESC;

下面我们看看使用该视图查询返回的结果。

# 不带x$前缀的视图
root@localhost : sys 12:41:25> select * from innodb_buffer_stats_by_table limit 3;
+---------------+-------------+-----------+-----------+-------+--------------+-----------+-------------+
| object_schema | object_name | allocated | data      | pages | pages_hashed | pages_old | rows_cached |
+---------------+-------------+-----------+-----------+-------+--------------+-----------+-------------+
| InnoDB System | SYS_TABLES  | 11.58 MiB | 10.63 MiB |  741 |            0 |        3 |      36692 |
| luoxiaobo    | t_luoxiaobo | 80.00 KiB | 29.21 KiB |    5 |            0 |        0 |        1658 |
| InnoDB System | SYS_COLUMNS | 48.00 KiB | 16.03 KiB |    3 |            0 |        3 |        239 |
+---------------+-------------+-----------+-----------+-------+--------------+-----------+-------------+
3 rows in set (0.12 sec)

# 带x$前缀的视图
root@localhost : sys 12:41:41> select * from x$innodb_buffer_stats_by_table limit 3;
+---------------+-------------+-----------+----------+-------+--------------+-----------+-------------+
| object_schema | object_name | allocated | data    | pages | pages_hashed | pages_old | rows_cached |
+---------------+-------------+-----------+----------+-------+--------------+-----------+-------------+
| InnoDB System | SYS_TABLES  |  12140544 | 11154757 |  741 |            0 |        3 |      36702 |
| luoxiaobo    | t_luoxiaobo |    81920 |    29913 |    5 |            0 |        0 |        1658 |
| InnoDB System | SYS_COLUMNS |    49152 |    16412 |    3 |            0 |        3 |        239 |
+---------------+-------------+-----------+----------+-------+--------------+-----------+-------------+
3 rows in set (0.12 sec)

视图字段含义如下：

object_name：表级别对象名称，通常是表名
其他字段含义与sys.innodb_buffer_stats_by_schema视图字段含义相同，详见 innodb_buffer_stats_by_schema,x$innodb_buffer_stats_by_schema视图解释部分。但这些字段是按照object_name表级别统计的

03.memory_by_host_by_current_bytes,x$memory_by_host_by_current_bytes

按照客户端主机名分组的内存使用统计信息，默认情况下按照当前内存使用量降序排序，数据来源：performance_schema.memory_summary_by_host_by_event_name

memory类型的事件默认情况下只启用了performance_schema自身的instruments，要监控用户访问，需要单独配置，如下：

* 开启所有的memory类型的instruments：update setup_instruments set enabled='yes' where name like '%memory/%';

下面我们看看使用该视图查询返回的结果。

# 不带x$前缀的视图
admin@localhost : sys 10:01:37> select * from memory_by_host_by_current_bytes limit 3;
+-------------+--------------------+-------------------+-------------------+-------------------+-----------------+
| host        | current_count_used | current_allocated | current_avg_alloc | current_max_alloc | total_allocated |
+-------------+--------------------+-------------------+-------------------+-------------------+-----------------+
| 10.10.20.14 |              58256 | 35.83 MiB        | 645 bytes        | 14.20 MiB        | 2.36 GiB        |
| localhost  |                32 | 903.11 KiB        | 28.22 KiB        | 819.00 KiB        | 7.69 MiB        |
| background  |                  5 | 176 bytes        | 35 bytes          | 160 bytes        | 352.57 KiB      |
+-------------+--------------------+-------------------+-------------------+-------------------+-----------------+
3 rows in set (0.01 sec)

# 带x$前缀的视图
admin@localhost : sys 10:02:19> select * from x$memory_by_host_by_current_bytes limit 3;
+-------------+--------------------+-------------------+-------------------+-------------------+-----------------+
| host        | current_count_used | current_allocated | current_avg_alloc | current_max_alloc | total_allocated |
+-------------+--------------------+-------------------+-------------------+-------------------+-----------------+
| 10.10.20.14 |              58256 |          37569266 |          644.8995 |          14885584 |      2538394110 |
| localhost  |                18 |            891658 |        49536.5556 |            838656 |        9821551 |
| background  |                  5 |              176 |          35.2000 |              160 |          361068 |
+-------------+--------------------+-------------------+-------------------+-------------------+-----------------+
3 rows in set (0.00 sec)

视图字段含义如下：

host：客户端连接的主机名或IP。在Performance Schema表中的HOST列为NULL的行在这里假定为后台线程，且在该视图host列显示为background
current_count_used：当前已分配的且未释放的内存块对应的内存分配次数（内存事件调用次数，该字段是快捷值，来自：performance_schema.memory_summary_by_host_by_event_name表的内存总分配次数字段COUNT_ALLOC - 内存释放次数COUNT_FREE）
current_allocated：当前已分配的且未释放的内存字节数
current_avg_alloc：当前已分配的且未释放的内存块对应的平均每次内存分配的内存字节数(current_allocated/current_count_used)
current_max_alloc：当前已分配的且未释放的单次最大内存分配字节数
total_allocated：总的已分配内存字节数

04.memory_by_thread_by_current_bytes,x$memory_by_thread_by_current_bytes

按照thread ID分组的内存使用统计信息（只统计前台线程），默认情况下按照当前内存使用量进行降序排序，数据来源：performance_schema.memory_summary_by_thread_by_event_name、performance_schema.threads

下面我们看看使用该视图查询返回的结果。

# 不带x$前缀的视图
admin@localhost : sys 10:04:07> select * from memory_by_thread_by_current_bytes limit 3;
+-----------+--------------------------+--------------------+-------------------+-------------------+-------------------+-----------------+
| thread_id | user                    | current_count_used | current_allocated | current_avg_alloc | current_max_alloc | total_allocated |
+-----------+--------------------------+--------------------+-------------------+-------------------+-------------------+-----------------+
|        45 | admin@localhost          |                34 | 4.91 MiB          | 147.98 KiB        | 4.00 MiB          | 29.36 MiB      |
|        41 | innodb/dict_stats_thread |                  5 | 176 bytes        | 35 bytes          | 160 bytes        | 346.31 KiB      |
|        47 | admin@localhost          |                  3 | 112 bytes        | 37 bytes          | 80 bytes          | 8.17 KiB        |
+-----------+--------------------------+--------------------+-------------------+-------------------+-------------------+-----------------+
3 rows in set (0.13 sec)

# 带x$前缀的视图
admin@localhost : sys 10:04:58> select * from x$memory_by_thread_by_current_bytes limit 3;
+-----------+--------------------------+--------------------+-------------------+-------------------+-------------------+-----------------+
| thread_id | user                    | current_count_used | current_allocated | current_avg_alloc | current_max_alloc | total_allocated |
+-----------+--------------------------+--------------------+-------------------+-------------------+-------------------+-----------------+
|        45 | admin@localhost          |                19 |          5102538 |      268554.6316 |          4194304 |        44995979 |
|        41 | innodb/dict_stats_thread |                  5 |              176 |          35.2000 |              160 |          354620 |
|        47 | admin@localhost          |                  3 |              112 |          37.3333 |                80 |            8368 |
+-----------+--------------------------+--------------------+-------------------+-------------------+-------------------+-----------------+
3 rows in set (0.12 sec)

视图字段含义如下：

thread_id：内部thread ID
user：对于前台线程，该字段显示为account名称，对于后台线程，该字段显示后台线程名称
其他字段含义与sys.memory_by_host_by_current_bytes视图的字段含义相同，详见 memory_by_host_by_current_bytes,x$memory_by_host_by_current_bytes视图解释部分。但是与该视图不同的是本视图是按照线程分组统计的

05.memory_by_user_by_current_bytes,x$memory_by_user_by_current_bytes

按照用户分组的内存使用统计信息，默认按照当前内存使用量进行降序排序，数据来源：performance_schema.memory_summary_by_user_by_event_name

下面我们看看使用该视图查询返回的结果。

# 不带x$前缀的视图
admin@localhost : sys 10:05:02> select * from memory_by_user_by_current_bytes limit 3;
+------------+--------------------+-------------------+-------------------+-------------------+-----------------+
| user      | current_count_used | current_allocated | current_avg_alloc | current_max_alloc | total_allocated |
+------------+--------------------+-------------------+-------------------+-------------------+-----------------+
| admin      |              58291 | 36.71 MiB        | 660 bytes        | 14.20 MiB        | 2.41 GiB        |
| background |                  5 | 176 bytes        | 35 bytes          | 160 bytes        | 358.17 KiB      |
| qfsys      |                  0 | 0 bytes          | 0 bytes          | 0 bytes          | 0 bytes        |
+------------+--------------------+-------------------+-------------------+-------------------+-----------------+
3 rows in set (0.01 sec)

# 带x$前缀的视图
admin@localhost : sys 10:05:21> select * from x$memory_by_user_by_current_bytes limit 3;
+------------+--------------------+-------------------+-------------------+-------------------+-----------------+
| user      | current_count_used | current_allocated | current_avg_alloc | current_max_alloc | total_allocated |
+------------+--------------------+-------------------+-------------------+-------------------+-----------------+
| admin      |              58278 |          38460932 |          659.9563 |          14885584 |      2586890836 |
| background |                  5 |              176 |          35.2000 |              160 |          366828 |
| qfsys      |                  0 |                0 |            0.0000 |                0 |              0 |
+------------+--------------------+-------------------+-------------------+-------------------+-----------------+
3 rows in set (0.01 sec)

视图字段含义如下：

user：客户端用户名。对于后台线程，该字段显示为background，对于前台线程，该字段显示user名称(不是account，不包含host部分)
其他字段含义与sys.memory_by_host_by_current_bytes视图的字段含义相同，详见 memory_by_host_by_current_bytes,x$memory_by_host_by_current_bytes视图解释部分。但是与该视图不同的是这里是按照用户名分组统计的

06.memory_global_by_current_bytes,x$memory_global_by_current_bytes

按照内存分配类型（事件类型）分组的内存使用统计信息，默认情况下按照当前内存使用量进行降序排序，数据来源：performance_schema.memory_summary_global_by_event_name

下面我们看看使用该视图查询返回的结果。

# 不带x$前缀的视图
admin@localhost : sys 10:05:24> select * from memory_global_by_current_bytes limit 3;
+-----------------------------------------------------------------+---------------+---------------+-------------------+------------+------------+----------------+
| event_name                                                      | current_count | current_alloc | current_avg_alloc | high_count | high_alloc | high_avg_alloc |
+-----------------------------------------------------------------+---------------+---------------+-------------------+------------+------------+----------------+
| memory/innodb/lock0lock                                        |          9166 | 14.20 MiB    | 1.59 KiB          |      9166 | 14.20 MiB  | 1.59 KiB      |
| memory/performance_schema/events_statements_history_long        |            1 | 13.66 MiB    | 13.66 MiB        |          1 | 13.66 MiB  | 13.66 MiB      |
| memory/performance_schema/events_statements_history_long.tokens |            1 | 9.77 MiB      | 9.77 MiB          |          1 | 9.77 MiB  | 9.77 MiB      |
+-----------------------------------------------------------------+---------------+---------------+-------------------+------------+------------+----------------+
3 rows in set (0.01 sec)

# 带x$前缀的视图
admin@localhost : sys 10:07:19> select * from x$memory_global_by_current_bytes limit 3;
+-----------------------------------------------------------------+---------------+---------------+-------------------+------------+------------+----------------+
| event_name                                                      | current_count | current_alloc | current_avg_alloc | high_count | high_alloc | high_avg_alloc |
+-----------------------------------------------------------------+---------------+---------------+-------------------+------------+------------+----------------+
| memory/innodb/lock0lock                                        |          9166 |      14885584 |        1624.0000 |      9166 |  14885584 |      1624.0000 |
| memory/performance_schema/events_statements_history_long        |            1 |      14320000 |    14320000.0000 |          1 |  14320000 |  14320000.0000 |
| memory/performance_schema/events_statements_history_long.tokens |            1 |      10240000 |    10240000.0000 |          1 |  10240000 |  10240000.0000 |
+-----------------------------------------------------------------+---------------+---------------+-------------------+------------+------------+----------------+
3 rows in set (0.00 sec)

视图字段含义如下：

EVENT_NAME：内存事件名称
CURRENT_COUNT：当前已分配内存且未释放的内存事件发生的总次数(内存分配次数)
current_alloc：当前已分配内存且未释放的内存字节数
current_avg_alloc：当前已分配内存且未释放的内存事件的平均内存字节数(平均每次内存分配的字节数)
high_count：内存事件发生的历史最高位(高水位)次数（来自performance_schema.memory_summary_global_by_event_name表中的HIGH_COUNT_USED字段：如果CURRENT_COUNT_USED增加1是一个新的最高值，则该字段值相应增加）
high_alloc：内存分配的历史最高位(高水位)字节数（来自performance_schema.memory_summary_global_by_event_name表中的HIGH_NUMBER_OF_BYTES_USED字段：如果CURRENT_NUMBER_OF_BYTES_USED增加N之后是一个新的最高值，则该字段值相应增加）
high_avg_alloc：内存事件发生的历史最高位(高水位)次数对应的平均每次内存分配的字节数(high_number_of_bytes_used/high_count_used)

07.memory_global_total,x$memory_global_total

当前总内存使用量统计（注意：只包含自memory类型的instruments启用以来被监控到的内存事件，在启用之前的无法监控，so..如果你不是在server启动之前就在配置文件中配置启动memory类型的instruments，那么此值可能并不可靠，当然如果你的server运行时间足够长，那么该值也具有一定参考价值），数据来源：performance_schema.memory_summary_global_by_event_name

下面我们看看使用该视图查询返回的结果。

# 不带x$前缀的视图
admin@localhost : sys 10:07:22> select * from memory_global_total limit 3;
+-----------------+
| total_allocated |
+-----------------+
| 168.91 MiB      |
+-----------------+
1 row in set (0.01 sec)

# 带x$前缀的视图
admin@localhost : sys 10:08:24> select * from x$memory_global_total limit 3;
+-----------------+
| total_allocated |
+-----------------+
|      177099388 |
+-----------------+
1 row in set (0.00 sec)