Python3日志记录模块
[TOC]
0x00 logging 模块
1.基础简述
描述: Logging 库是非常常用的记录日志库，通过logging模块存储各种格式的日志，主要用于输出运行日志，可以设置输出日志的等级、日志保存路径、日志文件回滚、日志按时分秒进行切割等.
模块导入: import logging
模块参考: https://docs.python.org/3/library/logging.html
模块优点
- 你可以控制消息的级别，过滤掉那些并不重要的消息。
- 你可决定输出到什么地方，以及怎么输出。有许多的重要性别级可供选择，debug、info、warning、error 以及 critical。通过赋予 logger 或者 handler 不同的级别，你就可以只输出错误消息到特定的记录文件中，或者在调试时只记录调试信息。
日志级别
描述: 以下描述了标准水平及其适用性（按严重程度的增加顺序）, 严重程度的级别依次是DEBUG < INFO < WARNING < ERROR < CRITICAL
温馨提示: 当进行日志输出时，如定义了最低日志记录等级，则只会记录严重程度在其之上的日志。
格式化日志信息
描述: Logging 模块有一系列可以用做格式化的属性，如format='%(asctime)s %(levelname)s:%(message)s'，如下：
Attribute name | Format | Description |
|---|---|---|
args | You shouldn’t need to format this yourself. | The tuple of arguments merged into msg to produce message, or a dict whose values are used for the merge (when there is only one argument, and it is a dictionary). |
asctime | %(asctime)s | Human-readable time when the LogRecord was created. By default this is of the form ‘2003-07-08 16:49:45,896’ (the numbers after the comma are millisecond portion of the time). |
created | %(created)f | Time when the LogRecord was created (as returned by time.time()). |
exc_info | You shouldn’t need to format this yourself. | Exception tuple (à la sys.exc_info) or, if no exception has occurred, None. |
filename | %(filename)s | Filename portion of pathname. |
funcName | %(funcName)s | Name of function containing the logging call. |
levelname | %(levelname)s | Text logging level for the message (‘DEBUG’, ‘INFO’, ‘WARNING’, ‘ERROR’, ‘CRITICAL’). |
levelno | %(levelno)s | Numeric logging level for the message (DEBUG, INFO, WARNING, ERROR, CRITICAL). |
lineno | %(lineno)d | Source line number where the logging call was issued (if available). |
message | %(message)s | The logged message, computed as msg % args. This is set when Formatter.format() is invoked. |
module | %(module)s | Module (name portion of filename). |
msecs | %(msecs)d | Millisecond portion of the time when the LogRecord was created. |
msg | You shouldn’t need to format this yourself. | The format string passed in the original logging call. Merged with args to produce message, or an arbitrary object (see Using arbitrary objects as messages). |
name | %(name)s | Name of the logger used to log the call. |
pathname | %(pathname)s | Full pathname of the source file where the logging call was issued (if available). |
process | %(process)d | Process ID (if available). |
processName | %(processName)s | Process name (if available). |
relativeCreated | %(relativeCreated)d | Time in milliseconds when the LogRecord was created, relative to the time the logging module was loaded. |
stack_info | You shouldn’t need to format this yourself. | Stack frame information (where available) from the bottom of the stack in the current thread, up to and including the stack frame of the logging call which resulted in the creation of this record. |
thread | %(thread)d | Thread ID (if available). |
threadName | %(threadName)s | Thread name (if available). |
2.语法参数
描述: logging库提供了模块化的方法和几个组件, 下列列出了模块定义的基础类和函数
- Loggers ：记录器公开应用程序代码直接使用的接口。
- Handlers ：处理程序将日志记录（由记录器创建）发送到相应的目标。
- Filters ：过滤器提供了更细粒度的工具，用于确定要输出哪些日志记录。
- Formatters： 格式化程序指定最终输出中日志记录的布局。
简单示例:
import logging
# 1、创建一个logger
logger = logging.getLogger(__name__) # 创建 logger
logger.setLevel(logging.DEBUG) # 写入内容的严重级别
# Handler方法有很多下面主要介绍两种：StreamHandler 和 FileHandler
# 2、创建一个handler，用于写入日志文件
fh=logging.FileHandler('test.log', encoding="utf-8") # 将日志写入到test.log文件
fh.setLevel(logging.DEBUG) # 并且需要指定写入的内容严重级别
# 再创建一个handler，用于输出到控制台
ch=logging.StreamHandler() # 将日志写入控制台
ch.setLevel(logging.DEBUG) # 并且需要指定写入的内容严重级别
# 3、定义handler的输出格式（formatter）
formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
# 4、给handler添加formatter
fh.setFormatter(formatter)
ch.setFormatter(formatter)
# 5、给logger添加handler
logger.addHandler(fh)
logger.addHandler(ch)
# 6、进行日志输出
logger.info("这是一条INFO级别的信息,来自 【WeiyiGeek.top】 触发。")
logger.debug('debug message')
logger.info('info message')
logger.warning('warn message')
logger.error('error message')
logger.critical('critical message')
WeiyiGeek.同时输出到文件或者终端中
那么为什么后续在定义Handler时又做了一次set Level操作呢？
原因是：Logger中设置的级别决定它将传递给Handler的消息严重性。每个Handler设置的setLevel()决定了该处理程序将发送哪些消息（记住：日志中消息是分严重程度的，当确定严重级别是某个层级时，该层级以下的消息不被发送或者记录，该层级以上的消息才被发送或者记录）。
温馨提示: 写入文件的三种filemode，即r (只读)/w (覆盖)/a (默认)
温馨提示: Logger 整个操作流（官网上down下来的）.
WeiyiGeek.Logger 操作流
3.动手实践
(1) 简单入门例子
import logging
import os
# 日志写入目录
os.chdir("/tmp/")
logging.basicConfig(format='%(asctime)s - %(name)s - %(levelname)s - %(message)ss', filename='example.log', level=logging.DEBUG)
# 注意：上面level设置的是显示的最低严重级别，小于level设置的最低严重级别将不会打印出来
logging.debug('Debug, This message should go to the log file')
logging.info('Info, So should this')
logging.warning('Warn, And this, too')
logging.error('Error, And non-ASCII stuff, too')(2) 实践之按日志大小分割
描述: python 提供了两个处理器，方便我们分割文件：
logging.handlers.RotatingFileHandler-> 按照大小自动分割日志文件，一旦达到指定的大小重新生成文件logging.handlers.TimedRotatingFileHandler-> 按照时间自动分割日志文件
# 设置日志的记录等级
logging.basicConfig(level=logging.DEBUG) # 调试debug级
# 创建日志记录器，指明日志保存的路径、每个日志文件的最大大小、保存的日志文件个数上限
file_log_handler = RotatingFileHandler("logs/weiyigeek.top.log", maxBytes=1024 * 1024 * 100, backupCount=10, encoding="utf-8")
# 创建日志记录的格式 日志等级 输入日志信息的文件名 行数 日志信息
formatter = logging.Formatter('%(levelname)s %(filename)s:%(lineno)d %(message)s')
# 为刚创建的日志记录器设置日志记录格式
file_log_handler.setFormatter(formatter)
# 为全局的日志工具对象（flask app使用的）添加日志记录器
logging.getLogger().addHandler(file_log_handler)执行效果：
-a---- 2022/8/10 13:46 79 weiyigeek.top.log
-a---- 2022/8/10 13:45 878 weiyigeek.top.1(3) 实践之日志按天分割
描述: 本示例代码可以将日志按天切割以及分别输出到_info.log与_error.log里面。
# -*- coding: utf-8 -*-
import os
import logging
from logging.handlers import TimedRotatingFileHandler
LOG_PATH = "log"
LOG_INFO = '_info.log'
LOG_ERROR = '_error.log'
class logger:
def __init__(self, prefix_name = "flask"):
if (not os.path.exists(LOG_PATH)):
os.makedirs(LOG_PATH)
self.prefix = prefix_name
# 创建logger日志对象
self.info_logger = logging.getLogger("info")
self.error_logger = logging.getLogger("error")
self.stream_logger = logging.getLogger("stream")
# 日志的最低输出级别
self.info_logger.setLevel(logging.DEBUG)
self.error_logger.setLevel(logging.ERROR)
# 日志格式化
self.format = logging.Formatter('[%(asctime)s][%(threadName)s:%(thread)d][task_id:%(name)s][%(filename)s:%(lineno)d]' '[%(levelname)s] : %(message)s')
# 按照时间切割文件 Handler 配置
TimeFileHandlerINFO = TimedRotatingFileHandler("%s/%s%s" % (LOG_PATH, prefix_name, LOG_INFO), when='MIDNIGHT', encoding="utf-8", backupCount=8760, delay=True)
TimeFileHandlerINFO.suffix = "%Y-%m-%d.log"
TimeFileHandlerERROR = TimedRotatingFileHandler("%s/%s%s" % (LOG_PATH, prefix_name, LOG_ERROR), when='MIDNIGHT', encoding="utf-8", backupCount=8760, delay=True)
TimeFileHandlerERROR.suffix = "%Y-%m-%d.log"
LoggerStream = logging.StreamHandler()
# 设置日志格式化
TimeFileHandlerINFO.setFormatter(self.format)
TimeFileHandlerERROR.setFormatter(self.format)
LoggerStream.setFormatter(self.format)
# 添加设置的句柄
self.info_logger.addHandler(TimeFileHandlerINFO)
self.info_logger.addHandler(LoggerStream)
self.error_logger.addHandler(TimeFileHandlerERROR)
self.error_logger.addHandler(LoggerStream)
def debug(self, msg, *args, **kwargs):
self.info_logger.debug(msg, *args, **kwargs)
def info(self, msg, *args, **kwargs):
self.info_logger.info(msg, *args, **kwargs)
def warn(self, msg, *args, **kwargs):
self.info_logger.warning(msg, *args, **kwargs)
def error(self, msg, *args, **kwargs):
self.error_logger.error(msg, *args, **kwargs)
def fatal(self, msg, *args, **kwargs):
self.error_logger.fatal(msg, *args, **kwargs)
def critical(self, msg, *args, **kwargs):
self.error_logger.critical(msg, *args, **kwargs)
log = logger("weiyigeek")
log.debug("debug-test ")
log.info("info-test")
log.warn("wrning-test")
log.error("error - test")
log.fatal("fatal - test")
log.critical("critical - test")执行结果:
python .\logger.py
[2022-08-10 10:21:21,068][MainThread:22372][task_id:info][logger.py:49][DEBUG] : debug-test
[2022-08-10 10:21:21,069][MainThread:22372][task_id:info][logger.py:52][INFO] : info-test
[2022-08-10 10:21:21,069][MainThread:22372][task_id:info][logger.py:55][WARNING] : wrning-test
[2022-08-10 10:21:21,070][MainThread:22372][task_id:error][logger.py:58][ERROR] : error - test
[2022-08-10 10:21:21,070][MainThread:22372][task_id:error][logger.py:61][CRITICAL] : fatal - test
[2022-08-10 10:21:21,070][MainThread:22372][task_id:error][logger.py:64][CRITICAL] : critical - test
# 生成的文件
/mnt/d/Study/Project/log$ ls
weiyigeek_error.log
weiyigeek_error.log.2022-08-09.log
weiyigeek_info.log
weiyigeek_info.log.2022-08-09.log补充说明: 上述代码中 when是一个字符串用于描述滚动周期的基本单位，字符串的值及意义如下：
'S': Seconds
'M': Minutes
'H': Hours
'D': Days
'W': Week day (0=Monday)
'midnight': Roll over at midnight(4) 补充示例
# -*- coding: utf-8 -*-
import logging
from flask.logging import default_handler
import os
from logging.handlers import RotatingFileHandler
from logging import StreamHandler
BASE_DIR = os.path.dirname(os.path.abspath(__file__))
LOG_PATH = os.path.join(BASE_DIR, 'logs')
LOG_PATH_ERROR = os.path.join(LOG_PATH, 'error.log')
LOG_PATH_INFO = os.path.join(LOG_PATH, 'info.log')
LOG_PATH_ALL = os.path.join(LOG_PATH, 'all.log')
# 日志文件最大 100MB
LOG_FILE_MAX_BYTES = 100 * 1024 * 1024
# 轮转数量是 10 个
LOG_FILE_BACKUP_COUNT = 10
class Logger(object):
def init_app(self, app):
# 移除默认的handler
app.logger.removeHandler(default_handler)
formatter = logging.Formatter(
'%(asctime)s [%(thread)d:%(threadName)s] [%(filename)s:%(module)s:%(funcName)s] '
'[%(levelname)s]: %(message)s'
)
# 将日志输出到文件
# 1 MB = 1024 * 1024 bytes
# 此处设置日志文件大小为500MB，超过500MB自动开始写入新的日志文件，历史文件归档
file_handler = RotatingFileHandler(
filename=LOG_PATH_ALL,
mode='a',
maxBytes=LOG_FILE_MAX_BYTES,
backupCount=LOG_FILE_BACKUP_COUNT,
encoding='utf-8'
)
file_handler.setFormatter(formatter)
file_handler.setLevel(logging.INFO)
stream_handler = StreamHandler()
stream_handler.setFormatter(formatter)
stream_handler.setLevel(logging.INFO)
for logger in (
# 这里自己还可以添加更多的日志模块，具体请参阅Flask官方文档
app.logger,
logging.getLogger('sqlalchemy'),
logging.getLogger('werkzeug')
):
logger.addHandler(file_handler)
logger.addHandler(stream_handler)扩展文件中添加log模块：
# encoding: utf-8
from log import Logger
logger = Logger()