专栏首页码匠的流水账聊聊dubbo的AccessLogFilter
原创

聊聊dubbo的AccessLogFilter

本文主要研究一下dubbo的AccessLogFilter

AccessLogFilter

dubbo-2.7.3/dubbo-rpc/dubbo-rpc-api/src/main/java/org/apache/dubbo/rpc/filter/AccessLogFilter.java

@Activate(group = PROVIDER, value = ACCESS_LOG_KEY)
public class AccessLogFilter implements Filter {
​
    private static final Logger logger = LoggerFactory.getLogger(AccessLogFilter.class);
​
    private static final String ACCESS_LOG_KEY = "dubbo.accesslog";
​
    private static final int LOG_MAX_BUFFER = 5000;
​
    private static final long LOG_OUTPUT_INTERVAL = 5000;
​
    private static final String FILE_DATE_FORMAT = "yyyyMMdd";
​
    // It's safe to declare it as singleton since it runs on single thread only
    private static final DateFormat FILE_NAME_FORMATTER = new SimpleDateFormat(FILE_DATE_FORMAT);
​
    private static final Map<String, Set<AccessLogData>> LOG_ENTRIES = new ConcurrentHashMap<String, Set<AccessLogData>>();
​
    private static final ScheduledExecutorService LOG_SCHEDULED = Executors.newSingleThreadScheduledExecutor(new NamedThreadFactory("Dubbo-Access-Log", true));
​
    /**
     * Default constructor initialize demon thread for writing into access log file with names with access log key
     * defined in url <b>accesslog</b>
     */
    public AccessLogFilter() {
        LOG_SCHEDULED.scheduleWithFixedDelay(this::writeLogToFile, LOG_OUTPUT_INTERVAL, LOG_OUTPUT_INTERVAL, TimeUnit.MILLISECONDS);
    }
​
    /**
     * This method logs the access log for service method invocation call.
     *
     * @param invoker service
     * @param inv     Invocation service method.
     * @return Result from service method.
     * @throws RpcException
     */
    @Override
    public Result invoke(Invoker<?> invoker, Invocation inv) throws RpcException {
        try {
            String accessLogKey = invoker.getUrl().getParameter(ACCESS_LOG_KEY);
            if (ConfigUtils.isNotEmpty(accessLogKey)) {
                AccessLogData logData = buildAccessLogData(invoker, inv);
                log(accessLogKey, logData);
            }
        } catch (Throwable t) {
            logger.warn("Exception in AccessLogFilter of service(" + invoker + " -> " + inv + ")", t);
        }
        return invoker.invoke(inv);
    }
​
    private void log(String accessLog, AccessLogData accessLogData) {
        Set<AccessLogData> logSet = LOG_ENTRIES.computeIfAbsent(accessLog, k -> new ConcurrentHashSet<>());
​
        if (logSet.size() < LOG_MAX_BUFFER) {
            logSet.add(accessLogData);
        } else {
            //TODO we needs use force writing to file so that buffer gets clear and new log can be written.
            logger.warn("AccessLog buffer is full skipping buffer ");
        }
    }
​
    private void writeLogToFile() {
        if (!LOG_ENTRIES.isEmpty()) {
            for (Map.Entry<String, Set<AccessLogData>> entry : LOG_ENTRIES.entrySet()) {
                try {
                    String accessLog = entry.getKey();
                    Set<AccessLogData> logSet = entry.getValue();
                    if (ConfigUtils.isDefault(accessLog)) {
                        processWithServiceLogger(logSet);
                    } else {
                        File file = new File(accessLog);
                        createIfLogDirAbsent(file);
                        if (logger.isDebugEnabled()) {
                            logger.debug("Append log to " + accessLog);
                        }
                        renameFile(file);
                        processWithAccessKeyLogger(logSet, file);
                    }
​
                } catch (Exception e) {
                    logger.error(e.getMessage(), e);
                }
            }
        }
    }
​
    private void processWithAccessKeyLogger(Set<AccessLogData> logSet, File file) throws IOException {
        try (FileWriter writer = new FileWriter(file, true)) {
            for (Iterator<AccessLogData> iterator = logSet.iterator();
                 iterator.hasNext();
                 iterator.remove()) {
                writer.write(iterator.next().getLogMessage());
                writer.write("\r\n");
            }
            writer.flush();
        }
    }
​
    private AccessLogData buildAccessLogData(Invoker<?> invoker, Invocation inv) {
        RpcContext context = RpcContext.getContext();
        AccessLogData logData = AccessLogData.newLogData();
        logData.setServiceName(invoker.getInterface().getName());
        logData.setMethodName(inv.getMethodName());
        logData.setVersion(invoker.getUrl().getParameter(VERSION_KEY));
        logData.setGroup(invoker.getUrl().getParameter(GROUP_KEY));
        logData.setInvocationTime(new Date());
        logData.setTypes(inv.getParameterTypes());
        logData.setArguments(inv.getArguments());
        return logData;
    }
​
    private void processWithServiceLogger(Set<AccessLogData> logSet) {
        for (Iterator<AccessLogData> iterator = logSet.iterator();
             iterator.hasNext();
             iterator.remove()) {
            AccessLogData logData = iterator.next();
            LoggerFactory.getLogger(ACCESS_LOG_KEY + "." + logData.getServiceName()).info(logData.getLogMessage());
        }
    }
​
    private void createIfLogDirAbsent(File file) {
        File dir = file.getParentFile();
        if (null != dir && !dir.exists()) {
            dir.mkdirs();
        }
    }
​
    private void renameFile(File file) {
        if (file.exists()) {
            String now = FILE_NAME_FORMATTER.format(new Date());
            String last = FILE_NAME_FORMATTER.format(new Date(file.lastModified()));
            if (!now.equals(last)) {
                File archive = new File(file.getAbsolutePath() + "." + last);
                file.renameTo(archive);
            }
        }
    }
}
  • AccessLogFilter实现了org.apache.dubbo.rpc.Filter接口,其构造器注册了一个定时任务,每隔LOG_OUTPUT_INTERVAL执行一次writeLogToFile
  • invoke方法从invoker.getUrl()获取accessLogKey,如果不为空则使用buildAccessLogData构建AccessLogData,然后放入到LOG_ENTRIES中,如果超出LOG_MAX_BUFFER则对其并打印warn日志
  • writeLogToFile方法遍历LOG_ENTRIES,将AccessLogData写入文件,如果是default的则使用processWithServiceLogger,否则使用processWithAccessKeyLogger方法

AccessLogFilterTest

dubbo-2.7.3/dubbo-rpc/dubbo-rpc-api/src/test/java/org/apache/dubbo/rpc/filter/AccessLogFilterTest.java

public class AccessLogFilterTest {
​
    Filter accessLogFilter = new AccessLogFilter();
​
    // Test filter won't throw an exception
    @Test
    public void testInvokeException() {
        Invoker<AccessLogFilterTest> invoker = new MyInvoker<AccessLogFilterTest>(null);
        Invocation invocation = new MockInvocation();
        LogUtil.start();
        accessLogFilter.invoke(invoker, invocation);
        assertEquals(1, LogUtil.findMessage("Exception in AccessLogFilter of service"));
        LogUtil.stop();
    }
​
    // TODO how to assert thread action
    @Test
    public void testDefault() {
        URL url = URL.valueOf("test://test:11/test?accesslog=true&group=dubbo&version=1.1");
        Invoker<AccessLogFilterTest> invoker = new MyInvoker<AccessLogFilterTest>(url);
        Invocation invocation = new MockInvocation();
        accessLogFilter.invoke(invoker, invocation);
    }
​
    @Test
    public void testCustom() {
        URL url = URL.valueOf("test://test:11/test?accesslog=custom-access.log");
        Invoker<AccessLogFilterTest> invoker = new MyInvoker<AccessLogFilterTest>(url);
        Invocation invocation = new MockInvocation();
        accessLogFilter.invoke(invoker, invocation);
    }
​
}
  • 这里验证了invokeException、default、custom场景

AccessLogData

dubbo-2.7.3/dubbo-rpc/dubbo-rpc-api/src/main/java/org/apache/dubbo/rpc/support/AccessLogData.java

public final class AccessLogData {
​
    private static final String MESSAGE_DATE_FORMAT = "yyyy-MM-dd HH:mm:ss";
    private static final DateFormat MESSAGE_DATE_FORMATTER = new SimpleDateFormat(MESSAGE_DATE_FORMAT);
​
    private static final String VERSION = "version";
    private static final String GROUP = "group";
    private static final String SERVICE = "service";
    private static final String METHOD_NAME = "method-name";
    private static final String INVOCATION_TIME = "invocation-time";
    private static final String TYPES = "types";
    private static final String ARGUMENTS = "arguments";
    private static final String REMOTE_HOST = "remote-host";
    private static final String REMOTE_PORT = "remote-port";
    private static final String LOCAL_HOST = "localhost";
    private static final String LOCAL_PORT = "local-port";
​
    /**
     * This is used to store log data in key val format.
     */
    private Map<String, Object> data;
​
    /**
     * Default constructor.
     */
    private AccessLogData() {
        RpcContext context = RpcContext.getContext();
        data = new HashMap<>();
        setLocalHost(context.getLocalHost());
        setLocalPort(context.getLocalPort());
        setRemoteHost(context.getRemoteHost());
        setRemotePort(context.getRemotePort());
    }
​
    //......
​
    public String getLogMessage() {
        StringBuilder sn = new StringBuilder();
​
        sn.append("[")
                .append(MESSAGE_DATE_FORMATTER.format(getInvocationTime()))
                .append("] ")
                .append(get(REMOTE_HOST))
                .append(":")
                .append(get(REMOTE_PORT))
                .append(" -> ")
                .append(get(LOCAL_HOST))
                .append(":")
                .append(get(LOCAL_PORT))
                .append(" - ");
​
        String group = get(GROUP) != null ? get(GROUP).toString() : "";
        if (StringUtils.isNotEmpty(group.toString())) {
            sn.append(group).append("/");
        }
​
        sn.append(get(SERVICE));
​
        String version = get(VERSION) != null ? get(VERSION).toString() : "";
        if (StringUtils.isNotEmpty(version.toString())) {
            sn.append(":").append(version);
        }
​
        sn.append(" ");
        sn.append(get(METHOD_NAME));
​
        sn.append("(");
        Class<?>[] types = get(TYPES) != null ? (Class<?>[]) get(TYPES) : new Class[0];
        boolean first = true;
        for (Class<?> type : types) {
            if (first) {
                first = false;
            } else {
                sn.append(",");
            }
            sn.append(type.getName());
        }
        sn.append(") ");
​
​
        Object[] args = get(ARGUMENTS) != null ? (Object[]) get(ARGUMENTS) : null;
        if (args != null && args.length > 0) {
            sn.append(JSON.toJSONString(args));
        }
​
        return sn.toString();
    }
​
    //......
}
  • AccessLogData定义了version、group、service、method-name、invocation-time、types、arguments、remote-host、remote-port、localhost、local-port常量;getLogMessage则构建log的输出

小结

  • AccessLogFilter实现了org.apache.dubbo.rpc.Filter接口,其构造器注册了一个定时任务,每隔LOG_OUTPUT_INTERVAL执行一次writeLogToFile
  • AccessLogFilter的invoke方法从invoker.getUrl()获取accessLogKey,如果不为空则使用buildAccessLogData构建AccessLogData,然后放入到LOG_ENTRIES中,如果超出LOG_MAX_BUFFER则对其并打印warn日志
  • AccessLogFilter的writeLogToFile方法遍历LOG_ENTRIES,将AccessLogData写入文件,如果是default的则使用processWithServiceLogger,否则使用processWithAccessKeyLogger方法

doc

原创声明,本文系作者授权云+社区发表,未经许可,不得转载。

如有侵权,请联系 yunjia_community@tencent.com 删除。

我来说两句

0 条评论
登录 后参与评论

相关文章

  • 聊聊dubbo的AccessLogFilter

    dubbo-2.7.3/dubbo-rpc/dubbo-rpc-api/src/main/java/org/apache/dubbo/rpc/filter/Ac...

    codecraft
  • 聊聊nacos ServiceManager的UpdatedServiceProcessor

    本文主要研究一下nacos ServiceManager的UpdatedServiceProcessor

    codecraft
  • 聊聊nacos ServiceManager的UpdatedServiceProcessor

    本文主要研究一下nacos ServiceManager的UpdatedServiceProcessor

    codecraft
  • 聊聊dubbo的AccessLogFilter

    dubbo-2.7.3/dubbo-rpc/dubbo-rpc-api/src/main/java/org/apache/dubbo/rpc/filter/Ac...

    codecraft
  • linux下C语言实现写日志功能

    使用tail 命令可以实现日志的查询,以及其他功能,不了解的话,自行查资料解决。

    砸漏
  • LWN: 在 Linux 上运行 macOS 程序

    目前有个名叫Darling的项目活跃度不断提升,这个项目是希望能在Linux上提供一个针对macOS软件的translation layer(翻译层),有点类似...

    用户6543014
  • 嵌入式linux之go语言开发(四)go语言8583协议报文解析

    原来的pos用c语言开发的,与银联后台通信走的是8583协议。那么用go来做,得实现个go语言8583协议报文解析

    特立独行的猫a
  • MySQL 的 help 命令你真的会用吗|全方位认识 mysql 系统库

    MySQL 的帮助信息重要吗?不太重要!有用吗?有!就好比你在家洗澡的时候,突然有人不停地按你家的门铃,能把你憋出心脏病来,嘿嘿。

    老叶茶馆
  • Linux 命令(110)—— help 命令(builtin)

    help 命令只能显示 Bash 内部命令的帮助信息,对于外部命令的帮助信息需要使用 man 或 info 命令查看。

    Dabelv
  • Python | “Python太火,我都不敢不把这些告诉你”

    之前说过,小编现在使用的环境是ubuntu server 16.04 LTS。默认安装的应用面没有python2.x,由于越来越多的平台弃用python2,所以...

    LogicPanda

扫码关注云+社区

领取腾讯云代金券