Flink Forward 2019--实战相关(14)--Arctic Wolf Networks分享事件流攻击

Hunting for Attack Chains in Event Streams -- Ray Ruvinskiy(Arctic Wolf Networks) & Jonathan Walsh(Arctic Wolf Networks)

Arctic Wolf Networks processes over 9 billion events a day across its customer base. These represent HTTP and DNS transactions from customer networks, log lines from customer infrastructure devices like firewalls and switches, Active Directory event logs, logs from cloud services such as Office 365, and more. It is critical for our security engineers to quickly pick out the small subset of these events that represent security threats facing our customers.

Arctic Wolf Networks 每天在其客户群中处理超过90亿个事件。这些代表来自客户网络的HTTP和DNS事务、来自客户基础设施设备（如防火墙和交换机）的日志行、Active Directory事件日志、来自云服务（如Office 365）的日志等等。对于我们的安全工程师来说，很重要的一点是要快速地找出这些事件中的一小部分，它们代表着我们客户面临的安全威胁。

Further, effective threat detection requires the ability to detect a sequence of related events. One example is detecting a certain threshold of login failures within a time period followed by a successful login, which might indicate a successful attempt to brute force a user account. Another is the download of an executable payload followed by an HTTP POST request to a suspicious site. In both these cases, the sequence of events taken together presents a stronger indicator of compromise than each of the individual events. Arctic Wolf Networks implemented this functionality by integrating Flink with EsperTech’s Esper Complex Event Processing streaming analytics engine.

此外，有效的威胁检测需要能够检测一系列相关事件。一个例子是在登录成功后的一段时间内检测到登录失败的某个阈值，这可能表示成功地尝试强制用户帐户。另一个是下载可执行负载，然后向可疑站点发送HTTP POST请求。在这两种情况下，事件的顺序综合起来比每个单独的事件都显示出更强的折衷指标。北极狼网络通过将Flink与Espertech的Esper复杂事件处理流分析引擎集成来实现此功能。

We greatly benefited from Flink’s ease of deployment and horizontal scalability, while its configurable thresholds for event lateness were crucial in enabling us to handle heterogeneous customer data sources that are not all in sync. Meanwhile, Esper offered a mature, highly expressive, and performant Complex Event Processing framework, a good fit for the flexibility required to express the logic desired by our security engineers. Together, Flink and Esper enhance our security engineers’ visibility into threats faced by our customers and reduce the time investment needed to identify these threats, allowing for more comprehensive and responsive customer service.

我们从Flink的易部署性和水平扩展性中受益匪浅，而其事件延迟的可配置阈值对于我们处理不完全同步的异构客户数据源至关重要。同时，ESPER提供了一个成熟、高表达性和高性能的复杂事件处理框架，非常适合于表达安全工程师所需逻辑所需的灵活性。Flink和Esper共同提高了我们的安全工程师对客户面临的威胁的可视性，并减少了识别这些威胁所需的时间投入，从而提供了更全面、更快速的客户服务。