Flink Forward 2019--实战相关(8)--Intuit公司分享识别图片

Building Financial Identity Platform using Apache Flink -- Vivek Thakre(Intuit.com)

To power financial prosperity around the world, Intuit needs to create personalized product experience and new data centric products. Some of these use cases include Enabling 360 Customer View for Personalization and Targeting, building Ecosystem for Data Exchange between internal and 3rd party and personalize financial offerings, creating platform for Personalized security experience based on risk factors of people and devices.

为了推动全球金融繁荣，Intuit需要创建个性化的产品体验和新的以数据为中心的产品。其中一些用例包括启用360度客户视图进行个性化和定位，构建内部和第三方之间数据交换的生态系统，个性化金融产品，根据人员和设备的风险因素创建个性化安全体验平台。

Unlike workflow centric products (for example, tax processing, accounting transactions), these use cases are often information-intensive and require real-time access to a large amount of connected data associated with people, organizations and things they own.

与以工作流为中心的产品（例如，税务处理、会计事务）不同，这些用例通常是信息密集型的，需要实时访问与人员、组织和他们拥有的东西相关联的大量数据。

To achieve this, we have created a platform called Unified Profile Service utilizing Flink. This platform is intended to provide the strategic data asset of a trusted, real-time, unified and connected view of people, organization and things they own. We have abstracted re-usable components such as sources, sinks, transformations etc and created a template. Utilizing this template our Product teams are able to rapidly test domain specific transformations and computations by creating and deploying Flink Jobs. This platform is running in production on AWS EMR, powering multiple use cases, ingesting and processing billions of events per day.

为了实现这一点，我们利用Flink创建了一个名为UnifiedProfileService的平台。该平台旨在为受信任、实时、统一和连接的人、组织和他们拥有的事物提供战略数据资产。我们抽象了可重用的组件，如源、接收器、转换等，并创建了一个模板。利用这个模板，我们的产品团队能够通过创建和部署Flink作业来快速测试特定于领域的转换和计算。这个平台正在AWS EMR上运行，为多个用例供电，每天接收和处理数十亿个事件。

In this talk, we will be discussing the design details of this Platform built leveraging Flink and Flink APIs as well as challenges faced along the way. We will begin by talking about the various components of the pipeline such as Identity Stitching, Entity Resolution, Reconciliation and Data Persistence. We will then dig in to the technical details of how we abstracted away these common components and created a template. We will also talk about how we update Consumer’s Financial Identity Graph in real-time through custom built AWS Dynamodb and Neptune Sink using Flink’s Connector API.

在本次讨论中，我们将讨论利用Flink和Flink API构建的平台的设计细节，以及沿途面临的挑战。我们将首先讨论管道的各个组件，如身份拼接、实体解析、协调和数据持久性。然后，我们将深入研究如何提取这些通用组件并创建模板的技术细节。我们还将讨论如何通过使用Flink的连接器API定制的AWS dynamodb和Neptune接收器实时更新消费者的财务身份图。

Finally we will touch on lessons learnt along the way as we deployed the platform in production and offer advice on things to avoid as well as how to take things to the next level.

最后，我们将在生产中部署平台的过程中吸取经验教训，并就要避免的事情以及如何将事情提升到下一个层次提供建议。