blocks|key|2136861|text|根据我们的经验，data+provenance仅为业务用户提供了系统的高级视图，因此他们可以大致导航其数据来自何处。它是由各种建模工具或简单的自定义表格和图表提供的。Data+lineage是一个更具体的术语，它包括两个方面-业务(数据)谱系和技术(数据)谱系。业务谱系图片数据在业务术语级别上流动，它是由Collibra、Alation等解决方案提供的。技术数据谱系从实际的技术元数据创建，并跟踪最低级别的数据流-实际的表、脚本和语句。MANTA或Informatica+Metadata+Manager等解决方案提供了技术数据谱系。|type|unstyled|depth|inlineStyleRanges|offset|length|style|BOLD|entityRanges|data|2136862|entityMap^0|8|F|2B|C|0^^$0|@$1|2|3|4|5|6|7|H|8|@$9|I|A|J|B|C]|$9|K|A|L|B|C]]|D|@]|E|$]]|$1|F|3|-4|5|6|7|M|8|@]|D|@]|E|$]]]|G|$]]

From our experience, data provenance includes only high level view of the system for business users, so they can roughly navigate where their data come from. It's provided by variety of modeling tools or just simple custom tables and charts. Data lineage is a more specific term and includes two sides - business (data) lineage and technical (data) lineage. Business lineage pictures data flows on a business-term level and it's provided by solutions like Collibra, Alation and many others. Technical data lineage is created from actual technical metadata and tracks data flows on the lowest level - actual tables, scripts and statements. Technical data lineage is being provided by solutions such as MANTA or Informatica Metadata Manager.

blocks|key|4248346|text|数据起源是数据术语的起始点，数据谱系是从系统中的起始点到当前观察点的完整数据转换过程。|type|unstyled|depth|inlineStyleRanges|entityRanges|data|4248347|entityMap^0|0^^$0|@$1|2|3|4|5|6|7|D|8|@]|9|@]|A|$]]|$1|B|3|-4|5|6|7|E|8|@]|9|@]|A|$]]]|C|$]]

Data Provenance is the point of origin for the data term, Data Lineage is the complete data transformation journey from point of origin to current observation point in system.

blocks|key|5665901|text|数据来源是，|type|unstyled|depth|inlineStyleRanges|entityRanges|data|5665902|数据谱系(什么是谱系，它的历程，它的历史，它是从哪里开始的，它是如何产生的，它是如何随着时间的推移而变化的，它去过哪里，它经历过的系统，任何损失或收益)(例如，面向数据的元数据)|5665903|加上|offset|length|style|BOLD|5665904|影响数据的输入、实体、系统和过程(即面向过程)，可用于复制数据。|5665905|entityMap^0|0|0|0|2|0|0^^$0|@$1|2|3|4|5|6|7|N|8|@]|9|@]|A|$]]|$1|B|3|C|5|6|7|O|8|@]|9|@]|A|$]]|$1|D|3|E|5|6|7|P|8|@$F|Q|G|R|H|I]]|9|@]|A|$]]|$1|J|3|K|5|6|7|S|8|@]|9|@]|A|$]]|$1|L|3|-4|5|6|7|T|8|@]|9|@]|A|$]]]|M|$]]

Data Provenance is,

data lineage (what is the genealogy,history of its journey, where did it begin, how did it come into being, how did it change over time, where has it been, systems it has traveled, any loss or gain) (i.e. data oriented, metadata)

PLUS 

the inputs, entities, systems and processes that influenced the data (i.e. process oriented) which can be used to reproduce the data.

blocks|key|2136966|text|请参阅维基百科上关于起源的文章：https://en.wikipedia.org/wiki/Provenance#Science中的这一节。它链接到关于起源的学术和行业工作的集合。|type|unstyled|depth|inlineStyleRanges|entityRanges|offset|length|data|2136967|简明扼要地回答您的问题:一般来说，没有足够的已知上下文来区分数据谱系和数据来源。在特定的上下文中，您可以查找或创建特定的或可能不同的定义。|2136968|entityMap|0|LINK|mutability|MUTABLE|url|https://en.wikipedia.org/wiki/Provenance#Science^0|G|1C|0|0|0^^$0|@$1|2|3|4|5|6|7|N|8|@]|9|@$A|O|B|P|1|Q]]|C|$]]|$1|D|3|E|5|6|7|R|8|@]|9|@]|C|$]]|$1|F|3|-4|5|6|7|S|8|@]|9|@]|C|$]]]|G|$H|$5|I|J|K|C|$L|M]]]]

See this section in the Wikipedia articl on provenance: <a href="https://en.wikipedia.org/wiki/Provenance#Science" rel="nofollow noreferrer">https://en.wikipedia.org/wiki/Provenance#Science</a>. It links to collections of academic and industry work on provenance.

To succinctly answer your question: in general, there's not enough context known to differentiate between data lineage and data provenance. Within a specific context, you could look for, or create, specific and possibly different, definitions.

blocks|key|4248439|text|我相信一个更简单的解释是谁拥有它，谁触摸了它，它将走向何方。|type|unstyled|depth|inlineStyleRanges|entityRanges|data|4248440|在业务意义上，这可以归结为数据流图。|4248441|从技术上讲，当数据从一个系统流动到另一个系统时，要开始添加到数据上，这是一个很大的负担。必须有一些巨大的理由才能带着这座山到处走，目的是什么？去看一些漂亮的图表？在大型现实环境中不会发生这种情况。$$$中的理由是什么？？|4248442|在数据从一个系统移动到另一个系统时，用一个简单2-4字节源代码来标记数据是一回事，但为了保持所有其他的技术巨型，系统性能下降/+dasd+/备份/等等的成本对于一个漂亮的图形？不可能的..。|4248443|entityMap^0|0|0|0|0^^$0|@$1|2|3|4|5|6|7|J|8|@]|9|@]|A|$]]|$1|B|3|C|5|6|7|K|8|@]|9|@]|A|$]]|$1|D|3|E|5|6|7|L|8|@]|9|@]|A|$]]|$1|F|3|G|5|6|7|M|8|@]|9|@]|A|$]]|$1|H|3|-4|5|6|7|N|8|@]|9|@]|A|$]]]|I|$]]

I believe a more simple explanation is who owns it, who touched it, and where is it going.

In a Business sense, that can be summed up in Data Flow Diagrams.

In a Technical sense, that's a whole lot of baggage to start adding onto data as it flows from system to system. There has to be some HUGE justification to carry that mountain around and for what purpose? To see some pretty graphs? Not going to happen in large real world environments. The justification in $$$ for what??

It's one thing to tag data with a simple 2 - 4 byte code of origin as it moves from system to system, but to keep all of that other technical jumbo, the cost in system performance degradation / dasd / backups / etc. for a pretty graph? No way....

blocks|key|5666013|text|数据沿袭Vs.数据起源:目标数据沿袭工具的关键目标是从数据起源到数据耗尽的数据生命周期管理。|type|unstyled|depth|inlineStyleRanges|offset|length|style|BOLD|entityRanges|data|5666014|另一方面，数据起源的关键目标是在三个关键阶段具体跟踪数据来源和分离数据。这些阶段是动态数据、进程数据和静态数据。|5666015|数据沿袭Vs.数据来源:组件数据谱系的关键组件包括web门户、数据捕获源和数据培育方法。这些组件还包括数据鉴定系统、CRM系统和ERP系统。|5666016|而另一方面，数据起源的关键组件包括所有数据谱系组件以及更多。这些附加组件正在跟踪捕获源和数据输入方法。|5666017|数据沿袭Vs.数据来源:挑战数据沿袭的主要挑战包括管理大量数据。它还包括维护数据谱系、跟踪跨渠道和统一不同的促销系统。|5666018|而数据起源的关键挑战包括数据谱系挑战等。其他挑战包括大型而复杂的工作流，以及为保留数据而重新执行。|5666019|Here's+the+link+to+the+complete+post。|5666020|entityMap|0|LINK|mutability|MUTABLE|url|https://www.knowledgenile.com/blogs/data-provenance-vs-data-lineage/^0|0|E|0|0|0|E|0|0|0|E|0|0|0|10|0|0^^$0|@$1|2|3|4|5|6|7|Z|8|@$9|10|A|11|B|C]]|D|@]|E|$]]|$1|F|3|G|5|6|7|12|8|@]|D|@]|E|$]]|$1|H|3|I|5|6|7|13|8|@$9|14|A|15|B|C]]|D|@]|E|$]]|$1|J|3|K|5|6|7|16|8|@]|D|@]|E|$]]|$1|L|3|M|5|6|7|17|8|@$9|18|A|19|B|C]]|D|@]|E|$]]|$1|N|3|O|5|6|7|1A|8|@]|D|@]|E|$]]|$1|P|3|Q|5|6|7|1B|8|@]|D|@$9|1C|A|1D|1|1E]]|E|$]]|$1|R|3|-4|5|6|7|1F|8|@]|D|@]|E|$]]]|S|$T|$5|U|V|W|E|$X|Y]]]]

Data Lineage Vs. Data Provenance: Goals
The key goal of a data lineage tool is data lifecycle management right from the data origination to the data exhaustion.
On the other hand, the key goal of data provenance is to specifically track the data origination and segregating data in three key stages. These stages are data-in-motion, data-in-process, and data-in-rest.
Data Lineage Vs. Data Provenance: Components
The key components of data lineage include a web portal, data capture sources, and data nurture methods. These components also include data qualification systems, CRM systems, and an ERP system.
While on the other hand, the key components of data provenance include all the data lineage components and some more. These additional components are tracking the capture sources and data input methods.
Data Lineage Vs. Data Provenance: Challenges
Key challenges of data lineage include managing large volumes of data. It also includes maintaining data lineage, tracking cross channels, and unifying disparate promotional systems.
While the key challenges of data provenance include the data lineage challenges and a few more. The additional challenges include large and complicated workflows, and reproducing the execution for data retention.
<a href="https://www.knowledgenile.com/blogs/data-provenance-vs-data-lineage/" rel="nofollow noreferrer">Here's the link to the complete post</a>.

blocks|key|2137099|text|定义|type|unstyled|depth|inlineStyleRanges|entityRanges|data|2137100|让我强调一下我认为是数据来源的关键部分，这是在数据谱系的定义中找不到的：|2137101|提供数据及其来源的历史记录的|2137102|blockquote|2137103|2137104|2137105|虽然措辞不同，但我相信这一补充是如何定义起源和血统的唯一相关区别。|2137106|释义|2137107|我遵循的解释，以及我经常在大数据环境中看到的解释是，谱系向您显示数据所采用的路径，但来源允许您了解数据在整个过程中的样子。|2137108|示例|2137109|如果您有执行此操作的工作流：|2137110|从源a，b>合并到c>以‘随机’的方式更新输入并存储在d中|2137111|然后我会说，谱系让你知道数据是从a，b到c再到d。深层谱系甚至可以让你看到用于这一点的逻辑。然而，这可能不会让你知道c是什么样子的，在理论上随机的例子中，这是希望清楚的，但在实践中，有更少的随机情况，但许多不可重现的情况，它也可能是随机的。|2137112|现在，provenance将跟踪所采用的路径，最重要的是，c中的数据是什么样子的。|2137113|关于实现的说明|2137114|正如其他人提到的，跟踪和存储起源可能是一个沉重的负担，但它可以很好地帮助开发，特别是流数据流的开发(这就像到处都有调试点)。此外，可能存在出处非常重要的情况(或者数据量和转换次数相对较低)，因此可能希望将出处保留一段时间。|2137115|在实践中，起源不会像谱系一样保持很长时间，但是像NiFi这样的一些工具确实会捕获它，在它最有价值的地方保存一段时间，同时跟踪正常的谱系。|2137116|2137117|完全披露和免责声明:尽管我是Cloudera的一名员工，但上面的描述是基于我的个人经验，以及与同事和客户关于血统和起源的交谈。|2137118|entityMap^0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0^^$0|@$1|2|3|4|5|6|7|1A|8|@]|9|@]|A|$]]|$1|B|3|C|5|6|7|1B|8|@]|9|@]|A|$]]|$1|D|3|E|5|6|7|1C|8|@]|9|@]|A|$]]|$1|F|3|-4|5|G|7|1D|8|@]|9|@]|A|$]]|$1|H|3|-4|5|6|7|1E|8|@]|9|@]|A|$]]|$1|I|3|-4|5|6|7|1F|8|@]|9|@]|A|$]]|$1|J|3|K|5|6|7|1G|8|@]|9|@]|A|$]]|$1|L|3|M|5|6|7|1H|8|@]|9|@]|A|$]]|$1|N|3|O|5|6|7|1I|8|@]|9|@]|A|$]]|$1|P|3|Q|5|6|7|1J|8|@]|9|@]|A|$]]|$1|R|3|S|5|6|7|1K|8|@]|9|@]|A|$]]|$1|T|3|U|5|6|7|1L|8|@]|9|@]|A|$]]|$1|V|3|W|5|6|7|1M|8|@]|9|@]|A|$]]|$1|X|3|Y|5|6|7|1N|8|@]|9|@]|A|$]]|$1|Z|3|10|5|6|7|1O|8|@]|9|@]|A|$]]|$1|11|3|12|5|6|7|1P|8|@]|9|@]|A|$]]|$1|13|3|14|5|6|7|1Q|8|@]|9|@]|A|$]]|$1|15|3|-4|5|6|7|1R|8|@]|9|@]|A|$]]|$1|16|3|17|5|6|7|1S|8|@]|9|@]|A|$]]|$1|18|3|-4|5|6|7|1T|8|@]|9|@]|A|$]]]|19|$]]

<h1>Definition</h1>
let me highlight what I believe to be the critical part of data provenance that is not found in the definition of in data lineage:
<blockquote>
providing a historical record of the data and its origins
</blockquote>
Though the wording is different, I believe this addition is the only relevant difference in how provenance and lineage are defined.
<h1>Interpretation</h1>
The interpretation that I follow, and that I have seen used often in a Big Data context, is that lineage shows you which path the data has taken, but provenance allows you to know what the data looked like along the way.
<h1>Example</h1>
If you have a workflow that does this:
Gather input from source a, b &gt; combine to c &gt; update in 'random' fashion and store in d
Then I would say lineage allows you to know that the data went from a, b through c into d. Deep lineage would even allow you to see the logic used for this. However, this may not let you know what c looked like, in the theoretical random example this is hopefully clear, but in practice there are less random situations, but many irreproducible situations to the point where it might as well have been random.
Now provenance would keep track of the path taken, and on top of this also what the data looked like in c.
<h1>Note on implementation</h1>
As others mentioned, tracking and storing provenance can be a heavy burden, but it can be great to assist in development, especially of streaming data flows (it is like having a debug point everywhere). Furthermore there may be cases where the provenance is so important (or the data volume and number of transformations comparatively low) that one may want to keep the provenance for a certain period of time.
In practice provenance is not kept as long as lineage, but some tools like NiFi do capture it out of the box, keep it for a short while where it is most valuable, and in parallel track the normal lineage.
<hr />
Full disclosure and disclaimer:
Though I am an employee of Cloudera, a company much involved with Governance, Lineage and products like NiFi, the description above is based on my personal experience, and from talking to colleagues and customers about Lineage and Provenance.

From wiki,

Data lineage is defined as a data life cycle that includes the data's origins and where it moves over time. It describes what happens to data as it goes through diverse processes. It helps provide visibility into the analytics pipeline and simplifies tracing errors back to their sources.

Data provenance documents the inputs, entities, systems, and processes that influence data of interest, in effect providing a historical record of the data and its origins.

It seems that both concepts are talking about about where the data comes from but I'm still confused about the differences. Are both the concepts the same? If they are different, can someone shares an example?

Thanks,

What are the differences between Data Lineage and Data Provenance?

翻译质量差，导致语言生硬或混乱。

没有提供实际的解决方法或示例。

解答不清晰，无法理解或解决问题。

页面排版不美观，阅读体验差。

文章

问答

视频

学习中心

腾讯云实验室

直播

竞赛

腾讯云代码分析专区

腾讯iOA零信任安全管理系统专区

腾讯云架构师技术同盟交流圈

腾讯云数据库专区

腾讯云顾问专区

腾讯云原生专区

腾讯混元专区

腾讯云TCE专区

腾讯云Lighthouse专区

腾讯云HAI专区

腾讯云Edgeone专区

腾讯云存储专区

腾讯云智能专区

腾讯轻联专区 

腾讯云开发专区

TAPD专区

腾讯轻量云游戏服专区

腾讯云最具价值专家

腾讯云架构师技术同盟

腾讯云创作之星

腾讯云开发者先锋

腾讯云代码助手

云原生构建

TAPD 敏捷项目管理

Cloud Studio

SDK中心

API中心

命令行工具

涵盖代码开发、场景应用、自动测试全流程，助你从零构建专属AI助手

一站式MCP教程库，解锁AI应用新玩法

来自维基，数据沿袭被定义为一个数据生命周期，它包括数据的来源及其随时间移动的位置。它描述了数据在经历不同的过程时发生了什么。它有助于提供对分析管道的可见性，并简化对错误来源的跟踪。数据起源记录了影响感兴趣数据的输入、实体、系统和过程，有效地提供了数据及其来源的历史记录。似乎这两个概念都在谈论数据来自哪里，但我仍然对其中...

问数据沿袭和数据来源之间有什么区别？
EN

Stack Overflow用户

社区

活动

圈层

关于

腾讯云开发者

热门产品

热门推荐

更多推荐

问数据沿袭和数据来源之间有什么区别？EN