blocks|key|1807279|text|根据Cloudera's+Impala+指南+(ClouderaEnterprise5.8)，但5.9保持不变|type|unstyled|depth|inlineStyleRanges|entityRanges|offset|length|data|1807280|失效元数据和刷新是对应的:当后续查询需要时，失效元数据等待重新加载元数据，但是重新加载表的所有元数据，这可能是一项昂贵的操作，特别是对于具有多个分区的大型表。REFRESH立即重新加载元数据，但只加载新添加的数据文件的块位置数据，从而降低了整个操作的开销。如果以更广泛的方式更改数据，例如由HDFS均衡器重新组织，则使用无效元数据来避免减少本地读取的性能损失。如果使用Impala+1.0版本，则无效元数据语句的工作方式与Impala+1.0+REFRESH语句一样，而Impala+1.1刷新是针对向现有表添加新数据文件的常见用例优化的，因此现在需要表名参数。|blockquote|1807281|与处理现有表格有关：|1807282|表名是刷新所需的参数。若要刷新所有表的元数据，请使用“无效元数据”命令。因为REFRESH+table_name只适用于当前Impala节点已经知道的表，所以当您在Hive中创建一个新表时，在您可以在INVALIDATE中看到新表之前输入无效元数据new_table。一旦该表为Impala所知，您可以在为该表添加数据文件后发出刷新table_name。|1807283|因此，它似乎确实保持不变。我相信CDH+5.9配备了Impala+2.7。|1807284|entityMap|0|LINK|mutability|MUTABLE|url|https://www.cloudera.com/documentation/enterprise/5-8-x/topics/impala_refresh.html|1|https://www.cloudera.com/documentation/enterprise/5-9-x/topics/impala_refresh.html^0|K|2|0|1C|3|1|0|0|0|0|0^^$0|@$1|2|3|4|5|6|7|W|8|@]|9|@$A|X|B|Y|1|Z]|$A|10|B|11|1|12]]|C|$]]|$1|D|3|E|5|F|7|13|8|@]|9|@]|C|$]]|$1|G|3|H|5|6|7|14|8|@]|9|@]|C|$]]|$1|I|3|J|5|F|7|15|8|@]|9|@]|C|$]]|$1|K|3|L|5|6|7|16|8|@]|9|@]|C|$]]|$1|M|3|-4|5|6|7|17|8|@]|9|@]|C|$]]]|N|$O|$5|P|Q|R|C|$S|T]]|U|$5|P|Q|R|C|$S|V]]]]

According to Cloudera's Impala <a href="https://www.cloudera.com/documentation/enterprise/5-8-x/topics/impala_refresh.html" rel="noreferrer">guide</a> (Cloudera Enterprise 5.8) but stayed the same for <a href="https://www.cloudera.com/documentation/enterprise/5-9-x/topics/impala_refresh.html" rel="noreferrer">5.9</a>:

<blockquote>
 INVALIDATE METADATA and REFRESH are counterparts: INVALIDATE METADATA
 waits to reload the metadata when needed for a subsequent query, but
 reloads all the metadata for the table, which can be an expensive
 operation, especially for large tables with many partitions. REFRESH
 reloads the metadata immediately, but only loads the block location
 data for newly added data files, making it a less expensive operation
 overall. If data was altered in some more extensive way, such as being
 reorganized by the HDFS balancer, use INVALIDATE METADATA to avoid a
 performance penalty from reduced local reads. If you used Impala
 version 1.0, the INVALIDATE METADATA statement works just like the
 Impala 1.0 REFRESH statement did, while the Impala 1.1 REFRESH is
 optimized for the common use case of adding new data files to an
 existing table, thus the table name argument is now required.
</blockquote>

and related to working on existing tables:

<blockquote>
 The table name is a required parameter [for REFRESH]. To flush the metadata for all
 tables, use the INVALIDATE METADATA command.
 Because REFRESH table_name only works for tables that the current
 Impala node is already aware of, when you create a new table in the
 Hive shell, enter INVALIDATE METADATA new_table before you can see the
 new table in impala-shell. Once the table is known by Impala, you can
 issue REFRESH table_name after you add data files for that table.
</blockquote>

So it seems like it indeed stayed the same. I believe CDH 5.9 comes with Impala 2.7.

blocks|key|88686|text|根据Impala文档使Metada无效和刷新|type|unstyled|depth|inlineStyleRanges|entityRanges|offset|length|data|88687|使元数据语句失效|style|BOLD|88688|失效元数据语句将一个或所有表的元数据标记为陈旧。下一次当Impala服务对元数据无效的表执行查询时，Impala将在查询进行之前重新加载相关的元数据。由于与REFRESH语句所做的增量元数据更新相比，这是一个非常昂贵的操作，因此在可能的情况下，更倾向于刷新而不是失效元数据。|88689|当在Impala之外、Hive和其他Hive客户端(如SparkSQL)中进行以下更改时，需要失效元数据：|88690|现有表的元数据更改。|unordered-list-item|88691|添加新表，Impala将使用这些表。|88692|服务器或数据库级别的Sentry特权被更改。|88693|块元数据更改，但文件保持不变(HDFS重新平衡)。UDF罐子变了。|88694|有些表不再被查询，您希望从目录和协调器缓存中删除它们的元数据，以减少内存需求。|88695|当impalad进行更改时，不需要无效元数据。|88696|刷新语句|88697|REFRESH语句从亚稳数据库中重新加载表的元数据，并对文件进行增量重新加载，并从HDFS+NameNode中阻止元数据。刷新用于避免Impala和外部元数据来源(即Hive和NameNodes+)之间的不一致。|88698|使用说明：|88699|表名是必需的参数，表必须已经存在并为Impala所知。|88700|只重新加载指定表的元数据。|88701|在Impala之外发生以下情况之一之后，使用REFRESH语句加载特定表的最新转移元数据：|88702|删除、添加或修改文件。
例如，在将新的数据文件加载到表的HDFS数据目录后，附加到现有的HDFS文件中，通过INSERT或LOAD从Hive插入数据。|88703|删除、添加或修改分区。
例如，在Hive中发出ALTER或其他表修改SQL语句之后|88704|entityMap|0|LINK|mutability|MUTABLE|url|https://impala.apache.org/docs/build/html/topics/impala_invalidate_metadata.html|1|https://impala.apache.org/docs/build/html/topics/impala_refresh.html^0|A|9|0|K|2|1|0|0|6|0|0|0|0|0|0|0|0|0|0|4|0|0|0|0|0|0|0|0^^$0|@$1|2|3|4|5|6|7|1O|8|@]|9|@$A|1P|B|1Q|1|1R]|$A|1S|B|1T|1|1U]]|C|$]]|$1|D|3|E|5|6|7|1V|8|@$A|1W|B|1X|F|G]]|9|@]|C|$]]|$1|H|3|I|5|6|7|1Y|8|@]|9|@]|C|$]]|$1|J|3|K|5|6|7|1Z|8|@]|9|@]|C|$]]|$1|L|3|M|5|N|7|20|8|@]|9|@]|C|$]]|$1|O|3|P|5|N|7|21|8|@]|9|@]|C|$]]|$1|Q|3|R|5|N|7|22|8|@]|9|@]|C|$]]|$1|S|3|T|5|N|7|23|8|@]|9|@]|C|$]]|$1|U|3|V|5|N|7|24|8|@]|9|@]|C|$]]|$1|W|3|X|5|6|7|25|8|@]|9|@]|C|$]]|$1|Y|3|Z|5|6|7|26|8|@$A|27|B|28|F|G]]|9|@]|C|$]]|$1|10|3|11|5|6|7|29|8|@]|9|@]|C|$]]|$1|12|3|13|5|6|7|2A|8|@]|9|@]|C|$]]|$1|14|3|15|5|6|7|2B|8|@]|9|@]|C|$]]|$1|16|3|17|5|6|7|2C|8|@]|9|@]|C|$]]|$1|18|3|19|5|6|7|2D|8|@]|9|@]|C|$]]|$1|1A|3|1B|5|N|7|2E|8|@]|9|@]|C|$]]|$1|1C|3|1D|5|N|7|2F|8|@]|9|@]|C|$]]|$1|1E|3|-4|5|6|7|2G|8|@]|9|@]|C|$]]]|1F|$1G|$5|1H|1I|1J|C|$1K|1L]]|1M|$5|1H|1I|1J|C|$1K|1N]]]]

As per Impala document <a href="https://impala.apache.org/docs/build/html/topics/impala_invalidate_metadata.html" rel="nofollow noreferrer">Invalidate Metada</a> and <a href="https://impala.apache.org/docs/build/html/topics/impala_refresh.html" rel="nofollow noreferrer">Refresh</a>
INVALIDATE METADATA Statement
The INVALIDATE METADATA statement marks the metadata for one or all tables as stale. The next time the Impala service performs a query against a table whose metadata is invalidated, Impala reloads the associated metadata before the query proceeds. As this is a very expensive operation compared to the incremental metadata update done by the REFRESH statement, when possible, prefer REFRESH rather than INVALIDATE METADATA.
INVALIDATE METADATA is required when the following changes are made outside of Impala, in Hive and other Hive client, such as SparkSQL:
<ul>
<li>Metadata of existing tables changes.</li>
<li>New tables are added, and Impala will use the tables.</li>
<li>The SERVER or DATABASE level Sentry privileges are changed.</li>
<li>Block metadata changes, but the files remain the same (HDFS rebalance).
UDF jars change.</li>
<li>Some tables are no longer queried, and you want to remove their metadata from the catalog and coordinator caches to reduce memory requirements.</li>
</ul>
No INVALIDATE METADATA is needed when the changes are made by impalad.
REFRESH Statement
The REFRESH statement reloads the metadata for the table from the metastore database and does an incremental reload of the file and block metadata from the HDFS NameNode. REFRESH is used to avoid inconsistencies between Impala and external metadata sources, namely Hive Metastore (HMS) and NameNodes.
Usage notes:
The table name is a required parameter, and the table must already exist and be known to Impala.
Only the metadata for the specified table is reloaded.
Use the REFRESH statement to load the latest metastore metadata for a particular table after one of the following scenarios happens outside of Impala:
<ul>
<li>Deleting, adding, or modifying files. 
For example, after loading new data files into the HDFS data directory for the table, appending to an existing HDFS file, inserting data from Hive via INSERT or LOAD DATA.</li>
<li>Deleting, adding, or modifying partitions. 
For example, after issuing ALTER TABLE or other table-modifying SQL statement in Hive</li>
</ul>

blocks|key|88728|text|失效元数据用于刷新亚稳态和数据(structure+&+data)==complete刷新。|type|ordered-list-item|depth|inlineStyleRanges|offset|length|style|CODE|entityRanges|data|88729|Refresh仅用于更新数据=轻量级刷新|88730|unstyled|entityMap^0|F|S|0|0^^$0|@$1|2|3|4|5|6|7|K|8|@$9|L|A|M|B|C]]|D|@]|E|$]]|$1|F|3|G|5|6|7|N|8|@]|D|@]|E|$]]|$1|H|3|-4|5|I|7|O|8|@]|D|@]|E|$]]]|J|$]]

<ol>
<li>Invalidate metadata is used to refresh the metastore and the data <code>(structure &amp; data)==complete</code> flush</li>
<li>Refresh is used to update only the data = lightweight flush</li>
</ol>

I saw at this link which affects <a href="https://issues.cloudera.org/browse/IMPALA-537" rel="noreferrer">Impala version 1.1</a>:

<blockquote>
 Since Impala 1.1, REFRESH statement only works for existing tables. For new tables you need to issue "INVALIDATE METADATA" statement.
</blockquote>

Does this still hold true for later versions of Impala?

Difference between invalidate metadata and refresh commands in Impala?

翻译质量差，导致语言生硬或混乱。

没有提供实际的解决方法或示例。

解答不清晰，无法理解或解决问题。

页面排版不美观，阅读体验差。

文章

问答

视频

教程

学习中心

腾讯云实验室

直播

竞赛

腾讯云代码分析专区

腾讯iOA零信任安全管理系统专区

腾讯云架构师技术同盟交流圈

腾讯云数据库专区

腾讯云智能顾问专区

腾讯云原生专区

腾讯混元专区

腾讯云TCE专区

腾讯云Lighthouse专区

腾讯云HAI专区

腾讯云Edgeone专区

腾讯云存储专区

腾讯云智能专区

腾讯轻联专区 

腾讯云开发专区

TAPD专区

腾讯轻量云游戏服专区

腾讯云最具价值专家

腾讯云架构师技术同盟

腾讯云创作之星

腾讯云开发者先锋

腾讯云代码助手

云原生构建

TAPD 敏捷项目管理

Cloud Studio

SDK中心

API中心

命令行工具

涵盖代码开发、场景应用、自动测试全流程，助你从零构建专属AI助手

一站式MCP教程库，解锁AI应用新玩法

聚焦“写作效率、视觉美观与运行性能”三方面进行全面升级，为您提供更高效、稳定的创作环境

社区富文本&Markdown编辑器全新改版上线，欢迎大家体验!

诚挚邀请您参与本次调研，分享您的真实使用感受与建议。您的反馈至关重要，感谢您的支持与参与！

社区新版编辑器体验调研

我在这个链接上看到了影响的链接从Impala 1.1开始，REFRESH语句只适用于现有表。对于新表，需要发出“失效元数据”语句。对于后来版本的Impala来说，这仍然成立吗？

问在Impala中失效元数据和刷新命令之间的区别？
EN

社区

活动

圈层

关于

腾讯云开发者

热门产品

热门推荐

更多推荐

问在Impala中失效元数据和刷新命令之间的区别？EN

社区

活动

圈层

关于

腾讯云开发者

热门产品

热门推荐

更多推荐

问在Impala中失效元数据和刷新命令之间的区别？
EN