blocks|key|138756|text|Hcatalog是Hadoop文件系统的元数据管理。Hcatalog可以通过webhcat访问，webhcat使用rest+api。无论在hcatalog中创建什么表，都可以通过hive和pig进行访问。|type|unstyled|depth|inlineStyleRanges|entityRanges|data|138757|entityMap^0|0^^$0|@$1|2|3|4|5|6|7|D|8|@]|9|@]|A|$]]|$1|B|3|-4|5|6|7|E|8|@]|9|@]|A|$]]]|C|$]]

Hcatalog is the metadata management of Hadoop File system. Hcatalog can be accessed through webhcat which makes use of rest api. Whatever the tables created in hcatalog can be accessed through hive and pig.

blocks|key|138796|text|简而言之，HCatalog向其他mapreduce工具开放了蜂窝元数据。每个mapreduce工具都有自己关于HDFS数据的概念(例如Pig将HDFS数据视为文件集，Hive将其视为表)。有了基于表的抽象，HCatalog支持的mapreduce工具不需要关心数据存储在哪里，以哪种格式和存储位置(HBase或HDFS)。|type|unstyled|depth|inlineStyleRanges|entityRanges|data|138797|如果您沿着Hcatalog配置WebHcat，我们确实可以获得webhcat以RESTful方式提交作业的功能。|138798|entityMap^0|0|0^^$0|@$1|2|3|4|5|6|7|F|8|@]|9|@]|A|$]]|$1|B|3|C|5|6|7|G|8|@]|9|@]|A|$]]|$1|D|3|-4|5|6|7|H|8|@]|9|@]|A|$]]]|E|$]]

In short, HCatalog opens up the hive metadata to other mapreduce tools. Every mapreduce tools has its own notion about HDFS data (example Pig sees the HDFS data as set of files, Hive sees it as tables). With having table based abstraction, HCatalog supported mapreduce tools do not need to care about where the data is stored, in which format and storage location (HBase or HDFS).

We do get the facility of WebHcat to submit jobs in an RESTful way if you configure webhcat along Hcatalog.

blocks|key|4645485|text|这是一个关于ho如何使用HCATALOG的非常基本的例子。|type|unstyled|depth|inlineStyleRanges|entityRanges|data|4645486|我在配置单元中有一个表，表名为STUDENT，它存储在HDFS位置之一：|4645487|neethu++90+malini++90+sunitha+98+mrinal++56+ravi++++90+joshua++8|offset|length|style|CODE|4645488|现在假设我想要将这个表加载到pig以进行进一步的数据转换，在这个场景中，我可以使用HCATALOG：|4645489|在对Pig使用Hive+metastore中的表信息时，请在调用pig时添加-useHCatalog选项：|4645490|pig+-useHCatalog|4645491|(您可能希望导出HCAT_HOME+'HCAT_HOME=/usr/lib/hive-hcatalog/')|4645492|现在将该表加载到pig：A+=+LOAD+'student'+USING+org.apache.hcatalog.pig.HCatLoader();|4645493|现在，您已经将表加载到pig.To，检查模式，只需对关系进行描述即可。|4645494|DESCRIBE+A|4645495|谢谢|4645496|entityMap^0|0|0|0|1S|0|0|0|0|G|0|0|C|1Q|0|0|0|A|0|0^^$0|@$1|2|3|4|5|6|7|11|8|@]|9|@]|A|$]]|$1|B|3|C|5|6|7|12|8|@]|9|@]|A|$]]|$1|D|3|E|5|6|7|13|8|@$F|14|G|15|H|I]]|9|@]|A|$]]|$1|J|3|K|5|6|7|16|8|@]|9|@]|A|$]]|$1|L|3|M|5|6|7|17|8|@]|9|@]|A|$]]|$1|N|3|O|5|6|7|18|8|@$F|19|G|1A|H|I]]|9|@]|A|$]]|$1|P|3|Q|5|6|7|1B|8|@]|9|@]|A|$]]|$1|R|3|S|5|6|7|1C|8|@$F|1D|G|1E|H|I]]|9|@]|A|$]]|$1|T|3|U|5|6|7|1F|8|@]|9|@]|A|$]]|$1|V|3|W|5|6|7|1G|8|@$F|1H|G|1I|H|I]]|9|@]|A|$]]|$1|X|3|Y|5|6|7|1J|8|@]|9|@]|A|$]]|$1|Z|3|-4|5|6|7|1K|8|@]|9|@]|A|$]]]|10|$]]

Here is a very basic example of how ho use HCATALOG.

I have a table in hive ,TABLE NAME is STUDENT which is stored in one of the HDFS location:

<code>neethu 90
malini 90
sunitha 98
mrinal 56
ravi 90
joshua 8</code>

Now suppose I want to load this table to pig for further transformation of data,
In this scenario I can use HCATALOG:

When using table information from the Hive metastore with Pig, add the -useHCatalog option when invoking pig:

<code>pig -useHCatalog</code>

(you may want to export HCAT_HOME 'HCAT_HOME=/usr/lib/hive-hcatalog/')

Now loading this table to pig:
<code>A = LOAD 'student' USING org.apache.hcatalog.pig.HCatLoader();</code>

Now you have loaded the table to pig.To check the schema , just do a DESCRIBE on the relation.

<code>DESCRIBE A</code>

Thanks

blocks|key|138923|text|HCatalog是一个用于Hadoop的表存储管理工具，它向其他Hadoop应用程序公开配置单元元存储的表格数据。它使具有不同数据处理工具的用户能够轻松地在表格网格中写入数据。它确保用户不必担心存储格式。|type|unstyled|depth|inlineStyleRanges|entityRanges|data|138924|entityMap^0|0^^$0|@$1|2|3|4|5|6|7|D|8|@]|9|@]|A|$]]|$1|B|3|-4|5|6|7|E|8|@]|9|@]|A|$]]]|C|$]]

HCatalog is a table storage management tool for Hadoop that exposes the tabular data of Hive metastore to other Hadoop applications. It enables users with different data processing tools to easily write data in a tabular grid. It ensures that users don’t have to worry about the storage format.

I'm new to Hadoop. I know that the HCatalog is a table and storage management layer for Hadoop. But how exactly it works and how to use it. Please give some simple example.

What is use of hcatalog in hadoop?

数据存储

数据管理

数据转换

存储管理

文件系统

Hadoop 

HBase

Hive

HDFS

我是Hadoop的新手。我知道HCatalog是Hadoop的一个表和存储管理层。但它到底是如何工作的，以及如何使用它。请举一些简单的例子。

问hcatalog在hadoop中的用途是什么？
EN

社区

活动

资源

关于

腾讯云开发者

热门产品

热门推荐

更多推荐

问hcatalog在hadoop中的用途是什么？EN

社区

活动

资源

关于

腾讯云开发者

热门产品

热门推荐

更多推荐

问hcatalog在hadoop中的用途是什么？
EN