blocks|key|459465|text|根据我使用Flink+Cassandra连接器编写数据Cassandra的经验，我认为最好的方法是使用Flink内置连接器。因为Flink+1.4.3，您可以使用HBase+Flink连接器。请参阅这里|type|unstyled|depth|inlineStyleRanges|offset|length|style|CODE|entityRanges|data|459466|entityMap|0|LINK|mutability|MUTABLE|url|https://github.com/apache/flink/tree/master/flink-connectors^0|1Y|5|2R|2|0|0^^$0|@$1|2|3|4|5|6|7|N|8|@$9|O|A|P|B|C]]|D|@$9|Q|A|R|1|S]]|E|$]]|$1|F|3|-4|5|6|7|T|8|@]|D|@]|E|$]]]|G|$H|$5|I|J|K|E|$L|M]]]]

<p>According to my experiences in writing data Cassandra using Flink Cassandra connector, I think the best way is to use Flink built-in connectors. Since Flink <code>1.4.3</code> you can use HBase Flink connector. See <a href="https://github.com/apache/flink/tree/master/flink-connectors" rel="nofollow noreferrer">here</a></p>


blocks|key|459478|text|我使用java连接到Flink中的HBase。只需在open中创建open连接对象，并在RichFunction的close方法(即RichSinkFunction)中关闭它。这些方法由每个flink时隙调用一次。|type|unstyled|depth|inlineStyleRanges|offset|length|style|CODE|entityRanges|data|459479|我认为在Scala中也可以这样做。|459480|entityMap^0|Q|4|X|4|1L|5|0|0^^$0|@$1|2|3|4|5|6|7|J|8|@$9|K|A|L|B|C]|$9|M|A|N|B|C]|$9|O|A|P|B|C]]|D|@]|E|$]]|$1|F|3|G|5|6|7|Q|8|@]|D|@]|E|$]]|$1|H|3|-4|5|6|7|R|8|@]|D|@]|E|$]]]|I|$]]

<p>I connect to HBase in Flink using java. Just create HBase Connection object in <code>open</code> and close it within <code>close</code> methods of RichFunction (i.e. RichSinkFunction). These methods are called once by each flink slot. </p>

<p>I think you can do something like this in Scala too.</p>


blocks|key|459565|text|取决于你所说的“推荐”是什么意思，我想。|type|unstyled|depth|inlineStyleRanges|entityRanges|data|459566|DIY|459567|鳗鱼|459568|如果您只想从Scala应用程序中访问HBase上的数据，那么您可能想看看鳗鱼，它包括与大数据景观中的许多存储格式和系统交互的库，并且是用Scala编写的。|offset|length|459569|您很可能对使用eel-hbase模块感兴趣，该模块在几个版本中包括一个HBaseSource类(以及一个HBaseSink)。实际上，最近我注意到README仍然提到不支持HBase。在Hive中没有明确的示例，但是源和接收器以类似的方式工作。|style|CODE|459570|风筝|459571|另一种选择可能是风筝，它也有您可以从其中获得灵感的一组相当广泛的例子+(包括HBase)，但是它看起来不像Eel那么活跃。|459572|大数据框架|459573|如果您想要一个框架来帮助您，而不是用库来酝酿您自己的解决方案。当然，你必须考虑一些学习曲线。|459574|火花|459575|Spark是一个相当成熟的项目，HBase项目本身是为Spark2.1.1(这里的Scaladocs)构建的连接器。这里是一个介绍性的演讲，可以帮助你。|459576|一般的想法是，您可以按照这个例子中的建议使用这个自定义数据源。|459577|sqlContext
++.read
++.options(Map(HBaseTableCatalog.tableCatalog->cat,+HBaseRelation.HBASE_CONFIGFILE+->+conf))
++.format("org.apache.spark.sql.execution.datasources.hbase")
++.load()|code-block|syntax|javascript|459578|允许您通过Spark访问HBase数据。下面是一个简短的摘录来自同一个例子|459579|val+df1+=+withCatalog(cat1,+conf1)
val+df2+=+withCatalog(cat2,+conf2)
val+s1+=+df1.filter($"col0"+<=+"row120"+&&+$"col0"+>+"row090").select("col0",+"col2")
val+s2+=+df2.filter($"col0"+<=+"row150"+&&+$"col0"+>+"row100").select("col0",+"col5")
val+result+=++s1.join(s2,+Seq("col0"))|459580|撇开性能方面的考虑，正如您可能看到的那样，这种语言对于数据操作来说是相当自然的。|459581|弗林克|459582|有两个答案已经涉及到Flink，所以我不会添加更多内容，除了在编写本报告(1.4.2)时来自最新稳定版本的指向示例的链接，您可能会对此感兴趣。|459583|entityMap|0|LINK|mutability|MUTABLE|url|https://github.com/51zero/eel-sdk|1|http://kitesdk.org/docs/1.1.0/|2|https://github.com/kite-sdk/kite-examples|3|http://hbase.apache.org/hbase-spark/scaladocs/#package|4|https://databricks.com/session/apache-spark-apache-hbase-connector-feature-rich-and-efficient-access-to-hbase-through-spark-sql|5|https://github.com/hortonworks-spark/shc/blob/master/examples/src/main/scala/org/apache/spark/sql/execution/datasources/hbase/JoinTablesFrom2Clusters.scala#L101-L107|6|https://github.com/hortonworks-spark/shc/blob/master/examples/src/main/scala/org/apache/spark/sql/execution/datasources/hbase/JoinTablesFrom2Clusters.scala#L129-L133|7|https://github.com/apache/flink/blob/release-1.4.2/flink-connectors/flink-hbase/src/test/java/org/apache/flink/addons/hbase/example/HBaseReadExample.java^0|0|0|0|10|2|0|0|7|9|Z|B|1G|9|21|6|0|0|8|2|1|P|9|2|0|0|0|0|15|9|3|1M|2|4|0|C|4|5|0|0|U|7|6|0|0|0|0|1H|7|7|0^^$0|@$1|2|3|4|5|6|7|22|8|@]|9|@]|A|$]]|$1|B|3|C|5|6|7|23|8|@]|9|@]|A|$]]|$1|D|3|E|5|6|7|24|8|@]|9|@]|A|$]]|$1|F|3|G|5|6|7|25|8|@]|9|@$H|26|I|27|1|28]]|A|$]]|$1|J|3|K|5|6|7|29|8|@$H|2A|I|2B|L|M]|$H|2C|I|2D|L|M]|$H|2E|I|2F|L|M]|$H|2G|I|2H|L|M]]|9|@]|A|$]]|$1|N|3|O|5|6|7|2I|8|@]|9|@]|A|$]]|$1|P|3|Q|5|6|7|2J|8|@]|9|@$H|2K|I|2L|1|2M]|$H|2N|I|2O|1|2P]]|A|$]]|$1|R|3|S|5|6|7|2Q|8|@]|9|@]|A|$]]|$1|T|3|U|5|6|7|2R|8|@]|9|@]|A|$]]|$1|V|3|W|5|6|7|2S|8|@]|9|@]|A|$]]|$1|X|3|Y|5|6|7|2T|8|@]|9|@$H|2U|I|2V|1|2W]|$H|2X|I|2Y|1|2Z]]|A|$]]|$1|Z|3|10|5|6|7|30|8|@]|9|@$H|31|I|32|1|33]]|A|$]]|$1|11|3|12|5|13|7|34|8|@]|9|@]|A|$14|15]]|$1|16|3|17|5|6|7|35|8|@]|9|@$H|36|I|37|1|38]]|A|$]]|$1|18|3|19|5|13|7|39|8|@]|9|@]|A|$14|15]]|$1|1A|3|1B|5|6|7|3A|8|@]|9|@]|A|$]]|$1|1C|3|1D|5|6|7|3B|8|@]|9|@]|A|$]]|$1|1E|3|1F|5|6|7|3C|8|@]|9|@$H|3D|I|3E|1|3F]]|A|$]]|$1|1G|3|-4|5|6|7|3G|8|@]|9|@]|A|$]]]|1H|$1I|$5|1J|1K|1L|A|$1M|1N]]|1O|$5|1J|1K|1L|A|$1M|1P]]|1Q|$5|1J|1K|1L|A|$1M|1R]]|1S|$5|1J|1K|1L|A|$1M|1T]]|1U|$5|1J|1K|1L|A|$1M|1V]]|1W|$5|1J|1K|1L|A|$1M|1X]]|1Y|$5|1J|1K|1L|A|$1M|1Z]]|20|$5|1J|1K|1L|A|$1M|21]]]]

<p>Depends on what do you mean by "recommended", I guess.</p>

<h2>DIY</h2>

<h3>Eel</h3>

<p>If you just want to access data on HBase from a Scala application, you may want to have a look at <a href="https://github.com/51zero/eel-sdk" rel="nofollow noreferrer">Eel</a>, which includes libraries to interact with many storage formats and systems in the <em>Big Data</em> landscape and is natively written in Scala.</p>

<p>You'll most likely be interested in using the <code>eel-hbase</code> module, which from a few releases includes an <code>HBaseSource</code> class (as well as an <code>HBaseSink</code>). It's actually so recent I just noticed the <code>README</code> still mentions that HBase is not supported. There are no explicit examples with Hive, but source and sinks work in similar ways.</p>

<h3>Kite</h3>

<p>Another alternative could be <a href="http://kitesdk.org/docs/1.1.0/" rel="nofollow noreferrer">Kite</a>, which also has <a href="https://github.com/kite-sdk/kite-examples" rel="nofollow noreferrer">a quite extensive set of examples</a> you can draw inspiration from (including with HBase), but it looks less active of a project than Eel.</p>

<hr>

<h2><em>Big Data</em> frameworks</h2>

<p>If you want a framework that helps you instead of brewing your own solution with libraries. Of course you'll have to account for some learning curve.</p>

<h3>Spark</h3>

<p>Spark is a fairly mature project and the HBase project itself as built a connector for Spark 2.1.1 (<a href="http://hbase.apache.org/hbase-spark/scaladocs/#package" rel="nofollow noreferrer">Scaladocs</a> here). <a href="https://databricks.com/session/apache-spark-apache-hbase-connector-feature-rich-and-efficient-access-to-hbase-through-spark-sql" rel="nofollow noreferrer">Here</a> is an introductory talk that can come to your help.</p>

<p>The general idea is that you could use this custom data source as suggested in <a href="https://github.com/hortonworks-spark/shc/blob/master/examples/src/main/scala/org/apache/spark/sql/execution/datasources/hbase/JoinTablesFrom2Clusters.scala#L101-L107" rel="nofollow noreferrer">this example</a>:</p>

<pre><code>sqlContext
  .read
  .options(Map(HBaseTableCatalog.tableCatalog-&gt;cat, HBaseRelation.HBASE_CONFIGFILE -&gt; conf))
  .format("org.apache.spark.sql.execution.datasources.hbase")
  .load()
</code></pre>

<p>Giving you access to HBase data through the Spark SQL API. Here is a short extract <a href="https://github.com/hortonworks-spark/shc/blob/master/examples/src/main/scala/org/apache/spark/sql/execution/datasources/hbase/JoinTablesFrom2Clusters.scala#L129-L133" rel="nofollow noreferrer">from the same example</a>:</p>

<pre><code>val df1 = withCatalog(cat1, conf1)
val df2 = withCatalog(cat2, conf2)
val s1 = df1.filter($"col0" &lt;= "row120" &amp;&amp; $"col0" &gt; "row090").select("col0", "col2")
val s2 = df2.filter($"col0" &lt;= "row150" &amp;&amp; $"col0" &gt; "row100").select("col0", "col5")
val result =  s1.join(s2, Seq("col0"))
</code></pre>

<p>Performance considerations aside, as you may see the language can feel pretty natural for data manipulation.</p>

<h3>Flink</h3>

<p>Two answers already dealt with Flink, so I won't add much more, except for <a href="https://github.com/apache/flink/blob/release-1.4.2/flink-connectors/flink-hbase/src/test/java/org/apache/flink/addons/hbase/example/HBaseReadExample.java" rel="nofollow noreferrer">a link to an example</a> from the latest stable release at the time of writing (1.4.2) that you may be interested in having a look at.</p>


<p>Now that <a href="https://github.com/ParallelAI/SpyGlass" rel="nofollow noreferrer">SpyGlass</a> is no longer being maintained, what is the recommended way to access HBase using Scala/Scalding? A <a href="https://stackoverflow.com/questions/16045543/alternatives-to-scalding-for-hbase-access-from-scala-or-java">similar question</a> was asked in 2013, but most of the suggested links are either dead or to defunct projects. The only link that seems useful is to <a href="https://flink.apache.org/" rel="nofollow noreferrer">Apache Flink</a>. Is that considered the best option nowadays? Are people still recommending SpyGlass for new projects even though it isn't been maintained? Performance (massively parallel) and testability are priorities.</p>


Recommended way to access HBase using Scala

翻译质量差，导致语言生硬或混乱。

没有提供实际的解决方法或示例。

解答不清晰，无法理解或解决问题。

页面排版不美观，阅读体验差。

文章

问答

视频

学习中心

腾讯云实验室

直播

竞赛

腾讯云代码分析专区

腾讯iOA零信任安全管理系统专区

腾讯云架构师技术同盟交流圈

腾讯云数据库专区

腾讯云顾问专区

腾讯云原生专区

腾讯混元专区

腾讯云TCE专区

腾讯云Lighthouse专区

腾讯云HAI专区

腾讯云Edgeone专区

腾讯云存储专区

腾讯云智能专区

腾讯轻联专区 

腾讯云开发专区

TAPD专区

腾讯轻量云游戏服专区

腾讯云最具价值专家

腾讯云架构师技术同盟

腾讯云创作之星

腾讯云开发者先锋

腾讯云代码助手

云原生构建

TAPD 敏捷项目管理

Cloud Studio

SDK中心

API中心

命令行工具

涵盖代码开发、场景应用、自动测试全流程，助你从零构建专属AI助手

一站式MCP教程库，解锁AI应用新玩法

既然不再维护，那么使用Scala/烫伤访问HBase的推荐方法是什么呢？2013年，被问到了，但大多数建议的链接要么已经失效，要么已经失效。唯一似乎有用的链接是到。这被认为是当今最好的选择吗？人们是否仍然推荐SpyGlass作为新项目，即使它没有被维护？性能(大规模并行)和可测试性是优先考虑的问题。

问使用Scala访问HBase的推荐方法
EN

Stack Overflow用户

社区

活动

圈层

关于

腾讯云开发者

热门产品

热门推荐

更多推荐

问使用Scala访问HBase的推荐方法EN