blocks|key|2548204|text|当我把叉子更新到最新的主人时，我也遇到了类似的问题。不幸的是，我们找不到办法让它从创意中发挥作用。我所做的就是从命令行编译这个项目。它生成了所需的antlr类。然后，我添加了生成源目标/生成源/antlr作为源目录。在那之后，我可以用Idea进行测试。理想情况下，Idea生成源应该生成代码。需要更多的检查为什么它没有。可能是因为我配置了May+3.3.3。|type|unstyled|depth|inlineStyleRanges|entityRanges|data|2548205|entityMap^0|0^^$0|@$1|2|3|4|5|6|7|D|8|@]|9|@]|A|$]]|$1|B|3|-4|5|6|7|E|8|@]|9|@]|A|$]]]|C|$]]

I also faced similar problem when I updated my fork to latest master. Unfortunately, could not find a way to make it work from IDEA. What I did is compiled the project from command line. It generated the antlr classes which is required. I then added the generated-source target/generated-source/antlr as source directory. After that I could run tests from Idea. Ideally Idea generate source should have generated the code. Need to check more why it did not. May be because I have maven3.3.3 configured.

blocks|key|521565|text|1)首先使用http://spark.apache.org/docs/latest/building-spark.html#building-with-buildmvn中给出的构建指令，从命令行构建星火|type|unstyled|depth|inlineStyleRanges|entityRanges|offset|length|data|521566|2)检查$SPARK_HOME/sql/catalyst/target/generated-sources/antlr3/org/apache/spark/sql/catalyst/parser文件夹。|521567|一些生成的类(如SparkSqlLexer.java+)就在那里。|521568|它生成的类的列表是|521569|++++SparkSqlLexer.java[enter+link+description+here][1]++++
++++SparkSqlParser.java
++++SparkSqlParser_ExpressionParser.java
++++SparkSqlParser_FromClauseParser.java
++++SparkSqlParser_IdentifiersParser.java
++++SparkSqlParser_KeywordParser.java
++++SparkSqlParser_SelectClauseParser.java|code-block|syntax|javascript|521570|3)开放模块设置。点击火花催化剂模块。转到右边的Source选项卡。将目标/生成的源作为源文件夹。|521571|📷|atomic|521572|entityMap|0|LINK|mutability|MUTABLE|url|http://spark.apache.org/docs/latest/building-spark.html#building-with-buildmvn|1|IMAGE|IMMUTABLE|imageUrl|https://developer.qcloudimg.com/http-save/yehe-900000/917581c09dd5abcd67d9e737ab825532.png|imageAlt^0|6|26|0|0|0|0|0|0|0|0|1|1|0^^$0|@$1|2|3|4|5|6|7|17|8|@]|9|@$A|18|B|19|1|1A]]|C|$]]|$1|D|3|E|5|6|7|1B|8|@]|9|@]|C|$]]|$1|F|3|G|5|6|7|1C|8|@]|9|@]|C|$]]|$1|H|3|I|5|6|7|1D|8|@]|9|@]|C|$]]|$1|J|3|K|5|L|7|1E|8|@]|9|@]|C|$M|N]]|$1|O|3|P|5|6|7|1F|8|@]|9|@]|C|$]]|$1|Q|3|R|5|S|7|1G|8|@]|9|@$A|1H|B|1I|1|1J]]|C|$]]|$1|T|3|-4|5|6|7|1K|8|@]|9|@]|C|$]]]|U|$V|$5|W|X|Y|C|$Z|10]]|11|$5|12|X|13|C|$14|15|16|-4]]]]

1) First build your Spark from command line using build instructions given in <a href="http://spark.apache.org/docs/latest/building-spark.html#building-with-buildmvn" rel="nofollow noreferrer">http://spark.apache.org/docs/latest/building-spark.html#building-with-buildmvn</a>

2) Then check 
$SPARK_HOME/sql/catalyst/target/generated-sources/antlr3/org/apache/spark/sql/catalyst/parser folder.

Some of the generated classes like SparkSqlLexer.java is there. 

List of classes it generates are 

<pre><code> SparkSqlLexer.java[enter link description here][1] 
 SparkSqlParser.java
 SparkSqlParser_ExpressionParser.java
 SparkSqlParser_FromClauseParser.java
 SparkSqlParser_IdentifiersParser.java
 SparkSqlParser_KeywordParser.java
 SparkSqlParser_SelectClauseParser.java
</code></pre>

3) Open Module Settings. Click on spark-catalyst module. Go to Source tab in the right. Make target/generated-source as a source folder. <a href="https://i.stack.imgur.com/i3Pui.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/i3Pui.png" alt="Attaching a pic to give an idea."></a>

blocks|key|521574|text|我做了从Rishitesh+Mishra进入和被困在第一步。在执行"build/mvn+-Pyarn+-Phadoop-2.4+-Dhadoop.version=2.4.0+-DskipTests干净包“时，我总是有错误。|type|unstyled|depth|inlineStyleRanges|entityRanges|data|521575|我已经尝试过来自https://spark.apache.org的源代码和github上的叉。|offset|length|521576|我已经附上了日志截图在一个新的答案如下。|521577|误差日志图像|521578|entityMap|0|LINK|mutability|MUTABLE|url|https://spark.apache.org/|1|http://i.stack.imgur.com/Qmfg3.jpg^0|0|8|O|0|0|0|0|6|1|0^^$0|@$1|2|3|4|5|6|7|T|8|@]|9|@]|A|$]]|$1|B|3|C|5|6|7|U|8|@]|9|@$D|V|E|W|1|X]]|A|$]]|$1|F|3|G|5|6|7|Y|8|@]|9|@]|A|$]]|$1|H|3|I|5|6|7|Z|8|@]|9|@$D|10|E|11|1|12]]|A|$]]|$1|J|3|-4|5|6|7|13|8|@]|9|@]|A|$]]]|K|$L|$5|M|N|O|A|$P|Q]]|R|$5|M|N|O|A|$P|S]]]]

I have did as the intruction from Rishitesh Mishra and get stuck in the first step. I have always errors when executing "build/mvn -Pyarn -Phadoop-2.4 -Dhadoop.version=2.4.0 -DskipTests clean package". 
 
I have both tried on source code from <a href="https://spark.apache.org" rel="nofollow">https://spark.apache.org</a> and fork on github. 
I have attached the log screenshot in a new answer as following.
 
<a href="http://i.stack.imgur.com/Qmfg3.jpg" rel="nofollow">error log image</a>

blocks|key|521620|text|我也遇到了同样的问题。这是我的解决方案：|type|unstyled|depth|inlineStyleRanges|entityRanges|data|521621|下载antlr4+(即antlr+v4)插件IntelliJ。然后您可以看到"spark-2.0.1\sql\catalyst\src\main\antlr4\org\apache\spark\sql\catalyst\parser\SqlBase.g4“文件可以被IntelliJ+IDEA识别。|ordered-list-item|521622|导航到View->Tool+Windows->Maven+Projects选项卡。选择项目“星火项目催化剂”。右击它。然后选择“生成源和更新文件夹”。|521623|之后，您可以看到添加到"spark-2.0.1\sql\catalyst\target\generated-sources\antlr4“中的一些文件|521624|然后你就可以建立成功的项目。|521625|希望它能帮到你。|521626|entityMap^0|0|0|0|0|0|0^^$0|@$1|2|3|4|5|6|7|O|8|@]|9|@]|A|$]]|$1|B|3|C|5|D|7|P|8|@]|9|@]|A|$]]|$1|E|3|F|5|D|7|Q|8|@]|9|@]|A|$]]|$1|G|3|H|5|D|7|R|8|@]|9|@]|A|$]]|$1|I|3|J|5|D|7|S|8|@]|9|@]|A|$]]|$1|K|3|L|5|6|7|T|8|@]|9|@]|A|$]]|$1|M|3|-4|5|6|7|U|8|@]|9|@]|A|$]]]|N|$]]

I meet the same problem. Here is my solution:

<ol>
<li>Download the antlr4 (i.e. antlr v4) plugin of IntelliJ. Then you can see the file "spark-2.0.1\sql\catalyst\src\main\antlr4\org\apache\spark\sql\catalyst\parser\SqlBase.g4" can be recognized by IntelliJ IDEA.</li>
<li>Navigate to View->Tool Windows->Maven Projects tab. select the project "Spark Project Catalyst". Right click on it. Then select "Generate sources and update folders"</li>
<li>After that you can see some files added into the "spark-2.0.1\sql\catalyst\target\generated-sources\antlr4"</li>
<li>Then you can build success of the project.</li>
</ol>

Hope it can help you.

blocks|key|521658|text|这里的建议对我都没有用。但是，我注意到生成的代码依赖于Antlr+3.x，而Antlr+4.x是依赖项中的内容(mvn依赖:tree)。我不知道为什么会这样。也许是因为我之前从命令行(?)构建了它。|type|unstyled|depth|inlineStyleRanges|entityRanges|data|521659|无论如何，尝试清理您的催化剂子项目，然后重建自动生成的资源。要在IntelliJ中做到这一点，请转到查看->工具窗口->+Maven项目。|521660|然后导航到"Maven+Project“选项卡中的”火花项目催化剂“。|521661|导航到清洁->干净:清洁和双击它。导航到插件->+antlr4+->+antlr4+4:antlr4+4并双击它。|521662|现在，您将看到Antlr类的自动生成源是不同的，它们应该编译。YMMV|521663|entityMap^0|0|0|0|0|0^^$0|@$1|2|3|4|5|6|7|L|8|@]|9|@]|A|$]]|$1|B|3|C|5|6|7|M|8|@]|9|@]|A|$]]|$1|D|3|E|5|6|7|N|8|@]|9|@]|A|$]]|$1|F|3|G|5|6|7|O|8|@]|9|@]|A|$]]|$1|H|3|I|5|6|7|P|8|@]|9|@]|A|$]]|$1|J|3|-4|5|6|7|Q|8|@]|9|@]|A|$]]]|K|$]]

None of the advice here worked for me. I noticed, however, that the generated code depends on Antlr 3.x while Antlr 4.x is what is in the dependencies (mvn dependecy:tree). I don't know why this was the case. Maybe because I had earlier built it from the command line (?).

Anyway, try cleaning your Catalyst sub-project then rebuild the autogenerated sources. To do this in IntelliJ, go to View -> Tools Window -> Maven Projects. 

Then navigate to the "Spark Project Catalyst" in the "Maven Project" tab. 

Navigate to clean -> clean:clean and double click it. Navigate to Plugins -> antlr4 -> antlr4:antlr4 and double click it. 

Now, you'll see the autogenerated sources of the Antlr classes are different and they should compile. YMMV.

I have installed spark-hadoop env in my Red Hat 64. And I also want to read and write code in spark source code project in intelliJ idea. I have downloaded spark source code and make everything ready. But I had some errors when compiling spark project in IntelliJ idea. 
Here are errors:

<blockquote>
 /home/xuch/IdeaProjects/spark/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/CatalystQI.scala
 
 Error:(809, 34) not found: value SparkSqlParser
 case ast if ast.tokenType == SparlSqlParser.TinyintLiteral =>
 
 Error:(812, 34) not found: value SparkSqlParser
 case ast if ast.tokenType == SparlSqlParser.SmallintLiteral =>
 
 ... ...
</blockquote>

But actually I did not find a file named SparkSqlParser.scala in the whole project neither a scala class named SparkSqlParser.

However, I had searched the web for some files named SparkSqlParser.scala, but they don't have attribute like "TinyintLiteral", "SmallintLiteral", etc.
Here are the files link:

<ul>
<li><a href="https://github.com/yjshen/zzzzobspk/blob/master/sql/core/src/main/scala/org/apache/spark/sql/SparkSQLParser.scala" rel="noreferrer">https://github.com/yjshen/zzzzobspk/blob/master/sql/core/src/main/scala/org/apache/spark/sql/SparkSQLParser.scala</a></li>
<li><a href="https://apache.googlesource.com/spark/+/c152dde78f73d5ce3a483fd60a47e7de1f1916da/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/SparkSQLParser.scala" rel="noreferrer">https://apache.googlesource.com/spark/+/c152dde78f73d5ce3a483fd60a47e7de1f1916da/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/SparkSQLParser.scala</a></li>
</ul>

Spark SQL has no SparkSqlParser.scala file when compiling in intelliJ idea

翻译质量差，导致语言生硬或混乱。

没有提供实际的解决方法或示例。

解答不清晰，无法理解或解决问题。

页面排版不美观，阅读体验差。

文章

问答

视频

学习中心

腾讯云实验室

直播

竞赛

腾讯云代码分析专区

腾讯iOA零信任安全管理系统专区

腾讯云架构师技术同盟交流圈

腾讯云数据库专区

腾讯云顾问专区

腾讯云原生专区

腾讯混元专区

腾讯云TCE专区

腾讯云Lighthouse专区

腾讯云HAI专区

腾讯云Edgeone专区

腾讯云存储专区

腾讯云智能专区

腾讯轻联专区 

腾讯云开发专区

TAPD专区

腾讯轻量云游戏服专区

腾讯云最具价值专家

腾讯云架构师技术同盟

腾讯云创作之星

腾讯云开发者先锋

腾讯云代码助手

云原生构建

TAPD 敏捷项目管理

Cloud Studio

SDK中心

API中心

命令行工具

涵盖代码开发、场景应用、自动测试全流程，助你从零构建专属AI助手

一站式MCP教程库，解锁AI应用新玩法

我在我的红帽64中安装了火花-hadoop env。我还想在spark源代码项目中读写intelliJ idea中的代码。我已经下载了星火源代码，并使一切就绪。但我在IntelliJ idea中编译spark项目时出现了一些错误。以下是一些错误：/home/xuch/IdeaProjects/spark/sql/cat...

问Spark在SparkSqlParser.scala idea中编译时没有intelliJ文件
EN

Stack Overflow用户

社区

活动

圈层

关于

腾讯云开发者

热门产品

热门推荐

更多推荐

问Spark在SparkSqlParser.scala idea中编译时没有intelliJ文件EN