无法在本地计算机上运行数据验证器2.11-0.12.1.jar
无法在本地计算机上运行数据验证器2.11-0.12.1.jar
提问于 2021-12-28 10:02:32
我试图使用目标数据验证器，但是当我在cmd中运行它时，它会显示一个错误，如下所示：
D:\Spark\DemoTime>spark-submit --master local data-validator_2.11-0.12.1.jar --help
21/12/28 15:05:41 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Exception in thread "main" org.apache.spark.SparkException: No main class set in JAR; please specify one with --class.
at org.apache.spark.deploy.SparkSubmit.error(SparkSubmit.scala:972)
at org.apache.spark.deploy.SparkSubmit.prepareSubmitEnvironment(SparkSubmit.scala:492)
at org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:898)
at org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:180)
at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:203)
at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:90)
at org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:1043)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:1052)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)但是如果我给出主类名，它显示如下：
D:\Spark\DemoTime>spark-submit --class com.target.data_validator.Main data-validator_2.11-0.13.0.jar --help
Exception in thread "main" java.lang.NoClassDefFoundError: com/typesafe/scalalogging/Logger
at java.lang.Class.getDeclaredMethods0(Native Method)
at java.lang.Class.privateGetDeclaredMethods(Class.java:2701)
at java.lang.Class.privateGetMethodRecursive(Class.java:3048)
at java.lang.Class.getMethod0(Class.java:3018)
at java.lang.Class.getMethod(Class.java:1784)
at org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:42)
at org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:955)
at org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:180)
at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:203)
at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:90)
at org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:1043)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:1052)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
Caused by: java.lang.ClassNotFoundException: com.typesafe.scalalogging.Logger
at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
... 13 more
log4j:WARN No appenders could be found for logger (org.apache.spark.util.ShutdownHookManager).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.我正在跟踪来自target的数据验证器工具：https://github.com/target/data-validator
但我没办法解决这件事。
您可以在：https://github.com/target/data-validator/releases找到jar文件。
因此，在运行此工具时，我将请求您的帮助。如果你还需要什么，请告诉我。
谢谢。
回答 1
Stack Overflow用户
发布于 2021-12-28 10:08:20
来自GitHub的自述：
组装脂肪罐：
sbt clean assembly
spark-submit --master local data-validator-assembly-0.13.0.jar --help
您需要大量的数据验证器+它的依赖项才能运行它。您只提供了数据验证器简单jar。
页面原文内容由Stack Overflow提供。腾讯云小微IT领域专用引擎提供翻译支持
原文链接：
https://stackoverflow.com/questions/70505508
复制相关文章
相似问题
腾讯云开发者
