如何在运行PySpark代码时禁用以下警告消息:
Setting default log level to "WARN".
To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use setLogLevel(newLevel).
18/06/08 21:04:55 WARN SparkConf: In Spark 1.0 and later spark.local.dir will be overridden by the value set by the cluster manager (via SPARK_LOCAL_DIRS in mesos/standalone and LOCAL_DIRS in YARN).
18/06/08 21:04:55 WARN Utils: Service 'SparkUI' could not bind on port 4040. Attempting port 4041.
我花了一些时间处理log4.properties,但不能准确地找出是哪个类记录了这些属性。
https://stackoverflow.com/questions/50768405
复制相似问题