我试图通过将gcloud命令转换为API来使用Dataproc,但是我在文档中找不到一个很好的例子。
%pip install google-cloud-dataproc
我发现的唯一一个好例子就是这个,它工作得很好:
from google.cloud import dataproc_v1
client = dataproc_v1.ClusterControllerClient()
project_id = 'test-project'
region = 'global'
for element in client.list_clusters(proje
我对这份文件感到很困惑
Service account requirements and Limitations:
* Service accounts can only be set when a cluster is created.
* You need to create a service account before creating the Cloud Dataproc cluster that will be associated with the service account.
* Once set, the service account used for a clust
您好,我应该如何修改我的代码,以正确读取dataset2?
%%writefile read_rdd.py
def read_RDD(argv):
parser = argparse.ArgumentParser() # get a parser object
parser.add_argument('--test_set', metavar='test_set', type =ParallelMapDataset)
args = parser.parse_args(argv) # read the value
args.test_set.