blocks|key|2224265|text|type|unstyled|depth|inlineStyleRanges|entityRanges|data|2224266|Goto+https://docs.aws.amazon.com/glue/latest/dg/add-job-python.html#create-python-extra-library.检查部分以创建Python+.egg或.whl文件，以获得“如何为python作业创建安装文件”setup.py文件中的|ordered-list-item|offset|length|style|BOLD|2224267|，添加行|2224268|2224269|2224270|安装程序(name=“”，version="0.1"，packages=‘，install_requires=’熊猫=0.25.1‘)|blockquote|2224271|2224272|2224273|我还编写了小的shell脚本来部署python作业，而无需手动步骤来创建鸡蛋文件并上传到s3，并通过cloudformation进行部署。脚本都是自动完成的。您可以在https://github.com/fatangare/aws-python-shell-deploy找到代码|2224274|entityMap|0|LINK|mutability|MUTABLE|url|https://docs.aws.amazon.com/glue/latest/dg/add-job-python.html#create-python-extra-library|1|https://github.com/fatangare/aws-python-shell-deploy^0|0|2Q|N|5|2I|0|0|0|0|0|0|0|0|2B|1G|1|0^^$0|@$1|2|3|-4|4|5|6|12|7|@]|8|@]|9|$]]|$1|A|3|B|4|C|6|13|7|@$D|14|E|15|F|G]]|8|@$D|16|E|17|1|18]]|9|$]]|$1|H|3|I|4|C|6|19|7|@]|8|@]|9|$]]|$1|J|3|-4|4|5|6|1A|7|@]|8|@]|9|$]]|$1|K|3|-4|4|5|6|1B|7|@]|8|@]|9|$]]|$1|L|3|M|4|N|6|1C|7|@]|8|@]|9|$]]|$1|O|3|-4|4|5|6|1D|7|@]|8|@]|9|$]]|$1|P|3|-4|4|5|6|1E|7|@]|8|@]|9|$]]|$1|Q|3|R|4|5|6|1F|7|@]|8|@$D|1G|E|1H|1|1I]]|9|$]]|$1|S|3|-4|4|5|6|1J|7|@]|8|@]|9|$]]]|T|$U|$4|V|W|X|9|$Y|Z]]|10|$4|V|W|X|9|$Y|11]]]]

<ol>
<li>Goto <a href="https://docs.aws.amazon.com/glue/latest/dg/add-job-python.html#create-python-extra-library" rel="nofollow noreferrer">https://docs.aws.amazon.com/glue/latest/dg/add-job-python.html#create-python-extra-library</a>. Check section
To create a Python .egg or .whl file for 'how to create setup file for python shell job'</li>
<li>In setup.py file, add line <code>install_requires=['pandas==0.25.1']</code>:</li>
</ol>

<blockquote>
<pre><code>setup(name="&lt;module name&gt;",
 version="0.1",
 packages=['&lt;package name if any or ignore&gt;'],
 install_requires=['pandas==0.25.1']
 )
</code></pre>
</blockquote>

I also wrote small shell script to deploy python shell job without manual steps to create egg file and upload to s3 and deploy via cloudformation. Script does all automatically.
You may find code at <a href="https://github.com/fatangare/aws-python-shell-deploy" rel="nofollow noreferrer">https://github.com/fatangare/aws-python-shell-deploy</a>

blocks|key|2224295|text|为了澄清桑迪普的回答，下面是对我有用的|type|unstyled|depth|inlineStyleRanges|entityRanges|data|2224296|1/忽略AWS文档|2224297|2/创建一个包含以下内容的setup.py文件：|2224298|from+setuptools+import+setup

setup(name="pandasmodule",
++++++++version="0.1",
++++++++packages=[],
++++++++install_requires=['pandas==0.25.1']
++++)|code-block|syntax|javascript|2224299|3/在包含文件的文件夹中运行此命令：|2224300|python+setup.py+bdist_wheel|2224301|4/将.whl文件上载到s3|2224302|5/将Glue+ETL作业中的"Python路径“配置为s3路径|2224303|您现在可以在Glue+ETL作业中使用"import+pandas+as+pd“|offset|length|style|CODE|2224304|entityMap^0|0|0|0|0|0|0|0|0|K|J|0^^$0|@$1|2|3|4|5|6|7|10|8|@]|9|@]|A|$]]|$1|B|3|C|5|6|7|11|8|@]|9|@]|A|$]]|$1|D|3|E|5|6|7|12|8|@]|9|@]|A|$]]|$1|F|3|G|5|H|7|13|8|@]|9|@]|A|$I|J]]|$1|K|3|L|5|6|7|14|8|@]|9|@]|A|$]]|$1|M|3|N|5|H|7|15|8|@]|9|@]|A|$I|J]]|$1|O|3|P|5|6|7|16|8|@]|9|@]|A|$]]|$1|Q|3|R|5|6|7|17|8|@]|9|@]|A|$]]|$1|S|3|T|5|6|7|18|8|@$U|19|V|1A|W|X]]|9|@]|A|$]]|$1|Y|3|-4|5|6|7|1B|8|@]|9|@]|A|$]]]|Z|$]]

Just to clarify Sandeep's answer, here is what worked for me

1/ Ignore AWS doc

2/ Create a setup.py file containing :

<pre><code>from setuptools import setup

setup(name="pandasmodule",
 version="0.1",
 packages=[],
 install_requires=['pandas==0.25.1']
 )
</code></pre>

3/ Run this command in the folder containing the file :

<pre><code>python setup.py bdist_wheel
</code></pre>

4/ Upload the .whl file to s3

5/ Configure the "Python lib path" in your Glue ETL Job to the s3 path

You can now use "<code>import pandas as pd</code>" in your Glue ETL Job

blocks|key|2201209|text|使用Glue+Python，以下脚本直接用于熊猫：|type|unstyled|depth|inlineStyleRanges|entityRanges|data|2201210|from+setuptools+import+setup

setup(name="pandasmodule",
++++++++version="0.1",
++++++++packages=[],
++++++++install_requires=['pandas==0.25.1']
++++)

#+use+pandas
import+numpy+as+np
import+pandas+as+pd

s+=+pd.Series([1,+3,+5,+np.nan,+6,+8])

print(s)|code-block|syntax|javascript|2201211|entityMap^0|0|0^^$0|@$1|2|3|4|5|6|7|I|8|@]|9|@]|A|$]]|$1|B|3|C|5|D|7|J|8|@]|9|@]|A|$E|F]]|$1|G|3|-4|5|6|7|K|8|@]|9|@]|A|$]]]|H|$]]

Using the Glue Python Shell, the following script works directly for pandas:
<pre><code>from setuptools import setup

setup(name=&quot;pandasmodule&quot;,
 version=&quot;0.1&quot;,
 packages=[],
 install_requires=['pandas==0.25.1']
 )

# use pandas
import numpy as np
import pandas as pd

s = pd.Series([1, 3, 5, np.nan, 6, 8])

print(s)
</code></pre>

blocks|key|2224315|text|不需要做任何事情，只要进口熊猫并开始使用它。|type|unstyled|depth|inlineStyleRanges|entityRanges|data|2224316|entityMap^0|0^^$0|@$1|2|3|4|5|6|7|D|8|@]|9|@]|A|$]]|$1|B|3|-4|5|6|7|E|8|@]|9|@]|A|$]]]|C|$]]

No need to do anything, just import pandas and start using it.

blocks|key|1269052|text|AWS+Glue+2.0支持熊猫-1.0.1+https://docs.aws.amazon.com/glue/latest/dg/reduced-start-times-spark-etl-jobs.html|type|unstyled|depth|inlineStyleRanges|entityRanges|offset|length|data|1269053|因此，在您的脚本中，您可以简单地编写:导入熊猫。如果您想使用Glue中没有提供的其他python模块，可以在.whl或.zip+->store中下载它在S3+->中的位置路径，在胶水作业中在"Python库路径“中使用，而在作业运行期间使用胶水将完成pip安装”you模块“。|1269054|entityMap|0|LINK|mutability|MUTABLE|url|https://docs.aws.amazon.com/glue/latest/dg/reduced-start-times-spark-etl-jobs.html^0|N|2A|0|0|0^^$0|@$1|2|3|4|5|6|7|N|8|@]|9|@$A|O|B|P|1|Q]]|C|$]]|$1|D|3|E|5|6|7|R|8|@]|9|@]|C|$]]|$1|F|3|-4|5|6|7|S|8|@]|9|@]|C|$]]]|G|$H|$5|I|J|K|C|$L|M]]]]

AWS Glue 2.0 supports pandas—1.0.1
<a href="https://docs.aws.amazon.com/glue/latest/dg/reduced-start-times-spark-etl-jobs.html" rel="nofollow noreferrer">https://docs.aws.amazon.com/glue/latest/dg/reduced-start-times-spark-etl-jobs.html</a>
so in your script you can simply write : import pandas.
If you want to use other python module that is not provided in Glue, you can download .whl or .zip -&gt;store it in S3 -&gt; place path of it in glue job in &quot;Python library path&quot; and glue during a job run will do a pip install &quot;yourmodule&quot;

The AWS Documentation 
<a href="https://docs.aws.amazon.com/glue/latest/dg/add-job-python.html" rel="nofollow noreferrer">https://docs.aws.amazon.com/glue/latest/dg/add-job-python.html</a>

mentions that 

<blockquote>
 The environment for running a Python shell job supports the following
 libraries: 
 
 ...
 
 pandas (required to be installed via the python setuptools
 configuration, setup.py)
</blockquote>

But it does not mention how to make the install.

How can I use Pandas in a AWS Glue Python Shell Jobs ?

Using Pandas AWS Glue Python Shell Jobs

5位重磅专家聚焦 AI 时代开发者核心成长需求，带你深入解锁全栈进化、人机协作关键路径！点击报名到场：<a href="https://cloud.tencent.com/developer/salon/salon-2403/signup" target="_blank">https://cloud.tencent.com/developer/salon/salon-2403/signup</a> 
 <img src="https://qcloudimg.tencent-cloud.cn/raw/487c82639d2338df3ef66dfb1450601a.png"/>

腾讯技术创作特训营「年度论坛」正在报名中！

翻译质量差，导致语言生硬或混乱。

没有提供实际的解决方法或示例。

解答不清晰，无法理解或解决问题。

页面排版不美观，阅读体验差。

文章

问答

视频

学习中心

腾讯云实验室

直播

竞赛

腾讯云代码分析专区

腾讯iOA零信任安全管理系统专区

腾讯云架构师技术同盟交流圈

腾讯云数据库专区

腾讯云智能顾问专区

腾讯云原生专区

腾讯混元专区

腾讯云TCE专区

腾讯云Lighthouse专区

腾讯云HAI专区

腾讯云Edgeone专区

腾讯云存储专区

腾讯云智能专区

腾讯轻联专区 

腾讯云开发专区

TAPD专区

腾讯轻量云游戏服专区

腾讯云最具价值专家

腾讯云架构师技术同盟

腾讯云创作之星

腾讯云开发者先锋

腾讯云代码助手

云原生构建

TAPD 敏捷项目管理

Cloud Studio

SDK中心

API中心

命令行工具

涵盖代码开发、场景应用、自动测试全流程，助你从零构建专属AI助手

一站式MCP教程库，解锁AI应用新玩法

聚焦“写作效率、视觉美观与运行性能”三方面进行全面升级，为您提供更高效、稳定的创作环境

社区富文本&Markdown编辑器全新改版上线，欢迎大家体验!

诚挚邀请您参与本次调研，分享您的真实使用感受与建议。您的反馈至关重要，感谢您的支持与参与！

社区新版编辑器体验调研

AWS文档提到运行Python作业的环境支持以下库：..。熊猫(需要通过python配置( setup.py)安装)但它没有提到如何进行安装。如何在AWS Glue Python Shell Jobs中使用Pandas？

问使用Pandas AWS Glue Python Shell Jobs
EN

回答 5

Stack Overflow用户

Stack Overflow用户

Stack Overflow用户

社区

活动

圈层

关于

腾讯云开发者

热门产品

热门推荐

更多推荐

问使用Pandas AWS Glue Python Shell JobsEN

回答 5

Stack Overflow用户

Stack Overflow用户

Stack Overflow用户

社区

活动

圈层

关于

腾讯云开发者

热门产品

热门推荐

更多推荐

问使用Pandas AWS Glue Python Shell Jobs
EN