本指导适用于在 TencentOS Server 3 上使用 LMDeploy 推理框架运行 LLaMA 模型的官方 Demo,以 Docker 方式启动。
前置环境条件
运行模型
模型环境准备
此时应该在
/opt/lmdeploy
路径下,创建 Demo 文件夹来存放模型代码。mkdir -p demo/LLaMAcd demo/LLaMA
创建模型代码
在 demo/LLaMA 文件夹下创建 LLaMA.py 文件,并写入以下代码,这里我们测试的是 llama_7b 模型:
from transformers import AutoTokenizer, AutoModelfrom lmdeploy import pipelineimport time# LMDeploy testing# load modelpipe = pipeline('openlm-research/open_llama_7b')# inferencestart = time.perf_counter()response1 = pipe(['The president of the United States is'])response2 = pipe(['Shanghai is'])response3 = pipe(['The future of AI is'])end = time.perf_counter() - start# print output and timeprint(f"{response1[0].text}\\n{response2[0].text}\\n{response3[0].text}")print(f"Processing time: {end:.4f}")
运行模型:
python3 LLaMA.py
运行该 Python 文件,则会自动开始下载 LLaMA_7b 模型并开始推理,输出结果如下(参见):
not the commander in chief of the armed forces of the United States. The president is the commander in chief of the armed forces of the United States in a very limited sense.He is the commander in chief of the armed forces of the United States, but he is not the commander in chief of the armed forces of the United States in the sense that he has any right to tell them what to do.The commander in chief of the armed forces of the United States is the Congress of the United States. The commander in chief of the armed forces of the United States is the Congress of the United States.When the Congress of the United States says that it is going to go to war, the commander in chief of the armed forces of the United States is the Congress of the United States.When the Congress of the United States says that it is going to go to war, the commander in chief of the armed forces of the United States is the Congress of the United States. The commander in chief of the armed forces of the United States is the Congress of the United States.When the Congress of the United States says that it is going to go to war, the commander in chief of the armed forces of the United States is the Congress of the United States. When the Congress of the United States says that it is going to go to war, the commander in chief of the armed forces of the United States is the Congress of the United States.When the Congress of the United States says that it is going to go to war, the commander in chief of the armed forces of the United States is the Congress of the United States. When the Congress of the United States says that it is going to go to war, the commander in chief of the armed forces of the United States is the Congress of the United States. When the Congress of the United States says that it is going to go to war, the commander in chief of the armed forces of the United States is the Congress of the United States.When the Congress of the United States says that it is going to go to war, the commander in chief of the armed forces of the United States is the Congress of the United States. When the Congress of the United States says that it is going to go to war, the commander in chief of the armed forces of the United States is the Congress of the United States. When the Congress of the United States says that it is going to go to war, the commander in chief of the armed forces of the United States is the Congressone of the most famous cities in China. It is also one of the largest cities in the world. The city is located on the Huangpu River and is surrounded by the Yangtze River Delta. The city is the largest city in China and is the second largest city in the world.Shanghai is a very popular tourist destination. The city has a lot of attractions, including the Bund, the Shanghai Museum, the Yu Garden, the Shanghai Tower, and the Shanghai Zoo. The city is also home to the Shanghai World Expo.The city is also home to the Shanghai International Film Festival. The city is also home to the Shanghai International Film Festival. The city is also home to the Shanghai International Film Festival. The city is also home to the Shanghai International Film Festival. The city is also home to the Shanghai International Film Festival. The city is also home to the Shanghai International Film Festival. The city is also home to the Shanghai International Film Festival. The city is also home to the Shanghai International Film Festival. The city is also home to the Shanghai International Film Festival. The city is also home to the Shanghai International Film Festival. The city is also home to the Shanghai International Film Festival. The city is also home to the Shanghai International Film Festival. The city is also home to the Shanghai International Film Festival. The city is also home to the Shanghai International Film Festival. The city is also home to the Shanghai International Film Festival. The city is also home to the Shanghai International Film Festival. The city is also home to the Shanghai International Film Festival. The city is also home to the Shanghai International Film Festival. The city is also home to the Shanghai International Film Festival. The city is also home to the Shanghai International Film Festival. The city is also home to the Shanghai International Film Festival. The city is also home to the Shanghai International Film Festival. The city is also home to the Shanghai International Film Festival. The city is also home to the Shanghai International Film Festival. The city is also home to the Shanghai International Film Festival. The city is also home to the Shanghai International Film Festival. The city is also home to the Shanghai International Film Festival. The city is also home to the Shanghai International Film Festival. The city is also home to the Shanghai International Film Festival. The city is also home to the Shanghai International Film Festival. The city is also home to the Shanghai International Film Festival. The city is also home to the Shanghai International Film Festival. The city is also home to the Shanghai International Film Festival. The city is also home toin the hands of the peopleThe AI revolution is here, but it is also not here. The promise of a world where humans and machines work together to solve our biggest problems is exciting and has the potential to transform every industry. However, the reality is that AI is still in its infancy and the risks associated with its misuse are real.As we enter a new era of AI, it is important that we consider the ethical implications of this technology and ensure that it is used for the benefit of all. This requires a multidisciplinary approach that includes the perspectives of experts from a variety of fields, including ethics, law, and technology.In this article, we will explore the ethical implications of AI and discuss how we can ensure that this technology is used for the good of all.AI’s impact on societyThe impact of AI on society is likely to be profound. It has the potential to transform how we live and work, but it also has the potential to cause harm. As we enter a new era of AI, it is important that we consider the ethical implications of this technology and ensure that it is used for the benefit of all.AI and the future of workAI is already having a significant impact on the future of work. As machines become increasingly capable of performing tasks that were once thought to be the domain of humans, it is likely that many jobs will be replaced by AI. This could lead to a decrease in employment and an increase in inequality.AI and the future of healthcareAI has the potential to revolutionize healthcare. It can be used to diagnose diseases, provide treatment recommendations, and even perform surgeries. However, it is important to consider the ethical implications of this technology.AI and the future of educationAI has the potential to transform education. It can be used to create personalized learning experiences for students, and it can also be used to assess student performance and provide feedback. However, it is important to consider the ethical implications of this technology.AI and the future of governmentAI has the potential to transform how governments operate. It can be used to make decisions about policy, allocate resources, and even provide personalized services to citizens. However, it is important to consider the ethical implications of this technology.AI and the future of securityAI has the potential to transform how we secure our homes, businesses, and countries. It can be used to detect and prevent crime, and it can also be used to identify potential threats. However, it is important to consider the ethicalProcessing time: 30.2124
可以看到输出结果和运行的时间,说明运行成功。
注意事项
说明:
由于 OpenCloudOS 是 TencentOS Server 的开源版本,理论上上述文档当中的所有操作同样适用于 OpenCloudOS。
参考文档