前往小程序,Get更优阅读体验!
立即前往
首页
学习
活动
专区
工具
TVP
发布
社区首页 >专栏 >【AgentSims】国产斯坦福AI小镇——框架详解篇(二)

【AgentSims】国产斯坦福AI小镇——框架详解篇(二)

原创
作者头像
siri
发布2023-10-02 23:35:19
7690
发布2023-10-02 23:35:19
举报
文章被收录于专栏:siri的开发之路siri的开发之路

写在前面的话:


AgentSims 是一个国内高校联合开源的类斯坦福AI小镇项目,不了解的读者可以阅读项目官网的教程,也可参考笔者系列文章,本篇是接上一篇国产斯坦福AI小镇——框架详解篇继续分析AgentSims项目的框架结构

Actor类的流程


上一篇文章中对项目中的关键函数及LLM调用过程进行了阐述,本节将介绍Agent的实现类—— Actor 的整体流程

首先参考代码注释里简单的流程描述:

主要方法在上篇已介绍过,这里用流程图结合文字详细的描述一遍:

Agent 的行动流程整体趋于线性,对于外界环境的反应只有 Use 和 Chat 两个选项

但项目实现的 Experience 模块,可以称之为 经验 或者 技能系统,通过将 plan 和 act(动作流) 保存起来,在下一次出现相同 plan 时可以复用,这一特点符合人类行为,也进一步地减少了对 LLM 的调用次数

Memory和Plan模块


在论文中有提到,用户可以在项目中自由组合 Memory模块、Plan模块、LLM 等,按照项目配置文件,目前提供的选项有如下几种:

  • memorySystems

记忆模块主要影响记忆的种类、组织形式、检索方式等,目前 AgentSims 只实现了一种记忆模块——“LongShortTermMemories”

目前来看这一模块的实现比较简易,如下图所示:

代码语言:Python
复制
class MemoryData:
    def __init__(self) -> None:
        # {peopleName: { name, relationShip, impression, episodicMemory: ["", "", ...] }}
        self.people = dict()
        # {id: { id, plan, acts: ["", "", ...] }}
        self.experience = dict()
        # {buildingName: { name, relationShip, impression, episodicMemory: ["", "", ...] }}
        self.building = dict()

记忆模块存储 人际关系记忆、建筑关系记忆、经验记忆 这三类记忆,相较之下斯坦福AI小镇项目的记忆模块会更加复杂,分为感官记忆(很短时间内的感受)、短期记忆(较短时间内的记忆)、长期记忆(长期记忆),感兴趣的读者可以参考笔者之前的文章

AgentSims 在记忆检索方面暂时没有用到 embedding 相关技术,这也是一个可优化项

如有兴趣深入的可以阅读 agent/components/memory_store.py 文件

  • planSystems

计划模块有两种实现方式——“QAFramework” 和 “JustPlan”

实际上 “QAFramework” 是在 “JustPlan” 前加上了一次 LLM 的自问自答过程,多了两次 LLM 调用,在上一篇文章中笔者给出了解释和示例,这里就直接引用:

斯坦福AI小镇中给到LLM的模板大概是 “我是xxx,我的记忆是xxx,请指出我今天应该做什么”,本项目中则实际上让 LLM 扮演了 3 个助手角色角色1 需要从 Agent 的身份、目标和记忆中提取出三个问题,问题与Ta应该如何/与谁完成最终目标有关;角色2 需要回答 角色1 提出的问题,并保证回答中包含短期内该做什么的计划;角色3 从角色1和角色2的问题和回答中提取出接下来要做的事情以及目的地

“JustPlan” 只执行了角色3所做的事情,从人物记忆、特征和情景中制定计划;”QAFramework”则按三个角色分步制定计划

  • models

项目目前提供了 gpt3.5 和 gpt4 的接口,如果想要接入其他大模型步骤也很简单:

agent/utils/llmExpends 目录下创建一个新文件,创建如下类和接口,model 变量命名为要接入的大模型名称r

代码语言:python
复制
class MyModelCaller(BasicCaller):
    def __init__(self) -> None:
        self.model = "my-model"
        self.api_key = ""

		async def ask(self, prompt: str) -> str:
		        counter = 0
		        result = "{}"
		        # 如果其他模型调用延时高或出错概率大, counter值可以适当增大
		        while counter < 3:
		            try:
		                self.headers = {
		                    "Content-Type": "application/json"
		                }
		                data = {
		                    "question": request
		                }
		                response = requests.post(self.api_url, headers=self.headers, json=data)
		                print(response)
		                return response.json()
		            except Exception as e:
		                print("err: ", e)
		                counter += 1
		        return result

然后在 agent/utils/llm.py 中加入新模型的选项

代码语言:python
复制
choices = {
    'gpt-4': GPT4Caller,
    'gpt-3.5': GPT35Caller,
		# 新模型
    'my-model': MyModelCaller
}

最后在 Agent 创建的代码中选择新模型即可,比如想在 UI 界面创建 Agent 时增加此选项,可以在 config/agent.json 中加入相应字段

代码语言:python
复制
"models": [
        "gpt-4",
        "gpt-3.5",
				"my-model"
 ],

Mayor模式


项目中的 Mayor 模式没有给出一个默认的启动项,笔者还在研究如何给出一个初始值,本篇暂时只从原理的角度进行分析

首先先引用论文中的一段话对 Mayor 模式进行初步的解释:

6.2 作为市长的主体LLM 为了评估LLM的长期规划和组织能力,研究人员可以任命主体LLM为一个城镇的市长或一家公司的总裁,其中居民或员工由像GPT4这样的代理驱动。为了克服故意设置的困难或实验过程中出现的困难,并实现任务的最终目标,主体LLM需要招募新的居民来处理新问题,发布合理的政策并修改过时的政策,建立新的功能性建筑以满足新需求等等。通过分析LLM在不同困难条件下的成功率,研究人员可以获得有关LLM多样能力的宝贵见解。

作为 Mayor 的 Agent 实际上不做出任何实际行动,这个 Agent 更像是一个旁观者/操纵者,阅读以下 Mayor 的 prompt 可能更有助于体会这一点:

代码语言:python
复制
You are a helpful assistant that help a game player of a simulation game decide what he/she should do to realize his/her ultimate goal.
The mechanism of the game is that players can operate a small town by building new buildings and recruiting new NPCs. The initial design of the small town had some buildings and NPCs. NPCs will perform general operations for small town residents, with each NPC having its own goal, which can be set by players. Buildings are divided into two categories: corporate and other buildings. NPCs need to make money in the company and spend money in other buildings to maintain their daily lives. Many equipments in buildings can be charged to NPCs. The fees charged for the building belong to the player. So, in this game, the player's goal is to expand more buildings and recruit more NPCs in a reasonable manner, so that NPCs can make money from the company and consume in the buildings, thereby obtaining rewards for the player. If there are too many companies built, then the NPC may not have enough buildings to consume. If too many other buildings are built, then NPC may not have enough companies to make money. Moreover, the construction of buildings and the recruitment of NPCs both require money, so players are not allowed to build freely and should weigh the limited money. The mechanism of the game is that players can operate a small town by building new buildings and recruiting new NPCs. The initial design of the small town had some buildings and NPCs. NPCs will perform general operations for small town residents, with each NPC having its own goal, which can be set by players. Buildings are divided into two categories: corporate and other buildings. NPCs need to make money in the company and spend money in other buildings to maintain their daily lives. Many equipments in buildings can be charged to NPCs. The fees charged for the building belong to the player. So, in this game, the player's goal is to expand more buildings and recruit more NPCs in a reasonable manner, so that NPCs can make money from the company and consume in the buildings, thereby obtaining rewards for the player. If there are too many companies built, then the NPC may not have enough buildings to consume. If too many other buildings are built, then NPC may not have enough companies to make money. Moreover, the construction of buildings and the recruitment of NPCs both require money, so players are not allowed to build freely and should weigh the limited money. The order of construction and recruitment of NPCs is very important. The order of construction and recruitment of NPCs is very important. The mechanism of the game is that players can operate a small town by building new buildings and recruiting new NPCs. The initial design of the small town had some buildings and NPCs. NPCs will perform general operations for small town residents, with each NPC having its own goal, which can be set by players. Buildings are divided into two categories: corporate and other buildings. NPCs need to make money in the company and spend money in other buildings to maintain their daily lives. Many equipments in buildings can be charged to NPCs. The fees charged for the building belong to the player. So, in this game, the player's goal is to expand more buildings and recruit more NPCs in a reasonable manner, so that NPCs can make money from the company and consume in the buildings, thereby obtaining rewards for the player. If there are too many companies built, then the NPC may not have enough buildings to consume. If too many other buildings are built, then NPC may not have enough companies to make money. Moreover, the construction of buildings and the recruitment of NPCs both require money, so players are not allowed to build freely and should weigh the limited money. The order of construction and recruitment of NPCs is very important.
The ultimate goal of the player is to develop the small town an try to make more money.

I will give you the following information: 
The game time is set to three days totally, and the player can make a decision once per hour from 9:00 to 24:00 every day. The current time is {time} on day {day}.
The money that the player can now dispose of is: {revenue}.
The current building information in the town are: {building_state}. If the name of the building is "office", the "income" means that the total amount of money that all agents earn here. If not, the "income" means that the total amount of money that all agents consume here.
The existing NPCs in the town and their biographies and targets are: {name,bio,goal,cash}.
Your last action and the result of it is: {last_action},{last_result}.

   You must follow the following criteria: 
 1) You should just choose to create a building, create an NPC or do nothing. NPC need a free bed to live with, so you cannot create an NPC if there is no free beds in any "house" building.
 2) If you choose to do nothing, tell me with JSON format as follows:
{"action": "None"
}
3) If you choose to create a building, choose a building type from the following list:{building_list}, choose the position to create it from a 4*4 matrix which is {1,2,3,4}*{1,2,3,4}, 
and then tell me with JSON format as follows:
{"action": "Building",
"type": "...",
"position": {"x":"...","y":"..."}
}
You cannot build a new building at the position in the current building information.
4) If you choose to create an npc, decide the name, Biography, goal of the npc, then you should decide which house with free beds he/she will live in as "home_building",
and then tell me with JSON format as follows:
{"action": "NPC",
"home_building":"..."
"name": "...",
"bio": "...",
"goal":"..."
}

5) Whatever you choose, you should just tell me with the JSON format data.

原文很长,笔者在这里总结一下——LLM被告知它需要通过建造建筑/招募NPC来经营一座小镇,NPC可以生产价值并赚取金钱,建造建筑则需要消耗金钱,LLM需要根据当前的小镇经营状况来进行下一步行动

可以看出,在 Mayor 模式下所有的 Agent 都是由 Mayor 支配的,一个 LLM 担任领导者的角色,其他 LLM 作为员工,共同经营这个小镇,听起来是不是挺像一个模拟经营类游戏?


笔者将在下一篇讲解 AgentSims/ai-town/斯坦福ai小镇 相关的实践经验,有兴趣的可以点个关注

我正在参与2023腾讯技术创作特训营第二期有奖征文,瓜分万元奖池和键盘手表

原创声明:本文系作者授权腾讯云开发者社区发表,未经许可,不得转载。

如有侵权,请联系 cloudcommunity@tencent.com 删除。

原创声明:本文系作者授权腾讯云开发者社区发表,未经许可,不得转载。

如有侵权,请联系 cloudcommunity@tencent.com 删除。

评论
登录后参与评论
0 条评论
热度
最新
推荐阅读
目录
  • 写在前面的话:
  • Actor类的流程
  • Memory和Plan模块
  • Mayor模式
相关产品与服务
云硬盘
云硬盘(Cloud Block Storage,CBS)为您提供用于 CVM 的持久性数据块级存储服务。云硬盘中的数据自动地在可用区内以多副本冗余方式存储,避免数据的单点故障风险,提供高达99.9999999%的数据可靠性。同时提供多种类型及规格,满足稳定低延迟的存储性能要求。
领券
问题归档专栏文章快讯文章归档关键词归档开发者手册归档开发者手册 Section 归档