carla 学习笔记
carla 学习笔记
1. What is CARLA?
Carla是一个用于自主驾驶研究的开源模拟器。与Torcs的关系，都属于模拟器，但是Torcs并非为自动驾驶研究而生，在仿真程度上，Carla可以说远胜Torcs，而且使用也更为方便。
CARLA （Car Learning to Act）has been developed from the ground up to support development, training, and validation of autonomous urban driving systems.
TORCS (The Open Racing Car Simulator) is an open source 3D car racing simulatoravailable for multiple platforms，originally developed by Eric Espié and Christophe Guionneau in 1997.
2. Carla的设计架构
CARLA is composed by the following modules
- Client side
- Python client API: "PythonClient/carla"
- Server side
- CarlaUE4 Unreal Engine project: "Unreal/CarlaUE4"
- Carla plugin for Unreal Engine: "Unreal/CarlaUE4/Plugins/Carla"
- CarlaServer: "Util/CarlaServer"
Python client API
The client API provides a Python module for communicating with the CARLA server. In the folder "PythonClient", we provide several examples for scripting a CARLA client using the "carla" module.
CarlaServer
External library for the networking communications.
CarlaUE4 Unreal Engine project
The Unreal project "CarlaUE4" contains all the assets and scenes for generating the CARLA binary. It uses the tools provided by the Carla plugin to assemble the cities and behavior of the agents in the scene.
3. System requirements
Carla是基于大名鼎鼎的游戏引擎Unreal Engine开发的，对系统要求比较高（反正我的MacBook Pro上装Ubuntu虚拟机是跑不起来的...）
CARLA is a very performance demanding software, at the very minimum you would need a computer with a dedicated GPU capable of running Unreal Engine. See Unreal Engine's recommended hardware.
Building CARLA from source requires about 15GB of disk space, not counting Unreal Engine installation. However, you will also need to build and install Unreal Engine, which on Linux requires much more disk space as it keeps all the intermediate files. (我build的时候用了大概46G，而且好多内容需要访问外国网站下载。)
4. 预编译版本：0.7.1
下载地址：Carla 0.7 baidu pan 链接: https://pan.baidu.com/s/1eSuBh5K 密码: dgqz
Credits to @xfqbuaa from Github, 解压就可以使用，如果想要从GitHub源码编译，会比较复杂，并且需要全局访问外国网站。折腾了一天多才编译成功，具体细节有兴趣的请私信。
run Carla as server:
./CarlaUE4.sh /Game/Maps/Town01 -carla-server -benchmark -fps=15
CARLA specific command-line options
-carla-serverLaunches CARLA as server, the execution hangs until a client connects.-carla-settings="Path/To/CarlaSettings.ini"Load settings from the given INI file. See Example.CarlaSettings.ini.-carla-world-port=NListen for client connections at port N, agent ports are set to N+1 and N+2 respectively. Activates server.-carla-no-hudDo not display the HUD by default.-carla-no-networkingDisable networking. Overrides-carla-serverif present.
另外Carla也可以在无图形界面环境下运行，但仍然会使用默认GPU进行渲染。
run Python client:
cd PythonClient/
./client_example.py or ./manual_control.py
需要注意的是，如果自己从源码编译Carla，没有预编译版本中的启动脚步：
There is no "CarlaUE4.sh" script in the source version of CARLA
编译好之后 open the project in the Unreal Editor, you can hit Play to test CARLA.
5. 游戏参数（Measurements）
Carla server会把每一帧画面及其对应的各种measurements都一并返回给客户端。这些Measurements包括：
Key Type Description
transform Transform World transform of the player.
acceleration Vector3D Current acceleration of the player.
forward_speed float Forward speed in km/h.
collision_vehicles float Collision intensity with other vehicles.
collision_pedestrians float Collision intensity with pedestrians.
collision_other float General collision intensity (everything --else but pedestrians and vehicles).
intersection_otherlane float Percentage of the car invading other ----lanes.
intersection_offroad float Percentage of the car off-road.
autopilot_control Control Vehicle's autopilot control that would ---apply this frame.
其中Transform是由player的location和rotation组成的, Vector3D + Rotation3D
Collision variables keep an accumulation of all the collisions occurred during this episode. Every collision contributes proportionally to the intensity of the collision (norm of the normal impulse between the two colliding objects). Three different counts are kept (pedestrians, vehicles, and other). Colliding objects are classified based on their tag (same as for semantic segmentation).
在Carla中，steer, throttle, brake/hand brake, reverse都是控制动作，属于autopilot ccontrol，需要client端发送给server：
steer float Steering angle between [-1.0, 1.0] (*)
throttle float Throttle input between [ 0.0, 1.0]
brake float Brake input between [ 0.0, 1.0]
hand_brake bool Whether the hand-brake is engaged
reverse bool Whether the vehicle is in reverse gear
measurements, sensor_data = carla_client.read_data()control = measurements.player_measurements.autopilot_control# modify here control if wanted.carla_client.send_control(control)
另外在Carla设置中可以打开非玩家的信息，server会将其返回给client。
Non-player agents info
- Vehicle Contains its transform, box-extent, and forward speed.
- Pedestrian Contains its transform, box-extent, and forward speed. (*)
- Traffic light Contains its transform and state (green, yellow, red).
- Speed-limit sign Contains its transform and speed-limit.
Carla中所有物体都有bounding box，可准确定位
6. 摄像头与传感器
Cameras and sensors can be added to the player vehicle by defining them in the settings file sent by the client on every new episode.
CarlaSettings.ini
; Use it with `./CarlaUE4.sh -carla-settings=Path/To/This/File`.
[CARLA/Server]
; If set to false, a mock controller will be used instead of waiting for a real
; client to connect.
UseNetworking=true
; Ports to use for the server-client communication. This can be overridden by
; the command-line switch `-world-port=N`, write and read ports will be set to
; N+1 and N+2 respectively.
WorldPort=2000
; Time-out in milliseconds for the networking operations.
ServerTimeOut=10000
; In synchronous mode, CARLA waits every frame until the control from the client
; is received.
SynchronousMode=true
; Send info about every non-player agent in the scene every frame, the
; information is attached to the measurements message. This includes other
; vehicles, pedestrians and traffic signs. Disabled by default to improve
; performance.
SendNonPlayerAgentsInfo=false
[CARLA/LevelSettings]
; Path of the vehicle class to be used for the player. Leave empty for default.
; Paths follow the pattern "/Game/Blueprints/Vehicles/Mustang/Mustang.Mustang_C"
PlayerVehicle=
; Number of non-player vehicles to be spawned into the level.
NumberOfVehicles=15
; Number of non-player pedestrians to be spawned into the level.
NumberOfPedestrians=30
; Index of the weather/lighting presets to use. If negative, the default presets
; of the map will be used.
WeatherId=1
; Seeds for the pseudo-random number generators.
SeedVehicles=123456789
SeedPedestrians=123456789
[CARLA/SceneCapture]
; Names of the cameras to be attached to the player, comma-separated, each of
; them should be defined in its own subsection. E.g., Uncomment next line to add
; a camera called MyCamera to the vehicle
; Cameras=MyCamera
; Now, every camera we added needs to be defined it in its own subsection.
[CARLA/SceneCapture/MyCamera]
; Post-processing effect to be applied. Valid values:
; * None No effects applied.
; * SceneFinal Post-processing present at scene (bloom, fog, etc).
; * Depth Depth map ground-truth only.
; * SemanticSegmentation Semantic segmentation ground-truth only.
PostProcessing=SceneFinal
; Size of the captured image in pixels.
ImageSizeX=800
ImageSizeY=600
; Camera (horizontal) field of view in degrees.
CameraFOV=90
; Position of the camera relative to the car in centimeters.
CameraPositionX=15
CameraPositionY=0
CameraPositionZ=123
; Rotation of the camera relative to the car in degrees.
CameraRotationPitch=8
CameraRotationRoll=0
CameraRotationYaw=0
; Stereo setup example:
;
; [CARLA/SceneCapture]
; Cameras=CameraStereoLeft/RGB,CameraStereoLeft/Depth,CameraStereoRight/RGB,CameraStereoRight/Depth
; ImageSizeX=720
; ImageSizeY=512
; CameraFOV=90
; [CARLA/SceneCapture/CameraStereoLeft]
; CameraPositionX=170
; CameraPositionY=-30
; CameraPositionZ=150
; CameraRotationPitch=0
; CameraRotationRoll=0
; CameraRotationYaw=0
; [CARLA/SceneCapture/CameraStereoLeft/RGB]
; PostProcessing=SceneFinal
; [CARLA/SceneCapture/CameraStereoLeft/Depth]
; PostProcessing=Depth
; [CARLA/SceneCapture/CameraStereoRight]
; CameraPositionX=170
; CameraPositionY=30
; CameraPositionZ=150
; CameraRotationPitch=0
; CameraRotationRoll=0
; CameraRotationYaw=0
; [CARLA/SceneCapture/CameraStereoRight/RGB]
; PostProcessing=SceneFinal
; [CARLA/SceneCapture/CameraStereoRight/Depth]
; PostProcessing=Depth
sensor目前只有主要的三种, implemented as different post-processing effects applied to scene capture cameras:Scene final, Depth map, Semantic segmentation
7. Running benchmarks
The "carla" api provides a basic benchmarking system, that allows making several tests on a certain agent. We already provide the same benchmark used in the CoRL 2017 paper. By running this benchmark you can compare the results of your agent to the results obtained by the agents show in the paper.
To run benchmark:
corl = CoRL2017(city_name=args.city_name, name_to_save=args.log_name)
agent = Manual(args.city_name)
results = corl.benchmark_agent(agent, client)
如果想测试自己实现的agent，继承Manual的父类，实现对应run_step方法即可。
def run_step(self, measurements, sensor_data, target):
""" Function to run a control step in the CARLA vehicle. :param measurements: object of the Measurements type :param sensor_data: images list object :param target: target position of Transform type :return: an object of the control type. """
control = VehicleControl()
control.throttle = 0.9
return control
如果需要自定义agent路线，可以在carla/benchmarks/CoRL2017.py中找到对应设置方法:
def _poses_town01(self):
"""
Each matrix is a new task. We have all the four tasks
"""
def _poses_straight():
return [[36, 40], [39, 35], [110, 114], [7, 3], [0, 4],
[68, 50], [61, 59], [47, 64], [147, 90], [33, 87],
[26, 19], [80, 76], [45, 49], [55, 44], [29, 107],
[95, 104], [84, 34], [53, 67], [22, 17], [91, 148],
[20, 107], [78, 70], [95, 102], [68, 44], [45, 69]]
def _poses_one_curve():
return [[138, 17], [47, 16], [26, 9], [42, 49], [140, 124],
[85, 98], [65, 133], [137, 51], [76, 66], [46, 39],
[40, 60], [0, 29], [4, 129], [121, 140], [2, 129],
[78, 44], [68, 85], [41, 102], [95, 70], [68, 129],
[84, 69], [47, 79], [110, 15], [130, 17], [0, 17]]
def _poses_navigation():
return [[105, 29], [27, 130], [102, 87], [132, 27], [24, 44],
[96, 26], [34, 67], [28, 1], [140, 134], [105, 9],
[148, 129], [65, 18], [21, 16], [147, 97], [42, 51],
[30, 41], [18, 107], [69, 45], [102, 95], [18, 145],
[111, 64], [79, 45], [84, 69], [73, 31], [37, 81]]
return [_poses_straight(),
_poses_one_curve(),
_poses_navigation(),
_poses_navigation()]
后续踩坑后再补充更新。