Android 框架提供了各种用 2D 和 3D 图形渲染的 API 与制造商的图形驱动程序实现方法交互,在Android平台上应用开发者可通过三种方式将图像绘制到屏幕上:Canvas、OpenGLES、Vulkan 无论使用什么方式进行内容的生产,这个离用户最近的图形系统都扮演者一个非常重要的角色,在此系统一系列关键组件的协同帮助下,最终按照我们的预期将画面展示给用户。
无论是从系统工程师角度或者应用开发者角度来看,这个离用户最近的系统都是非常值得我们去一探其工作原理的,通过了解Android图形系统的工作流程,可以帮助我们在实际生产设计应用或者分析图形性能问题时做出高效明智的选择。
正所谓窥一斑而知全豹,由于整个图形子系统过于庞大,直接啃代码不知从何下手,所以我希望通过观察Demo中一个具体图像帧的整个生命周期可以一窥整个Android系统的图形系统的工作流程以及工作模式。
Demo
在跟踪代码流程之前先看一个简单的动图直观的了解一下各个关键器件之间如何进行协同工作有个基本的概念:
显示器在显示动态画面时,每一帧图像的显示实际上是由上到下逐行扫描的,当扫描完最后一行时,需要将扫描点挪到左上角继续下一次扫描,而这个重置扫描点的动作称为vblank,在vblank之前将会产生一个信号称为vsync,对于Android系统而言,此信号将会驱动图形生产的逻辑代码在CPU上运行,而工作过程中CPU偏向于收集应用的绘制意图,收集完成后将指令一通刷到GPU中,GPU则是将这些指令再次展开执行,将一帧新的图像渲染到后缓冲区中,这个过程相对CPU是异步的.
而作为首篇文章,要跟踪的流程自然是应用如何发出首帧绘制请求的.毕竟有了vsync,我们应用的图形生产代码才会工作起来.
根据跟踪代码绘制的时序图:
在Android系统中,当用户通过桌面点击应用的图标后,桌面程序是通过系统API启动一个我们事先在应用清单文件里注册的Activity,所以对于用户来说,看到的第一个关键组件就是Activity,所以一切的一切,我打算从这个响应界面的Activity的生命周期作为起点进行跟踪.
函数位于:Activity.java
final void attach(...) {
...
mWindow = new PhoneWindow(this, window, activityConfigCallback);
mWindow.setWindowControllerCallback(this);
mWindow.setCallback(this);
mWindow.setOnWindowDismissedCallback(this);
mWindow.getLayoutInflater().setPrivateFactory(this);
mWindow.setWindowManager(
(WindowManager)context.getSystemService(Context.WINDOW_SERVICE),
mToken, mComponent.flattenToString(),
(info.flags & ActivityInfo.FLAG_HARDWARE_ACCELERATED) != 0);
...
}
补充一些个人理解:
该函数的作用就是指定内容区域参与图像生成的View,实际上是通过PhoneWindow提供了逻辑支持.具体的这里不展开,网上有很多大佬文章讲的非常细.
函数位于:ActivityThread.java
@Override
public void handleResumeActivity(...) {
...
if (a.mVisibleFromClient) {
if (!a.mWindowAdded) {
a.mWindowAdded = true;
wm.addView(decor, l);
} else {
...
}
}
...
关键信息:
该函数位于:WindowManagerImpl.java
public void addView(View view, ViewGroup.LayoutParams params,
Display display, Window parentWindow) {
...
root = new ViewRootImpl(view.getContext(), display);
view.setLayoutParams(wparams);
mViews.add(view);
mRoots.add(root);
mParams.add(wparams);
// do this last because it fires off messages to start doing things
try {
root.setView(view, wparams, panelParentView);
} catch (RuntimeException e) {
// BadTokenException or InvalidDisplayException, clean up.
if (index >= 0) {
removeViewLocked(index, true);
}
throw e;
}
}
}
关键信息:
关于ViewRootImpl,我们先看一下该类的构造函数中也有一些关键信息:
public ViewRootImpl(Context context, Display display) {
mContext = context;
mWindowSession = WindowManagerGlobal.getWindowSession();
...
mChoreographer = Choreographer.getInstance();
关键信息:
另外,Surface以及SurfaceControl也是在声明的地方进行了初始化,但仅仅只是java对象的初始化,真正的nativeObj并未在这里执行构造,所以这里直接略过.至于这两个对象的具体作用这里先不需要了解,后续进行剖析.
此函数代码较长,以下只将我认为与图像生成相关的关键代码进行了保留:
public void setView(View view, WindowManager.LayoutParams attrs, View panelParentView) {
...
enableHardwareAcceleration(attrs);
...
int res; /* = WindowManagerImpl.ADD_OKAY; */
// Schedule the first layout -before- adding to the window
// manager, to make sure we do the relayout before receiving
// any other events from the system.
requestLayout();
...
try {
mOrigWindowType = mWindowAttributes.type;
mAttachInfo.mRecomputeGlobalAttributes = true;
collectViewAttributes();
res = mWindowSession.addToDisplay(mWindow, mSeq, mWindowAttributes,
getHostVisibility(), mDisplay.getDisplayId(), mTmpFrame,
mAttachInfo.mContentInsets, mAttachInfo.mStableInsets,
mAttachInfo.mOutsets, mAttachInfo.mDisplayCutout, mInputChannel,
mTempInsets);
setFrame(mTmpFrame);
} catch (RemoteException e) {
mAdded = false;
mView = null;
mAttachInfo.mRootView = null;
mInputChannel = null;
mFallbackEventHandler.setView(null);
unscheduleTraversals();
setAccessibilityFocus(null, null);
throw new RuntimeException("Adding window failed", e);
} finally {
if (restore) {
attrs.restore();
}
}
mPendingOverscanInsets.set(0, 0, 0, 0);
mPendingContentInsets.set(mAttachInfo.mContentInsets);
mPendingStableInsets.set(mAttachInfo.mStableInsets);
mPendingDisplayCutout.set(mAttachInfo.mDisplayCutout);
mPendingVisibleInsets.set(0, 0, 0, 0);
mAttachInfo.mAlwaysConsumeSystemBars =
(res & WindowManagerGlobal.ADD_FLAG_ALWAYS_CONSUME_SYSTEM_BARS) != 0;
mPendingAlwaysConsumeSystemBars = mAttachInfo.mAlwaysConsumeSystemBars;
mInsetsController.onStateChanged(mTempInsets);
mAppVisible = (res & WindowManagerGlobal.ADD_FLAG_APP_VISIBLE) != 0;
...
}
}
}
关键信息:
此函数是非常关键的一步,因为从这里开始,我们已经进入了第一帧的触发流程:
@Override
public void requestLayout() {
if (!mHandlingLayoutInLayoutRequest) {
checkThread();
mLayoutRequested = true;
scheduleTraversals();
}
}
...
void scheduleTraversals() {
if (!mTraversalScheduled) {
mTraversalScheduled = true;
mTraversalBarrier = mHandler.getLooper().getQueue().postSyncBarrier();
mChoreographer.postCallback(
Choreographer.CALLBACK_TRAVERSAL, mTraversalRunnable, null);
if (!mUnbufferedInputDispatch) {
scheduleConsumeBatchedInput();
}
notifyRendererOfFramePending();
pokeDrawLockIfNeeded();
}
}
关键信息:
这个类是线程单例的,在ViewRootImpl构造阶段首次触发主线程的单例创建,我们还是看一下这个类的构造函数,这里也有一些不可忽略的关键组件的创建:
private Choreographer(Looper looper, int vsyncSource) {
mLooper = looper;
mHandler = new FrameHandler(looper);
mDisplayEventReceiver = USE_VSYNC
? new FrameDisplayEventReceiver(looper, vsyncSource)
: null;
mLastFrameTimeNanos = Long.MIN_VALUE;
mFrameIntervalNanos = (long)(1000000000 / getRefreshRate());
mCallbackQueues = new CallbackQueue[CALLBACK_LAST + 1];
for (int i = 0; i <= CALLBACK_LAST; i++) {
mCallbackQueues[i] = new CallbackQueue();
}
// b/68769804: For low FPS experiments.
setFPSDivisor(SystemProperties.getInt(ThreadedRenderer.DEBUG_FPS_DIVISOR, 1));
}
关键信息:
继续跟着requestLayout的流程往下走到postCallback函数中,其最终执行函数如下:
private void postCallbackDelayedInternal(int callbackType,
Object action, Object token, long delayMillis) {
...
synchronized (mLock) {
final long now = SystemClock.uptimeMillis();
final long dueTime = now + delayMillis;
mCallbackQueues[callbackType].addCallbackLocked(dueTime, action, token);
if (dueTime <= now) {
scheduleFrameLocked(now);
} else {
Message msg = mHandler.obtainMessage(MSG_DO_SCHEDULE_CALLBACK, action);
msg.arg1 = callbackType;
msg.setAsynchronous(true);
mHandler.sendMessageAtTime(msg, dueTime);
}
}
}
这里虽然有两个分支,但最后执行的函数都是一样的,只是根据dalayMillis来确定一个合适的实际执行时间:
private void scheduleFrameLocked(long now) {
if (!mFrameScheduled) {
mFrameScheduled = true;
if (USE_VSYNC) {
if (DEBUG_FRAMES) {
Log.d(TAG, "Scheduling next frame on vsync.");
}
// If running on the Looper thread, then schedule the vsync immediately,
// otherwise post a message to schedule the vsync from the UI thread
// as soon as possible.
if (isRunningOnLooperThreadLocked()) {
scheduleVsyncLocked();
} else {
Message msg = mHandler.obtainMessage(MSG_DO_SCHEDULE_VSYNC);
msg.setAsynchronous(true);
mHandler.sendMessageAtFrontOfQueue(msg);
}
} else {
final long nextFrameTime = Math.max(
mLastFrameTimeNanos / TimeUtils.NANOS_PER_MS + sFrameDelay, now);
if (DEBUG_FRAMES) {
Log.d(TAG, "Scheduling next frame in " + (nextFrameTime - now) + " ms.");
}
Message msg = mHandler.obtainMessage(MSG_DO_FRAME);
msg.setAsynchronous(true);
mHandler.sendMessageAtTime(msg, nextFrameTime);
}
}
}
private void scheduleVsyncLocked() {
mDisplayEventReceiver.scheduleVsync();
}
关键信息:
该类是FrameDisplayEventReceiver的父类,java层只有访问接口,所有逻辑均通过JNI调用到CPP,对应的类为NativeDisplayEventReceiver,位于该文件中: frameworks/base/core/jni/android_view_DisplayEventReceiver.cpp 所以我们后面直接分析NativeDisplayEventReceiver这个类.
这个类的构造函数很简单,JNI调用到CPP会为其构造一个NativeDisplayEventReceiver对象,并返回给java层:
public DisplayEventReceiver(Looper looper, int vsyncSource, int configChanged) {
...
mReceiverPtr = nativeInit(new WeakReference<DisplayEventReceiver>(this), mMessageQueue,
vsyncSource, configChanged);
}
static jlong nativeInit(JNIEnv* env, jclass clazz, jobject receiverWeak,
jobject messageQueueObj, jint vsyncSource, jint configChanged) {
...
sp<NativeDisplayEventReceiver> receiver = new NativeDisplayEventReceiver(env,
receiverWeak, messageQueue, vsyncSource, configChanged);
status_t status = receiver->initialize();
}
NativeDisplayEventReceiver的构造函数中并没有什么重要的内容,而其父类则不同,其在创建时,会将mReceiver(DisplayEventReceiver(native))变量一同初始化
DisplayEventDispatcher::DisplayEventDispatcher(const sp<Looper>& looper,
ISurfaceComposer::VsyncSource vsyncSource,
ISurfaceComposer::ConfigChanged configChanged) :
mLooper(looper), mReceiver(vsyncSource, configChanged), mWaitingForVsync(false) {
ALOGV("dispatcher %p ~ Initializing display event dispatcher.", this);
}
而这个DisplayEventReceiver便是真正负责搭建起SF与APP之间通信管道的一个类,因为在它的构造函数中将会创建BitTube并传递给SF进程,用于SF与APP进程通信:
DisplayEventReceiver::DisplayEventReceiver(ISurfaceComposer::VsyncSource vsyncSource,
ISurfaceComposer::ConfigChanged configChanged) {
sp<ISurfaceComposer> sf(ComposerService::getComposerService());
if (sf != nullptr) {
mEventConnection = sf->createDisplayEventConnection(vsyncSource, configChanged);
if (mEventConnection != nullptr) {
mDataChannel = std::make_unique<gui::BitTube>();
mEventConnection->stealReceiveChannel(mDataChannel.get());
}
}
}
那APP进程内如何接收这些数据呢?这需要我们再回头看到NativeDisplayEventReceiver构造后调用的initialize函数中:
status_t DisplayEventDispatcher::initialize() {
status_t result = mReceiver.initCheck();
if (result) {
ALOGW("Failed to initialize display event receiver, status=%d", result);
return result;
}
int rc = mLooper->addFd(mReceiver.getFd(), 0, Looper::EVENT_INPUT,
this, NULL);
if (rc < 0) {
return UNKNOWN_ERROR;
}
return OK;
}
关键信息:
在了解了DisplayEventReceiver初始化的工作后,就可以继续跟踪请求Vsync的流程了:
status_t DisplayEventDispatcher::scheduleVsync() {
if (!mWaitingForVsync) {
ALOGV("dispatcher %p ~ Scheduling vsync.", this);
// Drain all pending events.
nsecs_t vsyncTimestamp;
PhysicalDisplayId vsyncDisplayId;
uint32_t vsyncCount;
if (processPendingEvents(&vsyncTimestamp, &vsyncDisplayId, &vsyncCount)) {
ALOGE("dispatcher %p ~ last event processed while scheduling was for %" PRId64 "",
this, ns2ms(static_cast<nsecs_t>(vsyncTimestamp)));
}
status_t status = mReceiver.requestNextVsync();
if (status) {
ALOGW("Failed to request next vsync, status=%d", status);
return status;
}
mWaitingForVsync = true;
}
return OK;
}
bool DisplayEventDispatcher::processPendingEvents(
nsecs_t* outTimestamp, PhysicalDisplayId* outDisplayId, uint32_t* outCount) {
bool gotVsync = false;
DisplayEventReceiver::Event buf[EVENT_BUFFER_SIZE];
while ((n = mReceiver.getEvents(buf, EVENT_BUFFER_SIZE)) > 0) {
ALOGV("dispatcher %p ~ Read %d events.", this, int(n));
for (ssize_t i = 0; i < n; i++) {
const DisplayEventReceiver::Event& ev = buf[i];
switch (ev.header.type) {
case DisplayEventReceiver::DISPLAY_EVENT_VSYNC:
gotVsync = true;
*outTimestamp = ev.header.timestamp;
*outDisplayId = ev.header.displayId;
*outCount = ev.vsync.count;
break;
...
}
}
if (n < 0) {
ALOGW("Failed to get events from display event dispatcher, status=%d", status_t(n));
}
return gotVsync;
}
关键信息:
当SF将事件写入BitTube后,handleEvent将会被回调:
int DisplayEventDispatcher::handleEvent(int, int events, void*) {
if (events & (Looper::EVENT_ERROR | Looper::EVENT_HANGUP)) {
return 0; // remove the callback
}
if (!(events & Looper::EVENT_INPUT)) {
return 1; // keep the callback
}
...
if (processPendingEvents(&vsyncTimestamp, &vsyncDisplayId, &vsyncCount)) {
ALOGV("dispatcher %p ~ Vsync pulse: timestamp=%" PRId64 ", displayId=%"
ANDROID_PHYSICAL_DISPLAY_ID_FORMAT ", count=%d",
this, ns2ms(vsyncTimestamp), vsyncDisplayId, vsyncCount);
mWaitingForVsync = false;
dispatchVsync(vsyncTimestamp, vsyncDisplayId, vsyncCount);
}
return 1; // keep the callback
}
java:
private void dispatchVsync(long timestampNanos, long physicalDisplayId, int frame) {
//又转一手调用到onVsync中
onVsync(timestampNanos, physicalDisplayId, frame);
}
关键信息:
经过一圈的调用,响应信号终于还是回到java了,到现在为止已经完成了VSYNC的请求与接收,那么接下来事情就看下当初这个VSYNC的请求者,他会拿这个VSYNC做些什么.
public void onVsync(long timestampNanos, long physicalDisplayId, int frame) {
long now = System.nanoTime();
if (timestampNanos > now) {
timestampNanos = now;
}
if (mHavePendingVsync) {
Log.w(TAG, "Already have a pending vsync event. There should only be "
+ "one at a time.");
} else {
mHavePendingVsync = true;
}
mTimestampNanos = timestampNanos;
mFrame = frame;
Message msg = Message.obtain(mHandler, this);
msg.setAsynchronous(true);
mHandler.sendMessageAtTime(msg, timestampNanos / TimeUtils.NANOS_PER_MS);
}
public void run() {
mHavePendingVsync = false;
doFrame(mTimestampNanos, mFrame);
}
关键信息:
看到了吗,Vsync开始驱动Java层的一些组件工作了,而doFrame即是被Vsync实际驱动的第一个函数:
void doFrame(long frameTimeNanos, int frame) {
final long startNanos;
synchronized (mLock) {
...
long intendedFrameTimeNanos = frameTimeNanos;
startNanos = System.nanoTime();
final long jitterNanos = startNanos - frameTimeNanos;
if (jitterNanos >= mFrameIntervalNanos) {
final long skippedFrames = jitterNanos / mFrameIntervalNanos;
if (skippedFrames >= SKIPPED_FRAME_WARNING_LIMIT) {
Log.i(TAG, "Skipped " + skippedFrames + " frames! "
+ "The application may be doing too much work on its main thread.");
}
final long lastFrameOffset = jitterNanos % mFrameIntervalNanos;
if (DEBUG_JANK) {
Log.d(TAG, "Missed vsync by " + (jitterNanos * 0.000001f) + " ms "
+ "which is more than the frame interval of "
+ (mFrameIntervalNanos * 0.000001f) + " ms! "
+ "Skipping " + skippedFrames + " frames and setting frame "
+ "time to " + (lastFrameOffset * 0.000001f) + " ms in the past.");
}
frameTimeNanos = startNanos - lastFrameOffset;
}
if (frameTimeNanos < mLastFrameTimeNanos) {
if (DEBUG_JANK) {
Log.d(TAG, "Frame time appears to be going backwards. May be due to a "
+ "previously skipped frame. Waiting for next vsync.");
}
scheduleVsyncLocked();
return;
}
mFrameInfo.setVsync(intendedFrameTimeNanos, frameTimeNanos);
mFrameScheduled = false;
mLastFrameTimeNanos = frameTimeNanos;
}
try {
Trace.traceBegin(Trace.TRACE_TAG_VIEW, "Choreographer#doFrame");
AnimationUtils.lockAnimationClock(frameTimeNanos / TimeUtils.NANOS_PER_MS);
mFrameInfo.markInputHandlingStart();
doCallbacks(Choreographer.CALLBACK_INPUT, frameTimeNanos);
mFrameInfo.markAnimationsStart();
doCallbacks(Choreographer.CALLBACK_ANIMATION, frameTimeNanos);
doCallbacks(Choreographer.CALLBACK_INSETS_ANIMATION, frameTimeNanos);
mFrameInfo.markPerformTraversalsStart();
doCallbacks(Choreographer.CALLBACK_TRAVERSAL, frameTimeNanos);
doCallbacks(Choreographer.CALLBACK_COMMIT, frameTimeNanos);
} finally {
AnimationUtils.unlockAnimationClock();
Trace.traceEnd(Trace.TRACE_TAG_VIEW);
}
...
}
这个回调的作用就是将工作转一手给doTraversal
final class TraversalRunnable implements Runnable {
@Override
public void run() {
doTraversal();
}
}
void doTraversal() {
if (mTraversalScheduled) {
mTraversalScheduled = false;
mHandler.getLooper().getQueue().removeSyncBarrier(mTraversalBarrier);
if (mProfile) {
Debug.startMethodTracing("ViewAncestor");
}
performTraversals();
if (mProfile) {
Debug.stopMethodTracing();
mProfile = false;
}
}
}
这个函数的作用也很简单,执行到了就说明之前请求到的Vsync已经过来了,我们需要准备处理View调用了,所以在一开始的时候,先将之前设置的同步屏障移除掉.performTraversals正式开始遍历View树.
★ 到这里,应用进程中关于请求绘制信号的流程就跟踪结束了.但是这并不是整个完整的流程,涉及到与SF的信号是怎么来的? 与SF交互的详细逻辑并没有去跟踪,所以下一篇会跟进看一下SF里面响应交互时的逻辑.遍历View树的逻辑跟踪放到SF之后继续跟.”