导语 | Flutter作为一个跨平台开发框架,手势识别被放在Dart层。事件的收集和传递依赖各平台的不同实现,并将屏蔽事件对象的差异,统一转换为Flutter可识别的事件对象。Flutter采用竞技场模式对手势进行识别并决出最终获胜者。本文将从源码角度对Flutter事件传递及手势识别系统进行分析,希望与大家一同交流(本文论述基于Flutter 1.23.0)。文章作者:omegaxiao,腾讯PCG研发工程师。
一、事件传递
手势是对事件的语义化封装,手势的识别依赖于设备屏幕上所产生的各类事件(down/move/up/cancel)。无论在Android平台,还是iOS平台,事件均由平台原生父控件进行收集,在屏蔽平台差异后向Engine侧转发,事件最终将被发送至Framework层的GestureBinding中进行处理。在Android平台,事件传递的路径为java=>c++=>dart,具体流程如下图所示:
在Android平台事件最初由FlutterView(extends FrameLayout)接收, 随后对事件MotionEvent对象进行处理,提取其中关键数据(包括事件类型、坐标、时间戳等数据)序列化成ByteBuffer。
/*AndroidTouchProcessor.java**/private void addPointerForIndex(MotionEvent event, int pointerIndex, int pointerChange, int pointerData, Matrix transformMatrix, ByteBuffer packet long motionEventId = 0; if (trackMotionEvents) { MotionEventTracker.MotionEventId trackedEvent = motionEventTracker.track(event); motionEventId = trackedEvent.getId(); } int pointerKind = getPointerDeviceTypeForToolType(event.getToolType(pointerIndex)); int signalKind = event.getActionMasked() == MotionEvent.ACTION_SCROLL ? PointerSignalKind.SCROLL : PointerSignalKind.NONE; long timeStamp = event.getEventTime() * 1000; // Convert from milliseconds to micro packet.putLong(motionEventId); // motionEventId packet.putLong(timeStamp); // 时间戳 packet.putLong(pointerChange); // change packet.putLong(event.getPointerId(pointerIndex)); // device packet.putLong(0); // pointer_identifier float viewToScreenCoords[] = {event.getX(pointerIndex), event.getY(pointerIndex)}; transformMatrix.mapPoints(viewToScreenCoords); packet.putDouble(viewToScreenCoords[0]); // x坐标 packet.putDouble(viewToScreenCoords[1]); // y坐标 ... packet.putLong(buttons); // buttons ... packet.putDouble(0.0); // scroll_delta_x packet.putDouble(0.0); // scroll_delta_x }
在平台层处理完事件数据后,通过JNI接口将事件的bytebuffer发送至Engine层:
/**FlutterJNI.java*/ @UiThread public void dispatchPointerDataPacket(@NonNull ByteBuffer buffer, int position) { ensureRunningOnMainThread(); ensureAttachedToNative(); nativeDispatchPointerDataPacket(nativePlatformViewId, buffer, position); }
private native void nativeDispatchPointerDataPacket( long nativePlatformViewId, @NonNull ByteBuffer buffer, int position); // ------ End Touch Interaction Support ---
//platform_view_android_jni_impl.ccstatic void DispatchPointerDataPacket(JNIEnv* env, jobject jcaller, jlong shell_holder, jobject buffer, jint position) { uint8_t* data = static_cast<uint8_t*>(env->GetDirectBufferAddress(buffer)); auto packet = std::make_unique<flutter::PointerDataPacket>(data, position); ANDROID_SHELL_HOLDER->GetPlatformView()->DispatchPointerDataPacket( std::move(packet));}
在Engine层经过几道转发后,最终会在window中通过tonic将数据发送至dart层。
//window.ccvoid Window::DispatchPointerDataPacket(const PointerDataPacket& packet) { std::shared_ptr<tonic::DartState> dart_state = library_.dart_state().lock(); if (!dart_state) { return; } tonic::DartState::Scope scope(dart_state);
const std::vector<uint8_t>& buffer = packet.data(); Dart_Handle data_handle = tonic::DartByteData::Create(buffer.data(), buffer.size()); if (Dart_IsError(data_handle)) { return; } tonic::LogIfError(tonic::DartInvokeField( library_.value(), "_dispatchPointerDataPacket", {data_handle}));}
//hooks.dart@pragma('vm:entry-point')// ignore: unused_elementvoid _dispatchPointerDataPacket(ByteData packet) { if (window.onPointerDataPacket != null) _invoke1<PointerDataPacket>(window.onPointerDataPacket, window._onPointerDataPacketZone, _unpackPointerDataPacket(packet)); //此时会对事件数据进行反序列化为PointerDataPacket}
最终事件通过dart层的window对象传递到Gesturebinding中进行后续处理。
二、事件分发
前面说到事件数据经过层层转发,最终会到达dart层的GestureBinding中进行处理,在介绍下面事件分发流程之前,先了解下与之相关比较重要的几个类:
具体的处理流程如下(PointerDownEvent):
不同于Android的事件冒泡传递以及iOS的响应链机制,Flutter通过hitTest一次性获取该事件相关的所有组件,再逐一分发。在Flutter中,实际事件的响应者是这些组件所对应的RenderObject,并且通常为RenderBox对象。例如GestureDector所对应的RenderObject为RenderPointerListener。
void _handlePointerEventImmediately(PointerEvent event) { HitTestResult? hitTestResult; if (event is PointerDownEvent || event is PointerSignalEvent || event is PointerHoverEvent) { assert(!_hitTests.containsKey(event.pointer)); hitTestResult = HitTestResult(); //hitTest获取事件响应对象 hitTest(hitTestResult, event.position); if (event is PointerDownEvent) { _hitTests[event.pointer] = hitTestResult; } assert(() { if (debugPrintHitTestResults) debugPrint('$event: $hitTestResult'); return true; }()); } else if (event is PointerUpEvent || event is PointerCancelEvent) { hitTestResult = _hitTests.remove(event.pointer); } else if (event.down) { hitTestResult = _hitTests[event.pointer]; } assert(() { if (debugPrintMouseHoverEvents && event is PointerHoverEvent) debugPrint('$event'); return true; }()); //分发事件 if (hitTestResult != null || event is PointerAddedEvent || event is PointerRemovedEvent) { assert(event.position != null); dispatchEvent(event, hitTestResult); } }
GestureBinding的hitTest()会首先调用到RenderBinding中的hitTest()方法,该方法从根节点(renderView)开始,根据子节点(RenderBox)hitTest规则向下遍历,将满足条件的子节点加入HitTestResult中,HitTestResult中持有一个hitTestTarget列表,存有当前事件对应的所有对象。我们看到,在遍历完所有子节点后,GestureBinding的hitTest将自身也加入了hitTestResult对象中。
/**GestureBinding*/ @override // from HitTestable void hitTest(HitTestResult result, Offset position) { result.add(HitTestEntry(this)); //添加自身作为响应者 }
/**RenderBinding*/ @override void hitTest(HitTestResult result, Offset position) { assert(renderView != null); assert(result != null); assert(position != null); renderView.hitTest(result, position: position); //rootView开始hitTest super.hitTest(result, position); }
/**RenderBox*/bool hitTest(BoxHitTestResult result, { required Offset position }) { if (_size!.contains(position)) { if (hitTestChildren(result, position: position) || hitTestSelf(position)) { result.add(BoxHitTestEntry(this, position)); return true; } } return false; }
在RenderBox的hitTest中,首先判断当前事件的位置是否落在自身的布局范围内,若在则继续通过对孩子节点进行hitTest,同时会通过hitTestSelf方法决定是否将自身加入响应者列表。RenderBox的子类可以通过重写hitTest()、hitTestChild()、hitTestSelf()方法自定义规则。
在hitTest获取当前事件所有响应者后,通过dispatchEvent分发事件。
void dispatchEvent(PointerEvent event, HitTestResult? hitTestResult) { if (hitTestResult == null) { try { pointerRouter.route(event); } catch (exception, stack) {} return; } for (final HitTestEntry entry in hitTestResult.path) { //遍历hitTestResult try { entry.target.handleEvent(event.transformed(entry.transform), entry); } catch (exception, stack) {} } }
不同于Android的事件传递,在Flutter中事件在传递过程中无法被拦截,hitTestResult中的每个响应者都会收到事件。对于大多数的RenderObject来说,其handleEvent()都是空实现。因为大多数组件可以通过增加一层来Listener实现对事件的响应。
另外从源码可以看出,在前面提到GestureDetector所对应的RenderObject为RenderPointerListener,这里我们直接看RenderPointerListner的handleEvent实现。
/**handleEvent*/@override void handleEvent(PointerEvent event, HitTestEntry entry) { assert(debugHandleEvent(event, entry)); if (event is PointerDownEvent) return onPointerDown?.call(event); //处理PointerDownEvent事件 if (event is PointerMoveEvent) return onPointerMove?.call(event); if (event is PointerUpEvent) return onPointerUp?.call(event); if (event is PointerHoverEvent) return onPointerHover?.call(event); if (event is PointerCancelEvent) return onPointerCancel?.call(event); if (event is PointerSignalEvent) return onPointerSignal?.call(event); }
我们知道,每一种手势对应一个GestureRecognizer,在GestureDetector中注册了相应的回调方法,就会创建对应的GestureRecognizer。对于GestureDectector来说,只需要在PointerDownEvent事件发生时,将所有的GestureRecognizer添加到全局的路由中,之后的move/up之类的事件交由各个GestureRecognizer处理即可。因此这里只有onPointerDown方法被赋值,实际指向的是RawGestureDetectorState中的_handlePointerDown方法。
/**RawGestureDetectorState*/void _handlePointerDown(PointerDownEvent event) { assert(_recognizers != null); for (final GestureRecognizer recognizer in _recognizers!.values) recognizer.addPointer(event); }
在_handlePointerDown中会遍历当前GestureDetector监听的所有GestureRecognizer,通过addPointer方法将这些GestureRecognizer添加到GestureBinding的pointerRouter中,以OneSequenceGestureRecognizer及其子类为例,会在startTrackingPointer中将当前手势识别器的handleEvent方法注册到路由,同时也会将当前识别器加入到竞技场,竞技场的相关逻辑会在后面专门讲述。需要注意的是,recognizers里GestureRecognizer是有固定顺序的,可以直接通过查看GestureDetector中创建顺序看到。
@protected void startTrackingPointer(int pointer, [Matrix4? transform]) { GestureBinding.instance!.pointerRouter.addRoute(pointer, handleEvent, transform); //添加到路由中 _trackedPointers.add(pointer); assert(!_entries.containsValue(pointer)); _entries[pointer] = _addPointerToArena(pointer);//添加到竞技场 }
到这里,可以看到,每个GestureDetector对应的RenderPointerListner在被调用handleEvent后,会将监听的所有GestureRecognizer添加到GestureBindigng的pointerRouter中。这样一来,GestureBinding通过pointerRouter就能统一向所有的监听器转发事件。
前面说到,在hitTest遍历完所有子节点后,GestureBinding的hitTest将自身也加入了hitTestResult对象中。因此,在dispatchEvent的最后,会调用GestureBinding的handleEvent。
/**GestureBinding*/@override // from HitTestTarget void handleEvent(PointerEvent event, HitTestEntry entry) { pointerRouter.route(event); //向所有GestureRecognizer转发Event; if (event is PointerDownEvent) { gestureArena.close(event.pointer); } else if (event is PointerUpEvent) { gestureArena.sweep(event.pointer); } else if (event is PointerSignalEvent) { pointerSignalResolver.resolve(event); } }
GestureBinding的handleEvent首先会向所有的GestureRecognizer转发Event事件,调用GestureRecognizer的handleEvent()方法处理事件。然后根据事件类型,执行竞技场的相关操作。
三、手势识别
在实际的开发过程中,我们可能需要对一些手势做出响应,例如:点击、双击、长按、滑动、缩放等。对于某个控件,有时候需要同时对多种手势进行不同的响应。例如某个按钮需要支持对点击和长按的监听,那么当用户操作发生时,如何决策哪一个手势应该得到响应,这里就需要引出我们的手势竞争机制。Flutter通过竞技场模型在众多手势中筛选出唯一的获胜者,响应该手势。
首先我们来看一下手势竞争机制的几个重要角色:
上一节说到GestureBinding在handleEvent中最后会根据事件类型,执行竞技场的相关操作。当PointerDownEvent发生时,会执行竞技场的关闭操作,标识着竞技开始,手势可以开始竞争。当PointerUpEvent发生时,会强制执行清扫操作,决出获胜者。我们先看GestureArenaManager的关闭操作。
/**GestureArenaManager*/ void close(int pointer) { final _GestureArena? state = _arenas[pointer]; if (state == null) return; // This arena either never existed or has been resolved. state.isOpen = false; //设置竞技场状态为关闭 assert(_debugLogDiagnostic(pointer, 'Closing', state)); _tryToResolveArena(pointer, state); //尝试当前状态下能否决出获胜者 }
在关闭竞技场后会尝试一次,查看当前状态下能否决出获胜者。如果当前只存在一个手势,那么这个手势将直接成为获胜者,其acceptGesture被调用,此次竞争结束,竞技场被移除。
/**GestureArenaManager*/ void _resolveByDefault(int pointer, _GestureArena state) { if (!_arenas.containsKey(pointer)) return; // Already resolved earlier. assert(_arenas[pointer] == state); assert(!state.isOpen); final List<GestureArenaMember> members = state.members; assert(members.length == 1); _arenas.remove(pointer); //移除本次事件对应的竞技场 assert(_debugLogDiagnostic(pointer, 'Default winner: ${state.members.first}')); state.members.first.acceptGesture(pointer); //响应该手势 }
手势的竞争实质上是参与竞争的各类手势,在接收到事件流后,根据自身的规则做出接受和拒绝的响应。这套规则就直接定义在GestureRecognizer的handleEvent方法中。
总结一下竞技场的几个规则:
接下来我们以两个场景为例来分析一下多手势下的竞争流程:
先分别看下TapGestureRecognizer和DragGestureRecognizer的handleEvent()方法。TapGestureRecognizer的handleEvent()方法实现在父类PrimaryPointerGestureRecognizer中。
void handleEvent(PointerEvent event) { assert(state != GestureRecognizerState.ready); if (state == GestureRecognizerState.possible && event.pointer == primaryPointer) { final bool isPreAcceptSlopPastTolerance = !_gestureAccepted && preAcceptSlopTolerance != null && _getGlobalDistance(event) > preAcceptSlopTolerance!; final bool isPostAcceptSlopPastTolerance = _gestureAccepted && postAcceptSlopTolerance != null && _getGlobalDistance(event) > postAcceptSlopTolerance!;
if (event is PointerMoveEvent && (isPreAcceptSlopPastTolerance || isPostAcceptSlopPastTolerance)) { resolve(GestureDisposition.rejected); //拒绝,退出竞争 stopTrackingPointer(primaryPointer!); //停止事件路由,不再接收事件 } else { handlePrimaryPointer(event); } } stopTrackingIfPointerNoLongerDown(event); }
可以看出,在收到move事件后,如果移动的距离超过一定阈值,TapGestureRecognizer会选择拒绝,退出竞争,同时停止事件接收。否则继续调用handlePrimaryPointer()处理事件。
@override void handlePrimaryPointer(PointerEvent event) { if (event is PointerUpEvent) { //响应up事件 _up = event; _checkUp(); } else if (event is PointerCancelEvent) { //响应cancel事件 resolve(GestureDisposition.rejected);//拒绝,退出竞争 if (_sentTapDown) { _checkCancel(event, ''); } _reset(); } else if (event.buttons != _down!.buttons) { resolve(GestureDisposition.rejected); stopTrackingPointer(primaryPointer!); } }
当cancel事件到来时,TapGestureRecognizer会直接选择拒绝,退出竞争。当up事件到来,调用_checkUp()检查当前手势状态是否为接受(accept),如果是接受,则会回调到在GestureDetector中注册的onTap方法。
接着,我们再来看看DragGestureRecognizer的handleEvent()方法。
@override void handleEvent(PointerEvent event) { assert(_state != _DragState.ready); if (!event.synthesized && (event is PointerDownEvent || event is PointerMoveEvent)) { final VelocityTracker tracker = _velocityTrackers[event.pointer]!; assert(tracker != null); tracker.addPosition(event.timeStamp, event.localPosition); }
if (event is PointerMoveEvent) { //处理move事件,在其中决策是否接受 if (event.buttons != _initialButtons) { _giveUpPointer(event.pointer); return; } if (_state == _DragState.accepted) { //当前手势状态接受,直接检查更新 _checkUpdate( sourceTimeStamp: event.timeStamp, delta: _getDeltaForDetails(event.localDelta), primaryDelta: _getPrimaryValueFromOffset(event.localDelta), globalPosition: event.position, localPosition: event.localPosition, ); } else { _pendingDragOffset += OffsetPair(local: event.localDelta, global: event.delta); _lastPendingEventTimestamp = event.timeStamp; _lastTransform = event.transform; final Offset movedLocally = _getDeltaForDetails(event.localDelta); final Matrix4? localToGlobalTransform = event.transform == null ? null : Matrix4.tryInvert(event.transform!); _globalDistanceMoved += PointerEvent.transformDeltaViaPositions( transform: localToGlobalTransform, untransformedDelta: movedLocally, untransformedEndPosition: event.localPosition, ).distance * (_getPrimaryValueFromOffset(movedLocally) ?? 1).sign; if (_hasSufficientGlobalDistanceToAccept(event.kind)) //判断move的距离是否达到阈值 resolve(GestureDisposition.accepted); //接受手势 } } if (event is PointerUpEvent || event is PointerCancelEvent) { _giveUpPointer( event.pointer, reject: event is PointerCancelEvent || _state ==_DragState.possible, ); } }
一旦move事件移动的距离达到drag手势的阈值,DragGestureRecognizer便会选择接受,从而调用GestureDetector中注册的drag相关的事件回调。由于drag手势中move是一段连续的操作,因此在将当前手势置为接受状态后,再有新的move事件过来,直接触发drag相关的回调即可。
来看下完整的流程:
其中蓝色开头为move事件,红色开头为up事件。图中流程对应的是move事件偏移量超过tap手势允许的最小阈值(最小值kTouchSlop = 18.0)的情况,这个情况下,TapGestureRecognizer会通知到GestureAreanManager拒绝本次手势,不再参与竞争。
GestureAreanManager每一次在收到竞争者退出的通知后都会查看当前竞争者的数量,如果只剩下一个竞争者,则直接宣布其为获胜者。此处我们只设置了tap和drag手势,由于tap的退出,drag直接获得了胜利,随后drag的start和update被调用,最终该手势在GestureDetector注册的相关回调被调用。最后up事件到来,由于tap早早退出了竞争,不再接收事件。up事件就直接分发给了dragGestureRecognizer。
这里我们直接看DoubleTapGestureRecognizer的handleEvent()方法。
void _handleEvent(PointerEvent event) { final _TapTracker tracker = _trackers[event.pointer]!; if (event is PointerUpEvent) { if (_firstTap == null) _registerFirstTap(tracker); //记录首次点击 else _registerSecondTap(tracker);//记录第二次点击 } else if (event is PointerMoveEvent) { if (!tracker.isWithinGlobalTolerance(event, kDoubleTapTouchSlop)) _reject(tracker); } else if (event is PointerCancelEvent) { _reject(tracker); } }
在PointerUpEvent事件到来的时候,会通过registerFirstTap()和registerSecondTap()方法分别标记两次点击事件。
void _registerFirstTap(_TapTracker tracker) { _startDoubleTapTimer(); //启动定时器 GestureBinding.instance!.gestureArena.hold(tracker.pointer); //挂起竞技场 // Note, order is important below in order for the clear -> reject logic to // work properly. _freezeTracker(tracker); _trackers.remove(tracker.pointer); _clearTrackers(); _firstTap = tracker; }
void _registerSecondTap(_TapTracker tracker) { _firstTap!.entry.resolve(GestureDisposition.accepted); tracker.entry.resolve(GestureDisposition.accepted); //接受事件 _freezeTracker(tracker); _trackers.remove(tracker.pointer); _checkUp(tracker.initialButtons); _reset();}
在registerFirstTap()中看到,这里会启动一个定时器。Flutter中定义了构成双击事件的两次点击之间的最大时间间隔,默认值为300ms。紧接着立马调用竞技场的hold()方法,挂起竞技场,这意味着,在此之后300ms,竞技场都将处理挂起状态,即使PointerUpEvent到来,也不会触发竞技场的sweep操作。如果超过300md,下一次点击事件还未完成,则会将竞技场释放(release)。
void _reset() { _stopDoubleTapTimer(); //停止定时器 if (_firstTap != null) { if (_trackers.isNotEmpty) _checkCancel(); final _TapTracker tracker = _firstTap!; _firstTap = null; _reject(tracker); GestureBinding.instance!.gestureArena.release(tracker.pointer); //释放竞技场 } _clearTrackers(); }
这里我们就不画时序图,简单用文字梳理下tap手势和doubleTap手势的竞争过程:
四、结语
最后再总结一下,Flutter是通过hitTest找到所有事件响应者,并在down事件到来时,将监听的所有手势识别器GestureRecognizer加入事件路由以及竞技场中,并按顺序向他们转发事件。而GestureRecognizer接收事件后按自己对应的手势规则决定接受/拒绝响应,最终决出获胜的手势,回调GestureDetector中注册的相应方法。