通过源码角度看看AccessibilityService

简介

AccessibilityService的设计初衷是为了辅助有身体缺陷的群体使用Android应用,它的设计贯穿着Android的控件树View, ViewGroup, ViewRootImpl体系。借助于system_server进程的中转,能够注册Accessibility事件的客户端可以具备通过system_server提供的Accessibility服务来实现监听、操作其它应用视图的功能。这个功能十分强大,可以模拟用户的行为去操作其它APP,常常被用在自动化测试、微信抢红包、自动回复等功能实现中。

写这个的初衷有二:

  • 之前已经完成了Android View控件树的绘制、事件分发的源码分析,知识储备足够
  • 最近接触到了一些自动化方面的项目,并且对使用无障碍服务实现的自动微信抢红包功能原理十分好奇
  • 整体图

    类图

  • AccessibilityService: APP端直接继承的类,本质上是Service,通过onBind获取匿名Binder对象实现通信
  • IAccessibilityServiceClientWrapper: 用于和system_server通信的匿名Binder服务
  • AccessibilityInteractionClient: 本质上是个binder服务,用于获取Node信息
  • AccessibilityManagerService: 运行在system_server的实名binder服务,是整体的管理类
  • Service: AccessibilityManagerService的内部类,用于响应AccessibilityInteractionClient的binder通信请求
  • AccessibilityInteractionConnection: 运行在被监测的APP端,提供查找、点击视图等服务
  • AccessibilityManager: 运行在各个APP端,用于发送视图变化事件
  • AccessibilityInteractionController: 具体视图查找、点击服务的中间控制器
  • AccessibilityNodeProvider: 由客户端实现的视图节点内容提供者,最终操作的实现者
  • 整体设计图

    实例代码

    public class AutoDismissService extends AccessibilityService {
     @Override
     public void onAccessibilityEvent(AccessibilityEvent event) {
     if (event == null) {
     return;
     }
     
     // 自动将android系统弹出的其它crash dialog取消
     dismissAppErrorDialogIfExists(event);
     }
     
     private void dismissAppErrorDialogIfExists(AccessibilityEvent event) {
     // WINDOW视图变化才进行对应操作
     if ((event.getEventType() == AccessibilityEvent.TYPE_WINDOW_CONTENT_CHANGED
     && event.getPackageName().equals("android")) {
     // 查找带有"OK"字符的可点击Node
     AccessibilityNodeInfo nodeInfo = findViewByText("OK", true);
     if (nodeInfo != null) {
     // 查找到后执行点击操作
     performViewClick(nodeInfo);
     }
     }
     public AccessibilityNodeInfo findViewByText(String text, boolean clickable) {
     // 获取当前窗口父节点
     AccessibilityNodeInfo accessibilityNodeInfo = getRootInActiveWindow();
     if (accessibilityNodeInfo == null) {
     return null;
     }
     // 获取到满足字符要求的节点
     List<AccessibilityNodeInfo> nodeInfoList = accessibilityNodeInfo.findAccessibilityNodeInfosByText(text);
     if (nodeInfoList != null && !nodeInfoList.isEmpty()) {
     for (AccessibilityNodeInfo nodeInfo : nodeInfoList) {
     if (nodeInfo != null && (nodeInfo.isClickable() == clickable)) {
      return nodeInfo;
     }
     }
     }
     return null;
     }
     
     public void performViewClick(AccessibilityNodeInfo nodeInfo) {
     if (nodeInfo == null) {
     return;
     }
     // 由下至上进行查询,直到寻找到可点击的节点
     while (nodeInfo != null) {
     if (nodeInfo.isClickable()) {
     nodeInfo.performAction(AccessibilityNodeInfo.ACTION_CLICK);
     break;
     }
     nodeInfo = nodeInfo.getParent();
     }
     }
    }

    以上是一个典型的实现Accessibility功能的JAVA代码,主要涉及三点功能:

  • 当系统中有应用视图变化后,onAccessibilityEvent 方法会自动被system_server调用
  • 通过AccessibilityService的getRootInActiveWindow与findAccessibilityNodeInfosByText方法,可以获取到节点信息
  • 通过AccessibilityNodeInfo的performAction方法,最终会在被监听APP中执行对应操作
  • 本篇文章将会围绕着这三点主要功能进行源码分析

    源码分析

    常见 AccessibilityEvent 事件种类

    序号 种类名称 触发时机
    1
    TYPE_VIEW_CLICKED
    可点击的组件被点击
    2
    TYPE_VIEW_LONG_CLICKED
    可点击的组件被长按
    3
    TYPE_VIEW_SELECTED
    组件被选中
    4
    TYPE_VIEW_FOCUSED
    组件获取到了焦点
    5
    TYPE_VIEW_TEXT_CHANGED
    组件中的文本发生变化
    6
    TYPE_VIEW_SCROLLED
    组件被滑动
    7
    TYPE_WINDOW_STATE_CHANGED
    dialog等被打开
    8
    TYPE_NOTIFICATION_STATE_CHANGED
    通知弹出
    9
    TYPE_WINDOW_CONTENT_CHANGED
    组件树发生了变化

    onAccessibilityEvent 触发流程

    这里以TextView.setText触发事件变化流程为例进行分析

    TextView.setText

    应用组件状态发生变化

    frameworks/base/core/java/android/widget/TextView.java

    private void setText(CharSequence text, BufferType type,
       boolean notifyBefore, int oldlen) {
     ...
     notifyViewAccessibilityStateChangedIfNeeded(AccessibilityEvent.CONTENT_CHANGE_TYPE_TEXT);
     ...   
    }
    public void notifyViewAccessibilityStateChangedIfNeeded(int changeType) {
     if (!AccessibilityManager.getInstance(mContext).isEnabled() || mAttachInfo == null) {
     return;
     }
     if (mSendViewStateChangedAccessibilityEvent == null) {
     // 本质上是一个Runnable,意味着这里的流程会进入异步处理
     mSendViewStateChangedAccessibilityEvent =
      new SendViewStateChangedAccessibilityEvent();
     }
     mSendViewStateChangedAccessibilityEvent.runOrPost(changeType);
    }
    private class SendViewStateChangedAccessibilityEvent implements Runnable {
     ...
     @Override
     public void run() {
     mPosted = false;
     mPostedWithDelay = false;
     mLastEventTimeMillis = SystemClock.uptimeMillis();
     if (AccessibilityManager.getInstance(mContext).isEnabled()) {
      final AccessibilityEvent event = AccessibilityEvent.obtain();
      event.setEventType(AccessibilityEvent.TYPE_WINDOW_CONTENT_CHANGED);
      event.setContentChangeTypes(mChangeTypes);
      // 最终TYPE_WINDOW_CONTENT_CHANGED事件在这里异步发送
      sendAccessibilityEventUnchecked(event);
     }
     mChangeTypes = 0;
     }
     ...
    }
    public void sendAccessibilityEventUnchecked(AccessibilityEvent event) {
     if (mAccessibilityDelegate != null) {
     mAccessibilityDelegate.sendAccessibilityEventUnchecked(this, event);
     } else {
     sendAccessibilityEventUncheckedInternal(event);
     }
    }
    public void sendAccessibilityEventUnchecked(View host, AccessibilityEvent event) {
     host.sendAccessibilityEventUncheckedInternal(event);
    }
    public void sendAccessibilityEventUncheckedInternal(AccessibilityEvent event) {
     if (!isShown()) {
     return;
     }
     ...
     // 此处交由TextView所在父View进行处理,为责任链模式,事件经过层层向上传递,最终交由ViewRootImpl进行处理
     ViewParent parent = getParent();
     if (parent != null) {
     getParent().requestSendAccessibilityEvent(this, event);
     }
    }

    ViewRootImpl.requestSendAccessibilityEvent

    ViewRootImpl将事件派发到system_server

    frameworks/base/core/java/android/view/ViewRootImpl.java

    @Override
    public boolean requestSendAccessibilityEvent(View child, AccessibilityEvent event) {
     ...
     // 本地调用到AccessibilityManager进行事件发送
     mAccessibilityManager.sendAccessibilityEvent(event);
     return true;
    }

    frameworks/base/core/java/android/view/accessibility/AccessibilityManager.java

    public void sendAccessibilityEvent(AccessibilityEvent event) {
     final IAccessibilityManager service;
     final int userId;
     synchronized (mLock) {
     // 获取system_server的Accessibility实名服务
     service = getServiceLocked();
     ...
     }
     
     try {
     ...
     long identityToken = Binder.clearCallingIdentity();
     // binder call 到服务端,进行事件分发中转
     doRecycle = service.sendAccessibilityEvent(event, userId);
     Binder.restoreCallingIdentity(identityToken);
     ...
     } catch (RemoteException re) {
     Log.e(LOG_TAG, "Error during sending " + event + " ", re);
     } finally {
     ...
     }
    }

    AccessibilityManagerService.sendAccessibilityEvent

    system_server将事件分发到各个监听组件变化的Service

    frameworks/base/services/accessibility/java/com/android/server/accessibility/AccessibilityManagerService.java

    // binder call 到服务端,触发事件派发
    @Override
    public boolean sendAccessibilityEvent(AccessibilityEvent event, int userId) {
     synchronized (mLock) {
     ...
     if (mSecurityPolicy.canDispatchAccessibilityEventLocked(event)) {
      ...
      notifyAccessibilityServicesDelayedLocked(event, false);
      notifyAccessibilityServicesDelayedLocked(event, true);
     }
     ...
     }
     return (OWN_PROCESS_ID != Binder.getCallingPid());
    }
    private void notifyAccessibilityServicesDelayedLocked(AccessibilityEvent event,
     boolean isDefault) {
     try {
     UserState state = getCurrentUserStateLocked();
     for (int i = 0, count = state.mBoundServices.size(); i < count; i++) {
      Service service = state.mBoundServices.get(i);
      if (service.mIsDefault == isDefault) {
      if (canDispatchEventToServiceLocked(service, event)) {
       // 调用内部服务,以触发事件派发
       service.notifyAccessibilityEvent(event);
      }
      }
     }
     } catch (IndexOutOfBoundsException oobe) {
     ...
     }
    }
    class Service extends IAccessibilityServiceConnection.Stub
     implements ServiceConnection, DeathRecipient {
     public void notifyAccessibilityEvent(AccessibilityEvent event) {
     synchronized (mLock) {
      ...
      if ((mNotificationTimeout > 0)
       && (eventType != AccessibilityEvent.TYPE_WINDOW_CONTENT_CHANGED)) {
      ...
      // 按照惯例,异步分发到客户端进行派发
      message = mEventDispatchHandler.obtainMessage(eventType);
      } else {
      message = mEventDispatchHandler.obtainMessage(eventType, newEvent);
      }
      mEventDispatchHandler.sendMessageDelayed(message, mNotificationTimeout);
     }
     } 
    }
    public Handler mEventDispatchHandler = new Handler(mMainHandler.getLooper()) {
     @Override
     public void handleMessage(Message message) {
     final int eventType = message.what;
     AccessibilityEvent event = (AccessibilityEvent) message.obj;
     notifyAccessibilityEventInternal(eventType, event);
     }
    };
    private void notifyAccessibilityEventInternal(int eventType, AccessibilityEvent event) {
     IAccessibilityServiceClient listener;
     ...
     // mServiceInterface是通过bind客户端的AccessibilityService,在onServiceConnected连接成功后,获取到binder proxy转化来的,以这种方式实现了system_server与客户端的通信
     listener = mServiceInterface;
     ...
     try {
     listener.onAccessibilityEvent(event);
     if (DEBUG) {
      Slog.i(LOG_TAG, "Event " + event + " sent to " + listener);
     }
     } catch (RemoteException re) {
     Slog.e(LOG_TAG, "Error during sending " + event + " to " + listener, re);
     } finally {
     event.recycle();
     }
    }

    AccessibilityService.onAccessibilityEvent

    APP接收到组件变化的事件,并可以选择做出相应的处理

    frameworks/base/core/java/android/accessibilityservice/AccessibilityService.java

    // 抽象方法,模板模式,被系统主动调用
    public abstract void onAccessibilityEvent(AccessibilityEvent event);
    // 该service是被system_server主动绑定的,获取到IAccessibilityServiceClientWrapper的proxy来实现系统的主动调用
    @Override
    public final IBinder onBind(Intent intent) {
     return new IAccessibilityServiceClientWrapper(this, getMainLooper(), new Callbacks() {
     ...
     @Override
     public void onAccessibilityEvent(AccessibilityEvent event) {
      AccessibilityService.this.onAccessibilityEvent(event);
     }
     ...
     }
    }
    // 收到binder调用后,使用handler异步进行事件的处理
    public void onAccessibilityEvent(AccessibilityEvent event) {
     Message message = mCaller.obtainMessageO(DO_ON_ACCESSIBILITY_EVENT, event);
     mCaller.sendMessage(message);
    }
    @Override
    public void executeMessage(Message message) {
     switch (message.what) {
     case DO_ON_ACCESSIBILITY_EVENT: {
      AccessibilityEvent event = (AccessibilityEvent) message.obj;
      if (event != null) {
      AccessibilityInteractionClient.getInstance().onAccessibilityEvent(event);
      // 通过回调调用以触发事件
      mCallback.onAccessibilityEvent(event);
      ...
      }
     } return;
     }
    }

    getRootInActiveWindow 父节点获取流程

    在调用findAccessibilityNodeInfosByText之前,需要通过getRootInActiveWindow方法获取到父节点,才能通过调用父AccessibilityNodeInfo的方法进行其子节点信息查询

    AccessibilityService.getRootInActiveWindow

    frameworks/base/core/java/android/accessibilityservice/AccessibilityService.java

    public AccessibilityNodeInfo getRootInActiveWindow() {
     // 查找父节点的操作没有在自己的类中实现,而是交由了同一进程的Client管理类进行处理
     return AccessibilityInteractionClient.getInstance().getRootInActiveWindow(mConnectionId);
    }

    frameworks/base/core/java/android/view/accessibility/AccessibilityInteractionClient.java

    public AccessibilityNodeInfo getRootInActiveWindow(int connectionId) {
     return findAccessibilityNodeInfoByAccessibilityId(connectionId,
      AccessibilityNodeInfo.ACTIVE_WINDOW_ID, AccessibilityNodeInfo.ROOT_NODE_ID,
      false, AccessibilityNodeInfo.FLAG_PREFETCH_DESCENDANTS);
    }
    public AccessibilityNodeInfo findAccessibilityNodeInfoByAccessibilityId(int connectionId,
      int accessibilityWindowId, long accessibilityNodeId, boolean bypassCache,
      int prefetchFlags) {
     ...
     // 尝试binder call到system_server,请求中转到其它APP进程中查询父节点信息,注意的是这里AccessibilityInteractionClient本身是个binder服务端,把this传到system_server后,其它进程可以通过这个引用拿到binder proxy,以实现通信
     final boolean success = connection.findAccessibilityNodeInfoByAccessibilityId(
       accessibilityWindowId, accessibilityNodeId, interactionId, this,
       prefetchFlags, Thread.currentThread().getId());
     Binder.restoreCallingIdentity(identityToken);
     // If the scale is zero the call has failed.
     if (success) {
     // 调用成功后,这里会尝试同步获取结果
     List<AccessibilityNodeInfo> infos = getFindAccessibilityNodeInfosResultAndClear(
      interactionId);
     finalizeAndCacheAccessibilityNodeInfos(infos, connectionId);
     if (infos != null && !infos.isEmpty()) {
      return infos.get(0);
     }
     } 
     ... 
    }

    Service.findAccessibilityNodeInfoByAccessibilityId

    注意一下,这里的Service不是Android中的四大组件的Service,取名叫AccessiblitManagerServiceInternal其实更合适

    frameworks/base/services/accessibility/java/com/android/server/accessibility/AccessibilityManagerService.java

    @Override
    public boolean findAccessibilityNodeInfoByAccessibilityId(
     int accessibilityWindowId, long accessibilityNodeId, int interactionId,
     IAccessibilityInteractionConnectionCallback callback, int flags,
     long interrogatingTid) throws RemoteException {
     ...
     // 获取到其他APP的节点获取服务
     IAccessibilityInteractionConnection connection = null;
     ...
     resolvedWindowId = resolveAccessibilityWindowIdLocked(accessibilityWindowId);
     ...
     if (!permissionGranted) {
     return false;
     } else {
     connection = getConnectionLocked(resolvedWindowId);
     if (connection == null) {
      return false;
     }
     }
     ...
     // 这里的callback为之前应用的服务proxy句柄,将它传入是为了之后的信息通信不再需要经过system_server中转,而是直接可以APP对APP的进行通信
     connection.findAccessibilityNodeInfoByAccessibilityId(accessibilityNodeId,
      partialInteractiveRegion, interactionId, callback, mFetchFlags | flags,
      interrogatingPid, interrogatingTid, spec);
     ...
    }

    AccessibilityInteractionConnection.findAccessibilityNodeInfoByAccessibilityId

    这里调用到了APP端,其实同onAccessibilityEvent调用流程一样,是APP->SYSTEM->APP的调用顺序

    frameworks/base/core/java/android/view/ViewRootImpl.java

    @Override
    public void findAccessibilityNodeInfoByAccessibilityId(long accessibilityNodeId,
     Region interactiveRegion, int interactionId,
     IAccessibilityInteractionConnectionCallback callback, int flags,
     int interrogatingPid, long interrogatingTid, MagnificationSpec spec) {
     ViewRootImpl viewRootImpl = mViewRootImpl.get();
     if (viewRootImpl != null && viewRootImpl.mView != null) {
     // 这里也只是委托给控制类进行细节操作的处理
     viewRootImpl.getAccessibilityInteractionController()
      .findAccessibilityNodeInfoByAccessibilityIdClientThread(accessibilityNodeId,
       interactiveRegion, interactionId, callback, flags, interrogatingPid,
       interrogatingTid, spec);
     } else {
     ...
     }
    }

    frameworks/base/core/java/android/view/AccessibilityInteractionController.java

    private void findAccessibilityNodeInfoByAccessibilityIdUiThread(Message message) {
     ...
     // 初始化将会返回的节点
     List<AccessibilityNodeInfo> infos = mTempAccessibilityNodeInfoList;
     infos.clear();
     try {
     if (mViewRootImpl.mView == null || mViewRootImpl.mAttachInfo == null) {
      return;
     }
     mViewRootImpl.mAttachInfo.mAccessibilityFetchFlags = flags;
     View root = null;
     if (accessibilityViewId == AccessibilityNodeInfo.UNDEFINED_ITEM_ID) {
      root = mViewRootImpl.mView;
     } else {
      root = findViewByAccessibilityId(accessibilityViewId);
     }
     ...
     } finally {
     try {
      ...
      adjustIsVisibleToUserIfNeeded(infos, interactiveRegion);
      // 通过callback binder proxy句柄,将节点信息binder回应用
      callback.setFindAccessibilityNodeInfosResult(infos, interactionId);
      infos.clear();
     } catch (RemoteException re) {
      /* ignore - the other side will time out */
     }
     ...
     }
    }

    AccessibilityInteractionClient.setFindAccessibilityNodeInfosResult

    frameworks/base/core/java/android/view/accessibility/AccessibilityInteractionClient.java

    public void setFindAccessibilityNodeInfosResult(List<AccessibilityNodeInfo> infos,
      int interactionId) {
     synchronized (mInstanceLock) {
     if (interactionId > mInteractionId) {
      if (infos != null) {
      ...
      // 设置应用的返回节点信息
      if (!isIpcCall) {
       mFindAccessibilityNodeInfosResult = new ArrayList<>(infos);
      } else {
       mFindAccessibilityNodeInfosResult = infos;
      }
      } else {
      mFindAccessibilityNodeInfosResult = Collections.emptyList();
      }
      mInteractionId = interactionId;
     }
     // 释放锁,停止等待,节点信息已经取回
     mInstanceLock.notifyAll();
     }
    }

    findAccessibilityNodeInfosByText与performAction 对目标节点进行操作

    AccessibilityNodeInfo.findAccessibilityNodeInfosByText

    找到父节点信息后,就可以通过父节点获取对应的子节点信息了

    frameworks/base/core/java/android/view/accessibility/AccessibilityNodeInfo.java

    public List<AccessibilityNodeInfo> findAccessibilityNodeInfosByText(String text) {
     ...
     // 一样的流程,通过AccessibilityInteractionClient去获取信息
     AccessibilityInteractionClient client = AccessibilityInteractionClient.getInstance();
     return client.findAccessibilityNodeInfosByText(mConnectionId, mWindowId, mSourceNodeId,
      text);
    }
    ``` 

    以下的代码流程同getRootInActiveWindow大概一致,就不详细分析了

    #### AccessibilityNodeInfo.performAction

    获取到对应子节点后,通过performAction可以执行对应的操作了,如常用的点击

    最终回调用到AccessibilityInteractionController,获取到AccessibilityProvier后就可以执行performAction的最终操作了

    frameworks/base/core/java/android/view/AccessibilityInteractionController.java

    ```java
    private void performAccessibilityActionUiThread(Message message) {
     View target = null;
     if (accessibilityViewId != AccessibilityNodeInfo.UNDEFINED_ITEM_ID) {
     target = findViewByAccessibilityId(accessibilityViewId);
     } else {
     target = mViewRootImpl.mView;
     }
     if (target != null && isShown(target)) {
     AccessibilityNodeProvider provider = target.getAccessibilityNodeProvider();
     if (provider != null) {
      if (virtualDescendantId != AccessibilityNodeInfo.UNDEFINED_ITEM_ID) {
      // 在客户端执行performAction操作
      succeeded = provider.performAction(virtualDescendantId, action,
       arguments);
      } else {
      succeeded = provider.performAction(AccessibilityNodeProvider.HOST_VIEW_ID,
       action, arguments);
      }
     } else if (virtualDescendantId == AccessibilityNodeInfo.UNDEFINED_ITEM_ID) {
      succeeded = target.performAccessibilityAction(action, arguments);
     }
     }
    }

    frameworks/base/core/java/android/view/View.java

    public boolean performAccessibilityActionInternal(int action, Bundle arguments) {
     ...
     switch (action) {
     case AccessibilityNodeInfo.ACTION_CLICK: {
      if (isClickable()) {
      // 最终调用到我们熟悉的View.performClick方法
      performClick();
      return true;
      }
     } break;
     ...
    }

    分析到这里可以看到,Accessibility服务框架类似于hook在Android View组件树中的一套实现,它并不是独立的一套机制,而是”寄生”在View的显示、事件分发的流程中。

    总结

    功能实现依赖于ViewRootImpl, ViewGroup, View视图层级管理的基本架构。在视图变化时发出事件、当收到视图操作请求时也能够作出响应。

    system_server在实现该功能的过程中扮演着中间人的角色。当被监听APP视图变化时,APP首先会发出事件到system_server,随后再中转到监听者APP端。当监听者APP想要执行视图操作时,也是首先在system_server中找到对应的客户端binder proxy,再调用相应接口调用到被监听APP中。完成相关操作后,通过已经获取到的监听APP binder proxy句柄,直接binder call到对应的监听客户端。

    无障碍权限十分重要,切记不可滥用,APP自身也需要有足够的安全意识,防止恶意应用通过该服务获取用户隐私信息

    好了,以上就是这篇文章的全部内容了,希望本文的内容对大家的学习或者工作具有一定的参考学习价值,如果有疑问大家可以留言交流,谢谢大家对的支持。

    若文章对您有帮助,帮忙点个赞!

    0
    0
    发布时间 2018-06-12 12:09:10