基于 Android Studio 创建 AccessibilityService 与 Node.js 通信的手机自动化实现
摘要:
本文研究并实现了一种通过网络通信远程控制 Android AccessibilityService 的方法,以实现应用程序的自动化操作。利用 Node.js 的 net
模块建立客户端,与运行在 Android 设备上的 AccessibilityService 进行通信。通过发送 JSON 格式的指令,可以控制 AccessibilityService 执行查找节点、点击、长按、输入文本、全局操作(如返回)等动作。本文详细介绍了 Android 项目的创建、AccessibilityService 的实现、Node.js 客户端的编写,以及在低版本 Android 系统 (如 Android 10) 上可能遇到的无障碍服务需要重启才能生效的问题,并提供了使用较高 Java 和 JVM 版本的解决方案。
关键词: Android studio, AccessibilityService, Node.js, 网络通信, 自动化, 远程控制, 无障碍服务
1. 引言
Android 的 AccessibilityService 为开发者提供了一种强大的机制,可以访问和控制设备上的用户界面元素。传统的 AccessibilityService 应用主要集中在辅助功能领域,为视障或行动不便的用户提供便利。然而,AccessibilityService 的能力远不止于此,它也可以被用于应用程序的自动化测试、UI 自动化操作等场景。
直接在 Android 设备上编写和部署自动化脚本可能会受到限制。为了更灵活地控制 AccessibilityService,本文提出了一种基于网络通信的方法。通过将控制逻辑转移到 Node.js 客户端,我们可以利用 Node.js 强大的异步处理能力和丰富的第三方库,更方便地编写和管理自动化脚本。
2. 系统架构
本系统由两部分组成:
- Android AccessibilityService (服务端): 运行在 Android 设备上,监听特定端口,接收来自 Node.js 客户端的指令,并执行相应的操作。
- Node.js 客户端: 运行在 PC 或其他设备上,通过 TCP 连接与 Android AccessibilityService 通信,发送 JSON 格式的指令。
图 1. 系统架构图
+-----------------+ TCP 连接 +---------------------+
| Node.js 客户端 | <-----------> | Android 设备 |
| (运行在 PC) | | (AccessibilityService)|
+-----------------+ +---------------------+
项目结构
MyAccessibility/
├── app/
│ ├── src/main/
│ │ ├── java/com/example/myaccessibility/
│ │ │ └── MyAccessibility.kt # 无障碍服务实现
│ │ ├── res/xml/
│ │ │ └── accessibility_service_config.xml # 服务配置文件
│ │ └── AndroidManifest.xml
│ └── build.gradle.kts
└── node-client/ # Node.js客户端示例
└── client.js
3. Android AccessibilityService 实现
3.1 项目创建与配置 (MyAccessibility)
-
创建项目:
- 打开 Android Studio。
- 选择 "New Project"。
- 选择 "Empty Activity" 模板(或任何你喜欢的模板,但这里我们为了简洁起见使用 Empty Activity)。
- 项目名称:
MyAccessibility
。 - 包名:
com.example.myaccessibility
。 - 选择语言:
Kotlin
。 - Minimum SDK:
API 29: Android 10 (Q)
(为了演示低版本问题,初始设置为 29,后续会修改)。 - Build configuration language:
Kotlin DSL (build.gradle.kts)
-
配置
build.gradle.kts
://\MyAccessibility\app\build.gradle.kts plugins { alias(libs.plugins.android.application) alias(libs.plugins.kotlin.android) // 如果你使用的是 Jetpack Compose,则 *不要* 注释掉这一行 alias(libs.plugins.kotlin.compose) } android { namespace = "com.example.myaccessibility" compileSdk = 35 // 修改为 29 defaultConfig { applicationId = "com.example.myaccessibility" minSdk = 29 // 修改为 29 或更低 targetSdk = 35 // 修改为 35 versionCode = 1 versionName = "1.0" testInstrumentationRunner = "androidx.test.runner.AndroidJUnitRunner" } buildTypes { release { isMinifyEnabled = false proguardFiles( getDefaultProguardFile("proguard-android-optimize.txt"), "proguard-rules.pro" ) } } compileOptions { sourceCompatibility = JavaVersion.VERSION_17 // 或 JavaVersion.VERSION_17 targetCompatibility = JavaVersion.VERSION_17 // 或 JavaVersion.VERSION_17 } kotlinOptions { jvmTarget = "17" // 改为 11 } buildFeatures { compose = true // 保持为 true,因为你使用了 Compose } } dependencies { implementation(libs.androidx.core.ktx) implementation(libs.androidx.lifecycle.runtime.ktx) implementation(libs.androidx.activity.compose) implementation(platform(libs.androidx.compose.bom)) implementation(libs.androidx.ui) implementation(libs.androidx.ui.graphics) implementation(libs.androidx.ui.tooling.preview) implementation(libs.androidx.material3) // 保持 Material 3 依赖 testImplementation(libs.junit) androidTestImplementation(libs.androidx.junit) androidTestImplementation(libs.androidx.espresso.core) androidTestImplementation(platform(libs.androidx.compose.bom)) androidTestImplementation(libs.androidx.ui.test.junit4) debugImplementation(libs.androidx.ui.tooling) debugImplementation(libs.androidx.ui.test.manifest) implementation("androidx.appcompat:appcompat:1.6.1") // 确保包含 AppCompat 库 }
- 关键点:
compileSdk
,minSdk
,targetSdk
: 初始设置为29(为了演示问题)。sourceCompatibility
,targetCompatibility
,jvmTarget
: 设置为 11(为了解决低版本问题)。
- 关键点:
-
创建
MyAccessibility.java
(AccessibilityService)://\MyAccessibility\app\src\main\java\com\example\myaccessibility\MyAccessibility.java package com.example.myaccessibility; import android.accessibilityservice.AccessibilityService; import android.accessibilityservice.AccessibilityServiceInfo; import android.accessibilityservice.GestureDescription; import android.annotation.SuppressLint; import android.app.AlarmManager; import android.app.Notification; import android.app.NotificationChannel; import android.app.NotificationManager; import android.app.PendingIntent; import android.app.SearchManager; import android.content.ActivityNotFoundException; import android.content.Context; import android.content.Intent; import android.graphics.Path; import android.graphics.Point; import android.graphics.Rect; import android.os.Build; import android.os.Bundle; import android.os.Handler; import android.os.Looper; import android.os.PowerManager; import android.os.SystemClock; import android.provider.Settings; import android.util.Log; import android.view.Display; import android.view.WindowManager; import android.view.accessibility.AccessibilityEvent; import android.view.accessibility.AccessibilityNodeInfo; import androidx.core.app.NotificationCompat; import org.json.JSONException; import org.json.JSONObject; import java.io.BufferedReader; import java.io.IOException; import java.io.InputStreamReader; import java.io.PrintWriter; import java.net.ServerSocket; import java.net.Socket; import java.net.SocketException; import java.net.SocketTimeoutException; import java.util.ArrayList; import java.util.HashMap; import java.util.List; import java.util.Map; import java.util.concurrent.ExecutorService; import java.util.concurrent.Executors; import java.util.regex.Matcher; import java.util.regex.Pattern; public class MyAccessibility extends AccessibilityService { private static String TAG = "MyAccessibilityService"; private static final int SERVER_PORT = 6000; private static final int NOTIFICATION_ID = 1; private static final String CHANNEL_ID = "MyAccessibilityServiceChannel"; private static final long HEARTBEAT_TIMEOUT = 300000; private Handler mHandler = new Handler(Looper.getMainLooper()); private long mLastEventTime = 0; private final Handler mHeartbeatHandler = new Handler(Looper.getMainLooper()); private final Runnable mHeartbeatRunnable = this::checkHeartbeat; private ServerSocket serverSocket; private ExecutorService executorService; @SuppressLint("ForegroundServiceType") @Override protected void onServiceConnected() { super.onServiceConnected(); try { Log.i(TAG, "无障碍服务已连接"); // 配置无障碍服务 configureAccessibilityService(); // 启动前台服务并创建通知 startForeground(NOTIFICATION_ID, createNotification()); // 启动服务及心跳 startServer(); startHeartbeat(); // 配置无障碍服务信息 AccessibilityServiceInfo info = getServiceInfo(); if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.UPSIDE_DOWN_CAKE) { info.flags |= AccessibilityServiceInfo.FLAG_SEND_MOTION_EVENTS; // 根据需要调整 } setServiceInfo(info); } catch (Exception e) { Log.e(TAG, "无障碍服务连接失败: " + e.getMessage(), e); } } private void checkHeartbeat() { if (System.currentTimeMillis() - mLastEventTime > HEARTBEAT_TIMEOUT) { Log.w(TAG, "Heartbeat timeout!"); } else { mHeartbeatHandler.postDelayed(mHeartbeatRunnable, HEARTBEAT_TIMEOUT); } } private void scheduleRestart() { Intent restartIntent = new Intent(this, MyAccessibility.class); PendingIntent pendingIntent = PendingIntent.getService(this, 0, restartIntent, PendingIntent.FLAG_IMMUTABLE | PendingIntent.FLAG_UPDATE_CURRENT); AlarmManager alarmManager = (AlarmManager) getSystemService(Context.ALARM_SERVICE); if (alarmManager != null) { alarmManager.setInexactRepeating(AlarmManager.ELAPSED_REALTIME_WAKEUP, SystemClock.elapsedRealtime() + 60000, 60000, pendingIntent); } else { Log.e(TAG, "AlarmManager is null, cannot schedule restart."); } } private Notification createNotification() { createNotificationChannel(); // 确保在构建通知 *之前* 创建通道 Intent notificationIntent = new Intent(this, MainActivity.class); // 替换为你的实际 Activity PendingIntent pendingIntent = PendingIntent.getActivity(this, 0, notificationIntent, PendingIntent.FLAG_IMMUTABLE | PendingIntent.FLAG_UPDATE_CURRENT); return new NotificationCompat.Builder(this, CHANNEL_ID) .setContentTitle("无障碍服务") .setContentText("无障碍服务正在运行") .setSmallIcon(R.mipmap.ic_launcher) // 确保你有这个图标 .setContentIntent(pendingIntent) .setOngoing(true) .build(); } private void createNotificationChannel() { if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.O) { NotificationChannel channel = new NotificationChannel(CHANNEL_ID, "MyAccessibilityService", NotificationManager.IMPORTANCE_LOW); channel.setDescription("无障碍服务通知通道"); channel.setSound(null, null); channel.enableVibration(false); NotificationManager notificationManager = getSystemService(NotificationManager.class); if (notificationManager != null) { notificationManager.createNotificationChannel(channel); } else { Log.e(TAG, "NotificationManager is null, cannot create notification channel."); } } } @Override public int onStartCommand(Intent intent, int flags, int startId) { return START_STICKY; } @Override public void onAccessibilityEvent(AccessibilityEvent event) { updateLastEventTime(); } private void updateLastEventTime() { mLastEventTime = System.currentTimeMillis(); } @Override public void onInterrupt() { Log.w(TAG, "无障碍服务被中断"); stopServer(); stopHeartbeat(); } @Override public void onDestroy() { super.onDestroy(); Log.i(TAG, "服务被销毁"); scheduleRestart(); stopServer(); stopHeartbeat(); stopForeground(true); } private void configureAccessibilityService() { AccessibilityServiceInfo info = new AccessibilityServiceInfo(); info.eventTypes = AccessibilityEvent.TYPES_ALL_MASK; info.feedbackType = AccessibilityServiceInfo.FEEDBACK_GENERIC; info.flags = AccessibilityServiceInfo.DEFAULT | AccessibilityServiceInfo.FLAG_INCLUDE_NOT_IMPORTANT_VIEWS | AccessibilityServiceInfo.FLAG_REPORT_VIEW_IDS | AccessibilityServiceInfo.FLAG_REQUEST_TOUCH_EXPLORATION_MODE; info.notificationTimeout = 100; setServiceInfo(info); } private void startServer() { if (executorService == null || executorService.isShutdown()) { executorService = Executors.newCachedThreadPool(); Log.i(TAG, "ExecutorService 已创建"); new Thread(() -> { try (ServerSocket serverSocket = new ServerSocket(SERVER_PORT)) { Log.i(TAG, "服务器已启动,监听端口 " + SERVER_PORT); while (!Thread.currentThread().isInterrupted()) { try { Socket clientSocket = serverSocket.accept(); Log.i(TAG, "客户端已连接: " + clientSocket.getInetAddress()); clientSocket.setSoTimeout(10000); handleClient(clientSocket); } catch (SocketTimeoutException e) { Log.w(TAG, "客户端连接超时: " + e.getMessage()); } catch (SocketException e) { Log.i(TAG, "服务器socket已关闭"); break; } } } catch (IOException e) { Log.e(TAG, "服务器启动失败", e); } finally { stopServer(); } }).start(); } } private void handleClient(final Socket clientSocket) { executorService.execute(() -> { try (PrintWriter out = new PrintWriter(clientSocket.getOutputStream(), true); BufferedReader in = new BufferedReader(new InputStreamReader(clientSocket.getInputStream()))) { String inputLine; while ((inputLine = in.readLine()) != null) { processCommand(inputLine, out); updateLastEventTime(); } } catch (IOException e) { Log.e(TAG, "客户端连接异常: " + e.getMessage(), e); } finally { try { clientSocket.close(); } catch (IOException e) { Log.e(TAG, "关闭客户端连接失败", e); } } }); } private void processCommand(String command, PrintWriter out) { try { JSONObject jsonCommand = new JSONObject(command); switch (jsonCommand.getString("action")) { case "getText": processGetTextCommand(jsonCommand, out); break; case "global": processGlobalCommand(jsonCommand, out); break; case "getPageSource": getPageSource(out); break; case "performAllInOne": performAllInOne(out); break; default: sendResult(out, false, "未知操作"); } } catch (JSONException | RuntimeException e) { Log.e(TAG, "命令处理错误: " + e.getMessage(), e); sendResult(out, false, "命令处理错误: " + e.getMessage()); } } private void processGetTextCommand(JSONObject jsonCommand, PrintWriter out) throws JSONException { AccessibilityNodeInfo node = findNode(jsonCommand.getString("target")); String text = (node != null && node.getText() != null) ? node.getText().toString() : ""; sendResult(out, node != null, node != null ? text : "未找到节点"); } private void processGlobalCommand(JSONObject jsonCommand, PrintWriter out) throws JSONException { boolean success = false; switch (jsonCommand.getString("type")) { case "back": success = performGlobalAction(AccessibilityService.GLOBAL_ACTION_BACK); break; } sendResult(out, success, null); } private void sendResult(PrintWriter out, boolean success, String message) { try { JSONObject result = new JSONObject().put("result", success); if (message != null) result.put("message", message); out.println(result.toString()); } catch (JSONException e) { Log.e(TAG, "JSON 创建失败", e); out.println("{\"result\": false, \"message\": \"JSON 创建失败\"}"); } } private AccessibilityNodeInfo findNodeByTextRecursive(AccessibilityNodeInfo node, String text) { if (node == null) return null; // 检查当前节点文本 CharSequence nodeText = node.getText(); if (nodeText != null && nodeText.toString().contains(text)) { return node; } // 递归检查子节点 for (int i = 0; i < node.getChildCount(); i++) { AccessibilityNodeInfo child = node.getChild(i); AccessibilityNodeInfo found = findNodeByTextRecursive(child, text); if (found != null) { return found; } } return null; } private AccessibilityNodeInfo findNodeByText(String text) { AccessibilityNodeInfo rootNode = getRootInActiveWindow(); if (rootNode == null) return null; try { return findNodeByTextRecursive(rootNode, text); } finally { rootNode.recycle(); } } private AccessibilityNodeInfo findNodeById(String viewId) { Log.i("findNodeById", "viewId=" + viewId); AccessibilityNodeInfo rootNode = getRootInActiveWindow(); if (rootNode == null) { Log.w("findNodeById", "Root node is null, cannot find node."); return null; } try { List<AccessibilityNodeInfo> nodes = rootNode.findAccessibilityNodeInfosByViewId(viewId); if (nodes == null) { Log.w("findNodeById", "findAccessibilityNodeInfosByViewId returned null for viewId: " + viewId); return null; } if (nodes.isEmpty()) { Log.w("findNodeById", "No nodes found for viewId: " + viewId); return null; } AccessibilityNodeInfo node = nodes.get(0); Log.i("findNodeById", "Found node: " + node.toString()); // 打印节点信息 return node; } finally { rootNode.recycle(); } } private AccessibilityNodeInfo findNode(String target) { Log.i("findNode", "target=" + target ); return target.startsWith("text:") ? findNodeByText(target.substring(5)) : (target.startsWith("id:") ? findNodeById(target.substring(3)) : null); } private void stopServer() { if (executorService != null) { executorService.shutdownNow(); executorService = null; } } // 核心功能实现 /** * 获取当前界面节点结构(简化版) */ private void getPageSource(PrintWriter out) { AccessibilityNodeInfo root = getRootInActiveWindow(); if (root == null) { sendError(out, "No root node"); return; } StringBuilder sb = new StringBuilder(); buildNodeTree(root, sb, 0); sendSuccess(out, sb.toString()); root.recycle(); } /** * 递归构建节点树结构 */ private void buildNodeTree(AccessibilityNodeInfo node, StringBuilder sb, int depth) { if (node == null) return; String indent = " ".repeat(depth); sb.append(indent).append(node.getClassName()) .append(" [id:").append(node.getViewIdResourceName()) .append(", text:").append(node.getText()) .append("]\n"); for (int i = 0; i < node.getChildCount(); i++) { AccessibilityNodeInfo child = node.getChild(i); if (child != null) { buildNodeTree(child, sb, depth + 1); child.recycle(); } } } // 辅助方法 private void sendSuccess(PrintWriter out, String msg) { try { out.println(new JSONObject() .put("success", true) .put("message", msg)); } catch (JSONException e) { throw new RuntimeException(e); } } private void sendError(PrintWriter out, String error) { try { out.println(new JSONObject() .put("success", false) .put("message", error)); } catch (JSONException e) { throw new RuntimeException(e); } } public boolean performClickAt(int x, int y) { Path clickPath = new Path(); clickPath.moveTo(x, y); GestureDescription.Builder gestureBuilder = new GestureDescription.Builder(); gestureBuilder.addStroke(new GestureDescription.StrokeDescription(clickPath, 0, 100)); GestureDescription gestureDescription = gestureBuilder.build(); boolean result = dispatchGesture(gestureDescription, new GestureResultCallback() { @Override public void onCompleted(GestureDescription gestureDescription) { super.onCompleted(gestureDescription); } @Override public void onCancelled(GestureDescription gestureDescription) { super.onCancelled(gestureDescription); } }, null); return result; } public Map<String, Object> performClick(AccessibilityNodeInfo node) { Map<String, Object> result = new HashMap<>(); if (node == null) { result.put("success", false); result.put("message", "节点为 null"); return result; } boolean clickResult = node.isClickable() ? node.performAction(AccessibilityNodeInfo.ACTION_CLICK) : performClickAt(node); result.put("success", clickResult); result.put("message", clickResult ? "点击成功" : "点击失败"); return result; } private boolean performClickAt(AccessibilityNodeInfo node) { Rect bounds = new Rect(); node.getBoundsInScreen(bounds); int centerX = bounds.centerX(); int centerY = bounds.centerY(); return performClickAt(centerX, centerY); } public boolean performScrollByCoordinates(int x1, int y1, int x2, int y2, long durationMs) { Path scrollPath = new Path(); scrollPath.moveTo(x1, y1); scrollPath.lineTo(x2, y2); GestureDescription.Builder gestureBuilder = new GestureDescription.Builder(); gestureBuilder.addStroke(new GestureDescription.StrokeDescription(scrollPath, 0, durationMs)); GestureDescription gestureDescription = gestureBuilder.build(); boolean result = dispatchGesture(gestureDescription, new GestureResultCallback() { @Override public void onCompleted(GestureDescription gestureDescription) { super.onCompleted(gestureDescription); } @Override public void onCancelled(GestureDescription gestureDescription) { super.onCancelled(gestureDescription); } }, null); return result; } private void performAllInOne(PrintWriter out) { TAG = "performAllInOne"; // 1.唤醒屏幕 boolean wakeUpSuccess = wakeUpScreen(10 * 60 * 1000L); // 唤醒屏幕 10 分钟 if (wakeUpSuccess) { Log.i(TAG, "Screen successfully woken up."); } else { Log.e(TAG, "Failed to wake up screen."); } try { Thread.sleep(1000); } catch (InterruptedException ignored) { } AccessibilityNodeInfo sureButton = findNodeByText("确定"); // android.widget.Button 能直接点击 if (sureButton != null) { performClick(sureButton); } // 2. 打开搜索 launchGlobalSearch(); // 3.输入无线调试 String targetInputField = "id:com.android.quicksearchbox:id/query_text"; // 使用完整的 View ID (替换为实际的包名) 负一屏搜索 // String targetInputField = "id:android:id/input"; // 使用完整的 View ID (替换为实际的包名) String textToEnter = "无线调试"; mHandler.postDelayed(() -> { // 3. 输入文字 boolean entered = enterText(targetInputField, textToEnter); if (entered) { Log.d(TAG, "成功输入!"); } else { Log.e(TAG, "输入失败!"); } Log.i("performAllInOne", "搜索targetInputField:" + entered); }, 1000); // 3.点击"更多设置/开发者选项/无线调试" for (int i = 0; i < 20; i++) { AccessibilityNodeInfo switchNode = findNodeByText("更多设置/开发者选项/无线调试"); if (switchNode != null) { try { Thread.sleep(1000); } catch (InterruptedException ignored) {} // android.widget.TextView 不能直接点击要找父节点 performIntelligentClick(switchNode); switchNode.recycle(); break; } try { Thread.sleep(1000); } catch (InterruptedException ignored) {} } try { Thread.sleep(2000); } catch (InterruptedException ignored) { } Log.i("performAllInOne", "准备打开无线调试"); mHandler.postDelayed(() -> { AccessibilityNodeInfo wuxianNode = findNodeById("android:id/switch_widget"); Log.i("performAllInOne", "无线调试按钮" + wuxianNode); if (wuxianNode != null) { // android.widget.TextView 不能直接点击要找父节点 performIntelligentClick(wuxianNode); wuxianNode.recycle(); } }, 2000); // android:id/switch_widget try { Thread.sleep(2000); } catch (InterruptedException ignored) { } mHandler.postDelayed(() -> { // android:id/button1 AccessibilityNodeInfo cancelButton = findNodeByText("确定"); Log.i("performAllInOne", "确定按钮" + cancelButton); // android.widget.Button 能直接点击 if (cancelButton != null) { performClick(cancelButton); } }, 1000); // processGetTextCommand("id:android:id/summary", "ip端口号") try { Thread.sleep(4000); } catch (InterruptedException ignored) { } // 获取ip和端口号 Log.i("performAllInOne", "准备获取IP 地址和端口"); AccessibilityNodeInfo targetNode = findNodeByText("IP 地址和端口"); if (targetNode != null) { List<AccessibilityNodeInfo> allSiblings = getAllSiblings(targetNode); // ... 过滤 allSiblings, 获取 followingSiblings ... try{ for(AccessibilityNodeInfo sibling: allSiblings){ //... String siblingText = (String) sibling.getText(); Log.i("performAllInOne", "ip_port1:" + siblingText); if (siblingText != null){ if (siblingText.contains(":")) { Log.i("performAllInOne", "ip_port2:" + siblingText); sendSuccess(out, siblingText); sibling.recycle(); break; // 停止循环 } } } } finally { targetNode.recycle(); } } Log.i("performAllInOne", "准备获取配对码"); AccessibilityNodeInfo pei_dui = findNodeByText("使用配对码配对设备"); performIntelligentClick(pei_dui); pei_dui.recycle(); try { Thread.sleep(3000); } catch (InterruptedException ignored) { } // 获取配对 AccessibilityNodeInfo pair_ip_port = findNodeById("com.android.settings:id/ip_addr"); Log.i("performAllInOne", "ip_port:" + pair_ip_port); // sendSuccess(out, (String) pair_ip_port.getText()); AccessibilityNodeInfo pairing_code = findNodeById("com.android.settings:id/pairing_code"); Log.i("performAllInOne", "ip_port:" + pairing_code); sendSuccess(out, pair_ip_port.getText() + " " + pairing_code.getText()); // } public static List<AccessibilityNodeInfo> getAllSiblings(AccessibilityNodeInfo targetNode) { List<AccessibilityNodeInfo> siblings = new ArrayList<>(); if (targetNode == null) { return siblings; } AccessibilityNodeInfo parent = targetNode.getParent(); if (parent == null) { return siblings; } try{ int childCount = parent.getChildCount(); for(int i = 0; i < childCount; i++){ AccessibilityNodeInfo child = parent.getChild(i); if(child == null) continue; if(!child.equals(targetNode)){ siblings.add(child); } else { child.recycle(); // 回收目标节点本身. } } } finally { parent.recycle(); } return siblings; } private void launchGlobalSearch() { Intent intent = new Intent(SearchManager.INTENT_ACTION_GLOBAL_SEARCH); intent.addFlags(Intent.FLAG_ACTIVITY_NEW_TASK); try { startActivity(intent); } catch (ActivityNotFoundException e) { Log.e(TAG, "No global search activity found", e); } } public boolean wakeUpScreen(long durationMs) { TAG = "wakeUpScreen"; PowerManager powerManager = (PowerManager) getSystemService(Context.POWER_SERVICE); if (powerManager == null) { Log.e(TAG, "wakeUpScreen: PowerManager is null."); return false; } PowerManager.WakeLock wakeLock = null; try { wakeLock = powerManager.newWakeLock( PowerManager.FULL_WAKE_LOCK | PowerManager.ACQUIRE_CAUSES_WAKEUP | PowerManager.ON_AFTER_RELEASE, "MyApp:WakeLock"); wakeLock.acquire(durationMs > 0 ? durationMs : 1); // 至少 Acquire 1 毫秒 Log.i(TAG, "wakeUpScreen: Acquired WakeLock."); if (durationMs > 0) { // 如果指定了 durationMs, 则会自动在 durationMs 后释放 return true; } else { //需要手动 release Log.w(TAG, "wakeUpScreen: WakeLock 需要手动 release."); return true; // 需要调用者手动释放。 } } catch (SecurityException e) { Log.e(TAG, "wakeUpScreen: SecurityException - Missing WAKE_LOCK permission?", e); return false; } catch (Exception e) { Log.e(TAG, "wakeUpScreen: Exception during WakeLock acquisition.", e); return false; } finally { if(durationMs > 0 && wakeLock != null && wakeLock.isHeld()) { wakeLock.release(); Log.i(TAG, "wakeUpScreen: Released WakeLock in finally block."); } } } private void performIntelligentClick(AccessibilityNodeInfo node) { AccessibilityNodeInfo clickableAncestor = findClickableAncestor(node); if (clickableAncestor != null) { performClick(clickableAncestor); clickableAncestor.recycle(); } else { performClickAt(node); } } private AccessibilityNodeInfo findClickableAncestor(AccessibilityNodeInfo node) { AccessibilityNodeInfo parent = node.getParent(); while (parent != null) { if (parent.isClickable()) { return parent; } AccessibilityNodeInfo temp = parent.getParent(); parent.recycle(); parent = temp; } return null; } public boolean setText(AccessibilityNodeInfo node, String text) { if (node == null) { Log.w(TAG, "setText: Node is null, cannot set text."); return false; } if (!node.isFocusable()) { Log.w(TAG, "setText: Node is not focusable."); return false; } if (!node.isEnabled()) { Log.w(TAG, "setText: Node is not enabled."); return false; } if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.LOLLIPOP) { boolean focusResult = node.performAction(AccessibilityNodeInfo.ACTION_FOCUS); if (!focusResult) { Log.w(TAG, "setText: Failed to request focus."); } else { Log.i(TAG, "setText: Successfully requested focus."); } Bundle arguments = new Bundle(); arguments.putCharSequence(AccessibilityNodeInfo.ACTION_ARGUMENT_SET_TEXT_CHARSEQUENCE, text); boolean result = node.performAction(AccessibilityNodeInfo.ACTION_SET_TEXT, arguments); Log.i(TAG, "setText: Set text result: " + result); return result; } else { Log.w(TAG, "setText: ACTION_SET_TEXT requires API Level 21+"); return false; } } public boolean enterText(String target, String text) { // 输入 AccessibilityNodeInfo inputNode = findNode(target); if (inputNode == null) { Log.w(TAG, "enterText: Input node not found for target: " + target); return false; } Log.w(TAG, "enterText: Input target: " + target); boolean setTextResult = setText(inputNode, text); if (setTextResult) { Log.i(TAG, "enterText: Successfully entered text '" + text + "' into target: " + target); return true; } else { Log.e(TAG, "enterText: Failed to enter text '" + text + "' into target: " + target); return false; } } private Point getScreenSize(Context context) { WindowManager wm = (WindowManager) context.getSystemService(Context.WINDOW_SERVICE); if(wm == null) return new Point(0,0); Display display = wm.getDefaultDisplay(); Point size = new Point(); display.getRealSize(size); return size; } private void startHeartbeat() { mLastEventTime = System.currentTimeMillis(); mHeartbeatHandler.postDelayed(mHeartbeatRunnable, HEARTBEAT_TIMEOUT); } private void stopHeartbeat() { mHeartbeatHandler.removeCallbacks(mHeartbeatRunnable); } }
- 关键方法解释:
onServiceConnected()
: 服务连接成功后调用,初始化并启动服务器。onAccessibilityEvent()
: 处理无障碍事件(本例中留空)。onInterrupt()
: 服务被中断时调用,停止服务器。configureAccessibilityService()
: 配置 AccessibilityService 的基本信息,如事件类型、反馈类型等。startServer()
: 启动 TCP 服务器,监听客户端连接。handleClient()
: 处理客户端连接,读取指令并调用processCommand()
。processCommand()
: 解析 JSON 指令,根据指令类型执行相应的操作。handleFindNode()
: 处理findNode
的结果。findNodeByText()
,findNodeById()
: 根据文本或 ID 查找节点。performClick()
,performLongClick()
,performInput()
,performGlobalActionBack()
: 执行具体操作。sendResult()
: 将操作结果以 JSON 格式发送回客户端。stopServer()
: 停止服务器,关闭连接和线程池。
- 关键方法解释:
-
创建
accessibility_service_config.xml
:<!-- ./app/src/main/res/xml/accessibility_service_config.xml --> <?xml version="1.0" encoding="utf-8"?> <accessibility-service xmlns:android="http://schemas.android.com/apk/res/android" android:accessibilityEventTypes="typeAllMask" android:accessibilityFeedbackType="feedbackGeneric" android:accessibilityFlags="flagDefault|flagIncludeNotImportantViews|flagReportViewIds" android:canRetrieveWindowContent="true" android:description="@string/accessibility_service_description" android:notificationTimeout="100" android:packageNames="" />
android:packageNames=""
: 空字符串表示监听所有应用的事件。如果只想监听特定应用,可以填入应用的包名,多个包名用逗号分隔。
-
在
AndroidManifest.xml
中声明服务:<!-- ./app/src/main/AndroidManifest.xml --> <?xml version="1.0" encoding="utf-8"?> <manifest xmlns:android="http://schemas.android.com/apk/res/android" xmlns:tools="http://schemas.android.com/tools"> <uses-permission android:name="android.permission.FOREGROUND_SERVICE" /> <uses-permission android:name="android.permission.FOREGROUND_SERVICE_LOCATION"/> <uses-permission android:name="android.permission.INTERNET" /> <!-- <uses-permission android:name="android.permission.RECEIVE_BOOT_COMPLETED"/>--> <uses-permission android:name="android.permission.WAKE_LOCK" /> <uses-permission android:name="android.permission.ACCESS_NETWORK_STATE" /> <application android:allowBackup="true" android:icon="@mipmap/ic_launcher" android:label="@string/app_name" android:roundIcon="@mipmap/ic_launcher_round" android:supportsRtl="true" android:theme="@style/Theme.MyAccessibility"> <activity android:name=".MainActivity" android:exported="true"> <intent-filter> <action android:name="android.intent.action.MAIN" /> <category android:name="android.intent.category.LAUNCHER" /> </intent-filter> </activity> <service android:name=".MyAccessibility" android:label="@string/accessibility_service_label" android:permission="android.permission.BIND_ACCESSIBILITY_SERVICE" android:exported="true" tools:ignore="MissingClass"> <intent-filter> <action android:name="android.accessibilityservice.AccessibilityService" /> </intent-filter> <meta-data android:name="android.accessibilityservice" android:resource="@xml/accessibility_service_config" /> </service> </application> </manifest>
- 关键点:
<uses-permission android:name="android.permission.INTERNET" />
: 允许应用使用网络。<service ...>
: 声明 AccessibilityService。android:name=".MyAccessibility"
: 指定服务的类名。android:permission="android.permission.BIND_ACCESSIBILITY_SERVICE"
: 声明需要绑定无障碍服务的权限。android:resource="@xml/accessibility_service_config"
: 指向服务的配置文件。
- 关键点:
- strings.xml
<!-- .\app\src\main\res\values\strings.xml -->
<resources>
<string name="app_name">MyAccessibilityApp</string>
<string name="accessibility_service_label">我的Accessibility</string>
<string name="accessibility_service_description">This service provides automation capabilities.</string>
<string name="developer_name">by永生</string>
<string name="accessibility_service_status">无障碍服务(%1$s) 是否开启: %2$s</string>
</resources>
-
(可选) 创建一个简单的 MainActivity.kt (如果你使用了 Empty Activity 模板):
// ./app/src/main/java/com/example/myaccessibility/MainActivity.kt package com.example.myaccessibility import android.accessibilityservice.AccessibilityServiceInfo import android.content.Context import android.content.Intent import android.os.Build import android.os.Bundle import android.provider.Settings import android.view.accessibility.AccessibilityManager import androidx.activity.ComponentActivity import androidx.activity.compose.setContent import androidx.activity.enableEdgeToEdge import androidx.compose.foundation.layout.Arrangement import androidx.compose.foundation.layout.Column import androidx.compose.foundation.layout.fillMaxSize import androidx.compose.foundation.layout.padding import androidx.compose.material3.Button import androidx.compose.material3.MaterialTheme import androidx.compose.material3.Scaffold import androidx.compose.material3.Text import androidx.compose.runtime.Composable import androidx.compose.runtime.getValue import androidx.compose.runtime.mutableStateOf import androidx.compose.runtime.remember import androidx.compose.runtime.setValue import androidx.compose.ui.Alignment import androidx.compose.ui.Modifier import androidx.compose.ui.platform.LocalContext import androidx.compose.ui.res.stringResource import androidx.compose.ui.tooling.preview.Preview import androidx.compose.ui.unit.dp import com.example.myaccessibility.ui.theme.MyAccessibilityTheme class MainActivity : ComponentActivity() { override fun onCreate(savedInstanceState: Bundle?) { super.onCreate(savedInstanceState) enableEdgeToEdge() setContent { MyAccessibilityTheme { Scaffold(modifier = Modifier.fillMaxSize()) { innerPadding -> SystemInfoScreen(modifier = Modifier.padding(innerPadding)) } } } } private fun isAccessibilityServiceEnabled(context: Context, serviceClass: Class<*>): Boolean { val am = context.getSystemService(Context.ACCESSIBILITY_SERVICE) as AccessibilityManager val enabledServices = am.getEnabledAccessibilityServiceList(AccessibilityServiceInfo.FEEDBACK_ALL_MASK) for (enabledService in enabledServices) { val serviceInfo = enabledService.resolveInfo.serviceInfo if (serviceInfo.packageName == context.packageName && serviceInfo.name == serviceClass.name) return true } return false } @Composable fun SystemInfoScreen(modifier: Modifier = Modifier) { val context = LocalContext.current var isAdbEnabled by remember { mutableStateOf(isAdbEnabled(context)) } // 动态检查无障碍服务状态,getValue,setValue var isAccessibilityEnabled by remember { mutableStateOf( isAccessibilityServiceEnabled( context, MyAccessibility::class.java ) ) } val minSdkVersion = remember { if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.N) { context.packageManager.getApplicationInfo(context.packageName, 0).minSdkVersion } else { context.packageManager.getPackageInfo( context.packageName, 0 ).applicationInfo?.targetSdkVersion ?: 0 } } Column( modifier = modifier .fillMaxSize() .padding(16.dp), horizontalAlignment = Alignment.CenterHorizontally, // 水平居中 verticalArrangement = Arrangement.Center // 垂直居中 ) { // 显示构建此应用所使用的 Java 兼容性级别 (提示用户查看 build.gradle.kts) Text( text = "构建此应用的 Java 兼容性级别: 请查看 build.gradle.kts 文件中的 compileOptions,若无障碍服务打不开请重启手机", style = MaterialTheme.typography.bodyLarge ) Text( text = "Kotlin 版本: ${KotlinVersion.CURRENT}", style = MaterialTheme.typography.bodyLarge ) Text( text = "最低支持 Android 版本: $minSdkVersion", // 显示格式化后的 minSdkVersion style = MaterialTheme.typography.bodyLarge ) Text( text = "Android API 级别: ${Build.VERSION.SDK_INT}", style = MaterialTheme.typography.bodyLarge ) Text( text = "Android 版本: ${Build.VERSION.RELEASE}", style = MaterialTheme.typography.bodyLarge ) Text( text = "ADB 调试已开启: $isAdbEnabled", style = MaterialTheme.typography.bodyLarge ) // 使用 stringResource 和占位符 Text( text = stringResource( R.string.accessibility_service_status, stringResource(R.string.accessibility_service_label), if (isAccessibilityEnabled) "是" else "否" ), style = MaterialTheme.typography.bodyLarge ) Text( text = stringResource(R.string.developer_name), // 使用 stringResource style = MaterialTheme.typography.bodyLarge ) // 添加按钮,用于跳转到无障碍设置 Button( onClick = { // 打开无障碍设置界面 val intent = Intent(Settings.ACTION_ACCESSIBILITY_SETTINGS) context.startActivity(intent) }, enabled = !isAccessibilityEnabled // 根据服务状态启用/禁用按钮 ) { Text(text = "开启无障碍服务") } } } private fun isAdbEnabled(context: Context): Boolean { return Settings.Global.getInt(context.contentResolver, Settings.Global.ADB_ENABLED, 0) == 1 } // 将 API 级别转换为 Android 版本名称的辅助函数 private fun getAndroidVersionName(apiLevel: Int): String { return when (apiLevel) { 1 -> "Android 1.0 (API 1)" 2 -> "Android 1.1 (API 2)" 3 -> "Android 1.5 (Cupcake, API 3)" 4 -> "Android 1.6 (Donut, API 4)" 5 -> "Android 2.0 (Eclair, API 5)" 6 -> "Android 2.0.1 (Eclair, API 6)" 7 -> "Android 2.1 (Eclair, API 7)" 8 -> "Android 2.2 (Froyo, API 8)" 9 -> "Android 2.3 (Gingerbread, API 9)" 10 -> "Android 2.3.3 (Gingerbread, API 10)" 11 -> "Android 3.0 (Honeycomb, API 11)" 12 -> "Android 3.1 (Honeycomb, API 12)" 13 -> "Android 3.2 (Honeycomb, API 13)" 14 -> "Android 4.0 (Ice Cream Sandwich, API 14)" 15 -> "Android 4.0.3 (Ice Cream Sandwich, API 15)" 16 -> "Android 4.1 (Jelly Bean, API 16)" 17 -> "Android 4.2 (Jelly Bean, API 17)" 18 -> "Android 4.3 (Jelly Bean, API 18)" 19 -> "Android 4.4 (KitKat, API 19)" 20 -> "Android 4.4W (KitKat Wear, API 20)" 21 -> "Android 5.0 (Lollipop, API 21)" 22 -> "Android 5.1 (Lollipop, API 22)" 23 -> "Android 6.0 (Marshmallow, API 23)" 24 -> "Android 7.0 (Nougat, API 24)" 25 -> "Android 7.1 (Nougat, API 25)" 26 -> "Android 8.0 (Oreo, API 26)" 27 -> "Android 8.1 (Oreo, API 27)" 28 -> "Android 9 (Pie, API 28)" 29 -> "Android 10 (API 29)" 30 -> "Android 11 (API 30)" 31 -> "Android 12 (API 31)" 32 -> "Android 12L (API 32)" 33 -> "Android 13 (API 33)" 34 -> "Android 14 (API 34)" 35 -> "Android 15 (API 35)" // 根据需要更新 else -> "未知 Android 版本 (API $apiLevel)" } } @Preview(showBackground = true) @Composable fun SystemInfoScreenPreview() { MyAccessibilityTheme { SystemInfoScreen() } } }
- 这个 MainActivity 只是一个简单的占位符,你可以根据需要修改它。
8. Node.js 客户端实现
// client.js
const net = require('net');
const { exec } = require('child_process');
const util = require('util');
const execPromise = util.promisify(exec);
const commandsToSend = [
{ action: 'performAllInOne' },
// { action: 'getPageSource' }, // 获取页面源码
// 其他命令...
];
let responsesReceived = 0;
const results = [];
// 获取命令行参数中的 IP 地址,如果没有提供,则使用默认值
// 注意:这里仍然建议你通过命令行参数传入 IP 地址,而不是依赖 results 数组。
const host = process.argv[2] || '192.168.31.40';
const port = 6000;
const client = net.createConnection({ port: port, host: host }, () => {
console.log(`Connected to AccessibilityService at ${host}:${port}!`);
commandsToSend.forEach(command => sendCommand(command));
});
// ... (client.on('data'), client.on('end'), client.on('error'), sendCommand 与之前类似) ...
client.on('data', (data) => {
try {
const result = JSON.parse(data.toString());
console.log('Received from AccessibilityService:', result);
if (result.message) {
console.log('node输出(result.message):', result.message);
results.push(result.message);
}
responsesReceived++;
if(responsesReceived === commandsToSend.length){
client.end();
}
} catch (error) {
console.error('Error parsing JSON:', error);
console.log('Raw data received:', data.toString());
}
});
client.on('end', () => {
console.log('Disconnected from AccessibilityService');
processResponses(); // 在连接断开 *后* 执行
});
client.on('error', (err) => {
console.error('Connection error:', err);
});
function sendCommand(command) {
const commandString = JSON.stringify(command) + '\n';
client.write(commandString, err => {
if (err) {
console.error('Error sending command:', err);
} else {
console.log('Command sent:', commandString.trim());
}
});
}
async function processResponses() {
if (results.length < 2) {
console.error('Not enough responses received.');
return;
}
const pairingCode = results[results.length - 1];
const ipAndPort = results[results.length - 2];
// 错误的 adb pair 命令 (只使用配对码)
const pairCommand = `adb pair ${pairingCode}`;
try {
console.log("Executing command:", pairCommand);
const { stdout, stderr } = await execPromise(pairCommand);
console.log(`adb pair stdout: ${stdout}`);
console.error(`adb pair stderr: ${stderr}`);
// adb connect (使用完整的 ipAndPort)
const connectCommand = `adb connect ${ipAndPort}`;
console.log("Executing command:", connectCommand);
const { stdout: stdout2, stderr: stderr2 } = await execPromise(connectCommand);
console.log(`adb connect stdout: ${stdout2}`);
console.error(`adb connect stderr: ${stderr2}`);
// adb -s <device> tcpip 5555
const tcpipCommand = `adb -s ${ipAndPort} tcpip 5555`;
console.log("Executing command:", tcpipCommand);
const { stdout: stdout3, stderr: stderr3 } = await execPromise(tcpipCommand);
console.log(`adb tcpip stdout: ${stdout3}`);
console.error(`adb tcpip stderr: ${stderr3}`);
console.log("All ADB commands executed.");
} catch (error) {
console.error(`Error executing ADB command: ${error}`);
}
}
- 代码解释:把接收数据存入数组,进行adb pair connect tcpip 等操作
net.createConnection()
: 创建到 AccessibilityService 的 TCP 连接。port
: AccessibilityService 监听的端口 (6000)。host
: Android 设备的 IP 地址。
commandsToSend
: 要发送的命令数组,每个命令都是一个 JSON 对象。client.on('data', ...)
: 处理从 AccessibilityService 接收到的数据。- 解析 JSON 数据。
- 递增
responsesReceived
计数器。 - 如果收到了所有响应,则关闭连接。
client.on('end', ...)
: 处理连接关闭事件。client.on('error', ...)
: 处理连接错误事件。sendCommand()
: 将 JSON 命令转换为字符串并发送到 AccessibilityService。 重要: 在命令字符串末尾添加换行符 (\n
),因为 AccessibilityService 使用BufferedReader.readLine()
读取数据。
9. 运行与测试
-
在 Android 设备上安装并运行应用。
-
启用 AccessibilityService:
- 进入 "设置" -> "无障碍" (或 "辅助功能")。
- 找到 "MyAccessibility" 服务并启用它。
-
获取 Android 设备的 IP 地址。 (通常在 "设置" -> "关于手机" -> "状态信息" -> "IP 地址" 中)
-
修改
client.js
中的host
为你的 Android 设备的 IP 地址。 -
在 PC 上运行 Node.js 客户端:
node client.js
10. 低版本 Android 系统问题与解决方案
在 Android 10 (API 29) 及更低版本上,AccessibilityService 可能会遇到一个问题:在安装或更新应用后,即使在设置中启用了服务,服务也可能无法立即正常工作,需要重启手机才能生效。
原因:
这是由于低版本 Android 系统中 AccessibilityService 的生命周期管理机制与较高版本不同。在低版本中,系统可能不会立即加载或初始化新安装或更新的 AccessibilityService,导致服务无法接收事件或执行操作。
解决方案:
-
提高
minSdk
和targetSdk
(不推荐,仅用于演示):- 如果你将
build.gradle.kts
中的minSdk
和targetSdk
设置为 29,你 应该 能够复现这个问题。 这只是为了演示问题本身。 实际上 不推荐 将你的应用限制到这么低的版本。
- 如果你将
-
使用 Java 11 和 JVM 11 (或更高版本):
- 如
build.gradle.kts
中所示,将sourceCompatibility
、targetCompatibility
和jvmTarget
设置为 11 或更高版本。 经验表明,使用较新的 Java 和 JVM 版本可以解决这个问题,无需
- 如
11. 注意事项
-
无障碍服务未启动时的处理:
- 如果在设备上没有启动
AccessibilityService
,可以通过观察logcat
输出中的 “无障碍服务准备连接” 提示来确认服务是否已经启动。如果没有看到相关日志提示,可以尝试重启手机。 - 如果无障碍服务未启动,Node.js 客户端会报错,例如:
Connection error: Error: connect ECONNREFUSED 192.168.31.81:6000
。这是因为客户端尝试连接到 Android 设备时,连接被拒绝(ECONNREFUSED
),意味着没有服务在指定端口监听。
- 如果在设备上没有启动
-
重启手机:
- 若在
logcat
中没有看到无障碍服务相关日志,且重启手机后仍然无法连接,建议尝试重启手机,确保无障碍服务已正确启动,并且监听端口。
- 若在
-
调试提示:
- 在调试过程中,可以通过
logcat
观察 Android 设备的日志,确保无障碍服务正常启动并接收到命令。如果服务没有启动或出现异常,可以查看日志来获取更多信息。
- 在调试过程中,可以通过
通过以上步骤,可以确保无障碍服务正确启动,并且 Node.js 客户端能够成功连接到 Android 设备,实现手机自动化操作。
12. 未解决问题
- 一段时间无障碍服务会挂,重启手机才可以。而且和AXT有冲突,打开atx服务就挂了(待解决)
本文作者: 永生
本文链接: http://yys.zone:8080/detail/?id=375
版权声明: 本博客所有文章除特别声明外,均采用 CC BY-NC-SA 4.0 许可协议。转载请注明出处!
评论列表 (0 条评论)