ui-automation
pywinauto
Windows GUI Automation with Python (based on text properties)
- 6.5k
- Python
- BSD-3-Clause
- 2 wk ago
OpenAdapt
For EHRs, payer portals, and any app your team clicks through. A person or an agent shows the task; OpenAdapt turns it into a local program that repeats the steps with no model calls, checks each save, and stops when something doesn't match.
- 1.9k
- Python
- MIT
- 2 h ago
ios-mcp
让 Claude、Codex、Cursor 等 AI 助手看懂 iPhone 屏幕并直接操作手机的 iOS 自动化工具与 MCP Server,支持 UI 树获取、元素定位、截图与 OCR 识别、点击滑动、文字输入、应用管理和日志读取,适用于自动化测试与 iOS 逆向调试
- 698
- Objective-C
- MIT
- 8 wk ago
Interceptor
Browser automation and computer use for AI agents. One CLI and MCP server for your signed-in browser, native macOS apps, and real iPhones.
- 637
- TypeScript
- 4 d ago
assistant
🤖 Bring your Figma design & development pipeline to the next level - with design to code, in-design-content-management, component management, tools for faster design
- 619
- TypeScript
- 3 y ago
Droidline
Android automation from code. Test your apps on real phones, script the tasks you repeat, run a shelf of phones, or let an AI agent use one, from Python, Node.js, the CLI, HTTP or Appium clients. No ADB, no USB cable, no root.
- 558
- Go
- MIT
- 3 wk ago
sandbox-mcp
Fresh, throwaway Windows machines for testing your apps: an MCP server and CLI that automate Windows Sandbox
- 496
- JavaScript
- MIT
- 7 d ago
dsh-ios
DeepSeek Harness (DSH) plugin: a live iOS Simulator — and a USB-connected iPhone — inside the conversation. 22 agent tools for booting, building, driving the UI by accessibility identity, OCR text or list rows, plus a streaming sidebar panel you can tap and drag on.
- 476
- TypeScript
- MIT
- 12 d ago
dsh-android
DeepSeek Harness plugin for Android — build, run, and interact with a live emulator or USB device stream inside a conversation, driven entirely through adb.
- 362
- TypeScript
- MIT
- 12 d ago
phonedriver
MCP server and CLI that lets AI agents (Claude Code, Codex, Cursor) see and control Android phones and iPhones. Reads the screen as text. Asks before it pays.
- 301
- Rust
- AGPL-3.0
- 2 d ago
Jarvis
An autonomous, local-first on-device AI agent and Android OS automation framework featuring full accessibility control, │ continuous voice intelligence, and self-evolving workflow execution.
- 141
- Dart
- MIT
- 2 wk ago
ruyi-workbench-oss
开源 Windows 本地 AI 工作台|OpenAI-compatible + Claude CLI,99 个桌面/Office 工具,可回退文件操作与可视化多 Agent 工作流。
- 50
- JavaScript
- Apache-2.0
- 2 d ago
PC-Control
Let an AI see and operate Windows apps cheaply, without stealing focus or moving your mouse. MCP server for Claude Code.
- 37
- Python
- MIT
- 4 h ago
desktop-touch-mcp
Windows computer-use MCP server: drive any desktop app via semantic discover-then-act targeting (entities + leases, not pixel coordinates), with per-action perception guards, a native Rust UIA engine, Chrome CDP, and Key Locker credential autofill for ssh/sudo password prompts. Works with Claude, Cursor, and any MCP client.
- 27
- TypeScript
- MIT
- 9 h ago
app-auditor-plugin-claude
Claude plugin for auditing live web apps: capture screenshots, generate demo videos, document components, extract design tokens, map shadcn/ui, and find UI inconsistencies. Built by Legion Code Inc.
- 19
- HTML
- AGPL-3.0
- 4 wk ago
dsh-qa
DeepSeek Harness plugin that does QA: an agent explores your app like a real user and leaves evidence, then the explored path is exported as a deterministic replay scenario you run every release. Orchestrates the browser, macOS desktop, iOS and Android drivers — never a false green.
- 12
- JavaScript
- MIT
- 8 wk ago
pta-text-0.1
- 13
- 0
- gpl-3.0
- 3 y ago
AutoUI
📱 AI 零代码快准稳 UI 自动化测试平台 🚀 3 像素内自动精准定位,2 毫秒内自动精准等待,自动断言 UI 和数据、界定前/后端 bug,一键导出场景接口用例。用户包含腾讯,应微信团队邀请分享了 零代码测试工具与实践(API•单元•UI)
- 11
- Java
- Apache-2.0
- 2 wk ago
sidehand
MCP server and CLI that let AI agents (Claude, Codex, Cursor) operate Windows apps behind your window, without taking your mouse or keyboard.
- 8
- PowerShell
- 4 d ago
dsh-browser
DeepSeek Harness plugin for browser automation, headless first. Every agent gets its own throwaway Chrome/Edge/Chromium profile, short-lived semantic references instead of CSS selectors, scoped subtree observation, deterministic high-risk refusal, and bounded evidence.
- 8
- JavaScript
- MIT
- 8 wk ago
legends-jev
legends-jev: Python OCR, original-image screenshot pixel coordinates, Street View storefront signage verification & UI action advisor CLI for coding agents
- 5
- Python
- MIT
- 23 h ago
dockswipe
Synthesize macOS trackpad dock-swipe gestures from the CLI — trigger Mission Control, switch Spaces, App Exposé, Show Desktop & Launchpad through the real gesture pathway, with controllable speed. Built for end-to-end UI test automation.
- 4
- Objective-C
- MIT
- 5 mo ago
reflex-computer-use
Computer use MCP server for Claude Code, Codex and other AI agents. Reads the UI tree instead of taking a screenshot every step, with Jev (TypeSafe System One) as a fast judge that can also look at the screen to find icons and game objects. Local macOS, local Windows, or remote Windows over SSH.
- 3
- Python
- MIT
- 2 wk ago
POK-Agent
POK-Agent: a Windows computer-use AI agent. A large model plans (System 2) and a small fast model acts on screen (System 1: JEV or local Laya). Repeated tasks become muscle memory, so the big model is called less every time. Local-first, voice and vision, Rust + Tauri, open source.
- 3
- Rust
- Apache-2.0
- 2 wk ago
dsh-computer
DeepSeek Harness plugin for macOS desktop automation. Observes the accessibility tree, gates risky actions behind one-time approval, permanently refuses secure fields, and returns bounded visual evidence — through a native helper you build and grant locally.
- 3
- JavaScript
- MIT
- 8 wk ago
java-swing-mcp
Embedded MCP server that gives AI agents structured access to Java Swing applications
- 3
- Java
- MIT
- 7 mo ago
SmudgeAI
Windows-native autonomous AI agent for desktop automation with vision-based UI navigation, multi-step workflow execution, and LLM-powered reasoning.
- 3
- Python
- 8 mo ago
numalarm
牛马铃 numalarm —— Windows 下基于桌面 UI 自动化的 QQ 语音通话提醒工具与 Agent Skill:AI Agent 任务完成/中断/等待确认时自动拨打 QQ 电话强提醒;无人接听自动重拨、接听后播报语音(离线 TTS+虚拟声卡)、用户在电脑前自动静默。提供 Python SDK / HTTP API / CLI / 桌面快捷方式与打断 Hook 接入。
- 2
- Python
- MIT
- 7 wk ago
open-computer-use
Open-source computer-use AI agent for macOS: drives Mac apps through the accessibility tree with an indexed action space. The model picks from a numbered table of real elements; screenshots only as a fallback. Any OpenAI-compatible model or Jev. Port of browser-use/jev-ultrafast.
- 1
- Python
- MIT
- 3 d ago
knightsight
Share one window with your coding assistant over MCP. It looks, clicks and types there, and nowhere else.
- 1
- Python
- AGPL-3.0
- 1 d ago
Gliese-CUA-Tool-Call-8B-Demo
A Gradio-based demonstration for the prithivMLmods/Gliese-CUA-Tool-Call-8B model, a Computer Use Agent (CUA) specialized in GUI understanding and tool-calling actions.
- 1
- Python
- Apache-2.0
- 11 mo ago














