ui-automation

Open repos and models their makers tagged ui-automation, most starred first. The same for everyone.

AI agentsRepo

OpenAdapt

For EHRs, payer portals, and any app your team clicks through. A person or an agent shows the task; OpenAdapt turns it into a local program that repeats the steps with no model calls, checks each save, and stops when something doesn't match.

  • 1.9k
  • Python
  • MIT
  • 2 h ago
OpenAdaptAITake it
AI agentsRepo

ios-mcp

让 Claude、Codex、Cursor 等 AI 助手看懂 iPhone 屏幕并直接操作手机的 iOS 自动化工具与 MCP Server,支持 UI 树获取、元素定位、截图与 OCR 识别、点击滑动、文字输入、应用管理和日志读取,适用于自动化测试与 iOS 逆向调试

  • 698
  • Objective-C
  • MIT
  • 8 wk ago
witchanTake it
GitHubRepo

assistant

🤖 Bring your Figma design & development pipeline to the next level - with design to code, in-design-content-management, component management, tools for faster design

  • 619
  • TypeScript
  • 3 y ago
gridacoTake it
AI agentsRepo

dsh-ios

DeepSeek Harness (DSH) plugin: a live iOS Simulator — and a USB-connected iPhone — inside the conversation. 22 agent tools for booting, building, driving the UI by accessibility identity, OCR text or list rows, plus a streaming sidebar panel you can tap and drag on.

  • 476
  • TypeScript
  • MIT
  • 12 d ago
ZSeven-WTake it
AI agentsRepo

dsh-android

DeepSeek Harness plugin for Android — build, run, and interact with a live emulator or USB device stream inside a conversation, driven entirely through adb.

  • 362
  • TypeScript
  • MIT
  • 12 d ago
ZSeven-WTake it
AI agentsRepo

phonedriver

MCP server and CLI that lets AI agents (Claude Code, Codex, Cursor) see and control Android phones and iPhones. Reads the screen as text. Asks before it pays.

  • 301
  • Rust
  • AGPL-3.0
  • 2 d ago
hridaya423Take it
AI agentsRepo

desktop-touch-mcp

Windows computer-use MCP server: drive any desktop app via semantic discover-then-act targeting (entities + leases, not pixel coordinates), with per-action perception guards, a native Rust UIA engine, Chrome CDP, and Key Locker credential autofill for ssh/sudo password prompts. Works with Claude, Cursor, and any MCP client.

  • 27
  • TypeScript
  • MIT
  • 9 h ago
Harusame64Take it
AI agentsRepo

dsh-qa

DeepSeek Harness plugin that does QA: an agent explores your app like a real user and leaves evidence, then the explored path is exported as a deterministic replay scenario you run every release. Orchestrates the browser, macOS desktop, iOS and Android drivers — never a false green.

  • 12
  • JavaScript
  • MIT
  • 8 wk ago
ZSeven-WTake it
GitHubRepo

AutoUI

📱 AI 零代码快准稳 UI 自动化测试平台 🚀 3 像素内自动精准定位,2 毫秒内自动精准等待,自动断言 UI 和数据、界定前/后端 bug,一键导出场景接口用例。用户包含腾讯,应微信团队邀请分享了 零代码测试工具与实践(API•单元•UI)

  • 11
  • Java
  • Apache-2.0
  • 2 wk ago
TommyLemonTake it
AI agentsRepo

sidehand

MCP server and CLI that let AI agents (Claude, Codex, Cursor) operate Windows apps behind your window, without taking your mouse or keyboard.

  • 8
  • PowerShell
  • 4 d ago
Blue-BTake it
AI agentsRepo

dsh-browser

DeepSeek Harness plugin for browser automation, headless first. Every agent gets its own throwaway Chrome/Edge/Chromium profile, short-lived semantic references instead of CSS selectors, scoped subtree observation, deterministic high-risk refusal, and bounded evidence.

  • 8
  • JavaScript
  • MIT
  • 8 wk ago
ZSeven-WTake it
AutomationRepo

dockswipe

Synthesize macOS trackpad dock-swipe gestures from the CLI — trigger Mission Control, switch Spaces, App Exposé, Show Desktop & Launchpad through the real gesture pathway, with controllable speed. Built for end-to-end UI test automation.

  • 4
  • Objective-C
  • MIT
  • 5 mo ago
oomol-labTake it
AI agentsRepo

reflex-computer-use

Computer use MCP server for Claude Code, Codex and other AI agents. Reads the UI tree instead of taking a screenshot every step, with Jev (TypeSafe System One) as a fast judge that can also look at the screen to find icons and game objects. Local macOS, local Windows, or remote Windows over SSH.

  • 3
  • Python
  • MIT
  • 2 wk ago
8itlabTake it
AI on my own machineRepo

POK-Agent

POK-Agent: a Windows computer-use AI agent. A large model plans (System 2) and a small fast model acts on screen (System 1: JEV or local Laya). Repeated tasks become muscle memory, so the big model is called less every time. Local-first, voice and vision, Rust + Tauri, open source.

  • 3
  • Rust
  • Apache-2.0
  • 2 wk ago
AcekorneyaTake it
AI agentsRepo

dsh-computer

DeepSeek Harness plugin for macOS desktop automation. Observes the accessibility tree, gates risky actions behind one-time approval, permanently refuses secure fields, and returns bounded visual evidence — through a native helper you build and grant locally.

  • 3
  • JavaScript
  • MIT
  • 8 wk ago
ZSeven-WTake it
AI agentsRepo

numalarm

牛马铃 numalarm —— Windows 下基于桌面 UI 自动化的 QQ 语音通话提醒工具与 Agent Skill:AI Agent 任务完成/中断/等待确认时自动拨打 QQ 电话强提醒;无人接听自动重拨、接听后播报语音(离线 TTS+虚拟声卡)、用户在电脑前自动静默。提供 Python SDK / HTTP API / CLI / 桌面快捷方式与打断 Hook 接入。

  • 2
  • Python
  • MIT
  • 7 wk ago
AI on my own machineRepo

open-computer-use

Open-source computer-use AI agent for macOS: drives Mac apps through the accessibility tree with an indexed action space. The model picks from a numbered table of real elements; screenshots only as a fallback. Any OpenAI-compatible model or Jev. Port of browser-use/jev-ultrafast.

  • 1
  • Python
  • MIT
  • 3 d ago
max1874Take it
Search GitHub and Hugging Face for ui-automation

Get , open it once and sign in. Then come back here and press the button again.