Controlling the Mac OSX GUI via Python?
Christian Gollwitzer
auriocus at gmx.de
Fri Jul 1 00:58:58 EDT 2016
Am 01.07.16 um 01:12 schrieb Lawrence D’Oliveiro:
> On Friday, July 1, 2016 at 10:35:42 AM UTC+12, Chris Angelico wrote:
>>
>> On Fri, Jul 1, 2016 at 8:27 AM, Lawrence D’Oliveiro wrote:
>>>
>>> GUIs, by their nature, are not designed to be automated.
>>
>> Doesn't mean they can't be!
>
> It’s more trouble than it’s worth. Different (even minor) versions of
> the program will move UI elements around. Things that look and behave
> like buttons might have underlying code that is not quite the same as
> that for buttons. There will be subtle timing issues, where something
> doesn’t quite get enabled in time for your script to click on it. You
> will suffer mysterious intermittent problems, where a sequence works
> one time but not another time.
I think you misunderstood the request. Yes, simulating mouse clicks with
fixed coordinates etc. is prone to such failure, but there are better
methods. These mouse clicks and keyboard events usually trigger a method
call inside the GUI program. If there are any means to call that method
directly, you are independent from the graphics itself.
These bindings then usually call some higher-level functions. Many
applications export their internal structure via some general
communications mechanism. On Windows, COM is such a mechanism, CORBA is
another one, and on the Mac that are Apple Events. AppleScript is a
Mac-specific (strange) language which allows you to send arbitrary Apple
events to applications. This has nothing to do with simulating mouse
clicks. A deprecated Python package is found here:
http://appscript.sourceforge.net/py-appscript/
If you look at the first example:
app('TextEdit').documents['ReadMe'].paragraphs[1].get()
its immediately clear that this is on a very different level than
simulating Copy/Paste using the mouse to get the first paragraph.
According to my Google-Fu, howver, py-appscript stopped working with
some OSX update. One possibility would be to send the equivalent
applescript to the "osascript" program via subprocess.
> Again: GUIs are for manual operations to be performed by humans. For
> automation, you need a function-based scripting API and a command
> line.
Yes and no. Sometimes GUIs export an API which is just not obvious to
the user, sometimes they have an internal structure which can be exploited.
Christian
More information about the Python-list
mailing list