<!DOCTYPE html>
<html>
<head>
<title></title>
<style type="text/css">p.MsoNormal,p.MsoNoSpacing{margin:0}</style>
</head>
<body><div style="font-family:menlo, consolas, monospace;"><br></div>
<div><br></div>
<div><br></div>
<div>On Sun, Oct 28, 2018, at 11:58 PM, Ronald Oussoren wrote:<br></div>
<blockquote type="cite"><div style="font-family:menlo, consolas, monospace;"><br></div>
<div><div style="font-family:menlo, consolas, monospace;"><br></div>
<blockquote type="cite"><div>On 29 Oct 2018, at 00:56, Glyph <<a href="mailto:glyph@twistedmatrix.com">glyph@twistedmatrix.com</a>> wrote:<br></div>
<div style="font-family:menlo, consolas, monospace;"><br></div>
<div><div style="overflow-wrap:break-word;"><div style="font-family:menlo, consolas, monospace;"><br></div>
<div><div style="font-family:menlo, consolas, monospace;"><br></div>
<blockquote type="cite"><div>On Oct 28, 2018, at 2:57 PM, Glyph <<a href="mailto:glyph@twistedmatrix.com">glyph@twistedmatrix.com</a>> wrote:<br></div>
<div style="font-family:menlo, consolas, monospace;"><br></div>
<div><div><blockquote type="cite" style="font-family:Menlo-Regular;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-size-adjust:auto;-webkit-text-stroke-width:0px;text-decoration-line:none;text-decoration-style:initial;text-decoration-color:initial;"><div dir="ltr"><div>I wonder what the “hardened runtime” option actually does and enforces.   In 3.7 the line in ctypes/__init__.py that causes the exception is a call that creates a dummy C function, and likely triggers the first allocation for storing a libffi closure which could be something the hardened runtime doesn’t like (being writeable + executable memory). <br></div>
</div>
</blockquote><div style="font-family:Menlo-Regular;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;-webkit-text-stroke-width:0px;text-decoration-line:none;text-decoration-style:initial;text-decoration-color:initial;"><br></div>
<div style="font-family:Menlo-Regular;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;-webkit-text-stroke-width:0px;text-decoration-line:none;text-decoration-style:initial;text-decoration-color:initial;">Interesting. Perhaps what I want is simply <a href="https://developer.apple.com/documentation/security/com_apple_security_cs_allow-unsigned-executable-memory">https://developer.apple.com/documentation/security/com_apple_security_cs_allow-unsigned-executable-memory</a> then?  Any chance you know how to jam that into a `codesign` command line somehow? :-)<br></div>
</div>
<div style="font-family:menlo, consolas, monospace;"><br></div>
</div>
</blockquote></div>
<div style="font-family:menlo, consolas, monospace;"><br></div>
<div>Thank you so much for this tip, Ronald!  This was much easier than I anticipated, and things are working now!<br></div>
</div>
</div>
</blockquote><div><br></div>
<div style="font-family:menlo, consolas, monospace;">Great.<br></div>
</div>
<div><div style="font-family:menlo, consolas, monospace;"><br></div>
<blockquote type="cite"><div><div style="overflow-wrap:break-word;"><div><br></div>
<div>The relevant entitlements file is literally just:<br></div>
<div><br></div>
<blockquote style="margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:40px;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-left-width:initial;border-top-style:none;border-right-style:none;border-bottom-style:none;border-left-style:none;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-left-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;padding-top:0px;padding-right:0px;padding-bottom:0px;padding-left:0px;"><div><div><?xml version="1.0" encoding="UTF-8"?><br></div>
<div><!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "<a href="http://www.apple.com/DTDs/PropertyList-1.0.dtd">http://www.apple.com/DTDs/PropertyList-1.0.dtd</a>"><br></div>
<div><plist version="1.0"><br></div>
<div><dict><br></div>
<div><span style="white-space:pre;"></span><key>com.apple.security.cs.allow-unsigned-executable-memory</key><br></div>
<div><span style="white-space:pre;"></span><true/><br></div>
<div></dict><br></div>
<div></plist><br></div>
</div>
</blockquote><div style="font-family:menlo, consolas, monospace;"><br></div>
<div>I dropped that in a file, added `--entitlements=$THAT_FILE.plist` to my codesign invocations, removed all my workarounds for ctypes et. al. (except for the hard-coded 'import _cffi_backend' still necessary to convince modulegraph to include enough code for SSL to work), and then tried launching my app.  <br></div>
</div>
</div>
</blockquote><div><br></div>
<div style="font-family:menlo, consolas, monospace;">Which package needs _cffi_backend? I can add a recipe for that to py2app to do this automagically.<br></div>
</div>
</blockquote><div style="font-family:menlo, consolas, monospace;"><br></div>
<div style="font-family:menlo, consolas, monospace;">This may sound obvious, but: cffi :-).  In my case, pyOpenSSL -> cryptography -> cffi.</div>
<div style="font-family:menlo, consolas, monospace;"><br></div>
<blockquote type="cite"><div><div style="font-family:menlo, consolas, monospace;"><br></div>
<blockquote type="cite"><div><div style="overflow-wrap:break-word;"><div>Success!  Then I tried notarizing it: also success!  Time permitting, I'll be updating my blog post at <a href="https://glyph.twistedmatrix.com/2018/01/shipping-pygame-mac-app.html">https://glyph.twistedmatrix.com/2018/01/shipping-pygame-mac-app.html</a> with this information, and possibly publishing the now unfortunately somewhat complex tooling I use to do signing now.<br></div>
<div><br></div>
<div>So I don't know if I'm the first to <i>do</i> this, but looking at the archives for these lists I seem to be the first to <i>report</i> it: you can successfully codesign and notarize apps created with py2app and python 3.6!<br></div>
<div><br></div>
<div>It seems to me that whatever "MAP_JIT" is (an mmap flag, I'm guessing?) libffi needs to be using it for the memory it places synthetic closures into, so that this entitlement won't be necessary with some future version of Python.  But it looks like Apple is not pushing particularly hard to deprecate this one right now, thank goodness :-).<br></div>
</div>
</div>
</blockquote><div><br></div>
<div style="font-family:menlo, consolas, monospace;">MAP_JIT is a mmap flag that’s apparently introduced in 10.14. The slides at <a href="https://developer.apple.com/videos/play/wwdc2018/702/">https://developer.apple.com/videos/play/wwdc2018/702/</a> mention this flag and the hardened runtime.  <br></div>
</div>
<div><br></div>
<div>I guess we should add this flag to the code in <span class="font" style="font-family:Menlo"><span class="size" style="font-size:11px">Modules/_ctypes/malloc_closure.c</span></span><span class="font" style="font-family:Menlo"><span class="size" style="font-size:11px"> C</span></span><span class="font" style="font-family:Menlo"><span class="size" style="font-size:11px">Python) and in the similar code in PyObjC.  The annoying bit is that the flag is new in 10.14, and CPython installers are created on 10.9 which means those won’t include the new flag for a long time. </span></span><br></div>
<div><span class="font" style="font-family:Menlo"><span class="size" style="font-size:11px"></span></span><br></div>
<div><span class="font" style="font-family:Menlo"><span class="size" style="font-size:11px">I’ll have to check if using MAP_JIT is ok when deploying on older macOS versions, or if the code should do a runtime version check. </span></span><br></div>
</blockquote><div style="font-family:menlo, consolas, monospace;"><br></div>
<div style="font-family:menlo, consolas, monospace;">I can't shed any light on this, but I suspect the cffi folks will also have to figure this out, and may already have some sense of how this works.  I filed an issue with them here: <a href="https://bitbucket.org/cffi/cffi/issues/391/cffi-doesnt-work-inside-a-macos-app-bundle">https://bitbucket.org/cffi/cffi/issues/391/cffi-doesnt-work-inside-a-macos-app-bundle</a><br></div>
</body>
</html>