Docs: audit event table empty
Hi, the table with auditing events does not render on docs.python.org, https://docs.python.org/3.9/library/audit_events.html. Steve and I are going to present the auditing feature tomorrow at EuroPython. It would be helpful to have the table available. It works on Steve's and my local machine without any issues. I suspect it's either an outdated sphinx version or caching issue. Could somebody from the docs team or shell access to the docs machine please look into the matter? Thanks! Christian
Hi Christian,
the table with auditing events does not render on docs.python.org, https://docs.python.org/3.9/library/audit_events.html. Steve and I are going to present the auditing feature tomorrow at EuroPython. It would be helpful to have the table available.
This was not an easy one... and it may be a Sphinx issue, yet I'm still not sure, maybe Steve can shed some light on it: It's the "-j" option of sphinx-build (to parallelize) that causes the issue. I double checked it (full commands at the end of the message in case someone want to reproduce it): - Run with -j4 → No table - Run without -j → Table is here - Run again with -j4 → No table! - Run again without -j → Table is back! I'm patching docsbuild-scripts to stop using -j4 with is not really helpfull anyway as docsbuild script is parallelizing by starting multiple sphinx-build (for multiple languages / versions). I also copied the file and invalidated the cache, so https://docs.python.org/3.9/library/audit_events.html is good again. If I'm too slow testing locally and releasing a new docsbuild_script.py, the cron MAY break the file again, don't hesitate to poke me if it happen without me noticing first. Full test: docsbuild@docs:/srv/docsbuild/3.9/cpython-en/Doc$ /srv/docsbuild/venv/bin/sphinx-build -b html -d build/doctrees -D latex_elements.papersize= -D latex_engine=xelatex -D latex_elements.inputenc= -D latex_elements.fontenc= -j4 -q -Ea -A daily=1 -A switchers=1 . build/html docsbuild@docs:/srv/docsbuild/3.9/cpython-en/Doc$ grep breakpoint build/html/library/audit_events.html docsbuild@docs:/srv/docsbuild/3.9/cpython-en/Doc$ /srv/docsbuild/venv/bin/sphinx-build -b html -d build/doctrees -D latex_elements.papersize= -D latex_engine=xelatex -D latex_elements.inputenc= -D latex_elements.fontenc= -q -Ea -A daily=1 -A switchers=1 . build/html docsbuild@docs:/srv/docsbuild/3.9/cpython-en/Doc$ grep breakpoint build/html/library/audit_events.html <tr class="row-odd"><td><p>builtins.breakpoint</p></td> <td><p><code class="docutils literal notranslate"><span class="pre">breakpointhook</span></code></p></td> <td><p><a class="reference internal" href="functions.html#breakpoint">[1]</a></p></td> docsbuild@docs:/srv/docsbuild/3.9/cpython-en/Doc$ /srv/docsbuild/venv/bin/sphinx-build -b html -d build/doctrees -D latex_elements.papersize= -D latex_engine=xelatex -D latex_elements.inputenc= -D latex_elements.fontenc= -j4 -q -Ea -A daily=1 -A switchers=1 . build/html docsbuild@docs:/srv/docsbuild/3.9/cpython-en/Doc$ grep breakpoint build/html/library/audit_events.html docsbuild@docs:/srv/docsbuild/3.9/cpython-en/Doc$ /srv/docsbuild/venv/bin/sphinx-build -b html -d build/doctrees -D latex_elements.papersize= -D latex_engine=xelatex -D latex_elements.inputenc= -D latex_elements.fontenc= -q -Ea -A daily=1 -A switchers=1 . build/html docsbuild@docs:/srv/docsbuild/3.9/cpython-en/Doc$ grep breakpoint build/html/library/audit_events.html <tr class="row-odd"><td><p>builtins.breakpoint</p></td> <td><p><code class="docutils literal notranslate"><span class="pre">breakpointhook</span></code></p></td> <td><p><a class="reference internal" href="functions.html#breakpoint">[1]</a></p></td> Bests, -- Julien Palard https://mdk.fr
On 09/07/2019 14.02, Julien Palard wrote:
Hi Christian,
the table with auditing events does not render on docs.python.org, https://docs.python.org/3.9/library/audit_events.html. Steve and I are going to present the auditing feature tomorrow at EuroPython. It would be helpful to have the table available.
This was not an easy one... and it may be a Sphinx issue, yet I'm still not sure, maybe Steve can shed some light on it:
It's the "-j" option of sphinx-build (to parallelize) that causes the issue. I double checked it (full commands at the end of the message in case someone want to reproduce it):
- Run with -j4 → No table - Run without -j → Table is here - Run again with -j4 → No table! - Run again without -j → Table is back!
I'm patching docsbuild-scripts to stop using -j4 with is not really helpfull anyway as docsbuild script is parallelizing by starting multiple sphinx-build (for multiple languages / versions).
I also copied the file and invalidated the cache, so https://docs.python.org/3.9/library/audit_events.html is good again.
If I'm too slow testing locally and releasing a new docsbuild_script.py, the cron MAY break the file again, don't hesitate to poke me if it happen without me noticing first.
Fantastic! It'll do for now. Thank you! I'll follow up with you after EuroPython. Christian
On Tue, Jul 9, 2019 at 5:34 PM Julien Palard via Python-Dev < python-dev@python.org> wrote:
Hi Christian,
the table with auditing events does not render on docs.python.org, https://docs.python.org/3.9/library/audit_events.html. Steve and I are going to present the auditing feature tomorrow at EuroPython. It would be helpful to have the table available.
This was not an easy one... and it may be a Sphinx issue, yet I'm still not sure, maybe Steve can shed some light on it:
It's the "-j" option of sphinx-build (to parallelize) that causes the issue. I double checked it (full commands at the end of the message in case someone want to reproduce it):
- Run with -j4 → No table - Run without -j → Table is here - Run again with -j4 → No table! - Run again without -j → Table is back!
I can reproduce the same with table not appearing in parallel builds and table is successfully built on non-parallel builds. On adding -j4 to ALLSPHINXOPTS in my Makefile in Docs/ folder there is no table generated. Currently, the audit events are collected in a dictionary and are stored in self.state.document.settings.env that is initialized with empty dict in AuditEvent directive [0]. * On non-parallel builds when I use print statement with hasattr to check for all_audit_events, it's False for first time and then remains True with tuple set for rest of the collection. In process_audit_events for non-parallel builds the attribute is set with a dictionary of all audit events. * On parallel builds when I use print statement with hasattr to check for all_audit_events, it returns True and False randomly throughout the build. I guess it's due to attribute not being properly initialized. In process_audit_events for parallel builds the all_audit_events attribute is not at all set during table creation and falls back to empty tuple writing no rows to the table. I am not sure if modifying the env attribute on parallel builds is safe since I couldn't find any guarantees from the sphinx docs. The default docs Makefile is not parallel which could be the reason it was not reproducible locally. Thanks for fixing this. [0] https://github.com/python/cpython/blob/73ea54620a6f91c3f2e53880373dd47813691...
I'm patching docsbuild-scripts to stop using -j4 with is not really helpfull anyway as docsbuild script is parallelizing by starting multiple sphinx-build (for multiple languages / versions).
I also copied the file and invalidated the cache, so https://docs.python.org/3.9/library/audit_events.html is good again.
If I'm too slow testing locally and releasing a new docsbuild_script.py, the cron MAY break the file again, don't hesitate to poke me if it happen without me noticing first.
Full test:
docsbuild@docs:/srv/docsbuild/3.9/cpython-en/Doc$ /srv/docsbuild/venv/bin/sphinx-build -b html -d build/doctrees -D latex_elements.papersize= -D latex_engine=xelatex -D latex_elements.inputenc= -D latex_elements.fontenc= -j4 -q -Ea -A daily=1 -A switchers=1 . build/html docsbuild@docs:/srv/docsbuild/3.9/cpython-en/Doc$ grep breakpoint build/html/library/audit_events.html docsbuild@docs:/srv/docsbuild/3.9/cpython-en/Doc$ /srv/docsbuild/venv/bin/sphinx-build -b html -d build/doctrees -D latex_elements.papersize= -D latex_engine=xelatex -D latex_elements.inputenc= -D latex_elements.fontenc= -q -Ea -A daily=1 -A switchers=1 . build/html docsbuild@docs:/srv/docsbuild/3.9/cpython-en/Doc$ grep breakpoint build/html/library/audit_events.html <tr class="row-odd"><td><p>builtins.breakpoint</p></td> <td><p><code class="docutils literal notranslate"><span class="pre">breakpointhook</span></code></p></td> <td><p><a class="reference internal" href="functions.html#breakpoint">[1]</a></p></td> docsbuild@docs:/srv/docsbuild/3.9/cpython-en/Doc$ /srv/docsbuild/venv/bin/sphinx-build -b html -d build/doctrees -D latex_elements.papersize= -D latex_engine=xelatex -D latex_elements.inputenc= -D latex_elements.fontenc= -j4 -q -Ea -A daily=1 -A switchers=1 . build/html docsbuild@docs:/srv/docsbuild/3.9/cpython-en/Doc$ grep breakpoint build/html/library/audit_events.html docsbuild@docs:/srv/docsbuild/3.9/cpython-en/Doc$ /srv/docsbuild/venv/bin/sphinx-build -b html -d build/doctrees -D latex_elements.papersize= -D latex_engine=xelatex -D latex_elements.inputenc= -D latex_elements.fontenc= -q -Ea -A daily=1 -A switchers=1 . build/html docsbuild@docs:/srv/docsbuild/3.9/cpython-en/Doc$ grep breakpoint build/html/library/audit_events.html <tr class="row-odd"><td><p>builtins.breakpoint</p></td> <td><p><code class="docutils literal notranslate"><span class="pre">breakpointhook</span></code></p></td> <td><p><a class="reference internal" href="functions.html#breakpoint">[1]</a></p></td>
Bests, -- Julien Palard https://mdk.fr _______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/Q23O23HX...
-- Regards, Karthikeyan S
On 7/9/2019 6:44 AM, Christian Heimes wrote:
Hi,
the table with auditing events does not render on docs.python.org, https://docs.python.org/3.9/library/audit_events.html. Steve and I are going to present the auditing feature tomorrow at EuroPython. It would be helpful to have the table available.
It works on Steve's and my local machine without any issues. I suspect it's either an outdated sphinx version or caching issue. Could somebody from the docs team or shell access to the docs machine please look into the matter?
After they did so, this now works for me. Looks great. -- Terry Jan Reedy
On 7/9/2019 3:53 PM, Terry Reedy wrote:
https://docs.python.org/3.9/library/audit_events.html. Steve and I are going to present the auditing feature tomorrow at EuroPython.
That should be interesting. My experiment with the following
def audit(event, args): print(event, args)
import sys; sys.addaudithook(audit)
produced a line, on IDLE, like this: builtins.id (1501408069568,) builtins.id is not in the table. Should it be? -- Terry Jan Reedy
participants (4)
-
Christian Heimes -
Julien Palard -
Karthikeyan -
Terry Reedy