moin2 release with core functionality
Hello, many owners of a moin 1.9 wiki are waiting for a moin replacement based on Python3. If we limit the functionality of moin2 to the core features of moin1.9, IMO it should be possible to move to beta status or even better create a release candidate. For example, we can allow moinwiki markup and disable all other markup languages in the default configuration. They will then be treated as experimental. Wiki owners can test if they can migrate their wiki data to moin2. This could be an option to get more people interested in the development and to push the release of moin2. What is your opinion about this idea? I look forward to your feedback. With regards, Ulrich
On Monday, 14 August 2023 22:17:24 CEST Ulrich B. wrote:
many owners of a moin 1.9 wiki are waiting for a moin replacement based on Python3. If we limit the functionality of moin2 to the core features of moin1.9, IMO it should be possible to move to beta status or even better create a release candidate. For example, we can allow moinwiki markup and disable all other markup languages in the default configuration. They will then be treated as experimental.
Certainly, focus on the core features might help get the software into a state that people will be comfortable with. I shouldn't comment too substantially about this, though, since I have only been doing superficial testing of Moin 2, as discussed below. For all I know, it might be stable and usable enough already.
Wiki owners can test if they can migrate their wiki data to moin2. This could be an option to get more people interested in the development and to push the release of moin2.
I agree that migration is a great way of exposing deficiencies and making people more interested. When I converted the Mailman wiki from Confluence to Moin, I found that converting content exposed many things that I then had to fix, and doing so in bulk rather accelerated the process, highlighting multiple areas of improvement simultaneously.
What is your opinion about this idea? I look forward to your feedback.
I think I saw that you had updated the Moin wiki with instructions for installing Moin 2 on Debian. In fact, Debian packages have been created for Moin 2 already, but we didn't manage to get them into the latest stable release. At the moment, they are parked here: https://salsa.debian.org/moin-team/ (Ignore the "old" repositories.) What we really need to do is to generate packages for people who don't want to build them independently. I have also been packaging some other software for Debian and have used the "CI" facilities in the Salsa service which can build packages and test them, but I don't think such CI-built packages are published anywhere. Maybe such automated publishing can be done another way, though. I have performed cursory testing on the Moin packages and was able to get a Moin site up and running in a Debian unstable environment, so I have some confidence in the basic functionality. Paul
I love this idea! I consider myself a competent developer and was hoping I could at least get into just the basic functionality, enough to mimic 1.9, but I am not an experienced Python developer and wasn't able to get much beyond the launcher--besides life just interrupting and distracting me. It would also be nice to be able to set up an instance without going through all of the developer steps and environment; it is like every time I want to work on it, "OK, now what was it I did to setup that virtual python environment?...." hah. Scott. On 8/14/2023 14:17, Ulrich B. wrote:
Hello,
many owners of a moin 1.9 wiki are waiting for a moin replacement based on Python3. If we limit the functionality of moin2 to the core features of moin1.9, IMO it should be possible to move to beta status or even better create a release candidate. For example, we can allow moinwiki markup and disable all other markup languages in the default configuration. They will then be treated as experimental.
Wiki owners can test if they can migrate their wiki data to moin2. This could be an option to get more people interested in the development and to push the release of moin2.
What is your opinion about this idea? I look forward to your feedback.
With regards,
Ulrich
_______________________________________________ moin-devel mailing list moin-devel@python.org https://mail.python.org/mailman/listinfo/moin-devel
On Tuesday, 15 August 2023 23:13:57 CEST M. Scott Reynolds wrote:
It would also be nice to be able to set up an instance without going through all of the developer steps and environment; it is like every time I want to work on it, "OK, now what was it I did to setup that virtual python environment?...." hah.
The aim with regard to the Debian packaging is that you should be able to just invoke "apt install moin2" and get all the necessary software. Then, doing the following would get you started: moin create-instance moin index-create moin run There would be no need to mess around with whatever the Python packaging universe has decided today, nor with the peculiarities of a developer installation, at least if the focus is on testing the software or writing extensions. And for development of the software itself, the moin2 package and its dependencies could be installed, and then the moin2 package itself could be removed, making it possible to work directly with the development repository instead. Currently, I'm trying to update the packages but have encountered some problems with the test suite that didn't occur before. It is a tedious exercise, second-guessing overcomplicated packaging tools and presumably re- enacting the mainframe era by invoking programs that take minutes to run before they dump a transcript of arcane errors. But I imagine that updated packages will be ready eventually. Paul
On Wednesday, 16 August 2023 00:47:12 CEST Paul Boddie wrote:
The aim with regard to the Debian packaging is that you should be able to just invoke "apt install moin2" and get all the necessary software.
Correcting myself here, you would actually do this: apt install python3-moinmoin2
Currently, I'm trying to update the packages but have encountered some problems with the test suite that didn't occur before.
These are actually related to the way that the tests invoke the moin program which obviously needs the program to be available somehow. For some reason, this doesn't work, even though I have been packaging something else whose principal program can be invoked just fine to run its accompanying tests. Ignoring the test suite, the actual package is still installable and usable, although I am sure that improvements can be made. Resources like the following yield 404 errors, for instance: /+serve/autosize/jquery.autosize-min.js /+serve/font_awesome/css/all.css As far as the test suite is concerned, I have moved it to the Salsa CI system where it may well experience problems as it attempts to obtain other packages that are completely new to Debian, these being shown in the overview of the repositories: https://salsa.debian.org/moin-team/ If there is any documentation about the kinds of workarounds that have been available for things like pbuilder, or any concise, pertinent documentation about the CI system at all, I cannot find it. So, I think that there will be numerous pipeline failures until I figure something out: https://salsa.debian.org/moin-team/moin/-/pipelines Anyway, I hope this is vaguely interesting for some readers. Paul
@Paul, @Scott: thanks for your feedback. Am 16.08.23 um 17:40 schrieb Paul Boddie:
As far as the test suite is concerned, I have moved it to the Salsa CI system where it may well experience problems as it attempts to obtain other packages that are completely new to Debian, these being shown in the overview of the repositories:
I will take a look at the build process, but it will take quite a while. Regards, Ulrich
On Wednesday, 16 August 2023 23:06:38 CEST Ulrich B. wrote:
Am 16.08.23 um 17:40 schrieb Paul Boddie:
As far as the test suite is concerned, I have moved it to the Salsa CI system where it may well experience problems as it attempts to obtain other packages that are completely new to Debian, these being shown in the overview of the repositories:
I will take a look at the build process, but it will take quite a while.
Just to confirm, an experimental run of the test suite failed as expected because various dependencies are completely new to Debian: https://salsa.debian.org/moin-team/moin/-/jobs/4566286 Obviously, these dependencies are other packages that I have created but which have no way of being built and injected into the test environment. So, for testing in the Salsa CI environment, I need to discover a way of getting the dependencies recognised. Paul
On Wednesday, 16 August 2023 23:53:08 CEST Paul Boddie wrote:
Just to confirm, an experimental run of the test suite failed as expected because various dependencies are completely new to Debian:
https://salsa.debian.org/moin-team/moin/-/jobs/4566286
Obviously, these dependencies are other packages that I have created but which have no way of being built and injected into the test environment. So, for testing in the Salsa CI environment, I need to discover a way of getting the dependencies recognised.
Following up, I think I have probably managed to work around CI issues and have Debian's autopkgtest invoke the Moin test suite. The Scrapy-driven test seems to be quite demanding, requiring a server to be run and lots of HTTP connections to be made, so I have instructed pytest to ignore it for now. I also saw that "US/Arizona" is used as a time zone in two of the tests. This does not seem to be recognised by pytz, being some kind of legacy zone, but the equivalent "America/Phoenix" is recognised. For now, I have applied patches in my packaging to avoid any errors related to this issue. It is not currently very easy to publish the packages that are built for this exercise in a way that makes them convenient to acquire, at least not through the Salsa service. The build artefacts do include the packages, and I could provide a list of all the package URLs, but it would be preferable to just have some kind of repository containing them all. Now, it is possible to publish repositories of packages within Salsa, and this functionality is actually used to provide some new packages to the test environment that installs Moin and runs its tests. However, more work is needed to collect all of the new packages and to serve them up together. Anyway, I hope this is of interest! Paul
On Monday, 21 August 2023 00:16:05 CEST Paul Boddie wrote:
Now, it is possible to publish repositories of packages within Salsa, and this functionality is actually used to provide some new packages to the test environment that installs Moin and runs its tests. However, more work is needed to collect all of the new packages and to serve them up together.
Following up to myself again, I finally figured out a way of publishing a collection of packages on Debian's Salsa (GitLab) service. This should allow anyone running Debian unstable to try out the packages I have made for Moin 2. The Salsa pipeline and job responsible for this output are these: https://salsa.debian.org/moin-team/moin/-/pipelines/577973 https://salsa.debian.org/moin-team/moin/-/jobs/4683093 == Patches == Irritatingly, I had to fix various Flask-related incompatibilities to get the test suite working again. Since there has been a Debian release recently, people have bumped the versions of various packages and, of course, some software developers don't believe in API stability, so it broke Moin in various ways. So, the patches currently applied in the packaging should probably be applied upstream: https://salsa.debian.org/moin-team/moin/-/blob/debian/master/debian/patches/ 016-fix-flask-babel-3-breakage There are other patches that may benefit from application upstream. == Packages == Currently, the packages are available from a package repository at the following location: https://moin-team.pages.debian.net/-/moin/-/jobs/4683093/artifacts/aptly/ index.html Now, to use this repository, it is probably best to follow the "one-line format" instructions at the bottom of that page, this involving the downloading of the public key and the creation of a sources list file. However, I found that the source definition needs changing to specify an architecture of "all" as follows: deb [arch=all signed-by=...] ... In other words, "arch=all" needs adding, separated by a space. You should then be able to run the following: apt-get update apt-get install python3-moinmoin2 This will then download and install new and existing packages. Afterwards, you can then create and run an instance as follows: moin create-instance moin index-create moin run I hope this is useful to someone. I have had no feedback about this packaging work, despite spending a fair amount of time on it. If that continues, it will definitely influence how I spend my time in future. Paul
Hi Paul, Am 10.09.23 um 19:25 schrieb Paul Boddie:
== Patches ==
So, the patches currently applied in the packaging should probably be applied upstream:
https://salsa.debian.org/moin-team/moin/-/blob/debian/master/debian/patches/
016-fix-flask-babel-3-breakage
There are other patches that may benefit from application upstream.
I will go thru your patches and create issues and pull requests.
== Packages ==
Currently, the packages are available from a package repository at the following location:
I hope this is useful to someone. I have had no feedback about this packaging work, despite spending a fair amount of time on it.
Setting up a new Debian and testing your package will take me some time. thanks for all your work. I have great respect for your results, it definitely helps to keep the moinwiki project running. Kind regards, Ulrich
Ulrich,
I will go thru your patches and create issues and pull requests.
Thanks for offering to look at them! The ones that were required to adapt to package version changes were as follows: 001-pdfminer3-to-pdfminer - uses pdfminer instead of pdfminer3 002-log-logger - fixes UnboundLocalError in logging 015-fix-test-time-zones - use more conventional time zone names 016-fix-flask-babel-3-breakage - fix API breakage in Flask-Babel 3 There are numerous patches to setup.py, either adding files like intermap.txt or adjusting package details. The above pdfminer change introduces pdfminer.six instead of pdfminer3, for instance.
thanks for all your work. I have great respect for your results, it definitely helps to keep the moinwiki project running.
Thank you for all your hard work keeping Moin itself going! I would really like to see the project take off again, but I have limited time and energy available to direct towards it. Please let me know if you need any of the patches explaining or want to discuss them. I will attempt to track upstream development and hopefully eliminate patches over time. If you need to set up Debian, I can recommend doing so in a chroot environment or a virtual machine. I don't know what kind of system you are running, but something like GNOME Boxes should make a virtual machine installation fairly straightforward in a Linux environment. Myself, I use some scripts that create and manage chroot environments that have been useful to me over the years, but I cannot claim that they are "production quality": https://hg.boddie.org.uk/userinstall There are more cumbersome instructions for setting up a chroot on the Debian wiki: https://wiki.debian.org/chroot But a virtual machine might end up being the easiest approach to begin with, particularly if you are not running Linux. Regards, Paul
Hi Paul, Am 11.09.23 um 23:44 schrieb Paul Boddie:
Thanks for offering to look at them! The ones that were required to adapt to package version changes were as follows:
001-pdfminer3-to-pdfminer - uses pdfminer instead of pdfminer3 This pdfminer patch has been merged into master today. 002-log-logger - fixes UnboundLocalError in logging Can you provide an example how to reproduce this error? 015-fix-test-time-zones - use more conventional time zone names Is this issue a bug or is it just cosmetic? 016-fix-flask-babel-3-breakage - fix API breakage in Flask-Babel 3
A pull request has been created for flask-babel 3. Many thanks. Ulrich
On Friday, 15 September 2023 23:31:18 CEST Ulrich B. wrote:
001-pdfminer3-to-pdfminer - uses pdfminer instead of pdfminer3
This pdfminer patch has been merged into master today.
Excellent! I will eliminate the related patches in the packaging.
002-log-logger - fixes UnboundLocalError in logging
Can you provide an example how to reproduce this error?
I'll look into reproducing this when I merge from your upstream branch and try and rebuild.
015-fix-test-time-zones - use more conventional time zone names
Is this issue a bug or is it just cosmetic?
import pytz pytz.timezone("US/Arizona") Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/usr/lib/python3/dist-packages/pytz/__init__.py", line 200, in timezone raise UnknownTimeZoneError(zone)
I think it is something to do with the kind of time zone involved. US/Arizona is some kind of alias, whereas America/Phoenix is one of the fundamental zones. For some reason, in the package build environment, the pytz configuration doesn't recognise the alias whereas it does generally. It seems to be something that pytz in Debian unstable does not support: pytz.exceptions.UnknownTimeZoneError: 'US/Arizona' Looking further, it appears that pytz may be provided as some kind of wrapper around the zoneinfo module: https://packages.debian.org/sid/python3-pytz-deprecation-shim So, I guess this is yet another Python ecosystem regression.
016-fix-flask-babel-3-breakage - fix API breakage in Flask-Babel 3
A pull request has been created for flask-babel 3.
Great! Thanks for following up with that. I suspect that they have broken the API deliberately, but you never know. Paul
On Saturday, 16 September 2023 00:29:21 CEST Paul Boddie wrote:
On Friday, 15 September 2023 23:31:18 CEST Ulrich B. wrote:
002-log-logger - fixes UnboundLocalError in logging
Can you provide an example how to reproduce this error?
I'll look into reproducing this when I merge from your upstream branch and try and rebuild.
I have now eliminated this patch since it may only have been required as a consequence of another patch that has also been eliminated. I previously had difficulties getting Moin working when it needed to be run in the packaging environment. Here are the current patches: https://salsa.debian.org/moin-team/moin/-/tree/debian/master/debian/patches Apart from patches adjusting package versions and availability, some of these do merit a bit more explanation. First of all, I did experience problems copying package data, as can be seen from the 011-setup-package-data patch. Another problem was encountered with some kind of change to the Python packaging tools, where I got warnings about namespace packages, but it is hopefully addressed with the 012-search-namespace-packages patch. Testing with updated upstream code and some packaging adjustments was successful, with an updated package repository being made available here: https://moin-team.pages.debian.net/-/moin/-/jobs/4712648/artifacts/aptly/ index.html Again, the "old one-line format" approach can be used to access this repository with the following kind of amendment being made to the repository declaration lines: deb [arch=all signed-by=...] https... Here, "arch=all" has been added. Remember to acquire the public key! Paul
participants (3)
-
M. Scott Reynolds -
Paul Boddie -
Ulrich B.