Re: How strict are the dependencies on the django-compressor related backends?

On Fri, Mar 15, 2019 at 10:19 AM Jonas Smedegaard <dr@jones.dk> wrote:
Yeah I doubt that as well. In the meantime I also found 1 which seems to have come to the same conclusion for the majority of default setups. For upstream to disable it by default it was mentioned that there is a need for a customize kit which seems fine 1.
There is a further argument to disable the live compressors, at least as I read COMPRESS_OFFLINE that seems helpful for production setups anyway to speed up and safe CPU load.
So while upstream waits until a "customize kit" exists to switch defaults, as a downstream like Debian/Ubuntu I wonder if we already could: pulling it in at runtime
- drop sass/less compilers to a suggest
- disable the live compression in the default config that we ship
- keep the config commented-out to make it easy for admins that want to enable it later
- pre-compress at build time and enable compress_offline for speed
- maybe call
django_admin compress
3 or similar at build time? - having sass/less as build dependencies seems much safer than
- maybe call
@Jonas M.: what do you think about 2 (again maybe after Buster)? Sorry to not know more of this yet, but I hope suggesting what I think might work helps to get us the right way.
-- Christian Ehrhardt Software Engineer, Ubuntu Server Canonical Ltd

On Mon, Mar 18, 2019, at 1:04 AM, Christian Ehrhardt wrote:
+1 from me. It totally makes sense for any Mailman package to pre-compress the static files and not depend on any of those compilers.
Upstream doesn't declare any dependencies on sass or less compilers, since there is no way to do so, I don't think there is need to change anything except settings.py.
-- thanks, Abhilash Raj (maxking)

[I've put the related Debian bugreport in the loop]
Hello,
I finally found some time to look into this.
Disclaimer: I don't know much about CSS compression and the mailman3 web interface is the first time that I got into contact with either of sassc and node-less.
TL;DR:
- Hyperkitty doesn't need (or use) less.js/lessc at all. It can safely be removed from COMPRESS_PRECOMPILERS.
- The situation is different with sassc, it's actually used for compressing CSS by hyperkitty. Doing the compression at build-time should be possible but requires more work. I have a plan ;)
Christian Ehrhardt:
I don't think that any of the .less files you're referencing to will be processed by 'django-admin compress' on the mailman3-web project. The .less files are all outside the scope of the mailman3-web project. So I'm pretty sure that the node-less/lessc dependency can safely be dropped immediately.
Moreover, looking at the hyperkitty example_project/settings.py[1], it doesn't even reference the lessc compiler at all. Only settings.py from mailman-suite[2] references it, which probably is a copy-and-paste bug. There's even an open issue about that[3]. Since in Debian we took the mailman-suite settings.py as template, we suffer from this copy-and-paste bug as well.
[1] https://gitlab.com/mailman/hyperkitty/blob/master/example_project/settings.p... [2] https://gitlab.com/mailman/mailman-suite/blob/master/mailman-suite_project/s... [3] https://gitlab.com/mailman/mailman-suite/issues/7
Does anyone know of other uses of .less in the mailman3 stack that I have missed that we would have to think about?
Regarding sassc: Hyperkitty is the only part of the mailman3 stack that uses django-compressor. And apparently there's just one single html template that will be processed by sassc (well, to be precise, all html files that use this template):
/usr/lib/python3/dist-packages/hyperkitty/templates/hyperkitty/base.html
In this file, four CSS files are being compressed and packed together by sassc:
hyperkitty/libs/fonts/icomoon/icomoon.css hyperkitty/libs/fonts/droid/droid.css django-mailman3/css/main.css hyperkitty/sass/hyperkitty.scss
The last one is special as it's a scss file itself and needs to be processed by sassc itself.
I've done this. Result: removing node-less doesn't change anything (which is in line with my assumption that node-less/less.js/lessc is not required/used at all).
On the other hand, removing sassc breaks the setup, even with COMPRESS_OFFLINE being turned on:
CommandError: An error occurred during rendering /usr/lib/python3/dist-packages/django_mailman3/templates/django_mailman3/profile/delete_profile.html: /bin/sh: 1: sassc: not found
Christian Ehrhardt:
You're totally right. Therefore, COMPRESS_OFFLINE is already enabled per default in mailman3-web on Debian. The django-admin compress command is executed during package installation and upgrade by the package postinst script.
I'm absolutely in favour of getting rid of the sassc runtime dependency. After a chat with Jonas Smedegaard (thanks for his valuable input!), I think that I have an idea and a roadmap on how to get there:
- Process
static/hyperkittysass/hyperkitty.scss
with sassc at build- time and patchtemplates/hyperkitty/base.html
to reference the resulting CSS file. - Patch base.html at build-time to remove the '{% compress css %}' statements from it.
By then, we already can safely drop the run-time dependency on django-compressor from hyperkitty since it won't be needed anymore and consequently remove sassc dependency from mailman3-web (and all references to django-compressor from settings.py).
The downside will be some performance decrease as hyperkitty pages will reference four separate CSS file instead of a combined one. But that can be addressed as well:
- Make python3-django-mailman3 a build-dependency of mailman-suite (one of the referenced CSS files is shipped by django-mailman3) and combine all four CSS files into one by runing sassc at build-time.
- Further patch base.html at build-time to replace the four separate CSS references by one to the resulting combined CSS.
I plan to start working on the actual implementation soon but would be glad to hear your thoughts on it.
Cheers jonas

Hey again,
Jonas Meurer:
I've now done this in the Debian mailman3-web package and hope to get this into Buster before release.
For that, I now also have a working implementation as well. Due to the change being rather invasive, I will not push this into Buster though. And my plan slightly changed during implementing it, see below ;)
That's what I did now, and it already was enough to drop the build-time dependency on sassc. Django-compressor is still used to combine several CSS and JS files into one by running 'django-admin compress' in the postinst script of mailman3-web, but since the source files are already clean CSS and JS, no external processors (like sassc or lessc) are required.
I think this is perfectly fine and I don't see much benefit in combining the CSS and JS files at build-time. It would allow to drop the django-compressor dependency but with the cost of more heavy build-time adjustments that need to be maintained in future.
So I'm happy to now have a solution to drop both node-less and sassc from runtime dependencies in the Debian package. Thanks to everyone for their input on this topic.
I added a comment to upstream hyperkitty issue #120 where I proposed to do the same in the upstream hyperkitty release process and that way get rid of the sassc runtime dependency:
https://gitlab.com/mailman/hyperkitty/issues/120#note_179256370
Cheers jonas

[...]
I agree, the extra gain by this would be minimal at a rather high maintenance cost.
Thank you so much!
I think that also eases the maintenance of a live mailman deployment having much less complex components installed.
I understand that this won't be for Buster. Just curious for related Ubuntu planning - are you planning to do an upload to experimental ahead of time or are you waiting until Buster is released?
Great, that will help to keep things in sync.

On Mon, Mar 18, 2019, at 1:04 AM, Christian Ehrhardt wrote:
+1 from me. It totally makes sense for any Mailman package to pre-compress the static files and not depend on any of those compilers.
Upstream doesn't declare any dependencies on sass or less compilers, since there is no way to do so, I don't think there is need to change anything except settings.py.
-- thanks, Abhilash Raj (maxking)

[I've put the related Debian bugreport in the loop]
Hello,
I finally found some time to look into this.
Disclaimer: I don't know much about CSS compression and the mailman3 web interface is the first time that I got into contact with either of sassc and node-less.
TL;DR:
- Hyperkitty doesn't need (or use) less.js/lessc at all. It can safely be removed from COMPRESS_PRECOMPILERS.
- The situation is different with sassc, it's actually used for compressing CSS by hyperkitty. Doing the compression at build-time should be possible but requires more work. I have a plan ;)
Christian Ehrhardt:
I don't think that any of the .less files you're referencing to will be processed by 'django-admin compress' on the mailman3-web project. The .less files are all outside the scope of the mailman3-web project. So I'm pretty sure that the node-less/lessc dependency can safely be dropped immediately.
Moreover, looking at the hyperkitty example_project/settings.py[1], it doesn't even reference the lessc compiler at all. Only settings.py from mailman-suite[2] references it, which probably is a copy-and-paste bug. There's even an open issue about that[3]. Since in Debian we took the mailman-suite settings.py as template, we suffer from this copy-and-paste bug as well.
[1] https://gitlab.com/mailman/hyperkitty/blob/master/example_project/settings.p... [2] https://gitlab.com/mailman/mailman-suite/blob/master/mailman-suite_project/s... [3] https://gitlab.com/mailman/mailman-suite/issues/7
Does anyone know of other uses of .less in the mailman3 stack that I have missed that we would have to think about?
Regarding sassc: Hyperkitty is the only part of the mailman3 stack that uses django-compressor. And apparently there's just one single html template that will be processed by sassc (well, to be precise, all html files that use this template):
/usr/lib/python3/dist-packages/hyperkitty/templates/hyperkitty/base.html
In this file, four CSS files are being compressed and packed together by sassc:
hyperkitty/libs/fonts/icomoon/icomoon.css hyperkitty/libs/fonts/droid/droid.css django-mailman3/css/main.css hyperkitty/sass/hyperkitty.scss
The last one is special as it's a scss file itself and needs to be processed by sassc itself.
I've done this. Result: removing node-less doesn't change anything (which is in line with my assumption that node-less/less.js/lessc is not required/used at all).
On the other hand, removing sassc breaks the setup, even with COMPRESS_OFFLINE being turned on:
CommandError: An error occurred during rendering /usr/lib/python3/dist-packages/django_mailman3/templates/django_mailman3/profile/delete_profile.html: /bin/sh: 1: sassc: not found
Christian Ehrhardt:
You're totally right. Therefore, COMPRESS_OFFLINE is already enabled per default in mailman3-web on Debian. The django-admin compress command is executed during package installation and upgrade by the package postinst script.
I'm absolutely in favour of getting rid of the sassc runtime dependency. After a chat with Jonas Smedegaard (thanks for his valuable input!), I think that I have an idea and a roadmap on how to get there:
- Process
static/hyperkittysass/hyperkitty.scss
with sassc at build- time and patchtemplates/hyperkitty/base.html
to reference the resulting CSS file. - Patch base.html at build-time to remove the '{% compress css %}' statements from it.
By then, we already can safely drop the run-time dependency on django-compressor from hyperkitty since it won't be needed anymore and consequently remove sassc dependency from mailman3-web (and all references to django-compressor from settings.py).
The downside will be some performance decrease as hyperkitty pages will reference four separate CSS file instead of a combined one. But that can be addressed as well:
- Make python3-django-mailman3 a build-dependency of mailman-suite (one of the referenced CSS files is shipped by django-mailman3) and combine all four CSS files into one by runing sassc at build-time.
- Further patch base.html at build-time to replace the four separate CSS references by one to the resulting combined CSS.
I plan to start working on the actual implementation soon but would be glad to hear your thoughts on it.
Cheers jonas

Hey again,
Jonas Meurer:
I've now done this in the Debian mailman3-web package and hope to get this into Buster before release.
For that, I now also have a working implementation as well. Due to the change being rather invasive, I will not push this into Buster though. And my plan slightly changed during implementing it, see below ;)
That's what I did now, and it already was enough to drop the build-time dependency on sassc. Django-compressor is still used to combine several CSS and JS files into one by running 'django-admin compress' in the postinst script of mailman3-web, but since the source files are already clean CSS and JS, no external processors (like sassc or lessc) are required.
I think this is perfectly fine and I don't see much benefit in combining the CSS and JS files at build-time. It would allow to drop the django-compressor dependency but with the cost of more heavy build-time adjustments that need to be maintained in future.
So I'm happy to now have a solution to drop both node-less and sassc from runtime dependencies in the Debian package. Thanks to everyone for their input on this topic.
I added a comment to upstream hyperkitty issue #120 where I proposed to do the same in the upstream hyperkitty release process and that way get rid of the sassc runtime dependency:
https://gitlab.com/mailman/hyperkitty/issues/120#note_179256370
Cheers jonas

[...]
I agree, the extra gain by this would be minimal at a rather high maintenance cost.
Thank you so much!
I think that also eases the maintenance of a live mailman deployment having much less complex components installed.
I understand that this won't be for Buster. Just curious for related Ubuntu planning - are you planning to do an upload to experimental ahead of time or are you waiting until Buster is released?
Great, that will help to keep things in sync.
participants (3)
-
Abhilash Raj
-
Christian Ehrhardt
-
Jonas Meurer