Re: How strict are the dependencies on the django-compressor related backends?
On Fri, Mar 15, 2019 at 10:19 AM Jonas Smedegaard <dr@jones.dk> wrote:
Quoting Christian Ehrhardt (2019-03-15 06:39:08)
On Fri, Mar 15, 2019 at 12:08 AM Abhilash Raj <maxking@asynchronous.in> wrote:
On Thu, Mar 14, 2019, at 1:42 PM, Jonas Meurer wrote: Christian Ehrhardt:
I was evaluating the Dependencies of mailman3 in Ubuntu and I was wondering if I could cut them down a bit. One thing that got my attention is the dependency from mailman3-web 1 to nodejs for less 2 and ruby-sass 3.
I was wondering if we could by default not install node-less/ruby-sass and what exactly that would imply.
less is a reinvention of sass: It is often adaptable to sass with a simple patch.
Original sass processor was ruby-sass, but nowadays a better processor is sassc. Or if you want to integrated processing with Python then python3-libsass which uses same core library for its processing.
I have not looked closely on mailman3, but I highly doubt there is a real need for dynamic processing of less/sass during runtime.
Yeah I doubt that as well. In the meantime I also found 1 which seems to have come to the same conclusion for the majority of default setups. For upstream to disable it by default it was mentioned that there is a need for a customize kit which seems fine 1.
There is a further argument to disable the live compressors, at least as I read COMPRESS_OFFLINE that seems helpful for production setups anyway to speed up and safe CPU load.
So while upstream waits until a "customize kit" exists to switch defaults, as a downstream like Debian/Ubuntu I wonder if we already could: pulling it in at runtime
- drop sass/less compilers to a suggest
- disable the live compression in the default config that we ship
- keep the config commented-out to make it easy for admins that want to enable it later
- pre-compress at build time and enable compress_offline for speed
- maybe call
django_admin compress
3 or similar at build time? - having sass/less as build dependencies seems much safer than
- maybe call
@Jonas M.: what do you think about 2 (again maybe after Buster)? Sorry to not know more of this yet, but I hope suggesting what I think might work helps to get us the right way.
- Jonas
--
- Jonas Smedegaard - idealist & Internet-arkitekt
- Tlf.: +45 40843136 Website: http://dr.jones.dk/
[x] quote me freely [ ] ask before reusing [ ] keep private -----BEGIN PGP SIGNATURE-----
iQIzBAABCgAdFiEEn+Ppw2aRpp/1PMaELHwxRsGgASEFAlyLbfwACgkQLHwxRsGg ASHlqw//Vkkke6aulFYeLkq7vQOIt08j+DMMDBpJ/vXapwtQJ2WbWEYsHbbFv75v k80SILudsOyGMwQ0HKRGqDmhJw2Voi9mWoV2fBQeCQn5w/G9N2Ddw8hAMrzGsLxO C2FOV8Db0irJ6FS6AO0YQpAselKnnX387AvyWBCpVcWWuydf46pMkPpYajVJTvZg UZxtBh1epmv/VdQlw2OeLrV+bZJWfjULjr1qFPYdqFnEUnOA2cC44RMagxSV7kNh JmCc7DvL/VjX/dkPHJCW/VvRrvfxzOR4bimVr7BeuvcjNljlUnKhRwP+PQZt80gp CbOoVCeEOFJuHwqTlvasUpxo/aSZZPFeF8aQBtdyzefEx9QjjGwx8bYQFn+Zo0CX TVsB7qWyCwMZIR0DP5+A8ityXQb+FwOTXjySMzO+Z/rA9+h1PHZrK+WJMB0O9z5r hWYDk3jGSJIkwL+AemE06gUPuSaXZWhzoduI2Mbzs8a20qhrW08Ij338RVGalyhC QHJT4JoI3bXylaK+IEpH9xxUEW0i8gOge32dBFiqg9/dGFm06Tf3z8dUuBFyXgG3 BsNJaO6Cn3ghfNS/OfNN04nPS+luPqPQ1r9yZMztrIqwv8oOWyVj1YQpKW1vRN/v BK/2IpBA+6O3OE2X4RCW8pouwPmsSgMmXpIsuFE61dox/8tOPmE= =490O -----END PGP SIGNATURE-----
-- Christian Ehrhardt Software Engineer, Ubuntu Server Canonical Ltd
On Mon, Mar 18, 2019, at 1:04 AM, Christian Ehrhardt wrote:
On Fri, Mar 15, 2019 at 10:19 AM Jonas Smedegaard <dr@jones.dk> wrote:
Quoting Christian Ehrhardt (2019-03-15 06:39:08)
On Fri, Mar 15, 2019 at 12:08 AM Abhilash Raj <maxking@asynchronous.in> wrote:
On Thu, Mar 14, 2019, at 1:42 PM, Jonas Meurer wrote: Christian Ehrhardt:
I was evaluating the Dependencies of mailman3 in Ubuntu and I was wondering if I could cut them down a bit. One thing that got my attention is the dependency from mailman3-web 1 to nodejs for less 2 and ruby-sass 3.
I was wondering if we could by default not install node-less/ruby-sass and what exactly that would imply.
less is a reinvention of sass: It is often adaptable to sass with a simple patch.
Original sass processor was ruby-sass, but nowadays a better processor is sassc. Or if you want to integrated processing with Python then python3-libsass which uses same core library for its processing.
I have not looked closely on mailman3, but I highly doubt there is a real need for dynamic processing of less/sass during runtime.
Yeah I doubt that as well. In the meantime I also found 1 which seems to have come to the same conclusion for the majority of default setups. For upstream to disable it by default it was mentioned that there is a need for a customize kit which seems fine 1.
There is a further argument to disable the live compressors, at least as I read COMPRESS_OFFLINE that seems helpful for production setups anyway to speed up and safe CPU load.
So while upstream waits until a "customize kit" exists to switch defaults, as a downstream like Debian/Ubuntu I wonder if we already could: pulling it in at runtime
- drop sass/less compilers to a suggest
- disable the live compression in the default config that we ship
- keep the config commented-out to make it easy for admins that want to enable it later
- pre-compress at build time and enable compress_offline for speed
- maybe call
django_admin compress
3 or similar at build time?- having sass/less as build dependencies seems much safer than
+1 from me. It totally makes sense for any Mailman package to pre-compress the static files and not depend on any of those compilers.
Upstream doesn't declare any dependencies on sass or less compilers, since there is no way to do so, I don't think there is need to change anything except settings.py.
@Jonas M.: what do you think about 2 (again maybe after Buster)? Sorry to not know more of this yet, but I hope suggesting what I think might work helps to get us the right way.
- Jonas
--
- Jonas Smedegaard - idealist & Internet-arkitekt
- Tlf.: +45 40843136 Website: http://dr.jones.dk/
[x] quote me freely [ ] ask before reusing [ ] keep private -----BEGIN PGP SIGNATURE-----
iQIzBAABCgAdFiEEn+Ppw2aRpp/1PMaELHwxRsGgASEFAlyLbfwACgkQLHwxRsGg ASHlqw//Vkkke6aulFYeLkq7vQOIt08j+DMMDBpJ/vXapwtQJ2WbWEYsHbbFv75v k80SILudsOyGMwQ0HKRGqDmhJw2Voi9mWoV2fBQeCQn5w/G9N2Ddw8hAMrzGsLxO C2FOV8Db0irJ6FS6AO0YQpAselKnnX387AvyWBCpVcWWuydf46pMkPpYajVJTvZg UZxtBh1epmv/VdQlw2OeLrV+bZJWfjULjr1qFPYdqFnEUnOA2cC44RMagxSV7kNh JmCc7DvL/VjX/dkPHJCW/VvRrvfxzOR4bimVr7BeuvcjNljlUnKhRwP+PQZt80gp CbOoVCeEOFJuHwqTlvasUpxo/aSZZPFeF8aQBtdyzefEx9QjjGwx8bYQFn+Zo0CX TVsB7qWyCwMZIR0DP5+A8ityXQb+FwOTXjySMzO+Z/rA9+h1PHZrK+WJMB0O9z5r hWYDk3jGSJIkwL+AemE06gUPuSaXZWhzoduI2Mbzs8a20qhrW08Ij338RVGalyhC QHJT4JoI3bXylaK+IEpH9xxUEW0i8gOge32dBFiqg9/dGFm06Tf3z8dUuBFyXgG3 BsNJaO6Cn3ghfNS/OfNN04nPS+luPqPQ1r9yZMztrIqwv8oOWyVj1YQpKW1vRN/v BK/2IpBA+6O3OE2X4RCW8pouwPmsSgMmXpIsuFE61dox/8tOPmE= =490O -----END PGP SIGNATURE-----
-- Christian Ehrhardt Software Engineer, Ubuntu Server Canonical Ltd
-- thanks, Abhilash Raj (maxking)
[I've put the related Debian bugreport in the loop]
Hello,
I finally found some time to look into this.
Disclaimer: I don't know much about CSS compression and the mailman3 web interface is the first time that I got into contact with either of sassc and node-less.
TL;DR:
- Hyperkitty doesn't need (or use) less.js/lessc at all. It can safely be removed from COMPRESS_PRECOMPILERS.
- The situation is different with sassc, it's actually used for compressing CSS by hyperkitty. Doing the compression at build-time should be possible but requires more work. I have a plan ;)
Christian Ehrhardt:
I have checked a fully installed mailman3 stack in Debian/Ubuntu and there are only two sources of .less files that we have to be concerned of.
First there is the package fonts-font-awesome which is pulled in on the following path: [...] It seems different for the second case which is libjs-bootstrap from the source package twitter-bootstrap3
I don't think that any of the .less files you're referencing to will be processed by 'django-admin compress' on the mailman3-web project. The .less files are all outside the scope of the mailman3-web project. So I'm pretty sure that the node-less/lessc dependency can safely be dropped immediately.
Moreover, looking at the hyperkitty example_project/settings.py[1], it doesn't even reference the lessc compiler at all. Only settings.py from mailman-suite[2] references it, which probably is a copy-and-paste bug. There's even an open issue about that[3]. Since in Debian we took the mailman-suite settings.py as template, we suffer from this copy-and-paste bug as well.
[1] https://gitlab.com/mailman/hyperkitty/blob/master/example_project/settings.p... [2] https://gitlab.com/mailman/mailman-suite/blob/master/mailman-suite_project/s... [3] https://gitlab.com/mailman/mailman-suite/issues/7
Does anyone know of other uses of .less in the mailman3 stack that I have missed that we would have to think about?
Regarding sassc: Hyperkitty is the only part of the mailman3 stack that uses django-compressor. And apparently there's just one single html template that will be processed by sassc (well, to be precise, all html files that use this template):
/usr/lib/python3/dist-packages/hyperkitty/templates/hyperkitty/base.html
In this file, four CSS files are being compressed and packed together by sassc:
hyperkitty/libs/fonts/icomoon/icomoon.css hyperkitty/libs/fonts/droid/droid.css django-mailman3/css/main.css hyperkitty/sass/hyperkitty.scss
The last one is special as it's a scss file itself and needs to be processed by sassc itself.
OTOH there also is an experimental approach to this, could someone with a dev-stack of mailman3 just remove the node-less package and see if on a normal setup something breaks at all?
I've done this. Result: removing node-less doesn't change anything (which is in line with my assumption that node-less/less.js/lessc is not required/used at all).
On the other hand, removing sassc breaks the setup, even with COMPRESS_OFFLINE being turned on:
CommandError: An error occurred during rendering /usr/lib/python3/dist-packages/django_mailman3/templates/django_mailman3/profile/delete_profile.html: /bin/sh: 1: sassc: not found
Christian Ehrhardt:
On Fri, Mar 15, 2019 at 10:19 AM Jonas Smedegaard <dr@jones.dk> wrote:
Quoting Christian Ehrhardt (2019-03-15 06:39:08)
On Fri, Mar 15, 2019 at 12:08 AM Abhilash Raj <maxking@asynchronous.in> wrote:
On Thu, Mar 14, 2019, at 1:42 PM, Jonas Meurer wrote: Christian Ehrhardt:
I was evaluating the Dependencies of mailman3 in Ubuntu and I was wondering if I could cut them down a bit. One thing that got my attention is the dependency from mailman3-web [1] to nodejs for less [2] and ruby-sass [3].
I was wondering if we could by default not install node-less/ruby-sass and what exactly that would imply.
less is a reinvention of sass: It is often adaptable to sass with a simple patch.
Original sass processor was ruby-sass, but nowadays a better processor is sassc. Or if you want to integrated processing with Python then python3-libsass which uses same core library for its processing.
I have not looked closely on mailman3, but I highly doubt there is a real need for dynamic processing of less/sass during runtime.
Yeah I doubt that as well. In the meantime I also found [1] which seems to have come to the same conclusion for the majority of default setups.
For upstream to disable it by default it was mentioned that there is a need for a customize kit which seems fine [1].
There is a further argument to disable the live compressors, at least as I read COMPRESS_OFFLINE that seems helpful for production setups anyway to speed up and safe CPU load.
You're totally right. Therefore, COMPRESS_OFFLINE is already enabled per default in mailman3-web on Debian. The django-admin compress command is executed during package installation and upgrade by the package postinst script.
So while upstream waits until a "customize kit" exists to switch defaults, as a downstream like Debian/Ubuntu I wonder if we already could: pulling it in at runtime
- drop sass/less compilers to a suggest
- disable the live compression in the default config that we ship
- keep the config commented-out to make it easy for admins that want to enable it later
- pre-compress at build time and enable compress_offline for speed
- maybe call
django_admin compress
[3][4] or similar at build time?- having sass/less as build dependencies seems much safer than
@Jonas M.: what do you think about [2] (again maybe after Buster)? Sorry to not know more of this yet, but I hope suggesting what I think might work helps to get us the right way.
I'm absolutely in favour of getting rid of the sassc runtime dependency. After a chat with Jonas Smedegaard (thanks for his valuable input!), I think that I have an idea and a roadmap on how to get there:
- Process
static/hyperkittysass/hyperkitty.scss
with sassc at build- time and patchtemplates/hyperkitty/base.html
to reference the resulting CSS file. - Patch base.html at build-time to remove the '{% compress css %}' statements from it.
By then, we already can safely drop the run-time dependency on django-compressor from hyperkitty since it won't be needed anymore and consequently remove sassc dependency from mailman3-web (and all references to django-compressor from settings.py).
The downside will be some performance decrease as hyperkitty pages will reference four separate CSS file instead of a combined one. But that can be addressed as well:
- Make python3-django-mailman3 a build-dependency of mailman-suite (one of the referenced CSS files is shipped by django-mailman3) and combine all four CSS files into one by runing sassc at build-time.
- Further patch base.html at build-time to replace the four separate CSS references by one to the resulting combined CSS.
I plan to start working on the actual implementation soon but would be glad to hear your thoughts on it.
Cheers jonas
Hey again,
Jonas Meurer:
I finally found some time to look into this.
- Hyperkitty doesn't need (or use) less.js/lessc at all. It can safely be removed from COMPRESS_PRECOMPILERS.
I've now done this in the Debian mailman3-web package and hope to get this into Buster before release.
- The situation is different with sassc, it's actually used for compressing CSS by hyperkitty. Doing the compression at build-time should be possible but requires more work. I have a plan ;)
For that, I now also have a working implementation as well. Due to the change being rather invasive, I will not push this into Buster though. And my plan slightly changed during implementing it, see below ;)
I was wondering if we could by default not install node-less/ruby-sass and what exactly that would imply.
@Jonas M.: what do you think about [2] (again maybe after Buster)? Sorry to not know more of this yet, but I hope suggesting what I think might work helps to get us the right way.
I'm absolutely in favour of getting rid of the sassc runtime dependency. After a chat with Jonas Smedegaard (thanks for his valuable input!), I think that I have an idea and a roadmap on how to get there:
- Process
static/hyperkittysass/hyperkitty.scss
with sassc at build- time and patchtemplates/hyperkitty/base.html
to reference the resulting CSS file.
That's what I did now, and it already was enough to drop the build-time dependency on sassc. Django-compressor is still used to combine several CSS and JS files into one by running 'django-admin compress' in the postinst script of mailman3-web, but since the source files are already clean CSS and JS, no external processors (like sassc or lessc) are required.
I think this is perfectly fine and I don't see much benefit in combining the CSS and JS files at build-time. It would allow to drop the django-compressor dependency but with the cost of more heavy build-time adjustments that need to be maintained in future.
So I'm happy to now have a solution to drop both node-less and sassc from runtime dependencies in the Debian package. Thanks to everyone for their input on this topic.
I added a comment to upstream hyperkitty issue #120 where I proposed to do the same in the upstream hyperkitty release process and that way get rid of the sassc runtime dependency:
https://gitlab.com/mailman/hyperkitty/issues/120#note_179256370
Cheers jonas
[...]
I think this is perfectly fine and I don't see much benefit in combining the CSS and JS files at build-time. It would allow to drop the django-compressor dependency but with the cost of more heavy build-time adjustments that need to be maintained in future.
I agree, the extra gain by this would be minimal at a rather high maintenance cost.
So I'm happy to now have a solution to drop both node-less and sassc from runtime dependencies in the Debian package. Thanks to everyone for their input on this topic.
Thank you so much!
I think that also eases the maintenance of a live mailman deployment having much less complex components installed.
I understand that this won't be for Buster. Just curious for related Ubuntu planning - are you planning to do an upload to experimental ahead of time or are you waiting until Buster is released?
I added a comment to upstream hyperkitty issue #120 where I proposed to do the same in the upstream hyperkitty release process and that way get rid of the sassc runtime dependency:
https://gitlab.com/mailman/hyperkitty/issues/120#note_179256370
Great, that will help to keep things in sync.
participants (3)
-
Abhilash Raj
-
Christian Ehrhardt
-
Jonas Meurer