I'm repeating myself a bit, but my previous thread of this ended up being about something else, and also since then I've been on an expedition to the hostile waters of python-dev.
I'm crazy enough to believe that I'm proposing a technical solution to alleviate the problems we've faced as a community the past year. No, this will NOT be about NA, and certainly not governance, but do please allow me one paragraph of musings before the meaty stuff.
I believe the Achilles heel of NumPy is the C API and the PyArrayObject. The reliance we all have on the NumPy C API means there can in practice only be one "array" type per Python process. This makes people *very* afraid of creative forking or new competing array libraries (since they just can't live in parallel -- like Cython and Pyrex can!), and every new feature has to go into ndarray to fully realise itself. This in turn means that experimentation with new features has to happen within one or a few release cycles, it cannot happen in the wild and by competition and by seeing what works over the course of years before finally making it into upstream. Finally, if any new great idea can really only be implemented decently if it also impacts thousands of users...that's bad both for morale and developer recruitment.
The meat:
There's already of course been work on making the NumPy C API work through an indirection layer to make a more stable ABI. This is about changing the ideas of how that indirection should happen, so that you could in theory implement the C API independently of NumPy.
You could for instance make a "mini-NumPy" that only contains the bare essentials, and load that in the same process as the real NumPy, and use the C API against objects from both libraries.
I'll assume that we can get a PEP through by waving a magic wand, since that makes it easier to focus on essentials. There's many ugly or less ugly hacks to make it work on any existing CPython [1], and they wouldn't be so ugly if there's PEP blessing for the general idea.
Imagine if PyTypeObject grew an extra pointer "tp_customslots", which pointed to an array of these:
typedef struct { unsigned long tpe_id; void *tpe_data; } PyTypeObjectCustomSlot;
The ID space is partitioned to anyone who asks, and NumPy is given a large chunk. To insert a "custom slot", you stick it in this list. And you search it linearly for, say, PYTYPE_CUSTOM_NUMPY_SLOT (each type will typically have 0-3 entries so the search is very fast).
I've benchmarked something very similar recently, and the overhead in a "hot" situation is on the order of 4-6 cycles. (As for cache, you can at least stick the slot array right next to the type object in memory.)
Now, a NumPy array would populate this list with 1-2 entries pointing to tables of function pointers for the NumPy C API. This lookup through the PyTypeObject would in part replace the current import_array() mechanism.
I'd actually propose two such custom slots for ndarray for starters:
a) One PEP 3118-like binary description that exposes raw data pointers (without the PEP 3118 red tape)
b) A function pointer table for a suitable subset of the NumPy C API (obviously not array construction and so on)
The all-important PyArray_DATA/DIMS/... would be macros that try for a) first, but fall back to b). Things like PyArray_Check would actually check for support of these slots, "duck typing", rather than the Python type (of course, this could only be done at a major revision like NumPy 2.0 or 3.0).
The overhead should be on the order of 5 cycles per C API call. That should be fine for anything but the use of PyArray_DATA inside a tight loop (which is a bad idea anyway).
For now I just want to establish if there's support for this general idea, and see if I can get some weight behind a PEP (and ideally a co-author), which would make this a general approach and something more than an ugly NumPy specific hack. We'd also have good use for such a PEP in Cython (and, I believe, Numba/SciPy in CEP 1000).
Dag
[1] There's many ways of doing similar things in current Python, such as standardising across many participating projects on using a common metaclass. Here's another alternative that doesn't add such inter-project dependencies but is more renegade: http://wiki.cython.org/enhancements/cep1001
Dag Sverre Seljebotn d.s.seljebotn@astro.uio.no wrote:
I'm repeating myself a bit, but my previous thread of this ended up being about something else, and also since then I've been on an expedition to the hostile waters of python-dev.
I'm crazy enough to believe that I'm proposing a technical solution to alleviate the problems we've faced as a community the past year. No, this will NOT be about NA, and certainly not governance, but do please allow me one paragraph of musings before the meaty stuff.
I believe the Achilles heel of NumPy is the C API and the PyArrayObject. The reliance we all have on the NumPy C API means there can in practice
only be one "array" type per Python process. This makes people *very* afraid of creative forking or new competing array libraries (since they
just can't live in parallel -- like Cython and Pyrex can!), and every new feature has to go into ndarray to fully realise itself. This in turn means that experimentation with new features has to happen within one or a few release cycles, it cannot happen in the wild and by competition and by seeing what works over the course of years before finally making
it into upstream. Finally, if any new great idea can really only be implemented decently if it also impacts thousands of users...that's bad
both for morale and developer recruitment.
The meat:
There's already of course been work on making the NumPy C API work through an indirection layer to make a more stable ABI. This is about changing the ideas of how that indirection should happen, so that you could in theory implement the C API independently of NumPy.
You could for instance make a "mini-NumPy" that only contains the bare essentials, and load that in the same process as the real NumPy, and use the C API against objects from both libraries.
I'll assume that we can get a PEP through by waving a magic wand, since
that makes it easier to focus on essentials. There's many ugly or less ugly hacks to make it work on any existing CPython [1], and they wouldn't be so ugly if there's PEP blessing for the general idea.
Imagine if PyTypeObject grew an extra pointer "tp_customslots", which pointed to an array of these:
typedef struct { unsigned long tpe_id; void *tpe_data; } PyTypeObjectCustomSlot;
The ID space is partitioned to anyone who asks, and NumPy is given a large chunk. To insert a "custom slot", you stick it in this list. And you search it linearly for, say, PYTYPE_CUSTOM_NUMPY_SLOT (each type will typically have 0-3 entries so the search is very fast).
I've benchmarked something very similar recently, and the overhead in a
"hot" situation is on the order of 4-6 cycles. (As for cache, you can at least stick the slot array right next to the type object in memory.)
Now, a NumPy array would populate this list with 1-2 entries pointing to tables of function pointers for the NumPy C API. This lookup through the PyTypeObject would in part replace the current import_array() mechanism.
I'd actually propose two such custom slots for ndarray for starters:
a) One PEP 3118-like binary description that exposes raw data pointers
To be more clear: the custom-slot in the pytypeobject would contain an offset that you could add to your PyObject* to get to this information.
Dag
(without the PEP 3118 red tape)
b) A function pointer table for a suitable subset of the NumPy C API (obviously not array construction and so on)
The all-important PyArray_DATA/DIMS/... would be macros that try for a)
first, but fall back to b). Things like PyArray_Check would actually check for support of these slots, "duck typing", rather than the Python
type (of course, this could only be done at a major revision like NumPy
2.0 or 3.0).
The overhead should be on the order of 5 cycles per C API call. That should be fine for anything but the use of PyArray_DATA inside a tight loop (which is a bad idea anyway).
For now I just want to establish if there's support for this general idea, and see if I can get some weight behind a PEP (and ideally a co-author), which would make this a general approach and something more
than an ugly NumPy specific hack. We'd also have good use for such a PEP in Cython (and, I believe, Numba/SciPy in CEP 1000).
Dag
[1] There's many ways of doing similar things in current Python, such as standardising across many participating projects on using a common metaclass. Here's another alternative that doesn't add such inter-project dependencies but is more renegade: http://wiki.cython.org/enhancements/cep1001 _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Dag Sverre Seljebotn d.s.seljebotn@astro.uio.no wrote:
I'm repeating myself a bit, but my previous thread of this ended up being about something else, and also since then I've been on an expedition to the hostile waters of python-dev.
I'm crazy enough to believe that I'm proposing a technical solution to alleviate the problems we've faced as a community the past year. No, this will NOT be about NA, and certainly not governance, but do please allow me one paragraph of musings before the meaty stuff.
I believe the Achilles heel of NumPy is the C API and the PyArrayObject. The reliance we all have on the NumPy C API means there can in practice
only be one "array" type per Python process. This makes people *very* afraid of creative forking or new competing array libraries (since they
just can't live in parallel -- like Cython and Pyrex can!), and every new feature has to go into ndarray to fully realise itself. This in turn means that experimentation with new features has to happen within one or a few release cycles, it cannot happen in the wild and by competition and by seeing what works over the course of years before finally making
it into upstream. Finally, if any new great idea can really only be implemented decently if it also impacts thousands of users...that's bad
both for morale and developer recruitment.
Sorry for using the F-word so much, it is not accurate.
What I mean is complementary array features like resizeable arrays, arrays tuned for small n, arrays with metadata, lazy arrays, missing data concepts.. Today you can't just make a new array class and stick it on PyPI, because it wouldn't work seamlessly with C extensions. And that means, since it must be part of ndarray, you can't distribute it easily to others. So you can't depend on it for your own work and give it real testing without modifying ndarray, and that means everything new must be perfect on the first try, which is damaging.
Dag
The meat:
There's already of course been work on making the NumPy C API work through an indirection layer to make a more stable ABI. This is about changing the ideas of how that indirection should happen, so that you could in theory implement the C API independently of NumPy.
You could for instance make a "mini-NumPy" that only contains the bare essentials, and load that in the same process as the real NumPy, and use the C API against objects from both libraries.
I'll assume that we can get a PEP through by waving a magic wand, since
that makes it easier to focus on essentials. There's many ugly or less ugly hacks to make it work on any existing CPython [1], and they wouldn't be so ugly if there's PEP blessing for the general idea.
Imagine if PyTypeObject grew an extra pointer "tp_customslots", which pointed to an array of these:
typedef struct { unsigned long tpe_id; void *tpe_data; } PyTypeObjectCustomSlot;
The ID space is partitioned to anyone who asks, and NumPy is given a large chunk. To insert a "custom slot", you stick it in this list. And you search it linearly for, say, PYTYPE_CUSTOM_NUMPY_SLOT (each type will typically have 0-3 entries so the search is very fast).
I've benchmarked something very similar recently, and the overhead in a
"hot" situation is on the order of 4-6 cycles. (As for cache, you can at least stick the slot array right next to the type object in memory.)
Now, a NumPy array would populate this list with 1-2 entries pointing to tables of function pointers for the NumPy C API. This lookup through the PyTypeObject would in part replace the current import_array() mechanism.
I'd actually propose two such custom slots for ndarray for starters:
a) One PEP 3118-like binary description that exposes raw data pointers (without the PEP 3118 red tape)
b) A function pointer table for a suitable subset of the NumPy C API (obviously not array construction and so on)
The all-important PyArray_DATA/DIMS/... would be macros that try for a)
first, but fall back to b). Things like PyArray_Check would actually check for support of these slots, "duck typing", rather than the Python
type (of course, this could only be done at a major revision like NumPy
2.0 or 3.0).
The overhead should be on the order of 5 cycles per C API call. That should be fine for anything but the use of PyArray_DATA inside a tight loop (which is a bad idea anyway).
For now I just want to establish if there's support for this general idea, and see if I can get some weight behind a PEP (and ideally a co-author), which would make this a general approach and something more
than an ugly NumPy specific hack. We'd also have good use for such a PEP in Cython (and, I believe, Numba/SciPy in CEP 1000).
Dag
[1] There's many ways of doing similar things in current Python, such as standardising across many participating projects on using a common metaclass. Here's another alternative that doesn't add such inter-project dependencies but is more renegade: http://wiki.cython.org/enhancements/cep1001 _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
On 17 May 2012 23:53, Dag Sverre Seljebotn d.s.seljebotn@astro.uio.no wrote:
I'm repeating myself a bit, but my previous thread of this ended up being about something else, and also since then I've been on an expedition to the hostile waters of python-dev.
I'm crazy enough to believe that I'm proposing a technical solution to alleviate the problems we've faced as a community the past year. No, this will NOT be about NA, and certainly not governance, but do please allow me one paragraph of musings before the meaty stuff.
I believe the Achilles heel of NumPy is the C API and the PyArrayObject. The reliance we all have on the NumPy C API means there can in practice only be one "array" type per Python process. This makes people *very* afraid of creative forking or new competing array libraries (since they just can't live in parallel -- like Cython and Pyrex can!), and every new feature has to go into ndarray to fully realise itself. This in turn means that experimentation with new features has to happen within one or a few release cycles, it cannot happen in the wild and by competition and by seeing what works over the course of years before finally making it into upstream. Finally, if any new great idea can really only be implemented decently if it also impacts thousands of users...that's bad both for morale and developer recruitment.
The meat:
There's already of course been work on making the NumPy C API work through an indirection layer to make a more stable ABI. This is about changing the ideas of how that indirection should happen, so that you could in theory implement the C API independently of NumPy.
You could for instance make a "mini-NumPy" that only contains the bare essentials, and load that in the same process as the real NumPy, and use the C API against objects from both libraries.
I'll assume that we can get a PEP through by waving a magic wand, since that makes it easier to focus on essentials. There's many ugly or less ugly hacks to make it work on any existing CPython [1], and they wouldn't be so ugly if there's PEP blessing for the general idea.
Imagine if PyTypeObject grew an extra pointer "tp_customslots", which pointed to an array of these:
typedef struct { unsigned long tpe_id; void *tpe_data; } PyTypeObjectCustomSlot;
The ID space is partitioned to anyone who asks, and NumPy is given a large chunk. To insert a "custom slot", you stick it in this list. And you search it linearly for, say, PYTYPE_CUSTOM_NUMPY_SLOT (each type will typically have 0-3 entries so the search is very fast).
I've benchmarked something very similar recently, and the overhead in a "hot" situation is on the order of 4-6 cycles. (As for cache, you can at least stick the slot array right next to the type object in memory.)
Now, a NumPy array would populate this list with 1-2 entries pointing to tables of function pointers for the NumPy C API. This lookup through the PyTypeObject would in part replace the current import_array() mechanism.
I'd actually propose two such custom slots for ndarray for starters:
a) One PEP 3118-like binary description that exposes raw data pointers (without the PEP 3118 red tape)
b) A function pointer table for a suitable subset of the NumPy C API (obviously not array construction and so on)
The all-important PyArray_DATA/DIMS/... would be macros that try for a) first, but fall back to b). Things like PyArray_Check would actually check for support of these slots, "duck typing", rather than the Python type (of course, this could only be done at a major revision like NumPy 2.0 or 3.0).
The overhead should be on the order of 5 cycles per C API call. That should be fine for anything but the use of PyArray_DATA inside a tight loop (which is a bad idea anyway).
For now I just want to establish if there's support for this general idea, and see if I can get some weight behind a PEP (and ideally a co-author), which would make this a general approach and something more than an ugly NumPy specific hack. We'd also have good use for such a PEP in Cython (and, I believe, Numba/SciPy in CEP 1000).
Well, you have my vote, but you already knew that. I'd also be willing to co-author any PEP etc, but I'm sensing it may be more useful to have support from people from different projects. Personally, I think if this is to succeed, we first need to fix the design to work for subclasses (I think one may just want to memcpy the interface information over to the subclass, e.g. through a convenience function that allows one to add more as well). If we have a solid idea of the technical implementation, we should actually implement it and present the benchmarks, comparing the results to capsules as attributes (and to the _PyType_Lookup approach).
If we have come that far, then even at that point people may argue that one could do the same thing through a metaclass, and that this is too project specific. If people don't care for the standardization, then we could just keep this as a "scientific community proposal". That would be somewhat disappointing as it hurts standardization, adoption and elegance.
BTW, if we propose it, it would probably be a good idea to have concrete examples of how non-standardization, e.g. PyCUDA/PyOpenCL, Theano, Numpy, GNumpy, etc have different arrays. See http://deeplearning.net/software/theano/tutorial/gpu_data_convert.html , it may help drive the point through. (Of course GPU arrays can't be handled through a numpy-standardized duck-typing interface, but the ones that live in main memory can. Furthermore, this same proposal can help standardize conversion and operations between GPU arrays as well). If we can find even more examples, preferably outside of the scientific community, where related projects face a similar situation, it may help people understand that this is not "a Numpy problem".
Dag
[1] There's many ways of doing similar things in current Python, such as standardising across many participating projects on using a common metaclass. Here's another alternative that doesn't add such inter-project dependencies but is more renegade: http://wiki.cython.org/enhancements/cep1001 _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
On Fri, 2012-05-18 at 12:48 +0100, mark florisson wrote:
If we can find even more examples, preferably outside of the scientific community, where related projects face a similar situation, it may help people understand that this is not "a Numpy problem".
Buffer Objects in OpenGL?
Henry Gomersall heng@cantab.net wrote:
On Fri, 2012-05-18 at 12:48 +0100, mark florisson wrote:
If we can find even more examples, preferably outside of the scientific community, where related projects face a similar
situation,
it may help people understand that this is not "a Numpy problem".
Buffer Objects in OpenGL?
There is already PEP 3118 though.
I would focus on the 'polymorphic C API' spin. PyObject_GetItem is polymorphic, but there is no standard way for 3rd party libraries to make such functions.
So let's find a C API that's NOT about arrays at all and show how some polymorphism may help.
Of course there's a couple of non-array Cython usecases too.
Dag
NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
On Fri, 2012-05-18 at 14:45 +0200, Dag Sverre Seljebotn wrote:
I would focus on the 'polymorphic C API' spin. PyObject_GetItem is polymorphic, but there is no standard way for 3rd party libraries to make such functions.
So let's find a C API that's NOT about arrays at all and show how some polymorphism may help.
so memory mapped IO or something?
On 05/18/2012 05:00 PM, Henry Gomersall wrote:
On Fri, 2012-05-18 at 14:45 +0200, Dag Sverre Seljebotn wrote:
I would focus on the 'polymorphic C API' spin. PyObject_GetItem is polymorphic, but there is no standard way for 3rd party libraries to make such functions.
So let's find a C API that's NOT about arrays at all and show how some polymorphism may help.
so memory mapped IO or something?
A C API towards a Python extension that's in wide use; an analog to the NumPy C API in a different domain. Something like a web server or database Python extension module which also exports a C API for writing other Python extension modules against it.
I'm not even sure if such a thing exists, in which case NumPy would indeed be a special case.
Dag
On Fri, May 18, 2012 at 6:59 PM, Dag Sverre Seljebotn d.s.seljebotn@astro.uio.no wrote:
On 05/18/2012 05:00 PM, Henry Gomersall wrote:
On Fri, 2012-05-18 at 14:45 +0200, Dag Sverre Seljebotn wrote:
I would focus on the 'polymorphic C API' spin. PyObject_GetItem is polymorphic, but there is no standard way for 3rd party libraries to make such functions.
So let's find a C API that's NOT about arrays at all and show how some polymorphism may help.
so memory mapped IO or something?
A C API towards a Python extension that's in wide use; an analog to the NumPy C API in a different domain. Something like a web server or database Python extension module which also exports a C API for writing other Python extension modules against it.
I'm not even sure if such a thing exists, in which case NumPy would indeed be a special case.
numpy *is* pretty unique in this regard. The need for this style of polymorphism at this level is even rarer.
Hi,
The example with numpy array for small array, the speed problem is probably because NumPy have not been speed optimized for low overhead. For example, each c function should check first if the input is a NumPy array, if not jump to a function to make one. For example, currently in the c function(PyArray_Multiply?) that got called by dot(), a c function call is made to check if the array is a NumPy array. This is an extra overhead for the expected most frequent expected behavior that the input is a NumPy array. I'm pretty sure this happen at many place. In this particular function, there is many other function call before calling blas just for the simple case of vector x vector, vector x matrix or matrix x matrix dot product.
But this is probably for another thread if people want to discuss it more. Also, I didn't verify how frequently we could lower the overhead as we don't need it. So it could be just a few function that need those type of optimization.
For the comparison with the multiple type of array on the GPU, I think the first reason is that people worked isolated and that the only implemented the subset of the numpy ndarray they needed. As different project/groups need different part, reusing other people work was not trivial.
Otherwise, I see the problem, but I don't know what to say about it as I didn't experience it.
Fred
On 05/18/2012 01:48 PM, mark florisson wrote:
On 17 May 2012 23:53, Dag Sverre Seljebotnd.s.seljebotn@astro.uio.no wrote:
I'm repeating myself a bit, but my previous thread of this ended up being about something else, and also since then I've been on an expedition to the hostile waters of python-dev.
I'm crazy enough to believe that I'm proposing a technical solution to alleviate the problems we've faced as a community the past year. No, this will NOT be about NA, and certainly not governance, but do please allow me one paragraph of musings before the meaty stuff.
I believe the Achilles heel of NumPy is the C API and the PyArrayObject. The reliance we all have on the NumPy C API means there can in practice only be one "array" type per Python process. This makes people *very* afraid of creative forking or new competing array libraries (since they just can't live in parallel -- like Cython and Pyrex can!), and every new feature has to go into ndarray to fully realise itself. This in turn means that experimentation with new features has to happen within one or a few release cycles, it cannot happen in the wild and by competition and by seeing what works over the course of years before finally making it into upstream. Finally, if any new great idea can really only be implemented decently if it also impacts thousands of users...that's bad both for morale and developer recruitment.
The meat:
There's already of course been work on making the NumPy C API work through an indirection layer to make a more stable ABI. This is about changing the ideas of how that indirection should happen, so that you could in theory implement the C API independently of NumPy.
You could for instance make a "mini-NumPy" that only contains the bare essentials, and load that in the same process as the real NumPy, and use the C API against objects from both libraries.
I'll assume that we can get a PEP through by waving a magic wand, since that makes it easier to focus on essentials. There's many ugly or less ugly hacks to make it work on any existing CPython [1], and they wouldn't be so ugly if there's PEP blessing for the general idea.
Imagine if PyTypeObject grew an extra pointer "tp_customslots", which pointed to an array of these:
typedef struct { unsigned long tpe_id; void *tpe_data; } PyTypeObjectCustomSlot;
The ID space is partitioned to anyone who asks, and NumPy is given a large chunk. To insert a "custom slot", you stick it in this list. And you search it linearly for, say, PYTYPE_CUSTOM_NUMPY_SLOT (each type will typically have 0-3 entries so the search is very fast).
I've benchmarked something very similar recently, and the overhead in a "hot" situation is on the order of 4-6 cycles. (As for cache, you can at least stick the slot array right next to the type object in memory.)
Now, a NumPy array would populate this list with 1-2 entries pointing to tables of function pointers for the NumPy C API. This lookup through the PyTypeObject would in part replace the current import_array() mechanism.
I'd actually propose two such custom slots for ndarray for starters:
a) One PEP 3118-like binary description that exposes raw data pointers (without the PEP 3118 red tape)
b) A function pointer table for a suitable subset of the NumPy C API (obviously not array construction and so on)
The all-important PyArray_DATA/DIMS/... would be macros that try for a) first, but fall back to b). Things like PyArray_Check would actually check for support of these slots, "duck typing", rather than the Python type (of course, this could only be done at a major revision like NumPy 2.0 or 3.0).
The overhead should be on the order of 5 cycles per C API call. That should be fine for anything but the use of PyArray_DATA inside a tight loop (which is a bad idea anyway).
For now I just want to establish if there's support for this general idea, and see if I can get some weight behind a PEP (and ideally a co-author), which would make this a general approach and something more than an ugly NumPy specific hack. We'd also have good use for such a PEP in Cython (and, I believe, Numba/SciPy in CEP 1000).
Well, you have my vote, but you already knew that. I'd also be willing to co-author any PEP etc, but I'm sensing it may be more useful to have support from people from different projects. Personally, I think if this is to succeed, we first need to fix the design to work for subclasses (I think one may just want to memcpy the interface information over to the subclass, e.g. through a convenience function that allows one to add more as well). If we have a solid idea of the technical implementation, we should actually implement it and present the benchmarks, comparing the results to capsules as attributes (and to the _PyType_Lookup approach).
Unless there's any holes in my fresh metaclass implementation, I think that is good enough that we can wait a year and get actual adoption before pushing for a PEP. That would also make the PEP a lot stronger. I do believe it should happen eventually though.
Here's my post on the Cython list reposted to this list:
""" So I finally got around to implementing this:
https://github.com/dagss/pyextensibletype
Documentation now in a draft in the NumFOCUS SEP repo, which I believe is a better place to store cross-project standards like this. (The NumPy docstring standard will be SEP 100).
https://github.com/numfocus/sep/blob/master/sep200.rst
Summary:
- No common runtime dependency
- 1 ns overhead per lookup (that's for the custom slot *alone*, no fast-callable signature matching or similar)
- Slight annoyance: Types that want to use the metaclass must be a PyHeapExtensibleType, to make the binary layout work with how CPython makes subclasses from Python scripts
My conclusion: I think the metaclass approach should work really well. """
Dag
-
Dag Sverre Seljebotn -
Frédéric Bastien -
Henry Gomersall -
mark florisson -
Robert Kern