One item I admire from Julia and miss in python/numpy, I often use the power of python list comprehension to process data. This data often needs to be converted to numpy for other operations, for example fancy indexing. The fact that operations using comprehensions (which produce lists) and operations on numpy arrays use different incompatible data structures requires conversions between lists and numpy arrays. Comprehensions in Julia produce arrays directly (I believe), removing the need for conversions. I don't see any easy way to improve this. Any ideas? Thanks, Neal -- *Those who don't understand recursion are doomed to repeat it*
If you use Python's own arrays and generator expressions instead of list comprehension (by just dropping the `[ ]`s), you will get each number converted to the target type in memory as soon as it is calculated. (It will be a full Python float/int instance during the calculation itself, though). This won't work with the usual numpy array constructors as those need the array size beforehand - but if you know the size beforehand, there is probably a numpy constructor with no need to go through Python arrays first (but I don't know one by heart) ``` import numpy as np import array data = np.array(array.array("b", (int(127 * cos(i/100)) for i in range(628))), dtype="int8", copy=False) ``` On Fri, Jun 23, 2023 at 10:53 AM Neal Becker <ndbecker2@gmail.com> wrote:
One item I admire from Julia and miss in python/numpy,
I often use the power of python list comprehension to process data. This data often needs to be converted to numpy for other operations, for example fancy indexing. The fact that operations using comprehensions (which produce lists) and operations on numpy arrays use different incompatible data structures requires conversions between lists and numpy arrays. Comprehensions in Julia produce arrays directly (I believe), removing the need for conversions.
I don't see any easy way to improve this. Any ideas?
Thanks, Neal
-- *Those who don't understand recursion are doomed to repeat it* _______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/SRT377... Code of Conduct: http://python.org/psf/codeofconduct/
I think we can do better if we use fromiter and supply the size as count: np.fromiter ((u**2 for u in range(10)), dtype=float, count=10) On Fri, Jun 23, 2023 at 10:39 AM Joao S. O. Bueno <jsbueno@python.org.br> wrote:
If you use Python's own arrays and generator expressions instead of list comprehension (by just dropping the `[ ]`s), you will get each number converted to the target type in memory as soon as it is calculated. (It will be a full Python float/int instance during the calculation itself, though).
This won't work with the usual numpy array constructors as those need the array size beforehand - but if you know the size beforehand, there is probably a numpy constructor with no need to go through Python arrays first (but I don't know one by heart) ``` import numpy as np import array
data = np.array(array.array("b", (int(127 * cos(i/100)) for i in range(628))), dtype="int8", copy=False) ```
On Fri, Jun 23, 2023 at 10:53 AM Neal Becker <ndbecker2@gmail.com> wrote:
One item I admire from Julia and miss in python/numpy,
I often use the power of python list comprehension to process data. This data often needs to be converted to numpy for other operations, for example fancy indexing. The fact that operations using comprehensions (which produce lists) and operations on numpy arrays use different incompatible data structures requires conversions between lists and numpy arrays. Comprehensions in Julia produce arrays directly (I believe), removing the need for conversions.
I don't see any easy way to improve this. Any ideas?
Thanks, Neal
-- *Those who don't understand recursion are doomed to repeat it* _______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/SRT377... Code of Conduct: http://python.org/psf/codeofconduct/
-- *Those who don't understand recursion are doomed to repeat it*
participants (2)
-
Joao S. O. Bueno -
Neal Becker