fixing rockstar interface for python 3
Hi everyone,
I'm trying to close out this bug: https://github.com/yt-project/yt_astro_analysis/issues/18
After spending a fair amount of time on this, all I've been able to determine is that the values of the server address and port seem to go missing for the server. These two values are passed from yt/analysis_modules/halo_finding/rockstar/rockstar.py to rockstar_interface.pyx in the same directory. They are then assigned to two global variables internal to Rockstar. Rockstar instances are either readers, writers, or servers. Both the readers and writers manage to hold on to the correct values for the server address and port, but when I check these values for the server itself, they are just empty strings.
Somehow, this all works fine in python 2, i.e., all rockstar instances maintain the correct values. I am totally out of ideas for how to fix this. Does anyone have any thoughts?
Britton
Hi Britton,
This is very strange. Is there any way that you can manually set these, by making them public or making them operations that set global vars?
-Matt
On Wed, Mar 21, 2018 at 12:18 AM, Britton Smith brittonsmith@gmail.com wrote:
Hi everyone,
I'm trying to close out this bug: https://github.com/yt-project/yt_astro_analysis/issues/18
After spending a fair amount of time on this, all I've been able to determine is that the values of the server address and port seem to go missing for the server. These two values are passed from yt/analysis_modules/halo_finding/rockstar/rockstar.py to rockstar_interface.pyx in the same directory. They are then assigned to two global variables internal to Rockstar. Rockstar instances are either readers, writers, or servers. Both the readers and writers manage to hold on to the correct values for the server address and port, but when I check these values for the server itself, they are just empty strings.
Somehow, this all works fine in python 2, i.e., all rockstar instances maintain the correct values. I am totally out of ideas for how to fix this. Does anyone have any thoughts?
Britton
yt-dev mailing list -- yt-dev@python.org To unsubscribe send an email to yt-dev-leave@python.org
Hi Matt,
Everything I've tried that seems to successfully copy values within the cython code still gives me empty strings by the time it reaches server.c. I even tried using c functions like strcpy, but they seem to cause other memory errors. I've scrubbed the page below for clues, but doing things as they suggest just ends up with the empty string. http://cython.readthedocs.io/en/latest/src/tutorial/strings.html I still can't figure out why the string values make it just fine to client.c, but not server.c.
I think I'm ready to throw in the towel. If someone else has time to take a look, I would appreciate it. This is the last holdup for a yt_astro_analysis release. I'm also fine with just dropping support for this. It has never worked in Python 3, but no one has ever raised the issue, so maybe it's not worth it.
Britton
On Thu, Mar 22, 2018 at 10:28 AM, Matthew Turk matthewturk@gmail.com wrote:
Hi Britton,
This is very strange. Is there any way that you can manually set these, by making them public or making them operations that set global vars?
-Matt
On Wed, Mar 21, 2018 at 12:18 AM, Britton Smith brittonsmith@gmail.com wrote:
Hi everyone,
I'm trying to close out this bug: https://github.com/yt-project/yt_astro_analysis/issues/18
After spending a fair amount of time on this, all I've been able to determine is that the values of the server address and port seem to go missing for the server. These two values are passed from yt/analysis_modules/halo_finding/rockstar/rockstar.py to rockstar_interface.pyx in the same directory. They are then assigned to
two
global variables internal to Rockstar. Rockstar instances are either readers, writers, or servers. Both the readers and writers manage to
hold
on to the correct values for the server address and port, but when I
check
these values for the server itself, they are just empty strings.
Somehow, this all works fine in python 2, i.e., all rockstar instances maintain the correct values. I am totally out of ideas for how to fix
this.
Does anyone have any thoughts?
Britton
yt-dev mailing list -- yt-dev@python.org To unsubscribe send an email to yt-dev-leave@python.org
yt-dev mailing list -- yt-dev@python.org To unsubscribe send an email to yt-dev-leave@python.org
Hi Britton, I think the issue here is that Cython object is unable to track refs to the results of six.b() call and they're garbage collected. There's a simple workaround:
https://paste.pound-python.org/show/QdIThLDuq3YcqRnzwrED/
It doesn't get me a lot further (rockstar client is unable to connect to server), but at least R* server name/port are not mangled. Cheers, Kacper
On 03/24/2018 04:38 PM, Britton Smith wrote:
Hi Matt,
Everything I've tried that seems to successfully copy values within the cython code still gives me empty strings by the time it reaches server.c. I even tried using c functions like strcpy, but they seem to cause other memory errors. I've scrubbed the page below for clues, but doing things as they suggest just ends up with the empty string. http://cython.readthedocs.io/en/latest/src/tutorial/strings.html I still can't figure out why the string values make it just fine to client.c, but not server.c.
I think I'm ready to throw in the towel. If someone else has time to take a look, I would appreciate it. This is the last holdup for a yt_astro_analysis release. I'm also fine with just dropping support for this. It has never worked in Python 3, but no one has ever raised the issue, so maybe it's not worth it.
Britton
On Thu, Mar 22, 2018 at 10:28 AM, Matthew Turk <matthewturk@gmail.com mailto:matthewturk@gmail.com> wrote:
Hi Britton, This is very strange. Is there any way that you can manually set these, by making them public or making them operations that set global vars? -Matt On Wed, Mar 21, 2018 at 12:18 AM, Britton Smith <brittonsmith@gmail.com <mailto:brittonsmith@gmail.com>> wrote: > Hi everyone, > > I'm trying to close out this bug: > https://github.com/yt-project/yt_astro_analysis/issues/18 <https://github.com/yt-project/yt_astro_analysis/issues/18> > > After spending a fair amount of time on this, all I've been able to > determine is that the values of the server address and port seem to go > missing for the server. These two values are passed from > yt/analysis_modules/halo_finding/rockstar/rockstar.py to > rockstar_interface.pyx in the same directory. They are then assigned to two > global variables internal to Rockstar. Rockstar instances are either > readers, writers, or servers. Both the readers and writers manage to hold > on to the correct values for the server address and port, but when I check > these values for the server itself, they are just empty strings. > > Somehow, this all works fine in python 2, i.e., all rockstar instances > maintain the correct values. I am totally out of ideas for how to fix this. > Does anyone have any thoughts? > > Britton > > _______________________________________________ > yt-dev mailing list -- yt-dev@python.org <mailto:yt-dev@python.org> > To unsubscribe send an email to yt-dev-leave@python.org <mailto:yt-dev-leave@python.org> > _______________________________________________ yt-dev mailing list -- yt-dev@python.org <mailto:yt-dev@python.org> To unsubscribe send an email to yt-dev-leave@python.org <mailto:yt-dev-leave@python.org>
yt-dev mailing list -- yt-dev@python.org To unsubscribe send an email to yt-dev-leave@python.org
Well, pin a rose on Kacper's nose, it worked!
Kacper, with your diff, I was able to get the workers to connect and Rockstar to run. It doesn't find any halos when it should with this data, but that's a story for another time. Thanks for figuring that out.
Britton
On Mon, Mar 26, 2018 at 2:03 PM, Kacper Kowalik xarthisius.kk@gmail.com wrote:
Hi Britton, I think the issue here is that Cython object is unable to track refs to the results of six.b() call and they're garbage collected. There's a simple workaround:
https://paste.pound-python.org/show/QdIThLDuq3YcqRnzwrED/
It doesn't get me a lot further (rockstar client is unable to connect to server), but at least R* server name/port are not mangled. Cheers, Kacper
On 03/24/2018 04:38 PM, Britton Smith wrote:
Hi Matt,
Everything I've tried that seems to successfully copy values within the cython code still gives me empty strings by the time it reaches server.c. I even tried using c functions like strcpy, but they seem to cause other memory errors. I've scrubbed the page below for clues, but doing things as they suggest just ends up with the empty string. http://cython.readthedocs.io/en/latest/src/tutorial/strings.html I still can't figure out why the string values make it just fine to client.c, but not server.c.
I think I'm ready to throw in the towel. If someone else has time to take a look, I would appreciate it. This is the last holdup for a yt_astro_analysis release. I'm also fine with just dropping support for this. It has never worked in Python 3, but no one has ever raised the issue, so maybe it's not worth it.
Britton
On Thu, Mar 22, 2018 at 10:28 AM, Matthew Turk <matthewturk@gmail.com mailto:matthewturk@gmail.com> wrote:
Hi Britton, This is very strange. Is there any way that you can manually set these, by making them public or making them operations that setglobal
vars? -Matt On Wed, Mar 21, 2018 at 12:18 AM, Britton Smith <brittonsmith@gmail.com <mailto:brittonsmith@gmail.com>> wrote: > Hi everyone, > > I'm trying to close out this bug: > https://github.com/yt-project/yt_astro_analysis/issues/18 <https://github.com/yt-project/yt_astro_analysis/issues/18> > > After spending a fair amount of time on this, all I've been able to > determine is that the values of the server address and port seemto go
> missing for the server. These two values are passed from > yt/analysis_modules/halo_finding/rockstar/rockstar.py to > rockstar_interface.pyx in the same directory. They are then assigned to two > global variables internal to Rockstar. Rockstar instances areeither
> readers, writers, or servers. Both the readers and writers manage to hold > on to the correct values for the server address and port, but when I check > these values for the server itself, they are just empty strings. > > Somehow, this all works fine in python 2, i.e., all rockstarinstances
> maintain the correct values. I am totally out of ideas for how to fix this. > Does anyone have any thoughts? > > Britton > > _______________________________________________ > yt-dev mailing list -- yt-dev@python.org <mailto:yt-dev@python.org > To unsubscribe send an email to yt-dev-leave@python.org <mailto:yt-dev-leave@python.org> > _______________________________________________ yt-dev mailing list -- yt-dev@python.org <mailto:yt-dev@python.org> To unsubscribe send an email to yt-dev-leave@python.org <mailto:yt-dev-leave@python.org>
yt-dev mailing list -- yt-dev@python.org To unsubscribe send an email to yt-dev-leave@python.org
yt-dev mailing list -- yt-dev@python.org To unsubscribe send an email to yt-dev-leave@python.org
participants (3)
-
Britton Smith -
Kacper Kowalik -
Matthew Turk