That code doesn't look toooo scary... is trying to add a DeprecationWarning something that I could try to learn how to do and try my hand at? Maybe if someone else would like to address that more quickly, they'd be able to keep me in the loop so I can start learning how Things are Done? It looks as though all that is required is to add a warn message indicating the planned removal? I'm not sure if it's better for me to have a go at things quietly or to be more interactive about getting to grips with the code... Cheers, -T On Thu, Mar 5, 2009 at 12:04 PM, Benjamin Peterson <benjamin@python.org>wrote:

2009/3/4 Raymond Hettinger <python@rcn.com>:

...

Just notices that the empty() and full() methods were still there. IIRC, we agreed to take them out (but leaving qsize() exposed). The docs entries and test cases were taken out, but the actual methods were accidentally left in.

If so, the only thing to do is deprecate it in 3.1 for removal in 3.2.

-- Regards, Benjamin _______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/tleeuwenburg%40gmail.com

-- -------------------------------------------------- Tennessee Leeuwenburg http://myownhat.blogspot.com/ "Don't believe everything you think"

Hi all, just FYI... sorry for any list pollution I went ahead and tried adding a warning to my local checkout. The test suite raised a DeprecationWarning -- it appears those methods are currently in use by other Lib code: test_multibytecodec_support test_multiprocessing /home/tjl/python3/lib/python3.1/queue.py:98: DeprecationWarning: Queue.empty() is deprecated, and won't be in 3.2. DeprecationWarning) /home/tjl/python3/lib/python3.1/queue.py:109: DeprecationWarning: Queue.full() is deprecated, and won't be in 3.2. DeprecationWarning) /home/tjl/python3/lib/python3.1/queue.py:98: DeprecationWarning: Queue.empty() is deprecated, and won't be in 3.2. DeprecationWarning) /home/tjl/python3/lib/python3.1/queue.py:109: DeprecationWarning: Queue.full() is deprecated, and won't be in 3.2. DeprecationWarning) test_mutants test_netrc test_nis Regards, -T On Thu, Mar 5, 2009 at 12:30 PM, Raymond Hettinger <python@rcn.com> wrote:

Just noticed that the empty() and full() methods were still there.

...

...

IIRC, we agreed to take them out (but leaving qsize() exposed). The docs entries and test cases were taken out, but the actual methods were accidentally left in.

If so, the only thing to do is deprecate it in 3.1 for removal in 3.2.

I recommend adding a warning to 3.0.2 and removing in 3.1. Waiting for more 3.x uptake doesn't serve our users well. IIRC, that was the rationale for cmp() removal in 3.0.1. Even in 2.x, these methods were documented as being unreliable and were removed from the 3.0 docs entirely. We discussed removing them and most of the work was done.

Guido, any thoughts?

Raymond

_______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/tleeuwenburg%40gmail.com

-- -------------------------------------------------- Tennessee Leeuwenburg http://myownhat.blogspot.com/ "Don't believe everything you think"

Yup, I'd need to remove support in multiprocessing too.

...

ahead and tried adding a warning to my local checkout.

Thanks for quickly checking this out. Will be nice to get the API cleaned-up. Leaving out part of the clean-up was a mistake. Raymond

On Thu, Mar 5, 2009 at 1:43 AM, Tennessee Leeuwenburg <tleeuwenburg@gmail.com> wrote:

Hi Jesse,

I'm not sure what the most appropriate thing to do is. I could:   (a) leave any multiprocessing changes to you,   (b) alter the functioning of the method queue_empty inside test_multiprocessing to test for emptiness in a different manner

I'm happy to go ahead and try my hand at making the appropriate changes but I don't want to tread in areas of the code that other people have ownership of.

It appears as though the only place in multiprocessing which uses the deprecated call is in the test_multiprocessing file.

I also found a call to Queue.empty in wsgui.py. I don't see any authorship tags at the top of this file but the last commiter was Andrew Kuchling. Do I need to contact him regarding this or is it appropriate for me to make the appropriate modifications without consulting him?

Apologies to anyone if I appear to be overly tentative OR overly pushy -- I am aware that some people take a great deal of personal ownership of their works, while others are impatient with a soft approach.

Regards, -Tennessee

I'm an equal opportunity patch employer: I'll take it from anyone. That being said, I would open a new issue on the tracker (bugs.python.org) outlining the issue (removing these/deprecating them) and add me to the +noisy. There you can upload patches as you create them, and we can all coordinate work and discussion there. For the multiprocessing support - the first step is to identify why it's being used and remove it (I'm head-deep in pycon talk prep, so I haven't been able to look) and then replace the code. Second, I need to make sure the multiprocessing.queue API does not have it's own APIs for these (mp.queue is a clone of queue.queue) and deprecate those. -jesse

On Thu, Mar 5, 2009 at 2:27 PM, Steve Holden <steve@holdenweb.com> wrote:

Jesse Noller wrote: [...]I'll take it from anyone.

...

And we can *quote* you on that?

regards  Steve

As long as it's not on a t-shirt, I should be OK.

IIRC, that was the rationale for cmp() removal in 3.0.1.

And indeed, that removal already caused a bug report and broke the efforts of SWIG to support Python 3.0. I disagree that our users are served by constantly breaking the API, and removing stuff just because we can. I can't see how removing API can possibly serve a user. What's wrong with empty() and full() in the first place? Regards, Martin

I don't mind whether its "in" or "out", but as a language user I think it's best to minimise undocumented interfaces. According to that principle, if it's "in", then it should also work as documented (and be documented), and be "supported". If it's "out" then it should either be removed entirely or be marked "private" (i.e. leading underscore, unless I'm mistaking my style guidelines). Cheers, -T On Sat, Mar 7, 2009 at 10:19 AM, Raymond Hettinger <python@rcn.com> wrote:

[Martin v. Löwis]

...

I disagree that our users are served by constantly breaking the API, and removing stuff just because we can. I can't see how removing API can possibly serve a user.

Am not following you here. My suggestion was to remove the two methods in Py3.1 which isn't even in alpha yet. This is for a feature that has a simple substitute, was undocumented for Py3.0, and had long been documented in Py2.x as being unreliable.

It's seems silly to me that an incomplete patch from a year ago would need to wait another two years to ever see the light of day (am presuming that 3.1 goes final this summer and that 3.2 follows 18 months later). That being said, I don't really care much. We don't actually have to do anything. It could stay in forever and cause no harm.

Raymond _______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/tleeuwenburg%40gmail.com

-- -------------------------------------------------- Tennessee Leeuwenburg http://myownhat.blogspot.com/ "Don't believe everything you think"

On Fri, Mar 6, 2009 at 7:19 PM, "Martin v. Löwis" <martin@v.loewis.de> wrote:

...

...

I disagree that our users are served by constantly breaking the API, and removing stuff just because we can. I can't see how removing API can possibly serve a user.

Am not following you here.  My suggestion was to remove the two methods in Py3.1 which isn't even in alpha yet.

Your proposal was also to add a warning for 3.0.2. This is what I primarily object to.

...

This is for a feature that has a simple substitute, was undocumented for Py3.0, and had long been documented in Py2.x as being unreliable.

The latter is not true. It was not documented as unreliable. Instead, it was correctly documented as not being able, in general, to predict the result of a subsequent put operation. In that sense, it is as unreliable as the qsize() method, which remains supported and documented.

Interestingly enough, the usage of .empty() in test_multiprocessing is entirely safe, AFAICT. So whether the API is reliable or unreliable very much depends on the application (as is true for many multi-threading issues).

...

It's seems silly to me that an incomplete patch from a year ago would need to wait another two years to ever see the light of day

Right. So it might be better to revert the patch, and restore the documentation. I still fail to see the rationale for removing these two methods.

Regards, Martin

I would tend to agree with Martin, while it might be nice to straightjacket the API into completely reliable calls (really, is there anything like that with threads?) empty and the like when used correctly work just fine. I think anyone using Queue with threads will generally understand that size/empty calls will only be reliable when put calls are completed. We can continue to warn them about the issues with using it with continual producers, but as martin points out, qsize suffers the same issue. -jesse

...

Am not following you here. My suggestion was to remove the two methods in Py3.1 which isn't even in alpha yet.

Your proposal was also to add a warning for 3.0.2. This is what I primarily object to.

Okay, that's fine. Seemed prudent but it isn't essential.

...

This is for a feature that has a simple substitute, was undocumented for Py3.0, and had long been documented in Py2.x as being unreliable.

The latter is not true. It was not documented as unreliable.

You're right. It was the docstring that said it was unreliable. The regular docs were more specific about its limitations.

I still fail to see the rationale for removing these two methods.

I believe there was a thread (in January 2008) with a decision to keep qsize() but to drop empty() and full(). Raymond

[Guido van Rossum]

Based on our experiences so far, I think that of the two options we have, both of which are bad, it's better to keep things in 3.1 that we were planning to remove but forgot, than to make 3.1 have a whole slew of additional removals. We've removed cmp() in 3.0.1, and I think that was actually the right thing to do given its prominence and the clear decision to remove it, but for smaller stuff that didn't make the cut I think we should favor backwards compatibility over cleanup.

To some extent we get both by leaving them in the module but continuing to be left out of the docs. Raymond

[MvL]

At this point, I do request that the patch is reverted completely (i.e. that the documentation is restored), and that the qualification "not reliable!" is removed from the doc strings of the methods, as it is factually incorrect.

I would be happy to restore the documentation. You want the methods back and I think that's sufficient reason to bring it back. The "not reliable" wording in the docstrings was put there by Guido 16 years ago. http://svn.python.org/view/python/trunk/Lib/Queue.py?revision=3158&view=markup Speak to him about this. I'm done with this thread. Raymond

On Sat, Mar 7, 2009 at 6:54 AM, Raymond Hettinger <python@rcn.com> wrote:

...

[MvL]

...

At this point, I do request that the patch is reverted completely (i.e. that the documentation is restored), and that the qualification "not reliable!" is removed from the doc strings of the methods, as it is factually incorrect.

I would be happy to restore the documentation.  You want the methods back and I think that's sufficient reason to bring it back.

The "not reliable" wording in the docstrings was put there by Guido 16 years ago. http://svn.python.org/view/python/trunk/Lib/Queue.py?revision=3158&view=markup Speak to him about this.  I'm done with this thread.

Also, for years, the documentation also had a "not-reliable" notice:

  Return ``True`` if the queue is empty, ``False`` otherwise.   Because of multithreading semantics, this is not reliable.

Skip changed it to the present wording last year: http://svn.python.org/view/python/trunk/Doc/library/queue.rst?r1=59750&r2=59969

That was a long time ago, but I think I just meant the obvious warning about multi-threaded use: if you are using full() and empty() and other threads may also mutate the queue, then you cannot use them to reliably prevent you from blocking. That's why eventually the facilities for non-blocking get() and put() were added (I think by Tim Peters). -- --Guido van Rossum (home page: http://www.python.org/~guido/)

On Sat, Mar 7, 2009 at 2:56 PM, "Martin v. Löwis" <martin@v.loewis.de> wrote:

...

Skip changed it to the present wording last year: http://svn.python.org/view/python/trunk/Doc/library/queue.rst?r1=59750&r2=59969

I see. I agree that the change was for the better.

Agreed too, though it would seem that *if* there is a single reader, empty()==False would guarantee that one subsequent get() won't block, and similar *if* there is a single writer, full()==False would guarantee that the next put() won't block. My point being that the lack of guarantee has to do with the necessary consequences of the non-atomicity of making two calls, not with anything inherently buggy in the implementation. Because Queue is designed for multi-threaded use, it's important to emphasize the lacking guarantees; but I think it's even more important to explain *why* these guarantees cannot hold. After all we don't go around documenting e.g. that for a dictionary, after "x in d" returns True, "x[d]" may still raise a KeyError exception (in case another thread deleted it). I'm not sure how I would modify the Queue documentation to clarify all this; perhaps it would be helpful to add an introductory section explaining the general issues with multiple readers and writers, and refer to this section in the descriptions of qsize() (and empty() and full() if they remain documented)? -- --Guido van Rossum (home page: http://www.python.org/~guido/)