Proposed addtion to urllib.parse in 3.1 (and urlparse in 2.7)
Appending query parameters to a URL is a very common need. However, there's nothing in urllib.parse (and older urlparse) that caters for that need. Therefore, I propose adding the following to 2.7 and 3.1 in the respective libs: def add_query_params(url, **params): """ Adds additional query parameters to the given url, preserving original parameters. Usage: >>> add_query_params('http://foo.com', a='b') 'http://foo.com?a=b' >>> add_query_params('http://foo.com?a=b', b='c', d='q') 'http://foo.com?a=b&b=c&d=q' The real implementation should be more strict, e.g. raise on the following: >>> add_query_params('http://foo.com?a=b', a='b') 'http://foo.com?a=b&a=b' """ if not params: return url encoded = urllib.urlencode(params) url = urlparse.urlparse(url) return urlparse.urlunparse((url.scheme, url.netloc, url.path, url.params, (encoded if not url.query else url.query + '&' + encoded), url.fragment))
On Fri, Mar 27, 2009 at 8:56 PM, Mart Sõmermaa <mrts.pydev@gmail.com> wrote:
Usage: >>> add_query_params('http://foo.com', a='b') 'http://foo.com?a=b' >>> add_query_params('http://foo.com?a=b', b='c', d='q') 'http://foo.com?a=b&b=c&d=q'
The real implementation should be more strict, e.g. raise on the following: >>> add_query_params('http://foo.com?a=b', a='b') 'http://foo.com?a=b&a=b'
Well, this is not 'generic' - for eg. in Django sites the above would not be applicable. -V- http://twitter.com/venkasub
Why not? 2009/3/27 Venkatraman S <venkat83@gmail.com>
On Fri, Mar 27, 2009 at 8:56 PM, Mart Sõmermaa <mrts.pydev@gmail.com>wrote:
Usage: >>> add_query_params('http://foo.com', a='b') 'http://foo.com?a=b' >>> add_query_params('http://foo.com?a=b', b='c', d='q') 'http://foo.com?a=b&b=c&d=q'
The real implementation should be more strict, e.g. raise on the following: >>> add_query_params('http://foo.com?a=b', a='b') 'http://foo.com?a=b&a=b'
Well, this is not 'generic' - for eg. in Django sites the above would not be applicable.
-V- http://twitter.com/venkasub
_______________________________________________ Python-ideas mailing list Python-ideas@python.org http://mail.python.org/mailman/listinfo/python-ideas
On Fri, Mar 27, 2009 at 9:30 PM, Mart Sõmermaa <mrts.pydev@gmail.com> wrote:
Why not?
2009/3/27 Venkatraman S <venkat83@gmail.com>
On Fri, Mar 27, 2009 at 8:56 PM, Mart Sõmermaa <mrts.pydev@gmail.com>wrote:
Usage: >>> add_query_params('http://foo.com', a='b') 'http://foo.com?a=b' >>> add_query_params('http://foo.com?a=b', b='c', d='q') 'http://foo.com?a=b&b=c&d=q'
The real implementation should be more strict, e.g. raise on the following: >>> add_query_params('http://foo.com?a=b', a='b') 'http://foo.com?a=b&a=b'
Well, this is not 'generic' - for eg. in Django sites the above would not be applicable.
http://foo.com?a=b <http://foo.com/?a=b> != http://foo.com/a/b<http://foo.com/?a=b> . Semantically , both are same,but the framework rules are different. Not sure how you would this - by telling urllib that it is a 'pretty' django URL? (or am i missing out something?) -V-
You are definitely "missing out something". For the use case you describe, there's already ulrjoin(). add_query_params() is for a different use case, i.e. it *complements* urljoin(). 2009/3/27 Venkatraman S <venkat83@gmail.com>
On Fri, Mar 27, 2009 at 9:30 PM, Mart Sõmermaa <mrts.pydev@gmail.com>wrote:
Why not?
2009/3/27 Venkatraman S <venkat83@gmail.com>
On Fri, Mar 27, 2009 at 8:56 PM, Mart Sõmermaa <mrts.pydev@gmail.com>wrote:
Usage: >>> add_query_params('http://foo.com', a='b') 'http://foo.com?a=b' >>> add_query_params('http://foo.com?a=b', b='c', d='q') 'http://foo.com?a=b&b=c&d=q'
The real implementation should be more strict, e.g. raise on the following: >>> add_query_params('http://foo.com?a=b', a='b') 'http://foo.com?a=b&a=b'
Well, this is not 'generic' - for eg. in Django sites the above would not be applicable.
http://foo.com?a=b <http://foo.com/?a=b> != http://foo.com/a/b<http://foo.com/?a=b> . Semantically , both are same,but the framework rules are different. Not sure how you would this - by telling urllib that it is a 'pretty' django URL? (or am i missing out something?)
-V-
_______________________________________________ Python-ideas mailing list Python-ideas@python.org http://mail.python.org/mailman/listinfo/python-ideas
Mart Sõmermaa <mrts.pydev@gmail.com> wrote:
Appending query parameters to a URL is a very common need. However, there's nothing in urllib.parse (and older urlparse) that caters for that need.
Therefore, I propose adding the following to 2.7 and 3.1 in the respective libs:
>>> add_query_params('http://foo.com?a=b', b='c', d='q')
To begin with, I wouldn't use keyword params. They're syntactically more restrictive than the rules for application/x-www-form-urlencoded allow, so you start by ruling out whole classes of URLs. Bill
On Fri, Mar 27, 2009 at 6:13 PM, Bill Janssen <janssen@parc.com> wrote:
Mart Sõmermaa <mrts.pydev@gmail.com> wrote:
Appending query parameters to a URL is a very common need. However, there's nothing in urllib.parse (and older urlparse) that caters for that need.
Therefore, I propose adding the following to 2.7 and 3.1 in the respective libs:
>>> add_query_params('http://foo.com?a=b', b='c', d='q')
To begin with, I wouldn't use keyword params. They're syntactically more restrictive than the rules for application/x-www-form-urlencoded allow, so you start by ruling out whole classes of URLs.
Bill
Valid point, using an ordinary dict instead would resolve that (i.e. def add_query_params(url, param_dict)).
On 27 Mar 2009, at 16:17, Mart Sõmermaa wrote:
On Fri, Mar 27, 2009 at 6:13 PM, Bill Janssen <janssen@parc.com> wrote: Mart Sõmermaa <mrts.pydev@gmail.com> wrote:
Appending query parameters to a URL is a very common need. However, there's nothing in urllib.parse (and older urlparse) that caters for that need.
Therefore, I propose adding the following to 2.7 and 3.1 in the respective libs:
>>> add_query_params('http://foo.com?a=b', b='c', d='q')
To begin with, I wouldn't use keyword params. They're syntactically more restrictive than the rules for application/x-www-form-urlencoded allow, so you start by ruling out whole classes of URLs.
Bill
Valid point, using an ordinary dict instead would resolve that (i.e. def add_query_params(url, param_dict)).
Note that it's still not general enough as query fields can be repeated, e.g. http://foo.com/search/?q=spam&q=eggs -- Arnaud
Arnaud Delobelle wrote:
On 27 Mar 2009, at 16:17, Mart Sõmermaa wrote:
On Fri, Mar 27, 2009 at 6:13 PM, Bill Janssen <janssen@parc.com> wrote: Mart Sõmermaa <mrts.pydev@gmail.com> wrote:
Appending query parameters to a URL is a very common need. However, there's nothing in urllib.parse (and older urlparse) that caters for that need.
Therefore, I propose adding the following to 2.7 and 3.1 in the respective libs:
>>> add_query_params('http://foo.com?a=b', b='c', d='q')
To begin with, I wouldn't use keyword params. They're syntactically more restrictive than the rules for application/x-www-form-urlencoded allow, so you start by ruling out whole classes of URLs.
Bill
Valid point, using an ordinary dict instead would resolve that (i.e. def add_query_params(url, param_dict)).
Note that it's still not general enough as query fields can be repeated, e.g.
It's also possible that the order matters. I think an iterable of tuples (such as returned by dict.items(), but any iterable will do) would be an okay interface.
As far as I can see, people tend to agree that this is useful. So, unless someone steps up to oppose this, I'll file a feature request to the Python bug tracker. Will propose an implementation based on ordered dict (that will be in 2.7/3.1 anyway). On Fri, Mar 27, 2009 at 9:29 PM, Joel Bender <jjb5@cornell.edu> wrote:
It's also possible that the order matters. I think an iterable of tuples
(such as returned by dict.items(), but any iterable will do) would be an okay interface.
Ordered dict then :-)
_______________________________________________ Python-ideas mailing list Python-ideas@python.org http://mail.python.org/mailman/listinfo/python-ideas
2009/3/27 Mart Sõmermaa <mrts.pydev@gmail.com>:
As far as I can see, people tend to agree that this is useful. So, unless someone steps up to oppose this, I'll file a feature request to the Python bug tracker.
Will propose an implementation based on ordered dict (that will be in 2.7/3.1 anyway).
I hope by this you mean you'll provide a patch! -- --Guido van Rossum (home page: http://www.python.org/~guido/)
Joel Bender wrote:
It's also possible that the order matters. I think an iterable of tuples (such as returned by dict.items(), but any iterable will do) would be an okay interface.
Ordered dict then :-)
But that, unlike iterable of tuples, would exclude repeated fields, as in Arnaud's example
Note that it's still not general enough as query fields can be repeated, e.g. http://foo.com/search/?q=spam&q=eggs
tjr
On Fri, Mar 27, 2009 at 7:36 PM, Terry Reedy <tjreedy@udel.edu> wrote:
Joel Bender wrote:
It's also possible that the order matters. I think an iterable of tuples (such as returned by dict.items(), but any iterable will do) would be an okay interface.
Ordered dict then :-)
But that, unlike iterable of tuples, would exclude repeated fields, as in Arnaud's example
Note that it's still not general enough as query fields can be repeated, e.g. http://foo.com/search/?q=spam&q=eggs
Repeated fields can be packed together in a tuple/list: add_query_params('http://foo.com', dict(q=('spam', 'eggs'))) To which one might reply that this would exclude non-consecutive repeated fields,e g. '?q=spam&foo=bar&q=eggs. To which I would reply that for this 0.01% of cases that require this (a) do it by hand as now or (b) use the same signature as dict() (plus the host in the beginning): add_query_params(host, mapping_or_iterable=None, **params) George
There's way too much bikeshedding in this thread (not picking on you specifically). I think the originally proposed API is fine, except it should *not* reject duplicates. To add duplicates you'd just call it multiple times, e.g. add_query_params(add_query_params(url, a='x'), a='y'). It's a pretty minor use case anyways. --Guido On Fri, Mar 27, 2009 at 7:28 PM, George Sakkis <george.sakkis@gmail.com> wrote:
On Fri, Mar 27, 2009 at 7:36 PM, Terry Reedy <tjreedy@udel.edu> wrote:
Joel Bender wrote:
It's also possible that the order matters. I think an iterable of tuples (such as returned by dict.items(), but any iterable will do) would be an okay interface.
Ordered dict then :-)
But that, unlike iterable of tuples, would exclude repeated fields, as in Arnaud's example
Note that it's still not general enough as query fields can be repeated, e.g. http://foo.com/search/?q=spam&q=eggs
Repeated fields can be packed together in a tuple/list:
add_query_params('http://foo.com', dict(q=('spam', 'eggs')))
To which one might reply that this would exclude non-consecutive repeated fields,e g. '?q=spam&foo=bar&q=eggs.
To which I would reply that for this 0.01% of cases that require this (a) do it by hand as now or (b) use the same signature as dict() (plus the host in the beginning):
add_query_params(host, mapping_or_iterable=None, **params)
George _______________________________________________ Python-ideas mailing list Python-ideas@python.org http://mail.python.org/mailman/listinfo/python-ideas
-- --Guido van Rossum (home page: http://www.python.org/~guido/)
On Sat, Mar 28, 2009 at 5:26 AM, Guido van Rossum <guido@python.org> wrote:
There's way too much bikeshedding in this thread (not picking on you specifically). I think the originally proposed API is fine, except it should *not* reject duplicates. To add duplicates you'd just call it multiple times, e.g. add_query_params(add_query_params(url, a='x'), a='y'). It's a pretty minor use case anyways.
So be it. I'll open a ticket and provide a patch, tests and documentation. For people concerned about ordering -- you can always use an odict for passing the kwargs: add_query_params('http://foo.com', **odict('a' = 1, 'b' = 2)) For people concerned about syntactically more restrictive rules than application/x-www-form-urlencoded allows -- pass in the kwargs via ordinary dict: add_query_params('http://foo.com', **{'|"-/': 1, 'öäü': 2}) # note that py2k allows UTF-8 in argument names anyway The latter is bad practice anyway.
On Mon, Mar 30, 2009 at 1:04 PM, Mart Sõmermaa <mrts.pydev@gmail.com> wrote:
add_query_params('http://foo.com', **{'|"-/': 1, 'öäü': 2}) # note that py2k allows UTF-8 in argument names anyway
s/py2k/py3k/
For people concerned about ordering -- you can always use an odict for passing the kwargs:
add_query_params('http://foo.com <http://foo.com/>', **odict('a' = 1, 'b' = 2))
Not that I want to continue the discussion about this particular issue, but I'd like to correct this statement, since this statement is wrong (beyond the syntax of creating the odict being incorrect). "**" converts the parameters to an ordinary dict. The caller does not receive the same object you call the function with. So any ordering of the values in the odict will be lost. $ ./python.exe Python 2.7a0 (trunk:70598, Mar 25 2009, 17:30:54) [GCC 4.0.1 (Apple Inc. build 5465)] on darwin Type "help", "copyright", "credits" or "license" for more information.
from collections import OrderedDict as odict def foo(**kwargs): ... print type(kwargs) ... for k, v in kwargs.iteritems(): ... print k, v ... o1=odict(); o1['a']=1; o1['b']=2 o1 OrderedDict([('a', 1), ('b', 2)]) o2=odict(); o2['b']=2; o2['a']=1 o2 OrderedDict([('b', 2), ('a', 1)]) foo(**o1) <type 'dict'> a 1 b 2 foo(**o2) <type 'dict'> a 1 b 2
Further, when an odict is created and arguments are supplied, the ordering is also lost:
odict(a=1, b=2) OrderedDict([('a', 1), ('b', 2)]) odict(b=2, a=1) OrderedDict([('a', 1), ('b', 2)])
3.1 works the same way (once you change the print statement and use .items instead of .iteritems: I need to run 2to3 on my example!). I just want to make sure everyone realized the limitations. odict won't solve problems like this. I think these are both "gotchas" waiting to happen. Eric.
On Mon, Mar 30, 2009 at 1:28 PM, Eric Smith <eric@trueblade.com> wrote:
"**" converts the parameters to an ordinary dict. The caller does not receive the same object you call the function with. So any ordering of the values in the odict will be lost.
Right you are, sorry for the mental blunder. So what if the signature is as follows to support passing query parameters via an ordered dict: add_query_params(url, params_dict=None, **kwargs) with the following behaviour:
pd = odict() pd['a'] = 1 pd['b'] = 2 add_query_params('http://foo.com/?a=0', pd, a=3) 'http://foo.com/?a=0&a=1&b=2&a=3'
Mart Sõmermaa wrote:
Right you are, sorry for the mental blunder. So what if the signature is as follows to support passing query parameters via an ordered dict:
add_query_params(url, params_dict=None, **kwargs)
with the following behaviour:
pd = odict() pd['a'] = 1 pd['b'] = 2 add_query_params('http://foo.com/?a=0', pd, a=3) 'http://foo.com/?a=0&a=1&b=2&a=3 <http://foo.com/?a=0&a=1&b=2&a=3>'
When setting up a dict.update style interface like that, it is often better to use *args for the two positional arguments - it avoids accidental name conflicts between the positional arguments and arbitrary keyword arguments. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia ---------------------------------------------------------------
On Mon, Mar 30, 2009 at 2:28 PM, Nick Coghlan <ncoghlan@gmail.com> wrote:
Mart Sőmermaa wrote:
Right you are, sorry for the mental blunder. So what if the signature is as follows to support passing query parameters via an ordered dict:
add_query_params(url, params_dict=None, **kwargs)
with the following behaviour:
pd = odict() pd['a'] = 1 pd['b'] = 2 add_query_params('http://foo.com/?a=0', pd, a=3) 'http://foo.com/?a=0&a=1&b=2&a=3 <http://foo.com/?a=0&a=1&b=2&a=3>'
When setting up a dict.update style interface like that, it is often better to use *args for the two positional arguments - it avoids accidental name conflicts between the positional arguments and arbitrary keyword arguments.
Thanks, another good point.
The general consensus in python-ideas is that the following is needed, so I bring it to python-dev to final discussions before I file a feature request in bugs.python.org. Proposal: add add_query_params() for appending query parameters to an URL to urllib.parse and urlparse. Implementation: http://github.com/mrts/qparams/blob/83d1ec287ec10934b5e637455819cf796b1b421c... free to fork and comment). Behaviour (longish, guided by "simple things are simiple, complex things possible"): In the simplest form, parameters can be passed via keyword arguments: >>> add_query_params('foo', bar='baz') 'foo?bar=baz' >>> add_query_params('http://example.com/a/b/c?a=b', b='d') 'http://example.com/a/b/c?a=b&b=d' Note that '/', if given in arguments, is encoded: >>> add_query_params('http://example.com/a/b/c?a=b', b='d', foo='/bar') 'http://example.com/a/b/c?a=b&b=d&foo=%2Fbar' Duplicates are discarded: >>> add_query_params('http://example.com/a/b/c?a=b', a='b') 'http://example.com/a/b/c?a=b' >>> add_query_params('http://example.com/a/b/c?a=b&c=q', a='b', b='d', ... c='q') 'http://example.com/a/b/c?a=b&c=q&b=d' But different values for the same key are supported: >>> add_query_params('http://example.com/a/b/c?a=b', a='c', b='d') 'http://example.com/a/b/c?a=b&a=c&b=d' Pass different values for a single key in a list (again, duplicates are removed): >>> add_query_params('http://example.com/a/b/c?a=b', a=('q', 'b', 'c'), ... b='d') 'http://example.com/a/b/c?a=b&a=q&a=c&b=d' Keys with no value are respected, pass ``None`` to create one: >>> add_query_params('http://example.com/a/b/c?a', b=None) 'http://example.com/a/b/c?a&b' But if a value is given, the empty key is considered a duplicate (i.e. the case of a&a=b is considered nonsensical): >>> add_query_params('http://example.com/a/b/c?a', a='b', c=None) 'http://example.com/a/b/c?a=b&c' If you need to pass in key names that are not allowed in keyword arguments, pass them via a dictionary in second argument: >>> add_query_params('foo', {"+'|äüö": 'bar'}) 'foo?%2B%27%7C%C3%A4%C3%BC%C3%B6=bar' Order of original parameters is retained, although similar keys are grouped together. Order of keyword arguments is not (and can not be) retained: >>> add_query_params('foo?a=b&b=c&a=b&a=d', a='b') 'foo?a=b&a=d&b=c' >>> add_query_params('http://example.com/a/b/c?a=b&q=c&e=d', ... x='y', e=1, o=2) 'http://example.com/a/b/c?a=b&q=c&e=d&e=1&x=y&o=2' If you need to retain the order of the added parameters, use an :class:`OrderedDict` as the second argument (*params_dict*): >>> from collections import OrderedDict >>> od = OrderedDict() >>> od['xavier'] = 1 >>> od['abacus'] = 2 >>> od['janus'] = 3 >>> add_query_params('http://example.com/a/b/c?a=b', od) 'http://example.com/a/b/c?a=b&xavier=1&abacus=2&janus=3' If both *params_dict* and keyword arguments are provided, values from the former are used before the latter: >>> add_query_params('http://example.com/a/b/c?a=b', od, xavier=1.1, ... zorg='a', alpha='b', watt='c', borg='d') ' http://example.com/a/b/c?a=b&xavier=1&xavier=1.1&abacus=2&janus=3&zorg=a&borg=d&watt=c&alpha=b ' Do nothing with a single argument: >>> add_query_params('a') 'a' >>> add_query_params('arbitrary strange stuff?öäüõ*()+-=42') 'arbitrary strange stuff?\xc3\xb6\xc3\xa4\xc3\xbc\xc3\xb5*()+-=42'
Hi Mart I haven't really followed this thread closely, so I apologize if some of my comments below have already been addressed. Mart Sõmermaa wrote:
The general consensus in python-ideas is that the following is needed, so I bring it to python-dev to final discussions before I file a feature request in bugs.python.org <http://bugs.python.org>.
Proposal: add add_query_params() for appending query parameters to an URL to urllib.parse and urlparse.
Implementation: http://github.com/mrts/qparams/blob/83d1ec287ec10934b5e637455819cf796b1b421c... (feel free to fork and comment).
Behaviour (longish, guided by "simple things are simiple, complex things possible"):
In the simplest form, parameters can be passed via keyword arguments:
>>> add_query_params('foo', bar='baz') 'foo?bar=baz'
>>> add_query_params('http://example.com/a/b/c?a=b', b='d') 'http://example.com/a/b/c?a=b&b=d <http://example.com/a/b/c?a=b&b=d>'
Note that '/', if given in arguments, is encoded:
>>> add_query_params('http://example.com/a/b/c?a=b', b='d', foo='/bar') 'http://example.com/a/b/c?a=b&b=d&foo=%2Fbar <http://example.com/a/b/c?a=b&b=d&foo=%2Fbar>'
Duplicates are discarded:
Why discard duplicates? They are valid and have a well-defined meaning.
>>> add_query_params('http://example.com/a/b/c?a=b', a='b') 'http://example.com/a/b/c?a=b'
I would prefer: 'http://example.com/a/b/c?a=b&a=b'
>>> add_query_params('http://example.com/a/b/c?a=b&c=q <http://example.com/a/b/c?a=b&c=q>', a='b', b='d', ... c='q') 'http://example.com/a/b/c?a=b&c=q&b=d <http://example.com/a/b/c?a=b&c=q&b=d>'
I would prefer: 'http://example.com/a/b/c?a=b&c=q&a=b&b=d'
But different values for the same key are supported:
>>> add_query_params('http://example.com/a/b/c?a=b', a='c', b='d') 'http://example.com/a/b/c?a=b&a=c&b=d <http://example.com/a/b/c?a=b&a=c&b=d>'
Pass different values for a single key in a list (again, duplicates are removed):
>>> add_query_params('http://example.com/a/b/c?a=b', a=('q', 'b', 'c'), ... b='d') 'http://example.com/a/b/c?a=b&a=q&a=c&b=d <http://example.com/a/b/c?a=b&a=q&a=c&b=d>'
Keys with no value are respected, pass ``None`` to create one:
>>> add_query_params('http://example.com/a/b/c?a', b=None) 'http://example.com/a/b/c?a&b <http://example.com/a/b/c?a&b>'
But if a value is given, the empty key is considered a duplicate (i.e. the case of a&a=b is considered nonsensical):
Again, it is a valid url and this will change its meaning. Why?
>>> add_query_params('http://example.com/a/b/c?a', a='b', c=None) 'http://example.com/a/b/c?a=b&c <http://example.com/a/b/c?a=b&c>'
If you need to pass in key names that are not allowed in keyword arguments, pass them via a dictionary in second argument:
>>> add_query_params('foo', {"+'|äüö": 'bar'}) 'foo?%2B%27%7C%C3%A4%C3%BC%C3%B6=bar'
Order of original parameters is retained, although similar keys are grouped together.
Why the grouping? Is it a side effect of your desire to discard duplicates? Changing the order like that changes the meaning of the url. A concrete case where the order of field names matters is the ":records" converter in http://pypi.python.org/pypi/zope.httpform/1.0.1 (a small independent package extracted from the form handling code in zope).
Order of keyword arguments is not (and can not be) retained:
>>> add_query_params('foo?a=b&b=c&a=b&a=d', a='b') 'foo?a=b&a=d&b=c'
>>> add_query_params('http://example.com/a/b/c?a=b&q=c&e=d <http://example.com/a/b/c?a=b&q=c&e=d>', ... x='y', e=1, o=2) 'http://example.com/a/b/c?a=b&q=c&e=d&e=1&x=y&o=2 <http://example.com/a/b/c?a=b&q=c&e=d&e=1&x=y&o=2>'
If you need to retain the order of the added parameters, use an :class:`OrderedDict` as the second argument (*params_dict*):
>>> from collections import OrderedDict >>> od = OrderedDict() >>> od['xavier'] = 1 >>> od['abacus'] = 2 >>> od['janus'] = 3 >>> add_query_params('http://example.com/a/b/c?a=b', od) 'http://example.com/a/b/c?a=b&xavier=1&abacus=2&janus=3 <http://example.com/a/b/c?a=b&xavier=1&abacus=2&janus=3>'
If both *params_dict* and keyword arguments are provided, values from the former are used before the latter:
>>> add_query_params('http://example.com/a/b/c?a=b', od, xavier=1.1, ... zorg='a', alpha='b', watt='c', borg='d')
'http://example.com/a/b/c?a=b&xavier=1&xavier=1.1&abacus=2&janus=3&zorg=a&borg=d&watt=c&alpha=b <http://example.com/a/b/c?a=b&xavier=1&xavier=1.1&abacus=2&janus=3&zorg=a&borg=d&watt=c&alpha=b>'
Do nothing with a single argument:
>>> add_query_params('a') 'a'
>>> add_query_params('arbitrary strange stuff?öäüõ*()+-=42') 'arbitrary strange stuff?\xc3\xb6\xc3\xa4\xc3\xbc\xc3\xb5*()+-=42'
If you change it to keep duplicates and not unnecessarily mangle the field order I am +1, else I am -0. - Jacob
On Sun, Apr 12, 2009 at 3:23 PM, Jacob Holm <jh@improva.dk> wrote:
Hi Mart
add_query_params('http://example.com/a/b/c?a=b', b='d', foo='/bar') 'http://example.com/a/b/c?a=b&b=d&foo=%2Fbar < http://example.com/a/b/c?a=b&b=d&foo=%2Fbar>'
Duplicates are discarded:
Why discard duplicates? They are valid and have a well-defined meaning.
The bad thing about reasoning about query strings is that there is no comprehensive documentation about their meaning. Both RFC 1738 and RFC 3986 are rather vague in that matter. But I agree that duplicates actually have a meaning (an ordered list of identical values), so I'll remove the bits that prune them unless anyone opposes (which I doubt).
But if a value is given, the empty key is considered a duplicate (i.e. the case of a&a=b is considered nonsensical):
Again, it is a valid url and this will change its meaning. Why?
I'm uncertain whether a&a=b has a meaning, but don't see any harm in supporting it, so I'll add the feature.
add_query_params('http://example.com/a/b/c?a', a='b', c=None) 'http://example.com/a/b/c?a=b&c <http://example.com/a/b/c?a=b&c>'
If you need to pass in key names that are not allowed in keyword arguments, pass them via a dictionary in second argument:
add_query_params('foo', {"+'|äüö": 'bar'}) 'foo?%2B%27%7C%C3%A4%C3%BC%C3%B6=bar'
Order of original parameters is retained, although similar keys are grouped together.
Why the grouping? Is it a side effect of your desire to discard duplicates? Changing the order like that changes the meaning of the url. A concrete case where the order of field names matters is the ":records" converter in http://pypi.python.org/pypi/zope.httpform/1.0.1 (a small independent package extracted from the form handling code in zope).
It's also related to duplicate handling, but it mostly relates to the data structure used in the initial implementation (an OrderedDict). Re-grouping is removed now and not having to deal with duplicates simplified the code considerably (using a simple list of key-value tuples now). If you change it to keep duplicates and not unnecessarily mangle the field
order I am +1, else I am -0.
Thanks for your input! Changes pushed to github (see the updated behaviour there as well): http://github.com/mrts/qparams/blob/4f32670b55082f8d0ef01c33524145c3264c161a... MS
On 12Apr2009 16:15, Mart S?mermaa <mrts.pydev@gmail.com> wrote: | On Sun, Apr 12, 2009 at 3:23 PM, Jacob Holm <jh@improva.dk> wrote: | > Hi Mart | > >>> add_query_params('http://example.com/a/b/c?a=b', b='d', foo='/bar') | >> 'http://example.com/a/b/c?a=b&b=d&foo=%2Fbar < | >> http://example.com/a/b/c?a=b&b=d&foo=%2Fbar>' | >> | >> Duplicates are discarded: | > | > Why discard duplicates? They are valid and have a well-defined meaning. | | The bad thing about reasoning about query strings is that there is no | comprehensive documentation about their meaning. Both RFC 1738 and RFC 3986 | are rather vague in that matter. But I agree that duplicates actually have a | meaning (an ordered list of identical values), so I'll remove the bits that | prune them unless anyone opposes (which I doubt). +1 from me, with the following suggestion: it's probably worth adding the to doco that people working with dict-style query_string params should probably go make a dict or OrderedDict and use: add_query_params(..., **the_dict) just to make the other use case obvious. An alternative would be to have add_ and append_ methods with set and list behaviour. Feels a little like API bloat, though the convenience function can be nice. Cheers, -- Cameron Simpson <cs@zip.com.au> DoD#743 http://www.cskk.ezoshosting.com/cs/ The wonderous pulp and fibre of the brain had been substituted by brass and iron; he had taught wheelwork to think. - Harry Wilmot Buxton 1832, referring to Charles Babbage and his difference engine.
On Mon, Mar 30, 2009 at 5:04 AM, Mart Sõmermaa <mrts.pydev@gmail.com> wrote:
On Sat, Mar 28, 2009 at 5:26 AM, Guido van Rossum <guido@python.org> wrote:
There's way too much bikeshedding in this thread (not picking on you specifically). I think the originally proposed API is fine, except it should *not* reject duplicates. To add duplicates you'd just call it multiple times, e.g. add_query_params(add_query_params(url, a='x'), a='y'). It's a pretty minor use case anyways.
So be it. I'll open a ticket and provide a patch, tests and documentation.
For people concerned about ordering -- you can always use an odict for passing the kwargs:
add_query_params('http://foo.com', **odict('a' = 1, 'b' = 2))
Alas, that doesn't work -- f(**X) copies X into a real dict. But web apps that care about the order are crazy IMO.
For people concerned about syntactically more restrictive rules than application/x-www-form-urlencoded allows -- pass in the kwargs via ordinary dict:
add_query_params('http://foo.com', **{'|"-/': 1, 'öäü': 2}) # note that py2k allows UTF-8 in argument names anyway
The latter is bad practice anyway.
-- --Guido van Rossum (home page: http://www.python.org/~guido/)
On 27 Mar 2009, at 19:29, Joel Bender wrote:
It's also possible that the order matters. I think an iterable of tuples (such as returned by dict.items(), but any iterable will do) would be an okay interface.
Ordered dict then :-)
Why not use the same signature as dict.update()? update(...) D.update(E, **F) -> None. Update D from E and F: for k in E: D[k] = E[k] (if E has keys else: for (k, v) in E: D[k] = v) then: for k in F: D[k] = F[k] -- Arnaud
participants (12)
-
Arnaud Delobelle -
Bill Janssen -
Cameron Simpson -
Eric Smith -
George Sakkis -
Guido van Rossum -
Jacob Holm -
Joel Bender -
Mart Sõmermaa -
Nick Coghlan -
Terry Reedy -
Venkatraman S