Mailman 3 March 2011 - Python-Dev

Embedded Python startup is slow
by bruce bushby March 24, 2011

March 24, 2011

Hi I have previously asked this question in python-list, however I think it belongs here. I'm running python 2.7.1 on an embedded Linux board and noticed it takes 1.8 seconds to execute the most simple "Hello World" script. Platform: cpu: 200Mhz ARM (ARM926EJ-) kernel: 2.6.38 uClibc: 0.92.1-rc2 Using strace I found that python is attempting to open 168 non-existent files during startup, see trace: http://www.veritrack.co.uk/static/images/strace-hello-py.txt I think this is normal … [View More]

7 12

Re: [Python-Dev] [Python-checkins] r88796 - in sandbox/trunk/pep362: pep362.py setup.py test_pep362.py tests
by Eric Snow March 24, 2011

March 24, 2011

Shouldn't the Signature class also support building from args and not just a function object? Then I could use a Signature object when building a function object, or vice versa. Having the ability to validate args/kwargs against a Signature object, sort of like provided by the bind method, will be helpful, particularly with the decorator use case. What does the tuple form of Parameter.name represent? Did you come to any conclusion on the open issues in the PEP? With the exec_closure stuff I … [View More]have been looking pretty closely at function-related C-API capability that might be worth exposing better in python, and Signature objects would help facilitate in a much clearer way. Not only that, but even though CPython won't use Signature objects behind the scenes, they can help to make the internal operations of Python more clear by exposing the underlying mechanisms in pure python. For example, with a validate method you could express how args and kwargs are mapped onto the different parts of the code object which correspond to the parts of the Signature object. I am looking forward to the inclusion of the function signature stuff in 3.3. -eric On Wed, Mar 23, 2011 at 3:39 PM, brett.cannon <python-checkins(a)python.org>wrote: > Author: brett.cannon > Date: Wed Mar 23 22:39:32 2011 > New Revision: 88796 > > Log: > Update code for future inclusion in Python 3.3 (when I get around to it). > > Added: > sandbox/trunk/pep362/test_pep362.py > - copied, changed from r88415, > /sandbox/trunk/pep362/tests/test_pep362.py > Removed: > sandbox/trunk/pep362/setup.py > sandbox/trunk/pep362/tests/ > Modified: > sandbox/trunk/pep362/ (props changed) > sandbox/trunk/pep362/pep362.py > > Modified: sandbox/trunk/pep362/pep362.py > > ============================================================================== > --- sandbox/trunk/pep362/pep362.py (original) > +++ sandbox/trunk/pep362/pep362.py Wed Mar 23 22:39:32 2011 > @@ -8,7 +8,7 @@ > pass > > > -class Parameter(object): > +class Parameter: > > """Represent a parameter in a function signature. > > @@ -31,8 +31,9 @@ > > """ > > - def __init__(self, name, position, has_default=False, > default_value=None, > - keyword_only=False, has_annotation=False, > annotation=None): > + def __init__(self, name, position, *, has_default=False, > + default=None, keyword_only=False, has_annotation=False, > + annotation=None): > """Initialize a Parameter instance. > > For has_* arguments, if they are False then the corresponding * > @@ -42,13 +43,13 @@ > self.name = name > self.position = position > if has_default: > - self.default_value = default_value > + self.default = default > self.keyword_only = keyword_only > if has_annotation: > self.annotation = annotation > > > -class Signature(object): > +class Signature: > > """Object to represent the signature of a function/method. > > @@ -74,39 +75,19 @@ > > def __init__(self, func): > """Initialize from a function or method object.""" > - if hasattr(func, 'im_func'): > - func = func.im_func > - try: > - func_code = func.__code__ > - except AttributeError: > - # Compatibility for versions < 2.6. > - func_code = func.func_code > - > + func_code = func.__code__ > self.name = func.__name__ > - > - try: > - # Unneeded once 2.x support is removed; can easily get info > the > - # "hard" way. > - argspec = inspect.getfullargspec(func)[:4] > - except AttributeError: > - # Needed only for tuple parameters. > - argspec = inspect.getargspec(func) > + # XXX replace with own implementation > + argspec = inspect.getfullargspec(func)[:4] > parameters = {} > > # Parameter information. > pos_count = func_code.co_argcount > - if hasattr(func_code, 'co_kwonlyargcount'): > - keyword_only_count = func_code.co_kwonlyargcount > - else: > - keyword_only_count = 0 > + keyword_only_count = func_code.co_kwonlyargcount > positional = argspec[0] > keyword_only = func_code.co_varnames[pos_count: > > pos_count+keyword_only_count] > - try: > - fxn_defaults = func.__defaults__ > - except AttributeError: > - # Deal with old names prior to 2.6. > - fxn_defaults = func.func_defaults > + fxn_defaults = func.__defaults__ > if fxn_defaults: > pos_default_count = len(fxn_defaults) > else: > @@ -126,7 +107,7 @@ > has_annotation, annotation = self._find_annotation(func, name) > default_value = fxn_defaults[offset] > param = Parameter(name, offset+non_default_count, > - has_default=True, > default_value=default_value, > + has_default=True, default=default_value, > has_annotation=has_annotation, > annotation=annotation) > parameters[name] = param > @@ -134,16 +115,14 @@ > for offset, name in enumerate(keyword_only): > has_annotation, annotation = self._find_annotation(func, name) > has_default, default_value = False, None > - # hasattr check only needed for versions < 2.6. > - if (hasattr(func, '__kwdefaults__') and func.__kwdefaults__ > and > - name in func.__kwdefaults__): > + if func.__kwdefaults__ and name in func.__kwdefaults__: > has_default = True > default_value = func.__kwdefaults__[name] > param = Parameter(name, offset+pos_count, keyword_only=True, > - has_default=has_default, > - default_value=default_value, > - has_annotation=has_annotation, > - annotation=annotation) > + has_default=has_default, > + default=default_value, > + has_annotation=has_annotation, > + annotation=annotation) > parameters[name] = param > # Variable parameters. > index = pos_count + keyword_only_count > @@ -172,9 +151,8 @@ > self._parameters = parameters > > # Return annotation. > - if hasattr(func, '__annotations__'): > - if 'return' in func.__annotations__: > - self.return_annotation = func.__annotations__['return'] > + if 'return' in func.__annotations__: > + self.return_annotation = func.__annotations__['return'] > > def __getitem__(self, key): > return self._parameters[key] > @@ -186,10 +164,9 @@ > """Return True if an annotation exists for the named parameter > along > with its annotation, else return False and None.""" > has_annotation, annotation = False, None > - if hasattr(func, '__annotations__'): > - if name in func.__annotations__: > - has_annotation = True > - annotation = func.__annotations__[name] > + if name in func.__annotations__: > + has_annotation = True > + annotation = func.__annotations__[name] > return has_annotation, annotation > > def _convert_name(self, name): > @@ -234,7 +211,7 @@ > break > else: > raise BindError("too many positional arguments") > - self._tuple_bind(bindings, param.name, position_arg) > + bindings[param.name] = position_arg > args = args[1:] > # Keyword arguments & default values. > else: > @@ -247,9 +224,8 @@ > except KeyError: > raise BindError("%r unbound" % param_name) > else: > - if hasattr(positional_param, 'default_value'): > - self._tuple_bind(bindings, param_name, > - > positional_param.default_value) > + if hasattr(positional_param, 'default'): > + bindings[param_name] = positional_param.default > else: > raise BindError("%r parameter lacking default > value" % > param_name) > @@ -276,39 +252,14 @@ > # Keyword-only default values. > else: > for name, param in keyword_only.items(): > - if hasattr(param, 'default_value'): > - bindings[name] = param.default_value > + if hasattr(param, 'default'): > + bindings[name] = param.default > else: > raise BindError("%s parameter lacking a default value" > % > name) > > return bindings > > - def _tuple_bind(self, bindings, possible_tuple, value): > - """Where a tuple could be a parameter, handle binding the values > to the > - tuple and storing into the bindings mapping.""" > - if not isinstance(possible_tuple, tuple): > - bindings[possible_tuple] = value > - else: > - # Need to make sure that value is as long as the parameter, > but not > - # vice-versa. > - error_msg = "not enough values to unpack for %r" > - tuple_iter = iter(possible_tuple) > - try: > - value_iter = iter(value) > - except TypeError: > - raise BindError(error_msg % possible_tuple) > - while True: > - try: > - sub_param = tuple_iter.next() > - except StopIteration: > - break > - try: > - sub_value = value_iter.next() > - except StopIteration: > - raise BindError(error_msg % possible_tuple) > - self._tuple_bind(bindings, sub_param, sub_value) > - > > def signature(func): > """Return a Signature object for the function or method. > @@ -318,15 +269,9 @@ > attribute if possible (but is not required). > > """ > - if hasattr(func, 'im_func'): > - func = func.im_func > - sig = Signature(func) > - if not hasattr(func, '__signature__'): > - try: > - func.__signature__ = sig > - except AttributeError: > - pass > - else: > - sig = func.__signature__ > - > - return sig > + try: > + return func.__signature__ > + except AttributeError: > + sig = Signature(func) > + func.__signature__ = sig > + return sig > > Deleted: sandbox/trunk/pep362/setup.py > > ============================================================================== > --- sandbox/trunk/pep362/setup.py Wed Mar 23 22:39:32 2011 > +++ (empty file) > @@ -1,25 +0,0 @@ > -from distutils.core import setup > - > -setup( > - # Package metadata. > - name='pep362', > - version='0.6.2', > - description='Implementation of PEP 362 (Function Signature > objects)', > - author='Brett Cannon', > - author_email='brett(a)python.org', > - url='http://svn.python.org/view/sandbox/trunk/pep362/', > - # Files. > - py_modules=['pep362', 'examples'], > - packages=['tests'], > - data_files=['README'], > - classifiers=[ > - 'Programming Language :: Python :: 2', > - 'Programming Language :: Python :: 2.5', > - 'Programming Language :: Python :: 2.6', > - 'Programming Language :: Python :: 2.7', > - 'Programming Language :: Python :: 3', > - 'Programming Language :: Python :: 3.0', > - 'Programming Language :: Python :: 3.1', > - 'Programming Language :: Python :: 3.2', > - ] > - ) > > Copied: sandbox/trunk/pep362/test_pep362.py (from r88415, > /sandbox/trunk/pep362/tests/test_pep362.py) > > ============================================================================== > --- /sandbox/trunk/pep362/tests/test_pep362.py (original) > +++ sandbox/trunk/pep362/test_pep362.py Wed Mar 23 22:39:32 2011 > @@ -1,23 +1,49 @@ > import pep362 > > import unittest > -from tests import pep362_fodder > -try: > - from tests import pep362_py2_fodder > - from test import test_support > -except SyntaxError: > - from tests import pep362_py3k_fodder > - from test import support as test_support > +from test import support as test_support > from sys import version_info > > > -def version_specific(major_number): > - def inner(fxn): > - if version_info[0] == major_number: > - return fxn > - else: > - return lambda self: self > - return inner > +def no_args(): > + pass > + > +def var_args(*args): > + pass > + > +def var_kw_args(**kwargs): > + pass > + > +def no_default_args(a): > + pass > + > +def default_args(a=42): > + pass > + > +def keyword_only(*, a): > + pass > + > +def keyword_only_default(*, a=42): > + pass > + > +def arg_annotation(a:int): > + pass > + > +def arg_annotation_default(a:int=42): > + pass > + > +def arg_annotation_var(*args:int, **kwargs:str): > + pass > + > +def arg_annotation_keyword_only(*, a:int): > + pass > + > +def return_annotation() -> int: > + pass > + > +def all_args(a:int, d=0, *args:int, > + g:int, h:int=8, **kwargs:int) -> int: > + return a, d, g, h, args, kwargs > > > class ParameterObjectTests(unittest.TestCase): > @@ -41,13 +67,14 @@ > self.assertEqual(param.position, pos) > > def test_default_values(self): > - # Testing that 'default_value' is not set is handled in the > testing of > + # Testing that 'default' is not set is handled in the testing of > # that attribute. > default_value = 42 > - param = pep362.Parameter('_', 0, True, default_value) > - self.assertEqual(param.default_value, default_value) > - param = pep362.Parameter('_', 0, False) > - self.assertTrue(not hasattr(param, 'default_value')) > + param = pep362.Parameter('_', 0, has_default=True, > + default=default_value) > + self.assertEqual(param.default, default_value) > + param = pep362.Parameter('_', 0, has_default=False) > + self.assertTrue(not hasattr(param, 'default')) > > def test_keyword_only(self): > # Setting the value for keyword_only should create an attribute. > @@ -70,21 +97,21 @@ > def test_getitem(self): > # __getitem__() should return the Parameter object for the name > # parameter. > - sig = pep362.Signature(pep362_fodder.default_args) > + sig = pep362.Signature(default_args) > self.assertTrue(sig['a']) > param = sig['a'] > self.assertTrue(param.name, 'a') > > def test_iter(self): > # The iterator should return all Parameter objects in the proper > order. > - sig = pep362.Signature(pep362_fodder.default_args) > + sig = pep362.Signature(default_args) > params = list(sig) > self.assertEqual(len(params), 1) > self.assertEqual(params[0].name, 'a') > > def test_no_args(self): > # Test a function with no arguments. > - sig = pep362.Signature(pep362_fodder.no_args) > + sig = pep362.Signature(no_args) > self.assertEqual('no_args', sig.name) > self.assertTrue(not sig.var_args) > self.assertTrue(not sig.var_kw_args) > @@ -92,115 +119,87 @@ > > def test_var_args(self): > # Test the var_args attribute. > - sig = pep362.Signature(pep362_fodder.var_args) > + sig = pep362.Signature(var_args) > self.assertEqual('args', sig.var_args) > self.assertEqual(0, len(list(sig))) > - sig = pep362.Signature(pep362_fodder.no_args) > + sig = pep362.Signature(no_args) > self.assertEqual('', sig.var_args) > > def test_var_kw_args(self): > # Test the var_kw_args attribute and annotations. > - sig = pep362.Signature(pep362_fodder.var_kw_args) > + sig = pep362.Signature(var_kw_args) > self.assertEqual('var_kw_args', sig.name) > self.assertEqual('kwargs', sig.var_kw_args) > self.assertEqual(0, len(list(sig))) > - sig = pep362.Signature(pep362_fodder.no_args) > + sig = pep362.Signature(no_args) > self.assertEqual('', sig.var_kw_args) > > def test_parameter_positional(self): > # A function with positional arguments should work. > - sig = pep362.Signature(pep362_fodder.no_default_args) > + sig = pep362.Signature(no_default_args) > self.assertEqual('no_default_args', sig.name) > param = sig['a'] > self.assertEqual('a', param.name) > self.assertEqual(0, param.position) > - self.assertTrue(not hasattr(param, 'default_value')) > + self.assertTrue(not hasattr(param, 'default')) > > def test_parameter_default(self): > # Default parameters for a function should work. > - sig = pep362.Signature(pep362_fodder.default_args) > + sig = pep362.Signature(default_args) > self.assertEqual('default_args', sig.name) > param = sig['a'] > self.assertEqual('a', param.name) > self.assertEqual(0, param.position) > - self.assertEqual(42, param.default_value) > - > - @version_specific(2) > - def test_parameter_tuple(self): > - # A function with a tuple as a parameter should work. > - sig = pep362.Signature(pep362_py2_fodder.tuple_args) > - self.assertEqual('tuple_args', sig.name) > - param = list(sig)[0] > - self.assertTrue(isinstance(param.name, tuple)) > - self.assertEqual(('a', ('b',)), param.name) > - self.assertEqual(0, param.position) > - self.assertTrue(not hasattr(param, 'default_value')) > + self.assertEqual(42, param.default) > > - @version_specific(2) > - def test_parameter_tuple_default(self): > - # A default argument for a tuple parameter needs to work. > - sig = pep362.Signature(pep362_py2_fodder.default_tuple_args) > - self.assertEqual('default_tuple_args', sig.name) > - param = list(sig)[0] > - self.assertEqual(('a', ('b',)), param.name) > - self.assertEqual(0, param.position) > - self.assertEqual((1, (2,)), param.default_value) > - > - @version_specific(3) > def test_keyword_only(self): > # Is a function containing keyword-only parameters handled > properly? > - sig = pep362.Signature(pep362_py3k_fodder.keyword_only) > + sig = pep362.Signature(keyword_only) > param = sig['a'] > self.assertEqual(param.name, 'a') > self.assertTrue(param.keyword_only) > self.assertEqual(param.position, 0) > > - @version_specific(3) > def test_keyword_only_default(self): > # Default arguments can work for keyword-only parameters. > - sig = pep362.Signature(pep362_py3k_fodder.keyword_only_default) > + sig = pep362.Signature(keyword_only_default) > param = sig['a'] > self.assertEqual(param.name, 'a') > self.assertTrue(param.keyword_only) > self.assertEqual(param.position, 0) > - self.assertEqual(param.default_value, 42) > + self.assertEqual(param.default, 42) > > - @version_specific(3) > def test_annotations(self): > # Make sure the proper annotation is found. > - sig = pep362.Signature(pep362_py3k_fodder.arg_annotation) > + sig = pep362.Signature(arg_annotation) > param = sig['a'] > self.assertEqual(param.name, 'a') > self.assertEqual(param.annotation, int) > > - @version_specific(3) > def test_annotations_default(self): > # Annotations with a default value should work. > - sig = pep362.Signature(pep362_py3k_fodder.arg_annotation_default) > + sig = pep362.Signature(arg_annotation_default) > param = sig['a'] > self.assertEqual(param.name, 'a') > self.assertEqual(param.annotation, int) > - self.assertEqual(param.default_value, 42) > + self.assertEqual(param.default, 42) > > - @version_specific(3) > def test_annotation_keyword_only(self): > # Keyword-only parameters can have an annotation. > - sig = > pep362.Signature(pep362_py3k_fodder.arg_annotation_keyword_only) > + sig = pep362.Signature(arg_annotation_keyword_only) > param = sig['a'] > self.assertEqual(param.name, 'a') > self.assertEqual(param.annotation, int) > self.assertTrue(param.keyword_only) > > - @version_specific(3) > def test_return_annotation(self): > # The return value annotation. > - sig = pep362.Signature(pep362_py3k_fodder.return_annotation) > + sig = pep362.Signature(return_annotation) > self.assertEqual(sig.return_annotation, int) > > - @version_specific(3) > def test_var_annotations(self): > # Annotation on variable arguments (*args & **kwargs). > - sig = pep362.Signature(pep362_py3k_fodder.arg_annotation_var) > + sig = pep362.Signature(arg_annotation_var) > self.assertEqual(sig.var_annotations[sig.var_args], int) > self.assertEqual(sig.var_annotations[sig.var_kw_args], str) > > @@ -224,14 +223,14 @@ > """Test Signature.bind().""" > > def test_no_parameters(self): > - sig = pep362.Signature(pep362_fodder.no_args) > + sig = pep362.Signature(no_args) > binding = sig.bind() > self.assertEqual({}, binding) > self.assertRaises(pep362.BindError, sig.bind, 42) > self.assertRaises(pep362.BindError, sig.bind, a=0) > > def test_var_parameters(self): > - sig = pep362.Signature(pep362_fodder.var_args) > + sig = pep362.Signature(var_args) > binding = sig.bind(0, 1, 2) > self.assertEqual({'args':(0, 1, 2)}, binding) > binding = sig.bind() > @@ -239,7 +238,7 @@ > self.assertRaises(pep362.BindError, sig.bind, a=0) > > def test_var_kw_parameters(self): > - sig = pep362.Signature(pep362_fodder.var_kw_args) > + sig = pep362.Signature(var_kw_args) > binding = sig.bind(a=0) > self.assertEqual({'kwargs':{'a':0}}, binding) > binding = sig.bind() > @@ -247,7 +246,7 @@ > self.assertRaises(pep362.BindError, sig.bind, 42) > > def test_positional_parameters(self): > - sig = pep362.Signature(pep362_fodder.no_default_args) > + sig = pep362.Signature(no_default_args) > binding = sig.bind(42) > self.assertEqual({'a':42}, binding) > binding = sig.bind(a=42) > @@ -257,7 +256,7 @@ > self.assertRaises(pep362.BindError, sig.bind, b=0) > > def test_keyword_parameters(self): > - sig = pep362.Signature(pep362_fodder.default_args) > + sig = pep362.Signature(default_args) > binding = sig.bind() > self.assertEqual({'a':42}, binding) > binding = sig.bind(0) > @@ -268,73 +267,24 @@ > self.assertRaises(pep362.BindError, sig.bind, a=0, b=1) > self.assertRaises(pep362.BindError, sig.bind, b=1) > > - @version_specific(2) > - def test_tuple_parameter(self): > - sig = pep362.Signature(pep362_py2_fodder.tuple_args) > - arg = (1, ((2,),)) > - binding = sig.bind(arg) > - self.assertEqual({'a':1, 'b':(2,)}, binding) > - self.assertRaises(pep362.BindError, sig.bind, (1,2,3)) > - self.assertRaises(pep362.BindError, sig.bind, (1, 2)) > - > - @version_specific(2) > - def test_default_tuple_parameter(self): > - sig = pep362.Signature(pep362_py2_fodder.default_tuple_args) > - binding = sig.bind() > - self.assertEqual({'a':1, 'b':2}, binding) > - arg = (0, (1,)) > - binding = sig.bind(arg) > - self.assertEqual({'a':0, 'b':1}, binding) > - > - @version_specific(2) > - def test_py2_all_args(self): > - sig = pep362.Signature(pep362_py2_fodder.all_args) > - # a, (b, (c,)), d=0, (e, (f,))=(4, (5,)), *g, **h > - # name, position, has_default, default value > - expect = (('a', 0, False, None), > - (('b', ('c',)), 1, False, None), > - ('d', 2, True, 0), > - (('e', ('f',)), 3, True, (4, (5,)))) > - self.assertEqual(len(list(sig)), len(expect)) > - for param, check in zip(list(sig), expect): > - name, pos, has_default, default_value = check > - self.assertEqual(param.name, name) > - self.assertEqual(param.position, pos) > - if has_default: > - self.assertEqual(param.default_value, default_value) > - else: > - self.assertTrue(not hasattr(param, 'default_value')) > - self.assertTrue(not param.keyword_only) > - self.assertTrue(not hasattr(param, 'annotation')) > - self.assertEqual(sig.var_args, 'g') > - self.assertEqual(sig.var_kw_args, 'h') > - self.assertEqual(len(sig.var_annotations), 0) > - binding = sig.bind(0, (1, (2,)), d=3, i=7) > - expected = {'a':0, 'b':1, 'c':2, 'd':3, 'e':4, 'f':5, 'g':tuple(), > - 'h':{'i':7}} > - self.assertEqual(expected, binding) > - > - @version_specific(3) > def test_keyword_only(self): > - sig = pep362.Signature(pep362_py3k_fodder.keyword_only) > + sig = pep362.Signature(keyword_only) > binding = sig.bind(a=42) > self.assertEqual(binding, {'a':42}) > self.assertRaises(pep362.BindError, sig.bind) > self.assertRaises(pep362.BindError, sig.bind, 42) > > - @version_specific(3) > def test_keyword_only_default(self): > - sig = pep362.Signature(pep362_py3k_fodder.keyword_only_default) > + sig = pep362.Signature(keyword_only_default) > binding = sig.bind() > self.assertEqual(binding, {'a':42}) > binding = sig.bind(a=1) > self.assertEqual(binding, {'a':1}) > self.assertRaises(pep362.BindError, sig.bind, 1) > > - @version_specific(3) > def test_all_py3k_args(self): > # a:int, d=0, *args:int, g:int, h:int=8, **kwargs:int) -> int > - sig = pep362.Signature(pep362_py3k_fodder.all_args) > + sig = pep362.Signature(all_args) > # name, position, kw only, has_default, default, has anno, anno > expected = (('a', 0, False, False, None, True, int), > ('d', 1, False, True, 0, False, None), > @@ -353,9 +303,9 @@ > else: > self.assertTrue(not param.keyword_only) > if has_default: > - self.assertEqual(param.default_value, default) > + self.assertEqual(param.default, default) > else: > - self.assertTrue(not hasattr(param, 'default_value')) > + self.assertTrue(not hasattr(param, 'default')) > if has_anno: > self.assertEqual(param.annotation, anno) > else: > @@ -373,7 +323,7 @@ > > def test_too_many_arguments(self): > # Only one argument should pair up with a parameter. > - sig = pep362.Signature(pep362_fodder.no_default_args) > + sig = pep362.Signature(no_default_args) > self.assertRaises(pep362.BindError, sig.bind, 1, a=1) > > > _______________________________________________ > Python-checkins mailing list > Python-checkins(a)python.org > http://mail.python.org/mailman/listinfo/python-checkins > [View Less]

2 2

Re: [Python-Dev] [Python-checkins] cpython: Isolate the test_source() test in test_collections
by Benjamin Peterson March 24, 2011

March 24, 2011

2011/3/24 raymond.hettinger <python-checkins(a)python.org>: > http://hg.python.org/cpython/rev/4f1cd92fe835 > changeset: 68901:4f1cd92fe835 > user: Raymond Hettinger <python(a)rcn.com> > date: Thu Mar 24 09:45:43 2011 -0700 > summary: > Isolate the test_source() test in test_collections > > files: > Lib/test/test_collections.py | 13 ++++++++----- > 1 files changed, 8 … [View More]

1 0

Re: [Python-Dev] Second draft: PEP397: Python launcher for Windows
by Michael Foord March 24, 2011

March 24, 2011

On 23/03/2011 14:52, Mike Driscoll wrote: > On Tue, Mar 22, 2011 at 2:12 PM, Michael Foord > <fuzzyman(a)voidspace.org.uk> wrote: >> On 22/03/2011 07:21, Mark Hammond wrote: >> >> Hi all, >> I've made some changes to the draft PEP and checked it into the PEP >> repository as PEP397. The reference implementation is currently being >> tracked athttp://bugs.python.org/issue11629. >> >> >> Hey Mark, >> >> One way of … [View More]

4 6

Workflow proposal
by Jesus Cea March 24, 2011

March 24, 2011

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Most of our problems are related to trying to keep a lineal history, and races with pull-patch-commit-push cycle. I propose the following workflow. All branches (except 2.7) MUST be merged to default, all the time. This must be enforced by the push hook. The workflow would be something like this: 1. Update the local clone. 2. I patch 3.1. Test the patch locally. 3. Commit the patch locally. 4. Merge the patch to 3.2 locally. 5. Merge the patch … [View More]to "default" locally. 6. Push. If there are no other pushes in the meantime, we are done. If there are other pushes, we will have a "+1 head" and the push is rejected. Now we have two options. The easier and "mercurial" way would be: 6. Pull. 7. Merge the two heads. 8. Merge that merge to the other branches, as necessary. Since the two heads were already merged to other branches, this merge should be trivial. 9. Push. If we have some other pushes in the meantime, repeat again from 6. You will eventually win the race :). This head merging could be "ugly" to some people (not me), so other approach would be, if the original "merging" is easy: 6. Use "hg strip" (dangerous!) to delete the local merges to 3.2 and "default". Leave the original commit in "3.1" alone. 7. Pull and rebase your commit to the incoming head. Now your local history is lineal. 8. Do the merge to "3.2" and "default", locally. 9. Try to push. If there are more pushes in the meantime, repeat from 6. This second approach keep a lineal history, but it is more "dangerous" and can be a pain if the merges to "3.2" and "default" are not trivial, because you must do them several times. This workflow works well ONLY if we forbid to have active branches (except "default" and "2.7"). That is, forbidding having branches not merged to new versions. That is, 3.1 merged to 3.2 and default, 3.2 merged to default. This should be implemented in the push hook. If some patches in 3.1 should not be in 3.2/default, we MUST do a null merge anyway. The "no active branches" must not be broken. It should be a firm policy enforced by the push hook. So good so far. But there is a *legitime* reason for pushing a 3.1 patch without merging... trying the patch in the buildbots, maybe redoing it a few times, discussing it bugs.python.org, try several approaches, etc, before closing the issue and merging to 3.2 and "default". But that is an artifact of the way we interact with the buildbots. If I could do the following, it would be AMAZING: 1. Write a patch in my local clone. 2. Commit it locally. 3. Push my clone to a SANDBOX in hg.python.org ("hg push sandbox"). This sandbox is synchronized with the official HG repository. If my push create a "+1 head", we have a race. I must pull and merge, and try to push again. 4. When I get my patch in the sandbox. the sandbox accepts it, generate a patch relative to current tip from hg.python.org, and strip that commit. The sandbox must be always synchronized with hg.python.org. Its only mission is to validate that your patch applies cleanly to current tip, that you have push permissions (security). 5. The buildbots should checkout hg.python.org and apply your patch over the working directory. Run the buildbots testing. 6. You wait for the results. 7. If you are not ok with the result, your keep modifying your clone. When you want to try, you push to the sandbox, for trying. 8. When you are satisfied with the result, you do the merge to the other branches and push to "hg.python.org". Several people can use the sandbox at the same time, since it is synchronized with hg.python.org and it strips your patch inmediatelly, so nobody will see your patch. In fact, this sandbox could add your patch automatically to the assigned issue (bugs.python.org). We already have "cloned in the server" repositories, but I am not sure how to delete them, how to push them to the buildbots (is it efficient?, of the buildbots must pull the complete repo each time?), I must care about pushing it the last changes in hg.python.org, etc. Ideas?. - -- Jesus Cea Avion _/_/ _/_/_/ _/_/_/ jcea(a)jcea.es - http://www.jcea.es/ _/_/ _/_/ _/_/ _/_/ _/_/ jabber / xmpp:jcea@jabber.org _/_/ _/_/ _/_/_/_/_/ . _/_/ _/_/ _/_/ _/_/ _/_/ "Things are not so easy" _/_/ _/_/ _/_/ _/_/ _/_/ _/_/ "My name is Dump, Core Dump" _/_/_/ _/_/_/ _/_/ _/_/ "El amor es poner tu felicidad en la felicidad de otro" - Leibniz -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iQCVAwUBTYjehJlgi5GaxT1NAQInJAQAn61i1YFwUB/IXkNazJZfRi3dVYVTikbE t5qBpvFKZKA/s90y6fP6usa09DZVlYXsb95leMIaTc2BjXzgB5qd9I47rFlA+pbk mjHbVoDmsNcjjI93lRhZV4p7MSeWWSkNDpF/H3qi/bPw3MKqkJFpT9sVkboBs3iO UbtKE9+Dnno= =GVMW -----END PGP SIGNATURE----- [View Less]

9 15

Python3: speed efficiency vs user friendliness (my first experience)
by anatoly techtonik March 23, 2011

March 23, 2011

I didn't touch Python3 until PyCon, and my first user experience is not really good. I've got a feeling that Python3 became more ugly, because it doesn't allow me to think about the logic anymore, and requires more low-level workarounds even for basic user input/output. For example, now I need to remember that on Windows I need to flush output every time when I want the result of print() with end!='\n' to appear on the screen immediately. And for the most of my legacy scripts I used end='\n' … [View More]

9 13

__reduce__
by Kristján Valur Jónsson March 23, 2011

March 23, 2011

The __reduce__ protocol doesn't provide for keyword arguments to the constructor. Some constructor arguments are only available as keyword arguments. Annoying, isn't it? K

2 1

blogroll for the new blog
by Doug Hellmann March 23, 2011

March 23, 2011

We are nearly ready to launch the new blog for python-dev. If you have a personal blog where you talk about your Python work, we would like to include a link in the sidebar as part of our blog-roll. Send me the URL (off-list) and I'll make sure you are included. Doug

7 12

funky buildbot
by Bill Janssen March 23, 2011

March 23, 2011

I'm trying to get a new buildbot in the swim of things, and it keeps getting into this state where the buildslave process seems caught in an endless loop. Perhaps someone here knows why? It's a new Mac Mini running the latest Snow Leopard, with Python 2.6.1 (the /usr/bin/python) and buildslave 0.8.3, using Twisted 8.2.0. Bill

4 8

Second draft: PEP397: Python launcher for Windows
by Mark Hammond March 23, 2011

March 23, 2011

Hi all, I've made some changes to the draft PEP and checked it into the PEP repository as PEP397. The reference implementation is currently being tracked at http://bugs.python.org/issue11629. Most of the changes should not be too controversial, but of note: * I've added lots of discussion about the child-process issue and the wording now states, basically, that in-process is not feasible, but if we use the win32 Job API we can arrange for the child to be killed with the parent on … [View More]

4 6