From crk at godblessthe.us Wed Oct 1 00:54:42 2014 From: crk at godblessthe.us (Clayton Kirkwood) Date: Tue, 30 Sep 2014 15:54:42 -0700 Subject: [Tutor] could somebody please explain... Message-ID: <01eb01cfdd01$86034220$9209c660$@us> I don't understand the multiplicity of some tools. Namely, why is there a 'a+b', operator.add(a,b), operator.__add__(a,b), operator.iadd(a,b), operator.__iadd__(a,b) and their related operators? Also, I found something that I can't get my mind around. It is part of the time/date protocols. I've not seen it anywhere else. Datetime(year=blah, blah, blah).date/time() datetime(2013,3,6).date() #returns. datetime.date(2013,3,6) datetime(2013,3,6).time() #returns. datetime.time(0,0) This is one of the weirder things I've run across. Is this allowed/needed in other functions/classes, or is it a datetime thing only? Please spare my mind:<)) Clayton -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/tutor/attachments/20140930/b5705174/attachment.html> From alan.gauld at btinternet.com Wed Oct 1 02:58:16 2014 From: alan.gauld at btinternet.com (Alan Gauld) Date: Wed, 01 Oct 2014 01:58:16 +0100 Subject: [Tutor] could somebody please explain... In-Reply-To: <01eb01cfdd01$86034220$9209c660$@us> References: <01eb01cfdd01$86034220$9209c660$@us> Message-ID: <m0fjj8$me5$1@ger.gmane.org> On 30/09/14 23:54, Clayton Kirkwood wrote: > I don't understand the multiplicity of some tools. Namely, why is there a > 'a+b', operator.add(a,b), operator.__add__(a,b), operator.iadd(a,b), > operator.__iadd__(a,b) and their related operators? The operator module is there largely to allow you to pass operations to functions that take functions as parameters. For example map() which applies a function to a collection. total = map(operator.add, [1,2,3,4,5,6]) is the same result as total = sum([1,2,3,4,5,6]) You could do the same thing with a lambda: total = map(lambda a,b: a+b, [1,2,3,4,5,6]) But using operator.add makes it easier to read (and less error prone). The operator module has functions representing most of the builtin operations in Python. I'm not really sure why it implements the dunder methods (eg operatotor.__add__()) corresponding to the operations though. I'm sure somebody else can provide a good explanation... > Also, I found something that I can't get my mind around. It is part of the > time/date protocols. I've not seen it anywhere else. > > Datetime(year=blah, blah, blah).date/time() A Datetime object is an instance of a combined date and time. the date and time methods return just the date or time parts of that combined object. > This is one of the weirder things I've run across. Is this allowed/needed in > other functions/classes, or is it a datetime thing only? The nearest I can think of elsewhere is the complex number case where you can get the real/imaginary values from a complex number >>> n = (1+6j) >>> n.imag 6.0 >>> n.real 1.0 But similar concepts apply in most classes where attributes, methods or properties are used to return significant aspects of the object. An Address might have methods to return the city or street or house number for example. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.flickr.com/photos/alangauldphotos From steve at pearwood.info Wed Oct 1 03:38:14 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Wed, 1 Oct 2014 11:38:14 +1000 Subject: [Tutor] could somebody please explain... In-Reply-To: <01eb01cfdd01$86034220$9209c660$@us> References: <01eb01cfdd01$86034220$9209c660$@us> Message-ID: <20141001013814.GW19757@ando.pearwood.info> On Tue, Sep 30, 2014 at 03:54:42PM -0700, Clayton Kirkwood wrote: > I don't understand the multiplicity of some tools. Namely, why is there a > 'a+b', operator.add(a,b), operator.__add__(a,b), operator.iadd(a,b), > operator.__iadd__(a,b) and their related operators? The + operator is the public interface, but the implementation that makes + work are the special methods __add__ and __radd__ . When you write in your code: result = a + b how does Python know what to do with a and b? In principle, Python could hard-code into the language a handful of types that the interpreter knows how to add: int, float, str, list, etc. But that is not easily extended when a new type is supported, and Python supports "operator overloading" where any custom class can define "a + b" to do whatever the class developer wants. So when the Python interpreter executes a + b, when it does is look for a special "dunder" (Double UNDERscore) method on a, __add__, or a special dunder method __radd__ ("right add") on b, and calls that. Actually the rules are a bit more complicated than that, which I'm happy to explain if you would like, but for simplicity let's ignore __radd__ and just say that when Python sees "a + b" what actually gets called is a.__add__(b). So when you create a new class and want it to support the + operator, you write a __add__ method: class Spam: def __add__(self, other): ... and now Python knows how to add your Spam instances together. Sometimes it is useful to treat the + operator as a function, e.g. so that you can pass it to another function like reduce. But operators aren't values, you can't pass them to functions. This doesn't work: py> reduce(+, [1, 2, 3, 4]) File "<stdin>", line 1 reduce(+, [1, 2, 3, 4]) ^ SyntaxError: invalid syntax But you can wrap the operator in a function using lambda: py> reduce(lambda a, b: a+b, [1, 2, 3, 4]) 10 but a more efficient way is to use the pre-made functions in the operator module: py> import operator py> reduce(operator.add, [1, 2, 3, 4]) 10 So for every operator + - * / ** etc. there is a corresponding function version in the operator module, add(), sub() etc. [ Aside: you might not know about reduce(). It takes a function f, and a list [a, b, c, d, ...] and calls the function with the first two values: result = f(a, b) then takes that result and repeatedly calls the function again with the next value from the list: result = f(result, c) result = f(result, d) ... until there are no more values left, then returns the final result. These days, now that Python has a sum() function, reduce() doesn't get used very often. ] So for each operator that Python knows about, there is the operator itself, a function version, and one or two special dunder methods: Operator Function Dunder methods ========== ============== ===================== + operator.add __add__ __radd__ - operator.sub __sub__ __rsub__ * operator.mul __mul__ __rmul__ ** operator.pow __pow__ __rpow__ == operator.eq __eq__ != operator.ne __ne__ etc. Then there are the special "augmented assignment" operators, so that Python can support writing: x += 1 y -= x etc. Again, the syntax used is a combined operator-assignment += and that ends up calling a special dunder method, __iadd__. And again, there are special function versions in the operator module. In summary: (1) When you want to add two values, use a + b. (2) When you want a function that adds two values, use operator.add. (3) When you want to write a class that supports addition, give it the two special dunder methods __add__ and __radd__. (4) You almost never should call __add__ yourself. -- Steven From steve at pearwood.info Wed Oct 1 04:16:56 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Wed, 1 Oct 2014 12:16:56 +1000 Subject: [Tutor] could somebody please explain... In-Reply-To: <m0fjj8$me5$1@ger.gmane.org> References: <01eb01cfdd01$86034220$9209c660$@us> <m0fjj8$me5$1@ger.gmane.org> Message-ID: <20141001021656.GX19757@ando.pearwood.info> On Wed, Oct 01, 2014 at 01:58:16AM +0100, Alan Gauld wrote: > On 30/09/14 23:54, Clayton Kirkwood wrote: > >I don't understand the multiplicity of some tools. Namely, why is there a > >'a+b', operator.add(a,b), operator.__add__(a,b), operator.iadd(a,b), > >operator.__iadd__(a,b) and their related operators? > > The operator module is there largely to allow you to pass > operations to functions that take functions as parameters. > For example map() which applies a function to a collection. > > total = map(operator.add, [1,2,3,4,5,6]) > > is the same result as > > total = sum([1,2,3,4,5,6]) No, you're thinking of reduce(), not map(). reduce() takes a function and applies it all of the items, pairwise, gradually reducing the result down to a single value. map() takes a function and applies it to each of the items individually, returning a new list. [...] > The operator module has functions representing most > of the builtin operations in Python. > > I'm not really sure why it implements the dunder methods > (eg operatotor.__add__()) corresponding to the operations though. > I'm sure somebody else can provide a good explanation... They're just aliases. According to the documentation, the "official" versions are operator.__add__, etc. with the underscore-less versions just given for convenience. But in practice everybody uses the non-underscore versions. My advice is to ignore the operator.__add__ and similar double-underscore versions. They're longer to type, don't add any additional value, and in fact are actually misleading since they don't directly call the dunder methods. -- Steven From steve at pearwood.info Wed Oct 1 04:33:57 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Wed, 1 Oct 2014 12:33:57 +1000 Subject: [Tutor] could somebody please explain... In-Reply-To: <01eb01cfdd01$86034220$9209c660$@us> References: <01eb01cfdd01$86034220$9209c660$@us> Message-ID: <20141001023357.GY19757@ando.pearwood.info> On Tue, Sep 30, 2014 at 03:54:42PM -0700, Clayton Kirkwood wrote: > Also, I found something that I can't get my mind around. It is part of the > time/date protocols. I've not seen it anywhere else. > > Datetime(year=blah, blah, blah).date/time() > > datetime(2013,3,6).date() #returns. > datetime.date(2013,3,6) > > datetime(2013,3,6).time() #returns. > datetime.time(0,0) > > This is one of the weirder things I've run across. Is this allowed/needed in > other functions/classes, or is it a datetime thing only? I'm afraid I have no clue what part of this you consider weird. Is it that the date() and time() methods don't take an argument? That's quite common: py> "Hello".upper() 'Hello' Or is it that the result of calling date() or time() methods isn't the same type of thing as what you started with? Again, that's very common: py> {1: 'a', 2: 'b'}.keys() # Start with a dict, returns a list. [1, 2] Start with a datetime object. The date() method returns the date part alone, so it returns a date object. The time() method returns the time part alone, so it returns a time object. Or maybe you're weirded out by the leading "datetime" in the name. That's unfortunate, but not weird. The datetime module contains at least three classes. When you print the class, they show the module name. It is unfortunate that the module name happens to have the same name as one of those classes: py> datetime <module 'datetime' from '/usr/local/lib/python2.7/lib-dynload/datetime.so'> py> datetime.date <type 'datetime.date'> py> datetime.time <type 'datetime.time'> py> datetime.datetime <type 'datetime.datetime'> So when you see something like this: py> d = datetime.datetime(2000, 5, 22, 11, 5, 27) py> d datetime.datetime(2000, 5, 22, 11, 5, 27) the "datetime." means the module, and the "datetime(...)" means the class with its various arguments. Is this common? Sadly, there are quite a few modules where the main function or class in the module has the same, or very similar, name: dis.dis bisect.bisect decimal.Decimal fractions.Fraction etc. (P.S. it is better to raise each independent question in a separate email.) -- Steven From dyoo at hashcollision.org Wed Oct 1 09:11:07 2014 From: dyoo at hashcollision.org (Danny Yoo) Date: Wed, 1 Oct 2014 00:11:07 -0700 Subject: [Tutor] could somebody please explain... In-Reply-To: <01eb01cfdd01$86034220$9209c660$@us> References: <01eb01cfdd01$86034220$9209c660$@us> Message-ID: <CAGZAPF5FkcOk-vPH3ccYc-M6-EaOhfXZYKQcqW+hCfSHWhXORg@mail.gmail.com> > Also, I found something that I can?t get my mind around. It is part of the > time/date protocols. I?ve not seen it anywhere else. > > Datetime(year=blah, blah, blah).date/time() > > datetime(2013,3,6).date() #returns? > datetime.date(2013,3,6) > > datetime(2013,3,6).time() #returns? > datetime.time(0,0) > > This is one of the weirder things I?ve run across. Is this allowed/needed in > other functions/classes, or is it a datetime thing only? Can you say more about what you expect? It may help to be very explicit, even if it seems silly. The problem with talking with experienced tutors and programmers is that our perspective has warped slightly from extended exposure. :P So we may need a bit of hinting to tell what you're referring to by weirdness. The datetime library, if I recall correctly, combines two things: the date part, and the time part, each which are otherwise treated separately. It's a composite object. https://docs.python.org/2/library/datetime.html#datetime-objects When we construct a datetime.datetime, at the very least we need to provide its year, month, and day, but the other "time" components of it are optional. That's what the documentation is trying to say when it wraps the arguments in braces here: https://docs.python.org/2/library/datetime.html#datetime.datetime If you don't provide the time-related arguments, I think it assumes that those components are zeroed out. From alan.gauld at btinternet.com Wed Oct 1 12:05:27 2014 From: alan.gauld at btinternet.com (Alan Gauld) Date: Wed, 01 Oct 2014 11:05:27 +0100 Subject: [Tutor] could somebody please explain... In-Reply-To: <20141001021656.GX19757@ando.pearwood.info> References: <01eb01cfdd01$86034220$9209c660$@us> <m0fjj8$me5$1@ger.gmane.org> <20141001021656.GX19757@ando.pearwood.info> Message-ID: <m0gjl7$f0m$1@ger.gmane.org> On 01/10/14 03:16, Steven D'Aprano wrote: >> For example map() which applies a function to a collection. >> >> total = map(operator.add, [1,2,3,4,5,6]) >> >> is the same result as >> >> total = sum([1,2,3,4,5,6]) > > No, you're thinking of reduce(), not map(). Oops, you're quite right. Apologies. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.flickr.com/photos/alangauldphotos From Ben.Smith at arnoldkeqms.com Wed Oct 1 17:59:36 2014 From: Ben.Smith at arnoldkeqms.com (Ben Smith) Date: Wed, 1 Oct 2014 15:59:36 +0000 Subject: [Tutor] Whack-a-mole Message-ID: <89409F4193210C4EB6B405E88013078230CA429D@QUIMBY.CSCo.Org.UK> Hi, Can anyone help explain why you can keep hitting the Mole even when hittable should be False? from tkinter import * root = Tk() #root.state('zoomed') sec = 0 points=0 pic=PhotoImage(file='Dirt.gif') pic2=PhotoImage(file='Mole.gif') hittable=False def HIT(): if hittable==True: global points;points+=1 butty.configure(image=pic) labby.configure(text=points) def tick(): global sec, hittable sec += 1 if sec == 3: hittable=True butty.configure(image=pic2) if sec==6: hittable=False butty.configure(image=pic) time['text'] = sec time.after(1000, tick) time = Label(root) time.pack() labby = Label(root, text="POINTS");labby.pack() Button(root, text='Start', command=tick).pack() butty=Button(root, image=pic, command=HIT);butty.pack() root.mainloop() This email and any attachments sent with it are intended only for the named recipient. If you are not that person please contact us immediately through our website and delete this message from your computer. You should not disclose the content nor take, retain or distribute any copies. No responsibility is accepted by AKS, United Learning or any associated entity for the contents of e-mails unconnected with their business. No responsibility is accepted for any loss or damage caused due to any virus attached to this email. AKS is part of United Learning, comprising: UCST (Registered in England No: 2780748. Charity No. 1016538) and ULT (Registered in England No. 4439859. An Exempt Charity). Companies limited by guarantee. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/tutor/attachments/20141001/e1ccf06c/attachment.html> From crk at godblessthe.us Wed Oct 1 18:07:02 2014 From: crk at godblessthe.us (Clayton Kirkwood) Date: Wed, 1 Oct 2014 09:07:02 -0700 Subject: [Tutor] could somebody please explain... In-Reply-To: <CAGZAPF5FkcOk-vPH3ccYc-M6-EaOhfXZYKQcqW+hCfSHWhXORg@mail.gmail.com> References: <01eb01cfdd01$86034220$9209c660$@us> <CAGZAPF5FkcOk-vPH3ccYc-M6-EaOhfXZYKQcqW+hCfSHWhXORg@mail.gmail.com> Message-ID: <001b01cfdd91$bd1b8ed0$3752ac70$@us> !-----Original Message----- !From: Danny Yoo [mailto:dyoo at hashcollision.org] !Sent: Wednesday, October 01, 2014 12:11 AM !To: Clayton Kirkwood !Cc: Python Tutor Mailing List !Subject: Re: [Tutor] could somebody please explain... ! !> Also, I found something that I can?t get my mind around. It is part of !> the time/date protocols. I?ve not seen it anywhere else. !> !> Datetime(year=blah, blah, blah).date/time() !> !> datetime(2013,3,6).date() #returns? !> datetime.date(2013,3,6) !> !> datetime(2013,3,6).time() #returns? !> datetime.time(0,0) !> !> This is one of the weirder things I?ve run across. Is this !> allowed/needed in other functions/classes, or is it a datetime thing !only? ! ! !Can you say more about what you expect? It may help to be very !explicit, even if it seems silly. The problem with talking with !experienced tutors and programmers is that our perspective has warped !slightly from extended exposure. :P So we may need a bit of hinting to !tell what you're referring to by weirdness. ! Sure, the interest is regarding the '2013,3,6' in the first datetime. I've not seen something in the first set of parenthesis before. Is the first one a class or a function, how can you tell without looking at its internals or some documentation? ! !The datetime library, if I recall correctly, combines two things: the !date part, and the time part, each which are otherwise treated !separately. It's a composite object. ! ! https://docs.python.org/2/library/datetime.html#datetime-objects ! !When we construct a datetime.datetime, at the very least we need to !provide its year, month, and day, but the other "time" components of it !are optional. That's what the documentation is trying to say when it !wraps the arguments in braces here: ! ! https://docs.python.org/2/library/datetime.html#datetime.datetime ! !If you don't provide the time-related arguments, I think it assumes that !those components are zeroed out. Yes, but apparently you can also specify the specific handle in any order like a dict. Somewhat. Clayton Kirkwood From crk at godblessthe.us Wed Oct 1 18:10:13 2014 From: crk at godblessthe.us (Clayton Kirkwood) Date: Wed, 1 Oct 2014 09:10:13 -0700 Subject: [Tutor] could somebody please explain... In-Reply-To: <20141001023357.GY19757@ando.pearwood.info> References: <01eb01cfdd01$86034220$9209c660$@us> <20141001023357.GY19757@ando.pearwood.info> Message-ID: <001c01cfdd92$30badcb0$92309610$@us> !-----Original Message----- !From: Tutor [mailto:tutor-bounces+crk=godblessthe.us at python.org] On !Behalf Of Steven D'Aprano !Sent: Tuesday, September 30, 2014 7:34 PM !To: tutor at python.org !Subject: Re: [Tutor] could somebody please explain... ! !On Tue, Sep 30, 2014 at 03:54:42PM -0700, Clayton Kirkwood wrote: ! !> Also, I found something that I can't get my mind around. It is part of !> the time/date protocols. I've not seen it anywhere else. !> !> Datetime(year=blah, blah, blah).date/time() !> !> datetime(2013,3,6).date() #returns. !> datetime.date(2013,3,6) !> !> datetime(2013,3,6).time() #returns. !> datetime.time(0,0) !> !> This is one of the weirder things I've run across. Is this !> allowed/needed in other functions/classes, or is it a datetime thing !only? ! !I'm afraid I have no clue what part of this you consider weird. Is it !that the date() and time() methods don't take an argument? That's quite !common: ! !py> "Hello".upper() !'Hello' ! ! !Or is it that the result of calling date() or time() methods isn't the !same type of thing as what you started with? Again, that's very common: ! !py> {1: 'a', 2: 'b'}.keys() # Start with a dict, returns a list. ![1, 2] ! ! !Start with a datetime object. The date() method returns the date part !alone, so it returns a date object. The time() method returns the time !part alone, so it returns a time object. ! !Or maybe you're weirded out by the leading "datetime" in the name. !That's unfortunate, but not weird. The datetime module contains at least !three classes. When you print the class, they show the module name. It !is unfortunate that the module name happens to have the same name as one !of those classes: ! !py> datetime !<module 'datetime' from '/usr/local/lib/python2.7/lib- !dynload/datetime.so'> !py> datetime.date !<type 'datetime.date'> !py> datetime.time !<type 'datetime.time'> !py> datetime.datetime !<type 'datetime.datetime'> ! ! !So when you see something like this: ! !py> d = datetime.datetime(2000, 5, 22, 11, 5, 27) d !datetime.datetime(2000, 5, 22, 11, 5, 27) ! !the "datetime." means the module, and the "datetime(...)" means the !class with its various arguments. ! !Is this common? Sadly, there are quite a few modules where the main !function or class in the module has the same, or very similar, name: ! !dis.dis !bisect.bisect !decimal.Decimal !fractions.Fraction ! !etc. ! ! !(P.S. it is better to raise each independent question in a separate !email.) ! The part in question is the date components in the parentheses of the first datetime. Clayton Kirkwood ! ! !-- !Steven !_______________________________________________ !Tutor maillist - Tutor at python.org !To unsubscribe or change subscription options: !https://mail.python.org/mailman/listinfo/tutor From crk at godblessthe.us Wed Oct 1 18:43:29 2014 From: crk at godblessthe.us (Clayton Kirkwood) Date: Wed, 1 Oct 2014 09:43:29 -0700 Subject: [Tutor] could somebody please explain... In-Reply-To: <20141001013814.GW19757@ando.pearwood.info> References: <01eb01cfdd01$86034220$9209c660$@us> <20141001013814.GW19757@ando.pearwood.info> Message-ID: <003201cfdd96$d4a17e20$7de47a60$@us> !-----Original Message----- !From: Tutor [mailto:tutor-bounces+crk=godblessthe.us at python.org] On !Behalf Of Steven D'Aprano !Sent: Tuesday, September 30, 2014 6:38 PM !To: tutor at python.org !Subject: Re: [Tutor] could somebody please explain... ! !On Tue, Sep 30, 2014 at 03:54:42PM -0700, Clayton Kirkwood wrote: ! !> I don't understand the multiplicity of some tools. Namely, why is !> there a 'a+b', operator.add(a,b), operator.__add__(a,b), !> operator.iadd(a,b), !> operator.__iadd__(a,b) and their related operators? ! !The + operator is the public interface, but the implementation that !makes + work are the special methods __add__ and __radd__ . ! !When you write in your code: ! ! result = a + b ! !how does Python know what to do with a and b? In principle, Python could !hard-code into the language a handful of types that the interpreter !knows how to add: int, float, str, list, etc. But that is not easily !extended when a new type is supported, and Python supports "operator !overloading" where any custom class can define "a + b" to do whatever !the class developer wants. ! !So when the Python interpreter executes a + b, when it does is look for !a special "dunder" (Double UNDERscore) method on a, __add__, or a !special dunder method __radd__ ("right add") on b, and calls that. ! !Actually the rules are a bit more complicated than that, which I'm happy !to explain if you would like, but for simplicity let's ignore __radd__ !and just say that when Python sees "a + b" what actually gets called is !a.__add__(b). ! !So when you create a new class and want it to support the + operator, !you write a __add__ method: ! !class Spam: ! def __add__(self, other): ! ... ! ! !and now Python knows how to add your Spam instances together. ! !Sometimes it is useful to treat the + operator as a function, e.g. so !that you can pass it to another function like reduce. But operators !aren't values, you can't pass them to functions. This doesn't work: ! !py> reduce(+, [1, 2, 3, 4]) ! File "<stdin>", line 1 ! reduce(+, [1, 2, 3, 4]) ! ^ !SyntaxError: invalid syntax ! ! !But you can wrap the operator in a function using lambda: ! !py> reduce(lambda a, b: a+b, [1, 2, 3, 4]) !10 ! ! !but a more efficient way is to use the pre-made functions in the !operator module: ! !py> import operator !py> reduce(operator.add, [1, 2, 3, 4]) !10 ! ! !So for every operator + - * / ** etc. there is a corresponding function !version in the operator module, add(), sub() etc. ! ! ![ Aside: you might not know about reduce(). It takes a function f, and a !list [a, b, c, d, ...] and calls the function with the first two values: ! ! result = f(a, b) ! !then takes that result and repeatedly calls the function again with the !next value from the list: ! ! result = f(result, c) ! result = f(result, d) ! ... ! !until there are no more values left, then returns the final result. !These days, now that Python has a sum() function, reduce() doesn't get !used very often. ] ! !So for each operator that Python knows about, there is the operator !itself, a function version, and one or two special dunder methods: ! ! Operator Function Dunder methods ! ========== ============== ===================== ! + operator.add __add__ __radd__ ! - operator.sub __sub__ __rsub__ ! * operator.mul __mul__ __rmul__ ! ** operator.pow __pow__ __rpow__ ! == operator.eq __eq__ ! != operator.ne __ne__ ! !etc. ! !Then there are the special "augmented assignment" operators, so that !Python can support writing: ! ! x += 1 ! y -= x ! !etc. Again, the syntax used is a combined operator-assignment += and !that ends up calling a special dunder method, __iadd__. And again, there !are special function versions in the operator module. ! ! !In summary: ! !(1) When you want to add two values, use a + b. ! !(2) When you want a function that adds two values, use operator.add. ! !(3) When you want to write a class that supports addition, give it ! the two special dunder methods __add__ and __radd__. ! !(4) You almost never should call __add__ yourself. ! In an effort to learn and teach, I present a simple program which measures the time it takes to the various add functions with the appending results: # program to test time and count options import datetime,operator, sys from datetime import time, date, datetime date = datetime.now() dayofweek = date.strftime("%a, %b") print("Today is", dayofweek, date.day, "at ", date.time()) start = 0 count_max=int(input("give me a number")) start_time = datetime.now() print( start_time ) while start < count_max: start=start + 1 end_time = datetime.now() print( "s=s+1 time difference is:", (end_time - start_time) ) start=0 start_time = datetime.now() while( start < count_max ): start += 1 end_time = datetime.now() print( "the += time difference is:", (end_time - start_time) ) start_time = datetime.now() start = 0 while( start < count_max ): start = operator.add( start, 1) end_time = datetime.now() print( "the operator.add time difference is:", (end_time - start_time) ) start_time = datetime.now() start=0 while( start < count_max ): start = operator.iadd( start, 1) end_time = datetime.now() print( "the operator.iadd time difference is:", (end_time - start_time) ) start_time = datetime.now() start=0 while( start < count_max ): start = operator.__add__(start,1) end_time = datetime.now() print( "the operator.__add__ time difference is:", (end_time - start_time) ) start_time = datetime.now() start=0 while( start < count_max ): start = operator.__iadd__(start,1) end_time = datetime.now() print( "the operator.__iadd__ time difference is:", (end_time - start_time) ) As can be seen below, there is a definite pattern: s=s+1 and s+=1, are faster. There is some variability within the two schemes, but typically, the iadds are slowest. Today is Wed, Oct 1 at 09:19:05.671235 give me a number22222222 2014-10-01 09:19:18.485235 s=s+1 time difference is: 0:00:09.438000 the += time difference is: 0:00:09.072000 the operator.add time difference is: 0:00:17.172000 the operator.iadd time difference is: 0:00:17.325000 the operator.__add__ time difference is: 0:00:17.248000 the operator.__iadd__ time difference is: 0:00:17.673000 Today is Wed, Oct 1 at 09:35:06.525235 give me a number10000000 2014-10-01 09:35:17.915235 s=s+1 time difference is: 0:00:04.412000 the += time difference is: 0:00:04.343000 the operator.add time difference is: 0:00:07.499000 the operator.iadd time difference is: 0:00:07.734000 the operator.__add__ time difference is: 0:00:07.494000 the operator.__iadd__ time difference is: 0:00:07.906000 Today is Wed, Oct 1 at 09:39:07.830235 give me a number100000 2014-10-01 09:39:14.445235 s=s+1 time difference is: 0:00:00.042000 the += time difference is: 0:00:00.041000 the operator.add time difference is: 0:00:00.084000 the operator.iadd time difference is: 0:00:00.077000 the operator.__add__ time difference is: 0:00:00.076000 the operator.__iadd__ time difference is: 0:00:00.080000 Process finished with exit code 0 ! ! Clayton !-- !Steven !_______________________________________________ !Tutor maillist - Tutor at python.org !To unsubscribe or change subscription options: !https://mail.python.org/mailman/listinfo/tutor From dyoo at hashcollision.org Wed Oct 1 19:21:49 2014 From: dyoo at hashcollision.org (Danny Yoo) Date: Wed, 1 Oct 2014 10:21:49 -0700 Subject: [Tutor] could somebody please explain... In-Reply-To: <001b01cfdd91$bd1b8ed0$3752ac70$@us> References: <01eb01cfdd01$86034220$9209c660$@us> <CAGZAPF5FkcOk-vPH3ccYc-M6-EaOhfXZYKQcqW+hCfSHWhXORg@mail.gmail.com> <001b01cfdd91$bd1b8ed0$3752ac70$@us> Message-ID: <CAGZAPF6xwbFHECFjtxip5o7HpX25g25BO0aMX+xsqKWtT_ok2Q@mail.gmail.com> > !> Also, I found something that I can?t get my mind around. It is part of > !> the time/date protocols. I?ve not seen it anywhere else. > !> > !> Datetime(year=blah, blah, blah).date/time() > !> > !> datetime(2013,3,6).date() #returns? > !> datetime.date(2013,3,6) > !> > !> datetime(2013,3,6).time() #returns? > !> datetime.time(0,0) > !> > !> This is one of the weirder things I?ve run across. Is this > !> allowed/needed in other functions/classes, or is it a datetime thing > !only? > ! > ! > !Can you say more about what you expect? It may help to be very > !explicit, even if it seems silly. The problem with talking with > !experienced tutors and programmers is that our perspective has warped > !slightly from extended exposure. :P So we may need a bit of hinting to > !tell what you're referring to by weirdness. > ! > > Sure, the interest is regarding the '2013,3,6' in the first datetime. I've not seen something in the first set of parenthesis before. Is the first one a class or a function, how can you tell without looking at its internals or some documentation? Hi Clayton, I will assume that, at the very beginning of your program, you've done: from datetime import datetime or something equivalent to this. The expression: datetime(2013,3,6).date() can be rewritten as two separate pieces: t = datetime(2013, 3, 6) t.date() with the difference that, in the original expression, the result of the `datetime(2013,3,6)` is not given an explicit variable name, but is directly used as part of a larger expression. You will have seen this before. For example, in the function: ############################################# def c2f(c): """Returns conversion from celsius to fahrenheit.""" return (c * 9/5) + 32 ############################################# the mathematical expression: (c * 9/5) + 32 has two parts to it. We could have rewritten the c2f() function as this: ############################################# def c2f(c): """Returns conversion from celsius to fahrenheit.""" t = c * 9/5 f = t + 32 return f ############################################# where we store the value of each expression with a variable name. Similar meaning, but more verbose. Sometimes we don't need to name every value because otherwise the code is pedantic. But sometimes names help make large expressions easier to understand. Good taste is the judge. One of the key things about expressions is that they "compose": they act like lego in the sense that you can plug them into each other with very few restrictions. So this expression composition is what's happening in: datetime(2013,3,6).date() [More technical note: grammatically, the expression above is an "attribute reference", as defined in: https://docs.python.org/2/reference/expressions.html#attribute-references. The left hand side of an attribute reference expression can itself be a "primary" sub-expression as defined by the grammar.] As for the return value of datetime(2013,3,6): whether it returns an object or something else, you have to trust the documentation. In Python, object construction uses the same syntax as a function call. This is different than from a few other languages, where object construction has a distinct syntax. One of the advantages of having a uniform syntax is that it's easier to later swap out the object construction with something more sophisticated, such as: memoization, or pooling, or other managed work. The disadvantage is that it's harder to see from the source code alone where allocations occur. The documentation of: https://docs.python.org/2/library/datetime.html#datetime.datetime tells us that the return value of: datetime(2013,3,6) is an instance of the datetime class in the datetime module. (It's a bit unfortunate that the class name and the module name use the same name, so as to encourage confusion.) From alan.gauld at btinternet.com Wed Oct 1 23:44:32 2014 From: alan.gauld at btinternet.com (Alan Gauld) Date: Wed, 01 Oct 2014 22:44:32 +0100 Subject: [Tutor] Whack-a-mole In-Reply-To: <89409F4193210C4EB6B405E88013078230CA429D@QUIMBY.CSCo.Org.UK> References: <89409F4193210C4EB6B405E88013078230CA429D@QUIMBY.CSCo.Org.UK> Message-ID: <m0hsk0$fmb$1@ger.gmane.org> On 01/10/14 16:59, Ben Smith wrote: > Hi, Can anyone help explain why you can keep hitting the Mole even when hittable should be False? I can't. I can hit it 3 seconds after hitting Start then it turns un-hittable and the secs counter keeps counting but nothing else responds until you close the window. The only changes I made were to replace the images with text (I didn't have your gif files...) So if you make those changes and it works for you then it must be something about the way you are using the images... HTH -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.flickr.com/photos/alangauldphotos From stefan.sthilaire at gmail.com Wed Oct 1 23:34:49 2014 From: stefan.sthilaire at gmail.com (Stefan St-Hilaire) Date: Wed, 01 Oct 2014 17:34:49 -0400 Subject: [Tutor] VERY basic question Message-ID: <542C7379.1000702@gmail.com> Hello, I am just starting out with Python and ran into a problem day one. I am doing this statement: input("\n\nPress the enter key to exit.") I get the following error: >>> input("\n\nPress the enter key to exit.") Press the enter key to exit. Traceback (most recent call last): File "<stdin>", line 1, in <module> File "<string>", line 0 ^ SyntaxError: unexpected EOF while parsing I am using Linux (Shell) and PyCharm and get the same result when I run the command. I know this is stupid but any help would be appreciated. Thanks, Stefan From alan.gauld at btinternet.com Thu Oct 2 00:06:15 2014 From: alan.gauld at btinternet.com (Alan Gauld) Date: Wed, 01 Oct 2014 23:06:15 +0100 Subject: [Tutor] VERY basic question In-Reply-To: <542C7379.1000702@gmail.com> References: <542C7379.1000702@gmail.com> Message-ID: <m0htso$uu4$1@ger.gmane.org> On 01/10/14 22:34, Stefan St-Hilaire wrote: > >>> input("\n\nPress the enter key to exit.") > > > Press the enter key to exit. > Traceback (most recent call last): > File "<stdin>", line 1, in <module> > File "<string>", line 0 > > ^ > SyntaxError: unexpected EOF while parsing > > > I am using Linux (Shell) and PyCharm and get the same result when I run > the command. I know this is stupid but any help would be appreciated. Can you tell us more about how you run this in the Linux shell? You start Python by typing 'python' at a bash shell? You get the >>> prompt appearing? You type input("\n\nPress the enter key to exit.") at the >>> prompt? You get the Press the enter key to exit. prompt? What do you actually hit then - which specific key(s)? You then see the error message? Is that right? I can't reproduce the exact error message you are seeing, that's why I'm asking for the details... -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.flickr.com/photos/alangauldphotos From juan0christian at gmail.com Thu Oct 2 00:56:58 2014 From: juan0christian at gmail.com (Juan Christian) Date: Wed, 1 Oct 2014 19:56:58 -0300 Subject: [Tutor] Beautifulsoup4 question Message-ID: <CAAp0bGuM8StWSM+-bV4TesjeCVhT7cTX1UHqR=31zh1y__fQ9A@mail.gmail.com> I have this part of the code - full_item_list = self._soup.find_all('li', {'data-app': '440'}) - that gives me this: Very long output (~ 211 lines): http://pastebin.com/WLTtgVZz Now I need to filter this RAW data, what I need is to convert this data to something like a list of dicts in Python, so that I can do, let's say... for item in data: item['data-name'] > returns > 'Mann Co. Supply Crate' item['data-p-bptf'] > returns > '0.01 ref' item['image'] > returns > 'URL_TO_IMG' item['data-original-id'] > returns > '2713101947' and so on... It would be a list of dicts, each item in the list would be one "<li> item already parsed/filtered", and inside each list item I'd have a dict with these info. Is there something in bs4 that does that, or maybe a different module? -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/tutor/attachments/20141001/7a6a4e64/attachment.html> From __peter__ at web.de Thu Oct 2 01:16:36 2014 From: __peter__ at web.de (Peter Otten) Date: Thu, 02 Oct 2014 01:16:36 +0200 Subject: [Tutor] VERY basic question References: <542C7379.1000702@gmail.com> Message-ID: <m0i20l$gd6$1@ger.gmane.org> Stefan St-Hilaire wrote: > Hello, I am just starting out with Python and ran into a problem > day one. I am doing this statement: > > input("\n\nPress the enter key to exit.") > > I get the following error: > > >>> input("\n\nPress the enter key to exit.") > > > Press the enter key to exit. > Traceback (most recent call last): > File "<stdin>", line 1, in <module> > File "<string>", line 0 > > ^ > SyntaxError: unexpected EOF while parsing > > > I am using Linux (Shell) and PyCharm and get the same result when I run > the command. I know this is stupid but any help would be appreciated. You may be using Python 2 to run a code example written in Python 3. In Python 2 the string entered in input() was evaluated as a Python expression, and an empty string is a syntax error as you can verify with eval(): >>> eval("") Traceback (most recent call last): File "<stdin>", line 1, in <module> File "<string>", line 0 ^ SyntaxError: unexpected EOF while parsing To get the string as entered by the user you had to use raw_input() instead of input(): >>> raw_input("\n\nPress the enter key to exit.") Press the enter key to exit. '' >>> From dyoo at hashcollision.org Thu Oct 2 01:02:51 2014 From: dyoo at hashcollision.org (Danny Yoo) Date: Wed, 1 Oct 2014 16:02:51 -0700 Subject: [Tutor] Beautifulsoup4 question In-Reply-To: <CAAp0bGuM8StWSM+-bV4TesjeCVhT7cTX1UHqR=31zh1y__fQ9A@mail.gmail.com> References: <CAAp0bGuM8StWSM+-bV4TesjeCVhT7cTX1UHqR=31zh1y__fQ9A@mail.gmail.com> Message-ID: <CAGZAPF7F7VMn0AMwUpS9Xq2jXdKYyzHwoe=fGfLGsM6KC+MxjA@mail.gmail.com> Hi Juan, What you should be getting back from the call to find_all() should already be dictionary-like. Although they *print* like HTML, they're really soups. So you should already be able to do: ############################################# full_item_list = self._soup.find_all('li', {'data-app': '440'}) for item in full_item_list: print item['data-name'] ############################################# That is to say that what you're getting back from self._soup.find_all() is definitely not raw: it's parsed, it's soupy, and you can continue to deal with it structurally. From juan0christian at gmail.com Thu Oct 2 02:37:58 2014 From: juan0christian at gmail.com (Juan Christian) Date: Wed, 1 Oct 2014 21:37:58 -0300 Subject: [Tutor] Beautifulsoup4 question In-Reply-To: <CAGZAPF7F7VMn0AMwUpS9Xq2jXdKYyzHwoe=fGfLGsM6KC+MxjA@mail.gmail.com> References: <CAAp0bGuM8StWSM+-bV4TesjeCVhT7cTX1UHqR=31zh1y__fQ9A@mail.gmail.com> <CAGZAPF7F7VMn0AMwUpS9Xq2jXdKYyzHwoe=fGfLGsM6KC+MxjA@mail.gmail.com> Message-ID: <CAAp0bGub6PNinTwj4jTGbW_jgc89TWwOvS8pWkw2mbBV+3i=pA@mail.gmail.com> On Wed, Oct 1, 2014 at 8:02 PM, Danny Yoo <dyoo at hashcollision.org> wrote: > Hi Juan, > > What you should be getting back from the call to find_all() should > already be dictionary-like. Although they *print* like HTML, they're > really soups. > > So you should already be able to do: > > ############################################# > full_item_list = self._soup.find_all('li', {'data-app': '440'}) > > for item in full_item_list: > print item['data-name'] > ############################################# > > That is to say that what you're getting back from > self._soup.find_all() is definitely not raw: it's parsed, it's soupy, > and you can continue to deal with it structurally. > OH MY GOD! Super fail, hahaha. Thanks, bs4 is incredible. I thought they were RAW html data. Thank you! -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/tutor/attachments/20141001/beacd200/attachment.html> From juan0christian at gmail.com Thu Oct 2 03:44:18 2014 From: juan0christian at gmail.com (Juan Christian) Date: Wed, 1 Oct 2014 22:44:18 -0300 Subject: [Tutor] Beautifulsoup4 question In-Reply-To: <CAAp0bGub6PNinTwj4jTGbW_jgc89TWwOvS8pWkw2mbBV+3i=pA@mail.gmail.com> References: <CAAp0bGuM8StWSM+-bV4TesjeCVhT7cTX1UHqR=31zh1y__fQ9A@mail.gmail.com> <CAGZAPF7F7VMn0AMwUpS9Xq2jXdKYyzHwoe=fGfLGsM6KC+MxjA@mail.gmail.com> <CAAp0bGub6PNinTwj4jTGbW_jgc89TWwOvS8pWkw2mbBV+3i=pA@mail.gmail.com> Message-ID: <CAAp0bGvptLNgqAq+r__NRqTJ=XHKYbQpG8XbRof3pQX6pjy7OA@mail.gmail.com> On Wed, Oct 1, 2014 at 9:37 PM, Juan Christian <juan0christian at gmail.com> wrote: > OH MY GOD! Super fail, hahaha. > > Thanks, bs4 is incredible. I thought they were RAW html data. Thank you! > Not everything is that easy, hahaha. So, I can get everything I want, but this part: <li "="" WHATEVER WHATEVER WHATEVER WHATEVER"> <div class="item-icon" style="background-image:url( http://media.steampowered.com/apps/440/icons/wading_crate_2.dea09767ade382b0151eb6251d1e5b6deaf8ab75.png)"><div class="value">#86</div></div><span style="display: none"> WHATEVER </span></li> I need this, the image link: background-image:url(NEED_THIS_LINK) I need the image link, I have this in all "<li> items", how can I get that? Sometimes it has a .PNG in the end and sometimes it's a '94x94' in the end, as in here: http://cdn.steamcommunity.com/economy/image/iRulfx1JB6hWyBlnfvJwHzFXb85ZOQnoggbKfZoUOLhAEJKERFVBuvYRBZlYkiyRKVA0ilcmCeyEDc1vmwQTvkAThpJsU1Kx92AKl0faKM86RyzaVSQWs9RQlyrSVHW5FkTa0gJUB7nzWlSTA9l91jsSItxWdxDgyRLNew==/94x94 Thanks. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/tutor/attachments/20141001/ef1d958a/attachment.html> From alan.gauld at btinternet.com Thu Oct 2 13:56:31 2014 From: alan.gauld at btinternet.com (ALAN GAULD) Date: Thu, 2 Oct 2014 12:56:31 +0100 (BST) Subject: [Tutor] VERY basic question Message-ID: <1346078.12213.1412250991277.JavaMail.defaultUser@defaultHost> I've been using Python3 for a while now so forgot how Python 2 handled input errors. You could use Python 2 but you'd need to replace input() with raw_input(). But for learning I'd advise you to stick with Python3, just don't delete python2 from your PC since several Linux tools rely on it. Alan Gauld Author of the Learn To Program website http://www.alan-g.me.uk/ http://www.flickr.com/photos/alangauldphotos >----Original message---- >From : stefan.sthilaire at gmail.com >Date : 02/10/2014 - 00:30 (BST) >To : alan.gauld at btinternet.com >Subject : Re: [Tutor] VERY basic question > >You're a saint. I installed the latest version...3.4 and it executes >with no errors. > >Thanks Again. > > >Stefan > > >On 10/01/2014 06:25 PM, ALAN GAULD wrote: >>> Hi Alan, yes I just type python at the shell to get into interactive >>> mode. Yes all I do is hit the enter key and I get this response. Nothing >>> else. I am following a book to learn python and this is one of the first >>> exercises. The Hello World, followed by a prompt to hit enter to exit. >>> >> ok, its because you are using Python2 interpreter. >> >> Start python using python3 and see what happens. >> >> Alan g > > From crushed26 at gmail.com Thu Oct 2 17:41:49 2014 From: crushed26 at gmail.com (Bo Morris) Date: Thu, 2 Oct 2014 11:41:49 -0400 Subject: [Tutor] printing all text that begins with "25" Message-ID: <CAKKCnfezr_eUvJ18bS4+DOeL7Q8-un8KdHqNGnHSnbKdgtKmsg@mail.gmail.com> Hello all, hope everyone is doing well. When I run the linux command "hamachi list" i get something along the lines of the following output 087-888-279 Pandora 25.x.x.xxx alias: not set 096-779-867 AM1LaptopBD-PC 25.x.x.xxx alias: not set 097-552-220 OWS-Desktop 1 25.0.0.0 alias: not set 099-213-641 DESKTOP 25.0.0.0 alias: not set I am trying to write a python script that will run the above command and only print out the IP's that begin with 25. How do I strip out all other text except for the IP's that begin with "25?" Would it be best to send to a file first, then read the contents of the file? Would I need to use regex? I know how to run the above command in python. I also know how to send the output to a file and read the file; however I cannot figure out how to strip all text out except for the IPs that begin with "25." Thanks PS...Danny, still working on the "root" problem we discussed in previous emails. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/tutor/attachments/20141002/e16d9edb/attachment.html> From davea at davea.name Thu Oct 2 18:36:16 2014 From: davea at davea.name (Dave Angel) Date: Thu, 2 Oct 2014 12:36:16 -0400 (EDT) Subject: [Tutor] printing all text that begins with "25" References: <CAKKCnfezr_eUvJ18bS4+DOeL7Q8-un8KdHqNGnHSnbKdgtKmsg@mail.gmail.com> Message-ID: <m0juon$b7l$1@ger.gmane.org> Bo Morris <crushed26 at gmail.com> Wrote in message: (Thanks for starting a new thread when asking a new question. But please use text mode in your emails, not html.) For the first version, write it as a filter, and pipe the two commands together in the shell. So all you have to do is read a line from stdin, parse it, and conditionally write it to std. You don't provide a spec, just a short sample of data. So I'll have to guess that the leading whitespace is irrelevant and the first two fields cannot contain whitespace, and that each contain at least one non-whitespace character. Further that the fields are delimited by whitespace. So, use lstrip to get rid of leading junk, and split to split the line into fields. Then subscript into the resulting list to get the appropriate field. And use startswith to check the desired 3 character string. Notice that you don't just want to use "25", or you might accept an ip like 251.12.3.6 That still leaves you to deal with reporting invalid files, such as those with short or blank lines. -- DaveA From david at graniteweb.com Thu Oct 2 18:33:36 2014 From: david at graniteweb.com (David Rock) Date: Thu, 2 Oct 2014 11:33:36 -0500 Subject: [Tutor] printing all text that begins with "25" In-Reply-To: <CAKKCnfezr_eUvJ18bS4+DOeL7Q8-un8KdHqNGnHSnbKdgtKmsg@mail.gmail.com> References: <CAKKCnfezr_eUvJ18bS4+DOeL7Q8-un8KdHqNGnHSnbKdgtKmsg@mail.gmail.com> Message-ID: <20141002163336.GH3591@wdfs> * Bo Morris <crushed26 at gmail.com> [2014-10-02 11:41]: > Hello all, hope everyone is doing well. > > When I run the linux command "hamachi list" i get something along the lines > of the following output > > 087-888-279 Pandora 25.x.x.xxx alias: not set > 096-779-867 AM1LaptopBD-PC 25.x.x.xxx alias: not set > 097-552-220 OWS-Desktop 1 25.0.0.0 alias: not set > 099-213-641 DESKTOP 25.0.0.0 alias: not set > > I am trying to write a python script that will run the above command and > only print out the IP's that begin with 25. How do I strip out all other > text except for the IP's that begin with "25?" There are a few assumptions that need to be made, for starters. Is the format always the same (ie, is the IP address always in column 3 separated by whitespace)? Looking at the line with "OWS-Desktop 1", the answer is no. That complicates things a bit. If it was true, you could use the string split method to get column 3. Maybe the fields are separated by a tab? A regex may be possible, but you will have similar issues to using split. I'm also assuming it's possible for there to be IP addresses that do not start with 25. Are you looking to isolate those? It's not necessary to write out to a file first. You can get the output from commands and work on it directly. Another approach would be to change the command you are running. I've never heard of hamachi list before; does it have any commandline options to display only IP addresses? -- David Rock david at graniteweb.com From davea at davea.name Thu Oct 2 19:35:20 2014 From: davea at davea.name (Dave Angel) Date: Thu, 02 Oct 2014 13:35:20 -0400 Subject: [Tutor] print date and skip any repeats In-Reply-To: <CAN_=oguHEiK-hcq+ziDM=e5yJCLKFmepc3LpWb+9=NmRcY5x7A@mail.gmail.com> References: <CAN_=oguBjMn9rGe7rrfj1aWK=oUK7PGaBZbEn_eORwRQnwVpAA@mail.gmail.com> <20140917093505.GA9051@cskk.homeip.net> <CAN_=ogt6jgP-o2XVMyANUbpH=Ds2AhdT_num=JFk4tq-9w3Qfw@mail.gmail.com> <lvr5v7$msq$1@ger.gmane.org> <CAN_=ogsOTkqktBsC_WW=5ckd2-SgD+LFKjPaB07eoermZS97wA@mail.gmail.com> <CAN_=oguHEiK-hcq+ziDM=e5yJCLKFmepc3LpWb+9=NmRcY5x7A@mail.gmail.com> Message-ID: <542D8CD8.5040002@davea.name> On 09/24/2014 05:19 AM, questions anon wrote: > Ok, I am continuing to get stuck. I think I was asking the wrong question > so I have posted the entire script (see below). > What I really want to do is find the daily maximum for a dataset (e.g. > Temperature) that is contained in monthly netcdf files where the data are > separated by hour. > The major steps are: > open monthly netcdf files and use the timestamp to extract the hourly data > for the first day. > Append the data for each hour of that day to a list, concatenate, find max > and plot > then loop through and do the following day. > > I can do some of the steps separately but I run into trouble in working out > how to loop through each hour and separate the data into each day and then > repeat all the steps for the following day. > > Any feedback will be greatly appreciated! > > This is NOT the whole program. You don't seem to define ncvariablename, Dataset, np, Basemap, plt, and probably others. > > oneday=[] > all_the_dates=[] > onedateperday=[] > > > #open folders and open relevant netcdf files that are monthly files > containing hourly data across a region > for (path, dirs, files) in os.walk(MainFolder): > for ncfile in files: If there are more than one file, or more than one directory, then there are no guarantees that this will give you the files in the order you are likely to want. You may need to do some sorting to make it work out. But after further study, you seem to assume you'll read everything in from all the files, and then process it. Have you studied how big that might get, and whether you'll have enough memory to make it viable? > if ncfile.endswith(ncvariablename+'.nc'): > print "dealing with ncfiles:", path+ncfile > ncfile=os.path.join(path,ncfile) > ncfile=Dataset(ncfile, 'r+', 'NETCDF4') > variable=ncfile.variables[ncvariablename][:,:,:] > TIME=ncfile.variables['time'][:] Since you've given ncfile MANY different meanings, this close() will not close the file. Variables are cheap, use lots of them, and give them good names. In this particular case, maybe using a with statement would make sense. > ncfile.close() > > #combine variable and time so I can make calculations based > on hours or days > for v, time in zip((variable[:,:,:]),(TIME[:])): > > cdftime=utime('seconds since 1970-01-01 00:00:00') > ncfiletime=cdftime.num2date(time) > timestr=str(ncfiletime) > utc_dt = dt.strptime(timestr, '%Y-%m-%d %H:%M:%S') > au_tz = pytz.timezone('Australia/Sydney') > local_dt = > utc_dt.replace(tzinfo=pytz.utc).astimezone(au_tz) > strp_local=local_dt.strftime('%Y-%m-%d_%H') #strips > time down to date and hour > local_date=local_dt.strftime('%Y-%m-%d') #strips time > down to just date > > all_the_dates.append(local_date) > > #make a list that strips down the dates to only have one date per day > (rather than one for each hour) > onedateperday = sorted ( list (set (all_the_dates))) > > #loop through each day and combine data (v) where the hours occur on the > same day > for days in onedateperday: > if strp_local.startswith(days): > oneday.append(v) > And just where does v come from? You're no longer in the loop that says "for v, time in..." > big_array=np.ma.dstack(oneday) #concatenate data > v_DailyMax=big_array.max(axis=2) # find max > > #then go on to plot v_DailyMax for each day > map = Basemap(projection='merc',llcrnrlat=-40,urcrnrlat=-33, > llcrnrlon=139.0,urcrnrlon=151.0,lat_ts=0,resolution='i') > map.drawcoastlines() > map.drawstates() > map.readshapefile(shapefile1, 'REGIONS') > x,y=map(*np.meshgrid(LON,LAT)) > plottitle=ncvariablename+'v_DailyMax'+days > cmap=plt.cm.jet > CS = map.contourf(x,y,v_DailyMax, 15, cmap=cmap) > l,b,w,h =0.1,0.1,0.8,0.8 > cax = plt.axes([l+w+0.025, b, 0.025, h]) > plt.colorbar(CS,cax=cax, drawedges=True) > plt.savefig((os.path.join(OutputFolder, plottitle+'.png'))) > plt.show() > plt.close() > > It looks to me like you've jumbled up a bunch of different code fragments. Have you learned about writing functions yet? Or generators? By making everything global, you're masking a lot of mistakes you've made. For example, v has a value, but just the data from the last record of the last file. I think you need to back off and design this, without worrying much at first about how to code it. It's generally a bad idea to try to collect all data before processing it and winnowing it down. If you know something about the filenaming (or directory naming), and can assume that all data from one day will be processed before encountering the next day, you can greatly simplify things. Likewise if one hour is complete before the next begins. Is there one file per day? Is there one record per hour and are they in order? If some such assumptions can be made, then you might be able to factor the problem down to a set of less grandiose functions. Whoever wrote Dataset() has produced some data from a single file that you can work with. What does that data look like, and how much of it are you really needing to save past the opening of the next one? Sometimes when the data is large and not sorted, it pays off to make two or more passes through it. In the extreme, you might take one pass to make a sorted list of all the hours involved in all the files. Then for each of those hours, you'd take an additional pass looking for data related to that particular hour. Of course if you're talking years, you'll be opening each file many thousands of times. So if you know ANYTHING about the sorting of the data, you can make that more efficient. I can't help you at all with numpy (presumably what np stands for), or the plotting stuff. -- DaveA From crushed26 at gmail.com Thu Oct 2 22:20:43 2014 From: crushed26 at gmail.com (Crush) Date: Thu, 2 Oct 2014 16:20:43 -0400 Subject: [Tutor] printing all text that begins with 25" In-Reply-To: <mailman.83299.1412271339.18129.tutor@python.org> References: <mailman.83299.1412271339.18129.tutor@python.org> Message-ID: <07182624-2758-4E48-B846-6AEF788AF581@gmail.com> Yes the format is always the same and the IPs will always be in the 3rd collumn; although, the amount of whitspace that seperates column 2 and 3 may be different depending on how long the name is in column 2. Also all of the IPs will begin with a "25," so there would be no fear of having to deal with other IP addresses that start with anything else. Hamachi is VPN software and unfortunately, there is no command line argument that allows one to isolate the IPs. Bo From dantheman5457 at gmail.com Thu Oct 2 17:47:44 2014 From: dantheman5457 at gmail.com (John Doe) Date: Thu, 2 Oct 2014 11:47:44 -0400 Subject: [Tutor] Iterating Lines in File and Export Results Message-ID: <CAFdOivu0HyNpuDRfmp7BWTP+u1yXu9wZNLx85bjuFcmwZ3pVBg@mail.gmail.com> Hello List, I am in need of your assistance. I have a text file with random words in it. I want to write all the lines to a new file. Additionally, I am using Python 2.7 on Ubuntu 12.04: Here is my code: def loop_extract(): with open('words.txt', 'r') as f: for lines in f: #print lines (I confirmed that each line is successfully printed) with open('export.txt', 'w') as outf: outf.write(lines) #outf.write(lines) #outf.write('{}\n'.format(lines)) #outf.write('{}\n'.format(line for line in lines)) For some reason, the second file only contains the last line from the original file -- I have tried multiple variations (.read, .readlines, .writelines, other examples preceded by comment from above and many more) and tried to use the module, fileinput, but I still get the same results. I do understand there is another way to copy the file over, but to provide additional background information on my purpose -- I want to read a file and save successful regex matches to a file; exporting specific data. There doesn't appear to be anything wrong with my expression as it prints the expected results without failure. I then decided to just write the export function by itself in its basic form, per the code above, which the same behavior occurred; only copying the last line. I've googled for hours and, unfortunately, at loss. Thank you in advance for your help! From dantheman5457 at gmail.com Thu Oct 2 18:58:54 2014 From: dantheman5457 at gmail.com (John Doe) Date: Thu, 2 Oct 2014 12:58:54 -0400 Subject: [Tutor] printing all text that begins with "25" In-Reply-To: <20141002163336.GH3591@wdfs> References: <CAKKCnfezr_eUvJ18bS4+DOeL7Q8-un8KdHqNGnHSnbKdgtKmsg@mail.gmail.com> <20141002163336.GH3591@wdfs> Message-ID: <CAFdOivu6eMKAmqUg3g9DPGVNtQi20gKyoJbqLVkdrO0t1sZN2Q@mail.gmail.com> Hello, If you want to accomplish what you are looking for within linux (perhaps a bash script, instead?): $ hamachi list | grep -oP '25\.\d+\.\d+\.\d+' 25.0.0.0 25.255.255.255 For your python script, you want to group your regex: reg = re.compile(r'(25\.\d+\.\d+\.\d+)', re.MULTILINE) So when you call group(1) or group(0), it'll grab just the addresses. On Thu, Oct 2, 2014 at 12:33 PM, David Rock <david at graniteweb.com> wrote: > * Bo Morris <crushed26 at gmail.com> [2014-10-02 11:41]: >> Hello all, hope everyone is doing well. >> >> When I run the linux command "hamachi list" i get something along the lines >> of the following output >> >> 087-888-279 Pandora 25.x.x.xxx alias: not set >> 096-779-867 AM1LaptopBD-PC 25.x.x.xxx alias: not set >> 097-552-220 OWS-Desktop 1 25.0.0.0 alias: not set >> 099-213-641 DESKTOP 25.0.0.0 alias: not set >> >> I am trying to write a python script that will run the above command and >> only print out the IP's that begin with 25. How do I strip out all other >> text except for the IP's that begin with "25?" > > There are a few assumptions that need to be made, for starters. > > Is the format always the same (ie, is the IP address always in column 3 > separated by whitespace)? Looking at the line with "OWS-Desktop 1", the > answer is no. That complicates things a bit. If it was true, you could > use the string split method to get column 3. Maybe the fields are > separated by a tab? > > A regex may be possible, but you will have similar issues to using > split. > > I'm also assuming it's possible for there to be IP addresses that do not > start with 25. Are you looking to isolate those? > > It's not necessary to write out to a file first. You can get the output > from commands and work on it directly. > > Another approach would be to change the command you are running. I've > never heard of hamachi list before; does it have any commandline options > to display only IP addresses? > > -- > David Rock > david at graniteweb.com > _______________________________________________ > Tutor maillist - Tutor at python.org > To unsubscribe or change subscription options: > https://mail.python.org/mailman/listinfo/tutor From sebastian at fuentelibre.org Thu Oct 2 18:50:59 2014 From: sebastian at fuentelibre.org (Sebastian Silva) Date: Thu, 02 Oct 2014 11:50:59 -0500 Subject: [Tutor] printing all text that begins with "25" In-Reply-To: <20141002163336.GH3591@wdfs> References: <20141002163336.GH3591@wdfs> Message-ID: <1412268659.26956.2@smtp.gmail.com> El jue, 2 de oct 2014 a las 11:33 AM, David Rock <david at graniteweb.com> escribi?: > > A regex may be possible, but you will have similar issues to using > split. In my humble experience, a regex is the way to go: import re ip = re.findall( r'[0-9]+(?:\.[0-9]+){3}', s ) you will get a list of IP addresses and can filter from there which ones start with "25." -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/tutor/attachments/20141002/0679f524/attachment.html> From alan.gauld at btinternet.com Fri Oct 3 00:04:00 2014 From: alan.gauld at btinternet.com (Alan Gauld) Date: Thu, 02 Oct 2014 23:04:00 +0100 Subject: [Tutor] Iterating Lines in File and Export Results In-Reply-To: <CAFdOivu0HyNpuDRfmp7BWTP+u1yXu9wZNLx85bjuFcmwZ3pVBg@mail.gmail.com> References: <CAFdOivu0HyNpuDRfmp7BWTP+u1yXu9wZNLx85bjuFcmwZ3pVBg@mail.gmail.com> Message-ID: <m0ki4g$r9$1@ger.gmane.org> On 02/10/14 16:47, John Doe wrote: > def loop_extract(): > with open('words.txt', 'r') as f: > for lines in f: > #print lines (I confirmed that each line is successfully printed) > with open('export.txt', 'w') as outf: This opens and closes the file for each iteration of the inner loop. You need this outside the loop beside the other with statement. > outf.write(lines) HTH -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.flickr.com/photos/alangauldphotos From __peter__ at web.de Fri Oct 3 00:08:13 2014 From: __peter__ at web.de (Peter Otten) Date: Fri, 03 Oct 2014 00:08:13 +0200 Subject: [Tutor] Iterating Lines in File and Export Results References: <CAFdOivu0HyNpuDRfmp7BWTP+u1yXu9wZNLx85bjuFcmwZ3pVBg@mail.gmail.com> Message-ID: <m0kice$4l1$1@ger.gmane.org> John Doe wrote: > Hello List, > I am in need of your assistance. I have a text file with random words > in it. I want to write all the lines to a new file. Additionally, I am > using Python 2.7 on Ubuntu 12.04: > > Here is my code: > > def loop_extract(): > with open('words.txt', 'r') as f: > for lines in f: The name `lines` is misleading, you are reading one line at a time. > #print lines (I confirmed that each line is successfully > #printed) > with open('export.txt', 'w') as outf: > outf.write(lines) > #outf.write(lines) > #outf.write('{}\n'.format(lines)) > #outf.write('{}\n'.format(line for line in lines)) > > > For some reason, the second file only contains the last line from the > original file -- I have tried multiple variations (.read, .readlines, > .writelines, other examples preceded by comment from above and many > more) and tried to use the module, fileinput, but I still get the same > results. Every time the line > with open('export.txt', 'w') as outf: is executed the file "export.txt" is truncated: https://docs.python.org/dev/library/functions.html#open To avoid the loss of data open the file once, outside the loop: with open("words.txt") as infile, open("export.txt", "w") as outfile: for line in infile: outfile.write(line) > I do understand there is another way to copy the file over, but to > provide additional background information on my purpose -- I want to > read a file and save successful regex matches to a file; exporting > specific data. There doesn't appear to be anything wrong with my > expression as it prints the expected results without failure. I then > decided to just write the export function by itself in its basic form, > per the code above, which the same behavior occurred; That is a good approach! Reduce the code until only the source of the problem is left. > only copying the > last line. I've googled for hours and, unfortunately, at loss. I do that too, but not "for hours" ;) > I want to read a file and save successful regex matches to a file; > exporting specific data. An experienced user of Python might approach this scenario with a generator: def process_lines(infile): for line in infile: line = process(line) # your line processing if meets_condition(line): # your filter condition yield line with open("words.txt") as infile: with open("export.txt", "w") as outfile: outfile.writelines( process_lines(infile)) From alan.gauld at btinternet.com Fri Oct 3 00:06:06 2014 From: alan.gauld at btinternet.com (Alan Gauld) Date: Thu, 02 Oct 2014 23:06:06 +0100 Subject: [Tutor] printing all text that begins with "25" In-Reply-To: <CAKKCnfezr_eUvJ18bS4+DOeL7Q8-un8KdHqNGnHSnbKdgtKmsg@mail.gmail.com> References: <CAKKCnfezr_eUvJ18bS4+DOeL7Q8-un8KdHqNGnHSnbKdgtKmsg@mail.gmail.com> Message-ID: <m0ki8e$r9$2@ger.gmane.org> On 02/10/14 16:41, Bo Morris wrote: > of the following output > > 087-888-279 Pandora 25.x.x.xxx alias: not > set > > 096-779-867 AM1LaptopBD-PC 25.x.x.xxx alias: not set > > > 097-552-220 OWS-Desktop 1 25.0.0.0 alias: not set > > 099-213-641 DESKTOP 25.0.0.0 alias: not > set > > I am trying to write a python script that will run the above command and > only print out the IP's that begin with 25. How do I strip out all other > text except for the IP's that begin with "25?" Use split() to get the 'columns' in a list then use strip() to get rid of whitespace. HTH -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.flickr.com/photos/alangauldphotos From martin at linux-ip.net Fri Oct 3 01:01:50 2014 From: martin at linux-ip.net (Martin A. Brown) Date: Thu, 2 Oct 2014 16:01:50 -0700 Subject: [Tutor] printing all text that begins with "25" In-Reply-To: <CAKKCnfezr_eUvJ18bS4+DOeL7Q8-un8KdHqNGnHSnbKdgtKmsg@mail.gmail.com> References: <CAKKCnfezr_eUvJ18bS4+DOeL7Q8-un8KdHqNGnHSnbKdgtKmsg@mail.gmail.com> Message-ID: <alpine.LNX.2.00.1410021545270.12468@dagger.wonderfrog.net> Hi Bo, > I am trying to write a python script that will run the above > command and only print out the IP's that begin with 25. How do I > strip out all other text except for the IP's that begin with "25?" I liked the suggestion by John Doe earlier that this is a pretty good case for 'grep', but perhaps you want to do more than simply see the results on the terminal. So, you seem to want to be able to 'grep' for IPs that match a particular prefix, 25.0.0.0/8. Do you, perchance work for org-name: DINSA, Ministry of Defence [0] or are you using 25.0.0.0/8 in a private network. If the latter, are you sure you don't want to use one of the RFC 1918 networks? > Would it be best to send to a file first, then read the contents > of the file? Would I need to use regex? Others have addressed some of this. > I know how to run the above command in python. I also know how to > send the output to a file and read the file; however I cannot > figure out how to strip all text out except for the IPs that begin > with "25." Ok, so I can't help but observe that you are working with IP-oriented data. While you can perform tests like: ipstr.startswith('25') # -- yep, '250', '251', '253', '254', also or similar tests by writing a regular expression and using one of the heavier tools (re.findall, re.compile, re.match), I think that merely helps you locate the text that you think is the IP address. If you are asking is the IP within the 25.0.0.0/8 prefix, then you probably want to use the ipaddr (Python 2.x from PyPI) or ipaddress (Python 3.x stdlib) module to validate the IP and make sure that the IP is in a prefix of interest. I made one liberal change to the format of your data--I made it tab-separated. If it is not tab-separated, then you can see which line would probably need to have your regex line-splitter. The below, is more general than finding every IP that starts with '25.', because now you can "ipaddr-grep" for what you want. #! /usr/bin/python from __future__ import print_function import sys try: # -- Python2.x import ipaddr as ipaddress except ImportError: # -- Python3.x import ipaddress separator = '\t' def ipaddr_grep(prefix, fin): for line in fin: line = line.strip() if not line or line.startswith('#'): continue parts = line.strip().split(separator) # -- tab separated ip = ipaddress.IPv4Address(parts[2]) if ip in prefix: yield(line) def ipaddr_grep_main(prefix, fnames): prefix = ipaddress.IPv4Network(prefix) while fnames: fname = fnames.pop() with open(fname) as fin: for line in ipaddr_grep(prefix, fin): print(line) if __name__ == '__main__': ipaddr_grep_main(sys.argv[1], sys.argv[2:]) I happen to be the sort of person who always wants to point out the IP-related tools available in Python hence my reply to your post. Happy trails and good luck, -Martin [0] https://apps.db.ripe.net/search/query.html?searchtext=25.0.0.0/8&source=RIPE#resultsAnchor -- Martin A. Brown http://linux-ip.net/ From dantheman5457 at gmail.com Fri Oct 3 00:51:12 2014 From: dantheman5457 at gmail.com (John Doe) Date: Thu, 2 Oct 2014 18:51:12 -0400 Subject: [Tutor] Iterating Lines in File and Export Results In-Reply-To: <m0kice$4l1$1@ger.gmane.org> References: <CAFdOivu0HyNpuDRfmp7BWTP+u1yXu9wZNLx85bjuFcmwZ3pVBg@mail.gmail.com> <m0kice$4l1$1@ger.gmane.org> Message-ID: <CAFdOivs6OLRObhNkog78H-6_spFm_x00=4Nhgr1+=SD122JGJw@mail.gmail.com> Alan, Peter, et al: Thank you all very much! Staring at this problem for hours was driving me crazy and I am very appreciative for your guys' time in looking into my silly error -- I have thoroughly reviewed both the responses and it makes perfect sense (*sigh of relief*). On Thu, Oct 2, 2014 at 6:08 PM, Peter Otten <__peter__ at web.de> wrote: > John Doe wrote: > >> Hello List, >> I am in need of your assistance. I have a text file with random words >> in it. I want to write all the lines to a new file. Additionally, I am >> using Python 2.7 on Ubuntu 12.04: >> >> Here is my code: >> >> def loop_extract(): >> with open('words.txt', 'r') as f: >> for lines in f: > > The name `lines` is misleading, you are reading one line at a time. > >> #print lines (I confirmed that each line is successfully >> #printed) >> with open('export.txt', 'w') as outf: >> outf.write(lines) >> #outf.write(lines) >> #outf.write('{}\n'.format(lines)) >> #outf.write('{}\n'.format(line for line in lines)) >> >> >> For some reason, the second file only contains the last line from the >> original file -- I have tried multiple variations (.read, .readlines, >> .writelines, other examples preceded by comment from above and many >> more) and tried to use the module, fileinput, but I still get the same >> results. > > Every time the line > >> with open('export.txt', 'w') as outf: > > is executed the file "export.txt" is truncated: > > https://docs.python.org/dev/library/functions.html#open > > To avoid the loss of data open the file once, outside the loop: > > with open("words.txt") as infile, open("export.txt", "w") as outfile: > for line in infile: > outfile.write(line) > > >> I do understand there is another way to copy the file over, but to >> provide additional background information on my purpose -- I want to >> read a file and save successful regex matches to a file; exporting >> specific data. There doesn't appear to be anything wrong with my >> expression as it prints the expected results without failure. I then >> decided to just write the export function by itself in its basic form, >> per the code above, which the same behavior occurred; > > That is a good approach! Reduce the code until only the source of the > problem is left. > >> only copying the >> last line. I've googled for hours and, unfortunately, at loss. > > I do that too, but not "for hours" ;) > >> I want to read a file and save successful regex matches to a file; >> exporting specific data. > > An experienced user of Python might approach this scenario with a generator: > > def process_lines(infile): > for line in infile: > line = process(line) # your line processing > if meets_condition(line): # your filter condition > yield line > > with open("words.txt") as infile: > with open("export.txt", "w") as outfile: > outfile.writelines( > process_lines(infile)) > > > _______________________________________________ > Tutor maillist - Tutor at python.org > To unsubscribe or change subscription options: > https://mail.python.org/mailman/listinfo/tutor From steve at pearwood.info Fri Oct 3 13:41:26 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 3 Oct 2014 21:41:26 +1000 Subject: [Tutor] could somebody please explain... In-Reply-To: <003201cfdd96$d4a17e20$7de47a60$@us> References: <01eb01cfdd01$86034220$9209c660$@us> <20141001013814.GW19757@ando.pearwood.info> <003201cfdd96$d4a17e20$7de47a60$@us> Message-ID: <20141003114126.GC19757@ando.pearwood.info> On Wed, Oct 01, 2014 at 09:43:29AM -0700, Clayton Kirkwood wrote: > In an effort to learn and teach, I present a simple program which measures > the time it takes to the various add functions with the appending results: Well done for making the effort! Now I'm going to tell you all the things you've done wrong! Sorry. But seriously, I am very pleased to see you making the effort to develop this on your own, but *accurately* timing fast-running code on modern computers is very tricky. The problem is, when you run some code, it isn't the only program running! The operating system is running, and these days all computers are multi-tasking, which means that anything up to hundreds of other programs could be running at the same time. At any one instant, most of them will be idle, doing nothing, but there's no way to be sure. Furthermore, there are now complexities with CPU caches. Running a bit of code will be much slower the first time, since it is not in the CPU cache. If the code it too big, it won't fit in the cache. The end result is that when you time how long a piece of code takes to run, there will always be two components: - the actually time taken for your code to run; - random "noise" caused by CPU cache effects, other processes running, the operating system, your anti-virus suddenly starting a scan in the middle of the run, etc. The noise can be quite considerable, possibly a few seconds. Now obviously if your code took ten minutes to run, then a few seconds either way is no big deal. But imagine that your timing test says that it took 2 seconds. That could mean: - 0.001 seconds for your code, and 1.999 seconds worth of noise; - 1.999 seconds for your code, and 0.001 seconds worth of noise; - or anything in between. That measurement is clearly quite useless. Does this mean that timing Python code is impossible? No, not really, but you have to do it carefully. The best way is to use Python's "timeit" module, which is carefully crafted to be as accurate as possible. First I'll show some results with timeit, then come back for a second post where I explain what you can do to be (nearly) as accurate. I'm going to compare four different ways of adding two numbers: (1) Using the + operator (2) Using operator.add (3) Using operator.__add__ (4) Using a hand-written function, made with lambda Here's the plus operator: from the command shell, I tell Python to use the timeit module to time some code. I give it some setup code to initialise two variables, then I time adding them together: [steve at ando ~]$ python3.3 -m timeit -s "x = 1; y = 2" "x + y" 10000000 loops, best of 3: 0.0971 usec per loop [steve at ando ~]$ python3.3 -m timeit -s "x = 1; y = 2" "x + y" 10000000 loops, best of 3: 0.0963 usec per loop So timeit measures how long it takes to run "x + y" ten million times. It does that three times, and picks the fastest of the three. The fastest will have the least amount of noise. I ran it twice, and the two results are fairly close: 0.0971 microseconds, and 0.0963 microseconds. [steve at ando ~]$ python3.3 -m timeit -s "x = 1; y = 2" -s "import operator" "operator.add(x, y)" 1000000 loops, best of 3: 0.369 usec per loop [steve at ando ~]$ python3.3 -m timeit -s "x = 1; y = 2" -s "import operator" "operator.add(x, y)" 1000000 loops, best of 3: 0.317 usec per loop This time I use operator.add, and get a speed of about 0.3 microseconds. So operator.add is about three times slower than the + operator. [steve at ando ~]$ python3.3 -m timeit -s "x = 1; y = 2" -s "import operator" "operator.__add__(x, y)" 1000000 loops, best of 3: 0.296 usec per loop [steve at ando ~]$ python3.3 -m timeit -s "x = 1; y = 2" -s "import operator" "operator.__add__(x, y)" 1000000 loops, best of 3: 0.383 usec per loop This time I use operator.__add__, and get about the same result as operator.add. You can see the variability in the results: 0.296 to 0.383 microsecond, that's a variation of about 30%. [steve at ando ~]$ python3.3 -m timeit -s "x = 1; y = 2" -s "add = lambda a,b: a+b" "add(x, y)" 1000000 loops, best of 3: 0.296 usec per loop [steve at ando ~]$ python3.3 -m timeit -s "x = 1; y = 2" -s "add = lambda a,b: a+b" "add(x, y)" 1000000 loops, best of 3: 0.325 usec per loop Finally, I try it with a hand-made function using lambda, and I get about the same 0.3 microseconds again, with considerable variability. Of course, the results you get on your computer may be completely different. More to follow... -- Steven From steve at pearwood.info Fri Oct 3 15:21:15 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 3 Oct 2014 23:21:15 +1000 Subject: [Tutor] could somebody please explain... In-Reply-To: <003201cfdd96$d4a17e20$7de47a60$@us> References: <01eb01cfdd01$86034220$9209c660$@us> <20141001013814.GW19757@ando.pearwood.info> <003201cfdd96$d4a17e20$7de47a60$@us> Message-ID: <20141003132115.GD19757@ando.pearwood.info> On Wed, Oct 01, 2014 at 09:43:29AM -0700, Clayton Kirkwood wrote: > # program to test time and count options > > import datetime,operator, sys > from datetime import time, date, datetime > date = datetime.now() > dayofweek = date.strftime("%a, %b") > print("Today is", dayofweek, date.day, "at ", date.time()) > > start = 0 > count_max=int(input("give me a number")) > start_time = datetime.now() > > print( start_time ) > while start < count_max: > start=start + 1 > > end_time = datetime.now() > print( "s=s+1 time difference is:", (end_time - start_time) ) The first problem you have here is that you are not actually timing how long it takes to add "start + 1". You're actually timing eight things: - lookup the value of start; - lookup the value of count_max; - check whether the first is less than the second; - decide whether to loop, or exit the loop; - if we're still inside the loop, lookup start again; - add 1 to it; - store the result in start; - jump back to the top of the loop. So the results you get don't tell you much about the speed of start+1. Analogy: you want to know how long it takes you to drive to work in the morning. So you wake up, eat breakfast, brush your teeth, start the stopwatch, have a shower, get dressed, get in the car, drive to the gas station, fill up, buy a newspaper, and drive the rest of the way to work, and finally stop the stopwatch. The time you get is neither accurate as "driving time", nor "total time it takes to get to work" time. Ideally, we want to do as little extra work as possible inside the timing loop, so we can get a figure as close as possible to the time actually taken by + as we can. The second problem is that you are using datetime.now() as your clock. That's not a high-precision clock. It might be only be accurate to a second, or a millisecond. It certainly isn't accurate enough to measure a single addition: py> from datetime import datetime py> x = 1 py> t = datetime.now(); x + 1; datetime.now() - t 2 datetime.timedelta(0, 0, 85) This tells me that it supposedly took 85 microseconds to add two numbers, but as I showed before with timeit, the real figure is closer to 0.09 microseconds. That's a lot of noise! About 85000% noise! Unfortunately, it is tricky to know which clock to use. On Windows, time.clock() used to be the best one; on Linux, time.time() was the best. Starting in Python 3.3, there are a bunch more accurate clocks in the time module. But if you use the timeit module, it already picks the best clock for the job. But if in doubt, time.time() will normally be acceptable. https://docs.python.org/3/library/time.html https://docs.python.org/3/library/timeit.html -- Steven From crk at godblessthe.us Fri Oct 3 17:38:46 2014 From: crk at godblessthe.us (Clayton Kirkwood) Date: Fri, 3 Oct 2014 08:38:46 -0700 Subject: [Tutor] could somebody please explain... In-Reply-To: <20141003132115.GD19757@ando.pearwood.info> References: <01eb01cfdd01$86034220$9209c660$@us> <20141001013814.GW19757@ando.pearwood.info> <003201cfdd96$d4a17e20$7de47a60$@us> <20141003132115.GD19757@ando.pearwood.info> Message-ID: <033501cfdf20$1f8178c0$5e846a40$@us> Steven, I don't disagree with most of your analysis, I didn't know of other timing routines, and all of the superfluous stuff adds up. However, for a simple test, the route that I took was adequate I think. Yes I timed the whole wakeup to get to work, but the important element is that whatever I timed, was accurate between runs. And that is all that was import: to see the relative times.I also ran the complete program multiple times and found the test to be relatively consistent. I appreciate your notice of timeit(), I'll have to look into that, thanks. Thanks for taking the time to review and comment. Clayton !-----Original Message----- !From: Tutor [mailto:tutor-bounces+crk=godblessthe.us at python.org] On !Behalf Of Steven D'Aprano !Sent: Friday, October 03, 2014 6:21 AM !To: tutor at python.org !Subject: Re: [Tutor] could somebody please explain... ! !On Wed, Oct 01, 2014 at 09:43:29AM -0700, Clayton Kirkwood wrote: ! !> # program to test time and count options !> !> import datetime,operator, sys !> from datetime import time, date, datetime date = datetime.now() !> dayofweek = date.strftime("%a, %b") print("Today is", dayofweek, !> date.day, "at ", date.time()) !> !> start = 0 !> count_max=int(input("give me a number")) start_time = datetime.now() !> !> print( start_time ) !> while start < count_max: !> start=start + 1 !> !> end_time = datetime.now() !> print( "s=s+1 time difference is:", (end_time - start_time) ) ! ! !The first problem you have here is that you are not actually timing how !long it takes to add "start + 1". !You're actually timing eight things: ! !- lookup the value of start; !- lookup the value of count_max; !- check whether the first is less than the second; !- decide whether to loop, or exit the loop; !- if we're still inside the loop, lookup start again; !- add 1 to it; !- store the result in start; !- jump back to the top of the loop. ! ! !So the results you get don't tell you much about the speed of start+1. ! !Analogy: you want to know how long it takes you to drive to work in the !morning. So you wake up, eat breakfast, brush your teeth, start the !stopwatch, have a shower, get dressed, get in the car, drive to the gas !station, fill up, buy a newspaper, and drive the rest of the way to !work, and finally stop the stopwatch. The time you get is neither !accurate as "driving time", nor "total time it takes to get to work" !time. ! !Ideally, we want to do as little extra work as possible inside the !timing loop, so we can get a figure as close as possible to the time !actually taken by + as we can. ! !The second problem is that you are using datetime.now() as your clock. !That's not a high-precision clock. It might be only be accurate to a !second, or a millisecond. It certainly isn't accurate enough to measure !a single addition: ! !py> from datetime import datetime !py> x = 1 !py> t = datetime.now(); x + 1; datetime.now() - t !2 !datetime.timedelta(0, 0, 85) ! ! !This tells me that it supposedly took 85 microseconds to add two !numbers, but as I showed before with timeit, the real figure is closer !to 0.09 microseconds. That's a lot of noise! About 85000% noise! ! !Unfortunately, it is tricky to know which clock to use. On Windows, !time.clock() used to be the best one; on Linux, time.time() was the !best. Starting in Python 3.3, there are a bunch more accurate clocks in !the time module. But if you use the timeit module, it already picks the !best clock for the job. But if in doubt, time.time() will normally be !acceptable. ! !https://docs.python.org/3/library/time.html ! !https://docs.python.org/3/library/timeit.html ! ! ! !-- !Steven !_______________________________________________ !Tutor maillist - Tutor at python.org !To unsubscribe or change subscription options: !https://mail.python.org/mailman/listinfo/tutor From azzairob at gmail.com Fri Oct 3 23:27:31 2014 From: azzairob at gmail.com (Rob Ward) Date: Fri, 3 Oct 2014 14:27:31 -0700 Subject: [Tutor] pygame module Message-ID: <CAJ7YhN9R9jR3hozOmG8rgz9GOU+V0uaSBNQXPE9c9RNnoHREDQ@mail.gmail.com> i downloaded the 3.4 version of python but there is no matching binary file for pygame ive tried every 1.9.1 file and still cant import pygame would an older version of python work rob -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/tutor/attachments/20141003/c5dc83ff/attachment.html> From dyoo at hashcollision.org Sat Oct 4 03:04:36 2014 From: dyoo at hashcollision.org (Danny Yoo) Date: Fri, 3 Oct 2014 18:04:36 -0700 Subject: [Tutor] pygame module In-Reply-To: <CAJ7YhN9R9jR3hozOmG8rgz9GOU+V0uaSBNQXPE9c9RNnoHREDQ@mail.gmail.com> References: <CAJ7YhN9R9jR3hozOmG8rgz9GOU+V0uaSBNQXPE9c9RNnoHREDQ@mail.gmail.com> Message-ID: <CAGZAPF7PvZ8h=hvAurjJZD1ujLUCm5ofHgpTZRdt29BSW03=Hw@mail.gmail.com> On Fri, Oct 3, 2014 at 2:27 PM, Rob Ward <azzairob at gmail.com> wrote: > i downloaded the 3.4 version of python but there is no matching binary file > for pygame ive tried every 1.9.1 file and still cant import pygame would an > older version of python work You might have better results contacting the Pygame community for this question, as you're asking an installation question on a third-party library. http://pygame.org/wiki/info According to their FAQ, Pygame 1.9.2 should support Python 3: http://www.pygame.org/wiki/FrequentlyAskedQuestions#Does Pygame work with Python 3? From robertvstepp at gmail.com Sat Oct 4 02:46:02 2014 From: robertvstepp at gmail.com (boB Stepp) Date: Fri, 3 Oct 2014 19:46:02 -0500 Subject: [Tutor] pygame module In-Reply-To: <CAJ7YhN9R9jR3hozOmG8rgz9GOU+V0uaSBNQXPE9c9RNnoHREDQ@mail.gmail.com> References: <CAJ7YhN9R9jR3hozOmG8rgz9GOU+V0uaSBNQXPE9c9RNnoHREDQ@mail.gmail.com> Message-ID: <CANDiX9KPnGxOdYjWTWqYbM6=6fW=tT9epQ_Rt32Fp7hCQbi6xw@mail.gmail.com> On Fri, Oct 3, 2014 at 4:27 PM, Rob Ward <azzairob at gmail.com> wrote: > i downloaded the 3.4 version of python but there is no matching binary file > for pygame ive tried every 1.9.1 file and still cant import pygame would an > older version of python work > If you have windows try: http://www.lfd.uci.edu/~gohlke/pythonlibs/#pygame -- boB From spauldingfamily at twc.com Mon Oct 6 00:40:39 2014 From: spauldingfamily at twc.com (Mike Spaulding) Date: Sun, 5 Oct 2014 18:40:39 -0400 Subject: [Tutor] New at Python Message-ID: <007301cfe0ed$63d8d760$2b8a8620$@twc.com> Given that n refers to a positive int use a while loop to compute the sum of the cubes of the first n counting numbers, and associate this value with total . Use no <http://pearson.turingscraft.com/codelab/jsp/tutor.jsp?glIndex=1&lang=PYTHON 3> variables other than n , k , and total . Hi, I am really a rookie at Python and I am just learning some of the language and some of the functions etc. I am using Pearson's Programming lab and the accompanying online book. The biggest problem that I am having is trying to figure out how to start off on problems such as the one above. I have been out of school for over 40 years and I am struggling, needless to say. I know this problem will be laughably easy for most of you, but do you have any suggestions as to how I need to look at a problem and to "identify" what exactly it is asking for, and how I can pick out the elements that I need to start with? Any and all help will be greatly appreciated. Thanks -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/tutor/attachments/20141005/a533e49b/attachment.html> From amonroe at columbus.rr.com Mon Oct 6 02:10:13 2014 From: amonroe at columbus.rr.com (R. Alan Monroe) Date: Sun, 5 Oct 2014 20:10:13 -0400 Subject: [Tutor] New at Python In-Reply-To: <007301cfe0ed$63d8d760$2b8a8620$@twc.com> References: <007301cfe0ed$63d8d760$2b8a8620$@twc.com> Message-ID: <1474428057.20141005201013@columbus.rr.com> > while loop to compute > the sum of the cubes of the first n counting numbers > do you have any suggestions as to how I need to look at a problem > and to ?identify? what exactly it is asking for, Can you do it on paper? If I gave you "5" as a starting point, could you write down on your paper 1 cubed plus 2 cubed plus 3 cubed plus 4 cubed plus 5 cubed and arrive at a correct sum? Alan From alan.gauld at btinternet.com Mon Oct 6 02:14:23 2014 From: alan.gauld at btinternet.com (Alan Gauld) Date: Mon, 06 Oct 2014 01:14:23 +0100 Subject: [Tutor] New at Python In-Reply-To: <007301cfe0ed$63d8d760$2b8a8620$@twc.com> References: <007301cfe0ed$63d8d760$2b8a8620$@twc.com> Message-ID: <m0smsv$6pe$1@ger.gmane.org> On 05/10/14 23:40, Mike Spaulding wrote: > Given that n refers to a positive int use a while loop to compute the sum of > the cubes of the first n counting numbers, and associate this value with > total . Use no variables other than n , k , and total . > lab and the accompanying online book. The biggest problem that I am having > is trying to figure out how to start off on problems such as the one above. You need to break the problem into sub problems until you wind up with things you can do. For example in the above exercise: 1) can you associate a variable called n with an integer? 2) can you use a while loop to iterate a number of times? 3) can you calculate the cube of a number? 4) can you total two numbers and store the result? 5) can you display the final total? Assuming you can do each of those bits you need to organize them into a structure to form your program. You need to store a value in n and use a while loop to loop that many times. You need a counter to store the number of loops. For each time through the loop you need to calculate the cube of the counter. You then need to add that result to the running total. When the loop ends you need to report the total. Its the same with most beginner exercises, you divide it into hunks and solve each chunk. Then assemble the bits to solve the whole. Once the problems get a lot bigger you need to apply some more advanced techniques which we call analysis and design. But for beginner stuff the divide and conquer approach works just fine. Does that help? -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.flickr.com/photos/alangauldphotos From dyoo at hashcollision.org Mon Oct 6 02:15:27 2014 From: dyoo at hashcollision.org (Danny Yoo) Date: Sun, 5 Oct 2014 17:15:27 -0700 Subject: [Tutor] New at Python In-Reply-To: <007301cfe0ed$63d8d760$2b8a8620$@twc.com> References: <007301cfe0ed$63d8d760$2b8a8620$@twc.com> Message-ID: <CAGZAPF4uCg5QGG1s53Oc0M-_GjnYri26CabtCNN6ui2nV9Bu0g@mail.gmail.com> On Sun, Oct 5, 2014 at 3:40 PM, Mike Spaulding <spauldingfamily at twc.com> wrote: > Given that n refers to a positive int use a while loop to compute the sum of > the cubes of the first n counting numbers, and associate this value with > total . Use no variables other than n , k , and total . > > Hi, I am really a rookie at Python and I am just learning some of the > language and some of the functions etc. I am using Pearson?s Programming > lab and the accompanying online book. The biggest problem that I am having > is trying to figure out how to start off on problems such as the one above. Hi Mike, If you can point out previous exercises that you've done, that may help us point out similarities between the things you've done before, and this particular problem. (The material you're pointing out is not freely available, so I can not search for it not see what problems preceded the one you're working on now.) Generally, a problem solving strategy such as: http://en.wikipedia.org/wiki/How_to_Solve_It should be helpful in breaking this down. R. Alan Monroe suggests that you try it "by hand" first. This is a good approach to start: you need to _understand_ what's being asked before trying to instruct a computer by hand. Do you understand the problem statement? What is the "input", and what is the expected "output"? Let's get that cleared up before talking about the code. Good luck! From steve at pearwood.info Mon Oct 6 04:08:48 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Mon, 6 Oct 2014 13:08:48 +1100 Subject: [Tutor] New at Python In-Reply-To: <007301cfe0ed$63d8d760$2b8a8620$@twc.com> References: <007301cfe0ed$63d8d760$2b8a8620$@twc.com> Message-ID: <20141006020848.GM19757@ando.pearwood.info> On Sun, Oct 05, 2014 at 06:40:39PM -0400, Mike Spaulding wrote: > Given that n refers to a positive int use a while loop to compute the sum of > the cubes of the first n counting numbers, and associate this value with > total . Use no > <http://pearson.turingscraft.com/codelab/jsp/tutor.jsp?glIndex=1&lang=PYTHON > 3> variables other than n , k , and total . Here is the thought process I would use to convert this into code. Actually writing code happens quite late in the process, first I try to understand the problem, and then and only then do I write code. We want to "compute the sum of ...", associating that value with total. So write it out in English (or whatever your native language is): total = sum of ... What do the ... represent? The cubes of the first n counting numbers. Do you remember the definition of "counting numbers"? If not, google for it. Understanding the problem, and the language used in the problem, is critical to solving the problem. https://duckduckgo.com/html/?q=counting+numbers The counting numbers are simply 1, 2, 3, 4, 5, ... which goes on forever, but fortunately for this problem we only want "the first n" of them. If you're familiar with mathematics, that's obvious, but what if you're not? Let's reason inductively. If we wanted the first *three* numbers, we would have: 1, 2, 3. If we wanted the first *four* numbers, we would have: 1, 2, 3, 4. If we wanted the first *ten* numbers, we would have: 1, 2, 3, 4, 5, 6, 7, 8, 9, 10. I'm seeing a pattern here. So if you use a pronumeral "n" to represent the, er, number of numbers we want, we'll have something like this: 1, 2, 3, 4, ... n. and we want the *cubes* of those numbers: 1**3, 2**3, 3**3, 4**3, ... n**3. and they go into our sum: total = sum of 1**3, 2**3, 3**3, 4**3, ... n**3 We're told to solve this with a while loop, using variables n, k and total. Let's ignore the part about the while loop for now, and concentrate on the variables. n and total are obviously the input and the output: n is the number we're working on, total is the number we calculate. What's k? That gives us a strong hint that we need a third variable. What could it be? Since we can only add two numbers at a time (a + b) and we know that total is going to be one of them (total + ???) and n is *not* going to be the other one, that gives us a clue that k will be the other one: k = 1 total = total + k**3 k = 2 total = total + k**3 etc. Let's start with a simpler problem: ignore the requirement to cube the numbers, and let's just sum the natural numbers themselves: total = sum of 1, 2, 3, 4, ... n We can write that as: total = 0 # We start with nothing. total = total + 1 # Add the first number. total = total + 2 # Add the second number. total = total + 3 # Add the third number. total = total + 4 # I'm starting to see a pattern here. and so on. Whenever you see repeated code like that, you should think of a loop. A for loop is actually the right solution here, but the question insists you use a while loop, so let's start with a little bit of almost-but-not-quite Python code. We use k as the loop variable, it will start at the first number we want to add, and increase by one each time through the loop: total = 0 k = 1 # The number we're going to add to the total, starting at 1. while there are still numbers to be added: total = total + k k = k + 1 # Go on to the next number to be added. There's a shorter way to add an increment to a variable: total = 0 k = 1 while there are still numbers to be added: total += k k += 1 Obviously "while there are still numbers to be added" is not Python code. In Python, we need "while condition". What is the condition that decides whether there are still numbers to be added or not? Let's take a concrete example: n = 4. We're going to loop with: k = 1 is used. k = 2 is used. k = 3 is used. k = 4 is used. <-- n = 4 k = 5 is NOT used. So our condition is that k is less than or equal to n. So now we can insert that condition into the Python code: total = 0 k = 1 while k <= n: total += k k += 1 Don't forget that we actually want to sum the cubes! total = 0 k = 1 while k <= n: total += k**3 k += 1 And we're done here and can move on to the next exercise. Or, we can continue on this one and do it *right*. Earlier, I said that a for loop was the right solution for this, rather than a while loop. Using a for loop simplifies the management of the k variable: total = 0 for k in (1, 2, 3, 4, ..., n) # not quite legal Python! total = total + k**3 To get the values (1, 2, 3, 4, ..., n) we use the built-in Python function "range", which returns a sequence of numbers, given an optional starting value, a stopping value, and an optional stepsize or stride: range(6) -> 0, 1, 2, 3, 4, 5 range(1, 6) -> 1, 2, 3, 4, 5 range(1, 6, 2) -> 1, 3, 5 So in this case, we want to start at 1, and end with n *included*, so our stopping value is n+1, and the step is just the default 1: total = 0 for k in range(1, n+1): total = total + k**3 That can be simplified using a generator expression and the built-in sum() function: total = sum(k**3 for k in range(1, n+1)) only you probably haven't learned about them yet. But you can probably see the similarity between the explicit for-loop and the generator expression: for k in range(1, n+1): <blah blah blah> k**3 vs. <blah blah blah>(k**3 for k in range(1, n+1)) The order is slightly different, but I hope you can see the similarity. Can we improve this solution? Here it is again: total = sum(k**3 for k in range(1, n+1)) Yes, we can improve it, thanks to mathematics! Can we find a simple equation that gives us the sum of the first n cubes? Googling for "sum of first n cubes" gets me to here: https://en.wikipedia.org/wiki/Squared_triangular_number which simplifies my code to: total = sum(k for k in range(1, n+1))**2 I can simplify that even more, since there is a marvelous equation that gives the sum of the first n counting numbers. These two pages will give you a good feel for how to think like a mathematician and find patterns, which is an excellent skill for programmers too: http://www.comfsm.fm/~dleeling/math/cogapp/sumofn.html http://proofsfromthebook.com/2013/01/29/first-n-positive-integers/ Using that formula gives us: total = (n*(n+1)//2)**2 # sum of the first n cubes. which requires no other variables, no loops, and only four maths operations: one each addition, multiplication, division and power. For large values of n it will be *much* faster than summing them by hand. The only downside is that it isn't *obvious* that it is the sum of the first n cubes, but that's why # comments are available. -- Steven From crk at godblessthe.us Mon Oct 6 04:19:36 2014 From: crk at godblessthe.us (Clayton Kirkwood) Date: Sun, 5 Oct 2014 19:19:36 -0700 Subject: [Tutor] grave confusion Message-ID: <04db01cfe10b$fb217f00$f1647d00$@us> Here's my problem; my code snippet reads a file(presumably an _io.readline, I'll question this later), with the file.readline(). The output shows individual characters being read and printed out followed by the "here" being printed. Also, see below. Source data file: <!-- saved from url=(0040)https://finance.yahoo.com/portfolio/pf_2 --> html class="yui3-js-enabled" id="yui_3_10_3_4_1412203661632_7637"><div id="yui3-css-stamp" style="position: absolute !important; visibility: hidden !important" class=""></div><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8"> Code snippet: import re, os for line_in in file.readline(): print( line_in ) print("here") continue Output: < here ! here - here - here here s here a here v here e Why the individual characters and not the full line? Also, it is very unclear to me which readline I am using. I have imported os. I am using pycharm IDE and supposedly it will show me where readline is coming from by right clicking and choosing find usages. However, it brings up three readlines which are place holders. It doesn't bring up the os.readline. I am getting really tired of trying to figure out which function is being used! So, what is happening that I am missing? TIA, Clayton -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/tutor/attachments/20141005/0bfdffda/attachment.html> From alan.gauld at btinternet.com Mon Oct 6 12:48:19 2014 From: alan.gauld at btinternet.com (Alan Gauld) Date: Mon, 06 Oct 2014 11:48:19 +0100 Subject: [Tutor] grave confusion In-Reply-To: <04db01cfe10b$fb217f00$f1647d00$@us> References: <04db01cfe10b$fb217f00$f1647d00$@us> Message-ID: <m0ts1k$6gv$1@ger.gmane.org> On 06/10/14 03:19, Clayton Kirkwood wrote: > Here's my problem; my code snippet reads a file(presumably an _io.readline, > I'll question this later), with the file.readline(). Nope, it reads a line from a file. That's quite a different thing. > The output shows > individual characters being read and printed out followed by the "here" > being printed. Also, see below. Because a for loop iterates over a string character by character And because the print 'here' is inside the loop. I'm not sure if thats deliberate or not. > for line_in in file.readline(): > print( line_in ) > print("here") > continue You don't need the continue since the loop will continue from the end of the block automatically. Thats what loops do. > Why the individual characters and not the full line? Because you are iterating over the line which is a collection of characters. So the for loop breaks it down to each character. > Also, it is very unclear to me which readline I am using. We can't be certain since you only show a snippet which does not include you opening the file. But based on the output you are using a file readline and in Python 3 a text file is an instance of class _io.TextIOWrapper But you don't need to know that, its just a "file-like object". > I have imported os. I am using pycharm IDE and supposedly it > will show me where readline is coming from ... it brings up three > readlines which are place holders. It doesn't bring up the os.readline. I don't think there is an os.readline. You don't need the IDE to tell you where things are coming from just look at your code. What is file? Where does it come from? Thats the readline() you are using. If you type help(file) it will give you the documentation of file including its superclasses if any. > getting really tired of trying to figure out which function is being used! Mostly you don't need to. Use them. If in doubt use help(). In this case help(file.readline) > So, what is happening that I am missing? Mostly the fact that readline() reads a line() not the entire file. That would be readlines() But nowadays you rarely need readlines(() either. You should just iterate over the file. So your code should be: for line in file: print(line) print('help') -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.flickr.com/photos/alangauldphotos From alan.gauld at btinternet.com Mon Oct 6 13:23:27 2014 From: alan.gauld at btinternet.com (Alan Gauld) Date: Mon, 06 Oct 2014 12:23:27 +0100 Subject: [Tutor] grave confusion In-Reply-To: <m0ts1k$6gv$1@ger.gmane.org> References: <04db01cfe10b$fb217f00$f1647d00$@us> <m0ts1k$6gv$1@ger.gmane.org> Message-ID: <m0tu3f$vu1$1@ger.gmane.org> On 06/10/14 11:48, Alan Gauld wrote: > Mostly the fact that readline() reads a line() ... Oops, Got a bit over enamoured with the parentheses there. Should just be plain 'line' of course. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.flickr.com/photos/alangauldphotos From alan.gauld at btinternet.com Mon Oct 6 18:43:41 2014 From: alan.gauld at btinternet.com (Alan Gauld) Date: Mon, 06 Oct 2014 17:43:41 +0100 Subject: [Tutor] Sqliute Row row factory Message-ID: <m0ugrt$pv8$1@ger.gmane.org> I just discovered the SQLite Row row_factory which as the docs say: ----- If returning a tuple doesn?t suffice and you want name-based access to columns, you should consider setting row_factory to the highly-optimized sqlite3.Row type. Row provides both index-based and case-insensitive name-based access to columns with almost no memory overhead. ----- Now my question is: Does this exist in the DBAPI or is it SQLite specific? In other words, if I use this feature will it screw me up if I try to convert from SQLite to Oracle or Postgres later? Anyone know? -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.flickr.com/photos/alangauldphotos From dyoo at hashcollision.org Mon Oct 6 19:46:53 2014 From: dyoo at hashcollision.org (Danny Yoo) Date: Mon, 6 Oct 2014 10:46:53 -0700 Subject: [Tutor] grave confusion In-Reply-To: <04db01cfe10b$fb217f00$f1647d00$@us> References: <04db01cfe10b$fb217f00$f1647d00$@us> Message-ID: <CAGZAPF4iqg6h+xJOdf8W4k99+XgcTK4MKTTLpF8qk7sQLPQNUA@mail.gmail.com> Alan has pointed out that your loop here: for line_in in file.readline(): ... has a much different meaning that the one you intend. It means: "for every character in the first line of the file: ..." The reason is because "file.readline()" returns a line of your file as a string. A string is a sequence of characters. Loops work on sequences of things, so in the loop above, it will walk over the sequence of characters. That being said, if you're trying to parse HTML with regular expressions, you probably want to reconsider. Instead, you might want to look into a dedicated parser for that task such as Beautiful Soup. http://www.crummy.com/software/BeautifulSoup/ The problem with a regular expressions approach is that it's easy to code up a fragile, broken solution. See: http://stackoverflow.com/questions/1732348/regex-match-open-tags-except-xhtml-self-contained-tags From dyoo at hashcollision.org Mon Oct 6 20:25:41 2014 From: dyoo at hashcollision.org (Danny Yoo) Date: Mon, 6 Oct 2014 11:25:41 -0700 Subject: [Tutor] Fwd: grave confusion In-Reply-To: <CAGZAPF6-_HJ9Neq8S=XA_6L5sUMnjyo9W-_Eyx2hfHGEqVSYjQ@mail.gmail.com> References: <04db01cfe10b$fb217f00$f1647d00$@us> <CAGZAPF4iqg6h+xJOdf8W4k99+XgcTK4MKTTLpF8qk7sQLPQNUA@mail.gmail.com> <058601cfe191$5ae52d30$10af8790$@us> <CAGZAPF6-_HJ9Neq8S=XA_6L5sUMnjyo9W-_Eyx2hfHGEqVSYjQ@mail.gmail.com> Message-ID: <CAGZAPF6sjVC2UKoz=8h10TMG4=0QtqiBq2rUfDkXTo2KFB1BfA@mail.gmail.com> ---------- Forwarded message ---------- From: Danny Yoo <dyoo at hashcollision.org> Date: Mon, Oct 6, 2014 at 11:23 AM Subject: Re: [Tutor] grave confusion To: Clayton Kirkwood <crk at godblessthe.us> > Well the guide certainly doesn't suggest that the read is one character at a time, it implies one line at a time. However, it's hard to argue against the one character because that is what the output is looking at. I thought it would read one line in per iteration. Why didn't they call it readchar()? I think you might still be confused. Here, let's look at it again from a different angle. #################### for line in file.readline(): ... #################### I'm going to make a small change to this: #################### seq = file.readline() for line in seq: ... #################### I've introduced a temporary variable. This should preserve the rough meaning by just giving a name to the thing we're walking across. One more change: #################### seq = file.readline() for thing in seq: ... #################### Again, meaning preserving, if we change every instance of "line" in "..." with "thing". But "thing" is a poor name for this as well. What's its type? If seq is a string, then thing has to be a character. Let's change the code one more time. ###################### seq = file.readline() for ch in seq: ... ####################### Contrast this with: ####################### for line in file.readlines(): ... ####################### This has a *totally* different meaning. The "delta" is a single character in terms of the physical source code, but in terms of what the program means, high impact. From lists at mostrom.pp.se Tue Oct 7 00:08:03 2014 From: lists at mostrom.pp.se (=?UTF-8?Q?Jan_Erik_Mostr=C3=B6m?=) Date: Tue, 7 Oct 2014 00:08:03 +0200 Subject: [Tutor] Scaling photos Message-ID: <CALGRW_yxT6n1z90vjc4Db9CNRr_x6U37m1fuksheubgCkApgqQ@mail.gmail.com> I want to write a small script that scales photos. I want the scaled photos to keep all meta data available in the originals. I also want to keep this python only and easily installed on linux and OS X machines. What library would you people recommend for doing this? - jem From dyoo at hashcollision.org Tue Oct 7 00:16:23 2014 From: dyoo at hashcollision.org (Danny Yoo) Date: Mon, 6 Oct 2014 15:16:23 -0700 Subject: [Tutor] Scaling photos In-Reply-To: <CALGRW_yxT6n1z90vjc4Db9CNRr_x6U37m1fuksheubgCkApgqQ@mail.gmail.com> References: <CALGRW_yxT6n1z90vjc4Db9CNRr_x6U37m1fuksheubgCkApgqQ@mail.gmail.com> Message-ID: <CAGZAPF5nSmwJ1dhvNMS4nOQeZ1Qy1gv__jpm=ngDgY5TuVmCjQ@mail.gmail.com> On Mon, Oct 6, 2014 at 3:08 PM, Jan Erik Mostr?m <lists at mostrom.pp.se> wrote: > I want to write a small script that scales photos. I want the scaled > photos to keep all meta data available in the originals. I also want > to keep this python only and easily installed on linux and OS X > machines. > > What library would you people recommend for doing this? Hi Jan, Python Imaging Library (PIL) is the first thing that comes to mind. http://www.pythonware.com/products/pil/ Good luck! From alan.gauld at btinternet.com Tue Oct 7 03:10:57 2014 From: alan.gauld at btinternet.com (Alan Gauld) Date: Tue, 07 Oct 2014 02:10:57 +0100 Subject: [Tutor] Scaling photos In-Reply-To: <CALGRW_yxT6n1z90vjc4Db9CNRr_x6U37m1fuksheubgCkApgqQ@mail.gmail.com> References: <CALGRW_yxT6n1z90vjc4Db9CNRr_x6U37m1fuksheubgCkApgqQ@mail.gmail.com> Message-ID: <m0vej1$v0o$1@ger.gmane.org> On 06/10/14 23:08, Jan Erik Mostr?m wrote: > I want to write a small script that scales photos. I want the scaled > photos to keep all meta data available in the originals. I also want > to keep this python only and easily installed on linux and OS X > machines. > > What library would you people recommend for doing this? Pillow Its the Python 3 version of PIL but works with 2.7 too I think. Most PIL tutorials will work for Pillow too. Also consider ImageMagick. Its usually available in the package system for Linux and can be downloaded for Macs too. It has a Python library. HTH -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.flickr.com/photos/alangauldphotos From crk at godblessthe.us Tue Oct 7 01:55:02 2014 From: crk at godblessthe.us (Clayton Kirkwood) Date: Mon, 6 Oct 2014 16:55:02 -0700 Subject: [Tutor] search/match file position q Message-ID: <05e101cfe1c0$f2adf190$d809d4b0$@us> Howdy I haven't been able to find an definitive answer. I am looking through a file(stream:<), for several matching strings. Match definitively starts at the beginning of the stream. As I search, or match, do I start over at the beginning of the stream for each match or do I start at the end of the last search/match? Apparently the file object knows where it is in a stream, but it is unclear whether line_location = readline() keeps track of that location. I see advantages to both protocols, but overall, I would prefer that the next search starts at the end of the last start TIA, Clayton -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/tutor/attachments/20141006/6999b8be/attachment.html> From pughpl at me.com Tue Oct 7 00:42:34 2014 From: pughpl at me.com (Phillip Pugh) Date: Mon, 06 Oct 2014 17:42:34 -0500 Subject: [Tutor] Suggestions Please Message-ID: <B44E0FEB-52A6-4774-94BD-A769ACF3ED9A@me.com> I am trying to decide if Python is the right toolset for me. I do a lot of data analytics. Over the years I have used a lot of SQL and VBA, but the data sources are getting bigger. I am thinking Python may be what I need to use, but I am in the early stages of getting to know Python. Can you point me to a really good, intuitive resource for getting up to speed with data. Something that would help with the following. I have one text file that is 500,000 + records.. I need to read the file, move "structured" data around and then write it to a new file. The txt file has several data elements and is 300 characters per line. I am only interested in the first two fields. The first data element is 19 characters. The second data element is 6 characters. I want to rearrange the data by moving the 6 characters data in front of the 19 characters data and then write the 25 character data to a new file. I have spent some time digging for the correct resource, However being new to Python and the syntax for the language makes it slow going. I would like to see if I can speed up the learning curve. Hopefully can get some results quickly then I will know how deep I want to go into Python. Thanks in advance Phillip From davea at davea.name Tue Oct 7 03:56:49 2014 From: davea at davea.name (Dave Angel) Date: Mon, 6 Oct 2014 21:56:49 -0400 (EDT) Subject: [Tutor] search/match file position q References: <05e101cfe1c0$f2adf190$d809d4b0$@us> Message-ID: <m0vh3f$kpi$1@ger.gmane.org> "Clayton Kirkwood" <crk at godblessthe.us> Wrote in message: > > haven?t been able to find an definitive answer You should make your question clearer. Are you asking what your code does, or what you should like it to do? Either way, you should start with some sample code, and refer to that in your questions. As it is, we can't tell whether you're searching a line at a time, whether you're using regex, index, or what. And please use text message, not html. -- DaveA From davea at davea.name Tue Oct 7 04:03:12 2014 From: davea at davea.name (Dave Angel) Date: Mon, 6 Oct 2014 22:03:12 -0400 (EDT) Subject: [Tutor] Suggestions Please References: <B44E0FEB-52A6-4774-94BD-A769ACF3ED9A@me.com> Message-ID: <m0vhff$tuj$1@ger.gmane.org> Phillip Pugh <pughpl at me.com> Wrote in message: > I am trying to decide if Python is the right toolset for me. I do a lot of data analytics. Over the years I have used a lot of SQL and VBA, but the data sources are getting bigger. I am thinking Python may be what I need to use, but I am in the early stages of getting to know Python. Can you point me to a really good, intuitive resource for getting up to speed with data. Something that would help with the following. > > I have one text file that is 500,000 + records.. I need to read the file, move "structured" data around and then write it to a new file. The txt file has several data elements and is 300 characters per line. I am only interested in the first two fields. The first data element is 19 characters. The second data element is 6 characters. I want to rearrange the data by moving the 6 characters data in front of the 19 characters data and then write the 25 character data to a new file. > > I have spent some time digging for the correct resource, However being new to Python and the syntax for the language makes it slow going. I would like to see if I can speed up the learning curve. Hopefully can get some results quickly then I will know how deep I want to go into Python. > Open the two files, using the 'with' statement. Using a for loop, iterate over the input file. For each line, use substring operators to fetch each field. Then write second + first. Maybe 6 lines. If you can get most of this, show it to us, and tell us where you're stuck. -- DaveA From martin at linux-ip.net Tue Oct 7 06:17:15 2014 From: martin at linux-ip.net (Martin A. Brown) Date: Mon, 6 Oct 2014 21:17:15 -0700 Subject: [Tutor] Suggestions Please In-Reply-To: <B44E0FEB-52A6-4774-94BD-A769ACF3ED9A@me.com> References: <B44E0FEB-52A6-4774-94BD-A769ACF3ED9A@me.com> Message-ID: <alpine.LNX.2.00.1410061956320.1349@dagger.wonderfrog.net> Greetings Phillip, > I am trying to decide if Python is the right toolset for me. It might be. That depends on you and also the environment in which you operate. > I do a lot of data analytics. Over the years I have used a lot of > SQL and VBA, but the data sources are getting bigger. Data sets have a very annoying habit of growing. > I am thinking Python may be what I need to use, but I am in the > early stages of getting to know Python. Can you point me to a > really good, intuitive resource for getting up to speed with data. > Something that would help with the following. Ooof. So, the killer in your question is the word 'intuitive'. Intuitive for whom? That depends entirely on you. There are a variety of tutorials, and I point out one below, but I do not know if it will be intuitive for you. Something that is intuitive for one person is opaque to the next. > I have one text file that is 500,000 + records.. There's always somebody who has dealt with bigger files. 500k records (at only 300 chars per line)? I'd read that straight into memory and do something with it. Given recent CPUs and amounts of available RAM, the biggest cost I see here is disk (and possibly algorithm). > I need to read the file, move "structured" data around and then > write it to a new file. The txt file has several data elements and > is 300 characters per line. I am only interested in the first two > fields. The first data element is 19 characters. The second data > element is 6 characters. I want to rearrange the data by moving > the 6 characters data in front of the 19 characters data and then > write the 25 character data to a new file. Dave Angel gave you some suggestions on how to do start. I'll make an attempt at translating his English into a Python block for you. Specifically, in Python, he suggested something like this: with open('yourfile.txt') as f: for line in f: first, second = line[:19], line[19:19+6] print second, first # Python-2.x #print(second, first) # Python-3.x Try the above. Does that do what you expected? If not, then have a look at substring operations that you can do once you have open()'d up a file and have read a line of data into a variable, "line" above, which is of type string: https://docs.python.org/2/library/stdtypes.html#string-methods If you can control the serialized format of data on disk, that might help open up a much richer set of tools for you. Plain text has the benefit and liability that it is amazingly flexible. If you are accustomed to performing data analytics with SQL and VBA, then here are some tools to examine. For people accustomed to working with data analytics in R, the Python pandas toolkit is a great fit: http://pandas.pydata.org/pandas-docs/stable/tutorials.html http://pandas.pydata.org/ This sounds much more like strict text-handling than like data analytics, though. Some of us may be able to help you more if you have a specific known-format you deal with regularly. For example, Python has modules for handling JSON and CSV (or TSV) out of the box: https://docs.python.org/2/library/json.html https://docs.python.org/2/library/csv.html Given that many SQL implementations (e.g. MySQL, Postgres, Oracle, SQLite) can produce outputs in CSV format, you may find generating exported data from your SQL engine of choice and then importing using the csv library is easier than parsing a fixed format. Why did you quote the word "structured"? It almost seems that you do not like your (peculiar) fixed-width field format. If you have a follow-up question, please post a small relevant snippet of the Python, the data and explain what you expected. Anyway, good luck--I have found Python is a fantastic tool, readable and it grew with my sophistication. I still reach for SQL for many reasons, but I like the flexibility, richness and tools that Python offers me. -Martin P.S. The Python online documentation is pretty good.....though the language changed very little between Python-2.x and Python-3.x, there are a few stumbling blocks, so run 'python -V' to make sure you are reading the correct documentation: https://docs.python.org/2/ # -- Python-2.7.x https://docs.python.org/3/ # -- Python-3.4.x -- Martin A. Brown http://linux-ip.net/ From crk at godblessthe.us Tue Oct 7 07:58:23 2014 From: crk at godblessthe.us (Clayton Kirkwood) Date: Mon, 6 Oct 2014 22:58:23 -0700 Subject: [Tutor] search/match file position q In-Reply-To: <m0vh3f$kpi$1@ger.gmane.org> References: <05e101cfe1c0$f2adf190$d809d4b0$@us> <m0vh3f$kpi$1@ger.gmane.org> Message-ID: <060a01cfe1f3$b52ea7f0$1f8bf7d0$@us> I was trying to keep it generic. Wrapped data file: <tr data-row-symbol="SWKS"><td class="col-symbol txt"><span class="wrapper " data-model="name:DatumModel;id:null;" data-tmpl=""><a data-ylk="cat:portfolio;cpos:1" href="http://finance.yahoo.com/q?s=SWKS" data-rapid_p="18">SWKS</a></span></td><td class="col-fiftytwo_week_low cell-raw:23.270000"><span class="wrapper " data-model="name:DatumModel;id:SWKS:qsi:wk52:low;" data-tmpl="change:yfin.datum">23.27</span></td><td class="col-prev_close cell-raw:58.049999"><span class="wrapper " data-model="name:DatumMo import re, os line_in = file.readline() # read in humongous html line stock = re.search('\s*<tr data-row-symbol="([A-Z]+)">', line_in) #scan to SWKS"> in data #line, stock should be SWKS low_52 = re.search('.+wk52:low.+([\d\.]+)<', line_in) #want to pick up from #SWKS">, low_52 should be 23.27 I am trying to figure out if each re.match starts scanning at the beginning of the same line over and over or does each scan start at the end of the last match. It appears to start over?? This is stock: <_sre.SRE_Match object; span=(0, 47), match=' <tr data-row-symbol="SWKS">'> This is low_52: <_sre.SRE_Match object; span=(0, 502875), match=' <tr data-row-symbol="SWKS"><t> If necessary, how do I pick up and move forward to the point right after the previous match? File.tell() and file.__sizeof__(), don't seem to play a useful role. TIA, Clayton From alan.gauld at btinternet.com Tue Oct 7 11:39:08 2014 From: alan.gauld at btinternet.com (Alan Gauld) Date: Tue, 07 Oct 2014 10:39:08 +0100 Subject: [Tutor] Suggestions Please In-Reply-To: <B44E0FEB-52A6-4774-94BD-A769ACF3ED9A@me.com> References: <B44E0FEB-52A6-4774-94BD-A769ACF3ED9A@me.com> Message-ID: <m10cbs$eem$1@ger.gmane.org> On 06/10/14 23:42, Phillip Pugh wrote: > I am trying to decide if Python is the right toolset for me. > I do a lot of data analytics. It can almost certainly do what you want but there may be other tools that do it better. However, data analytics is quite vague. It depends on what kind of data and what kind of analysis. > Can you point me to a really good, intuitive resource intuitive depends on the student. But we also need to know what kind of data. Is it stored in flat files? in a SQL database(which one?) In a NoSQL database(which one?) Python can handle all of those but the tutorials involved will all be different. If you want a general introduction with some SQL database specifics you can try my tutorial(see sig). Whether you find it intuitive is another matter. > I have one text file that is 500,000 + records.. Thats not very big in modern computing terms. You could probably just read that straight into memory. > I need to read the file, What kind of file? A database file such as Foxpro? or Access? or a CSV export? Or something else? > move "structured" data around and then write it to a new file. What is structured about it? Fixed column width? Fixed relative position? Binary format? > The txt file has several data elements and is > 300 characters per line. > I am only interested in the first two fields. > The first data element is 19 characters. > The second data element is 6 characters. There are two ways in Python to extract 'columns' from a file. If you know the separators you can use either the csv module(best) or string.split(<sep list>) to create a list of fields. If its a fixed length record (with potentially no seperator) you can use string slicing. In your case that would be field1 = string[:19]; field2 = string[19:25] > I want to rearrange the data by moving the 6 characters data > in front of the 19 characters data Do you need a separator? > and then write the 25 character data to a new file. the reading and writing of the files is straightforward, any tutorial will show you that. > I have spent some time digging for the correct resource, > However being new to Python and the syntax for the language > makes it slow going. I would like to see if I can speed up > the learning curve. So far it sounds like you don't need any of the high powered data analysis tools like R or Pandas, you are just doing basic data extraction and manipulation. For that standard Python should be fine and most tutorials include all you need. If you look at mine the most relevant topics from the contents are: The raw materials - variables & data types Looping - basic loops in Python Branching - basic selection in python Handling Files - files Handling Text - text and possibly Working with Databases - using SQL in Python You probably should read the CSV module documentation too. I suspect it will do a lot of what you want. HTH -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.flickr.com/photos/alangauldphotos From alan.gauld at btinternet.com Tue Oct 7 11:47:00 2014 From: alan.gauld at btinternet.com (Alan Gauld) Date: Tue, 07 Oct 2014 10:47:00 +0100 Subject: [Tutor] search/match file position q In-Reply-To: <05e101cfe1c0$f2adf190$d809d4b0$@us> References: <05e101cfe1c0$f2adf190$d809d4b0$@us> Message-ID: <m10cqk$k3j$1@ger.gmane.org> On 07/10/14 00:55, Clayton Kirkwood wrote: > I haven't been able to find an definitive answer. There isn't one. Each technique has its place. It depends on what you are doing and why. And to answer that you need to ask a definitive question. > I am looking through a file(stream:<), for several matching strings. Using what? regex? Or string search methods? Or character by character? or a parser? > Match definitively starts at the beginning of the stream. Sounds like regex except regex doesn't work on streams it works on strings. You could read the string(s) from a stream but that's not what regex does. > As I search, or match, do I start over at the beginning of the stream > for each match or do I start at the end of the last > search/match? Whichever fits your design best. Usually using regex I just use findall() and let Python do the work. > Apparently the file object knows where it is in a stream, but > it is unclear whether line_location = readline() keeps track of that > location. All read operations move the cursor forward and the file keeps track of it. But that's all irrelevant for match() and search() they don't care about the file. > I see advantages to both protocols, but overall, I would prefer > that the next search starts at the end of the last start If that's what you want then that's what you should do. Now what is preventing you? Do you have code where you try to do it? If so we might see the problem. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.flickr.com/photos/alangauldphotos From __peter__ at web.de Tue Oct 7 12:49:39 2014 From: __peter__ at web.de (Peter Otten) Date: Tue, 07 Oct 2014 12:49:39 +0200 Subject: [Tutor] search/match file position q References: <05e101cfe1c0$f2adf190$d809d4b0$@us> <m0vh3f$kpi$1@ger.gmane.org> <060a01cfe1f3$b52ea7f0$1f8bf7d0$@us> Message-ID: <m10gg5$5c4$1@ger.gmane.org> Clayton Kirkwood wrote: > I was trying to keep it generic. > Wrapped data file: > <tr data-row-symbol="SWKS"><td class="col-symbol > txt"><span class="wrapper " > data-model="name:DatumModel;id:null;" data-tmpl=""><a > data-ylk="cat:portfolio;cpos:1" > href="http://finance.yahoo.com/q?s=SWKS" > data-rapid_p="18">SWKS</a></span></td><td > class="col-fiftytwo_week_low cell-raw:23.270000"><span > class="wrapper " > data-model="name:DatumModel;id:SWKS:qsi:wk52:low;" > data-tmpl="change:yfin.datum">23.27</span></td><td > class="col-prev_close cell-raw:58.049999"><span > class="wrapper " data-model="name:DatumMo Doesn't Yahoo make the data available as CSV? That would be the way to go then. Anyway, regular expressions are definitely the wrong tool here, and reading the file one line at a time only makes it worse. > import re, os > line_in = file.readline() # read in humongous html line > stock = re.search('\s*<tr data-row-symbol="([A-Z]+)">', line_in) > #scan to SWKS"> in data #line, stock should be SWKS > low_52 = re.search('.+wk52:low.+([\d\.]+)<', line_in) #want to > pick up from #SWKS">, low_52 should be 23.27 > > I am trying to figure out if each re.match starts scanning at the > beginning of the same line over and over or does each scan start at the > end of the last match. It appears to start over?? > > This is stock: > <_sre.SRE_Match object; span=(0, 47), match=' <tr > data-row-symbol="SWKS">'> This is low_52: > <_sre.SRE_Match object; span=(0, 502875), match=' <tr > data-row-symbol="SWKS"><t> > If necessary, how do I pick up and move forward to the point right after > the previous match? File.tell() and file.__sizeof__(), don't seem to play > a useful role. You should try BeautifulSoup. Let's play: >>> from bs4 import BeautifulSoup >>> soup = BeautifulSoup("""<tr data-row-symbol="SWKS"><td class="col-symbol txt"><span class="wrapper " data-model="name:DatumModel;id:null;" data- tmpl=""><a data-ylk="cat:portfolio;cpos:1" href="http://finance.yahoo.com/q?s=SWKS" data- rapid_p="18">SWKS</a></span></td><td class="col-fiftytwo_week_low cell- raw:23.270000"><span class="wrapper " data- model="name:DatumModel;id:SWKS:qsi:wk52:low;" data- tmpl="change:yfin.datum">23.27</span></td><td class="col-prev_close cell- raw:58.049999">""") >>> soup.find("tr") <tr data-row-symbol="SWKS"><td class="col-symbol txt"><span class="wrapper " data-model="name:DatumModel;id:null;" data-tmpl=""><a data-rapid_p="18" data-ylk="cat:portfolio;cpos:1" href="http://finance.yahoo.com/q?s=SWKS">SWKS</a></span></td><td class="col- fiftytwo_week_low cell-raw:23.270000"><span class="wrapper " data- model="name:DatumModel;id:SWKS:qsi:wk52:low;" data- tmpl="change:yfin.datum">23.27</span></td><td class="col-prev_close cell- raw:58.049999"></td></tr> >>> tr = soup.find("tr") >>> tr["data-row-symbol"] 'SWKS' >>> tr.find_all("span") [<span class="wrapper " data-model="name:DatumModel;id:null;" data- tmpl=""><a data-rapid_p="18" data-ylk="cat:portfolio;cpos:1" href="http://finance.yahoo.com/q?s=SWKS">SWKS</a></span>, <span class="wrapper " data-model="name:DatumModel;id:SWKS:qsi:wk52:low;" data- tmpl="change:yfin.datum">23.27</span>] >>> span = tr.find_all("span")[1] >>> span["data-model"] 'name:DatumModel;id:SWKS:qsi:wk52:low;' >>> span.text '23.27' Note that normally soup would hold the complete html and you'd need a few more iterations to get to the element of interest. From francois.dion at gmail.com Tue Oct 7 13:02:23 2014 From: francois.dion at gmail.com (Francois Dion) Date: Tue, 7 Oct 2014 07:02:23 -0400 Subject: [Tutor] Suggestions Please In-Reply-To: <B44E0FEB-52A6-4774-94BD-A769ACF3ED9A@me.com> References: <B44E0FEB-52A6-4774-94BD-A769ACF3ED9A@me.com> Message-ID: <2E58A5F4-12DB-4E57-A4E7-20895B028A41@gmail.com> El Oct 6, 2014, a las 6:42 PM, Phillip Pugh <pughpl at me.com> escribi?: > I am trying to decide if Python is the right toolset for me. I do a lot of data analytics. Over the years I have used a lot of SQL and VBA, but the data sources are getting bigger. I am thinking Python may be what I need to use, but I am in the early stages of getting to know Python. Can you point me to a really good, intuitive resource for getting up to speed with data. If you are doing data analytics, particularly iterative stuff, get to know ipython notebook and pandas. http://nbviewer.ipython.org/github/twiecki/financial-analysis-python-tutorial/blob/master/1.%20Pandas%20Basics.ipynb To have an overall feel for ipython notebook and python in general for matlab style notebooks: https://github.com/ipython/ipython/wiki/A-gallery-of-interesting-IPython-Notebooks Francois -- raspberry-python.blogspot.com - www.pyptug.org - @f_dion From ljetibo at gmail.com Tue Oct 7 13:30:58 2014 From: ljetibo at gmail.com (=?ISO-8859-2?Q?Dino_Bekte=B9evi=E6?=) Date: Tue, 7 Oct 2014 13:30:58 +0200 Subject: [Tutor] Suggestions Please (Martin A. Brown) Message-ID: <CAMGeA2XwH4MNKXsH2LYNTn_h4-s9S+7KsvxHn5E2MMvJJBSr1A@mail.gmail.com> Hey, Most of your answers you already got. I'm just willing to share a short story of mine considering big data and python. I've done computer vision on images ~15Mb big each. There's 6 filters of which each can reach 7TB in total size. I ran detection algorithms per 2 filters at a time that totalled around 14TB. I've also had to read a lot of "side-data" in from SQL db's. I chose python for easy of writing and it's force-to-write-pretty-or-die syntax. I also chose it because it's more easily portable anywhere than any other languages are (some of them dependant even on the 3rd number of compiler version code!) I did all per-pixel operations with wrapped C code. That means I only did loops from 0 to width/height of the image with minimal calculations in C because the numerics there is just inherently faster than loops in python. With the new Python >3 versions a lot has been improved in that area and you no longer have to think about the implementation differences behind things like zip/izip, range/xrange, read/realine/readlines in example. Output files were easily killing 10GB in size. I used various generator expressions and iterators, that are heavily supported and encouraged in python>3 unlike python<3, and I have never run into space problems. And I have never had issues with RAM (swapping comes to mind). A short clarifications: generators are functions that behave like iterators, that means, for a generator reading a file you only allocate the memory the size of that 1 single line but are still able to preform calculations on them. These expressions are hardly ever longer than a line of code. And even better is that today you rarely have to write them yourself even, they've been written for you and hidden in the underbelly of python implementations. Combine the amazing compatibility python offers with other programs (how it's relatively easy to wrap and call outside programs in different languages, execute them and catch their return) and how easy it is to communicate with the OS in question (hardly any worries if you use python functions and not subprocess module) and how clean that code looks (compared i.e. if you ever had to pass a function as an object in C, or writing C documentation) and you're fairly well armed to combat anything. I've had my code run on Win, Ubuntu, Suse, I've run it on clusters (hybrids/IBMs) and I've never ever regretted that I chose to write my code in python. Now that we hopefully have the motivation question out of the way. There's only one way to learn anything. Set aside some free time, start coding, if you like it it's good for you. If you don't move on. One thing you will just have to learn when doing python, is how to learn really fast. All these things have pretty much been pre-written for you, but you have to get the info of where (numpy/scipy/swig in my case) and you're going to have to keep track of modules, after all that's where python really shines. I recommend books: "Learning python the hard way" and "Dive into python". also as a disclaimer, I've ran over a lot of things and some I've really bludgeoned in short don't hold any of the half-assed explanations it against me. > To: Phillip Pugh <pughpl at me.com> > Cc: Python Tutor Mailing List <tutor at python.org> > Subject: Re: [Tutor] Suggestions Please > Message-ID: <alpine.LNX.2.00.1410061956320.1349 at dagger.wonderfrog.net> > Content-Type: TEXT/PLAIN; format=flowed; charset=US-ASCII > >I am trying to decide if Python is the right toolset for me. I do a lot of data >analytics. Over the years I have used a lot of SQL and VBA, but the data >sources are getting bigger. I am thinking Python may be what I need to >use, but I am in the early stages of getting to know Python. Can you point >me to a really good, intuitive resource for getting up to speed with data. >Something that would help with the following. > >I have one text file that is 500,000 + records.. I need to read the file, move >"structured" data around and then write it to a new file. The txt file has >several data elements and is 300 characters per line. I am only interested >in the first two fields. The first data element is 19 characters. The second >data element is 6 characters. I want to rearrange the data by moving the 6 >characters data in front of the 19 characters data and then write the 25 >character data to a new file. > >I have spent some time digging for the correct resource, However being >new to Python and the syntax for the language makes it slow going. I >would like to see if I can speed up the learning curve. Hopefully can get >some results quickly then I will know how deep I want to go into Python. > >Thanks in advance > >Phillip From davea at davea.name Tue Oct 7 14:39:41 2014 From: davea at davea.name (Dave Angel) Date: Tue, 7 Oct 2014 08:39:41 -0400 (EDT) Subject: [Tutor] search/match file position q References: <05e101cfe1c0$f2adf190$d809d4b0$@us> <m0vh3f$kpi$1@ger.gmane.org> <060a01cfe1f3$b52ea7f0$1f8bf7d0$@us> Message-ID: <m10moq$ocb$1@ger.gmane.org> "Clayton Kirkwood" <crk at godblessthe.us> Wrote in message: > I was trying to keep it generic. > Wrapped data file: > <tr data-row-symbol="SWKS"><td class="col-symbol txt"><span class="wrapper " data-model="name:DatumModel;id:null;" data-tmpl=""><a data-ylk="cat:portfolio;cpos:1" href="http://finance.yahoo.com/q?s=SWKS" data-rapid_p="18">SWKS</a></span></td><td class="col-fiftytwo_week_low cell-raw:23.270000"><span class="wrapper " data-model="name:DatumModel;id:SWKS:qsi:wk52:low;" data-tmpl="change:yfin.datum">23.27</span></td><td class="col-prev_close cell-raw:58.049999"><span class="wrapper " data-model="name:DatumMo > > > import re, os > line_in = file.readline() # read in humongous html line > stock = re.search('\s*<tr data-row-symbol="([A-Z]+)">', line_in) #scan to SWKS"> in data #line, stock should be SWKS > low_52 = re.search('.+wk52:low.+([\d\.]+)<', line_in) #want to pick up from #SWKS">, low_52 should be 23.27 > > I am trying to figure out if each re.match starts scanning at the beginning of the same line over and over or does each scan start at the end of the last match. It appears to start over?? > > This is stock: > <_sre.SRE_Match object; span=(0, 47), match=' <tr data-row-symbol="SWKS">'> > This is low_52: > <_sre.SRE_Match object; span=(0, 502875), match=' <tr data-row-symbol="SWKS"><t> > If necessary, how do I pick up and move forward to the point right after the previous match? File.tell() and file.__sizeof__(), don't seem to play a useful role. > The best solution is ANYTHING but html scraping. If the website offers an api like csf, use it. Html is too prone to changing at the whim of the developers. If you must use html, get beautiful soup. Regex can mess up suddenly even if the developers don't change anything. Regex should only be used on html if you're the one generating the website, and you coordinate it to be parseable. If regex were the best solution you could read the following example pasted from the online docs. re.findall searches a string, not a file, so file position is irrelevant. The numbers below can be used to subscript your string, either for saving the results or for removing the part already searched. Something like line_in = line_in[span [0] +span [1]: ] Ref: https://docs.python.org/3.4/howto/regex.html findall() has to create the entire list before it can be returned as the result. The finditer() method returns a sequence of match object instances as an iterator: >>> >>> iterator = p.finditer('12 drummers drumming, 11 ... 10 ...') >>> iterator <callable_iterator object at 0x...> >>> for match in iterator: ... print(match.span()) ... (0, 2) (22, 24) (29, 31) -- DaveA From dillonrw at comcast.net Tue Oct 7 15:40:24 2014 From: dillonrw at comcast.net (Richard Dillon) Date: Tue, 7 Oct 2014 06:40:24 -0700 Subject: [Tutor] format command help Message-ID: <6B65AD97-1216-43D2-B84B-7E64C103BC7B@comcast.net> I create column headings using \t print('base1\tbase2\theight\tarea') and I would like the numbers to align with the headings. I think that I need to use format instead of doing this: print(A,' ',B,' ',C,' ',int(area1)) print(D,' ',E,' ',F,' ',int(area2)) but I don't know how. I've looked at code examples and I can't figure out how. Thanks From alan.gauld at btinternet.com Tue Oct 7 17:45:38 2014 From: alan.gauld at btinternet.com (Alan Gauld) Date: Tue, 07 Oct 2014 16:45:38 +0100 Subject: [Tutor] format command help In-Reply-To: <6B65AD97-1216-43D2-B84B-7E64C103BC7B@comcast.net> References: <6B65AD97-1216-43D2-B84B-7E64C103BC7B@comcast.net> Message-ID: <m111r2$ggk$1@ger.gmane.org> On 07/10/14 14:40, Richard Dillon wrote: > I create column headings using \t > > print('base1\tbase2\theight\tarea') > > and I would like the numbers to align with the headings. > I think that I need to use format instead of doing this: Yes, using tabs is a bit hit and miss depemding on the length of the fields. If they are all similar sizes its OK but otherwise it gets messy. > print(A,' ',B,' ',C,' ',int(area1)) > print(D,' ',E,' ',F,' ',int(area2)) > > but I don't know how. At its simplest you just use braces like this: fmtString = "{}\t{}\t{}\t{}" print(fmtString.format('base1','base2',height','area')) And that gives you your header line then print(fmtString.format(A,B,C,int(area1)) gives you your first line and so on. By using a constant fmtString you avoid any risk of mixing things up. But formatting gives you more options. You can specify width and justification. By using a fixed width you are more likely to get the alignment consistent than by using tabs. Here is an example: fmtString="{:20}{:20}{:10}{:6}" Then your prints stay the same but the formatting will now use the fixed lengths you gave rather than rely on tabs. You can specify which end of the space you want the data by using the < and > justification characters. So to pull the integer to the left hand siode of its column use: fmtString="{:20}{:20}{:10}{:<6}" You can specify the type too but often you don't need to since Python will do conversion to string for you. There's lots more. Read the documentation on format and play with different format strings in the interpreter. HTH -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.flickr.com/photos/alangauldphotos From crk at godblessthe.us Tue Oct 7 17:47:37 2014 From: crk at godblessthe.us (Clayton Kirkwood) Date: Tue, 7 Oct 2014 08:47:37 -0700 Subject: [Tutor] search/match file position q In-Reply-To: <m10gg5$5c4$1@ger.gmane.org> References: <05e101cfe1c0$f2adf190$d809d4b0$@us> <m0vh3f$kpi$1@ger.gmane.org> <060a01cfe1f3$b52ea7f0$1f8bf7d0$@us> <m10gg5$5c4$1@ger.gmane.org> Message-ID: <06aa01cfe246$08799e90$196cdbb0$@us> !-----Original Message----- !From: Tutor [mailto:tutor-bounces+crk=godblessthe.us at python.org] On !Behalf Of Peter Otten !Sent: Tuesday, October 07, 2014 3:50 AM !To: tutor at python.org !Subject: Re: [Tutor] search/match file position q ! !Clayton Kirkwood wrote: ! !> I was trying to keep it generic. !> Wrapped data file: !> <tr data-row-symbol="SWKS"><td class="col-symbol !> txt"><span class="wrapper " !> data-model="name:DatumModel;id:null;" data- !tmpl=""><a !> data-ylk="cat:portfolio;cpos:1" !> href="http://finance.yahoo.com/q?s=SWKS" !> data-rapid_p="18">SWKS</a></span></td><td !> class="col-fiftytwo_week_low cell- !raw:23.270000"><span !> class="wrapper " !> data-model="name:DatumModel;id:SWKS:qsi:wk52:low;" !> data-tmpl="change:yfin.datum">23.27</span></td><td !> class="col-prev_close cell-raw:58.049999"><span !> class="wrapper " data-model="name:DatumMo ! !Doesn't Yahoo make the data available as CSV? That would be the way to !go then. Yes, Yahoo has a few columns that are csv, but I have maybe 15 fields that aren't provided. Besides, what fun would that be, I try to find tasks that allow me to expand my knowledge"<))) ! !Anyway, regular expressions are definitely the wrong tool here, and !reading the file one line at a time only makes it worse. Why is it making it only worse? I don't think a char by char would be helpful, the line happens to be very long, and I don't have a way of peeking around the corner to the next line so to speak. If I broke it into shorter strings, it would be much more onerous to jump over the end of the current to potentially many next strings. ! !> import re, os !> line_in = file.readline() ! # read in humongous html line !> stock = re.search('\s*<tr data-row-symbol="([A-Z]+)">', !line_in) !> #scan to SWKS"> in data ! #line, stock !should be SWKS !> low_52 = re.search('.+wk52:low.+([\d\.]+)<', line_in) !#want to !> pick up from ! #SWKS">, !low_52 should be 23.27 !> !> I am trying to figure out if each re.match starts scanning at the !> beginning of the same line over and over or does each scan start at !> the end of the last match. It appears to start over?? !> !> This is stock: !> <_sre.SRE_Match object; span=(0, 47), match=' <tr !> data-row-symbol="SWKS">'> This is low_52: !> <_sre.SRE_Match object; span=(0, 502875), match=' !<tr !> data-row-symbol="SWKS"><t> !> If necessary, how do I pick up and move forward to the point right !> after the previous match? File.tell() and file.__sizeof__(), don't !> seem to play a useful role. ! !You should try BeautifulSoup. Let's play: ! !>>> from bs4 import BeautifulSoup !>>> soup = BeautifulSoup("""<tr data-row-symbol="SWKS"><td !>>> class="col-symbol !txt"><span class="wrapper " data-model="name:DatumModel;id:null;" data- !tmpl=""><a data-ylk="cat:portfolio;cpos:1" !href="http://finance.yahoo.com/q?s=SWKS" data- !rapid_p="18">SWKS</a></span></td><td class="col-fiftytwo_week_low cell- !raw:23.270000"><span class="wrapper " data- !model="name:DatumModel;id:SWKS:qsi:wk52:low;" data- !tmpl="change:yfin.datum">23.27</span></td><td class="col-prev_close !cell- !raw:58.049999">""") !>>> soup.find("tr") !<tr data-row-symbol="SWKS"><td class="col-symbol txt"><span !class="wrapper " !data-model="name:DatumModel;id:null;" data-tmpl=""><a data-rapid_p="18" !data-ylk="cat:portfolio;cpos:1" !href="http://finance.yahoo.com/q?s=SWKS">SWKS</a></span></td><td !class="col- fiftytwo_week_low cell-raw:23.270000"><span class="wrapper " !data- model="name:DatumModel;id:SWKS:qsi:wk52:low;" data- !tmpl="change:yfin.datum">23.27</span></td><td class="col-prev_close !cell- raw:58.049999"></td></tr> !>>> tr = soup.find("tr") !>>> tr["data-row-symbol"] !'SWKS' !>>> tr.find_all("span") ![<span class="wrapper " data-model="name:DatumModel;id:null;" data- !tmpl=""><a data-rapid_p="18" data-ylk="cat:portfolio;cpos:1" !href="http://finance.yahoo.com/q?s=SWKS">SWKS</a></span>, <span !class="wrapper " data-model="name:DatumModel;id:SWKS:qsi:wk52:low;" !data- tmpl="change:yfin.datum">23.27</span>] !>>> span = tr.find_all("span")[1] !>>> span["data-model"] !'name:DatumModel;id:SWKS:qsi:wk52:low;' !>>> span.text !'23.27' So, what makes regex wrong for this job? question still remains: does the search start at the beginning of the line each time or does it step forward from the last search? I will check out beautiful soup as suggested in a subsequent mail; I'd still like to finish this process:<}} Clayton From __peter__ at web.de Tue Oct 7 20:10:11 2014 From: __peter__ at web.de (Peter Otten) Date: Tue, 07 Oct 2014 20:10:11 +0200 Subject: [Tutor] search/match file position q References: <05e101cfe1c0$f2adf190$d809d4b0$@us> <m0vh3f$kpi$1@ger.gmane.org> <060a01cfe1f3$b52ea7f0$1f8bf7d0$@us> <m10gg5$5c4$1@ger.gmane.org> <06aa01cfe246$08799e90$196cdbb0$@us> Message-ID: <m11aa4$9d2$1@ger.gmane.org> Clayton Kirkwood wrote: > > > !-----Original Message----- > !From: Tutor [mailto:tutor-bounces+crk=godblessthe.us at python.org] On > !Behalf Of Peter Otten > !Sent: Tuesday, October 07, 2014 3:50 AM > !To: tutor at python.org > !Subject: Re: [Tutor] search/match file position q > ! > !Clayton Kirkwood wrote: > ! > !> I was trying to keep it generic. > !> Wrapped data file: > !> <tr data-row-symbol="SWKS"><td class="col-symbol > !> txt"><span class="wrapper " > !> data-model="name:DatumModel;id:null;" data- > !tmpl=""><a > !> data-ylk="cat:portfolio;cpos:1" > !> href="http://finance.yahoo.com/q?s=SWKS" > !> data-rapid_p="18">SWKS</a></span></td><td > !> class="col-fiftytwo_week_low cell- > !raw:23.270000"><span > !> class="wrapper " > !> data-model="name:DatumModel;id:SWKS:qsi:wk52:low;" > !> data-tmpl="change:yfin.datum">23.27</span></td><td > !> class="col-prev_close cell-raw:58.049999"><span > !> class="wrapper " data-model="name:DatumMo > ! > !Doesn't Yahoo make the data available as CSV? That would be the way to > !go then. > > > Yes, Yahoo has a few columns that are csv, but I have maybe 15 fields that > aren't provided. Besides, what fun would that be, I try to find tasks that > allow me to expand my knowledge"<))) > > ! > !Anyway, regular expressions are definitely the wrong tool here, and > !reading the file one line at a time only makes it worse. > > > Why is it making it only worse? I don't think a char by char would be > helpful, the line happens to be very long, and I don't have a way of > peeking around the corner to the next line so to speak. If I broke it into > shorter strings, it would be much more onerous to jump over the end of the > current to potentially many next strings. I meant you should slurp in the whole file instead of reading it line after line. That way you'd at least have a chance to find elements that spread over more than one line like <a href="example.com">Example</a> > !> import re, os > !> line_in = file.readline() > ! # read in humongous html line > !> stock = re.search('\s*<tr data-row-symbol="([A-Z]+)">', > !line_in) > !> #scan to SWKS"> in data > ! #line, stock > !should be SWKS > !> low_52 = re.search('.+wk52:low.+([\d\.]+)<', line_in) > !#want to > !> pick up from > ! #SWKS">, > !low_52 should be 23.27 > !> > !> I am trying to figure out if each re.match starts scanning at the > !> beginning of the same line over and over or does each scan start at > !> the end of the last match. It appears to start over?? > !> > !> This is stock: > !> <_sre.SRE_Match object; span=(0, 47), match=' <tr > !> data-row-symbol="SWKS">'> This is low_52: > !> <_sre.SRE_Match object; span=(0, 502875), match=' > !<tr > !> data-row-symbol="SWKS"><t> > !> If necessary, how do I pick up and move forward to the point right > !> after the previous match? File.tell() and file.__sizeof__(), don't > !> seem to play a useful role. > ! > !You should try BeautifulSoup. Let's play: > ! > !>>> from bs4 import BeautifulSoup > !>>> soup = BeautifulSoup("""<tr data-row-symbol="SWKS"><td > !>>> span.text > !'23.27' > > > So, what makes regex wrong for this job? A regex doesn't understand the structure of an html document. For example you need to keep track of the nesting level manually to find the cells of the inner of two nested tables. > question still remains: does the > search start at the beginning of the line each time or does it step > forward from the last search? re.search() doesn't keep track of prior searches; whatever string you feed it (in your case a line cut out of an html document) is searched. > I will check out beautiful soup as suggested > in a subsequent mail; I'd still like to finish this process:<}} Do you say that when someone points out that you are eating your shoe? From dyoo at hashcollision.org Tue Oct 7 20:13:32 2014 From: dyoo at hashcollision.org (Danny Yoo) Date: Tue, 7 Oct 2014 11:13:32 -0700 Subject: [Tutor] search/match file position q In-Reply-To: <06aa01cfe246$08799e90$196cdbb0$@us> References: <05e101cfe1c0$f2adf190$d809d4b0$@us> <m0vh3f$kpi$1@ger.gmane.org> <060a01cfe1f3$b52ea7f0$1f8bf7d0$@us> <m10gg5$5c4$1@ger.gmane.org> <06aa01cfe246$08799e90$196cdbb0$@us> Message-ID: <CAGZAPF6YX5DKVpE7EUS0pGdL7-NmAvmpwD1d3Ev68BrYYwG_0g@mail.gmail.com> > So, what makes regex wrong for this job? question still remains: does the > search start at the beginning of the line each time or does it step forward > from the last search? I will check out beautiful soup as suggested in a > subsequent mail; I'd still like to finish this process:<}} Mathematically, regular expressions can capture a certain class of text called the "regular languages". Regular languages have a few characteristics. As a concrete example of a limitation: you can't write a pattern that properly does parentheses matching with a regular expression alone. This isn't a challenge to your machismo: it's a matter of mathematics! For the precise details on the impossibility proof, you'd need to take a CS theory class, and in particular, learn about the "pumping lemma for regular expressions". Sipser's "Introduction to the Theory of Computation" has a good presentation. This is one reason why CS theory matters: it can tell you when some approach is not a good idea. :P HTML is not a regular language: it has nested substructure. The same problem about matching balanced parentheses is essentially that of matching start and end tags. So that's the objections from the purely mathematical point of view. This is not to say that regular expressions are useless: they work well for breaking down HTML into a sequence of tokens. If you only care about processing individual tokens at a time, regexes might be appropriate. They're just not the best tool for everything. From a practical point of view: HTML parsing libraries such as Beautiful Soup are nicer to work with than plain regular expressions. From juan0christian at gmail.com Tue Oct 7 21:08:33 2014 From: juan0christian at gmail.com (Juan Christian) Date: Tue, 7 Oct 2014 16:08:33 -0300 Subject: [Tutor] Using xml.etree ElementTree Message-ID: <CAAp0bGuZK1_kAYE7t-p7ew9Gs5LiAzA990yC9frghp4BEZ4_fQ@mail.gmail.com> I have this XML scheme: http://steamcommunity.com/profiles/76561198084537782?xml=1 My code: self._xml = ElementTree.fromstring(requests.get(url + '?xml=1').content) print(self._xml.tag) > returns > profile It's working, the thing is that I don't know how to "navigate" inside the XML, I read the doc but everything there regarding key-value is on for-loops, I need something more direct, like: XML.steamID64 XML.steamID XML.onlineState XML.vacBanned ... Something like that, where I provide the key (<xmltag>) and get the value (<xmltag>value</xmltag>). Is that possible? -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/tutor/attachments/20141007/8d4e28e8/attachment-0001.html> From crk at godblessthe.us Tue Oct 7 21:40:54 2014 From: crk at godblessthe.us (Clayton Kirkwood) Date: Tue, 7 Oct 2014 12:40:54 -0700 Subject: [Tutor] search/match file position q In-Reply-To: <m11aa4$9d2$1@ger.gmane.org> References: <05e101cfe1c0$f2adf190$d809d4b0$@us> <m0vh3f$kpi$1@ger.gmane.org> <060a01cfe1f3$b52ea7f0$1f8bf7d0$@us> <m10gg5$5c4$1@ger.gmane.org> <06aa01cfe246$08799e90$196cdbb0$@us> <m11aa4$9d2$1@ger.gmane.org> Message-ID: <007201cfe266$9c5261e0$d4f725a0$@us> !> !> So, what makes regex wrong for this job? ! !A regex doesn't understand the structure of an html document. For !example !you need to keep track of the nesting level manually to find the cells !of !the inner of two nested tables. ! !> question still remains: does the !> search start at the beginning of the line each time or does it step !> forward from the last search? ! !re.search() doesn't keep track of prior searches; whatever string you !feed !it (in your case a line cut out of an html document) is searched. ! So, you are saying that each regex starts at the beginning of the long line? Is there a way to start the next search at the end of the last one? !> I will check out beautiful soup as suggested !> in a subsequent mail; I'd still like to finish this process:<}} ! !Do you say that when someone points out that you are eating your shoe? Depends on the flavor of the shoe:<))) Clayton From crk at godblessthe.us Tue Oct 7 21:53:19 2014 From: crk at godblessthe.us (Clayton Kirkwood) Date: Tue, 7 Oct 2014 12:53:19 -0700 Subject: [Tutor] search/match file position q In-Reply-To: <CAGZAPF6YX5DKVpE7EUS0pGdL7-NmAvmpwD1d3Ev68BrYYwG_0g@mail.gmail.com> References: <05e101cfe1c0$f2adf190$d809d4b0$@us> <m0vh3f$kpi$1@ger.gmane.org> <060a01cfe1f3$b52ea7f0$1f8bf7d0$@us> <m10gg5$5c4$1@ger.gmane.org> <06aa01cfe246$08799e90$196cdbb0$@us> <CAGZAPF6YX5DKVpE7EUS0pGdL7-NmAvmpwD1d3Ev68BrYYwG_0g@mail.gmail.com> Message-ID: <007c01cfe268$58b189a0$0a149ce0$@us> !-----Original Message----- !From: Danny Yoo [mailto:dyoo at hashcollision.org] !Sent: Tuesday, October 07, 2014 11:14 AM !To: Clayton Kirkwood !Cc: Python Tutor Mailing List !Subject: Re: [Tutor] search/match file position q ! !> So, what makes regex wrong for this job? question still remains: does !> the search start at the beginning of the line each time or does it !> step forward from the last search? I will check out beautiful soup as !> suggested in a subsequent mail; I'd still like to finish this !> process:<}} ! ! !Mathematically, regular expressions can capture a certain class of text !called the "regular languages". Regular languages have a few !characteristics. As a concrete example of a limitation: you can't write !a pattern that properly does parentheses matching with a regular !expression alone. ! !This isn't a challenge to your machismo: it's a matter of mathematics! ! For the precise details on the impossibility proof, you'd need to take !a CS theory class, and in particular, learn about the "pumping lemma for !regular expressions". Sipser's "Introduction to the Theory of !Computation" has a good presentation. This is one reason why CS theory !matters: it can tell you when some approach is not a good idea. !:P ! !HTML is not a regular language: it has nested substructure. The same !problem about matching balanced parentheses is essentially that of !matching start and end tags. ! !So that's the objections from the purely mathematical point of view. !This is not to say that regular expressions are useless: they work well !for breaking down HTML into a sequence of tokens. If you only care !about processing individual tokens at a time, regexes might be !appropriate. They're just not the best tool for everything. From a !practical point of view: HTML parsing libraries such as Beautiful Soup !are nicer to work with than plain regular expressions. In this case, I was able to determine which line I was interested in because it had a specific marker. From that point, I knew specific markers to look for for each desired field. I thought the desired parenthesis couple was assigned to the variable at the beginning of the match line. I thought that regex's Were meant to skip over unwanted detritus and grab only the desired match with in the parentheses. Wrong? TIA, Clayton From juan0christian at gmail.com Wed Oct 8 00:03:32 2014 From: juan0christian at gmail.com (Juan Christian) Date: Tue, 7 Oct 2014 19:03:32 -0300 Subject: [Tutor] Using xml.etree ElementTree In-Reply-To: <CAAp0bGuZK1_kAYE7t-p7ew9Gs5LiAzA990yC9frghp4BEZ4_fQ@mail.gmail.com> References: <CAAp0bGuZK1_kAYE7t-p7ew9Gs5LiAzA990yC9frghp4BEZ4_fQ@mail.gmail.com> Message-ID: <CAAp0bGuZArGDhL8BDytCDWyTHmXiJX2rJuWqkT-oBWR2faUJuQ@mail.gmail.com> On Tue, Oct 7, 2014 at 4:08 PM, Juan Christian <juan0christian at gmail.com> wrote: > I have this XML scheme: > http://steamcommunity.com/profiles/76561198084537782?xml=1 > > My code: > > self._xml = ElementTree.fromstring(requests.get(url + '?xml=1').content) > print(self._xml.tag) > returns > profile > > It's working, the thing is that I don't know how to "navigate" inside the > XML, I read the doc but everything there regarding key-value is on > for-loops, I need something more direct, like: > > XML.steamID64 > XML.steamID > XML.onlineState > XML.vacBanned > ... > > Something like that, where I provide the key (<xmltag>) and get the value > (<xmltag>value</xmltag>). Is that possible? > So.. I found that I can do it with 'self._xml[index].text', but that's not a good and 'visible' way to do things. Explicit is better than implicit, and this XML has some data that might or might not be there, so I can't trust in the index. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/tutor/attachments/20141007/66924fa9/attachment.html> From martin at linux-ip.net Wed Oct 8 00:05:07 2014 From: martin at linux-ip.net (Martin A. Brown) Date: Tue, 7 Oct 2014 15:05:07 -0700 Subject: [Tutor] search/match file position q In-Reply-To: <007201cfe266$9c5261e0$d4f725a0$@us> References: <05e101cfe1c0$f2adf190$d809d4b0$@us> <m0vh3f$kpi$1@ger.gmane.org> <060a01cfe1f3$b52ea7f0$1f8bf7d0$@us> <m10gg5$5c4$1@ger.gmane.org> <06aa01cfe246$08799e90$196cdbb0$@us> <m11aa4$9d2$1@ger.gmane.org> <007201cfe266$9c5261e0$d4f725a0$@us> Message-ID: <alpine.LNX.2.00.1410071434330.1349@dagger.wonderfrog.net> Good afternoon Clayton, > !A regex doesn't understand the structure of an html document. For > !example > !you need to keep track of the nesting level manually to find the cells > !of > !the inner of two nested tables. > ! > !> question still remains: does the > !> search start at the beginning of the line each time or does it step > !> forward from the last search? > ! > !re.search() doesn't keep track of prior searches; whatever string you > !feed > !it (in your case a line cut out of an html document) is searched. > ! > > So, you are saying that each regex starts at the beginning of the > long line? Is there a way to start the next search at the end of > the last one? Well, it depends on how you are using the re module (if you really want to do that). Have a look at: https://docs.python.org/2/library/re.html#re.RegexObject.search But...I'll add my voice to the admonition against using regex here. Consider the following events that could happen in the future after you have labored over your program and are able to get it to work, based on today's HTML. 1. Somebody inserts a line-break in the middle of the element you were searching for with regex. 2. A week from now, somebody runs 'tidy' on the HTML or changes or removes the the line endings. 3. Somebody adds an HTML comment which causes your regex to match. These are the first three reasons that occur to me for why regex is the wrong tool for the job here, given that you know precisely the format of the data. It is HTML. The good thing is that there are other tools for processing HTML. Anyway, if you want to use regexes, nobody can stop you, so see below, a bit of nonsense text which you can search for 2 distinct instances of the string "ei" [0]. > !> I will check out beautiful soup as suggested > !> in a subsequent mail; I'd still like to finish this process:<}} > !Do you say that when someone points out that you are eating your shoe? > Depends on the flavor of the shoe:<))) Root beer float. -Martin [0] If you really, really want to use regex, here's an example of how to keep track of where you last sought, and how to search from that place in the string. from __future__ import print_function import re def main(): s = 'Wo lattenzaun aneinander erhaltenen vorpfeifen grasgarten.' pattern = re.compile('ei', re.IGNORECASE) matched = pattern.search(s,0) while matched: endpos = matched.end() print(matched.group(0), matched.start(), matched.end()) matched = pattern.search(s, endpos) -- Martin A. Brown http://linux-ip.net/ From dyoo at hashcollision.org Wed Oct 8 00:50:46 2014 From: dyoo at hashcollision.org (Danny Yoo) Date: Tue, 7 Oct 2014 15:50:46 -0700 Subject: [Tutor] Using xml.etree ElementTree In-Reply-To: <CAAp0bGuZArGDhL8BDytCDWyTHmXiJX2rJuWqkT-oBWR2faUJuQ@mail.gmail.com> References: <CAAp0bGuZK1_kAYE7t-p7ew9Gs5LiAzA990yC9frghp4BEZ4_fQ@mail.gmail.com> <CAAp0bGuZArGDhL8BDytCDWyTHmXiJX2rJuWqkT-oBWR2faUJuQ@mail.gmail.com> Message-ID: <CAGZAPF5-LTwAPqJCQ86-cQS3+tgWurc9vBhqxTZP15K2eA=Bmw@mail.gmail.com> >> It's working, the thing is that I don't know how to "navigate" inside the >> XML, I read the doc but everything there regarding key-value is on >> for-loops, I need something more direct, like: Hi Juan, I think you're looking for: http://effbot.org/zone/element.htm#searching-for-subelements where you should be able to do something like: sml._xml.findtext('steamID64') to get the text "76561198084537782". Let's check: ################################################################### >>> import xml.etree.ElementTree >>> import urllib >>> f = urllib.urlopen("http://steamcommunity.com/profiles/76561198084537782?xml=1") >>> tree = xml.etree.ElementTree.ElementTree(file=f) >>> tree.findtext('steamID64') '76561198084537782' ################################################################### Yup, this looks relatively straightforward. Let us know if you hit any snags. From alan.gauld at btinternet.com Wed Oct 8 01:18:32 2014 From: alan.gauld at btinternet.com (Alan Gauld) Date: Wed, 08 Oct 2014 00:18:32 +0100 Subject: [Tutor] search/match file position q In-Reply-To: <007c01cfe268$58b189a0$0a149ce0$@us> References: <05e101cfe1c0$f2adf190$d809d4b0$@us> <m0vh3f$kpi$1@ger.gmane.org> <060a01cfe1f3$b52ea7f0$1f8bf7d0$@us> <m10gg5$5c4$1@ger.gmane.org> <06aa01cfe246$08799e90$196cdbb0$@us> <CAGZAPF6YX5DKVpE7EUS0pGdL7-NmAvmpwD1d3Ev68BrYYwG_0g@mail.gmail.com> <007c01cfe268$58b189a0$0a149ce0$@us> Message-ID: <m11sc8$8iu$1@ger.gmane.org> On 07/10/14 20:53, Clayton Kirkwood wrote: > In this case, I was able to determine which line I > was interested in because it had a > specific marker. > From that point, I knew specific markers to look > for for each desired field. > I thought the desired parenthesis couple was assigned > to the variable at the beginning of the match line. I'm not quite sure what you mean by that. > I thought that regex's Were meant to skip over > unwanted detritus and grab only the desired > match with in the parentheses. Not quite. They don't "skip over" unwanted detritus. The detritus is either part of the matched pattern or it's not. regex find defined patterns they don't find undefined patterns. The skipping over happens as a side effect of looking for the pattern, it's not a design goal of itself. And the reason regex are often described as a weapon of last resort is because it's infernally difficult to define exactly the right pattern for anything non-trivial. And especially in things like HTML where it may even be impossible to get it right. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.flickr.com/photos/alangauldphotos From juan0christian at gmail.com Wed Oct 8 02:02:45 2014 From: juan0christian at gmail.com (Juan Christian) Date: Tue, 7 Oct 2014 21:02:45 -0300 Subject: [Tutor] Using xml.etree ElementTree In-Reply-To: <CAGZAPF5-LTwAPqJCQ86-cQS3+tgWurc9vBhqxTZP15K2eA=Bmw@mail.gmail.com> References: <CAAp0bGuZK1_kAYE7t-p7ew9Gs5LiAzA990yC9frghp4BEZ4_fQ@mail.gmail.com> <CAAp0bGuZArGDhL8BDytCDWyTHmXiJX2rJuWqkT-oBWR2faUJuQ@mail.gmail.com> <CAGZAPF5-LTwAPqJCQ86-cQS3+tgWurc9vBhqxTZP15K2eA=Bmw@mail.gmail.com> Message-ID: <CAAp0bGvM1MnprBifM-o1-PPLXKQzeA2pLLztvk0MawhfoezKeA@mail.gmail.com> On Tue, Oct 7, 2014 at 7:50 PM, Danny Yoo <dyoo at hashcollision.org> wrote: > Hi Juan, > > I think you're looking for: > > http://effbot.org/zone/element.htm#searching-for-subelements > > where you should be able to do something like: > > sml._xml.findtext('steamID64') > > to get the text "76561198084537782". > > > Let's check: > > ################################################################### > >>> import xml.etree.ElementTree > >>> import urllib > >>> f = urllib.urlopen(" > http://steamcommunity.com/profiles/76561198084537782?xml=1") > >>> tree = xml.etree.ElementTree.ElementTree(file=f) > >>> tree.findtext('steamID64') > '76561198084537782' > ################################################################### > > > Yup, this looks relatively straightforward. Let us know if you hit any > snags. > Exactly what I needed, thanks! -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/tutor/attachments/20141007/057f3768/attachment.html> From lists at mostrom.pp.se Wed Oct 8 07:47:13 2014 From: lists at mostrom.pp.se (=?UTF-8?Q?Jan_Erik_Mostr=C3=B6m?=) Date: Wed, 8 Oct 2014 07:47:13 +0200 Subject: [Tutor] Scaling photos In-Reply-To: <m0vej1$v0o$1@ger.gmane.org> References: <CALGRW_yxT6n1z90vjc4Db9CNRr_x6U37m1fuksheubgCkApgqQ@mail.gmail.com> <m0vej1$v0o$1@ger.gmane.org> Message-ID: <CALGRW_wMzUZumdyA41GMnf5+ViRxshQ71QThZGD_VZYGN6sOUQ@mail.gmail.com> Thanks for the suggestions On Tue, Oct 7, 2014 at 3:10 AM, Alan Gauld <alan.gauld at btinternet.com> wrote: > On 06/10/14 23:08, Jan Erik Mostr?m wrote: >> >> I want to write a small script that scales photos. I want the scaled >> photos to keep all meta data available in the originals. I also want >> to keep this python only and easily installed on linux and OS X >> machines. >> >> What library would you people recommend for doing this? > > > Pillow > Its the Python 3 version of PIL but works with 2.7 too I think. > Most PIL tutorials will work for Pillow too. > > Also consider ImageMagick. Its usually available in the package system for > Linux and can be downloaded for Macs too. It has a Python library. > > HTH > -- > Alan G > Author of the Learn to Program web site > http://www.alan-g.me.uk/ > http://www.flickr.com/photos/alangauldphotos > > _______________________________________________ > Tutor maillist - Tutor at python.org > To unsubscribe or change subscription options: > https://mail.python.org/mailman/listinfo/tutor From alan.gauld at btinternet.com Wed Oct 8 08:49:21 2014 From: alan.gauld at btinternet.com (Alan Gauld) Date: Wed, 08 Oct 2014 07:49:21 +0100 Subject: [Tutor] Suggestions Please In-Reply-To: <B2ED4870-EE82-4A35-BE72-D1E8A91B2F8A@me.com> References: <B44E0FEB-52A6-4774-94BD-A769ACF3ED9A@me.com> <m10cbs$eem$1@ger.gmane.org> <B2ED4870-EE82-4A35-BE72-D1E8A91B2F8A@me.com> Message-ID: <5434DE71.8020009@btinternet.com> On 08/10/14 02:02, Phillip Pugh wrote: > with open("InputTest.txt","r") as f: > with open("Outputt.txt", "w") as fw: > for line in f: > first,second = line[:32], line[32:37] > > if first.isspace()== False: > fw.write (second.strip()+ first.strip()+"\n") > > f.close() > fw.close() > > You don't need the close()s the with construct does that for you automatically. Alan G. From pughpl at me.com Wed Oct 8 03:02:09 2014 From: pughpl at me.com (Phillip Pugh) Date: Tue, 07 Oct 2014 20:02:09 -0500 Subject: [Tutor] Suggestions Please In-Reply-To: <m10cbs$eem$1@ger.gmane.org> References: <B44E0FEB-52A6-4774-94BD-A769ACF3ED9A@me.com> <m10cbs$eem$1@ger.gmane.org> Message-ID: <B2ED4870-EE82-4A35-BE72-D1E8A91B2F8A@me.com> Thank you All!! I am impressed with the support. It was very helpful and timely. I was able to put together a script to do what I wanted. I know now that I wont be wasting time learning Python. As with any language, it is about understanding the syntax. As I mentioned before, I want to make sure I am focusing my time on something useful and this was a big help. Here is what I came up with (with your help) . I expect there is a more efficient way to do it, but hey... it was my first try with data. And FYI, I work with over one hundred data sources, I wanted to test on something small. Phillip with open("InputTest.txt","r") as f: with open("Outputt.txt", "w") as fw: for line in f: first,second = line[:32], line[32:37] if first.isspace()== False: fw.write (second.strip()+ first.strip()+"\n") f.close() fw.close() On Oct 7, 2014, at 4:39 AM, Alan Gauld <alan.gauld at btinternet.com> wrote: > On 06/10/14 23:42, Phillip Pugh wrote: >> I am trying to decide if Python is the right toolset for me. > > I do a lot of data analytics. > > It can almost certainly do what you want but there may be other > tools that do it better. However, data analytics is quite vague. > It depends on what kind of data and what kind of analysis. > > > Can you point me to a really good, intuitive resource > > intuitive depends on the student. > But we also need to know what kind of data. > Is it stored in flat files? > in a SQL database(which one?) > In a NoSQL database(which one?) > Python can handle all of those but the tutorials involved > will all be different. > > If you want a general introduction with some SQL database > specifics you can try my tutorial(see sig). Whether you > find it intuitive is another matter. > >> I have one text file that is 500,000 + records.. > > Thats not very big in modern computing terms. > You could probably just read that straight into memory. > >> I need to read the file, > > What kind of file? A database file such as Foxpro? > or Access? or a CSV export? Or something else? > >> move "structured" data around and then write it to a new file. > > What is structured about it? Fixed column width? > Fixed relative position? Binary format? > >> The txt file has several data elements and is > > 300 characters per line. > >> I am only interested in the first two fields. > > The first data element is 19 characters. > > The second data element is 6 characters. > > There are two ways in Python to extract 'columns' from a file. > > If you know the separators you can use either the csv module(best) > or string.split(<sep list>) to create a list of fields. > > If its a fixed length record (with potentially no seperator) > you can use string slicing. In your case that would be > field1 = string[:19]; field2 = string[19:25] > >> I want to rearrange the data by moving the 6 characters data > > in front of the 19 characters data > > Do you need a separator? > >> and then write the 25 character data to a new file. > > the reading and writing of the files is straightforward, any tutorial will show you that. > >> I have spent some time digging for the correct resource, > > However being new to Python and the syntax for the language > > makes it slow going. I would like to see if I can speed up > > the learning curve. > > So far it sounds like you don't need any of the high powered data analysis tools like R or Pandas, you are just doing basic data extraction and manipulation. For that standard Python should > be fine and most tutorials include all you need. > > If you look at mine the most relevant topics from the contents > are: > The raw materials - variables & data types > Looping - basic loops in Python > Branching - basic selection in python > Handling Files - files > Handling Text - text > > and possibly > Working with Databases - using SQL in Python > > You probably should read the CSV module documentation too. > I suspect it will do a lot of what you want. > > HTH > -- > Alan G > Author of the Learn to Program web site > http://www.alan-g.me.uk/ > http://www.flickr.com/photos/alangauldphotos > > _______________________________________________ > Tutor maillist - Tutor at python.org > To unsubscribe or change subscription options: > https://mail.python.org/mailman/listinfo/tutor From davea at davea.name Wed Oct 8 10:38:18 2014 From: davea at davea.name (Dave Angel) Date: Wed, 8 Oct 2014 04:38:18 -0400 (EDT) Subject: [Tutor] Suggestions Please References: <B44E0FEB-52A6-4774-94BD-A769ACF3ED9A@me.com> <m10cbs$eem$1@ger.gmane.org> <B2ED4870-EE82-4A35-BE72-D1E8A91B2F8A@me.com> Message-ID: <m12t04$o8r$1@ger.gmane.org> Phillip Pugh <pughpl at me.com> Wrote in message: > Thank you All!! > > I am impressed with the support. It was very helpful and timely. I was able to put together a script to do what I wanted. I know now that I wont be wasting time learning Python. As with any language, it is about understanding the syntax. As I mentioned before, I want to make sure I am focusing my time on something useful and this was a big help. > > Here is what I came up with (with your help) . I expect there is a more efficient way to do it, but hey... it was my first try with data. And FYI, I work with over one hundred data sources, I wanted to test on something small. > > Phillip > > > with open("InputTest.txt","r") as f: > with open("Outputt.txt", "w") as fw: > for line in f: > first,second = line[:32], line[32:37] > > if first.isspace()== False: > fw.write (second.strip()+ first.strip()+"\n") > > f.close() > fw.close() > > > Some comments, first the minor stuff. No need to close f or fw; the with statements already took care of it before you got to those lines. And the if statement would normally be spelled if not first.isspace (): But the important thing is that by stripping those two strings, you're almost certainly damaging the output data. Unless you know some specific reason why you can get away with it. Also, please don't top-post. > > On Oct 7, 2014, at 4:39 AM, Alan Gauld <alan.gauld at btinternet.com> wrote: > >> On 06/10/14 23:42, Phillip Pugh wrote: >>> I am trying to decide if Python is the right toolset for me. >> > I do a lot of data analytics. > >> >>> move "structured" data around and then write it to a new file. >> >> What is structured about it? Fixed column width? >> Fixed relative position? Binary format? >> >>> The txt file has several data elements and is >> > 300 characters per line. >> >>> I am only interested in the first two fields. >> > The first data element is 19 characters. >> > The second data element is 6 characters. >> >> >>> I want to rearrange the data by moving the 6 characters data >> > in front of the 19 characters data >> >> Do you need a separator? >> >>> and then write the 25 character data to a new file. -- DaveA From robertvstepp at gmail.com Wed Oct 8 16:56:02 2014 From: robertvstepp at gmail.com (boB Stepp) Date: Wed, 8 Oct 2014 09:56:02 -0500 Subject: [Tutor] How best to structure a plain text data file for use in program(s) and later updating with new data? Message-ID: <CANDiX9L=t0K_Eba=isTJo6rUHrDs_GWuDL6XDp19VM3J6p9rBg@mail.gmail.com> About two years ago I wrote my most ambitious program to date, a hodge-podge collection of proprietary scripting, perl and shell files that collectively total about 20k lines of code. Amazingly it actually works and has saved my colleagues and I much time and effort. At the time I created this mess, I was playing "guess the correct proprietary syntax to do something" and "hunt and peck perl" games and squeezing this programming work into brief snippets of time away from what I am actually paid to do. I did not give much thought to design at the time and knew I would regret it later, which is now today! So now in my current few snippets of time I wish to redesign this program from scratch and make it much, ... , much easier to maintain the code and update the data tables, which change from time to time. And now that I have some version of python available on all of our current Solaris 10 systems (python versions 2.4.4 and 2.6.4), it seems like a fine time to (finally!) do some serious python learning. Right now I have separated my data into their own files. Previously I had integrated the data with my source code files (Horrors!). Currently, a snippet from one of these data files is: NUMBER_FX:ONE; DATA_SOURCE:Timmerman; RELEASE_DATE:(11-2012); SERIAL_ROI:Chiasm; TEST_VOLUME:< 0.2 cc; VOLUME_MAX_GY:8.0; MAX_PT_DOSE_GY:10.0; MAX_MEAN_DOSE: ; SERIAL_ROI:Optic_Nerve_R; TEST_VOLUME:< 0.2 cc; VOLUME_MAX_GY:8.0; MAX_PT_DOSE_GY:10.0; MAX_MEAN_DOSE: ; SERIAL_ROI:Optic_Nerve_L; TEST_VOLUME:< 0.2 cc; VOLUME_MAX_GY:8.0; MAX_PT_DOSE_GY:10.0; MAX_MEAN_DOSE: ; [...] PARALLEL_ROI:Lungs_Bilateral; CRITICAL_VOLUME_CC:1500.0; CRITICAL_VOLUME_DOSE_MAX_GY:7.0; V8GY: ; V20GY: ; MAX_MEAN_DOSE: ; PARALLEL_ROI:Lungs_Bilateral; CRITICAL_VOLUME_CC:1000.0; CRITICAL_VOLUME_DOSE_MAX_GY:7.6; V8GY:< 37.0%; V20GY: ; MAX_MEAN_DOSE: ; PARALLEL_ROI:Liver; CRITICAL_VOLUME_CC:700.0; CRITICAL_VOLUME_DOSE_MAX_GY:11.0; V8GY: ; V20GY: ; MAX_MEAN_DOSE: ; PARALLEL_ROI:Renal_Cortex_Bilateral; CRITICAL_VOLUME_CC:200.0; CRITICAL_VOLUME_DOSE_MAX_GY:9.5; V8GY: ; V20GY: ; MAX_MEAN_DOSE: ; [EOF] I just noticed that copying from my data file into my Google email resulted in all extra spaces being condensed into a single space. I do not know why this has just happened. Note that there are no tab characters. The [...] indicates omitted lines of serial tissue data and [EOF] just notes the end-of-file. I am far from ready to write any code at this point. I am trying to organize my data files, so that they will be easy to use by the programs that will process the data and also to be easily updated every time these data values get improved upon. For the latter, I envision writing a second program to enable anyone to update the data tables when we are given new values. But until that second program gets written, the data files would have to be opened and edited manually, which is why I have labels included in all-caps ending in a colon. This is so the editor will know what he is editing. So, basically the actual data fields fall between ":" and ";" . String representations of numbers will need to get converted to floats by the program. Some fields containing numbers are of a form like "< 0.2 cc" . These will get copied as is into a GUI display, while the "0.2" will be used in a computation and/or comparison. Also notice that in each data file there are two distinct groupings of records--one for serial tissue (SERIAL_ROI:) and one for parallel tissue (PARALLEL_ROI). The fields used are different for each grouping. Also, notice that some fields will have no values, but in other data files they will have values. And finally the header line at the top of the file identifies for what number of fractions (FX) the data is to be used for as well as the source of the data and date that the data was released by that source. Finally the questions! Will I easily be able to use python to parse this data as currently structured, or do I need to restructure this? I am not at the point where I am aware of what possibilities python offers to handle these data files. Also, my efforts to search the 'net did not turn up anything that really clicked for me as the way to go. I could not seem to come up with a search string that would bring up what I was really interested in: What are the best practices for organizing plain text data? Thanks! -- boB From joel.goldstick at gmail.com Wed Oct 8 17:02:42 2014 From: joel.goldstick at gmail.com (Joel Goldstick) Date: Wed, 8 Oct 2014 11:02:42 -0400 Subject: [Tutor] How best to structure a plain text data file for use in program(s) and later updating with new data? In-Reply-To: <CANDiX9L=t0K_Eba=isTJo6rUHrDs_GWuDL6XDp19VM3J6p9rBg@mail.gmail.com> References: <CANDiX9L=t0K_Eba=isTJo6rUHrDs_GWuDL6XDp19VM3J6p9rBg@mail.gmail.com> Message-ID: <CAPM-O+ziugLyTda5a2xR9oNC+O557d2O=tnP4N5o_aZFSwjWuQ@mail.gmail.com> On Wed, Oct 8, 2014 at 10:56 AM, boB Stepp <robertvstepp at gmail.com> wrote: > About two years ago I wrote my most ambitious program to date, a > hodge-podge collection of proprietary scripting, perl and shell files > that collectively total about 20k lines of code. Amazingly it actually > works and has saved my colleagues and I much time and effort. At the > time I created this mess, I was playing "guess the correct proprietary > syntax to do something" and "hunt and peck perl" games and squeezing > this programming work into brief snippets of time away from what I am > actually paid to do. I did not give much thought to design at the time > and knew I would regret it later, which is now today! So now in my > current few snippets of time I wish to redesign this program from > scratch and make it much, ... , much easier to maintain the code and > update the data tables, which change from time to time. And now that I > have some version of python available on all of our current Solaris 10 > systems (python versions 2.4.4 and 2.6.4), it seems like a fine time > to (finally!) do some serious python learning. > > Right now I have separated my data into their own files. Previously I > had integrated the data with my source code files (Horrors!). > Currently, a snippet from one of these data files is: > > NUMBER_FX:ONE; DATA_SOURCE:Timmerman; RELEASE_DATE:(11-2012); > > SERIAL_ROI:Chiasm; TEST_VOLUME:< 0.2 cc; VOLUME_MAX_GY:8.0; > MAX_PT_DOSE_GY:10.0; MAX_MEAN_DOSE: ; > SERIAL_ROI:Optic_Nerve_R; TEST_VOLUME:< 0.2 cc; VOLUME_MAX_GY:8.0; > MAX_PT_DOSE_GY:10.0; MAX_MEAN_DOSE: ; > SERIAL_ROI:Optic_Nerve_L; TEST_VOLUME:< 0.2 cc; VOLUME_MAX_GY:8.0; > MAX_PT_DOSE_GY:10.0; MAX_MEAN_DOSE: ; > > [...] > > PARALLEL_ROI:Lungs_Bilateral; CRITICAL_VOLUME_CC:1500.0; > CRITICAL_VOLUME_DOSE_MAX_GY:7.0; V8GY: ; V20GY: ; MAX_MEAN_DOSE: ; > PARALLEL_ROI:Lungs_Bilateral; CRITICAL_VOLUME_CC:1000.0; > CRITICAL_VOLUME_DOSE_MAX_GY:7.6; V8GY:< 37.0%; V20GY: ; MAX_MEAN_DOSE: > ; > PARALLEL_ROI:Liver; CRITICAL_VOLUME_CC:700.0; > CRITICAL_VOLUME_DOSE_MAX_GY:11.0; V8GY: ; V20GY: ; MAX_MEAN_DOSE: ; > PARALLEL_ROI:Renal_Cortex_Bilateral; CRITICAL_VOLUME_CC:200.0; > CRITICAL_VOLUME_DOSE_MAX_GY:9.5; V8GY: ; V20GY: ; MAX_MEAN_DOSE: ; > [EOF] > > I just noticed that copying from my data file into my Google email > resulted in all extra spaces being condensed into a single space. I do > not know why this has just happened. Note that there are no tab > characters. The [...] indicates omitted lines of serial tissue data > and [EOF] just notes the end-of-file. > > I am far from ready to write any code at this point. I am trying to > organize my data files, so that they will be easy to use by the > programs that will process the data and also to be easily updated > every time these data values get improved upon. For the latter, I > envision writing a second program to enable anyone to update the data > tables when we are given new values. But until that second program > gets written, the data files would have to be opened and edited > manually, which is why I have labels included in all-caps ending in a > colon. This is so the editor will know what he is editing. So, > basically the actual data fields fall between ":" and ";" . String > representations of numbers will need to get converted to floats by the > program. Some fields containing numbers are of a form like "< 0.2 cc" > . These will get copied as is into a GUI display, while the "0.2" will > be used in a computation and/or comparison. Also notice that in each > data file there are two distinct groupings of records--one for serial > tissue (SERIAL_ROI:) and one for parallel tissue (PARALLEL_ROI). The > fields used are different for each grouping. Also, notice that some > fields will have no values, but in other data files they will have > values. And finally the header line at the top of the file identifies > for what number of fractions (FX) the data is to be used for as well > as the source of the data and date that the data was released by that > source. > > Finally the questions! Will I easily be able to use python to parse > this data as currently structured, or do I need to restructure this? I > am not at the point where I am aware of what possibilities python > offers to handle these data files. Also, my efforts to search the 'net > did not turn up anything that really clicked for me as the way to go. > I could not seem to come up with a search string that would bring up > what I was really interested in: What are the best practices for > organizing plain text data? > > Thanks! > > -- > boB > _______________________________________________ > Tutor maillist - Tutor at python.org > To unsubscribe or change subscription options: > https://mail.python.org/mailman/listinfo/tutor It looks like you have csv like data. Except you have a semicolon as a separator. Look at the csv module. That should work for you -- Joel Goldstick http://joelgoldstick.com From robertvstepp at gmail.com Wed Oct 8 17:47:36 2014 From: robertvstepp at gmail.com (boB Stepp) Date: Wed, 8 Oct 2014 10:47:36 -0500 Subject: [Tutor] How best to structure a plain text data file for use in program(s) and later updating with new data? In-Reply-To: <CAPM-O+ziugLyTda5a2xR9oNC+O557d2O=tnP4N5o_aZFSwjWuQ@mail.gmail.com> References: <CANDiX9L=t0K_Eba=isTJo6rUHrDs_GWuDL6XDp19VM3J6p9rBg@mail.gmail.com> <CAPM-O+ziugLyTda5a2xR9oNC+O557d2O=tnP4N5o_aZFSwjWuQ@mail.gmail.com> Message-ID: <CANDiX9+xZLWynJ0zWTFBBQHsdHnO4pNXm=BUBHAyQaKk=30QTw@mail.gmail.com> On Wed, Oct 8, 2014 at 10:02 AM, Joel Goldstick <joel.goldstick at gmail.com> wrote: [...] > It looks like you have csv like data. Except you have a semicolon as > a separator. Look at the csv module. That should work for you > Joel, will the labels (like SERIAL_ROI:) cause me difficulties? I will need to strip these off to get to the actual data. But when I implement a data editor later, these labels will be needed (I think.). I just now have located the documentation for python 2.4.4. It does not seem to be as friendly or easy to read as for the current version documentation. But I will persevere... -- boB From alan.gauld at btinternet.com Wed Oct 8 18:27:23 2014 From: alan.gauld at btinternet.com (Alan Gauld) Date: Wed, 08 Oct 2014 17:27:23 +0100 Subject: [Tutor] How best to structure a plain text data file for use in program(s) and later updating with new data? In-Reply-To: <CANDiX9+xZLWynJ0zWTFBBQHsdHnO4pNXm=BUBHAyQaKk=30QTw@mail.gmail.com> References: <CANDiX9L=t0K_Eba=isTJo6rUHrDs_GWuDL6XDp19VM3J6p9rBg@mail.gmail.com> <CAPM-O+ziugLyTda5a2xR9oNC+O557d2O=tnP4N5o_aZFSwjWuQ@mail.gmail.com> <CANDiX9+xZLWynJ0zWTFBBQHsdHnO4pNXm=BUBHAyQaKk=30QTw@mail.gmail.com> Message-ID: <m13olc$e6b$1@ger.gmane.org> On 08/10/14 16:47, boB Stepp wrote: >> It looks like you have csv like data. Except you have a semicolon as >> a separator. Look at the csv module. That should work for you >> > Joel, will the labels (like SERIAL_ROI:) cause me difficulties? I will > need to strip these off to get to the actual data. But when I > implement a data editor later, these labels will be needed (I think.). You will get the name:value fields as strings. You can then use string.split(':') to separate them and then either lose the label or convert them to a dictionary. If its not too big a task you could even convert the data structure to JSON which is quite a close match to what you have now and the json module will help you read/write to them. > I just now have located the documentation for python 2.4.4. It does > not seem to be as friendly or easy to read as for the current version > documentation. Shouldn't be that much different. What kind of anomalies are you seeing? -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.flickr.com/photos/alangauldphotos From aakazemaa at gmail.com Wed Oct 8 18:51:52 2014 From: aakazemaa at gmail.com (Kazem Ardekanian) Date: Wed, 8 Oct 2014 09:51:52 -0700 Subject: [Tutor] Welcome to the "Tutor" mailing list In-Reply-To: <mailman.0.1412786890.15411.tutor@python.org> References: <mailman.0.1412786890.15411.tutor@python.org> Message-ID: <CAFF6B_OY0rTENSD9MBmSRV4LHuVNX8n_uNCWdfnZT0-hPwBb3g@mail.gmail.com> Hi, I was reading IDLE tutorial. The following document dose not exist. http://www.python.org/idle/doc/ It was referenced from https://hkn.eecs.berkeley.edu/~dyoo/python/idle_intro/ Thanks, Kazem On Wed, Oct 8, 2014 at 9:48 AM, <tutor-request at python.org> wrote: > Welcome to the Tutor at python.org mailing list! This list is for folks > who want to ask (and/or answer) questions from folks who wish to learn > how to program with Python. Feel free to ask even the most basic of > questions -- that's what the list is for! > > For best results when asking a question on this list: - Try to write > some code to solve your problem - Show the code you have written - > Describe what the code does and what you want it to do - If the code > generates an error, copy and paste the entire error message, including > the traceback, into your email. - Tell us what OS and Python version > you are using. > > - Don't ask us to do your homework. - Don't assume we know what you > are talking about. If you are having trouble with a third-party > library, include a link to the library home page. > > When replying to a posting: - Use Reply All to reply to the entire > list - Don't top post - put your reply after the text to which you are > replying > > For all posts: - Format your email as plain text, not HTML > > > To post to this list, send your message to: > > tutor at python.org > > General information about the mailing list is at: > > https://mail.python.org/mailman/listinfo/tutor > > If you ever want to unsubscribe or change your options (eg, switch to > or from digest mode, change your password, etc.), visit your > subscription page at: > > https://mail.python.org/mailman/options/tutor/aakazemaa%40gmail.com > > You can also make such adjustments via email by sending a message to: > > Tutor-request at python.org > > with the word `help' in the subject or body (don't include the > quotes), and you will get back a message with instructions. > > You must know your password to change your options (including changing > the password, itself) or to unsubscribe without confirmation. It is: > > 123zak > > Normally, Mailman will remind you of your python.org mailing list > passwords once every month, although you can disable this if you > prefer. This reminder will also include instructions on how to > unsubscribe or change your account options. There is also a button on > your options page that will email your current password to you. > -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/tutor/attachments/20141008/b793af93/attachment-0001.html> From robertvstepp at gmail.com Wed Oct 8 19:51:08 2014 From: robertvstepp at gmail.com (boB Stepp) Date: Wed, 8 Oct 2014 12:51:08 -0500 Subject: [Tutor] How best to structure a plain text data file for use in program(s) and later updating with new data? In-Reply-To: <m13olc$e6b$1@ger.gmane.org> References: <CANDiX9L=t0K_Eba=isTJo6rUHrDs_GWuDL6XDp19VM3J6p9rBg@mail.gmail.com> <CAPM-O+ziugLyTda5a2xR9oNC+O557d2O=tnP4N5o_aZFSwjWuQ@mail.gmail.com> <CANDiX9+xZLWynJ0zWTFBBQHsdHnO4pNXm=BUBHAyQaKk=30QTw@mail.gmail.com> <m13olc$e6b$1@ger.gmane.org> Message-ID: <CANDiX9LC0bEHoYojKBz7qmASQFPdRKRJ99cPE0jtOdXTyMr1Fw@mail.gmail.com> On Wed, Oct 8, 2014 at 11:27 AM, Alan Gauld <alan.gauld at btinternet.com> wrote: > On 08/10/14 16:47, boB Stepp wrote: > >>> It looks like you have csv like data. Except you have a semicolon as >>> a separator. Look at the csv module. That should work for you >>> >> Joel, will the labels (like SERIAL_ROI:) cause me difficulties? I will >> need to strip these off to get to the actual data. But when I >> implement a data editor later, these labels will be needed (I think.). > > > You will get the name:value fields as strings. > You can then use string.split(':') to separate them and > then either lose the label or convert them to a dictionary. Ah! Thanks. That should work well. > If its not too big a task you could even convert the data > structure to JSON which is quite a close match to what you > have now and the json module will help you read/write > to them. I am not seeing JSON listed among python's standard libraries for version 2.4.4. Is this something that has to be independently installed? >> I just now have located the documentation for python 2.4.4. It does >> not seem to be as friendly or easy to read as for the current version >> documentation. > > > Shouldn't be that much different. > What kind of anomalies are you seeing? Compare the current library index, https://docs.python.org/2/library/index.html, with the 2.4.4 one, https://docs.python.org/release/2.4.4/lib/lib.html . On the former I was able to immediately find the csv module with a quick scroll, but in the latter I found myself using my browser's find function to locate the link. The other improvement that the newer presentation has that the older does not, is an actual brief example along with the description in the module contents. Perhaps these are minor points... -- boB From martin at linux-ip.net Wed Oct 8 21:19:11 2014 From: martin at linux-ip.net (Martin A. Brown) Date: Wed, 8 Oct 2014 12:19:11 -0700 Subject: [Tutor] How best to structure a plain text data file for use in program(s) and later updating with new data? In-Reply-To: <CANDiX9LC0bEHoYojKBz7qmASQFPdRKRJ99cPE0jtOdXTyMr1Fw@mail.gmail.com> References: <CANDiX9L=t0K_Eba=isTJo6rUHrDs_GWuDL6XDp19VM3J6p9rBg@mail.gmail.com> <CAPM-O+ziugLyTda5a2xR9oNC+O557d2O=tnP4N5o_aZFSwjWuQ@mail.gmail.com> <CANDiX9+xZLWynJ0zWTFBBQHsdHnO4pNXm=BUBHAyQaKk=30QTw@mail.gmail.com> <m13olc$e6b$1@ger.gmane.org> <CANDiX9LC0bEHoYojKBz7qmASQFPdRKRJ99cPE0jtOdXTyMr1Fw@mail.gmail.com> Message-ID: <alpine.LNX.2.00.1410081207210.1349@dagger.wonderfrog.net> Good afternoon, >> If its not too big a task you could even convert the data >> structure to JSON which is quite a close match to what you >> have now and the json module will help you read/write >> to them. I would agree with the JSON recommendation (until your data set grows to more than 10GB in size). Also, if you are operating with JSON, you can add elements and delete elements and the serialization bits don't care. This makes it more dynamic than a typical csv/tsv format. The application still has to know about the data type, of course. > I am not seeing JSON listed among python's standard libraries for > version 2.4.4. Is this something that has to be independently > installed? Yes. The json module was 'only' added in Python 2.6. If you wanted to operate on JSON in Python 2.4, you had to use something called simplejson [0]. According to this post [0], the standard library json module is a(n older) release of the simplejson module. Anyway, when I found myself stuck in Python-2.4 land (on the stock Python shipped with CentOS-5.x, for example), I often saw and wrote snippets like this [1]: try: # -- if Python-2.6+, it's in STDLIB import json except ImportError: # -- Python-2.4, simplejson import simplejson as json And, then the rest of the program can operate just the same, regardless of which one you used. Good luck, -Martin [0] https://pypi.python.org/pypi/simplejson/ [1] http://stackoverflow.com/questions/712791/what-are-the-differences-between-json-and-simplejson-python-modules -- Martin A. Brown http://linux-ip.net/ From danny.yoo at gmail.com Wed Oct 8 21:16:47 2014 From: danny.yoo at gmail.com (Danny Yoo) Date: Wed, 8 Oct 2014 12:16:47 -0700 Subject: [Tutor] Welcome to the "Tutor" mailing list In-Reply-To: <CAFF6B_OY0rTENSD9MBmSRV4LHuVNX8n_uNCWdfnZT0-hPwBb3g@mail.gmail.com> References: <mailman.0.1412786890.15411.tutor@python.org> <CAFF6B_OY0rTENSD9MBmSRV4LHuVNX8n_uNCWdfnZT0-hPwBb3g@mail.gmail.com> Message-ID: <CAGZAPF7oCKfaRRMdayNrr_QtZxO4yyGZuyzffm07GYP=C8HWNw@mail.gmail.com> > I was reading IDLE tutorial. > The following document dose not exist. > http://www.python.org/idle/doc/ > > It was referenced from https://hkn.eecs.berkeley.edu/~dyoo/python/idle_intro/ > Ok. I'll correct the broken link as soon as I have time. Thanks. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/tutor/attachments/20141008/be36775d/attachment.html> From dyoo at hashcollision.org Wed Oct 8 22:45:46 2014 From: dyoo at hashcollision.org (Danny Yoo) Date: Wed, 8 Oct 2014 13:45:46 -0700 Subject: [Tutor] Welcome to the "Tutor" mailing list In-Reply-To: <CAGZAPF7oCKfaRRMdayNrr_QtZxO4yyGZuyzffm07GYP=C8HWNw@mail.gmail.com> References: <mailman.0.1412786890.15411.tutor@python.org> <CAFF6B_OY0rTENSD9MBmSRV4LHuVNX8n_uNCWdfnZT0-hPwBb3g@mail.gmail.com> <CAGZAPF7oCKfaRRMdayNrr_QtZxO4yyGZuyzffm07GYP=C8HWNw@mail.gmail.com> Message-ID: <CAGZAPF6o8hAQVM972VGUpvOCmDBrmwi83gQdn3ium_0c61qVwg@mail.gmail.com> On Wed, Oct 8, 2014 at 12:16 PM, Danny Yoo <danny.yoo at gmail.com> wrote: > >> I was reading IDLE tutorial. >> The following document dose not exist. >> http://www.python.org/idle/doc/ >> >> It was referenced from >> https://hkn.eecs.berkeley.edu/~dyoo/python/idle_intro/ >> > > Ok. I'll correct the broken link as soon as I have time. Thanks. Link should be corrected now to: https://docs.python.org/2/library/idle.html. From robertvstepp at gmail.com Wed Oct 8 23:07:39 2014 From: robertvstepp at gmail.com (boB Stepp) Date: Wed, 8 Oct 2014 16:07:39 -0500 Subject: [Tutor] How best to structure a plain text data file for use in program(s) and later updating with new data? In-Reply-To: <alpine.LNX.2.00.1410081207210.1349@dagger.wonderfrog.net> References: <CANDiX9L=t0K_Eba=isTJo6rUHrDs_GWuDL6XDp19VM3J6p9rBg@mail.gmail.com> <CAPM-O+ziugLyTda5a2xR9oNC+O557d2O=tnP4N5o_aZFSwjWuQ@mail.gmail.com> <CANDiX9+xZLWynJ0zWTFBBQHsdHnO4pNXm=BUBHAyQaKk=30QTw@mail.gmail.com> <m13olc$e6b$1@ger.gmane.org> <CANDiX9LC0bEHoYojKBz7qmASQFPdRKRJ99cPE0jtOdXTyMr1Fw@mail.gmail.com> <alpine.LNX.2.00.1410081207210.1349@dagger.wonderfrog.net> Message-ID: <CANDiX9+yQHEgwFhaYCiWbThaocmBbp1y34ke=9PGMRJp0ZdMJg@mail.gmail.com> On Wed, Oct 8, 2014 at 2:19 PM, Martin A. Brown <martin at linux-ip.net> wrote: > > Good afternoon, > >>> If its not too big a task you could even convert the data >>> structure to JSON which is quite a close match to what you >>> have now and the json module will help you read/write >>> to them. Looking at some examples of JSON-structured data, I see that I formatted my data very similarly, as Alan noted. Purely a lucky coincidence! > > I would agree with the JSON recommendation (until your data set grows to > more than 10GB in size). Also, if you are operating with JSON, you can add > elements and delete elements and the serialization bits don't care. This > makes it more dynamic than a typical csv/tsv format. The application still > has to know about the data type, of course. > My data files are rather small and should remain so, about 30 to 40 lines at the most. Definitely forever less than 100 lines for each file. This is because there are only so many human organs in the vicinity of our radiation treatments that we might want to track dose for. >> I am not seeing JSON listed among python's standard libraries for version >> 2.4.4. Is this something that has to be independently installed? > > > Yes. The json module was 'only' added in Python 2.6. If you wanted to > operate on JSON in Python 2.4, you had to use something called simplejson > [0]. According to this post [0], the standard library json module is a(n > older) release of the simplejson module. > I looked at [0] and it states that it is backwards compatible to version 2.5, which is one iteration higher than my lowest installed python version at work. They provide a link to an older simplejson that is from the python 2.2 era, but state that this should be done only as a last resort. > Anyway, when I found myself stuck in Python-2.4 land (on the stock Python > shipped with CentOS-5.x, for example), I often saw and wrote snippets like > this [1]: > > try: # -- if Python-2.6+, it's in STDLIB > import json > except ImportError: # -- Python-2.4, simplejson > import simplejson as json > > And, then the rest of the program can operate just the same, regardless of > which one you used. > Is there a typo on [0]? Will it in fact work for python 2.4? > > [0] https://pypi.python.org/pypi/simplejson/ > [1] > http://stackoverflow.com/questions/712791/what-are-the-differences-between-json-and-simplejson-python-modules > -- boB From martin at linux-ip.net Wed Oct 8 23:53:07 2014 From: martin at linux-ip.net (Martin A. Brown) Date: Wed, 8 Oct 2014 14:53:07 -0700 Subject: [Tutor] How best to structure a plain text data file for use in program(s) and later updating with new data? In-Reply-To: <CANDiX9+yQHEgwFhaYCiWbThaocmBbp1y34ke=9PGMRJp0ZdMJg@mail.gmail.com> References: <CANDiX9L=t0K_Eba=isTJo6rUHrDs_GWuDL6XDp19VM3J6p9rBg@mail.gmail.com> <CAPM-O+ziugLyTda5a2xR9oNC+O557d2O=tnP4N5o_aZFSwjWuQ@mail.gmail.com> <CANDiX9+xZLWynJ0zWTFBBQHsdHnO4pNXm=BUBHAyQaKk=30QTw@mail.gmail.com> <m13olc$e6b$1@ger.gmane.org> <CANDiX9LC0bEHoYojKBz7qmASQFPdRKRJ99cPE0jtOdXTyMr1Fw@mail.gmail.com> <alpine.LNX.2.00.1410081207210.1349@dagger.wonderfrog.net> <CANDiX9+yQHEgwFhaYCiWbThaocmBbp1y34ke=9PGMRJp0ZdMJg@mail.gmail.com> Message-ID: <alpine.LNX.2.00.1410081448360.1349@dagger.wonderfrog.net> Good afternoon again, > I looked at [0] and it states that it is backwards compatible to > version 2.5, which is one iteration higher than my lowest installed > python version at work. They provide a link to an older simplejson > that is from the python 2.2 era, but state that this should be done > only as a last resort. > Is there a typo on [0]? Will it in fact work for python 2.4? Probably not. I don't recall what version of simplejson I used with Python-2.4. Note: If they provide a version that will work back to Python-2.2 and the next option is something that only works with Python-2.5+, then the obvious choice is to use the old version of software. The developers of simplejson (and most developers) will probably not want to support very old releases of their software, which is why they suggest using this as a last resort. If you can install or use a newer Python, then maybe that's a better option for you. If you can not do so, then take this older version of simplejson. You are at that place of last resort to which the simplejson authors allude. How do you like it at that resort? Would I want to go on vacation there? -Martin -- Martin A. Brown http://linux-ip.net/ From felisha.lawrence at gmail.com Thu Oct 9 01:58:38 2014 From: felisha.lawrence at gmail.com (Felisha Lawrence) Date: Wed, 8 Oct 2014 19:58:38 -0400 Subject: [Tutor] Renaming Files in Directory Message-ID: <CALcsL=GbNKcujrSZ69Zfj-AHonh+fPgkgUqKMsFkpqtuQSPvqw@mail.gmail.com> Hello, I have the following program import os path = '/Users/felishalawrence/testswps/vol1' for file in os.listdir(path): newFile = path+file[:file.rindex("v")]+"v20" print newFile and I want to output the results of the 'newFile' variable into the directory specified by the 'path' variable. There are current files in this directory, but I would like tho replace them with different names. Can anyone point me in the right direction? Thanks, Felisha Lawrence -- Felisha Lawrence Howard University Program for Atmospheric Sciences(HUPAS), Graduate Student NASA URC/BCCSO Graduate Fellow NOAA NCAS Graduate Fellow Graduate Student Association for Atmospheric Sciences(GSAAS), Treasurer (240)-535-6665 (cell) felisha.lawrence at gmail.com (email) -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/tutor/attachments/20141008/9c11308a/attachment.html> From __peter__ at web.de Thu Oct 9 11:36:18 2014 From: __peter__ at web.de (Peter Otten) Date: Thu, 09 Oct 2014 11:36:18 +0200 Subject: [Tutor] Renaming Files in Directory References: <CALcsL=GbNKcujrSZ69Zfj-AHonh+fPgkgUqKMsFkpqtuQSPvqw@mail.gmail.com> Message-ID: <m15kuj$85h$1@ger.gmane.org> Felisha Lawrence wrote: > I have the following program > import os > > path = '/Users/felishalawrence/testswps/vol1' > for file in os.listdir(path): > newFile = path+file[:file.rindex("v")]+"v20" > > print newFile > and I want to output the results of the 'newFile' variable into the > directory specified by the 'path' variable. There are current files in > this directory, but I would like tho replace them with different names. > Can anyone point me in the right direction? So you can no longer defer that dreadful task? ;) To rename a file you need its old and its new name, both preferrably with their complete path, for example for old_name in os.listdir(path): old_file = os.path.join(path, old_name) new_name = old_name[:old_name.rindex("v")] + "v20" new_file = os.path.join(path, new_name) os.rename(old_file, new_file) If there are only files containing a "v" in your /Users/felishalawrence/testswps/vol1 folder the above should already work. Here are a few ideas to make it more robust (or rather less brittle): Verify that old_name contains a "v" using the `in` operator Verify that old_file is a file using os.path.isfile() Verify that new_file doesn't exist with os.path.exists() From alan.gauld at btinternet.com Thu Oct 9 13:02:34 2014 From: alan.gauld at btinternet.com (Alan Gauld) Date: Thu, 09 Oct 2014 12:02:34 +0100 Subject: [Tutor] Renaming Files in Directory In-Reply-To: <CALcsL=GbNKcujrSZ69Zfj-AHonh+fPgkgUqKMsFkpqtuQSPvqw@mail.gmail.com> References: <CALcsL=GbNKcujrSZ69Zfj-AHonh+fPgkgUqKMsFkpqtuQSPvqw@mail.gmail.com> Message-ID: <m15q0a$7i1$1@ger.gmane.org> On 09/10/14 00:58, Felisha Lawrence wrote: > Hello, > I have the following program > > > import os > > path = '/Users/felishalawrence/testswps/vol1' > for file in os.listdir(path): > newFile = path+file[:file.rindex("v")]+"v20" > > print newFile > > and I want to output the results of the 'newFile' variable into the > directory specified by the 'path' variable. You want the os.rename function. Also you should use os.path.join() to create the path rather than string addition. It will ensure the correct separators are used for the OS. You might also want to look at glob.glob() rather than listdir to get a listing of files matching a wildcard pattern. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.flickr.com/photos/alangauldphotos From fomcl at yahoo.com Thu Oct 9 20:32:01 2014 From: fomcl at yahoo.com (Albert-Jan Roskam) Date: Thu, 9 Oct 2014 18:32:01 +0000 (UTC) Subject: [Tutor] alternative Python 2.6 install? Message-ID: <1529317176.81747.1412879521526.JavaMail.yahoo@jws10743.mail.gq1.yahoo.com> Hi, I need to install Python 2.6 on my Debian system to check some code.*) What is the easiest way to do this? Simply "sudo apt-get install python2.6"? I know I can also compile it and then do make altinstall, but I prefer apt-get. I am kinda paranoid that I might accidentally change my system Python version. Thank you! Regards, Albert-Jan *) albertjan at debian:~$ uname -a Linux debian 3.2.0-4-amd64 #1 SMP Debian 3.2.60-1+deb7u3 x86_64 GNU/Linux albertjan at debian:~$ python -c "import sys; print sys.version_info" sys.version_info(major=2, minor=7, micro=3, releaselevel='final', serial=0) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ All right, but apart from the sanitation, the medicine, education, wine, public order, irrigation, roads, a fresh water system, and public health, what have the Romans ever done for us? ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ From wbecerra1 at gmail.com Thu Oct 9 20:38:02 2014 From: wbecerra1 at gmail.com (William Becerra) Date: Thu, 9 Oct 2014 20:38:02 +0200 Subject: [Tutor] (no subject) Message-ID: <CAF1DC4VdVnjRNxuf5gmRseS2bEv2TLVN6Vb2Qrn=Tiax3CcaQQ@mail.gmail.com> I'm new to programming. Started reading the book 'How to think like a computer Scientist-learning with python'. I'm now in chapter 3 sub-chapter 3.4 Math functions. When I write the following code: import maths; decibel = math.log10 (17.0); angle = 1.5; height = math.sin(angle); print height; I get the following error: Traceback (most recent call last): File "C:/Python27/test", line 1, in <module> import maths; ImportError: No module named maths I don't know what I'm doing wrong? >From what I've read the maths module is supposed to come with the python installation package. I'm using a windows 8 operating system python 2.7.8 please help? -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/tutor/attachments/20141009/5670d430/attachment.html> From joel.goldstick at gmail.com Fri Oct 10 02:03:23 2014 From: joel.goldstick at gmail.com (Joel Goldstick) Date: Thu, 9 Oct 2014 20:03:23 -0400 Subject: [Tutor] (no subject) In-Reply-To: <CAF1DC4VdVnjRNxuf5gmRseS2bEv2TLVN6Vb2Qrn=Tiax3CcaQQ@mail.gmail.com> References: <CAF1DC4VdVnjRNxuf5gmRseS2bEv2TLVN6Vb2Qrn=Tiax3CcaQQ@mail.gmail.com> Message-ID: <CAPM-O+w6=NrkYw6uNtAYkfttot+Ot3x1taF3oa5ew0kEWYDSPw@mail.gmail.com> On Oct 9, 2014 8:00 PM, "William Becerra" <wbecerra1 at gmail.com> wrote: > > I'm new to programming. Started reading the book 'How to think like a computer Scientist-learning with python'. I'm now in chapter 3 sub-chapter 3.4 Math functions. > > When I write the following code: > > import maths; import math You added s > decibel = math.log10 (17.0); > angle = 1.5; > height = math.sin(angle); > print height; > > I get the following error: > > Traceback (most recent call last): > File "C:/Python27/test", line 1, in <module> > import maths; > ImportError: No module named maths > > I don't know what I'm doing wrong? > From what I've read the maths module is supposed to come with the python installation package. > I'm using a windows 8 operating system > python 2.7.8 > please help? > > > > > _______________________________________________ > Tutor maillist - Tutor at python.org > To unsubscribe or change subscription options: > https://mail.python.org/mailman/listinfo/tutor > -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/tutor/attachments/20141009/787fbf3e/attachment.html> From martin at linux-ip.net Fri Oct 10 02:13:51 2014 From: martin at linux-ip.net (Martin A. Brown) Date: Thu, 9 Oct 2014 17:13:51 -0700 Subject: [Tutor] (no subject) In-Reply-To: <CAF1DC4VdVnjRNxuf5gmRseS2bEv2TLVN6Vb2Qrn=Tiax3CcaQQ@mail.gmail.com> References: <CAF1DC4VdVnjRNxuf5gmRseS2bEv2TLVN6Vb2Qrn=Tiax3CcaQQ@mail.gmail.com> Message-ID: <alpine.LNX.2.00.1410091704080.1349@dagger.wonderfrog.net> Hi there and welcome! > import maths; > decibel = math.log10 (17.0); > angle = 1.5; > height = math.sin(angle); > print height; > > Traceback (most recent call last): > File "C:/Python27/test", line 1, in <module> > import maths; > ImportError: No module named maths Oops! It's a nice error report, though! Python tried to locate a module called 'maths' and was not able to find it. What happens if you try: import math N.B. You say 'import maths'--assume that this import succeeded. A few lines later, there's a line 'math.log10(17.0)' which seems to be trying to use something from a module called 'math' not 'maths'. > I don't know what I'm doing wrong? Computers are so picky. > From what I've read the maths module is supposed to come with the python > installation package. The 'math' library is a standard library module for quite awhile now. Here's a possibly useful online link, which describes that module: https://docs.python.org/2/library/math.html This is just more documentation support, in addition to the book you are reading. > I'm using a windows 8 operating system > python 2.7.8 > please help? One other issue I might point out. The semicolon at the end of the line (statement) is a feature of other programming languages with which you may be familiar (C, Java, Perl), but it is not necessary and, in fact, discouraged in Python. So, rid yourself of the semicolons and enjoy the benefits of a trivially cleaner syntax. Enjoy! -Martin P.S. Thanks for your clear question and letting us know your OS and Python version, as well. -- Martin A. Brown http://linux-ip.net/ From alan.gauld at btinternet.com Fri Oct 10 02:36:25 2014 From: alan.gauld at btinternet.com (Alan Gauld) Date: Fri, 10 Oct 2014 01:36:25 +0100 Subject: [Tutor] (no subject) In-Reply-To: <CAF1DC4VdVnjRNxuf5gmRseS2bEv2TLVN6Vb2Qrn=Tiax3CcaQQ@mail.gmail.com> References: <CAF1DC4VdVnjRNxuf5gmRseS2bEv2TLVN6Vb2Qrn=Tiax3CcaQQ@mail.gmail.com> Message-ID: <m179m9$clv$1@ger.gmane.org> On 09/10/14 19:38, William Becerra wrote: > import maths; Python, like most languages speaks American English so its math not maths. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.flickr.com/photos/alangauldphotos From robertvstepp at gmail.com Fri Oct 10 05:57:24 2014 From: robertvstepp at gmail.com (boB Stepp) Date: Thu, 9 Oct 2014 22:57:24 -0500 Subject: [Tutor] Installing both Python 2.7 and Python 3.4 on Windows 7 Pro 64-bit: Install Python 2.7 FIRST! Message-ID: <CANDiX9LG2y7XNGLsruqjNtY7GZYN5VWEm-E2sha1o0LLqB2Xsw@mail.gmail.com> I am hoping to save other people the grief I just worked through. I wanted to run both Python 2 and 3 on my windows PC, and, after googling this topic found that with Python 3.3 or later one could easily do both. So I merrily installed Python 3.4.2 first and then Python 2.7.8. A Python 3 program that had been working fine suddenly stopped working. After working down to a test portion of code that isolated the culprit I realized my Python 3 program was being interpreted by Python 2. I soon found that installing Python 2 first and then 3 enabled both to happily coexist. If there was a mention about the order of installation anywhere during my searches, I missed it. Anyway, I hope that my experience helps some other newbie who wants to play around with both major versions. -- boB From wbecerra1 at gmail.com Fri Oct 10 05:55:38 2014 From: wbecerra1 at gmail.com (William Becerra) Date: Fri, 10 Oct 2014 05:55:38 +0200 Subject: [Tutor] (no subject) In-Reply-To: <m179m9$clv$1@ger.gmane.org> References: <CAF1DC4VdVnjRNxuf5gmRseS2bEv2TLVN6Vb2Qrn=Tiax3CcaQQ@mail.gmail.com> <m179m9$clv$1@ger.gmane.org> Message-ID: <CAF1DC4UDEg2F7uOZdL-d_GvjAO0=3JuKohuoarFuXC8_1LJFUQ@mail.gmail.com> It is working now. Thank you everyone. It was very helpfull. On Fri, Oct 10, 2014 at 2:36 AM, Alan Gauld <alan.gauld at btinternet.com> wrote: > On 09/10/14 19:38, William Becerra wrote: > > import maths; >> > > Python, like most languages speaks American English > so its math not maths. > > -- > Alan G > Author of the Learn to Program web site > http://www.alan-g.me.uk/ > http://www.flickr.com/photos/alangauldphotos > > > _______________________________________________ > Tutor maillist - Tutor at python.org > To unsubscribe or change subscription options: > https://mail.python.org/mailman/listinfo/tutor > -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/tutor/attachments/20141010/40100121/attachment.html> From ljetibo at gmail.com Fri Oct 10 09:24:41 2014 From: ljetibo at gmail.com (=?ISO-8859-2?Q?Dino_Bekte=B9evi=E6?=) Date: Fri, 10 Oct 2014 09:24:41 +0200 Subject: [Tutor] Installing both Python 2.7 and Python 3.4 on Windows 7 Pro 64-bit: Install Python 2.7 FIRST! (boB Stepp) Message-ID: <CAMGeA2WncEg=FNL+oJDemMi+OUXJut2kaMAioF5o=6Yjj=rBDg@mail.gmail.com> In the lieu of the same thing. I did an install of both python 2.6 and 3.3 on Win7. The key was to start using virtualenv (http://docs.python-guide.org/en/latest/dev/virtualenvs/) and set it up to the python install of my choice with: virtualenv -p /usr/bin/python2.7 venv although virtualenv has somewhat mixed reviews, some say it's good some don't really like it, I find that it makes handling of python versions really easy and I do recommend it. Upsides are: clean enviroment every time, you get to use pip even in Win. Downsides: sometimes is a bugger to install. You want to isolate each virtualenv by not using local packages, that means for each virtual env you want to install all the packages from scratch, but some packages don't really want to install easily in virtualenv (matplotlib I'm looking at you!). The --relocatable flag to can be a real nightmare, don't know if they fixed it as of recently.... As far as it goes without virtualenv, it's best to structure your code so that it's terminal friendly even in Win, and then start each with C:\python2.6\python program.py or whatever your full path may be. > ------------------------------ > > Message: 7 > Date: Thu, 9 Oct 2014 22:57:24 -0500 > From: boB Stepp <robertvstepp at gmail.com> > To: tutor <tutor at python.org> > Subject: [Tutor] Installing both Python 2.7 and Python 3.4 on Windows > 7 Pro 64-bit: Install Python 2.7 FIRST! > Message-ID: > <CANDiX9LG2y7XNGLsruqjNtY7GZYN5VWEm-E2sha1o0LLqB2Xsw at mail.gmail.com> > Content-Type: text/plain; charset=UTF-8 > > I am hoping to save other people the grief I just worked through. I > wanted to run both Python 2 and 3 on my windows PC, and, after > googling this topic found that with Python 3.3 or later one could > easily do both. So I merrily installed Python 3.4.2 first and then > Python 2.7.8. A Python 3 program that had been working fine suddenly > stopped working. After working down to a test portion of code that > isolated the culprit I realized my Python 3 program was being > interpreted by Python 2. I soon found that installing Python 2 first > and then 3 enabled both to happily coexist. If there was a mention > about the order of installation anywhere during my searches, I missed > it. Anyway, I hope that my experience helps some other newbie who > wants to play around with both major versions. > > -- > boB > > > ------------------------------ > > Subject: Digest Footer > > _______________________________________________ > Tutor maillist - Tutor at python.org > https://mail.python.org/mailman/listinfo/tutor > > > ------------------------------ > > End of Tutor Digest, Vol 128, Issue 22 > ************************************** From wolfgang.maier at biologie.uni-freiburg.de Fri Oct 10 09:05:35 2014 From: wolfgang.maier at biologie.uni-freiburg.de (Wolfgang Maier) Date: Fri, 10 Oct 2014 09:05:35 +0200 Subject: [Tutor] Installing both Python 2.7 and Python 3.4 on Windows 7 Pro 64-bit: Install Python 2.7 FIRST! In-Reply-To: <CANDiX9LG2y7XNGLsruqjNtY7GZYN5VWEm-E2sha1o0LLqB2Xsw@mail.gmail.com> References: <CANDiX9LG2y7XNGLsruqjNtY7GZYN5VWEm-E2sha1o0LLqB2Xsw@mail.gmail.com> Message-ID: <5437853F.5050309@biologie.uni-freiburg.de> On 10/10/2014 05:57 AM, boB Stepp wrote: > I am hoping to save other people the grief I just worked through. I > wanted to run both Python 2 and 3 on my windows PC, and, after > googling this topic found that with Python 3.3 or later one could > easily do both. So I merrily installed Python 3.4.2 first and then > Python 2.7.8. A Python 3 program that had been working fine suddenly > stopped working. After working down to a test portion of code that > isolated the culprit I realized my Python 3 program was being > interpreted by Python 2. It would help if you could share details about how you tried to run the Python 3 program (command line call, double-click, Python launcher for Windows, ..). > I soon found that installing Python 2 first > and then 3 enabled both to happily coexist. If there was a mention > about the order of installation anywhere during my searches, I missed > it. Anyway, I hope that my experience helps some other newbie who > wants to play around with both major versions. > From henry at sa-hk.com Fri Oct 10 10:32:09 2014 From: henry at sa-hk.com (Henry) Date: Fri, 10 Oct 2014 16:32:09 +0800 Subject: [Tutor] Tutor Digest, Vol 128, Issue 22 In-Reply-To: <mailman.85151.1412913889.18129.tutor@python.org> References: <mailman.85151.1412913889.18129.tutor@python.org> Message-ID: <CAC7HCj-q7N3Ox7eb1URRGXFUv7pzHrzDwe0YLrjM6NzR9g7hZw@mail.gmail.com> Hi I am new to programming. After I created two text files(the text file is most consist of numbers), its class is "<class '_io.TextIOWrapper'>", how can I compare this class with two text files? Please give me a hint which area I should look under? Set? List? Thanks Henry On Fri, Oct 10, 2014 at 12:04 PM, <tutor-request at python.org> wrote: > Send Tutor mailing list submissions to > tutor at python.org > > To subscribe or unsubscribe via the World Wide Web, visit > https://mail.python.org/mailman/listinfo/tutor > or, via email, send a message with subject or body 'help' to > tutor-request at python.org > > You can reach the person managing the list at > tutor-owner at python.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of Tutor digest..." > > > Today's Topics: > > 1. Re: Renaming Files in Directory (Alan Gauld) > 2. alternative Python 2.6 install? (Albert-Jan Roskam) > 3. (no subject) (William Becerra) > 4. Re: (no subject) (Joel Goldstick) > 5. Re: (no subject) (Martin A. Brown) > 6. Re: (no subject) (Alan Gauld) > 7. Installing both Python 2.7 and Python 3.4 on Windows 7 Pro > 64-bit: Install Python 2.7 FIRST! (boB Stepp) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Thu, 09 Oct 2014 12:02:34 +0100 > From: Alan Gauld <alan.gauld at btinternet.com> > To: tutor at python.org > Subject: Re: [Tutor] Renaming Files in Directory > Message-ID: <m15q0a$7i1$1 at ger.gmane.org> > Content-Type: text/plain; charset=windows-1252; format=flowed > > On 09/10/14 00:58, Felisha Lawrence wrote: > > Hello, > > I have the following program > > > > > > import os > > > > path = '/Users/felishalawrence/testswps/vol1' > > for file in os.listdir(path): > > newFile = path+file[:file.rindex("v")]+"v20" > > > > print newFile > > > > and I want to output the results of the 'newFile' variable into the > > directory specified by the 'path' variable. > > You want the os.rename function. > > Also you should use os.path.join() to create the path rather than string > addition. It will ensure the correct separators are used > for the OS. > > You might also want to look at glob.glob() rather than listdir > to get a listing of files matching a wildcard pattern. > > -- > Alan G > Author of the Learn to Program web site > http://www.alan-g.me.uk/ > http://www.flickr.com/photos/alangauldphotos > > > > ------------------------------ > > Message: 2 > Date: Thu, 9 Oct 2014 18:32:01 +0000 (UTC) > From: Albert-Jan Roskam <fomcl at yahoo.com> > To: Python Tutor Mailing List <tutor at python.org> > Subject: [Tutor] alternative Python 2.6 install? > Message-ID: > < > 1529317176.81747.1412879521526.JavaMail.yahoo at jws10743.mail.gq1.yahoo.com> > > Content-Type: text/plain; charset=UTF-8 > > Hi, > > > I need to install Python 2.6 on my Debian system to check some code.*) > What is the easiest way to do this? Simply "sudo apt-get install > python2.6"? I know I can also compile it and then do make altinstall, but I > prefer apt-get. I am kinda paranoid that I might accidentally change my > system Python version. > > > > Thank you! > > > > Regards, > > Albert-Jan > > > *) > > albertjan at debian:~$ uname -a > Linux debian 3.2.0-4-amd64 #1 SMP Debian 3.2.60-1+deb7u3 x86_64 GNU/Linux > albertjan at debian:~$ python -c "import sys; print sys.version_info" > sys.version_info(major=2, minor=7, micro=3, releaselevel='final', serial=0) > > > > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > > All right, but apart from the sanitation, the medicine, education, wine, > public order, irrigation, roads, a > > fresh water system, and public health, what have the Romans ever done for > us? > > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > > > ------------------------------ > > Message: 3 > Date: Thu, 9 Oct 2014 20:38:02 +0200 > From: William Becerra <wbecerra1 at gmail.com> > To: tutor at python.org > Subject: [Tutor] (no subject) > Message-ID: > <CAF1DC4VdVnjRNxuf5gmRseS2bEv2TLVN6Vb2Qrn= > Tiax3CcaQQ at mail.gmail.com> > Content-Type: text/plain; charset="utf-8" > > I'm new to programming. Started reading the book 'How to think like a > computer Scientist-learning with python'. I'm now in chapter 3 sub-chapter > 3.4 Math functions. > > When I write the following code: > > import maths; > decibel = math.log10 (17.0); > angle = 1.5; > height = math.sin(angle); > print height; > > I get the following error: > > Traceback (most recent call last): > File "C:/Python27/test", line 1, in <module> > import maths; > ImportError: No module named maths > > I don't know what I'm doing wrong? > >From what I've read the maths module is supposed to come with the python > installation package. > I'm using a windows 8 operating system > python 2.7.8 > please help? > -------------- next part -------------- > An HTML attachment was scrubbed... > URL: < > http://mail.python.org/pipermail/tutor/attachments/20141009/5670d430/attachment-0001.html > > > > ------------------------------ > > Message: 4 > Date: Thu, 9 Oct 2014 20:03:23 -0400 > From: Joel Goldstick <joel.goldstick at gmail.com> > Cc: tutor at python.org > Subject: Re: [Tutor] (no subject) > Message-ID: > <CAPM-O+w6= > NrkYw6uNtAYkfttot+Ot3x1taF3oa5ew0kEWYDSPw at mail.gmail.com> > Content-Type: text/plain; charset="utf-8" > > On Oct 9, 2014 8:00 PM, "William Becerra" <wbecerra1 at gmail.com> wrote: > > > > I'm new to programming. Started reading the book 'How to think like a > computer Scientist-learning with python'. I'm now in chapter 3 sub-chapter > 3.4 Math functions. > > > > When I write the following code: > > > > import maths; > import math > > You added s > > > decibel = math.log10 (17.0); > > angle = 1.5; > > height = math.sin(angle); > > print height; > > > > I get the following error: > > > > Traceback (most recent call last): > > File "C:/Python27/test", line 1, in <module> > > import maths; > > ImportError: No module named maths > > > > I don't know what I'm doing wrong? > > From what I've read the maths module is supposed to come with the python > installation package. > > I'm using a windows 8 operating system > > python 2.7.8 > > please help? > > > > > > > > > > _______________________________________________ > > Tutor maillist - Tutor at python.org > > To unsubscribe or change subscription options: > > https://mail.python.org/mailman/listinfo/tutor > > > -------------- next part -------------- > An HTML attachment was scrubbed... > URL: < > http://mail.python.org/pipermail/tutor/attachments/20141009/787fbf3e/attachment-0001.html > > > > ------------------------------ > > Message: 5 > Date: Thu, 9 Oct 2014 17:13:51 -0700 > From: "Martin A. Brown" <martin at linux-ip.net> > To: William Becerra <wbecerra1 at gmail.com> > Cc: tutor at python.org > Subject: Re: [Tutor] (no subject) > Message-ID: <alpine.LNX.2.00.1410091704080.1349 at dagger.wonderfrog.net> > Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed > > > Hi there and welcome! > > > import maths; > > decibel = math.log10 (17.0); > > angle = 1.5; > > height = math.sin(angle); > > print height; > > > > Traceback (most recent call last): > > File "C:/Python27/test", line 1, in <module> > > import maths; > > ImportError: No module named maths > > Oops! It's a nice error report, though! Python tried to locate a > module called 'maths' and was not able to find it. > > What happens if you try: > > import math > > N.B. You say 'import maths'--assume that this import succeeded. A > few lines later, there's a line 'math.log10(17.0)' which seems to be > trying to use something from a module called 'math' not 'maths'. > > > I don't know what I'm doing wrong? > > Computers are so picky. > > > From what I've read the maths module is supposed to come with the python > > installation package. > > The 'math' library is a standard library module for quite awhile > now. Here's a possibly useful online link, which describes that > module: > > https://docs.python.org/2/library/math.html > > This is just more documentation support, in addition to the book you > are reading. > > > I'm using a windows 8 operating system > > python 2.7.8 > > please help? > > One other issue I might point out. The semicolon at the end of the > line (statement) is a feature of other programming languages with > which you may be familiar (C, Java, Perl), but it is not necessary > and, in fact, discouraged in Python. > > So, rid yourself of the semicolons and enjoy the benefits of a > trivially cleaner syntax. > > Enjoy! > > -Martin > > P.S. Thanks for your clear question and letting us know your OS and > Python version, as well. > > -- > Martin A. Brown > http://linux-ip.net/ > > > ------------------------------ > > Message: 6 > Date: Fri, 10 Oct 2014 01:36:25 +0100 > From: Alan Gauld <alan.gauld at btinternet.com> > To: tutor at python.org > Subject: Re: [Tutor] (no subject) > Message-ID: <m179m9$clv$1 at ger.gmane.org> > Content-Type: text/plain; charset=windows-1252; format=flowed > > On 09/10/14 19:38, William Becerra wrote: > > > import maths; > > Python, like most languages speaks American English > so its math not maths. > > -- > Alan G > Author of the Learn to Program web site > http://www.alan-g.me.uk/ > http://www.flickr.com/photos/alangauldphotos > > > > ------------------------------ > > Message: 7 > Date: Thu, 9 Oct 2014 22:57:24 -0500 > From: boB Stepp <robertvstepp at gmail.com> > To: tutor <tutor at python.org> > Subject: [Tutor] Installing both Python 2.7 and Python 3.4 on Windows > 7 Pro 64-bit: Install Python 2.7 FIRST! > Message-ID: > < > CANDiX9LG2y7XNGLsruqjNtY7GZYN5VWEm-E2sha1o0LLqB2Xsw at mail.gmail.com> > Content-Type: text/plain; charset=UTF-8 > > I am hoping to save other people the grief I just worked through. I > wanted to run both Python 2 and 3 on my windows PC, and, after > googling this topic found that with Python 3.3 or later one could > easily do both. So I merrily installed Python 3.4.2 first and then > Python 2.7.8. A Python 3 program that had been working fine suddenly > stopped working. After working down to a test portion of code that > isolated the culprit I realized my Python 3 program was being > interpreted by Python 2. I soon found that installing Python 2 first > and then 3 enabled both to happily coexist. If there was a mention > about the order of installation anywhere during my searches, I missed > it. Anyway, I hope that my experience helps some other newbie who > wants to play around with both major versions. > > -- > boB > > > ------------------------------ > > Subject: Digest Footer > > _______________________________________________ > Tutor maillist - Tutor at python.org > https://mail.python.org/mailman/listinfo/tutor > > > ------------------------------ > > End of Tutor Digest, Vol 128, Issue 22 > ************************************** > -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/tutor/attachments/20141010/0c68dc1a/attachment-0001.html> From alan.gauld at btinternet.com Fri Oct 10 11:55:35 2014 From: alan.gauld at btinternet.com (Alan Gauld) Date: Fri, 10 Oct 2014 10:55:35 +0100 Subject: [Tutor] Tutor Digest, Vol 128, Issue 22 In-Reply-To: <CAC7HCj-q7N3Ox7eb1URRGXFUv7pzHrzDwe0YLrjM6NzR9g7hZw@mail.gmail.com> References: <mailman.85151.1412913889.18129.tutor@python.org> <CAC7HCj-q7N3Ox7eb1URRGXFUv7pzHrzDwe0YLrjM6NzR9g7hZw@mail.gmail.com> Message-ID: <m18aen$tbb$1@ger.gmane.org> On 10/10/14 09:32, Henry wrote: > I am new to programming. Welcome to tutor. Please set a meaningful subject line in your messages. It helps people doing a search and will attract interested readers so you get more chance of help. > After I created two text files(the text file is most consist of > numbers), its class is "<class '_io.TextIOWrapper'>", how can I compare > this class with two text files? I'm not really sure what you mean. How did you "create" the text files? On the hard disk using a text editor? Or by opening the files in python using open()? The _io.TextWrapper is just the class that python uses to model text files, you should hardly ever need to know anything about it. You should never need to be comparing it to actual files - I'm not even sure what that would mean... You can compare two text file objects, but that's rarely meaningful. Or you can compare the contents of two text files, which is more common. For the latter there is the difflib module to help. > Please give me a hint which area I should look under? Set? List? You need to explain in more detail what you are trying to do. It might be good to give short example data, say a couple of lines from each file and what you expect the output to look like. Also any code you have written, because it shows us the way you are tackling the problem. Finally always include the fill text of any errors, plus a note of the python version and OS you are using too. > On Fri, Oct 10, 2014 at 12:04 PM, <tutor-request at python.org > <mailto:tutor-request at python.org>> wrote: > > Send Tutor mailing list submissions to > tutor at python.org <mailto:tutor at python.org> And do not send the full text of the digest. We've all seen the messages already and some people pay by the byte so it costs them money. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.flickr.com/photos/alangauldphotos From fomcl at yahoo.com Fri Oct 10 13:08:49 2014 From: fomcl at yahoo.com (Albert-Jan Roskam) Date: Fri, 10 Oct 2014 11:08:49 +0000 (UTC) Subject: [Tutor] alternative Python 2.6 install? In-Reply-To: <1529317176.81747.1412879521526.JavaMail.yahoo@jws10743.mail.gq1.yahoo.com> References: <1529317176.81747.1412879521526.JavaMail.yahoo@jws10743.mail.gq1.yahoo.com> Message-ID: <770191121.141799.1412939329664.JavaMail.yahoo@jws10762.mail.gq1.yahoo.com> ----- Original Message ----- > From: Albert-Jan Roskam <fomcl at yahoo.com.dmarc.invalid> > To: Python Tutor Mailing List <tutor at python.org> > Cc: > Sent: Thursday, October 9, 2014 8:32 PM > Subject: [Tutor] alternative Python 2.6 install? > > Hi, > > > I need to install Python 2.6 on my Debian system to check some code.*) What is > the easiest way to do this? Simply "sudo apt-get install python2.6"? I > know I can also compile it and then do make altinstall, but I prefer apt-get. I > am kinda paranoid that I might accidentally change my system Python version. Ok, I decided to take the altinstall route anyway. I am now a getting message 'failed to find necessary bits to build these modules', even though I did install a whole bunch of packages (zlib1g is the first one that causes problems). Does anyone know how to solve this? I don't understand why e.g. 'zlib' still is missing while I clearly installed it! Alternatively: which Ubuntu (or Debian) version was the last version to use Python 2.6? I could simply install it in VirtualBox... Lame, but I am in kind of a hurry. This is what I did (outcommented lines are what won't work, or commands that are pointless at this point): sudo apt-get install libsqlite3-dev libbz2-dev libgdbm-dev libncurses5-dev tk-dev zlib1g-dev wget https://www.python.org/ftp/python/2.6.8/Python-2.6.8.tgz tar -zxvf Python-2.6.8.tgz cd Python-2.6.8/ ./configure --prefix=/usr/local make # [a] see 'failed stuff' below #sudo make altinstall #mkvirtualenv -p /usr/local/bin/python2.6 python26 # ImportError: No module named zlib #workon python26 #pip install mysql-python # [a] Failed stuff Failed to find the necessary bits to build these modules: _bsddb _curses _curses_panel _hashlib _sqlite3 _ssl bsddb185 bz2 dbm dl gdbm imageop linuxaudiodev ossaudiodev readline sunaudiodev zlib To find the necessary bits, look in setup.py in detect_modules() for the module's name. Failed to build these modules: crypt nis running build_scripts creating build/scripts-2.6 copying and adjusting /home/albertjan/Downloads/Python-2.6.8/Tools/scripts/pydoc -> build/scripts-2.6 copying and adjusting /home/albertjan/Downloads/Python-2.6.8/Tools/scripts/idle -> build/scripts-2.6 copying and adjusting /home/albertjan/Downloads/Python-2.6.8/Tools/scripts/2to3 -> build/scripts-2.6 copying and adjusting /home/albertjan/Downloads/Python-2.6.8/Lib/smtpd.py -> build/scripts-2.6 changing mode of build/scripts-2.6/pydoc from 644 to 755 changing mode of build/scripts-2.6/idle from 644 to 755 changing mode of build/scripts-2.6/2to3 from 644 to 755 changing mode of build/scripts-2.6/smtpd.py from 644 to 755 thanks in advance! Albert-Jan From fomcl at yahoo.com Fri Oct 10 14:36:40 2014 From: fomcl at yahoo.com (Albert-Jan Roskam) Date: Fri, 10 Oct 2014 12:36:40 +0000 (UTC) Subject: [Tutor] alternative Python 2.6 install? In-Reply-To: <770191121.141799.1412939329664.JavaMail.yahoo@jws10762.mail.gq1.yahoo.com> References: <770191121.141799.1412939329664.JavaMail.yahoo@jws10762.mail.gq1.yahoo.com> Message-ID: <1652645073.149046.1412944600267.JavaMail.yahoo@jws10781.mail.gq1.yahoo.com> ----- Original Message ----- > From: Albert-Jan Roskam <fomcl at yahoo.com> > To: Albert-Jan Roskam <fomcl at yahoo.com>; Python Tutor Mailing List <tutor at python.org> > Cc: > Sent: Friday, October 10, 2014 1:08 PM > Subject: Re: [Tutor] alternative Python 2.6 install? > > > > ----- Original Message ----- > >> From: Albert-Jan Roskam <fomcl at yahoo.com.dmarc.invalid> >> To: Python Tutor Mailing List <tutor at python.org> >> Cc: >> Sent: Thursday, October 9, 2014 8:32 PM >> Subject: [Tutor] alternative Python 2.6 install? >> >> Hi, >> >> >> I need to install Python 2.6 on my Debian system to check some code.*) What > is >> the easiest way to do this? Simply "sudo apt-get install > python2.6"? I >> know I can also compile it and then do make altinstall, but I prefer > apt-get. I >> am kinda paranoid that I might accidentally change my system Python > version. > > > Ok, I decided to take the altinstall route anyway. I am now a getting message > 'failed to find necessary bits to build these modules', even though I > did install a whole bunch of packages (zlib1g is the first one that causes > problems). Does anyone know how to solve this? I don't understand why e.g. > 'zlib' still is missing while I clearly installed it! > > Alternatively: which Ubuntu (or Debian) version was the last version to use > Python 2.6? I could simply install it in VirtualBox... Lame, but I am in kind of > a hurry. > > > This is what I did (outcommented lines are what won't work, or commands that > are pointless at this point): > > sudo apt-get install libsqlite3-dev libbz2-dev libgdbm-dev libncurses5-dev > tk-dev zlib1g-dev > wget https://www.python.org/ftp/python/2.6.8/Python-2.6.8.tgz > tar -zxvf Python-2.6.8.tgz > cd Python-2.6.8/ > ./configure --prefix=/usr/local > make # [a] see 'failed stuff' below > #sudo make altinstall > #mkvirtualenv -p /usr/local/bin/python2.6 python26 # ImportError: No module > named zlib > #workon python26 > #pip install mysql-python The first of the symlinks solved the zlib problem. The there was a problem with _sha256... ln -s /lib/x86_64-linux-gnu/libz.so /lib/libz.so ln -s /lib/x86_64-linux-gnu/libssl.so /lib/libssl.so ln -s /lib/x86_64-linux-gnu/libcrypt.so /lib/libcrypt.so ln -s /lib/x86_64-linux-gnu/libcrypto.so /lib/libcrypto.so ln -s /lib/x86_64-linux-gnu/libbz2.so /lib/libbz2.so ln -s /lib/x86_64-linux-gnu/libgdbm.so /lib/libgdbm.so ln -s /lib/x86_64-linux-gnu/libcurses.so /lib/libcurses.so ln -s /lib/x86_64-linux-gnu/libsqlite3.so /lib/libsqlite3.so OK, I gave up. I used Vagrant to install a Ubuntu 10.0.4 LTS virtualbox (which has Python 2.6 by default). Now I can SSH to that machine and do whatever I want. Sheesh. I am *still* very much interested to hear what might have gone wrong, though. Albert-Jan From robertvstepp at gmail.com Fri Oct 10 14:43:47 2014 From: robertvstepp at gmail.com (boB Stepp) Date: Fri, 10 Oct 2014 07:43:47 -0500 Subject: [Tutor] Installing both Python 2.7 and Python 3.4 on Windows 7 Pro 64-bit: Install Python 2.7 FIRST! In-Reply-To: <5437853F.5050309@biologie.uni-freiburg.de> References: <CANDiX9LG2y7XNGLsruqjNtY7GZYN5VWEm-E2sha1o0LLqB2Xsw@mail.gmail.com> <5437853F.5050309@biologie.uni-freiburg.de> Message-ID: <CANDiX9LqfGoTdU1fKVuAsmyS1Vuv_MXXBaCwr2SjP-+gspOkcw@mail.gmail.com> On Fri, Oct 10, 2014 at 2:05 AM, Wolfgang Maier <wolfgang.maier at biologie.uni-freiburg.de> wrote: > On 10/10/2014 05:57 AM, boB Stepp wrote: >> >> I am hoping to save other people the grief I just worked through. I >> wanted to run both Python 2 and 3 on my windows PC, and, after >> googling this topic found that with Python 3.3 or later one could >> easily do both. So I merrily installed Python 3.4.2 first and then >> Python 2.7.8. A Python 3 program that had been working fine suddenly >> stopped working. After working down to a test portion of code that >> isolated the culprit I realized my Python 3 program was being >> interpreted by Python 2. > > > It would help if you could share details about how you tried to run the > Python 3 program (command line call, double-click, Python launcher for > Windows, ..). > Initially, I ran the program in question by double-clicking on its icon, which was my normal way of running it. This is when I realized I had a problem. The program would run without error, but give erroneous print and input statement results. I next put Python 3 into my path variable, but the same error persisted. From the command line I found that if I ran it normally (With Python 3 still in the path like it was on my previous 3.4.1 installation.), meaning python program_name.py it behaved the same way as double-clicking on the icon. However, if I ran it as py -3 program_name.py then all was well. Apparently Python 2.7.8 became the system default installation if I installed it last, but after uninstalling both and installing Python 3.4.2 last, it became the default. Or, at least when double-clicked on it ran normally. I have since added shebang lines to my programs specifying Python 3. I hope I have not forgotten any relevant details! -- boB From niyanaxx95 at gmail.com Fri Oct 10 17:46:09 2014 From: niyanaxx95 at gmail.com (niyanaxx95 at gmail.com) Date: Fri, 10 Oct 2014 15:46:09 +0000 Subject: [Tutor] =?utf-8?q?Tile_Code_Program?= Message-ID: <5437ffaf.632c8c0a.8d0f.6e95@mx.google.com> This is the prompt: Write an algorithm / pseudocode that: ? Reads in a two integers; the width and length of a room. i. The valid range for the width (x-axis) of the room is between 100 and 1000 pixels. ii. The valid range for the length (y-axis) of the room is between 100 and 900 pixels. ? Validate the inputs, prompting the user to try again if they enter an invalid integer. ? Compute the number of tiles in each row (the number of columns) ? Compute the number of tiles in each column (the number of rows) ? Print the number of columns ? Print the number of rows ? Print the total number of tiles needed ? Print the gap at the end of each row ? Print the gap at the end of each column ? Print the pattern of tiles centered in a graphics window My Code: #This is a program to compute tiles # Constants # from graphics import GraphicsWindow TILESIZE = 20 roomWidth = 100.0 roomLength = 90.0 while roomWidth < 100 or roomWidth > 1000: roomWidth = float(input("Please enter a room width between 100 and 1000: ")) if roomWidth >= 100 or roomWidth <= 1000: print("Invalid entry") while roomLength < 100 or roomLength > 900: roomLength = float(input("Please enter a room length between 100 and 900: ")) if roomLength >= 100 or roomLength <= 900: print("Invalid entry.") win = GraphicsWindow(roomWidth, roomLength) canvas = win.canvas() #Calculate the number of pairs of tiles # the number of pairs = interger part of (total width - tile width) / (2 * tile width) numberOfPairsWidth = int((roomWidth - TILESIZE) // (2 * TILESIZE)) numberOfPairsLength = int((roomLength - TILESIZE) // (2 * TILESIZE)) #Calculate the number of columns and rows numberOfCol = int(1 + (2 * numberOfPairsWidth)) numberOfRow = int(1 + (2 * numberOfPairsLength)) #Calculate the gap # the gap = (the total width - the number of tiles * tile width / 2 gapCol = (roomWidth - numberOfCol * TILESIZE) / 2 gapRow = (roomLength - numberOfRow * TILESIZE) / 2 # Draw Tiles for i in range(numberOfCol) : if i % 2 == 0 : for j in range(numberOfRow) : if j % 2 == 0 : canvas.setColor("black") else : canvas.setColor("yellow") canvas.drawRect(gapCol + i * TILESIZE, gapRow + j * TILESIZE, TILESIZE, TILESIZE) else : for j in range(numberOfRow) : if j % 2 == 0 : canvas.setColor("yellow") else : canvas.setColor("black") canvas.drawRect(gapCol + i * TILESIZE, gapRow + j * TILESIZE, TILESIZE, TILESIZE) # Print results print("The number of columns is: ", numberOfCol) print("The number of rows is: ", numberOfRow) print("The total number of tiles needed is: ", numberOfCol * numberOfRow) print("The gap at the end of each rows is: ", gapRow) print("The gap at the end of each column is: ", gapCol) The output: Please enter a room length between 100 and 900: 200 Invalid entry. The number of columns is: 5 The number of rows is: 9 The total number of tiles needed is: 45 The gap at the end of each rows is: 10.0 The gap at the end of each column is: 0.0 Sent from Windows Mail -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/tutor/attachments/20141010/a3f7eed2/attachment-0001.html> From twitarch at gmail.com Fri Oct 10 19:47:03 2014 From: twitarch at gmail.com (Z) Date: Fri, 10 Oct 2014 13:47:03 -0400 Subject: [Tutor] Tutorials for experimental psycology Message-ID: <CAHR2oARSmQawa6GCPh8_fa0HjC-HFN_5fmPNvVioAwgF46KDEQ@mail.gmail.com> I am familiar with basics of programming, and wanted to learn how to code psychology experiments in python. Are there standard libraries/tutorials and/or GUI-based software? I especially want to be able to assign probability distribution functions to variables. I would eventually be working with looking times, reaction times, and integrating these different measures to measure behavior. Thanks -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/tutor/attachments/20141010/4b06fec9/attachment.html> From davea at davea.name Fri Oct 10 22:57:18 2014 From: davea at davea.name (Dave Angel) Date: Fri, 10 Oct 2014 16:57:18 -0400 (EDT) Subject: [Tutor] Tile Code Program References: <5437ffaf.632c8c0a.8d0f.6e95@mx.google.com> Message-ID: <m19h1j$qng$1@ger.gmane.org> <niyanaxx95 at gmail.com> Wrote in message: > > (Please use text mail to post here) >>>>>>>> roomWidth = 100.0. #BUGBUG roomLength = 90.0 while roomWidth < 100 or roomWidth > 1000: roomWidth = float(input("Please enter a room width between 100 and 1000: ")) if roomWidth >= 100 or roomWidth <= 1000: print("Invalid entry") while roomLength < 100 or roomLength > 900: roomLength = float(input("Please enter a room length between 100 and 900: ")) if roomLength >= 100 or roomLength <= 900: print("Invalid entry.") >>>>>>>> a room length of 100 causes the first while to be skipped entirely. Both the if statements are inside out. You need the same conditional as you used for the while loops. -- DaveA From dyoo at hashcollision.org Sat Oct 11 00:03:25 2014 From: dyoo at hashcollision.org (Danny Yoo) Date: Fri, 10 Oct 2014 15:03:25 -0700 Subject: [Tutor] Tile Code Program In-Reply-To: <5437ffaf.632c8c0a.8d0f.6e95@mx.google.com> References: <5437ffaf.632c8c0a.8d0f.6e95@mx.google.com> Message-ID: <CAGZAPF6A3vt60FSy_CJhm0wQoQXfg2F01PA7tojk+pfM9Vwa=Q@mail.gmail.com> On Fri, Oct 10, 2014 at 8:46 AM, <niyanaxx95 at gmail.com> wrote: > This is the prompt: Write an algorithm / pseudocode that: > ? Reads in a two integers; the width and length of a room. [code cut] Can you talk a little bit about what problem you're having, if anything? Or are you asking for a code review? It's unclear, so more context would be helpful. From alan.gauld at btinternet.com Sat Oct 11 00:41:36 2014 From: alan.gauld at btinternet.com (Alan Gauld) Date: Fri, 10 Oct 2014 23:41:36 +0100 Subject: [Tutor] Tutorials for experimental psycology In-Reply-To: <CAHR2oARSmQawa6GCPh8_fa0HjC-HFN_5fmPNvVioAwgF46KDEQ@mail.gmail.com> References: <CAHR2oARSmQawa6GCPh8_fa0HjC-HFN_5fmPNvVioAwgF46KDEQ@mail.gmail.com> Message-ID: <m19nb0$a9d$1@ger.gmane.org> On 10/10/14 18:47, Z wrote: > I am familiar with basics of programming, and wanted to learn how to > code psychology experiments in python. Are there standard > libraries/tutorials and/or GUI-based software? Yes, but a lot depends on your specific requirements. You should probably research SciPy/Scikit and in particular NumPy, Pandas and IPython. But I'm no expert on their details and their use is a bit out of scope for this mailing list. But there are dedicated fora for most of the SciP{y components. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.flickr.com/photos/alangauldphotos From dyoo at hashcollision.org Sat Oct 11 00:45:05 2014 From: dyoo at hashcollision.org (Danny Yoo) Date: Fri, 10 Oct 2014 15:45:05 -0700 Subject: [Tutor] Tutorials for experimental psycology In-Reply-To: <m19nb0$a9d$1@ger.gmane.org> References: <CAHR2oARSmQawa6GCPh8_fa0HjC-HFN_5fmPNvVioAwgF46KDEQ@mail.gmail.com> <m19nb0$a9d$1@ger.gmane.org> Message-ID: <CAGZAPF4fUUuqB-rWhr_Bpcw9RhUyVhBRvOar_kmtT9SJtGGr5w@mail.gmail.com> On Fri, Oct 10, 2014 at 3:41 PM, Alan Gauld <alan.gauld at btinternet.com> wrote: > On 10/10/14 18:47, Z wrote: >> >> I am familiar with basics of programming, and wanted to learn how to >> code psychology experiments in python. Are there standard >> libraries/tutorials and/or GUI-based software? > > > Yes, but a lot depends on your specific requirements. > > You should probably research SciPy/Scikit and in particular > NumPy, Pandas and IPython. > > But I'm no expert on their details and their use is > a bit out of scope for this mailing list. But there are > dedicated fora for most of the SciP{y components. Agreed; it's a bit out of scope for Python-Tutor. A web search for [experimental psychology python] does hit a few intriguing leads: http://www.psychopy.org/ but I suspect none of us here have direct experience with this or other related libraries. From robertvstepp at gmail.com Sat Oct 11 06:34:35 2014 From: robertvstepp at gmail.com (boB Stepp) Date: Fri, 10 Oct 2014 23:34:35 -0500 Subject: [Tutor] Installing both Python 2.7 and Python 3.4 on Windows 7 Pro 64-bit: Install Python 2.7 FIRST! In-Reply-To: <CANDiX9LqfGoTdU1fKVuAsmyS1Vuv_MXXBaCwr2SjP-+gspOkcw@mail.gmail.com> References: <CANDiX9LG2y7XNGLsruqjNtY7GZYN5VWEm-E2sha1o0LLqB2Xsw@mail.gmail.com> <5437853F.5050309@biologie.uni-freiburg.de> <CANDiX9LqfGoTdU1fKVuAsmyS1Vuv_MXXBaCwr2SjP-+gspOkcw@mail.gmail.com> Message-ID: <CANDiX9JwjaUcjydL_dMtFWigWeuEeH82UeJU4u7hHEnLLwo-dQ@mail.gmail.com> On Fri, Oct 10, 2014 at 7:43 AM, boB Stepp <robertvstepp at gmail.com> wrote: > On Fri, Oct 10, 2014 at 2:05 AM, Wolfgang Maier > <wolfgang.maier at biologie.uni-freiburg.de> wrote: >> On 10/10/2014 05:57 AM, boB Stepp wrote: [...] >> >> It would help if you could share details about how you tried to run the >> Python 3 program (command line call, double-click, Python launcher for >> Windows, ..). >> > Initially, I ran the program in question by double-clicking on its > icon, which was my normal way of running it. This is when I realized I > had a problem. The program would run without error, but give erroneous > print and input statement results. I next put Python 3 into my path > variable, but the same error persisted. From the command line I found > that if I ran it normally (With Python 3 still in the path like it was > on my previous 3.4.1 installation.), meaning > > python program_name.py > > it behaved the same way as double-clicking on the icon. However, if I ran it as > > py -3 program_name.py > > then all was well. > > Apparently Python 2.7.8 became the system default installation if I > installed it last, but after uninstalling both and installing Python > 3.4.2 last, it became the default. Or, at least when double-clicked on > it ran normally. > > I have since added shebang lines to my programs specifying Python 3. > > I hope I have not forgotten any relevant details! > Apparently I must have forgotten relevant details because I have concluded tonight that I have not solved anything. Despite thinking that I had made Python 3.4.2 my default Python, I am now certain that Python 2.7.8 is, in fact, my current default Python. So switching the order of installation did not work and whatever I did last night in my presumably sleepy or lame state was not what I thought I was doing. If I double-click on a file or icon to run a program, it runs under Python 2.7.8. If I run it from the command line with py program_name.py, then it runs under Python 2.7.8. This is with shebang lines removed from all programs. So I apologize for what I wrote earlier. I can live with 2.7.8 being the default Python, but if I wanted to make 3.4.2 the default, how would I go about doing it? boB From robertvstepp at gmail.com Sat Oct 11 07:24:49 2014 From: robertvstepp at gmail.com (boB Stepp) Date: Sat, 11 Oct 2014 00:24:49 -0500 Subject: [Tutor] Installing both Python 2.7 and Python 3.4 on Windows 7 Pro 64-bit: Install Python 2.7 FIRST! In-Reply-To: <CAKJDb-NHmBjP-H_h1qKhsxnWYFnwcPEs6WFeQDDeHH=37b1P1g@mail.gmail.com> References: <CANDiX9LG2y7XNGLsruqjNtY7GZYN5VWEm-E2sha1o0LLqB2Xsw@mail.gmail.com> <5437853F.5050309@biologie.uni-freiburg.de> <CANDiX9LqfGoTdU1fKVuAsmyS1Vuv_MXXBaCwr2SjP-+gspOkcw@mail.gmail.com> <CANDiX9JwjaUcjydL_dMtFWigWeuEeH82UeJU4u7hHEnLLwo-dQ@mail.gmail.com> <CAKJDb-NHmBjP-H_h1qKhsxnWYFnwcPEs6WFeQDDeHH=37b1P1g@mail.gmail.com> Message-ID: <CANDiX9Jcunt2oOw_Xev0Z8GDcU2e-+Dspbdg26G71ZVBRUJ=0g@mail.gmail.com> On Fri, Oct 10, 2014 at 11:54 PM, Zachary Ware <zachary.ware+pytut at gmail.com> wrote: > On Fri, Oct 10, 2014 at 11:34 PM, boB Stepp <robertvstepp at gmail.com> wrote: >> I can live with 2.7.8 being the default Python, but if I wanted to >> make 3.4.2 the default, how would I go about doing it? > > Check the output of "ftype Python.File", it should be: > > C:\>ftype Python.File > Python.File="C:\Windows\py.exe" "%1" %* > This checked out sat. > If it's not, make it so :). Then configure the Python Launcher > (py.exe) as described here: > https://docs.python.org/3/using/windows.html#customization > > (You'll probably want your py.ini to look like this: > > [defaults] > python=3 > Cannot locate either of the mentioned py.ini files. I did a search for these on my PC and came up empty. I am going to try to create my own py.ini file and place it in C:\WINDOWS and see what happens. -- boB From robertvstepp at gmail.com Sat Oct 11 07:30:03 2014 From: robertvstepp at gmail.com (boB Stepp) Date: Sat, 11 Oct 2014 00:30:03 -0500 Subject: [Tutor] Installing both Python 2.7 and Python 3.4 on Windows 7 Pro 64-bit: Install Python 2.7 FIRST! In-Reply-To: <CANDiX9Jcunt2oOw_Xev0Z8GDcU2e-+Dspbdg26G71ZVBRUJ=0g@mail.gmail.com> References: <CANDiX9LG2y7XNGLsruqjNtY7GZYN5VWEm-E2sha1o0LLqB2Xsw@mail.gmail.com> <5437853F.5050309@biologie.uni-freiburg.de> <CANDiX9LqfGoTdU1fKVuAsmyS1Vuv_MXXBaCwr2SjP-+gspOkcw@mail.gmail.com> <CANDiX9JwjaUcjydL_dMtFWigWeuEeH82UeJU4u7hHEnLLwo-dQ@mail.gmail.com> <CAKJDb-NHmBjP-H_h1qKhsxnWYFnwcPEs6WFeQDDeHH=37b1P1g@mail.gmail.com> <CANDiX9Jcunt2oOw_Xev0Z8GDcU2e-+Dspbdg26G71ZVBRUJ=0g@mail.gmail.com> Message-ID: <CANDiX9LHowt3M94_Chun7fW78detF3yBNwP3S7BW_G1yfm+pZA@mail.gmail.com> On Sat, Oct 11, 2014 at 12:24 AM, boB Stepp <robertvstepp at gmail.com> wrote: > On Fri, Oct 10, 2014 at 11:54 PM, Zachary Ware > <zachary.ware+pytut at gmail.com> wrote: >> On Fri, Oct 10, 2014 at 11:34 PM, boB Stepp <robertvstepp at gmail.com> wrote: >>> I can live with 2.7.8 being the default Python, but if I wanted to >>> make 3.4.2 the default, how would I go about doing it? >> >> Check the output of "ftype Python.File", it should be: >> >> C:\>ftype Python.File >> Python.File="C:\Windows\py.exe" "%1" %* >> > > This checked out sat. > >> If it's not, make it so :). Then configure the Python Launcher >> (py.exe) as described here: >> https://docs.python.org/3/using/windows.html#customization >> >> (You'll probably want your py.ini to look like this: >> >> [defaults] >> python=3 >> > Cannot locate either of the mentioned py.ini files. I did a search for > these on my PC and came up empty. I am going to try to create my own > py.ini file and place it in C:\WINDOWS and see what happens. > This did the trick! Many thanks, Zach! -- boB From zachary.ware+pytut at gmail.com Sat Oct 11 07:37:15 2014 From: zachary.ware+pytut at gmail.com (Zachary Ware) Date: Sat, 11 Oct 2014 00:37:15 -0500 Subject: [Tutor] Fwd: Installing both Python 2.7 and Python 3.4 on Windows 7 Pro 64-bit: Install Python 2.7 FIRST! In-Reply-To: <CAKJDb-NHmBjP-H_h1qKhsxnWYFnwcPEs6WFeQDDeHH=37b1P1g@mail.gmail.com> References: <CANDiX9LG2y7XNGLsruqjNtY7GZYN5VWEm-E2sha1o0LLqB2Xsw@mail.gmail.com> <5437853F.5050309@biologie.uni-freiburg.de> <CANDiX9LqfGoTdU1fKVuAsmyS1Vuv_MXXBaCwr2SjP-+gspOkcw@mail.gmail.com> <CANDiX9JwjaUcjydL_dMtFWigWeuEeH82UeJU4u7hHEnLLwo-dQ@mail.gmail.com> <CAKJDb-NHmBjP-H_h1qKhsxnWYFnwcPEs6WFeQDDeHH=37b1P1g@mail.gmail.com> Message-ID: <CAKJDb-MB0gdkKPYLWEMae9raNvvYc6F5jSb83s=b61G=oSJYNw@mail.gmail.com> <Forwarding my original to the list, as I failed earlier> On Fri, Oct 10, 2014 at 11:34 PM, boB Stepp <robertvstepp at gmail.com> wrote: > I can live with 2.7.8 being the default Python, but if I wanted to > make 3.4.2 the default, how would I go about doing it? Check the output of "ftype Python.File", it should be: C:\>ftype Python.File Python.File="C:\Windows\py.exe" "%1" %* If it's not, make it so :). Then configure the Python Launcher (py.exe) as described here: https://docs.python.org/3/using/windows.html#customization (You'll probably want your py.ini to look like this: [defaults] python=3 ) Hope this helps, -- Zach From zachary.ware+pytut at gmail.com Sat Oct 11 07:39:57 2014 From: zachary.ware+pytut at gmail.com (Zachary Ware) Date: Sat, 11 Oct 2014 00:39:57 -0500 Subject: [Tutor] Installing both Python 2.7 and Python 3.4 on Windows 7 Pro 64-bit: Install Python 2.7 FIRST! In-Reply-To: <CANDiX9LHowt3M94_Chun7fW78detF3yBNwP3S7BW_G1yfm+pZA@mail.gmail.com> References: <CANDiX9LG2y7XNGLsruqjNtY7GZYN5VWEm-E2sha1o0LLqB2Xsw@mail.gmail.com> <5437853F.5050309@biologie.uni-freiburg.de> <CANDiX9LqfGoTdU1fKVuAsmyS1Vuv_MXXBaCwr2SjP-+gspOkcw@mail.gmail.com> <CANDiX9JwjaUcjydL_dMtFWigWeuEeH82UeJU4u7hHEnLLwo-dQ@mail.gmail.com> <CAKJDb-NHmBjP-H_h1qKhsxnWYFnwcPEs6WFeQDDeHH=37b1P1g@mail.gmail.com> <CANDiX9Jcunt2oOw_Xev0Z8GDcU2e-+Dspbdg26G71ZVBRUJ=0g@mail.gmail.com> <CANDiX9LHowt3M94_Chun7fW78detF3yBNwP3S7BW_G1yfm+pZA@mail.gmail.com> Message-ID: <CAKJDb-PURzZXHS9g8=G+u-AdT3FQ9m4h+wkvH1kcF7cL2m=VOg@mail.gmail.com> On Sat, Oct 11, 2014 at 12:30 AM, boB Stepp <robertvstepp at gmail.com> wrote: > On Sat, Oct 11, 2014 at 12:24 AM, boB Stepp <robertvstepp at gmail.com> wrote: >> Cannot locate either of the mentioned py.ini files. I did a search for >> these on my PC and came up empty. I am going to try to create my own >> py.ini file and place it in C:\WINDOWS and see what happens. Correct, those files don't exist by default; they only exist if you want to change the launcher's defaults. > This did the trick! Many thanks, Zach! Glad I could help :) -- Zach From wbecerra1 at gmail.com Sat Oct 11 10:44:18 2014 From: wbecerra1 at gmail.com (William Becerra) Date: Sat, 11 Oct 2014 10:44:18 +0200 Subject: [Tutor] keyword colors disappear Message-ID: <CAF1DC4UimMfyDAfDyiLvN5WtEGYZkjemboce-p6W3nn54YR_YA@mail.gmail.com> Hey, I'm new to programming. Only have about 2 weeks of experience. Using Python 2.7.8 and running Windows 8 I'm having the following problem. I open Python shell press file, new file and write my code(any code) then all the Python keywords appear in their different *colors*, for example print appears in orange, strings in green numbers in blue, etc. Now here is the problem, after I press F5 and i run my code. Then i try go back to my code and add some more code or change the code. Now all ll the colors of the Python keywords are gone. Everything appears in normal black and white text. Note: weather the code is correct or there is any errors I still have the same problem. What i want to know is. Is there something i can do to keep the colors of the keywords? because it makes it easier for me to keep track of my code. Hope what i asked is clear. Thank You -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/tutor/attachments/20141011/f2298ac9/attachment.html> From __peter__ at web.de Sat Oct 11 14:14:32 2014 From: __peter__ at web.de (Peter Otten) Date: Sat, 11 Oct 2014 14:14:32 +0200 Subject: [Tutor] keyword colors disappear References: <CAF1DC4UimMfyDAfDyiLvN5WtEGYZkjemboce-p6W3nn54YR_YA@mail.gmail.com> Message-ID: <m1b6v9$q9f$1@ger.gmane.org> William Becerra wrote: > Hey, I'm new to programming. > Only have about 2 weeks of experience. > Using Python 2.7.8 and running Windows 8 > I'm having the following problem. > > I open Python shell press file, new file and write my code(any code) > then all the Python keywords appear in their different *colors*, for > example print appears in orange, strings in green numbers in blue, etc. >From your description I conclude that you are using IDLE. Is that correct? (If you are not sure look into the help menu, there should be an "About IDLE" entry) > Now here is the problem, after I press F5 and i run my code. Then i try go > back to my code and add some more code or change the code. Now all ll the > colors of the Python keywords are gone. Everything appears in normal > black and white text. > Note: weather the code is correct or there is any errors I still have the > same problem. > > What i want to know is. > Is there something i can do to keep the colors of the keywords? because it > makes it easier for me to keep track of my code. > > Hope what i asked is clear. What is the suffix of the saved file? It should be '.py'. If the file name ends with with something else (on my linux machine even '.PY' doesn't work) IDLE may mistake the code for arbitrary text. From davea at davea.name Sat Oct 11 15:35:12 2014 From: davea at davea.name (Dave Angel) Date: Sat, 11 Oct 2014 09:35:12 -0400 (EDT) Subject: [Tutor] Installing both Python 2.7 and Python 3.4 on Windows 7 Pro 64-bit: Install Python 2.7 FIRST! References: <CANDiX9LG2y7XNGLsruqjNtY7GZYN5VWEm-E2sha1o0LLqB2Xsw@mail.gmail.com> <5437853F.5050309@biologie.uni-freiburg.de> <CANDiX9LqfGoTdU1fKVuAsmyS1Vuv_MXXBaCwr2SjP-+gspOkcw@mail.gmail.com> <CANDiX9JwjaUcjydL_dMtFWigWeuEeH82UeJU4u7hHEnLLwo-dQ@mail.gmail.com> Message-ID: <m1bbgj$b90$1@ger.gmane.org> boB Stepp <robertvstepp at gmail.com> Wrote in message: > > I can live with 2.7.8 being the default Python, but if I wanted to > make 3.4.2 the default, how would I go about doing it? > I haven't used Windows in a long while. When I did, I used assoc.exe and ftype.exe to set my python to run py.exe. That program is installed by pthon 3.3 and later. In turn, it interprets the shebang line, launching whatever version needed. However somebody corrected me, saying that per-user mappings in HKEY_CURRENT_USER\Software\Classes can override the ones set with ftype and assoc. -- DaveA From juan0christian at gmail.com Sun Oct 12 04:29:13 2014 From: juan0christian at gmail.com (Juan Christian) Date: Sat, 11 Oct 2014 23:29:13 -0300 Subject: [Tutor] How to use custom protocol with requests? Message-ID: <CAAp0bGsZKmyA+hyPcnrMS4w3b3DnsHbF7H=-a-iMs6H6MfAa5Q@mail.gmail.com> I need to call this URL: steam://friends/add/<ID> If I put it in my browser and press enter it works as expected, now I need to do this on my code, I don't need to retrieve anything, I just need it to be "executed", is there a way to do that in Python with requests or any other lib? Python 3.4.1 -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/tutor/attachments/20141011/a06abfbc/attachment.html> From dyoo at hashcollision.org Sun Oct 12 04:47:36 2014 From: dyoo at hashcollision.org (Danny Yoo) Date: Sat, 11 Oct 2014 19:47:36 -0700 Subject: [Tutor] How to use custom protocol with requests? In-Reply-To: <CAAp0bGsZKmyA+hyPcnrMS4w3b3DnsHbF7H=-a-iMs6H6MfAa5Q@mail.gmail.com> References: <CAAp0bGsZKmyA+hyPcnrMS4w3b3DnsHbF7H=-a-iMs6H6MfAa5Q@mail.gmail.com> Message-ID: <CAGZAPF6iB2yi7v32ev=7APFsGVsNZtWTWcJN2PvyHgwNvO3KJA@mail.gmail.com> On Sat, Oct 11, 2014 at 7:29 PM, Juan Christian <juan0christian at gmail.com> wrote: > I need to call this URL: steam://friends/add/<ID> > > If I put it in my browser and press enter it works as expected, now I need > to do this on my code, I don't need to retrieve anything, I just need it to > be "executed", is there a way to do that in Python with requests or any > other lib? This may be more difficult to do; it's unclear whether or not this is exposed in the web API that Valve provides. Unlike the other methods described in: https://developer.valvesoftware.com/wiki/Steam_Web_API which are all query-based, you're asking for something that _mutates_ a user. This functionality would probably not be exposed as a RESTful web-based API: imagine the kind of havok that a malicious user could do if they could add arbitrary friends on any other person. So I would expect it to require some sort of user authentication, at the very least. Searching... I see no such documentation from web searches on this. You'll probably need to ask the Steam folks on this one. From juan0christian at gmail.com Sun Oct 12 04:55:44 2014 From: juan0christian at gmail.com (Juan Christian) Date: Sat, 11 Oct 2014 23:55:44 -0300 Subject: [Tutor] How to use custom protocol with requests? In-Reply-To: <CAGZAPF6RvBjR1iuHCAGiQh9U40T+jhGL6eiQBUc_KC+gxCh9yA@mail.gmail.com> References: <CAAp0bGsZKmyA+hyPcnrMS4w3b3DnsHbF7H=-a-iMs6H6MfAa5Q@mail.gmail.com> <CAGZAPF6iB2yi7v32ev=7APFsGVsNZtWTWcJN2PvyHgwNvO3KJA@mail.gmail.com> <CAGZAPF6RvBjR1iuHCAGiQh9U40T+jhGL6eiQBUc_KC+gxCh9yA@mail.gmail.com> Message-ID: <CAAp0bGsgFwMws8oGAbaZgKBty-zSFZAexKCuhN35VwsyfN6AdQ@mail.gmail.com> On Sat, Oct 11, 2014 at 11:53 PM, Danny Yoo <dyoo at hashcollision.org> wrote: > >> I need to call this URL: steam://friends/add/<ID> > >> > >> If I put it in my browser and press enter it works as expected, now I > need > >> to do this on my code, I don't need to retrieve anything, I just need > it to > >> be "executed", is there a way to do that in Python with requests or any > >> other lib? > > Thinking about this a bit more. If your browser is already doing the > "right thing", maybe the webbrowser module is enough to get this > behavior: > > https://docs.python.org/2/library/webbrowser.html > > webbrowser should invoke the browser on your behalf. Maybe your web > browser is set up to dispatch this to the steam application? Hard to > tell without testing this myself, but I'm not in an environment that > lets me try this out at the moment. > YES, the browser redirects it to the Steam software installed on the PC! -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/tutor/attachments/20141011/321d5aad/attachment-0001.html> From juan0christian at gmail.com Sun Oct 12 04:56:42 2014 From: juan0christian at gmail.com (Juan Christian) Date: Sat, 11 Oct 2014 23:56:42 -0300 Subject: [Tutor] How to use custom protocol with requests? In-Reply-To: <CAAp0bGsgFwMws8oGAbaZgKBty-zSFZAexKCuhN35VwsyfN6AdQ@mail.gmail.com> References: <CAAp0bGsZKmyA+hyPcnrMS4w3b3DnsHbF7H=-a-iMs6H6MfAa5Q@mail.gmail.com> <CAGZAPF6iB2yi7v32ev=7APFsGVsNZtWTWcJN2PvyHgwNvO3KJA@mail.gmail.com> <CAGZAPF6RvBjR1iuHCAGiQh9U40T+jhGL6eiQBUc_KC+gxCh9yA@mail.gmail.com> <CAAp0bGsgFwMws8oGAbaZgKBty-zSFZAexKCuhN35VwsyfN6AdQ@mail.gmail.com> Message-ID: <CAAp0bGsoax1fRn3vmjciS0sQiAGfSwsLiuwbGUT4FYo7HWmBrw@mail.gmail.com> On Sat, Oct 11, 2014 at 11:47 PM, Danny Yoo <dyoo at hashcollision.org> wrote: > This may be more difficult to do; it's unclear whether or not this is > exposed in the web API that Valve provides. > > Unlike the other methods described in: > > https://developer.valvesoftware.com/wiki/Steam_Web_API > > which are all query-based, you're asking for something that _mutates_ > a user. This functionality would probably not be exposed as a RESTful > web-based API: imagine the kind of havok that a malicious user could > do if they could add arbitrary friends on any other person. So I > would expect it to require some sort of user authentication, at the > very least. Searching... I see no such documentation from web > searches on this. You'll probably need to ask the Steam folks on this > one. > Indeed the person HAS TO BE logged in on Steam, it already exists in sites like backpack.tf, you have a +ADD button in all profiles there, I just want to mimic this action on my script when needed. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/tutor/attachments/20141011/3a96bfdb/attachment.html> From juan0christian at gmail.com Sun Oct 12 04:58:58 2014 From: juan0christian at gmail.com (Juan Christian) Date: Sat, 11 Oct 2014 23:58:58 -0300 Subject: [Tutor] How to use custom protocol with requests? In-Reply-To: <CAAp0bGsoax1fRn3vmjciS0sQiAGfSwsLiuwbGUT4FYo7HWmBrw@mail.gmail.com> References: <CAAp0bGsZKmyA+hyPcnrMS4w3b3DnsHbF7H=-a-iMs6H6MfAa5Q@mail.gmail.com> <CAGZAPF6iB2yi7v32ev=7APFsGVsNZtWTWcJN2PvyHgwNvO3KJA@mail.gmail.com> <CAGZAPF6RvBjR1iuHCAGiQh9U40T+jhGL6eiQBUc_KC+gxCh9yA@mail.gmail.com> <CAAp0bGsgFwMws8oGAbaZgKBty-zSFZAexKCuhN35VwsyfN6AdQ@mail.gmail.com> <CAAp0bGsoax1fRn3vmjciS0sQiAGfSwsLiuwbGUT4FYo7HWmBrw@mail.gmail.com> Message-ID: <CAAp0bGth16gKeiLfFMi2TfoarUKGWBoYSp-YPm6DVOSduzh1wQ@mail.gmail.com> Sorry for triple post, but yes, webbrowser worked 100%. Exactly what I needed! Thanks! -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/tutor/attachments/20141011/dee692cd/attachment.html> From dyoo at hashcollision.org Sun Oct 12 05:17:28 2014 From: dyoo at hashcollision.org (Danny Yoo) Date: Sat, 11 Oct 2014 20:17:28 -0700 Subject: [Tutor] How to use custom protocol with requests? In-Reply-To: <CAAp0bGth16gKeiLfFMi2TfoarUKGWBoYSp-YPm6DVOSduzh1wQ@mail.gmail.com> References: <CAAp0bGsZKmyA+hyPcnrMS4w3b3DnsHbF7H=-a-iMs6H6MfAa5Q@mail.gmail.com> <CAGZAPF6iB2yi7v32ev=7APFsGVsNZtWTWcJN2PvyHgwNvO3KJA@mail.gmail.com> <CAGZAPF6RvBjR1iuHCAGiQh9U40T+jhGL6eiQBUc_KC+gxCh9yA@mail.gmail.com> <CAAp0bGsgFwMws8oGAbaZgKBty-zSFZAexKCuhN35VwsyfN6AdQ@mail.gmail.com> <CAAp0bGsoax1fRn3vmjciS0sQiAGfSwsLiuwbGUT4FYo7HWmBrw@mail.gmail.com> <CAAp0bGth16gKeiLfFMi2TfoarUKGWBoYSp-YPm6DVOSduzh1wQ@mail.gmail.com> Message-ID: <CAGZAPF7ouiu6ef=saDCaxuet-HbaubsGYOW-pg08uB=SZ=3-rg@mail.gmail.com> On Sat, Oct 11, 2014 at 7:58 PM, Juan Christian <juan0christian at gmail.com> wrote: > Sorry for triple post, but yes, webbrowser worked 100%. Exactly what I > needed! Huh. Wow. That actually worked? :P --- Frankly speaking though, this sounds like a horrible XSRF-style attack in waiting, if I understand what has just happened. (http://en.wikipedia.org/wiki/Cross-site_request_forgery) Usually, requests to do mutation operations are protected so that, in order to make the request, you have to have some knowledge in the request that's specific to the user, and not public knowledge. The URL you've described is missing this basic information, an "XSRF token" as its commonly known (though I would have assumed it would be called an "anti-XSRF" token, but oh well.) I'm not sure how your web browser is handling the 'steam://' URL class, but I would very much hope that, in the interface between the browser and your Steam client, it's doing something to mitigate what looks like an XSRF exploit. From dyoo at hashcollision.org Sun Oct 12 04:53:11 2014 From: dyoo at hashcollision.org (Danny Yoo) Date: Sat, 11 Oct 2014 19:53:11 -0700 Subject: [Tutor] How to use custom protocol with requests? In-Reply-To: <CAGZAPF6iB2yi7v32ev=7APFsGVsNZtWTWcJN2PvyHgwNvO3KJA@mail.gmail.com> References: <CAAp0bGsZKmyA+hyPcnrMS4w3b3DnsHbF7H=-a-iMs6H6MfAa5Q@mail.gmail.com> <CAGZAPF6iB2yi7v32ev=7APFsGVsNZtWTWcJN2PvyHgwNvO3KJA@mail.gmail.com> Message-ID: <CAGZAPF6RvBjR1iuHCAGiQh9U40T+jhGL6eiQBUc_KC+gxCh9yA@mail.gmail.com> >> I need to call this URL: steam://friends/add/<ID> >> >> If I put it in my browser and press enter it works as expected, now I need >> to do this on my code, I don't need to retrieve anything, I just need it to >> be "executed", is there a way to do that in Python with requests or any >> other lib? Thinking about this a bit more. If your browser is already doing the "right thing", maybe the webbrowser module is enough to get this behavior: https://docs.python.org/2/library/webbrowser.html webbrowser should invoke the browser on your behalf. Maybe your web browser is set up to dispatch this to the steam application? Hard to tell without testing this myself, but I'm not in an environment that lets me try this out at the moment. From wbecerra1 at gmail.com Sun Oct 12 09:41:40 2014 From: wbecerra1 at gmail.com (William Becerra) Date: Sun, 12 Oct 2014 09:41:40 +0200 Subject: [Tutor] Infinite Recursion Message-ID: <CAF1DC4VFm0oZFLLOvNtxx1J5-bmbwUvY7AVuXGK94aGgnQ0d4A@mail.gmail.com> Hey, I'm new to programming. Using python 2.7.8 and running windows8 OS I'm reading 'How to think like a computer scientist, learning with Python' I'm in chapter 4 sub-chapter 4.11 Infinite recursion According to the book if I write def recurse(): recurse() I should get the following error File"<stdin>", line2, in recurse ( 98 repetitions omittted) File "<stdin>", line 2, in recurse RuntimeError: Maximum recursion depth exceeded. I don't get that error, instead the Python shell prints out two blank lines. >From what i understand if i don't get the error the infinite recursion is not been tried by the shell. Am I missing anything in the code? and If anything is wrong. How can I write a easy Infinite recursion to help me grasp the concept? Thank You. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/tutor/attachments/20141012/9a3e739d/attachment.html> From steve at pearwood.info Sun Oct 12 11:03:52 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Sun, 12 Oct 2014 20:03:52 +1100 Subject: [Tutor] Infinite Recursion In-Reply-To: <CAF1DC4VFm0oZFLLOvNtxx1J5-bmbwUvY7AVuXGK94aGgnQ0d4A@mail.gmail.com> References: <CAF1DC4VFm0oZFLLOvNtxx1J5-bmbwUvY7AVuXGK94aGgnQ0d4A@mail.gmail.com> Message-ID: <20141012090352.GE23419@ando.pearwood.info> On Sun, Oct 12, 2014 at 09:41:40AM +0200, William Becerra wrote: > Hey, I'm new to programming. > Using python 2.7.8 and running windows8 OS > I'm reading 'How to think like a computer scientist, learning with Python' > I'm in chapter 4 sub-chapter 4.11 Infinite recursion > > According to the book if I write > def recurse(): > recurse() > I should get the following error > File"<stdin>", line2, in recurse > ( 98 repetitions omittted) > File "<stdin>", line 2, in recurse > RuntimeError: Maximum recursion depth exceeded. > > I don't get that error, instead the Python shell prints out two blank lines. *Two* blank lines? I can't reproduce that. Are you perhaps using IPython? Or IDLE? Either way, I expect that you have forgotten the brackets (parentheses) on the *inner* call to `recurse`. If I type this instead: def recurse(): recurse recurse() # Call the function. notice that there are no parens on the inner call, so what happens is that when I enter "recurse()", Python executes the function. Inside the body of the function, Python grabs a reference to the "recurse" function, but *does not call it*. Because it's not called, the recursion stops immediately. It may help you understand the difference to play around with these functions. Copy and paste them into the shell: def recurse1(): # Actually a lie, not really recursing at all. print "Look up the name 'recurse1' and see if it exists:" recurse1 print "Done!" def recurse2(): print "Look up the name 'recurse2' and call it:" recurse2() print "Done!" -- Steven From fomcl at yahoo.com Sun Oct 12 14:35:48 2014 From: fomcl at yahoo.com (Albert-Jan Roskam) Date: Sun, 12 Oct 2014 12:35:48 +0000 (UTC) Subject: [Tutor] what is the easiest way to install different Python versions? Message-ID: <1798207457.287910.1413117348399.JavaMail.yahoo@jws10769.mail.gq1.yahoo.com> Hi, (sorry for cross-posting, sort of) A few days ago I needed to check whether some Python code ran with Python 2.6. What is the easiest way to install another Python version along side the default Python version? My own computer is Debian Linux 64 bit, but a platform-independent solution would be best. Possible solutions that I am aware of -make altinstall *). This is what I tried (see below), but not all modules could be built. I gave up because I was in a hurry -Pythonbrew. This project is dead -Deadsnakes -Anaconda -Tox? I only know this is as a cross-version/implementation test runner -Vagrant. This is what I eventually did, and this was very simple. I ran Ubuntu 10.0.4 LTS, which uses Python 2.6, and used Vagrant SSH to run and check my code in Python 2.6 (and I replaced a dict comprehension with a list comprehension, for example) - ... What is the recommended way? I don't expect/hope that I'd ever need something lower than Python 2.5 Thank you. Albert-Jan *) Make altinstall sudo apt-get install libsqlite3-dev libbz2-dev libgdbm-dev libncurses5-dev tk-dev zlib1g-dev wget https://www.python.org/ftp/python/2.6.8/Python-2.6.8.tgz tar -zxvf Python-2.6.8.tgz cd Python-2.6.8/ ./configure --prefix=/usr/local make # see 'failed stuff' below sudo make altinstall mkvirtualenv -p /usr/local/bin/python2.6 python26 # ImportError: No module named zlib # Failed stuff Failed to find the necessary bits to build these modules: _bsddb _curses _curses_panel _hashlib _sqlite3 _ssl bsddb185 bz2 dbm dl gdbm imageop linuxaudiodev ossaudiodev readline sunaudiodev zlib ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ All right, but apart from the sanitation, the medicine, education, wine, public order, irrigation, roads, a fresh water system, and public health, what have the Romans ever done for us? ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ From joel.goldstick at gmail.com Sun Oct 12 15:15:28 2014 From: joel.goldstick at gmail.com (Joel Goldstick) Date: Sun, 12 Oct 2014 09:15:28 -0400 Subject: [Tutor] what is the easiest way to install different Python versions? In-Reply-To: <1798207457.287910.1413117348399.JavaMail.yahoo@jws10769.mail.gq1.yahoo.com> References: <1798207457.287910.1413117348399.JavaMail.yahoo@jws10769.mail.gq1.yahoo.com> Message-ID: <CAPM-O+wtA2cBGvxU7wZSsKBFX5rBDu24FSiC=XGQ=Sj5dKEFMw@mail.gmail.com> On Sun, Oct 12, 2014 at 8:35 AM, Albert-Jan Roskam <fomcl at yahoo.com.dmarc.invalid> wrote: > Hi, > > (sorry for cross-posting, sort of) > > A few days ago I needed to check whether some Python code ran with Python 2.6. What is the easiest way to install another Python version along side the default Python version? My own computer is Debian Linux 64 bit, but a platform-independent solution would be best. > > Possible solutions that I am aware of > > -make altinstall *). This is what I tried (see below), but not all modules could be built. I gave up because I was in a hurry > -Pythonbrew. This project is dead > -Deadsnakes > -Anaconda > -Tox? I only know this is as a cross-version/implementation test runner > -Vagrant. This is what I eventually did, and this was very simple. I ran Ubuntu 10.0.4 LTS, which uses Python 2.6, and used Vagrant SSH to run and check my code in Python 2.6 (and I replaced a dict comprehension with a list comprehension, for example) > - ... > > What is the recommended way? I don't expect/hope that I'd ever need something lower than Python 2.5 > > Using virtualenvwrapper is easy and isolates your environment http://virtualenvwrapper.readthedocs.org/en/latest/index.html > > Thank you. > > Albert-Jan > > > > > *) Make altinstall > sudo apt-get install libsqlite3-dev libbz2-dev libgdbm-dev libncurses5-dev tk-dev zlib1g-dev > wget https://www.python.org/ftp/python/2.6.8/Python-2.6.8.tgz > tar -zxvf Python-2.6.8.tgz > cd Python-2.6.8/ > ./configure --prefix=/usr/local > make # see 'failed stuff' below > sudo make altinstall > mkvirtualenv -p /usr/local/bin/python2.6 python26 # ImportError: No module named zlib > > > # Failed stuff > Failed to find the necessary bits to build these modules: > _bsddb _curses _curses_panel > _hashlib _sqlite3 _ssl > bsddb185 bz2 dbm > dl gdbm imageop > linuxaudiodev ossaudiodev readline > sunaudiodev zlib > > > > > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > > All right, but apart from the sanitation, the medicine, education, wine, public order, irrigation, roads, a > > fresh water system, and public health, what have the Romans ever done for us? > > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > _______________________________________________ > Tutor maillist - Tutor at python.org > To unsubscribe or change subscription options: > https://mail.python.org/mailman/listinfo/tutor -- Joel Goldstick http://joelgoldstick.com From fomcl at yahoo.com Sun Oct 12 15:18:47 2014 From: fomcl at yahoo.com (Albert-Jan Roskam) Date: Sun, 12 Oct 2014 13:18:47 +0000 (UTC) Subject: [Tutor] what is the easiest way to install different Python versions? In-Reply-To: <CAPM-O+wtA2cBGvxU7wZSsKBFX5rBDu24FSiC=XGQ=Sj5dKEFMw@mail.gmail.com> References: <CAPM-O+wtA2cBGvxU7wZSsKBFX5rBDu24FSiC=XGQ=Sj5dKEFMw@mail.gmail.com> Message-ID: <998879688.288775.1413119927126.JavaMail.yahoo@jws10753.mail.gq1.yahoo.com> Regards, Albert-Jan ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ All right, but apart from the sanitation, the medicine, education, wine, public order, irrigation, roads, a fresh water system, and public health, what have the Romans ever done for us? ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ----- Original Message ----- > From: Joel Goldstick <joel.goldstick at gmail.com> > To: Albert-Jan Roskam <fomcl at yahoo.com> > Cc: Python Mailing List <tutor at python.org> > Sent: Sunday, October 12, 2014 3:15 PM > Subject: Re: [Tutor] what is the easiest way to install different Python versions? > > On Sun, Oct 12, 2014 at 8:35 AM, Albert-Jan Roskam > <fomcl at yahoo.com.dmarc.invalid> wrote: >> Hi, >> >> (sorry for cross-posting, sort of) >> >> A few days ago I needed to check whether some Python code ran with Python > 2.6. What is the easiest way to install another Python version along side the > default Python version? My own computer is Debian Linux 64 bit, but a > platform-independent solution would be best. >> >> Possible solutions that I am aware of >> >> -make altinstall *). This is what I tried (see below), but not all modules > could be built. I gave up because I was in a hurry >> -Pythonbrew. This project is dead >> -Deadsnakes >> -Anaconda >> -Tox? I only know this is as a cross-version/implementation test runner >> -Vagrant. This is what I eventually did, and this was very simple. I ran > Ubuntu 10.0.4 LTS, which uses Python 2.6, and used Vagrant SSH to run and check > my code in Python 2.6 (and I replaced a dict comprehension with a list > comprehension, for example) >> - ... >> >> What is the recommended way? I don't expect/hope that I'd ever need > something lower than Python 2.5 >> >> > > Using virtualenvwrapper is easy and isolates your environment > > http://virtualenvwrapper.readthedocs.org/en/latest/index.html But then it's still required that Python 2.6 is installed on my system, right? This seems to confirm this: http://stackoverflow.com/questions/1534210/use-different-python-version-with-virtualenv (I like the --python or -p flag of virtualenv and virtualenvwrapper though) From juan0christian at gmail.com Sun Oct 12 15:26:12 2014 From: juan0christian at gmail.com (Juan Christian) Date: Sun, 12 Oct 2014 10:26:12 -0300 Subject: [Tutor] How to use custom protocol with requests? In-Reply-To: <CAGZAPF7ouiu6ef=saDCaxuet-HbaubsGYOW-pg08uB=SZ=3-rg@mail.gmail.com> References: <CAAp0bGsZKmyA+hyPcnrMS4w3b3DnsHbF7H=-a-iMs6H6MfAa5Q@mail.gmail.com> <CAGZAPF6iB2yi7v32ev=7APFsGVsNZtWTWcJN2PvyHgwNvO3KJA@mail.gmail.com> <CAGZAPF6RvBjR1iuHCAGiQh9U40T+jhGL6eiQBUc_KC+gxCh9yA@mail.gmail.com> <CAAp0bGsgFwMws8oGAbaZgKBty-zSFZAexKCuhN35VwsyfN6AdQ@mail.gmail.com> <CAAp0bGsoax1fRn3vmjciS0sQiAGfSwsLiuwbGUT4FYo7HWmBrw@mail.gmail.com> <CAAp0bGth16gKeiLfFMi2TfoarUKGWBoYSp-YPm6DVOSduzh1wQ@mail.gmail.com> <CAGZAPF7ouiu6ef=saDCaxuet-HbaubsGYOW-pg08uB=SZ=3-rg@mail.gmail.com> Message-ID: <CAAp0bGtbfg1KgsVXdp0Jic-JDJg74T9q+TtzTAWSCUZiVO9qZQ@mail.gmail.com> On Sun, Oct 12, 2014 at 12:17 AM, Danny Yoo <dyoo at hashcollision.org> wrote: > Huh. Wow. That actually worked? > > :P > > --- > > Frankly speaking though, this sounds like a horrible XSRF-style attack > in waiting, if I understand what has just happened. > (http://en.wikipedia.org/wiki/Cross-site_request_forgery) > > Usually, requests to do mutation operations are protected so that, in > order to make the request, you have to have some knowledge in the > request that's specific to the user, and not public knowledge. The > URL you've described is missing this basic information, an "XSRF > token" as its commonly known (though I would have assumed it would be > called an "anti-XSRF" token, but oh well.) > > I'm not sure how your web browser is handling the 'steam://' URL > class, but I would very much hope that, in the interface between the > browser and your Steam client, it's doing something to mitigate what > looks like an XSRF exploit. > Well, the person needs to be logged in the browser (maybe cookies are set for that), when I trigger that in the browser it automatically opens the Steam software installed in the computer and add the person. I don't know if it's a flaw, but it's very useful for what I'm doing. If you go to ANY profile on Steam (after logged in), let's say ' http://steamcommunity.com/profiles/<ID_HERE>', you can add the person, that simple. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/tutor/attachments/20141012/d818bae7/attachment-0001.html> From fomcl at yahoo.com Sun Oct 12 15:24:08 2014 From: fomcl at yahoo.com (Albert-Jan Roskam) Date: Sun, 12 Oct 2014 13:24:08 +0000 (UTC) Subject: [Tutor] what is the easiest way to install different Python versions? In-Reply-To: <1798207457.287910.1413117348399.JavaMail.yahoo@jws10769.mail.gq1.yahoo.com> References: <1798207457.287910.1413117348399.JavaMail.yahoo@jws10769.mail.gq1.yahoo.com> Message-ID: <1081777067.291567.1413120248648.JavaMail.yahoo@jws10795.mail.gq1.yahoo.com> ----- Original Message ----- > From: Albert-Jan Roskam <fomcl at yahoo.com.dmarc.invalid> > To: Python Mailing List <tutor at python.org> > Cc: > Sent: Sunday, October 12, 2014 2:35 PM > Subject: [Tutor] what is the easiest way to install different Python versions? > > Hi, > > (sorry for cross-posting, sort of) Oops, now I see that I posted it to Python Tutor again. Sorry. I intended to send this to the main Python list as this question is perhaps a bit out of scope for Tutor. From wbecerra1 at gmail.com Sun Oct 12 16:38:54 2014 From: wbecerra1 at gmail.com (William Becerra) Date: Sun, 12 Oct 2014 16:38:54 +0200 Subject: [Tutor] Return Statement error Message-ID: <CAF1DC4Vt_p1o9jJgdVb4sYso-be7tsBTSgXUcKe7Ocj-wXug_Q@mail.gmail.com> Hello, I'm new to programming. I'm using Python 2.7.8 and Windows 8 OS I was making an application to see if I understand how the return statement works I want my application to compare x and y and return either 1, -1 or 0. I'm using IDLE Here is my code: print"Please write a value for x" x = raw_input() print "Please write a value for y" y = raw_input() if x > y: return 1 elif x < y: return -1 elif x == y: return 0 else: return "Please insert a Valid character" When I press F5 to run my code a new window called syntax error appears The window says the following There's an error in your program: ***'return' outside function (C:/Users/William/Desktop/Python Files/Function compare.py, line6) What am I doing Wrong? I noticed that if i substitute all the return keywords with print the code runs correctly. However I want to use return as I am trying to learn how it works. Thank You. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/tutor/attachments/20141012/094c92be/attachment.html> From joel.goldstick at gmail.com Sun Oct 12 19:02:55 2014 From: joel.goldstick at gmail.com (Joel Goldstick) Date: Sun, 12 Oct 2014 13:02:55 -0400 Subject: [Tutor] Return Statement error In-Reply-To: <CAF1DC4Vt_p1o9jJgdVb4sYso-be7tsBTSgXUcKe7Ocj-wXug_Q@mail.gmail.com> References: <CAF1DC4Vt_p1o9jJgdVb4sYso-be7tsBTSgXUcKe7Ocj-wXug_Q@mail.gmail.com> Message-ID: <CAPM-O+xFvtn=UEoaVWtZL_sGRd+C4qYxGvbXPxnkaBNXybkxSg@mail.gmail.com> On Sun, Oct 12, 2014 at 10:38 AM, William Becerra <wbecerra1 at gmail.com> wrote: > Hello, I'm new to programming. I'm using Python 2.7.8 and Windows 8 OS > I was making an application to see if I understand how the return statement > works > I want my application to compare x and y and return either 1, -1 or 0. > I'm using IDLE > > Here is my code: > print"Please write a value for x" > x = raw_input() > print "Please write a value for y" > y = raw_input() > if x > y: > return 1 > elif x < y: > return -1 > elif x == y: > return 0 > else: > return "Please insert a Valid character" > > > When I press F5 to run my code a new window called syntax error appears > The window says the following > There's an error in your program: > ***'return' outside function (C:/Users/William/Desktop/Python Files/Function > compare.py, line6) > > What am I doing Wrong? return is only allowed inside a function -- not in main line code You can wrap your code into a function and then just call it to see what it returns > I noticed that if i substitute all the return keywords with print the code > runs correctly. > However I want to use return as I am trying to learn how it works. > Thank You. > > _______________________________________________ > Tutor maillist - Tutor at python.org > To unsubscribe or change subscription options: > https://mail.python.org/mailman/listinfo/tutor > -- Joel Goldstick http://joelgoldstick.com From danny.yoo at gmail.com Sun Oct 12 20:24:43 2014 From: danny.yoo at gmail.com (Danny Yoo) Date: Sun, 12 Oct 2014 11:24:43 -0700 Subject: [Tutor] Return Statement error In-Reply-To: <CAF1DC4Vt_p1o9jJgdVb4sYso-be7tsBTSgXUcKe7Ocj-wXug_Q@mail.gmail.com> References: <CAF1DC4Vt_p1o9jJgdVb4sYso-be7tsBTSgXUcKe7Ocj-wXug_Q@mail.gmail.com> Message-ID: <CAGZAPF4XUqBh8GwYg0uHsFtsZixUmw7nL=Ej2B3nQ9fWPp7J=w@mail.gmail.com> On Oct 12, 2014 9:52 AM, "William Becerra" <wbecerra1 at gmail.com> wrote: > > Hello, I'm new to programming. I'm using Python 2.7.8 and Windows 8 OS > I was making an application to see if I understand how the return statement works > I want my application to compare x and y and return either 1, -1 or 0. > I'm using IDLE The return statement is intimately tied to functions: look for tutorial content about functions. For example: http://www.greenteapress.com/thinkpython/html/thinkpython004.html If you have questions, please feel free to ask. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/tutor/attachments/20141012/3f228c5e/attachment.html> From steve at pearwood.info Mon Oct 13 01:01:53 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Mon, 13 Oct 2014 10:01:53 +1100 Subject: [Tutor] Return Statement error In-Reply-To: <CAF1DC4Vt_p1o9jJgdVb4sYso-be7tsBTSgXUcKe7Ocj-wXug_Q@mail.gmail.com> References: <CAF1DC4Vt_p1o9jJgdVb4sYso-be7tsBTSgXUcKe7Ocj-wXug_Q@mail.gmail.com> Message-ID: <20141012230153.GI23419@ando.pearwood.info> On Sun, Oct 12, 2014 at 04:38:54PM +0200, William Becerra wrote: > Hello, I'm new to programming. I'm using Python 2.7.8 and Windows 8 OS > I was making an application to see if I understand how the return statement > works The `return` statement can only be used inside a function. That means you have to start off with a `def` line, and indent your code. Have you learned about functions yet? If not, perhaps you might prefer to forget about `return` until you do. Otherwise, if you take the code you wrote, indent it, and put a function declaration at the top, you should be able to use `return` successfully: def compare(): print "Please write a value for x" x = raw_input() print "Please write a value for y" y = raw_input() if x > y: return 1 elif x < y: return -1 elif x == y: return 0 else: return "this will never happen" Then, once you have defined your function, you can call it: result = compare() # don't forget the parentheses () print "And the result is", result -- Steven From wbecerra1 at gmail.com Mon Oct 13 07:48:53 2014 From: wbecerra1 at gmail.com (William Becerra) Date: Mon, 13 Oct 2014 07:48:53 +0200 Subject: [Tutor] Return Statement error In-Reply-To: <20141012230153.GI23419@ando.pearwood.info> References: <CAF1DC4Vt_p1o9jJgdVb4sYso-be7tsBTSgXUcKe7Ocj-wXug_Q@mail.gmail.com> <20141012230153.GI23419@ando.pearwood.info> Message-ID: <CAF1DC4XPqnjst5r_xVA2JF_0FAOmEYqSTnHQ2moNMEXeAKHxZw@mail.gmail.com> I am familiar with funtions, i didn't realize i had to write the return statement inside a function...Thank you all..that was very helpful On 13 Oct 2014 01:03, "Steven D'Aprano" <steve at pearwood.info> wrote: > On Sun, Oct 12, 2014 at 04:38:54PM +0200, William Becerra wrote: > > Hello, I'm new to programming. I'm using Python 2.7.8 and Windows 8 OS > > I was making an application to see if I understand how the return > statement > > works > > The `return` statement can only be used inside a function. That means > you have to start off with a `def` line, and indent your code. > > Have you learned about functions yet? If not, perhaps you might prefer > to forget about `return` until you do. Otherwise, if you take the code > you wrote, indent it, and put a function declaration at the top, you > should be able to use `return` successfully: > > def compare(): > print "Please write a value for x" > x = raw_input() > print "Please write a value for y" > y = raw_input() > if x > y: > return 1 > elif x < y: > return -1 > elif x == y: > return 0 > else: > return "this will never happen" > > > Then, once you have defined your function, you can call it: > > result = compare() # don't forget the parentheses () > print "And the result is", result > > > > -- > Steven > _______________________________________________ > Tutor maillist - Tutor at python.org > To unsubscribe or change subscription options: > https://mail.python.org/mailman/listinfo/tutor > -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/tutor/attachments/20141013/402d6b0a/attachment-0001.html> From crusier at gmail.com Mon Oct 13 09:54:59 2014 From: crusier at gmail.com (Crusier) Date: Mon, 13 Oct 2014 15:54:59 +0800 Subject: [Tutor] Compare two text files Message-ID: <CAC7HCj-z0a1g-HoZiuBLTKivO43oP2a2bsVkzmWzRb9e9nHz=Q@mail.gmail.com> Hi Alan, Attached are the two text files (stocklist.txt & stocklist1.txt) which I want to do a comparison with the content of the file, Basically, I want to see if there are any new numbers added to the file. Please comment on the sequence of the file: 1. First, Open up the File No. 1 and put the string into a list. 2. Second, Open the File No. 2 and put the string into a list. 3. Use difflib to compare This is some of the code I had written. #Remove ".HK" from the stock list def remove_HK(): f = open('stock_list.txt', "r") output = open('stock_lista.txt', "w") output.write(f.read().replace(".hk","").replace(".HK","")) f.close() output.close() remove_HK() My thinking is first remove ".HK" so I could compare in as int to another text file. Thank you Henry -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/tutor/attachments/20141013/14f8b5a6/attachment.html> -------------- next part -------------- 1728.HK 1033.HK 2393.HK 0968.HK 3378.HK 3049.HK 1661.HK 8269.HK 3393.HK 0151.HK 0303.HK 0345.HK 0913.HK 0220.HK 0696.HK 0570.HK 3886.HK 2283.HK 3382.HK 0882.HK 1065.HK 0826.HK 3823.HK 1613.HK 1228.HK 2382.HK 1089.HK 0981.HK 0598.HK 1099.HK 0361.HK 1177.HK 0750.HK 0444.HK 0460.HK 2877.HK 2313.HK 0152.HK 0747.HK 2607.HK 0563.HK 2727.HK 0205.HK 8047.HK 1004.HK 2010.HK 8201.HK 1345.HK 2328.HK 1515.HK 8311.HK 0402.HK 1323.HK 8180.HK 0553.HK 1618.HK 0231.HK 2186.HK 1108.HK 8058.HK 8237.HK 1212.HK 0381.HK 6136.HK 1638.HK 3336.HK 0419.HK 2211.HK 0923.HK 0438.HK 0091.HK 0167.HK 1886.HK 1071.HK 0336.HK 2811.HK 6823.HK 8292.HK 0911.HK 0566.HK 1367.HK 2208.HK 0283.HK 0530.HK 0175.HK 3800.HK 0451.HK 0500.HK 0038.HK 8123.HK 8018.HK 3360.HK 0729.HK 1856.HK 1808.HK 1330.HK 0895.HK 1072.HK 2880.HK 3898.HK 0080.HK 0867.HK 0471.HK 2722.HK 1060.HK 1313.HK 1333.HK 0728.HK 2198.HK 2380.HK 0572.HK 1185.HK 0085.HK 0217.HK 0370.HK 0031.HK 1196.HK 2623.HK 0476.HK 1375.HK 0996.HK 2324.HK 3188.HK 1848.HK 6828.HK 8321.HK 0285.HK 0154.HK 2357.HK 0232.HK 0161.HK 1803.HK 0899.HK 2020.HK 1131.HK -------------- next part -------------- 0471.HK 3800.HK 0728.HK 1033.HK 1099.HK 2357.HK 0566.HK 2328.HK 0232.HK 0729.HK 2208.HK 0598.HK 2186.HK 0231.HK 0175.HK 0981.HK 0285.HK 0460.HK 0553.HK 2382.HK 0031.HK 0747.HK 3188.HK 1071.HK 3382.HK 3823.HK 3898.HK 0451.HK 2727.HK 0968.HK 0750.HK 1680.HK 6136.HK 1072.HK 6823.HK 1177.HK 2020.HK 0419.HK 6828.HK 1060.HK 8047.HK 0867.HK 0336.HK 1848.HK 1856.HK 1313.HK 2607.HK 3886.HK 8292.HK 1618.HK 0572.HK 2211.HK 3336.HK 2313.HK 0220.HK 1323.HK 1638.HK 1185.HK 1004.HK 1808.HK 8321.HK 0205.HK 2623.HK 2393.HK 0161.HK 1613.HK 0855.HK 8201.HK 0882.HK 1212.HK 0696.HK 1375.HK 0091.HK 0038.HK 0911.HK 3360.HK 0085.HK 1333.HK 0152.HK 1522.HK 0570.HK 0938.HK 1330.HK 2880.HK 3049.HK 0546.HK 2198.HK 1108.HK 8237.HK 2380.HK 0996.HK 0402.HK 0036.HK 0732.HK 0444.HK 0895.HK 3393.HK 1345.HK 0476.HK 1369.HK 1131.HK 1228.HK 0154.HK 0548.HK 8123.HK 0899.HK 0718.HK 2322.HK 0926.HK 1661.HK 1089.HK 0811.HK 0433.HK 83188.HK 0303.HK 1728.HK 0260.HK 0107.HK 2348.HK 1599.HK 1065.HK 8311.HK 8018.HK 0530.HK 8207.HK 0440.HK 1308.HK 0564.HK 0568.HK From davea at davea.name Mon Oct 13 11:30:53 2014 From: davea at davea.name (Dave Angel) Date: Mon, 13 Oct 2014 05:30:53 -0400 (EDT) Subject: [Tutor] Compare two text files References: <CAC7HCj-z0a1g-HoZiuBLTKivO43oP2a2bsVkzmWzRb9e9nHz=Q@mail.gmail.com> Message-ID: <m1g5uc$iq6$1@ger.gmane.org> Crusier <crusier at gmail.com> Wrote in message: > Attached are the two text files (stocklist.txt & stocklist1.txt) which I want to do a comparison with the content of the file, Basically, I want to see if there are any new numbers added to the file. > Please comment on the sequence of the file: 1. First, Open up the File No. 1 and put the string into a list. 2. Second, Open the File No. 2 and put the string into a list. 3. Use difflib to compare I don't see what the included code had to do with the problem, since difflib doesn?t care about numbers. It compares sequences (like lists) of strings. So you have a couple of files to read in. Normally you might use readlines, but there don't seem to be any newlines in the files. So you'll need split, or something similar. Come back when you've made an attempt at the problem, and ask a question about your code. Otherwise you're just asking others to do your homework for you. That's no way to learn. And please post in plain text. Html messages can cause lots of problems. -- DaveA From steve at pearwood.info Mon Oct 13 11:32:16 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Mon, 13 Oct 2014 20:32:16 +1100 Subject: [Tutor] Compare two text files In-Reply-To: <CAC7HCj-z0a1g-HoZiuBLTKivO43oP2a2bsVkzmWzRb9e9nHz=Q@mail.gmail.com> References: <CAC7HCj-z0a1g-HoZiuBLTKivO43oP2a2bsVkzmWzRb9e9nHz=Q@mail.gmail.com> Message-ID: <20141013093216.GJ23419@ando.pearwood.info> On Mon, Oct 13, 2014 at 03:54:59PM +0800, Crusier wrote: > Hi Alan, > > Attached are the two text files (stocklist.txt & stocklist1.txt) which I > want to do a comparison with the content of the file, Basically, I want to > see if there are any new numbers added to the file. > > Please comment on the sequence of the file: > 1. First, Open up the File No. 1 and put the string into a list. > 2. Second, Open the File No. 2 and put the string into a list. > 3. Use difflib to compare Sounds like a good plan to me. Something like this would work: py> import difflib py> a = '1728.HK 1033.HK 2393.HK 0968.HK 3378.HK'.split() py> b = '1728.HK 1036.HK 2393.HK 0968.HK 2784.HK 3378.HK'.split() py> print '\n'.join(difflib.unified_diff(a, b)) --- +++ @@ -1,5 +1,6 @@ 1728.HK -1033.HK +1036.HK 2393.HK 0968.HK +2784.HK 3378.HK Obviously you don't type in the strings '1728.HK...', you read them from the files you wish to compare. -- Steven From ofir_l at walla.com Mon Oct 13 12:40:45 2014 From: ofir_l at walla.com (=?UTF-8?Q?=D7=90=D7=95=D7=A4=D7=99=D7=A8=20=D7=9C=D7=99=D7=A8=D7=95=D7=9F?=) Date: Mon, 13 Oct 2014 13:40:45 +0300 Subject: [Tutor] =?utf-8?q?Help_with_guess_my_number_game?= Message-ID: <1413196845.709000-19643650-15260@walla.com> An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/tutor/attachments/20141013/b03ba823/attachment.html> From dyoo at hashcollision.org Mon Oct 13 22:06:26 2014 From: dyoo at hashcollision.org (Danny Yoo) Date: Mon, 13 Oct 2014 13:06:26 -0700 Subject: [Tutor] Help with guess my number game In-Reply-To: <1413196845.709000-19643650-15260@walla.com> References: <1413196845.709000-19643650-15260@walla.com> Message-ID: <CAGZAPF5NyV2j-KfbKr6m_+4Ae1cEXZ4agVKgRkgn7RQLJqVCEA@mail.gmail.com> > > if guess != the_number: > > print ("you failed, the number was", the_number) > > elif guess==the_number: > > print("You guessed it! The number was", the_number) > > print("And it only took you", tries, "tries!\n") This block of code appears to be applied for every iteration through your loop. Is that your intention? From dyoo at hashcollision.org Mon Oct 13 22:43:10 2014 From: dyoo at hashcollision.org (Danny Yoo) Date: Mon, 13 Oct 2014 13:43:10 -0700 Subject: [Tutor] Fwd: Re: Help with guess my number game In-Reply-To: <1413231639.190000-93791214-29830@walla.com> References: <1413231639.190000-93791214-29830@walla.com> Message-ID: <CAGZAPF75-OV3gW55aj1Ay9MOLsNRTxMoJsgADax9CMGAkB954Q@mail.gmail.com> ---------- Forwarded message ---------- From: ????? ????? <ofir_l at walla.com> Date: Mon, Oct 13, 2014 at 1:20 PM Subject: Re: Re: [Tutor] Help with guess my number game To: Danny Yoo <dyoo at hashcollision.org> Hi Danny Thanks for your response, I think it is part of the problem, but my intention was to end the first loop only after 5 tries. I don't understand why the code go to the print section after only two gusses and not five. and it also reveal the_number before the game end. Again thank for your help. Ofir ________________________________ ????: Danny Yoo, ????: Re: [Tutor] Help with guess my number game > > if guess != the_number: > > print ("you failed, the number was", the_number) > > elif guess==the_number: > > print("You guessed it! The number was", the_number) > > print("And it only took you", tries, "tries!n") This block of code appears to be applied for every iteration through your loop. Is that your intention? ________________________________ Walla! Mail - Get your free unlimited mail today From alan.gauld at btinternet.com Tue Oct 14 01:39:00 2014 From: alan.gauld at btinternet.com (Alan Gauld) Date: Tue, 14 Oct 2014 00:39:00 +0100 Subject: [Tutor] Help with guess my number game In-Reply-To: <1413196845.709000-19643650-15260@walla.com> References: <1413196845.709000-19643650-15260@walla.com> Message-ID: <m1hnqk$i2o$1@ger.gmane.org> On 13/10/14 11:40, ????? ????? wrote: > # set the initial values > > the_number = random.randint(1, 100) > guess = int(input("Take a guess: ")) > tries = 1 > > # guessing loop > while guess != the_number: > if guess > the_number: > print("Lower...") > else: > print("Higher...") > guess = int(input("Take a guess: ")) > > tries += 1 > if tries > 5: > break so far so good.... almost... > > if guess != the_number: > print ("you failed, the number was", the_number) This is still inside the loop. You want to remove the indentation so this only happens after you exit the loop. Otherwise you tell the user the answer before they guess it (or have 5 goes) and it doesn't work right if the first guess is correct... > input("\n\nPress the enter key to exit.") You need the if/else to look like this. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.flickr.com/photos/alangauldphotos From ofir_l at walla.com Tue Oct 14 18:23:33 2014 From: ofir_l at walla.com (=?UTF-8?Q?=D7=90=D7=95=D7=A4=D7=99=D7=A8=20=D7=9C=D7=99=D7=A8=D7=95=D7=9F?=) Date: Tue, 14 Oct 2014 19:23:33 +0300 Subject: [Tutor] =?utf-8?q?Tutor_Digest=2C_Vol_128=2C_Issue_33?= Message-ID: <1413303813.070000-70028564-17186@walla.com> An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/tutor/attachments/20141014/6124f548/attachment.html> From crk at godblessthe.us Thu Oct 16 00:27:30 2014 From: crk at godblessthe.us (Clayton Kirkwood) Date: Wed, 15 Oct 2014 15:27:30 -0700 Subject: [Tutor] well I finally got finished with this part of my exercise Message-ID: <007901cfe8c7$35fd6a90$a1f83fb0$@us> No, it hasn't taken the whole time, but it did take longer than I wanted, but I did progress as seen below. 1st pass - get the idea down: #program to capture streaming quotes from yahoo import re out_file = open("output", mode='w') in_file = open('Yahoo!_Finance_Portfolios.htm', encoding= "utf-8" ) if not (in_file.readline().startswith("<!DOCTYPE html>")): #first line isn't an HTML exit(44) line_in, line_no = ' ', 0 for line_in in in_file: print( line_in[:100], flush = True ) # check out the line for debug. [:100] cuz charset error on long string line_no += 1 cap_end = 0 print(line_no, '\"', line_in[:120], '\"', flush = True) while True: pattern = re.compile(r'.+?<tr data-row-symbol=\"([A-Z\.\-\^]+?)\"') #continuing on the line a = pattern.search( line_in, cap_end ) if not a: break print( "line_no=", line_no, 'pattern=\"', pattern, '\"', ", a = \'", a, '\"', flush=True) stock = a.group(1) stock_end = a.end() pattern = re.compile(r'.+?wk52:low.+?>([\d\.]+?)<') a = pattern.search( line_in, stock_end ) low_52 = a.group(1) low_end = a.end() pattern = re.compile(r'.+?prevclose:.+?>([\d\.\,]+?)<') a = pattern.search( line_in, low_end ) prev = a.group(1) prev_end = a.end() pattern = re.compile(r'.+?mkt:open.+?>([\d\.\,]+?)<') a = pattern.search( line_in, prev_end ) mkt_open = a.group(1) mkt_end = a.end() pattern = re.compile(r'.+?mkt:low.+?>([\d\.\,]+?)<' ) a = pattern.search(line_in, mkt_end) mkt_low = a.group(1) mkt_end = a.end() pattern = re.compile(r'.+?mkt:lsttrdtime.+?>(.+?)<' ) a = pattern.search( line_in, mkt_end ) last_time = a.group(1) last_end = a.end() pattern = re.compile(r'.+?tquote:value.+?>([\d\.\,]+?)<' ) a = pattern.search( line_in, last_end ) stock_price = a.group(1) stock_end = a.end() pattern =re.compile(r'.+?mkt:chg.+?>([\-\+][\d\.\,]+?)<' ) a = pattern.search(line_in, stock_end ) mkt_change = a.group(1) mkt_end = a.end() pattern = re.compile(r'.+?mkt:pctchg.+?>([\+\-][\d\.\,]+?%)<' ) a = pattern.search( line_in, mkt_end ) pct_change = a.group(1) pct_end = a.end() pattern = re.compile(r'.+?mkt:high.+?>([\d\.\,]+?)<' ) a = pattern.search( line_in, pct_end ) mkt_high = a.group(1) mkt_end = a.end() pattern = re.compile(r'.+?wk52:high.+?>([\d\.\,]+?)<' ) a = pattern.search( line_in, mkt_end ) high_52 = a.group(1) high_end = a.end() pattern = re.compile(r'.+?mkt:vol.+?>([\d\,]+?)<' ) a = pattern.search( line_in, high_end ) vol = a.group(1) vol_end = a.end() pattern = re.compile(r'.+?mkt.avgVol.+?>([\d\,]+?)<' ) a = pattern.search( line_in, vol_end ) vol_avg = a.group(1) vol_end = a.end() pattern = re.compile(r'.+?dividend_per_share.+?datum">([\d\.\,-]+?)<' ) a = pattern.search( line_in, vol_end ) div_share = a.group(1) div_end = a.end() pattern = re.compile(r'.+?dividend_yield.+?>([\d\.-]+?)<' ) a = pattern.search(line_in, div_end ) div_yield = a.group(1) div_end = a.end() pattern = re.compile(r".+?market_cap.+?datum.?>([\d\.\,-]+?[A-Z])<") a = pattern.search(line_in, div_end ) cap = a.group(1) cap_end = a.end() print( stock, low_52, prev, mkt_open, mkt_low, last_time, stock_price, mkt_change, pct_change, mkt_high, high_52, vol, vol_avg, div_share, div_yield, cap, flush=True ) out_file.closed 2nd pass: #program to capture streaming quotes from yahoo import re out_file = open("output", mode='w') in_file = open('Yahoo!_Finance_Portfolios.htm', encoding= "utf-8" ) if not (in_file.readline().startswith("<!DOCTYPE html>")): #first line isn't an HTML exit(44) line_in = ' ' for line_in in in_file: # print( line_in[:100], flush = True ) # check out the line for debug. [:100] cuz charset error on long string string_end = 0 # print(line_no, '\"', line_in[:120], '\"', flush = True) #debug, flush needed cuz otherwise confused output while True: pattern = re.compile(r'.+?<tr data-row-symbol=\"([A-Z\.\-\^]+?)\"') #continuing on the line match = pattern.search( line_in, string_end ) if not match: # nothing here: look at next line break # out of inner loop stock = match.group(1) # pick up stock name string_end = match.end() pattern = re.compile(r'.+?wk52:low.+?>([\d\.]+?)<') match = pattern.search( line_in, string_end ) low_52 = match.group(1) string_end = match.end() pattern = re.compile(r'.+?prevclose:.+?>([\d\.\,]+?)<') match = pattern.search( line_in, string_end ) prev = match.group(1) string_end = match.end() pattern = re.compile(r'.+?mkt:open.+?>([\d\.\,]+?)<') match = pattern.search( line_in, string_end ) mkt_open = match.group(1) string_end = match.end() pattern = re.compile(r'.+?mkt:low.+?>([\d\.\,]+?)<' ) match = pattern.search(line_in, string_end) mkt_low = match.group(1) string_end = match.end() pattern = re.compile(r'.+?mkt:lsttrdtime.+?>(.+?)<' ) maatch = pattern.search( line_in, string_end ) last_time = match.group(1) string_end = match.end() pattern = re.compile(r'.+?tquote:value.+?>([\d\.\,]+?)<' ) match = pattern.search( line_in, string_end ) stock_price = match.group(1) string_end = match.end() pattern =re.compile(r'.+?mkt:chg.+?>([\-\+][\d\.\,]+?)<' ) match = pattern.search(line_in, string_end ) mkt_change = match.group(1) string_end = match.end() pattern = re.compile(r'.+?mkt:pctchg.+?>([\+\-][\d\.\,]+?%)<' ) match = pattern.search( line_in, string_end ) pct_change = match.group(1) string_end = match.end() pattern = re.compile(r'.+?mkt:high.+?>([\d\.\,]+?)<' ) match = pattern.search( line_in, string_end ) mkt_high = match.group(1) string_end = match.end() pattern = re.compile(r'.+?wk52:high.+?>([\d\.\,]+?)<' ) match = pattern.search( line_in, string_end ) high_52 = match.group(1) string_end = match.end() pattern = re.compile(r'.+?mkt:vol.+?>([\d\,]+?)<' ) match = pattern.search( line_in, string_end ) vol = match.group(1) string_end = match.end() pattern = re.compile(r'.+?mkt.avgVol.+?>([\d\,]+?)<' ) match = pattern.search( line_in, string_end ) vol_avg = match.group(1) string_end = match.end() pattern = re.compile(r'.+?dividend_per_share.+?datum">([\d\.\,-]+?)<' ) match = pattern.search( line_in, string_end ) div_share = match.group(1) string_end = match.end() pattern = re.compile(r'.+?dividend_yield.+?>([\d\.-]+?)<' ) match = pattern.search(line_in, string_end ) div_yield = match.group(1) string_end = match.end() pattern = re.compile(r".+?market_cap.+?datum.?>([\d\.\,-]+?[A-Z])<") match = pattern.search(line_in, string_end ) cap = match.group(1) string_end = match.end() # we'll use string_end at the beginning of loop print( stock, low_52, prev, mkt_open, mkt_low, last_time, stock_price, mkt_change, pct_change, mkt_high, high_52, vol, vol_avg, div_share, div_yield, cap, flush=True ) out_file.closed # always do the right thing final pass: #program to capture streaming quotes from yahoo import re def match_string( pattern, string_end ): pattern = re.compile( pattern ) match = pattern.search( line_in, string_end ) return(match.group(1), match.end()) out_file = open("output", mode='w') in_file = open('Yahoo!_Finance_Portfolios.htm', encoding= "utf-8" ) if not (in_file.readline().startswith("<!DOCTYPE html>")): #first line isn't an HTML exit(44) line_in = ' ' for line_in in in_file: # print( line_in[:100], flush = True ) # check out the line for debug. [:100] cuz charset error on long string string_end = 0 # print(line_no, '\"', line_in[:120], '\"', flush = True) #debug, flush needed cuz otherwise confused output while True: pattern = re.compile(r'.+?<tr data-row-symbol=\"([A-Z\.\-\^]+?)\"') #continuing on the line match = pattern.search( line_in, string_end ) if not match: # nothing here: look at next line break # out of inner loop stock = match.group(1) # pick up stock name string_end = match.end() # pattern = re.compile(r'.+?wk52:low.+?>([\d\.]+?)<') # match = pattern.search( line_in, string_end ) # low_52 = match.group(1) # string_end = match.end() (low_52, string_end) = match_string( r'.+?wk52:low.+?>([\d\.]+?)<', string_end) (prev, string_end) = match_string(r'.+?prevclose:.+?>([\d\.\,]+?)<', string_end) (mkt_open, string_end) = match_string(r'.+?mkt:open.+?>([\d\.\,]+?)<', string_end) (mkt_low, string_end) = match_string(r'.+?mkt:low.+?>([\d\.\,]+?)<', string_end) (last_time, string_end) = match_string(r'.+?mkt:lsttrdtime.+?>(.+?)<', string_end) (stock_price, string_end) = match_string(r'.+?tquote:value.+?>([\d\.\,]+?)<', string_end) (mkt_change, string_end) = match_string(r'.+?mkt:chg.+?>([\-\+][\d\.\,]+?)<', string_end) (pct_change, string_end) = match_string(r'.+?mkt:pctchg.+?>([\+\-][\d\.\,]+?%)<', string_end) (mkt_high, string_end) = match_string(r'.+?mkt:high.+?>([\d\.\,]+?)<', string_end ) (high_52, string_end) = match_string(r'.+?wk52:high.+?>([\d\.\,]+?)<', string_end) (vol, string_end) = match_string(r'.+?mkt:vol.+?>([\d\,]+?)<', string_end) (vol_avg, string_end) = match_string(r'.+?mkt.avgVol.+?>([\d\,]+?)<', string_end) (div_share, string_end) = match_string(r'.+?dividend_per_share.+?datum">([\d\.\,-]+?)<', string_end ) (div_yield, string_end) = match_string(r'.+?dividend_yield.+?>([\d\.-]+?)<', string_end ) (cap, string_end) = match_string(r".+?market_cap.+?datum.?>([\d\.\,-]+?[A-Z])<", string_end) print( stock, low_52, prev, mkt_open, mkt_low, last_time, stock_price, mkt_change, pct_change, mkt_high, high_52, vol, vol_avg, div_share, div_yield, cap, flush=True ) out_file.close() # always do the right thing So, I learned quite a bit about matching and found that .+? is very useful: it is the question mark. Swallows as little as possible. Learned something about a simple function and returns. All put out the same thing: EKSO 0.75 1.38 1.39 1.26 03:57pm EST 1.3000 -0.0800 -5.80% 1.49 8.22 660,578 596,448 - - 102.05M SWKS 23.27 52.23 50.13 44.06 04:00pm EST 47.01 -5.22 -9.99% 50.13 59.25 12,383,347 3,448,690 0.44 0.80 8.91B EROC 2.79 3.01 2.98 2.79 04:00pm EST 2.95 -0.06 -1.99% 3.05 7.88 716,374 608,405 0.60 18.80 464.19M RKUS 8.65 12.54 12.36 11.78 04:04pm EST 11.84 -0.70 -5.58% 12.39 18.58 1,243,329 1,345,410 - - 979.58M FES 2.93 3.18 3.14 3.07 04:00pm EST 3.08 -0.10 -3.14% 3.15 5.75 15,514 43,111 - - 67.12M WLB 13.38 35.60 35.41 34.37 04:00pm EST 34.45 -1.15 -3.23% 36.33 45.19 213,917 189,145 0.00 0.00 585.17M DANG 7.69 11.00 10.67 10.33 04:02pm EST 10.33 -0.67 -6.09% 11.02 19.05 1,767,816 2,327,650 - - 833.54M UEPS 7.03 11.77 11.72 11.26 04:00pm EST 11.29 -0.48 -4.08% 11.82 14.24 212,876 431,726 - - 535.52M GY 15.01 15.46 15.37 15.12 04:02pm EST 15.17 -0.29 -1.88% 15.53 19.77 339,871 505,269 0.00 0.00 890.48M NR 10.43 11.00 10.89 10.59 04:05pm EST 10.89 -0.11 -1.00% 11.06 13.64 1,005,357 740,434 - - 911.13M PQ 3.64 4.54 4.50 4.32 04:02pm EST 4.50 -0.04 -0.88% 4.65 7.82 893,841 799,475 - - 297.42M PLG 0.76 0.7873 0.7899 0.7823 04:00pm EST 0.7942 +0.0069 +0.88% 0.8143 1.37 159,580 161,960 - - 440.8642M CALX 7.12 8.93 8.85 8.27 04:05pm EST 8.31 -0.62 -6.94% 8.85 13.36 371,015 295,832 - - 423.1M SWC 10.42 14.08 13.93 13.82 04:03pm EST 13.85 -0.23 -1.63% 14.44 19.42 1,531,009 1,828,360 - - 1.66B BBG 16.06 17.26 16.84 16.06 04:04pm EST 16.10 -1.16 -6.72% 17.21 30.69 1,993,888 1,333,430 - - 773.22M IAU 11.51 11.84 11.85 11.80 04:00pm EST 11.85 +0.01 +0.08% 11.86 13.46 6,511,530 2,368,960 - 0.00 1.62B IDTI 9.18 14.04 13.74 12.62 04:00pm EST 12.685 -1.355 -9.65% 13.84 17.32 8,615,394 3,095,850 - - 1.887B GRH 0.8 1.30 1.16 1.16 04:00pm EST 1.1650 -0.1350 -10.38% 1.25 3.62 131,138 631,260 - - 42.93M ORBC 5.4 5.85 5.82 5.75 04:00pm EST 5.80 -0.05 -0.85% 5.97 8.21 148,377 174,626 - - 320.47M PTX 1.68 8.09 8.04 7.89 04:00pm EST 8.20 +0.11 +1.36% 8.45 9.56 890,188 425,780 0.00 0.00 311.95M AMAT 16.4 20.87 20.55 19.50 04:15pm EST 19.83 -1.04 -4.98% 20.59 23.46 33,135,379 12,219,800 0.40 2.00 24.16B ^DJI 0 1,928.21 1,925.63 1,906.05 04:28pm EST 1,906.13 -22.08 -1.15% 1,936.98 2,019.26 757,485,255 27,888,833 0.92 1.59 121.77M ADEP 6.85 7.34 7.29 6.85 04:00pm EST 6.89 -0.45 -6.13% 7.29 21.9 146,339 230,198 - - 90.02M CIMT 5.38 5.51 5.50 5.50 03:59pm EST 5.50 -0.01 -0.18% 5.54 10.75 45,323 79,577 0.00 0.00 59.24M DDD 39.09 40.86 40.31 39.09 04:00pm EST 39.14 -1.72 -4.21% 40.92 97.28 3,237,974 3,035,590 0.11 0.60 4.3B IRBT 28.9 31.82 31.65 31.42 04:00pm EST 32.00 +0.18 +0.57% 32.32 48.36 892,183 661,142 - - 945.7M ONVO 5.12 6.00 5.86 5.53 04:00pm EST 5.56 -0.44 -7.33% 5.98 13.65 962,106 1,057,330 - - 436.34M VJET 11.9 12.84 12.55 11.90 04:00pm EST 12.09 -0.75 -5.84% 12.87 70 408,697 453,477 - - 221.28M XONE 16 18.21 18.11 17.81 04:00pm EST 19.78 +1.57 +8.62% 20.75 70.25 941,813 577,675 - - 285.66M AXAS 2.81 4.13 4.09 3.79 04:00pm EST 3.84 -0.29 -7.02% 4.13 6.45 4,336,860 2,590,600 - - 400.34M CMLP 19.01 19.92 19.79 19.01 04:03pm EST 19.03 -0.89 -4.47% 19.79 24.94 1,212,528 614,751 1.64 7.70 3.58B CORR 6.11 7.33 7.27 7.19 04:02pm EST 7.20 -0.13 -1.77% 7.39 8.54 103,846 132,263 0.50 7.70 10.58B FET 24.32 28.50 28.34 27.58 04:03pm EST 27.66 -0.84 -2.95% 28.51 37.03 343,356 423,854 - - 2.56B GDP 10.61 11.22 11.20 10.62 04:02pm EST 10.83 -0.39 -3.48% 11.82 30.52 2,935,299 1,686,260 - - 481.14M LNCGY 6.63 7.30 6.95 6.63 03:59pm EST 6.7499 -0.5501 -7.54% 7.00 17.25 123,277 38,138 - - 396.8334M NAMG 0.0251 0.0272 0.0251 0.0251 01:20pm EST 0.0261 -0.0011 -4.04% 0.0299 1.3 14,610 92,791 - - 1.6408M NRG 26.3 31.04 31.16 30.08 04:02pm EST 30.14 -0.90 -2.90% 31.38 38.09 5,711,982 4,186,300 0.56 1.80 10.18B QTWW 2.99 3.34 3.27 3.22 04:00pm EST 3.51 +0.17 +5.09% 3.60 11.25 248,630 284,014 - - 81.77M SDLP 26.69 29.37 29.01 28.19 04:05pm EST 29.98 +0.61 +2.08% 30.42 36.07 817,735 315,097 2.17 7.50 2.51B SDRL 22.01 24.11 23.54 22.01 04:05pm EST 22.83 -1.28 -5.31% 23.64 47.29 15,010,230 6,057,380 4.00 16.10 10.7B SLCA 24.28 48.95 47.50 44.00 04:03pm EST 45.62 -3.33 -6.80% 48.60 73.43 6,377,504 1,900,830 0.50 1.00 2.46B TRP 42.21 47.77 47.71 47.19 04:04pm EST 47.72 -0.05 -0.10% 48.77 58.4 3,454,796 1,501,470 1.73 3.40 33.79B TSL 9.55 10.09 9.97 9.55 04:05pm EST 9.92 -0.17 -1.68% 10.41 18.77 7,281,208 5,037,110 - - 811.15M UGA 49.9 50.35 50.38 49.90 04:00pm EST 50.25 -0.10 -0.20% 50.56 64.27 20,892 23,802 - 0.00 1.84B WDGJF 10.14 10.92 10.749 10.53 12:19pm EST 10.533 -0.387 -3.54% 10.749 13.93 2,983 2,040 - - 3.842B WPRT 5.7 6.23 6.23 5.70 04:00pm EST 6.05 -0.18 -2.89% 6.28 25.1 1,692,057 812,189 - - 381.98M FCSMF 0.24 0.39 0.41 0.40 03:43pm EST 0.40 +0.01 +3.09% 0.42 0.65 43,300 99,049 - - 42.83M GTI 3.31 3.58 3.52 3.47 04:04pm EST 3.72 +0.14 +3.91% 3.75 13.01 2,016,591 1,878,670 - - 506.43M MCP 1.14 1.49 1.49 1.36 04:03pm EST 1.38 -0.11 -7.38% 1.52 7.24 4,485,912 5,543,760 - - 337.79M NGPHF 0.53 0.774 0.75 0.745 03:59pm EST 0.7460 -0.0280 -3.62% 0.77 1.32 35,188 144,531 - - 36.689M TAS 0.55 0.605 0.60 0.55 04:00pm EST 0.5680 -0.0370 -6.12% 0.6099 1.89 286,580 95,643 - - 36.5892M PALL 67.6 77.35 76.27 75.85 04:00pm EST 76.34 -1.01 -1.31% 76.45 88.42 38,892 57,737 - - 69.2618M SPPP 8.35 8.92 8.84 8.77 04:00pm EST 8.80 -0.12 -1.35% 8.86 10.34 83,538 130,674 - - 15.7B ANV 2.29 2.69 2.64 2.51 04:02pm EST 2.5100 -0.1800 -6.69% 2.76 6.7 2,492,399 2,747,350 - - 261.87M GDX 20.11 21.22 21.13 20.67 04:00pm EST 20.77 -0.45 -2.12% 21.61 28.03 52,019,404 33,993,100 0.11 0.20 18.88B IAG 2.22 2.41 2.40 2.31 04:02pm EST 2.32 -0.09 -3.73% 2.47 5.59 8,204,914 6,909,500 0.00 0.00 874.41M PZG 0.68 0.75 0.75 0.70 04:01pm EST 0.70 -0.05 -6.53% 0.75 1.48 444,793 487,271 - - 112.91M SBGL 4.39 8.68 8.66 8.39 04:02pm EST 8.47 -0.21 -2.42% 8.95 11.29 1,031,828 762,377 0.28 3.30 1.9B SIL 9.72 10.30 10.19 9.96 04:00pm EST 10.14 -0.16 -1.55% 10.53 15.28 533,074 235,908 0.24 - 6.95B SVM 1.41 1.55 1.52 1.52 04:01pm EST 1.58 +0.03 +1.94% 1.59 3.6 806,644 860,260 0.02 1.20 270M AFOP 11.11 11.71 11.66 11.11 04:00pm EST 11.22 -0.49 -4.18% 11.75 22.6 267,475 482,178 0.15 1.30 208.87M AMD 2.71 2.95 2.95 2.71 04:00pm EST 2.72 -0.23 -7.80% 2.95 4.8 41,013,929 30,318,300 - - 2.08B AMKR 4.42 8.18 8.09 7.18 04:00pm EST 7.18 -1.00 -12.22% 8.20 12.27 2,336,091 1,478,200 - - 1.7B ASX 4.44 6.20 6.16 5.90 04:02pm EST 5.95 -0.25 -4.03% 6.16 6.89 1,765,849 732,037 0.15 2.40 9.12B EBIX 11.13 13.42 13.38 13.14 04:00pm EST 13.30 -0.12 -0.89% 13.56 17.95 328,446 448,305 0.30 2.30 508.69M GTAT 0.75 1.29 1.00 0.76 04:00pm EST 0.81 -0.48 -37.21% 1.09 20.54 70,540,435 22,427,900 - - 111.41M HIMX 5.7 9.52 9.28 8.85 04:00pm EST 9.09 -0.43 -4.52% 9.45 16.15 5,445,526 5,650,400 0.27 2.90 1.55B INTC 22.82 33.62 32.63 30.50 04:00pm EST 31.91 -1.71 -5.09% 32.86 35.56 80,910,985 31,394,900 0.90 2.70 157.99B INVN 15.2 19.09 18.85 17.66 04:00pm EST 18.74 -0.35 -1.83% 19.21 26.78 3,113,288 3,267,400 - - 1.67B INVT 1.22 1.31 1.32 1.28 03:58pm EST 1.3000 -0.0100 -0.76% 1.40 14.98 76,880 163,928 0.00 0.00 31.32M MU 16.17 30.64 29.75 27.59 04:15pm EST 27.79 -2.85 -9.30% 29.87 34.85 85,676,857 24,427,600 0.00 0.00 29.68B PCRFY 9.32 11.40 11.33 11.10 03:58pm EST 11.13 -0.27 -2.35% 11.33 13.14 89,081 196,168 - - 25.73B RFMD 4.5 10.34 10.09 9.30 04:00pm EST 9.62 -0.72 -6.96% 10.23 12.98 23,946,024 11,723,600 - - 2.77B STM 6.93 7.39 7.20 6.93 04:00pm EST 6.96 -0.43 -5.82% 7.21 10 2,775,384 775,592 0.34 4.50 6.22B CXDC 4.51 5.16 5.18 5.06 04:00pm EST 5.21 +0.05 +0.97% 5.25 13.24 100,879 353,680 - - 258.36M FRM 6.57 7.16 7.10 6.86 04:02pm EST 6.90 -0.26 -3.63% 7.18 12.7 243,022 200,508 0.00 0.00 259.7M JBLU 6.77 10.68 10.59 10.04 04:00pm EST 10.06 -0.62 -5.81% 10.84 12.83 13,438,950 7,708,150 - - 2.94B FTR 4.28 5.93 5.97 5.87 04:15pm EST 5.90 -0.03 -0.51% 6.01 7.24 7,605,322 10,940,100 0.40 6.50 5.91B SSW 19.13 19.72 19.62 19.13 04:02pm EST 19.45 -0.27 -1.37% 19.86 24.48 335,848 163,789 1.38 6.70 1.84B WIN 7.18 10.33 10.36 10.09 04:00pm EST 10.11 -0.22 -2.08% 10.415 13.3 6,950,101 8,084,240 1.00 9.40 6.096B ACRX 5.22 6.07 6.02 5.80 04:00pm EST 6.06 -0.01 -0.16% 6.32 13.64 509,193 1,145,230 - - 262.86M ADHD 5.88 6.53 6.41 5.95 04:00pm EST 6.06 -0.47 -7.20% 6.50 25.44 447,337 385,751 - - 82.79M AFFX 6.25 7.21 7.15 7.10 04:00pm EST 7.10 -0.11 -1.53% 7.37 9.8 541,914 549,582 - - 521.01M CRY 7.22 10.39 10.30 10.24 04:01pm EST 10.30 -0.09 -0.87% 10.61 12.14 84,549 114,891 0.12 1.10 287.63M DRTX 8.6 23.60 23.56 23.51 04:00pm EST 23.56 -0.04 -0.17% 23.85 24.33 741,953 571,268 - - 630.87M GALE 1.66 1.81 1.78 1.69 04:00pm EST 1.71 -0.10 -5.52% 1.83 7.77 1,887,999 1,904,160 - - 202.13M HZNP 3.65 12.24 12.16 11.66 04:00pm EST 11.97 -0.27 -2.21% 12.44 18.3 1,876,080 2,035,960 - - 894.98M INO 6.52 10.09 10.23 9.92 04:00pm EST 10.04 -0.05 -0.50% 10.42 15.8 822,957 1,428,130 - - 606.4M KERX 8.61 15.82 15.85 14.92 04:00pm EST 14.98 -0.84 -5.31% 16.21 18.48 2,889,907 2,435,450 - - 1.38B KLH.V 0.65 1.33 1.31 1.24 03:59pm EST 1.28 -0.05 -3.76% 1.32 2.55 230,485 514,989 - - 100M SBOTF 0.6 1.196 1.165 1.10 03:57pm EST 1.1310 -0.0648 -5.42% 1.18 2.36 306,955 590,542 - - 88.357M MBII 1.85 2.29 2.22 1.85 04:00pm EST 1.99 -0.30 -13.10% 2.25 20 571,556 261,028 - - 48.55M NBIX 8.57 15.94 15.85 15.71 04:00pm EST 15.93 -0.01 -0.06% 16.32 20.29 651,053 527,702 0.00 0.00 1.21B NEO 2.71 4.69 4.63 4.53 04:00pm EST 4.79 +0.10 +2.13% 4.93 6.1 161,449 532,298 - - 240.33M NKTR 8.87 12.67 12.58 12.55 04:00pm EST 12.65 -0.02 -0.16% 13.00 15.34 1,984,022 1,252,300 - - 1.61B NVAX 2.68 4.41 4.33 4.33 04:00pm EST 4.40 -0.01 -0.23% 4.59 6.95 4,341,420 2,963,340 - - 1.05B OPK 7.32 8.27 8.22 8.05 04:01pm EST 8.16 -0.11 -1.33% 8.33 12.95 2,310,886 2,344,180 - - 3.5B PGNX 3.1 4.55 4.51 4.41 04:00pm EST 4.4900 -0.0600 -1.32% 4.72 7.45 712,642 1,115,430 - - 312.3M POZN 5.35 8.01 7.98 7.91 04:00pm EST 7.95 -0.06 -0.75% 8.18 9.9 253,740 258,711 0.00 0.00 250.65M RDNT 1.5 7.64 7.63 7.40 04:00pm EST 7.60 -0.04 -0.46% 8.00 8.46 406,024 453,480 - - 324.32M SPPI 6.36 7.35 7.29 7.21 04:00pm EST 7.33 -0.02 -0.27% 7.51 10.32 1,000,626 882,048 0.00 0.00 473.58M THCZ 0.015 0.016 0.016 0.015 03:50pm EST 0.0180 +0.0024 +15.38% 0.018 0.018 399,453 0 - - 3.349M XOMA 3.42 3.72 3.76 3.65 04:00pm EST 3.70 -0.02 -0.54% 3.92 9.57 1,598,839 1,283,950 - - 396.18M I'm pretty happy with the outcome, but as I said, way to long. Next step is to play with http calls. Clayton -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/tutor/attachments/20141015/60ce9ae0/attachment-0001.html> From cmferras at estudiantes.uci.cu Thu Oct 16 17:21:38 2014 From: cmferras at estudiantes.uci.cu (C@rlos) Date: Thu, 16 Oct 2014 11:21:38 -0400 (CDT) Subject: [Tutor] Convert Qstring to string in windows In-Reply-To: <1549164247.13980291.1413472329833.JavaMail.zimbra@estudiantes.uci.cu> Message-ID: <969535236.13984943.1413472898089.JavaMail.zimbra@estudiantes.uci.cu> I have been tryed to convert a Qstring text to string on python, in linux that work fine but in windows when qstring contine ?,?,?,?,? the converted text is not correct, contine extranger characters, this qstring text is an url from qdialogtext. in linux i do for this way: pythonstringtext=qstringtext.text().toUtf8.data() and it return a python string correctly. i need some help plese... .....C at rlos III Escuela Internacional de Invierno en la UCI del 17 al 28 de febrero del 2014. Ver www.uci.cu -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/tutor/attachments/20141016/eb4b6861/attachment.html> From dyoo at hashcollision.org Thu Oct 16 20:14:22 2014 From: dyoo at hashcollision.org (Danny Yoo) Date: Thu, 16 Oct 2014 11:14:22 -0700 Subject: [Tutor] Convert Qstring to string in windows In-Reply-To: <969535236.13984943.1413472898089.JavaMail.zimbra@estudiantes.uci.cu> References: <1549164247.13980291.1413472329833.JavaMail.zimbra@estudiantes.uci.cu> <969535236.13984943.1413472898089.JavaMail.zimbra@estudiantes.uci.cu> Message-ID: <CAGZAPF4OC404UC7uFcT8m0FKNutQw0f5aWgaF-VDqN0DCpCnYg@mail.gmail.com> On Thu, Oct 16, 2014 at 8:21 AM, C at rlos <cmferras at estudiantes.uci.cu> wrote: > > I have been tryed to convert a Qstring text to string on python, in linux > that work fine but in windows when qstring contine ?,?,?,?,? the converted > text is not correct, contine extranger characters, > this qstring text is an url from qdialogtext. > > in linux i do for this way: > pythonstringtext=qstringtext.text().toUtf8.data() > and it return a python string correctly. Hi Carlos, This seems like a question that's very specific to Qt: you may want to ask on a Qt-Python mailing list. Tutor is intended for beginner programmers, and the question you're asking seems a bit specialized for the intended audience. In absence of this information, I have to make a few guesses. My best guesses so far are that you're working with Qt, which provides its own Unicode string class: http://qt-project.org/doc/qt-4.8/qstring.html Are you using PyQt 4 or PyQt 5, or something else entirely? Are you using Python 2 or Python 3? According to the PyQt5 documentation, it automatically handles the string conversion: http://pyqt.sourceforge.net/Docs/PyQt5/gotchas.html#python-strings-qt-strings-and-unicode and according to the PyQt 4 documentation, it also handles the conversion automatically for you: http://pyqt.sourceforge.net/Docs/PyQt4/python_v3.html#qstring and because the mapping is done by the library, you should not have to be doing anything on your own end to convert Qstrings to Python strings. Yeah, I am not sure what you are doing yet, because the documentation says that it handles conversions for you. The fact that you're doing this manually suggests that you might be doing something unusual. We need more information. But I think you may get better help on a Qt-specific mailing list; I suspect very few of us here have Qt experience. From fomcl at yahoo.com Thu Oct 16 23:08:45 2014 From: fomcl at yahoo.com (Albert-Jan Roskam) Date: Thu, 16 Oct 2014 14:08:45 -0700 Subject: [Tutor] Convert Qstring to string in windows Message-ID: <1413493725.22928.BPMail_high_carrier@web163805.mail.gq1.yahoo.com> ----------------------------- On Thu, Oct 16, 2014 10:59 PM CEST Alan Gauld wrote: >On 16/10/14 19:14, Danny Yoo wrote: > >> need more information. But I think you may get better help on a >> Qt-specific mailing list; I suspect very few of us here have Qt >> experience. > >There are at least 2 Python Qt mailing lists and also two for >Side which is Nokia's public domain fork of Qt. That's probably >worth a look too. > >Definitely in the minority interest camp on the tutor list. > I am not 100 % sure but I think you can use str() on a qstring. In other words, it has its own __str__ method. From Pete.Wilson at atmel.com Fri Oct 17 01:35:02 2014 From: Pete.Wilson at atmel.com (Wilson, Pete) Date: Thu, 16 Oct 2014 23:35:02 +0000 Subject: [Tutor] Registering callbacks and .DLL Message-ID: <1A99A1517680884DAE440F5341907717872B88ED@DVRMBX02.corp.atmel.com> I'm having problems registering a call-back with a .DLL. Maybe someone has some advice on registering call-backs from .dlls. I'm using Windows 7 and Python 2.6 (32-bits). The .DLL was written in C++ is working with C++ apps calling it. I tried the methods in section 15.17.1.17 with the qsort() and CFUNCTYPE, but it is not working. My code and the .dll are attached. Pete -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/tutor/attachments/20141016/182c333a/attachment-0001.html> -------------- next part -------------- A non-text attachment was scrubbed... Name: py_cb.7z Type: application/octet-stream Size: 955723 bytes Desc: py_cb.7z URL: <http://mail.python.org/pipermail/tutor/attachments/20141016/182c333a/attachment-0001.obj> From alan.gauld at btinternet.com Fri Oct 17 02:24:07 2014 From: alan.gauld at btinternet.com (Alan Gauld) Date: Fri, 17 Oct 2014 01:24:07 +0100 Subject: [Tutor] Registering callbacks and .DLL In-Reply-To: <1A99A1517680884DAE440F5341907717872B88ED@DVRMBX02.corp.atmel.com> References: <1A99A1517680884DAE440F5341907717872B88ED@DVRMBX02.corp.atmel.com> Message-ID: <m1pnj7$hie$1@ger.gmane.org> On 17/10/14 00:35, Wilson, Pete wrote: > I?m having problems registering a call-back with a .DLL. Maybe someone > has some advice on registering call-backs from .dlls. > > I?m using Windows 7 and Python 2.6 (32-bits). The .DLL was written in > C++ is working with C++ apps calling it. OK As I understand it, You are trying to register a Python function as a call back on a C++ function in a DLL is that correct? Do you have any code? I don't see any attachment. Many lists/servers strip out binary attachments as potential viruses so it may not have made it through. Code is best included inline as text provided its not too long, or alternatively, in a pastebin. > I tried the methods in section 15.17.1.17 with the qsort() and > CFUNCTYPE, but it is not working. My code and the .dll are attached. Section 15.17.1.17 of what? Not the tutorial, not the library reference and not the language reference. So what? This question may be a tad too complex for the tutor list which is targeted to those learning Python, you probably want the main list for this. But you are going to have to provide some code and a more detailed description of what you are trying to do. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.flickr.com/photos/alangauldphotos From Pete.Wilson at atmel.com Fri Oct 17 18:02:36 2014 From: Pete.Wilson at atmel.com (Wilson, Pete) Date: Fri, 17 Oct 2014 16:02:36 +0000 Subject: [Tutor] Registering callbacks and .DLL In-Reply-To: <m1pnj7$hie$1@ger.gmane.org> References: <1A99A1517680884DAE440F5341907717872B88ED@DVRMBX02.corp.atmel.com> <m1pnj7$hie$1@ger.gmane.org> Message-ID: <1A99A1517680884DAE440F5341907717872BA414@DVRMBX02.corp.atmel.com> Hi AG. I guess the attachment py_cb.7z was stripped. The statement below is correct, I am trying to register a Python function as the Callback (Confirmation) For a C++ function in the .dll. When I register the call-back with prod_bat_vol_read_request I should get a machine readable 'token' from send_serial_data.tx_data. I send this token to the battery tester to start it. When the test is done, my_conf_func should be triggered. The code is in line below. Section 15.17.1.17 is in the 2.6/library docs see the URL in the comments below. I tried several different ways without success. Thanks for your advice. Pete #********************************************************************************* ''' bat_read_15_17_1_17 this program is attempting ot register a call back function the ProductionTest.dll using the techniques outlined in section 15.17.1.17 of the python docs https://docs.python.org/2.6/library/ctypes.html#module-ctypes ''' '''open and register SendSerialData this part looks like it's working ''' from ctypes import * pt_dll = pt_dll = cdll.LoadLibrary("c:/py_stuff/ProductionTest.dll") reg_send_serial_data = pt_dll.RegSendSerialData class SendSerialData_t(Structure): _fields_ = [("tx_data", c_void_p), ("size", c_uint8)] send_serial_data = SendSerialData_t() try: reg_send_serial_data(send_serial_data) except: print "reg_send_serial_data fail" print send_serial_data.tx_data print send_serial_data.size '''set up ProdBatVolRequest and register my_conf_func as the call back this is NOT working. ''' prod_bat_vol_read_request = pt_dll.ProdBatVolReadRequest #MY_FUNC = CFUNCTYPE(None, POINTER(c_uint16),POINTER(c_uint8)) #MY_FUNC = CFUNCTYPE(None, c_uint16,c_uint8) MY_FUNC = CFUNCTYPE(c_uint16,c_uint8) #MY_FUNC = WINFUNCTYPE(c_uint16,c_uint8) #MY_FUNC = WINFUNCTYPE(None, POINTER(c_uint16),POINTER(c_uint8)) def my_conf_func(bat_vol, status): print "my_conf_func", bat_vol, status #return 0 conf_func = MY_FUNC(my_conf_func) print "breakpoint" #prod_bat_vol_read_request(conf_func) #prod_bat_vol_read_request(my_conf_func) #prod_bat_vol_read_request(POINTER(my_conf_func)) #************************************************************************** > -----Original Message----- > From: Tutor [mailto:tutor-bounces+pete.wilson=atmel.com at python.org] On > Behalf Of Alan Gauld > Sent: Thursday, October 16, 2014 5:24 PM > To: tutor at python.org > Subject: Re: [Tutor] Registering callbacks and .DLL > > On 17/10/14 00:35, Wilson, Pete wrote: > > I'm having problems registering a call-back with a .DLL. Maybe > someone > > has some advice on registering call-backs from .dlls. > > > > I'm using Windows 7 and Python 2.6 (32-bits). The .DLL was written in > > C++ is working with C++ apps calling it. > > OK As I understand it, You are trying to register a Python function as > a call back on a C++ function in a DLL is that correct? > > Do you have any code? I don't see any attachment. Many lists/servers > strip out binary attachments as potential viruses so it may not have > made it through. Code is best included inline as text provided its not > too long, or alternatively, in a pastebin. > > > I tried the methods in section 15.17.1.17 with the qsort() and > > CFUNCTYPE, but it is not working. My code and the .dll are attached. > > Section 15.17.1.17 of what? Not the tutorial, not the library reference > and not the language reference. So what? > > This question may be a tad too complex for the tutor list which is > targeted to those learning Python, you probably want the main list for > this. But you are going to have to provide some code and a more > detailed description of what you are trying to do. > > -- > Alan G > Author of the Learn to Program web site > http://www.alan-g.me.uk/ > http://www.flickr.com/photos/alangauldphotos > > _______________________________________________ > Tutor maillist - Tutor at python.org > To unsubscribe or change subscription options: > https://mail.python.org/mailman/listinfo/tutor From eryksun at gmail.com Sat Oct 18 05:53:11 2014 From: eryksun at gmail.com (eryksun) Date: Fri, 17 Oct 2014 22:53:11 -0500 Subject: [Tutor] Registering callbacks and .DLL In-Reply-To: <1A99A1517680884DAE440F5341907717872B88ED@DVRMBX02.corp.atmel.com> References: <1A99A1517680884DAE440F5341907717872B88ED@DVRMBX02.corp.atmel.com> Message-ID: <CACL+1auCwAt++_2DxR2va9L-UjO1WoO6KAN86cX_EU5G7osUBQ@mail.gmail.com> On Thu, Oct 16, 2014 at 6:35 PM, Wilson, Pete <Pete.Wilson at atmel.com> wrote: > > The .DLL was written in C++ is working with C++ apps calling it. ctypes doesn't support the platform C++ ABI (I don't think the VC++ ABI is even stable), classes, STL containers, or exceptions [*]. It isn't "cpptypes". To work with ctypes, a C++ library needs an extern "C" interface. The library you're using probably qualifies, but try a simple test program in C before jumping into ctypes. [*] On Windows ctypes has limited support for Structured Exception Handling (SEH), a Microsoft extension of ANSI C. Inadvertently it also handles any exception raised by Win32 RaiseException, such as VC++ exceptions. The code for VC++ exceptions is 0xE06D7363, i.e. "\xE0" "msc". ctypes isn't looking for this code and doesn't delve deeper to get the C++ exception type, so the OSError it raises is almost useless. Pretend this doesn't exist. SEH support is only implemented for the few cases ctypes handles explicitly such as access violations. > I tried the methods in section 15.17.1.17 with the qsort() and CFUNCTYPE, > but it is not working. My code and the .dll are attached. > > from ctypes import * > > pt_dll = cdll.LoadLibrary("c:/py_stuff/ProductionTest.dll") You can use CDLL instead. It's fewer keystrokes. from ctypes import * pt_dll = CDLL("c:/py_stuff/ProductionTest.dll") If the functions use the stdcall convention, substitute WinDLL for CDLL. If there's a mix of calling conventions you can simply load the library twice, once as CDLL and again as WinDLL. They'll each have the same _handle attribute. You can also define prototypes manually via CFUNCTYPE and WINFUNCTYPE. Then instantiate them with a 2-tuple (name_or_ordinal, library), e.g. libc = CDLL('msvcr100') atoi_t = CFUNCTYPE(c_int, c_char_p) atoi = atoi_t(('atoi', libc)) >>> atoi(b'42') 42 FYI, 64-bit Windows has a single calling convention, so if you switch to 64-bit Python you don't have to worry about cdecl vs stdcall. http://en.wikipedia.org/wiki/X86_calling_conventions > reg_send_serial_data = pt_dll.RegSendSerialData > > class SendSerialData_t(Structure): > _fields_ = [("tx_data", c_void_p), > ("size", c_uint8)] > > send_serial_data = SendSerialData_t() SendSerialData_t is a function pointer type, not a data structure. Here are the C prototypes from the attached PDF: typedef void (*SendSerialData_t) (uint8_t *tx_data, uint8_t size); void RegSendSerialData(SendSerialData_t SendSerialData); A SenedSerialData_t function takes two parameters (uint8_t *, uint8_t) and returns nothing (void). ctypes declarations: SendSerialData_t = CFUNCTYPE(None, POINTER(c_uint8), c_uint8) reg_send_serial_data = pt_dll.RegSendSerialData reg_send_serial_data.argtypes = [SendSerialData_t] reg_send_serial_data.restype = None The first argument to CFUNCTYPE is the return type. Use None for void. Next define the Python callback. def send_serial_data(tx_data, size): # testing print tx_data, size print tx_data[:size] cb_send_serial_data = SendSerialData_t(send_serial_data) Finally, register the callback with the library: reg_send_serial_data(cb_send_serial_data) It's vital that you keep a reference to cb_send_serial_data (as a global, an instance attribute, in a container, etc). This prevents the callback from being deallocated while it's possible the library can call it. Otherwise at best you'll get an access violation (or segfault on POSIX systems), but probably a less obvious error. Next your test code sets up ProdBatVolRequest, which is prototyped as follows: typedef void (*BatVolReadRequest_cb)(uint16_t bat_vol, uint8_t status); typedef struct { BatVolReadRequest_cb BatVolReadConf; } BatVolReadRequest_t; void ProdBatVolReadRequest(BatVolReadRequest_t BatVolReadParam); ProdBatVolReadRequest is passed a struct by value that consists of a single function pointer. You can skip defining this struct and just pass the function pointer. It's the same ABI for x86 and x64. BatVolReadRequest_t = CFUNCTYPE(None, c_uint16, c_uint8) prod_bat_vol_read_request = pt_dll.ProdBatVolReadRequest prod_bat_vol_read_request.argtypes = [BatVolReadRequest_t] prod_bat_vol_read_request.restype = None def bat_vol_read(bat_vol, status): # testing print bat_vol, status cb_bat_vol_read = BatVolReadRequest_t(bat_vol_read) prod_bat_vol_read_request(cb_bat_vol_read) Remember to keep a reference to cb_bat_vol_read. HTH From eryksun at gmail.com Sat Oct 18 06:02:36 2014 From: eryksun at gmail.com (eryksun) Date: Fri, 17 Oct 2014 23:02:36 -0500 Subject: [Tutor] Convert Qstring to string in windows In-Reply-To: <969535236.13984943.1413472898089.JavaMail.zimbra@estudiantes.uci.cu> References: <1549164247.13980291.1413472329833.JavaMail.zimbra@estudiantes.uci.cu> <969535236.13984943.1413472898089.JavaMail.zimbra@estudiantes.uci.cu> Message-ID: <CACL+1avuPzGtjSOhqk+8auowr_8GVEYEBk1jS4U5q72hvQFKmQ@mail.gmail.com> On Thu, Oct 16, 2014 at 10:21 AM, C at rlos <cmferras at estudiantes.uci.cu> wrote: > in linux i do for this way: > pythonstringtext=qstringtext.text().toUtf8.data() > and it return a python string correctly. pythonstringtext is a byte string that has to be decoded as UTF-8. Here's the 'mojibake' result when it gets decoded as UTF-16LE or Windows codepages 850 and 1252: >>> s = u'?????' >>> b = s.encode('utf-8') >>> print b.decode('utf-16le') ????? >>> print b.decode('850') ?????????? >>> print b.decode('1252') ?????????? The native character size in the Windows kernel is 2 bytes, so UTF-16 is the path of least resistance for in-memory strings used with GUI controls and file/registry paths. UTF-8 is preferred for text data in files and network communication. From grgoffe at yahoo.com Sat Oct 18 20:36:43 2014 From: grgoffe at yahoo.com (George R Goffe) Date: Sat, 18 Oct 2014 11:36:43 -0700 Subject: [Tutor] A question about using stdin/out/err vs named files Message-ID: <1413657403.99078.YahooMailNeo@web310209.mail.ne1.yahoo.com> Hi, When you run a python program, it appears that stdin, stdout, and stderr are opened automatically. I've been trying to find out how you tell if there's data in stdin (like when you pipe data to a python program) rather than in a named input file. It seems like most/all the Unix/Linux commands are able to figure this out. Do you know how Python programs do this or might do this? MANY thanks for any/all help/hints/tips/suggestions, George... From ben+python at benfinney.id.au Sun Oct 19 02:46:13 2014 From: ben+python at benfinney.id.au (Ben Finney) Date: Sun, 19 Oct 2014 11:46:13 +1100 Subject: [Tutor] A question about using stdin/out/err vs named files References: <1413657403.99078.YahooMailNeo@web310209.mail.ne1.yahoo.com> Message-ID: <85tx30nay2.fsf@benfinney.id.au> George R Goffe <grgoffe at yahoo.com.dmarc.invalid> writes: > When you run a python program, it appears that stdin, stdout, and > stderr are opened automatically. That's true of any program on a POSIX-compliant operating system. > I've been trying to find out how you tell if there's data in stdin > (like when you pipe data to a python program) rather than in a named > input file. What does ?there's data in [a stream]? mean here, and how is it distinct from there being ?data in ? a named input file?? The advantage of the standard POSIX streams is that processes can treat them as very nearly normal files. I don't doubt you have a distinction you want to detect, but can you be clearer about what that distinction is? > It seems like most/all the Unix/Linux commands are able to figure this > out. Do you know how Python programs do this or might do this? Hmm. The standard input stream is a separate object from any named input file you might otherwise open. That's a trivially obvious difference to detect, so I guess you must not mean that. Perhaps you mean ?is the ?stdin? stream connected to an interactive terminal??. That's quite a different question from what you seem to be asking, so I don't know. -- \ ?The best is the enemy of the good.? ?Voltaire, _Dictionnaire | `\ Philosophique_ | _o__) | Ben Finney From wolfrage8765 at gmail.com Sun Oct 19 02:52:12 2014 From: wolfrage8765 at gmail.com (wolfrage8765 at gmail.com) Date: Sat, 18 Oct 2014 20:52:12 -0400 Subject: [Tutor] A question about using stdin/out/err vs named files In-Reply-To: <1413657403.99078.YahooMailNeo@web310209.mail.ne1.yahoo.com> References: <1413657403.99078.YahooMailNeo@web310209.mail.ne1.yahoo.com> Message-ID: <CAOhNYv=_B62uwy68LgyCF6J=fgmeRNPEZZqjktO6kEWeJ8Pv1Q@mail.gmail.com> Are you planning to pipe data to a python program? If so please specify and you will get more complete answers. Specifically I am thinking you want information pertaining to subprocess in the standard library. https://docs.python.org/3/library/subprocess.html On Sat, Oct 18, 2014 at 2:36 PM, George R Goffe <grgoffe at yahoo.com.dmarc.invalid> wrote: > Hi, > > When you run a python program, it appears that stdin, stdout, and stderr are opened automatically. > > I've been trying to find out how you tell if there's data in stdin (like when you pipe data to a python program) rather > than in a named input file. It seems like most/all the Unix/Linux > commands are able to figure this out. Do you know how Python programs do this or might do this? > > MANY thanks for any/all help/hints/tips/suggestions, > > George... > _______________________________________________ > Tutor maillist - Tutor at python.org > To unsubscribe or change subscription options: > https://mail.python.org/mailman/listinfo/tutor From steve at pearwood.info Sun Oct 19 05:04:59 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Sun, 19 Oct 2014 14:04:59 +1100 Subject: [Tutor] A question about using stdin/out/err vs named files In-Reply-To: <1413657403.99078.YahooMailNeo@web310209.mail.ne1.yahoo.com> References: <1413657403.99078.YahooMailNeo@web310209.mail.ne1.yahoo.com> Message-ID: <20141019030459.GI18740@ando.pearwood.info> On Sat, Oct 18, 2014 at 11:36:43AM -0700, George R Goffe wrote: > Hi, > > When you run a python program, it appears that stdin, stdout, and > stderr are opened automatically. > > I've been trying to find out how you tell if there's data in stdin > (like when you pipe data to a python program) rather than in a named > input file. It seems like most/all the Unix/Linux commands are able to > figure this out. Do you know how Python programs do this or might do > this? Hmmm, good question. I've never actually done this before, but I think the right way is to just try reading from stdin and see if anything is there. Let's try it, using Python 2.7: # stdin_demo.py import sys c = sys.stdin.read(1) if c: text = c + sys.stdin.read() # Slurp everything. print "Read text from stdin:" print text else: print "Nothing in stdin." Alas, that doesn't do what I expected. It works if I pipe something into stdin: [steve at ando ~]$ echo "Some data here" | python2.7 stdin_demo.py Read text from stdin: Some data here but if stdin is empty, the call to read() blocks, waiting for input. I have to manually type something (or nothing, as the case may be) then type Ctrl-D to force end-of-file: [steve at ando ~]$ python2.7 stdin_demo.py # enter something, then Ctrl-D hello world Read text from stdin: hello world [steve at ando ~]$ python2.7 stdin_demo.py # immediately Ctrl-D Nothing in stdin. Here's my second attempt: # stdin_demo.py import sys if sys.stdin.isatty(): print "Ignoring stdin." else: c = sys.stdin.read(1) if c: text = c + sys.stdin.read() # Slurp everything. print "Read text from stdin:" print text else: print "Nothing in stdin." This version seems to do what I think you want: [steve at ando ~]$ python2.7 stdin_demo.py Ignoring stdin. [steve at ando ~]$ echo "Some data here" | python2.7 stdin_demo.py Read text from stdin: Some data here [steve at ando ~]$ echo -n "" | python2.7 stdin_demo.py Nothing in stdin. I haven't tried this under Windows, and I'm not sure if it is the "correct" way to do it under POSIX systems like Unix, Linux or Mac, but it seems to work. Oh, in case it wasn't obvious... it's probably not a good idea to slurb everything as I do above, but I leave processing stdin line-by-line as an exercise. -- Steven From __peter__ at web.de Sun Oct 19 10:05:15 2014 From: __peter__ at web.de (Peter Otten) Date: Sun, 19 Oct 2014 10:05:15 +0200 Subject: [Tutor] A question about using stdin/out/err vs named files References: <1413657403.99078.YahooMailNeo@web310209.mail.ne1.yahoo.com> Message-ID: <m1vrbs$acf$1@ger.gmane.org> George R Goffe wrote: > When you run a python program, it appears that stdin, stdout, and stderr > are opened automatically. > > I've been trying to find out how you tell if there's data in stdin (like > when you pipe data to a python program) rather than in a named input file. > It seems like most/all the Unix/Linux commands are able to figure this > out. Do you know how Python programs do this or might do this? I don't think there is a way to guess that. Instead there is an optional commandline argument; if that is missing or '-' the script assumes stdin as the default. With argparse you spell it like this: $ cat upper.py #!/usr/bin/env python3 import argparse import sys parser = argparse.ArgumentParser(description="Convert input to uppercase") parser.add_argument( "input", type=argparse.FileType("r"), default=sys.stdin, nargs="?", help="Input file or '-' for stdin. Default: stdin.") for line in parser.parse_args().input: print(line.upper(), end="") $ ./upper.py Hello HELLO $ ./upper.py upper.py #!/USR/BIN/ENV PYTHON3 IMPORT ARGPARSE IMPORT SYS PARSER = ARGPARSE.ARGUMENTPARSER(DESCRIPTION="CONVERT INPUT TO STDIN") PARSER.ADD_ARGUMENT( "INPUT", TYPE=ARGPARSE.FILETYPE("R"), DEFAULT=SYS.STDIN, NARGS="?", HELP="INPUT FILE OR '-' FOR STDIN. DEFAULT: STDIN.") FOR LINE IN PARSER.PARSE_ARGS().INPUT: PRINT(LINE.UPPER(), END="") There is also the fileinput module. From alan.gauld at btinternet.com Sun Oct 19 10:28:01 2014 From: alan.gauld at btinternet.com (Alan Gauld) Date: Sun, 19 Oct 2014 09:28:01 +0100 Subject: [Tutor] A question about using stdin/out/err vs named files In-Reply-To: <1413657403.99078.YahooMailNeo@web310209.mail.ne1.yahoo.com> References: <1413657403.99078.YahooMailNeo@web310209.mail.ne1.yahoo.com> Message-ID: <m1vsmi$n73$1@ger.gmane.org> On 18/10/14 19:36, George R Goffe wrote: > When you run a python program, it appears that stdin, stdout, and stderr are opened automatically. correct. > I've been trying to find out how you tell if there's data in stdin Same way you tell if there's data in any file/stream - you read from it. In the old days when I programmed in C there were a pair of calls often used for this: getc() and ungetc(). You read a character with getc() then pout it back with ungetc(). Unfortunately they don't exist in Python stdlib. But, there is a getch()/ungetch() in the msvcrt for Windows, so you could use that. The curses module on linux has an ungetch() but no getch() - which seems bizarre... Steven has posted a solution using read(1) but that blocks so you need to use the isatty() method which all seems a bit messy and doesn't work with tty input. On further investigation the curses.screen object has a getch() method, but its not clear how the curses.ungetch() relates to that (there's no help() documentation) and I've run out of time to experiment. But if you need a linux solution that works with any kind of stdin that might be worth exploring. Sorry, not a great answer but hopefully its of some use. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.flickr.com/photos/alangauldphotos From jarod_v6 at libero.it Sun Oct 19 13:32:27 2014 From: jarod_v6 at libero.it (jarod_v6 at libero.it) Date: Sun, 19 Oct 2014 13:32:27 +0200 (CEST) Subject: [Tutor] python draw christman tree using loops Message-ID: <12201833.2224411413718347686.JavaMail.httpd@webmail-35.iol.local> Dear All, or improve my understanding o python I would like to learn how to draw simple figures using loops. I start from this code in java: import java.util.Scanner; public class Eserc2_8 { public static void main(String args[]) { Scanner s = new Scanner(System.in); // Altezza del triangolo int altezza = s.nextInt(); for (int riga = 1; riga <= altezza; riga++) { int i = 0; while (i<altezza-riga) { System.out.print(" "); i += 1; } i = 0; while (i< (riga*2)-1) { System.out.print("*"); i += 1; } System.out.println(""); } } } and I want to transfer to python loops: for i in range(10): #space for t in range(6-i): print "" for item in range(i+1): print "." But I ha not succed. So someone could you please explain how can work' thanks a lot bw From joel.goldstick at gmail.com Sun Oct 19 14:10:13 2014 From: joel.goldstick at gmail.com (Joel Goldstick) Date: Sun, 19 Oct 2014 08:10:13 -0400 Subject: [Tutor] python draw christman tree using loops In-Reply-To: <12201833.2224411413718347686.JavaMail.httpd@webmail-35.iol.local> References: <12201833.2224411413718347686.JavaMail.httpd@webmail-35.iol.local> Message-ID: <CAPM-O+yG5PRUOAZyHnp9n7UmpOdyU9MHetqL4eB5X16Gnbg37A@mail.gmail.com> On Sun, Oct 19, 2014 at 7:32 AM, jarod_v6 at libero.it <jarod_v6 at libero.it> wrote: > > Dear All, > or improve my understanding o python I would like to learn how to draw simple > figures using loops. > I start from this code in java: > import java.util.Scanner; > > public class Eserc2_8 { > public static void main(String args[]) { > Scanner s = new Scanner(System.in); > // Altezza del triangolo > int altezza = s.nextInt(); > > for (int riga = 1; riga <= altezza; riga++) { > int i = 0; > while (i<altezza-riga) { > System.out.print(" "); > i += 1; > } > i = 0; > while (i< (riga*2)-1) { > System.out.print("*"); > i += 1; > } > System.out.println(""); > } > } > } > > and I want to transfer to python loops: > > for i in range(10): > #space > for t in range(6-i): > print "" > for item in range(i+1): > print "." > > > But I ha not succed. So someone could you please explain how can work' > thanks a lot > bw > what is your output? What errors (print complete traceback) You probably want print " ", and print ".", so that there is no added newline. > > > _______________________________________________ > Tutor maillist - Tutor at python.org > To unsubscribe or change subscription options: > https://mail.python.org/mailman/listinfo/tutor -- Joel Goldstick http://joelgoldstick.com From david at graniteweb.com Sun Oct 19 15:10:53 2014 From: david at graniteweb.com (David Rock) Date: Sun, 19 Oct 2014 08:10:53 -0500 Subject: [Tutor] A question about using stdin/out/err vs named files In-Reply-To: <m1vrbs$acf$1@ger.gmane.org> References: <1413657403.99078.YahooMailNeo@web310209.mail.ne1.yahoo.com> <m1vrbs$acf$1@ger.gmane.org> Message-ID: <20141019131053.GB1276@wdfs> * Peter Otten <__peter__ at web.de> [2014-10-19 10:05]: > George R Goffe wrote: > > > When you run a python program, it appears that stdin, stdout, and stderr > > are opened automatically. > > > > I've been trying to find out how you tell if there's data in stdin (like > > when you pipe data to a python program) rather than in a named input file. > > It seems like most/all the Unix/Linux commands are able to figure this > > out. Do you know how Python programs do this or might do this? > > There is also the fileinput module. I use fileinput all the time. "This iterates over the lines of all files listed in sys.argv[1:], defaulting to sys.stdin if the list is empty. If a filename is '-', it is also replaced by sys.stdin. To specify an alternative list of filenames, pass it as the first argument to input(). A single file name is also allowed." It gives a fairly clean way to just "do the Right Thing" whether you are feeding files, or reading from stdin. -- David Rock david at graniteweb.com From grgoffe at yahoo.com Sun Oct 19 14:32:16 2014 From: grgoffe at yahoo.com (George R Goffe) Date: Sun, 19 Oct 2014 05:32:16 -0700 Subject: [Tutor] A question about using stdin/out/err vs named files In-Reply-To: <mailman.88215.1413707295.18129.tutor@python.org> References: <mailman.88215.1413707295.18129.tutor@python.org> Message-ID: <1413721936.75294.YahooMailNeo@web310204.mail.ne1.yahoo.com> Hi, Wow. Lots of feedback. REALLY GOOD FEEDBACK! This was my first question to this list. Let me clarify my question. I want to use tst.py as follows: tst.py input-file output-file OR cat data-file | tst.py - output-file OR cat data-file | tst.py output-file tst.py input-file output-file works well tst.py - output-file works well The question boils down to "tst.py output-file"... which is a "parsing the args" question which you "guys" have caused me to think more clearly about. If there's just 1 arg, consider it an output-file and read from stdin and write to output-file ONLY if output-file does not exist. Thank you all for your very helpful and informative responses. Regards, George... From joel.goldstick at gmail.com Sun Oct 19 15:43:26 2014 From: joel.goldstick at gmail.com (Joel Goldstick) Date: Sun, 19 Oct 2014 09:43:26 -0400 Subject: [Tutor] python draw christman tree using loops In-Reply-To: <235009006.1195201413725265309.JavaMail.defaultUser@defaultHost> References: <235009006.1195201413725265309.JavaMail.defaultUser@defaultHost> Message-ID: <CAPM-O+wtD794QX1XT+j24sxYYSVk6ZrGSBT4JjYsrZjjGqRJMA@mail.gmail.com> On Sun, Oct 19, 2014 at 9:27 AM, jarod_v6 at libero.it <jarod_v6 at libero.it> wrote: > thanks for the help: > In [60]: for i in range(10): > ....: for t in range(6-i): > ....: print " " > ....: for item in range(i+1): > ....: print "*" > ....: > > * > > * > > * > > * > > * > > * > > * > * > > * > * > > * > * > > * > * > > * > * > > * > * > * > > * > * > * > > * > * > * > > * > * > * > > * > * > * > * > > * > * > * > * > > * > * > * > * > > * > * > * > * > * > > * > * > * > * > * > > * > * > * > * > * > * > > this i the output.. it is not a tree.. > > >>----Messaggio originale---- >>Da: joel.goldstick at gmail.com >>Data: 19/10/2014 14.10 >>A: "jarod_v6 at libero.it"<jarod_v6 at libero.it> >>Cc: <tutor-request at python.org>, "tutor at python.org"<tutor at python.org> >>Ogg: Re: [Tutor] python draw christman tree using loops >> >>On Sun, Oct 19, 2014 at 7:32 AM, jarod_v6 at libero.it <jarod_v6 at libero.it> > wrote: >>> >>> Dear All, >>> or improve my understanding o python I would like to learn how to draw > simple >>> figures using loops. >>> I start from this code in java: >>> import java.util.Scanner; >>> >>> public class Eserc2_8 { >>> public static void main(String args[]) { >>> Scanner s = new Scanner(System.in); >>> // Altezza del triangolo >>> int altezza = s.nextInt(); >>> >>> for (int riga = 1; riga <= altezza; riga++) { >>> int i = 0; >>> while (i<altezza-riga) { >>> System.out.print(" "); >>> i += 1; >>> } >>> i = 0; >>> while (i< (riga*2)-1) { >>> System.out.print("*"); >>> i += 1; >>> } >>> System.out.println(""); >>> } >>> } >>> } >>> >>> and I want to transfer to python loops: >>> >>> for i in range(10): >>> #space >>> for t in range(6-i): >>> print "" >>> for item in range(i+1): >>> print "." >>> >>> >>> But I ha not succed. So someone could you please explain how can work' >>> thanks a lot >>> bw >>> >>what is your output? What errors (print complete traceback) >> >>You probably want print " ", and print ".", so that there is no added > newline. >>> In python 2.x you can add a comma to the print statement so that it will not generate a newline. That way your *s will print across. After you loop use plain print to go to next line. Also, reply to the list, not to me. >>> >>> _______________________________________________ >>> Tutor maillist - Tutor at python.org >>> To unsubscribe or change subscription options: >>> https://mail.python.org/mailman/listinfo/tutor >> >> >> >>-- >>Joel Goldstick >>http://joelgoldstick.com >> > > -- Joel Goldstick http://joelgoldstick.com From dyoo at hashcollision.org Sun Oct 19 21:00:14 2014 From: dyoo at hashcollision.org (Danny Yoo) Date: Sun, 19 Oct 2014 12:00:14 -0700 Subject: [Tutor] python draw christman tree using loops In-Reply-To: <12201833.2224411413718347686.JavaMail.httpd@webmail-35.iol.local> References: <12201833.2224411413718347686.JavaMail.httpd@webmail-35.iol.local> Message-ID: <CAGZAPF62-wAW_UWPKS7g-tGbdDJNZrPLnyWVTdmCseqJFZ==Mg@mail.gmail.com> In Java, System.out represents the standard output device. In Python, there's a similar value in sys.stdout. https://docs.python.org/2/library/sys.html#sys.stdout In your Python program, you should be able to say: import sys at the beginning of your program, and then use: sys.stdout.write(" ") and: sys.stdout.write("*") with the expected results. From alruheili at berkeley.edu Sun Oct 19 16:27:25 2014 From: alruheili at berkeley.edu (AMNA MOHAMMED ALRUHEILI) Date: Sun, 19 Oct 2014 07:27:25 -0700 Subject: [Tutor] Need help to convert pdf to excel Message-ID: <CAJS+Pa72-=A2_adJnG+vNudOFEZk-V5ssGpbiOUe08WO64Ujmw@mail.gmail.com> Hell, My name is Amna and I am totally new to python world with zero experience in programming. I am facing the challenge of converting data from pdf to excel. The data within pdf is numbers separated by space not within a table. I need a help to figure out a code that help me to convert these pdf to excel. How can I do that. Thank you, Amna -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/tutor/attachments/20141019/234d9e66/attachment.html> From hanzer at riseup.net Sun Oct 19 18:38:43 2014 From: hanzer at riseup.net (Adam Jensen) Date: Sun, 19 Oct 2014 12:38:43 -0400 Subject: [Tutor] A question about using stdin/out/err vs named files In-Reply-To: <1413657403.99078.YahooMailNeo@web310209.mail.ne1.yahoo.com> References: <1413657403.99078.YahooMailNeo@web310209.mail.ne1.yahoo.com> Message-ID: <5443E913.9030607@riseup.net> On 10/18/2014 02:36 PM, George R Goffe wrote: > Hi, > > When you run a python program, it appears that stdin, stdout, and stderr are opened automatically. > > I've been trying to find out how you tell if there's data in stdin (like when you pipe data to a python program) rather > than in a named input file. It seems like most/all the Unix/Linux > commands are able to figure this out. Do you know how Python programs do this or might do this? > > MANY thanks for any/all help/hints/tips/suggestions, > > George... Command line argument parsing aside, perhaps something like this would be useful: script.py ------------------------------------------- #!/usr/bin/env python3 import os, stat mode = os.fstat(0).st_mode if stat.S_ISFIFO(mode): print("stdin is a pipe") elif stat.S_ISREG(mode): print("stdin is a redirected file") elif stat.S_ISCHR(mode): print("stdin is terminal") else: print("stdin is weird") ------------------------------------------- $ ./script.py stdin is terminal $ cat script.py | ./script.py stdin is a pipe $ ./script.py < script.py stdin is a redirected file From dyoo at hashcollision.org Sun Oct 19 23:59:49 2014 From: dyoo at hashcollision.org (Danny Yoo) Date: Sun, 19 Oct 2014 14:59:49 -0700 Subject: [Tutor] Need help to convert pdf to excel In-Reply-To: <CAJS+Pa72-=A2_adJnG+vNudOFEZk-V5ssGpbiOUe08WO64Ujmw@mail.gmail.com> References: <CAJS+Pa72-=A2_adJnG+vNudOFEZk-V5ssGpbiOUe08WO64Ujmw@mail.gmail.com> Message-ID: <CAGZAPF578whzV=0NJp6ZufSti96GLL4jrPznLdP70PNUC0MMgw@mail.gmail.com> On Sun, Oct 19, 2014 at 7:27 AM, AMNA MOHAMMED ALRUHEILI <alruheili at berkeley.edu> wrote: > My name is Amna and I am totally new to python world with zero experience in > programming. I am facing the challenge of converting data from pdf to excel. > The data within pdf is numbers separated by space not within a table. > I need a help to figure out a code that help me to convert these pdf to > excel. Can you get the original data in a format that is not PDF? PDF is a human-centric published format: its concerns are typographic. It is not intended primarily for computers to extract data. Certainly you can extract data from it, but it won't be structured in a way that makes it particularly easy. That being said: you might try an approach that gets text from a PDF: http://stackoverflow.com/questions/25665/python-module-for-converting-pdf-to-text and then write some processing program that takes that text and inserts into an excel spreadsheet. There are several libraries that other programmers have written to write Excel documents from Python. For example: https://xlsxwriter.readthedocs.org/ As a side comment: this task actually does require a significant bit of programming knowledge. I believe it would be an unfair and unreasonable request to ask a complete programming beginner to do this without some basic programming training. I would strongly recommend taking an introductory programming class first, before trying to tackle your PDF->Excel problem head on. From crk at godblessthe.us Mon Oct 20 00:26:44 2014 From: crk at godblessthe.us (Clayton Kirkwood) Date: Sun, 19 Oct 2014 15:26:44 -0700 Subject: [Tutor] what am I not understanding? Message-ID: <050a01cfebeb$c3d4d160$4b7e7420$@us> raw_table = (''' a: Ask y: Dividend Yield b: Bid d: Dividend per Share b2: Ask (Realtime) r1: Dividend Pay Date b3: Bid (Realtime) q: Ex-Dividend Date p: Previous Close o: Open''') key_name = raw_table.rstrip('\t') print(key_name) a: Ask y: Dividend Yield b: Bid d: Dividend per Share b2: Ask (Realtime) r1: Dividend Pay Date b3: Bid (Realtime) q: Ex-Dividend Date p: Previous Close o: Open #why is the tab not being removed? key_name = raw_table.rstrip('\n') print(key_name) a: Ask y: Dividend Yield b: Bid d: Dividend per Share b2: Ask (Realtime) r1: Dividend Pay Date b3: Bid (Realtime) q: Ex-Dividend Date p: Previous Close o: Open # why is the \n not being removed? key_name = raw_table.split('\t') print(key_name) ['\na: Ask', 'y: Dividend Yield\nb: Bid', 'd: Dividend per Share\nb2: Ask (Realtime)', 'r1: Dividend Pay Date\nb3: Bid (Realtime)'. #great the tab is being removed but now I have to remove the \n but it is no longer a string key_name = raw_table.split('\n') print(key_name) ['', 'a: Ask\ty: Dividend Yield', 'b: Bid\td: Dividend per Share', 'b2: Ask (Realtime)\tr1: Dividend Pay Date', 'b3: Bid (Realtime)\tq: Ex-Dividend Date'. #great, the \n is being "removed" but now I have to remove the \t, but it is no longer a string key_name = raw_table.split('\t\n') print(key_name) ['\na: Ask\ty: Dividend Yield\nb: Bid\td: Dividend per Share\nb2: Ask (Realtime)\tr1: Dividend Pay Date\nb3: Bid (Realtime)\tq: Ex-Dividend Date\n. #why isn't the \t and \n not both "removed" key_name = raw_table.rstrip('[\t\n]') a: Ask y: Dividend Yield b: Bid d: Dividend per Share b2: Ask (Realtime) r1: Dividend Pay Date b3: Bid (Realtime) q: Ex-Dividend Date p: Previous Close o: Open #why aren't both the \t and \n being removed? (tried with and without the square brackets) I am trying to get to where I can create a dict using the ':' separator Thanks Clayton -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/tutor/attachments/20141019/4cd2fed3/attachment.html> From alan.gauld at btinternet.com Mon Oct 20 02:03:12 2014 From: alan.gauld at btinternet.com (Alan Gauld) Date: Mon, 20 Oct 2014 01:03:12 +0100 Subject: [Tutor] what am I not understanding? In-Reply-To: <050a01cfebeb$c3d4d160$4b7e7420$@us> References: <050a01cfebeb$c3d4d160$4b7e7420$@us> Message-ID: <m21jg1$e0j$1@ger.gmane.org> On 19/10/14 23:26, Clayton Kirkwood wrote: > raw_table = (''' > a: Ask y: Dividend Yield > b: Bid d: Dividend per Share > b2: Ask (Realtime) r1: Dividend Pay Date ... > o: Open???) > > key_name = raw_table.rstrip('\t') rstrip() strips characters from the *right* hand side. When it finds a character that is not a strippable character(the n of Open in this case) it stops. If you want to replace all tabs with spaces or nulls then you need to use string.replace() (or sub for regex) HTH -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.flickr.com/photos/alangauldphotos From steve at pearwood.info Mon Oct 20 03:16:23 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Mon, 20 Oct 2014 12:16:23 +1100 Subject: [Tutor] what am I not understanding? In-Reply-To: <050a01cfebeb$c3d4d160$4b7e7420$@us> References: <050a01cfebeb$c3d4d160$4b7e7420$@us> Message-ID: <20141020011623.GM18740@ando.pearwood.info> On Sun, Oct 19, 2014 at 03:26:44PM -0700, Clayton Kirkwood wrote: > raw_table = (''' > a: Ask y: Dividend Yield [...] > o: Open''') > key_name = raw_table.rstrip('\t') > print(key_name) [...] > #why is the tab not being removed? How do you know that the raw_table contains tabs rather than spaces? As far as I can see, it contains spaces, although that might just be an artifact of copying and pasting text into your email. You think you have a problem with tabs not being removed, but it seems to me that your *actual* problem is that you're not really sure whether there are tabs in the string in the first place! Just because you press the TAB key on your keyboard doesn't mean that a tab is inserted. Your system might be confirgured to insert a number of spaces instead. Your editor might automatically convert the tab into spaces. Who knows? Instead of typing TAB, the best way to get tabs into a Python string is to use the \t (backslash-t) escape code. And the best way to see that they are inside a string is to print the repr() of the string: py> raw = " some text " py> print raw # Looks like a leading tab, trailing tab is invisible. some text py> print repr(raw) # But repr() reveals the truth, no tabs here! ' some text ' py> raw = "\tsome text\t" # Actually are tabs. py> print raw some text py> print repr(raw) '\tsome text\t' So the first thing for you to do is to satisfy yourself that what you think are tabs actually are tabs. Then, the second thing is to understand what the rstrip() method actually does. It only removes characters from the right hand end of the string, not everywhere in the string: py> raw = "--some-text----" py> print raw.rstrip("-") --some-text (There is also a lstrip() to do only the left hand end of the string, and a strip() method to do both.) "Right hand end" doesn't mean the right hand end of *each line*, only of the entire string: py> raw = """---some-text---- ... --more--text--- ... -and-even-more--text-----""" py> print raw.rstrip("-") ---some-text---- --more--text--- -and-even-more--text If you want to strip something from the end of each line, first you have to break the string up into individual lines, strip each one, then put them back together again: py> lines = raw.split('\n') py> lines = [line.rstrip('-') for line in lines] py> print '\n'.join(lines) ---some-text --more--text -and-even-more--text To remove from the middle of the string, use the replace() method: py> print raw.replace("-", "") sometext moretext andevenmoretext > key_name = raw_table.split('\t') > print(key_name) [...] > #great the tab is being removed but now I > have to remove the \n but it is no longer a string Right. The split() method isn't a "remove" method, that's why it's called "split" and not "remove". It splits the string into pieces, and returns a list of substrings. You can always join the pieces back together, if you want, or find a better way to remove things. > key_name = raw_table.split('\t\n') > print(key_name) [...] > #why isn't the \t and \n not both "removed" Because you didn't tell Python to remove \t and \n separately. You told it to split the string everywhere it sees the pair of characters \t\n. The argument to split is not a list of individual characters to split on, but an exact substring to split on. If it doesn't match that substring exactly, it won't split and you'll only get one piece: py> print raw ---some-text---- --more--text--- -and-even-more--text----- py> raw.split('ex') ['---some-t', 't----\n--more--t', 't---\n-and-even-more--t', 't-----'] py> raw.split('xe') ['---some-text----\n--more--text---\n-and-even-more--text-----'] > I am trying to get to where I can create a dict using the ':' separator It takes about three steps (excluding print statements, which I have only included so you can see the individual steps in action): # Step 1: split the string into "key:value" substrings. py> raw = "a : something, b: another thing, c:yet a third thing" py> items = [s.strip() for s in raw.split(',')] py> print items ['a : something', 'b: another thing', 'c:yet a third thing'] # Step 2: split each substring into a separate (key, value) tuple, # cleaning up any whitespace around them. py> items = [s.split(':') for s in items] py> print items [['a ', ' something'], ['b', ' another thing'], ['c', 'yet a third thing']] py> items = [(key.strip(), value.strip()) for key,value in items] py> print items [('a', 'something'), ('b', 'another thing'), ('c', 'yet a third thing')] # Step 3: convert into a dict. py> d = dict(items) py> print d {'a': 'something', 'c': 'yet a third thing', 'b': 'another thing'} -- Steven From __peter__ at web.de Mon Oct 20 10:30:47 2014 From: __peter__ at web.de (Peter Otten) Date: Mon, 20 Oct 2014 10:30:47 +0200 Subject: [Tutor] what am I not understanding? References: <050a01cfebeb$c3d4d160$4b7e7420$@us> Message-ID: <m22h7p$u1f$1@ger.gmane.org> Clayton Kirkwood wrote: > raw_table = (''' > a: Ask y: Dividend Yield > b: Bid d: Dividend per Share > b2: Ask (Realtime) r1: Dividend Pay Date > b3: Bid (Realtime) q: Ex-Dividend Date > p: Previous Close > o: Open''') > o: Open #why aren't both the \t and \n > being removed? (tried with and without the square brackets) > I am trying to get to where I can create a dict using the ':' separator You never state your actual problem with sufficient precision, so I'm going to assume: - Create a Python dict from some text - Key-value pairs are separated by tabs or newlines - Keys are separated from values by a colon - Remaining whitespace surrounding keys and values needs to be stripped So >>> raw_table = '''\ ... a: Ask\t y: Dividend Yield ... b: Bid\td: Dividend per Share ... b2: Ask (Realtime)\t r1: Dividend Pay Date ... b3: Bid (Realtime)\tq: Ex-Dividend Date ... p: Previous Close ... o: Open''' I used \t as tabs, but literal tabs will work, too. The backslash in the first line is to avoid the empty first line. alternatively remove it and other leading/trailing whitespace with raw_table = raw_table.strip() >>> pairs = [pair for line in raw_table.splitlines() for pair in line.split("\t")] >>> pairs ['a: Ask', ' y: Dividend Yield', 'b: Bid', 'd: Dividend per Share', 'b2: Ask (Realtime)', ' r1: Dividend Pay Date', 'b3: Bid (Realtime)', 'q: Ex-Dividend Date', 'p: Previous Close', 'o: Open'] >>> pairs = [pair.partition(":")[::2] for pair in pairs] pair.split(":") instead of partition()[::2] would work, too. >>> pairs [('a', ' Ask'), (' y', ' Dividend Yield'), ('b', ' Bid'), ('d', ' Dividend per Share'), ('b2', ' Ask (Realtime)'), (' r1', ' Dividend Pay Date'), ('b3', ' Bid (Realtime)'), ('q', ' Ex-Dividend Date'), ('p', ' Previous Close'), ('o', ' Open')] >>> d = {k.strip(): v.strip() for k, v in pairs} >>> d {'b2': 'Ask (Realtime)', 'q': 'Ex-Dividend Date', 'b3': 'Bid (Realtime)', 'r1': 'Dividend Pay Date', 'd': 'Dividend per Share', 'y': 'Dividend Yield', 'p': 'Previous Close', 'b': 'Bid', 'a': 'Ask', 'o': 'Open'} From taylor.ruzgys at hotmail.com Mon Oct 20 04:56:57 2014 From: taylor.ruzgys at hotmail.com (Taylor Ruzgys) Date: Sun, 19 Oct 2014 22:56:57 -0400 Subject: [Tutor] Python Help Message-ID: <BLU172-W1349F134E13C6B1CD8F9E3EC970@phx.gbl> Hi, I was wondering if you could help me with an assignment that I'm doing involving Python? -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/tutor/attachments/20141019/a7fe389c/attachment.html> From alan.gauld at btinternet.com Mon Oct 20 12:31:02 2014 From: alan.gauld at btinternet.com (Alan Gauld) Date: Mon, 20 Oct 2014 11:31:02 +0100 Subject: [Tutor] Python Help In-Reply-To: <BLU172-W1349F134E13C6B1CD8F9E3EC970@phx.gbl> References: <BLU172-W1349F134E13C6B1CD8F9E3EC970@phx.gbl> Message-ID: <m22o96$knb$1@ger.gmane.org> On 20/10/14 03:56, Taylor Ruzgys wrote: > Hi, I was wondering if you could help me with an assignment that I'm > doing involving Python? Yes, we can help you do it. We won't do it for you. You need to tell us what the assignment is, how you have tried to solve it, including any code you've written. Explain where and how you are stuck. Include the full text of any error messages. Tell us the Python version and OS you use. Tell us if you are using any non standard libraries/modules -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.flickr.com/photos/alangauldphotos From crushed26 at gmail.com Mon Oct 20 14:34:34 2014 From: crushed26 at gmail.com (Bo Morris) Date: Mon, 20 Oct 2014 08:34:34 -0400 Subject: [Tutor] Insert time into email Message-ID: <CAKKCnfdf2+VVPyJyNK3JfQXZJ1VXf06Sj8At1yKyJUooTVorzQ@mail.gmail.com> hello all, hope everyone is doing well. The below code works, however I am going back and trying to enter the time and date and I cant quite figure out how to do this without breaking the code. #!/usr/bin/python import smtplib from email.MIMEMultipart import MIMEMultipart from email.MIMEText import MIMEText from email.MIMEImage import MIMEImage import time strFrom = "HourlyReport.com" #strTo = "engineering at oneconnxt.com" #strTo = "mmedley at onemediacorpinc.com" strTo = "bo at onemediacorpinc.com" l = ['3102EHD-01108.png', '3102DHD-01109.png','3102EHD-01082.png', '3102DHD-01033.png', '3102EHD-01302.png', '3102DHD-01149.png', '3102EHD-01125.png', '3102DHD-01144.png', '3102EHD-01105.png'] t = time.strftime("%H:%M:%S") d = time.strftime("%d/%m/%Y") msgRoot = MIMEMultipart('related') msgRoot['Subject'] = 'Test Hourly Report' msgRoot['From'] = strFrom msgRoot['To'] = strTo msgRoot.preamble = 'This is a multi-part message in MIME format.' msgAlternative = MIMEMultipart('alternative') msgRoot.attach(msgAlternative) msgText = MIMEText('This is the alternative plain text message.') msgAlternative.attach(msgText) msgText = MIMEText('<table cellspacing="15" border="1"><tr><td><img src="cid:3102EHD-01108.png" width="400" height="300"></img><table><tr><td>TIME<td><td>DATE</td></td></tr></table></td><td><img src="cid:3102DHD-01109.png" width="400" height="300"></img><table><tr><td>TIME<td><td>DATE</td></td></tr></table></td></tr><table cellspacing="15" border="1"><tr><td><img src="cid:3102EHD-01082.png" width="400" height="300"></img><table><tr><td>TIME</td><td>DATE</td></tr></table></td><td><img src="cid:3102DHD-01033.png" width="400" height="300"></img><table><tr><td>TIME</td><td>DATE</td></tr></table></td></tr></table><table cellspacing="15" border="1"><tr><td><img src="cid:3102EHD-01302.png" width="400" height="300"></img><table><tr><td>TIME</td><td>DATE</td></tr></table></td><td><img src="cid:3102DHD-01149.png" width="400" height="300"></img><table><tr><td>TIME</td><td>DATE</td></tr></table></td></tr><table cellspacing="15" border="1"><tr><td><img src="cid:3102EHD-01125.png" width="400" height="300"></img><table><tr><td>TIME</td><td>DATE</td></tr></table></td><td><img src="cid:3102DHD-01144.png" width="400" height="300"></img><table><tr><td>TIME</td><td>DATE</td></tr></table></td></tr></td></tr></table><table cellspacing="15" border="1"><tr><td><img src="cid:3102DHD-01144.png" width="400" height="300"></img><table><tr><td>TIME</td><td>DATE</td></tr></table></td></tr></table>', 'html') msgAlternative.attach(msgText) for image in l: with open(image, 'rb') as fh: msgImage = MIMEImage(fh.read()) msgImage.add_header('Content-ID', '<{0}>'.format(image)) msgRoot.attach(msgImage) try: smtpObj = smtplib.SMTP('localhost') smtpObj.sendmail(strFrom, strTo, msgRoot.as_string()) print "Successfully sent email" except smtplib.SMTPException: print "Error: unable to send email" I need to enter the time and date in the html where "TIME" and "DATE" are. I imagine I can do this by adding "cid: t" and "cid:d" which just refers back to t = "time.strftime("%H:%M:%S")" "d = time.strftime("%d/%m/%Y")"? Thanks in advance for any help. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/tutor/attachments/20141020/2599e487/attachment.html> From kwpolska at gmail.com Mon Oct 20 18:02:14 2014 From: kwpolska at gmail.com (=?UTF-8?B?Q2hyaXMg4oCcS3dwb2xza2HigJ0gV2Fycmljaw==?=) Date: Mon, 20 Oct 2014 18:02:14 +0200 Subject: [Tutor] Insert time into email In-Reply-To: <CAKKCnfdf2+VVPyJyNK3JfQXZJ1VXf06Sj8At1yKyJUooTVorzQ@mail.gmail.com> References: <CAKKCnfdf2+VVPyJyNK3JfQXZJ1VXf06Sj8At1yKyJUooTVorzQ@mail.gmail.com> Message-ID: <CAMw+j7K_UF26Y9=DN55PGH2LsSWSw_S_0v6-3ePC01X9ptXSzg@mail.gmail.com> On Mon, Oct 20, 2014 at 2:34 PM, Bo Morris <crushed26 at gmail.com> wrote: > hello all, hope everyone is doing well. > > The below code works, however I am going back and trying to enter the time > and date and I cant quite figure out how to do this without breaking the > code. > > #!/usr/bin/python > > import smtplib > from email.MIMEMultipart import MIMEMultipart > from email.MIMEText import MIMEText > from email.MIMEImage import MIMEImage > import time > > strFrom = "HourlyReport.com" PS. You may want to use a real e-mail address here. Or, at the very least, something that looks like one. > #strTo = "engineering at oneconnxt.com" > #strTo = "mmedley at onemediacorpinc.com" > strTo = "bo at onemediacorpinc.com" > > l = ['3102EHD-01108.png', '3102DHD-01109.png','3102EHD-01082.png', > '3102DHD-01033.png', '3102EHD-01302.png', '3102DHD-01149.png', > '3102EHD-01125.png', '3102DHD-01144.png', '3102EHD-01105.png'] > > t = time.strftime("%H:%M:%S") > d = time.strftime("%d/%m/%Y") > > msgRoot = MIMEMultipart('related') > msgRoot['Subject'] = 'Test Hourly Report' > msgRoot['From'] = strFrom > msgRoot['To'] = strTo > msgRoot.preamble = 'This is a multi-part message in MIME format.' > > msgAlternative = MIMEMultipart('alternative') > msgRoot.attach(msgAlternative) > > msgText = MIMEText('This is the alternative plain text message.') > msgAlternative.attach(msgText) > > msgText = MIMEText('<table cellspacing="15" border="1"><tr><td><img > src="cid:3102EHD-01108.png" width="400" > height="300"></img><table><tr><td>TIME<td><td>DATE</td></td></tr></table></td><td><img > src="cid:3102DHD-01109.png" width="400" > height="300"></img><table><tr><td>TIME<td><td>DATE</td></td></tr></table></td></tr><table > cellspacing="15" border="1"><tr><td><img src="cid:3102EHD-01082.png" > width="400" > height="300"></img><table><tr><td>TIME</td><td>DATE</td></tr></table></td><td><img > src="cid:3102DHD-01033.png" width="400" > height="300"></img><table><tr><td>TIME</td><td>DATE</td></tr></table></td></tr></table><table > cellspacing="15" border="1"><tr><td><img src="cid:3102EHD-01302.png" > width="400" > height="300"></img><table><tr><td>TIME</td><td>DATE</td></tr></table></td><td><img > src="cid:3102DHD-01149.png" width="400" > height="300"></img><table><tr><td>TIME</td><td>DATE</td></tr></table></td></tr><table > cellspacing="15" border="1"><tr><td><img src="cid:3102EHD-01125.png" > width="400" > height="300"></img><table><tr><td>TIME</td><td>DATE</td></tr></table></td><td><img > src="cid:3102DHD-01144.png" width="400" > height="300"></img><table><tr><td>TIME</td><td>DATE</td></tr></table></td></tr></td></tr></table><table > cellspacing="15" border="1"><tr><td><img src="cid:3102DHD-01144.png" > width="400" > height="300"></img><table><tr><td>TIME</td><td>DATE</td></tr></table></td></tr></table>', > 'html') > msgAlternative.attach(msgText) > > for image in l: > with open(image, 'rb') as fh: > msgImage = MIMEImage(fh.read()) > msgImage.add_header('Content-ID', '<{0}>'.format(image)) > msgRoot.attach(msgImage) > > > try: > smtpObj = smtplib.SMTP('localhost') > smtpObj.sendmail(strFrom, strTo, msgRoot.as_string()) > print "Successfully sent email" > except smtplib.SMTPException: > print "Error: unable to send email" > > I need to enter the time and date in the html where "TIME" and "DATE" are. I > imagine I can do this by adding "cid: t" and "cid:d" which just refers back > to t = "time.strftime("%H:%M:%S")" "d = time.strftime("%d/%m/%Y")"? No, not really. cid: is for images. You want to insert actual text. Use string formatting. '?snip?<tr><td>{time}</td><td>{date}</tr>?snip?'.format(time=t, date=d) This is assuming every date is the same. If it isn?t, you?d have to use different identifiers and values. -- Chris ?Kwpolska? Warrick <http://chriswarrick.com/> PGP: 5EAAEA16 stop html mail | always bottom-post | only UTF-8 makes sense From juan0christian at gmail.com Mon Oct 20 19:04:56 2014 From: juan0christian at gmail.com (Juan Christian) Date: Mon, 20 Oct 2014 15:04:56 -0200 Subject: [Tutor] Python sqlite3 issue Message-ID: <CAAp0bGtcWgVm118pwqr0nnrQeZ=uhDwaM1Gc+yqNTwgj8fgK7A@mail.gmail.com> I have this code (http://pastebin.com/Q21vQdHZ): import sqlite3 db = sqlite3.connect('db.sqlite') def create_db(): db.execute(''' CREATE TABLE TOPICS( ID INT PRIMARY KEY NOT NULL, URL VARCHAR NOT NULL, AUTHOR VARCHAR NOT NULL, MESSAGE VARCHAR NOT NULL ); ''') def insert_db(_id, url, author, message): db.execute("INSERT INTO TOPICS (ID, URL, AUTHOR, MESSAGE) VALUES ({}, {}, {}, {})".format(_id, url, author, message)) db.commit() def get_db(_id): cursor = db.execute("SELECT ID, URL, AUTHOR, MESSAGE FROM TOPICS WHERE ID = {}".format(_id)) return cursor.fetchone() if __name__ == '__main__': create_db() insert_db(12, 'abc.com', 'a', 'b') get_db(12) db.close() And when I try to execute it I get: First time executing: Traceback (most recent call last): File ".\sql.py", line 29, in <module> insert_db(12, 'abc.com', 'a', 'b') File ".\sql.py", line 18, in insert_db db.execute("INSERT INTO TOPICS (ID, URL, AUTHOR, MESSAGE) VALUES ({}, {}, {}, {})".format(_id, url, author, message) ) sqlite3.OperationalError: no such column: abc.com Second time executing: Traceback (most recent call last): File ".\sql.py", line 28, in <module> create_db() File ".\sql.py", line 14, in create_db ''') sqlite3.OperationalError: table TOPICS already exists What's the problem? It's just a test script just to learn sqlite3 with python. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/tutor/attachments/20141020/647fd507/attachment-0001.html> From dyoo at hashcollision.org Mon Oct 20 19:32:12 2014 From: dyoo at hashcollision.org (Danny Yoo) Date: Mon, 20 Oct 2014 10:32:12 -0700 Subject: [Tutor] Python sqlite3 issue In-Reply-To: <CAAp0bGtcWgVm118pwqr0nnrQeZ=uhDwaM1Gc+yqNTwgj8fgK7A@mail.gmail.com> References: <CAAp0bGtcWgVm118pwqr0nnrQeZ=uhDwaM1Gc+yqNTwgj8fgK7A@mail.gmail.com> Message-ID: <CAGZAPF4_B-MQEZZi_WMtUJd2goH06hx6QMhWTB7k9q6L2ABvyQ@mail.gmail.com> Hi Juan, On Mon, Oct 20, 2014 at 10:04 AM, Juan Christian <juan0christian at gmail.com> wrote: > I have this code (http://pastebin.com/Q21vQdHZ): > > import sqlite3 [code cut] > def insert_db(_id, url, author, message): > db.execute("INSERT INTO TOPICS (ID, URL, AUTHOR, MESSAGE) VALUES ({}, > {}, {}, {})".format(_id, url, author, message)) [code cut] Ah. Do not use normal string formatting when you're dealing with SQL. In web programming, we can introduce script injection problems when we use naive string concatenation. Unsurprisingly, the same class of injection attack can be done in SQL when we use naive string concatenation to build SQL statements. See: http://xkcd.com/327/ I'm being serious! :P Look for the concept of "prepared statements" or "parameter substitution" or "placeholder" in the sqllite3 Python bindings: it should let you safely construct the statement templates. In the documentation here: https://docs.python.org/2/library/sqlite3.html search for the word "placeholder", and you should see the relevant material. From kwpolska at gmail.com Mon Oct 20 19:38:59 2014 From: kwpolska at gmail.com (=?UTF-8?B?Q2hyaXMg4oCcS3dwb2xza2HigJ0gV2Fycmljaw==?=) Date: Mon, 20 Oct 2014 19:38:59 +0200 Subject: [Tutor] Insert time into email In-Reply-To: <323D4157-6573-4BC8-9F65-78534C334152@gmail.com> References: <CAKKCnfdf2+VVPyJyNK3JfQXZJ1VXf06Sj8At1yKyJUooTVorzQ@mail.gmail.com> <CAMw+j7K_UF26Y9=DN55PGH2LsSWSw_S_0v6-3ePC01X9ptXSzg@mail.gmail.com> <323D4157-6573-4BC8-9F65-78534C334152@gmail.com> Message-ID: <CAMw+j7LzH0KCouyqJeGLNKO+PPabTCU__GQCzKAaFdji94VTig@mail.gmail.com> Forwarding to list ? please use Reply All? next time. On Mon, Oct 20, 2014 at 6:28 PM, Crush <crushed26 at gmail.com> wrote: > Where does the ."format(time=t,date=d)" go? > > I would assume something like > > msgTime = MIMEText() > msgTime.add_header('not sure what to put here?' '<time>' .format(time=t)) > msgRoot.attach(msgTime) > > then in the actual html, something like... > <table><tr><td>{time}</td></tr></table>? > > and yes, for simplicity sake, the date and time will be the same. Do not bother with MIME for this. It WON?T be simpler. (it?s most likely impossible, in fact.) msgText = MIMEText('<table cellspacing="15" border="1">?rest of your HTML here, complete with {time} and {date} tags?</table>'.format(time=t, date=d), 'html') msgAlternative.attach(msgText) -- Chris ?Kwpolska? Warrick <http://chriswarrick.com/> PGP: 5EAAEA16 stop html mail | always bottom-post | only UTF-8 makes sense From joel.goldstick at gmail.com Mon Oct 20 19:17:02 2014 From: joel.goldstick at gmail.com (Joel Goldstick) Date: Mon, 20 Oct 2014 13:17:02 -0400 Subject: [Tutor] Python sqlite3 issue In-Reply-To: <CAAp0bGtcWgVm118pwqr0nnrQeZ=uhDwaM1Gc+yqNTwgj8fgK7A@mail.gmail.com> References: <CAAp0bGtcWgVm118pwqr0nnrQeZ=uhDwaM1Gc+yqNTwgj8fgK7A@mail.gmail.com> Message-ID: <CAPM-O+zGMDi9vb05HVQfHc9nfZHsdV_e5FpFK6shoNrhxxwXcQ@mail.gmail.com> On Mon, Oct 20, 2014 at 1:04 PM, Juan Christian <juan0christian at gmail.com> wrote: > I have this code (http://pastebin.com/Q21vQdHZ): > > import sqlite3 > > db = sqlite3.connect('db.sqlite') > > > def create_db(): > db.execute(''' > CREATE TABLE TOPICS( > ID INT PRIMARY KEY NOT NULL, > URL VARCHAR NOT NULL, > AUTHOR VARCHAR NOT NULL, > MESSAGE VARCHAR NOT NULL > ); > ''') > > > def insert_db(_id, url, author, message): > db.execute("INSERT INTO TOPICS (ID, URL, AUTHOR, MESSAGE) VALUES ({}, > {}, {}, {})".format(_id, url, author, message)) The above line looks suspect. I rewrote like so: >>> def insert_db(_id, url, author, message): print ("INSERT INTO TOPICS (ID, URL, AUTHOR, MESSAGE) VALUES ({}, {}, {}, {})".format(_id, url, author, message)) ... >>> insert_db(12, "abc.com", "author", "message") INSERT INTO TOPICS (ID, URL, AUTHOR, MESSAGE) VALUES (12, abc.com, author, message) >>> I've never used format like that. It looks like you need to quote the strings. I don't know if you can tell format to do that or if you have to escape them. > db.commit() > > > def get_db(_id): > cursor = db.execute("SELECT ID, URL, AUTHOR, MESSAGE FROM TOPICS WHERE ID = > {}".format(_id)) > return cursor.fetchone() > > > if __name__ == '__main__': > create_db() > insert_db(12, 'abc.com', 'a', 'b') > get_db(12) > db.close() > > And when I try to execute it I get: > > First time executing: > > Traceback (most recent call last): > File ".\sql.py", line 29, in <module> > insert_db(12, 'abc.com', 'a', 'b') > File ".\sql.py", line 18, in insert_db > db.execute("INSERT INTO TOPICS (ID, URL, AUTHOR, MESSAGE) VALUES ({}, > {}, {}, {})".format(_id, url, author, message) > ) > sqlite3.OperationalError: no such column: abc.com > > > Second time executing: > > Traceback (most recent call last): > File ".\sql.py", line 28, in <module> > create_db() > File ".\sql.py", line 14, in create_db > ''') > sqlite3.OperationalError: table TOPICS already exists This makes sense since the create sql code created TABLE the first time > > > What's the problem? It's just a test script just to learn sqlite3 with > python. > > _______________________________________________ > Tutor maillist - Tutor at python.org > To unsubscribe or change subscription options: > https://mail.python.org/mailman/listinfo/tutor > -- Joel Goldstick http://joelgoldstick.com From dyoo at hashcollision.org Mon Oct 20 20:08:08 2014 From: dyoo at hashcollision.org (Danny Yoo) Date: Mon, 20 Oct 2014 11:08:08 -0700 Subject: [Tutor] Python sqlite3 issue In-Reply-To: <CAPM-O+zGMDi9vb05HVQfHc9nfZHsdV_e5FpFK6shoNrhxxwXcQ@mail.gmail.com> References: <CAAp0bGtcWgVm118pwqr0nnrQeZ=uhDwaM1Gc+yqNTwgj8fgK7A@mail.gmail.com> <CAPM-O+zGMDi9vb05HVQfHc9nfZHsdV_e5FpFK6shoNrhxxwXcQ@mail.gmail.com> Message-ID: <CAGZAPF5K2W1iRaXgS_PADpsJSSXAvPERdRUjto-9rTLMDBCaOg@mail.gmail.com> >>>> insert_db(12, "abc.com", "author", "message") > INSERT INTO TOPICS (ID, URL, AUTHOR, MESSAGE) VALUES (12, abc.com, > author, message) >>>> > > I've never used format like that. It looks like you need to quote the > strings. I don't know if you can tell format to do that or if you > have to escape them. In normal situations, this might be good advice. When the string being produced is itself to be interpreted as code, though, we want to see if there's already some library to do the templating and quoting for us already. Otherwise, it is extraordinarily easy to leave an "injection attack" vulnerability. It doesn't even have to be one with malicious intent. See the following from way back in 2005: https://mail.python.org/pipermail/tutor/2005-June/039213.html In this case, getting it wrong just means that certain good inputs are treated incorrectly. So there's good reason to do this just so that our programs work. This is one of those issues that a programmer has to be aware of, to treat data with the proper respect and wariness. "Code is data, and data is code," is one of the mantras that the Lisp programmers use. When data becomes code, that's when we have to be especially careful. From juan0christian at gmail.com Mon Oct 20 21:05:16 2014 From: juan0christian at gmail.com (Juan Christian) Date: Mon, 20 Oct 2014 17:05:16 -0200 Subject: [Tutor] Python sqlite3 issue In-Reply-To: <CAGZAPF5K2W1iRaXgS_PADpsJSSXAvPERdRUjto-9rTLMDBCaOg@mail.gmail.com> References: <CAAp0bGtcWgVm118pwqr0nnrQeZ=uhDwaM1Gc+yqNTwgj8fgK7A@mail.gmail.com> <CAPM-O+zGMDi9vb05HVQfHc9nfZHsdV_e5FpFK6shoNrhxxwXcQ@mail.gmail.com> <CAGZAPF5K2W1iRaXgS_PADpsJSSXAvPERdRUjto-9rTLMDBCaOg@mail.gmail.com> Message-ID: <CAAp0bGvnJMPqM8tr4waOsg1tKA1DxjX53fz7ZJ1gaTDNuU6QSA@mail.gmail.com> Ok, new code using ?: import sqlite3 db = sqlite3.connect('db.sqlite') def create_db(): db.execute(''' CREATE TABLE TOPICS( ID INT PRIMARY KEY NOT NULL, URL VARCHAR NOT NULL, AUTHOR VARCHAR NOT NULL, MESSAGE VARCHAR NOT NULL ); ''') def insert_db(_id, url, author, message): db.execute("INSERT INTO TOPICS (ID, URL, AUTHOR, MESSAGE) VALUES (?, ?, ?, ?)", (_id, url, author, message)) db.commit() def get_db(_id): cursor = db.execute("SELECT ID, URL, AUTHOR, MESSAGE FROM TOPICS WHERE ID = ?", (_id)) return cursor.fetchone() if __name__ == '__main__': create_db() insert_db(12, 'abc.com', 'a', 'b') print(get_db(12)) db.close() ------------- But now when I execute I get Traceback (most recent call last): File ".\sql.py", line 30, in <module> print(get_db(12)) File ".\sql.py", line 23, in get_db cursor = db.execute("SELECT ID, URL, AUTHOR, MESSAGE FROM TOPICS WHERE ID = ?", (_id)) ValueError: parameters are of unsupported type ------------- And the second time, again, I get Traceback (most recent call last): File ".\sql.py", line 28, in <module> create_db() File ".\sql.py", line 14, in create_db ''') sqlite3.OperationalError: table TOPICS already exists On Mon, Oct 20, 2014 at 4:08 PM, Danny Yoo <dyoo at hashcollision.org> wrote: > >>>> insert_db(12, "abc.com", "author", "message") > > INSERT INTO TOPICS (ID, URL, AUTHOR, MESSAGE) VALUES (12, abc.com, > > author, message) > >>>> > > > > I've never used format like that. It looks like you need to quote the > > strings. I don't know if you can tell format to do that or if you > > have to escape them. > > > In normal situations, this might be good advice. When the string > being produced is itself to be interpreted as code, though, we want to > see if there's already some library to do the templating and quoting > for us already. Otherwise, it is extraordinarily easy to leave an > "injection attack" vulnerability. > > It doesn't even have to be one with malicious intent. See the > following from way back in 2005: > > https://mail.python.org/pipermail/tutor/2005-June/039213.html > > In this case, getting it wrong just means that certain good inputs are > treated incorrectly. So there's good reason to do this just so that > our programs work. > > This is one of those issues that a programmer has to be aware of, to > treat data with the proper respect and wariness. "Code is data, and > data is code," is one of the mantras that the Lisp programmers use. > When data becomes code, that's when we have to be especially careful. > _______________________________________________ > Tutor maillist - Tutor at python.org > To unsubscribe or change subscription options: > https://mail.python.org/mailman/listinfo/tutor > -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/tutor/attachments/20141020/ffefdd4c/attachment-0001.html> From alan.gauld at btinternet.com Mon Oct 20 23:59:35 2014 From: alan.gauld at btinternet.com (Alan Gauld) Date: Mon, 20 Oct 2014 22:59:35 +0100 Subject: [Tutor] Python sqlite3 issue In-Reply-To: <CAAp0bGtcWgVm118pwqr0nnrQeZ=uhDwaM1Gc+yqNTwgj8fgK7A@mail.gmail.com> References: <CAAp0bGtcWgVm118pwqr0nnrQeZ=uhDwaM1Gc+yqNTwgj8fgK7A@mail.gmail.com> Message-ID: <m240k7$l7q$1@ger.gmane.org> On 20/10/14 18:04, Juan Christian wrote: > CREATE TABLE TOPICS( > ID INT PRIMARY KEY NOT NULL, This means SQLite will automatically create the ID value, you do not need to provide one. > URL VARCHAR NOT NULL, > AUTHOR VARCHAR NOT NULL, > MESSAGE VARCHAR NOT NULL > ); > ''') > > > def insert_db(_id, url, author, message): > db.execute("INSERT INTO TOPICS (ID, URL, AUTHOR, MESSAGE) VALUES > ({}, {}, {}, {})".format(_id, url, author, message)) > db.commit() So in the insert you should only provide the 3 values URL, AUTHOR, MESSAGE The use of format is also a mistake but I see that Danny has already pointed that out. Finally when creating tables you can use: DROP TABLE IF EXISTS TOPICS; Prior to creating it. That will ensure you don't get an existing table error. Some other databases provide other, more conservative, mechanisms for safely creating tables but SQLite doesn't support them so dropping the table and then recreating it is your best option. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.flickr.com/photos/alangauldphotos From alan.gauld at btinternet.com Tue Oct 21 00:04:54 2014 From: alan.gauld at btinternet.com (Alan Gauld) Date: Mon, 20 Oct 2014 23:04:54 +0100 Subject: [Tutor] Python sqlite3 issue In-Reply-To: <CAAp0bGtcWgVm118pwqr0nnrQeZ=uhDwaM1Gc+yqNTwgj8fgK7A@mail.gmail.com> References: <CAAp0bGtcWgVm118pwqr0nnrQeZ=uhDwaM1Gc+yqNTwgj8fgK7A@mail.gmail.com> Message-ID: <m240u6$ofe$1@ger.gmane.org> On 20/10/14 18:04, Juan Christian wrote: > What's the problem? It's just a test script just to learn sqlite3 with > python. If you are learning SQLite3 as well as how to access it from Python I strongly recommend installing the sqlite3 interpreter and typing the SQL commands directly into that. Once you are happy with the SQL you can easily transfer that to Python. In practice I rarely issue CREATE, DROP or ALTER commands from within Python. I put those in SQL scripts and execute them from the sqlite3 interpreter. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.flickr.com/photos/alangauldphotos From crushed26 at gmail.com Tue Oct 21 01:28:27 2014 From: crushed26 at gmail.com (Crush) Date: Mon, 20 Oct 2014 19:28:27 -0400 Subject: [Tutor] Insert time into email In-Reply-To: <CAMw+j7LzH0KCouyqJeGLNKO+PPabTCU__GQCzKAaFdji94VTig@mail.gmail.com> References: <CAKKCnfdf2+VVPyJyNK3JfQXZJ1VXf06Sj8At1yKyJUooTVorzQ@mail.gmail.com> <CAMw+j7K_UF26Y9=DN55PGH2LsSWSw_S_0v6-3ePC01X9ptXSzg@mail.gmail.com> <323D4157-6573-4BC8-9F65-78534C334152@gmail.com> <CAMw+j7LzH0KCouyqJeGLNKO+PPabTCU__GQCzKAaFdji94VTig@mail.gmail.com> Message-ID: <9D28743A-44A4-4CE1-9F64-124E529F1B7C@gmail.com> Hey thanks for the help, it worked like a charm. I will post the complete code soon, so others may benefit should they ever need it. Thanks again. From __peter__ at web.de Tue Oct 21 10:18:23 2014 From: __peter__ at web.de (Peter Otten) Date: Tue, 21 Oct 2014 10:18:23 +0200 Subject: [Tutor] Python sqlite3 issue References: <CAAp0bGtcWgVm118pwqr0nnrQeZ=uhDwaM1Gc+yqNTwgj8fgK7A@mail.gmail.com> <m240k7$l7q$1@ger.gmane.org> Message-ID: <m254sf$btp$1@ger.gmane.org> Alan Gauld wrote: > Finally when creating tables you can use: > > DROP TABLE IF EXISTS TOPICS; > > Prior to creating it. That will ensure you don't get an existing > table error. Another option is to only create the table if it does not already exist: create table if not exists topics ... From alan.gauld at btinternet.com Tue Oct 21 11:43:30 2014 From: alan.gauld at btinternet.com (Alan Gauld) Date: Tue, 21 Oct 2014 10:43:30 +0100 Subject: [Tutor] Python sqlite3 issue In-Reply-To: <m254sf$btp$1@ger.gmane.org> References: <CAAp0bGtcWgVm118pwqr0nnrQeZ=uhDwaM1Gc+yqNTwgj8fgK7A@mail.gmail.com> <m240k7$l7q$1@ger.gmane.org> <m254sf$btp$1@ger.gmane.org> Message-ID: <m259s2$ige$1@ger.gmane.org> On 21/10/14 09:18, Peter Otten wrote: > Another option is to only create the table if it does not already exist: > > create table if not exists topics ... The danger with that one is that if you are changing the structure of the table you can wind up keeping the old structure by accident and your queries no longer match the table. This is especially common in SQLite since the ALTER TABLE command is so weak in capability that you often have to recreate tables to make changes. In more powerful SQL databases I use the "if not exists" quite often for creates because I can do most restructuring via ALTER, but in SQLite I usually drop and then create. The exception to that is of course if you are working on a pre-populated database with many tables. Then the exists check can prevent you wiping out a key tabl;e and its data by accidental name duplication. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.flickr.com/photos/alangauldphotos From crk at godblessthe.us Mon Oct 20 23:18:49 2014 From: crk at godblessthe.us (Clayton Kirkwood) Date: Mon, 20 Oct 2014 14:18:49 -0700 Subject: [Tutor] what am I not understanding? In-Reply-To: <m22h7p$u1f$1@ger.gmane.org> References: <050a01cfebeb$c3d4d160$4b7e7420$@us> <m22h7p$u1f$1@ger.gmane.org> Message-ID: <007901cfecab$719dbf40$54d93dc0$@us> Thanks all for the insight. I'm not sure I fully understand all of the code snippets, but in time... This is finally what I came up with: raw_table = (''' a: Ask y: Dividend Yield b: Bid d: Dividend per Share b2: Ask (Realtime) r1: Dividend Pay Date b3: Bid (Realtime) q: Ex-Dividend Date p: Previous Close o: Open Date ''') import re, string dict={} key_name = raw_table.replace('\t','\n') for each_line in key_name.splitlines(): if ':' in each_line: #this design had to do with a few different lines for key, value in [each_line.split(':')]: #see the last line in the data. I still don't fully dict[key.strip()] = value.strip() #understand the second for and the square brackets. #I presume that they force the source to look and feel like a tuple or list, but not sure. I think they force two strings into a two items of a tuple??? Please let me know if I munged bad code together:<)) Clayton From Pete.Wilson at atmel.com Tue Oct 21 02:16:21 2014 From: Pete.Wilson at atmel.com (Wilson, Pete) Date: Tue, 21 Oct 2014 00:16:21 +0000 Subject: [Tutor] Registering callbacks and .DLL In-Reply-To: <CACL+1auCwAt++_2DxR2va9L-UjO1WoO6KAN86cX_EU5G7osUBQ@mail.gmail.com> References: <1A99A1517680884DAE440F5341907717872B88ED@DVRMBX02.corp.atmel.com> <CACL+1auCwAt++_2DxR2va9L-UjO1WoO6KAN86cX_EU5G7osUBQ@mail.gmail.com> Message-ID: <1A99A1517680884DAE440F5341907717872BBC5D@DVRMBX02.corp.atmel.com> This worked. Yeah! Thank you so much. Pete > -----Original Message----- > From: eryksun [mailto:eryksun at gmail.com] > Sent: Friday, October 17, 2014 8:53 PM > To: Wilson, Pete > Cc: tutor at python.org > Subject: Re: [Tutor] Registering callbacks and .DLL > > On Thu, Oct 16, 2014 at 6:35 PM, Wilson, Pete <Pete.Wilson at atmel.com> > wrote: > > > > The .DLL was written in C++ is working with C++ apps calling it. > > ctypes doesn't support the platform C++ ABI (I don't think the VC++ ABI > is even stable), classes, STL containers, or exceptions [*]. It isn't > "cpptypes". To work with ctypes, a C++ library needs an extern "C" > interface. The library you're using probably qualifies, but try a > simple test program in C before jumping into ctypes. > > [*] On Windows ctypes has limited support for Structured Exception > Handling (SEH), a Microsoft extension of ANSI C. Inadvertently it also > handles any exception raised by Win32 RaiseException, such as VC++ > exceptions. The code for VC++ exceptions is 0xE06D7363, i.e. "\xE0" > "msc". ctypes isn't looking for this code and doesn't delve deeper to > get the C++ exception type, so the OSError it raises is almost useless. > Pretend this doesn't exist. SEH support is only implemented for the few > cases ctypes handles explicitly such as access violations. > > > I tried the methods in section 15.17.1.17 with the qsort() and > > CFUNCTYPE, but it is not working. My code and the .dll are attached. > > > > from ctypes import * > > > > pt_dll = cdll.LoadLibrary("c:/py_stuff/ProductionTest.dll") > > You can use CDLL instead. It's fewer keystrokes. > > from ctypes import * > > pt_dll = CDLL("c:/py_stuff/ProductionTest.dll") > > If the functions use the stdcall convention, substitute WinDLL for > CDLL. If there's a mix of calling conventions you can simply load the > library twice, once as CDLL and again as WinDLL. They'll each have the > same _handle attribute. > > You can also define prototypes manually via CFUNCTYPE and WINFUNCTYPE. > Then instantiate them with a 2-tuple (name_or_ordinal, library), e.g. > > libc = CDLL('msvcr100') > > atoi_t = CFUNCTYPE(c_int, c_char_p) > atoi = atoi_t(('atoi', libc)) > > >>> atoi(b'42') > 42 > > FYI, 64-bit Windows has a single calling convention, so if you switch > to 64-bit Python you don't have to worry about cdecl vs stdcall. > > http://en.wikipedia.org/wiki/X86_calling_conventions > > > reg_send_serial_data = pt_dll.RegSendSerialData > > > > class SendSerialData_t(Structure): > > _fields_ = [("tx_data", c_void_p), > > ("size", c_uint8)] > > > > send_serial_data = SendSerialData_t() > > SendSerialData_t is a function pointer type, not a data structure. > Here are the C prototypes from the attached PDF: > > typedef void (*SendSerialData_t) (uint8_t *tx_data, uint8_t size); > > void RegSendSerialData(SendSerialData_t SendSerialData); > > A SenedSerialData_t function takes two parameters (uint8_t *, uint8_t) > and returns nothing (void). > > ctypes declarations: > > SendSerialData_t = CFUNCTYPE(None, POINTER(c_uint8), c_uint8) > > reg_send_serial_data = pt_dll.RegSendSerialData > reg_send_serial_data.argtypes = [SendSerialData_t] > reg_send_serial_data.restype = None > > The first argument to CFUNCTYPE is the return type. Use None for void. > > Next define the Python callback. > > def send_serial_data(tx_data, size): > # testing > print tx_data, size > print tx_data[:size] > > cb_send_serial_data = SendSerialData_t(send_serial_data) > > Finally, register the callback with the library: > > reg_send_serial_data(cb_send_serial_data) > > It's vital that you keep a reference to cb_send_serial_data (as a > global, an instance attribute, in a container, etc). This prevents the > callback from being deallocated while it's possible the library can > call it. Otherwise at best you'll get an access violation (or segfault > on POSIX systems), but probably a less obvious error. > > Next your test code sets up ProdBatVolRequest, which is prototyped as > follows: > > typedef void (*BatVolReadRequest_cb)(uint16_t bat_vol, uint8_t > status); > > typedef struct { > BatVolReadRequest_cb BatVolReadConf; > } BatVolReadRequest_t; > > void ProdBatVolReadRequest(BatVolReadRequest_t BatVolReadParam); > > ProdBatVolReadRequest is passed a struct by value that consists of a > single function pointer. You can skip defining this struct and just > pass the function pointer. It's the same ABI for x86 and x64. > > BatVolReadRequest_t = CFUNCTYPE(None, c_uint16, c_uint8) > > prod_bat_vol_read_request = pt_dll.ProdBatVolReadRequest > prod_bat_vol_read_request.argtypes = [BatVolReadRequest_t] > prod_bat_vol_read_request.restype = None > > def bat_vol_read(bat_vol, status): > # testing > print bat_vol, status > > cb_bat_vol_read = BatVolReadRequest_t(bat_vol_read) > > prod_bat_vol_read_request(cb_bat_vol_read) > > Remember to keep a reference to cb_bat_vol_read. > > HTH From alan.gauld at btinternet.com Tue Oct 21 14:39:08 2014 From: alan.gauld at btinternet.com (Alan Gauld) Date: Tue, 21 Oct 2014 13:39:08 +0100 Subject: [Tutor] what am I not understanding? In-Reply-To: <007901cfecab$719dbf40$54d93dc0$@us> References: <050a01cfebeb$c3d4d160$4b7e7420$@us> <m22h7p$u1f$1@ger.gmane.org> <007901cfecab$719dbf40$54d93dc0$@us> Message-ID: <m25k5c$rem$1@ger.gmane.org> On 20/10/14 22:18, Clayton Kirkwood wrote: > for each_line in key_name.splitlines(): > if ':' in each_line: #this design had to > for key, value in [each_line.split(':')]: #see the last line You shouldn't need the [] around split(), it generates a list for you. In fact I'd expect the [] to break it since there will only be a single element(a list) in the outer list. > #I presume that they force the source to look and feel like a tuple or list, No, they create a list with whatever strip() produces. Here is a simpler example: mylist = [1,2,3] outerlist = [mylist] outerlist is now [[1,2,3]] > I think they force two strings into a two items of a tuple??? I think you might be referring to the unpacking of the list into key,value above? key,value = [2,3] makes key=2 and value=3 -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.flickr.com/photos/alangauldphotos From davea at davea.name Tue Oct 21 15:50:38 2014 From: davea at davea.name (Dave Angel) Date: Tue, 21 Oct 2014 09:50:38 -0400 (EDT) Subject: [Tutor] what am I not understanding? References: <050a01cfebeb$c3d4d160$4b7e7420$@us> <m22h7p$u1f$1@ger.gmane.org> <007901cfecab$719dbf40$54d93dc0$@us> Message-ID: <m25o4r$nm3$1@ger.gmane.org> "Clayton Kirkwood" <crk at godblessthe.us> Wrote in message: > Thanks all for the insight. I'm not sure I fully understand all of the code > snippets, but in time... > > This is finally what I came up with: > > raw_table = (''' > a: Ask y: Dividend Yield > b: Bid d: Dividend per Share > b2: Ask (Realtime) r1: Dividend Pay Date > b3: Bid (Realtime) q: Ex-Dividend Date > p: Previous Close > o: Open > Date > ''') > > > import re, string > dict={} > key_name = raw_table.replace('\t','\n') > for each_line in key_name.splitlines(): > if ':' in each_line: #this design had to > do with a few different lines > for key, value in [each_line.split(':')]: #see the last line > in the data. I still don't fully > dict[key.strip()] = value.strip() #understand the second for > and the square brackets. > > #I presume that they force the source to look and feel like a tuple or list, > but not sure. I think they force two strings into a two items of a tuple??? > Please let me know if I munged bad code together:<)) I dont think you want the for loop at all. It just undoes the mistake of the extra brackets. Try replacing the for loop with : key, value in each_line.split(':') And dedent the following line -- DaveA From a.bull at pubdmgroup.com Tue Oct 21 20:57:57 2014 From: a.bull at pubdmgroup.com (Al Bull) Date: Tue, 21 Oct 2014 13:57:57 -0500 Subject: [Tutor] Question on a select statement with ODBC Message-ID: <014301cfed60$edb83730$c928a590$@pubdmgroup.com> Windows 7.0 Python 3.3.4 I am accessing a database table via ODBC. The table I am accessing can have multiple records per ord_dbasub. Is there a way I can structure the select statement to retrieve only the most current record (based on ord_date)? I used to program in PL/I and assembly in the old days but my knowledge of SQL and Python is still pretty limited. Here is what I have so far.. import pyodbc cnxn = pyodbc.connect("DSN=Quickfill DEMO Database") cursor = cnxn.cursor() ord_rows = cursor.execute("select ord_dbasub, ord_pub, ord_date,ord_service," "ord_agency, ord_woa, ord_status," "ord_channel, ord_source, ord_giftcomp," "ord_cnreason " "from ord " "Where ord_pub='QWKFIL'" "order by ord_dbasub, ord_date").fetchall() for row in ord_rows: # Print for testing. Remove later print (row.ord_dbasub, row.ord_date, row.ord_pub) # Add code here to find the most recent order per DBASUB and delete other orders # Also add exclusion code to be applied to survivor record This code works so far. I just need to determine how to remove all but the most current record per ord_dbasub. Assuming I can't remover it via the select statement I'll have to traverse the array (Tuples in Python, correct?) and remove the ones I don't need. What's the best way to do that? I have to admit that the concept of tuples & dictionaries has me a little bit confused. I'm used to working with arrays and arrays of structures. Thanks in advance for your help! Al Bull, Chief Technology Officer/Owner Publishers Data Management Group a.bull at pubdmgroup.com 815-732-5297 From crk at godblessthe.us Wed Oct 22 00:43:46 2014 From: crk at godblessthe.us (Clayton Kirkwood) Date: Tue, 21 Oct 2014 15:43:46 -0700 Subject: [Tutor] what am I not understanding? In-Reply-To: <m25o4r$nm3$1@ger.gmane.org> References: <050a01cfebeb$c3d4d160$4b7e7420$@us> <m22h7p$u1f$1@ger.gmane.org> <007901cfecab$719dbf40$54d93dc0$@us> <m25o4r$nm3$1@ger.gmane.org> Message-ID: <01ff01cfed80$79966550$6cc32ff0$@us> !-----Original Message----- !From: Tutor [mailto:tutor-bounces+crk=godblessthe.us at python.org] On !Behalf Of Dave Angel !Sent: Tuesday, October 21, 2014 6:51 AM !To: tutor at python.org !Subject: Re: [Tutor] what am I not understanding? ! !"Clayton Kirkwood" <crk at godblessthe.us> Wrote in message: !> Thanks all for the insight. I'm not sure I fully understand all of the !> code snippets, but in time... !> !> This is finally what I came up with: !> !> raw_table = (''' !> a: Ask y: Dividend Yield !> b: Bid d: Dividend per Share !> b2: Ask (Realtime) r1: Dividend Pay Date !> b3: Bid (Realtime) q: Ex-Dividend Date !> p: Previous Close !> o: Open !> Date !> ''') !> !> !> import re, string !> dict={} !> key_name = raw_table.replace('\t','\n') for each_line in !> key_name.splitlines(): !> if ':' in each_line: #this design had to !> do with a few different lines !> for key, value in [each_line.split(':')]: #see the last line !> in the data. I still don't fully !> dict[key.strip()] = value.strip() #understand the second !for !> and the square brackets. !> !> #I presume that they force the source to look and feel like a tuple or !> list, but not sure. I think they force two strings into a two items of !a tuple??? !> Please let me know if I munged bad code together:<)) ! !I dont think you want the for loop at all. It just undoes the mistake !of the extra brackets. Try replacing the for loop with ! : ! ! key, value in each_line.split(':') ! !And dedent the following line Yep, that worked well. Thanks. I was lead to my initial line by something else not working. Clayton ! ! !-- !DaveA ! !_______________________________________________ !Tutor maillist - Tutor at python.org !To unsubscribe or change subscription options: !https://mail.python.org/mailman/listinfo/tutor From njs64 at pitt.edu Tue Oct 21 23:14:57 2014 From: njs64 at pitt.edu (Nathan Spencer) Date: Tue, 21 Oct 2014 17:14:57 -0400 Subject: [Tutor] Full screen mode Message-ID: <CANtncHqgmetbqdRQp=ZZpi2ZtyOqtXG55URkbLx+rJqjGjxdJg@mail.gmail.com> Hi there, I'm running python on Scientific Linux. The problem is that I'm permanently stuck in full screen mode. Exiting out of that mode (via F11) doesn't do anything. Do you have any suggestions? Thanks, Nathan -- Nathaniel J. Spencer, PhD Post-Doctoral Research Associate Psychoacoustics Lab School of Health and Rehabilitative Sciences University of Pittsburgh -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/tutor/attachments/20141021/3efd5d1a/attachment.html> From alan.gauld at btinternet.com Wed Oct 22 01:41:40 2014 From: alan.gauld at btinternet.com (Alan Gauld) Date: Wed, 22 Oct 2014 00:41:40 +0100 Subject: [Tutor] Question on a select statement with ODBC In-Reply-To: <014301cfed60$edb83730$c928a590$@pubdmgroup.com> References: <014301cfed60$edb83730$c928a590$@pubdmgroup.com> Message-ID: <m26qvl$cb2$1@ger.gmane.org> On 21/10/14 19:57, Al Bull wrote: > have multiple records per ord_dbasub. Is there a way I can structure the > select statement to retrieve only the most current record (based on > ord_date)? Yes, the cursor can be told to only retrieve N records, in your case 1. SELECT ord_dbasub, ord_pub,ord_date,ord_service, ... FROM ord WHERE ord_pub='QWKFIL' ORDER BY ord_dbasub, ord_date LIMIT 1 If the sort order is wrong you can specify ASC or DESC to reverse it as needed. > ord_rows = cursor.execute("select ord_dbasub, ord_pub, > ord_date,ord_service," > "ord_agency, ord_woa, ord_status," > "ord_channel, ord_source, ord_giftcomp," > "ord_cnreason " > "from ord " > "Where ord_pub='QWKFIL'" > "order by ord_dbasub, ord_date").fetchall() Rather than all those quotes you can use triple quotes: ord_rows = cursor.execute('''select ord_dbasub, ord_pub, ord_date,ord_service, ord_agency, ord_woa, ord_status, etc... order by ord_dbasub, ord_date limit 1''').fetchall() > for row in ord_rows: > print (row.ord_dbasub, row.ord_date, row.ord_pub) > # Add code here to find the most recent order per DBASUB and delete other > orders If it's in order you could just access the first row using an index. print (ord_rows[0]) > I have to admit that the concept of tuples & dictionaries has me a little > bit confused. I'm used to working with arrays and arrays of structures. tuples are just read-only lists, which, in turn, are arrays that can hold any data type. tuples are also like records without named fields. You can use a named tuple from the collections module which is even more like a record. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.flickr.com/photos/alangauldphotos From dyoo at hashcollision.org Wed Oct 22 01:44:56 2014 From: dyoo at hashcollision.org (Danny Yoo) Date: Tue, 21 Oct 2014 16:44:56 -0700 Subject: [Tutor] Full screen mode In-Reply-To: <CANtncHqgmetbqdRQp=ZZpi2ZtyOqtXG55URkbLx+rJqjGjxdJg@mail.gmail.com> References: <CANtncHqgmetbqdRQp=ZZpi2ZtyOqtXG55URkbLx+rJqjGjxdJg@mail.gmail.com> Message-ID: <CAGZAPF5752LBSquJm+OqtF1f1fs8+YXCaZKex7PjuKtLe17TkA@mail.gmail.com> On Tue, Oct 21, 2014 at 2:14 PM, Nathan Spencer <njs64 at pitt.edu> wrote: > Hi there, > > I'm running python on Scientific Linux. The problem is that I'm permanently > stuck in full screen mode. Exiting out of that mode (via F11) doesn't do > anything. Do you have any suggestions? Hmmm... unfortunately, I do not know what you mean by full screen mode, and am unfamiliar with the Scientific Linux environment. The standard Python distribution, itself, should not include anything that uses full screen mode, so you are probably running some third-party IDE or editor. The problem you're having doesn't sound like a Python problem. You may have better luck asking on a forum that's specialized toward your computing environment. You should probably check with the scientific-linux-users mailing list: https://www.scientificlinux.org/community/ From alan.gauld at btinternet.com Wed Oct 22 01:45:27 2014 From: alan.gauld at btinternet.com (Alan Gauld) Date: Wed, 22 Oct 2014 00:45:27 +0100 Subject: [Tutor] Full screen mode In-Reply-To: <CANtncHqgmetbqdRQp=ZZpi2ZtyOqtXG55URkbLx+rJqjGjxdJg@mail.gmail.com> References: <CANtncHqgmetbqdRQp=ZZpi2ZtyOqtXG55URkbLx+rJqjGjxdJg@mail.gmail.com> Message-ID: <m26r6n$fb5$1@ger.gmane.org> On 21/10/14 22:14, Nathan Spencer wrote: > I'm running python on Scientific Linux. I don't know it but assume it starts in normal X Windows fashion and you can create a normal Terminal/Console window? eg an xterm? > permanently stuck in full screen mode. Exiting out of that mode (via > F11) doesn't do anything. Is this for everything or just Python? If its just Python how are you running it? Are you using an IDE(eg IDLE/Eclipse?) or maybe IPython? Or are you just typing python at the shell? If its everything then it sounds like you have switched to a virtual console. Try hitting Ctrl-Alt-F7 to get back to your X session. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.flickr.com/photos/alangauldphotos From steve at pearwood.info Wed Oct 22 02:52:45 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Wed, 22 Oct 2014 11:52:45 +1100 Subject: [Tutor] Full screen mode In-Reply-To: <CANtncHqgmetbqdRQp=ZZpi2ZtyOqtXG55URkbLx+rJqjGjxdJg@mail.gmail.com> References: <CANtncHqgmetbqdRQp=ZZpi2ZtyOqtXG55URkbLx+rJqjGjxdJg@mail.gmail.com> Message-ID: <20141022005245.GA20010@ando.pearwood.info> On Tue, Oct 21, 2014 at 05:14:57PM -0400, Nathan Spencer wrote: > Hi there, > > I'm running python on Scientific Linux. The problem is that I'm > permanently stuck in full screen mode. Exiting out of that mode (via F11) > doesn't do anything. Do you have any suggestions? I don't know what "full screen mode" means. Do you mean maximized? It will help if you can tell us how you are starting Python. Python is a programming language, and there are many different environments that you might be using to run code in that programming language, e.g.: - via the terminal - in the Python interactive interpreter - via IDLE - via some third party IDE (integrated development environment) - embedded in another application What command do you type or button or menu do you click to start Python? If you can see the standard window title bar, what happens if you click the "maximise" box next to the close box? -- Steven From Pete.Wilson at atmel.com Wed Oct 22 02:04:18 2014 From: Pete.Wilson at atmel.com (Wilson, Pete) Date: Wed, 22 Oct 2014 00:04:18 +0000 Subject: [Tutor] Passing Data to .DLL Message-ID: <1A99A1517680884DAE440F5341907717872BBE55@DVRMBX02.corp.atmel.com> I am having problems passing a pointer and uint8 to a .DLL. I have been successful registering call -backs with this .DLL. So it is all 32-bit and ctypes are working. Everything is working up to the line #set-up ProcessingIncomingSerialData. I tried the direct approach and then morphing the call-back style but yummy_thing is not so yummy... The python code is below.. The definition from the .DLL header is [cid:image002.png at 01CFED51.0C6D7E70] # Python code starts here..... ''' read_bat.py ''' from serial import * from string import * from time import * system_state = "G2G" ''' --- REMOVED Set-up the COMM port ---- print("Enter COM port number") user_comm_port = raw_input() try: dut_serial_port = Serial(port="COM"+user_comm_port, baudrate=115200, timeout=1) except: system_state = "FUBAR" print "Serial Port Problem" try: dut_serial_port.isOpen() print("COM" + user_comm_port + " is open") except: print "Serial Port Problem" system_state = "FUBAR" -------------------------------------------''' #Set-up and register cb_send_serial_data from ctypes import * pt_dll = CDLL("c:/py_stuff/ProductionTest.dll") SendSerialData_t = CFUNCTYPE(None, POINTER(c_uint8), c_uint8) reg_send_serial_data = pt_dll.RegSendSerialData reg_send_serial_data.argtypes = [SendSerialData_t] reg_send_serial_data.restype = None global new_serial_tx_data global new_serial_size def send_serial_data(tx_data, size): # testing print "tx_data = ", tx_data print "size = ", size print "tx_data[:size] = ", tx_data[:size] global new_serial_tx_data new_serial_tx_data = tx_data[:size] global new_serial_size new_serial_size = size cb_send_serial_data = SendSerialData_t(send_serial_data) global cb_send_serial_data reg_send_serial_data(cb_send_serial_data) print "reg_send_serial_data done" #Set-up and register cb_bat_vol_read #this triggers send_serial_data when it is registered. BatVolReadRequest_t = CFUNCTYPE(None, c_uint16, c_uint8) prod_bat_vol_read_request = pt_dll.ProdBatVolReadRequest prod_bat_vol_read_request.argtypes = [BatVolReadRequest_t] prod_bat_vol_read_request.restype = None def bat_vol_read(bat_vol, status): print "bat_vol_read()" # testing print bat_vol, status cb_bat_vol_read = BatVolReadRequest_t(bat_vol_read) prod_bat_vol_read_request(cb_bat_vol_read) print "prod_bat_vol_read_request done" ''' ------------REMOVED serial access for example ----- #push new_serial_tx_data out the serial port to the DUT. print new_serial_tx_data dut_serial_port.write(new_serial_tx_data) dut_serial_port.write("\r \n") sleep(1) #and check to see the DUT's reply... global new_serial_rx_data #new_serial_rx_data ="" global rx_buf rx_buf = [] try: string_buf = dut_serial_port.readline() except: system_state = "FUBAR" print "serial read problem" rx_buf = list(string_buf) print "string_buf = ", string_buf print "rx_buf = ", rx_buf -----------------------------------------------''' #set-up ProcessingIncomingSerialData print "breakpoint" class rx_data_t: def _init_(self): #self.ret = None self.data = [] self.size = '' fake_rx_data = rx_data_t() fake_rx_data.data = ['\x01', '\x05', '\x00', '\x1c', '\x00', '\x99', '\x0c', '\x04'] fake_rx_data.size = 8 print "fake_rx_data.data = ", fake_rx_data.data print "fake_rx_data.size = ", fake_rx_data.size ProcessIncomingSerialData_t = CFUNCTYPE(None, POINTER(c_uint8), c_uint16) process_incoming_serial_data = pt_dll.ProcessIncomingSerialData process_incoming_serial_data.argtypes = [ProcessIncomingSerialData_t] #process_incoming_serial_data.argtypes = [POINTER(c_uint8), c_uint16] process_incoming_serial_data.restype = None yummy_thing = ProcessIncomingSerialData_t(fake_rx_data)passing pointers to process_incoming_serial_data(yummy_thing) #process_incoming_serial_data(fake_rx_data) print "Done." print "system_state = ", system_state #print "Closing COM port", user_comm_port #dut_serial_port.close() -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/tutor/attachments/20141022/8f661e80/attachment-0001.html> -------------- next part -------------- A non-text attachment was scrubbed... Name: image002.png Type: image/png Size: 62037 bytes Desc: image002.png URL: <http://mail.python.org/pipermail/tutor/attachments/20141022/8f661e80/attachment-0001.png> From crk at godblessthe.us Wed Oct 22 01:54:33 2014 From: crk at godblessthe.us (Clayton Kirkwood) Date: Tue, 21 Oct 2014 16:54:33 -0700 Subject: [Tutor] A couple of somewhat esoteric questions Message-ID: <022801cfed8a$5e6ee770$1b4cb650$@us> As I've contemplated the usage of dictionaries, I face the question of efficiency. Going back before most of you were probably born;<)) if I remember correctly dictionaries(assoc. arrays), having hashes, are efficient for storing sparse arrays with the added benefit of hiding the traversal of the dictionary looking for the proper key, and being much less efficient if looking for the value and trying to retrieve its key. But it also depends on the hash key and algorithm and how many spaces are "available" for filling. If a hash is created for a small array, which is known ahead of time with some possible hints, the array is efficient but has the distinct difficulty of use when more and more pairs are created, because either the new pair matches an already used hash point and a trail must be descended off of that particular hash point or a mechanism must be used to determine how the new pair should be stored elsewhere in the array like at the end and matching becomes time consuming. And/or, a new hash key and perhaps a different algorithm must be used for creating the larger array. Also, since the array allows for multiple keys pointing to possibly identical values, the array is not necessarily 1 to 1 (could be 1 to multiple although we use replacement of the value normally) and definitely cannot be onto: there is no rule that says the mapping from value back to key is singular and unique. I've not had much use of dictionaries in the past so maybe I am missing something. As I contemplated dictionary use, it seems changing the order of entries is best left to double single arrays - which can be modified together when changing order. Dicts don't seem to support moving pairs around and in fact would be broken if changes were attempted. Kind of like using a real dictionary, you can have a word (key) have multiple meanings (values), and you can have multiple words (keys) having the same meaning (value). Again, with the difference from a real dictionary, our dicts can't have keys pointing to different values. But there are definitely uses for dicts. Seem right??? Clayton -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/tutor/attachments/20141021/e39aa199/attachment.html> From crk at godblessthe.us Wed Oct 22 06:54:49 2014 From: crk at godblessthe.us (Clayton Kirkwood) Date: Tue, 21 Oct 2014 21:54:49 -0700 Subject: [Tutor] yet another misunderstanding on my part Message-ID: <026001cfedb4$510d0e20$f3272a60$@us> __author__ = 'SYSTEM' import string #Pricing Dividends raw_table = (''' a: Ask y: Dividend Yield b: Bid d: Dividend per Share b2: Ask (Realtime) r1: Dividend Pay Date b3: Bid (Realtime) q: Ex-Dividend Date p: Previous Close o: Open import re, string col_position, code, description = 0, [], [] key_name = raw_table.replace('\t','\n') for each_line in key_name.splitlines(): if ':' in each_line: code[col_position], description.append() = each_line.split(':') #neither works; first one is out of range error, 2nd, can't assign to #function. I've used square brackets around various sections, and it doesn't like it c, d = each_line.split(':') #this works code[col_position] = each_line.split(':') #why doesn't this. It looks like what is in the tutorial 5.1. I get out of range error code.append(c) #part of the first 'this works' line code[col_position] = 5 #this works, yet I am using the same col_position element description.append(d) print( col_position, code[col_position], description[col_position]) col_position += 1 I've seen an example of description.strip() work from a for in loop assignment with out the can't assign to function. I'd like to see something like: code[col_position].strip(), description[col_position].stripe() = each_line.split(':') but for some reason, the [col_position] doesn't work, and I can't strip(). What am I not seeing? I *do* spend hours trying different options, and study various documentation which confuse me and don't seem consistent. Thanks, Clayton -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/tutor/attachments/20141021/5c5e7827/attachment.html> From alan.gauld at btinternet.com Wed Oct 22 11:52:25 2014 From: alan.gauld at btinternet.com (Alan Gauld) Date: Wed, 22 Oct 2014 10:52:25 +0100 Subject: [Tutor] Passing Data to .DLL In-Reply-To: <1A99A1517680884DAE440F5341907717872BBE55@DVRMBX02.corp.atmel.com> References: <1A99A1517680884DAE440F5341907717872BBE55@DVRMBX02.corp.atmel.com> Message-ID: <m27uoq$5nq$1@ger.gmane.org> On 22/10/14 01:04, Wilson, Pete wrote: > The python code is below.. > > ''' read_bat.py > ''' You are using triple quoted strings as a commenting feature. Thats good practice where you want a docstring to appear in the help() screen but its not so good for comments like the above or where you are commenting out code blocks because the quotes are esy to miss and hard to spot the closing match. (OK in an IDE because it colours the string but in a mail message its much harder to see) Could you, when posting long bits of code, please remove the commented out sections - it makes us (well me at least) much more willing to study the code. I've tried to remove the redundant sections below, and made a few style comments. But apart from the spurious 'comment' at the end of the calling line I don't see anything obvious. > from serial import * > from string import * Are you sure you need string? It is very rarely used and I don't see it anywhere in your code? > from time import * Also in general its bad practice to use this style of import. The risk of name collisions is quite high. If you don;t like the long names use an alias: import serial as ser for example. > system_state = "G2G" > > #Set-up and register cb_send_serial_data > from ctypes import * And its best to keep the imports all together > pt_dll = CDLL("c:/py_stuff/ProductionTest.dll") > SendSerialData_t = CFUNCTYPE(None, POINTER(c_uint8), c_uint8) > > reg_send_serial_data = pt_dll.RegSendSerialData > reg_send_serial_data.argtypes = [SendSerialData_t] > reg_send_serial_data.restype = None > > global new_serial_tx_data > global new_serial_size You don't need the global keyword at the module level, its only needed inside the function to indicate that the variable is non-local. > def send_serial_data(tx_data, size): > # testing > print "tx_data = ", tx_data > print "size = ", size > print "tx_data[:size] = ", tx_data[:size] > > global new_serial_tx_data > > new_serial_tx_data = tx_data[:size] > > global new_serial_size Its traditional to put all the global declarations together at the top of the function. > new_serial_size = size > > cb_send_serial_data = SendSerialData_t(send_serial_data) > global cb_send_serial_data > reg_send_serial_data(cb_send_serial_data) > print "reg_send_serial_data done" > > #Set-up and register cb_bat_vol_read > #this triggers send_serial_data when it is registered. > > BatVolReadRequest_t = CFUNCTYPE(None, c_uint16, c_uint8) > prod_bat_vol_read_request = pt_dll.ProdBatVolReadRequest > prod_bat_vol_read_request.argtypes = [BatVolReadRequest_t] > prod_bat_vol_read_request.restype = None > > def bat_vol_read(bat_vol, status): > print "bat_vol_read()" > print bat_vol, status > > cb_bat_vol_read = BatVolReadRequest_t(bat_vol_read) > > prod_bat_vol_read_request(cb_bat_vol_read) > print "prod_bat_vol_read_request done" > > #set-up ProcessingIncomingSerialData > > print "breakpoint" > > class rx_data_t: > def _init_(self): > self.data = [] > self.size = '' > > fake_rx_data = rx_data_t() > fake_rx_data.data = ['\x01', '\x05', '\x00', '\x1c', '\x00', '\x99', > '\x0c', '\x04'] > fake_rx_data.size = 8 I'm not sure why you used a class here. But if you are using one you might as well make the __init__() do something useful: class rx_data_t: def _init_(self,data=[],size=None): self.data = data self.size = size And then initialize it on creation: fake_rx_data = rx_data_t(['\x01','\x05','\x00','\x1c', '\x00','\x99', '\x0c','\x04'], 8) But since the class doesn't do anything(no methods) you might as well just use a tuple or dictionary. fake_rx_data = {data: ['\x01','\x05','\x00','\x1c', '\x00','\x99', '\x0c','\x04'], size: 8} > print "fake_rx_data.data = ", fake_rx_data.data > print "fake_rx_data.size = ", fake_rx_data.size > > ProcessIncomingSerialData_t = CFUNCTYPE(None, POINTER(c_uint8), c_uint16) > process_incoming_serial_data = pt_dll.ProcessIncomingSerialData > process_incoming_serial_data.argtypes = [ProcessIncomingSerialData_t] > process_incoming_serial_data.restype = None > > yummy_thing = ProcessIncomingSerialData_t(fake_rx_data)passing pointers to Is there supposed to be comment marker after the parens? > process_incoming_serial_data(yummy_thing) > print "Done." > print "system_state = ", system_state -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.flickr.com/photos/alangauldphotos From alan.gauld at btinternet.com Wed Oct 22 11:59:10 2014 From: alan.gauld at btinternet.com (Alan Gauld) Date: Wed, 22 Oct 2014 10:59:10 +0100 Subject: [Tutor] A couple of somewhat esoteric questions In-Reply-To: <022801cfed8a$5e6ee770$1b4cb650$@us> References: <022801cfed8a$5e6ee770$1b4cb650$@us> Message-ID: <m27v5f$clt$1@ger.gmane.org> On 22/10/14 00:54, Clayton Kirkwood wrote: > As I?ve contemplated the usage of dictionaries, I face the question of > efficiency. Don;t worry about it. Python dictionaries are highly efficient and used throughout the language. namespaces and classes are both effectively dictionaries so every time you use a variable name or class attribute (whether your own or a built-in) you are using a dictionary. > remember correctly dictionaries(assoc. arrays), having hashes, are > efficient for storing sparse arrays They are sparce arrays using hashing for lookup. > traversal of the dictionary looking for the proper key, and being much > less efficient if looking for the value and trying to retrieve its key. Indeed, that's why dictionaries don't have a get key for value operation. > But it also depends on the hash key and algorithm and how many spaces > are ?available? for filling. Yes, and Python's hashing has been tuned and optimised over many years. > I?ve not had much use of dictionaries in the past so maybe I am missing > something. You are using them all the time. Python is built on dictionaries. They are very powerful and efficient. > As I contemplated dictionary use, it seems changing the order > of entries is best left to double single arrays There is no order. The order may change in use and you have no control over that. It's a Python internal detail and you as a programmer cannot mess with it. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.flickr.com/photos/alangauldphotos From alan.gauld at btinternet.com Wed Oct 22 12:11:04 2014 From: alan.gauld at btinternet.com (Alan Gauld) Date: Wed, 22 Oct 2014 11:11:04 +0100 Subject: [Tutor] yet another misunderstanding on my part In-Reply-To: <026001cfedb4$510d0e20$f3272a60$@us> References: <026001cfedb4$510d0e20$f3272a60$@us> Message-ID: <m27vrp$nra$1@ger.gmane.org> On 22/10/14 05:54, Clayton Kirkwood wrote: > __author__ = 'SYSTEM' Its best not to invent your own dunder names. You might run into a problem if a future Python release decided to use the same name under the covers. > import string Do you need this? Its very unusual these days to use the string module. > #Pricing Dividends > raw_table = (''' > a: Ask y: Dividend Yield > b: Bid d: Dividend per Share > b2: Ask (Realtime) r1: Dividend Pay Date > b3: Bid (Realtime) q: Ex-Dividend Date > p: Previous Close > o: Open Seems to be missing a closing '''') ? BTW the above would be a great place to use a dictionary, its already almost in the right format... > import re, string Keep imports together, it would make it easier to see the duplication of string... > col_position, code, description = 0, [], [] > key_name = raw_table.replace('\t','\n') > > for each_line in key_name.splitlines(): > if ':' in each_line: > code[col_position], description.append() = > each_line.split(?:?) #neither works; first one is out of range Where did 'code' come from? You are trying to index into a non existent variable. You need to declare the list and then use it. The usual way to add items to a list is to use the append() method. > code[col_position] = each_line.split(':') #why doesn?t Again code is still undeclared. > code.append(c) And even this won't work until you declare code to be a list. > I?ve seen an example of description.strip() work from a for in loop > assignment with out the can?t assign to function. I can't quite parse that but I think you are looking at the wrong problem. Its not the strip() thats causing problems its the use of code before declaring it. code = [] # or code = list() each declares code to be an empty list code.append(item) # this now adds an item to the list If you want to use indexing to assign values then you need to assign dummy values to create the 'spaces' first. This is often done using a list comprehension: # create a list of length size each containing zero code = [0 for n in range(size)] or cell multiplication code = [0] * size HTH -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.flickr.com/photos/alangauldphotos From steve at pearwood.info Wed Oct 22 13:02:56 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Wed, 22 Oct 2014 22:02:56 +1100 Subject: [Tutor] yet another misunderstanding on my part In-Reply-To: <026001cfedb4$510d0e20$f3272a60$@us> References: <026001cfedb4$510d0e20$f3272a60$@us> Message-ID: <20141022110251.GB20010@ando.pearwood.info> On Tue, Oct 21, 2014 at 09:54:49PM -0700, Clayton Kirkwood wrote: > col_position, code, description = 0, [], [] > key_name = raw_table.replace('\t','\n') > for each_line in key_name.splitlines(): > if ':' in each_line: > code[col_position], description.append() = each_line.split(':') > #neither works; first one is out of range error, 2nd, can't assign to > #function. I've used square brackets around various sections, and it doesn't > like it The trick to learning how to program is NOT to randomly make changes to your code and hope that you get lucky, but to understand why you are getting the errors you get. Start here: position = 0 codes = [] codes[position] = 999 This fails with an error: IndexError: list assignment index out of range What does that mean? It's an English sentence, a little bit terse but still understandable: "list assignment index" -- what's that? It's the index used inside the square brackets, being used for list assignment. In this case, the index is 0. "out of range" -- what's that mean? It tells you that the index you have provided (in this case, 0) is too big. Let's experiment to see what "too big" means: py> alist = ['a', 'b', 'c'] # three items py> alist[0] = 'A' py> alist[1] = 'B' py> alist[2] = 'C' py> alist[3] = 'D' Traceback (most recent call last): File "<stdin>", line 1, in <module> IndexError: list assignment index out of range Aha! You can only assign to items in a list which already exist. In the experiment, I had a list with three items, item 0, item 1 and item 2, and assigning to index 0, 1 and 2 succeeded. But assigning to item 3 (which doesn't exist) fails. Go back to your list. It is an empty list, [], which means it has *no* items in it. Since there are no items in it, you cannot assign to any (non-existent) item. If you can't assign to an item, since there aren't any, you have to append to the list. I'm not sure where you got the idea of writing description.append() = something from. There's nothing in Python that I've ever seen that suggests that would work, and the error message should be clear: py> x() = 23 File "<stdin>", line 1 SyntaxError: can't assign to function call Try this instead: py> descriptions = [] py> descriptions.append("something blue") py> descriptions.append("something round") py> descriptions.append("something flat") py> print(descriptions) ['something blue', 'something round', 'something flat'] Like the name suggests, "append" appends something to the end of the list. So your code, which started like this: # doesn't work col_position, code, description = 0, [], [] key_name = raw_table.replace('\t','\n') for each_line in key_name.splitlines(): if ':' in each_line: code[col_position], description.append() = each_line.split(':') could be written something like this: # note plural names codes, descriptions = [], [] key_name = raw_table.replace('\t','\n') for each_line in key_name.splitlines(): if ':' in each_line: code, description = each_line.split(':') codes.append(code) descriptions.append(description) [...] > What am I not seeing? I *do* spend hours trying different options, and study > various documentation which confuse me and don't seem consistent. It seems to me that you are probably failing to isolate the problem to *one* thing. When you get an error, you should ignore EVERYTHING else and just play around with that one function until you understand it. Open the interactive interpreter, and experiment, like I did above with the "alist[0]" assignments. -- Steven From eryksun at gmail.com Wed Oct 22 13:16:17 2014 From: eryksun at gmail.com (eryksun) Date: Wed, 22 Oct 2014 06:16:17 -0500 Subject: [Tutor] Passing Data to .DLL In-Reply-To: <1A99A1517680884DAE440F5341907717872BBE55@DVRMBX02.corp.atmel.com> References: <1A99A1517680884DAE440F5341907717872BBE55@DVRMBX02.corp.atmel.com> Message-ID: <CACL+1asL7YuR6bD3wkCVJyW8aqN+wA7=-TvwUc4BUvVS8zSG_w@mail.gmail.com> On Tue, Oct 21, 2014 at 7:04 PM, Wilson, Pete <Pete.Wilson at atmel.com> wrote: > > ProcessIncomingSerialData_t = CFUNCTYPE(None, POINTER(c_uint8), c_uint16) > process_incoming_serial_data = pt_dll.ProcessIncomingSerialData > process_incoming_serial_data.argtypes = [ProcessIncomingSerialData_t] ProcessIncomingSerialData takes two parameters: uint8_t *rx_buf and uint16_t size. process_incoming_serial_data.argtypes = [POINTER(c_uint8), c_uint16] process_incoming_serial_data.restype = None The buffer parameter isn't const, so pass a copy of string_buf. size = len(string_buf) rx_buf = (c_uint8 * size).from_buffer_copy(string_buf) process_incoming_serial_data(rx_buf, size) I'm going to meander off topic a bit to give you a warning... Python strings are immutable by contract, so don't pass them directly to C functions that might modify them. String immutability makes interning possible, sometimes just locally in the code object and sometimes globally in the interpreter. Here's an example of both in CPython 2.7: import abc from ctypes import * def test(): s = 'abc' a = cast(s, POINTER(c_char * 3))[0] a[:] = 'cba' print 'abc' >>> test.__code__.co_consts (None, 'abc', 3, 0, 'cba') Notice that the 2nd constant in co_consts is 'abc'. This gets stored to s and later printed. In between I use ctypes to reverse it. So what gets printed? If you guessed "cba", you're right. >>> test() cba Look at the constants now: >>> test.__code__.co_consts (None, 'cba', 3, 0, 'cba') So how about referencing the abc module? >>> abc Traceback (most recent call last): File "<stdin>", line 1, in <module> NameError: name 'abc' is not defined OK, but do you think we can use cba instead? >>> cba Traceback (most recent call last): File "<stdin>", line 1, in <module> NameError: name 'cba' is not defined This is yet another problem with mucking with the internal state of an immutable object. Equal objects are supposed to hash the same. 'cba' is now equal to the interned 'abc' string that I reversed, but it probes to a different slot in the dict's hash table. >>> test.__code__.co_consts[1] == 'cba' True >>> hash(test.__code__.co_consts[1]) == hash('cba') False OTOH, a new 'abc' string probes to the same slot in the hash table, but it's no longer equal (i.e. it gets handled as a hash collision). >>> test.__code__.co_consts[1] == 'abc' False >>> hash(test.__code__.co_consts[1]) == hash('abc') True This breaks dict lookup unless we use the interned string itself:. >>> globals()[test.__code__.co_consts[1]] <module 'abc' from '/usr/lib/python2.7/abc.pyc'> So the moral is only pass Python strings to C functions that promise (scout's honor) to not modify them. If the parameter isn't const, err on the side of caution; copy the string to a ctypes buffer. From steve at pearwood.info Wed Oct 22 13:40:16 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Wed, 22 Oct 2014 22:40:16 +1100 Subject: [Tutor] A couple of somewhat esoteric questions In-Reply-To: <022801cfed8a$5e6ee770$1b4cb650$@us> References: <022801cfed8a$5e6ee770$1b4cb650$@us> Message-ID: <20141022114016.GC20010@ando.pearwood.info> On Tue, Oct 21, 2014 at 04:54:33PM -0700, Clayton Kirkwood wrote: > As I've contemplated the usage of dictionaries, I face the question of > efficiency. Going back before most of you were probably born;<)) if I > remember correctly dictionaries(assoc. arrays), having hashes, are efficient > for storing sparse arrays with the added benefit of hiding the traversal of > the dictionary looking for the proper key, and being much less efficient if > looking for the value and trying to retrieve its key. But it also depends on > the hash key and algorithm and how many spaces are "available" for filling. > If a hash is created for a small array, which is known ahead of time with > some possible hints, the array is efficient but has the distinct difficulty > of use when more and more pairs are created, because either the new pair > matches an already used hash point and a trail must be descended off of that > particular hash point or a mechanism must be used to determine how the new > pair should be stored elsewhere in the array like at the end and matching > becomes time consuming. This is standard for hash tables. Python, if I recall correctly, uses linear addressing rather than chaining. Python's hash function can return approximately 4294967296 different values (2**32), which means that for random or arbitrary keys, you won't expect more than a tiny handful of collisions. Of course a hostile adversary might, sometimes, be able to feed you keys that all hash the same, but in practice that's very unlikely to happen. And even if it does happen, the worst that will occur is that dict look-ups will degenerate to being proportional to the number of items, which isn't too bad. > And/or, a new hash key and perhaps a different > algorithm must be used for creating the larger array. Python always uses the same hash algorithm. It doesn't try to change algorithms as the table gets bigger, it just scales the hash value to the size of the table. As the table gets full, the table is doubled in size to make room. (To be pedantic, I'm talking here about the standard CPython reference implementation. There are other implementations, such as Jython and IronPython and PyPy, which may do other things.) > Also, since the array > allows for multiple keys pointing to possibly identical values, the array is > not necessarily 1 to 1 (could be 1 to multiple although we use replacement > of the value normally) and definitely cannot be onto: there is no rule that > says the mapping from value back to key is singular and unique. Yes, but I don't see your point. Hash tables are designed to be one-to-many. If you want to understand the *precise* details of how Python dicts work, you will need to read the source code written in C. But as an over simplification: - when you create an empty dict, it is created with a certain number of free slots in a table; - as you add items to the dict, the dict knows how many free slots remain; - when the table is half full, the dict resizes; very small dicts are increased to four times bigger, while larger dicts are doubled in size; - that means that *on average* adding new items to a dict takes the same amount of time (amortized over the lifetime of the dict). > I've not had much use of dictionaries in the past so maybe I am missing > something. As I contemplated dictionary use, it seems changing the order of > entries is best left to double single arrays - which can be modified > together when changing order. Dicts don't seem to support moving pairs > around and in fact would be broken if changes were attempted. I don't understand what you are trying to say here. Hash tables are unordered, and "changing the order of entries" makes no sense for them. Can you give a concrete example of what you're trying to do? > Kind of like > using a real dictionary, you can have a word (key) have multiple meanings > (values), In Python, the way to do that is by associating a list of meanings with the one key: d = {'key': ['first', 'second', 'third'], 'another key': ['alpha', 'beta']} > and you can have multiple words (keys) having the same meaning > (value). Again, with the difference from a real dictionary, our dicts can't > have keys pointing to different values. They certainly can. > But there are definitely uses for dicts. One or two, one or two. Hundred. Thousand. Seriously, dicts are probably the most commonly used data structure in Python, by far. They are used for modules, classes, instances, global variables are stored inside a dict, built-in functions are stored inside a dict, modules are cached inside a dict. They are efficient and fast and highly tuned, and there is almost never any need to try to re-invent the wheel. -- Steven From crushed26 at gmail.com Wed Oct 22 16:58:47 2014 From: crushed26 at gmail.com (Bo Morris) Date: Wed, 22 Oct 2014 10:58:47 -0400 Subject: [Tutor] Insert time into email...final code Message-ID: <CAKKCnfcEU42+D_Op-EP+o0Rem29qBuVWWZp4=fA6kBJBrCSQYw@mail.gmail.com> Just in case anyone else can benefit from this, here is my working code up to this point #!/usr/bin/python import smtplib from email.MIMEMultipart import MIMEMultipart from email.MIMEText import MIMEText from email.MIMEImage import MIMEImage import time strFrom = "HourlyReport.com" strTo = "myEmail at server.com" t = time.strftime("%H:%M:%S %m/%d/%y") l = ['3102EHD-01108.png', '3102DHD-01109.png','3102EHD-01082.png', '3102DHD-01033.png', '3102EHD-01302.png', '3102DHD-01149.png', '3102EHD-01125.png', '3102DHD-01144.png', '3102EHD-01105.png'] msgRoot = MIMEMultipart('related') msgRoot['Subject'] = 'Test Hourly Report' msgRoot['From'] = strFrom msgRoot['To'] = strTo msgRoot.preamble = 'This is a multi-part message in MIME format.' msgAlternative = MIMEMultipart('alternative') msgRoot.attach(msgAlternative) msgText = MIMEText('This is the alternative plain text message.') msgAlternative.attach(msgText) msgText = MIMEText('<table cellspacing="15" border="0"><tr><td align="center"><img src="cid:3102EHD-01108.png"><br/>3102EHD-01108<br/>{time}</td><td align="center"><img src="cid:3102DHD-01109.png"><br/>3102DHD-01109<br/>{time}</td></tr><tr><td align="center"><img src="cid:3102EHD-01082.png"><br/>3102EHD-01082<br/>{time}</td><td align="center"><img src="cid:3102DHD-01033.png"><br/>3102DHD-01033<br/>{time}</td></tr><tr><td align="center"><img src="cid:3102EHD-01302.png"><br/>3102EHD-01302<br/>{time}</td><td align="center"><img src="cid:3102DHD-01149.png"><br/>3102DHD-01149<br/>{time}</td></tr><tr><td align="center"><img src="cid:3102EHD-01125.png"><br/>3102EHD-01125<br/>{time}</td><td align="center"><img src="cid:3102DHD-01144.png"><br/>3102DHD-01144<br/>{time}</td></tr><tr><td align="center"><img src="cid:3102EHD-01105.png"><br/>3102EHD-01105<br/>{time}</td></tr></table>'.format(time=t), 'html') msgAlternative.attach(msgText) for image in l: with open(image, 'rb') as fh: msgImage = MIMEImage(fh.read()) msgImage.add_header('Content-ID', '<{0}>'.format(image)) msgRoot.attach(msgImage) try: smtpObj = smtplib.SMTP('localhost') smtpObj.sendmail(strFrom, strTo, msgRoot.as_string()) print "Successfully sent email" except smtplib.SMTPException: print "Error: unable to send email" -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/tutor/attachments/20141022/2ea84627/attachment.html> From a.bull at pubdmgroup.com Wed Oct 22 17:06:32 2014 From: a.bull at pubdmgroup.com (Al Bull) Date: Wed, 22 Oct 2014 10:06:32 -0500 Subject: [Tutor] Question on a select statement with ODBC In-Reply-To: <m26qvl$cb2$1@ger.gmane.org> References: <014301cfed60$edb83730$c928a590$@pubdmgroup.com> <m26qvl$cb2$1@ger.gmane.org> Message-ID: <00aa01cfee09$c42e2ab0$4c8a8010$@pubdmgroup.com> -----Original Message----- From: Tutor [mailto:tutor-bounces+a.bull=pubdmgroup.com at python.org] On Behalf Of Alan Gauld Sent: Tuesday, October 21, 2014 6:42 PM To: tutor at python.org Subject: Re: [Tutor] Question on a select statement with ODBC On 21/10/14 19:57, Al Bull wrote: > have multiple records per ord_dbasub. Is there a way I can structure > the select statement to retrieve only the most current record (based > on ord_date)? Yes, the cursor can be told to only retrieve N records, in your case 1. SELECT ord_dbasub, ord_pub,ord_date,ord_service, ... FROM ord WHERE ord_pub='QWKFIL' ORDER BY ord_dbasub, ord_date LIMIT 1 If the sort order is wrong you can specify ASC or DESC to reverse it as needed. > ord_rows = cursor.execute("select ord_dbasub, ord_pub, > ord_date,ord_service," > "ord_agency, ord_woa, ord_status," > "ord_channel, ord_source, ord_giftcomp," > "ord_cnreason " > "from ord " > "Where ord_pub='QWKFIL'" > "order by ord_dbasub, ord_date").fetchall() Rather than all those quotes you can use triple quotes: ord_rows = cursor.execute('''select ord_dbasub, ord_pub, ord_date,ord_service, ord_agency, ord_woa, ord_status, etc... order by ord_dbasub, ord_date limit 1''').fetchall() > for row in ord_rows: > print (row.ord_dbasub, row.ord_date, row.ord_pub) > # Add code here to find the most recent order per DBASUB and > delete other orders If it's in order you could just access the first row using an index. print (ord_rows[0]) > I have to admit that the concept of tuples & dictionaries has me a little > bit confused. I'm used to working with arrays and arrays of structures. tuples are just read-only lists, which, in turn, are arrays that can hold any data type. tuples are also like records without named fields. You can use a named tuple from the collections module which is even more like a record. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.flickr.com/photos/alangauldphotos _______________________________________________ Tutor maillist - Tutor at python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor [Al Bull] I don't think I explained the problem properly. I have several hundred thousand records in the ORD table. There are many instances of records with identical ORD_DBASUB values. Where duplicates exist, I only want to keep the most current record. There could be 1-20 or more records with the same ORD_DBASUB value. I am close to having this working. I added the following statement: ord_rows.reverse() to reverse the order of the table. Now for each ORD_DBASUB, the most current record will be the first one. I then added this code: savedbasub = 0 for row in ord_rows: if savedbasub == row.ord_dbasub: ord_rows.remove(row) delcount += 1 else: savedbasub = row.ord_dbasub This code works except in very specific cases. Take the following example: ORD_DBASUB DATE 1) 100000360 2004-11-02 2) 100000360 2004-09-03 3) 100000334 2004-04-05 4) 100000334 2004-03-08 Record #1 is saved, as it should be. Record #2 is correctly removed. Record #3 is correctly saved, but record #4 is not removed. It appears that ROW is being moved to the next entry after the ord_rows.remove statement, then being moved again at the top of the loop causing me to drop down into the else clause for record #4. Al Bull, Chief Technology Officer/Owner Publishers Data Management Group a.bull at pubdmgroup.com 815-732-5297 From alan.gauld at btinternet.com Wed Oct 22 19:12:53 2014 From: alan.gauld at btinternet.com (Alan Gauld) Date: Wed, 22 Oct 2014 18:12:53 +0100 Subject: [Tutor] Question on a select statement with ODBC In-Reply-To: <00aa01cfee09$c42e2ab0$4c8a8010$@pubdmgroup.com> References: <014301cfed60$edb83730$c928a590$@pubdmgroup.com> <m26qvl$cb2$1@ger.gmane.org> <00aa01cfee09$c42e2ab0$4c8a8010$@pubdmgroup.com> Message-ID: <m28oil$98v$1@ger.gmane.org> On 22/10/14 16:06, Al Bull wrote: > I don't think I explained the problem properly. I have several hundred > thousand records in the ORD table. There are many instances of records with > identical ORD_DBASUB values. Where duplicates exist, I only want to keep > the most current record. Ah, OK thats very different. You can do it in SQL but it gets messy and depends on the details of the ODBC SQL, which I don't know... It would involve a nested select with an inner join I suspect. > This code works except in very specific cases. Take the following example: > ORD_DBASUB DATE > 1) 100000360 2004-11-02 > 2) 100000360 2004-09-03 > 3) 100000334 2004-04-05 > 4) 100000334 2004-03-08 > > Record #3 is correctly saved, but record #4 is not removed. It appears > that ROW is being moved to the next entry after the ord_rows.remove That's correct you should never modify the collection that you are iterating over with a for loop. Instead convert to using a while loop and only increment the index if you don't remove an thing. Alternatively make a copy of the collection and iterate over that, but a while is usually preferable IMHO. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.flickr.com/photos/alangauldphotos From juan0christian at gmail.com Wed Oct 22 19:38:04 2014 From: juan0christian at gmail.com (Juan Christian) Date: Wed, 22 Oct 2014 15:38:04 -0200 Subject: [Tutor] Python sqlite3 issue In-Reply-To: <m259s2$ige$1@ger.gmane.org> References: <CAAp0bGtcWgVm118pwqr0nnrQeZ=uhDwaM1Gc+yqNTwgj8fgK7A@mail.gmail.com> <m240k7$l7q$1@ger.gmane.org> <m254sf$btp$1@ger.gmane.org> <m259s2$ige$1@ger.gmane.org> Message-ID: <CAAp0bGuwhXhXQNddqc+h96UzP8n4MLPhHnpPaMr-bZXRXmfQQA@mail.gmail.com> So guys, now I have this code that is fully working: import sqlite3 db = sqlite3.connect('db.sqlite') def ini_db(): db.execute(''' CREATE TABLE IF NOT EXISTS TOPICS( ID INTEGER NOT NULL, URL VARCHAR NOT NULL, AUTHOR VARCHAR NOT NULL, MESSAGE VARCHAR NOT NULL ); ''') def insert_db(_id, url, author, message): db.execute("INSERT INTO TOPICS (ID, URL, AUTHOR, MESSAGE) VALUES (?, ?, ?, ?)", (_id, url, author, message)) db.commit() def get_db(_id): cursor = db.execute("SELECT ID, URL, AUTHOR, MESSAGE FROM TOPICS WHERE ID = ?", (_id,)) return cursor.fetchone() if __name__ == '__main__': ini_db() insert_db(12, 'abc.com', 'a', 'b') insert_db(20, 'abc2.com', 'a2', 'c') insert_db(1, 'abc3.com', 'a3', 'd') for row in db.execute('SELECT * FROM TOPICS ORDER BY ID'): print(row) db.close() ------ Anything else that I need to improve/change? Regarding the 'DROP TABLE' before creating, It wouldn't be good for me, because I do want the 'old' data from the table there, I don't want to drop them! -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/tutor/attachments/20141022/4e7544eb/attachment-0001.html> From crk at godblessthe.us Wed Oct 22 20:18:07 2014 From: crk at godblessthe.us (Clayton Kirkwood) Date: Wed, 22 Oct 2014 11:18:07 -0700 Subject: [Tutor] A couple of somewhat esoteric questions In-Reply-To: <20141022114016.GC20010@ando.pearwood.info> References: <022801cfed8a$5e6ee770$1b4cb650$@us> <20141022114016.GC20010@ando.pearwood.info> Message-ID: <034a01cfee24$87d46230$977d2690$@us> !-----Original Message----- !From: Tutor [mailto:tutor-bounces+crk=godblessthe.us at python.org] On !Behalf Of Steven D'Aprano !Sent: Wednesday, October 22, 2014 4:40 AM !To: tutor at python.org !Subject: Re: [Tutor] A couple of somewhat esoteric questions ! !On Tue, Oct 21, 2014 at 04:54:33PM -0700, Clayton Kirkwood wrote: ! !> As I've contemplated the usage of dictionaries, I face the question of !> efficiency. Going back before most of you were probably born;<)) if I !> remember correctly dictionaries(assoc. arrays), having hashes, are !> efficient for storing sparse arrays with the added benefit of hiding !> the traversal of the dictionary looking for the proper key, and being !> much less efficient if looking for the value and trying to retrieve !> its key. But it also depends on the hash key and algorithm and how !many spaces are "available" for filling. !> If a hash is created for a small array, which is known ahead of time !> with some possible hints, the array is efficient but has the distinct !> difficulty of use when more and more pairs are created, because either !> the new pair matches an already used hash point and a trail must be !> descended off of that particular hash point or a mechanism must be !> used to determine how the new pair should be stored elsewhere in the !> array like at the end and matching becomes time consuming. ! !This is standard for hash tables. Python, if I recall correctly, uses !linear addressing rather than chaining. ! !Python's hash function can return approximately 4294967296 different !values (2**32), which means that for random or arbitrary keys, you won't !expect more than a tiny handful of collisions. Of course a hostile !adversary might, sometimes, be able to feed you keys that all hash the !same, but in practice that's very unlikely to happen. And even if it !does happen, the worst that will occur is that dict look-ups will !degenerate to being proportional to the number of items, which isn't too !bad. I would hope that the full array is not created? ! !> And/or, a new hash key and perhaps a different algorithm must be used !> for creating the larger array. ! !Python always uses the same hash algorithm. It doesn't try to change !algorithms as the table gets bigger, it just scales the hash value to !the size of the table. As the table gets full, the table is doubled in !size to make room. Out of curiousity, does a new array get built in a larger home and everything redistributed, or does more memory get added on? I think the second would be difficult because a new seed and size value would be needed to properly distribute. ! !(To be pedantic, I'm talking here about the standard CPython reference !implementation. There are other implementations, such as Jython and !IronPython and PyPy, which may do other things.) ! ! !> Also, since the array !> allows for multiple keys pointing to possibly identical values, the !> array is not necessarily 1 to 1 (could be 1 to multiple although we !> use replacement of the value normally) and definitely cannot be onto: !> there is no rule that says the mapping from value back to key is !singular and unique. ! !Yes, but I don't see your point. Hash tables are designed to be one-to- !many. For clarification, a key only has one value which can be changed. Multiple keys can have the same value (hopefully not to the same memory location, because one would usually not want the change of one key's value to alter another key's value. As to the hash table, yes, I agree that this would be one to many, hence the chains. ! !If you want to understand the *precise* details of how Python dicts !work, you will need to read the source code written in C. But as an over !simplification: ! !- when you create an empty dict, it is created with a certain number ! of free slots in a table; !- as you add items to the dict, the dict knows how many free slots ! remain; !- when the table is half full, the dict resizes; very small dicts ! are increased to four times bigger, while larger dicts are ! doubled in size; !- that means that *on average* adding new items to a dict takes ! the same amount of time (amortized over the lifetime of the ! dict). ! ! !> I've not had much use of dictionaries in the past so maybe I am !> missing something. As I contemplated dictionary use, it seems changing !> the order of entries is best left to double single arrays - which can !> be modified together when changing order. Dicts don't seem to support !> moving pairs around and in fact would be broken if changes were !attempted. ! !I don't understand what you are trying to say here. Hash tables are !unordered, and "changing the order of entries" makes no sense for them. !Can you give a concrete example of what you're trying to do? The changing of the order is necessary when you want numeric indexing, say for columns or rows. ! ! !> Kind of like !> using a real dictionary, you can have a word (key) have multiple !meanings !> (values), ! !In Python, the way to do that is by associating a list of meanings with !the one key: ! !d = {'key': ['first', 'second', 'third'], ! 'another key': ['alpha', 'beta']} ! ! !> and you can have multiple words (keys) having the same meaning !> (value). Again, with the difference from a real dictionary, our dicts !can't !> have keys pointing to different values. ! !They certainly can. Ah, I see from your previous snippet! ! ! !> But there are definitely uses for dicts. ! !One or two, one or two. Hundred. Thousand. ! !Seriously, dicts are probably the most commonly used data structure in !Python, by far. They are used for modules, classes, instances, global !variables are stored inside a dict, built-in functions are stored inside !a dict, modules are cached inside a dict. They are efficient and fast !and highly tuned, and there is almost never any need to try to re-invent !the wheel. ! ! That's an interesting use of dictionaries. Makes sense. Clayton !-- !Steven !_______________________________________________ !Tutor maillist - Tutor at python.org !To unsubscribe or change subscription options: !https://mail.python.org/mailman/listinfo/tutor From alan.gauld at btinternet.com Wed Oct 22 20:37:11 2014 From: alan.gauld at btinternet.com (Alan Gauld) Date: Wed, 22 Oct 2014 19:37:11 +0100 Subject: [Tutor] Python sqlite3 issue In-Reply-To: <CAAp0bGuwhXhXQNddqc+h96UzP8n4MLPhHnpPaMr-bZXRXmfQQA@mail.gmail.com> References: <CAAp0bGtcWgVm118pwqr0nnrQeZ=uhDwaM1Gc+yqNTwgj8fgK7A@mail.gmail.com> <m240k7$l7q$1@ger.gmane.org> <m254sf$btp$1@ger.gmane.org> <m259s2$ige$1@ger.gmane.org> <CAAp0bGuwhXhXQNddqc+h96UzP8n4MLPhHnpPaMr-bZXRXmfQQA@mail.gmail.com> Message-ID: <m28tgo$tsp$1@ger.gmane.org> On 22/10/14 18:38, Juan Christian wrote: > def ini_db(): > db.execute(''' > CREATE TABLE IF NOT EXISTS TOPICS( > ID INTEGER NOT NULL, > URL VARCHAR NOT NULL, > AUTHOR VARCHAR NOT NULL, > MESSAGE VARCHAR NOT NULL > ); > ''') So you no longer have a primary key. You are taking on management of uniqueness yourself. That's OK provided you understand fully the consequences of that decision. Incidentally you don't need the semi-colon inside the execute. It can only execute the entire string so if there's only one command you don't need the terminator. > def insert_db(_id, url, author, message): > db.execute("INSERT INTO TOPICS (ID, URL, AUTHOR, MESSAGE) VALUES > (?, ?, ?, ?)", (_id, url, author, message)) > db.commit() Note that this makes no checks for unique ID so you put the onus on the inserting code to provide a unique ID. > def get_db(_id): > cursor = db.execute("SELECT ID, URL, AUTHOR, MESSAGE FROM TOPICS > WHERE ID = ?", (_id,)) > return cursor.fetchone() What happens if there are multiple rows returned (non unique IDs)? You only get the first (and SQL does not guarantee order so it might be a different one each time...), is that sufficient? > if __name__ == '__main__': > ini_db() > > insert_db(12, 'abc.com <http://abc.com>', 'a', 'b') > insert_db(20, 'abc2.com <http://abc2.com>', 'a2', 'c') > insert_db(1, 'abc3.com <http://abc3.com>', 'a3', 'd') > > for row in db.execute('SELECT * FROM TOPICS ORDER BY ID'): > print(row) > > db.close() > > > ------ > > Anything else that I need to improve/change? Regarding the 'DROP TABLE' > before creating, It wouldn't be good for me, because I do want the 'old' > data from the table there, I don't want to drop them! That's fine but if you ever try to change the structure of your table (other than adding a column to the end) you will have to recreate the table; which you won't be able to do without dropping it first. (You can rename the original however and then copy the data from it to the new table, before dropping the renamed version.) Finally, in production code you should not use select *. Always provide the field names in the order you want them. That's because if somebody else adds columns or changes their order (not likely in SQLite but common in other DBs) your select will not be broken. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.flickr.com/photos/alangauldphotos From fomcl at yahoo.com Wed Oct 22 21:35:23 2014 From: fomcl at yahoo.com (Albert-Jan Roskam) Date: Wed, 22 Oct 2014 12:35:23 -0700 Subject: [Tutor] Insert time into email Message-ID: <1414006523.11761.BPMail_high_carrier@web163804.mail.gq1.yahoo.com> ---------------------------- On Mon, Oct 20, 2014 6:02 PM CEST Chris ?Kwpolska? Warrick wrote: >On Mon, Oct 20, 2014 at 2:34 PM, Bo Morris <crushed26 at gmail.com> wrote: >> hello all, hope everyone is doing well. >> >> The below code works, however I am going back and trying to enter the time >> and date and I cant quite figure out how to do this without breaking the >> code. >> >> #!/usr/bin/python >> >> import smtplib >> from email.MIMEMultipart import MIMEMultipart >> from email.MIMEText import MIMEText >> from email.MIMEImage import MIMEImage >> import time >> >> strFrom = "HourlyReport.com" > >PS. You may want to use a real e-mail address here. Or, at the very >least, something that looks like one. > >> #strTo = "engineering at oneconnxt.com" >> #strTo = "mmedley at onemediacorpinc.com" >> strTo = "bo at onemediacorpinc.com" >> >> l = ['3102EHD-01108.png', '3102DHD-01109.png','3102EHD-01082.png', >> '3102DHD-01033.png', '3102EHD-01302.png', '3102DHD-01149.png', >> '3102EHD-01125.png', '3102DHD-01144.png', '3102EHD-01105.png'] >> >> t = time.strftime("%H:%M:%S") >> d = time.strftime("%d/%m/%Y") >> >> msgRoot = MIMEMultipart('related') >> msgRoot['Subject'] = 'Test Hourly Report' >> msgRoot['From'] = strFrom >> msgRoot['To'] = strTo >> msgRoot.preamble = 'This is a multi-part message in MIME format.' >> Shouldn't date be specified too? msgRoot['Date'] = datetime... The other day I forgot to do this and the date was Jan 1970 iirc (Linux, I have never seen it on Windows) From juan0christian at gmail.com Thu Oct 23 00:30:46 2014 From: juan0christian at gmail.com (Juan Christian) Date: Wed, 22 Oct 2014 20:30:46 -0200 Subject: [Tutor] Python sqlite3 issue In-Reply-To: <m28tgo$tsp$1@ger.gmane.org> References: <CAAp0bGtcWgVm118pwqr0nnrQeZ=uhDwaM1Gc+yqNTwgj8fgK7A@mail.gmail.com> <m240k7$l7q$1@ger.gmane.org> <m254sf$btp$1@ger.gmane.org> <m259s2$ige$1@ger.gmane.org> <CAAp0bGuwhXhXQNddqc+h96UzP8n4MLPhHnpPaMr-bZXRXmfQQA@mail.gmail.com> <m28tgo$tsp$1@ger.gmane.org> Message-ID: <CAAp0bGtm2L4rFrmKNhp_efxb+xcxh7XhCAZ6c-vkb8MbC5bUJg@mail.gmail.com> On Wed, Oct 22, 2014 at 4:37 PM, Alan Gauld <alan.gauld at btinternet.com> wrote: > > Incidentally you don't need the semi-colon inside the execute. It can only > execute the entire string so if there's only one command you > don't need the terminator. Ok, thanks! Note that this makes no checks for unique ID so you put the onus > on the inserting code to provide a unique ID. > > What happens if there are multiple rows returned (non unique IDs)? > You only get the first (and SQL does not guarantee order so it might > be a different one each time...), is that sufficient? > I'll get this id from here (http://steamcommunity.com/app/440/tradingforum/), every topic has a unique ID. That's fine but if you ever try to change the structure of your table > (other than adding a column to the end) you will have to recreate the > table; which you won't be able to do without dropping it first. (You can > rename the original however and then copy the data from it to the new > table, before dropping the renamed version.) > I won't need to change the structure, but if I do, I can DROP the table, no problem with that. Finally, in production code you should not use select *. Always provide the > field names in the order you want them. That's because if somebody else > adds columns or changes their order (not likely in SQLite but common in > other DBs) your select will not be broken. Ok, thanks! NEW CODE: import sqlite3 db = sqlite3.connect('db.sqlite') def ini_db(): db.execute('''CREATE TABLE IF NOT EXISTS TOPICS ( ID INTEGER NOT NULL, URL VARCHAR NOT NULL, AUTHOR VARCHAR NOT NULL, MESSAGE VARCHAR NOT NULL )''' ) def insert(topic_id, url, author, message): db.execute("INSERT INTO TOPICS (ID, URL, AUTHOR, MESSAGE) VALUES (?, ?, ?, ?)", (topic_id, url, author, message)) db.commit() def get(topic_id): cursor = db.execute("SELECT ID, URL, AUTHOR, MESSAGE FROM TOPICS WHERE ID = ?", (topic_id,)) return cursor.fetchone() if __name__ == '__main__': ini_db() insert(12, 'abc.com', 'a', 'b') insert(20, 'abc2.com', 'a2', 'c') insert(1, 'abc3.com', 'a3', 'd') for row in db.execute('SELECT ID, URL, AUTHOR, MESSAGE FROM TOPICS'): print(row) db.close() The only thing left now is that the topics in this forum has a one/two weeks lifespan, and I think Steam reuses the same ID for new topics that was used in, lets say a 1 month-old topic (already closed and gone for weeks), I don't know for sure, but their Inventory/User API is a mess in some parts, so I don't trust them in this matter either. How would be a good approach regarding that? Use UPDATE? Use if-else? -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/tutor/attachments/20141022/2d203083/attachment-0001.html> From eryksun at gmail.com Thu Oct 23 01:05:33 2014 From: eryksun at gmail.com (eryksun) Date: Wed, 22 Oct 2014 18:05:33 -0500 Subject: [Tutor] Passing Data to .DLL In-Reply-To: <1A99A1517680884DAE440F5341907717872BCD83@DVRMBX02.corp.atmel.com> References: <1A99A1517680884DAE440F5341907717872BBE55@DVRMBX02.corp.atmel.com> <CACL+1asL7YuR6bD3wkCVJyW8aqN+wA7=-TvwUc4BUvVS8zSG_w@mail.gmail.com> <1A99A1517680884DAE440F5341907717872BCD83@DVRMBX02.corp.atmel.com> Message-ID: <CACL+1auX6vFSh9LtmL=KhEuLhdAqqtV5i2XPHtU7=Lc+ZWFQWw@mail.gmail.com> On Wed, Oct 22, 2014 at 3:50 PM, Wilson, Pete <Pete.Wilson at atmel.com> wrote: > I don't understand the line > rx_buf = (c_uint8 * rx_buf_size).from_buffer_copy(string_buf) A buffer is a block of memory used to pass data between functions, processes, or systems. Specifically its use as a 'buffer' comes from using a block of memory to buffer I/O operations. There are also buffers implemented in hardware. They're typically FIFO (first in, first out) queues or circular buffers (e.g. the sample buffer in an ADC). In ctypes, the expression `c_uint8 * rx_buf_size` creates an array type that consists of rx_buf_size elements of type c_uint8. Note that an array in C is an aggregate type. It's not a pointer, even though it degenerates to a pointer to the first element when passed as an argument to a function (look back at how I defined it in process_incoming_serial_data.argtypes). An instance of this array type has a contiguous buffer that consists of rx_buf_size elements that are each sizeof(c_uint8) bytes: >>> rx_buf_size = 8 >>> rx_buf_t = c_uint8 * rx_buf_size >>> sizeof(c_uint8) * 8 8 >>> sizeof(rx_buf_t) 8 > or where .from_buffer_copy() came from but it works... from_buffer is a class method of ctypes data types. It creates an instance that shares the same base address as the target buffer, but only if it's writable and at least the same size. In this case 'buffer' is referring to Python's buffer protocol. Python byte strings implement this protocol, but not unicode strings. Immutable strings are read-only buffers: >>> rx_buf_t.from_buffer('01234567')[:] Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: Cannot use string as modifiable buffer A bytearray is writable: >>> string_buf = bytearray('01234567') >>> rx_buf = rx_buf_t.from_buffer(string_buf) >>> rx_buf[:] [48, 49, 50, 51, 52, 53, 54, 55] Note that the buffer is shared by the bytearray and ctypes array: >>> string_buf[0] = '7' >>> rx_buf[:] [55, 49, 50, 51, 52, 53, 54, 55] from_buffer_copy is similar, accept instead of sharing the buffer it makes a private copy. Thus you can use it with an immutable string: >>> string_buf = '01234567' >>> rx_buf = rx_buf_t.from_buffer_copy(string_buf) >>> rx_buf[:] [48, 49, 50, 51, 52, 53, 54, 55] In this case the buffer is *not* shared: >>> rx_buf[0] = ord('7') >>> string_buf '01234567' These class methods are implemented by the metaclass, PyCArrayType. >>> PyCArrayType = type(rx_buf_t) >>> PyCArrayType <type '_ctypes.PyCArrayType'> >>> print '\n'.join(sorted(vars(PyCArrayType))) __doc__ __mul__ __new__ __rmul__ from_address from_buffer from_buffer_copy from_param in_dll from_address is particularly dangerous. Misusing it will probably crash the interpreter. in_dll gives you access to a library's data exports. For example, the flag Py_HashRandomizationFlag indicates whether the interpreter is randomizing string hashes (command-line option -R). In Python 3 it's on by default: >>> sys.flags.hash_randomization 1 >>> c_int.in_dll(pythonapi, 'Py_HashRandomizationFlag') c_long(1) > I promise I will not knowingly pass Python strings to C. It's fine if you see something such as `const char *var` in the function prototype. The const qualifier lets you know the function promises to not modify the buffer. In C, a promise is the best you can get. Better still the docs will tell you whether or not the buffer needs to be writable. This is an important distinction to make for the shared library itself. A memory block could literally be read only (not just by contract), such that trying to write to it will raise a Windows access violation or POSIX segfault. From alan.gauld at btinternet.com Thu Oct 23 01:51:39 2014 From: alan.gauld at btinternet.com (Alan Gauld) Date: Thu, 23 Oct 2014 00:51:39 +0100 Subject: [Tutor] Python sqlite3 issue In-Reply-To: <CAAp0bGtm2L4rFrmKNhp_efxb+xcxh7XhCAZ6c-vkb8MbC5bUJg@mail.gmail.com> References: <CAAp0bGtcWgVm118pwqr0nnrQeZ=uhDwaM1Gc+yqNTwgj8fgK7A@mail.gmail.com> <m240k7$l7q$1@ger.gmane.org> <m254sf$btp$1@ger.gmane.org> <m259s2$ige$1@ger.gmane.org> <CAAp0bGuwhXhXQNddqc+h96UzP8n4MLPhHnpPaMr-bZXRXmfQQA@mail.gmail.com> <m28tgo$tsp$1@ger.gmane.org> <CAAp0bGtm2L4rFrmKNhp_efxb+xcxh7XhCAZ6c-vkb8MbC5bUJg@mail.gmail.com> Message-ID: <m29fub$hqf$1@ger.gmane.org> On 22/10/14 23:30, Juan Christian wrote: > The only thing left now is that the topics in this forum has a one/two > weeks lifespan, and I think Steam reuses the same ID for new topics that > was used in, lets say a 1 month-old topic In that case I'd go back to using a primary key ID set by SQLite and have a specific field for SteamID. Then you can have multiple Steam IDs but still have a unique ID for your records. You might need a date field as well to distinguish/sort the various topics with similar IDs. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.flickr.com/photos/alangauldphotos From davea at davea.name Thu Oct 23 02:31:38 2014 From: davea at davea.name (Dave Angel) Date: Wed, 22 Oct 2014 20:31:38 -0400 (EDT) Subject: [Tutor] A couple of somewhat esoteric questions References: <022801cfed8a$5e6ee770$1b4cb650$@us> <20141022114016.GC20010@ando.pearwood.info> <034a01cfee24$87d46230$977d2690$@us> Message-ID: <m29i2l$gnb$1@ger.gmane.org> "Clayton Kirkwood" <crk at godblessthe.us> Wrote in message: > > > !-----Original Message----- > !From: Tutor [mailto:tutor-bounces+crk=godblessthe.us at python.org] On > !Behalf Of Steven D'Aprano ... > > For clarification, a key only has one value which can be changed. No, because the key has to be immutable, like a string or an int. > Multiple > keys can have the same value (hopefully not to the same memory location, > because one would usually not want the change of one key's value to alter > another key's value. As to the hash table, yes, I agree that this would be > one to many, hence the chains. Memory locations are an implementation detail. We're talking in this paragraph about keys and values. A value is an object, the dict provides a one-way mapping from immutable key to an arbitrary value object. If the value objects happen to be immutable, it makes no difference if a program uses the same value for multiple keys. If the values are not immutable, python makes no assurance whether two values are equal, or identical. If the latter is undesirable for a particular use, it's up to the application logic to prevent it. It's probably more often useful than problematic. Python makes no such assurances. > !Can you give a concrete example of what you're trying to do? > > The changing of the order is necessary when you want numeric indexing, say > for columns or rows. > No idea how this relates to dicts, which have no order, nor rows nor columns. Make it concrete. -- DaveA From eryksun at gmail.com Thu Oct 23 02:35:52 2014 From: eryksun at gmail.com (eryksun) Date: Wed, 22 Oct 2014 19:35:52 -0500 Subject: [Tutor] Passing Data to .DLL In-Reply-To: <CACL+1auX6vFSh9LtmL=KhEuLhdAqqtV5i2XPHtU7=Lc+ZWFQWw@mail.gmail.com> References: <1A99A1517680884DAE440F5341907717872BBE55@DVRMBX02.corp.atmel.com> <CACL+1asL7YuR6bD3wkCVJyW8aqN+wA7=-TvwUc4BUvVS8zSG_w@mail.gmail.com> <1A99A1517680884DAE440F5341907717872BCD83@DVRMBX02.corp.atmel.com> <CACL+1auX6vFSh9LtmL=KhEuLhdAqqtV5i2XPHtU7=Lc+ZWFQWw@mail.gmail.com> Message-ID: <CACL+1avpSLbh-vwfyu7Qm6m_UEr3BMEQMPRvwjVKY9eGOt5HZg@mail.gmail.com> On Wed, Oct 22, 2014 at 6:05 PM, eryksun <eryksun at gmail.com> wrote: > from_buffer_copy is similar, accept instead of sharing the buffer That should be ex-cept (conjunction for an exception clause), not ac-cept (verb, to receive). I missed that in my initial proofread. It takes a while to clear my mental buffer enough for a fresh look. From davea at davea.name Thu Oct 23 03:22:53 2014 From: davea at davea.name (Dave Angel) Date: Wed, 22 Oct 2014 21:22:53 -0400 (EDT) Subject: [Tutor] A couple of somewhat esoteric questions References: <022801cfed8a$5e6ee770$1b4cb650$@us> <20141022114016.GC20010@ando.pearwood.info> <034a01cfee24$87d46230$977d2690$@us> <m29i2l$gnb$1@ger.gmane.org> Message-ID: <m29l2m$rnu$1@ger.gmane.org> Dave Angel <davea at davea.name> Wrote in message: > "Clayton Kirkwood" <crk at godblessthe.us> Wrote in message: >> >> >> !-----Original Message----- >> !From: Tutor [mailto:tutor-bounces+crk=godblessthe.us at python.org] On >> !Behalf Of Steven D'Aprano > ... >> >> For clarification, a key only has one value which can be changed. > > No, because the key has to be immutable, like a string or an int. My error for reading your statement wrong. > >> Multiple >> keys can have the same value (hopefully not to the same memory location, >> because one would usually not want the change of one key's value to alter >> another key's value. As to the hash table, yes, I agree that this would be >> one to many, hence the chains. > > Memory locations are an implementation detail. We're talking in > this paragraph about keys and values. A value is an object, the > dict provides a one-way mapping from immutable key to an > arbitrary value object. If the value objects happen to be > immutable, it makes no difference if a program uses the same > value for multiple keys. If the values are not immutable, python > makes no assurance whether two values are equal, or identical. If > the latter is undesirable for a particular use, it's up to the > application logic to prevent it. It's probably more often useful > than problematic. Python makes no such assurances. > > > > > >> !Can you give a concrete example of what you're trying to do? >> >> The changing of the order is necessary when you want numeric indexing, say >> for columns or rows. >> > > No idea how this relates to dicts, which have no order, nor rows > nor columns. Make it concrete. > > > -- > DaveA > > _______________________________________________ > Tutor maillist - Tutor at python.org > To unsubscribe or change subscription options: > https://mail.python.org/mailman/listinfo/tutor > > -- DaveA From a.bull at pubdmgroup.com Wed Oct 22 20:43:27 2014 From: a.bull at pubdmgroup.com (Al Bull) Date: Wed, 22 Oct 2014 13:43:27 -0500 Subject: [Tutor] Question on a select statement with ODBC In-Reply-To: <m28oil$98v$1@ger.gmane.org> References: <014301cfed60$edb83730$c928a590$@pubdmgroup.com> <m26qvl$cb2$1@ger.gmane.org> <00aa01cfee09$c42e2ab0$4c8a8010$@pubdmgroup.com> <m28oil$98v$1@ger.gmane.org> Message-ID: <010401cfee28$113ff360$33bfda20$@pubdmgroup.com> -----Original Message----- From: Tutor [mailto:tutor-bounces+a.bull=pubdmgroup.com at python.org] On Behalf Of Alan Gauld Sent: Wednesday, October 22, 2014 12:13 PM To: tutor at python.org Subject: Re: [Tutor] Question on a select statement with ODBC On 22/10/14 16:06, Al Bull wrote: > I don't think I explained the problem properly. I have several hundred > thousand records in the ORD table. There are many instances of records with > identical ORD_DBASUB values. Where duplicates exist, I only want to keep > the most current record. Ah, OK thats very different. You can do it in SQL but it gets messy and depends on the details of the ODBC SQL, which I don't know... It would involve a nested select with an inner join I suspect. > This code works except in very specific cases. Take the following example: > ORD_DBASUB DATE > 1) 100000360 2004-11-02 > 2) 100000360 2004-09-03 > 3) 100000334 2004-04-05 > 4) 100000334 2004-03-08 > > Record #3 is correctly saved, but record #4 is not removed. It appears > that ROW is being moved to the next entry after the ord_rows.remove That's correct you should never modify the collection that you are iterating over with a for loop. Instead convert to using a while loop and only increment the index if you don't remove an thing. Alternatively make a copy of the collection and iterate over that, but a while is usually preferable IMHO. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.flickr.com/photos/alangauldphotos _______________________________________________ [Al Bull] Quick question then... Does this do the trick? Currentrecord = 1 While currentrecord <= len(ord_rows): if savedbasub == currentrecord.ord_dbasub: ord_rows.remove(currentrecord) delcount += 1 else: savedbasub = currentrecord.ord_dbasub currentrecord =+ 1 From crk at godblessthe.us Wed Oct 22 20:48:44 2014 From: crk at godblessthe.us (Clayton Kirkwood) Date: Wed, 22 Oct 2014 11:48:44 -0700 Subject: [Tutor] yet another misunderstanding on my part In-Reply-To: <20141022110251.GB20010@ando.pearwood.info> References: <026001cfedb4$510d0e20$f3272a60$@us> <20141022110251.GB20010@ando.pearwood.info> Message-ID: <035401cfee28$ce970520$6bc50f60$@us> !-----Original Message----- !From: Tutor [mailto:tutor-bounces+crk=godblessthe.us at python.org] On !Behalf Of Steven D'Aprano !Sent: Wednesday, October 22, 2014 4:03 AM !To: tutor at python.org !Subject: Re: [Tutor] yet another misunderstanding on my part ! !On Tue, Oct 21, 2014 at 09:54:49PM -0700, Clayton Kirkwood wrote: ! !> col_position, code, description = 0, [], [] key_name = !> raw_table.replace('\t','\n') for each_line in key_name.splitlines(): !> if ':' in each_line: !> code[col_position], description.append() = !> each_line.split(':') #neither works; first one is out of range error, !> 2nd, can't assign to #function. I've used square brackets around !> various sections, and it doesn't like it ! !The trick to learning how to program is NOT to randomly make changes to !your code and hope that you get lucky, but to understand why you are !getting the errors you get. Trust me, I iterate and look for examples instead of randomly running around. Regarding the index out of range, I know what it meant, I was just kind of surprised that Python didn't automatically create the node and stuff a value in. ! !Start here: ! !position = 0 !codes = [] !codes[position] = 999 ! ! !This fails with an error: ! !IndexError: list assignment index out of range ! ! !What does that mean? It's an English sentence, a little bit terse but !still understandable: ! !"list assignment index" -- what's that? It's the index used inside the !square brackets, being used for list assignment. In this case, the index !is 0. ! !"out of range" -- what's that mean? It tells you that the index you have !provided (in this case, 0) is too big. Let's experiment to see what "too !big" means: ! !py> alist = ['a', 'b', 'c'] # three items alist[0] = 'A' !py> alist[1] = 'B' !py> alist[2] = 'C' !py> alist[3] = 'D' !Traceback (most recent call last): ! File "<stdin>", line 1, in <module> !IndexError: list assignment index out of range ! ! !Aha! You can only assign to items in a list which already exist. In the !experiment, I had a list with three items, item 0, item 1 and item 2, !and assigning to index 0, 1 and 2 succeeded. But assigning to item 3 !(which doesn't exist) fails. ! !Go back to your list. It is an empty list, [], which means it has *no* !items in it. Since there are no items in it, you cannot assign to any !(non-existent) item. ! !If you can't assign to an item, since there aren't any, you have to !append to the list. ! !I'm not sure where you got the idea of writing ! !description.append() = something ! ! !from. There's nothing in Python that I've ever seen that suggests that !would work, and the error message should be clear: ! !py> x() = 23 ! File "<stdin>", line 1 !SyntaxError: can't assign to function call >>> freshfruit = [' banana', ' loganberry ', 'passion fruit '] >>> [weapon.strip() for weapon in freshfruit] ['banana', 'loganberry', 'passion fruit'] >From 5.1.3 in the Python3 tutorial. Although not an = assignment, some value is being stored in weapon after being strip of whitespace. I realize they are different, maybe just not sure how different:<) In this case, somehow Python seems to keep track of the location so that the modified value(dropping spaces) replaces the original location. ! ! !Try this instead: ! !py> descriptions = [] !py> descriptions.append("something blue") descriptions.append("something !py> round") descriptions.append("something flat") !py> print(descriptions) !['something blue', 'something round', 'something flat'] ! ! !Like the name suggests, "append" appends something to the end of the !list. So your code, which started like this: ! !# doesn't work !col_position, code, description = 0, [], [] key_name = !raw_table.replace('\t','\n') for each_line in key_name.splitlines(): ! if ':' in each_line: ! code[col_position], description.append() = each_line.split(':') ! ! !could be written something like this: ! !# note plural names !codes, descriptions = [], [] !key_name = raw_table.replace('\t','\n') !for each_line in key_name.splitlines(): ! if ':' in each_line: ! code, description = each_line.split(':') ! codes.append(code) ! descriptions.append(description) So two questions remain. Why can't codes.append(code) just replace the code in the previous line and descriptions.append(description) replace the description in the previous line. Previous emails have seen something like value.strip() = some value . Second question, why can't a numeric index be used to make assignment to a specific location like a[1] = "some value"? If the mechanism is to use a.index(1,"some value"), it seems somewhat clumsy. It is clear that once the array is created, it is possible to access the value by numeric indexing such as: print( col_position, code[col_position], description[col_position]) which produces: 83 r7 Price / EPS Estimate Next Year 84 s7 Short Ratio Also, >>> # create a list of 2-tuples like (number, square) >>> [(x, x**2) for x in range(6)] [(0, 0), (1, 1), (2, 4), (3, 9), (4, 16), (5, 25)] Why does this not need the .append or .insert? Square brackets around the whole line? Clayton ! ! ![...] !> What am I not seeing? I *do* spend hours trying different options, and !> study various documentation which confuse me and don't seem !consistent. ! !It seems to me that you are probably failing to isolate the problem to !*one* thing. When you get an error, you should ignore EVERYTHING else !and just play around with that one function until you understand it. !Open the interactive interpreter, and experiment, like I did above with !the "alist[0]" assignments. ! ! !-- !Steven !_______________________________________________ !Tutor maillist - Tutor at python.org !To unsubscribe or change subscription options: !https://mail.python.org/mailman/listinfo/tutor From Pete.Wilson at atmel.com Wed Oct 22 22:50:13 2014 From: Pete.Wilson at atmel.com (Wilson, Pete) Date: Wed, 22 Oct 2014 20:50:13 +0000 Subject: [Tutor] Passing Data to .DLL In-Reply-To: <CACL+1asL7YuR6bD3wkCVJyW8aqN+wA7=-TvwUc4BUvVS8zSG_w@mail.gmail.com> References: <1A99A1517680884DAE440F5341907717872BBE55@DVRMBX02.corp.atmel.com> <CACL+1asL7YuR6bD3wkCVJyW8aqN+wA7=-TvwUc4BUvVS8zSG_w@mail.gmail.com> Message-ID: <1A99A1517680884DAE440F5341907717872BCD83@DVRMBX02.corp.atmel.com> It's working! I don't understand the line rx_buf = (c_uint8 * rx_buf_size).from_buffer_copy(string_buf) or where .from_buffer_copy() came from but it works... I promise I will not knowingly pass Python strings to C. Thanks. > -----Original Message----- > From: eryksun [mailto:eryksun at gmail.com] > Sent: Wednesday, October 22, 2014 4:16 AM > To: Wilson, Pete > Cc: tutor at python.org > Subject: Re: [Tutor] Passing Data to .DLL > > On Tue, Oct 21, 2014 at 7:04 PM, Wilson, Pete <Pete.Wilson at atmel.com> > wrote: > > > > ProcessIncomingSerialData_t = CFUNCTYPE(None, POINTER(c_uint8), > > c_uint16) process_incoming_serial_data = > > pt_dll.ProcessIncomingSerialData > process_incoming_serial_data.argtypes > > = [ProcessIncomingSerialData_t] > > ProcessIncomingSerialData takes two parameters: uint8_t *rx_buf and > uint16_t size. > > process_incoming_serial_data.argtypes = [POINTER(c_uint8), > c_uint16] > process_incoming_serial_data.restype = None > > The buffer parameter isn't const, so pass a copy of string_buf. > > size = len(string_buf) > rx_buf = (c_uint8 * size).from_buffer_copy(string_buf) > process_incoming_serial_data(rx_buf, size) > > I'm going to meander off topic a bit to give you a warning... > > Python strings are immutable by contract, so don't pass them directly > to C functions that might modify them. String immutability makes > interning possible, sometimes just locally in the code object and > sometimes globally in the interpreter. Here's an example of both in > CPython 2.7: > > import abc > from ctypes import * > > def test(): > s = 'abc' > a = cast(s, POINTER(c_char * 3))[0] > a[:] = 'cba' > print 'abc' > > >>> test.__code__.co_consts > (None, 'abc', 3, 0, 'cba') > > Notice that the 2nd constant in co_consts is 'abc'. This gets stored to > s and later printed. In between I use ctypes to reverse it. So what > gets printed? If you guessed "cba", you're right. > > >>> test() > cba > > Look at the constants now: > > >>> test.__code__.co_consts > (None, 'cba', 3, 0, 'cba') > > So how about referencing the abc module? > > >>> abc > Traceback (most recent call last): > File "<stdin>", line 1, in <module> > NameError: name 'abc' is not defined > > OK, but do you think we can use cba instead? > > >>> cba > Traceback (most recent call last): > File "<stdin>", line 1, in <module> > NameError: name 'cba' is not defined > > This is yet another problem with mucking with the internal state of an > immutable object. Equal objects are supposed to hash the same. 'cba' > is now equal to the interned 'abc' string that I reversed, but it > probes to a different slot in the dict's hash table. > > >>> test.__code__.co_consts[1] == 'cba' > True > >>> hash(test.__code__.co_consts[1]) == hash('cba') > False > > OTOH, a new 'abc' string probes to the same slot in the hash table, but > it's no longer equal (i.e. it gets handled as a hash collision). > > >>> test.__code__.co_consts[1] == 'abc' > False > >>> hash(test.__code__.co_consts[1]) == hash('abc') > True > > This breaks dict lookup unless we use the interned string itself:. > > >>> globals()[test.__code__.co_consts[1]] > <module 'abc' from '/usr/lib/python2.7/abc.pyc'> > > So the moral is only pass Python strings to C functions that promise > (scout's honor) to not modify them. If the parameter isn't const, err > on the side of caution; copy the string to a ctypes buffer. From Pete.Wilson at atmel.com Thu Oct 23 06:34:09 2014 From: Pete.Wilson at atmel.com (Wilson, Pete) Date: Thu, 23 Oct 2014 04:34:09 +0000 Subject: [Tutor] Passing Data to .DLL In-Reply-To: <CACL+1avpSLbh-vwfyu7Qm6m_UEr3BMEQMPRvwjVKY9eGOt5HZg@mail.gmail.com> References: <1A99A1517680884DAE440F5341907717872BBE55@DVRMBX02.corp.atmel.com> <CACL+1asL7YuR6bD3wkCVJyW8aqN+wA7=-TvwUc4BUvVS8zSG_w@mail.gmail.com> <1A99A1517680884DAE440F5341907717872BCD83@DVRMBX02.corp.atmel.com> <CACL+1auX6vFSh9LtmL=KhEuLhdAqqtV5i2XPHtU7=Lc+ZWFQWw@mail.gmail.com> <CACL+1avpSLbh-vwfyu7Qm6m_UEr3BMEQMPRvwjVKY9eGOt5HZg@mail.gmail.com> Message-ID: <1A99A1517680884DAE440F5341907717872BD861@DVRMBX02.corp.atmel.com> No worries, if I could spell I would have been a Lawyer. Pete > -----Original Message----- > From: eryksun [mailto:eryksun at gmail.com] > Sent: Wednesday, October 22, 2014 5:36 PM > To: Wilson, Pete > Cc: tutor at python.org > Subject: Re: [Tutor] Passing Data to .DLL > > On Wed, Oct 22, 2014 at 6:05 PM, eryksun <eryksun at gmail.com> wrote: > > from_buffer_copy is similar, accept instead of sharing the buffer > > That should be ex-cept (conjunction for an exception clause), not ac- > cept (verb, to receive). I missed that in my initial proofread. It > takes a while to clear my mental buffer enough for a fresh look. From davea at davea.name Thu Oct 23 10:52:48 2014 From: davea at davea.name (Dave Angel) Date: Thu, 23 Oct 2014 04:52:48 -0400 (EDT) Subject: [Tutor] yet another misunderstanding on my part References: <026001cfedb4$510d0e20$f3272a60$@us> <20141022110251.GB20010@ando.pearwood.info> <035401cfee28$ce970520$6bc50f60$@us> Message-ID: <m2afea$6jg$1@ger.gmane.org> "Clayton Kirkwood" <crk at godblessthe.us> Wrote in message: (Somehow, your email program seems to be using the exclamation point to identify quoted lines, instead of the standard greater-than symbol. Is that something you can correct, prrhaps using "settings"?) > > > !-----Original Message----- > !From: Tutor [mailto:tutor-bounces+crk=godblessthe.us at python.org] On > !Behalf Of Steven D'Aprano > >>>> freshfruit = [' banana', ' loganberry ', 'passion fruit '] >>>> [weapon.strip() for weapon in freshfruit] > ['banana', 'loganberry', 'passion fruit'] > From 5.1.3 in the Python3 tutorial. Although not an = assignment, some value > is being stored in weapon after being strip of whitespace. I realize they > are different, maybe just not sure how different:<) In this case, somehow > Python seems to keep track of the location so that the modified > value(dropping spaces) replaces the original location. > > ! > ! > !Try this instead: > ! > !py> descriptions = [] > !py> descriptions.append("something blue") descriptions.append("something > !py> round") descriptions.append("something flat") > !py> print(descriptions) > !['something blue', 'something round', 'something flat'] > ! > ! > !Like the name suggests, "append" appends something to the end of the > !list. So your code, which started like this: > ! > !# doesn't work > !col_position, code, description = 0, [], [] key_name = > !raw_table.replace('\t','\n') for each_line in key_name.splitlines(): > ! if ':' in each_line: > ! code[col_position], description.append() = each_line.split(':') > ! > ! > !could be written something like this: > ! > !# note plural names > !codes, descriptions = [], [] > !key_name = raw_table.replace('\t','\n') > !for each_line in key_name.splitlines(): > ! if ':' in each_line: > ! code, description = each_line.split(':') > ! codes.append(code) > ! descriptions.append(description) > > So two questions remain. Why can't codes.append(code) just replace the code > in the previous line and descriptions.append(description) replace the > description in the previous line. Previous emails have seen something like > value.strip() = some value . That last line won't work either. If you did try to do it on one line, you'd need to put the argument to append inside the parens, not on the other side of an equal sign. Something like (untested ) codes.append (each_line.split()[0]) descriptions.append (each_line.split()[1] Second question, why can't a numeric index be > used to make assignment to a specific location like a[1] = "some value"? If > the mechanism is to use a.index(1,"some value"), The index() method does not change the object a, at least not for list objects. So that is not a replacement for subscripted assignment. The form a[1] = "some_value" works fine, as long as there is already an object in that list element. In other words, it can be used to replace an item, but not to change the size. Python does not support sparse lists, so rather than permitting a size increase of exactly one, it was decided that the syntax would not permit any size change. And to grow by 1, append works very well. If you know ahead of time what size you need, you could prefill it with something like: a = [] * 20 But you might want to check later that you replaced them all. > it seems somewhat clumsy. > It is clear that once the array We have been talking about lists here, not arrays. The type array.array behaves similarly, but not the same. > is created, it is possible to access the > value by numeric indexing such as: > print( col_position, code[col_position], description[col_position]) > which produces: > 83 r7 Price / EPS Estimate Next Year > 84 s7 Short Ratio > > Also, >>>> # create a list of 2-tuples like (number, square) >>>> [(x, x**2) for x in range(6)] > [(0, 0), (1, 1), (2, 4), (3, 9), (4, 16), (5, 25)] > > Why does this not need the .append or .insert? Square brackets around the > whole line? > Those square brackets, with the for keyword inside, comprise a more advanced technique called a list comprehension. It's syntactic sugar for a for loop with an append. But the list object being built is anonymous, so it can frequently be used inside other expressions. I recommend ignoring list comprehensions till you can reliably write the loop. You seem determined to cram things onto a single line that would probably be more readable as separate ones. And intermediate variables give an opportunity to make up useful names, to make the code easier to read. -- DaveA From __peter__ at web.de Thu Oct 23 13:14:25 2014 From: __peter__ at web.de (Peter Otten) Date: Thu, 23 Oct 2014 13:14:25 +0200 Subject: [Tutor] Question on a select statement with ODBC References: <014301cfed60$edb83730$c928a590$@pubdmgroup.com> <m26qvl$cb2$1@ger.gmane.org> <00aa01cfee09$c42e2ab0$4c8a8010$@pubdmgroup.com> <m28oil$98v$1@ger.gmane.org> <010401cfee28$113ff360$33bfda20$@pubdmgroup.com> Message-ID: <m2anuh$i7s$1@ger.gmane.org> Al Bull wrote: > Quick question then... > > Does this do the trick? I may have misunderstood your original question; do you want to delete records from the database or from the Python list? Your code below will do the latter, once you have fixed the bugs. > Currentrecord = 1 > > While currentrecord <= len(ord_rows): > if savedbasub == currentrecord.ord_dbasub: > ord_rows.remove(currentrecord) > delcount += 1 > else: > savedbasub = currentrecord.ord_dbasub > currentrecord =+ 1 As the general idea is correct I'll encourage you to try to fix these bugs yourself. The first thing you have to resolve: is currentrecord an index or an object representing a database row? From alan.gauld at btinternet.com Thu Oct 23 13:17:51 2014 From: alan.gauld at btinternet.com (Alan Gauld) Date: Thu, 23 Oct 2014 12:17:51 +0100 Subject: [Tutor] yet another misunderstanding on my part In-Reply-To: <035401cfee28$ce970520$6bc50f60$@us> References: <026001cfedb4$510d0e20$f3272a60$@us> <20141022110251.GB20010@ando.pearwood.info> <035401cfee28$ce970520$6bc50f60$@us> Message-ID: <m2ao50$o5s$1@ger.gmane.org> On 22/10/14 19:48, Clayton Kirkwood wrote: > Regarding the index out of range, I know what it meant, I was just kind of > surprised that Python didn't automatically create the node and stuff a value > in. But what should Python do in this case aNewList[1000000] = 42 Should Python create a 1 million element list with only one value? That would be a slow operation. What should the other values be set to? None, presumably? > !description.append() = something > ! > ! > !from. There's nothing in Python that I've ever seen that suggests that > !would work, and the error message should be clear: >>>> freshfruit = [' banana', ' loganberry ', 'passion fruit '] >>>> [weapon.strip() for weapon in freshfruit] > ['banana', 'loganberry', 'passion fruit'] > From 5.1.3 in the Python3 tutorial. > Although not an = assignment, some value > is being stored in weapon after being strip of whitespace. > I realize they are different, maybe just not sure how different:<) This is a list comprehension with a very specific syntax. It is equivalent to aList = [] for weapon in freshfruit: aList.append(weapon.strip()) It is very different from assigning a value to a function call. > In this case, somehow > Python seems to keep track of the location so that the modified > value(dropping spaces) replaces the original location. No it does not replace the original location it is appended to the list. > So two questions remain. Why can't codes.append(code) just replace the code > in the previous line and descriptions.append(description) replace the > description in the previous line. Because the append() adds it at the end it doesn't replace anything. > value.strip() = some value . Second question, why can't a numeric index be > used to make assignment to a specific location like a[1] = "some value"? It can if that entry already exists. listA = [0,1,2,3,4,5] #initialize with elements listA[3] = 9 print(listA) # -> [0,1,2,9,4,5] listB = [] listB[3] = 9 # error because listB has no elements yet. listB += [0,1,2,3,4] # add some elements listB[3] = 9 # OK now. > It is clear that once the array is created, Its not an array, its a list. Most specifically it is not a C style block of memory that is just filled in with a set of bytes. It is a dynamic sequence of objects of potentially mixed(and changing) type. >>>> # create a list of 2-tuples like (number, square) >>>> [(x, x**2) for x in range(6)] > [(0, 0), (1, 1), (2, 4), (3, 9), (4, 16), (5, 25)] > > Why does this not need the .append or .insert? Square brackets around the > whole line? Because the list comprehension implicitly does an append for you (see above) -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.flickr.com/photos/alangauldphotos From davea at davea.name Thu Oct 23 14:58:13 2014 From: davea at davea.name (Dave Angel) Date: Thu, 23 Oct 2014 08:58:13 -0400 (EDT) Subject: [Tutor] yet another misunderstanding on my part References: <026001cfedb4$510d0e20$f3272a60$@us> <20141022110251.GB20010@ando.pearwood.info> <035401cfee28$ce970520$6bc50f60$@us> <m2afea$6jg$1@ger.gmane.org> Message-ID: <m2atqe$rco$1@ger.gmane.org> Dave Angel <davea at davea.name> Wrote in message: > "Clayton Kirkwood" <crk at godblessthe.us> Wrote in message: > > > Second question, why can't a numeric index be >> used to make assignment to a specific location like a[1] = "some value"? If >> the mechanism is to use a.index(1,"some value"), > > The index() method does not change the object a, at least not for > list objects. So that is not a replacement for subscripted > assignment. In particular, the index () method searches the list. > > The form a[1] = "some_value" works fine, as long as there is > already an object in that list element. In other words, it can be > used to replace an item, but not to change the size. Python does > not support sparse lists, so rather than permitting a size > increase of exactly one, it was decided that the syntax would not > permit any size change. And to grow by 1, append works very > well. > > If you know ahead of time what size you need, you could prefill it > with something like: > > a = [] * 20 Oops. I meant: A = [None] * 20 > > But you might want to check later that you replaced them all. > > > -- DaveA From robertvstepp at gmail.com Fri Oct 24 05:35:17 2014 From: robertvstepp at gmail.com (boB Stepp) Date: Thu, 23 Oct 2014 22:35:17 -0500 Subject: [Tutor] Is there a convenient table of Python 3.4 exceptions? Message-ID: <CANDiX9Kq997q6dawd2OpU0Cpk=htd0r8_crfqo5a_SQd334B8w@mail.gmail.com> I am reading a brief intro to exception handling in Mark Summerfield's "Programming in Python 3, 2nd ed." He gives the basic syntax as: try: try_suite except exception1 as variable1: exception_suite1 ... except exceptionN as variableN: exception_suiteN My current understanding is that exception1, ... , exceptionN should match one of Python's exception classes or subclasses in order to perform a match and execute the corresponding exception suite. I found this in the docs: Exceptions are identified by class instances. The except clause is selected depending on the class of the instance: it must reference the class of the instance or a base class thereof. The instance can be received by the handler and can carry additional information about the exceptional condition. Note Exception messages are not part of the Python API. Their contents may change from one version of Python to the next without warning and should not be relied on by code which will run under multiple versions of the interpreter. ?I have so far been unable to find a list of these class/subclass names. Of course I can force an error to occur in the interpreter and see what comes up for each type of error I wish to catch. Is there such a table or list?? -- boB -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/tutor/attachments/20141023/f17815eb/attachment.html> From ben+python at benfinney.id.au Fri Oct 24 06:04:54 2014 From: ben+python at benfinney.id.au (Ben Finney) Date: Fri, 24 Oct 2014 15:04:54 +1100 Subject: [Tutor] Is there a convenient table of Python 3.4 exceptions? References: <CANDiX9Kq997q6dawd2OpU0Cpk=htd0r8_crfqo5a_SQd334B8w@mail.gmail.com> Message-ID: <85fvee3yft.fsf@benfinney.id.au> boB Stepp <robertvstepp at gmail.com> writes: > My current understanding is that exception1, ... , exceptionN should > match one of Python's exception classes or subclasses in order to > perform a match and execute the corresponding exception suite. Correct. Bear in mind that ?? or subclasses? covers user-defined exception classes, so you'll often see non-standard exception classes in use by particular code bases, that are nevertheless subclasses of the exception classes in the standard library. > ?I have so far been unable to find a list of these class/subclass > names. The standard library documentation's chapter on exceptions <URL:https://docs.python.org/3/library/exceptions.html> shows <URL:https://docs.python.org/3/library/exceptions.html#exception-hierarchy>. But you won't find an exhaustive list of what can raise those exceptions. > Of course I can force an error to occur in the interpreter and see > what comes up for each type of error I wish to catch. Yes, that's the right thing to do; that, and read the documentation for whatever library code you are invoking. It should say if there are important situations where a particular exception will be raised. There will, of course, be a great many other situations that can raise an exception. This is a normal part of the flow of code in Python, and the standard library does a lot of it. More importantly, though, you should consider *why* you're attempting to catch a lot of exception classes. Will you be meaningfully handling every one of those situations? That's rather doubtful. Instead, you should consider which exceptional states your code can meaningfully handle, discover what excpetion classes are raised for those few situations, and catch *only* those classes. Any other exception that gets raised isn't something you can do anything useful with, so it should propagate back up to higher levels, either to be handled or to exit with a useful error message. -- \ ?Science is a way of trying not to fool yourself. The first | `\ principle is that you must not fool yourself, and you are the | _o__) easiest person to fool.? ?Richard P. Feynman, 1964 | Ben Finney From dyoo at hashcollision.org Fri Oct 24 05:57:16 2014 From: dyoo at hashcollision.org (Danny Yoo) Date: Thu, 23 Oct 2014 20:57:16 -0700 Subject: [Tutor] Is there a convenient table of Python 3.4 exceptions? In-Reply-To: <CANDiX9Kq997q6dawd2OpU0Cpk=htd0r8_crfqo5a_SQd334B8w@mail.gmail.com> References: <CANDiX9Kq997q6dawd2OpU0Cpk=htd0r8_crfqo5a_SQd334B8w@mail.gmail.com> Message-ID: <CAGZAPF79FThguBsStQVAoCbgM4Ffbw+92ZMVMXPoCgEas=_Vkw@mail.gmail.com> > I have so far been unable to find a list of these class/subclass names. Of > course I can force an error to occur in the interpreter and see what comes > up for each type of error I wish to catch. Is there such a table or list? > Hi Bob, You can find the ones used in the Standard Library here: https://docs.python.org/3.4/library/exceptions.html Usually the documentation should say what kinds of exceptions to expect. If you find an exception to this, please bring it up, and one of us can investigate: maybe the documentation can be improved. If you have questions, please feel free to ask! Good luck. From ben+python at benfinney.id.au Fri Oct 24 06:50:34 2014 From: ben+python at benfinney.id.au (Ben Finney) Date: Fri, 24 Oct 2014 15:50:34 +1100 Subject: [Tutor] Is there a convenient table of Python 3.4 exceptions? References: <CANDiX9Kq997q6dawd2OpU0Cpk=htd0r8_crfqo5a_SQd334B8w@mail.gmail.com> <CAGZAPF79FThguBsStQVAoCbgM4Ffbw+92ZMVMXPoCgEas=_Vkw@mail.gmail.com> Message-ID: <85bnp23wbp.fsf@benfinney.id.au> (Danny, please preserve attribution lines for the quoted text.) Danny Yoo <dyoo at hashcollision.org> writes: > > Of course I can force an error to occur in the interpreter and see > > what comes up for each type of error I wish to catch. Is there such > > a table or list? > > [?] > Usually the documentation should say what kinds of exceptions to > expect. If you find an exception to this, please bring it up, and one > of us can investigate: maybe the documentation can be improved. That's rather too sweeping a statement, and it's definitely not true for Python's standard library documentation:: >>> import signal >>> signal.getsignal(None) Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: an integer is required (got type NoneType) >>> signal.getsignal(500000) Traceback (most recent call last): File "<stdin>", line 1, in <module> ValueError: signal number out of range There's nothing in the documentation of ?signal.getsignal? which says it will raise either of those exceptions. I'd say it's a safe bet there are numerous other exception classes that will be raised calling that function, too. That's just one, chosen quickly and arbitrarily to prove the point. The same goes for the vast majority of functions in the standard library; most possible exception classes go unmentioned in the documentation for that function1, because exceptions are a normal part of operating on Python values (think of the wealth of possible exception classes that can come from passing in a file-like object to many functions). Indeed, most functions can't possibly have exhaustive documentation for what exception classes might be raised, because most of those possible exceptions are to be raised by the code implementing the *parameter* values, not raised by the code implementing that function. Should every function in every module of the standard library document every exception class that could be raised by that function? I'd say clearly not; the result would be uselessly cluttered. Where should the line be drawn? I think, in the case of the standard library, the line is already drawn at the right place. The documentation should describe situations that *someone already familiar with Python* ? and with the types of objects they're passing to a function ? would not know exactly what exception class might be raised. But for most functions, most of the exception classes they might raise are not documented, because when they are raised it's obvious. Is this a hurdle for newcomers? Yes, and that's why the standard library API documentation is not a tutorial. We have separate tutorial documentation for that. It's not a problem with the documentation of ?signal.getsignal? that it doesn't have an exhaustive list of all exception classes that might be raised. This is Python, not Java; exceptions are used prolifically as a flow control method, for code to let its call stack know something unusual occurred. -- \ ?I went camping and borrowed a circus tent by mistake. I didn't | `\ notice until I got it set up. People complained because they | _o__) couldn't see the lake.? ?Steven Wright | Ben Finney From hanzer at riseup.net Fri Oct 24 03:45:48 2014 From: hanzer at riseup.net (Adam Jensen) Date: Thu, 23 Oct 2014 21:45:48 -0400 Subject: [Tutor] Standard Library Performance (3.4.1) Message-ID: <5449AF4C.50601@riseup.net> I'm tinkering this evening and I've noticed that math.factorial() is much faster than my plain python implementations. -------------------------------------------- import math def factorial(n): temp = 1 for k in range(0,n): temp = temp * (n - k) return(temp) def fac(n): return 1 if (n == 0) else n * fac(n-1) -------------------------------------------- >From IPython: In [21]: %timeit factorial(9) 100000 loops, best of 3: 5.31 ?s per loop In [22]: %timeit fac(9) 100000 loops, best of 3: 6.86 ?s per loop In [23]: %timeit math.factorial(9) 1000000 loops, best of 3: 416 ns per loop -------------------------------------------- Is this kind of performance difference typical of the standard library functions (compared to plain python user implementations)? From alan.gauld at btinternet.com Fri Oct 24 13:03:41 2014 From: alan.gauld at btinternet.com (Alan Gauld) Date: Fri, 24 Oct 2014 12:03:41 +0100 Subject: [Tutor] Standard Library Performance (3.4.1) In-Reply-To: <5449AF4C.50601@riseup.net> References: <5449AF4C.50601@riseup.net> Message-ID: <m2dbmd$aph$1@ger.gmane.org> On 24/10/14 02:45, Adam Jensen wrote: > I'm tinkering this evening and I've noticed that math.factorial() is > much faster than my plain python implementations. > > Is this kind of performance difference typical of the standard library > functions (compared to plain python user implementations)? Yes, its often the case that the library functions are written in C and compiled into libraries. That makes them much faster than your Python code being executed within the Python interpreter. In addition the library code has typically been fine tuned and optimised over many releases so its about as fast as it can get. (Your Python code could be optimised a bit but it would still not be as fast as a C implementation.) Not all library modules are C based however so it doesn't always apply. But they are usually optimised and thoroughly debugged so it is still worth using them rather than building your own. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.flickr.com/photos/alangauldphotos From stefan_ml at behnel.de Fri Oct 24 14:01:51 2014 From: stefan_ml at behnel.de (Stefan Behnel) Date: Fri, 24 Oct 2014 14:01:51 +0200 Subject: [Tutor] Standard Library Performance (3.4.1) In-Reply-To: <m2dbmd$aph$1@ger.gmane.org> References: <5449AF4C.50601@riseup.net> <m2dbmd$aph$1@ger.gmane.org> Message-ID: <m2df3j$3o2$1@ger.gmane.org> Alan Gauld schrieb am 24.10.2014 um 13:03: > Not all library modules are C based however so it doesn't > always apply. But they are usually optimised and thoroughly > debugged so it is still worth using them rather than building > your own. It's worth stressing this point a bit more. Lots of people have been using this code, and some found bugs in it that are now fixed. This means that whatever implementation of similar functionality you can come up with yourself will most likely have more bugs and be less generally versatile. And even if the standard library code doesn't fit your needs, start by taking a deep look at what the Python Package Index (PyPI) offers instead of writing your own. Developer time is much better spent reusing other people's code and helping to squash the remaining bugs in it than having everyone write their own buggy code over and over. Stefan From crushed26 at gmail.com Fri Oct 24 14:03:41 2014 From: crushed26 at gmail.com (Bo Morris) Date: Fri, 24 Oct 2014 08:03:41 -0400 Subject: [Tutor] Code critique Message-ID: <CAKKCnffLcKGL6+krd2rJ+5i8c3rV-sn_WthCmW7Q6Umadhn6Ng@mail.gmail.com> Hello all, May I please get a little instructional criticism. The code below works. It logs into 9 different Linux computers, runs a couple commands, and then transfers a file back to the server. I want to become a better Python coder; therefore, I was hoping for some ways to make the below code better, more efficient, or if I am doing something incorrectly, a correct way of doing it. Thanks #!/usr/bin/python import paramiko l = ['ip-1', 'ip-2', 'ip-3', 'ip-4', 'ip-5', 'ip-6', 'ip-7', 'ip-8', 'ip-9'] def connect(ip): user = 'user' passwd = '' command = 'echo $HOSTNAME' s = paramiko.SSHClient() s.set_missing_host_key_policy(paramiko.AutoAddPolicy()) s.connect(ip,22,user,passwd,timeout=4) stdin, stdout, stderr = s.exec_command('echo $HOSTNAME') out = stdout.read() if '3102EHD-Lanka-1108' in out: s.exec_command('export DISPLAY=:0.0; cd /Downloads/Hourly/win.sh') sftp = s.open_sftp() sftp.get('/Downloads/Hourly/3102EHD-01108/3102EHD-01108.png', '/Downloads/Hourly/3102EHD-01108.png') sftp.close() print 'file recieved' elif '3102EHD-01109' in out: s.exec_command('export DISPLAY=:0.0; cd /Downloads/Hourly/win.sh') sftp = s.open_sftp() sftp.get('/Downloads/Hourly/3102DHD-01109/3102DHD-01109.png', '/Downloads/Hourly/3102DHD-01109.png') sftp.close() print 'file recieved' elif '3102EHD-MUTV-1082' in out: s.exec_command('export DISPLAY=:0.0; cd /Downloads/Hourly/win.sh') sftp = s.open_sftp() sftp.get('/Downloads/Hourly/3102EHD-01082/3102EHD-01082.png', '/Downloads/Hourly/3102EHD-01082.png') sftp.close() print 'file recieved' elif '3102DHD-MUTV-1033' in out: s.exec_command('export DISPLAY=:0.0; cd /Downloads/Hourly/win.sh') sftp = s.open_sftp() sftp.get('/Downloads/Hourly/3102DHD-01033/3102DHD-01033.png', '/Downloads/Hourly/3102DHD-01033.png') sftp.close() print 'file recieved' elif 'Encoder' in out: s.exec_command('export DISPLAY=:0.0; cd /Downloads/Hourly/win.sh') sftp = s.open_sftp() sftp.get('/Downloads/Hourly/3102EHD-01302/3102EHD-01302.png', '/Downloads/Hourly/3102EHD-01302.png') sftp.close() print 'file recieved' elif '3102DHD-01149' in out: s.exec_command('export DISPLAY=:0.0; cd /Downloads/Hourly/win.sh') sftp = s.open_sftp() sftp.get('/Downloads/Hourly/3102DHD-01149/3102DHD-01149.png', '/Downloads/Hourly/3102DHD-01149.png') sftp.close() print 'file recieved' elif '3102EHD-01125' in out: s.exec_command('export DISPLAY=:0.0; cd /Downloads/Hourly/win.sh') sftp = s.open_sftp() sftp.get('/Downloads/Hourly/3102EHD-01125/3102EHD-01125.png', '/Downloads/Hourly/3102EHD-01125.png') sftp.close() print 'file recieved' elif '3102DHD-01144' in out: s.exec_command('export DISPLAY=:0.0; cd /Downloads/Hourly/win.sh') sftp = s.open_sftp() sftp.get('/Downloads/Hourly/3102DHD-01144/3102DHD-01144.png', '/Downloads/Hourly/3102DHD-01144.png') sftp.close() print 'file recieved' elif '3102EHD-01105' in out: s.exec_command('export DISPLAY=:0.0; cd /Downloads/Hourly/win.sh') sftp = s.open_sftp() sftp.get('/Downloads/Hourly/3102EHD-01105/3102EHD-01105.png', '/Downloads/Hourly/3102EHD-01105.png') sftp.close() print 'file recieved' con = map(connect, l) -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/tutor/attachments/20141024/3b0e2ae7/attachment.html> From stefan_ml at behnel.de Fri Oct 24 15:03:49 2014 From: stefan_ml at behnel.de (Stefan Behnel) Date: Fri, 24 Oct 2014 15:03:49 +0200 Subject: [Tutor] Code critique In-Reply-To: <CAKKCnffLcKGL6+krd2rJ+5i8c3rV-sn_WthCmW7Q6Umadhn6Ng@mail.gmail.com> References: <CAKKCnffLcKGL6+krd2rJ+5i8c3rV-sn_WthCmW7Q6Umadhn6Ng@mail.gmail.com> Message-ID: <m2dino$dr$1@ger.gmane.org> Bo Morris schrieb am 24.10.2014 um 14:03: > May I please get a little instructional criticism. The code below works. It > logs into 9 different Linux computers, runs a couple commands, and then > transfers a file back to the server. I want to become a better Python > coder; therefore, I was hoping for some ways to make the below code better, > more efficient, or if I am doing something incorrectly, a correct way of > doing it. Thanks A quick comment, not related to coding style: take a look at fabric. It's made for doing these things. http://www.fabfile.org/ Regarding your program, instead of writing long sequences of repetitive if conditions, I would write one function for each of the different operations and store them in a dict, mapping each host name to a function (and multiple host names may map to the same function). Then, look up the host name in the dict and call the corresponding function to run the right operations on that host. Using functions will make it easier to factor out similar code. If you look at the different operations that you do on the different hosts, you will notice that most of them do the same thing, just with different files. Instead of duplicating your code for each host, extract the file names that each host needs and then pass it into the function that reads the file from the host. The function will then be the same for all hosts, only the input arguments change. Stefan From friedhelm.peters at gmx.net Fri Oct 24 16:31:02 2014 From: friedhelm.peters at gmx.net (Friedhelm Peters) Date: Fri, 24 Oct 2014 16:31:02 +0200 Subject: [Tutor] Problem by installing Python 3.4.2 64 Bit Message-ID: <!&!AAAAAAAAAAAYAAAAAAAAABCyioDND+tMrewuNGNEUL7CgAAAEAAAAKWLqIqVyCJHk+Hwn5KsAjwBAAAAAA==@gmx.net> Dear Mr. and Mrs., Sorry, I'm from Germany and my English isn't so good. By installing python 3.4.2 (64bit) I get the following error message: "There is a problem with this Windows Installer package. A program required for this install the complete could not be run. Contact your support personnel or package vendor." My system: Windows 7 Ultimate x64 SP1, Intel i7 3770, 16 GB RAM The Version 3.4.1150.1013 is installed Greetings Friedhelm Peters Bockholter Str. 29 45659 Recklinghausen Tel.: +49 (171) 823 899 1 Fax: +49 (2361) 905 326 Mail: <mailto:friedhelm.peters.1009 at epost.de> friedhelm.peters.1009 at epost.de <mailto:Friedhelm-peters at arcor.de> Friedhelm-peters at arcor.de -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/tutor/attachments/20141024/ebd91b18/attachment-0001.html> -------------- next part -------------- A non-text attachment was scrubbed... Name: image001.jpg Type: image/jpeg Size: 10569 bytes Desc: not available URL: <http://mail.python.org/pipermail/tutor/attachments/20141024/ebd91b18/attachment-0001.jpg> From crushed26 at gmail.com Fri Oct 24 20:50:09 2014 From: crushed26 at gmail.com (Bo Morris) Date: Fri, 24 Oct 2014 14:50:09 -0400 Subject: [Tutor] Code critique Message-ID: <CAKKCnfdvRu7hhUMGfyLF-EgW3LJVkWevt+N-uJWF8RWu_=D0dg@mail.gmail.com> "...Regarding your program, instead of writing long sequences of repetitive if conditions, I would write one function for each of the different operations and store them in a dict, mapping each host name to a function (and multiple host names may map to the same function). Then, look up the host name in the dict and call the corresponding function to run the right operations on that host..." Thank you Stefan for your input. Would please be willing to provide an example? -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/tutor/attachments/20141024/1647384b/attachment.html> From __peter__ at web.de Sat Oct 25 01:39:51 2014 From: __peter__ at web.de (Peter Otten) Date: Sat, 25 Oct 2014 01:39:51 +0200 Subject: [Tutor] Code critique References: <CAKKCnfdvRu7hhUMGfyLF-EgW3LJVkWevt+N-uJWF8RWu_=D0dg@mail.gmail.com> Message-ID: <m2eo0c$56k$1@ger.gmane.org> Bo Morris wrote: > "...Regarding your program, instead of writing long sequences of > repetitive if > conditions, I would write one function for each of the different > operations and store them in a dict, mapping each host name to a function > (and multiple host names may map to the same function). Then, look up the > host name in the dict and call the corresponding function to run the right > operations on that host..." > > Thank you Stefan for your input. Would please be willing to provide an > example? Start small. Look at this > if '3102EHD-Lanka-1108' in out: > s.exec_command('export DISPLAY=:0.0; cd /Downloads/Hourly/win.sh') > sftp = s.open_sftp() > sftp.get('/Downloads/Hourly/3102EHD-01108/3102EHD-01108.png', > '/Downloads/Hourly/3102EHD-01108.png') > sftp.close() > print 'file recieved' > elif '3102EHD-01109' in out: > s.exec_command('export DISPLAY=:0.0; cd /Downloads/Hourly/win.sh') > sftp = s.open_sftp() > sftp.get('/Downloads/Hourly/3102DHD-01109/3102DHD-01109.png', > '/Downloads/Hourly/3102DHD-01109.png') > sftp.close() > print 'file recieved' > and try to move the code in the if-suite into a function > if '3102EHD-Lanka-1108' in out: process() Then try to determine the argument(s) needed to make the same process() function work for both the if and the elif suite: > if '3102EHD-Lanka-1108' in out: process(?) > elif '3102EHD-01109' in out: process(?) Once that works see if you can preprocess `out` so that it only contains the hostname. Do you still need the if...elif...elif...? Come back for more hints if you think you do. Random remarks: > command = 'echo $HOSTNAME' The sooner you remove unused code, the better. > cd /Downloads/Hourly/win.sh That looks like you are cd-ing into a shell script. > con = map(connect, l) That should be a for loop; also, use a descriptive variable name instead of `l`. PS: a dict would come into play once you want to treat machines differently. Say, your computers are called one, two, three, four, ..., and you have a default procedure def process_default(hostname): ... and some special treatment for three, seven, and eight: def process_three(hostname): ... def process_seven_or_eight(hostname): ... Then you can build a lookup table lookup_process = { "three": process_three, "seven": process_seven_or_eight, "eight": process_seven_or_eight, } and find the applicable function with hostname = ... process = lookup_process.get(hostname, process_default) process(hostname) From hanzer at riseup.net Fri Oct 24 20:13:50 2014 From: hanzer at riseup.net (Adam Jensen) Date: Fri, 24 Oct 2014 14:13:50 -0400 Subject: [Tutor] Standard Library Performance (3.4.1) In-Reply-To: <m2df3j$3o2$1@ger.gmane.org> References: <5449AF4C.50601@riseup.net> <m2dbmd$aph$1@ger.gmane.org> <m2df3j$3o2$1@ger.gmane.org> Message-ID: <544A96DE.3020804@riseup.net> On 10/24/2014 08:01 AM, Stefan Behnel wrote: > Alan Gauld schrieb am 24.10.2014 um 13:03: >> Not all library modules are C based however so it doesn't >> always apply. But they are usually optimised and thoroughly >> debugged so it is still worth using them rather than building >> your own. > > It's worth stressing this point a bit more. Lots of people have been using > this code, and some found bugs in it that are now fixed. This means that > whatever implementation of similar functionality you can come up with > yourself will most likely have more bugs and be less generally versatile. > And even if the standard library code doesn't fit your needs, start by > taking a deep look at what the Python Package Index (PyPI) offers instead > of writing your own. If the standard library is thoughtfully designed, tested, tuned and released in a stable way then I agree, it's worth spending development time to search and explore the library for potentially useful components then examine the component's interface, decipher the documentation and tinker with the component to discover its characteristics. There is some cost in doing this (developer time and energy). Sometimes it seems like there will be less expense in just writing a few lines of pure python but a ~10x difference in performance settles that - the library route is probably worth the effort. > Developer time is much better spent reusing other people's code and helping > to squash the remaining bugs in it than having everyone write their own > buggy code over and over. I haven't studied the PyPI engineering process yet but my limited experience with just installing third party packages (e.g., pip install scipy) makes me a bit wary about naively using these packages as foundational components. BTW - I did manage to get all of the scipy dependencies built and installed but damn, that was a grueling process. I'm surprised it wasn't automated and thoroughly shocked by the number of compilation warnings. (I once heard someone describe the current state of software engineering as being like "building a skyscraper out of bananas"). From japhy at pearachute.com Fri Oct 24 21:32:56 2014 From: japhy at pearachute.com (Japhy Bartlett) Date: Fri, 24 Oct 2014 14:32:56 -0500 Subject: [Tutor] Code critique In-Reply-To: <CAKKCnfdvRu7hhUMGfyLF-EgW3LJVkWevt+N-uJWF8RWu_=D0dg@mail.gmail.com> References: <CAKKCnfdvRu7hhUMGfyLF-EgW3LJVkWevt+N-uJWF8RWu_=D0dg@mail.gmail.com> Message-ID: <CANTsVHKn1yABU_EdKpf=Z6zvM_vd_eAVPrrq78=e+xMfinyHkQ@mail.gmail.com> out = stdout.read() if '3102EHD-Lanka-1108' in out: s.exec_command('export DISPLAY=:0.0; cd /Downloads/Hourly/win.sh') sftp = s.open_sftp() sftp.get('/Downloads/Hourly/3102EHD-01108/3102EHD-01108.png', '/Downloads/Hourly/3102EHD-01108.png') sftp.close() print 'file recieved' elif '3102EHD-01109' in out: s.exec_command('export DISPLAY=:0.0; cd /Downloads/Hourly/win.sh') sftp = s.open_sftp() sftp.get('/Downloads/Hourly/3102DHD-01109/3102DHD-01109.png', '/Downloads/Hourly/3102DHD-01109.png') sftp.close() print 'file recieved' ... ---- could be something like: ---- params = { '3102EHD-Lanka-1108': '3102EHD-01108', '3102EHD-01109': '3102DHD-0119', # ... etc, for each of those elif choices } # one function with a variable for the parts that change function dosomething(identifier): s.exec_command('export DISPLAY=:0.0; cd /Downloads/Hourly/win.sh') sftp = s.open_sftp() sftp.get('/Downloads/Hourly/{}/{}.png'.format(identifier,identifier), '/Downloads/Hourly/{}.png'.format(identifier)) sftp.close() print 'file recieved' # do this as usual out = stdout.read() # then call the function with the variable dosomething(params[out]) ---- with a bit of work, it seems like that could be simplified even more, but the general idea is that the elif blocks with lots of boilerplate repeated over and over is generally a sign of bad/lazy code. On Fri, Oct 24, 2014 at 1:50 PM, Bo Morris <crushed26 at gmail.com> wrote: > "...Regarding your program, instead of writing long sequences of > repetitive if > conditions, I would write one function for each of the different operations > and store them in a dict, mapping each host name to a function (and > multiple host names may map to the same function). Then, look up the host > name in the dict and call the corresponding function to run the right > operations on that host..." > > Thank you Stefan for your input. Would please be willing to provide an > example? > > _______________________________________________ > Tutor maillist - Tutor at python.org > To unsubscribe or change subscription options: > https://mail.python.org/mailman/listinfo/tutor > > -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/tutor/attachments/20141024/26062e70/attachment-0001.html> From cs at zip.com.au Sat Oct 25 02:26:46 2014 From: cs at zip.com.au (Cameron Simpson) Date: Sat, 25 Oct 2014 11:26:46 +1100 Subject: [Tutor] Is there a convenient table of Python 3.4 exceptions? In-Reply-To: <85bnp23wbp.fsf@benfinney.id.au> References: <85bnp23wbp.fsf@benfinney.id.au> Message-ID: <20141025002646.GA35746@cskk.homeip.net> On 24Oct2014 15:50, Ben Finney <ben+python at benfinney.id.au> wrote: >Is this a hurdle for newcomers? Yes, and that's why the standard library >API documentation is not a tutorial. We have separate tutorial >documentation for that. > >It's not a problem with the documentation of ?signal.getsignal? that it >doesn't have an exhaustive list of all exception classes that might be >raised. This is Python, not Java; [...] Further to this, in particular WRT to Java and other strongly typed static languages, in Java one must define what exceptions a function may throw or the code won't even compile. This is just great in one sense: it lets you write interface definitions with complete confidence about the types of failure the caller must accomodate. However, this only works rigorously because the same information is available on everything the function itself calls, courtesy of the same requirements and the staticly typed nature of the language. With Python, this goes out the window. Because _names_ are untyped (they may be bound to any object) you can't make the same guarentees. Newcomers will find this disorienting; I did. Because of this, not only can't one document everything a function might throw, one can return to writing relevant documents that says what a function does, what it expected to receive, and specific exceptions representing important failure modes special to the operation. Pass it something else, and undocumented badness may ensue (GIGO). One thing the newcomer should keep in mind is that with library wrapper functions (such as almost everything in the "os" module), the python documentation says what the function does and what underlying library function is involved; detailed failure modes often are to be found in the external library documentation. >exceptions are used prolifically as a >flow control method, for code to let its call stack know something >unusual occurred. This, again, is surprising to newcomers, who will often think of exceptions only in "something went wrong" scenarios. Raising an exception, particularly StopIteration from iterators, frees one from choosing a sentinel value for every single function to indicate some value not in the legitimate domain. Consider an "iteration" in C where the values from from an external call. It might naturally be written like this: while ( (value=function(....)) != -1) { loop body here... } Here, "-1" is a sentinel value returned by function() when there are no more values for the iteration. It needs to be some naturally "invalid" return value, and in C is often -1 for intergers and NULL for pointers including strings. Regardless, a special value needs to be chosen and exposed in the loop condition. Comared with almost every Python iteration loop: for value in generator_function(): loop body here... Here the generator (usually automaticly) raises StopIteration, and the Python "for" loop implementation automatically catches that exception and exits the loop. No sentinel values required. Again with sentinel values, for non-iteration: Compare with, say, POSIX system calls in C. They mostly return -1 on error, and every piece of code calling them has to look like this: if (unlink(some_pathname) == -1) { consult errno, recover or complain, maybe return or abort, etc ... else: ... continue after successful operation... With exceptions one can often write clumps of operations all of which success without explicit checking verbiage: os.unlink(some_pathname) os.rename(this, that) because a failed operation will raise an exception. Cheers, Cameron Simpson <cs at zip.com.au> Mike was a sportbike rider, He'd say "El grande numero one!" With a 'No Fear' sticker on his zx11, he'd lightem'up just for fun... - Mike Hardcore DoD#5010 <moike at netcom.com> From crushed26 at gmail.com Sat Oct 25 03:26:47 2014 From: crushed26 at gmail.com (Bo Morris) Date: Fri, 24 Oct 2014 21:26:47 -0400 Subject: [Tutor] Code critique Message-ID: <CAKKCnffLx37PQCFhb_So7wFL3oAiR7b3HdW18Cynffu1B3FLBQ@mail.gmail.com> Thank you all for the helpful criticism. I wish I was able to catch on to what you are suggesting more quickly. Based on your recommendations, I have come up with the following so far, however I just dont see it as easily as I did while using the if/elif statements. This is what I have so far. I can not figure out how to iterate through the dictionary inserting each value where "png_file" should be and execute the code for each ip address in the list. I was able to do it using the if/elif statements, but I am afraid I am lost trying to do it another way. ipList = ['ip-1', 'ip-2', 'ip-3', 'ip-4', 'ip-5', 'ip-6', 'ip-7', 'ip-8', 'ip-9'] host = {'3102EHD-01108':'3102EHD-01108.png', '3102EHD-01109':'3102DHD-01109.png', '3102EHD-MUTV-1082':'3102EHD-01082.png', '3102DHD-01033':'3102DHD-MUTV-1033.png', 'Encoder':'3102EHD-01302.png', '3102DHD-01149':'3102DHD-01149.png', '3102EHD-01125':'3102EHD-01125.png', '3102DHD-01144':'3102DHD-01144.png', '3102EHD-01105':'3102EHD-01105.png'} # iterate through the dictionary inserting the png file def get_png_file(?): process = get_png_file.get(hostname, png_file) s.exec_command('export DISPLAY=:0.0; /Downloads/Hourly/win.sh') sftp = s.open_sftp() sftp.get('/Downloads/Hourly/'png_file,'/Downloads/Hourly/'png_file) sftp.close() print 'file recieved' user = 'user' passwd = 'password' s = paramiko.SSHClient() s.set_missing_host_key_policy(paramiko.AutoAddPolicy()) # iterate through the list and do the below for each for ip in ipList: s.connect(ip,22,user,passwd,timeout=4) ####since I have all the hostnames in the dic, and I am removing ####the if/elif statments, do I need the below command 'echo $HOSTNAME'? stdin, stdout, stderr = s.exec_command('echo $HOSTNAME') out = stdout.read() get_png_file(?) **original code below** #############connect to each machine and retrieve the image############# l = ['ip-1', 'ip-2', 'ip-3', 'ip-4', 'ip-5', 'ip-6', 'ip-7', 'ip-8', 'ip-9'] def connect(ip): user = 'user' passwd = 'password' command = 'echo $HOSTNAME' s = paramiko.SSHClient() s.set_missing_host_key_policy(paramiko.AutoAddPolicy()) s.connect(ip,22,user,passwd,timeout=4) stdin, stdout, stderr = s.exec_command('echo $HOSTNAME') out = stdout.read() if '3102EHD-Lanka-1108' in out: s.exec_command('export DISPLAY=:0.0; /Downloads/Hourly/win.sh') sftp = s.open_sftp() sftp.get('/Downloads/Hourly/3102EHD-01108/3102EHD-01108.png', '/Downloads/Hourly/3102EHD-01108.png') sftp.close() elif '3102EHD-01109' in out: s.exec_command('export DISPLAY=:0.0; /Downloads/Hourly/win.sh') sftp = s.open_sftp() sftp.get('/Downloads/Hourly/3102DHD-01109/3102DHD-01109.png', '/Downloads/Hourly/3102DHD-01109.png') sftp.close() elif '3102EHD-MUTV-1082' in out: s.exec_command('export DISPLAY=:0.0; /Downloads/Hourly/win.sh') sftp = s.open_sftp() sftp.get('/Downloads/Hourly/3102EHD-01082/3102EHD-01082.png', '/Downloads/Hourly/3102EHD-01082.png') sftp.close() elif '3102DHD-MUTV-1033' in out: s.exec_command('export DISPLAY=:0.0; /Downloads/Hourly/win.sh') sftp = s.open_sftp() sftp.get('/Downloads/Hourly/3102DHD-01033/3102DHD-01033.png', '/Downloads/Hourly/3102DHD-01033.png') sftp.close() elif 'Encoder' in out: s.exec_command('export DISPLAY=:0.0; /Downloads/Hourly/win.sh') sftp = s.open_sftp() sftp.get('/Downloads/Hourly/3102EHD-01302/3102EHD-01302.png', '/Downloads/Hourly/3102EHD-01302.png') sftp.close() elif '3102DHD-01149' in out: s.exec_command('export DISPLAY=:0.0; /Downloads/Hourly/win.sh') sftp = s.open_sftp() sftp.get('/Downloads/Hourly/3102DHD-01149/3102DHD-01149.png', '/Downloads/Hourly/3102DHD-01149.png') sftp.close() elif '3102EHD-01125' in out: s.exec_command('export DISPLAY=:0.0; /Downloads/Hourly/win.sh') sftp = s.open_sftp() sftp.get('/Downloads/Hourly/3102EHD-01125/3102EHD-01125.png', '/Downloads/Hourly/3102EHD-01125.png') sftp.close() elif '3102DHD-01144' in out: s.exec_command('export DISPLAY=:0.0; /Downloads/Hourly/win.sh') sftp = s.open_sftp() sftp.get('/Downloads/Hourly/3102DHD-01144/3102DHD-01144.png', '/Downloads/Hourly/3102DHD-01144.png') sftp.close() elif '3102EHD-01105' in out: s.exec_command('export DISPLAY=:0.0; /Downloads/Hourly/win.sh') sftp = s.open_sftp() sftp.get('/Downloads/Hourly/3102EHD-01105/3102EHD-01105.png', '/Downloads/Hourly/3102EHD-01105.png') sftp.close() con = map(connect, l) -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/tutor/attachments/20141024/8e4cad86/attachment.html> From robertvstepp at gmail.com Sat Oct 25 03:55:39 2014 From: robertvstepp at gmail.com (boB Stepp) Date: Fri, 24 Oct 2014 20:55:39 -0500 Subject: [Tutor] Is there a convenient table of Python 3.4 exceptions? In-Reply-To: <CAGZAPF79FThguBsStQVAoCbgM4Ffbw+92ZMVMXPoCgEas=_Vkw@mail.gmail.com> References: <CANDiX9Kq997q6dawd2OpU0Cpk=htd0r8_crfqo5a_SQd334B8w@mail.gmail.com> <CAGZAPF79FThguBsStQVAoCbgM4Ffbw+92ZMVMXPoCgEas=_Vkw@mail.gmail.com> Message-ID: <CANDiX9+665msDOY7++j4LQLkvHqnkU_B-sQecz_+zyUpQs7t4w@mail.gmail.com> On Thu, Oct 23, 2014 at 10:57 PM, Danny Yoo <dyoo at hashcollision.org> wrote: > > > I have so far been unable to find a list of these class/subclass names. Of > > course I can force an error to occur in the interpreter and see what comes > > up for each type of error I wish to catch. Is there such a table or list? > > > > Hi Bob, > > You can find the ones used in the Standard Library here: > > https://docs.python.org/3.4/library/exceptions.html > Thanks, Danny. That is exactly what I was looking for. Not being familiar with exception-related vocabulary yet, I was not typing in the search strings to get me what I wanted. I wound up in "The Python Language Reference", which is where the quote I gave came from. Apparently my answers were in the Standard Library Reference. Sigh. -- boB From robertvstepp at gmail.com Sat Oct 25 04:14:03 2014 From: robertvstepp at gmail.com (boB Stepp) Date: Fri, 24 Oct 2014 21:14:03 -0500 Subject: [Tutor] Is there a convenient table of Python 3.4 exceptions? In-Reply-To: <85fvee3yft.fsf@benfinney.id.au> References: <CANDiX9Kq997q6dawd2OpU0Cpk=htd0r8_crfqo5a_SQd334B8w@mail.gmail.com> <85fvee3yft.fsf@benfinney.id.au> Message-ID: <CANDiX9LdYKoNtV-uGXDqqYidfhHevZ3=Gav=QQ+XEcOXiXrwAw@mail.gmail.com> On Thu, Oct 23, 2014 at 11:04 PM, Ben Finney <ben+python at benfinney.id.au> wrote: > boB Stepp <robertvstepp at gmail.com> writes: > [...] > >> I have so far been unable to find a list of these class/subclass >> names. > > The standard library documentation's chapter on exceptions > <URL:https://docs.python.org/3/library/exceptions.html> shows > <URL:https://docs.python.org/3/library/exceptions.html#exception-hierarchy>. > As with Danny, thanks Ben. This is what I was trying to find [...] > > More importantly, though, you should consider *why* you're attempting to > catch a lot of exception classes. Will you be meaningfully handling > every one of those situations? That's rather doubtful. > > Instead, you should consider which exceptional states your code can > meaningfully handle, discover what excpetion classes are raised for > those few situations, and catch *only* those classes. > In the programs I have been dabbling in at work, I am often "surprised" by the situations my users stumble into that I did not have sufficient imagination to consider up front. And in line with your statements I have often wondered if I had done enough error checking, or whether in some instances I was doing too much. Trying to imagine what the user might do is often a difficult exercise, especially as my programs have been becoming more lengthy and complex. I suppose this gets easier with more knowledge and experience? > Any other exception that gets raised isn't something you can do anything > useful with, so it should propagate back up to higher levels, either to > be handled or to exit with a useful error message. > As a user of software I have often wished that the error messages generated were understandable and help me to avoid the condition which triggered that message in the future. -- boB From ben+python at benfinney.id.au Sat Oct 25 05:42:32 2014 From: ben+python at benfinney.id.au (Ben Finney) Date: Sat, 25 Oct 2014 14:42:32 +1100 Subject: [Tutor] How to anticipate and deal with unexpected exceptions (was: Is there a convenient table of Python 3.4 exceptions?) References: <CANDiX9Kq997q6dawd2OpU0Cpk=htd0r8_crfqo5a_SQd334B8w@mail.gmail.com> <85fvee3yft.fsf@benfinney.id.au> <CANDiX9LdYKoNtV-uGXDqqYidfhHevZ3=Gav=QQ+XEcOXiXrwAw@mail.gmail.com> Message-ID: <857fzo4xxz.fsf_-_@benfinney.id.au> boB Stepp <robertvstepp at gmail.com> writes: > In the programs I have been dabbling in at work, I am often > "surprised" by the situations my users stumble into that I did not > have sufficient imagination to consider up front. This is a good thing to focus on. Improving the robustness of your code is a matter of experience, but you can learn from the experience (i.e. mistakes) of the several decades of programmers who have come before you, and the approaches that have arisen in response to those mistakes. > And in line with your statements I have often wondered if I had done > enough error checking, or whether in some instances I was doing too > much. It's interesting that you raise that dilemma. The Python style definitely leans away from LBYL (Look Before You Leap, i.e. lots of checking for errors before performing the task at hand), and much more toward EAFP (it is Easier to Ask Forgiveness than Permission). EAFP means that if you're going to perform a task in your code, you make a clear clean API, document that API, and then trust that the other programmers using that API will get the preconditions and parameters correct. If they don't, allow errors to propagate all the way back to that code and let the user of your API handle the errors. So, when you're using the Python standard library, you're expected not to know every possible error that can occur; but rather, to read and understand the API you're writing to, and set appropriate preconditions and parameters. If you don't, you can rely on getting an exception object that tells you about the problem. > Trying to imagine what the user might do is often a difficult > exercise, especially as my programs have been becoming more lengthy > and complex. I suppose this gets easier with more knowledge and > experience? It also gets easier if you record every bug-fix as a unit test, and automate a full run of all the existing unit tests after changing your program behaviour. That way, you: * Don't have to keep re-discovering bugs you thought you'd fixed. * Continually ratchet up the robustness of your code. * Never get to the point where you feel like you can't trust the old code. * Begin to internalise the habit of anticipating what possible conditions you should test for, and write those tests *first*. Once you're comfortable with always writing a battery of unit tests for any change you make, and accumulating a suite of unit tests that run before every release (and ideally much more often than that), and never release any code that breaks any of the tests, you will gain a confidence in making changes that is priceless. -- \ ?Very few things happen at the right time, and the rest do not | `\ happen at all. The conscientious historian will correct these | _o__) defects.? ?Mark Twain, _A Horse's Tale_ | Ben Finney From steve at pearwood.info Sat Oct 25 07:07:37 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Sat, 25 Oct 2014 16:07:37 +1100 Subject: [Tutor] Problem by installing Python 3.4.2 64 Bit In-Reply-To: <!&!AAAAAAAAAAAYAAAAAAAAABCyioDND+tMrewuNGNEUL7CgAAAEAAAAKWLqIqVyCJHk+Hwn5KsAjwBAAAAAA==@gmx.net> References: <!&!AAAAAAAAAAAYAAAAAAAAABCyioDND+tMrewuNGNEUL7CgAAAEAAAAKWLqIqVyCJHk+Hwn5KsAjwBAAAAAA==@gmx.net> Message-ID: <20141025050736.GF20010@ando.pearwood.info> On Fri, Oct 24, 2014 at 04:31:02PM +0200, Friedhelm Peters wrote: > > Sorry, I'm from Germany and my English isn't so good. By installing python > 3.4.2 (64bit) I get the following error message: > > "There is a problem with this Windows Installer package. A program required > for this install the complete could not be run. Contact your support > personnel or package vendor." Which installer package are you using? -- Steven From __peter__ at web.de Sat Oct 25 09:38:59 2014 From: __peter__ at web.de (Peter Otten) Date: Sat, 25 Oct 2014 09:38:59 +0200 Subject: [Tutor] Code critique References: <CAKKCnffLx37PQCFhb_So7wFL3oAiR7b3HdW18Cynffu1B3FLBQ@mail.gmail.com> Message-ID: <m2fk2k$8gj$1@ger.gmane.org> Bo Morris wrote: > Thank you all for the helpful criticism. I wish I was able to catch on to > what you are suggesting more quickly. > > Based on your recommendations, I have come up with the following so far, > however I just dont see it as easily as I did while using the if/elif > statements. > > This is what I have so far. I can not figure out how to iterate through > the dictionary inserting each value where "png_file" should be and execute > the code for each ip address in the list. I was able to do it using the > if/elif statements, but I am afraid I am lost trying to do it another way. > > ipList = ['ip-1', 'ip-2', 'ip-3', 'ip-4', 'ip-5', 'ip-6', 'ip-7', 'ip-8', > 'ip-9'] > > host = {'3102EHD-01108':'3102EHD-01108.png', > '3102EHD-01109':'3102DHD-01109.png', > '3102EHD-MUTV-1082':'3102EHD-01082.png', > '3102DHD-01033':'3102DHD-MUTV-1033.png', > 'Encoder':'3102EHD-01302.png', > '3102DHD-01149':'3102DHD-01149.png', > '3102EHD-01125':'3102EHD-01125.png', > '3102DHD-01144':'3102DHD-01144.png', > '3102EHD-01105':'3102EHD-01105.png'} > > # iterate through the dictionary inserting the png file > def get_png_file(?): > process = get_png_file.get(hostname, png_file) > s.exec_command('export DISPLAY=:0.0; /Downloads/Hourly/win.sh') > sftp = s.open_sftp() > sftp.get('/Downloads/Hourly/'png_file,'/Downloads/Hourly/'png_file) > sftp.close() > print 'file recieved' > > user = 'user' > passwd = 'password' > s = paramiko.SSHClient() > s.set_missing_host_key_policy(paramiko.AutoAddPolicy()) > > # iterate through the list and do the below for each > for ip in ipList: > s.connect(ip,22,user,passwd,timeout=4) > ####since I have all the hostnames in the dic, and I am removing > ####the if/elif statments, do I need the below command 'echo $HOSTNAME'? > stdin, stdout, stderr = s.exec_command('echo $HOSTNAME') > out = stdout.read() > get_png_file(?) > As hinted in my previous post I don't think you need a dict here. Once you have the hostname you can build the filename from it with hostname + ".png". Here's what I had in mind, unfortunately totally untested: #!/usr/bin/python import os import paramiko ip_list = [ 'ip-1', 'ip-2', 'ip-3', 'ip-4', 'ip-5', 'ip-6', 'ip-7', 'ip-8', 'ip-9'] def connect(ip): user = 'user' passwd = 'password' s = paramiko.SSHClient() s.set_missing_host_key_policy(paramiko.AutoAddPolicy()) s.connect(ip, 22, user, passwd, timeout=4) stdin, stdout, stderr = s.exec_command('echo $HOSTNAME') hostname = stdout.read().strip() filename = hostname + ".png" png_source = os.path.join("/Downloads/Hourly", filename) png_dest = os.path.join("/Downloads/Hourly", hostname, filename) s.exec_command('export DISPLAY=:0.0; /Downloads/Hourly/win.sh') sftp = s.open_sftp() sftp.get(png_dest, png_source) sftp.close() print "file", png_dest, "received" for ip in ip_list: connect(ip) From alan.gauld at btinternet.com Sat Oct 25 10:44:00 2014 From: alan.gauld at btinternet.com (Alan Gauld) Date: Sat, 25 Oct 2014 09:44:00 +0100 Subject: [Tutor] Is there a convenient table of Python 3.4 exceptions? In-Reply-To: <CANDiX9LdYKoNtV-uGXDqqYidfhHevZ3=Gav=QQ+XEcOXiXrwAw@mail.gmail.com> References: <CANDiX9Kq997q6dawd2OpU0Cpk=htd0r8_crfqo5a_SQd334B8w@mail.gmail.com> <85fvee3yft.fsf@benfinney.id.au> <CANDiX9LdYKoNtV-uGXDqqYidfhHevZ3=Gav=QQ+XEcOXiXrwAw@mail.gmail.com> Message-ID: <m2fnsg$u0i$1@ger.gmane.org> On 25/10/14 03:14, boB Stepp wrote: > In the programs I have been dabbling in at work, I am often > "surprised" by the situations my users stumble into that I did not > have sufficient imagination to consider up front. That's always a problem. And just when you think you've seen everything the users will astound you by thinking up another way to screw your code. That's one reason for having specialist system test teams completely divorced from the developers. Developers know how its supposed to work so its very difficult for a developer to break their own code. System testers exist to break code, they will not be happy until they have found a way to make it crash or freeze. That's a good thing. > As a user of software I have often wished that the error messages > generated were understandable and help me to avoid the condition which > triggered that message in the future. Some people like to raise Python error messages to user level but that can be dangerous. I've had users turn off their PC and call the help desk and wait several hours for help because of a fairly innocuous exception message that spooked them. That cost the business a lot of down time. So translating any uncaught errors into logged messages and displaying something more human friendly on the screen is a good thing IMHO. But that doesn't mean you need to predict every possible individual error or handle it, that's just not possible, especially in a dynamic language like Python. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.flickr.com/photos/alangauldphotos From crk at godblessthe.us Sat Oct 25 05:17:59 2014 From: crk at godblessthe.us (Clayton Kirkwood) Date: Fri, 24 Oct 2014 20:17:59 -0700 Subject: [Tutor] solution for for loop? Message-ID: <000001cff002$4b23ec40$e16bc4c0$@us> description_string=code_string='' description = code = 'a' for (description_position, code_position) in (description, code): print(description_position,code_position) I have tried variations on this for statement, and it doesn't work:<))) Both description and code have the same size array. I was hoping that some derivative of this for would bring in a new description_position value, and code_position value. Amongst various trials I have tried dp in d && cp in c; dp, cp in d,c. etc. This is the error report: Traceback (most recent call last): File "C:/Users/Dad/python/stock tracker/raw yahoo scraper codes.py", line 80, in <module> for (description_position, code_position) in (description, code): ValueError: too many values to unpack (expected 2) Is there something like what I want? Thanks, CLayton -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/tutor/attachments/20141024/d5317b94/attachment.html> From __peter__ at web.de Sat Oct 25 13:41:18 2014 From: __peter__ at web.de (Peter Otten) Date: Sat, 25 Oct 2014 13:41:18 +0200 Subject: [Tutor] solution for for loop? References: <000001cff002$4b23ec40$e16bc4c0$@us> Message-ID: <m2g28u$g3k$1@ger.gmane.org> Clayton Kirkwood wrote: > description_string=code_string='' > > description = code = 'a' > > for (description_position, code_position) in (description, code): > > print(description_position,code_position) > I have tried variations on this for statement, and it doesn't work:<))) > Both description and code have the same size array. I was hoping that some > derivative of this for would bring in a new description_position value, > and code_position value. You want zip(): >>> colors = [ "red", "green", "yellow"] >>> fruit_list = ["cherry", "apple", "banana"] >>> for color, fruit in zip(colors, fruit_list): ... print(color, fruit) ... red cherry green apple yellow banana From alan.gauld at btinternet.com Sat Oct 25 14:17:46 2014 From: alan.gauld at btinternet.com (Alan Gauld) Date: Sat, 25 Oct 2014 13:17:46 +0100 Subject: [Tutor] solution for for loop? In-Reply-To: <000001cff002$4b23ec40$e16bc4c0$@us> References: <000001cff002$4b23ec40$e16bc4c0$@us> Message-ID: <m2g4da$iih$1@ger.gmane.org> On 25/10/14 04:17, Clayton Kirkwood wrote: > description_string=code_string='' > description = code = ?a? > > for (description_position, code_position) in (description, code): > print(description_position,code_position) > > I have tried variations on this for statement, and it doesn?t work:<))) No surprise there, it doesn't make sense. > Both description and code have the same size array. There are no arrays in sight. Both names refer to the same object: the letter 'a'. You then put the names in a tuple which is what I assume you mean? . But tuples are very different to arrays. > I was hoping that some derivative of this for > would bring in a new description_position > value, and code_position value. From where? You set them up to point to a fixed string. Where do you suppose Python would find any other values? > Amongst various trials I have tried dp in d && cp in c; dp, cp in d,c. etc. > > This is the error report: > Traceback (most recent call last): > File "C:/Users/Dad/python/stock tracker/raw yahoo scraper codes.py", > line 80, in <module> > for (description_position, code_position) in (description, code): > ValueError: too many values to unpack (expected 2) Read the error description then look at your for line closely. for (description_position, code_position) in (description, code): That reads: for aTupleOfNames in aTupleOfValues: It's trying to iterate over aTupleofValues and then unpack the first value into the the two names. You need a collection as the for loop target to give the for something to iterate over. It only needs to contain your tuple (a single element collection) but it needs to have something to iterate over. An example might help: Using shorter names and literals for brevity: This is your code: >>> for x,y in (1,2): print('ok') which gives an error because it picks out the first element of the (1,2) tuple which is 1, and tries to unpack that into x,y - which it can't. That's what is happening in your case too. Now look at: >>> for x,y in [(1,2)]: print('ok') This time the loop picks out the (1,2) tuple as the first (and only) element of the target list and unpacks to x and y. So we get ok printed in this case. > Is there something like what I want? I don't know because I don't really know what you were trying to do. Your code as it stands could just have been written as description_position, code_position = description, code But I don't think that really is what you were tying to do. HTH, -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.flickr.com/photos/alangauldphotos From mmeanwell at gmail.com Sat Oct 25 18:08:36 2014 From: mmeanwell at gmail.com (Mark Meanwell) Date: Sat, 25 Oct 2014 11:08:36 -0500 Subject: [Tutor] Help with running an API Message-ID: <CACKS5hHi-KnhreyBbtU_rL8mEumtg4+3xkDn40m6tV0q9Ese0w@mail.gmail.com> Hi Folks - new to python and trying to run an API. Running version 2.7.3. on Windows 7 machine. Here is the scenario for the given API (FRED API in this case): easy_install Fred from C:\ - this installs to C:\site packages then I fire up the python shell and run file created for fred api: from fredapi import Fred fred = Fred(api_key='my api key') data = fred.get_series('SP500') keep getting below error message: ImportError: cannot import name Fred thanks for any suggestions! -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/tutor/attachments/20141025/cb181d3b/attachment.html> From alan.gauld at btinternet.com Sat Oct 25 20:27:09 2014 From: alan.gauld at btinternet.com (Alan Gauld) Date: Sat, 25 Oct 2014 19:27:09 +0100 Subject: [Tutor] Help with running an API In-Reply-To: <CACKS5hHi-KnhreyBbtU_rL8mEumtg4+3xkDn40m6tV0q9Ese0w@mail.gmail.com> References: <CACKS5hHi-KnhreyBbtU_rL8mEumtg4+3xkDn40m6tV0q9Ese0w@mail.gmail.com> Message-ID: <m2gq1t$ks6$1@ger.gmane.org> On 25/10/14 17:08, Mark Meanwell wrote: > Hi Folks - new to python and trying to run an API. Running version > 2.7.3. on Windows 7 machine. > > Here is the scenario for the given API (FRED API in this case): > > easy_install Fred from C:\ - this installs to C:\site packages > > then I fire up the python shell and run file created for fred api: I'm not sure what you mean by this bit? Do you mean you run the file using python somefile.py Or do you start the interpreter to get the >>> prompt and then somehow execute the file? (If so how do you run it?) Or are you using some kind of IDE such as IDLE? In which case how do you run the file? > from fredapi import Fred > fred = Fred(api_key='my api key') > data = fred.get_series('SP500') > > keep getting below error message: > ImportError: cannot import name Fred Taking it back to basics start the Python interpreter and get a >>> prompt. Then type >>> import fredapi Does that work without errors? Also check that your Python version is compatible with your module version. Third party APIs are often version specific. Finally, since Fred is not part of the standard library, you might be better asking on a Fred specific forum, if one exists. This list is mainly for the language and standard libraries. Anything beyond that will have less chance of a good answer. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.flickr.com/photos/alangauldphotos From joel.goldstick at gmail.com Sat Oct 25 20:52:45 2014 From: joel.goldstick at gmail.com (Joel Goldstick) Date: Sat, 25 Oct 2014 14:52:45 -0400 Subject: [Tutor] Help with running an API In-Reply-To: <CACKS5hHi-KnhreyBbtU_rL8mEumtg4+3xkDn40m6tV0q9Ese0w@mail.gmail.com> References: <CACKS5hHi-KnhreyBbtU_rL8mEumtg4+3xkDn40m6tV0q9Ese0w@mail.gmail.com> Message-ID: <CAPM-O+yfXkHmjNX5huNic_q2aONmLneFhqgkHW-Y945pJAsqrw@mail.gmail.com> On Sat, Oct 25, 2014 at 12:08 PM, Mark Meanwell <mmeanwell at gmail.com> wrote: > Hi Folks - new to python and trying to run an API. Running version 2.7.3. on > Windows 7 machine. > > Here is the scenario for the given API (FRED API in this case): > > easy_install Fred from C:\ - this installs to C:\site packages > > then I fire up the python shell and run file created for fred api: > > from fredapi import Fred > fred = Fred(api_key='my api key') > data = fred.get_series('SP500') > > keep getting below error message: > ImportError: cannot import name Fred > > thanks for any suggestions! I'm not familiar with FRED, but a quick look on stackoverflow used the lowercase name to import > > > _______________________________________________ > Tutor maillist - Tutor at python.org > To unsubscribe or change subscription options: > https://mail.python.org/mailman/listinfo/tutor > -- Joel Goldstick http://joelgoldstick.com From crushed26 at gmail.com Sat Oct 25 21:09:39 2014 From: crushed26 at gmail.com (Crush) Date: Sat, 25 Oct 2014 15:09:39 -0400 Subject: [Tutor] Code critique In-Reply-To: <mailman.19.1414231202.5020.tutor@python.org> References: <mailman.19.1414231202.5020.tutor@python.org> Message-ID: <02EF603B-51FE-4F7B-8353-11473053E25E@gmail.com> Thank you Peter for your example. I have the code working now and will post soon for eveyones benefit. Thank you all who took the time to help. Bo From crk at godblessthe.us Sun Oct 26 00:40:54 2014 From: crk at godblessthe.us (Clayton Kirkwood) Date: Sat, 25 Oct 2014 15:40:54 -0700 Subject: [Tutor] solution for for loop? In-Reply-To: <m2g28u$g3k$1@ger.gmane.org> References: <000001cff002$4b23ec40$e16bc4c0$@us> <m2g28u$g3k$1@ger.gmane.org> Message-ID: <015c01cff0a4$bd240170$376c0450$@us> Ding, ding, ding. Winner, winner, winner. Peter wins the prize, suggesting the use of zip(). This allowed me to pull two different strings from two different lists at one time in a for loop. for description_position, code_position in zip(description, code): Yes!! Thank you Peter Clayton !-----Original Message----- !From: Tutor [mailto:tutor-bounces+crk=godblessthe.us at python.org] On !Behalf Of Peter Otten !Sent: Saturday, October 25, 2014 4:41 AM !To: tutor at python.org !Subject: Re: [Tutor] solution for for loop? ! !Clayton Kirkwood wrote: ! !> description_string=code_string='' !> !> description = code = 'a' !> !> for (description_position, code_position) in (description, code): !> !> print(description_position,code_position) ! !> I have tried variations on this for statement, and it doesn't !> work:<))) Both description and code have the same size array. I was !> hoping that some derivative of this for would bring in a new !> description_position value, and code_position value. ! !You want zip(): ! !>>> colors = [ "red", "green", "yellow"] fruit_list = ["cherry", !>>> "apple", "banana"] for color, fruit in zip(colors, fruit_list): !... print(color, fruit) !... !red cherry !green apple !yellow banana ! ! !_______________________________________________ !Tutor maillist - Tutor at python.org !To unsubscribe or change subscription options: !https://mail.python.org/mailman/listinfo/tutor From anish.tambe.lists at gmail.com Sun Oct 26 06:31:28 2014 From: anish.tambe.lists at gmail.com (Anish Tambe) Date: Sun, 26 Oct 2014 11:01:28 +0530 Subject: [Tutor] Help with running an API In-Reply-To: <CAPM-O+yfXkHmjNX5huNic_q2aONmLneFhqgkHW-Y945pJAsqrw@mail.gmail.com> References: <CACKS5hHi-KnhreyBbtU_rL8mEumtg4+3xkDn40m6tV0q9Ese0w@mail.gmail.com> <CAPM-O+yfXkHmjNX5huNic_q2aONmLneFhqgkHW-Y945pJAsqrw@mail.gmail.com> Message-ID: <CAMxoHv10KVZnFpFUVNnpqEYNdH=2zYF+f9+D1McDU=P3muYh=Q@mail.gmail.com> The usage of the api as documented here - https://github.com/zachwill/fred - suggests : >>> import fred # Save your FRED API key. >>> fred.key('my_fred_api_key') # Interact with economic data categories. >>> fred.category() ... Cheers, Anish Tambe On 26 Oct 2014 00:23, "Joel Goldstick" <joel.goldstick at gmail.com> wrote: > On Sat, Oct 25, 2014 at 12:08 PM, Mark Meanwell <mmeanwell at gmail.com> > wrote: > > Hi Folks - new to python and trying to run an API. Running version > 2.7.3. on > > Windows 7 machine. > > > > Here is the scenario for the given API (FRED API in this case): > > > > easy_install Fred from C:\ - this installs to C:\site packages > > > > then I fire up the python shell and run file created for fred api: > > > > from fredapi import Fred > > fred = Fred(api_key='my api key') > > data = fred.get_series('SP500') > > > > keep getting below error message: > > ImportError: cannot import name Fred > > > > thanks for any suggestions! > > I'm not familiar with FRED, but a quick look on stackoverflow used the > lowercase name to import > > > > > > > _______________________________________________ > > Tutor maillist - Tutor at python.org > > To unsubscribe or change subscription options: > > https://mail.python.org/mailman/listinfo/tutor > > > > > > -- > Joel Goldstick > http://joelgoldstick.com > _______________________________________________ > Tutor maillist - Tutor at python.org > To unsubscribe or change subscription options: > https://mail.python.org/mailman/listinfo/tutor > -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/tutor/attachments/20141026/e5ff8ab5/attachment.html> From crk at godblessthe.us Sun Oct 26 00:46:30 2014 From: crk at godblessthe.us (Clayton Kirkwood) Date: Sat, 25 Oct 2014 15:46:30 -0700 Subject: [Tutor] if you're interested in the code thus far... Message-ID: <015d01cff0a5$856d0be0$904723a0$@us> __author__ = 'SYSTEM' import string #Pricing Dividends raw_table = (''' a: Ask y: Dividend Yield b: Bid d: Dividend per Share b2: Ask (Realtime) r1: Dividend Pay Date b3: Bid (Realtime) q: Ex-Dividend Date p: Previous Close o: Open Date c1: Change d1: Last Trade Date c: Change & Percent Change d2: Trade Date c6: Change (Realtime) t1: Last Trade Time k2: Change Percent (Realtime) p2: Change in Percent Averages c8: After Hours Change (Realtime) m5: Change From 200 Day Moving Average c3: Commission m6: Percent Change From 200 Day Moving Average g: Day?s Low m7: Change From 50 Day Moving Average h: Day?s High m8: Percent Change From 50 Day Moving Average k1: Last Trade (Realtime) With Time m3: 50 Day Moving Average l: Last Trade (With Time) m4: 200 Day Moving Average l1: Last Trade (Price Only) t8: 1 yr Target Price Misc w1: Day?s Value Change g1: Holdings Gain Percent w4: Day?s Value Change (Realtime) g3: Annualized Gain p1: Price Paid g4: Holdings Gain m: Day?s Range g5: Holdings Gain Percent (Realtime) m2: Day?s Range (Realtime) g6: Holdings Gain (Realtime) 52 Week Pricing Symbol Info k: 52 Week High v: More Info j: 52 week Low j1: Market Capitalization j5: Change From 52 Week Low j3: Market Cap (Realtime) k4: Change From 52 week High f6: Float Shares j6: Percent Change From 52 week Low n: Name k5: Percent Change From 52 week High n4: Notes w: 52 week Range s: Symbol s1: Shares Owned x: Stock Exchange j2: Shares Outstanding Volume v: Volume a5: Ask Size b6: Bid Size Misc k3: Last Trade Size t7: Ticker Trend a2: Average Daily Volume t6: Trade Links i5: Order Book (Realtime) Ratios l2: High Limit e: Earnings per Share l3: Low Limit e7: EPS Estimate Current Year v1: Holdings Value e8: EPS Estimate Next Year v7: Holdings Value (Realtime) e9: EPS Estimate Next Quarter s6 Revenue b4: Book Value j4: EBITDA p5: Price / Sales p6: Price / Book r: P/E Ratio r2: P/E Ratio (Realtime) r5: PEG Ratio r6: Price / EPS Estimate Current Year r7: Price / EPS Estimate Next Year s7: Short Ratio ''') import re, string col_position, code, description = 0, [], [] key_name = raw_table.replace('\t','\n') for each_line in key_name.splitlines(): if ':' in each_line: c, d = each_line.split(':') code.append(c) description.append(d.strip()) print( col_position, code[col_position], description[col_position]) col_position += 1 output_line_len = 120 current_output_pos = index = 0 description_output_string = code_output_string = '' for description_position, code_position in zip(description, code): #for description_position in description: # code_position = code[index] description_position_len = len(description_position) current_output_pos += description_position_len # index += 1 if current_output_pos >= output_line_len: print(description_output_string) print(code_output_string) code_output_string=description_output_string='' current_output_pos=-1 #start new line description_output_string += '{:^}|'.format(description_position) code_output_string+='{0:^{1}}|'.format(code_position, description_position_len) current_output_pos+=1 #takes care of '|' at end of string and a subset of output: Ask|Dividend Yield|Bid|Dividend per Share|Ask (Realtime)|Dividend Pay Date|Bid (Realtime)|Ex-Dividend Date| a | y | b | d | b2 | r1 | b3 | q | Previous Close|Open|Change|Last Trade Date|Change & Percent Change|Trade Date|Change (Realtime)|Last Trade Time| p | o | c1 | d1 | c | d2 | c6 | t1 | Change Percent (Realtime)|Change in Percent|After Hours Change (Realtime)|Change From 200 Day Moving Average|Commission| k2 | p2 | c8 | m5 | c3 | P Spacing not correct on above output obviously!! -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/tutor/attachments/20141025/194467df/attachment-0001.html> From wcb_rlb at bellsouth.net Sun Oct 26 02:05:22 2014 From: wcb_rlb at bellsouth.net (Bill Bright) Date: Sat, 25 Oct 2014 21:05:22 -0400 Subject: [Tutor] Python GPIO Code Help Needed Message-ID: <F5CD3C49-EF37-4453-84E9-DC7301A5EC97@bellsouth.net> I have been working on a piece of code that I got from another tutorial. The code polls the GPIO pins on a Raspberry Pi. When it detects a switch being flipped, it plays a corresponding audio file. My problem is, if the switch remains flipped, the audio repeats again and again. What I would like to do is when a switch is flipped have the audio play, then have the code pause until the switch is returned to normal (not repeating the audio), and then return to the while loop to poll for the next switch flip. I am apparently not smart enough to figure out the correct code. I really need some help with this and I would appreciate any assistance. Here is the code I'm working with. #!/usr/bin/env python from time import sleep import os import RPi.GPIO as GPIO GPIO.setmode(GPIO.BCM) GPIO.setup(23, GPIO.IN) GPIO.setup(24, GPIO.IN) GPIO.setup(25, GPIO.IN) while True: if ( GPIO.input(23) == False ): os.system('mpg321 -g 95 a.mp3') if ( GPIO.input(24) == False ): os.system('mpg321 -g 95 b.mp3') if ( GPIO.input(25)== False ): os.system('mpg321 -g 95 c.mp3') sleep(1.1); From alan.gauld at btinternet.com Sun Oct 26 10:10:34 2014 From: alan.gauld at btinternet.com (Alan Gauld) Date: Sun, 26 Oct 2014 09:10:34 +0000 Subject: [Tutor] if you're interested in the code thus far... In-Reply-To: <015d01cff0a5$856d0be0$904723a0$@us> References: <015d01cff0a5$856d0be0$904723a0$@us> Message-ID: <m2idqb$7nf$1@ger.gmane.org> On 25/10/14 23:46, Clayton Kirkwood wrote: > __author__ = 'SYSTEM' You are still setting __author__ which is a bit suspect. Leave double underscores to python. > import string You are still importing string twice, and you don't use it anywhere that I can see. > #Pricing Dividends > > raw_table = (''' I assume this will eventually come from a file? You are not really going to store it with your code? If you are then using a string is pointless and causing you lots of extra work. > a: Ask y: Dividend Yield > b: Bid d: Dividend per Share > b2: Ask (Realtime) r1: Dividend Pay Date > b3: Bid (Realtime) q: Ex-Dividend Date ... > s7: Short Ratio > ''') You don't need parens as well as triple quotes. The quotes alone are sufficient. > import re, string second string import... And you don't seem to be using re either? > col_position, code, description = 0, [], [] > key_name = raw_table.replace('\t','\n') I assume this is because you don't control the file format? Since otherwise you would just use newlines in the file, right? > > for each_line in key_name.splitlines(): > if ':' in each_line: > c, d = each_line.split(':') > code.append(c) > description.append(d.strip()) > print( col_position, code[col_position], description[col_position]) > col_position += 1 You could use enumerate in the for loop and that would set the col_position value for you: for col_position,each_line in enumerate(key_name.splitlines()): > output_line_len = 120 > current_output_pos = index = 0 > description_output_string = code_output_string = '' > for description_position, code_position in zip(description, code): Why not just put the codes and descriptions in tuples when you read them in the loop above? Why use zip? In other words where you do > c, d = each_line.split(':') > code.append(c) > description.append(d.strip()) Why not just join the pair there: > c, d = each_line.split(':') values.append((c,d)) or even just > values.append(each_line.split(':')) It seems as if you are splitting the values into two lists only to zip those lists together again in the next loop? > description_position_len = len(description_position) > current_output_pos += description_position_len > > if current_output_pos >= output_line_len: > print(description_output_string) > print(code_output_string) > code_output_string=description_output_string='' > current_output_pos=-1 #start new line > > description_output_string += '{:^}|'.format(description_position) > > code_output_string+='{0:^{1}}|'.format(code_position, > description_position_len) > > current_output_pos+=1 #takes care of '|' at end of string HTH -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.flickr.com/photos/alangauldphotos From __peter__ at web.de Sun Oct 26 11:52:46 2014 From: __peter__ at web.de (Peter Otten) Date: Sun, 26 Oct 2014 11:52:46 +0100 Subject: [Tutor] if you're interested in the code thus far... References: <015d01cff0a5$856d0be0$904723a0$@us> Message-ID: <m2ijpv$tm7$1@ger.gmane.org> Clayton Kirkwood wrote: > for description_position, code_position in zip(description, code): > description_position_len = len(description_position) > current_output_pos += description_position_len > if current_output_pos >= output_line_len: > print(description_output_string) > print(code_output_string) > code_output_string=description_output_string='' > current_output_pos=-1 #start new line > > description_output_string += '{:^}|'.format(description_position) > code_output_string+='{0:^{1}}|'.format(code_position, > description_position_len) > > current_output_pos+=1 #takes care of '|' at end of string If you want to ensure that the actual line length is never more than output_line_len -- the above doesn't do that. Hint: reconsider this line: > current_output_pos=-1 #start new line > and a subset of output: Some pairs are not printed. Can you figure out which and why? From nik at naturalnet.de Sun Oct 26 13:10:16 2014 From: nik at naturalnet.de (Dominik George) Date: Sun, 26 Oct 2014 13:10:16 +0100 Subject: [Tutor] Python GPIO Code Help Needed In-Reply-To: <F5CD3C49-EF37-4453-84E9-DC7301A5EC97@bellsouth.net> References: <F5CD3C49-EF37-4453-84E9-DC7301A5EC97@bellsouth.net> Message-ID: <51694114-B74B-4975-8205-20547501BD6B@naturalnet.de> Hi, > if ( GPIO.input(23) == False ): > os.system('mpg321 -g 95 a.mp3') while not GPIO.input(23): pass ... would be the simplest solution. -nik From davea at davea.name Sun Oct 26 13:27:59 2014 From: davea at davea.name (Dave Angel) Date: Sun, 26 Oct 2014 08:27:59 -0400 (EDT) Subject: [Tutor] Python GPIO Code Help Needed References: <F5CD3C49-EF37-4453-84E9-DC7301A5EC97@bellsouth.net> Message-ID: <m2ip5l$d6h$1@ger.gmane.org> Bill Bright <wcb_rlb at bellsouth.net> Wrote in message: > I have been working on a piece of code that I got from another tutorial. The code polls the GPIO pins on a Raspberry Pi. When it detects a switch being flipped, it plays a corresponding audio file. My problem is, if the switch remains flipped, the audio repeats again and again. What I would like to do is when a switch is flipped have the audio play, then have the code pause until the switch is returned to normal (not repeating the audio), and then return to the while loop to poll for the next switch flip. I am apparently not smart enough to figure out the correct code. I really need some help with this and I would appreciate any assistance. > > Here is the code I'm working with. > > #!/usr/bin/env python > from time import sleep > import os > import RPi.GPIO as GPIO > GPIO.setmode(GPIO.BCM) > GPIO.setup(23, GPIO.IN) > GPIO.setup(24, GPIO.IN) > GPIO.setup(25, GPIO.IN) > while True: > if ( GPIO.input(23) == False ): > os.system('mpg321 -g 95 a.mp3') > if ( GPIO.input(24) == False ): > os.system('mpg321 -g 95 b.mp3') > if ( GPIO.input(25)== False ): > os.system('mpg321 -g 95 c.mp3') > sleep(1.1); > _______________________________________________ > Tutor maillist - Tutor at python.org > To unsubscribe or change subscription options: > https://mail.python.org/mailman/listinfo/tutor > > So what you're saying is that you want t do an action anytime you notice a change in the input state from True to False. So you just need to store the old state in your own variable. Whever old is True and new is False, perform the action. In your initialization, create a new list of bools, length at least 26. oldstates = [True] * 32 Now write a function changed, that detects the state change for one of the inputs : def changed(num): newstate = GPIO.input(num) change = oldstates[num] and not newstate oldstates[num] = newstate return change Now each of the if clauses can be written : if changed(23): os.system.... Hopefully this will give you some ideas how you could generalize and/or streamline the code. If you decide to use more than 3 switches, or if some want to detect the opposite transition, or ... -- DaveA From crushed26 at gmail.com Sun Oct 26 14:12:50 2014 From: crushed26 at gmail.com (Crush) Date: Sun, 26 Oct 2014 09:12:50 -0400 Subject: [Tutor] Python GPIO Code Help Needed In-Reply-To: <mailman.21.1414321202.15493.tutor@python.org> References: <mailman.21.1414321202.15493.tutor@python.org> Message-ID: <C4312AC1-3651-46CC-96E7-DD98FB38F042@gmail.com> Could this work? /usr/bin/env python import os import RPi.GPIO as GPIO GPIO.setmode(GPIO.BCM) GPIO.setup(23, GPIO.IN) GPIO.setup(24, GPIO.IN) GPIO.setup(25, GPIO.IN) pin_list = ['GPIO.input(23)', 'GPIO.input(23)', 'GPIO.input(23)'] for item in pin_list: if item == false: os.system('mpg321 -g 95 a.mp3') Bo From anish.tambe.lists at gmail.com Sun Oct 26 10:36:13 2014 From: anish.tambe.lists at gmail.com (Anish Tambe) Date: Sun, 26 Oct 2014 15:06:13 +0530 Subject: [Tutor] Python GPIO Code Help Needed In-Reply-To: <F5CD3C49-EF37-4453-84E9-DC7301A5EC97@bellsouth.net> References: <F5CD3C49-EF37-4453-84E9-DC7301A5EC97@bellsouth.net> Message-ID: <CAMxoHv0+CL4Zi6ijP--6HYWP+Yx=rV3aZ4foHC21dM5RtczK8g@mail.gmail.com> On Sun, Oct 26, 2014 at 6:35 AM, Bill Bright <wcb_rlb at bellsouth.net> wrote: > I have been working on a piece of code that I got from another tutorial. > The code polls the GPIO pins on a Raspberry Pi. When it detects a switch > being flipped, it plays a corresponding audio file. My problem is, if the > switch remains flipped, the audio repeats again and again. What I would > like to do is when a switch is flipped have the audio play, then have the > code pause until the switch is returned to normal (not repeating the > audio), and then return to the while loop to poll for the next switch flip. > I am apparently not smart enough to figure out the correct code. I really > need some help with this and I would appreciate any assistance. > > Here is the code I'm working with. > > #!/usr/bin/env python > from time import sleep > import os > import RPi.GPIO as GPIO > GPIO.setmode(GPIO.BCM) > GPIO.setup(23, GPIO.IN) > GPIO.setup(24, GPIO.IN) > GPIO.setup(25, GPIO.IN) > while True: > if ( GPIO.input(23) == False ): > os.system('mpg321 -g 95 a.mp3') > if ( GPIO.input(24) == False ): > os.system('mpg321 -g 95 b.mp3') > if ( GPIO.input(25)== False ): > os.system('mpg321 -g 95 c.mp3') > sleep(1.1); > _______________________________________________ > Tutor maillist - Tutor at python.org > To unsubscribe or change subscription options: > https://mail.python.org/mailman/listinfo/tutor > Using callbacks instead of polling might help. You can do something like this - def play(channel) : #your action here if channel == 23 : os.system('mpg321 -g 95 a.mp3') GPIO.add_event_detect(23, GPIO.RISING, callback=play) and similarly for the other pins. Check the "Events and Callback Functions" section of this tutorial - http://makezine.com/projects/tutorial-raspberry-pi-gpio-pins-and-python/ Cheers, Anish Tambe -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/tutor/attachments/20141026/c63711f8/attachment.html> From wbecerra1 at gmail.com Sun Oct 26 10:01:52 2014 From: wbecerra1 at gmail.com (William Becerra) Date: Sun, 26 Oct 2014 11:01:52 +0200 Subject: [Tutor] Differentiating vowels from consonants Message-ID: <CAF1DC4WLvpXD3AJWqGydLPmoOqCVZVEZe0SqYbrLhyzUQZ2C9A@mail.gmail.com> Hello, I'm new to programming Running Python 2.7.8 on Windows 8 OS I was reading http://www.sthurlow.com/python/lesson07/ Here there is an example of the for loop with a Cheerleader program but the program is not able to print grammatically correct. word = raw_input("Who do you go for? ") for letter in word: call = "Gimme a " + letter + "!" print call print letter + "!" print "What does that spell?" print word + "!" I tried changing the code so that the program can recognize vowels from consonants and write the correct article (a or an) here is my code. word = raw_input("Who do You Support: ") vowels = ('a', 'e', 'i', 'o', 'u') for letter in word: if letter == vowels: call = 'Give me an ' + letter + '!' print call print letter + '!' else : call = 'Give me a ' + letter + '!' print call print letter + '!' print 'What does that say' print word + '!!' My code also isn't able to recognise vowels. For example if i write Manchester as the raw_input i get : give me a M give me a a etc. I would like it to say give me a M give an a So here is my question. What is wrong with my code and how can I change it to get the result I want. Thank You -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/tutor/attachments/20141026/e8a00922/attachment-0001.html> From alan.gauld at btinternet.com Sun Oct 26 14:31:23 2014 From: alan.gauld at btinternet.com (Alan Gauld) Date: Sun, 26 Oct 2014 13:31:23 +0000 Subject: [Tutor] Differentiating vowels from consonants In-Reply-To: <CAF1DC4WLvpXD3AJWqGydLPmoOqCVZVEZe0SqYbrLhyzUQZ2C9A@mail.gmail.com> References: <CAF1DC4WLvpXD3AJWqGydLPmoOqCVZVEZe0SqYbrLhyzUQZ2C9A@mail.gmail.com> Message-ID: <m2it3c$7e8$1@ger.gmane.org> On 26/10/14 09:01, William Becerra wrote: > word = raw_input("Who do You Support: ") > vowels = ('a', 'e', 'i', 'o', 'u') You could just use a string vowels = 'aeiou' > for letter in word: > if letter == vowels: you want if letter in vowels: > call = 'Give me an ' + letter + '!' > print call > print letter + '!' > else : > call = 'Give me a ' + letter + '!' > print call But I'd use a helper function instead: def article(letter): return 'an' if letter in 'aeiou' else 'a' and write the call assignment as call = 'Give me ' + article(letter) + letter + '!' Which saves the if/else structure with its repeated code. Of course you may not have covered functions yet in which case your solution is fine. HTH -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.flickr.com/photos/alangauldphotos From alan.gauld at btinternet.com Sun Oct 26 14:41:13 2014 From: alan.gauld at btinternet.com (Alan Gauld) Date: Sun, 26 Oct 2014 13:41:13 +0000 Subject: [Tutor] Python GPIO Code Help Needed In-Reply-To: <F5CD3C49-EF37-4453-84E9-DC7301A5EC97@bellsouth.net> References: <F5CD3C49-EF37-4453-84E9-DC7301A5EC97@bellsouth.net> Message-ID: <m2itlp$fsr$1@ger.gmane.org> On 26/10/14 01:05, Bill Bright wrote: > The code polls the GPIO pins on a Raspberry Pi. > When it detects a switch being flipped, it plays > a corresponding audio file. Does that mean in either direction? Or does it only play when the switch goes ON? (I'm not familiar with the Pi (although somebody recently bought me one as a gift!) so don't know what these switches look like - I'm guessing they are DIPs?) > What I would like to do is when a switch is flipped have > the audio play, then have the code pause until the switch > is returned to normal (not repeating the audio), > and then return to the while loop to poll for the next switch flip. What happens if one of the other switches is flipped while you are waiting? You need to think through all the possible combinations that could happen at any one time and the resultant actions. (In professional coding this would usually involve a "state machine") So the questions I have: 1) If I flip a switch a tune plays. - What if I flip another switch while the first is playing? - What if I flip another switch after the tune finishes? - What if I restore the switch? - do I play it again? Do I play a different tune? or None? 2) What if I flip multiple switches together? - Do all the related tunes play? or none? HTH -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.flickr.com/photos/alangauldphotos From nik at naturalnet.de Sun Oct 26 18:18:12 2014 From: nik at naturalnet.de (Dominik George) Date: Sun, 26 Oct 2014 18:18:12 +0100 Subject: [Tutor] Auto-response for your message to the "Tutor" mailing list In-Reply-To: <mailman.89571.1414326013.18129.tutor@python.org> References: <mailman.89571.1414326013.18129.tutor@python.org> Message-ID: <A5CA9D94-761A-4DAD-88E5-9688AB1881E1@naturalnet.de> Hi, Am 26. Oktober 2014 13:20:13 MEZ, schrieb tutor-bounces at python.org: >Your message for tutor at python.org, the Python programming tutor list, >has been received and is being delivered. This automated response is >sent to those of you new to the Tutor list, to point out a few >resources that can help with answering your own questions, or improve >the chances of getting a useful answer from the other subscribers. why is it I get this messages repeatedly despite having been reading and writing on the list for over a year now? Cheers, Nik From alan.gauld at btinternet.com Sun Oct 26 20:40:50 2014 From: alan.gauld at btinternet.com (Alan Gauld) Date: Sun, 26 Oct 2014 19:40:50 +0000 Subject: [Tutor] Auto-response for your message to the "Tutor" mailing list In-Reply-To: <A5CA9D94-761A-4DAD-88E5-9688AB1881E1@naturalnet.de> References: <mailman.89571.1414326013.18129.tutor@python.org> <A5CA9D94-761A-4DAD-88E5-9688AB1881E1@naturalnet.de> Message-ID: <m2jio2$j3b$1@ger.gmane.org> On 26/10/14 17:18, Dominik George wrote: > Hi, > > Am 26. Oktober 2014 13:20:13 MEZ, schrieb tutor-bounces at python.org: >> Your message for tutor at python.org, the Python programming tutor list, >> has been received and is being delivered. This automated response is >> sent to those of you new to the Tutor list, to point out a few >> resources that can help with answering your own questions, or improve >> the chances of getting a useful answer from the other subscribers. > > why is it I get this messages repeatedly despite having been reading and writing on the list for over a year now? The list server puts you on moderation when you first join. Usually I recognise regular posters names after a while and take them off - less work for me! However, on looking at the admin screens you are not on moderation... The other thing that might trigger it is if you are sending from a different address than the one you registered with? It all looks OK at first glance,.. -- Alan G Tutor list moderator. From crk at godblessthe.us Sun Oct 26 20:32:53 2014 From: crk at godblessthe.us (Clayton Kirkwood) Date: Sun, 26 Oct 2014 12:32:53 -0700 Subject: [Tutor] if you're interested in the code thus far... In-Reply-To: <m2ijpv$tm7$1@ger.gmane.org> References: <015d01cff0a5$856d0be0$904723a0$@us> <m2ijpv$tm7$1@ger.gmane.org> Message-ID: <01ec01cff153$a99c3500$fcd49f00$@us> Good eyes on the logic !-----Original Message----- !From: Tutor [mailto:tutor-bounces+crk=godblessthe.us at python.org] On !Behalf Of Peter Otten !Sent: Sunday, October 26, 2014 3:53 AM !To: tutor at python.org !Subject: Re: [Tutor] if you're interested in the code thus far... ! !Clayton Kirkwood wrote: ! !> for description_position, code_position in zip(description, code): !> description_position_len = len(description_position) !> current_output_pos += description_position_len !> if current_output_pos >= output_line_len: !> print(description_output_string) !> print(code_output_string) !> code_output_string=description_output_string='' !> current_output_pos=-1 #start new line Changing this line to current_ouput_pos = description_position_len I believe I was losing the next lines final field on each subsequent line Thank-you Clayton !> !> description_output_string += '{:^}|'.format(description_position) !> code_output_string+='{0:^{1}}|'.format(code_position, !> description_position_len) !> !> current_output_pos+=1 #takes care of '|' at end of string ! !If you want to ensure that the actual line length is never more than !output_line_len -- the above doesn't do that. ! !Hint: reconsider this line: ! !> current_output_pos=-1 #start new line ! ! !> and a subset of output: ! !Some pairs are not printed. Can you figure out which and why? ! !_______________________________________________ !Tutor maillist - Tutor at python.org !To unsubscribe or change subscription options: !https://mail.python.org/mailman/listinfo/tutor From danny.yoo at gmail.com Sun Oct 26 21:15:10 2014 From: danny.yoo at gmail.com (Danny Yoo) Date: Sun, 26 Oct 2014 13:15:10 -0700 Subject: [Tutor] Auto-response for your message to the "Tutor" mailing list In-Reply-To: <m2jio2$j3b$1@ger.gmane.org> References: <mailman.89571.1414326013.18129.tutor@python.org> <A5CA9D94-761A-4DAD-88E5-9688AB1881E1@naturalnet.de> <m2jio2$j3b$1@ger.gmane.org> Message-ID: <CAGZAPF5ywji8Emze7M+WPWfM=Cu7XySzD-nZbCk0nB4rp8LXwg@mail.gmail.com> >> why is it I get this messages repeatedly despite having been reading and writing on the list for over a year now? This has happened to me once in a while too. I conjecture that it might be a bug with Mailman, but I'd have to dive into the code to be certain. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/tutor/attachments/20141026/d9373fea/attachment.html> From mbrahimi02 at gmail.com Sun Oct 26 21:25:09 2014 From: mbrahimi02 at gmail.com (Malik Brahimi) Date: Sun, 26 Oct 2014 16:25:09 -0400 Subject: [Tutor] User Input with Multiple Lines Message-ID: <CABe8r0J5KoHp+r6_uxgKYEp6N6+CRLQf4jum634K0fpC6JWTRQ@mail.gmail.com> So I am working with input that is pasted from a PDF file, and I was wondering if there is any way I can do so without the program terminating abruptly because of the newlines. I know that multiple lines of input can be achieved via a loop with string concatenation, but this is not what I'm looking for. Remember, I am trying to directly cut and paste a PDF paragraph with newlines into a console program without an unexpected exit. Take a look at the following PDF for an example: Example PDF <http://www.energy.umich.edu/sites/default/files/pdf-sample.pdf> -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/tutor/attachments/20141026/bf044f89/attachment-0001.html> From marc.tompkins at gmail.com Sun Oct 26 22:37:15 2014 From: marc.tompkins at gmail.com (Marc Tompkins) Date: Sun, 26 Oct 2014 14:37:15 -0700 Subject: [Tutor] Auto-response for your message to the "Tutor" mailing list In-Reply-To: <CAGZAPF5ywji8Emze7M+WPWfM=Cu7XySzD-nZbCk0nB4rp8LXwg@mail.gmail.com> References: <mailman.89571.1414326013.18129.tutor@python.org> <A5CA9D94-761A-4DAD-88E5-9688AB1881E1@naturalnet.de> <m2jio2$j3b$1@ger.gmane.org> <CAGZAPF5ywji8Emze7M+WPWfM=Cu7XySzD-nZbCk0nB4rp8LXwg@mail.gmail.com> Message-ID: <CAKK8jXZ6wj8pHyNH=QjHox9mh6VRSz+teBvjpZ3NH9ADkBmtPQ@mail.gmail.com> On Sun, Oct 26, 2014 at 1:15 PM, Danny Yoo <danny.yoo at gmail.com> wrote: > > >> why is it I get this messages repeatedly despite having been reading > and writing on the list for over a year now? > > This has happened to me once in a while too. I conjecture that it might > be a bug with Mailman, but I'd have to dive into the code to be certain. > > I get these every few months, especially (but not exclusively) if I've been silent for a while. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/tutor/attachments/20141026/5d3df1f6/attachment.html> From ben+python at benfinney.id.au Sun Oct 26 23:27:25 2014 From: ben+python at benfinney.id.au (Ben Finney) Date: Mon, 27 Oct 2014 09:27:25 +1100 Subject: [Tutor] User Input with Multiple Lines References: <CABe8r0J5KoHp+r6_uxgKYEp6N6+CRLQf4jum634K0fpC6JWTRQ@mail.gmail.com> Message-ID: <858uk231rm.fsf@benfinney.id.au> Malik Brahimi <mbrahimi02 at gmail.com> writes: > So I am working with input that is pasted from a PDF file, and I was > wondering if there is any way I can do so without the program > terminating abruptly because of the newlines. Can you construct a *very* short and simple example program, which demonstrates the problem, and show it here? -- \ ?The cost of education is trivial compared to the cost of | `\ ignorance.? ?Thomas Jefferson | _o__) | Ben Finney From carolineharlow0 at gmail.com Sun Oct 26 23:15:51 2014 From: carolineharlow0 at gmail.com (Caroline H) Date: Sun, 26 Oct 2014 15:15:51 -0700 Subject: [Tutor] Python Questions Help Message-ID: <CAHy4Z5kkBjP1AL9JeZT_8vRC+s2Ov3b0Ngw5GU7UP5gouuhc6g@mail.gmail.com> Hi Python Tutor, I'm having a lot of trouble with this python problem and I'm wondering if you can help me. "Given the lists, lst1 and lst2 , create a new sorted list consisting of all the elements of lst1 that also appears in lst2 . For example, if lst1 is [4, 3, 2, 6, 2] and lst2 is [1, 2, 4], then the new list would be [2, 2, 4]. Note that duplicate elements in lst1 that appear in lst2 are also duplicated in the new list. Associate the new list with the variable new_list , and don't forget to sort the new list." The code I wrote almost works, it just needs to be able to add a number more than once in the third list. For example, lst1 = [2,5,6,7,2] lst2 = [2,4] it comes up with new_list = [2] when I need it to come up with new_list = [2,2] The code I have so far is: new_list = [] i = 0 j = 0 if len(lst1)<=len(lst2): for i in range(len(lst1)): if lst1[i] in lst2: new_list.append(lst1[i]) else: for j in range(len(lst2)): if lst2[j] in lst1: new_list.append(lst2[j]) -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/tutor/attachments/20141026/2d455589/attachment.html> From crk at godblessthe.us Sun Oct 26 23:12:21 2014 From: crk at godblessthe.us (Clayton Kirkwood) Date: Sun, 26 Oct 2014 15:12:21 -0700 Subject: [Tutor] if you're interested in the code thus far... In-Reply-To: <m2idqb$7nf$1@ger.gmane.org> References: <015d01cff0a5$856d0be0$904723a0$@us> <m2idqb$7nf$1@ger.gmane.org> Message-ID: <022301cff169$ecd2ece0$c678c6a0$@us> !-----Original Message----- !From: Tutor [mailto:tutor-bounces+crk=godblessthe.us at python.org] On !Behalf Of Alan Gauld !Sent: Sunday, October 26, 2014 2:11 AM !To: tutor at python.org !Subject: Re: [Tutor] if you're interested in the code thus far... ! !On 25/10/14 23:46, Clayton Kirkwood wrote: !> __author__ = 'SYSTEM' ! !You are still setting __author__ which is a bit suspect. !Leave double underscores to python. This is something being created when I started with this file. ! !> import string ! !You are still importing string twice, and you don't use it anywhere that !I can see. ! !> #Pricing Dividends !> !> raw_table = (''' ! !I assume this will eventually come from a file? !You are not really going to store it with your code? !If you are then using a string is pointless and causing you lots of !extra work. ! !> a: Ask y: Dividend Yield !> b: Bid d: Dividend per Share !> b2: Ask (Realtime) r1: Dividend Pay Date !> b3: Bid (Realtime) q: Ex-Dividend Date !... !> s7: Short Ratio !> ''') ! !You don't need parens as well as triple quotes. !The quotes alone are sufficient. Thanks. ! !> import re, string ! !second string import... And you don't seem to be using re either? ! Used them before and then removed the need for them. !> col_position, code, description = 0, [], [] key_name = !> raw_table.replace('\t','\n') ! !I assume this is because you don't control the file format? Since !otherwise you would just use newlines in the file, right? Correct. I cut and pasted the data. Not going to change (probably) no sense in putting it in a file ! !> !> for each_line in key_name.splitlines(): !> if ':' in each_line: !> c, d = each_line.split(':') !> code.append(c) !> description.append(d.strip()) !> print( col_position, code[col_position], !description[col_position]) !> col_position += 1 ! !You could use enumerate in the for loop and that would set the !col_position value for you: ! !for col_position,each_line in enumerate(key_name.splitlines()): Good info, but I tried it and the debugger indicated an IndexError: list.index out of range. Interesting because when I cursor over the various items on the line, they seem fine: col_position was 1 However, I don't think that this solution will work for me, because there are lines with no : these being category type lines such as Date toward the top of the data. But I now have new info on enumerate. ! ! !> output_line_len = 120 !> current_output_pos = index = 0 !> description_output_string = code_output_string = '' !> for description_position, code_position in zip(description, code): ! !Why not just put the codes and descriptions in tuples when you read them !in the loop above? Why use zip? In other words where you do ! ! > c, d = each_line.split(':') ! > code.append(c) ! > description.append(d.strip()) ! !Why not just join the pair there: ! ! > c, d = each_line.split(':') ! values.append((c,d)) ! !or even just ! ! > values.append(each_line.split(':')) ! Uh, because I didn't know about it....:<))) !It seems as if you are splitting the values into two lists only to zip !those lists together again in the next loop? ! !> description_position_len = len(description_position) !> current_output_pos += description_position_len !> !> if current_output_pos >= output_line_len: !> print(description_output_string) !> print(code_output_string) !> code_output_string=description_output_string='' !> current_output_pos=-1 #start new line !> !> description_output_string += '{:^}|'.format(description_position) !> !> code_output_string+='{0:^{1}}|'.format(code_position, !> description_position_len) !> !> current_output_pos+=1 #takes care of '|' at end of string ! This has led to a problem, however. How do I determine if a value is in one of the tuples: Key in [a, word], [key, word] I thought there might be a comprehension or something like a list.key or something. Thanks, Clayton !HTH !-- !Alan G !Author of the Learn to Program web site !http://www.alan-g.me.uk/ !http://www.flickr.com/photos/alangauldphotos ! !_______________________________________________ !Tutor maillist - Tutor at python.org !To unsubscribe or change subscription options: !https://mail.python.org/mailman/listinfo/tutor From wcb_rlb at bellsouth.net Sun Oct 26 23:37:37 2014 From: wcb_rlb at bellsouth.net (Bill Bright) Date: Sun, 26 Oct 2014 18:37:37 -0400 Subject: [Tutor] Python GPIO Code Help Needed Message-ID: <942ABD1F-C071-4077-9D1F-F2EEE770EE5F@bellsouth.net> Thanks to everyone for the help. The code below seems to work. My question is why does oldstates need to multiplied by 32? Does this mean that the code will only work for 32 switches before I have to reset the Pi? Code: #!/usr/bin/env python from time import sleep import os import RPi.GPIO as GPIO GPIO.setmode(GPIO.BCM) GPIO.setup(23, GPIO.IN) GPIO.setup(24, GPIO.IN) GPIO.setup(25, GPIO.IN) oldstates=[True]*32 def changed(channel): newstate = GPIO.input(channel) change = oldstates[channel] and not newstate oldstates[channel] = newstate return change while True: if changed(23): os.system('mpg321 -g 95 a.mp3') if changed(24): os.system('mpg321 -g 95 b.mp3') if changed(25): os.system('mpg321 -g 95 c.mp3') sleep(1.1); -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/tutor/attachments/20141026/dacb6046/attachment-0001.html> From wcb_rlb at bellsouth.net Sun Oct 26 23:42:57 2014 From: wcb_rlb at bellsouth.net (Bill Bright) Date: Sun, 26 Oct 2014 18:42:57 -0400 Subject: [Tutor] Python GPIO Code Help Needed Message-ID: <60B0E92C-8ACC-45B9-BCC5-AF076F6E78FF@bellsouth.net> Thanks to all for the help. The code below seems to work. My question is why does oldstates need to be multiplied by 32? Does this means the code will only read 32 switch changes? That would be a problem. Code: #!/usr/bin/env python from time import sleep import os import RPi.GPIO as GPIO GPIO.setmode(GPIO.BCM) GPIO.setup(23, GPIO.IN) GPIO.setup(24, GPIO.IN) GPIO.setup(25, GPIO.IN) oldstates=[True]*32 def changed(channel): newstate = GPIO.input(channel) change = oldstates[channel] and not newstate oldstates[channel] = newstate return change while True: if changed(23): os.system('mpg321 -g 95 a.mp3') if changed(24): os.system('mpg321 -g 95 b.mp3') if changed(25): os.system('mpg321 -g 95 c.mp3') sleep(1.1); -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/tutor/attachments/20141026/c0de7f82/attachment.html> From alan.gauld at btinternet.com Mon Oct 27 01:16:09 2014 From: alan.gauld at btinternet.com (Alan Gauld) Date: Mon, 27 Oct 2014 00:16:09 +0000 Subject: [Tutor] Python GPIO Code Help Needed In-Reply-To: <60B0E92C-8ACC-45B9-BCC5-AF076F6E78FF@bellsouth.net> References: <60B0E92C-8ACC-45B9-BCC5-AF076F6E78FF@bellsouth.net> Message-ID: <m2k2s9$504$1@ger.gmane.org> On 26/10/14 22:42, Bill Bright wrote: > Thanks to all for the help. The code below seems to work. My question is > why does oldstates need to be multiplied by 32? Its creating 32 instances of True. So you can cater for up to 32 different switch positions. > will only read 32 switch changes? That would be a problem. No it can manage up to 32 different switches. Each switch can toggle between True/False as often as it likes but oldstates will store the previous state for each switch. > oldstates=[True]*32 > > def changed(channel): > newstate = GPIO.input(channel) > change = oldstates[channel] and not newstate > oldstates[channel] = newstate Here is where it compares the old and new state of the given switch/channel > while True: > if changed(23): > os.system('mpg321 -g 95 a.mp3') > if changed(24): > os.system('mpg321 -g 95 b.mp3') > if changed(25): > os.system('mpg321 -g 95 c.mp3') > sleep(1.1); HTH -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.flickr.com/photos/alangauldphotos From eryksun at gmail.com Mon Oct 27 01:18:41 2014 From: eryksun at gmail.com (eryksun) Date: Sun, 26 Oct 2014 19:18:41 -0500 Subject: [Tutor] User Input with Multiple Lines In-Reply-To: <CABe8r0J5KoHp+r6_uxgKYEp6N6+CRLQf4jum634K0fpC6JWTRQ@mail.gmail.com> References: <CABe8r0J5KoHp+r6_uxgKYEp6N6+CRLQf4jum634K0fpC6JWTRQ@mail.gmail.com> Message-ID: <CACL+1avRyO1BZHK22khHO3x+STMpady5yvNa4Xxb-PY6rdyurw@mail.gmail.com> On Sun, Oct 26, 2014 at 3:25 PM, Malik Brahimi <mbrahimi02 at gmail.com> wrote: > So I am working with input that is pasted from a PDF file, and I was > wondering if there is any way I can do so without the program terminating > abruptly because of the newlines. input() and raw_input() grab a single line of text. The rest of the lines, if any, remain in the input buffer. As you know, you can loop to build a list and then '\n'.join() the list. To instead grab the whole buffer, you can use sys.stdin.read(). In this case you need to manually write EOF to the buffer at the start of a new line. In a POSIX terminal that's Ctrl-D. In the Windows console use Ctrl-Z+Return. You may already be familiar with these as a shortcut for exiting the shell. From alan.gauld at btinternet.com Mon Oct 27 01:22:05 2014 From: alan.gauld at btinternet.com (Alan Gauld) Date: Mon, 27 Oct 2014 00:22:05 +0000 Subject: [Tutor] Python Questions Help In-Reply-To: <CAHy4Z5kkBjP1AL9JeZT_8vRC+s2Ov3b0Ngw5GU7UP5gouuhc6g@mail.gmail.com> References: <CAHy4Z5kkBjP1AL9JeZT_8vRC+s2Ov3b0Ngw5GU7UP5gouuhc6g@mail.gmail.com> Message-ID: <m2k37e$adn$1@ger.gmane.org> On 26/10/14 22:15, Caroline H wrote: > For example, > lst1 = [2,5,6,7,2] > lst2 = [2,4] > > it comes up with new_list = [2] when I need it to come up with new_list > = [2,2] Check the logic in the if statement. Walk through it and se if it does what you expect. You are basically doing the right thing, except... > The code I have so far is: > > new_list = [] > i = 0 > j = 0 > if len(lst1)<=len(lst2): > for i in range(len(lst1)): > if lst1[i] in lst2: > new_list.append(lst1[i]) > else: > for j in range(len(lst2)): > if lst2[j] in lst1: > new_list.append(lst2[j]) Rather than using indexing you should use the 'for' loop directly on the lists: for item in list if item in otherlist: newlist.append(item) Its much easier to read and leaves Python to do all the index management for you. Its rare to need to use indexes in conjunction with a 'for' loop. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.flickr.com/photos/alangauldphotos From alan.gauld at btinternet.com Mon Oct 27 01:33:54 2014 From: alan.gauld at btinternet.com (Alan Gauld) Date: Mon, 27 Oct 2014 00:33:54 +0000 Subject: [Tutor] if you're interested in the code thus far... In-Reply-To: <022301cff169$ecd2ece0$c678c6a0$@us> References: <015d01cff0a5$856d0be0$904723a0$@us> <m2idqb$7nf$1@ger.gmane.org> <022301cff169$ecd2ece0$c678c6a0$@us> Message-ID: <m2k3ti$lpq$1@ger.gmane.org> On 26/10/14 22:12, Clayton Kirkwood wrote: > !On 25/10/14 23:46, Clayton Kirkwood wrote: > !> __author__ = 'SYSTEM' > ! > !You are still setting __author__ which is a bit suspect. > !Leave double underscores to python. > > This is something being created when I started with this file. How are you creating your code? Are you using some kind of IDE? > !> raw_table = (''' > ! > !> a: Ask y: Dividend Yield > !> b: Bid d: Dividend per Share > !> b2: Ask (Realtime) r1: Dividend Pay Date > !> b3: Bid (Realtime) q: Ex-Dividend Date > !... > !> s7: Short Ratio > !> ''') > !> col_position, code, description = 0, [], [] key_name = > !> raw_table.replace('\t','\n') > ! > !I assume this is because you don't control the file format? Since > !otherwise you would just use newlines in the file, right? > > Correct. I cut and pasted the data. Not going to change (probably) no sense > in putting it in a file Are you saying the data will stay with your code? If that's the case then reformat the text into a Python data structure and avoid all the parsing that makes up about half your code. > !> for each_line in key_name.splitlines(): > !> if ':' in each_line: > !> c, d = each_line.split(':') > !> code.append(c) > !> description.append(d.strip()) > !> print( col_position, code[col_position], > !description[col_position]) > !> col_position += 1 > ! > !You could use enumerate in the for loop and that would set the > !col_position value for you: > ! ... > > However, I don't think that this solution will work for me, because there > are lines with no : Ah yes, I didn't notice you only increment the counter inside the if. > !> output_line_len = 120 > !> current_output_pos = index = 0 > !> description_output_string = code_output_string = '' > !> for description_position, code_position in zip(description, code): > ! > !Why not just put the codes and descriptions in tuples when you read them > !in the loop above? Why use zip? In other words where you do > ! > ! > c, d = each_line.split(':') > ! > code.append(c) > ! > description.append(d.strip()) > ! > !Why not just join the pair there: > ! > ! > c, d = each_line.split(':') > ! values.append((c,d)) > ! > !or even just > ! > ! > values.append(each_line.split(':')) > ! > > Uh, because I didn't know about it....:<))) > > !It seems as if you are splitting the values into two lists only to zip > !those lists together again in the next loop? > ! ... > > This has led to a problem, however. How do I determine if a value is in one > of the tuples: > Key in [a, word], [key, word] I'm not sure exactly what you mean, I'd probably need to see how you implemented it but, using my example above, you can do key in values[index] So if values looks like: values = [(1,2),(2,3),....] then the test if 2 in values[0]: will be true and if 2 in values[1]: will also be true. If you need to test the specific tuple item you could do: if 2 == values[0][0]: will be false because you only test 1st item and if 2 == values[0][1]: will be true coz you test the 2nd item. If you wanted to extract all the tuples containing 2 you could use a comprehension like: value2 = [tup for tup in values if 2 in tup] HTH -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.flickr.com/photos/alangauldphotos From davea at davea.name Mon Oct 27 01:43:50 2014 From: davea at davea.name (Dave Angel) Date: Sun, 26 Oct 2014 20:43:50 -0400 (EDT) Subject: [Tutor] Python GPIO Code Help Needed References: <60B0E92C-8ACC-45B9-BCC5-AF076F6E78FF@bellsouth.net> Message-ID: <m2k498$rgk$1@ger.gmane.org> Please use text mail for posting, and please use reply-list, or whatever your mailer calls it when adding to a thread. So far, you've got at least 3 threads covering a single topic. Bill Bright <wcb_rlb at bellsouth.net> Wrote in message: > > The code below seems to work. My question is why does oldstates need to be multiplied by 32? Does this means the code will only read 32 switch changes? That would be a problem. If you're just learning Python, you need to actually play with each new concept you encounter. When you multiply a list by an int n, you create a new list, n times as big. Experiment with this in the interpreter, till you understand. ["A", "f", 42] *3 [] * 4000 [True, False] * 10 So you wind up with 32 flags in the oldstates list. Why 32? I don't have a rpi, so I took the wild guess that a Raspberry Pi might have 32 inputs. You're only using 3, so you could have used a list of 3, or a dict. But I was trying for the minimum complication. Others have mentioned an event loop in GPIO. Assuming such exists, it would be better to use it. -- DaveA From dyoo at hashcollision.org Mon Oct 27 01:41:12 2014 From: dyoo at hashcollision.org (Danny Yoo) Date: Sun, 26 Oct 2014 17:41:12 -0700 Subject: [Tutor] Python GPIO Code Help Needed In-Reply-To: <m2k2s9$504$1@ger.gmane.org> References: <60B0E92C-8ACC-45B9-BCC5-AF076F6E78FF@bellsouth.net> <m2k2s9$504$1@ger.gmane.org> Message-ID: <CAGZAPF41OYEaj6pVWCRv6FmFirGKLiKnKWozwX7AqY5X6Y7Zqw@mail.gmail.com> >> def changed(channel): >> newstate = GPIO.input(channel) >> change = oldstates[channel] and not newstate >> oldstates[channel] = newstate Hi Bill, I would suggest documenting the intent of this function, as it isn't obvious at first. A comment string would be appropriate. I would also suggest that the function should be named "isTurnedOn" or "isTurnedOff", as it is returning True only under a specific transition: when the old state of the channel is True, and the new state of the channel is False. It doesn't check the transition going the other direction, from False to True. In contrast, I think of the word "changed" to mean both directions for the transition. To use more technical terms: the "XOR" boolean operator is what I was expecting to see in the definition of changed(), and I was surprised when didn't see it. :P From davea at davea.name Mon Oct 27 01:55:10 2014 From: davea at davea.name (Dave Angel) Date: Sun, 26 Oct 2014 20:55:10 -0400 (EDT) Subject: [Tutor] Python Questions Help References: <CAHy4Z5kkBjP1AL9JeZT_8vRC+s2Ov3b0Ngw5GU7UP5gouuhc6g@mail.gmail.com> Message-ID: <m2k4ug$55o$1@ger.gmane.org> Please don't use html mail in a text newsgroup. And especially don't select black on black for your foreground and background colors. Caroline H <carolineharlow0 at gmail.com> Wrote in message: > create a new sorted list consisting of all the elements of lst1 that also appears in lst2 . Are you permitted at this stage of your class to use sets? If so, study their methods, and see if you can come up with a oneliner. If you cannot use sets, please post your code again in a text message and I'll try to see what the problem is. It can't be what your words say, since a list allows any number of dups. You also might need to establish some constraints on your input data. For example what are you supposed to do if either or both of the original lists has duplicates? -- DaveA From crk at godblessthe.us Mon Oct 27 04:17:34 2014 From: crk at godblessthe.us (Clayton Kirkwood) Date: Sun, 26 Oct 2014 20:17:34 -0700 Subject: [Tutor] if you're interested in the code thus far... In-Reply-To: <m2k3ti$lpq$1@ger.gmane.org> References: <015d01cff0a5$856d0be0$904723a0$@us> <m2idqb$7nf$1@ger.gmane.org> <022301cff169$ecd2ece0$c678c6a0$@us> <m2k3ti$lpq$1@ger.gmane.org> Message-ID: <024501cff194$8f5c2bf0$ae1483d0$@us> !-----Original Message----- !From: Tutor [mailto:tutor-bounces+crk=godblessthe.us at python.org] On !Behalf Of Alan Gauld !Sent: Sunday, October 26, 2014 5:34 PM !To: tutor at python.org !Subject: Re: [Tutor] if you're interested in the code thus far... ! !On 26/10/14 22:12, Clayton Kirkwood wrote: ! !> !On 25/10/14 23:46, Clayton Kirkwood wrote: !> !> __author__ = 'SYSTEM' !> ! !> !You are still setting __author__ which is a bit suspect. !> !Leave double underscores to python. !> !> This is something being created when I started with this file. ! !How are you creating your code? Are you using some kind of IDE? ! I snarfed it up from somewhere. Right now, it's where it's easy to access and I don't have to worry about it. The code was instructional and it gives me something to refer to and be real proud of:<}} !> !> raw_table = (''' !> ! !> !> a: Ask y: Dividend Yield !> !> b: Bid d: Dividend per Share !> !> b2: Ask (Realtime) r1: Dividend Pay Date !> !> b3: Bid (Realtime) q: Ex-Dividend Date !> !... !> !> s7: Short Ratio !> !> ''') !> !> col_position, code, description = 0, [], [] key_name = !> !> raw_table.replace('\t','\n') ! !> !I assume this is because you don't control the file format? Since !> !otherwise you would just use newlines in the file, right? !> !> Correct. I cut and pasted the data. Not going to change (probably) no !> sense in putting it in a file ! !Are you saying the data will stay with your code? If that's the case !then reformat the text into a Python data structure and avoid all the !parsing that makes up about half your code. ! !> !> for each_line in key_name.splitlines(): !> !> if ':' in each_line: !> !> c, d = each_line.split(':') !> !> code.append(c) !> !> description.append(d.strip()) !> !> print( col_position, code[col_position], !> !description[col_position]) !> !> col_position += 1 !> ! !> !You could use enumerate in the for loop and that would set the !> !col_position value for you: !> ! !... !> !> However, I don't think that this solution will work for me, because !> there are lines with no : ! !Ah yes, I didn't notice you only increment the counter inside the if. ! !> !> output_line_len = 120 !> !> current_output_pos = index = 0 !> !> description_output_string = code_output_string = '' !> !> for description_position, code_position in zip(description, code): !> ! !> !Why not just put the codes and descriptions in tuples when you read !> them !in the loop above? Why use zip? In other words where you do ! !> ! > c, d = each_line.split(':') !> ! > code.append(c) !> ! > description.append(d.strip()) !> ! !> !Why not just join the pair there: !> ! !> ! > c, d = each_line.split(':') !> ! values.append((c,d)) !> ! !> !or even just !> ! !> ! > values.append(each_line.split(':')) !> ! !> !> Uh, because I didn't know about it....:<))) !> !> !It seems as if you are splitting the values into two lists only to !> zip !those lists together again in the next loop? !> ! !... !> !> This has led to a problem, however. How do I determine if a value is !> in one of the tuples: !> Key in [a, word], [key, word] ! !I'm not sure exactly what you mean, I'd probably need to see how you !implemented it but, using my example above, you can do ! !key in values[index] ! !So if values looks like: ! !values = [(1,2),(2,3),....] ! !then the test ! !if 2 in values[0]: will be true ! !and ! !if 2 in values[1]: will also be true. ! !If you need to test the specific tuple item you could do: ! !if 2 == values[0][0]: will be false because you only test 1st item ! !and ! !if 2 == values[0][1]: will be true coz you test the 2nd item. ! !If you wanted to extract all the tuples containing 2 you could use a !comprehension like: ! !value2 = [tup for tup in values if 2 in tup] ! get_new_list = True print(values) #[('a', 'Ask'), ('y', 'Dividend Yield')] while get_new_list: key_list = input('Please enter space separated keys in the order you want: ').split() print(key_list) #['a', 'y'] for key in key_list: print(key) #a if key not in values[0]: #I am trying to ensure that 'a' is in the first 'column' of values print("Error:", key, "not available, start again") get_new_list = True break else: get_new_list = False I don't know how to check if 'key' is in the first position of values. Is it straightforward to check each loop, or should I pull the first part of the tuple out before the loop? Clayton ! !HTH !-- !Alan G !Author of the Learn to Program web site !http://www.alan-g.me.uk/ !http://www.flickr.com/photos/alangauldphotos ! !_______________________________________________ !Tutor maillist - Tutor at python.org !To unsubscribe or change subscription options: !https://mail.python.org/mailman/listinfo/tutor From alan.gauld at btinternet.com Mon Oct 27 10:20:48 2014 From: alan.gauld at btinternet.com (Alan Gauld) Date: Mon, 27 Oct 2014 09:20:48 +0000 Subject: [Tutor] if you're interested in the code thus far... In-Reply-To: <024501cff194$8f5c2bf0$ae1483d0$@us> References: <015d01cff0a5$856d0be0$904723a0$@us> <m2idqb$7nf$1@ger.gmane.org> <022301cff169$ecd2ece0$c678c6a0$@us> <m2k3ti$lpq$1@ger.gmane.org> <024501cff194$8f5c2bf0$ae1483d0$@us> Message-ID: <m2l2pg$219$1@ger.gmane.org> On 27/10/14 03:17, Clayton Kirkwood wrote: > get_new_list = True > print(values) #[('a', 'Ask'), ('y', 'Dividend Yield')] > while get_new_list: > key_list = input('Please enter space separated keys in the order you > want: ').split() > print(key_list) #['a', 'y'] > for key in key_list: > print(key) #a > if key not in values[0]: #I am trying to ensure that 'a' is > in the first 'column' of values Are you only interested in the first pair of values or if its in any pair? I'll assume the latter. You could do: [pair for pair in values if key in pair] Or if you want to be sure you are testing the first element of the pair: [pair for pair in values if key == pair[0]] That will return a list of pairs containing your key. Hopefully that will usually be only one pair... You can then test for an empty list in you if statement like: if not [pair for pair in values if key in pair]: > print("Error:", key, "not available, start again") > get_new_list = True > break > else: get_new_list = False If you want to process the pairs containing the key then you could store the comprehension result in a variable, say key_pairs or somesuch. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.flickr.com/photos/alangauldphotos From __peter__ at web.de Mon Oct 27 10:39:46 2014 From: __peter__ at web.de (Peter Otten) Date: Mon, 27 Oct 2014 10:39:46 +0100 Subject: [Tutor] if you're interested in the code thus far... References: <015d01cff0a5$856d0be0$904723a0$@us> <m2ijpv$tm7$1@ger.gmane.org> <01ec01cff153$a99c3500$fcd49f00$@us> Message-ID: <m2l3t5$k1a$1@ger.gmane.org> Clayton Kirkwood wrote: > Good eyes on the logic > > > !-----Original Message----- > !From: Tutor [mailto:tutor-bounces+crk=godblessthe.us at python.org] On > !Behalf Of Peter Otten > !Sent: Sunday, October 26, 2014 3:53 AM > !To: tutor at python.org > !Subject: Re: [Tutor] if you're interested in the code thus far... > ! > !Clayton Kirkwood wrote: > ! > !> for description_position, code_position in zip(description, code): > !> description_position_len = len(description_position) > !> current_output_pos += description_position_len > !> if current_output_pos >= output_line_len: > !> print(description_output_string) > !> print(code_output_string) > !> code_output_string=description_output_string='' > !> current_output_pos=-1 #start new line > > Changing this line to current_ouput_pos = description_position_len > I believe I was losing the next lines final field on each subsequent line There's no need to guess. Run the code with a small test dataset and compare the actual output with manually determined output. To make this approach feasible it is better to put the formatting algorithm into a function rather than to print() the lines directly. Here's a sketch: def format_output(pairs, width): ... # your code building lines from the pairs expected = [ "abc|de|", "x |y |", "1234567|" "z |" ] got = format_output([ ("abc", "x"), ("de", "y"), ("1234567", "z"), ], 9) if got != expected: print("expected", expected) print("got", got) raise AssertionError Once you start doing this for all your code you should have a look at https://docs.python.org/dev/library/unittest.html From hanzer at riseup.net Mon Oct 27 19:24:45 2014 From: hanzer at riseup.net (Adam Jensen) Date: Mon, 27 Oct 2014 14:24:45 -0400 Subject: [Tutor] subprocess.Popen basics Message-ID: <544E8DED.5090600@riseup.net> Hi, I'm exploring Popen today and I seem to be having some trouble deciphering the [documentation][1]. [1]: docs.python.org/3/library/subprocess.html#popen-constructor In this example (below), I expect to start a shell script as a separate process, send a line of text through a pipe (to the shell script's stdin) then read the shell script's response (a line of text) from another pipe (attached to the shell script's stdout). --------------------------------------------------------- #!/usr/bin/env python3.4 import subprocess as sp parrot = sp.Popen(['./parrot.sh'], stdin=sp.PIPE, stdout=sp.PIPE, stderr=sp.PIPE, universal_newlines=True, start_new_session=True) parrot.stdin.write('Pushing up daisies.\n') print('Parrot said: ', parrot.stdout.readline()) parrot.kill() --------------------------------------------------------- Where "parrot.sh" is: --------------------------------------------------------- #!/bin/bash while true do read foo echo "Squawk: $foo" done --------------------------------------------------------- It hangs at the print statement and, from the sound of the fans in the computer, I suspect it spirals off into an infinite loop somewhere / somehow. Does anyone have any ideas about what it is that I might be misunderstanding? From david at pythontoo.com Mon Oct 27 20:40:08 2014 From: david at pythontoo.com (David Abbott) Date: Mon, 27 Oct 2014 15:40:08 -0400 Subject: [Tutor] subprocess.Popen basics In-Reply-To: <544E8DED.5090600@riseup.net> References: <544E8DED.5090600@riseup.net> Message-ID: <CACs9S6Z8_j_bsppnM+pNa3U8Ekwjq+ktKMdDM8b3qLm0xP-wTw@mail.gmail.com> > It hangs at the print statement and, from the sound of the fans in the > computer, I suspect it spirals off into an infinite loop somewhere / > somehow. Does anyone have any ideas about what it is that I might be > misunderstanding? Works here. david at heater ~/python_practice $ ./subprocess_pipe.py Parrot said: Squawk: Pushing up daisies. david at heater ~/python_practice $ python Python 3.3.5 (default, Oct 2 2014, 07:55:01) [GCC 4.7.3] on linux Type "help", "copyright", "credits" or "license" for more information. >>> quit() -- David Abbott From david at pythontoo.com Mon Oct 27 20:41:55 2014 From: david at pythontoo.com (David Abbott) Date: Mon, 27 Oct 2014 15:41:55 -0400 Subject: [Tutor] subprocess.Popen basics In-Reply-To: <CACs9S6Z8_j_bsppnM+pNa3U8Ekwjq+ktKMdDM8b3qLm0xP-wTw@mail.gmail.com> References: <544E8DED.5090600@riseup.net> <CACs9S6Z8_j_bsppnM+pNa3U8Ekwjq+ktKMdDM8b3qLm0xP-wTw@mail.gmail.com> Message-ID: <CACs9S6bPbmbhWL5wTip-FvDvD+cuH6qfhk4inmtrett9KFyJpw@mail.gmail.com> I did do this also; david at heater ~/python_practice $ chmod a+x parrot.sh david at heater ~/python_practice $ chmod a+x subprocess_pipe.py From alan.gauld at btinternet.com Tue Oct 28 00:12:45 2014 From: alan.gauld at btinternet.com (Alan Gauld) Date: Mon, 27 Oct 2014 23:12:45 +0000 Subject: [Tutor] subprocess.Popen basics In-Reply-To: <544E8DED.5090600@riseup.net> References: <544E8DED.5090600@riseup.net> Message-ID: <m2mjhd$gfh$1@ger.gmane.org> On 27/10/14 18:24, Adam Jensen wrote: > It hangs at the print statement and, from the sound of the fans in the > computer, I suspect it spirals off into an infinite loop somewhere It works fine on my Lubuntu 14 with Python3.4. How exactly are you running it? If I don't make parrot.sh executable I get a permission error. But definitely no hang-ups. Are you using an IDE by any chance? That can often do strange things. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.flickr.com/photos/alangauldphotos From hanzer at riseup.net Mon Oct 27 21:26:22 2014 From: hanzer at riseup.net (Adam Jensen) Date: Mon, 27 Oct 2014 16:26:22 -0400 Subject: [Tutor] subprocess.Popen basics In-Reply-To: <CACs9S6Z8_j_bsppnM+pNa3U8Ekwjq+ktKMdDM8b3qLm0xP-wTw@mail.gmail.com> References: <544E8DED.5090600@riseup.net> <CACs9S6Z8_j_bsppnM+pNa3U8Ekwjq+ktKMdDM8b3qLm0xP-wTw@mail.gmail.com> Message-ID: <544EAA6E.3000605@riseup.net> On 10/27/2014 03:40 PM, David Abbott wrote: >> It hangs at the print statement and, from the sound of the fans in the >> computer, I suspect it spirals off into an infinite loop somewhere / >> somehow. Does anyone have any ideas about what it is that I might be >> misunderstanding? > > Works here. > > david at heater ~/python_practice $ ./subprocess_pipe.py > Parrot said: Squawk: Pushing up daisies. > > david at heater ~/python_practice $ python > Python 3.3.5 (default, Oct 2 2014, 07:55:01) > [GCC 4.7.3] on linux > Type "help", "copyright", "credits" or "license" for more information. >>>> quit() > That's a bit bizarre. I too have the execution bit set for both the python script and the shell script but the same (no joy) behavior occurs on both: Python 3.4.1 (default, Jun 20 2014, 01:51:12) [GCC 4.4.7 20120313 (Red Hat 4.4.7-4)] on linux Python 3.4.1 |Anaconda 2.1.0 (32-bit)| (default, Sep 10 2014, 17:21:42) [GCC 4.4.7 20120313 (Red Hat 4.4.7-1)] on linux $ ./subprocess_pipe.py # It hangs here until Ctrl-c ^CTraceback (most recent call last): File "./subprocess_pipe.py", line 10, in <module> print('Parrot said: ', parrot.stdout.readline()) KeyboardInterrupt From hanzer at riseup.net Mon Oct 27 22:26:27 2014 From: hanzer at riseup.net (Adam Jensen) Date: Mon, 27 Oct 2014 17:26:27 -0400 Subject: [Tutor] subprocess.Popen basics In-Reply-To: <CACs9S6Z8_j_bsppnM+pNa3U8Ekwjq+ktKMdDM8b3qLm0xP-wTw@mail.gmail.com> References: <544E8DED.5090600@riseup.net> <CACs9S6Z8_j_bsppnM+pNa3U8Ekwjq+ktKMdDM8b3qLm0xP-wTw@mail.gmail.com> Message-ID: <544EB883.9060605@riseup.net> On 10/27/2014 03:40 PM, David Abbott wrote: >> It hangs at the print statement and, from the sound of the fans in the >> computer, I suspect it spirals off into an infinite loop somewhere / >> somehow. Does anyone have any ideas about what it is that I might be >> misunderstanding? > > Works here. > > david at heater ~/python_practice $ ./subprocess_pipe.py > Parrot said: Squawk: Pushing up daisies. > > david at heater ~/python_practice $ python > Python 3.3.5 (default, Oct 2 2014, 07:55:01) > [GCC 4.7.3] on linux > Type "help", "copyright", "credits" or "license" for more information. >>>> quit() > It works on OpenBSD. Python 3.3.2 (default, Mar 4 2014, 09:37:25) [GCC 4.2.1 20070719 ] on openbsd5 $ ./subprocess_pipe.py Parrot said: Squawk: Pushing up daisies. Thanks for the sanity check, David. I guess there is something wrong with my python3 build on CentOS. From hanzer at riseup.net Tue Oct 28 02:05:50 2014 From: hanzer at riseup.net (Adam Jensen) Date: Mon, 27 Oct 2014 21:05:50 -0400 Subject: [Tutor] subprocess.Popen basics In-Reply-To: <m2mjhd$gfh$1@ger.gmane.org> References: <544E8DED.5090600@riseup.net> <m2mjhd$gfh$1@ger.gmane.org> Message-ID: <544EEBEE.7040406@riseup.net> On 10/27/2014 07:12 PM, Alan Gauld wrote: > On 27/10/14 18:24, Adam Jensen wrote: >> It hangs at the print statement and, from the sound of the fans in the >> computer, I suspect it spirals off into an infinite loop somewhere > > It works fine on my Lubuntu 14 with Python3.4. > > How exactly are you running it? If I don't make parrot.sh executable > I get a permission error. But definitely no hang-ups. > > Are you using an IDE by any chance? That can often do strange things. > Thanks for giving it a try. The mailing list moderation delay is making an interactive conversation a bit difficult. My current theory is that there is something wrong with the python3 installation on my CentOS-6.5 machine. I'm rebuilding from source now and I will definitely have a close look at the build logs this time and, of course, the test suite logs. When writing scripts, I've been using gedit or vi (depending on the machine I'm using), or I interact directly with the python interpreter (through a terminal). I've tinkered with IPython and Spyder but for now (while learning) they mostly just seem to introduce an unnecessary layer of complexity and obfuscation. Summary: The python and shell scripts that I posted seem to be okay (in the sense that they do what I expected them to do). I got them to run on OpenBSD without any problems. I don't know why my python3.4 build on CentOS6.5 is so wonky. After getting a clean python build from source (sometime tonight or tomorrow), hopefully the problem will go away. If not, I guess it will become a bug hunt. From alan.gauld at btinternet.com Tue Oct 28 02:27:06 2014 From: alan.gauld at btinternet.com (Alan Gauld) Date: Tue, 28 Oct 2014 01:27:06 +0000 Subject: [Tutor] subprocess.Popen basics In-Reply-To: <544EEBEE.7040406@riseup.net> References: <544E8DED.5090600@riseup.net> <m2mjhd$gfh$1@ger.gmane.org> <544EEBEE.7040406@riseup.net> Message-ID: <m2mrdb$fgk$1@ger.gmane.org> On 28/10/14 01:05, Adam Jensen wrote: > Thanks for giving it a try. The mailing list moderation delay is making > an interactive conversation a bit difficult. Well, that much I can help with, you are now off moderation :-) -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.flickr.com/photos/alangauldphotos From alan.gauld at btinternet.com Tue Oct 28 02:31:34 2014 From: alan.gauld at btinternet.com (Alan Gauld) Date: Tue, 28 Oct 2014 01:31:34 +0000 Subject: [Tutor] subprocess.Popen basics In-Reply-To: <544EAA6E.3000605@riseup.net> References: <544E8DED.5090600@riseup.net> <CACs9S6Z8_j_bsppnM+pNa3U8Ekwjq+ktKMdDM8b3qLm0xP-wTw@mail.gmail.com> <544EAA6E.3000605@riseup.net> Message-ID: <m2mrlm$jv6$1@ger.gmane.org> On 27/10/14 20:26, Adam Jensen wrote: > That's a bit bizarre. I too have the execution bit set for both the > python script and the shell script but the same (no joy) behavior occurs > on both: > $ ./subprocess_pipe.py Its a long shot but try explicitly invoking the interpreter: $ python3 ./subprocess_pipe.py The shebang line should make in unnecessary but you never know... HTH -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.flickr.com/photos/alangauldphotos From hanzer at riseup.net Tue Oct 28 02:50:39 2014 From: hanzer at riseup.net (Adam Jensen) Date: Mon, 27 Oct 2014 21:50:39 -0400 Subject: [Tutor] subprocess.Popen basics In-Reply-To: <m2mrlm$jv6$1@ger.gmane.org> References: <544E8DED.5090600@riseup.net> <CACs9S6Z8_j_bsppnM+pNa3U8Ekwjq+ktKMdDM8b3qLm0xP-wTw@mail.gmail.com> <544EAA6E.3000605@riseup.net> <m2mrlm$jv6$1@ger.gmane.org> Message-ID: <544EF66F.7060604@riseup.net> On 10/27/2014 09:31 PM, Alan Gauld wrote: > On 27/10/14 20:26, Adam Jensen wrote: > >> That's a bit bizarre. I too have the execution bit set for both the >> python script and the shell script but the same (no joy) behavior occurs >> on both: > >> $ ./subprocess_pipe.py > > Its a long shot but try explicitly invoking the interpreter: > > $ python3 ./subprocess_pipe.py > > The shebang line should make in unnecessary but you > never know... > What's weird is that I have two different python3.4 installations on this CentOS-6.5 machine and both have the same behavior (script hangs until Ctrl+C). I built this one (/opt/bin/python3.4) from source: $ /opt/bin/python3.4 subprocess_pipe.py ^CTraceback (most recent call last): File "subprocess_pipe.py", line 10, in <module> print('Parrot said: ', parrot.stdout.readline()) KeyboardInterrupt But this one (~/anaconda3/bin/python3.4) was a binary distribution (if I recall correctly): $ ~/anaconda3/bin/python3.4 subprocess_pipe.py ^CTraceback (most recent call last): File "subprocess_pipe.py", line 10, in <module> print('Parrot said: ', parrot.stdout.readline()) KeyboardInterrupt From hanzer at riseup.net Tue Oct 28 16:31:13 2014 From: hanzer at riseup.net (Adam Jensen) Date: Tue, 28 Oct 2014 11:31:13 -0400 Subject: [Tutor] subprocess.Popen basics In-Reply-To: <544EF66F.7060604@riseup.net> References: <544E8DED.5090600@riseup.net> <CACs9S6Z8_j_bsppnM+pNa3U8Ekwjq+ktKMdDM8b3qLm0xP-wTw@mail.gmail.com> <544EAA6E.3000605@riseup.net> <m2mrlm$jv6$1@ger.gmane.org> <544EF66F.7060604@riseup.net> Message-ID: <544FB6C1.7040904@riseup.net> Update: On 10/27/2014 09:50 PM, Adam Jensen wrote: > What's weird is that I have two different python3.4 installations on > this CentOS-6.5 machine and both have the same behavior (script hangs > until Ctrl+C). > > I built this one (/opt/bin/python3.4) from source: ... > But this one (~/anaconda3/bin/python3.4) was a binary distribution (if I > recall correctly): This ^ undermines the "build problem" theory. I found a little blurb in the documentation about bufsize: ------------------------------------------------------------------------- bufsize will be supplied as the corresponding argument to the open() function when creating the stdin/stdout/stderr pipe file objects: * 0 means unbuffered (read and write are one system call and can return short) * 1 means line buffered (only usable if universal_newlines=True i.e., in a text mode) * any other positive value means use a buffer of approximately that size * negative bufsize (the default) means the system default of io.DEFAULT_BUFFER_SIZE will be used. ------------------------------------------------------------------------- Some tinkering: ------------------------------------------------------------------------- #!/usr/bin/env python3.4 import subprocess as sp parrot = sp.Popen(['./parrot.sh'], bufsize=0, stdin=sp.PIPE, stdout=sp.PIPE, stderr=sp.PIPE, universal_newlines=True, start_new_session=True) parrot.stdin.write('Pushing up daisies.\n') print('Parrot said: ', parrot.stdout.readline()) parrot.kill() ------------------------------------------------------------------------- And I get these results (on CentOS-6.5-x86): | bufsize | results | |---------+---------| | default | hangs | | -1 | hangs | | 0 | works | | 1 | hangs | | >=2 | works | From alan.gauld at btinternet.com Tue Oct 28 19:32:47 2014 From: alan.gauld at btinternet.com (Alan Gauld) Date: Tue, 28 Oct 2014 18:32:47 +0000 Subject: [Tutor] subprocess.Popen basics In-Reply-To: <544FB6C1.7040904@riseup.net> References: <544E8DED.5090600@riseup.net> <CACs9S6Z8_j_bsppnM+pNa3U8Ekwjq+ktKMdDM8b3qLm0xP-wTw@mail.gmail.com> <544EAA6E.3000605@riseup.net> <m2mrlm$jv6$1@ger.gmane.org> <544EF66F.7060604@riseup.net> <544FB6C1.7040904@riseup.net> Message-ID: <m2ongg$s36$1@ger.gmane.org> On 28/10/14 15:31, Adam Jensen wrote: > ------------------------------------------------------------------------- > bufsize will be supplied as the corresponding argument to the open() > function when creating the stdin/stdout/stderr pipe file objects: > And I get these results (on CentOS-6.5-x86): > > | bufsize | results | > |---------+---------| > | default | hangs | > | -1 | hangs | > | 0 | works | > | 1 | hangs | > | >=2 | works | I tried -1 and 1 on my Lubuntu and it still works fine. Definitely weird, it begins to look like a CentOS build issue but what is CentOS doing different to Lubuntu/Suse/OpenBSD etc? From memory CentOS is basically a free version of Red Hat Enterprise Linux? I can't see why that would be significantly different in stdin/out behaviour. weirder and weirder -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.flickr.com/photos/alangauldphotos From hanzer at riseup.net Tue Oct 28 20:23:40 2014 From: hanzer at riseup.net (Adam Jensen) Date: Tue, 28 Oct 2014 15:23:40 -0400 Subject: [Tutor] subprocess.Popen basics In-Reply-To: <m2ongg$s36$1@ger.gmane.org> References: <544E8DED.5090600@riseup.net> <CACs9S6Z8_j_bsppnM+pNa3U8Ekwjq+ktKMdDM8b3qLm0xP-wTw@mail.gmail.com> <544EAA6E.3000605@riseup.net> <m2mrlm$jv6$1@ger.gmane.org> <544EF66F.7060604@riseup.net> <544FB6C1.7040904@riseup.net> <m2ongg$s36$1@ger.gmane.org> Message-ID: <544FED3C.2040200@riseup.net> On 10/28/2014 02:32 PM, Alan Gauld wrote: > I tried -1 and 1 on my Lubuntu and it still works fine. > Definitely weird, it begins to look like a CentOS build issue > but what is CentOS doing different to Lubuntu/Suse/OpenBSD etc? > > From memory CentOS is basically a free version of Red Hat > Enterprise Linux? I can't see why that would be significantly > different in stdin/out behaviour. > > weirder and weirder Yes, CentOS is a RHEL clone. Scientific-Linux is another RHEL clone (assembled and maintained by several of the national labs). I'm fairly certain that Python is in heavy use at the labs so it seems especially weird that there is something so wonky going on with it on that platform. This is what I've discovered so far: | | CentOS-6.5 | OpenBSD-5.5 | DragonFly-3.8.2 | | bufsize | Python-3.4.1 | Python-3.3.2 | Python-3.3.3 | |---------------------+--------------+--------------+-----------------| | default | hangs | works | works | | -1 | hangs | works | works | | 0 | works | works | works | | 1 | hangs | works | works | | >=2 & <(# of bytes) | works | works | works | | >=(# of bytes) | hangs | works | works | (The number of bytes in 'Pushing up daisies.\n' is 20). io.DEFAULT_BUFFER_SIZE == 8192 on all three platforms. From alan.gauld at btinternet.com Tue Oct 28 21:04:37 2014 From: alan.gauld at btinternet.com (Alan Gauld) Date: Tue, 28 Oct 2014 20:04:37 +0000 Subject: [Tutor] subprocess.Popen basics In-Reply-To: <544FED3C.2040200@riseup.net> References: <544E8DED.5090600@riseup.net> <CACs9S6Z8_j_bsppnM+pNa3U8Ekwjq+ktKMdDM8b3qLm0xP-wTw@mail.gmail.com> <544EAA6E.3000605@riseup.net> <m2mrlm$jv6$1@ger.gmane.org> <544EF66F.7060604@riseup.net> <544FB6C1.7040904@riseup.net> <m2ongg$s36$1@ger.gmane.org> <544FED3C.2040200@riseup.net> Message-ID: <m2ossl$l0g$1@ger.gmane.org> On 28/10/14 19:23, Adam Jensen wrote: > platform. This is what I've discovered so far: > > | | CentOS-6.5 | OpenBSD-5.5 | DragonFly-3.8.2 | > | bufsize | Python-3.4.1 | Python-3.3.2 | Python-3.3.3 | > |---------------------+--------------+--------------+-----------------| > | default | hangs | works | works | > | -1 | hangs | works | works | > | 0 | works | works | works | > | 1 | hangs | works | works | > | >=2 & <(# of bytes) | works | works | works | > | >=(# of bytes) | hangs | works | works | > Looks like you might be best trying this on a CentOS forum/list. Its not exactly a mainstream distro so the number of folks on this list who can help is probably limited and even on the main Python list there probably are only a handful of CentOS users. And it looks increasingly like a CentOS configuration issue. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.flickr.com/photos/alangauldphotos From norrist at gmail.com Tue Oct 28 21:27:28 2014 From: norrist at gmail.com (Todd) Date: Tue, 28 Oct 2014 15:27:28 -0500 Subject: [Tutor] subprocess.Popen basics In-Reply-To: <m2ossl$l0g$1@ger.gmane.org> References: <544E8DED.5090600@riseup.net> <CACs9S6Z8_j_bsppnM+pNa3U8Ekwjq+ktKMdDM8b3qLm0xP-wTw@mail.gmail.com> <544EAA6E.3000605@riseup.net> <m2mrlm$jv6$1@ger.gmane.org> <544EF66F.7060604@riseup.net> <544FB6C1.7040904@riseup.net> <m2ongg$s36$1@ger.gmane.org> <544FED3C.2040200@riseup.net> <m2ossl$l0g$1@ger.gmane.org> Message-ID: <CAL=nqr6249JwhgFqqf7kTncehnfbh1QS2sHLVMZepnc4HYsLaw@mail.gmail.com> Centos has SELinux enabled by default. I dont know if SELinux is causing your problem, but it is always worth looking at. SELinux can keep a process from accessing files or executing another process. Try temporarily disabling SELinux by running setenforce=0 as root. Then see if python does what you expect. On Tue, Oct 28, 2014 at 3:04 PM, Alan Gauld <alan.gauld at btinternet.com> wrote: > On 28/10/14 19:23, Adam Jensen wrote: > > platform. This is what I've discovered so far: >> >> | | CentOS-6.5 | OpenBSD-5.5 | DragonFly-3.8.2 | >> | bufsize | Python-3.4.1 | Python-3.3.2 | Python-3.3.3 | >> |---------------------+--------------+--------------+-----------------| >> | default | hangs | works | works | >> | -1 | hangs | works | works | >> | 0 | works | works | works | >> | 1 | hangs | works | works | >> | >=2 & <(# of bytes) | works | works | works | >> | >=(# of bytes) | hangs | works | works | >> >> > Looks like you might be best trying this on a CentOS forum/list. > Its not exactly a mainstream distro so the number of folks on this list > who can help is probably limited and even on the main Python list there > probably are only a handful of CentOS users. And it looks increasingly like > a CentOS configuration issue. > > -- > Alan G > Author of the Learn to Program web site > http://www.alan-g.me.uk/ > http://www.flickr.com/photos/alangauldphotos > > _______________________________________________ > Tutor maillist - Tutor at python.org > To unsubscribe or change subscription options: > https://mail.python.org/mailman/listinfo/tutor > -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/tutor/attachments/20141028/7ebaea3d/attachment.html> From hanzer at riseup.net Tue Oct 28 21:34:08 2014 From: hanzer at riseup.net (Adam Jensen) Date: Tue, 28 Oct 2014 16:34:08 -0400 Subject: [Tutor] subprocess.Popen basics In-Reply-To: <CAL=nqr6249JwhgFqqf7kTncehnfbh1QS2sHLVMZepnc4HYsLaw@mail.gmail.com> References: <544E8DED.5090600@riseup.net> <CACs9S6Z8_j_bsppnM+pNa3U8Ekwjq+ktKMdDM8b3qLm0xP-wTw@mail.gmail.com> <544EAA6E.3000605@riseup.net> <m2mrlm$jv6$1@ger.gmane.org> <544EF66F.7060604@riseup.net> <544FB6C1.7040904@riseup.net> <m2ongg$s36$1@ger.gmane.org> <544FED3C.2040200@riseup.net> <m2ossl$l0g$1@ger.gmane.org> <CAL=nqr6249JwhgFqqf7kTncehnfbh1QS2sHLVMZepnc4HYsLaw@mail.gmail.com> Message-ID: <544FFDC0.7040207@riseup.net> On 10/28/2014 04:27 PM, Todd wrote: > Centos has SELinux enabled by default. I dont know if SELinux is > causing your problem, but it is always worth looking at. > > SELinux can keep a process from accessing files or executing another > process. > > Try temporarily disabling SELinux by running setenforce=0 as root. > Then see if python does what you expect. Yep, that occurred to me too. Earlier today I set 'SELINUX=disabled' in /etc/selinux/config and did a 'sudo touch /.autorelabel' then rebooted. No Joy, same behavior from subprocess.Popen(). From crk at godblessthe.us Wed Oct 29 00:13:19 2014 From: crk at godblessthe.us (Clayton Kirkwood) Date: Tue, 28 Oct 2014 16:13:19 -0700 Subject: [Tutor] Would somebody kindly... Message-ID: <052701cff304$c48b5c80$4da21580$@us> Explain this double speak(>: [pair for pair in values if key == pair[0]] I understand the 'for pair in values'. I assume the first 'pair' creates the namespace (although I am not sure how Python knows it's a tuple yet). I think the outer [] make the line a comprehension ( If so, I don't seem to get how/why a comprehension works), and probably make the line either a tuple or list. The comprehension seems to act as a subroutine or macro( yes I know those two words are meaningless in Python). I definitely am missing the full meaning of the if statement. The example comes courtesy of Alan. Thanks, Clayton -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/tutor/attachments/20141028/a9d4ff3f/attachment.html> From dyoo at hashcollision.org Wed Oct 29 00:23:14 2014 From: dyoo at hashcollision.org (Danny Yoo) Date: Tue, 28 Oct 2014 16:23:14 -0700 Subject: [Tutor] Would somebody kindly... In-Reply-To: <052701cff304$c48b5c80$4da21580$@us> References: <052701cff304$c48b5c80$4da21580$@us> Message-ID: <CAGZAPF5inn1_CZ76sf1Z+sPUwJuRLgE0RK2ALFY6pU5PftKqDg@mail.gmail.com> On Tue, Oct 28, 2014 at 4:13 PM, Clayton Kirkwood <crk at godblessthe.us> wrote: > Explain this double speak(>: > > [pair for pair in values if key == pair[0]] Hi Clayton, Here is a rewording of that expression to an approximately equivalent statement: ####################### some_list = [] for pair in values: if key == pair[0]: some_list.append(pair) ####################### From alan.gauld at btinternet.com Wed Oct 29 00:37:09 2014 From: alan.gauld at btinternet.com (Alan Gauld) Date: Tue, 28 Oct 2014 23:37:09 +0000 Subject: [Tutor] Would somebody kindly... In-Reply-To: <052701cff304$c48b5c80$4da21580$@us> References: <052701cff304$c48b5c80$4da21580$@us> Message-ID: <m2p9b5$5jo$1@ger.gmane.org> On 28/10/14 23:13, Clayton Kirkwood wrote: > Explain this double speak(>: > > [pair for pair in values if key == pair[0]] A list comprehension is a specific form of a more general construction called a generator expression. Its specific in that it creates a list, hence the [] around it. The general shape of a generator expression is: <result expression> for value in <iterator> if <condition> and the comprehension version looks like [ <result expression> for value in <iterator> if <condition> ] The if condition part is optional in both cases. This is syntactically equivalent to aList = [] for value in iterator: if condition: aList.append(result expression) Let's look at a specific example using literals: [item for item in [1,2,3,4,5] if item > 3] The effect of which is the list [4,5] So it acts as a filter that extracts the items in the inner list that are greater than 3 (the if condition). In your case I had proposed using a list of tuples(pairs) like so: values = [(1,2),(2,3),(3,4),(4,5)] So the expression: [pair for pair in values if 2 in pair] will return [(1,2),(2,3)] and the expression [pair for pair in values if 2 == pair[0]] will return [(2,3)] - the only pair with 2 in the first position Now, coming back to your original question: [pair for pair in values if key == pair[0]] this returns the tuples that have the first element equal to key - whatever the variable key happens to be. PS. For bonus points you can research how generator expressions can encapsulate nested loops, but that's probably a step too far for just now! HTH -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.flickr.com/photos/alangauldphotos From alan.gauld at btinternet.com Wed Oct 29 00:41:00 2014 From: alan.gauld at btinternet.com (Alan Gauld) Date: Tue, 28 Oct 2014 23:41:00 +0000 Subject: [Tutor] Would somebody kindly... In-Reply-To: <052701cff304$c48b5c80$4da21580$@us> References: <052701cff304$c48b5c80$4da21580$@us> Message-ID: <m2p9ic$9al$1@ger.gmane.org> On 28/10/14 23:13, Clayton Kirkwood wrote: > line either a tuple or list. The comprehension seems to act as a > subroutine or macro In a sense yes, that's true. You can find another explanation of list comprehensions and other related functions in my tutorial (see .sig) in the Functional Programming topic. It has several more examples. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.flickr.com/photos/alangauldphotos From crk at godblessthe.us Wed Oct 29 02:02:21 2014 From: crk at godblessthe.us (Clayton Kirkwood) Date: Tue, 28 Oct 2014 18:02:21 -0700 Subject: [Tutor] Would somebody kindly... In-Reply-To: <CAGZAPF5inn1_CZ76sf1Z+sPUwJuRLgE0RK2ALFY6pU5PftKqDg@mail.gmail.com> References: <052701cff304$c48b5c80$4da21580$@us> <CAGZAPF5inn1_CZ76sf1Z+sPUwJuRLgE0RK2ALFY6pU5PftKqDg@mail.gmail.com> Message-ID: <054d01cff314$00c83510$02589f30$@us> Ah, Alan sent an answer also, but this one answers the last tidbit. Alan had the some_list and pair the same name, presumably creating a temporary tuple and when the loop is done, the temporary replaces the original. Thanks Clayton !-----Original Message----- !From: Danny Yoo [mailto:dyoo at hashcollision.org] !Sent: Tuesday, October 28, 2014 4:23 PM !To: Clayton Kirkwood !Cc: Python Tutor Mailing List !Subject: Re: [Tutor] Would somebody kindly... ! !On Tue, Oct 28, 2014 at 4:13 PM, Clayton Kirkwood <crk at godblessthe.us> !wrote: !> Explain this double speak(>: !> !> [pair for pair in values if key == pair[0]] ! !Hi Clayton, ! !Here is a rewording of that expression to an approximately equivalent !statement: ! !####################### !some_list = [] !for pair in values: ! if key == pair[0]: ! some_list.append(pair) !####################### From alan.gauld at btinternet.com Wed Oct 29 02:14:39 2014 From: alan.gauld at btinternet.com (Alan Gauld) Date: Wed, 29 Oct 2014 01:14:39 +0000 Subject: [Tutor] Would somebody kindly... In-Reply-To: <054d01cff314$00c83510$02589f30$@us> References: <052701cff304$c48b5c80$4da21580$@us> <CAGZAPF5inn1_CZ76sf1Z+sPUwJuRLgE0RK2ALFY6pU5PftKqDg@mail.gmail.com> <054d01cff314$00c83510$02589f30$@us> Message-ID: <m2pf20$t3l$1@ger.gmane.org> On 29/10/14 01:02, Clayton Kirkwood wrote: > Ah, Alan sent an answer also, but this one answers the last tidbit. > Alan had the some_list and pair the same name, presumably creating > a temporary tuple and when the loop is done, the temporary > replaces the original. Nope. If you re-read my post you'll seee that my examples are just expressions, they don;t assign the result. Dannys code is actually the equivalent of: some_list = [pair for pair in values if key == pair[0]] some_list is an arbitrary name Danny used to represent the list that is built up by the comprehension. It doesn't actually have a name in the real comprehension. The name 'pair' in my code is used exactly as in Danny's 'for' loop below. > !####################### > !some_list = [] > !for pair in values: > ! if key == pair[0]: > ! some_list.append(pair) > !####################### There is no temporary tuples being created the list is built up inside the comprehension as the for part executes in the same way as in the expanded version. There is no magic here it's just a short-cut notation for the expanded loop (and it usually executes a bit faster). -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.flickr.com/photos/alangauldphotos From davea at davea.name Wed Oct 29 02:34:30 2014 From: davea at davea.name (Dave Angel) Date: Tue, 28 Oct 2014 21:34:30 -0400 (EDT) Subject: [Tutor] Would somebody kindly... References: <052701cff304$c48b5c80$4da21580$@us> Message-ID: <m2pg03$b0d$1@ger.gmane.org> > Explain this double speak(>: > [pair for pair in values if key == pair[0]] > I understand the ?for pair in values?. I assume the first > ?pair? creates the namespace The namespace question depends on the version of Python. Python 2.x does not do any scoping. But in version 3.x, the variable pair will go away. So please tell us the version you're asking about. The list comprehension always creates a list, not a tuple. Pair is a tuple if and only if the corresponding element of values is a tuple. In fact there's no requirement that those elements of values have the same type. Of course if they're not of length 2, then pair is a lousy name. And if one of them is not subscriptable, then the if expression will blow up. -- DaveA From cs at zip.com.au Wed Oct 29 02:31:14 2014 From: cs at zip.com.au (Cameron Simpson) Date: Wed, 29 Oct 2014 12:31:14 +1100 Subject: [Tutor] Would somebody kindly... In-Reply-To: <054d01cff314$00c83510$02589f30$@us> References: <054d01cff314$00c83510$02589f30$@us> Message-ID: <20141029013114.GA14604@cskk.homeip.net> On 28Oct2014 18:02, Clayton Kirkwood <crk at godblessthe.us> wrote: >!> Explain this double speak(>: >!> >Ah, Alan sent an answer also, but this one answers the last tidbit. Alan had the some_list and pair the same name, presumably creating a temporary tuple and when the loop is done, the temporary replaces the original. Let me try a less wordy diagram. You will need to be displaying in a constant width font :-) [ pair for pair in values if key == pair[0] ] ^^^^-- the expression that accrues in the resulting list ^^^^-- the loop variable, taken from the loop source values ^^^^^^-- the loop source values ^^^^^^^^^^^^^^-- condition for including the expression in the resulting list So that first "pair" could be any expression, it is almost only coincidence that it is the same as the loop variable. It is the same in this case because this is the idiomatic way to select particular values form an existing list. If we'd wanted the new list to contain double the original values we'd write: [ pair*2 for pair in values if key == pair[0] ] Cheers, Cameron Simpson <cs at zip.com.au> ERROR 155 - You can't do that. - Data General S200 Fortran error code list From crk at godblessthe.us Wed Oct 29 05:33:10 2014 From: crk at godblessthe.us (Clayton Kirkwood) Date: Tue, 28 Oct 2014 21:33:10 -0700 Subject: [Tutor] Would somebody kindly... In-Reply-To: <20141029013114.GA14604@cskk.homeip.net> References: <054d01cff314$00c83510$02589f30$@us> <20141029013114.GA14604@cskk.homeip.net> Message-ID: <056001cff331$72e4ad50$58ae07f0$@us> !-----Original Message----- !From: Cameron Simpson [mailto:cs at zip.com.au] !Sent: Tuesday, October 28, 2014 6:31 PM !To: Clayton Kirkwood !Cc: 'Danny Yoo'; 'Python Tutor Mailing List' !Subject: Re: [Tutor] Would somebody kindly... ! !On 28Oct2014 18:02, Clayton Kirkwood <crk at godblessthe.us> wrote: !>!> Explain this double speak(>: !>!> !>Ah, Alan sent an answer also, but this one answers the last tidbit. !Alan had the some_list and pair the same name, presumably creating a !temporary tuple and when the loop is done, the temporary replaces the !original. ! !Let me try a less wordy diagram. You will need to be displaying in a !constant width font :-) ! ! [ pair for pair in values if key == pair[0] ] ! ^^^^-- the expression that accrues in the resulting list ! ^^^^-- the loop variable, taken from the loop source !values ! ^^^^^^-- the loop source values ! ^^^^^^^^^^^^^^-- condition for including !the ! expression in the resulting !list ! !So that first "pair" could be any expression, it is almost only !coincidence that it is the same as the loop variable. It is the same in !this case because this is the idiomatic way to select particular values !form an existing list. ! !If we'd wanted the new list to contain double the original values we'd !write: ! ! [ pair*2 for pair in values if key == pair[0] ] Ok, I am somewhat confused again. In the original example up above, it appears that the pair list or tuple gets overridden. In this one right above, once again, the list gets overwritten again, but what is being doubled?, each part of the tuple? Clayton ! !Cheers, !Cameron Simpson <cs at zip.com.au> ! !ERROR 155 - You can't do that. - Data General S200 Fortran error code !list From crk at godblessthe.us Wed Oct 29 05:42:14 2014 From: crk at godblessthe.us (Clayton Kirkwood) Date: Tue, 28 Oct 2014 21:42:14 -0700 Subject: [Tutor] Would somebody kindly... In-Reply-To: <m2pg03$b0d$1@ger.gmane.org> References: <052701cff304$c48b5c80$4da21580$@us> <m2pg03$b0d$1@ger.gmane.org> Message-ID: <056101cff332$b8180510$28480f30$@us> !-----Original Message----- !From: Tutor [mailto:tutor-bounces+crk=godblessthe.us at python.org] On !Behalf Of Dave Angel !Sent: Tuesday, October 28, 2014 6:34 PM !To: tutor at python.org !Subject: Re: [Tutor] Would somebody kindly... ! ! !> ! Explain this double speak(>: !> [pair for pair in values if key == pair[0]] ! !> I understand the ?for pair in values?. I assume the first ?pair? !> creates the namespace ! !The namespace question depends on the version of Python. Python 2.x !does not do any scoping. ! !But in version 3.x, the variable pair will go away. ! !So please tell us the version you're asking about. I am using 3.4.1. ! !The list comprehension always creates a list, not a tuple. Pair is a !tuple if and only if the corresponding element of values is a tuple. In !fact there's no requirement that those elements of values have the same !type. Of course if they're not of length 2, then pair is a lousy name. !And if one of them is not subscriptable, then the if expression will !blow up. ! ! !-- !DaveA ! !_______________________________________________ !Tutor maillist - Tutor at python.org !To unsubscribe or change subscription options: !https://mail.python.org/mailman/listinfo/tutor From cs at zip.com.au Wed Oct 29 07:58:51 2014 From: cs at zip.com.au (Cameron Simpson) Date: Wed, 29 Oct 2014 17:58:51 +1100 Subject: [Tutor] Would somebody kindly... In-Reply-To: <056001cff331$72e4ad50$58ae07f0$@us> References: <056001cff331$72e4ad50$58ae07f0$@us> Message-ID: <20141029065851.GA59331@cskk.homeip.net> On 28Oct2014 21:33, Clayton Kirkwood <crk at godblessthe.us> wrote: >!From: Cameron Simpson [mailto:cs at zip.com.au] >!Let me try a less wordy diagram. You will need to be displaying in a >!constant width font :-) >! >! [ pair for pair in values if key == pair[0] ] >! ^^^^-- the expression that accrues in the resulting list >! ^^^^-- the loop variable, taken from the loop source values >! ^^^^^^-- the loop source values >! ^^^^^^^^^^^^^^-- condition for including the >! expression in the resulting list >! >!So that first "pair" could be any expression, it is almost only >!coincidence that it is the same as the loop variable. It is the same in >!this case because this is the idiomatic way to select particular values >!form an existing list. >! >!If we'd wanted the new list to contain double the original values we'd >!write: >! >! [ pair*2 for pair in values if key == pair[0] ] > >Ok, I am somewhat confused again. In the original example up above, it >appears that the pair list or tuple gets overridden. In this one right >above, once again, the list gets overwritten again, but what is being >doubled?, each part of the tuple? First up, this makes a list, not a tuple. (Hence the outermost [] instead of (). No, there is not a tuple equivalenti:-) So, to get things straight: - values is a list of pairs (2-tuples) - the list comprehension above creates a new list consisting the first expression computed for each pair in "values" matching the condition expression And I had not noticed that "pair" was a 2-tuple. So "pair*2" is not a sensible example. The point is that the first "pair" (or bad example "2*pair") is an arbitrary expression, used to compute each value in the new list. Cheers, Cameron Simpson <cs at zip.com.au> Don't have awk? Use this simple sh emulation: #!/bin/sh echo 'Awk bailing out!' >&2 exit 2 - Tom Horsley <tahorsley at csd.harris.com> From cs at zip.com.au Wed Oct 29 08:34:04 2014 From: cs at zip.com.au (Cameron Simpson) Date: Wed, 29 Oct 2014 18:34:04 +1100 Subject: [Tutor] Would somebody kindly... In-Reply-To: <056001cff331$72e4ad50$58ae07f0$@us> References: <056001cff331$72e4ad50$58ae07f0$@us> Message-ID: <20141029073404.GA75498@cskk.homeip.net> On 28Oct2014 21:33, Clayton Kirkwood <crk at godblessthe.us> wrote: >!If we'd wanted the new list to contain double the original values we'd >!write: >! >! [ pair*2 for pair in values if key == pair[0] ] > >Ok, I am somewhat confused again. In the original example up above, it >appears that the pair list or tuple gets overridden. In this one right >above, once again, the list gets overwritten again, but what is being >doubled?, each part of the tuple? A different try. Sometimes it helsp to see a concrete example. Do this at the command line: Python 3.4.2 (default, Oct 9 2014, 11:02:09) [GCC 4.2.1 Compatible Apple LLVM 5.1 (clang-503.0.40)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> values = [ (1,2), (3,4) ] >>> values [(1, 2), (3, 4)] >>> result = [ pair for pair in values if pair[0] == 3 ] >>> result [(3, 4)] >>> result = [ pair[1] for pair in values if pair[0] == 3 ] >>> result [4] You're thinking, I think, that "pair" a tuple which is overwritten with a copy of each tuple from the list in turn. It is not. The list comprehension above is equivalnt to a for loop starting: for pair in values: During this loop, the _name_ "pair" becomes a direct reference to each tuple from the list "values". No tuples are copied or changed. So if the list looks like this: values --> [ (1, 2), (3, 4) ] During the first loop iteration, "pair" gets bound to the first tuple, like this: values --> [ pair -----> (1, 2), (3, 4) ] See? One list, two tuples, with "values" bound to the list, and "pair" bound to the first tuple. On the second loop iteration, "pair" is bound to the second tuple (the arrow moves). No copying. The result of the list comprehension is the first expression (eg "pair[1]") evaluated for each iteration (where the "if" expression is true). So, the expression in the examples above is true only for the second tuple. So the expression "pair[1]" is only used for that tuple. And so the final result is a single element list with a "4", taken from the second tuple. Maybe this makes things clearer. Cheers, Cameron Simpson <cs at zip.com.au> Draw little boxes with arrows. It helps. - Michael J. Eager From alan.gauld at btinternet.com Wed Oct 29 10:49:59 2014 From: alan.gauld at btinternet.com (Alan Gauld) Date: Wed, 29 Oct 2014 09:49:59 +0000 Subject: [Tutor] Would somebody kindly... In-Reply-To: <056001cff331$72e4ad50$58ae07f0$@us> References: <054d01cff314$00c83510$02589f30$@us> <20141029013114.GA14604@cskk.homeip.net> <056001cff331$72e4ad50$58ae07f0$@us> Message-ID: <m2qd87$464$1@ger.gmane.org> On 29/10/14 04:33, Clayton Kirkwood wrote: > !If we'd wanted the new list to contain double the original values we'd > !write: > ! > ! [ pair*2 for pair in values if key == pair[0] ] > > Ok, I am somewhat confused again. In the original example up above, it > appears that the pair list or tuple gets overridden. The point Cameron is making is that the first expression is completely arbitrary it does not even need to contain the variable used in the for loop. Here is an example of how to create a random number for each matching pair: from random import random ... random_list = [random() for pair in values if pair[0] == 2] Notice the resultant list will contain one random number for each pair found that has 2 as its first element. Again the equivalent code: random_list = [] for pair in values: if pair[0] == 2: random_list.append(random()) Another example, this time using numbers and a more complex expression. Let's say we want the square root of all the even numbers in a list: from math import sqrt roots = [sqrt(n) for n in [1,3,5,4,6,9,7,8,34,67,22] if n % 2 == 0] yields: [2.0, 2.449, 2.828, 5.830, 4.690] So the expression that is put in the result is not necessarily the same as the value in the initial for loop, and in fact may be completely unrelated. It just happens to be a very common usage that we want to extract a sub set of the values of a list. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.flickr.com/photos/alangauldphotos From steve at pearwood.info Wed Oct 29 13:04:18 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Wed, 29 Oct 2014 23:04:18 +1100 Subject: [Tutor] Would somebody kindly... In-Reply-To: <052701cff304$c48b5c80$4da21580$@us> References: <052701cff304$c48b5c80$4da21580$@us> Message-ID: <20141029120417.GA26866@ando.pearwood.info> On Tue, Oct 28, 2014 at 04:13:19PM -0700, Clayton Kirkwood wrote: > Explain this double speak(>: > > [pair for pair in values if key == pair[0]] Translated to a regular for-loop: result = [] for pair in values: if key == pair[0]: result.append(pair) It iterates over the sequence `values`, extracting something called `pair`. If the first item of `pair` == key, then the pair is placed in the resulting list. py> values = [ ('a', 1), ('b', 2), ('a', 5), ('c', 7)] py> key = 'a' py> [pair for pair in values if key == pair[0]] [('a', 1), ('a', 5)] -- Steven From davea at davea.name Wed Oct 29 13:29:56 2014 From: davea at davea.name (Dave Angel) Date: Wed, 29 Oct 2014 08:29:56 -0400 (EDT) Subject: [Tutor] Would somebody kindly... References: <052701cff304$c48b5c80$4da21580$@us> <m2pg03$b0d$1@ger.gmane.org> <056101cff332$b8180510$28480f30$@us> Message-ID: <m2qmd2$ji$1@ger.gmane.org> "Clayton Kirkwood" <crk at godblessthe.us> Wrote in message: > > > !-----Original Message----- > !From: Tutor [mailto:tutor-bounces+crk=godblessthe.us at python.org] On > !Behalf Of Dave Angel > !Sent: Tuesday, October 28, 2014 6:34 PM > !To: tutor at python.org > !Subject: Re: [Tutor] Would somebody kindly... > ! > ! > !> > ! Explain this double speak(>: > !> [pair for pair in values if key == pair[0]] > ! > !> I understand the ?for pair in values?. I assume the first ?pair? > !> creates the namespace > ! > !The namespace question depends on the version of Python. Python 2.x > !does not do any scoping. > ! > !But in version 3.x, the variable pair will go away. > ! > !So please tell us the version you're asking about. > > I am using 3.4.1. > Have you somehow configured your email program to use exclamation points for quoting instead of the standard greater-than symbol? "!" instead of ">" ? If so, do you mind changing it back? In 3.4.1, let's consider the following code. thingie = 2 mylist = [(2,55), "charlie", [2, "item2", 12]] x = [78 for item in mylist if item[0] == thingie] What will happen in the list comprehension, and what will be the final value of x ? First an anonymous list object will be created. This eventually will be bound to x, but not till the comprehension is successfully completed. Next a locally scoped variable item is created. This goes away at the end of the comprehension, regardless of how we exit. Next the 0th value from mylist is bound to item. It happens to be a tuple, but not from anything the comprehension decides. Next the expression item [0] == thingie is evaluated. If it's true, then the int 78 is appended to the anonymous list. Now the previous group of actions is repeated for the 1th value of mylist. So now item is a string, and the zeroth character of the string is compared with the int 2. Not equal, so 72 doesn't get appended. Similarly for the 2th item. The first element of that list is equal to 2, so another 72 is appended. Now the anonymous list is bound to x. print (x) [72, 72] -- DaveA From jarod_v6 at libero.it Wed Oct 29 17:08:28 2014 From: jarod_v6 at libero.it (jarod_v6 at libero.it) Date: Wed, 29 Oct 2014 17:08:28 +0100 (CET) Subject: [Tutor] question on array Operation Message-ID: <700633221.1075021414598908994.JavaMail.defaultUser@defaultHost> Dear All, I have a long matrix where I have the samples (1to n) and then I have (1to j )elements. I would like to count how many times each j element are present on each samples. So the question is which is the best algoritm for obtain the results. The result I want is a table like that. A,B,D,E,F AA,1,0,1,0 BB1,1,1,1 CC0,0,1,0 DD01,0,1,0 Do you have any suggestion on how to do this? only using a loop like: A is a list with some names A=[AA,BB,ZZ,TT,NN] if AA in A: print 1 else : print 0 thanks in advance for any help -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/tutor/attachments/20141029/ca8bdc9c/attachment.html> From __peter__ at web.de Wed Oct 29 17:49:06 2014 From: __peter__ at web.de (Peter Otten) Date: Wed, 29 Oct 2014 17:49:06 +0100 Subject: [Tutor] question on array Operation References: <700633221.1075021414598908994.JavaMail.defaultUser@defaultHost> Message-ID: <m2r5q2$ibm$1@ger.gmane.org> jarod_v6 at libero.it wrote: > Dear All, > > I have a long matrix where I have the samples (1to n) and then I have (1to > j )elements. > I would like to count how many times each j element are present on each > samples. > > So the question is which is the best algoritm for obtain the results. The > result I want is a table like that. > > > > A,B,D,E,F > AA,1,0,1,0 > BB1,1,1,1 > CC0,0,1,0 > DD01,0,1,0 > > > Do you have any suggestion on how to do this? only using a loop like: > > A is a list with some names A=[AA,BB,ZZ,TT,NN] > > if AA in A: > print 1 > else : > print 0 > > thanks in advance for any help I'm sorry I have no idea what you are trying to do. I can't believe that this is your "best effort" to explain the problem, so I won't bother to guess. From alan.gauld at btinternet.com Wed Oct 29 18:29:12 2014 From: alan.gauld at btinternet.com (Alan Gauld) Date: Wed, 29 Oct 2014 17:29:12 +0000 Subject: [Tutor] question on array Operation In-Reply-To: <700633221.1075021414598908994.JavaMail.defaultUser@defaultHost> References: <700633221.1075021414598908994.JavaMail.defaultUser@defaultHost> Message-ID: <m2r858$2d5$1@ger.gmane.org> On 29/10/14 16:08, jarod_v6 at libero.it wrote: > I have a long matrix where I have the samples (1to n) and then I have > (1to j )elements. > I would like to count how many times each j element are present on each > samples. That probably makes sense to you but it doesn't to me. Lets assuyme you have n=10 and j=5. That would suggest you have a 10x5 matrix of values? Now what do you mean by 'each j' element? The last one in the row? So for this small example: 1,2,3,4,5, -> count = 1 - there is only one 5 1,2,1,2,1, -> count = 3 - there are 3 ones A,3,B,2,A -> count = 2 - there are 2 As Is that what you want? > So the question is which is the best algoritm for obtain the results. > The result I want is a table like that. > > A,B,D,E,F > AA,1,0,1,0 > BB1,1,1,1 > CC0,0,1,0 > DD01,0,1,0 That doesn';t senm to match your description above. You will need to show us both input and output data before we can understand the transformation you are looking for. > Do you have any suggestion on how to do this? Not until I understand what you want. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.flickr.com/photos/alangauldphotos From steve at pearwood.info Wed Oct 29 23:06:29 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Thu, 30 Oct 2014 09:06:29 +1100 Subject: [Tutor] question on array Operation In-Reply-To: <700633221.1075021414598908994.JavaMail.defaultUser@defaultHost> References: <700633221.1075021414598908994.JavaMail.defaultUser@defaultHost> Message-ID: <20141029220629.GB26866@ando.pearwood.info> On Wed, Oct 29, 2014 at 05:08:28PM +0100, jarod_v6 at libero.it wrote: > Dear All, > > I have a long matrix where I have the samples (1to n) and then I have (1to j )elements. > I would like to count how many times each j element are present on each samples. Jarod, looking at your email address, I am guessing that English is not your native language. I'm afraid that I cannot understand what you are trying to do. Please show a *simple* example. I will make a *guess* that you have something like this: matrix = [ # three samples of seven values each [2, 4, 6, 8, 6, 9, 5], [3, 5, 1, 7, 9, 8, 8], [1, 2, 0, 6, 6, 2, 1], ] and then you want to count how many each element [0, 1, 2, 3, 4, 5, 6, 7, 8, 9] appear in each sample: from collections import Counter for i, sample in enumerate(matrix, 1): c = Counter(sample) print("Sample %d" % i) print(c) which gives me: Sample 1 Counter({6: 2, 2: 1, 4: 1, 5: 1, 8: 1, 9: 1}) Sample 2 Counter({8: 2, 1: 1, 3: 1, 5: 1, 7: 1, 9: 1}) Sample 3 Counter({1: 2, 2: 2, 6: 2, 0: 1}) Does that help? If not, you have to explain what you need better. -- Steven From crk at godblessthe.us Thu Oct 30 06:39:53 2014 From: crk at godblessthe.us (Clayton Kirkwood) Date: Wed, 29 Oct 2014 22:39:53 -0700 Subject: [Tutor] Would somebody kindly... In-Reply-To: <m2qmd2$ji$1@ger.gmane.org> References: <052701cff304$c48b5c80$4da21580$@us> <m2pg03$b0d$1@ger.gmane.org> <056101cff332$b8180510$28480f30$@us> <m2qmd2$ji$1@ger.gmane.org> Message-ID: <06b801cff403$eed61340$cc8239c0$@us> >-----Original Message----- >From: Tutor [mailto:tutor-bounces+crk=godblessthe.us at python.org] On >Behalf Of Dave Angel >Sent: Wednesday, October 29, 2014 5:30 AM >To: tutor at python.org >Subject: Re: [Tutor] Would somebody kindly... > >"Clayton Kirkwood" <crk at godblessthe.us> Wrote in message: >> >> >> !-----Original Message----- >> !From: Tutor [mailto:tutor-bounces+crk=godblessthe.us at python.org] On >> !Behalf Of Dave Angel >> !Sent: Tuesday, October 28, 2014 6:34 PM >> !To: tutor at python.org >> !Subject: Re: [Tutor] Would somebody kindly... >> ! >> ! >> !> >> ! Explain this double speak(>: >> !> [pair for pair in values if key == pair[0]] ! >> !> I understand the ?for pair in values?. I assume the first ?pair? >> !> creates the namespace >> ! >> !The namespace question depends on the version of Python. Python 2.x >> !does not do any scoping. >> ! >> !But in version 3.x, the variable pair will go away. >> ! >> !So please tell us the version you're asking about. >> >> I am using 3.4.1. >> > >Have you somehow configured your email program to use exclamation >points for quoting instead of the standard greater-than symbol? > "!" instead of ">" ? If so, do you mind changing it back? > >In 3.4.1, let's consider the following code. > >thingie = 2 >mylist = [(2,55), "charlie", [2, "item2", 12]] x = [78 for item in >mylist if item[0] == thingie] > >What will happen in the list comprehension, and what will be the final >value of x ? > >First an anonymous list object will be created. This eventually will >be bound to x, but not till the comprehension is successfully >completed. Next a locally scoped variable item is created. This goes >away at the end of the comprehension, regardless of how we exit. > >Next the 0th value from mylist is bound to item. It happens to be a >tuple, but not from anything the comprehension decides. >Next the expression item [0] == thingie is evaluated. If it's true, >then the int 78 is appended to the anonymous list. > >Now the previous group of actions is repeated for the 1th value of >mylist. So now item is a string, and the zeroth character of the string >is compared with the int 2. Not equal, so 72 doesn't get appended. > >Similarly for the 2th item. The first element of that list is equal to >2, so another 72 is appended. > >Now the anonymous list is bound to x. > >print (x) >[72, 72] So, in this case, the assignment to x is external. Often I don't see an external assignment, but there is an item in the first position within the comprehension. You don't have that here. When you have [item for item in [list] if item[0] == key], after the iteration completes does item equal the matched entities or does it have the original item? I understand that if we had x = [dsfasdfasdf] x will be a list (never a tuple?) with the matches, but what happens to the first item? This is from a previous email-- When I run: values = [ ('a', 1), ('b', 2), ('a', 5), ('c', 7)] key = 'a' pair=[] [pair for pair in values if key == pair[0]] print(pair) I get []. When I run: values = [ ('a', 1), ('b', 2), ('a', 5), ('c', 7)] key = 'a' pair=[] x=[pair for pair in values if key == pair[0]] print(x) I get [('a', 1), ('a', 5)] So, what does that first pair do? I see and have used the first comprehension. Clayton > > > >-- >DaveA > >_______________________________________________ >Tutor maillist - Tutor at python.org >To unsubscribe or change subscription options: >https://mail.python.org/mailman/listinfo/tutor From marc.tompkins at gmail.com Thu Oct 30 08:10:53 2014 From: marc.tompkins at gmail.com (Marc Tompkins) Date: Thu, 30 Oct 2014 00:10:53 -0700 Subject: [Tutor] Would somebody kindly... In-Reply-To: <06b801cff403$eed61340$cc8239c0$@us> References: <052701cff304$c48b5c80$4da21580$@us> <m2pg03$b0d$1@ger.gmane.org> <056101cff332$b8180510$28480f30$@us> <m2qmd2$ji$1@ger.gmane.org> <06b801cff403$eed61340$cc8239c0$@us> Message-ID: <CAKK8jXau6efFYzAS86D-Fc3aaU1sagWtdUYkXUqwCrFkewNnJA@mail.gmail.com> On Wed, Oct 29, 2014 at 10:39 PM, Clayton Kirkwood <crk at godblessthe.us> wrote: > When I run: > values = [ ('a', 1), ('b', 2), ('a', 5), ('c', 7)] > key = 'a' > pair=[] > You just created an empty list and called it "pair". > [pair for pair in values if key == pair[0]] > Two things to bear in mind here: - The stepper variable in a list comprehension lives in a distinct namespace. To the human reader, the list called "pair" outside the comprehension and the object called "pair" inside look identical - but Bad Things would happen if Python were as easily confused as humans, so it isn't. To Python, they are no more alike than two humans who happen to have the same first name. - As soon as the list comprehension stops executing - from the reader's perspective, as soon as you read past the rightmost square bracket - it, and its stepper variable, disappear. You didn't assign the list that the comprehension generated _to_ anything, so it evaporates. (Of course, nothing is really gone until it's garbage collected - but since you don't have any control over that, consider it gone.) print(pair) > > I get []. > As expected - it's the same empty list you started out with, and you didn't do anything to it. > > When I run: > values = [ ('a', 1), ('b', 2), ('a', 5), ('c', 7)] > key = 'a' > pair=[] > You're still not doing anything with that list... > x=[pair for pair in values if key == pair[0]] > This time, you ARE doing something with the list that the comprehension generates - you're naming it "x". > print(x) > And now you're printing it. > > I get [('a', 1), ('a', 5)] > Again, as expected. I can't help thinking that part of the difficulty here is that you're using object names that aren't meaningful to Python, but that seem to be meaningful to _you_. "pair" is just a name; it doesn't mean anything to Python. Neither do "key" or "values" - but they SEEM meaningful, and they're kinda close to actual Python keywords. This is a bad and dangerous practice! Try changing it up: > whatsits = [ ('a', 1), ('b', 2), ('a', 5), ('c', 7)] > thingy = 'a' > twofer=[] > x=[twofer for twofer in whatsits if thingy == twofer[0]] Obviously you needn't make your variable names as silly as that, but - for your own sake - don't make them official-sounding either. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/tutor/attachments/20141030/020a41db/attachment.html> From martin at linux-ip.net Thu Oct 30 08:05:15 2014 From: martin at linux-ip.net (Martin A. Brown) Date: Thu, 30 Oct 2014 00:05:15 -0700 Subject: [Tutor] Would somebody kindly... In-Reply-To: <06b801cff403$eed61340$cc8239c0$@us> References: <052701cff304$c48b5c80$4da21580$@us> <m2pg03$b0d$1@ger.gmane.org> <056101cff332$b8180510$28480f30$@us> <m2qmd2$ji$1@ger.gmane.org> <06b801cff403$eed61340$cc8239c0$@us> Message-ID: <alpine.LNX.2.00.1410292324470.1349@dagger.wonderfrog.net> Hi there Clayton, > values = [ ('a', 1), ('b', 2), ('a', 5), ('c', 7)] > key = 'a' > pair=[] # -- this assignment is unnecessary > x=[pair for pair in values if key == pair[0]] > print(x) > > I get [('a', 1), ('a', 5)] I also get that result. Good. > So, what does that first pair do? I see and have used the first comprehension. I'll point out that somebody (Cameron Simpson, specifically) already gave you a hint that might have helped, but maybe you missed it as you were trying to understand list comprehensions. Let's play a little game....I'm going to assume that the variables values and key are initialized as you have initialized them above. Game #1: Recognize that the name of the variable in the list comprehension is ephemeral. >>> [frobnitz for frobnitz in values if key == frobnitz[0]] [('a', 1), ('a', 5)] Yes, I guess that's obvious now. So, this is why people often use 'x' in these situations. >>> [x for x in values if key == x[0]] [('a', 1), ('a', 5)] The variable will probably contain the final element of the input sequence after the list comprehension terminates. >>> [x for x in values if key == x[0]] [('a', 1), ('a', 5)] >>> x ('c', 7) If the input sequence had no contents, then the variable will be undefined. >>> del x >>> [x for x in []] [] >>> x Traceback (most recent call last): File "<stdin>", line 1, in <module> NameError: name 'x' is not defined ...but, it is safer, I think, always to treat that variable as visible in that one line only. I have never seen this, but can imagine those who are unusually organized or fixated on code cleanliness might remove the variable inline after using it. This is a rare invertebrate--I have never seen it in the gardens I have explored: [x for x in values if key == x[0]] del x Game #2: Wait, what? The variable x is ephemeral? But, I put something important into it! x = "password to my cousin's turnip ranching high voltage override" >>> x "password to my cousin's turnip ranching high voltage override" >>> [x for x in values if key == x[0]] [('a', 1), ('a', 5)] >>> x ('c', 7) By using x in the list comprehension, you are assigning to a variable which may (or may not) have already existed. So, if you expect the stuff that was in the variable (x, here) to be there when you get back from lunch, you'll be sad, because the Python has eaten it just like you told it to. Game #3: Observe that you can manipulate what gets returned in the output expression from the list comprehension. >>> [(x[1], x[0]) for x in values if key == x[0]] [(1, 'a'), (5, 'a')] Hey! We flipped the position of the number and the alphachar in the tuple! That was fun. Game #4: Drop a function into the output expression. Not only can you perform simple sleight of hand like manipulating tuples, but you can perform function calls. So multiply the second element of each tuple by a random number. >>> import random >>> [(x[0], x[1], x[1] * random.randint(0,7000)) for x in values if key == x[0]] [('a', 1, 2656), ('a', 5, 4510)] Good luck, -Martin -- Martin A. Brown http://linux-ip.net/ From davea at davea.name Thu Oct 30 08:38:45 2014 From: davea at davea.name (Dave Angel) Date: Thu, 30 Oct 2014 03:38:45 -0400 (EDT) Subject: [Tutor] Would somebody kindly... References: <052701cff304$c48b5c80$4da21580$@us> <m2pg03$b0d$1@ger.gmane.org> <056101cff332$b8180510$28480f30$@us> <m2qmd2$ji$1@ger.gmane.org> <06b801cff403$eed61340$cc8239c0$@us> <alpine.LNX.2.00.1410292324470.1349@dagger.wonderfrog.net> Message-ID: <m2spmv$6d4$1@ger.gmane.org> "Martin A. Brown" <martin at linux-ip.net> Wrote in message: > > Hi there Clayton, > >> values = [ ('a', 1), ('b', 2), ('a', 5), ('c', 7)] >> key = 'a' >> pair=[] # -- this assignment is unnecessary >> x=[pair for pair in values if key == pair[0]] >> print(x) >> >> I get [('a', 1), ('a', 5)] > > I also get that result. Good. > >> So, what does that first pair do? I see and have used the first comprehension. > > I'll point out that somebody (Cameron Simpson, specifically) already > gave you a hint that might have helped, but maybe you missed it as > you were trying to understand list comprehensions. > > Let's play a little game....I'm going to assume that the variables > values and key are initialized as you have initialized them above. > > > Game #1: Recognize that the name of the variable in the list > comprehension is ephemeral. > > >>> [frobnitz for frobnitz in values if key == frobnitz[0]] > [('a', 1), ('a', 5)] > > Yes, I guess that's obvious now. So, this is why people often use > 'x' in these situations. > > >>> [x for x in values if key == x[0]] > [('a', 1), ('a', 5)] > > The variable will probably contain the final element of the input > sequence after the list comprehension terminates. NO, NO, NO. The OP is using Python 3.4, and has consistently shown results accordingly. x does NOT exist after the list comprehension. That was a flaw in python 2.x which has been fixed. -- DaveA From davea at davea.name Thu Oct 30 08:53:46 2014 From: davea at davea.name (Dave Angel) Date: Thu, 30 Oct 2014 03:53:46 -0400 (EDT) Subject: [Tutor] Would somebody kindly... References: <052701cff304$c48b5c80$4da21580$@us> <m2pg03$b0d$1@ger.gmane.org> <056101cff332$b8180510$28480f30$@us> <m2qmd2$ji$1@ger.gmane.org> <06b801cff403$eed61340$cc8239c0$@us> Message-ID: <m2sqj6$kf2$1@ger.gmane.org> "Clayton Kirkwood" <crk at godblessthe.us> Wrote in message: > > >>-----Original Message----- >>From: Tutor [mailto:tutor-bounces+crk=godblessthe.us at python.org] On >>Behalf Of Dave Angel >>Sent: Wednesday, October 29, 2014 5:30 AM >>To: tutor at python.org >>Subject: Re: [Tutor] Would somebody kindly... >> >>"Clayton Kirkwood" <crk at godblessthe.us> Wrote in message: >>> >>> >>> !-----Original Message----- >>> !From: Tutor [mailto:tutor-bounces+crk=godblessthe.us at python.org] On >>> !Behalf Of Dave Angel >>> !Sent: Tuesday, October 28, 2014 6:34 PM >>> !To: tutor at python.org >>> !Subject: Re: [Tutor] Would somebody kindly... >>> ! >>> ! >>> !> >>> ! Explain this double speak(>: >>> !> [pair for pair in values if key == pair[0]] ! >>> !> I understand the ?for pair in values?. I assume the first ?pair? >>> !> creates the namespace >>> ! >>> !The namespace question depends on the version of Python. Python 2.x >>> !does not do any scoping. >>> ! >>> !But in version 3.x, the variable pair will go away. >>> ! >>> !So please tell us the version you're asking about. >>> >>> I am using 3.4.1. >>> >> >>Have you somehow configured your email program to use exclamation >>points for quoting instead of the standard greater-than symbol? >> "!" instead of ">" ? If so, do you mind changing it back? >> >>In 3.4.1, let's consider the following code. >> >>thingie = 2 >>mylist = [(2,55), "charlie", [2, "item2", 12]] x = [78 for item in >>mylist if item[0] == thingie] >> >>What will happen in the list comprehension, and what will be the final >>value of x ? >> >>First an anonymous list object will be created. This eventually will >>be bound to x, but not till the comprehension is successfully >>completed. Next a locally scoped variable item is created. This goes >>away at the end of the comprehension, regardless of how we exit. >> >>Next the 0th value from mylist is bound to item. It happens to be a >>tuple, but not from anything the comprehension decides. >>Next the expression item [0] == thingie is evaluated. If it's true, >>then the int 78 is appended to the anonymous list. >> >>Now the previous group of actions is repeated for the 1th value of >>mylist. So now item is a string, and the zeroth character of the string >>is compared with the int 2. Not equal, so 72 doesn't get appended. >> >>Similarly for the 2th item. The first element of that list is equal to >>2, so another 72 is appended. >> >>Now the anonymous list is bound to x. >> >>print (x) >>[72, 72] > > So, in this case, the assignment to x is external. Often I don't see an external assignment, but there is an item in the first position within the comprehension. The thing before the 'for' keyword is an expression. It's evaluated once for each piece of the result list. If it happens to be a simple variable, nothing changes. It is not a name to be bound to, just a name to be evaluated. > You don't have that here. When you have [item for item in [list] if item[0] == key], after the iteration completes does item equal the matched entities or does it have the original item? 'item' that was used in the comprehension is gone, and if you happened to have an earlier one of same name, it's visible again. Like happens when a function returns, all the variables of that namespace are gone. > I understand that if we had x = [dsfasdfasdf] x will be a list (never a tuple?) with the matches, but what happens to the first item? > > This is from a previous email-- > When I run: > values = [ ('a', 1), ('b', 2), ('a', 5), ('c', 7)] > key = 'a' > pair=[] > [pair for pair in values if key == pair[0]] > print(pair) > > I get []. Naturally because the two variables are in separate namespaces. > > When I run: > values = [ ('a', 1), ('b', 2), ('a', 5), ('c', 7)] > key = 'a' > pair=[] > x=[pair for pair in values if key == pair[0]] > print(x) > > I get [('a', 1), ('a', 5)] > > So, what does that first pair do? I see and have used the first comprehension. > The first use of the name 'pair' assigned an empty list to it. That's totally unaffected by anything inside your comprehension. -- DaveA From jarod_v6 at libero.it Thu Oct 30 09:32:01 2014 From: jarod_v6 at libero.it (jarod_v6 at libero.it) Date: Thu, 30 Oct 2014 09:32:01 +0100 (CET) Subject: [Tutor] R: re question on array Message-ID: <1903614574.1197511414657921768.JavaMail.defaultUser@defaultHost> Dear All, Sorry for my bad presentation of my problem!! I have this tipe of input: A file with a long liste of gene ad the occurence for sample: gene Samples FUS SampleA TP53 SampleA ATF4 SampleB ATF3 SampleC ATF4 SampleD FUS SampleE RORA SampleE RORA SampleC WHat I want to obtain is amtrix where I have the occurence for sample. SampleA SampleB SampleC SampleD SampleE FUS 1 0 0 0 1 TP53 1 0 0 0 0 ATF4 0 1 1 0 ATF3 0 0 1 0 0 RORA 0 0 1 0 In that way I count count the occurence in fast way! At the moment I only able to do the list of the rownames and the sample names. Unfortunately I don't know how to create this matrix. Cold you help me ? Thanks for the patience and the help >----Messaggio originale---- >Da: tutor-request at python.org >Data: 30/10/2014 6.41 >A: <tutor at python.org> >Ogg: Tutor Digest, Vol 128, Issue 74 > >Send Tutor mailing list submissions to > tutor at python.org > >To subscribe or unsubscribe via the World Wide Web, visit > https://mail.python.org/mailman/listinfo/tutor >or, via email, send a message with subject or body 'help' to > tutor-request at python.org > >You can reach the person managing the list at > tutor-owner at python.org > >When replying, please edit your Subject line so it is more specific >than "Re: Contents of Tutor digest..." > > >Today's Topics: > > 1. Re: Would somebody kindly... (Steven D'Aprano) > 2. Re: Would somebody kindly... (Dave Angel) > 3. question on array Operation (jarod_v6 at libero.it) > 4. Re: question on array Operation (Peter Otten) > 5. Re: question on array Operation (Alan Gauld) > 6. Re: question on array Operation (Steven D'Aprano) > 7. Re: Would somebody kindly... (Clayton Kirkwood) > > >---------------------------------------------------------------------- > >Message: 1 >Date: Wed, 29 Oct 2014 23:04:18 +1100 >From: Steven D'Aprano <steve at pearwood.info> >To: tutor at python.org >Subject: Re: [Tutor] Would somebody kindly... >Message-ID: <20141029120417.GA26866 at ando.pearwood.info> >Content-Type: text/plain; charset=us-ascii > >On Tue, Oct 28, 2014 at 04:13:19PM -0700, Clayton Kirkwood wrote: >> Explain this double speak(>: >> >> [pair for pair in values if key == pair[0]] > >Translated to a regular for-loop: > >result = [] >for pair in values: > if key == pair[0]: > result.append(pair) > > >It iterates over the sequence `values`, extracting something called >`pair`. If the first item of `pair` == key, then the pair is placed in >the resulting list. > >py> values = [ ('a', 1), ('b', 2), ('a', 5), ('c', 7)] >py> key = 'a' >py> [pair for pair in values if key == pair[0]] >[('a', 1), ('a', 5)] > > >-- >Steven > > >------------------------------ > >Message: 2 >Date: Wed, 29 Oct 2014 08:29:56 -0400 (EDT) >From: Dave Angel <davea at davea.name> >To: tutor at python.org >Subject: Re: [Tutor] Would somebody kindly... >Message-ID: <m2qmd2$ji$1 at ger.gmane.org> >Content-Type: text/plain; charset=UTF-8 > >"Clayton Kirkwood" <crk at godblessthe.us> Wrote in message: >> >> >> !-----Original Message----- >> !From: Tutor [mailto:tutor-bounces+crk=godblessthe.us at python.org] On >> !Behalf Of Dave Angel >> !Sent: Tuesday, October 28, 2014 6:34 PM >> !To: tutor at python.org >> !Subject: Re: [Tutor] Would somebody kindly... >> ! >> ! >> !> >> ! Explain this double speak(>: >> !> [pair for pair in values if key == pair[0]] >> ! >> !> I understand the ?for pair in values?. I assume the first ?pair? >> !> creates the namespace >> ! >> !The namespace question depends on the version of Python. Python 2.x >> !does not do any scoping. >> ! >> !But in version 3.x, the variable pair will go away. >> ! >> !So please tell us the version you're asking about. >> >> I am using 3.4.1. >> > >Have you somehow configured your email program to use exclamation > points for quoting instead of the standard greater-than symbol? > "!" instead of ">" ? If so, do you mind changing it > back? > >In 3.4.1, let's consider the following code. > >thingie = 2 >mylist = [(2,55), "charlie", [2, "item2", 12]] >x = [78 for item in mylist if item[0] == thingie] > >What will happen in the list comprehension, and what will be the > final value of x ? > >First an anonymous list object will be created. This eventually > will be bound to x, but not till the comprehension is > successfully completed. Next a locally scoped variable item is > created. This goes away at the end of the comprehension, > regardless of how we exit. > >Next the 0th value from mylist is bound to item. It happens to be > a tuple, but not from anything the comprehension > decides. >Next the expression item [0] == thingie is evaluated. If it's > true, then the int 78 is appended to the anonymous > list. > >Now the previous group of actions is repeated for the 1th value of > mylist. So now item is a string, and the zeroth character of the > string is compared with the int 2. Not equal, so 72 doesn't get > appended. > >Similarly for the 2th item. The first element of that list is > equal to 2, so another 72 is appended. > >Now the anonymous list is bound to x. > >print (x) >[72, 72] > > > >-- >DaveA > > > >------------------------------ > >Message: 3 >Date: Wed, 29 Oct 2014 17:08:28 +0100 (CET) >From: "jarod_v6 at libero.it" <jarod_v6 at libero.it> >To: tutor at python.org >Subject: [Tutor] question on array Operation >Message-ID: > <700633221.1075021414598908994.JavaMail.defaultUser at defaultHost> >Content-Type: text/plain; charset="utf-8" > >Dear All, > >I have a long matrix where I have the samples (1to n) and then I have (1to j ) elements. >I would like to count how many times each j element are present on each samples. > >So the question is which is the best algoritm for obtain the results. The result I want is a table like that. > > > >A,B,D,E,F >AA,1,0,1,0 >BB1,1,1,1 >CC0,0,1,0 >DD01,0,1,0 > > >Do you have any suggestion on how to do this? only using a loop like: > >A is a list with some names A=[AA,BB,ZZ,TT,NN] > >if AA in A: > print 1 >else : > print 0 > >thanks in advance for any help >-------------- next part -------------- >An HTML attachment was scrubbed... >URL: <http://mail.python. org/pipermail/tutor/attachments/20141029/ca8bdc9c/attachment-0001.html> > >------------------------------ > >Message: 4 >Date: Wed, 29 Oct 2014 17:49:06 +0100 >From: Peter Otten <__peter__ at web.de> >To: tutor at python.org >Subject: Re: [Tutor] question on array Operation >Message-ID: <m2r5q2$ibm$1 at ger.gmane.org> >Content-Type: text/plain; charset="ISO-8859-1" > >jarod_v6 at libero.it wrote: > >> Dear All, >> >> I have a long matrix where I have the samples (1to n) and then I have (1to >> j )elements. >> I would like to count how many times each j element are present on each >> samples. >> >> So the question is which is the best algoritm for obtain the results. The >> result I want is a table like that. >> >> >> >> A,B,D,E,F >> AA,1,0,1,0 >> BB1,1,1,1 >> CC0,0,1,0 >> DD01,0,1,0 >> >> >> Do you have any suggestion on how to do this? only using a loop like: >> >> A is a list with some names A=[AA,BB,ZZ,TT,NN] >> >> if AA in A: >> print 1 >> else : >> print 0 >> >> thanks in advance for any help > >I'm sorry I have no idea what you are trying to do. > >I can't believe that this is your "best effort" to explain the problem, so I >won't bother to guess. > > > >------------------------------ > >Message: 5 >Date: Wed, 29 Oct 2014 17:29:12 +0000 >From: Alan Gauld <alan.gauld at btinternet.com> >To: tutor at python.org >Subject: Re: [Tutor] question on array Operation >Message-ID: <m2r858$2d5$1 at ger.gmane.org> >Content-Type: text/plain; charset=windows-1252; format=flowed > >On 29/10/14 16:08, jarod_v6 at libero.it wrote: > >> I have a long matrix where I have the samples (1to n) and then I have >> (1to j )elements. >> I would like to count how many times each j element are present on each >> samples. > >That probably makes sense to you but it doesn't to me. >Lets assuyme you have n=10 and j=5. >That would suggest you have a 10x5 matrix of values? > >Now what do you mean by 'each j' element? The last one in the row? > >So for this small example: > >1,2,3,4,5, -> count = 1 - there is only one 5 >1,2,1,2,1, -> count = 3 - there are 3 ones >A,3,B,2,A -> count = 2 - there are 2 As > >Is that what you want? > >> So the question is which is the best algoritm for obtain the results. >> The result I want is a table like that. >> >> A,B,D,E,F >> AA,1,0,1,0 >> BB1,1,1,1 >> CC0,0,1,0 >> DD01,0,1,0 > >That doesn';t senm to match your description above. You will >need to show us both input and output data before we can >understand the transformation you are looking for. > >> Do you have any suggestion on how to do this? > >Not until I understand what you want. > > >-- >Alan G >Author of the Learn to Program web site >http://www.alan-g.me.uk/ >http://www.flickr.com/photos/alangauldphotos > > > >------------------------------ > >Message: 6 >Date: Thu, 30 Oct 2014 09:06:29 +1100 >From: Steven D'Aprano <steve at pearwood.info> >To: tutor at python.org >Subject: Re: [Tutor] question on array Operation >Message-ID: <20141029220629.GB26866 at ando.pearwood.info> >Content-Type: text/plain; charset=us-ascii > >On Wed, Oct 29, 2014 at 05:08:28PM +0100, jarod_v6 at libero.it wrote: >> Dear All, >> >> I have a long matrix where I have the samples (1to n) and then I have (1to j )elements. >> I would like to count how many times each j element are present on each samples. > >Jarod, looking at your email address, I am guessing that English is not >your native language. I'm afraid that I cannot understand what you are >trying to do. Please show a *simple* example. > >I will make a *guess* that you have something like this: > >matrix = [ # three samples of seven values each > [2, 4, 6, 8, 6, 9, 5], > [3, 5, 1, 7, 9, 8, 8], > [1, 2, 0, 6, 6, 2, 1], > ] > >and then you want to count how many each element [0, 1, 2, 3, 4, 5, 6, >7, 8, 9] appear in each sample: > > >from collections import Counter >for i, sample in enumerate(matrix, 1): > c = Counter(sample) > print("Sample %d" % i) > print(c) > > >which gives me: > >Sample 1 >Counter({6: 2, 2: 1, 4: 1, 5: 1, 8: 1, 9: 1}) >Sample 2 >Counter({8: 2, 1: 1, 3: 1, 5: 1, 7: 1, 9: 1}) >Sample 3 >Counter({1: 2, 2: 2, 6: 2, 0: 1}) > > >Does that help? > >If not, you have to explain what you need better. > > >-- >Steven > > >------------------------------ > >Message: 7 >Date: Wed, 29 Oct 2014 22:39:53 -0700 >From: "Clayton Kirkwood" <crk at godblessthe.us> >To: <tutor at python.org> >Subject: Re: [Tutor] Would somebody kindly... >Message-ID: <06b801cff403$eed61340$cc8239c0$@us> >Content-Type: text/plain; charset="UTF-8" > > > >>-----Original Message----- >>From: Tutor [mailto:tutor-bounces+crk=godblessthe.us at python.org] On >>Behalf Of Dave Angel >>Sent: Wednesday, October 29, 2014 5:30 AM >>To: tutor at python.org >>Subject: Re: [Tutor] Would somebody kindly... >> >>"Clayton Kirkwood" <crk at godblessthe.us> Wrote in message: >>> >>> >>> !-----Original Message----- >>> !From: Tutor [mailto:tutor-bounces+crk=godblessthe.us at python.org] On >>> !Behalf Of Dave Angel >>> !Sent: Tuesday, October 28, 2014 6:34 PM >>> !To: tutor at python.org >>> !Subject: Re: [Tutor] Would somebody kindly... >>> ! >>> ! >>> !> >>> ! Explain this double speak(>: >>> !> [pair for pair in values if key == pair[0]] ! >>> !> I understand the ?for pair in values?. I assume the first ?pair? >>> !> creates the namespace >>> ! >>> !The namespace question depends on the version of Python. Python 2.x >>> !does not do any scoping. >>> ! >>> !But in version 3.x, the variable pair will go away. >>> ! >>> !So please tell us the version you're asking about. >>> >>> I am using 3.4.1. >>> >> >>Have you somehow configured your email program to use exclamation >>points for quoting instead of the standard greater-than symbol? >> "!" instead of ">" ? If so, do you mind changing it back? >> >>In 3.4.1, let's consider the following code. >> >>thingie = 2 >>mylist = [(2,55), "charlie", [2, "item2", 12]] x = [78 for item in >>mylist if item[0] == thingie] >> >>What will happen in the list comprehension, and what will be the final >>value of x ? >> >>First an anonymous list object will be created. This eventually will >>be bound to x, but not till the comprehension is successfully >>completed. Next a locally scoped variable item is created. This goes >>away at the end of the comprehension, regardless of how we exit. >> >>Next the 0th value from mylist is bound to item. It happens to be a >>tuple, but not from anything the comprehension decides. >>Next the expression item [0] == thingie is evaluated. If it's true, >>then the int 78 is appended to the anonymous list. >> >>Now the previous group of actions is repeated for the 1th value of >>mylist. So now item is a string, and the zeroth character of the string >>is compared with the int 2. Not equal, so 72 doesn't get appended. >> >>Similarly for the 2th item. The first element of that list is equal to >>2, so another 72 is appended. >> >>Now the anonymous list is bound to x. >> >>print (x) >>[72, 72] > >So, in this case, the assignment to x is external. Often I don't see an external assignment, but there is an item in the first position within the comprehension. You don't have that here. When you have [item for item in [list] if item[0] == key], after the iteration completes does item equal the matched entities or does it have the original item? I understand that if we had x = [dsfasdfasdf] x will be a list (never a tuple?) with the matches, but what happens to the first item? > >This is from a previous email-- >When I run: >values = [ ('a', 1), ('b', 2), ('a', 5), ('c', 7)] >key = 'a' >pair=[] >[pair for pair in values if key == pair[0]] >print(pair) > >I get []. > >When I run: >values = [ ('a', 1), ('b', 2), ('a', 5), ('c', 7)] >key = 'a' >pair=[] >x=[pair for pair in values if key == pair[0]] >print(x) > >I get [('a', 1), ('a', 5)] > >So, what does that first pair do? I see and have used the first comprehension. > > >Clayton > > >> >> >> >>-- >>DaveA >> >>_______________________________________________ >>Tutor maillist - Tutor at python.org >>To unsubscribe or change subscription options: >>https://mail.python.org/mailman/listinfo/tutor > > > > >------------------------------ > >Subject: Digest Footer > >_______________________________________________ >Tutor maillist - Tutor at python.org >https://mail.python.org/mailman/listinfo/tutor > > >------------------------------ > >End of Tutor Digest, Vol 128, Issue 74 >************************************** > From __peter__ at web.de Thu Oct 30 10:14:27 2014 From: __peter__ at web.de (Peter Otten) Date: Thu, 30 Oct 2014 10:14:27 +0100 Subject: [Tutor] R: re question on array References: <1903614574.1197511414657921768.JavaMail.defaultUser@defaultHost> Message-ID: <m2svhk$46u$1@ger.gmane.org> jarod_v6 at libero.it wrote: > Dear All, > Sorry for my bad presentation of my problem!! > I have this tipe of input: > A file with a long liste of gene ad the occurence for sample: > > gene Samples > FUS SampleA > TP53 SampleA > ATF4 SampleB > ATF3 SampleC > ATF4 SampleD > FUS SampleE > RORA SampleE > RORA SampleC > > WHat I want to obtain is amtrix where I have the occurence for sample. > SampleA SampleB SampleC SampleD SampleE > FUS 1 0 0 0 1 > TP53 1 0 0 0 0 > ATF4 0 1 1 0 > ATF3 0 0 1 0 0 > RORA 0 0 1 0 > > In that way I count count the occurence in fast way! > > At the moment I only able to do the list of the rownames and the sample > names. Unfortunately I don't know how to create this matrix. > Cold you help me ? > Thanks for the patience and the help Open the file, skip the first line and convert the remaining lines into (gene, sample) tuples. I assume that you know enough Python to do that. Then build dict that maps (gene, sample) tuples to the number of occurences: pivot = { ("FUS", "SampleA"): 1, ... ("RORA", "SampleC"): 1, } Remember to handle both the case when the tuple is already in the dict and when it's not in the dict. (Once you did it successfully have a look at the collections.Counter class). Now you need the row/column labels. You can extract them from the dict with rows = sorted(set(row for row, column in pivot)) # use set(...) to avoid duplicates columns = ... # something very similar You can then print the table with print([""] + columns) for row in rows: print([row] + [pivot.get((row, column), 0) for column in columns]) Use the str.format() method on the table cells to prettify the output and you're done. From alan.gauld at btinternet.com Thu Oct 30 10:27:23 2014 From: alan.gauld at btinternet.com (Alan Gauld) Date: Thu, 30 Oct 2014 09:27:23 +0000 Subject: [Tutor] R: re question on array In-Reply-To: <1903614574.1197511414657921768.JavaMail.defaultUser@defaultHost> References: <1903614574.1197511414657921768.JavaMail.defaultUser@defaultHost> Message-ID: <m2t09s$h8b$1@ger.gmane.org> On 30/10/14 08:32, jarod_v6 at libero.it wrote: > Sorry for my bad presentation of my problem!! Thats OK, and this explanation is much better, thanks. > A file with a long liste of gene ad the occurence for sample: > > gene Samples > FUS SampleA > TP53 SampleA > ATF4 SampleB > ATF3 SampleC > ATF4 SampleD > FUS SampleE > > WHat I want to obtain is amtrix where I have the occurence for sample. > SampleA SampleB SampleC SampleD SampleE > FUS 1 0 0 0 1 > TP53 1 0 0 0 0 > ATF4 0 1 1 0 > ATF3 0 0 1 0 0 > > In that way I count count the occurence in fast way! You probably want a dictionary keyed on the gene and with a list of samples as the value. Using the data above the final result would be data = { 'FUS' : [1,0,0,0,1] 'TP53' : [1,0,0,0,0] 'ATF4' : [0,1,0,1,0] 'ATF3' : [0,0,1,0,0] You would need to initialise each entry to all zeros when you create it. Then overwrite the sample positions as you discover them. PS. When replying to a digest please remove the irrelevant material from the end of your post. Some people pay by the byte to receive mail. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.flickr.com/photos/alangauldphotos From r.sokolewicz at gmail.com Thu Oct 30 15:01:51 2014 From: r.sokolewicz at gmail.com (Robert Sokolewicz) Date: Thu, 30 Oct 2014 15:01:51 +0100 Subject: [Tutor] passing named arguments through command line Message-ID: <CAP+R9sm03O0i=pRXgwNWZW06pNPT07kO4mTUi+NGSDhSw7iiqQ@mail.gmail.com> I have a function with optional arguments x, y and I would like to pass y or z using a named variable through the command line. Inside a python script main(y=3) would work, but I have trouble passing y=3 as an argument in command line. I have tried the following: ---- import sys def main(x=1, y=2): print x print y if __name__ == '__main__': main(*sys.argv[1:]) ----- from my terminal I get: $ python script.py 1 2 $ python script.py y=3 y=3 2 whereas I would like the following to happen: $ python script.py y=3 1 3 Is this doable in any convenient way? thanks in advance! -Robert -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/tutor/attachments/20141030/bf1c18e2/attachment.html> From lu.nemec at gmail.com Thu Oct 30 15:58:19 2014 From: lu.nemec at gmail.com (Lukas Nemec) Date: Thu, 30 Oct 2014 15:58:19 +0100 Subject: [Tutor] passing named arguments through command line In-Reply-To: <CAP+R9sm03O0i=pRXgwNWZW06pNPT07kO4mTUi+NGSDhSw7iiqQ@mail.gmail.com> References: <CAP+R9sm03O0i=pRXgwNWZW06pNPT07kO4mTUi+NGSDhSw7iiqQ@mail.gmail.com> Message-ID: <5452520B.3070606@gmail.com> Hello, take a look at argparse library. ------- import argparse parser = argparse.ArgumentParser(description="My prgoram") parser.add_argument('-y', '--y', help="Y value", required=True) parser.add_argument('-x', '--x', help="X value", required=True) def main(x=1, y=2): print x print y if __name__ == '__main__': args = parser.parse_args() main(x=args.x, y=args.y) Enjoy! Lukas On 10/30/2014 03:01 PM, Robert Sokolewicz wrote: > I have a function with optional arguments x, y and I would like to > pass y or z using a named variable through the command line. Inside a > python script main(y=3) would work, but I have trouble passing y=3 as > an argument in command line. > > I have tried the following: > > ---- > import sys > > def main(x=1, y=2): > print x > print y > > if __name__ == '__main__': > main(*sys.argv[1:]) > > ----- > > from my terminal I get: > > $ python script.py > 1 > 2 > $ python script.py y=3 > y=3 > 2 > > whereas I would like the following to happen: > $ python script.py y=3 > 1 > 3 > > Is this doable in any convenient way? > > thanks in advance! > > -Robert > > > > > > > > > > _______________________________________________ > Tutor maillist - Tutor at python.org > To unsubscribe or change subscription options: > https://mail.python.org/mailman/listinfo/tutor -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/tutor/attachments/20141030/b5d1feb7/attachment.html> From martin at linux-ip.net Thu Oct 30 16:45:22 2014 From: martin at linux-ip.net (Martin A. Brown) Date: Thu, 30 Oct 2014 08:45:22 -0700 Subject: [Tutor] Would somebody kindly... In-Reply-To: <m2spmv$6d4$1@ger.gmane.org> References: <052701cff304$c48b5c80$4da21580$@us> <m2pg03$b0d$1@ger.gmane.org> <056101cff332$b8180510$28480f30$@us> <m2qmd2$ji$1@ger.gmane.org> <06b801cff403$eed61340$cc8239c0$@us> <alpine.LNX.2.00.1410292324470.1349@dagger.wonderfrog.net> <m2spmv$6d4$1@ger.gmane.org> Message-ID: <alpine.LNX.2.00.1410300844241.1349@dagger.wonderfrog.net> > NO, NO, NO. The OP is using Python 3.4, and has consistently > shown results accordingly. x does NOT exist after the list > comprehension. That was a flaw in python 2.x which has been > fixed. Sorry about that, all. I still use Python 2.x most of the time, so defaulted to that. I am glad that this was fixed. Apologies! -Martin -- Martin A. Brown http://linux-ip.net/ From dyoo at hashcollision.org Thu Oct 30 19:24:35 2014 From: dyoo at hashcollision.org (Danny Yoo) Date: Thu, 30 Oct 2014 18:24:35 +0000 Subject: [Tutor] passing named arguments through command line References: <CAP+R9sm03O0i=pRXgwNWZW06pNPT07kO4mTUi+NGSDhSw7iiqQ@mail.gmail.com> <5452520B.3070606@gmail.com> Message-ID: <CAGZAPF46gh3733O3TYFcE2POEcfM4m9nEZ2F9dwOfu0jri_4=g@mail.gmail.com> On Thu Oct 30 2014 at 7:58:32 AM Lukas Nemec <lu.nemec at gmail.com> wrote: > Hello, > > take a look at argparse library. > Hi Robert, As Lukas mentions, it sounds like you're looking for a "flag parsing" library. A flag parsing library reads a set of key/value pairs that are encoded in sys.argv, so they let command-line programs provide variable values through the use of these flags. There are a few of these flag libraries in Python due to Python's long history. The one that Lukas recommends, 'argparse', is probably the one you want to use. You can find documentation for argparse at: https://docs.python.org/2/howto/argparse.html#id1 https://docs.python.org/2/library/argparse.html Good luck! -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/tutor/attachments/20141030/e1bd9381/attachment.html> From steve at pearwood.info Fri Oct 31 01:09:08 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 31 Oct 2014 11:09:08 +1100 Subject: [Tutor] Would somebody kindly... In-Reply-To: <06b801cff403$eed61340$cc8239c0$@us> References: <052701cff304$c48b5c80$4da21580$@us> <m2pg03$b0d$1@ger.gmane.org> <056101cff332$b8180510$28480f30$@us> <m2qmd2$ji$1@ger.gmane.org> <06b801cff403$eed61340$cc8239c0$@us> Message-ID: <20141031000908.GG26866@ando.pearwood.info> On Wed, Oct 29, 2014 at 10:39:53PM -0700, Clayton Kirkwood wrote: > When you have > [item for item in [list] if item[0] == key], after the iteration > completes does item equal the matched entities or does it have the > original item? Why don't you try it for yourself and find out? At the interactive interpreter -- do you know how to run the interactive interpreter? if not, you should ask -- simply try it: py> key = 23 py> result = [item for item in [(20, 1), (21, 2), (23, 4)] if item[0] == key] py> print(item) Traceback (most recent call last): File "<stdin>", line 1, in <module> NameError: name 'item' is not defined So in Python 3, the list comprehension variable `item` is *not* visible outside of the list comprehension. If we set a variable `item` ahead of time, then try it again, we'll see that it is not changed. py> item = "something" py> result = [item for item in [(20, 1), (21, 2), (23, 4)] if item[0] == key] py> print(item) something [Note: this is different from Python 2, where the list comprehension variable "leaks" outside of the comprehension. That was considered an accident, if not a bug, and fixed in Python 3.] What is happening here? This may not be *exactly* what Python does, but it will give you a reasonable idea of how list comprehensions work. When you write something like: result = [2*x + 1 for x in sequence if condition(x)] the Python compiler builds a hidden function: def _list_comp(): tmp = [] for x in sequence: if condition(x): tmp.append(2*x + 1) return tmp then executes it: result = _list_comp() (except that the hidden function is completely invisible to Python code, only the compiler can access it). So you can see that the list comp variable x is a local variable, localised to the list comprehension, and doesn't change any existing x outside of the list comprehension. > This is from a previous email-- > When I run: > values = [ ('a', 1), ('b', 2), ('a', 5), ('c', 7)] > key = 'a' > pair=[] > [pair for pair in values if key == pair[0]] > print(pair) > > I get []. Correct. The result of the list comprehension [pair for ...] is built, then thrown away because you don't do anything with it. The existing variable `pair` is unchanged, since the `pair` inside the list comp lives inside it's own scope, separate from the outside `pair`. This is no different from this situation: pair = 'something' # global scope def f(): pair = 23 # local to function f def g(): pair = 42 # local to function g [pair for pair in sequence] # local to list comp In this case, there are four *different* `pair` variables, different because they live in different scopes. > When I run: > values = [ ('a', 1), ('b', 2), ('a', 5), ('c', 7)] > key = 'a' > pair=[] > x=[pair for pair in values if key == pair[0]] > print(x) > > I get [('a', 1), ('a', 5)] That's because you build the list, and instead of throwing it away, you assign it to the variable `x`, then print `x`. The variable `pair` remains unchanged. -- Steven From alan.gauld at btinternet.com Fri Oct 31 02:52:42 2014 From: alan.gauld at btinternet.com (Alan Gauld) Date: Fri, 31 Oct 2014 01:52:42 +0000 Subject: [Tutor] Would somebody kindly... In-Reply-To: <06b801cff403$eed61340$cc8239c0$@us> References: <052701cff304$c48b5c80$4da21580$@us> <m2pg03$b0d$1@ger.gmane.org> <056101cff332$b8180510$28480f30$@us> <m2qmd2$ji$1@ger.gmane.org> <06b801cff403$eed61340$cc8239c0$@us> Message-ID: <m2uq1a$813$1@ger.gmane.org> On 30/10/14 05:39, Clayton Kirkwood wrote: > So, in this case, the assignment to x is external. > Often I don't see an external assignment, but there > is an item in the first position within the comprehension. I've been reviewing this thread and there might be a concept that you are missing. Several of us have spoken about expressions and that the first "item" in a comprehension is an expression. Indeed the comprehension as a whole is an expression. But do you understand the significance of "an expression" in Python (or general programming terms) as opposed to simple English language terms? If not, much of the significance of what has been said so far will have been missed. An expression is basically a piece of code that returns a value. So X X+3 X*Y 4**2 pow(4,2) round((3*2)/(4+7)) [item for item in values if item > 42] are all expressions: they return values. Someone (Steven?) mentioned that the comprehension can be thought of as a kind of anonymous function that returns a value and the values inside are effectively hidden from the outer program. And a function call(like pow() above) are expressions too. The point of expressions is that the resulting value can be assigned to variables or used in tests or included in data. Anywhere that a literal value can be used you can use an expression. Including a list comprehension. The significant thing about a list comprehension is that it returns a new list value. It does not modify the collection inside the comprehension and it does not modify any lists outside the comprehension. It returns a new list comprising all the expression values of the first element in the comprehension that match the optional conditional expression. result = [result_expression for variable in collection if filter_expression] Now, maybe I'm misreading things and you already understood all of that, in which case I apologize. On the other hand it might just clear up what we mean when we talk about expressions. HTH -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.flickr.com/photos/alangauldphotos From matt.l.varner at gmail.com Fri Oct 31 12:07:25 2014 From: matt.l.varner at gmail.com (Matt Varner) Date: Fri, 31 Oct 2014 04:07:25 -0700 Subject: [Tutor] Newbie Trouble Processing SRT Strings In Text File Message-ID: <CAAOjOuc2D6ubPKTrOF2SG7KZopovMomdpcVqWfhUqyzAXO9-ag@mail.gmail.com> TL:DR - Skip to "My Script: "subtrans.py" <beg> Optional Links to (perhaps) Helpful Images: 1. The SRT download button: http://i70.photobucket.com/albums/i82/RavingNoah/Python%20Help/tutor1_zps080f20f7.png 2. A visual comparison of my current problem (see 'Desire Versus Reality' below): http://i70.photobucket.com/albums/i82/RavingNoah/Python%20Help/newline_problem_zps307f8cab.jpg ============ The SRT File ============ The SRT file that you can download for every lesson that has a video contains the caption transcript data and is organized according to text snippets with some connecting time data. ======================================== Reading the SRT File and Outputting Something Useful ======================================== There may be a hundred different ways to read one of these file types. The reliable method I chose was to use a handy editor for the purpose called Aegisub. It will open the SRT file and let me immediately export a version of it, without the time data (which I don't need...yet). The result of the export is a plain-text file containing each string snippet and a newline character. ========================== Dealing with the Text File ========================== One of these text files can be anywhere between 130 to 500 lines or longer, depending (obviously) on the length of its attendant video. For my purposes, as a springboard for extending my own notes for each module, I need to concatenate each string with an acceptable format. My desire for this is to interject spaces where I need them and kill all the newline characters so that I get just one big lump of properly spaced paragraph text. From here, I can divide up the paragraphs how I see fit and I'm golden... ============================== My first Python script: Issues ============================== I did my due diligence. I have read the tutorial at www.python.org. I went to my local library and have a copy of "Python Programming for the Absolute Beginner, 3rd Edition by Michael Dawson." I started collecting what seemed like logical little bits here and there from examples found using Uncle Google, but none of the examples anywhere were close enough, contextually, to be automatically picked up by my dense 'noobiosity.' For instance, when discussing string methods...almost all operations taught to beginners are done on strings generated "on the fly," directly inputted into IDLE, but not on strings that are contained in an external file. There are other examples for file operations, but none of them involved doing string operations afterward. After many errors about not being able to directly edit strings in a file object, I finally figured out that lists are used to read and store strings kept in a file like the one I'm sourcing from...so I tried using that. Then I spent hours unsuccessfully trying to call strings using index numbers from the list object (I guess I'm dense). Anyhow, I put together my little snippets and have been banging my head against the wall for a couple of days now. After many frustrating attempts, I have NEARLY produced what I'm looking to achieve in my test file. ================ Example - Source ================ My Test file contains just twelve lines of a much larger (but no more complex) file that is typical for the SRT subtitle caption file, of which I expect to have to process a hundred...or hundreds, depending on how many there are in all of the courses I plan to take (coincidentally, there is one on Python) Line 01: # Exported by Aegisub 3.2.1 Line 02: [Deep Dive] Line 03: [CSS Values & Units Numeric and Textual Data Types with Guil Hernandez] Line 04: In this video, we'll go over the Line 05: common numeric and textual values Line 06: that CSS properties can accept. Line 07: Let's get started. Line 08: So, here we have a simple HTML page Line 09: containing a div and a paragraph Line 10: element nested inside. Line 11: It's linked to a style sheet named style.css Line 12: and this is where we'll be creating our new CSS rules. ======================== My Script: "subtrans.py" ======================== # Open the target file, create file object f = open('tmp.txt', 'r') # Create an output file to write the changed strings to o = open('result.txt', 'w') # Create a list object that holds all the strings in the file object lns = f.readlines() # Close the source file you no longer # need now that you have your strings f.close() # Import sys to get at stdout (standard output) - "print" results will be written to file import sys # Associate stdout with the output file sys.stdout = o # Try to print strings to output file using loopback variable (line) and the list object for line in lns: if ".\n" in line: a = line.replace('.\n','. ') print(a.strip('\n')) else: b = line.strip('\n') print(b + " ") # Close your output file o.close() ================= Desire Versus Reality ================= The source file contains a series of strings with newline characters directly following whatever the last character in the snippet...with absolutely no whitespace. This is a problem for me if I want to concatentate it back together into paragraph text to use as the jumping off point for my additional notes. I've been literally taking four hours to type explicitly the dialogue from the videos I've been watching...and I know this is going to save me a lot of time and get me interacting with the lessons faster and more efficiently. However... My script succeeds in processing the source file and adding the right amount of spaces for each line, the rule being "two spaces added following a period, and one space added following a string with no period in it (technically, a period/newline pairing (which was the only way I could figure out not target the period in 'example.css' or 'version 2.3.2'. But, even though it successfully kills these additional newlines that seem to form in the list-making process...I end up with basically a non-concatenated file of strings...with the right spaces I need, but not one big chunk of text, like I expect using the s.strip('\n') functionality. ============================================================ What I'm Holding Out For - This is what my output should look like (for the test file) ============================================================ # Exported by Aegisub 3.2.1 [Deep Dive] [CSS Values & Units Numeric and Textual Data Types with Guil Hernandez] In this video, we'll go over the common numeric and textual values that CSS properties can accept. Let's get started. So, here we have a simple HTML page containing a div and a paragraph element nested inside. It's linked to a style sheet named style.css and this is where we'll be creating our new CSS rules. =========================== Thank You For Your Time and Efforts! =========================== </beg> From r.sokolewicz at gmail.com Fri Oct 31 12:14:15 2014 From: r.sokolewicz at gmail.com (Robert Sokolewicz) Date: Fri, 31 Oct 2014 12:14:15 +0100 Subject: [Tutor] passing named arguments through command line In-Reply-To: <CAGZAPF46gh3733O3TYFcE2POEcfM4m9nEZ2F9dwOfu0jri_4=g@mail.gmail.com> References: <CAP+R9sm03O0i=pRXgwNWZW06pNPT07kO4mTUi+NGSDhSw7iiqQ@mail.gmail.com> <5452520B.3070606@gmail.com> <CAGZAPF46gh3733O3TYFcE2POEcfM4m9nEZ2F9dwOfu0jri_4=g@mail.gmail.com> Message-ID: <CAP+R9smu95Hygk23UFEyNes3gYF=oaawDx-yCMPsaCOw2_tO-Q@mail.gmail.com> cool, thanks guys :) -Robert On Thu, Oct 30, 2014 at 7:24 PM, Danny Yoo <dyoo at hashcollision.org> wrote: > > > On Thu Oct 30 2014 at 7:58:32 AM Lukas Nemec <lu.nemec at gmail.com> wrote: > >> Hello, >> >> take a look at argparse library. >> > > > Hi Robert, > > As Lukas mentions, it sounds like you're looking for a "flag parsing" > library. A flag parsing library reads a set of key/value pairs that are > encoded in sys.argv, so they let command-line programs provide variable > values through the use of these flags. > > There are a few of these flag libraries in Python due to Python's long > history. The one that Lukas recommends, 'argparse', is probably the one > you want to use. > > You can find documentation for argparse at: > > https://docs.python.org/2/howto/argparse.html#id1 > https://docs.python.org/2/library/argparse.html > > Good luck! > -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/tutor/attachments/20141031/765ff979/attachment.html> From alan.gauld at btinternet.com Fri Oct 31 18:13:54 2014 From: alan.gauld at btinternet.com (Alan Gauld) Date: Fri, 31 Oct 2014 17:13:54 +0000 Subject: [Tutor] Newbie Trouble Processing SRT Strings In Text File In-Reply-To: <CAAOjOuc2D6ubPKTrOF2SG7KZopovMomdpcVqWfhUqyzAXO9-ag@mail.gmail.com> References: <CAAOjOuc2D6ubPKTrOF2SG7KZopovMomdpcVqWfhUqyzAXO9-ag@mail.gmail.com> Message-ID: <m30g0i$jfv$1@ger.gmane.org> On 31/10/14 11:07, Matt Varner wrote: > # Import sys to get at stdout (standard output) - "print" results will > be written to file > import sys This is a bad idea. Instead, write your strings directly to o o.write(s) Print adds newlines automatically(unless you explicitly suppress them). But printing to a file is messy compared to writing directly to the file. (And also means you cant print debug messages while developing your code!) HTH -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.flickr.com/photos/alangauldphotos From crushed26 at gmail.com Fri Oct 31 18:28:48 2014 From: crushed26 at gmail.com (Bo Morris) Date: Fri, 31 Oct 2014 13:28:48 -0400 Subject: [Tutor] Practicing with sockets Message-ID: <CAKKCnffw7CdKaOpxx3Q4p2fZOH8nJNfbrnvZYBBnrEc3fy5vaA@mail.gmail.com> Hello all, hope everyone is doing well. I have been practicing with sockets and I am trying to send a small png from the client to the server. the client code is... import socket f = open('/Users/Bo/Desktop/logo_ONEConnxt.png', 'rb') strf = f.read() client_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM) client_socket.connect(("ip.ip.ip.ip", 8999)) client_socket.sendall(strf) f.close() exit() and the server code is... import socket f = open('img.png', 'wb') s = socket.socket(socket.AF_INET, socket.SOCK_STREAM) port = 8999 s.bind(('', port)) s.listen(5) client_socket, address = s.accept() data = client_socket.recv(4029) f.write(data) client_socket.close() Both the above client and server code runs without error, however the "img.png" file that is placed on the server shows zero bytes? Will someone please show me what I am doing wrong? Thank you, Bo -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/tutor/attachments/20141031/65b9f3af/attachment.html> From dyoo at hashcollision.org Fri Oct 31 18:18:16 2014 From: dyoo at hashcollision.org (Danny Yoo) Date: Fri, 31 Oct 2014 17:18:16 +0000 Subject: [Tutor] Newbie Trouble Processing SRT Strings In Text File References: <CAAOjOuc2D6ubPKTrOF2SG7KZopovMomdpcVqWfhUqyzAXO9-ag@mail.gmail.com> Message-ID: <CAGZAPF4DxTgqFOeUfomhqrU8FFR6E9iok6m499ki_Os2cdWNUw@mail.gmail.com> [code cut] Hi Matt, It looks like you're trying to write your own srt parser as part of this problem. If you're in a hurry, you may want to look at existing parsers that people have written. For example: https://github.com/byroot/pysrt But, even though it successfully kills these additional newlines that > seem to form in the list-making process...I end up with basically a > non-concatenated file of strings...with the right spaces I need, but > not one big chunk of text, like I expect using the s.strip('\n') > functionality. > Rather than immediately print the string, you may want to accumulate your results in a list. You can then do some processing on your list of strings. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/tutor/attachments/20141031/de836b52/attachment-0001.html> From dyoo at hashcollision.org Fri Oct 31 19:38:45 2014 From: dyoo at hashcollision.org (Danny Yoo) Date: Fri, 31 Oct 2014 18:38:45 +0000 Subject: [Tutor] Practicing with sockets References: <CAKKCnffw7CdKaOpxx3Q4p2fZOH8nJNfbrnvZYBBnrEc3fy5vaA@mail.gmail.com> Message-ID: <CAGZAPF7S43OCw=vSiyj0udwwVqEYS6ZkPF+t+XvAB0dfGgKGHg@mail.gmail.com> On Fri Oct 31 2014 at 10:31:20 AM Bo Morris <crushed26 at gmail.com> wrote: > Hello all, hope everyone is doing well. > > I have been practicing with sockets and I am trying to send a small png > from the client to the server. > Hey Bo, Very cool! Socket programming is fun, because it lets your programs start talking to other programs. But it can get frustrating at times too, since it's all about communication, and we know communcation can fail for so many different reasons. :P We'll try to help where we can. Just to make sure, you are probably following the Socket HOWTO: https://docs.python.org/2/howto/sockets.html Reading code... > import socket > > f = open('/Users/Bo/Desktop/logo_ONEConnxt.png', 'rb') > strf = f.read() > client_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM) > client_socket.connect(("ip.ip.ip.ip", 8999)) > client_socket.sendall(strf) > f.close() > exit() > > This is problematic: the server code won't know up front how many bytes it should expect to read from the socket. That is, the code here is sending "variable-length" message, and variable lengths are difficult to work with. One common solution is to prefix the payload with a fixed-size byte length. That way, the server can read the fixed-size length first, and then run a loop that reads the rest of the bytes. This looks something like: import struct # ... # Send the length... client_socket.send(struct.pack("!I", len(strf))) # followed by the content client_socket.sendall(strf) Your client code will symmetrically read the first four bytes, use struct.unpack() to find how how large the rest of the message is going to be, and then do a loop until it reads the exact number of bytes. Ok, I'm reading through the client code a bit more... data = client_socket.recv(4029) > f.write(data) > client_socket.close() > You probably want to open the output file _after_ the socket has accepted. Otherwise, it seems a bit premature to open that "f" file. Also, don't forget to close the "f" file once you've finished reading the bytes. Also note here that since recv() doesn't guarantee how many bytes you'll read at a time, the byte-reading code needs to be in a loop. Also, I strongly suggest that you place some logging messages in both your client and server to trace where your programs are. One distinguishing feature of network programs is that they are typically long-running, and so logs help to expose what the heck they're doing at a given time. See: https://docs.python.org/2/howto/logging.html#logging-basic-tutorial https://docs.python.org/2/library/logging.html As it stands, your server might not have ever accepted a message from your client, and you'll still see an empty file, since the code is opening the file for writing before listening for a request. That's the other reason why you want to move the file opening to _after_ the socket is accepted. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/tutor/attachments/20141031/fa1e3031/attachment.html> From crushed26 at gmail.com Fri Oct 31 20:42:20 2014 From: crushed26 at gmail.com (Bo Morris) Date: Fri, 31 Oct 2014 15:42:20 -0400 Subject: [Tutor] Practicing with sockets In-Reply-To: <CAGZAPF7S43OCw=vSiyj0udwwVqEYS6ZkPF+t+XvAB0dfGgKGHg@mail.gmail.com> References: <CAKKCnffw7CdKaOpxx3Q4p2fZOH8nJNfbrnvZYBBnrEc3fy5vaA@mail.gmail.com> <CAGZAPF7S43OCw=vSiyj0udwwVqEYS6ZkPF+t+XvAB0dfGgKGHg@mail.gmail.com> Message-ID: <CAKKCnfdgC81Ppec1N_HmFJDfyv7sazDkj50D8UfvqoL06GOoSQ@mail.gmail.com> Hey Danny, yes I have been having quite a bit of fun learning to work with sockets. Thank you for your response. I have applied what you suggested with the exception of the "logging." I read through the logging docs and figured logging would be learning for another day. I have a hard time enough staying focused on one task at time haha. I did however insert some print statements into the code so I could keep track of where it was at, but to keep my email short, I omitted them here. After implementing what you suggested, the image fie that is saved on the server is now 4 bytes, but I assume that is due to... "Your client code will symmetrically read the first four bytes, use struct.unpack() to find how how large the rest of the message is going to be, and then do a loop until it reads the exact number of bytes" and I have not quite got the correct loop to read all the bytes? I also reread the docs at https://docs.python.org/2/howto/sockets.html and decided to remove the "b" from "open('myfile.png', 'wb') open('myfile.png', 'rb') seeing how binary could be different depending on the machine and I have not yet learned how to deal with this. Would I be better off converting the image to base64 prior to sending it to the server, then decoding it on the server? Here is my updated code...for brevity sake, I have omitted the "import" statments... Client: f = open('/Users/Bo/Desktop/SIG.png', 'r') strf = f.read() client_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM) client_socket.connect(("ip,ip,ip,ip", 8999)) payload = client_socket.send(struct.pack("!I", len(strf))) for data in payload: client_socket.sendall(strf) f.close() exit() Server: s = socket.socket(socket.AF_INET, socket.SOCK_STREAM) port = 8999 s.bind(('', port)) s.listen(5) client_socket, address = s.accept() data = client_socket.recv(4029) f = open('img.png', 'w') for item in data: f.write(item) f.flush() f.close() client_socket.close() At least I am getting 4 bytes in oppose to 0 like I was getting before. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/tutor/attachments/20141031/7f65b197/attachment.html> From crushed26 at gmail.com Fri Oct 31 21:00:24 2014 From: crushed26 at gmail.com (Bo Morris) Date: Fri, 31 Oct 2014 16:00:24 -0400 Subject: [Tutor] Practicing with sockets In-Reply-To: <CAKKCnfdgC81Ppec1N_HmFJDfyv7sazDkj50D8UfvqoL06GOoSQ@mail.gmail.com> References: <CAKKCnffw7CdKaOpxx3Q4p2fZOH8nJNfbrnvZYBBnrEc3fy5vaA@mail.gmail.com> <CAGZAPF7S43OCw=vSiyj0udwwVqEYS6ZkPF+t+XvAB0dfGgKGHg@mail.gmail.com> <CAKKCnfdgC81Ppec1N_HmFJDfyv7sazDkj50D8UfvqoL06GOoSQ@mail.gmail.com> Message-ID: <CAKKCnfeqX+FxFSbbZdC9+aaHvtYhs0EXOok73BjpCXpEQ7jXFA@mail.gmail.com> ok so I finally got all the bytes to be transfered to the server, however I am unable to open the image on the server; although the filed is saved as a png file on the server, the server does not recognize the file as png format? I changed the loops to the following... Client: f = open('/Users/Bo/Desktop/SIG.png', 'r') strf = f.read() client_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM) client_socket.connect(("25.78.28.110", 8999)) while True: client_socket.send(struct.pack("!I", len(strf))) data = client_socket.sendall(strf) if not data: break f.close() print "Data Received successfully" exit() Server: s = socket.socket(socket.AF_INET, socket.SOCK_STREAM) port = 8999 s.bind(('', port)) s.listen(5) client_socket, address = s.accept() f = open('img.png', 'w') while True: data = client_socket.recv(4029) f.write(data) if not data: break #f.flush() f.close() client_socket.close() On Fri, Oct 31, 2014 at 3:42 PM, Bo Morris <crushed26 at gmail.com> wrote: > Hey Danny, yes I have been having quite a bit of fun learning to work with > sockets. Thank you for your response. I have applied what you suggested > with the exception of the "logging." I read through the logging docs and > figured logging would be learning for another day. I have a hard time > enough staying focused on one task at time haha. I did however insert some > print statements into the code so I could keep track of where it was at, > but to keep my email short, I omitted them here. > > After implementing what you suggested, the image fie that is saved on the > server is now 4 bytes, but I assume that is due to... > > "Your client code will symmetrically read the first four bytes, use > struct.unpack() to find how how large the rest of the message is going to > be, and then do a loop until it reads the exact number of bytes" > > and I have not quite got the correct loop to read all the bytes? > > I also reread the docs at https://docs.python.org/2/howto/sockets.html and > decided to remove the "b" from "open('myfile.png', 'wb') open('myfile.png', > 'rb') seeing how binary could be different depending on the machine and I > have not yet learned how to deal with this. Would I be better off > converting the image to base64 prior to sending it to the server, then > decoding it on the server? > > Here is my updated code...for brevity sake, I have omitted the "import" > statments... > > Client: > > f = open('/Users/Bo/Desktop/SIG.png', 'r') > strf = f.read() > client_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM) > client_socket.connect(("ip,ip,ip,ip", 8999)) > payload = client_socket.send(struct.pack("!I", len(strf))) > for data in payload: > client_socket.sendall(strf) > f.close() > exit() > > Server: > > s = socket.socket(socket.AF_INET, socket.SOCK_STREAM) > port = 8999 > s.bind(('', port)) > s.listen(5) > client_socket, address = s.accept() > data = client_socket.recv(4029) > f = open('img.png', 'w') > for item in data: > f.write(item) > f.flush() > f.close() > client_socket.close() > > At least I am getting 4 bytes in oppose to 0 like I was getting before. > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/tutor/attachments/20141031/4d53f8bd/attachment-0001.html> From dyoo at hashcollision.org Fri Oct 31 22:42:48 2014 From: dyoo at hashcollision.org (Danny Yoo) Date: Fri, 31 Oct 2014 21:42:48 +0000 Subject: [Tutor] Practicing with sockets References: <CAKKCnffw7CdKaOpxx3Q4p2fZOH8nJNfbrnvZYBBnrEc3fy5vaA@mail.gmail.com> <CAGZAPF7S43OCw=vSiyj0udwwVqEYS6ZkPF+t+XvAB0dfGgKGHg@mail.gmail.com> <CAKKCnfdgC81Ppec1N_HmFJDfyv7sazDkj50D8UfvqoL06GOoSQ@mail.gmail.com> Message-ID: <CAGZAPF6zadQPw4-Yd6OAuQQke21VEMZERhJ8Ck184vHuDJGVRA@mail.gmail.com> > > > I also reread the docs at https://docs.python.org/2/howto/sockets.html and > decided to remove the "b" from "open('myfile.png', 'wb') open('myfile.png', > 'rb') seeing how binary could be different depending on the machine and I > have not yet learned how to deal with this. > Whoa, wait. I think you're misunderstanding the point of binary mode. You _definitely_ need binary mode on when working with binary file formats like PNG. Otherwise, your operating system environment may do funny things to the file content like treat the 0-character (NULL) as a terminator, or try to transparently translate line ending sequences. Would I be better off converting the image to base64 prior to sending it to > the server, then decoding it on the server? > The socket approach is low-level: all you've got is a pipe that can send and receive bytes. It's _all_ binary from the perspective of the network layer. base64-encoding and decoding these bytes won't harm anything, of course, but I don't see it helping either. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/tutor/attachments/20141031/56ad90b0/attachment.html>