From c-gschuette at neogov.net Thu Oct 3 16:48:35 2013 From: c-gschuette at neogov.net (c-gschuette at neogov.net) Date: Thu, 3 Oct 2013 13:48:35 -0700 (PDT) Subject: wil anyone ressurect medusa and pypersist? In-Reply-To: <50e6d4bc-ce2f-4c55-97b0-ec0cad793682@googlegroups.com> References: <50e6d4bc-ce2f-4c55-97b0-ec0cad793682@googlegroups.com> Message-ID: On Thursday, May 16, 2013 11:15:45 AM UTC-7, vispha... at gmail.com wrote: > www.prevayler.org in python = pypersist > > > > medusa = python epoll web server and ftp server eventy and async wow interesting sprevayler ?? cl-prevalence From jkn_gg at nicorp.f9.co.uk Fri Oct 4 04:28:20 2013 From: jkn_gg at nicorp.f9.co.uk (jkn) Date: Fri, 4 Oct 2013 01:28:20 -0700 (PDT) Subject: wil anyone ressurect medusa and pypersist? In-Reply-To: References: <50e6d4bc-ce2f-4c55-97b0-ec0cad793682@googlegroups.com> Message-ID: On Thursday, 3 October 2013 21:48:35 UTC+1, c-gsc... at neogov.net wrote: > On Thursday, May 16, 2013 11:15:45 AM UTC-7, vispha... at gmail.com wrote: > > > www.prevayler.org in python = pypersist > > > > > > > > > > > > medusa = python epoll web server and ftp server eventy and async > > > > wow interesting > > > > sprevayler ?? > > > > cl-prevalence Hmm - FWIW this looks a lot like it's from 'gavino', a troll-like entity on c.l.forth. J^n From neilc at norwich.edu Tue Oct 1 09:59:27 2013 From: neilc at norwich.edu (Neil Cerutti) Date: 1 Oct 2013 13:59:27 GMT Subject: Functional Programming and python References: <524a13d7$0$29974$c3e8da3$5496439d@news.astraweb.com> Message-ID: On 2013-10-01, Steven D'Aprano wrote: > On Mon, 30 Sep 2013 18:36:28 +0000, Neil Cerutti quoted: > >> Why can??t lambda forms contain statements? > > Gah! Please fix your news client! (I see you're using slrn.) > The \x92 bytes found in your message are apostrophes > (technically: right single quotation marks), encoded using the > legacy Windows-1252 codec, but your news client is falsely > advertising it as US-ASCII: > > Content-Type: text/plain; charset=US-ASCII > Content-Transfer-Encoding: 8bit > > It's almost 2014, it is unspeakably poor form that an > application is still making this mistake. Is there an updated > version of slrn that fixes this? Can you manually force it to > use UTF-8? Can you report this as a bug? > > In case you aren't too clear on the concepts, here are two Must > Read links: > > http://www.joelonsoftware.com/articles/Unicode.html > http://nedbatchelder.com/text/unipain.html Thanks, Steve. I'm aware my news setup is crap when it comes to character set/unicode support. I haven't been motivated to fix it, because everything else about it works great for me. I'm afraid slrn is stagnated, but it might provide at least some support and I need to find out what. -- Neil Cerutti From piet at vanoostrum.org Tue Oct 1 13:31:33 2013 From: piet at vanoostrum.org (Piet van Oostrum) Date: Tue, 01 Oct 2013 13:31:33 -0400 Subject: Functional Programming and python References: Message-ID: Antoon Pardon writes: > Op 30-09-13 20:55, Piet van Oostrum schreef: >> Franck Ditter writes: >> >>> Good approach of FP in Python, but two points make me crazy : >>> 1. Tail recursion is not optimized. We are in 2013, why ? This is known technology (since 1960). >>> And don't answer with "good programmers don't use recursion", this is bullshit. >> >> Tail recursion optimization throws away valuable stack trace information in case of an error. > > This is hardly relevant. Because what are we told to use instead of > tail calls? We are told to use loops. But when you use a loop the > stack trace doesn't contain the values of previous runs through > the loop. > > So how valuable is that stack frame information when the proposed > alternative doesn't produces it either. This is incorrect. Loops are not always a replacement for tail recursion optimization. Tail recursion optimization generally means that the last function call before a return in a function reuses the current stack frame instead of generating a new one. This is regardless of which function is called, otherwise mutually recursive calls will not be optimized. (Unless you do static analysis of all the functions calls involved, which is very hard or even impossible in python.) A better name for this process is therefore 'tail call optimization'. So any function call that is last in its flow will be optimized. This gives a lot of situations where stack trace information is thrown away, even in those situations where a loop would not be an alternative and where optimization would give little benefit. Anyway, deep recursion isn't used very often in Python, which diminishes the value of tail call optimization. -- Piet van Oostrum WWW: http://pietvanoostrum.com/ PGP key: [8DAE142BE17999C4] From rosuav at gmail.com Tue Oct 1 03:28:04 2013 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 1 Oct 2013 17:28:04 +1000 Subject: Functional Programming and python In-Reply-To: <08fef68f-dc0d-43a4-9e28-635272fb59cf@googlegroups.com> References: <20130930160218.326d518b@bigbox.christie.dr> <08fef68f-dc0d-43a4-9e28-635272fb59cf@googlegroups.com> Message-ID: On Tue, Oct 1, 2013 at 11:36 AM, rusi wrote: >> (But I do sometimes yearn for a goto.) > > Ha! In Scheme, a tail call IS a goto with parameter re-assignment Precisely. In fact, tail call optimization basically consists of that exact rewrite. I'm absolutely fine with it being completely explicit. ChrisA From petmertens at gmail.com Thu Oct 3 14:09:16 2013 From: petmertens at gmail.com (petmertens at gmail.com) Date: Thu, 3 Oct 2013 11:09:16 -0700 (PDT) Subject: Get the selected tab in a enthought traits application In-Reply-To: <7aba29d3-dc9e-4a9c-82d5-edfab19d767c@googlegroups.com> References: <7aba29d3-dc9e-4a9c-82d5-edfab19d767c@googlegroups.com> Message-ID: <6db8a695-d577-43a1-99f6-cf82793aa142@googlegroups.com> Here's the answer: from enthought.traits.api import HasTraits, Str, List, Button, Any from enthought.traits.ui.api import View, Item from enthought.traits.ui.api import ListEditor class A(HasTraits): StringA = Str view = View(Item('StringA')) class B(HasTraits): StringB = Str view = View(Item('StringB')) class C(HasTraits): MyList = List(HasTraits) MyButton = Button(label="Test") SelectedTab = Any def _MyButton_fired(self): if self.SelectedTab == self.MyList[0]: print self.MyList[0].StringA if self.SelectedTab == self.MyList[1]: print self.MyList[1].StringB view = View(Item('MyList', style='custom', show_label=False, editor=ListEditor(use_notebook=True, deletable=False, dock_style='tab', selected='SelectedTab')), Item('MyButton', show_label=False)) a = A() b = B() c = C() c.MyList = [a, b] c.configure_traits() From kjakupak at gmail.com Tue Oct 1 13:53:26 2013 From: kjakupak at gmail.com (kjakupak at gmail.com) Date: Tue, 1 Oct 2013 10:53:26 -0700 (PDT) Subject: Help with python functions? In-Reply-To: References: <5240489d$0$29992$c3e8da3$5496439d@news.astraweb.com> <3af43a58-fc56-46d1-9e1e-cb69f8aa520b@googlegroups.com> <5d5e5c4d-a79c-4b1a-b9da-ee2ad766ded8@googlegroups.com> Message-ID: <1b025b07-3c9c-4390-80e5-8b5661f00d96@googlegroups.com> I ended up with these. I know they're only like half right... I was wondering if any of you had to do this, what would you end up with? # Question 1.a def temp(T, from_unit, to_unit): if from_unit == 'C' or from_unit == 'c': return 32 + (9/5)*T elif from_unit == 'K' or from_unit == 'k': return T + 273.15 elif from_unit == 'F' or from_unit == 'f': return (5/9)*(T - 32) else: return to_unit # Question 1.b def comp(T1, u1, T2, u2): if u1 != u2: T1 = temp(T1, u1) elif T2 > T1: return -1 elif T1 > T2: return 1 else: return 0 # Question 2 def P(p_0, t, i): Amount = P(1 + (i/100)) return P(1 + (t * i/12)) From random832 at fastmail.us Tue Oct 1 15:02:32 2013 From: random832 at fastmail.us (random832 at fastmail.us) Date: Tue, 01 Oct 2013 15:02:32 -0400 Subject: Help with python functions? In-Reply-To: <1b025b07-3c9c-4390-80e5-8b5661f00d96@googlegroups.com> References: <5240489d$0$29992$c3e8da3$5496439d@news.astraweb.com> <3af43a58-fc56-46d1-9e1e-cb69f8aa520b@googlegroups.com> <5d5e5c4d-a79c-4b1a-b9da-ee2ad766ded8@googlegroups.com> <1b025b07-3c9c-4390-80e5-8b5661f00d96@googlegroups.com> Message-ID: <1380654152.4713.28764133.6B91C072@webmail.messagingengine.com> On Tue, Oct 1, 2013, at 13:53, kjakupak at gmail.com wrote: > I ended up with these. I know they're only like half right... > I was wondering if any of you had to do this, what would you end up with? > > # Question 1.a > def temp(T, from_unit, to_unit): Assuming this is temperature conversion. You should add a docstring. > if from_unit == 'C' or from_unit == 'c': Consider normalizing the unit with .upper() at the top of the function so you don't have to do this "or" case in every single section. > return 32 + (9/5)*T You are ignoring the value of to_unit and returning the value in fahrenheit. > elif from_unit == 'K' or from_unit == 'k': > return T + 273.15 This conversion is simply wrong - it's the conversion _from_ celsius _to_ kelvin. > elif from_unit == 'F' or from_unit == 'f': > return (5/9)*(T - 32) You are ignoring the value of to_unit and returning the value in celsius > else: > return to_unit I don't know what this is. It's probably wrong. To implement a temperature conversion function, I would convert from the unit given to kelvin, then convert from kelvin to the desired unit - that way you don't have to implement every combination separately. > # Question 1.b > def comp(T1, u1, T2, u2): > if u1 != u2: > T1 = temp(T1, u1) You're not passing in u2 here. > elif T2 > T1: > return -1 > elif T1 > T2: > return 1 > else: > return 0 > # Question 2 > def P(p_0, t, i): > Amount = P(1 + (i/100)) > return P(1 + (t * i/12)) I don't know what this is. Is this for homework? From denismfmcmahon at gmail.com Tue Oct 1 17:45:24 2013 From: denismfmcmahon at gmail.com (Denis McMahon) Date: Tue, 1 Oct 2013 21:45:24 +0000 (UTC) Subject: Help with python functions? References: <5240489d$0$29992$c3e8da3$5496439d@news.astraweb.com> <3af43a58-fc56-46d1-9e1e-cb69f8aa520b@googlegroups.com> <5d5e5c4d-a79c-4b1a-b9da-ee2ad766ded8@googlegroups.com> <1b025b07-3c9c-4390-80e5-8b5661f00d96@googlegroups.com> Message-ID: On Tue, 01 Oct 2013 10:53:26 -0700, kjakupak wrote: > I ended up with these. I know they're only like half right... > I was wondering if any of you had to do this, what would you end up > with? > # Question 1.a > def temp(T, from_unit, to_unit): I suspect that this doesn't work properly for all cases of from_unit, to_unit. As a general case: def temp ( T, u1, u2 ): # from and to units the same, return T unchanged # else use a conversion table ct = { (a, b):lambda x: formula, .... } return ct[ (u1, u2 ) ]( T ) Note you may need to build in case independence! > # Question 1.b > def comp(T1, u1, T2, u2): You completely missed the point of my earlier posts, and I suspect the reason both these questions were included. Firstly, consider that if temp (from q1a) works properly you can use temp to convert the units of T2 to the units of T1, by calling: temp( T2, u2, u1 ) q1b can be implemented in one line if temp from q1a works properly! > # Question 2 > def P(p_0, t, i): > Amount = P(1 + (i/100)) > return P(1 + (t * i/12)) First calculate the annual interest as 1 + fraction where fraction is interest % / 100 The calculate the compounded interest as annual ^ years Finally multiply the compounded interest by the principal Mathematically: principal * ( ( 1 + ( period_interest_% / 100 ) ) ^ periods ) Again, this should be possible as a single line function. All you have to do is turn the math into python code. -- Denis McMahon, denismfmcmahon at gmail.com From antoon.pardon at rece.vub.ac.be Tue Oct 1 03:27:22 2013 From: antoon.pardon at rece.vub.ac.be (Antoon Pardon) Date: Tue, 01 Oct 2013 09:27:22 +0200 Subject: UnicodeDecodeError: 'utf-8' codec can't decode byte 0xb6 in position 0: invalid start byte In-Reply-To: References: <5247f6bb$0$29988$c3e8da3$5496439d@news.astraweb.com> Message-ID: <524A795A.4050202@rece.vub.ac.be> Op 01-10-13 01:14, ????? schreef: > ???? 1/10/2013 1:56 ??, ?/? Chris Angelico ??????: >> But what you're doing >> is charging your customers while you learn the very basics. > > I designed their websites and they are up and running. > Yes i have charged some money, but they gain what they paid for, a > running website, all of them. > > So, its not like i'm ripping off someone here. Yes you are. People don't just pay for a running website, with "running" meaning some vague: "is mostly accesible." People pay for some kind of guaranteed uptime. Since you don't have the skills to deliver that guarantee, you are in fact ripping them off. -- Antoon Pardon From nikos.gr33k at gmail.com Tue Oct 1 04:39:07 2013 From: nikos.gr33k at gmail.com (=?UTF-8?B?zp3Or866zr/Pgg==?=) Date: Tue, 01 Oct 2013 11:39:07 +0300 Subject: UnicodeDecodeError: 'utf-8' codec can't decode byte 0xb6 in position 0: invalid start byte In-Reply-To: References: <5247f6bb$0$29988$c3e8da3$5496439d@news.astraweb.com> Message-ID: ???? 1/10/2013 10:27 ??, ?/? Antoon Pardon ??????: > Op 01-10-13 01:14, ????? schreef: >> ???? 1/10/2013 1:56 ??, ?/? Chris Angelico ??????: >>> But what you're doing >>> is charging your customers while you learn the very basics. >> >> I designed their websites and they are up and running. >> Yes i have charged some money, but they gain what they paid for, a >> running website, all of them. >> >> So, its not like i'm ripping off someone here. > > Yes you are. People don't just pay for a running website, with "running" > meaning some vague: "is mostly accesible." People pay for some kind of > guaranteed uptime. Since you don't have the skills to deliver that > guarantee, you are in fact ripping them off. But it has uptime, VPS is always online, i dont make system wide changes except for the fat that i installed Python 3.3.2 for my personal account needs. If i encounter some problem i ask, bu the sad thing is that my provider doesn't care to help. From antoon.pardon at rece.vub.ac.be Tue Oct 1 05:10:40 2013 From: antoon.pardon at rece.vub.ac.be (Antoon Pardon) Date: Tue, 01 Oct 2013 11:10:40 +0200 Subject: UnicodeDecodeError: 'utf-8' codec can't decode byte 0xb6 in position 0: invalid start byte In-Reply-To: References: <5247f6bb$0$29988$c3e8da3$5496439d@news.astraweb.com> Message-ID: <524A9190.2030507@rece.vub.ac.be> Op 01-10-13 10:39, ????? schreef: > ???? 1/10/2013 10:27 ??, ?/? Antoon Pardon ??????: >> Op 01-10-13 01:14, ????? schreef: >>> ???? 1/10/2013 1:56 ??, ?/? Chris Angelico ??????: >>>> But what you're doing >>>> is charging your customers while you learn the very basics. >>> >>> I designed their websites and they are up and running. >>> Yes i have charged some money, but they gain what they paid for, a >>> running website, all of them. >>> >>> So, its not like i'm ripping off someone here. >> >> Yes you are. People don't just pay for a running website, with "running" >> meaning some vague: "is mostly accesible." People pay for some kind of >> guaranteed uptime. Since you don't have the skills to deliver that >> guarantee, you are in fact ripping them off. > > But it has uptime, VPS is always online, i dont make system wide changes > except for the fat that i installed Python 3.3.2 for my personal account > needs. That is has uptime is not enough. The question is: Should something go wrong, are you skilled enough to fix it within a reasonable time? In other words, when the side does go down, how long will it take you to have it up again? Going by the skill level you have shown here, you are unable to cope with such situations in a way that can be expected. > If i encounter some problem i ask, bu the sad thing is that my provider > doesn't care to help. Then either you have the wrong provider or you are so lacking in skill that your provider is fed up with spoon feeding you the basic solutions. Going with your history here, I'll go with the latter. Especially as I think it entirely possible that your provider has already helped you and provided you the necessary answers but you rejected them because you didn't like the particular style of the answer. -- Antoon Pardon From steve+comp.lang.python at pearwood.info Tue Oct 1 21:36:56 2013 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: 02 Oct 2013 01:36:56 GMT Subject: UnicodeDecodeError: 'utf-8' codec can't decode byte 0xb6 in position 0: invalid start byte References: <5247f6bb$0$29988$c3e8da3$5496439d@news.astraweb.com> Message-ID: <524b78b8$0$29974$c3e8da3$5496439d@news.astraweb.com> On Tue, 01 Oct 2013 09:27:22 +0200, Antoon Pardon wrote: > People pay for some kind of guaranteed uptime. You have *no idea* what sort of contract Nikos has with his customers. Nor do you know have any idea what fees he charges. For all we know, he is promising, and charging for, 99% uptime while delivering 99.9% uptime. I know you are getting off on hating Nikos, but take it elsewhere. -- Steven From antoon.pardon at rece.vub.ac.be Wed Oct 2 03:23:03 2013 From: antoon.pardon at rece.vub.ac.be (Antoon Pardon) Date: Wed, 02 Oct 2013 09:23:03 +0200 Subject: UnicodeDecodeError: 'utf-8' codec can't decode byte 0xb6 in position 0: invalid start byte In-Reply-To: <524b78b8$0$29974$c3e8da3$5496439d@news.astraweb.com> References: <5247f6bb$0$29988$c3e8da3$5496439d@news.astraweb.com> <524b78b8$0$29974$c3e8da3$5496439d@news.astraweb.com> Message-ID: <524BC9D7.2090901@rece.vub.ac.be> Op 02-10-13 03:36, Steven D'Aprano schreef: > On Tue, 01 Oct 2013 09:27:22 +0200, Antoon Pardon wrote: > >> People pay for some kind of guaranteed uptime. > > You have *no idea* what sort of contract Nikos has with his customers. > Nor do you know have any idea what fees he charges. For all we know, he > is promising, and charging for, 99% uptime while delivering 99.9% uptime. Which is beside the point. It is very well possible to rip someone of and in the mean time have a contract that makes ripping that person of, legal. It is also possible one behaves in a way similar as if ripping others off, but that your "victims" are lucky and don't experience a bad outcome (yet). > I know you are getting off on hating Nikos, but take it elsewhere. I know you are getting off insinuating hate of those who dare to critisize Nikos harshly, but take it elsewhere. -- Antoon Pardon From nikos.gr33k at gmail.com Wed Oct 2 04:49:30 2013 From: nikos.gr33k at gmail.com (=?UTF-8?B?zp3Or866zr/Pgg==?=) Date: Wed, 02 Oct 2013 11:49:30 +0300 Subject: UnicodeDecodeError: 'utf-8' codec can't decode byte 0xb6 in position 0: invalid start byte In-Reply-To: <524b78b8$0$29974$c3e8da3$5496439d@news.astraweb.com> References: <5247f6bb$0$29988$c3e8da3$5496439d@news.astraweb.com> <524b78b8$0$29974$c3e8da3$5496439d@news.astraweb.com> Message-ID: ???? 2/10/2013 4:36 ??, ?/? Steven D'Aprano ??????: > On Tue, 01 Oct 2013 09:27:22 +0200, Antoon Pardon wrote: > >> People pay for some kind of guaranteed uptime. > > You have *no idea* what sort of contract Nikos has with his customers. > Nor do you know have any idea what fees he charges. For all we know, he > is promising, and charging for, 99% uptime while delivering 99.9% uptime. > > I know you are getting off on hating Nikos, but take it elsewhere. It is good to know that some people understand me better then others. Thank you steven. From nikos.gr33k at gmail.com Wed Oct 2 04:52:54 2013 From: nikos.gr33k at gmail.com (=?UTF-8?B?zp3Or866zr/Pgg==?=) Date: Wed, 02 Oct 2013 11:52:54 +0300 Subject: UnicodeDecodeError: 'utf-8' codec can't decode byte 0xb6 in position 0: invalid start byte In-Reply-To: References: <5247f6bb$0$29988$c3e8da3$5496439d@news.astraweb.com> <524b78b8$0$29974$c3e8da3$5496439d@news.astraweb.com> Message-ID: ???? 2/10/2013 10:23 ??, ?/? Antoon Pardon ??????: > Op 02-10-13 03:36, Steven D'Aprano schreef: >> On Tue, 01 Oct 2013 09:27:22 +0200, Antoon Pardon wrote: >> >>> People pay for some kind of guaranteed uptime. >> >> You have *no idea* what sort of contract Nikos has with his customers. >> Nor do you know have any idea what fees he charges. For all we know, he >> is promising, and charging for, 99% uptime while delivering 99.9% uptime. > > Which is beside the point. It is very well possible to rip someone of > and in the mean time have a contract that makes ripping that person of, > legal. > > It is also possible one behaves in a way similar as if ripping > others off, but that your "victims" are lucky and don't experience > a bad outcome (yet). > >> I know you are getting off on hating Nikos, but take it elsewhere. > > I know you are getting off insinuating hate of those who dare to > critisize Nikos harshly, but take it elsewhere. I'am not ripping anyone off. I designed all their websites and i host all their websiutes. All their websites do work properly as expected. I only trial and error with my perosnal domain, personal account only. The only thing i did change system wide was installing Python 3.3.2 for perosnal reasons and rewrite the code for that. But, wy iam a sitting here and explainign myself to you..... From invalid at invalid.invalid Tue Oct 1 10:29:01 2013 From: invalid at invalid.invalid (Grant Edwards) Date: Tue, 1 Oct 2013 14:29:01 +0000 (UTC) Subject: UnicodeDecodeError: 'utf-8' codec can't decode byte 0xb6 in position 0: invalid start byte References: <5247f6bb$0$29988$c3e8da3$5496439d@news.astraweb.com> Message-ID: On 2013-09-30, ?????????? wrote: > I learn during the process. That's fine as long as your customers are told up front that what they're paying for is _not_ a working usable service, but rahter a training program for you personally (a training program that's failing rather badly, IMO). > That's how i deal with the situation. I challedge my self and then > try to confront the given situation _live_. That's a lousy attitude to have if your customers expect something that works rather than some in-progress hacked-up POS you're using for practice. -- Grant Edwards grant.b.edwards Yow! I want to mail a at bronzed artichoke to gmail.com Nicaragua! From dihedral88888 at gmail.com Tue Oct 1 03:01:01 2013 From: dihedral88888 at gmail.com (88888 Dihedral) Date: Tue, 1 Oct 2013 00:01:01 -0700 (PDT) Subject: class implementation In-Reply-To: References: <6e338858-b9e6-4745-9959-35d7c0c7724e@googlegroups.com> <02724244-c924-4cf0-9656-71ade6e435c2@googlegroups.com> Message-ID: <14031e5f-584f-4796-971f-c0b562351b9b@googlegroups.com> On Tuesday, October 1, 2013 3:34:08 AM UTC+8, Dave Angel wrote: > On 30/9/2013 08:41, markotaht at gmail.com wrote: > > > > > under variables, i mean, the int's and lists and strings and floats that the parent class uses. IF in parent class there is variable called location, then can i use the same variable in my sub class. > > > > Python doesn't actually have variables, but the things it documents as > > variables are local names within a method. Those are not visible > > outside of the method, regardless of whether you're in a class or a > > subclass. > > > > But perhaps you mean attributes. There are both class attributes and > > instance attributes, and the behavior is quite different. Roughly > > speaking a class attribute occurs only once per class, and all code can > > read its value with either Class.my_attrib or instance.my_attrib. It > > can be written with Class.my_attrib. > > > > On the other hand, instance attributes are usable by > > instance.my_attrib, regardless of whether the instance is a base class > An instance is an object of some class that could have its own attributes i.e. instance priviate properties during the run time. > or a child class. Each instance of the class gets a separate copy of > > such an attribute. They are normally defined in the __init__() method. > > > > If you don't happen to know the difference between a class an an > > instance of that class, then all the above will look like gibberish, and > > you need to do some studying first. > > > > -- > > DaveA From markotaht at gmail.com Sun Oct 6 09:15:51 2013 From: markotaht at gmail.com (markotaht at gmail.com) Date: Sun, 6 Oct 2013 06:15:51 -0700 (PDT) Subject: class implementation In-Reply-To: <6e338858-b9e6-4745-9959-35d7c0c7724e@googlegroups.com> References: <6e338858-b9e6-4745-9959-35d7c0c7724e@googlegroups.com> Message-ID: <052c735c-0d79-4a1b-91d9-b0676acf9ba8@googlegroups.com> There is this class file, it has its functions and variables. Now im greating my program, that uses the funcions from the class. BUt there are some functions missing from the class. So i want to add some extra funtions to the class, whidout altering the original source code, but by extending it in my code. But for that i need to use some variables that exsist in the original class. Is there a way i can acccsess them? From tjreedy at udel.edu Sun Oct 6 15:52:12 2013 From: tjreedy at udel.edu (Terry Reedy) Date: Sun, 06 Oct 2013 15:52:12 -0400 Subject: class implementation In-Reply-To: <052c735c-0d79-4a1b-91d9-b0676acf9ba8@googlegroups.com> References: <6e338858-b9e6-4745-9959-35d7c0c7724e@googlegroups.com> <052c735c-0d79-4a1b-91d9-b0676acf9ba8@googlegroups.com> Message-ID: On 10/6/2013 9:15 AM, markotaht at gmail.com wrote: > There is this class file, it has its functions and variables. Now im greating my program, that uses the funcions from the class. BUt there are some functions missing from the class. So i want to add some extra funtions to the class, whidout altering the original source code, but by extending it in my code. But for that i need to use some variables that exsist in the original class. Is there a way i can acccsess them? I though you already got an answer: subclass the class. -- Terry Jan Reedy From steve+comp.lang.python at pearwood.info Sun Oct 6 21:24:45 2013 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: 07 Oct 2013 01:24:45 GMT Subject: class implementation References: <6e338858-b9e6-4745-9959-35d7c0c7724e@googlegroups.com> <052c735c-0d79-4a1b-91d9-b0676acf9ba8@googlegroups.com> Message-ID: <52520d5c$0$29984$c3e8da3$5496439d@news.astraweb.com> On Sun, 06 Oct 2013 06:15:51 -0700, markotaht wrote: > There is this class file, it has its functions and variables. What's a class file? Do you mean a file containing only a single class? > Now im > greating my program, that uses the funcions from the class. They are called "methods". > BUt there > are some functions missing from the class. So i want to add some extra > funtions to the class, whidout altering the original source code, but by > extending it in my code. But for that i need to use some variables that > exsist in the original class. Is there a way i can acccsess them? When you say "variables", we prefer to say "attributes". A string variable is a variable that holds a string. An int variable is a variable that holds an int. A list variable is a variable that holds a list. A float variable is a variable that holds a float. So... ...a class variable is a variable that holds a class, and an instance variable is a variable that holds an instance. When extending classes, you access attributes exactly the same way you would access them if you were writing the class in the first place. class Test(object): def __init__(self, arg): self.arg = arg # Store the arg as an instance attribute. def method(self): print("arg is", self.arg) class NewTest(Test): # subclass to add new methods def add_one(self): return self.arg+1 For experts only: sometimes you need to extend the class itself. You can do that by adding methods to the class: def minus_one(self): return self.arg-1 NewTest.minus_one = minus_one -- Steven From markotaht at gmail.com Tue Oct 8 04:20:00 2013 From: markotaht at gmail.com (markotaht at gmail.com) Date: Tue, 8 Oct 2013 01:20:00 -0700 (PDT) Subject: class implementation In-Reply-To: <6e338858-b9e6-4745-9959-35d7c0c7724e@googlegroups.com> References: <6e338858-b9e6-4745-9959-35d7c0c7724e@googlegroups.com> Message-ID: <99e763bd-9757-4d44-ad8d-10fd8a2dc3bd@googlegroups.com> I cant just subclassing doesent work. It seem the init method of the source class also calls out another class. And the problem is, i can subclass the other class to with the required function but the end result is that it doesent work, since the source class cant accsess the subclass functions. The source code is pykkar. https://courses.cs.ut.ee/all/MTAT.03.100/2012_fall/uploads/opik/_downloads/pykkar.py I want to add it a new ability called left(). I cant manipulate the source class, cause then my comp will be the only one where the program runs. class pykkar_l(Pykkar): def left(self): self._world.execute("left") def _cmd_left(self): headings = (N,E,S,W) cur_tile = self._get_current_tile() cur_heading_index = headings.index(cur_tile.pykkar_heading) new_heading_index = (cur_heading_index - 1) % 4 cur_tile.pykkar_heading = headings[new_heading_index] self._update_pykkar_image(cur_tile) class world_l(World): def left(self): self._world.execute("left") These are my subclasses. For it to work. Class World, must obtain the method from subclass world_l From breamoreboy at yahoo.co.uk Tue Oct 8 04:41:47 2013 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Tue, 08 Oct 2013 09:41:47 +0100 Subject: class implementation In-Reply-To: <99e763bd-9757-4d44-ad8d-10fd8a2dc3bd@googlegroups.com> References: <6e338858-b9e6-4745-9959-35d7c0c7724e@googlegroups.com> <99e763bd-9757-4d44-ad8d-10fd8a2dc3bd@googlegroups.com> Message-ID: On 08/10/2013 09:20, markotaht at gmail.com wrote: To whom and to what are you replying? -- Roses are red, Violets are blue, Most poems rhyme, But this one doesn't. Mark Lawrence From davea at davea.name Tue Oct 8 06:31:20 2013 From: davea at davea.name (Dave Angel) Date: Tue, 8 Oct 2013 10:31:20 +0000 (UTC) Subject: class implementation References: <6e338858-b9e6-4745-9959-35d7c0c7724e@googlegroups.com> <99e763bd-9757-4d44-ad8d-10fd8a2dc3bd@googlegroups.com> Message-ID: On 8/10/2013 04:20, markotaht at gmail.com wrote: > I cant just subclassing doesent work. I can't parse that "sentence." > It seem the init method of the source class also calls out another class. And the problem is, i can subclass the other class to with the required function but the end result is that it doesent work, since the source class cant accsess the subclass functions. What's a "source class"? If you mean parent class, then say so. Otherwise, if you give it a name, we might be able to follow. But "source" and "other" don't narrow the field very much. A parent class can certainly access the child class (subclass) methods (not functions). But only if the instance (self) is an instance of the child class. That's the whole point of subclassing. > > The source code is pykkar. > > https://courses.cs.ut.ee/all/MTAT.03.100/2012_fall/uploads/opik/_downloads/pykkar.py > > I want to add it a new ability called left(). I cant manipulate the source class, cause then my comp will be the only one where the program runs. > > class pykkar_l(Pykkar): > def left(self): > self._world.execute("left") > > def _cmd_left(self): > headings = (N,E,S,W) > cur_tile = self._get_current_tile() > > cur_heading_index = headings.index(cur_tile.pykkar_heading) > new_heading_index = (cur_heading_index - 1) % 4 > cur_tile.pykkar_heading = headings[new_heading_index] > > self._update_pykkar_image(cur_tile) > > class world_l(World): > def left(self): > self._world.execute("left") > > These are my subclasses. For it to work. Class World, must obtain the method from subclass world_l Then it sounds like you should make sure that the global value "world" in that module is an instance of your world_l class, rather than an instance or World. And that the proxy is an instance of pykkar_l rather than of Pykkar. import pykkar layout = "fdlkjdsljdslfkjsdljfdsf" pykkar.world = world_I(layout) ??? = pykkar_l(pykkar.world) You don't show your own top-level code, so I can't integrate it in. By the way, it's conventional to use uppercase for class names, and lowercase for instances of those classes. I'm astounded that your class is using eval and multiprocessing before understanding classes and subclasses. -- DaveA From cs at zip.com.au Tue Oct 8 19:55:28 2013 From: cs at zip.com.au (Cameron Simpson) Date: Wed, 9 Oct 2013 10:55:28 +1100 Subject: class implementation In-Reply-To: <99e763bd-9757-4d44-ad8d-10fd8a2dc3bd@googlegroups.com> References: <99e763bd-9757-4d44-ad8d-10fd8a2dc3bd@googlegroups.com> Message-ID: <20131008235528.GA87129@cskk.homeip.net> On 08Oct2013 01:20, markotaht at gmail.com wrote: > I cant just subclassing doesent work. It seem the init method of the source class also calls out another class. And the problem is, i can subclass the other class to with the required function but the end result is that it doesent work, since the source class cant accsess the subclass functions. > > The source code is pykkar. > > https://courses.cs.ut.ee/all/MTAT.03.100/2012_fall/uploads/opik/_downloads/pykkar.py > > I want to add it a new ability called left(). I cant manipulate the source class, cause then my comp will be the only one where the program runs. > > class pykkar_l(Pykkar): > def left(self): > self._world.execute("left") [...] You normally need to call the superclasses' __init__ method as well. Example: def __init__(self): Pykkar.__init__(self) ... any of your own init stuff ... Likewise for your world_l class. BTW, it is conventional to start class names with an upper case letters. Just style, but it helps other people when reading your code. Cheers, -- Cameron Simpson It looks like you've got Mister Bus Error installed. - tjc From markotaht at gmail.com Thu Oct 10 14:34:53 2013 From: markotaht at gmail.com (markotaht at gmail.com) Date: Thu, 10 Oct 2013 11:34:53 -0700 (PDT) Subject: class implementation In-Reply-To: References: <99e763bd-9757-4d44-ad8d-10fd8a2dc3bd@googlegroups.com> Message-ID: <706f150c-add5-492b-833e-f8813ce6a96c@googlegroups.com> kolmap?ev, 9. oktoober 2013 2:55.28 UTC+3 kirjutas Cameron Simpson: > On 08Oct2013 01:20, wrote: > > > I cant just subclassing doesent work. It seem the init method of the source class also calls out another class. And the problem is, i can subclass the other class to with the required function but the end result is that it doesent work, since the source class cant accsess the subclass functions. > > > > > > The source code is pykkar. > > > > > > https://courses.cs.ut.ee/all/MTAT.03.100/2012_fall/uploads/opik/_downloads/pykkar.py > > > > > > I want to add it a new ability called left(). I cant manipulate the source class, cause then my comp will be the only one where the program runs. > > > > > > class pykkar_l(Pykkar): > > > def left(self): > > > self._world.execute("left") > > [...] > > > > You normally need to call the superclasses' __init__ method as well. > > Example: > > > > def __init__(self): > > Pykkar.__init__(self) > > ... any of your own init stuff ... > > > > Likewise for your world_l class. > > > > BTW, it is conventional to start class names with an upper case letters. Just > > style, but it helps other people when reading your code. > > > > Cheers, > > -- > > > It looks like you've got Mister Bus Error installed. - tjc OK so I did a took time of and read the pykkar code through. abd I found that there is a third class i have to implement. This Is the pykkar sourcecode # coding=UTF-8 """ Intro and usage =================== Pykkar is a virtual robot living in a virtual world. It can step, turn 90 degrees right, paint floor tiles, take and put down traffic cones and push boxes. It can detect whether there is a wall or other obstacle ahead or whether it's standing on a painted tile. It can be commanded by Python code (either procedural or OOP style) Pykkar's world can be created by a call to ``create_world`` (when using procedural style) or by instantiating class ``World`` (OOP style). Both take single argument, which is a string representation of the world. Lines in the string represent the rows in the world map. Each character represents one tile. Meaning of characters: # wall plain floor . painted floor b box on plain floor B box on painted floor ^ > v < pykkar on plain floor without cone (caret, greater-than, lowercase v, less-than) N E S W pykkar on painted floor without cone 1 2 3 4 5 6 7 8 9 cone stack on plain floor C single cone on painted floor Sample: create_world(''' #### #> # # 3# #### ''') this creates a world where 2x2 floor are is padded with walls. Pykkar is in north-west corner of the floor, looking east and in south-east corner there is a stack of 3 traffic cones. In procedural style, Pykkar can be commanded by calling global functions defined in this module (eg. ``step()``, ``right()``, etc). There are also functions for querying the world (``is_wall()``, ``is_box()``, etc). New compound commands can be defined by defining new functions in client module. In OOP style, Pykkar is represented by a separate object of class ``Pykkar``. In the start of the program, client code is supposed to create new world (eg. ``w = World(layout)``) and a Pykkar living in that world (eg ``p = Pykkar(w)``). Commands are given by calling the methods of Pykkar object. New commands should be defined by subclassing ``Pykkar``. """ """ Technical stuff ================ In order to reserve the main thread for executing commands (this way respective function calls can be written in client module's toplevel), tkinter window must run in a different thread. Unfortunately, tkinter runs reliably only in the main thread of the process. For this reason the execution is divided into 2 processes: the "main" process, which is just a shallow command proxy and the child process, which runs actual program logic and presents the world state in a tkinter window. Main process (ie. user module) normally begins by creating the world (with either ``create_world(...)`` or ``World(...)``). This spawns a child process which creates tkinter window representing the world. Main process then continues by executing user-provided function/method calls, which amounts to writing respective command strings to child process' stdin and reading results back from child process' stdout. Main player in child process is an object of class ``_WorldProper``. It keeps the data structures about world layout, responds to commands that alter the world state and runs a tkinter window. It reads periodically next command from stdin, acts upon it and writes result (may be None) to stdout. NB! as stdout from tkinter process is parsed, you can't just print out debug information to sys.stdout. Use sys.stderr instead! Reading from stdin blocks, as usual. This would make window temporariliy unresponsive when commands are given interactively (including stepping in a debugger). One case, where this can be annoying is when pykkar is run in a debugger and user wants to move the window. For this reason, the child process is divided into 2 threads. Main thread runs tkinter mainloop, and secondary thread takes care of stding and stdout. They communicate via two Queues (see _WorldProper.execute and _WordProper._process_commands). Creating new bitmaps ======================== TODO * embed base gif-s as base64 strings * http://effbot.org/tkinterbook/photoimage.htm * http://effbot.org/librarybook/base64.htm * http://www.tcl.tk/man/tcl8.4/TkCmd/photo.htm#M17 * load user provided gif-s, when present """ import sys import subprocess import os.path import traceback from threading import Thread try: import tkinter as tk # works in Python 3 except ImportError: import Tkinter as tk # works in Python 2 try: from queue import Queue # works in Python 3 except ImportError: from Queue import Queue # works in Python 2 N = (0,-1) W = (-1,0) E = (1,0) S = (0,1) class World(): """ Object, which creates GUI and mediates commands and results between Pykkar and GUI""" # This is actually a proxy for the actual world (_WorldProper, # which lives in another process) def __init__(self, layout_str): """ """ self.proc = subprocess.Popen ( (sys.executable, '-u', '-m', 'pykkar', repr(layout_str)), stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE, bufsize=0 ) # Check initialization result result_str = self.proc.stdout.readline().decode() if result_str.strip() != "OK": raise RuntimeError("World creation failed: " + eval(result_str)) def execute(self, command_str): if self.proc.poll() != None: return self.proc.stdin.write((command_str + "\n").encode()) self.proc.stdin.flush() result_str = self.proc.stdout.readline().decode() if result_str == "": return result = eval(result_str) if isinstance(result, Exception): raise result else: return result def create_world(layout_str): global world world = World(layout_str) def step(): world.execute("step") def right(): world.execute("right") def take(): world.execute("take") def put(): world.execute("put") def push(): world.execute("push") def paint(): world.execute("paint") def get_heading(): return world.execute("get_heading") def get_direction(): return world.execute("get_direction") def get_speed(): return world.execute("get_speed") def set_speed(value): world.execute("set_speed " + str(value)) def is_wall(): return world.execute("is_wall") def is_box(): return world.execute("is_box") def is_cone(): return world.execute("is_cone") def is_painted(): return world.execute("is_painted") class Pykkar: """ Just a handle to the world, it passes all commands to its world """ def __init__(self, world): self._world = world def step(self): self._world.execute("step") def right(self): self._world.execute("right") def take(self): self._world.execute("take") def put(self): self._world.execute("put") def push(self): self._world.execute("push") def paint(self): self._world.execute("paint") def get_heading(self): return self._world.execute("get_heading") def get_direction(self): return self._world.execute("get_direction") def get_speed(self): return self._world.execute("get_speed") def set_speed(self, value): self._world.execute("set_speed " + str(value)) def is_wall(self): return self._world.execute("is_wall") def is_box(self): return self._world.execute("is_box") def is_cone(self): return self._world.execute("is_cone") def is_painted(self): return self._world.execute("is_painted") class _WorldProper: _block_size = 32 def __init__(self, layout_str): self._command_queue = Queue(maxsize=1) self._result_queue = Queue(maxsize=1) self._setup_layout(layout_str) self._setup_ui() self._speed = 5 self.closed = False def run(self): try: self.root.after_idle(self._process_commands) self.root.mainloop() finally: # give a result in case there's a pending command from client self.closed = True if not self._command_queue.empty(): self._result_queue.put(Exception("World has ended"), block=False) def execute(self, command_str): """ meant for calling by outsiders (from another thread) """ if self.closed: # ignore command and return constant answer result = Exception("Window is closed") else: # put won't normally block, because commands are given and processed synchronously self._command_queue.put(command_str, block=False) # get blocks until mainloop handles the command result = self._result_queue.get() return result def _setup_layout(self, layout_str): self.layout = [] self.px = None self.py = None lines = layout_str.strip("\r\n").split("\n") longest_row_width = 0 for y in range(len(lines)): line = list(lines[y].strip("\r")) row = [] for x in range(len(line)): code = line[x] tile = _Tile.create_from_code(code) if tile.pykkar_heading != None: if self.px != None: raise RuntimeError("Only one pykkar is allowed") self.px = x self.py = y row.append(tile) self.layout.append(row) longest_row_width = max(longest_row_width, len(row)) if self.px == None: raise Exception("Pykkar is missing") # pad the layout for line in self.layout: # extend with floor in the right line.extend(list((longest_row_width - len(line)) * [_Tile('plain_floor')])) self.width = len(self.layout[0]) self.height = len(self.layout) def _setup_ui(self): self.root = tk.Tk() self.root.title("Pykkar") self.px_width = self.width * _WorldProper._block_size self.px_height = self.height * _WorldProper._block_size self.root.geometry("%dx%d" % (self.px_width, self.px_height)) self.root.resizable(0,0) self.canvas = tk.Canvas(self.root, width=self.px_width, height=self.px_height, highlightthickness=0) self.canvas.grid() # load images self.images = {} for name in ('wall', 'box', 'cone', 'plain_floor', 'painted_floor', 'pykkar_n', 'pykkar_e', 'pykkar_s', 'pykkar_w', 'pykkar_n_cone', 'pykkar_e_cone', 'pykkar_s_cone', 'pykkar_w_cone'): self.images[name] = self._load_image(name) # draw tiles for y in range(len(self.layout)): row = self.layout[y] for x in range(len(row)): tile = row[x] # base image tile.base_image_id = self.canvas.create_image ( x * _WorldProper._block_size, y * _WorldProper._block_size, image=self.images[tile.base_kind], anchor=tk.NW ) # item image if tile.item_kind != None and tile.pykkar_heading == None: tile.item_image_id = self.canvas.create_image ( x * _WorldProper._block_size, y * _WorldProper._block_size, image=self.images[tile.item_kind], anchor=tk.NW ) self._update_item_image(tile) # pykkar if tile.pykkar_heading != None: self.pykkar_id = self.canvas.create_image ( self.px * _WorldProper._block_size, self.py * _WorldProper._block_size, image=self._get_pykkar_img(tile.pykkar_heading, tile.item_kind), anchor=tk.NW ) # done with row # done with all rows self.canvas.tag_raise(self.pykkar_id) def _load_image(self, name): filename = "./" + name + ".gif" if os.path.exists(filename): return tk.PhotoImage(file=filename) else: return tk.PhotoImage(data=_image_data[name]) def _process_commands(self): """ is called periodically be WorldProper itself """ delay = _WorldProper._delays[self._speed-1] if not self._command_queue.empty(): try: command_str = self._command_queue.get() parts = command_str.strip().split() cmd = parts[0] args = tuple(parts[1:]) # look up corresponding method f = getattr(self, "_cmd_" + cmd) result = f(*args) #self._result_queue.put(result, block=False) self._result_queue.put(result) # reduce delay for some commands if cmd in ['set_speed', 'get_speed']: delay = 10 except BaseException as e: traceback.print_exc(file=sys.stderr) # no command may stay without result, otherwise client remains blocked self._result_queue.put(e) # schedule next processing self.root.after(delay, self._process_commands) def _cmd_get_x(self): return self.px def _cmd_get_y(self): return self.py def _cmd_is_wall(self): (next_x, next_y) = self._get_next_pos() # wall means either actuall wall block or outside of window return not self._is_valid_pos(next_x, next_y) \ or self.layout[next_y][next_x].base_kind == 'wall' def _cmd_is_box(self): return self._is_item("box") def _cmd_is_cone(self): return self._is_item("cone") def _cmd_is_painted(self): return self._get_current_tile().base_kind == 'painted_floor' def _is_item(self, item_kind): (next_x, next_y) = self._get_next_pos() return self._is_valid_pos(next_x, next_y) \ and self.layout[next_y][next_x].item_kind == item_kind def _cmd_step(self): (next_x, next_y) = self._get_next_pos() next_tile = self.layout[next_y][next_x] if not self._is_valid_pos(next_x, next_y) \ or next_tile.base_kind not in ('plain_floor', 'painted_floor') \ or next_tile.item_kind != None: raise Exception("Can't go to (%d,%d)" % (next_x, next_y)) current_tile = self._get_current_tile() next_tile.pykkar_heading = current_tile.pykkar_heading next_tile.item_kind = current_tile.item_kind next_tile.item_count = current_tile.item_count current_tile.pykkar_heading = None current_tile.item_kind = None current_tile.item_count = None # move pykkar image self.canvas.coords(self.pykkar_id, next_x * _WorldProper._block_size, next_y * _WorldProper._block_size) # update current position self.px = next_x self.py = next_y def _cmd_right(self): headings = (N,E,S,W) cur_tile = self._get_current_tile() cur_heading_index = headings.index(cur_tile.pykkar_heading) new_heading_index = (cur_heading_index + 1) % 4 cur_tile.pykkar_heading = headings[new_heading_index] self._update_pykkar_image(cur_tile) def _cmd_with_cone(self): tile = self._get_current_tile() return tile.item_kind == 'cone' def _cmd_take(self): cur_tile = self._get_current_tile() next_tile = self._get_next_tile() if cur_tile.item_kind != None: raise Exception("Pykkar already carries something") if next_tile.item_kind != 'cone': raise Exception("Pykkar can take only cones") cur_tile.item_kind = next_tile.item_kind cur_tile.item_count = 1 next_tile.item_count -= 1 if next_tile.item_count == 0: next_tile.item_kind = None next_tile.item_count = None self._update_pykkar_image(cur_tile) self._update_item_image(next_tile) def _cmd_put(self): cur_tile = self._get_current_tile() next_tile = self._get_next_tile() if cur_tile.item_kind == None: raise Exception("Not carrying anything") if next_tile.base_kind not in ['plain_floor', 'painted_floor'] \ or (next_tile.item_kind != cur_tile.item_kind \ and next_tile.item_kind != None): raise Exception("Can't put it there") if cur_tile.item_kind != 'cone' and next_tile.item_kind != None: raise Exception("There is one already") if cur_tile.item_kind == 'cone' and next_tile.item_count == 9: raise Exception("Can't stack more than 9 cones") next_tile.item_kind = cur_tile.item_kind next_tile.item_count = 1 if next_tile.item_count == None else next_tile.item_count + 1 cur_tile.item_kind = None cur_tile.item_count = None self._update_pykkar_image(cur_tile) self._update_item_image(next_tile) def _cmd_push(self): if self._cmd_with_cone(): raise Exception("Can't push when carrying something") (next_x, next_y) = self._get_next_pos() if not self._is_valid_pos(next_x, next_y): raise Exception("Nothing to push") cur_tile = self._get_current_tile() next_tile = self._get_next_tile() if next_tile.item_kind == None: raise Exception("Nothing to push") next_next_x = self.px + (cur_tile.pykkar_heading[0]*2) next_next_y = self.py + (cur_tile.pykkar_heading[1]*2) if not self._is_valid_pos(next_next_x, next_next_y): raise Exception("Nowhere to push") next_next_tile = self.layout[next_next_y][next_next_x] if next_next_tile.base_kind not in ['plain_floor', 'painted_floor']: raise Exception("Nowhere to push") if next_next_tile.item_kind != None: raise Exception("No room to push") # move item next_next_tile.item_kind = next_tile.item_kind next_next_tile.item_count = next_tile.item_count next_tile.item_kind = None next_tile.item_count = None self._update_item_image(next_tile) self._update_item_image(next_next_tile) # move pykkar self._cmd_step() def _cmd_paint(self): tile = self._get_current_tile() if tile.item_kind != None: raise Exception("Can't paint when carrying something") self.canvas.itemconfig(tile.base_image_id, image=self.images['painted_floor']) tile.base_kind = 'painted_floor' def _cmd_get_heading(self): return self._get_current_tile().pykkar_heading def _cmd_get_direction(self): heading = self._cmd_get_heading() if heading == N: return "N" elif heading == E: return "E" elif heading == S: return "S" else: assert heading == W return "W" def _cmd_set_speed(self, value): self._speed = int(value) def _cmd_get_speed(self): return self._speed def _get_current_tile(self): return self.layout[self.py][self.px] def _get_next_tile(self): (next_x, next_y) = self._get_next_pos() if not self._is_valid_pos(next_x, next_y): raise Exception("Not valid position") return self.layout[next_y][next_x] def _get_next_pos(self): tile = self._get_current_tile() next_x = self.px + tile.pykkar_heading[0] next_y = self.py + tile.pykkar_heading[1] return (next_x, next_y) def _is_valid_pos(self, x, y): return x >= 0 and x < self.width \ and y >= 0 and y < self.height def _update_pykkar_image(self, tile): new_image = self._get_pykkar_img(tile.pykkar_heading, tile.item_kind) self.canvas.itemconfig(self.pykkar_id, image=new_image) def _get_tile_pos(self, tile): for y in range(self.height): for x in range(self.width): if self.layout[y][x] is tile: return (x, y) def _update_item_image(self, tile): (x, y) = self._get_tile_pos(tile) # update image if tile.item_kind == None: #if tile.item_image_id != None: self.canvas.delete(tile.item_image_id) else: img = self.images[tile.item_kind] if tile.item_image_id == None: tile.item_image_id = self.canvas.create_image( x * _WorldProper._block_size, y * _WorldProper._block_size, image=img, anchor=tk.NW ) else: self.canvas.itemconfig(tile.item_image_id, image=img) # update count text if tile.item_count == None or tile.item_count <= 1: if tile.text_id != None: self.canvas.delete(tile.text_id) else: if tile.text_id == None: tile.text_id = self.canvas.create_text ( x * _WorldProper._block_size+3, y * _WorldProper._block_size+1, text=str(tile.item_count), anchor=tk.NW ) else: self.canvas.itemconfig(tile.text_id, text=str(tile.item_count)) def _get_pykkar_img(self, heading, item_kind): img_name = "pykkar" if heading == N: img_name += '_n' elif heading == E: img_name += '_e' elif heading == S: img_name += '_s' else: img_name += '_w' if item_kind != None: img_name += '_' + item_kind return self.images[img_name] _delays = (500, 300, 200, 150, 100, 75, 50, 30, 25, 20) class _CmdBroker(Thread): """ Mediates between stdin/stdout/stderr and World""" def __init__(self, world_proper): Thread.__init__(self) self.world_proper = world_proper def run(self): while True: command_str = sys.stdin.readline() if command_str == "": # client finished break result = self.world_proper.execute(command_str.strip()) print(repr(result)) class _Tile: @staticmethod def create_from_code(code): proto = _content_codes[code] return _Tile(proto.base_kind, proto.pykkar_heading, proto.item_kind, proto.item_count) def __init__(self, base_kind, pykkar_heading=None, item_kind=None, item_count=None): self.base_kind = base_kind self.pykkar_heading = pykkar_heading self.item_kind = item_kind self.item_count = item_count self.base_image_id = None self.item_image_id = None self.text_id = None def __eq__(self, other): return self.base_kind == other.base_kind \ and self.pykkar_heading == other.pykkar_heading \ and self.item_kind == other.item_kind \ and self.item_count == other.item_count def __ne__(self, other): return not self.__eq__(other) def __str__(self): return self.get_content_code() def get_content_code(self): for key in _content_codes: if _content_codes[key] == self: return key raise LookupError("No code found") _content_codes = { '#' : _Tile('wall'), ' ' : _Tile('plain_floor'), '.' : _Tile('painted_floor'), 'b' : _Tile('plain_floor', None, 'box', 1), 'B' : _Tile('painted_floor', None, 'box', 1), '^' : _Tile('plain_floor', N), '>' : _Tile('plain_floor', E), 'v' : _Tile('plain_floor', S), '<' : _Tile('plain_floor', W), 'N' : _Tile('painted_floor', N), 'E' : _Tile('painted_floor', E), 'S' : _Tile('painted_floor', S), 'W' : _Tile('painted_floor', W), '1' : _Tile('plain_floor', None, 'cone', 1), '2' : _Tile('plain_floor', None, 'cone', 2), '3' : _Tile('plain_floor', None, 'cone', 3), '4' : _Tile('plain_floor', None, 'cone', 4), '5' : _Tile('plain_floor', None, 'cone', 5), '6' : _Tile('plain_floor', None, 'cone', 6), '7' : _Tile('plain_floor', None, 'cone', 7), '8' : _Tile('plain_floor', None, 'cone', 8), '9' : _Tile('plain_floor', None, 'cone', 9), 'C' : _Tile('painted_floor', None, 'cone', 1), } _image_data = { 'box' : """ R0lGODlhIAAgALMAAAAAAMxVAMyAAP+AAP+qM/+qZv/VM//VZv/Vmf/VzP//mf//zP//// ///////////yH5BAEAAA8ALAAAAAAgACAAAwT/8MlJq70V6M27/5oEIEhClqdJquiaEoCY GAVx1LdNFMfeFzRgQSiIPUa1JG2H2zGXylpRZhMIBtZrFmvlagXMgRFJKILOmsVuekTcxF aAl0svxt3lMUIqLyAAX1laSGJ7BWwAM4eAO38CSU4FAGqLCjViInt5jz0JgENBZGZ7B4iK Yp+NAD8winaienxFAz8KADywWEiLIpaLugE/f4k0dot4mG1AdlepB39rfXkJ0VS/j8Y1ns Y0hcWxm2A1gEsL0nwJfDLdcgI9BxrBqjjH7JmyAUMjszh/SUWGELm5BsOXsR22vuERKKsI tUUH/dxyWO2IKzkjvg3Y8enGH28FWZLBSjTE2DtGPGCQzJPpDaM8A6BwfBQNwbd7m3QJ6h LnC6x1i9Cc8SVSUwEFJlQoTeECxbJYrIRIDbLkR1SGPiLJlOrkRxBEh3byFORl7JYxQtN6 wMC2rYUIADs=""", 'cone' : """ R0lGODlhIAAgALMAAOyVC/n5+f////T09AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAACH5BAEAAAIALAAAAAAgACAAAwRTUMhJq7046827/2AoTgAwbqV5Xqm6Vu1r ta5M128Z7KUsAbsgbnRL2YpDENJ4AgafPaLzKUxylrEQlqbUUb9WjPdLjV63yAx6Hfa53/ C4fE6fRwAAOw==""", 'painted_floor' : """ R0lGODlhIAAgAPcAAAAAAAAABgAAHQAATAAAlgAA/wAEAAAEBgAEHQAETAAElgAE/wATAA ATBgATHQATTAATlgAT/wAxAAAxBgAxHQAxTAAxlgAx/wBhAABhBgBhHQBhTABhlgBh/wCm AACmBgCmHQCmTACmlgCm/wD/AAD/BgD/HQD/TAD/lgD//wYAAAYABgYAHQYATAYAlgYA/w YEAAYEBgYEHQYETAYElgYE/wYTAAYTBgYTHQYTTAYTlgYT/wYxAAYxBgYxHQYxTAYxlgYx /wZhAAZhBgZhHQZhTAZhlgZh/wamAAamBgamHQamTAamlgam/wb/AAb/Bgb/HQb/TAb/lg b//x0AAB0ABh0AHR0ATB0Alh0A/x0EAB0EBh0EHR0ETB0Elh0E/x0TAB0TBh0THR0TTB0T lh0T/x0xAB0xBh0xHR0xTB0xlh0x/x1hAB1hBh1hHR1hTB1hlh1h/x2mAB2mBh2mHR2mTB 2mlh2m/x3/AB3/Bh3/HR3/TB3/lh3//0wAAEwABkwAHUwATEwAlkwA/0wEAEwEBkwEHUwE TEwElkwE/0wTAEwTBkwTHUwTTEwTlkwT/0wxAEwxBkwxHUwxTEwxlkwx/0xhAExhBkxhHU xhTExhlkxh/0ymAEymBkymHUymTEymlkym/0z/AEz/Bkz/HUz/TEz/lkz//5YAAJYABpYA HZYATJYAlpYA/5YEAJYEBpYEHZYETJYElpYE/5YTAJYTBpYTHZYTTJYTlpYT/5YxAJYxBp YxHZYxTJYxlpYx/5ZhAJZhBpZhHZZhTJZhlpZh/5amAJamBpamHZamTJamlpam/5b/AJb/ Bpb/HZb/TJb/lpb///8AAP8ABv8AHf8ATP8Alv8A//8EAP8EBv8EHf8ETP8Elv8E//8TAP 8TBv8THf8TTP8Tlv8T//8xAP8xBv8xHf8xTP8xlv8x//9hAP9hBv9hHf9hTP9hlv9h//+m AP+mBv+mHf+mTP+mlv///////////////////////////////////////////yH5BAAAAA AALAAAAAAgACAABwj/AAEoS0aPYLJkwxAWVDaP4cCCBRFKhAggWcODBjFaTDZwXsaMDA8+ 5OhxYcOGJjkmHHbxJMeCJUmGdLjSoMeBDg3OvHiwZsSbN0lm/NlxZNCIJh0CVSkx50WcHi UK7WnR6MGlDE+OtLm1qMyRTr1qlWkwZUiiG5OK9NiSY9qqVT3WtEk1YU6IOumtxOk24kiz WON2pIsXr9CQEhOSbdk1qNaLEWVGVXuxJmSMO0NeHirUpE+4C93OdXo4KMaUMMe6bOryLO eBOjeCXSp59mDAO0vDzj3QLmjdOiuzLh23LmnNGZfC5Iv4qmSSl++uzfrXKFRliJEeXO72 8tPiifuGRh+mOWzdzTDJducbk+v2q7PfM274mSfS7GS5R4UqPmLCm6MFJZ12UuWEHFftrU SPasQd5dVW2nlFID0ArKYdVkBJ+B4AAQEAOw==""", 'plain_floor' : """ R0lGODlhIAAgAOf1AAAAAAAAMwAAZgAAmQAAzAAA/wArAAArMwArZgArmQArzAAr/wBVAA BVMwBVZgBVmQBVzABV/wCAAACAMwCAZgCAmQCAzACA/wCqAACqMwCqZgCqmQCqzACq/wDV AADVMwDVZgDVmQDVzADV/wD/AAD/MwD/ZgD/mQD/zAD//zMAADMAMzMAZjMAmTMAzDMA/z MrADMrMzMrZjMrmTMrzDMr/zNVADNVMzNVZjNVmTNVzDNV/zOAADOAMzOAZjOAmTOAzDOA /zOqADOqMzOqZjOqmTOqzDOq/zPVADPVMzPVZjPVmTPVzDPV/zP/ADP/MzP/ZjP/mTP/zD P//2YAAGYAM2YAZmYAmWYAzGYA/2YrAGYrM2YrZmYrmWYrzGYr/2ZVAGZVM2ZVZmZVmWZV zGZV/2aAAGaAM2aAZmaAmWaAzGaA/2aqAGaqM2aqZmaqmWaqzGaq/2bVAGbVM2bVZmbVmW bVzGbV/2b/AGb/M2b/Zmb/mWb/zGb//5kAAJkAM5kAZpkAmZkAzJkA/5krAJkrM5krZpkr mZkrzJkr/5lVAJlVM5lVZplVmZlVzJlV/5mAAJmAM5mAZpmAmZmAzJmA/5mqAJmqM5mqZp mqmZmqzJmq/5nVAJnVM5nVZpnVmZnVzJnV/5n/AJn/M5n/Zpn/mZn/zJn//8wAAMwAM8wA ZswAmcwAzMwA/8wrAMwrM8wrZswrmcwrzMwr/8xVAMxVM8xVZsxVmcxVzMxV/8yAAMyAM8 yAZsyAmcyAzMyA/8yqAMyqM8yqZsyqmcyqzMyq/8zVAMzVM8zVZszVmczVzMzV/8z/AMz/ M8z/Zsz/mcz/zMz///8AAP8AM/8AZv8Amf8AzP8A//8rAP8rM/8rZv8rmf8rzP8r//9VAP 9VM/9VZv9Vmf9VzP9V//+AAP+AM/+AZv+Amf+AzP+A//+qAP+qM/+qZv+qmf+qzP+q///V AP/VM//VZv/Vmf/VzP///////////////////////////////////////////yH+EUNyZW F0ZWQgd2l0aCBHSU1QACwAAAAAIAAgAAAI/gABKEtGj2CyZMMQFlQ2j+HAggURSoQIIFnD gwYxWkw2cF7GjAwPPuTocWHDhiY5Jhx28STHgiVJhnS40qDHgQ4Nzrx4sGbEmzdJZvzZcW TQiCYdAlUpMedFnB4lCu1p0ejBpQxPjrS5tajMkU69apVpMGVIohuTivTYkmPaqlU91rRJ NWFOiDrprcTpNuJIs1jjdqSLF6/QkBITkm3ZNajWixFlRlV7sSZkjDtDXh4q1KRPuAvdzn V6OCjGlDDHumzq8izngTo3gl0qefZgwDtLw8490C5o3Torsy4dty5pzRmXwuSL+Kpkkpfv rs361yhUZYiRHlzu9vLT4on7R4Yfpjls3c0wyXbnG5Pr9quz3zNu+Jkn0uxkuUeFKj5iwp ujBSWddlLlhBxX7a1Ej2rEHeXVVtp5RSA9AKymHVZASfgeAAEBADs=""", 'pykkar_e' : """ R0lGODlhIAAgALMAAAAAAJ83G8febf///wAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAACH5BAEAAAMALAAAAAAgACAAAwSMcMhJq7046827/6AWjKRXXmRaAagqAXCc sVY6DrGgCzIFiyTAbriL/X4bGHFJm/QsyqW0mRRKr1SMFWsscrY8Y5TnTV7JRWv2Aia21d 8zOLqGytNlTXu47ddXezpdeU5sOWeESDNjWDhPFTYvgWEsipCRkkyAGJiSMS5Ofxw2AZsh pCGpqqusra6vFREAOw==""", 'pykkar_e_cone' : """ R0lGODlhIAAgALMAAAAAAJ83Gw2VC8l/CeyVC8febfT09PT09AAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAACH5BAEAAAIALAAAAAAgACAAAwS2UMhJq7046827H14oDaQVnKiHDkQ5oXAF XDDbDkCeZ7NZE0BSYaiT9TArIMFAGgyJutluY1sym8+jpHjJVQ0GpZDoyRW+YeDVWdBiAE +0EnzDceBDuTrdBGnwZ3NpVnt1PHGCiVZNf4iFj2Auh3mKhEuSk4GPliRuXXh6i06engI6 oWNtFFMZcqlkUUhJaq9QUqUCMAFVtaqruD83UQAxv1QtPsUTrBeYEroBIiPS1NXW19jZGh EAOw==""", 'pykkar_n' : """ R0lGODlhIAAgALMAAAAAAJ83G8febf///wAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAACH5BAEAAAMALAAAAAAgACAAAwSecMhJq724grxp59N3fYCIbWU2pKkammPr SnA8a3PtdmXfh7dAgLYSGI8CX2eYEdIAyOhxI2RenEVkTwqoNofQaSkZRnoxznJSqxacr4 HxlKvFotvt9XRgPUXlc3s5XEpqOhp0UkYyKoCLimQ3NH+QhyOUXJIeZXlJmnxCnWJvX2mQ nqRoXo5aoH2qWJ0pqaqbhRK0aJqvn72+v8DBwBEAOw==""", 'pykkar_n_cone' : """ R0lGODlhIAAgALMAAAAAAJ83Gw2VC8l/CeyVC8febfT09PT09AAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAACH5BAEAAAIALAAAAAAgACAAAwS5UMhJq724grxp59N3fYCIbWUmpKkamoNV mio9xB7tivftuSNJaUDokYCCQEC4KtwM0Nuss8wohYCCcwCNagsbZfVybW4J6PTgCxBbl1 ktMa3+FtyYa/zZ7a+1eGQBJVt9hn9gZXl7dI1of2E7Z450kEhZN5SNazoyfIagUkifoKE4 KpmamqcYqaqrqK+yrBQ9tre4uW96dr1hirtwcb1gScB5eMN2KYEqwFNTEs3OSNLV19jZ2t vc2REAOw==""", 'pykkar_s' : """ R0lGODlhIAAgALMAAAAAAJ83G8febf///wAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAACH5BAEAAAMALAAAAAAgACAAAwSZcMhJq7046827/2DwiV1gSkCqptTJmWIq zLQASDCpwQNQ16zc6+T70W7CDUxmPPI+xV+UBeo1o7MbtGmTQrFAqXbDvBrHGuxKde6oxd 6d9XiW6i6mtxQIcOFNcHRHA34WMGc+bIOFFUuIYFqMLURcfDh3higrVz2XbhNrKihVnWRV VKYeomRoaTeoFyytsWOzFLWpoBm2pCARADs=""", 'pykkar_s_cone' : """ R0lGODlhIAAgALMAAAAAAJ83Gw2VC8l/CeyVC8febfT09PT09AAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAACH5BAEAAAIALAAAAAAgACAAAwS8UMhJq7046827/2DwiV1gSkCqptTJmWJa zHQBSDCpwQJQ16zc6+T70W5CzGDJbDqfTSVhSq1arwPN4MrFZrXd8JKzNJjP6PSYAyin34 bljV3YhqdLW6doFw/0Oz0zfV1/NjoXJkV1cGZ5KS6JJjQDjXl6kRYwNYRUhpiIFTCLnQSX c5miRJRop6ihoigqbnF/N697E21Lpiw9ICgUUbrAvhNrFHN7xscVzBosz863bMrKF9Yb17 8Y28DfAhEAOw==""", 'pykkar_w' : """ R0lGODlhIAAgALMAAAAAAJ83G8febf///wAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAACH5BAEAAAMALAAAAAAgACAAAwSJcMhJq7046827/2AIBmQpAkBVmuKAqiUq U2uAvRQg7Dw+1LbLS8YrCnC120xnNM40PldzKktZrLlidXrEZlC9MNd76zGP3COHCRZT17 tzPE0WxuVyJ3x+r+L3XUR0gGladWWFZkpkeU1RSi5bdCtQjG19P0lXX34TmhVkQEEWLBxA LaipqqusIREAOw==""", 'pykkar_w_cone' : """ R0lGODlhIAAgALMAAAAAAJ83Gw2VC8l/CeyVC8febfT09PT09AAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAACH5BAEAAAIALAAAAAAgACAAAwS4UMhJq7046827/6A3hEJgntYwZgBQnWg1 ECvrUnCrqwQNY60KoEAs8noDWADo0hWNA0PPF7Poms8CzyBFboKTIfFI4E4HusuNMlSZe2 8aEawBHOPxQXGdccO7ZYB6cxtuXXiCexoqUYeAeYoYjIF/U5CEi5OUlUiRdZOIZ54sY36c cnRfUJaCqWyuYlqMeRxBV7GyUWdVanwCTqW7P1Y2OzNUJmovSksTPEnJHcwpKiQdNdbZ2t vZEQA7""", 'wall' : """ R0lGODlhIAAgAIQAAAAAAGpsN2tVRWBgYHNzcwFhuCiX/owtALplDZlvQNl8ALqOV6quYv +1Us/RbYSEhJaWlrmoktWWhsa9lN+uot7Wrf/aqOvnkv//hMDAwMbGxtjY2AAAAAAAAAAA AAAAACH/C05FVFNDQVBFMi4wAwEAAAAh+QQAZAAAACwAAAAAIAAgAIcAAAAAADMAAGYAAJ kAAMwAAP8AKwAAKzMAK2YAK5kAK8wAK/8AVQAAVTMAVWYAVZkAVcwAVf8AgAAAgDMAgGYA gJkAgMwAgP8AqgAAqjMAqmYAqpkAqswAqv8A1QAA1TMA1WYA1ZkA1cwA1f8A/wAA/zMA/2 YA/5kA/8wA//8zAAAzADMzAGYzAJkzAMwzAP8zKwAzKzMzK2YzK5kzK8wzK/8zVQAzVTMz VWYzVZkzVcwzVf8zgAAzgDMzgGYzgJkzgMwzgP8zqgAzqjMzqmYzqpkzqswzqv8z1QAz1T Mz1WYz1Zkz1cwz1f8z/wAz/zMz/2Yz/5kz/8wz//9mAABmADNmAGZmAJlmAMxmAP9mKwBm KzNmK2ZmK5lmK8xmK/9mVQBmVTNmVWZmVZlmVcxmVf9mgABmgDNmgGZmgJlmgMxmgP9mqg BmqjNmqmZmqplmqsxmqv9m1QBm1TNm1WZm1Zlm1cxm1f9m/wBm/zNm/2Zm/5lm/8xm//+Z AACZADOZAGaZAJmZAMyZAP+ZKwCZKzOZK2aZK5mZK8yZK/+ZVQCZVTOZVWaZVZmZVcyZVf +ZgACZgDOZgGaZgJmZgMyZgP+ZqgCZqjOZqmaZqpmZqsyZqv+Z1QCZ1TOZ1WaZ1ZmZ1cyZ 1f+Z/wCZ/zOZ/2aZ/5mZ/8yZ///MAADMADPMAGbMAJnMAMzMAP/MKwDMKzPMK2bMK5nMK8 zMK//MVQDMVTPMVWbMVZnMVczMVf/MgADMgDPMgGbMgJnMgMzMgP/MqgDMqjPMqmbMqpnM qszMqv/M1QDM1TPM1WbM1ZnM1czM1f/M/wDM/zPM/2bM/5nM/8zM////AAD/ADP/AGb/AJ n/AMz/AP//KwD/KzP/K2b/K5n/K8z/K///VQD/VTP/VWb/VZn/Vcz/Vf//gAD/gDP/gGb/ gJn/gMz/gP//qgD/qjP/qmb/qpn/qsz/qv//1QD/1TP/1Wb/1Zn/1cz/1f///wD//zP//2 b//5n//8z///8AAAAAAAAAAAAAAAAI/wABAECjbBKaSWIATFJ28GDCgQUdKmRoEOHEimgS EsSYcCFHiB8pNuxY0WBCkSYVlsy48CBCMQnFjOxI8WXMmQoPZrrxUGBGmR1d9ug5EGZGAD DF8ASQSRlDgVCRwlza9GlUgTB9Eiup0SlSNFq5TvTK1GnDSUsjKhO4EGXas18xGpwIV6Zc sBvrTgrpsiHSvW5SRsQoNTCaTCf3ws14NrHLSW5iwnw5sGImiUkR4l1JeeDSv2cjPxQzFK tcmTmhXkZ6eW/W1WzBirmcFGTXvVKV3r49KTfYlhZbHsT6myJNNMOhEjwYGaLL5s4h8w5M U3HwwUc9Vjy+HUBEh0G3n5lsrJGjGOAIS593WHzk0fWaB+KEajRrQ8RE60sGWtZq1KSvmX WVTw+hsZVOXa111EAHToLYWGuNdVZaBkXY1oTjVRRXY3RpaBeHeXlo2VE1+QUfbd7dVZR1 iclVFIeDKRbTUPGdBZlFQDFWWV/4YfUZfPPVBpp4TIEVG2suoTbXkbP9tNtEgeW2FEEPUt lbUu1ZFCKBHR6XXEAAOw==""", } #def _create_rotated_image(source, source_height, source_width): # Does not work ... but idea is good :) # """ returns new tk.PhotoImage with source pixels rotated 90deg clockwise. # Assuming source image has square dimensions # """ # #source_height = int(source['height']) # #source_width = int(source['width']) # #print(source['height'], source['height'], file=sys.stderr) # dest = tk.PhotoImage(width=source_height, height=source_width) # for x in range(source_width): # for y in range(source_height): # px = source.get(x, y) # print("pixel: ", px, file=sys.stderr) # dest.put(px, to=(source_height-y, x)) # # return dest if __name__ == '__main__': if len(sys.argv) < 2: layout_str = "####\n#> #\n# #\n####" else: layout_str = eval(sys.argv[1]) try: wp = _WorldProper(layout_str) cb = _CmdBroker(wp) cb.start() print("OK") except: print(repr(traceback.format_exc())) # run mainloop wp.run() So I have come up with this code from pykkar import * create_world(""" ######## # # # v# # # # # # # ######## """) class world_t?iend(World): def left(self): world.excecute("left") class pykkar_t?iend(Pykkar): def left(self): self._world.excecute("left") class _WorldProper_t?iend(_WorldProper): def _cmd_right(self): headings = (N,E,S,W) cur_tile = self._get_current_tile() cur_heading_index = headings.index(cur_tile.pykkar_heading) new_heading_index = (cur_heading_index - 1) % 4 cur_tile.pykkar_heading = headings[new_heading_index] self._update_pykkar_image(cur_tile) left() When I run my code I get this error. Traceback (most recent call last): File "C:\Users\MarkoPC\Desktop\python\pykkar_test.py", line 21, in class _WorldProper_t?iend(_WorldProper): NameError: name '_WorldProper' is not defined I did not wan to but the source in here because it is just so god damn long. But some of you wanted it, so here it is :D From cs at zip.com.au Thu Oct 10 18:07:59 2013 From: cs at zip.com.au (Cameron Simpson) Date: Fri, 11 Oct 2013 09:07:59 +1100 Subject: class implementation In-Reply-To: <706f150c-add5-492b-833e-f8813ce6a96c@googlegroups.com> References: <706f150c-add5-492b-833e-f8813ce6a96c@googlegroups.com> Message-ID: <20131010220759.GA20768@cskk.homeip.net> On 10Oct2013 11:34, markotaht at gmail.com wrote: > OK so I did a took time of and read the pykkar code through. abd > I found that there is a third class i have to implement. > This Is the pykkar sourcecode [... lots and lots of docstring and code ...] [... and finally a little more messgae ...] > I did not wan to but the source in here because it is just so god > damn long. But some of you wanted it, so here it is :D As a matter of readability, if I really need to include a huge body of text I append it below the end of my message, and say something like this (pretending I were writing your message): OK so I did a took time of and read the pykkar code through. abd I found that there is a third class i have to implement. I've appended the relevant pykkar source below this message. So I have come up with this code: [...] That way your message does not get hidden by the (overly long IMO) included material and readers can get on with looking at your stuff, knowing that if necessary they can wade through the other stuff. Cheers, -- Cameron Simpson Any profit should go to Arnie's `get the daemon carved on Mount Rushmore' fund. - Marty Albini, DOD0550, martya at sdd.hp.com From piet at vanoostrum.org Thu Oct 10 19:29:04 2013 From: piet at vanoostrum.org (Piet van Oostrum) Date: Thu, 10 Oct 2013 19:29:04 -0400 Subject: class implementation References: <99e763bd-9757-4d44-ad8d-10fd8a2dc3bd@googlegroups.com> <706f150c-add5-492b-833e-f8813ce6a96c@googlegroups.com> Message-ID: markotaht at gmail.com writes: > > OK so I did a took time of and read the pykkar code through. abd I found that there is a third class i have to implement. [...] > So I have come up with this code > from pykkar import * > > create_world(""" > ######## > # # > # v# > # # > # # > # # > ######## > """) > > class world_t?iend(World): > def left(self): > world.excecute("left") > > class pykkar_t?iend(Pykkar): > def left(self): > self._world.excecute("left") > > class _WorldProper_t?iend(_WorldProper): > def _cmd_right(self): Should that not be _cmd_left? > headings = (N,E,S,W) > cur_tile = self._get_current_tile() > > cur_heading_index = headings.index(cur_tile.pykkar_heading) > new_heading_index = (cur_heading_index - 1) % 4 > cur_tile.pykkar_heading = headings[new_heading_index] > > self._update_pykkar_image(cur_tile) > > > left() > > When I run my code I get this error. > Traceback (most recent call last): > File "C:\Users\MarkoPC\Desktop\python\pykkar_test.py", line 21, in > class _WorldProper_t?iend(_WorldProper): > NameError: name '_WorldProper' is not defined from import * doesn't import names that start with underscore (_). So therefore _WorldProper is not defined. from import * is considered bad practice anyway. It is better just to import the things you need. from pykkar import World, Pykkar, _WorldProper I have looked a bit in this pykkar.py and I think it is badly structured for extension. The three classes are too strongly connected and it is difficult to get three subclasses connected in the proper way without duplicating code. But anyway you will have to do that when you create your world. -- Piet van Oostrum WWW: http://pietvanoostrum.com/ PGP key: [8DAE142BE17999C4] From ben+python at benfinney.id.au Thu Oct 10 19:51:04 2013 From: ben+python at benfinney.id.au (Ben Finney) Date: Fri, 11 Oct 2013 10:51:04 +1100 Subject: class implementation References: <99e763bd-9757-4d44-ad8d-10fd8a2dc3bd@googlegroups.com> <706f150c-add5-492b-833e-f8813ce6a96c@googlegroups.com> Message-ID: <7wbo2wofbr.fsf@benfinney.id.au> Piet van Oostrum writes: > from import * is considered bad practice anyway. It is better just to import the things you need. > > from pykkar import World, Pykkar, _WorldProper Or, even better, be explicit: import pykkar ? foo = pykkar.World() -- \ ?Computer perspective on Moore's Law: Human effort becomes | `\ twice as expensive roughly every two years.? ?anonymous | _o__) | Ben Finney From markotaht at gmail.com Tue Oct 8 10:05:26 2013 From: markotaht at gmail.com (markotaht at gmail.com) Date: Tue, 8 Oct 2013 07:05:26 -0700 (PDT) Subject: class implementation In-Reply-To: <6e338858-b9e6-4745-9959-35d7c0c7724e@googlegroups.com> References: <6e338858-b9e6-4745-9959-35d7c0c7724e@googlegroups.com> Message-ID: Parent class is at the link. From rhodri at wildebst.demon.co.uk Tue Oct 8 19:41:33 2013 From: rhodri at wildebst.demon.co.uk (Rhodri James) Date: Wed, 09 Oct 2013 00:41:33 +0100 Subject: class implementation References: <6e338858-b9e6-4745-9959-35d7c0c7724e@googlegroups.com> Message-ID: On Tue, 08 Oct 2013 15:05:26 +0100, wrote: > Parent class is at the link. Please quote some context when you reply. What link? Then again, I'm not about to click on some random link someone posts to a newsgroup. Apart from being one of the classic ways to get a virus onto my computer, it's rather selfish of you. While your newsgroup posting will stay around essentially forever, the link will eventually rot. At some point in the future, no one will be able to follow that link, so no one will be able to learn from what you have done. Please show your working, it makes it so much easier for the rest of us to understand what you meant when you use terms so loosely! -- Rhodri James *-* Wildebeest Herder to the Masses From rustompmody at gmail.com Tue Oct 1 13:12:52 2013 From: rustompmody at gmail.com (rusi) Date: Tue, 1 Oct 2013 10:12:52 -0700 (PDT) Subject: VERY BASIC HELP In-Reply-To: <62dfe971-f28a-4b01-a696-b2099dacd91b@googlegroups.com> References: <380132bc-bc9c-4d57-95d8-dc01f26f47a5@googlegroups.com> <62dfe971-f28a-4b01-a696-b2099dacd91b@googlegroups.com> Message-ID: On Monday, September 30, 2013 11:20:16 PM UTC+5:30, vignesh.h... at gmail.com wrote: > Thank you both so much! I'll be sure to make more pertinent subject lines now :) Thanks for the detailed explanations! Clearly, I've just started learning this language ~20 minutes before I made this post, and am still learning the basics. Do you guys know of any guides for a beginner? I am definitely willing to take the time to learn in depth :) Have you seen http://docs.python.org/2/tutorial/ ?? Its kind of required reading for beginners. A little time spent on that will save a lot on head-scratchers avoided. After that there are the language and the library references http://docs.python.org/2/reference/index.html#reference-index http://docs.python.org/2/library/index.html#library-index The library is ok if you stick to modules that make sense to you. The language-ref is too heavy-going for a beginner -- other material/books may be preferable. From breamoreboy at yahoo.co.uk Tue Oct 1 13:25:05 2013 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Tue, 01 Oct 2013 18:25:05 +0100 Subject: VERY BASIC HELP In-Reply-To: <62dfe971-f28a-4b01-a696-b2099dacd91b@googlegroups.com> References: <380132bc-bc9c-4d57-95d8-dc01f26f47a5@googlegroups.com> <62dfe971-f28a-4b01-a696-b2099dacd91b@googlegroups.com> Message-ID: On 30/09/2013 18:50, vignesh.harikrishna at gmail.com wrote: > Thank you both so much! I'll be sure to make more pertinent subject lines now :) Thanks for the detailed explanations! Clearly, I've just started learning this language ~20 minutes before I made this post, and am still learning the basics. Do you guys know of any guides for a beginner? I am definitely willing to take the time to learn in depth :) > As a newbie you might like to try the tutor mailing list see https://mail.python.org/mailman/listinfo/tutor -- Cheers. Mark Lawrence From breamoreboy at yahoo.co.uk Tue Oct 1 04:26:55 2013 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Tue, 01 Oct 2013 09:26:55 +0100 Subject: I haev fixed it In-Reply-To: <20130930235914.GB13553@secure.superhost.gr> References: <20130930235914.GB13553@secure.superhost.gr> Message-ID: On 01/10/2013 00:59, nikos at secure.superhost.gr wrote: > http://superhost.gr/warning.html > I believe this is set at a suitable level http://www.youtube.com/watch?v=HyTPFyyA-mA -- Cheers. Mark Lawrence From rosuav at gmail.com Tue Oct 1 04:59:09 2013 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 1 Oct 2013 18:59:09 +1000 Subject: I haev fixed it In-Reply-To: <20130930235914.GB13553@secure.superhost.gr> References: <20130930235914.GB13553@secure.superhost.gr> Message-ID: On Tue, Oct 1, 2013 at 9:59 AM, wrote: > http://superhost.gr/warning.html Okay, who posted this? I'm thinking possibly someone may have access to your server again. ChrisA From nikos.gr33k at gmail.com Tue Oct 1 06:16:59 2013 From: nikos.gr33k at gmail.com (=?UTF-8?B?zp3Or866zr/Pgg==?=) Date: Tue, 01 Oct 2013 13:16:59 +0300 Subject: I haev fixed it In-Reply-To: References: <20130930235914.GB13553@secure.superhost.gr> Message-ID: ???? 1/10/2013 11:59 ??, ?/? Chris Angelico ??????: > On Tue, Oct 1, 2013 at 9:59 AM, wrote: >> http://superhost.gr/warning.html > > Okay, who posted this? I'm thinking possibly someone may have access > to your server again. > > ChrisA > Hi chris, I just saw thios thread, although half an hour earlier i opened anew one myself. Just logged in via FTP to my server and i saw an uploade file named "Warnign html" Contents were: WARNING I am incompetent. Do not hire me! Question: WHO AND MOST IMPORTNTANLY HOW DID HE MANAGED TO UPLOAD THIS FILE ON MY ACCOUNT? PLEASE ANSWER ME, I WONT GET MAD, BUT THIS IS AN IMPORTANT SECURITY RISK. SOMEONES MUST HAVE ACCESS TO MY ACCOUNT, DOES THE SOURCE CODE OF MY MAIN PYTHON SCRIPT APPEARS SOMEPLACE AGAIN?!?! From rosuav at gmail.com Tue Oct 1 06:47:55 2013 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 1 Oct 2013 20:47:55 +1000 Subject: I haev fixed it In-Reply-To: References: <20130930235914.GB13553@secure.superhost.gr> Message-ID: On Tue, Oct 1, 2013 at 8:16 PM, ????? wrote: > WHO AND MOST IMPORTNTANLY HOW DID HE MANAGED TO UPLOAD THIS FILE ON MY > ACCOUNT? > > PLEASE ANSWER ME, I WONT GET MAD, BUT THIS IS AN IMPORTANT SECURITY RISK. > > SOMEONES MUST HAVE ACCESS TO MY ACCOUNT, DOES THE SOURCE CODE OF MY MAIN > PYTHON SCRIPT APPEARS SOMEPLACE AGAIN?!?! Whoever did it, he seems also to have gained access to your emails as well. Congratulations to him, I think he's done well. ChrisA From breamoreboy at yahoo.co.uk Tue Oct 1 09:19:13 2013 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Tue, 01 Oct 2013 14:19:13 +0100 Subject: I haev fixed it In-Reply-To: References: <20130930235914.GB13553@secure.superhost.gr> Message-ID: On 01/10/2013 11:47, Chris Angelico wrote: > On Tue, Oct 1, 2013 at 8:16 PM, ????? wrote: >> WHO AND MOST IMPORTNTANLY HOW DID HE MANAGED TO UPLOAD THIS FILE ON MY >> ACCOUNT? >> >> PLEASE ANSWER ME, I WONT GET MAD, BUT THIS IS AN IMPORTANT SECURITY RISK. >> >> SOMEONES MUST HAVE ACCESS TO MY ACCOUNT, DOES THE SOURCE CODE OF MY MAIN >> PYTHON SCRIPT APPEARS SOMEPLACE AGAIN?!?! > > Whoever did it, he seems also to have gained access to your emails as > well. Congratulations to him, I think he's done well. > > ChrisA > I'm sorry Chris but I must disagree with you. It's like claiming that somebody has done well breaking into a field that has neither gate nor fence. -- Cheers. Mark Lawrence From joel.goldstick at gmail.com Tue Oct 1 09:27:02 2013 From: joel.goldstick at gmail.com (Joel Goldstick) Date: Tue, 1 Oct 2013 09:27:02 -0400 Subject: I haev fixed it In-Reply-To: References: <20130930235914.GB13553@secure.superhost.gr> Message-ID: all your base are belong to us On Tue, Oct 1, 2013 at 9:19 AM, Mark Lawrence wrote: > On 01/10/2013 11:47, Chris Angelico wrote: > >> On Tue, Oct 1, 2013 at 8:16 PM, ????? wrote: >> >>> WHO AND MOST IMPORTNTANLY HOW DID HE MANAGED TO UPLOAD THIS FILE ON MY >>> ACCOUNT? >>> >>> PLEASE ANSWER ME, I WONT GET MAD, BUT THIS IS AN IMPORTANT SECURITY RISK. >>> >>> SOMEONES MUST HAVE ACCESS TO MY ACCOUNT, DOES THE SOURCE CODE OF MY MAIN >>> PYTHON SCRIPT APPEARS SOMEPLACE AGAIN?!?! >>> >> >> Whoever did it, he seems also to have gained access to your emails as >> well. Congratulations to him, I think he's done well. >> >> ChrisA >> >> > I'm sorry Chris but I must disagree with you. It's like claiming that > somebody has done well breaking into a field that has neither gate nor > fence. > > > -- > Cheers. > > Mark Lawrence > > -- > https://mail.python.org/**mailman/listinfo/python-list > -- Joel Goldstick http://joelgoldstick.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From nikos.gr33k at gmail.com Tue Oct 1 09:43:54 2013 From: nikos.gr33k at gmail.com (=?UTF-8?B?zp3Or866zr/Pgg==?=) Date: Tue, 01 Oct 2013 16:43:54 +0300 Subject: I haev fixed it In-Reply-To: References: <20130930235914.GB13553@secure.superhost.gr> Message-ID: ???? 1/10/2013 4:27 ??, ?/? Joel Goldstick ??????: > all your base are belong to us Keep it safe, because its high unlikely you will see it again. From steve+comp.lang.python at pearwood.info Tue Oct 1 21:52:38 2013 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: 02 Oct 2013 01:52:38 GMT Subject: Stop posting HTML [was Re: I haev fixed it] References: <20130930235914.GB13553@secure.superhost.gr> Message-ID: <524b7c66$0$29974$c3e8da3$5496439d@news.astraweb.com> Joel, you've been asked repeatedly to please stop posting HTML. On a non- binary newsgroup like this, it is rude and inconsiderate of others to persist. On Tue, 01 Oct 2013 09:27:02 -0400, Joel Goldstick wrote: >
all your base are belong to us
class="gmail_extra">

On Tue, Oct 1, 2013 > at 9:19 AM, Mark Lawrence < href="mailto:breamoreboy at yahoo.co.uk" > target="_blank">breamoreboy at yahoo.co.uk> wrote:
>
On 01/10/2013 11:47, Chris > Angelico wrote:
On Tue, Oct 1, 2013 > at 8:16 PM, ????? < target="_blank">nikos.gr33k at gmail.com> wrote:
class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc [rest of the HTML gunk deleted] -- Steven From joel.goldstick at gmail.com Tue Oct 1 22:02:36 2013 From: joel.goldstick at gmail.com (Joel Goldstick) Date: Tue, 1 Oct 2013 22:02:36 -0400 Subject: Stop posting HTML [was Re: I haev fixed it] In-Reply-To: <524b7c66$0$29974$c3e8da3$5496439d@news.astraweb.com> References: <20130930235914.GB13553@secure.superhost.gr> <524b7c66$0$29974$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Tue, Oct 1, 2013 at 9:52 PM, Steven D'Aprano < steve+comp.lang.python at pearwood.info> wrote: > Joel, you've been asked repeatedly to please stop posting HTML. On a non- > binary newsgroup like this, it is rude and inconsiderate of others to > persist. > > On Tue, 01 Oct 2013 09:27:02 -0400, Joel Goldstick wrote: > > >
all your base are belong to us
> class="gmail_extra">

On Tue, Oct 1, 2013 > > at 9:19 AM, Mark Lawrence < > href="mailto:breamoreboy at yahoo.co.uk" > > target="_blank">breamoreboy at yahoo.co.uk> wrote:
> >
On 01/10/2013 11:47, Chris > > Angelico wrote:
On Tue, Oct 1, 2013 > > at 8:16 PM, ????? < > target="_blank">nikos.gr33k at gmail.com> wrote:
> class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc > > [rest of the HTML gunk deleted] > > > > -- > Steven > -- > https://mail.python.org/mailman/listinfo/python-list > I'm very sorry. I didn't realize I was. I use gmail for this list, and I believe it replies with html if the previous message was written with html. I'll be more vigilant. Mea culpa to all. This is something that changed apparently with gmail as in the past I have set all outgoing messages to text. I agree its annoying. -- Joel Goldstick http://joelgoldstick.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From skip.montanaro at gmail.com Tue Oct 1 22:28:38 2013 From: skip.montanaro at gmail.com (Skip Montanaro) Date: Tue, 1 Oct 2013 21:28:38 -0500 Subject: Stop posting HTML [was Re: I haev fixed it] In-Reply-To: References: <20130930235914.GB13553@secure.superhost.gr> <524b7c66$0$29974$c3e8da3$5496439d@news.astraweb.com> Message-ID: > I use gmail for this list, and I believe it replies with html if the previous message was written with html. I can verify that. Definitely need to pay attention when using Gmail. It loves to top post and quote entire messages by default as well. Skip (who has a love/hate relationship with Gmail) -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve+comp.lang.python at pearwood.info Tue Oct 1 22:40:15 2013 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: 02 Oct 2013 02:40:15 GMT Subject: Stop posting HTML [was Re: I haev fixed it] References: <20130930235914.GB13553@secure.superhost.gr> <524b7c66$0$29974$c3e8da3$5496439d@news.astraweb.com> Message-ID: <524b878f$0$29984$c3e8da3$5496439d@news.astraweb.com> On Tue, 01 Oct 2013 22:02:36 -0400, Joel Goldstick wrote: > On Tue, Oct 1, 2013 at 9:52 PM, Steven D'Aprano < > steve+comp.lang.python at pearwood.info> wrote: > >> Joel, you've been asked repeatedly to please stop posting HTML. [...] >

class="gmail_extra">I'm very sorry.? I didn't realize I was.? I > use gmail for this list, and I believe it replies with html if the > previous message was written with html.? I'll be more vigilant. Not vigilant enough, it seems. -- Steven From daniel.stjnv+python-list at gmail.com Tue Oct 1 22:57:14 2013 From: daniel.stjnv+python-list at gmail.com (Daniel Stojanov) Date: Wed, 2 Oct 2013 12:57:14 +1000 Subject: Stop posting HTML [was Re: I haev fixed it] In-Reply-To: <524b878f$0$29984$c3e8da3$5496439d@news.astraweb.com> References: <20130930235914.GB13553@secure.superhost.gr> <524b7c66$0$29974$c3e8da3$5496439d@news.astraweb.com> <524b878f$0$29984$c3e8da3$5496439d@news.astraweb.com> Message-ID: On 2 October 2013 12:40, Steven D'Aprano wrote: >>> Joel, you've been asked repeatedly to please stop posting HTML. > [...] > >>

> class="gmail_extra">I'm very sorry. I didn't realize I was. I >> use gmail for this list, and I believe it replies with html if the >> previous message was written with html. I'll be more vigilant. > > Not vigilant enough, it seems. > > > > -- > Steven > -- > https://mail.python.org/mailman/listinfo/python-list I have noticed that you need to set plain text mode each time you compose from Gmail (it sometimes reverts back) and there is no option for plain text for the mobile app. It's also awkward about replying below the previous text. From torriem at gmail.com Tue Oct 1 23:33:42 2013 From: torriem at gmail.com (Michael Torrie) Date: Tue, 01 Oct 2013 21:33:42 -0600 Subject: Stop posting HTML [was Re: I haev fixed it] In-Reply-To: <524b878f$0$29984$c3e8da3$5496439d@news.astraweb.com> References: <20130930235914.GB13553@secure.superhost.gr> <524b7c66$0$29974$c3e8da3$5496439d@news.astraweb.com> <524b878f$0$29984$c3e8da3$5496439d@news.astraweb.com> Message-ID: <524B9416.1080900@gmail.com> On 10/01/2013 08:40 PM, Steven D'Aprano wrote: > On Tue, 01 Oct 2013 22:02:36 -0400, Joel Goldstick wrote: > >> On Tue, Oct 1, 2013 at 9:52 PM, Steven D'Aprano < >> steve+comp.lang.python at pearwood.info> wrote: >> >>> Joel, you've been asked repeatedly to please stop posting HTML. > [...] > >>

> class="gmail_extra">I'm very sorry. I didn't realize I was. I >> use gmail for this list, and I believe it replies with html if the >> previous message was written with html. I'll be more vigilant. > > Not vigilant enough, it seems. Near as I can tell Gmail is posting the message in both html and plain text parts in a multi-part message. Thunderbird has no problem with the plain text part. That's all I see right now, unless I force html view, or look at the message source. From davea at davea.name Wed Oct 2 08:05:59 2013 From: davea at davea.name (Dave Angel) Date: Wed, 2 Oct 2013 12:05:59 +0000 (UTC) Subject: Stop posting HTML [was Re: I haev fixed it] References: <20130930235914.GB13553@secure.superhost.gr> <524b7c66$0$29974$c3e8da3$5496439d@news.astraweb.com> <524b878f$0$29984$c3e8da3$5496439d@news.astraweb.com> <524B9416.1080900@gmail.com> Message-ID: On 1/10/2013 23:33, Michael Torrie wrote: > On 10/01/2013 08:40 PM, Steven D'Aprano wrote: >> On Tue, 01 Oct 2013 22:02:36 -0400, Joel Goldstick wrote: >> >>> On Tue, Oct 1, 2013 at 9:52 PM, Steven D'Aprano < >>> steve+comp.lang.python at pearwood.info> wrote: >>> >>>> Joel, you've been asked repeatedly to please stop posting HTML. >> [...] >> >>>

>> class="gmail_extra">I'm very sorry. I didn't realize I was. I >>> use gmail for this list, and I believe it replies with html if the >>> previous message was written with html. I'll be more vigilant. >> >> Not vigilant enough, it seems. > > Near as I can tell Gmail is posting the message in both html and plain > text parts in a multi-part message. > > Thunderbird has no problem with the plain text part. That's all I see > right now, unless I force html view, or look at the message source. You say that as though you think that makes html posting okay. The html to text conversion done at posting time frequently loses something in the message, so that people viewing the text in a newsreader don't see the same thing the author typed. Simplest example is color or bold. But the author can avoid using these. More importantly, indentation and paragraph boundaries are sometimes completely lost, and worse, sometimes just lost in some places. Sometimes the html and text are both stuffed into the same part of the mail message. This is annoying and misleading. I don't know if gmail is guilty of this. Sometimes a message has html only, and the receiving program sometimes attempts to convert it to plain text. That's even more likely to have problems with formatting. Whenever html is included, it's a redundant waste of space, and many of us pay for our internet usage by the kilobyte. Worse, I assume that all the archives are also bigger by a big percentage. -- DaveA From nikos.gr33k at gmail.com Tue Oct 1 09:22:37 2013 From: nikos.gr33k at gmail.com (=?UTF-8?B?zp3Or866zr/Pgg==?=) Date: Tue, 01 Oct 2013 16:22:37 +0300 Subject: I haev fixed it In-Reply-To: References: <20130930235914.GB13553@secure.superhost.gr> Message-ID: ???? 1/10/2013 4:19 ??, ?/? Mark Lawrence ??????: > On 01/10/2013 11:47, Chris Angelico wrote: >> On Tue, Oct 1, 2013 at 8:16 PM, ????? wrote: >>> WHO AND MOST IMPORTNTANLY HOW DID HE MANAGED TO UPLOAD THIS FILE ON MY >>> ACCOUNT? >>> >>> PLEASE ANSWER ME, I WONT GET MAD, BUT THIS IS AN IMPORTANT SECURITY >>> RISK. >>> >>> SOMEONES MUST HAVE ACCESS TO MY ACCOUNT, DOES THE SOURCE CODE OF MY MAIN >>> PYTHON SCRIPT APPEARS SOMEPLACE AGAIN?!?! >> >> Whoever did it, he seems also to have gained access to your emails as >> well. Congratulations to him, I think he's done well. >> >> ChrisA >> > > I'm sorry Chris but I must disagree with you. It's like claiming that > somebody has done well breaking into a field that has neither gate nor > fence. Now it hance a fence. Jump again to break in. From nikos.gr33k at gmail.com Tue Oct 1 06:54:32 2013 From: nikos.gr33k at gmail.com (=?UTF-8?B?zp3Or866zr/Pgg==?=) Date: Tue, 01 Oct 2013 13:54:32 +0300 Subject: I haev fixed it In-Reply-To: References: <20130930235914.GB13553@secure.superhost.gr> Message-ID: ???? 1/10/2013 1:47 ??, ?/? Chris Angelico ??????: > On Tue, Oct 1, 2013 at 8:16 PM, ????? wrote: >> WHO AND MOST IMPORTNTANLY HOW DID HE MANAGED TO UPLOAD THIS FILE ON MY >> ACCOUNT? >> >> PLEASE ANSWER ME, I WONT GET MAD, BUT THIS IS AN IMPORTANT SECURITY RISK. >> >> SOMEONES MUST HAVE ACCESS TO MY ACCOUNT, DOES THE SOURCE CODE OF MY MAIN >> PYTHON SCRIPT APPEARS SOMEPLACE AGAIN?!?! > > Whoever did it, he seems also to have gained access to your emails as > well. Congratulations to him, I think he's done well. How did this happened i asked. I must know how did this happen so i take action to prevent it from happening again. I you want to congratulate Mark Lawrence do it in private. From rosuav at gmail.com Tue Oct 1 06:58:24 2013 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 1 Oct 2013 20:58:24 +1000 Subject: I haev fixed it In-Reply-To: References: <20130930235914.GB13553@secure.superhost.gr> Message-ID: On Tue, Oct 1, 2013 at 8:54 PM, ????? wrote: > How did this happened i asked. > I must know how did this happen so i take action to prevent it from > happening again. > > I you want to congratulate Mark Lawrence do it in private. You know it was Mark, then? Okay. In that case, ask him directly. If not, I advise you to refrain from making bald statements that you can't back. You really need to sort out your own security. Don't go begging someone else for help - not everyone is courteous enough to just put an HTML file down and send an email from your account. Some people will actually destroy things. ChrisA From ned at nedbatchelder.com Tue Oct 1 06:59:49 2013 From: ned at nedbatchelder.com (Ned Batchelder) Date: Tue, 01 Oct 2013 06:59:49 -0400 Subject: I haev fixed it In-Reply-To: References: <20130930235914.GB13553@secure.superhost.gr> Message-ID: <524AAB25.6020206@nedbatchelder.com> On 10/1/13 6:54 AM, ????? wrote: > ???? 1/10/2013 1:47 ??, ?/? Chris Angelico ??????: >> On Tue, Oct 1, 2013 at 8:16 PM, ????? wrote: >>> WHO AND MOST IMPORTNTANLY HOW DID HE MANAGED TO UPLOAD THIS FILE ON MY >>> ACCOUNT? >>> >>> PLEASE ANSWER ME, I WONT GET MAD, BUT THIS IS AN IMPORTANT SECURITY >>> RISK. >>> >>> SOMEONES MUST HAVE ACCESS TO MY ACCOUNT, DOES THE SOURCE CODE OF MY >>> MAIN >>> PYTHON SCRIPT APPEARS SOMEPLACE AGAIN?!?! >> >> Whoever did it, he seems also to have gained access to your emails as >> well. Congratulations to him, I think he's done well. > > How did this happened i asked. > I must know how did this happen so i take action to prevent it from > happening again. > > I you want to congratulate Mark Lawrence do it in private. Nikos, I am sorry you have been hacked. But how it happened is not a Python question. It's not an appropriate topic of discussion on this mailing list. Please don't make a bad situation worse by continuing to talk about it here. --Ned. From nikos.gr33k at gmail.com Tue Oct 1 07:06:18 2013 From: nikos.gr33k at gmail.com (=?UTF-8?B?zp3Or866zr/Pgg==?=) Date: Tue, 01 Oct 2013 14:06:18 +0300 Subject: I haev fixed it In-Reply-To: References: <20130930235914.GB13553@secure.superhost.gr> Message-ID: ???? 1/10/2013 1:58 ??, ?/? Chris Angelico ??????: > On Tue, Oct 1, 2013 at 8:54 PM, ????? wrote: >> How did this happened i asked. >> I must know how did this happen so i take action to prevent it from >> happening again. >> >> I you want to congratulate Mark Lawrence do it in private. > > You know it was Mark, then? Okay. In that case, ask him directly. If > not, I advise you to refrain from making bald statements that you > can't back. He started the thread didn't he? He also posted couple days agon if i have fixes a link form a domain i host which provided my source code in plain text. Let alone his hatred agaisnt me. Considerign the above I think its safe to say it was him. > You really need to sort out your own security. Don't go begging > someone else for help - not everyone is courteous enough to just put > an HTML file down and send an email from your account. Some people > will actually destroy things. I'am not begging anyone. I have aksed a question as to HOW this might have happened. But it seems you don't want to provide an explanation although i think you might have a theory. From modelnine at modelnine.org Tue Oct 1 07:29:14 2013 From: modelnine at modelnine.org (Heiko Wundram) Date: Tue, 01 Oct 2013 13:29:14 +0200 Subject: I haev fixed it In-Reply-To: References: <20130930235914.GB13553@secure.superhost.gr> Message-ID: <524AB20A.8080803@modelnine.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Am 01.10.2013 13:06, schrieb ?????: > But it seems you don't want to provide an explanation although i > think you might have a theory. You need a theory? 1) Your password(s) is/are leaked (see the URL referenced somewhere before, and IIRC you also posted your GMail password sometime ago), and 2) you did password-reuse, so that by an attacker getting access to one password, more than one of your accounts was compromised. - -- - --- Heiko. -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.20 (MingW32) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQEcBAEBAgAGBQJSSrIKAAoJEDMqpHf921/SO+UH/iujBSt7ZXmXAIAHgHXoKH0Q Qxvzi2L1pCXcXvEE4yeUI0g0TiYD9B88Q5eRyCegWWm2BwpOjx7KLNBkMqQeiI6H M52L/ulXwMkwVq0HTn6YPNncReQrPMu2V5xQaKWhfVhBnWLZnZYm1n7WZse9M2Sr 9KaAkZ4j2jlHozJ9tAGXnIt/9bj6MM3SQPuG1b68qSWThisUhvTcbrDkm3e4KDoq I9i9kEF93XPLYeOMefEOksm51vKjpDWFlRu20Vqy5quYxDHpUU/5e04Z6doz0py8 6XhR892g4zetQ3OwtzxQOKunwaLOvSg9VtXfIn7ElBkCE0v/XbCxTnO/oBLcb7g= =I1kO -----END PGP SIGNATURE----- From nikos.gr33k at gmail.com Tue Oct 1 08:06:13 2013 From: nikos.gr33k at gmail.com (=?UTF-8?B?zp3Or866zr/Pgg==?=) Date: Tue, 01 Oct 2013 15:06:13 +0300 Subject: I haev fixed it In-Reply-To: References: <20130930235914.GB13553@secure.superhost.gr> Message-ID: ???? 1/10/2013 2:29 ??, ?/? Heiko Wundram ??????: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Am 01.10.2013 13:06, schrieb ?????: >> But it seems you don't want to provide an explanation although i >> think you might have a theory. > > You need a theory? > > 1) Your password(s) is/are leaked (see the URL referenced somewhere > before, and IIRC you also posted your GMail password sometime ago), and Hello, i know about the link you mentioned and i have deleted the source code from there. From modelnine at modelnine.org Tue Oct 1 08:12:01 2013 From: modelnine at modelnine.org (Heiko Wundram) Date: Tue, 01 Oct 2013 14:12:01 +0200 Subject: I haev fixed it In-Reply-To: References: <20130930235914.GB13553@secure.superhost.gr> Message-ID: <524ABC11.7070603@modelnine.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Am 01.10.2013 14:06, schrieb ?????: > i know about the link you mentioned and i have deleted the source > code from there. Guess what: Google keeps a cache. See here: http://webcache.googleusercontent.com/search?q=cache:http://superhost.gr/~dauwin/cgi-bin/metrites.py So if you haven't changed your password(s), you'd better do that now. - -- - --- Heiko. -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.20 (MingW32) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQEcBAEBAgAGBQJSSrwRAAoJEDMqpHf921/SY/MH/3pf9ZdHCXuu84urCodUyBrQ RVRbVN1lXAzCXY1nyPGfzANOsraXLzRDe0j9ZBfHbEaZR19Hvl4DOf8+RJfRl8jg LWCsgIIVb2fWWVLrx1CU3oz47sVfy1vGOp8XRiIqjcDKa+zOtyqqlxIolKCFM6CL /YsHnb1/9JE1zn07WaKYJTi1/9+uptaQPR9kNzOssv1TpvRiJ+4H1oO67Px6tpdj VchpEirkV7CaD39mD9BLEoB24FhEX+NSNYPRJx89ivC+MENpNUp6n5vVjQ+ciXI/ NvJJxBalypi/DLNaCR/up2B2018ebH+3ByDv3xO+UnbS6MYx5YVppstilvkvr1c= =2VTx -----END PGP SIGNATURE----- From nikos.gr33k at gmail.com Tue Oct 1 08:21:07 2013 From: nikos.gr33k at gmail.com (=?UTF-8?B?zp3Or866zr/Pgg==?=) Date: Tue, 01 Oct 2013 15:21:07 +0300 Subject: I haev fixed it In-Reply-To: References: <20130930235914.GB13553@secure.superhost.gr> Message-ID: ???? 1/10/2013 3:12 ??, ?/? Heiko Wundram ??????: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Am 01.10.2013 14:06, schrieb ?????: >> i know about the link you mentioned and i have deleted the source >> code from there. > > Guess what: Google keeps a cache. See here: > > http://webcache.googleusercontent.com/search?q=cache:http://superhost.gr/~dauwin/cgi-bin/metrites.py > > So if you haven't changed your password(s), you'd better do that now. Tahnk you Heiko i didnt knew Google was keeping cached version of files. But i have deleted the file metrites.py 2 days ago when i saw Mark Lawrence mentioned it and i also have changed the passwords from my personal account and root as well. From nikos.gr33k at gmail.com Tue Oct 1 08:25:13 2013 From: nikos.gr33k at gmail.com (=?UTF-8?B?zp3Or866zr/Pgg==?=) Date: Tue, 01 Oct 2013 15:25:13 +0300 Subject: I haev fixed it In-Reply-To: References: <20130930235914.GB13553@secure.superhost.gr> Message-ID: ???? 1/10/2013 3:21 ??, ?/? ????? ??????: > ???? 1/10/2013 3:12 ??, ?/? Heiko Wundram ??????: >> -----BEGIN PGP SIGNED MESSAGE----- >> Hash: SHA1 >> >> Am 01.10.2013 14:06, schrieb ?????: >>> i know about the link you mentioned and i have deleted the source >>> code from there. >> >> Guess what: Google keeps a cache. See here: >> >> http://webcache.googleusercontent.com/search?q=cache:http://superhost.gr/~dauwin/cgi-bin/metrites.py >> >> >> So if you haven't changed your password(s), you'd better do that now. > > Tahnk you Heiko i didnt knew Google was keeping cached version of files. > > But i have deleted the file metrites.py 2 days ago when i saw Mark > Lawrence mentioned it and i also have changed the passwords from my > personal account and root as well. I know how he did it, he sees it form here: http://superhost.gr/~nikos/cgi-bin/metrites.py I must somehow use an apache directive not to allow such view. From ishish at domhain.de Tue Oct 1 08:50:30 2013 From: ishish at domhain.de (ishish) Date: Tue, 01 Oct 2013 13:50:30 +0100 Subject: I haev fixed it In-Reply-To: References: <20130930235914.GB13553@secure.superhost.gr> Message-ID: <13b84cb099cc331c4a596c7fdf9d0e46@home.minuskel.de> Am 01.10.2013 13:25, schrieb ?????: > ???? 1/10/2013 3:21 ??, ?/? ????? ??????: >> ???? 1/10/2013 3:12 ??, ?/? Heiko Wundram ??????: >>> -----BEGIN PGP SIGNED MESSAGE----- >>> Hash: SHA1 >>> >>> Am 01.10.2013 14:06, schrieb ?????: >>>> i know about the link you mentioned and i have deleted the source >>>> code from there. >>> >>> Guess what: Google keeps a cache. See here: >>> >>> >>> http://webcache.googleusercontent.com/search?q=cache:http://superhost.gr/~dauwin/cgi-bin/metrites.py >>> >>> >>> So if you haven't changed your password(s), you'd better do that >>> now. >> >> Tahnk you Heiko i didnt knew Google was keeping cached version of >> files. >> >> But i have deleted the file metrites.py 2 days ago when i saw Mark >> Lawrence mentioned it and i also have changed the passwords from my >> personal account and root as well. > > I know how he did it, he sees it form here: > > http://superhost.gr/~nikos/cgi-bin/metrites.py > > I must somehow use an apache directive not to allow such view. I agree. Just keep folders for scripts and templates above the actual public_html. Sas From nikos.gr33k at gmail.com Tue Oct 1 09:14:21 2013 From: nikos.gr33k at gmail.com (=?UTF-8?B?zp3Or866zr/Pgg==?=) Date: Tue, 01 Oct 2013 16:14:21 +0300 Subject: I haev fixed it In-Reply-To: References: <20130930235914.GB13553@secure.superhost.gr> Message-ID: ???? 1/10/2013 3:50 ??, ?/? ishish ??????: > Am 01.10.2013 13:25, schrieb ?????: >> ???? 1/10/2013 3:21 ??, ?/? ????? ??????: >>> ???? 1/10/2013 3:12 ??, ?/? Heiko Wundram ??????: >>>> -----BEGIN PGP SIGNED MESSAGE----- >>>> Hash: SHA1 >>>> >>>> Am 01.10.2013 14:06, schrieb ?????: >>>>> i know about the link you mentioned and i have deleted the source >>>>> code from there. >>>> >>>> Guess what: Google keeps a cache. See here: >>>> >>>> >>>> http://webcache.googleusercontent.com/search?q=cache:http://superhost.gr/~dauwin/cgi-bin/metrites.py >>>> >>>> >>>> >>>> So if you haven't changed your password(s), you'd better do that now. >>> >>> Tahnk you Heiko i didnt knew Google was keeping cached version of files. >>> >>> But i have deleted the file metrites.py 2 days ago when i saw Mark >>> Lawrence mentioned it and i also have changed the passwords from my >>> personal account and root as well. >> >> I know how he did it, he sees it form here: >> >> http://superhost.gr/~nikos/cgi-bin/metrites.py >> >> I must somehow use an apache directive not to allow such view. > > I agree. Just keep folders for scripts and templates above the actual > public_html. python scripts need to be placed inside the 'cgi-bin/' folder which is located at '~/public_html/'. Othwerise they wont work. From ishish at domhain.de Tue Oct 1 09:26:11 2013 From: ishish at domhain.de (ishish) Date: Tue, 01 Oct 2013 14:26:11 +0100 Subject: I haev fixed it In-Reply-To: References: <20130930235914.GB13553@secure.superhost.gr> Message-ID: <97ae80b12fa6a85ca08fb43bd7e7fc99@home.minuskel.de> Am 01.10.2013 14:14, schrieb ?????: > ???? 1/10/2013 3:50 ??, ?/? ishish ??????: >> Am 01.10.2013 13:25, schrieb ?????: >>> ???? 1/10/2013 3:21 ??, ?/? ????? ??????: >>>> ???? 1/10/2013 3:12 ??, ?/? Heiko Wundram ??????: >>>>> -----BEGIN PGP SIGNED MESSAGE----- >>>>> Hash: SHA1 >>>>> >>>>> Am 01.10.2013 14:06, schrieb ?????: >>>>>> i know about the link you mentioned and i have deleted the >>>>>> source >>>>>> code from there. >>>>> >>>>> Guess what: Google keeps a cache. See here: >>>>> >>>>> >>>>> >>>>> http://webcache.googleusercontent.com/search?q=cache:http://superhost.gr/~dauwin/cgi-bin/metrites.py >>>>> >>>>> >>>>> >>>>> So if you haven't changed your password(s), you'd better do that >>>>> now. >>>> >>>> Tahnk you Heiko i didnt knew Google was keeping cached version of >>>> files. >>>> >>>> But i have deleted the file metrites.py 2 days ago when i saw Mark >>>> Lawrence mentioned it and i also have changed the passwords from >>>> my >>>> personal account and root as well. >>> >>> I know how he did it, he sees it form here: >>> >>> http://superhost.gr/~nikos/cgi-bin/metrites.py >>> >>> I must somehow use an apache directive not to allow such view. >> >> I agree. Just keep folders for scripts and templates above the >> actual >> public_html. > > python scripts need to be placed inside the 'cgi-bin/' folder which > is located at '~/public_html/'. > > Othwerise they wont work. That's due to your (or whoever set these up) configurations. I have never used the public_html to store perl, python or php scripts and it works perfectly fine for me. Sas From joel.goldstick at gmail.com Tue Oct 1 09:35:19 2013 From: joel.goldstick at gmail.com (Joel Goldstick) Date: Tue, 1 Oct 2013 09:35:19 -0400 Subject: I haev fixed it In-Reply-To: <97ae80b12fa6a85ca08fb43bd7e7fc99@home.minuskel.de> References: <20130930235914.GB13553@secure.superhost.gr> <97ae80b12fa6a85ca08fb43bd7e7fc99@home.minuskel.de> Message-ID: On Tue, Oct 1, 2013 at 9:26 AM, ishish wrote: > Am 01.10.2013 14:14, schrieb ?????: > > ???? 1/10/2013 3:50 ??, ?/? ishish ??????: >> >>> Am 01.10.2013 13:25, schrieb ?????: >>> >>>> ???? 1/10/2013 3:21 ??, ?/? ????? ??????: >>>> >>>>> ???? 1/10/2013 3:12 ??, ?/? Heiko Wundram ??????: >>>>> >>>>>> -----BEGIN PGP SIGNED MESSAGE----- >>>>>> Hash: SHA1 >>>>>> >>>>>> Am 01.10.2013 14:06, schrieb ?????: >>>>>> >>>>>>> i know about the link you mentioned and i have deleted the source >>>>>>> code from there. >>>>>>> >>>>>> >>>>>> Guess what: Google keeps a cache. See here: >>>>>> >>>>>> >>>>>> >>>>>> http://webcache.**googleusercontent.com/search?** >>>>>> q=cache:http://superhost.gr/~**dauwin/cgi-bin/metrites.py >>>>>> >>>>>> >>>>>> >>>>>> So if you haven't changed your password(s), you'd better do that now. >>>>>> >>>>> >>>>> Tahnk you Heiko i didnt knew Google was keeping cached version of >>>>> files. >>>>> >>>>> But i have deleted the file metrites.py 2 days ago when i saw Mark >>>>> Lawrence mentioned it and i also have changed the passwords from my >>>>> personal account and root as well. >>>>> >>>> >>>> I know how he did it, he sees it form here: >>>> >>>> http://superhost.gr/~nikos/**cgi-bin/metrites.py >>>> >>>> I must somehow use an apache directive not to allow such view. >>>> >>> >>> I agree. Just keep folders for scripts and templates above the actual >>> public_html. >>> >> >> python scripts need to be placed inside the 'cgi-bin/' folder which >> is located at '~/public_html/'. >> >> Othwerise they wont work. >> > > That's due to your (or whoever set these up) configurations. I have never > used the public_html to store perl, python or php scripts and it works > perfectly fine for me. > > Sas > -- > https://mail.python.org/**mailman/listinfo/python-list > Thanks for that note. I haven't done this sort of thing for a while, but as I recall you NEVER put code in public_html. So that was odd (well not really since its from Nikos!) to see a rule that code must be put there. On a personal note, i'm bummed out I woke up too late to see the warning page. -- Joel Goldstick http://joelgoldstick.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From nikos.gr33k at gmail.com Tue Oct 1 09:39:56 2013 From: nikos.gr33k at gmail.com (=?UTF-8?B?zp3Or866zr/Pgg==?=) Date: Tue, 01 Oct 2013 16:39:56 +0300 Subject: I haev fixed it In-Reply-To: References: <20130930235914.GB13553@secure.superhost.gr> <97ae80b12fa6a85ca08fb43bd7e7fc99@home.minuskel.de> Message-ID: ???? 1/10/2013 4:35 ??, ?/? Joel Goldstick ??????: > Thanks for that note. I haven't done this sort of thing for a while, > but as I recall you NEVER put code in public_html. So that was odd > (well not really since its from Nikos!) to see a rule that code must be > put there. public_html as well as cgi-bin are there for a reason you know. So people put files within those folders. Tehy can be dafe there as well, no nee to place them above '../public_html'. I still have them there and they are safe now. From nikos.gr33k at gmail.com Tue Oct 1 09:37:01 2013 From: nikos.gr33k at gmail.com (=?UTF-8?B?zp3Or866zr/Pgg==?=) Date: Tue, 01 Oct 2013 16:37:01 +0300 Subject: I haev fixed it In-Reply-To: References: <20130930235914.GB13553@secure.superhost.gr> Message-ID: ???? 1/10/2013 4:26 ??, ?/? ishish ??????: > Am 01.10.2013 14:14, schrieb ?????: >> ???? 1/10/2013 3:50 ??, ?/? ishish ??????: >>> Am 01.10.2013 13:25, schrieb ?????: >>>> ???? 1/10/2013 3:21 ??, ?/? ????? ??????: >>>>> ???? 1/10/2013 3:12 ??, ?/? Heiko Wundram ??????: >>>>>> -----BEGIN PGP SIGNED MESSAGE----- >>>>>> Hash: SHA1 >>>>>> >>>>>> Am 01.10.2013 14:06, schrieb ?????: >>>>>>> i know about the link you mentioned and i have deleted the source >>>>>>> code from there. >>>>>> >>>>>> Guess what: Google keeps a cache. See here: >>>>>> >>>>>> >>>>>> >>>>>> http://webcache.googleusercontent.com/search?q=cache:http://superhost.gr/~dauwin/cgi-bin/metrites.py >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> So if you haven't changed your password(s), you'd better do that now. >>>>> >>>>> Tahnk you Heiko i didnt knew Google was keeping cached version of >>>>> files. >>>>> >>>>> But i have deleted the file metrites.py 2 days ago when i saw Mark >>>>> Lawrence mentioned it and i also have changed the passwords from my >>>>> personal account and root as well. >>>> >>>> I know how he did it, he sees it form here: >>>> >>>> http://superhost.gr/~nikos/cgi-bin/metrites.py >>>> >>>> I must somehow use an apache directive not to allow such view. >>> >>> I agree. Just keep folders for scripts and templates above the actual >>> public_html. >> >> python scripts need to be placed inside the 'cgi-bin/' folder which >> is located at '~/public_html/'. >> >> Othwerise they wont work. > > That's due to your (or whoever set these up) configurations. I have > never used the public_html to store perl, python or php scripts and it > works perfectly fine for me. To which folders do you store your cgi scripts for safety? From wuwei23 at gmail.com Tue Oct 1 19:25:54 2013 From: wuwei23 at gmail.com (alex23) Date: Wed, 02 Oct 2013 09:25:54 +1000 Subject: I haev fixed it In-Reply-To: References: <20130930235914.GB13553@secure.superhost.gr> Message-ID: On 1/10/2013 9:06 PM, ????? wrote: > ???? 1/10/2013 1:58 ??, ?/? Chris Angelico ??????: >> On Tue, Oct 1, 2013 at 8:54 PM, ????? wrote: >>> I you want to congratulate Mark Lawrence do it in private. >> >> You know it was Mark, then? Okay. In that case, ask him directly. If >> not, I advise you to refrain from making bald statements that you >> can't back. > > He started the thread didn't he? > > He also posted couple days agon if i have fixes a link form a domain i > host which provided my source code in plain text. > > Let alone his hatred agaisnt me. > > Considerign the above I think its safe to say it was him. No, it's defamation. Unless you have _proof_ it was Mark Lawrence, you really should just shut up for once. As it stands, without an apology (and even _with_ one), Mark has the legal right to sue you. http://en.wikipedia.org/wiki/Defamation#Internationally http://www.aaronkellylaw.com/internet-defamation-laws/serving-an-international-defamation-subpoena/ From nikos.gr33k at gmail.com Tue Oct 1 19:30:25 2013 From: nikos.gr33k at gmail.com (=?UTF-8?B?zp3Or866zr/Pgg==?=) Date: Wed, 02 Oct 2013 02:30:25 +0300 Subject: I haev fixed it In-Reply-To: References: <20130930235914.GB13553@secure.superhost.gr> Message-ID: ???? 2/10/2013 2:25 ??, ?/? alex23 ??????: > On 1/10/2013 9:06 PM, ????? wrote: >> ???? 1/10/2013 1:58 ??, ?/? Chris Angelico ??????: >>> On Tue, Oct 1, 2013 at 8:54 PM, ????? wrote: >>>> I you want to congratulate Mark Lawrence do it in private. >>> >>> You know it was Mark, then? Okay. In that case, ask him directly. If >>> not, I advise you to refrain from making bald statements that you >>> can't back. >> >> He started the thread didn't he? >> >> He also posted couple days agon if i have fixes a link form a domain i >> host which provided my source code in plain text. >> >> Let alone his hatred agaisnt me. >> >> Considerign the above I think its safe to say it was him. > > No, it's defamation. Unless you have _proof_ it was Mark Lawrence, you > really should just shut up for once. As it stands, without an apology > (and even _with_ one), Mark has the legal right to sue you. > > http://en.wikipedia.org/wiki/Defamation#Internationally > http://www.aaronkellylaw.com/internet-defamation-laws/serving-an-international-defamation-subpoena/ Wooow! Now i'am reaaaaly scared! Please don't put me behind bars....lol! From wuwei23 at gmail.com Tue Oct 1 19:38:30 2013 From: wuwei23 at gmail.com (alex23) Date: Wed, 02 Oct 2013 09:38:30 +1000 Subject: I haev fixed it In-Reply-To: References: <20130930235914.GB13553@secure.superhost.gr> Message-ID: On 2/10/2013 9:30 AM, ????? wrote: > ???? 2/10/2013 2:25 ??, ?/? alex23 ??????: >> No, it's defamation. Unless you have _proof_ it was Mark Lawrence, you >> really should just shut up for once. As it stands, without an apology >> (and even _with_ one), Mark has the legal right to sue you. > > Wooow! Now i'am reaaaaly scared! Please don't put me behind bars....lol! Yes, it's so funny. You do understand there's a good chance your customers would be able to sue you as well? From breamoreboy at yahoo.co.uk Tue Oct 1 09:23:30 2013 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Tue, 01 Oct 2013 14:23:30 +0100 Subject: I haev fixed it In-Reply-To: References: <20130930235914.GB13553@secure.superhost.gr> Message-ID: On 01/10/2013 11:54, ????? wrote: > ???? 1/10/2013 1:47 ??, ?/? Chris Angelico ??????: >> On Tue, Oct 1, 2013 at 8:16 PM, ????? wrote: >>> WHO AND MOST IMPORTNTANLY HOW DID HE MANAGED TO UPLOAD THIS FILE ON MY >>> ACCOUNT? >>> >>> PLEASE ANSWER ME, I WONT GET MAD, BUT THIS IS AN IMPORTANT SECURITY >>> RISK. >>> >>> SOMEONES MUST HAVE ACCESS TO MY ACCOUNT, DOES THE SOURCE CODE OF MY MAIN >>> PYTHON SCRIPT APPEARS SOMEPLACE AGAIN?!?! >> >> Whoever did it, he seems also to have gained access to your emails as >> well. Congratulations to him, I think he's done well. > > How did this happened i asked. > I must know how did this happen so i take action to prevent it from > happening again. > > I you want to congratulate Mark Lawrence do it in private. Brilliantly funny on the gounds that I haven't got the faintest idea about how web systems are put together, let alone hacking them. But am I replying to the real, really incompetant Greek moron or the person who has pinched his email account? I'll happily admit to being far too lazy to try and find out. -- Cheers. Mark Lawrence From nikos.gr33k at gmail.com Tue Oct 1 09:34:26 2013 From: nikos.gr33k at gmail.com (=?UTF-8?B?zp3Or866zr/Pgg==?=) Date: Tue, 01 Oct 2013 16:34:26 +0300 Subject: I haev fixed it In-Reply-To: References: <20130930235914.GB13553@secure.superhost.gr> Message-ID: ???? 1/10/2013 4:23 ??, ?/? Mark Lawrence ??????: > On 01/10/2013 11:54, ????? wrote: >> ???? 1/10/2013 1:47 ??, ?/? Chris Angelico ??????: >>> On Tue, Oct 1, 2013 at 8:16 PM, ????? wrote: >>>> WHO AND MOST IMPORTNTANLY HOW DID HE MANAGED TO UPLOAD THIS FILE ON MY >>>> ACCOUNT? >>>> >>>> PLEASE ANSWER ME, I WONT GET MAD, BUT THIS IS AN IMPORTANT SECURITY >>>> RISK. >>>> >>>> SOMEONES MUST HAVE ACCESS TO MY ACCOUNT, DOES THE SOURCE CODE OF MY >>>> MAIN >>>> PYTHON SCRIPT APPEARS SOMEPLACE AGAIN?!?! >>> >>> Whoever did it, he seems also to have gained access to your emails as >>> well. Congratulations to him, I think he's done well. >> >> How did this happened i asked. >> I must know how did this happen so i take action to prevent it from >> happening again. >> >> I you want to congratulate Mark Lawrence do it in private. > > Brilliantly funny on the gounds that I haven't got the faintest idea > about how web systems are put together, let alone hacking them. > > But am I replying to the real, really incompetant Greek moron or the > person who has pinched his email account? I'll happily admit to being > far too lazy to try and find out. You started the thread didn't you? And the file's contents that was uploaded speaks out loud your kind of insults. How tough ws to break in since you had the account password by seeing it from the source code? Don't try to blame someone else for this and pretend you didn't ahd anythign to dop with it. Who pinched my mail account? i never posted by mistake any of my mail accounts' passwords, except once from gmail's which i chnaged it immeditely.What are you talking about? As far accusing me for being a moron i have i have fixed it immediately. Since you are so clever jump the fense again grasshoper. From breamoreboy at yahoo.co.uk Tue Oct 1 09:44:23 2013 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Tue, 01 Oct 2013 14:44:23 +0100 Subject: I haev fixed it In-Reply-To: References: <20130930235914.GB13553@secure.superhost.gr> Message-ID: On 01/10/2013 14:34, ????? wrote: > ???? 1/10/2013 4:23 ??, ?/? Mark Lawrence ??????: >> On 01/10/2013 11:54, ????? wrote: >>> ???? 1/10/2013 1:47 ??, ?/? Chris Angelico ??????: >>>> On Tue, Oct 1, 2013 at 8:16 PM, ????? wrote: >>>>> WHO AND MOST IMPORTNTANLY HOW DID HE MANAGED TO UPLOAD THIS FILE ON MY >>>>> ACCOUNT? >>>>> >>>>> PLEASE ANSWER ME, I WONT GET MAD, BUT THIS IS AN IMPORTANT SECURITY >>>>> RISK. >>>>> >>>>> SOMEONES MUST HAVE ACCESS TO MY ACCOUNT, DOES THE SOURCE CODE OF MY >>>>> MAIN >>>>> PYTHON SCRIPT APPEARS SOMEPLACE AGAIN?!?! >>>> >>>> Whoever did it, he seems also to have gained access to your emails as >>>> well. Congratulations to him, I think he's done well. >>> >>> How did this happened i asked. >>> I must know how did this happen so i take action to prevent it from >>> happening again. >>> >>> I you want to congratulate Mark Lawrence do it in private. >> >> Brilliantly funny on the gounds that I haven't got the faintest idea >> about how web systems are put together, let alone hacking them. >> >> But am I replying to the real, really incompetant Greek moron or the >> person who has pinched his email account? I'll happily admit to being >> far too lazy to try and find out. > > You started the thread didn't you? > And the file's contents that was uploaded speaks out loud your kind of > insults. > > How tough ws to break in since you had the account password by seeing it > from the source code? > > Don't try to blame someone else for this and pretend you didn't ahd > anythign to dop with it. > > Who pinched my mail account? i never posted by mistake any of my mail > accounts' passwords, except once from gmail's which i chnaged it > immeditely.What are you talking about? > > As far accusing me for being a moron i have i have fixed it immediately. > > Since you are so clever jump the fense again grasshoper. > Okay I'm not falling for this. This has to be the pseudo Nikos as there aren't enough typos for it to be the real one. -- Cheers. Mark Lawrence From nikos.gr33k at gmail.com Tue Oct 1 09:52:13 2013 From: nikos.gr33k at gmail.com (=?UTF-8?B?zp3Or866zr/Pgg==?=) Date: Tue, 01 Oct 2013 16:52:13 +0300 Subject: I haev fixed it In-Reply-To: References: <20130930235914.GB13553@secure.superhost.gr> Message-ID: ???? 1/10/2013 4:44 ??, ?/? Mark Lawrence ??????: > On 01/10/2013 14:34, ????? wrote: >> ???? 1/10/2013 4:23 ??, ?/? Mark Lawrence ??????: >>> On 01/10/2013 11:54, ????? wrote: >>>> ???? 1/10/2013 1:47 ??, ?/? Chris Angelico ??????: >>>>> On Tue, Oct 1, 2013 at 8:16 PM, ????? wrote: >>>>>> WHO AND MOST IMPORTNTANLY HOW DID HE MANAGED TO UPLOAD THIS FILE >>>>>> ON MY >>>>>> ACCOUNT? >>>>>> >>>>>> PLEASE ANSWER ME, I WONT GET MAD, BUT THIS IS AN IMPORTANT SECURITY >>>>>> RISK. >>>>>> >>>>>> SOMEONES MUST HAVE ACCESS TO MY ACCOUNT, DOES THE SOURCE CODE OF MY >>>>>> MAIN >>>>>> PYTHON SCRIPT APPEARS SOMEPLACE AGAIN?!?! >>>>> >>>>> Whoever did it, he seems also to have gained access to your emails as >>>>> well. Congratulations to him, I think he's done well. >>>> >>>> How did this happened i asked. >>>> I must know how did this happen so i take action to prevent it from >>>> happening again. >>>> >>>> I you want to congratulate Mark Lawrence do it in private. >>> >>> Brilliantly funny on the gounds that I haven't got the faintest idea >>> about how web systems are put together, let alone hacking them. >>> >>> But am I replying to the real, really incompetant Greek moron or the >>> person who has pinched his email account? I'll happily admit to being >>> far too lazy to try and find out. >> >> You started the thread didn't you? >> And the file's contents that was uploaded speaks out loud your kind of >> insults. >> >> How tough ws to break in since you had the account password by seeing it >> from the source code? >> >> Don't try to blame someone else for this and pretend you didn't ahd >> anythign to dop with it. >> >> Who pinched my mail account? i never posted by mistake any of my mail >> accounts' passwords, except once from gmail's which i chnaged it >> immeditely.What are you talking about? >> >> As far accusing me for being a moron i have i have fixed it immediately. >> >> Since you are so clever jump the fense again grasshoper. >> > > Okay I'm not falling for this. This has to be the pseudo Nikos as there > aren't enough typos for it to be the real one. I can type properly you know if i'am not in a hurry. Come on, don't be a coward try again to break in! Jump the fence, unlock the door...or admin that you can't. From breamoreboy at yahoo.co.uk Tue Oct 1 10:00:44 2013 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Tue, 01 Oct 2013 15:00:44 +0100 Subject: I haev fixed it In-Reply-To: References: <20130930235914.GB13553@secure.superhost.gr> Message-ID: On 01/10/2013 14:52, ????? wrote: > ???? 1/10/2013 4:44 ??, ?/? Mark Lawrence ??????: >> On 01/10/2013 14:34, ????? wrote: >>> ???? 1/10/2013 4:23 ??, ?/? Mark Lawrence ??????: >>>> On 01/10/2013 11:54, ????? wrote: >>>>> ???? 1/10/2013 1:47 ??, ?/? Chris Angelico ??????: >>>>>> On Tue, Oct 1, 2013 at 8:16 PM, ????? wrote: >>>>>>> WHO AND MOST IMPORTNTANLY HOW DID HE MANAGED TO UPLOAD THIS FILE >>>>>>> ON MY >>>>>>> ACCOUNT? >>>>>>> >>>>>>> PLEASE ANSWER ME, I WONT GET MAD, BUT THIS IS AN IMPORTANT SECURITY >>>>>>> RISK. >>>>>>> >>>>>>> SOMEONES MUST HAVE ACCESS TO MY ACCOUNT, DOES THE SOURCE CODE OF MY >>>>>>> MAIN >>>>>>> PYTHON SCRIPT APPEARS SOMEPLACE AGAIN?!?! >>>>>> >>>>>> Whoever did it, he seems also to have gained access to your emails as >>>>>> well. Congratulations to him, I think he's done well. >>>>> >>>>> How did this happened i asked. >>>>> I must know how did this happen so i take action to prevent it from >>>>> happening again. >>>>> >>>>> I you want to congratulate Mark Lawrence do it in private. >>>> >>>> Brilliantly funny on the gounds that I haven't got the faintest idea >>>> about how web systems are put together, let alone hacking them. >>>> >>>> But am I replying to the real, really incompetant Greek moron or the >>>> person who has pinched his email account? I'll happily admit to being >>>> far too lazy to try and find out. >>> >>> You started the thread didn't you? >>> And the file's contents that was uploaded speaks out loud your kind of >>> insults. >>> >>> How tough ws to break in since you had the account password by seeing it >>> from the source code? >>> >>> Don't try to blame someone else for this and pretend you didn't ahd >>> anythign to dop with it. >>> >>> Who pinched my mail account? i never posted by mistake any of my mail >>> accounts' passwords, except once from gmail's which i chnaged it >>> immeditely.What are you talking about? >>> >>> As far accusing me for being a moron i have i have fixed it immediately. >>> >>> Since you are so clever jump the fense again grasshoper. >>> >> >> Okay I'm not falling for this. This has to be the pseudo Nikos as there >> aren't enough typos for it to be the real one. > > I can type properly you know if i'am not in a hurry. > Come on, don't be a coward try again to break in! > Jump the fence, unlock the door...or admin that you can't. > Yes very subtle, now come on and admit who you really are, the tension is killing me. -- Cheers. Mark Lawrence From feedthetroll at gmx.de Tue Oct 1 10:08:24 2013 From: feedthetroll at gmx.de (feedthetroll at gmx.de) Date: Tue, 1 Oct 2013 07:08:24 -0700 (PDT) Subject: I haev fixed it In-Reply-To: References: <20130930235914.GB13553@secure.superhost.gr> Message-ID: Am Dienstag, 1. Oktober 2013 15:34:26 UTC+2 schrieb Ferrous Cranus: > ???? 1/10/2013 4:23 ??, ?/? Mark Lawrence ??????: >> ... >> But am I replying to the real, really incompetant Greek moron or the >> person who has pinched his email account? I'll happily admit to being >> far too lazy to try and find out. > > You started the thread didn't you? Your are wrong (again) Nikos. Mark did not start the thread. YOU did: The thread was started by nikos at secure.superhost.gr ( see https://groups.google.com/forum/#!topic/comp.lang.python/_J_3e6_LIa8) So stop accusing others! > And the file's contents that was uploaded speaks out loud your kind of > insults. Mark is not the only one telling you the truth in this group. > ... From nikos.gr33k at gmail.com Tue Oct 1 10:32:57 2013 From: nikos.gr33k at gmail.com (=?UTF-8?B?zp3Or866zr/Pgg==?=) Date: Tue, 01 Oct 2013 17:32:57 +0300 Subject: I haev fixed it In-Reply-To: References: <20130930235914.GB13553@secure.superhost.gr> Message-ID: ???? 1/10/2013 5:08 ??, ?/? feedthetroll at gmx.de ??????: > Am Dienstag, 1. Oktober 2013 15:34:26 UTC+2 schrieb Ferrous Cranus: >> ???? 1/10/2013 4:23 ??, ?/? Mark Lawrence ??????: >>> ... >>> But am I replying to the real, really incompetant Greek moron or the >>> person who has pinched his email account? I'll happily admit to being >>> far too lazy to try and find out. >> >> You started the thread didn't you? > Your are wrong (again) Nikos. Mark did not start the thread. YOU did: > The thread was started by nikos at secure.superhost.gr > ( see https://groups.google.com/forum/#!topic/comp.lang.python/_J_3e6_LIa8) > So stop accusing others! > >> And the file's contents that was uploaded speaks out loud your kind of >> insults. > Mark is not the only one telling you the truth in this group. > >> ... Yeah right..... X-Spam-Evidence: '*H*': 0.46; '*S*': 0.11; 'header:User-Agent:1': 0.23; 'received:84': 0.35; 'charset:us-ascii': 0.36; 'to:addr :python-list': 0.38; 'to:addr:python.org': 0.39; 'from:no real name:2**0': 0.61; 'content-disposition:inline': 0.62 Date: Mon, 30 Sep 2013 23:59:14 +0000 From: ni... at secure.superhost.gr To: pytho... at python.org Subject: I haev fixed it MIME-Version: 1.0 User-Agent: Mutt/1.5.20 (2009-12-10) X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - secure.superhost.gr X-AntiAbuse: Original Domain - python.org X-AntiAbuse: Originator/Caller UID/GID - [515 515] / [47 12] X-AntiAbuse: Sender Address Domain - secure.superhost.gr X-Get-Message-Sender-Via: secure.superhost.gr: authenticated_id: nikos/primary_hostname/system user As you see here, here the origiv of the mail is from my server and the forger used the mail agent Mutt to send the mail. *I* use ThunderBird, not Mutt mail client. Don't cover him up. The message of the file also contain insulting languge like he uses all th etime against me. From feedthetroll at gmx.de Tue Oct 1 11:53:36 2013 From: feedthetroll at gmx.de (feedthetroll at gmx.de) Date: Tue, 1 Oct 2013 08:53:36 -0700 (PDT) Subject: I haev fixed it In-Reply-To: References: <20130930235914.GB13553@secure.superhost.gr> Message-ID: <07f9768c-0d38-4b7f-b451-8d8dd7b721ce@googlegroups.com> Am Dienstag, 1. Oktober 2013 16:32:57 UTC+2 schrieb Ferrous Cranus: > ???? 1/10/2013 5:08 ??, ?/? feedthetroll ??????: >> Am Dienstag, 1. Oktober 2013 15:34:26 UTC+2 schrieb Ferrous Cranus: >>> ???? 1/10/2013 4:23 ??, ?/? Mark Lawrence ??????: >>>> ... >>>> But am I replying to the real, really incompetant Greek moron or the >>>> person who has pinched his email account? I'll happily admit to being >>>> far too lazy to try and find out. >>> >>> You started the thread didn't you? >> Your are wrong (again) Nikos. Mark did not start the thread. YOU did: >> The thread was started by nikos at secure.superhost.gr >> ( see https://groups.google.com/forum/#!topic/comp.lang.python/_J_3e6_LIa8) >> So stop accusing others! >> >>> And the file's contents that was uploaded speaks out loud your kind of >>> insults. >> Mark is not the only one telling you the truth in this group. >> >>> ... >Yeah right..... I am very happy, that you admit, that not Mark but YOU started the thread. You said, that the one who started the thread, broke into your system. So YOU broke into your system and are fooling us (again). > ... > As you see here, here the origiv of the mail is from my server and the > forger used the mail agent Mutt to send the mail. > > *I* use ThunderBird, not Mutt mail client. You also use gmail and even systems of other people (as you told us months ago when someone asked why you were using so many different FROM - addresses), so we can not know, that you are not also using mutt. > > ... > The message of the file also contain insulting languge like he uses all > th etime against me. And the original subject was full of typos. You are the only one committing thousands of typos. So, following your arguments, this is the proof, that YOU started the thread. From nikos.gr33k at gmail.com Tue Oct 1 11:57:25 2013 From: nikos.gr33k at gmail.com (=?UTF-8?B?zp3Or866zr/Pgg==?=) Date: Tue, 01 Oct 2013 18:57:25 +0300 Subject: I haev fixed it In-Reply-To: <07f9768c-0d38-4b7f-b451-8d8dd7b721ce@googlegroups.com> References: <20130930235914.GB13553@secure.superhost.gr> <07f9768c-0d38-4b7f-b451-8d8dd7b721ce@googlegroups.com> Message-ID: ???? 1/10/2013 6:53 ??, ?/? feedthetroll at gmx.de ??????: > Am Dienstag, 1. Oktober 2013 16:32:57 UTC+2 schrieb Ferrous Cranus: >> ???? 1/10/2013 5:08 ??, ?/? feedthetroll ??????: >>> Am Dienstag, 1. Oktober 2013 15:34:26 UTC+2 schrieb Ferrous Cranus: >>>> ???? 1/10/2013 4:23 ??, ?/? Mark Lawrence ??????: >>>>> ... >>>>> But am I replying to the real, really incompetant Greek moron or the >>>>> person who has pinched his email account? I'll happily admit to being >>>>> far too lazy to try and find out. >>>> >>>> You started the thread didn't you? >>> Your are wrong (again) Nikos. Mark did not start the thread. YOU did: >>> The thread was started by nikos at secure.superhost.gr >>> ( see https://groups.google.com/forum/#!topic/comp.lang.python/_J_3e6_LIa8) >>> So stop accusing others! >>> >>>> And the file's contents that was uploaded speaks out loud your kind of >>>> insults. >>> Mark is not the only one telling you the truth in this group. >>> >>>> ... >> Yeah right..... > I am very happy, that you admit, that not Mark but YOU started the thread. You said, that the one who started the thread, broke into your system. So YOU broke into your system and are fooling us (again). > >> ... >> As you see here, here the origiv of the mail is from my server and the >> forger used the mail agent Mutt to send the mail. >> >> *I* use ThunderBird, not Mutt mail client. > You also use gmail and even systems of other people (as you told us months ago when someone asked why you were using so many different FROM - addresses), so we can not know, that you are not also using mutt. > >> >> ... >> The message of the file also contain insulting languge like he uses all >> th etime against me. > And the original subject was full of typos. You are the only one committing thousands of typos. So, following your arguments, this is the proof, that YOU started the thread. > You are a liar and a bad one. From feedthetroll at gmx.de Tue Oct 1 12:40:51 2013 From: feedthetroll at gmx.de (feedthetroll at gmx.de) Date: Tue, 1 Oct 2013 09:40:51 -0700 (PDT) Subject: I haev fixed it In-Reply-To: References: <20130930235914.GB13553@secure.superhost.gr> <07f9768c-0d38-4b7f-b451-8d8dd7b721ce@googlegroups.com> Message-ID: <7f277c2d-6345-49a3-a0fd-1616f1704be8@googlegroups.com> Am Dienstag, 1. Oktober 2013 17:57:25 UTC+2 schrieb Ferrous Cranus: > ???? 1/10/2013 6:53 ??, ?/? feedthetroll ??????: >> Am Dienstag, 1. Oktober 2013 16:32:57 UTC+2 schrieb Ferrous Cranus: >>> ???? 1/10/2013 5:08 ??, ?/? feedthetroll ??????: >>>> Am Dienstag, 1. Oktober 2013 15:34:26 UTC+2 schrieb Ferrous Cranus: >>>>> ???? 1/10/2013 4:23 ??, ?/? Mark Lawrence ??????: >>>>>> ... >>>>>> But am I replying to the real, really incompetant Greek moron or the >>>>>> person who has pinched his email account? I'll happily admit to being >>>>>> far too lazy to try and find out. >>>>> >>>>> You started the thread didn't you? >>>> Your are wrong (again) Nikos. Mark did not start the thread. YOU did: >>>> The thread was started by nikos at secure.superhost.gr >>>> ( see https://groups.google.com/forum/#!topic/comp.lang.python/_J_3e6_LIa8) >>>> So stop accusing others! >>>> >>>>> And the file's contents that was uploaded speaks out loud your kind of >>>>> insults. >>>> Mark is not the only one telling you the truth in this group. >>>> >>>>> ... >>> Yeah right..... >> I am very happy, that you admit, that not Mark but YOU started the thread. >> You said, that the one who started the thread, broke into your system. So >> YOU broke into your system and are fooling us (again). >> >>> ... >>> As you see here, here the origiv of the mail is from my server and the >>> forger used the mail agent Mutt to send the mail. >>> >>> *I* use ThunderBird, not Mutt mail client. >> You also use gmail and even systems of other people (as you told us months >> ago when someone asked why you were using so many different FROM - >> addresses), so we can not know, that you are not also using mutt. >>> ... >>> The message of the file also contain insulting languge like he uses all >>> th etime against me. >> And the original subject was full of typos. You are the only one committing >> thousands of typos. So, following your arguments, this is the proof, that >> YOU started the thread. >> > You are a liar and a bad one. Do you have any points / evidence or are you just trolling again. btw: If you write "Yeah right....." (see 23 lines above) you should read what you agree to. But I forgot, you do not read answers to your posts. You just write arbitrary things. Just like a troll does. From nikos.gr33k at gmail.com Tue Oct 1 18:46:44 2013 From: nikos.gr33k at gmail.com (=?UTF-8?B?zp3Or866zr/Pgg==?=) Date: Wed, 02 Oct 2013 01:46:44 +0300 Subject: I haev fixed it In-Reply-To: <7f277c2d-6345-49a3-a0fd-1616f1704be8@googlegroups.com> References: <20130930235914.GB13553@secure.superhost.gr> <07f9768c-0d38-4b7f-b451-8d8dd7b721ce@googlegroups.com> <7f277c2d-6345-49a3-a0fd-1616f1704be8@googlegroups.com> Message-ID: ???? 1/10/2013 7:40 ??, ?/? feedthetroll at gmx.de ??????: > Am Dienstag, 1. Oktober 2013 17:57:25 UTC+2 schrieb Ferrous Cranus: >> ???? 1/10/2013 6:53 ??, ?/? feedthetroll ??????: >>> Am Dienstag, 1. Oktober 2013 16:32:57 UTC+2 schrieb Ferrous Cranus: >>>> ???? 1/10/2013 5:08 ??, ?/? feedthetroll ??????: >>>>> Am Dienstag, 1. Oktober 2013 15:34:26 UTC+2 schrieb Ferrous Cranus: >>>>>> ???? 1/10/2013 4:23 ??, ?/? Mark Lawrence ??????: >>>>>>> ... >>>>>>> But am I replying to the real, really incompetant Greek moron or the >>>>>>> person who has pinched his email account? I'll happily admit to being >>>>>>> far too lazy to try and find out. >>>>>> >>>>>> You started the thread didn't you? >>>>> Your are wrong (again) Nikos. Mark did not start the thread. YOU did: >>>>> The thread was started by nikos at secure.superhost.gr >>>>> ( see https://groups.google.com/forum/#!topic/comp.lang.python/_J_3e6_LIa8) >>>>> So stop accusing others! >>>>> >>>>>> And the file's contents that was uploaded speaks out loud your kind of >>>>>> insults. >>>>> Mark is not the only one telling you the truth in this group. >>>>> >>>>>> ... >>>> Yeah right..... >>> I am very happy, that you admit, that not Mark but YOU started the thread. >>> You said, that the one who started the thread, broke into your system. So >>> YOU broke into your system and are fooling us (again). >>> >>>> ... >>>> As you see here, here the origiv of the mail is from my server and the >>>> forger used the mail agent Mutt to send the mail. >>>> >>>> *I* use ThunderBird, not Mutt mail client. >>> You also use gmail and even systems of other people (as you told us months >>> ago when someone asked why you were using so many different FROM - >>> addresses), so we can not know, that you are not also using mutt. >>>> ... >>>> The message of the file also contain insulting languge like he uses all >>>> th etime against me. >>> And the original subject was full of typos. You are the only one committing >>> thousands of typos. So, following your arguments, this is the proof, that >>> YOU started the thread. >>> >> You are a liar and a bad one. > Do you have any points / evidence or are you just trolling again. > > btw: If you write "Yeah right....." (see 23 lines above) you should read what you agree to. But I forgot, you do not read answers to your posts. You just write arbitrary things. Just like a troll does. > The hacker created all the above situation to made it look it was me. But it wasn't me. And i was never used "Mutt' as MUA. If only there was a log file that could show the connection made by the hacker's host, so to post it here. From breamoreboy at yahoo.co.uk Tue Oct 1 18:57:08 2013 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Tue, 01 Oct 2013 23:57:08 +0100 Subject: I haev fixed it In-Reply-To: References: <20130930235914.GB13553@secure.superhost.gr> <07f9768c-0d38-4b7f-b451-8d8dd7b721ce@googlegroups.com> <7f277c2d-6345-49a3-a0fd-1616f1704be8@googlegroups.com> Message-ID: On 01/10/2013 23:46, ????? wrote: > > If only there was a log file that could show the connection made by the > hacker's host, so to post it here. I'd write a serious letter of complaint to all of your customers asking them why they've not provided you with log files that list what they do on the system that you've provided them with. At least I think that's how these things usually work, isn't it? -- Cheers. Mark Lawrence From nikos.gr33k at gmail.com Tue Oct 1 19:28:49 2013 From: nikos.gr33k at gmail.com (=?UTF-8?B?zp3Or866zr/Pgg==?=) Date: Wed, 02 Oct 2013 02:28:49 +0300 Subject: I haev fixed it In-Reply-To: References: <20130930235914.GB13553@secure.superhost.gr> <07f9768c-0d38-4b7f-b451-8d8dd7b721ce@googlegroups.com> <7f277c2d-6345-49a3-a0fd-1616f1704be8@googlegroups.com> Message-ID: ???? 2/10/2013 1:57 ??, ?/? Mark Lawrence ??????: > On 01/10/2013 23:46, ????? wrote: >> >> If only there was a log file that could show the connection made by the >> hacker's host, so to post it here. > > I'd write a serious letter of complaint to all of your customers asking > them why they've not provided you with log files that list what they do > on the system that you've provided them with. At least I think that's > how these things usually work, isn't it? My customer's are not hackers. Neither are you. You show my password in plain text somehow and decided to break in. The contents of the warning.html file also fits your style of insult. Stop pretending it wasn't you. From breamoreboy at yahoo.co.uk Wed Oct 2 10:01:31 2013 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Wed, 02 Oct 2013 15:01:31 +0100 Subject: I haev fixed it In-Reply-To: References: <20130930235914.GB13553@secure.superhost.gr> <07f9768c-0d38-4b7f-b451-8d8dd7b721ce@googlegroups.com> <7f277c2d-6345-49a3-a0fd-1616f1704be8@googlegroups.com> Message-ID: On 02/10/2013 00:28, ????? wrote: > ???? 2/10/2013 1:57 ??, ?/? Mark Lawrence ??????: >> On 01/10/2013 23:46, ????? wrote: >>> >>> If only there was a log file that could show the connection made by the >>> hacker's host, so to post it here. >> >> I'd write a serious letter of complaint to all of your customers asking >> them why they've not provided you with log files that list what they do >> on the system that you've provided them with. At least I think that's >> how these things usually work, isn't it? > > My customer's are not hackers. > Neither are you. > You show my password in plain text somehow and decided to break in. > The contents of the warning.html file also fits your style of insult. > Stop pretending it wasn't you. > I want a full apology to this entire group now for your behaviour, and a very specific formal apology to myself for your completely unfounded allegations. I expect to see this by 23:59 2nd October 2013 BST. -- Cheers. Mark Lawrence From denismfmcmahon at gmail.com Wed Oct 2 11:48:02 2013 From: denismfmcmahon at gmail.com (Denis McMahon) Date: Wed, 2 Oct 2013 15:48:02 +0000 (UTC) Subject: I haev fixed it References: <20130930235914.GB13553@secure.superhost.gr> <07f9768c-0d38-4b7f-b451-8d8dd7b721ce@googlegroups.com> <7f277c2d-6345-49a3-a0fd-1616f1704be8@googlegroups.com> Message-ID: On Wed, 02 Oct 2013 15:01:31 +0100, Mark Lawrence wrote: > I want a full apology to this entire group now for your behaviour, and a > very specific formal apology to myself for your completely unfounded > allegations. I expect to see this by 23:59 2nd October 2013 BST. Honestly? Or you'll do what? Sulk? Cry like a baby? Whinge and whine loudly? Or are you going to throw money away on this? Because if you are, I have a bridge for sale. He's an idiot. Everyone can see he's an idiot. I don't think anyone here believes that he has any ability at all in identifying who used what method to upload a file to his server. If you want to make yourself appear to be as stupid as he makes himself appear all the time, carry on making demands. Because when push comes to shove, the only satisfaction you'll ever get is by taking civil action against him, and that's going to mean you putting your money up front, and hoping you can recover those costs from him afterwards. If he hasn't got the money, your costs judgement against him is just a piece of worthless paper, but your legal costs in bringing the action are your responsibility. So before starting down the path of threatening legal action, I suggest you run a comprehensive credit check on your target. -- Denis McMahon, denismfmcmahon at gmail.com From breamoreboy at yahoo.co.uk Wed Oct 2 12:07:43 2013 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Wed, 02 Oct 2013 17:07:43 +0100 Subject: I haev fixed it In-Reply-To: References: <20130930235914.GB13553@secure.superhost.gr> <07f9768c-0d38-4b7f-b451-8d8dd7b721ce@googlegroups.com> <7f277c2d-6345-49a3-a0fd-1616f1704be8@googlegroups.com> Message-ID: On 02/10/2013 16:48, Denis McMahon wrote: > On Wed, 02 Oct 2013 15:01:31 +0100, Mark Lawrence wrote: > >> I want a full apology to this entire group now for your behaviour, and a >> very specific formal apology to myself for your completely unfounded >> allegations. I expect to see this by 23:59 2nd October 2013 BST. > > So before starting down the path of threatening legal action, I suggest > you run a comprehensive credit check on your target. > Pardon my ignorance but how does asking for an apology translate into threatening legal action, something that I've already stated that I won't do for several reasons, the main one of which is that I find these threads hilarious? -- Roses are red, Violets are blue, Most poems rhyme, But this one doesn't. Mark Lawrence From denismfmcmahon at gmail.com Wed Oct 2 14:57:47 2013 From: denismfmcmahon at gmail.com (Denis McMahon) Date: Wed, 2 Oct 2013 18:57:47 +0000 (UTC) Subject: I haev fixed it References: <20130930235914.GB13553@secure.superhost.gr> <07f9768c-0d38-4b7f-b451-8d8dd7b721ce@googlegroups.com> <7f277c2d-6345-49a3-a0fd-1616f1704be8@googlegroups.com> Message-ID: On Wed, 02 Oct 2013 17:07:43 +0100, Mark Lawrence wrote: > Pardon my ignorance but how does asking for an apology translate into > threatening legal action, something that I've already stated that I > won't do for several reasons, the main one of which is that I find these > threads hilarious? You're making the sort of demand for an apology that's usually followed by a comment such as "or I will sue you for defamation". Personally I think demanding an apology is a waste of time and ng bandwidth, but *shrugs* whatever. -- Denis McMahon, denismfmcmahon at gmail.com From steve+comp.lang.python at pearwood.info Tue Oct 1 22:35:18 2013 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: 02 Oct 2013 02:35:18 GMT Subject: I haev fixed it References: <20130930235914.GB13553@secure.superhost.gr> Message-ID: <524b8665$0$29984$c3e8da3$5496439d@news.astraweb.com> On Tue, 01 Oct 2013 13:54:32 +0300, ????? wrote: > How did this happened i asked. > I must know how did this happen so i take action to prevent it from > happening again. If I recall, some of your haters (Mark Lawrence, Antoon Pardon and Chris Warrick, if I remember correctly) were making lots of noise about reporting you to the Greek police and having you shut down for incompetence, as if incompetence was a criminal matter. And now it appears that somebody now *actually has* broken the law to try to shut you down. This follows death threats made against you. Under post 9/11 laws, death threats almost certainly would be classified as "terrorism". I recommend that you report this to your hosting provider, your local police, and Interpol. If your Gmail account was hacked, you should also report it to Google. Your local police are probably too busy to care, but stick to it. At the very least make sure they take your complaint. Get it on file. Be careful not to destroy evidence. Don't go deleting log files, htaccess files or anything else that can be used to identifier the criminal. Interpol have a division specialising in international computer crime and if you stick to it with your usual bull-headedness I'm sure you can convince them to investigate. Your ISP can probably tell you the IP address of who logged into your account. If your Gmail account was accessed, Google can do the same. Nikos, I would treat this seriously. Who knows what this criminal will do next? Those hating you have talked about getting you arrested, or having you killed. Now somebody has escalated into breaking into your email account. They've already proven that they want to destroy your business and your livelihood. What will they do next? Try to frame you for child molestation by planting illegal material? Nikos, if I were you I would not stop until you have tracked down this criminal hacker and had him arrested, if it takes you a year. Put all your energy and famous bull-headedness into this, and I'm sure you will be successful. I am totally serious here. You have a strong case -- a group of people who have publicly harassed you, publicly tried to shut down your business, made death threats against you, and now somebody has hacked into your account and tried to scare off your customers. And then, in the name of all that's holy, get yourself some professional help to secure your website. -- Steven From nikos.gr33k at gmail.com Wed Oct 2 06:12:28 2013 From: nikos.gr33k at gmail.com (=?UTF-8?B?zp3Or866zr/Pgg==?=) Date: Wed, 02 Oct 2013 13:12:28 +0300 Subject: I haev fixed it In-Reply-To: <524b8665$0$29984$c3e8da3$5496439d@news.astraweb.com> References: <20130930235914.GB13553@secure.superhost.gr> <524b8665$0$29984$c3e8da3$5496439d@news.astraweb.com> Message-ID: ???? 2/10/2013 5:35 ??, ?/? Steven D'Aprano ??????: > On Tue, 01 Oct 2013 13:54:32 +0300, ????? wrote: > >> How did this happened i asked. >> I must know how did this happen so i take action to prevent it from >> happening again. > > If I recall, some of your haters (Mark Lawrence, Antoon Pardon and Chris > Warrick, if I remember correctly) were making lots of noise about > reporting you to the Greek police and having you shut down for > incompetence, as if incompetence was a criminal matter. And now it > appears that somebody now *actually has* broken the law to try to shut > you down. This follows death threats made against you. Under post 9/11 > laws, death threats almost certainly would be classified as "terrorism". > > I recommend that you report this to your hosting provider, your local > police, and Interpol. If your Gmail account was hacked, you should also > report it to Google. Your local police are probably too busy to care, but > stick to it. At the very least make sure they take your complaint. Get it > on file. > > Be careful not to destroy evidence. Don't go deleting log files, htaccess > files or anything else that can be used to identifier the criminal. > > Interpol have a division specialising in international computer crime and > if you stick to it with your usual bull-headedness I'm sure you can > convince them to investigate. Your ISP can probably tell you the IP > address of who logged into your account. If your Gmail account was > accessed, Google can do the same. > > Nikos, I would treat this seriously. Who knows what this criminal will do > next? Those hating you have talked about getting you arrested, or having > you killed. Now somebody has escalated into breaking into your email > account. They've already proven that they want to destroy your business > and your livelihood. What will they do next? Try to frame you for child > molestation by planting illegal material? > > Nikos, if I were you I would not stop until you have tracked down this > criminal hacker and had him arrested, if it takes you a year. Put all > your energy and famous bull-headedness into this, and I'm sure you will > be successful. > > I am totally serious here. You have a strong case -- a group of people > who have publicly harassed you, publicly tried to shut down your > business, made death threats against you, and now somebody has hacked > into your account and tried to scare off your customers. > > And then, in the name of all that's holy, get yourself some professional > help to secure your website. You tell me Steven, If someone was to make a lawsuit that would be me against them, not the other way around. From piet at vanoostrum.org Tue Oct 1 14:20:53 2013 From: piet at vanoostrum.org (Piet van Oostrum) Date: Tue, 01 Oct 2013 14:20:53 -0400 Subject: I haev fixed it References: <20130930235914.GB13553@secure.superhost.gr> Message-ID: ????? writes: > Just logged in via FTP to my server and i saw an uploade file named > "Warnign html" > > Contents were: > > WARNING > > I am incompetent. Do not hire me! > > Question: > > WHO AND MOST IMPORTNTANLY HOW DID HE MANAGED TO UPLOAD THIS FILE ON MY > ACCOUNT? > > PLEASE ANSWER ME, I WONT GET MAD, BUT THIS IS AN IMPORTANT SECURITY RISK. > > SOMEONES MUST HAVE ACCESS TO MY ACCOUNT, DOES THE SOURCE CODE OF MY MAIN > PYTHON SCRIPT APPEARS SOMEPLACE AGAIN?!?! These questions show that the one who put that warning on your site was right. -- Piet van Oostrum WWW: http://pietvanoostrum.com/ PGP key: [8DAE142BE17999C4] From ron.eggler at gmail.com Tue Oct 1 00:14:53 2013 From: ron.eggler at gmail.com (cerr) Date: Mon, 30 Sep 2013 21:14:53 -0700 (PDT) Subject: extraction tool using CRF++ Message-ID: <90b8ca83-fb81-40d6-a864-f1c0e07bca76@googlegroups.com> Hi, I want to write an extraction tool using CRF++ (http://crfpp.googlecode.com/svn/trunk/doc/index.html). I have written a trainings file and a template: training: banana FOOD B-NP bread FOOD I-NP template: U01:%x[0,1] U02:%x[1,1] and now I want to go ahead and extract the foods from a sentence like "how do I make a banana bread". Also, I'm unsure how I interface to crf++ with python, I compiled and installed it from source as described on the above website but I don't have a crf module available in python... From vlastimil.brom at gmail.com Tue Oct 1 10:24:37 2013 From: vlastimil.brom at gmail.com (Vlastimil Brom) Date: Tue, 1 Oct 2013 16:24:37 +0200 Subject: extraction tool using CRF++ In-Reply-To: <90b8ca83-fb81-40d6-a864-f1c0e07bca76@googlegroups.com> References: <90b8ca83-fb81-40d6-a864-f1c0e07bca76@googlegroups.com> Message-ID: 2013/10/1 cerr : > Hi, > > I want to write an extraction tool using CRF++ (http://crfpp.googlecode.com/svn/trunk/doc/index.html). > I have written a trainings file and a template: > training: > banana FOOD B-NP > bread FOOD I-NP > template: > U01:%x[0,1] > U02:%x[1,1] > > and now I want to go ahead and extract the foods from a sentence like "how do I make a banana bread". Also, I'm unsure how I interface to crf++ with python, I compiled and installed it from source as described on the above website but I don't have a crf module available in python... > -- > https://mail.python.org/mailman/listinfo/python-list Hi, I have unfortunately no experience with CRF++; if there is no python wrapper for it available, the usage might not be (easily) possible - depending on the character of this library, you may try accessing it e.g. via ctypes. Alternatively, you may try another packages already available, e.g. NLTK: http://nltk.org/ >>> import nltk >>> any(synset.lexname == "noun.food" for synset in nltk.corpus.wordnet.synsets("apple")) True >>> any(synset.lexname == "noun.food" for synset in nltk.corpus.wordnet.synsets("bread")) True >>> any(synset.lexname == "noun.food" for synset in nltk.corpus.wordnet.synsets("wine")) True >>> any(synset.lexname == "noun.food" for synset in nltk.corpus.wordnet.synsets("book")) False >>> any(synset.lexname == "noun.food" for synset in nltk.corpus.wordnet.synsets("pencil")) False # of course there might be some surprise, probably due to polysemy ore some specifics of the semantic description... >>> any(synset.lexname == "noun.food" for synset in nltk.corpus.wordnet.synsets("dog")) True >>> any(synset.lexname == "noun.food" for synset in nltk.corpus.wordnet.synsets("white")) True >>> cf. http://nltk.org/ http://nltk.googlecode.com/svn/trunk/doc/howto/wordnet.html http://www.velvetcache.org/2010/03/01/looking-up-words-in-a-dictionary-using-python http://wordnet.princeton.edu/man/lexnames.5WN.html hth, vbr From j.j.molenaar at gmail.com Tue Oct 1 11:04:00 2013 From: j.j.molenaar at gmail.com (Joost Molenaar) Date: Tue, 1 Oct 2013 17:04:00 +0200 Subject: extraction tool using CRF++ In-Reply-To: References: <90b8ca83-fb81-40d6-a864-f1c0e07bca76@googlegroups.com> Message-ID: Hi Ron, In the python/ subdirectory of the CRF++ source package there's a README with instructions on how to use the CRFPP python module. HTH, Joost On Tue, Oct 1, 2013 at 4:24 PM, Vlastimil Brom wrote: > 2013/10/1 cerr : >> Hi, >> >> I want to write an extraction tool using CRF++ (http://crfpp.googlecode.com/svn/trunk/doc/index.html). >> I have written a trainings file and a template: >> training: >> banana FOOD B-NP >> bread FOOD I-NP >> template: >> U01:%x[0,1] >> U02:%x[1,1] >> >> and now I want to go ahead and extract the foods from a sentence like "how do I make a banana bread". Also, I'm unsure how I interface to crf++ with python, I compiled and installed it from source as described on the above website but I don't have a crf module available in python... >> -- >> https://mail.python.org/mailman/listinfo/python-list > > > Hi, > I have unfortunately no experience with CRF++; if there is no python > wrapper for it available, the usage might not be (easily) possible - > depending on the character of this library, you may try accessing it > e.g. via ctypes. > > Alternatively, you may try another packages already available, e.g. > NLTK: http://nltk.org/ > >>>> import nltk >>>> any(synset.lexname == "noun.food" for synset in nltk.corpus.wordnet.synsets("apple")) > True >>>> any(synset.lexname == "noun.food" for synset in nltk.corpus.wordnet.synsets("bread")) > True >>>> any(synset.lexname == "noun.food" for synset in nltk.corpus.wordnet.synsets("wine")) > True >>>> any(synset.lexname == "noun.food" for synset in nltk.corpus.wordnet.synsets("book")) > False >>>> any(synset.lexname == "noun.food" for synset in nltk.corpus.wordnet.synsets("pencil")) > False > > # of course there might be some surprise, probably due to polysemy ore > some specifics of the semantic description... > >>>> any(synset.lexname == "noun.food" for synset in nltk.corpus.wordnet.synsets("dog")) > True >>>> any(synset.lexname == "noun.food" for synset in nltk.corpus.wordnet.synsets("white")) > True >>>> > > cf. > http://nltk.org/ > http://nltk.googlecode.com/svn/trunk/doc/howto/wordnet.html > http://www.velvetcache.org/2010/03/01/looking-up-words-in-a-dictionary-using-python > http://wordnet.princeton.edu/man/lexnames.5WN.html > > hth, > vbr > -- > https://mail.python.org/mailman/listinfo/python-list From ron.eggler at gmail.com Tue Oct 1 11:36:52 2013 From: ron.eggler at gmail.com (cerr) Date: Tue, 1 Oct 2013 08:36:52 -0700 (PDT) Subject: extraction tool using CRF++ In-Reply-To: References: <90b8ca83-fb81-40d6-a864-f1c0e07bca76@googlegroups.com> Message-ID: <6267fcd8-69ea-4968-975c-e45d1779c880@googlegroups.com> On Tuesday, October 1, 2013 3:04:00 PM UTC, Joost Molenaar wrote: > Hi Ron, > > In the python/ subdirectory of the CRF++ source package there's a > > README with instructions on how to use the CRFPP python module. > Joost, Hoops, didn't see that! Yes, Thanks! :) From wjw15129 at gmail.com Tue Oct 1 01:47:05 2013 From: wjw15129 at gmail.com (YetToCome) Date: Mon, 30 Sep 2013 22:47:05 -0700 (PDT) Subject: I have a problem when creating a django project Message-ID: <59215b57-a6a6-436b-aa5e-6b437612b3e9@googlegroups.com> I can't create a django project, I think the usage of the commend is correct...here is the error information. h:\jcode>django-admin.py startproject mysite Usage: django-admin.py subcommand [options] [args] Options: -v VERBOSITY, --verbosity=VERBOSITY Verbosity level; 0=minimal output, 1=normal output, 2=verbose output, 3=very verbose output --settings=SETTINGS The Python path to a settings module, e.g. "myproject.settings.main". If this isn't provided, the DJANGO_SETTINGS_MODULE environment variable will be used. --pythonpath=PYTHONPATH A directory to add to the Python path, e.g. "/home/djangoprojects/myproject". --traceback Print traceback on exception --version show program's version number and exit -h, --help show this help message and exit Type 'django-admin.py help ' for help on a specific subcommand. From wjw15129 at gmail.com Tue Oct 1 02:54:53 2013 From: wjw15129 at gmail.com (YetToCome) Date: Mon, 30 Sep 2013 23:54:53 -0700 (PDT) Subject: I have a problem when creating a django project In-Reply-To: <59215b57-a6a6-436b-aa5e-6b437612b3e9@googlegroups.com> References: <59215b57-a6a6-436b-aa5e-6b437612b3e9@googlegroups.com> Message-ID: <8d55b6e7-a7ff-4df8-be4a-bcf1d2d9b5c0@googlegroups.com> ? 2013?10?1????UTC+8??1?47?05??YetToCome??? > I can't create a django project, I think the usage of the commend is correct...here is the error information. > > > > > > h:\jcode>django-admin.py startproject mysite > > Usage: django-admin.py subcommand [options] [args] > > > > Options: > > -v VERBOSITY, --verbosity=VERBOSITY > > Verbosity level; 0=minimal output, 1=normal output, > > 2=verbose output, 3=very verbose output > > --settings=SETTINGS The Python path to a settings module, e.g. > > "myproject.settings.main". If this isn't provided, the > > DJANGO_SETTINGS_MODULE environment variable will be > > used. > > --pythonpath=PYTHONPATH > > A directory to add to the Python path, e.g. > > "/home/djangoprojects/myproject". > > --traceback Print traceback on exception > > --version show program's version number and exit > > -h, --help show this help message and exit > > > > Type 'django-admin.py help ' for help on a specific subcommand. And what happened now... h:\jcode>django-admin.py help Type 'django-admin.py help' for usage. From wjw15129 at gmail.com Tue Oct 1 03:24:23 2013 From: wjw15129 at gmail.com (YetToCome) Date: Tue, 1 Oct 2013 00:24:23 -0700 (PDT) Subject: I have a problem when creating a django project In-Reply-To: <8d55b6e7-a7ff-4df8-be4a-bcf1d2d9b5c0@googlegroups.com> References: <59215b57-a6a6-436b-aa5e-6b437612b3e9@googlegroups.com> <8d55b6e7-a7ff-4df8-be4a-bcf1d2d9b5c0@googlegroups.com> Message-ID: <12a4d429-c4d6-4f27-a8eb-cc88cf70cb8c@googlegroups.com> ? 2013?10?1????UTC+8??2?54?53??YetToCome??? > ? 2013?10?1????UTC+8??1?47?05??YetToCome??? > > > I can't create a django project, I think the usage of the commend is correct...here is the error information. > > > > > > > > > > > > > > > > > > h:\jcode>django-admin.py startproject mysite > > > > > > Usage: django-admin.py subcommand [options] [args] > > > > > > > > > > > > Options: > > > > > > -v VERBOSITY, --verbosity=VERBOSITY > > > > > > Verbosity level; 0=minimal output, 1=normal output, > > > > > > 2=verbose output, 3=very verbose output > > > > > > --settings=SETTINGS The Python path to a settings module, e.g. > > > > > > "myproject.settings.main". If this isn't provided, the > > > > > > DJANGO_SETTINGS_MODULE environment variable will be > > > > > > used. > > > > > > --pythonpath=PYTHONPATH > > > > > > A directory to add to the Python path, e.g. > > > > > > "/home/djangoprojects/myproject". > > > > > > --traceback Print traceback on exception > > > > > > --version show program's version number and exit > > > > > > -h, --help show this help message and exit > > > > > > > > > > > > Type 'django-admin.py help ' for help on a specific subcommand. > > > > And what happened now... > > > > h:\jcode>django-admin.py help > > Type 'django-admin.py help' for usage. ok...i solve the problem, clean the Registry and reinstall python From joel.goldstick at gmail.com Tue Oct 1 09:25:42 2013 From: joel.goldstick at gmail.com (Joel Goldstick) Date: Tue, 1 Oct 2013 09:25:42 -0400 Subject: I have a problem when creating a django project In-Reply-To: <12a4d429-c4d6-4f27-a8eb-cc88cf70cb8c@googlegroups.com> References: <59215b57-a6a6-436b-aa5e-6b437612b3e9@googlegroups.com> <8d55b6e7-a7ff-4df8-be4a-bcf1d2d9b5c0@googlegroups.com> <12a4d429-c4d6-4f27-a8eb-cc88cf70cb8c@googlegroups.com> Message-ID: did you know there is a django mailing list/news group. On Tue, Oct 1, 2013 at 3:24 AM, YetToCome wrote: > ? 2013?10?1????UTC+8??2?54?53??YetToCome??? > > ? 2013?10?1????UTC+8??1?47?05??YetToCome??? > > > > > I can't create a django project, I think the usage of the commend is > correct...here is the error information. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > h:\jcode>django-admin.py startproject mysite > > > > > > > > > > Usage: django-admin.py subcommand [options] [args] > > > > > > > > > > > > > > > > > > > > Options: > > > > > > > > > > -v VERBOSITY, --verbosity=VERBOSITY > > > > > > > > > > Verbosity level; 0=minimal output, 1=normal > output, > > > > > > > > > > 2=verbose output, 3=very verbose output > > > > > > > > > > --settings=SETTINGS The Python path to a settings module, e.g. > > > > > > > > > > "myproject.settings.main". If this isn't > provided, the > > > > > > > > > > DJANGO_SETTINGS_MODULE environment variable > will be > > > > > > > > > > used. > > > > > > > > > > --pythonpath=PYTHONPATH > > > > > > > > > > A directory to add to the Python path, e.g. > > > > > > > > > > "/home/djangoprojects/myproject". > > > > > > > > > > --traceback Print traceback on exception > > > > > > > > > > --version show program's version number and exit > > > > > > > > > > -h, --help show this help message and exit > > > > > > > > > > > > > > > > > > > > Type 'django-admin.py help ' for help on a specific > subcommand. > > > > > > > > And what happened now... > > > > > > > > h:\jcode>django-admin.py help > > > > Type 'django-admin.py help' for usage. > > ok...i solve the problem, clean the Registry and reinstall python > -- > https://mail.python.org/mailman/listinfo/python-list > -- Joel Goldstick http://joelgoldstick.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From nikos.gr33k at gmail.com Tue Oct 1 05:58:50 2013 From: nikos.gr33k at gmail.com (=?UTF-8?B?zp3Or866zr/Pgg==?=) Date: Tue, 01 Oct 2013 12:58:50 +0300 Subject: JUST GOT HACKED Message-ID: Just logged in via FTP to my server and i saw an uploade file named "Warnign html" Contents were: WARNING I am incompetent. Do not hire me! Question: WHO AND MOST IMPORTNTANLY HOW DID HE MANAGED TO UPLOAD THIS FILE ON MY ACCOUNT? PLEASE ANSWER ME, I WONT GET MAD, BUT THIS IS AN IMPORTANT SECURITY RISK. SOMEONES MUST HAVE ACCESS TO MY ACCOUNT, DOES THE SOURCE CODE OF MY MAIN PYTHON SCRIPT APPEARS SOMEPLACE AGAIN?!?! From breamoreboy at yahoo.co.uk Tue Oct 1 09:06:51 2013 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Tue, 01 Oct 2013 14:06:51 +0100 Subject: JUST GOT HACKED In-Reply-To: References: Message-ID: On 01/10/2013 10:58, ????? wrote: > Just logged in via FTP to my server and i saw an uploade file named > "Warnign html" > > Contents were: > > WARNING > > I am incompetent. Do not hire me! > > Question: > > WHO AND MOST IMPORTNTANLY HOW DID HE MANAGED TO UPLOAD THIS FILE ON MY > ACCOUNT? > > PLEASE ANSWER ME, I WONT GET MAD, BUT THIS IS AN IMPORTANT SECURITY RISK. > > SOMEONES MUST HAVE ACCESS TO MY ACCOUNT, DOES THE SOURCE CODE OF MY MAIN > PYTHON SCRIPT APPEARS SOMEPLACE AGAIN?!?! Would you please stop posting, I've almost burst my stomach laughing at this. You definetely have a ready made career writing comedy. -- Cheers. Mark Lawrence From nikos.gr33k at gmail.com Tue Oct 1 09:15:40 2013 From: nikos.gr33k at gmail.com (=?UTF-8?B?zp3Or866zr/Pgg==?=) Date: Tue, 01 Oct 2013 16:15:40 +0300 Subject: JUST GOT HACKED In-Reply-To: References: Message-ID: ???? 1/10/2013 4:06 ??, ?/? Mark Lawrence ??????: > On 01/10/2013 10:58, ????? wrote: >> Just logged in via FTP to my server and i saw an uploade file named >> "Warnign html" >> >> Contents were: >> >> WARNING >> >> I am incompetent. Do not hire me! >> >> Question: >> >> WHO AND MOST IMPORTNTANLY HOW DID HE MANAGED TO UPLOAD THIS FILE ON MY >> ACCOUNT? >> >> PLEASE ANSWER ME, I WONT GET MAD, BUT THIS IS AN IMPORTANT SECURITY RISK. >> >> SOMEONES MUST HAVE ACCESS TO MY ACCOUNT, DOES THE SOURCE CODE OF MY MAIN >> PYTHON SCRIPT APPEARS SOMEPLACE AGAIN?!?! > > Would you please stop posting, I've almost burst my stomach laughing at > this. You definetely have a ready made career writing comedy. Okey smartass, Try to do it again, if you be successfull again i'll even congratulate you myself. From kwpolska at gmail.com Tue Oct 1 09:27:57 2013 From: kwpolska at gmail.com (=?UTF-8?B?Q2hyaXMg4oCcS3dwb2xza2HigJ0gV2Fycmljaw==?=) Date: Tue, 1 Oct 2013 15:27:57 +0200 Subject: JUST GOT HACKED In-Reply-To: References: Message-ID: On Tue, Oct 1, 2013 at 3:15 PM, ????? wrote: > ???? 1/10/2013 4:06 ??, ?/? Mark Lawrence ??????: >> >> On 01/10/2013 10:58, ????? wrote: >>> >>> Just logged in via FTP to my server and i saw an uploade file named >>> "Warnign html" >>> >>> Contents were: >>> >>> WARNING >>> >>> I am incompetent. Do not hire me! >>> >>> Question: >>> >>> WHO AND MOST IMPORTNTANLY HOW DID HE MANAGED TO UPLOAD THIS FILE ON MY >>> ACCOUNT? >>> >>> PLEASE ANSWER ME, I WONT GET MAD, BUT THIS IS AN IMPORTANT SECURITY RISK. >>> >>> SOMEONES MUST HAVE ACCESS TO MY ACCOUNT, DOES THE SOURCE CODE OF MY MAIN >>> PYTHON SCRIPT APPEARS SOMEPLACE AGAIN?!?! >> >> >> Would you please stop posting, I've almost burst my stomach laughing at >> this. You definetely have a ready made career writing comedy. > > > Okey smartass, > > Try to do it again, if you be successfull again i'll even congratulate you > myself. > > -- > https://mail.python.org/mailman/listinfo/python-list It looks like you are accusing someone of doing something without any proof whatsoever. Would you like help with the fallout of the lawsuit that I hope Mark might (should!) come up with? Speaking of ?try again?, I doubt it would be hard? As long as a FTP daemon is running somewhere (and you clearly do not know better); or even you have a SSH daemon and you do not know better, an attacker can: a) wait for you to publish your password yet again; b) get you to download an exploit/keylogger/whatever; c) brute-force. Well, considering it?s unlikely you actually have a long-as-shit password, (c) is the best option. Unless your password is very long, in which case is not. I?m also wondering what language your password is in. If you actually used a Greek phrase, how long will it take you to get locked out due to encoding bullshit? -- Chris ?Kwpolska? Warrick PGP: 5EAAEA16 stop html mail | always bottom-post | only UTF-8 makes sense From breamoreboy at yahoo.co.uk Tue Oct 1 14:42:18 2013 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Tue, 01 Oct 2013 19:42:18 +0100 Subject: JUST GOT HACKED In-Reply-To: References: Message-ID: On 01/10/2013 14:27, Chris ?Kwpolska? Warrick wrote: > > It looks like you are accusing someone of doing something without any > proof whatsoever. Would you like help with the fallout of the lawsuit > that I hope Mark might (should!) come up with? > Why would I want to sue someone who's very kindly given me the longest, loudest laugh I've had in years? Besides I can't afford a lawyer, don't want to put any money the way of ThievingScumbagLawyers.co.uk, and what is the point of sueing a person who almost certainly is going bust and hence can't afford to pay up. Finally it was most certainly *NOT* me who did the hacking. If it had of been I would have taken the same type of action just to teach Nikos a lesson, but I would have owned up to my trickery after a few hours. As it happens my skills are so outdated that I wouldn't have the faintest idea where to start, unless of course I'd have taken the trouble of wading through the gazillions of posts talking about websites here with the occasional mention of how (not) to write Python. -- Cheers. Mark Lawrence From breamoreboy at yahoo.co.uk Tue Oct 1 09:28:56 2013 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Tue, 01 Oct 2013 14:28:56 +0100 Subject: JUST GOT HACKED In-Reply-To: References: Message-ID: On 01/10/2013 14:15, ????? wrote: > ???? 1/10/2013 4:06 ??, ?/? Mark Lawrence ??????: >> On 01/10/2013 10:58, ????? wrote: >>> Just logged in via FTP to my server and i saw an uploade file named >>> "Warnign html" >>> >>> Contents were: >>> >>> WARNING >>> >>> I am incompetent. Do not hire me! >>> >>> Question: >>> >>> WHO AND MOST IMPORTNTANLY HOW DID HE MANAGED TO UPLOAD THIS FILE ON MY >>> ACCOUNT? >>> >>> PLEASE ANSWER ME, I WONT GET MAD, BUT THIS IS AN IMPORTANT SECURITY >>> RISK. >>> >>> SOMEONES MUST HAVE ACCESS TO MY ACCOUNT, DOES THE SOURCE CODE OF MY MAIN >>> PYTHON SCRIPT APPEARS SOMEPLACE AGAIN?!?! >> >> Would you please stop posting, I've almost burst my stomach laughing at >> this. You definetely have a ready made career writing comedy. > > Okey smartass, > > Try to do it again, if you be successfull again i'll even congratulate > you myself. > Oh boy is this fun!!! Is this Nikos or is this not Nikos, that is the question? Thinking that I could hack a website, there's more likelihood that it's the latest member of the UK Royal Family. -- Cheers. Mark Lawrence From nikos.gr33k at gmail.com Tue Oct 1 09:42:31 2013 From: nikos.gr33k at gmail.com (=?UTF-8?B?zp3Or866zr/Pgg==?=) Date: Tue, 01 Oct 2013 16:42:31 +0300 Subject: JUST GOT HACKED In-Reply-To: References: Message-ID: ???? 1/10/2013 4:27 ??, ?/? Chris ?Kwpolska? Warrick ??????: > On Tue, Oct 1, 2013 at 3:15 PM, ????? wrote: >> ???? 1/10/2013 4:06 ??, ?/? Mark Lawrence ??????: >>> >>> On 01/10/2013 10:58, ????? wrote: >>>> >>>> Just logged in via FTP to my server and i saw an uploade file named >>>> "Warnign html" >>>> >>>> Contents were: >>>> >>>> WARNING >>>> >>>> I am incompetent. Do not hire me! >>>> >>>> Question: >>>> >>>> WHO AND MOST IMPORTNTANLY HOW DID HE MANAGED TO UPLOAD THIS FILE ON MY >>>> ACCOUNT? >>>> >>>> PLEASE ANSWER ME, I WONT GET MAD, BUT THIS IS AN IMPORTANT SECURITY RISK. >>>> >>>> SOMEONES MUST HAVE ACCESS TO MY ACCOUNT, DOES THE SOURCE CODE OF MY MAIN >>>> PYTHON SCRIPT APPEARS SOMEPLACE AGAIN?!?! >>> >>> >>> Would you please stop posting, I've almost burst my stomach laughing at >>> this. You definetely have a ready made career writing comedy. >> >> >> Okey smartass, >> >> Try to do it again, if you be successfull again i'll even congratulate you >> myself. >> >> -- >> https://mail.python.org/mailman/listinfo/python-list > > It looks like you are accusing someone of doing something without any > proof whatsoever. Would you like help with the fallout of the lawsuit > that I hope Mark might (should!) come up with?i'am > > Speaking of ?try again?, I doubt it would be hard? As long as a FTP > daemon is running somewhere (and you clearly do not know better); or > even you have a SSH daemon and you do not know better, an attacker > can: > > a) wait for you to publish your password yet again; > b) get you to download an exploit/keylogger/whatever; > c) brute-force. > > Well, considering it?s unlikely you actually have a long-as-shit > password, (c) is the best option. Unless your password is very long, > in which case is not. > > I?m also wondering what language your password is in. If you actually > used a Greek phrase, how long will it take you to get locked out due > to encoding bullshit? Like i use grek letter for my passwords or like i'am gonna fall for any of your 3 dumbass reasons. I already foudn the weakness and corrected it. From kwpolska at gmail.com Tue Oct 1 09:56:31 2013 From: kwpolska at gmail.com (=?UTF-8?B?Q2hyaXMg4oCcS3dwb2xza2HigJ0gV2Fycmljaw==?=) Date: Tue, 1 Oct 2013 15:56:31 +0200 Subject: Fwd: JUST GOT HACKED In-Reply-To: References: Message-ID: Why is this list not setting Reply-To correctly again? ---------- Forwarded message ---------- From: Chris ?Kwpolska? Warrick Date: Tue, Oct 1, 2013 at 3:55 PM Subject: Re: JUST GOT HACKED To: ????? On Tue, Oct 1, 2013 at 3:42 PM, ????? wrote: > ???? 1/10/2013 4:27 ??, ?/? Chris ?Kwpolska? Warrick ??????: >> >> On Tue, Oct 1, 2013 at 3:15 PM, ????? wrote: >>> >>> ???? 1/10/2013 4:06 ??, ?/? Mark Lawrence ??????: >>>> >>>> >>>> On 01/10/2013 10:58, ????? wrote: >>>>> >>>>> >>>>> Just logged in via FTP to my server and i saw an uploade file named >>>>> "Warnign html" >>>>> >>>>> Contents were: >>>>> >>>>> WARNING >>>>> >>>>> I am incompetent. Do not hire me! >>>>> >>>>> Question: >>>>> >>>>> WHO AND MOST IMPORTNTANLY HOW DID HE MANAGED TO UPLOAD THIS FILE ON MY >>>>> ACCOUNT? >>>>> >>>>> PLEASE ANSWER ME, I WONT GET MAD, BUT THIS IS AN IMPORTANT SECURITY >>>>> RISK. >>>>> >>>>> SOMEONES MUST HAVE ACCESS TO MY ACCOUNT, DOES THE SOURCE CODE OF MY >>>>> MAIN >>>>> PYTHON SCRIPT APPEARS SOMEPLACE AGAIN?!?! >>>> >>>> >>>> >>>> Would you please stop posting, I've almost burst my stomach laughing at >>>> this. You definetely have a ready made career writing comedy. >>> >>> >>> >>> Okey smartass, >>> >>> Try to do it again, if you be successfull again i'll even congratulate >>> you >>> myself. >>> >>> -- >>> https://mail.python.org/mailman/listinfo/python-list >> >> >> It looks like you are accusing someone of doing something without any >> proof whatsoever. Would you like help with the fallout of the lawsuit >> that I hope Mark might (should!) come up with?i'am >> >> >> Speaking of ?try again?, I doubt it would be hard? As long as a FTP >> daemon is running somewhere (and you clearly do not know better); or >> even you have a SSH daemon and you do not know better, an attacker >> can: >> >> a) wait for you to publish your password yet again; >> b) get you to download an exploit/keylogger/whatever; >> c) brute-force. >> >> Well, considering it?s unlikely you actually have a long-as-shit >> password, (c) is the best option. Unless your password is very long, >> in which case is not. >> >> I?m also wondering what language your password is in. If you actually >> used a Greek phrase, how long will it take you to get locked out due >> to encoding bullshit? > > > Like i use grek letter for my passwords Did you know that you just lowered the amount of characters an attacker should check while brute-forcing your password from 256/164 (UTF-*/ISO-8859-7) to just 95? No? Congratulations anyways, Nikos! -- Chris ?Kwpolska? Warrick PGP: 5EAAEA16 stop html mail | always bottom-post | only UTF-8 makes sense -- Chris ?Kwpolska? Warrick PGP: 5EAAEA16 stop html mail | always bottom-post | only UTF-8 makes sense From nikos.gr33k at gmail.com Tue Oct 1 09:58:15 2013 From: nikos.gr33k at gmail.com (=?UTF-8?B?zp3Or866zr/Pgg==?=) Date: Tue, 01 Oct 2013 16:58:15 +0300 Subject: Fwd: JUST GOT HACKED In-Reply-To: References: Message-ID: ???? 1/10/2013 4:56 ??, ?/? Chris ?Kwpolska? Warrick ??????: > Why is this list not setting Reply-To correctly again? > > ---------- Forwarded message ---------- > From: Chris ?Kwpolska? Warrick > Date: Tue, Oct 1, 2013 at 3:55 PM > Subject: Re: JUST GOT HACKED > To: ????? > > > On Tue, Oct 1, 2013 at 3:42 PM, ????? wrote: >> ???? 1/10/2013 4:27 ??, ?/? Chris ?Kwpolska? Warrick ??????: >>> >>> On Tue, Oct 1, 2013 at 3:15 PM, ????? wrote: >>>> >>>> ???? 1/10/2013 4:06 ??, ?/? Mark Lawrence ??????: >>>>> >>>>> >>>>> On 01/10/2013 10:58, ????? wrote: >>>>>> >>>>>> >>>>>> Just logged in via FTP to my server and i saw an uploade file named >>>>>> "Warnign html" >>>>>> >>>>>> Contents were: >>>>>> >>>>>> WARNING >>>>>> >>>>>> I am incompetent. Do not hire me! >>>>>> >>>>>> Question: >>>>>> >>>>>> WHO AND MOST IMPORTNTANLY HOW DID HE MANAGED TO UPLOAD THIS FILE ON MY >>>>>> ACCOUNT? >>>>>> >>>>>> PLEASE ANSWER ME, I WONT GET MAD, BUT THIS IS AN IMPORTANT SECURITY >>>>>> RISK. >>>>>> >>>>>> SOMEONES MUST HAVE ACCESS TO MY ACCOUNT, DOES THE SOURCE CODE OF MY >>>>>> MAIN >>>>>> PYTHON SCRIPT APPEARS SOMEPLACE AGAIN?!?! >>>>> >>>>> >>>>> >>>>> Would you please stop posting, I've almost burst my stomach laughing at >>>>> this. You definetely have a ready made career writing comedy. >>>> >>>> >>>> >>>> Okey smartass, >>>> >>>> Try to do it again, if you be successfull again i'll even congratulate >>>> you >>>> myself. >>>> >>>> -- >>>> https://mail.python.org/mailman/listinfo/python-list >>> >>> >>> It looks like you are accusing someone of doing something without any >>> proof whatsoever. Would you like help with the fallout of the lawsuit >>> that I hope Mark might (should!) come up with?i'am >>> >>> >>> Speaking of ?try again?, I doubt it would be hard? As long as a FTP >>> daemon is running somewhere (and you clearly do not know better); or >>> even you have a SSH daemon and you do not know better, an attacker >>> can: >>> >>> a) wait for you to publish your password yet again; >>> b) get you to download an exploit/keylogger/whatever; >>> c) brute-force. >>> >>> Well, considering it?s unlikely you actually have a long-as-shit >>> password, (c) is the best option. Unless your password is very long, >>> in which case is not. >>> >>> I?m also wondering what language your password is in. If you actually >>> used a Greek phrase, how long will it take you to get locked out due >>> to encoding bullshit? >> >> >> Like i use grek letter for my passwords > > Did you know that you just lowered the amount of characters an > attacker should check while brute-forcing your password from 256/164 > (UTF-*/ISO-8859-7) to just 95? No? Congratulations anyways, Nikos! Yes' iam aware of that, iam helping you as you see. Brute force then, after a few fail attempts you will be fobribben to even try a a new connection. From alister.ware at ntlworld.com Tue Oct 1 09:57:44 2013 From: alister.ware at ntlworld.com (Alister) Date: Tue, 01 Oct 2013 13:57:44 GMT Subject: JUST GOT HACKED References: Message-ID: On Tue, 01 Oct 2013 16:42:31 +0300, ????? wrote: > ???? 1/10/2013 4:27 ??, ?/? Chris ?Kwpolska? Warrick ??????: >> On Tue, Oct 1, 2013 at 3:15 PM, ????? wrote: >>> ???? 1/10/2013 4:06 ??, ?/? Mark Lawrence ??????: >>>> >>>> On 01/10/2013 10:58, ????? wrote: >>>>> >>>>> Just logged in via FTP to my server and i saw an uploade file named >>>>> "Warnign html" >>>>> >>>>> Contents were: >>>>> >>>>> WARNING >>>>> >>>>> I am incompetent. Do not hire me! >>>>> >>>>> Question: >>>>> >>>>> WHO AND MOST IMPORTNTANLY HOW DID HE MANAGED TO UPLOAD THIS FILE ON >>>>> MY ACCOUNT? >>>>> >>>>> PLEASE ANSWER ME, I WONT GET MAD, BUT THIS IS AN IMPORTANT SECURITY >>>>> RISK. >>>>> >>>>> SOMEONES MUST HAVE ACCESS TO MY ACCOUNT, DOES THE SOURCE CODE OF MY >>>>> MAIN PYTHON SCRIPT APPEARS SOMEPLACE AGAIN?!?! >>>> >>>> >>>> Would you please stop posting, I've almost burst my stomach laughing >>>> at this. You definetely have a ready made career writing comedy. >>> >>> >>> Okey smartass, >>> >>> Try to do it again, if you be successfull again i'll even congratulate >>> you myself. >>> >>> -- >>> https://mail.python.org/mailman/listinfo/python-list >> >> It looks like you are accusing someone of doing something without any >> proof whatsoever. Would you like help with the fallout of the lawsuit >> that I hope Mark might (should!) come up with?i'am >> >> Speaking of ?try again?, I doubt it would be hard? As long as a FTP >> daemon is running somewhere (and you clearly do not know better); or >> even you have a SSH daemon and you do not know better, an attacker can: >> >> a) wait for you to publish your password yet again; >> b) get you to download an exploit/keylogger/whatever; >> c) brute-force. >> >> Well, considering it?s unlikely you actually have a long-as-shit >> password, (c) is the best option. Unless your password is very long, >> in which case is not. >> >> I?m also wondering what language your password is in. If you actually >> used a Greek phrase, how long will it take you to get locked out due to >> encoding bullshit? > > Like i use grek letter for my passwords or like i'am gonna fall for any > of your 3 dumbass reasons. > > I already foudn the weakness and corrected it. i hope whoever is taking on your roll has a better basic understating of programming & systems administration. good luck with you new career -- This place just isn't big enough for all of us. We've got to find a way off this planet. From nikos.gr33k at gmail.com Tue Oct 1 10:00:02 2013 From: nikos.gr33k at gmail.com (=?UTF-8?B?zp3Or866zr/Pgg==?=) Date: Tue, 01 Oct 2013 17:00:02 +0300 Subject: JUST GOT HACKED In-Reply-To: References: Message-ID: ???? 1/10/2013 4:57 ??, ?/? Alister ??????: > On Tue, 01 Oct 2013 16:42:31 +0300, ????? wrote: > >> ???? 1/10/2013 4:27 ??, ?/? Chris ?Kwpolska? Warrick ??????: >>> On Tue, Oct 1, 2013 at 3:15 PM, ????? wrote: >>>> ???? 1/10/2013 4:06 ??, ?/? Mark Lawrence ??????: >>>>> >>>>> On 01/10/2013 10:58, ????? wrote: >>>>>> >>>>>> Just logged in via FTP to my server and i saw an uploade file named >>>>>> "Warnign html" >>>>>> >>>>>> Contents were: >>>>>> >>>>>> WARNING >>>>>> >>>>>> I am incompetent. Do not hire me! >>>>>> >>>>>> Question: >>>>>> >>>>>> WHO AND MOST IMPORTNTANLY HOW DID HE MANAGED TO UPLOAD THIS FILE ON >>>>>> MY ACCOUNT? >>>>>> >>>>>> PLEASE ANSWER ME, I WONT GET MAD, BUT THIS IS AN IMPORTANT SECURITY >>>>>> RISK. >>>>>> >>>>>> SOMEONES MUST HAVE ACCESS TO MY ACCOUNT, DOES THE SOURCE CODE OF MY >>>>>> MAIN PYTHON SCRIPT APPEARS SOMEPLACE AGAIN?!?! >>>>> >>>>> >>>>> Would you please stop posting, I've almost burst my stomach laughing >>>>> at this. You definetely have a ready made career writing comedy. >>>> >>>> >>>> Okey smartass, >>>> >>>> Try to do it again, if you be successfull again i'll even congratulate >>>> you myself. >>>> >>>> -- >>>> https://mail.python.org/mailman/listinfo/python-list >>> >>> It looks like you are accusing someone of doing something without any >>> proof whatsoever. Would you like help with the fallout of the lawsuit >>> that I hope Mark might (should!) come up with?i'am >>> >>> Speaking of ?try again?, I doubt it would be hard? As long as a FTP >>> daemon is running somewhere (and you clearly do not know better); or >>> even you have a SSH daemon and you do not know better, an attacker can: >>> >>> a) wait for you to publish your password yet again; >>> b) get you to download an exploit/keylogger/whatever; >>> c) brute-force. >>> >>> Well, considering it?s unlikely you actually have a long-as-shit >>> password, (c) is the best option. Unless your password is very long, >>> in which case is not. >>> >>> I?m also wondering what language your password is in. If you actually >>> used a Greek phrase, how long will it take you to get locked out due to >>> encoding bullshit? >> >> Like i use grek letter for my passwords or like i'am gonna fall for any >> of your 3 dumbass reasons. >> >> I already foudn the weakness and corrected it. > > > i hope whoever is taking on your roll has a better basic understating of > programming & systems administration. > > good luck with you new career Carred remaisn and it will remain the same. Thanks for visting my website: you help me increase my google page rank without actually utilizing SEO. Here: http://superhost.gr/?show=log&page=index.html From daniel.stjnv at gmail.com Tue Oct 1 10:24:35 2013 From: daniel.stjnv at gmail.com (Daniel Stojanov) Date: Wed, 2 Oct 2013 00:24:35 +1000 Subject: JUST GOT HACKED In-Reply-To: References: Message-ID: On 02/10/2013 12:05 AM, "?????" wrote: > Thanks for visting my website: you help me increase my google page rank without actually utilizing SEO. > > -- > https://mail.python.org/mailman/listinfo/python-list 1) You need links, not page views to improve your Google rank. 2) I just signed up the this mailing list. To the regulars, is this what normally happens on this list? 3) I'm a bit late to the party. Is Nikos a real sysadmin or is this some horrible inside joke I don't get? -------------- next part -------------- An HTML attachment was scrubbed... URL: From python.list at tim.thechases.com Tue Oct 1 10:56:13 2013 From: python.list at tim.thechases.com (Tim Chase) Date: Tue, 1 Oct 2013 09:56:13 -0500 Subject: JUST GOT HACKED In-Reply-To: References: Message-ID: <20131001095613.1882d506@bigbox.christie.dr> Daniel, I'm sorry your initial interactions with the list were tainted by this experience. Modulo these degenerative threads (usually started by Nikos), it *really* is a helpful and friendly place. On 2013-10-02 00:24, Daniel Stojanov wrote: > 2) I just signed up the this mailing list. To the regulars, is this > what normally happens on this list? There's "all the other very helpful, very on-topic traffic" and there's the "Nikos flailing around trying to get other people to write his code and solve his problems for him without actually taking the time to understand the actual problem/solution, and demanding that helpful/working solutions be contorted to fit his perspective of what the solution *should* look like" threads. When things boil up sufficiently, I tend to just use my mail/usenet client's "kill-thread" feature to auto-block threads where the FROM header contains Nikos's gmail or "superhost.gr", and suddenly the list reverts mostly to the "very helpful, very on-topic traffic". > 3) I'm a bit late to the party. Is Nikos a real sysadmin or is this > some horrible inside joke I don't get? You don't leave much wiggle-room there. I'm afraid he is a "real" "sysadmin" (for some definition of "real" and "sysadmin") or at least he has development/deployment access on a shared-hosting system where he alleges to have suckersactual clients depending on his "services". I wish it was a horrible joke (or maybe it is). The site "hacking" referred to in this thread appears to be the result of his repeated antagonization of the list through his intentional disregard for advice given; also a result of his failure to heed instructions on securing the site--especially with regards to publishing passwords on mailing-lists (or the URLs to the code containing those plain-text credentials). Again, I'm sorry this is how you meet the list. -tkc From ned at nedbatchelder.com Tue Oct 1 10:52:43 2013 From: ned at nedbatchelder.com (Ned Batchelder) Date: Tue, 01 Oct 2013 10:52:43 -0400 Subject: JUST GOT HACKED In-Reply-To: References: Message-ID: <524AE1BB.8090505@nedbatchelder.com> On 10/1/13 10:24 AM, Daniel Stojanov wrote: > > 2) I just signed up the this mailing list. To the regulars, is this > what normally happens on this list? > This is not what normally happens here. Usually we have concise and helpful conversations. Unfortunately, every online community has to struggle with the occasional troublemaker. We are currently struggling with that. The best approach is to simply ignore people that you either can't help or don't care to help. > 3) I'm a bit late to the party. Is Nikos a real sysadmin or is this > some horrible inside joke I don't get? > Please don't contribute to the problem by discussing Nikos. Thanks, --Ned. From nikos.gr33k at gmail.com Tue Oct 1 11:34:16 2013 From: nikos.gr33k at gmail.com (=?UTF-8?B?zp3Or866zr/Pgg==?=) Date: Tue, 01 Oct 2013 18:34:16 +0300 Subject: JUST GOT HACKED In-Reply-To: References: Message-ID: ???? 1/10/2013 5:52 ??, ?/? Ned Batchelder ??????: > On 10/1/13 10:24 AM, Daniel Stojanov wrote: >> >> 2) I just signed up the this mailing list. To the regulars, is this >> what normally happens on this list? >> > > This is not what normally happens here. Usually we have concise and > helpful conversations. > > Unfortunately, every online community has to struggle with the > occasional troublemaker. We are currently struggling with that. The > best approach is to simply ignore people that you either can't help or > don't care to help. > >> 3) I'm a bit late to the party. Is Nikos a real sysadmin or is this >> some horrible inside joke I don't get? >> > > Please don't contribute to the problem by discussing Nikos. > > Thanks, > > --Ned. Excuse me...but i;am no troublemaker, i ask question and read the answers and comment on those. And also i was the one being hacked here, not ther other way around, i did not started this. From wuwei23 at gmail.com Tue Oct 1 19:28:26 2013 From: wuwei23 at gmail.com (alex23) Date: Wed, 02 Oct 2013 09:28:26 +1000 Subject: JUST GOT HACKED In-Reply-To: References: Message-ID: On 2/10/2013 1:34 AM, ????? wrote: > i ask question and read the answers and comment on those. Citation needed. From ben+python at benfinney.id.au Tue Oct 1 18:06:38 2013 From: ben+python at benfinney.id.au (Ben Finney) Date: Wed, 02 Oct 2013 08:06:38 +1000 Subject: JUST GOT HACKED References: Message-ID: <7wioxgsl1t.fsf@benfinney.id.au> Daniel Stojanov writes: > 2) I just signed up the this mailing list. Welcome! Feel free to engage and/or begin some Python-related threads :-) > To the regulars, is this what normally happens on this list? This is an unmoderated forum, so we have occasional spates of persistent nuisances, and those who respond with the maturity level and impulse control of an average six-year-old. We have guidelines on community conduct, but they are enforced through social pressure and hence are not perfect. > 3) I'm a bit late to the party. Is Nikos a real sysadmin or is this some > horrible inside joke I don't get? Nikos has turned out to be a particularly persistent nuisance here, but is relatively easy to avoid once you realise that. -- \ ?An idea isn't responsible for the people who believe in it.? | `\ ?Donald Robert Perry Marquis | _o__) | Ben Finney From antoon.pardon at rece.vub.ac.be Wed Oct 2 02:29:07 2013 From: antoon.pardon at rece.vub.ac.be (Antoon Pardon) Date: Wed, 02 Oct 2013 08:29:07 +0200 Subject: JUST GOT HACKED In-Reply-To: <7wioxgsl1t.fsf@benfinney.id.au> References: <7wioxgsl1t.fsf@benfinney.id.au> Message-ID: <524BBD33.1060601@rece.vub.ac.be> Op 02-10-13 00:06, Ben Finney schreef: > Daniel Stojanov writes: > >> 2) I just signed up the this mailing list. > > Welcome! Feel free to engage and/or begin some Python-related threads > :-) > >> To the regulars, is this what normally happens on this list? > > This is an unmoderated forum, so we have occasional spates of persistent > nuisances, and those who respond with the maturity level and impulse > control of an average six-year-old. I think this is an unfair characterisation. People venting their frustration after the umpteenth thread started by Nikos in which he rejects the answers given to him and him repeatedly coming back with the same question, that is not the impulse control of a six year old. And what about the impuls control and the maturity of people who can't stop answering Nikos, knowing they contribute to the nuisance to the group? > We have guidelines on community conduct, but they are enforced through > social pressure and hence are not perfect. That social pressure is enforced rather one-side. People can endlessly contribute to the nuisance by keeping engaging with Nikos no matter how many times his nuisance his exposed and you will here very few complains. But then others start venting their frustration and suddenly the social pressure is put on. -- Antoon Pardon From ben+python at benfinney.id.au Wed Oct 2 02:49:22 2013 From: ben+python at benfinney.id.au (Ben Finney) Date: Wed, 02 Oct 2013 16:49:22 +1000 Subject: JUST GOT HACKED References: <7wioxgsl1t.fsf@benfinney.id.au> <524BBD33.1060601@rece.vub.ac.be> Message-ID: <7w1u44rwul.fsf@benfinney.id.au> Antoon Pardon writes: > Op 02-10-13 00:06, Ben Finney schreef: > > This is an unmoderated forum, so we have occasional spates of > > persistent nuisances, and those who respond with the maturity level > > and impulse control of an average six-year-old. [?] > > And what about the impuls control and the maturity of people who can't > stop answering [a nuisance], knowing they contribute to the nuisance > to the group? Yes, we are in firm agreement here. -- \ ?It is far better to grasp the universe as it really is than to | `\ persist in delusion, however satisfying and reassuring.? ?Carl | _o__) Sagan | Ben Finney From ganeshsahni07 at gmail.com Wed Oct 2 03:02:57 2013 From: ganeshsahni07 at gmail.com (Ravi Sahni) Date: Wed, 2 Oct 2013 12:32:57 +0530 Subject: JUST GOT HACKED In-Reply-To: <7w1u44rwul.fsf@benfinney.id.au> References: <7wioxgsl1t.fsf@benfinney.id.au> <524BBD33.1060601@rece.vub.ac.be> <7w1u44rwul.fsf@benfinney.id.au> Message-ID: On Wed, Oct 2, 2013 at 12:19 PM, Ben Finney wrote: > Antoon Pardon writes: > >> Op 02-10-13 00:06, Ben Finney schreef: >> > This is an unmoderated forum, so we have occasional spates of >> > persistent nuisances, and those who respond with the maturity level >> > and impulse control of an average six-year-old. > [?] >> >> And what about the impuls control and the maturity of people who can't >> stop answering [a nuisance], knowing they contribute to the nuisance >> to the group? > > Yes, we are in firm agreement here. So Ben,Antoon you are saying that Nikos is a minor problem -- spam-like -- Whereas people answering him are a bigger problem??! I find this real confused!! Why they are answering then?!?! As far as I can make out everyone who is answering (helping!) doing it frustratation and disgust. But still they keep answering and answering!! Makes no sense [Sorry -- old programmer (C,C++ etc) -- new to python. If there is some secret to this list's culture that I missed will be pleased to be educated! ] -- - Ravi From ben+python at benfinney.id.au Wed Oct 2 03:24:33 2013 From: ben+python at benfinney.id.au (Ben Finney) Date: Wed, 02 Oct 2013 17:24:33 +1000 Subject: JUST GOT HACKED References: <7wioxgsl1t.fsf@benfinney.id.au> <524BBD33.1060601@rece.vub.ac.be> <7w1u44rwul.fsf@benfinney.id.au> Message-ID: <7wwqlwqgni.fsf@benfinney.id.au> Ravi Sahni writes: > So Ben,Antoon you are saying that [demands for off-topic help with > demonstrated history of unwillingness to learn] is a minor problem [?] > Whereas [baiting and enabling that behaviour is] a bigger problem??! (I edited the above to focus on behaviour, not people. Let's not vilify a person when what is objectionable is the behaviour.) No, I'm not saying that. I'm saying that both those behaviours are significant nuisances, both are against our community guidelines of mutual respect, and both should stop. Comparing the magnitude of those problems to see which is worse isn't of interest to me, they're both objectionable to the point of noise and disruption. I'd like them both to stop, in the interest of keeping this forum functional for its intended purposes. -- \ ?If you make people think they're thinking, they'll love you; | `\ but if you really make them think, they'll hate you.? ?Donald | _o__) Robert Perry Marquis | Ben Finney From ganeshsahni07 at gmail.com Wed Oct 2 03:37:49 2013 From: ganeshsahni07 at gmail.com (Ravi Sahni) Date: Wed, 2 Oct 2013 13:07:49 +0530 Subject: JUST GOT HACKED In-Reply-To: <7wwqlwqgni.fsf@benfinney.id.au> References: <7wioxgsl1t.fsf@benfinney.id.au> <524BBD33.1060601@rece.vub.ac.be> <7w1u44rwul.fsf@benfinney.id.au> <7wwqlwqgni.fsf@benfinney.id.au> Message-ID: On Wed, Oct 2, 2013 at 12:54 PM, Ben Finney wrote: > > Ravi Sahni writes: > > > So Ben,Antoon you are saying that [demands for off-topic help with > > demonstrated history of unwillingness to learn] is a minor problem [?] > > Whereas [baiting and enabling that behaviour is] a bigger problem??! > > (I edited the above to focus on behaviour, not people. Let's not vilify > a person when what is objectionable is the behaviour.) Good... Sorry > > No, I'm not saying that. I'm saying that both those behaviours are > significant nuisances, both are against our community guidelines of > mutual respect, and both should stop. > > Comparing the magnitude of those problems to see which is worse isn't of > interest to me, they're both objectionable to the point of noise and > disruption. > > I'd like them both to stop, in the interest of keeping this forum > functional for its intended purposes. > Thanks Ben for clarification and understanding -- - Ravi From antoon.pardon at rece.vub.ac.be Wed Oct 2 03:51:26 2013 From: antoon.pardon at rece.vub.ac.be (Antoon Pardon) Date: Wed, 02 Oct 2013 09:51:26 +0200 Subject: JUST GOT HACKED In-Reply-To: References: <7wioxgsl1t.fsf@benfinney.id.au> <524BBD33.1060601@rece.vub.ac.be> <7w1u44rwul.fsf@benfinney.id.au> Message-ID: <524BD07E.3060606@rece.vub.ac.be> Op 02-10-13 09:02, Ravi Sahni schreef: > On Wed, Oct 2, 2013 at 12:19 PM, Ben Finney wrote: >> Antoon Pardon writes: >> >>> Op 02-10-13 00:06, Ben Finney schreef: >>>> This is an unmoderated forum, so we have occasional spates of >>>> persistent nuisances, and those who respond with the maturity level >>>> and impulse control of an average six-year-old. >> [?] >>> >>> And what about the impuls control and the maturity of people who can't >>> stop answering [a nuisance], knowing they contribute to the nuisance >>> to the group? >> >> Yes, we are in firm agreement here. > > So Ben,Antoon you are saying that Nikos is a minor problem -- spam-like -- > Whereas people answering him are a bigger problem??! > > I find this real confused!! Why they are answering then?!?! > As far as I can make out everyone who is answering (helping!) doing it > frustratation and disgust. But still they keep answering and > answering!! You should understand that what is a bigger problem and what is a minor problem is a personal, subjective judgement and people come to different conclusions. So group1 finds Nikos a minor nuisance and is willing to answer him. Probably because it gives them warm fuzzy feelings knowing they tried to help someone or because they found the problem interresting to solve. Now group2 may find Nikos himself not that big a nuisance but they certainly find Nikos in combination with group1 a major nuisance. Because it keeps the cycle going and even if they kill file Nikos, they keep being confronted with his contributions through the responses of group1. So frustration builds for those in group2, until it reaches a level that some of them feel the need to vent that frustration. That can sometimes be rather ugly to observe and I am sure that some venters weren't that happy with their own reaction afterwards, but I think it is an understandable, human reaction. Now for a number of people in group1, the venting of group2 is a major nuisance and they start venting their own frustration with that. Unfortunately, their own need for venting doesn't create any empathy for the need of group2 for venting. They only see groups2 as the cause for their own frustration with very little willingness to see their own contribution to the original built up. -- Antoon Pardon From ganeshsahni07 at gmail.com Wed Oct 2 09:17:46 2013 From: ganeshsahni07 at gmail.com (Ravi Sahni) Date: Wed, 2 Oct 2013 18:47:46 +0530 Subject: JUST GOT HACKED In-Reply-To: <524BD07E.3060606@rece.vub.ac.be> References: <7wioxgsl1t.fsf@benfinney.id.au> <524BBD33.1060601@rece.vub.ac.be> <7w1u44rwul.fsf@benfinney.id.au> <524BD07E.3060606@rece.vub.ac.be> Message-ID: On Wed, Oct 2, 2013 at 1:21 PM, Antoon Pardon wrote: > Op 02-10-13 09:02, Ravi Sahni schreef: >> On Wed, Oct 2, 2013 at 12:19 PM, Ben Finney wrote: >>> Antoon Pardon writes: >>> >>>> Op 02-10-13 00:06, Ben Finney schreef: >>>>> This is an unmoderated forum, so we have occasional spates of >>>>> persistent nuisances, and those who respond with the maturity level >>>>> and impulse control of an average six-year-old. >>> [?] >>>> >>>> And what about the impuls control and the maturity of people who can't >>>> stop answering [a nuisance], knowing they contribute to the nuisance >>>> to the group? >>> >>> Yes, we are in firm agreement here. >> >> So Ben,Antoon you are saying that Nikos is a minor problem -- spam-like -- >> Whereas people answering him are a bigger problem??! >> >> I find this real confused!! Why they are answering then?!?! >> As far as I can make out everyone who is answering (helping!) doing it >> frustratation and disgust. But still they keep answering and >> answering!! > > You should understand that what is a bigger problem and what is a minor > problem is a personal, subjective judgement and people come to different > conclusions. > > So group1 finds Nikos a minor nuisance and is willing to answer him. > Probably because it gives them warm fuzzy feelings knowing they tried > to help someone or because they found the problem interresting to solve. > > Now group2 may find Nikos himself not that big a nuisance but they > certainly find Nikos in combination with group1 a major nuisance. > Because it keeps the cycle going and even if they kill file Nikos, > they keep being confronted with his contributions through the responses > of group1. > > So frustration builds for those in group2, until it reaches a level > that some of them feel the need to vent that frustration. That can > sometimes be rather ugly to observe and I am sure that some venters > weren't that happy with their own reaction afterwards, but I think > it is an understandable, human reaction. > > Now for a number of people in group1, the venting of group2 is a > major nuisance and they start venting their own frustration with that. > Unfortunately, their own need for venting doesn't create any empathy > for the need of group2 for venting. They only see groups2 as the > cause for their own frustration with very little willingness to see > their own contribution to the original built up. Thanks Antoon for explaining so clearly and taking trouble to explain. As said above, Im newbie to python and to this group, (done C, C++ before) and was too confused by the BS to ask/speak. Daniel's post gave me courage to ask. Hope to get back to python now! -- Ravi From walterhurry at lavabit.com Wed Oct 2 17:13:16 2013 From: walterhurry at lavabit.com (Walter Hurry) Date: Wed, 2 Oct 2013 21:13:16 +0000 (UTC) Subject: Goodbye: was JUST GOT HACKED References: < sxA2u.17678$Jd4.7466@fx12.am4> < CAEvez=xfCV76bWR2UmMKokiBPOCrux6vEfQ8=WzSPXLjghE25A@mail.gmail.com> < 7wioxgsl1t.fsf@benfinney.id.au> <524BBD33.1060601@rece.vub.ac.be> < 7w1u44rwul.fsf@benfinney.id.au> < CAAbhUAyKFdod=Wm=d6mfKCTb0btWPne_j1KRR7Qz+nsRQUP4ng@mail.gmail.com> < mailman.598.1380700287.18130.python-list@python.org> Message-ID: On Wed, 02 Oct 2013 09:51:26 +0200, Antoon Pardon wrote: > Op 02-10-13 09:02, Ravi Sahni schreef: >> On Wed, Oct 2, 2013 at 12:19 PM, Ben Finney >> wrote: >>> Antoon Pardon writes: >>> >>>> Op 02-10-13 00:06, Ben Finney schreef: >>>>> This is an unmoderated forum, so we have occasional spates of >>>>> persistent nuisances, and those who respond with the maturity level >>>>> and impulse control of an average six-year-old. >>> [?] >>>> >>>> And what about the impuls control and the maturity of people who >>>> can't stop answering [a nuisance], knowing they contribute to the >>>> nuisance to the group? >>> >>> Yes, we are in firm agreement here. >> >> So Ben,Antoon you are saying that Nikos is a minor problem -- spam-like >> -- Whereas people answering him are a bigger problem??! >> >> I find this real confused!! Why they are answering then?!?! >> As far as I can make out everyone who is answering (helping!) doing it >> frustratation and disgust. But still they keep answering and >> answering!! > > You should understand that what is a bigger problem and what is a minor > problem is a personal, subjective judgement and people come to different > conclusions. > > So group1 finds Nikos a minor nuisance and is willing to answer him. > Probably because it gives them warm fuzzy feelings knowing they tried to > help someone or because they found the problem interresting to solve. > > Now group2 may find Nikos himself not that big a nuisance but they > certainly find Nikos in combination with group1 a major nuisance. > Because it keeps the cycle going and even if they kill file Nikos, > they keep being confronted with his contributions through the responses > of group1. > > So frustration builds for those in group2, until it reaches a level that > some of them feel the need to vent that frustration. That can sometimes > be rather ugly to observe and I am sure that some venters weren't that > happy with their own reaction afterwards, but I think it is an > understandable, human reaction. > > Now for a number of people in group1, the venting of group2 is a major > nuisance and they start venting their own frustration with that. > Unfortunately, their own need for venting doesn't create any empathy for > the need of group2 for venting. They only see groups2 as the cause for > their own frustration with very little willingness to see their own > contribution to the original built up. Ding ding! Nikos is simply trolling. It's easy enough to killfile him but inconvenient to skip all the answers to his lengthy threads. If only people would just ignore him! Anyway, the NG/list is spoiled for me. Goodbye. From tjreedy at udel.edu Wed Oct 2 19:05:19 2013 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 02 Oct 2013 19:05:19 -0400 Subject: Goodbye: was JUST GOT HACKED In-Reply-To: References: < sxA2u.17678$Jd4.7466@fx12.am4> < CAEvez=xfCV76bWR2UmMKokiBPOCrux6vEfQ8=WzSPXLjghE25A@mail.gmail.com> < 7wioxgsl1t.fsf@benfinney.id.au> <524BBD33.1060601@rece.vub.ac.be> < 7w1u44rwul.fsf@benfinney.id.au> < CAAbhUAyKFdod=Wm=d6mfKCTb0btWPne_j1KRR7Qz+nsRQUP4ng@mail.gmail.com> < mailman.598.1380700287.18130.python-list@python.org> Message-ID: <524CA6AF.2020205@udel.edu> On 10/2/2013 5:13 PM, Walter Hurry wrote: > Ding ding! Nikos is simply trolling. It's easy enough to killfile him but > inconvenient to skip all the answers to his lengthy threads. Use news.gmane.org newsgroup gmane.comp.python.general and current Thunderbird or other newsreader with 'Ignore thread' (right-click context menu) option. Even without that, a read should have the option to expand or condense threads. I just read what I want and mark all as read when done. > If only people would just ignore him! I view troll-feeding as a form of trolling. > Anyway, the NG/list is spoiled for me. Goodbye. I hope you try again someday. -- Terry Jan Reedy From ganeshsahni07 at gmail.com Wed Oct 2 23:51:08 2013 From: ganeshsahni07 at gmail.com (Ravi Sahni) Date: Thu, 3 Oct 2013 09:21:08 +0530 Subject: Goodbye: was JUST GOT HACKED In-Reply-To: References: <524BBD33.1060601@rece.vub.ac.be> Message-ID: On Thu, Oct 3, 2013 at 2:43 AM, Walter Hurry wrote: > Ding ding! Nikos is simply trolling. It's easy enough to killfile him but > inconvenient to skip all the answers to his lengthy threads. If only > people would just ignore him! Hello Walter Hurry please wait! Did I do/say something wrong?! If one of us should go it should be me -- Im just a newbie here. I have little time/efforts invested in python anyway. I was for a long time wasting time choosing upgrading myself from C/C++ to python or javascript. javascript -- universal and unavoidable in today's web world, but a mess python -- looks cleaner and well-designed (and not for heavyweight phds like FP languages like haskell ) So I finally went with python Now given the mess out here I need to rethink anyway! -- Ravi From steve+comp.lang.python at pearwood.info Thu Oct 3 07:35:00 2013 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: 03 Oct 2013 11:35:00 GMT Subject: Goodbye: was JUST GOT HACKED References: <524BBD33.1060601@rece.vub.ac.be> Message-ID: <524d5664$0$29984$c3e8da3$5496439d@news.astraweb.com> On Thu, 03 Oct 2013 09:21:08 +0530, Ravi Sahni wrote: > On Thu, Oct 3, 2013 at 2:43 AM, Walter Hurry > wrote: >> Ding ding! Nikos is simply trolling. It's easy enough to killfile him >> but inconvenient to skip all the answers to his lengthy threads. If >> only people would just ignore him! > > Hello Walter Hurry please wait! > > Did I do/say something wrong?! Don't worry about it Ravi, you haven't done anything wrong. Walter is not a regular here. At best he is a lurker who neither asks Python questions nor answers them. In the last four months, I can see four posts from him: three are complaining about Nikos, and one is a two- line "Me to!" response to a post about defensive programming. > If one of us should go it should be me -- Im just a newbie here. No, you are welcome here. You've posted more in just a few days than Walter has in months. We need more people like you. -- Steven From ganeshsahni07 at gmail.com Thu Oct 3 08:01:44 2013 From: ganeshsahni07 at gmail.com (Ravi Sahni) Date: Thu, 3 Oct 2013 17:31:44 +0530 Subject: Goodbye: was JUST GOT HACKED In-Reply-To: <524d5664$0$29984$c3e8da3$5496439d@news.astraweb.com> References: <524BBD33.1060601@rece.vub.ac.be> <524d5664$0$29984$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Thu, Oct 3, 2013 at 5:05 PM, Steven D'Aprano wrote: > On Thu, 03 Oct 2013 09:21:08 +0530, Ravi Sahni wrote: > >> On Thu, Oct 3, 2013 at 2:43 AM, Walter Hurry >> wrote: >>> Ding ding! Nikos is simply trolling. It's easy enough to killfile him >>> but inconvenient to skip all the answers to his lengthy threads. If >>> only people would just ignore him! >> >> Hello Walter Hurry please wait! >> >> Did I do/say something wrong?! > > Don't worry about it Ravi, you haven't done anything wrong. > > Walter is not a regular here. At best he is a lurker who neither asks > Python questions nor answers them. In the last four months, I can see > four posts from him: three are complaining about Nikos, and one is a two- > line "Me to!" response to a post about defensive programming. > > > >> If one of us should go it should be me -- Im just a newbie here. > > No, you are welcome here. You've posted more in just a few days than > Walter has in months. We need more people like you. Thanks for the welcome! But No thanks for the non-welcome -- I dont figure why Walter Hurry (or anyone else) should be unwelcome just because I am welcome. The world (and the python list hopefully!!) is big enough for all of us -- Ravi From steve+comp.lang.python at pearwood.info Thu Oct 3 22:03:07 2013 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: 04 Oct 2013 02:03:07 GMT Subject: Goodbye: was JUST GOT HACKED References: <524BBD33.1060601@rece.vub.ac.be> <524d5664$0$29984$c3e8da3$5496439d@news.astraweb.com> Message-ID: <524e21da$0$29984$c3e8da3$5496439d@news.astraweb.com> On Thu, 03 Oct 2013 17:31:44 +0530, Ravi Sahni wrote: > On Thu, Oct 3, 2013 at 5:05 PM, Steven D'Aprano > wrote: >> No, you are welcome here. You've posted more in just a few days than >> Walter has in months. We need more people like you. > > Thanks for the welcome! > > But No thanks for the non-welcome -- I dont figure why Walter Hurry (or > anyone else) should be unwelcome just because I am welcome. ???? Who said Walter was unwelcome? It's *his* choice to leave, nobody is kicking him out. Regards, -- Steven From walterhurry at lavabit.com Mon Oct 7 08:26:03 2013 From: walterhurry at lavabit.com (Walter Hurry) Date: Mon, 7 Oct 2013 12:26:03 +0000 (UTC) Subject: Goodbye: was JUST GOT HACKED References: < l2ejg6$8un$5@dont-email.me> <524BBD33.1060601@ rece.vub.ac.be> <524d5664$0$29984$c3e8da3$5496439d@news. astraweb.com> Message-ID: On Thu, 03 Oct 2013 11:35:00 +0000, Steven D'Aprano wrote: > On Thu, 03 Oct 2013 09:21:08 +0530, Ravi Sahni wrote: > >> On Thu, Oct 3, 2013 at 2:43 AM, Walter Hurry >> wrote: >>> Ding ding! Nikos is simply trolling. It's easy enough to killfile him >>> but inconvenient to skip all the answers to his lengthy threads. If >>> only people would just ignore him! >> >> Hello Walter Hurry please wait! >> >> Did I do/say something wrong?! > > Don't worry about it Ravi, you haven't done anything wrong. > > Walter is not a regular here. At best he is a lurker who neither asks > Python questions nor answers them. In the last four months, I can see > four posts from him: three are complaining about Nikos, and one is a > two- > line "Me to!" response to a post about defensive programming. > > > >> If one of us should go it should be me -- Im just a newbie here. > > No, you are welcome here. You've posted more in just a few days than > Walter has in months. We need more people like you. Steven, You make a fair point. I have posted very little recently, for the following reasons: a) I'm not really competent enough to answer python questions, at least not yet. b) I try not to post my own Python questions unless as a last resort. I prefer to try to solve my own problems by reading the fine documentation, and DuckDuckGoing. However, I do lurk assiduously and have learned much by reading excellent 'answering' posts from many such as you. The 'Goodbye' post was made in rather a fit of pique, for which I apologise. If I am allowed a second chance, there is actually something puzzling me at the moment. It's a UnicodeDecodeError, but I shall start a separate thread about it. Sorry again. Walter From rosuav at gmail.com Mon Oct 7 08:34:42 2013 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 7 Oct 2013 23:34:42 +1100 Subject: Goodbye: was JUST GOT HACKED In-Reply-To: References: Message-ID: On Mon, Oct 7, 2013 at 11:26 PM, Walter Hurry wrote: > The 'Goodbye' post was made in rather a fit of pique, for which I > apologise. If I am allowed a second chance, there is actually something > puzzling me at the moment. It's a UnicodeDecodeError, but I shall start > a separate thread about it. You're allowed a second chance, and a third, and a seventh, and a seventy-times-seventh (or a seventy-seventh, depending on which translation you read). Welcome back. And yes, starting a separate thread for the actual question is definitely the best way :) ChrisA From python.list at tim.thechases.com Mon Oct 7 09:12:39 2013 From: python.list at tim.thechases.com (Tim Chase) Date: Mon, 7 Oct 2013 08:12:39 -0500 Subject: Goodbye: was JUST GOT HACKED In-Reply-To: References: <524BBD33.1060601@ rece.vub.ac.be> <524d5664$0$29984$c3e8da3$5496439d@news. astraweb.com> Message-ID: <20131007081239.0f6bbc2c@bigbox.christie.dr> On 2013-10-07 12:26, Walter Hurry wrote: > The 'Goodbye' post was made in rather a fit of pique, for which I > apologise. If I am allowed a second chance, there is actually > something puzzling me at the moment. It's a UnicodeDecodeError, but > I shall start a separate thread about it. Indeed, the list eagerly welcomes folks who want to learn Python (not just have the answers spoon-fed to them **ehem**), so based on your "I prefer to try to solve my own problems by reading the fine documentation, and DuckDuckGoing," you're the ideal person we *want* here. You've already done the grunt work debugging/diagnosis, and leave the interesting problems for the mailing-list to have fun with. :-) So UnicodeDecodeErrors? Bring it on! :-) -tkc From ganeshsahni07 at gmail.com Mon Oct 7 09:10:43 2013 From: ganeshsahni07 at gmail.com (Ravi Sahni) Date: Mon, 7 Oct 2013 18:40:43 +0530 Subject: Goodbye: was JUST GOT HACKED In-Reply-To: References: Message-ID: On Mon, Oct 7, 2013 at 5:56 PM, Walter Hurry wrote: > On Thu, 03 Oct 2013 11:35:00 +0000, Steven D'Aprano wrote: > >> On Thu, 03 Oct 2013 09:21:08 +0530, Ravi Sahni wrote: >> >>> On Thu, Oct 3, 2013 at 2:43 AM, Walter Hurry >>> wrote: >>>> Ding ding! Nikos is simply trolling. It's easy enough to killfile him >>>> but inconvenient to skip all the answers to his lengthy threads. If >>>> only people would just ignore him! >>> >>> Hello Walter Hurry please wait! >>> >>> Did I do/say something wrong?! >> >> Don't worry about it Ravi, you haven't done anything wrong. >> >> Walter is not a regular here. At best he is a lurker who neither asks >> Python questions nor answers them. In the last four months, I can see >> four posts from him: three are complaining about Nikos, and one is a >> two- >> line "Me to!" response to a post about defensive programming. >> >> >> >>> If one of us should go it should be me -- Im just a newbie here. >> >> No, you are welcome here. You've posted more in just a few days than >> Walter has in months. We need more people like you. > > Steven, > > You make a fair point. I have posted very little recently, for the > following reasons: > > a) I'm not really competent enough to answer python questions, at least > not yet. > > b) I try not to post my own Python questions unless as a last resort. I > prefer to try to solve my own problems by reading the fine documentation, > and DuckDuckGoing. > > However, I do lurk assiduously and have learned much by reading excellent > 'answering' posts from many such as you. > > The 'Goodbye' post was made in rather a fit of pique, for which I > apologise. If I am allowed a second chance, there is actually something > puzzling me at the moment. It's a UnicodeDecodeError, but I shall start > a separate thread about it. > > Sorry again. Thanks! For changing decision! From steve at pearwood.info Wed Oct 2 05:08:01 2013 From: steve at pearwood.info (Steven D'Aprano) Date: 02 Oct 2013 09:08:01 GMT Subject: JUST GOT HACKED References: <7wioxgsl1t.fsf@benfinney.id.au> <524BBD33.1060601@rece.vub.ac.be> <7w1u44rwul.fsf@benfinney.id.au> Message-ID: <524be270$0$2865$c3e8da3$76491128@news.astraweb.com> On Wed, 02 Oct 2013 12:32:57 +0530, Ravi Sahni wrote: > I find this real confused!! Why they are answering then?!?! As far as I > can make out everyone who is answering (helping!) doing it frustratation > and disgust. But still they keep answering and answering!! > > Makes no sense If you want to ask why people are answering Nikos' questions, you should ask them directly. I won't speak for others, but I'll answer for myself: I answer Nikos' questions because: #1 He is a member of our community who needs help with Python, and this is a welcoming community, not an elitist one. #2 Some of his questions are interesting technical questions, like his Unicode problems. I have learnt things from answering his questions. I'm sure other people have learnt things from reading those answers. #3 Even his uninteresting questions deserve answers. Everybody here, I am sure, has asked boring or stupid or trivial questions at some stage, especially the newbies. Even when the answer is just "This answer is the same as last time you asked", the question deserves an answer. This is a matter of simple respect. We should treat others in the way we would hope to be treated if we were in their shoes. #4 Explicit is better than implicit. Even if nobody has an answer, or if it is off-topic, it is better to explicitly say that we have no answer and remove all doubt than to respond with nothing but silence and leave open the hope that if you just ask again more loudly someone will answer. #5 In the long run, encouraging good behaviour is more effective at changing people's behaviour than merely punishing bad behaviour. #6 Consider the first impression of a newbie, joining this community with questions about Python. The first thing they see is Nikos asking questions, and being ignored, or worse, being abused. Does that send the message that we want, that their questions are welcome? -- Steven From antoon.pardon at rece.vub.ac.be Wed Oct 2 07:28:11 2013 From: antoon.pardon at rece.vub.ac.be (Antoon Pardon) Date: Wed, 02 Oct 2013 13:28:11 +0200 Subject: JUST GOT HACKED In-Reply-To: <524be270$0$2865$c3e8da3$76491128@news.astraweb.com> References: <7wioxgsl1t.fsf@benfinney.id.au> <524BBD33.1060601@rece.vub.ac.be> <7w1u44rwul.fsf@benfinney.id.au> <524be270$0$2865$c3e8da3$76491128@news.astraweb.com> Message-ID: <524C034B.2090500@rece.vub.ac.be> Op 02-10-13 11:08, Steven D'Aprano schreef: > On Wed, 02 Oct 2013 12:32:57 +0530, Ravi Sahni wrote: > >> I find this real confused!! Why they are answering then?!?! As far as I >> can make out everyone who is answering (helping!) doing it frustratation >> and disgust. But still they keep answering and answering!! >> >> Makes no sense > > > If you want to ask why people are answering Nikos' questions, you should > ask them directly. I won't speak for others, but I'll answer for myself: > I answer Nikos' questions because: > > #1 He is a member of our community who needs help with Python, and this > is a welcoming community, not an elitist one. Come on Steve. You have kill filed people. So you don't think that merely being a member is enough. Or are you being elistist when you kill file someone? > #2 Some of his questions are interesting technical questions, like his > Unicode problems. I have learnt things from answering his questions. I'm > sure other people have learnt things from reading those answers. Sure, but not the fourth or fith time the same question comes up in the same thread. > #3 Even his uninteresting questions deserve answers. Everybody here, I am > sure, has asked boring or stupid or trivial questions at some stage, > especially the newbies. Even when the answer is just "This answer is the > same as last time you asked", the question deserves an answer. This is a > matter of simple respect. We should treat others in the way we would hope > to be treated if we were in their shoes. Steve with Nikos's MO this means you risk a thread with nothing but a cycle of Nikos repeating his question and others repeating the question was already answered in the thread. And you don't treat all others in the way you hope to be treated if you would be in their shoes. I suspect that should you one day feel so frustrated you need to vent, you will hope to get treated differently than how you treat those that need to vent now. You are very selective about the people in whose shoes you can imagine yourself. > #4 Explicit is better than implicit. Even if nobody has an answer, or if > it is off-topic, it is better to explicitly say that we have no answer > and remove all doubt than to respond with nothing but silence and leave > open the hope that if you just ask again more loudly someone will answer. > > #5 In the long run, encouraging good behaviour is more effective at > changing people's behaviour than merely punishing bad behaviour. But you don't encourage good behaviour by rewarding bad behaviour. Even when Nikos has ignored previous answers and has ignored advise to read up on a particular subject, you are still inclined to answer him. That is not encouraging good bahaviour, that is rewarding bad behaviour. All these reasons you have given above, don't differentiate between Nikos behaving good or badly while asking his questions. You giving a reason that depends on making such a differentiation doesn't make any sense after that. > #6 Consider the first impression of a newbie, joining this community with > questions about Python. The first thing they see is Nikos asking > questions, and being ignored, or worse, being abused. Does that send the > message that we want, that their questions are welcome? Newbie may be new to python, that doesn't mean they are new to social interaction. I think people can understand someone has been behaving badly enough so people won't engage with him anymore. This reason here implies, that no matter what a nuissance Nikos is, we should still answer his questions, which blantantly contradicts your "encourage good behaviour" reason above. There is no encouragement for good behaviour is you provide an answer anyway, whether the asker is showing good behaviour or not. -- Antoon Pardon From steve+comp.lang.python at pearwood.info Wed Oct 2 09:17:20 2013 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: 02 Oct 2013 13:17:20 GMT Subject: JUST GOT HACKED References: <7wioxgsl1t.fsf@benfinney.id.au> <524BBD33.1060601@rece.vub.ac.be> <7w1u44rwul.fsf@benfinney.id.au> <524be270$0$2865$c3e8da3$76491128@news.astraweb.com> Message-ID: <524c1ce0$0$29984$c3e8da3$5496439d@news.astraweb.com> On Wed, 02 Oct 2013 13:28:11 +0200, Antoon Pardon wrote: > Op 02-10-13 11:08, Steven D'Aprano schreef: >> On Wed, 02 Oct 2013 12:32:57 +0530, Ravi Sahni wrote: >> >>> I find this real confused!! Why they are answering then?!?! As far as >>> I can make out everyone who is answering (helping!) doing it >>> frustratation and disgust. But still they keep answering and >>> answering!! >>> >>> Makes no sense >> >> >> If you want to ask why people are answering Nikos' questions, you >> should ask them directly. I won't speak for others, but I'll answer for >> myself: I answer Nikos' questions because: >> >> #1 He is a member of our community who needs help with Python, and this >> is a welcoming community, not an elitist one. > > Come on Steve. You have kill filed people. So you don't think that > merely being a member is enough. Or are you being elistist when you kill > file someone? When I kill-file somebody, I tell them, and I always make it temporary. This is not a matter of elitism, it is a matter of responding to bad behaviour and sending a message that it is inappropriate -- if you behave badly, I will not see your messages. Think of it as "time out", or for sports fans, "the sin bin". I expect some people have probably kill-filed me, although (to the best of me knowledge) none of them have had the elementary decency to tell me. [...] > And you don't treat all others in the way you hope to be treated if you > would be in their shoes. I suspect that should you one day feel so > frustrated you need to vent, you will hope to get treated differently > than how you treat those that need to vent now. You are very selective > about the people in whose shoes you can imagine yourself. I am only an imperfect human being. I don't always live up to my ideals. Sometimes I behave poorly. I have a tendency to react to newbies' poor questions with sarcasm. Perhaps a little bit of sarcasm is okay, but there is a fine line between making a point and being unnecessarily nasty. If I cross that line, I hope that somebody will call me out on it. -- Steven From neilc at norwich.edu Wed Oct 2 12:05:00 2013 From: neilc at norwich.edu (Neil Cerutti) Date: 2 Oct 2013 16:05:00 GMT Subject: JUST GOT HACKED References: <7wioxgsl1t.fsf@benfinney.id.au> <524BBD33.1060601@rece.vub.ac.be> <7w1u44rwul.fsf@benfinney.id.au> <524be270$0$2865$c3e8da3$76491128@news.astraweb.com> <524c1ce0$0$29984$c3e8da3$5496439d@news.astraweb.com> Message-ID: On 2013-10-02, Steven D'Aprano wrote: > When I kill-file somebody, I tell them, and I always make it > temporary. This is not a matter of elitism, it is a matter of > responding to bad behaviour and sending a message that it is > inappropriate -- if you behave badly, I will not see your > messages. Think of it as "time out", or for sports fans, "the > sin bin". > > I expect some people have probably kill-filed me, although (to > the best of me knowledge) none of them have had the elementary > decency to tell me. I do think there is some value in telling someone why you might killfile them. But actual *plonks* are, I think, manifestation of spotlight syndrome. -- Neil Cerutti From antoon.pardon at rece.vub.ac.be Thu Oct 3 03:01:29 2013 From: antoon.pardon at rece.vub.ac.be (Antoon Pardon) Date: Thu, 03 Oct 2013 09:01:29 +0200 Subject: JUST GOT HACKED In-Reply-To: <524c1ce0$0$29984$c3e8da3$5496439d@news.astraweb.com> References: <7wioxgsl1t.fsf@benfinney.id.au> <524BBD33.1060601@rece.vub.ac.be> <7w1u44rwul.fsf@benfinney.id.au> <524be270$0$2865$c3e8da3$76491128@news.astraweb.com> <524c1ce0$0$29984$c3e8da3$5496439d@news.astraweb.com> Message-ID: <524D1649.8050905@rece.vub.ac.be> Op 02-10-13 15:17, Steven D'Aprano schreef: > [...] >> And you don't treat all others in the way you hope to be treated if you >> would be in their shoes. I suspect that should you one day feel so >> frustrated you need to vent, you will hope to get treated differently >> than how you treat those that need to vent now. You are very selective >> about the people in whose shoes you can imagine yourself. > > I am only an imperfect human being. I don't always live up to my ideals. > Sometimes I behave poorly. I have a tendency to react to newbies' poor > questions with sarcasm. Perhaps a little bit of sarcasm is okay, but > there is a fine line between making a point and being unnecessarily > nasty. If I cross that line, I hope that somebody will call me out on it. Steve it is not about failing. It is about not even trying. You don't follow the principle of treating others in the way you hope to be treated if you were in their shoes. You just mention the principle if you think it can support your behaviour. Your paragraph above is a nice illustration. Suppose you develop a new interest in which you are now the newbie and you go to a newsgroup or forum where as a nebie you ask a poor question. Are you hoping they will answer with sarcasm? I doubt that very much. Yet here you are telling us sarcasm is okay as long as you don't cross the line instead of you showing us that you would like to follow this priciple although you find it hard in a number of circumstances. -- Antoon Pardon From steve+comp.lang.python at pearwood.info Thu Oct 3 07:30:03 2013 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: 03 Oct 2013 11:30:03 GMT Subject: JUST GOT HACKED References: <7wioxgsl1t.fsf@benfinney.id.au> <524BBD33.1060601@rece.vub.ac.be> <7w1u44rwul.fsf@benfinney.id.au> <524be270$0$2865$c3e8da3$76491128@news.astraweb.com> <524c1ce0$0$29984$c3e8da3$5496439d@news.astraweb.com> Message-ID: <524d553a$0$29984$c3e8da3$5496439d@news.astraweb.com> On Thu, 03 Oct 2013 09:01:29 +0200, Antoon Pardon wrote: > You don't > follow the principle of treating others in the way you hope to be > treated if you were in their shoes. [...] > Suppose you develop a new > interest in which you are now the newbie and you go to a newsgroup or > forum where as a nebie you ask a poor question. Are you hoping they will > answer with sarcasm? I doubt that very much. Then you would be wrong. You don't know me very well at all. If I asked a dumb question -- not an ignorant question, but a dumb question -- then I hope somebody will rub my nose in it. Sarcasm strikes me as a good balance between being too namby-pamby to correct me for wasting everyone's time, and being abusive. An ignorant question would be: "I don't understand closures, can somebody help me?" or even: "I wrote this function: def f(arg=[]): arg.append(1); return arg and it behaves strangely. Is that a bug in Python?" This, on the other hand, is a dumb question: "I wrote a function to print prime numbers, and it didn't work. What did I do wrong?" In the last case, the question simply is foolish. Short of mind-reading, how is anyone supposed to know which of the infinite number of errors I made? In this case, I would *much* prefer a gentle, or even not-so- gentle, reminder of my foolishness via a sarcastic retort about looking in crystal balls or reading minds, than either being ignored or being abused. And quite frankly, although I might *prefer* a gentle request asking for more information, I might *need* something harsher for the lesson to really sink in. Negative reinforcement is a legitimate teaching tool, provided it doesn't cross the line into abuse. -- Steven From antoon.pardon at rece.vub.ac.be Fri Oct 4 09:48:25 2013 From: antoon.pardon at rece.vub.ac.be (Antoon Pardon) Date: Fri, 04 Oct 2013 15:48:25 +0200 Subject: JUST GOT HACKED In-Reply-To: <524d553a$0$29984$c3e8da3$5496439d@news.astraweb.com> References: <7wioxgsl1t.fsf@benfinney.id.au> <524BBD33.1060601@rece.vub.ac.be> <7w1u44rwul.fsf@benfinney.id.au> <524be270$0$2865$c3e8da3$76491128@news.astraweb.com> <524c1ce0$0$29984$c3e8da3$5496439d@news.astraweb.com> <524d553a$0$29984$c3e8da3$5496439d@news.astraweb.com> Message-ID: <524EC729.5090603@rece.vub.ac.be> Op 03-10-13 13:30, Steven D'Aprano schreef: > On Thu, 03 Oct 2013 09:01:29 +0200, Antoon Pardon wrote: > >> You don't >> follow the principle of treating others in the way you hope to be >> treated if you were in their shoes. > [...] >> Suppose you develop a new >> interest in which you are now the newbie and you go to a newsgroup or >> forum where as a nebie you ask a poor question. Are you hoping they will >> answer with sarcasm? I doubt that very much. > > Then you would be wrong. You don't know me very well at all. > > If I asked a dumb question -- not an ignorant question, but a dumb > question -- then I hope somebody will rub my nose in it. Sarcasm strikes > me as a good balance between being too namby-pamby to correct me for > wasting everyone's time, and being abusive. You are contradicting yourself with previous contributions. If you yourself want to be treated with sarcasm when you ask a poor question and you follow the priciple of trying to treat people the same way as you like to be treated should you be in their shoes. Then it doesn't make sense that you should judge yourself as not living up to your ideals because of having a tendency to react to newbies' poor questions with sarcasm, as you did earlier. That tendency would then make you live up to your ideals. I have the impression you haven't thought this through much and are just making it up as you go along. > And quite frankly, although I might *prefer* a gentle request asking for > more information, I might *need* something harsher for the lesson to > really sink in. Negative reinforcement is a legitimate teaching tool, > provided it doesn't cross the line into abuse. But need was orginally not a consideration. You certainly never seem to consider Nikos might need something harsher than a polite answer, labeling almost any criticism almost automaticcaly as hate. -- Antoon Pardon From invalid at invalid.invalid Wed Oct 2 09:34:49 2013 From: invalid at invalid.invalid (Grant Edwards) Date: Wed, 2 Oct 2013 13:34:49 +0000 (UTC) Subject: JUST GOT HACKED References: <7wioxgsl1t.fsf@benfinney.id.au> <524BBD33.1060601@rece.vub.ac.be> <7w1u44rwul.fsf@benfinney.id.au> Message-ID: On 2013-10-02, Ravi Sahni wrote: > On Wed, Oct 2, 2013 at 12:19 PM, Ben Finney wrote: >> Antoon Pardon writes: >>> Op 02-10-13 00:06, Ben Finney schreef: >>> And what about the impuls control and the maturity of people who can't >>> stop answering [a nuisance], knowing they contribute to the nuisance >>> to the group? >> >> Yes, we are in firm agreement here. > > So Ben,Antoon you are saying that Nikos is a minor problem -- > spam-like -- Whereas people answering him are a bigger problem??! I think that's true. If people stopped responding to Nikos (either positively or negatively), he'd stop posting and go away. > I find this real confused!! Why they are answering then?!?! Many of them are trying to help Nikos. Others are just poking him with a stick to watch him jump [a Usenet tradition since before the Internet itself]. -- Grant Edwards grant.b.edwards Yow! I am covered with at pure vegetable oil and I am gmail.com writing a best seller! From rurpy at yahoo.com Wed Oct 2 12:44:53 2013 From: rurpy at yahoo.com (rurpy at yahoo.com) Date: Wed, 2 Oct 2013 09:44:53 -0700 (PDT) Subject: JUST GOT HACKED In-Reply-To: References: <7wioxgsl1t.fsf@benfinney.id.au> <524BBD33.1060601@rece.vub.ac.be> <7w1u44rwul.fsf@benfinney.id.au> Message-ID: On 10/02/2013 01:02 AM, Ravi Sahni wrote: > On Wed, Oct 2, 2013 at 12:19 PM, Ben Finney wrote: >> Antoon Pardon writes: >> >>> Op 02-10-13 00:06, Ben Finney schreef: >>> > This is an unmoderated forum, so we have occasional spates of >>> > persistent nuisances, and those who respond with the maturity level >>> > and impulse control of an average six-year-old. >> [?] >>> >>> And what about the impuls control and the maturity of people who can't >>> stop answering [a nuisance], knowing they contribute to the nuisance >>> to the group? >> >> Yes, we are in firm agreement here. > > So Ben,Antoon you are saying that Nikos is a minor problem -- spam-like -- > Whereas people answering him are a bigger problem??! > > I find this real confused!! Why they are answering then?!?! > As far as I can make out everyone who is answering (helping!) doing it > frustratation and disgust. But still they keep answering and > answering!! > > Makes no sense > > [Sorry -- old programmer (C,C++ etc) -- new to python. If there is > some secret to this list's culture that I missed will be pleased to be > educated!] Actually it does make sense when one thinks of the psychology. It is fun to bash other people on the internet. There are few consequences and it makes up for the lack of authority and control we experience in our real daily lives. When someone like Nikos appears and irritates enough people to exceed a critical mass, it becomes socially ok to bash him and one gets a _Lord of the Flies_ [*1] effect. Further it is nothing new -- this kind of spiral down into chaos and noise of an unmoderated online community has been happening since the earliest days of the internet. For decades a useful way to combat this has been summarized in the phase "don't feed the trolls". But that only works when people are able sacrifice their own fun (giving up the joy of joining in publicly bashing a scapegoat by simply not responding to inflammatory posts) for a common good (a mailing list with a good signal-to-noise ratio and non-hostile atmosphere.) It would seem that enough Python regulars here get enjoyment from the current state of affairs that the situation is likely to last indefinitely. The rest of us try to make do by using restraint, filtering and alternate forums (Stackoverflow, etc). ---- [*1] http://www.cliffsnotes.com/literature/l/lord-of-the-flies/lord-of-the-flies-at-a-glance From ganeshsahni07 at gmail.com Wed Oct 2 14:21:44 2013 From: ganeshsahni07 at gmail.com (Ravi Sahni) Date: Wed, 2 Oct 2013 23:51:44 +0530 Subject: JUST GOT HACKED In-Reply-To: References: <7wioxgsl1t.fsf@benfinney.id.au> <524BBD33.1060601@rece.vub.ac.be> <7w1u44rwul.fsf@benfinney.id.au> Message-ID: On Wed, Oct 2, 2013 at 10:14 PM, wrote: > On 10/02/2013 01:02 AM, Ravi Sahni wrote: >> On Wed, Oct 2, 2013 at 12:19 PM, Ben Finney wrote: >>> Antoon Pardon writes: >>> >>>> Op 02-10-13 00:06, Ben Finney schreef: >>>> > This is an unmoderated forum, so we have occasional spates of >>>> > persistent nuisances, and those who respond with the maturity level >>>> > and impulse control of an average six-year-old. >>> [?] >>>> >>>> And what about the impuls control and the maturity of people who can't >>>> stop answering [a nuisance], knowing they contribute to the nuisance >>>> to the group? >>> >>> Yes, we are in firm agreement here. >> >> So Ben,Antoon you are saying that Nikos is a minor problem -- spam-like -- >> Whereas people answering him are a bigger problem??! >> >> I find this real confused!! Why they are answering then?!?! >> As far as I can make out everyone who is answering (helping!) doing it >> frustratation and disgust. But still they keep answering and >> answering!! >> >> Makes no sense >> >> [Sorry -- old programmer (C,C++ etc) -- new to python. If there is >> some secret to this list's culture that I missed will be pleased to be >> educated!] > > Actually it does make sense when one thinks of the psychology. > It is fun to bash other people on the internet. There are few > consequences and it makes up for the lack of authority and > control we experience in our real daily lives. > > When someone like Nikos appears and irritates enough people > to exceed a critical mass, it becomes socially ok to bash > him and one gets a _Lord of the Flies_ [*1] effect. > > Further it is nothing new -- this kind of spiral down > into chaos and noise of an unmoderated online community > has been happening since the earliest days of the internet. > > For decades a useful way to combat this has been summarized > in the phase "don't feed the trolls". But that only works > when people are able sacrifice their own fun (giving up the > joy of joining in publicly bashing a scapegoat by simply not > responding to inflammatory posts) for a common good (a mailing > list with a good signal-to-noise ratio and non-hostile atmosphere.) > > It would seem that enough Python regulars here get enjoyment > from the current state of affairs that the situation is likely > to last indefinitely. > > The rest of us try to make do by using restraint, filtering and > alternate forums (Stackoverflow, etc). > > ---- > [*1] http://www.cliffsnotes.com/literature/l/lord-of-the-flies/lord-of-the-flies-at-a-glance That (link) is an ugly stupid view of humanness. Why should I want to piss on you and flame you and shoot you for fun? I have never met anyone like that and dont believe that anyone is like that. [We are told about Hitler and Stalin and so on. I have never met them :-) ] And if you believe everyone is like that -- sorry - please go to psychatrist -- serious! Basically I am a software engineer. A engineer believes in right design. Mess happens with wrong design. Something is making ppl behave crazy. What is it? If we are engineers we should do analysis. No I dont think it is Nikos. I think it is ppl answering nonsense questions and shouting and keep on answering nonsense with more nonsense and keep on shouting. Why this crazy behavior?? So far Anton has given me the best explanaton -- Ravi From piet at vanoostrum.org Fri Oct 4 17:23:39 2013 From: piet at vanoostrum.org (Piet van Oostrum) Date: Fri, 04 Oct 2013 17:23:39 -0400 Subject: JUST GOT HACKED References: <7wioxgsl1t.fsf@benfinney.id.au> <524BBD33.1060601@rece.vub.ac.be> <7w1u44rwul.fsf@benfinney.id.au> Message-ID: Ravi Sahni writes: > I find this real confused!! Why they are answering then?!?! > As far as I can make out everyone who is answering (helping!) doing it > frustratation and disgust. But still they keep answering and > answering!! I answered him because I wanted to help him. I also find it interesting to solve Python problems. I did not like Nikos' arrogance, ignorance and his refusal to use perfectly good solutions. -- Piet van Oostrum WWW: http://pietvanoostrum.com/ PGP key: [8DAE142BE17999C4] From steve at pearwood.info Wed Oct 2 03:29:54 2013 From: steve at pearwood.info (Steven D'Aprano) Date: 02 Oct 2013 07:29:54 GMT Subject: JUST GOT HACKED References: <7wioxgsl1t.fsf@benfinney.id.au> <524BBD33.1060601@rece.vub.ac.be> Message-ID: <524bcb71$0$2865$c3e8da3$76491128@news.astraweb.com> On Wed, 02 Oct 2013 16:49:22 +1000, Ben Finney wrote: > Antoon Pardon writes: > >> Op 02-10-13 00:06, Ben Finney schreef: >> > This is an unmoderated forum, so we have occasional spates of >> > persistent nuisances, and those who respond with the maturity level >> > and impulse control of an average six-year-old. > [?] >> >> And what about the impuls control and the maturity of people who can't >> stop answering [a nuisance], knowing they contribute to the nuisance to >> the group? > > Yes, we are in firm agreement here. Well that's a shame, and I think that your comment here is incompatible with your previous approval of the PSF Diversity Statement. The Python Software Foundation and the global Python community welcome and encourage participation by everyone. Our community is based on mutual respect, tolerance, and encouragement, and we are working to help each other live up to these principles. We want our community to be more diverse: whoever you are, and whatever your background, we welcome you. This doesn't say anything about bullying, mocking and belittling those we don't like. It says, *encourage* participation by *everyone*, and that includes annoyances like Nikos. Like it or not, he is part of the community of Python users too. I admit to serious reservations about the Diversity Statement and the Code of Conduct, but I think I am living up to their ideals. In my imperfect way, I am trying to encourage Nikos towards better behaviour by rewarding him for good behaviour (answer his sensible questions, encourage him to learn on his own) and discourage his bad behaviour. I don't mock him or belittle him, and I don't celebrate when he screws up. And if I slip up and do any of these things, then I would deserve to be called out on it. Ben, are you doing all you can to live up to the PSF ideals? Are you encouraging Nikos to be a good member of this community, or just standing idly by while others bully him? I can tell you this: if the PSF Code of Conduct applied here, Nikos would be a *distant* third on my list of people to be banned. He might be needy and annoying, but he is not abusive and hostile except when defending himself from the abuse and hostility of others. And I've called Nikos out on his retaliation too. He doesn't get a free pass from me. -- Steven From ben+python at benfinney.id.au Wed Oct 2 03:42:32 2013 From: ben+python at benfinney.id.au (Ben Finney) Date: Wed, 02 Oct 2013 17:42:32 +1000 Subject: Mutual respect, bullying, tolerance (was: JUST GOT HACKED) References: <7wioxgsl1t.fsf@benfinney.id.au> <524BBD33.1060601@rece.vub.ac.be> <524bcb71$0$2865$c3e8da3$76491128@news.astraweb.com> Message-ID: <7wsiwkqftj.fsf_-_@benfinney.id.au> Steven D'Aprano writes: > Well that's a shame, and I think that your comment here is > incompatible with your previous approval of the PSF Diversity > Statement. We should tolerate people, but we should not tolerate their bad behaviour. The Diversity Statement includes the principle of mutual respect, which I heartily endorse. I think the behaviour exhibited for months by the originator of this thread is very unrespectful of this whole community, and that those who bait and belittle this person are also disrespectful. Neither bad behaviour excuses the other, and I find both those behaviours to be against the principles in the PSF Diversity Statement. > Ben, are you doing all you can to live up to the PSF ideals? Are you > encouraging Nikos to be a good member of this community, or just > standing idly by while others bully him? I have several times responded with unequivocal public condemnation to those who called for violence to be done against him. No mere utterance of words can justify a call for violence in response, and I want no-one to suffer that in this forum regardless how much of a nuisance they have become. > And I've called Nikos out on his retaliation too. He doesn't get a > free pass from me. I have long since consigned him to a kill-file: his unrepentantly off-topic demands are far too voluminous for me to keep up with. -- \ ?Holy tintinnabulation, Batman!? ?Robin | `\ | _o__) | Ben Finney From nikos.gr33k at gmail.com Wed Oct 2 06:22:25 2013 From: nikos.gr33k at gmail.com (=?UTF-8?B?zp3Or866zr/Pgg==?=) Date: Wed, 02 Oct 2013 13:22:25 +0300 Subject: JUST GOT HACKED In-Reply-To: <524bcb71$0$2865$c3e8da3$76491128@news.astraweb.com> References: <7wioxgsl1t.fsf@benfinney.id.au> <524BBD33.1060601@rece.vub.ac.be> <524bcb71$0$2865$c3e8da3$76491128@news.astraweb.com> Message-ID: ???? 2/10/2013 10:29 ??, ?/? Steven D'Aprano ??????: > On Wed, 02 Oct 2013 16:49:22 +1000, Ben Finney wrote: > >> Antoon Pardon writes: >> >>> Op 02-10-13 00:06, Ben Finney schreef: >>>> This is an unmoderated forum, so we have occasional spates of >>>> persistent nuisances, and those who respond with the maturity level >>>> and impulse control of an average six-year-old. >> [?] >>> >>> And what about the impuls control and the maturity of people who can't >>> stop answering [a nuisance], knowing they contribute to the nuisance to >>> the group? >> >> Yes, we are in firm agreement here. > > > Well that's a shame, and I think that your comment here is incompatible > with your previous approval of the PSF Diversity Statement. > > The Python Software Foundation and the global Python community > welcome and encourage participation by everyone. Our community is > based on mutual respect, tolerance, and encouragement, and we are > working to help each other live up to these principles. We want our > community to be more diverse: whoever you are, and whatever your > background, we welcome you. > > > This doesn't say anything about bullying, mocking and belittling those we > don't like. It says, *encourage* participation by *everyone*, and that > includes annoyances like Nikos. Like it or not, he is part of the > community of Python users too. > > I admit to serious reservations about the Diversity Statement and the > Code of Conduct, but I think I am living up to their ideals. In my > imperfect way, I am trying to encourage Nikos towards better behaviour by > rewarding him for good behaviour (answer his sensible questions, > encourage him to learn on his own) and discourage his bad behaviour. I > don't mock him or belittle him, and I don't celebrate when he screws up. > > And if I slip up and do any of these things, then I would deserve to be > called out on it. > > Ben, are you doing all you can to live up to the PSF ideals? Are you > encouraging Nikos to be a good member of this community, or just standing > idly by while others bully him? > > I can tell you this: if the PSF Code of Conduct applied here, Nikos would > be a *distant* third on my list of people to be banned. He might be needy > and annoying, but he is not abusive and hostile except when defending > himself from the abuse and hostility of others. > > And I've called Nikos out on his retaliation too. He doesn't get a free > pass from me. My lawyer couldn't have said it better steven :) Thanks for the vote of support. The truth is _exactly_ as yoiu have stated it to be. I apologize though oif some times i loose my temper and use impoprer vocabulary on some people. The pressure is just too high at some point when i see so many people critisize and make ironic comments agianst me all the time in every single thread i start. I was even mocked because all i wanted to do was to optimize code and use the best solution there is to it. From rosuav at gmail.com Wed Oct 2 06:32:25 2013 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 2 Oct 2013 20:32:25 +1000 Subject: JUST GOT HACKED In-Reply-To: References: <7wioxgsl1t.fsf@benfinney.id.au> <524BBD33.1060601@rece.vub.ac.be> <524bcb71$0$2865$c3e8da3$76491128@news.astraweb.com> Message-ID: On Wed, Oct 2, 2013 at 8:22 PM, ????? wrote: > I apologize though oif some times i loose my temper and use impoprer > vocabulary on some people. The pressure is just too high at some point when > i see so many people critisize and make ironic comments agianst me all the > time in every single thread i start. > > I was even mocked because all i wanted to do was to optimize code and use > the best solution there is to it. Well, you weren't trying to use the best solution, for any meaning of "best" other than "the one that feels most like Perl or APL". Your primary problem, as I see it, is pressure. You cope VERY badly under pressure. That's not a big deal in itself - I know plenty of people like that - but if you're that sort of person, you need to organize your life such that situations don't become pressured. That's why I think it would be best for you, in the long run, to give up your web hosting business and code for pure pleasure - that way, if you run into a problem, you're not losing customers, and a delay of a day or two won't hurt anything. You'll be able to work through your coding issues at leisure, figure out what suits you, and actually spend the time to _read_ what people are saying to you. Life would be significantly better for you - and hugely better for this list, because there'd be no more of these sorts of threads. Slow down, you're movin' too fast. ChrisA From nikos.gr33k at gmail.com Wed Oct 2 06:43:32 2013 From: nikos.gr33k at gmail.com (=?UTF-8?B?zp3Or866zr/Pgg==?=) Date: Wed, 02 Oct 2013 13:43:32 +0300 Subject: JUST GOT HACKED In-Reply-To: References: <7wioxgsl1t.fsf@benfinney.id.au> <524BBD33.1060601@rece.vub.ac.be> <524bcb71$0$2865$c3e8da3$76491128@news.astraweb.com> Message-ID: ???? 2/10/2013 1:32 ??, ?/? Chris Angelico ??????: > On Wed, Oct 2, 2013 at 8:22 PM, ????? wrote: >> I apologize though oif some times i loose my temper and use impoprer >> vocabulary on some people. The pressure is just too high at some point when >> i see so many people critisize and make ironic comments agianst me all the >> time in every single thread i start. >> >> I was even mocked because all i wanted to do was to optimize code and use >> the best solution there is to it. > > Well, you weren't trying to use the best solution, for any meaning of > "best" other than "the one that feels most like Perl or APL". > > Your primary problem, as I see it, is pressure. You cope VERY badly > under pressure. That's not a big deal in itself - I know plenty of > people like that - but if you're that sort of person, you need to > organize your life such that situations don't become pressured. That's > why I think it would be best for you, in the long run, to give up your > web hosting business and code for pure pleasure - that way, if you run > into a problem, you're not losing customers, and a delay of a day or > two won't hurt anything. You'll be able to work through your coding > issues at leisure, figure out what suits you, and actually spend the > time to _read_ what people are saying to you. Life would be > significantly better for you - and hugely better for this list, > because there'd be no more of these sorts of threads. > > Slow down, you're movin' too fast. Please bare with me. I will improve more and more over time. I have stopped using Google groups because of the he new lines addition issues that you said it was annoying ans tarted using Thunderbird, and i have also corrected typo errors and try to type more slowly and take the time to correct typos. As you see i'm improving over time. I can assure you that i read all your answer carefully. I only re-ask the same thing if: 1. Di not understood what was provided or proposed to me as being a solution 2. Still feel that that the solution provided to me doesn't meet my needs and should have been re-written in a different way. Nevertheless we are all improving, especially the newbies, by seeing alternative way, best methods and wise practices of writing code for the specific problem and pick the one that does the job best. Just bare with me and you will see improving over time. From rosuav at gmail.com Wed Oct 2 06:54:04 2013 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 2 Oct 2013 20:54:04 +1000 Subject: JUST GOT HACKED In-Reply-To: References: <7wioxgsl1t.fsf@benfinney.id.au> <524BBD33.1060601@rece.vub.ac.be> <524bcb71$0$2865$c3e8da3$76491128@news.astraweb.com> Message-ID: On Wed, Oct 2, 2013 at 8:43 PM, ????? wrote: > Just bare with me and you will see improving over time. Tip: Start with this one... the word you want is 'bear'. In English, those two words have very different meanings, even though they're pronounced the same way; "bare with me" actually talks about getting naked, whereas "bear with me" means what you think it does. :) I'm quite okay with you improving over time, I'm even okay with it happening slowly. And I think everyone here would agree with me on that. But that's why I think you should code only as a hobby, because that means you're not disrupting anything while you learn. It's the safe option. It's like learning to drive a car out in the back paddock (field, for you non-Australians) - there's nobody else to get hurt, worst you can do is bang the car up a bit... and chances are it's an old beat-up car anyway, so that's not a big deal. You don't learn to drive by hopping into a taxi and collecting fares. ChrisA From nikos.gr33k at gmail.com Wed Oct 2 07:01:56 2013 From: nikos.gr33k at gmail.com (=?UTF-8?B?zp3Or866zr/Pgg==?=) Date: Wed, 02 Oct 2013 14:01:56 +0300 Subject: JUST GOT HACKED In-Reply-To: References: <7wioxgsl1t.fsf@benfinney.id.au> <524BBD33.1060601@rece.vub.ac.be> <524bcb71$0$2865$c3e8da3$76491128@news.astraweb.com> Message-ID: ???? 2/10/2013 1:54 ??, ?/? Chris Angelico ??????: > On Wed, Oct 2, 2013 at 8:43 PM, ????? wrote: >> Just bare with me and you will see improving over time. > > Tip: Start with this one... the word you want is 'bear'. In English, > those two words have very different meanings, even though they're > pronounced the same way; "bare with me" actually talks about getting > naked, whereas "bear with me" means what you think it does. :) > > I'm quite okay with you improving over time, I'm even okay with it > happening slowly. And I think everyone here would agree with me on > that. But that's why I think you should code only as a hobby, because > that means you're not disrupting anything while you learn. It's the > safe option. It's like learning to drive a car out in the back paddock > (field, for you non-Australians) - there's nobody else to get hurt, > worst you can do is bang the car up a bit... and chances are it's an > old beat-up car anyway, so that's not a big deal. You don't learn to > drive by hopping into a taxi and collecting fares. I know, but i have too. All i can say its that my visitor's websites are working smoothly and the DNS, Mail issues and Python experiments are happening to my personal account, i don't mess at all with my client's data and their settings. I have to make some money and that needs for some reason to happen now as we speak, so i have no alternative than to hop into a car and learn to drive during the process, hoping i will not bang the the From nikos.gr33k at gmail.com Wed Oct 2 07:03:25 2013 From: nikos.gr33k at gmail.com (=?UTF-8?B?zp3Or866zr/Pgg==?=) Date: Wed, 02 Oct 2013 14:03:25 +0300 Subject: JUST GOT HACKED In-Reply-To: References: <7wioxgsl1t.fsf@benfinney.id.au> <524BBD33.1060601@rece.vub.ac.be> <524bcb71$0$2865$c3e8da3$76491128@news.astraweb.com> Message-ID: ???? 2/10/2013 1:54 ??, ?/? Chris Angelico ??????: > On Wed, Oct 2, 2013 at 8:43 PM, ????? wrote: >> Just bare with me and you will see improving over time. > > Tip: Start with this one... the word you want is 'bear'. In English, > those two words have very different meanings, even though they're > pronounced the same way; "bare with me" actually talks about getting > naked, whereas "bear with me" means what you think it does. :) > > I'm quite okay with you improving over time, I'm even okay with it > happening slowly. And I think everyone here would agree with me on > that. But that's why I think you should code only as a hobby, because > that means you're not disrupting anything while you learn. It's the > safe option. It's like learning to drive a car out in the back paddock > (field, for you non-Australians) - there's nobody else to get hurt, > worst you can do is bang the car up a bit... and chances are it's an > old beat-up car anyway, so that's not a big deal. You don't learn to > drive by hopping into a taxi and collecting fares. I know, but i have too. All i can say its that my visitor's websites are working smoothly and the DNS, Mail issues and Python experiments are happening to my personal account, i don't mess at all with my client's data and their settings. I have to make some money and that needs for some reason to happen now as we speak, so i have no alternative than to hop into a car and learn to drive during the process, hoping i will not bang-smash the car. From rosuav at gmail.com Wed Oct 2 07:11:21 2013 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 2 Oct 2013 21:11:21 +1000 Subject: JUST GOT HACKED In-Reply-To: References: <7wioxgsl1t.fsf@benfinney.id.au> <524BBD33.1060601@rece.vub.ac.be> <524bcb71$0$2865$c3e8da3$76491128@news.astraweb.com> Message-ID: On Wed, Oct 2, 2013 at 9:03 PM, ????? wrote: > All i can say its that my visitor's websites are working smoothly and the > DNS, Mail issues and Python experiments are happening to my personal > account, i don't mess at all with my client's data and their settings. But when you expose your account to vulnerabilities, you expose your customers' web sites too. If you insist on doing this both ways, get yourself a separate computer - maybe just run your own server at home, it won't be the best but it'll be easy to do. That way, if you let every man and his dog into your test box, your customers aren't affected. However, this is a non-Python subject, and one you'll do well to research on Google rather than ask about. ChrisA From modelnine at modelnine.org Wed Oct 2 07:35:12 2013 From: modelnine at modelnine.org (Heiko Wundram) Date: Wed, 02 Oct 2013 13:35:12 +0200 Subject: JUST GOT HACKED In-Reply-To: References: <7wioxgsl1t.fsf@benfinney.id.au> <524BBD33.1060601@rece.vub.ac.be> <524bcb71$0$2865$c3e8da3$76491128@news.astraweb.com> Message-ID: <524C04F0.6060304@modelnine.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Am 02.10.2013 13:03, schrieb ?????: > I have to make some money and that needs for some reason to happen > now as we speak, so i have no alternative than to hop into a car > and learn to drive during the process, hoping i will not bang-smash > the car. I'm really sorry for the fact that it seems as though your livelyhood really does depend on your current mess, but: this is not the way to learn to administer servers or how to program. Do that first in one way or another - and then start making money off it. And, from my personal experience, you are exacerbating your problems by behaving and/or acting as you currently are, as generally at some point in time the mess you currently leave behind - which you permanently choose to ignore - will start to become a liability for the rest of your livelyhood, which _will_ then get you in real trouble. - -- - --- Heiko. -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.20 (MingW32) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQEcBAEBAgAGBQJSTATwAAoJEDMqpHf921/SK54IAJvUF+3nTJq5nKPN2s1WdQbz hOvqMThrhBE7BG6ybF8TfbKpLmZ+20cZExVzn4Xy9PPGe+WTrt6UR8+UizSst1Vs EgZ0DrmWb+WRN+nUZPyL45psDMaHdi1bQy0ReVGbav1faG9Y9tAMZ2KEQwfrnmZz CJ9mTJ95IbuB3iizCdlUOT2qCzhGPyCsx1ejR6IkKofKaO0QU712V7rHN9u/xdlJ v687pSzeNuRxWP9Rdlp25FIVDgj3oNGrK9HXrYUyra9TXSyZW3XTWbUjwriNMxer 8B00cngvLTEf14AmMeIkno7GvTP5QWq7yNul7n85Pq6ZXJKWLjVLodKwYndSf9I= =AOdw -----END PGP SIGNATURE----- From steve+comp.lang.python at pearwood.info Wed Oct 2 09:06:36 2013 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: 02 Oct 2013 13:06:36 GMT Subject: JUST GOT HACKED References: <7wioxgsl1t.fsf@benfinney.id.au> <524BBD33.1060601@rece.vub.ac.be> <524bcb71$0$2865$c3e8da3$76491128@news.astraweb.com> Message-ID: <524c1a5c$0$29984$c3e8da3$5496439d@news.astraweb.com> On Wed, 02 Oct 2013 14:03:25 +0300, ????? wrote: > I have to make some money and that needs for some reason to happen now > as we speak, so i have no alternative than to hop into a car and learn > to drive during the process, hoping i will not bang-smash the car. Yes you have alternatives. You have many alternatives, even if you are determined to stay in this IT business. We have told you some of these alternatives before. Some alternatives: - Can you access Google? You can search for guides to do with securing web sites. You don't have to ask here. We are not experts on that. This is a Python list, not a website administration list. - Books. Buy some books on system administration. Borrow them from the library. Even ten year old books going cheap are better than nothing, they will teach you the basic concepts. - Go do a course. - Set up a crappy old PC running Linux and put apache on it. Don't put anything important on it. You will probably find a million people on the internet who will break into it, if you give them permission, and tell you what they did. But we are not those people. Look elsewhere. But don't trust them with anything important. After they are done, erase the entire disk and reinstall from scratch. Not everyone is as kind and trustworthy as us. - If you can't find people to break into it for free, you can pay them. Surely there are Linux consultants in Greece that will secure your system for you for a fee? That's a legitimate business expense and will reduce your tax. - Find a business partner. You can't be the only person in Greece interested in building up a web hosting business. None of these things have anything to do with Python. This entire discussion is off-topic, I am only mentioning this to you as a courtesy, one human being to another. Please don't insult me by continuing this off- topic discussion here on this Python list. This is comp.lang.python, not comp.lang.teach.nikos.everything.he.needs.to.learn. Come back when you have Python questions. Everything else, take it away. -- Steven From python.list at tim.thechases.com Wed Oct 2 09:09:36 2013 From: python.list at tim.thechases.com (Tim Chase) Date: Wed, 2 Oct 2013 08:09:36 -0500 Subject: JUST GOT HACKED In-Reply-To: References: <7wioxgsl1t.fsf@benfinney.id.au> <524BBD33.1060601@rece.vub.ac.be> <524bcb71$0$2865$c3e8da3$76491128@news.astraweb.com> Message-ID: <20131002080936.08058574@bigbox.christie.dr> On 2013-10-02 13:43, ????? wrote: > 2. Still feel that that the solution provided to me doesn't meet my > needs and should have been re-written in a different way. This is part of the trouble people had recently in the IP-address/default-value thread. Python has what folks here call a "pythonic" style. Adhering to this style helps others read your code, and often makes it easy for other people to spot bugs. By breaking those conventions and demanding not only a solution, but one that looks the way you think it should, you make it harder for others to help you and verify the correctness of the solution. You will be better off concisely posing the problem with no expectation of how the solution will look. Look at the answers that come back: if similar solutions appear in multiple replies, then that's likely the best way to go. > Mysql pass != account's password As mentioned in other areas of this thread, with access to MySQL, there are often ways to execute arbitrary shell-code on the server. -tkc From waynejwerner at gmail.com Fri Oct 4 09:49:21 2013 From: waynejwerner at gmail.com (Wayne Werner) Date: Fri, 4 Oct 2013 06:49:21 -0700 (PDT) Subject: JUST GOT HACKED In-Reply-To: References: <7wioxgsl1t.fsf@benfinney.id.au> <524BBD33.1060601@rece.vub.ac.be> <524bcb71$0$2865$c3e8da3$76491128@news.astraweb.com> Message-ID: <3de69877-325f-4206-b009-8b0643ec9e5f@googlegroups.com> On Wednesday, October 2, 2013 5:43:32 AM UTC-5, Ferrous Cranus wrote: > > I only re-ask the same thing if: > > > 1. Di not understood what was provided or proposed to me as being a solution > > 2. Still feel that that the solution provided to me doesn't meet my > needs and should have been re-written in a different way. Nevertheless > we are all improving, especially the newbies, by seeing alternative way, > best methods and wise practices of writing code for the specific problem > and pick the one that does the job best. > If you feel that the provided solution doesn't meet your needs then the *most likely* answer is that you have asked the wrong question. Rather than, say, posting a new thread (unless your new question is clearly different and mostly unrelated), the appropriate course of action is to say something like, "Hey, apparently I'm asking the wrong question, because all these answers are about frobnosticating, but I really wanted to foo the baz." If you don't acknowledge that you goofed up, you come across as arrogant or ignorant (or both). -W From denismfmcmahon at gmail.com Wed Oct 2 11:50:30 2013 From: denismfmcmahon at gmail.com (Denis McMahon) Date: Wed, 2 Oct 2013 15:50:30 +0000 (UTC) Subject: JUST GOT HACKED References: <7wioxgsl1t.fsf@benfinney.id.au> <524BBD33.1060601@rece.vub.ac.be> <524bcb71$0$2865$c3e8da3$76491128@news.astraweb.com> Message-ID: On Wed, 02 Oct 2013 13:22:25 +0300, ????? wrote: > I was even mocked because all i wanted to do was to optimize code and > use the best solution there is to it. No. You were mocked because you insisted that your broken one line code was a better looking solution than anyone elses working multi line solution. The mocking was because you were failing to recognise that if it doesn't work, it's not a solution, no matter how nice it looks. -- Denis McMahon, denismfmcmahon at gmail.com From rosuav at gmail.com Tue Oct 1 18:15:31 2013 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 2 Oct 2013 08:15:31 +1000 Subject: JUST GOT HACKED In-Reply-To: References: Message-ID: On Wed, Oct 2, 2013 at 12:24 AM, Daniel Stojanov wrote: > 3) I'm a bit late to the party. Is Nikos a real sysadmin or is this some > horrible inside joke I don't get? He's a real sysadmin, in the sense that he has a system that he administers - this isn't some elaborate prank that we collectively pull on newcomers, no. His main problem is that he doesn't listen to advice, even advice he specifically requests; so his server has, on occasion, had major problems lasting a long time. What just happened here, I think, is that he had a password accidentally visible, and someone logged in as him. Tip: Learn from his mistakes, that'll save you the pain of making them yourself :) Fortunately, this sort of thing isn't all that common. Most of the time we have much more in the category of "Python-related and real good fun". Unless you absolutely detest Python and all that it stands for, stick around - we're sure to have something more interesting than this. :) ChrisA From breamoreboy at yahoo.co.uk Tue Oct 1 18:28:20 2013 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Tue, 01 Oct 2013 23:28:20 +0100 Subject: JUST GOT HACKED In-Reply-To: References: Message-ID: On 01/10/2013 15:24, Daniel Stojanov wrote: > > 2) I just signed up the this mailing list. To the regulars, is this what > normally happens on this list? > Nah, sadly part of it is due to me failing to keep Asperger Syndrome under control. Marks out of 10, -5, must try harder :( If you're into upping your game with Python please take a look here http://news.gmane.org/index.php?prefix=gmane.comp.python, there's a couple of mailing lists that could keep you occupied for a few minutes each day :) -- Cheers. Mark Lawrence From ben+python at benfinney.id.au Tue Oct 1 18:41:24 2013 From: ben+python at benfinney.id.au (Ben Finney) Date: Wed, 02 Oct 2013 08:41:24 +1000 Subject: Improving community discussion (was: JUST GOT HACKED) References: Message-ID: <7wa9issjfv.fsf_-_@benfinney.id.au> Mark Lawrence writes: > Nah, sadly part of it is due to me failing to keep Asperger Syndrome > under control. Marks out of 10, -5, must try harder :( Thank you for acknowledging this. I sincerely wish you strength in limiting antisocial behaviour. We need more people who can recognise and overcome their own impulses to behave badly :-) > If you're into upping your game with Python please take a look here > http://news.gmane.org/index.php?prefix=gmane.comp.python, there's a > couple of mailing lists that could keep you occupied for a few minutes > each day :) Also of benefit is to find ? or, if it doesn't exist, to found ? a local Python User Group . Meeting and discussing in person with like-minded fellows, on a regular schedule, works very well for a tool like Python, in my experience. -- \ ?He who laughs last, thinks slowest.? ?anonymous | `\ | _o__) | Ben Finney From waynejwerner at gmail.com Fri Oct 4 09:23:05 2013 From: waynejwerner at gmail.com (Wayne Werner) Date: Fri, 4 Oct 2013 06:23:05 -0700 (PDT) Subject: JUST GOT HACKED In-Reply-To: References: Message-ID: <4fe72ca2-61c7-4b95-acec-cddfa6f63d22@googlegroups.com> On Tuesday, October 1, 2013 5:06:38 PM UTC-5, Ben Finney wrote: > This is an unmoderated forum, so we have occasional spates of persistent > > nuisances, and those who respond with the maturity level and impulse > > control of an average six-year-old. Hey! That's so degrading! I don't know many six-year-olds (and certainly not mine, but maybe they're just not average) who would continue to respond in the same fashion that the OP of *this* thread did! ;-) -W From timothy.c.delaney at gmail.com Tue Oct 1 16:45:33 2013 From: timothy.c.delaney at gmail.com (Tim Delaney) Date: Wed, 2 Oct 2013 06:45:33 +1000 Subject: JUST GOT HACKED In-Reply-To: References: Message-ID: On 2 October 2013 00:00, ????? wrote: > > Thanks for visting my website: you help me increase my google page rank > without actually utilizing SEO. > > Here: http://superhost.gr/?show=log&**page=index.html Speaking of which, I would strongly advise against *anyone* going to Nikos' web site. With the length of time his credentials have been available for anyone in the world to obtain and use it's highly likely that by now his website is a malware-spewing zombie member of a botnet. Of course, I'm not going to risk it by going there to check myself ... Tim Delaney -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve+comp.lang.python at pearwood.info Tue Oct 1 22:30:20 2013 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: 02 Oct 2013 02:30:20 GMT Subject: JUST GOT HACKED References: Message-ID: <524b853b$0$29984$c3e8da3$5496439d@news.astraweb.com> On Wed, 02 Oct 2013 00:24:35 +1000, Daniel Stojanov wrote: > 2) I just signed up the this mailing list. To the regulars, is this what > normally happens on this list? No. > 3) I'm a bit late to the party. Is Nikos a real sysadmin or is this some > horrible inside joke I don't get? Nikos is not a real sysadmin. He is a wanna-be web developer who barely knows Python. He is arrogant and annoying. Unfortunately he is not the real problem here. The real problem is a bunch of vigilantes have appointed themselves the anti-Nikos lynch mob and take every opportunity they can to mock him, insult him, bait him into responding to their taunts, and even make public death threats against him. And now it appears that one of them may have hacked into his web site in an attempt to put him out of business. These vigilantes have decided to save this mailing list from Nikos, even if it means destroying it. Nikos at least does ask Python questions. The vigilantes hardly talk about Python at all, they're too busy laughing at Nikos and insulting him. Nikos doesn't learn from his errors; neither do the vigilantes, no matter how many times they have failed they are sure that if they mock him just a little bit harder he will go away. He won't, they keep baiting him even more, and this place is going to shit thanks to them. P.S. this mailing list is mirrored on Usenet as comp.lang.python, and it is considered rude to post HTML if you can avoid it. -- Steven From antoon.pardon at rece.vub.ac.be Wed Oct 2 02:51:14 2013 From: antoon.pardon at rece.vub.ac.be (Antoon Pardon) Date: Wed, 02 Oct 2013 08:51:14 +0200 Subject: JUST GOT HACKED In-Reply-To: <524b853b$0$29984$c3e8da3$5496439d@news.astraweb.com> References: <524b853b$0$29984$c3e8da3$5496439d@news.astraweb.com> Message-ID: <524BC262.8080304@rece.vub.ac.be> Op 02-10-13 04:30, Steven D'Aprano schreef: > On Wed, 02 Oct 2013 00:24:35 +1000, Daniel Stojanov wrote: > >> 2) I just signed up the this mailing list. To the regulars, is this what >> normally happens on this list? > > No. > >> 3) I'm a bit late to the party. Is Nikos a real sysadmin or is this some >> horrible inside joke I don't get? > > Nikos is not a real sysadmin. He is a wanna-be web developer who barely > knows Python. He is arrogant and annoying. Unfortunately he is not the > real problem here. The real problem is a bunch of vigilantes have > appointed themselves the anti-Nikos lynch mob and take every opportunity > they can to mock him, insult him, bait him into responding to their > taunts, and even make public death threats against him. And now it > appears that one of them may have hacked into his web site in an attempt > to put him out of business. That is your slant of things. My take is that the real problem is those that keep spoon feeding the help vampire, collaborating to the nuissance and encouraging Nikos to come back. > These vigilantes have decided to save this mailing list from Nikos, even > if it means destroying it. Nikos at least does ask Python questions. The > vigilantes hardly talk about Python at all, they're too busy laughing at > Nikos and insulting him. Nikos doesn't learn from his errors; neither do > the vigilantes, no matter how many times they have failed they are sure > that if they mock him just a little bit harder he will go away. He won't, > they keep baiting him even more, and this place is going to shit thanks > to them. Maybe the people venting their frustration think the mailing list is already partly gone to shit. So why should they care it is going to shit too for those who collaborate to the nuisance. The latter didn't care much the mailing list was going to shit for the first. The latter only started to care about the shit level when they themselves though it became too high for them. And then they complain about those venting their frustration but they won't look at how their own behaviour contributes to the frustration of those venting. -- Antoon. From breamoreboy at yahoo.co.uk Wed Oct 2 03:32:01 2013 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Wed, 02 Oct 2013 08:32:01 +0100 Subject: JUST GOT HACKED In-Reply-To: <524BC262.8080304@rece.vub.ac.be> References: <524b853b$0$29984$c3e8da3$5496439d@news.astraweb.com> <524BC262.8080304@rece.vub.ac.be> Message-ID: On 02/10/2013 07:51, Antoon Pardon wrote: > Op 02-10-13 04:30, Steven D'Aprano schreef: >> On Wed, 02 Oct 2013 00:24:35 +1000, Daniel Stojanov wrote: >> >>> 2) I just signed up the this mailing list. To the regulars, is this what >>> normally happens on this list? >> >> No. >> >>> 3) I'm a bit late to the party. Is Nikos a real sysadmin or is this some >>> horrible inside joke I don't get? >> >> Nikos is not a real sysadmin. He is a wanna-be web developer who barely >> knows Python. He is arrogant and annoying. Unfortunately he is not the >> real problem here. The real problem is a bunch of vigilantes have >> appointed themselves the anti-Nikos lynch mob and take every opportunity >> they can to mock him, insult him, bait him into responding to their >> taunts, and even make public death threats against him. And now it >> appears that one of them may have hacked into his web site in an attempt >> to put him out of business. > > That is your slant of things. My take is that the real problem is those > that keep spoon feeding the help vampire, collaborating to the nuissance > and encouraging Nikos to come back. > >> These vigilantes have decided to save this mailing list from Nikos, even >> if it means destroying it. Nikos at least does ask Python questions. The >> vigilantes hardly talk about Python at all, they're too busy laughing at >> Nikos and insulting him. Nikos doesn't learn from his errors; neither do >> the vigilantes, no matter how many times they have failed they are sure >> that if they mock him just a little bit harder he will go away. He won't, >> they keep baiting him even more, and this place is going to shit thanks >> to them. > > Maybe the people venting their frustration think the mailing list is > already partly gone to shit. So why should they care it is going to shit > too for those who collaborate to the nuisance. The latter didn't care > much the mailing list was going to shit for the first. The latter only > started to care about the shit level when they themselves though it > became too high for them. And then they complain about those venting > their frustration but they won't look at how their own behaviour > contributes to the frustration of those venting. > No guessing which camp I'm in. -- Cheers. Mark Lawrence From ganeshsahni07 at gmail.com Wed Oct 2 00:09:33 2013 From: ganeshsahni07 at gmail.com (Ravi Sahni) Date: Tue, 1 Oct 2013 21:09:33 -0700 (PDT) Subject: JUST GOT HACKED In-Reply-To: References: Message-ID: On Tuesday, October 1, 2013 7:54:35 PM UTC+5:30, Daniel Stojanov wrote: > 2) I just signed up the this mailing list. To the regulars, is this what normally happens on this list? > > 3) I'm a bit late to the party. Is Nikos a real sysadmin or is this some horrible inside joke I don't get? Thanks Daniel!!! Lurker here: I too was wondering whether I have got into the wrong place From rurpy at yahoo.com Wed Oct 2 12:41:45 2013 From: rurpy at yahoo.com (rurpy at yahoo.com) Date: Wed, 2 Oct 2013 09:41:45 -0700 (PDT) Subject: JUST GOT HACKED In-Reply-To: References: Message-ID: <148af794-6ef2-44a5-a5fa-403bae955857@googlegroups.com> On 10/01/2013 08:24 AM, Daniel Stojanov wrote: > On 02/10/2013 12:05 AM, "?????" wrote: > >> Thanks for visting my website: you help me increase my google page >> rank without actually utilizing SEO. >> >> -- https://mail.python.org/mailman/listinfo/python-list > > 1) You need links, not page views to improve your Google rank. > > 2) I just signed up the this mailing list. To the regulars, is this > what normally happens on this list? Recently, it seems to have become normal. > 3) I'm a bit late to the party. Is Nikos a real sysadmin or is this > some horrible inside joke I don't get? Nikos is running a website he wrote in Python and seems to be learning as he goes. He offends a number of people here by refusing to take "advice" such as hire someone, spend a few years learning python, system administration, webserver administration, and the like given without the slightest knowledge of Nikos' circumstances. He also repeatedly re- asks questions when he doesn't understand or like the answers received, seems to prefer to find answers to questions by asking here rather than researching himself (tho it is not clear how much being a non-native English speaker plays into that.) He is also willing to respond in kind to hostile remarks addressed to him, and does not display proper deference to the regulars here in other way too. All of the above irritates a number of people here, who, being rather like Nikos themselves in their complete disregard for the signal-to-noise ratio or atmosphere of the group, find in him a good excuse to vent their own frustrations by responding with more patently useless "advice", insults, ridicule, threats and other vitriolic noise. They rationalize this as applying social pressure blithely ignoring that it's shown no signs of working. In other words, many of Nikos' threads degenerate into a plain old-fashioned flame war. Probably the vast majority of readers do their best to simply ignore the trash posts but there is small (but large enough) group of regulars who enjoy participating in such flame wars to degrade the quality of the group far more than would be the case if they were able to follow the time-tested advice of "don't feed the trolls". While Stephen D'Aprano is often enough an abrasive poster in his own right, his comments on the current situation are the most sensible I've seen in this disscussion: https://mail.python.org/pipermail/python-list/2013-October/656691.html https://mail.python.org/pipermail/python-list/2013-October/656716.html From p.johnson125 at gmail.com Tue Oct 8 10:51:45 2013 From: p.johnson125 at gmail.com (Pat Johnson) Date: Tue, 8 Oct 2013 07:51:45 -0700 (PDT) Subject: JUST GOT HACKED In-Reply-To: References: Message-ID: I don't think you are allowed to use the word dumbass to describe anyone or anything buddy. On Tuesday, October 1, 2013 9:42:31 AM UTC-4, Ferrous Cranus wrote: > ???? 1/10/2013 4:27 ??, ?/? Chris ?Kwpolska? Warrick ??????: > > > On Tue, Oct 1, 2013 at 3:15 PM, ????? wrote: > > >> ???? 1/10/2013 4:06 ??, ?/? Mark Lawrence ??????: > > >>> > > >>> On 01/10/2013 10:58, ????? wrote: > > >>>> > > >>>> Just logged in via FTP to my server and i saw an uploade file named > > >>>> "Warnign html" > > >>>> > > >>>> Contents were: > > >>>> > > >>>> WARNING > > >>>> > > >>>> I am incompetent. Do not hire me! > > >>>> > > >>>> Question: > > >>>> > > >>>> WHO AND MOST IMPORTNTANLY HOW DID HE MANAGED TO UPLOAD THIS FILE ON MY > > >>>> ACCOUNT? > > >>>> > > >>>> PLEASE ANSWER ME, I WONT GET MAD, BUT THIS IS AN IMPORTANT SECURITY RISK. > > >>>> > > >>>> SOMEONES MUST HAVE ACCESS TO MY ACCOUNT, DOES THE SOURCE CODE OF MY MAIN > > >>>> PYTHON SCRIPT APPEARS SOMEPLACE AGAIN?!?! > > >>> > > >>> > > >>> Would you please stop posting, I've almost burst my stomach laughing at > > >>> this. You definetely have a ready made career writing comedy. > > >> > > >> > > >> Okey smartass, > > >> > > >> Try to do it again, if you be successfull again i'll even congratulate you > > >> myself. > > >> > > >> -- > > >> https://mail.python.org/mailman/listinfo/python-list > > > > > > It looks like you are accusing someone of doing something without any > > > proof whatsoever. Would you like help with the fallout of the lawsuit > > > that I hope Mark might (should!) come up with?i'am > > > > > > Speaking of ?try again?, I doubt it would be hard? As long as a FTP > > > daemon is running somewhere (and you clearly do not know better); or > > > even you have a SSH daemon and you do not know better, an attacker > > > can: > > > > > > a) wait for you to publish your password yet again; > > > b) get you to download an exploit/keylogger/whatever; > > > c) brute-force. > > > > > > Well, considering it?s unlikely you actually have a long-as-shit > > > password, (c) is the best option. Unless your password is very long, > > > in which case is not. > > > > > > I?m also wondering what language your password is in. If you actually > > > used a Greek phrase, how long will it take you to get locked out due > > > to encoding bullshit? > > > > Like i use grek letter for my passwords or like i'am gonna fall for any > > of your 3 dumbass reasons. > > > > I already foudn the weakness and corrected it. From piet at vanoostrum.org Tue Oct 1 14:21:28 2013 From: piet at vanoostrum.org (Piet van Oostrum) Date: Tue, 01 Oct 2013 14:21:28 -0400 Subject: JUST GOT HACKED References: Message-ID: ????? writes: > Just logged in via FTP to my server and i saw an uploade file named > "Warnign html" > > Contents were: > > WARNING > > I am incompetent. Do not hire me! > > Question: > > WHO AND MOST IMPORTNTANLY HOW DID HE MANAGED TO UPLOAD THIS FILE ON MY > ACCOUNT? > > PLEASE ANSWER ME, I WONT GET MAD, BUT THIS IS AN IMPORTANT SECURITY RISK. > > SOMEONES MUST HAVE ACCESS TO MY ACCOUNT, DOES THE SOURCE CODE OF MY MAIN > PYTHON SCRIPT APPEARS SOMEPLACE AGAIN?!?! This shows that the warning was correct. -- Piet van Oostrum WWW: http://pietvanoostrum.com/ PGP key: [8DAE142BE17999C4] From denismfmcmahon at gmail.com Tue Oct 1 18:05:43 2013 From: denismfmcmahon at gmail.com (Denis McMahon) Date: Tue, 1 Oct 2013 22:05:43 +0000 (UTC) Subject: JUST GOT HACKED References: Message-ID: On Tue, 01 Oct 2013 12:58:50 +0300, ????? wrote: > Just logged in via FTP to my server and i saw an uploade file named > "Warnign html" Yes, so we can add "basic internet security" to the growing list of things you know nothing about: python programming, etiquette, http, dns, tcp/ip, mimetypes, utf-8, basic internet security And this is just based on the last 30 days of your posts! -- Denis McMahon, denismfmcmahon at gmail.com From z at etiol.net Tue Oct 1 19:02:08 2013 From: z at etiol.net (Zero Piraeus) Date: Tue, 1 Oct 2013 20:02:08 -0300 Subject: JUST GOT HACKED In-Reply-To: References: Message-ID: <20131001230208.GA17776@piedra> : Imagine, if you will, a pub landlord. I'll call him Nick. The pub Nick manages is a run-of-the-mill affair: he leases it from a brewery, and they look after most of the technical aspects of the business. When it comes down to it, Nick is just a reseller of alcohol. If one of the regulars mentions that they like Joomlager[1], all he has to do is call the brewery and they'll handle the rest. After a little while, Nick decides that his six months experience behind a bar qualifies him for greater things. He's heard that the cool kids are all drinking artesanal, micro-brewed Python Ale these days, so thinking to himself "how hard can it be?", Nick throws syrup, yeast and water into a big bucket next to the bar (reading up on how to do it properly is too boring for Nick), and after a week or so, he's got a sludgy mess for his trouble. "Great!", thinks Nick. "I can sell this no problem". Unsurprisingly, Nick's "beer" is awful. His customers aren't that fussy (they're in Nick's bar, after all), but the stuff doesn't even seem to get you even slightly merry, and a couple of people have gotten ill after drinking it. Undaunted, Nick decides to stick at it. He still can't be bothered to learn anything about fermentation or any of that boring crap, but that's okay, because he's discovered the local Python Ale Brewing Club. The PABC is a friendly, helpful bunch, and a lot of the members really know their stuff. For example, when Nick asks "how do I avoid letting that scummy residue into the glass when I dunk it into the bucket to serve someone?", they try to explain to him that a) serving directly from the same bucket he brewed in is a bad idea, and b) if he'd brewed it right, there shouldn't be any scummy residue in the first place. Of course, Nick doesn't have time for any of that - he just wants an answer to his question. He resolutely ignores anyone who tells him things like "anyone could spit (or worse) in that open bucket; you need to think about safety", and if someone tells him he should add malt to improve the flavour, he just throws some ovaltine into the bucket he's serving from, along with any half-finished drinks left by his customers. Meanwhile, a lot of the members of the PABC are getting tired of Nick asking the same questions over and over again, and not listening to the answers - and especially his casual disregard for the safety of his customers. They're enthusiasts, after all, and he's the kind of guy that gives drinking establishments a bad name. So, one night just before closing time, one of them pours bright green food dye into Nick's bucket - nothing that would hurt anyone, but something that Nick couldn't fail to notice before opening up the next day. It's a little sketchy to adulterate his product like that, but he's proved impervious to everyone's attempts to get him to take safety seriously - maybe *this* will shock him into action. Sure enough, the next morning Nick starts crying about how someone has poisoned his beer. It's okay though; he's covered the bucket with a wet towel, and he challenges anyone to get past what he believes is his now-perfect security. - - - In other words: you weren't "hacked". You'd been repeatedly told that you had publicly visible source code on the net containing passwords in plain text; all anyone had to do was login to your server with the credentials you negligently exposed, and open a text editor. If that's hacking, I'm Neo. That's not to say someone else *hasn't* pissed in your bucket, but if they have, they won't have publicised the fact. By the way: if you haven't already, you'll want to remove the extra line from your .htaccess file. And in case it isn't obvious: no, it wasn't Mark Lawrence. -[]z. [1] "It's a bit rough, but it gets the job done. Gives you a terrible hangover, mind". -- Zero Piraeus: flagellum dei http://etiol.net/pubkey.asc From nikos.gr33k at gmail.com Tue Oct 1 19:28:41 2013 From: nikos.gr33k at gmail.com (=?UTF-8?B?zp3Or866zr/Pgg==?=) Date: Wed, 02 Oct 2013 02:28:41 +0300 Subject: JUST GOT HACKED In-Reply-To: References: Message-ID: ???? 2/10/2013 2:02 ??, ?/? Zero Piraeus ??????: > In other words: you weren't "hacked". You'd been repeatedly told that > you had publicly visible source code on the net containing passwords in > plain text; all anyone had to do was login to your server with the > credentials you negligently exposed, and open a text editor. If that's > hacking, I'm Neo. I'am aware of that fact, but the line you are refering too was just initiating a mysql connection: con = pymysql.connect( db = 'mypass', user = 'myuser', passwd = 'mysqlpass', charset = 'utf8', host = 'localhost' ) That was viewable by the link Mark have posted. But this wasnt my personal's account's login password, that was just the mysql password. Mysql pass != account's password > That's not to say someone else *hasn't* pissed in your bucket, but if > they have, they won't have publicised the fact. Ah, now i shoudl worry for more people breaking in? > By the way: if you haven't already, you'll want to remove the extra line > from your .htaccess file. Tell me the line you are referring to. Yes i added some line but i want you to tell me which line is that. > case it isn't obvious: no, it wasn't > Mark Lawrence. Who was it then, you? From timothy.c.delaney at gmail.com Tue Oct 1 19:48:30 2013 From: timothy.c.delaney at gmail.com (Tim Delaney) Date: Wed, 2 Oct 2013 09:48:30 +1000 Subject: JUST GOT HACKED In-Reply-To: References: Message-ID: On 2 October 2013 09:28, ????? wrote: > > con = pymysql.connect( db = 'mypass', user = 'myuser', passwd = > 'mysqlpass', charset = 'utf8', host = 'localhost' ) > > That was viewable by the link Mark have posted. > > But this wasnt my personal's account's login password, that was just the > mysql password. > > Mysql pass != account's password Because there's no chance with the brilliance you display that there could be any possibility of login details being kept in plaintext in your database. And of course your database is so well locked down that no attacker with a login to it could then execute arbitrary code on your system. And there's also zero chance that your personal account login details are also available in plaintext somewhere that you're unaware of. Tim Delaney -------------- next part -------------- An HTML attachment was scrubbed... URL: From python.list at tim.thechases.com Tue Oct 1 20:14:13 2013 From: python.list at tim.thechases.com (Tim Chase) Date: Tue, 1 Oct 2013 19:14:13 -0500 Subject: JUST GOT HACKED In-Reply-To: References: Message-ID: <20131001191413.0df23d84@bigbox.christie.dr> On 2013-10-02 09:48, Tim Delaney wrote: > Because there's no chance with the brilliance you display that > there could be any possibility of login details being kept in > plaintext in your database. > > And of course your database is so well locked down that no attacker > with a login to it could then execute arbitrary code on your system. > > And there's also zero chance that your personal account login > details are also available in plaintext somewhere that you're > unaware of. And there's no way an elephant-sized hole in the web application allowed for dropping files/scripts on the server to do the arbitrary bidding of anybody who read the source... -tkc From kwpolska at gmail.com Wed Oct 2 08:47:24 2013 From: kwpolska at gmail.com (=?UTF-8?B?Q2hyaXMg4oCcS3dwb2xza2HigJ0gV2Fycmljaw==?=) Date: Wed, 2 Oct 2013 14:47:24 +0200 Subject: JUST GOT HACKED In-Reply-To: References: Message-ID: On Wed, Oct 2, 2013 at 1:48 AM, Tim Delaney wrote: > On 2 October 2013 09:28, ????? wrote: >> >> >> con = pymysql.connect( db = 'mypass', user = 'myuser', passwd = >> 'mysqlpass', charset = 'utf8', host = 'localhost' ) >> >> That was viewable by the link Mark have posted. >> >> But this wasnt my personal's account's login password, that was just the >> mysql password. >> >> Mysql pass != account's password > > > Because there's no chance with the brilliance you display that there could > be any possibility of login details being kept in plaintext in your > database. Or the statement is a blatant lie and was meant to be mysql_password is not account_password as they have the same value, but are set independently. (too much Python Ale?) -- Chris ?Kwpolska? Warrick PGP: 5EAAEA16 stop html mail | always bottom-post | only UTF-8 makes sense From ishish at domhain.de Wed Oct 2 08:57:44 2013 From: ishish at domhain.de (ishish) Date: Wed, 02 Oct 2013 13:57:44 +0100 Subject: JUST GOT HACKED In-Reply-To: References: Message-ID: Am 02.10.2013 13:47, schrieb Chris ?Kwpolska? Warrick: > On Wed, Oct 2, 2013 at 1:48 AM, Tim Delaney > wrote: >> On 2 October 2013 09:28, ????? wrote: >>> >>> >>> con = pymysql.connect( db = 'mypass', user = 'myuser', passwd = >>> 'mysqlpass', charset = 'utf8', host = 'localhost' ) >>> >>> That was viewable by the link Mark have posted. >>> >>> But this wasnt my personal's account's login password, that was >>> just the >>> mysql password. >>> >>> Mysql pass != account's password >> >> >> Because there's no chance with the brilliance you display that there >> could >> be any possibility of login details being kept in plaintext in your >> database. > > Or the statement is a blatant lie and was meant to be > > mysql_password is not account_password > > as they have the same value, but are set independently. (too much > Python Ale?) Who cares... mysql> \! bash... job done. From nikos.gr33k at gmail.com Wed Oct 2 06:52:39 2013 From: nikos.gr33k at gmail.com (=?UTF-8?B?zp3Or866zr/Pgg==?=) Date: Wed, 02 Oct 2013 13:52:39 +0300 Subject: JUST GOT HACKED In-Reply-To: References: Message-ID: zero piraeus have said: > In other words: you weren't "hacked". You'd been repeatedly told that > you had publicly visible source code on the net containing passwords in > plain text; all anyone had to do was login to your server with the > credentials you negligently exposed, and open a text editor. If that's > hacking, I'm Neo. I'am aware of that fact, but the line you are refering too was just initiating a mysql connection: con = pymysql.connect( db = 'mypass', user = 'myuser', passwd = 'mysqlpass', charset = 'utf8', host = 'localhost' ) That was viewable by the link Mark have posted. But this wasnt my personal's account's login password, that was just the mysql password. Mysql pass != account's password > That's not to say someone else *hasn't* pissed in your bucket, but if > they have, they won't have publicised the fact. Ah, now i shoudl worry for more people breaking in? > By the way: if you haven't already, you'll want to remove the extra line > from your .htaccess file. Tell me the line you are referring to. Yes i added some line but i want you to tell me which line is that. > case it isn't obvious: no, it wasn't > Mark Lawrence. Who was it then, you? I wont get mad but i want you too answer all of my questions and: 1. state by which method you managed to break in since at noplace at my awareness did i psot my account's login pass, only the source code of my main script which is now fixed by me altering the httpd.conf file and placing extra lines into my main .htaccess file 2. Be sincere and tell me if you have created a backdoor on my server that allows you to remotely login and do stuff. I will even thank you for not destroying my system, but i want these questions i just types to be answered so i take action to fix things ven better. Please, this is a business server. From feedthetroll at gmx.de Wed Oct 2 07:42:32 2013 From: feedthetroll at gmx.de (feedthetroll at gmx.de) Date: Wed, 2 Oct 2013 04:42:32 -0700 (PDT) Subject: JUST GOT HACKED In-Reply-To: References: Message-ID: <9208d479-21ff-4da1-86f0-35cf46b13505@googlegroups.com> Am Mittwoch, 2. Oktober 2013 12:52:39 UTC+2 schrieb Ferrous Cranus: > ... >> By the way: if you haven't already, you'll want to remove the extra line >> from your .htaccess file. > Tell me the line you are referring to. I think it will be the line YOU did NOT enter. Just take a look yourself. The best way will be to compare it to a backup you made before the change. (Nikos and backups. Good joke, ins't it?) > ... > I wont get mad but i want you too answer all of my questions and: > 1. state by which method you managed to break in since at noplace at my > awareness did i psot my account's login pass, only the source code of my > main script which is now fixed by me altering the httpd.conf file and > placing extra lines into my main .htaccess file > 2. Be sincere and tell me if you have created a backdoor on my server > that allows you to remotely login and do stuff. > I will even thank you for not destroying my system, but i want these > questions i just types to be answered so i take action to fix things ven > better. ROTFL. You made my day. Again! > Please, this is a business server. No it ist not: Am Montag, 30. September 2013 20:03:32 UTC+2 schrieb Ferrous Cranus: > ... > I learn Python for personal pleasure because i like programming. "Learning for personal pleasure" and "business server" can't be true both. So one of the statements is wrong. Therefore you are a liar Nikos, I'm sorry. From nikos.gr33k at gmail.com Wed Oct 2 07:55:10 2013 From: nikos.gr33k at gmail.com (=?UTF-8?B?zp3Or866zr/Pgg==?=) Date: Wed, 02 Oct 2013 14:55:10 +0300 Subject: JUST GOT HACKED In-Reply-To: <9208d479-21ff-4da1-86f0-35cf46b13505@googlegroups.com> References: <9208d479-21ff-4da1-86f0-35cf46b13505@googlegroups.com> Message-ID: ???? 2/10/2013 2:42 ??, ?/? feedthetroll at gmx.de ??????: > Am Mittwoch, 2. Oktober 2013 12:52:39 UTC+2 schrieb Ferrous Cranus: >> ... >>> By the way: if you haven't already, you'll want to remove the extra line >>> from your .htaccess file. >> Tell me the line you are referring to. > I think it will be the line YOU did NOT enter. Just take a look yourself. The best way will be to compare it to a backup you made before the change. Except from the additional line i have added to the .htaccess myself there is nothing added extra. Also i'm under the impression Zeus Piraeus was referring at that line(the one i added myself) so to fix my scripts source code which could be seen as plain test. >> I learn Python for personal pleasure because i like programming. > "Learning for personal pleasure" and "business server" can't be true both. > So one of the statements is wrong. Therefore you are a liar Nikos, I'm sorry. Yes they work together. Take my website for example, i have started to leanr Python be reading a few things about it as pleasure and now i have a working website which has secret bultin abilities as well. From steve+comp.lang.python at pearwood.info Wed Oct 2 08:51:11 2013 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: 02 Oct 2013 12:51:11 GMT Subject: JUST GOT HACKED References: <9208d479-21ff-4da1-86f0-35cf46b13505@googlegroups.com> Message-ID: <524c16be$0$29984$c3e8da3$5496439d@news.astraweb.com> On Wed, 02 Oct 2013 04:42:32 -0700, feedthetroll wrote: > "Learning for personal pleasure" and "business server" can't be true > both. Utter nonsense. Some people are fortunate enough to be paid to do something that gives them pleasure. Many programmers and system administrators are in that lucky position. I am sorry that you (apparently) get no pleasure from learning new programming and sys admin skills, but don't imagine that everyone is like that. > So one of the statements is wrong. Therefore you are a liar Nikos, > I'm sorry. You should be sorry, because that was uncalled for as well as wrong. -- Steven From antoon.pardon at rece.vub.ac.be Wed Oct 2 09:12:42 2013 From: antoon.pardon at rece.vub.ac.be (Antoon Pardon) Date: Wed, 02 Oct 2013 15:12:42 +0200 Subject: JUST GOT HACKED In-Reply-To: <524c16be$0$29984$c3e8da3$5496439d@news.astraweb.com> References: <9208d479-21ff-4da1-86f0-35cf46b13505@googlegroups.com> <524c16be$0$29984$c3e8da3$5496439d@news.astraweb.com> Message-ID: <524C1BCA.5050900@rece.vub.ac.be> Op 02-10-13 14:51, Steven D'Aprano schreef: > On Wed, 02 Oct 2013 04:42:32 -0700, feedthetroll wrote: > >> "Learning for personal pleasure" and "business server" can't be true >> both. > > Utter nonsense. Some people are fortunate enough to be paid to do > something that gives them pleasure. Many programmers and system > administrators are in that lucky position. I am sorry that you > (apparently) get no pleasure from learning new programming and sys admin > skills, but don't imagine that everyone is like that. And how do you come to that conclusion? As far as I understand feedthetroll only implied you shouldn't do your *learning* for personal pleasure on the business server. -- Antoon Pardon From nikos.gr33k at gmail.com Wed Oct 2 09:30:01 2013 From: nikos.gr33k at gmail.com (=?UTF-8?B?zp3Or866zr/Pgg==?=) Date: Wed, 02 Oct 2013 16:30:01 +0300 Subject: JUST GOT HACKED In-Reply-To: References: <9208d479-21ff-4da1-86f0-35cf46b13505@googlegroups.com> <524c16be$0$29984$c3e8da3$5496439d@news.astraweb.com> Message-ID: ???? 2/10/2013 4:12 ??, ?/? Antoon Pardon ??????: > Op 02-10-13 14:51, Steven D'Aprano schreef: >> On Wed, 02 Oct 2013 04:42:32 -0700, feedthetroll wrote: >> >>> "Learning for personal pleasure" and "business server" can't be true >>> both. >> >> Utter nonsense. Some people are fortunate enough to be paid to do >> something that gives them pleasure. Many programmers and system >> administrators are in that lucky position. I am sorry that you >> (apparently) get no pleasure from learning new programming and sys admin >> skills, but don't imagine that everyone is like that. > > And how do you come to that conclusion? As far as I understand > feedthetroll only implied you shouldn't do your *learning* for > personal pleasure on the business server. You learn and you are forced to solve problems better when you deal with real time problems. An aprt of that, no matter well prepared you become you still have a lot to learn during the process, so in that context nobody can be "really ready" to start something. You just have some basic understanding and buil up from there. From rodperson at rodperson.com Wed Oct 2 10:31:35 2013 From: rodperson at rodperson.com (Rod Person) Date: Wed, 02 Oct 2013 10:31:35 -0400 Subject: JUST GOT HACKED In-Reply-To: References: <9208d479-21ff-4da1-86f0-35cf46b13505@googlegroups.com> <524c16be$0$29984$c3e8da3$5496439d@news.astraweb.com> Message-ID: <524C2E47.7000209@rodperson.com> On 10/2/2013 9:30 AM, ????? wrote: > You learn and you are forced to solve problems better when you deal > with real time problems. https://tinyurl.com/44teepw -- Rod The guide of millers uses only the finest grains: true Roman breads, for true Romans. From rosuav at gmail.com Wed Oct 2 09:06:40 2013 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 2 Oct 2013 23:06:40 +1000 Subject: JUST GOT HACKED In-Reply-To: <524c16be$0$29984$c3e8da3$5496439d@news.astraweb.com> References: <9208d479-21ff-4da1-86f0-35cf46b13505@googlegroups.com> <524c16be$0$29984$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Wed, Oct 2, 2013 at 10:51 PM, Steven D'Aprano wrote: > On Wed, 02 Oct 2013 04:42:32 -0700, feedthetroll wrote: > >> "Learning for personal pleasure" and "business server" can't be true >> both. > > Utter nonsense. Some people are fortunate enough to be paid to do > something that gives them pleasure. Many programmers and system > administrators are in that lucky position. I am sorry that you > (apparently) get no pleasure from learning new programming and sys admin > skills, but don't imagine that everyone is like that. > There's still a difference between getting paid to do something you enjoy, and getting paid to learn the utter basics. I enjoy networking, and I'm paid to do that (well, part of my job involves networking); this week I learned something new that I can do by simply setting a few ARP entries and some fixed routes, which let us de-hack some sections of our code. That's great! I love doing it, and I learn something, and I get paid to do so. Awesome! But if you hire someone to set you up a LAN, and he's still learning the fundamentals of IP addressing and so on, then you'd be a bit concerned. The terms, as stated, are far too broad. Learning at the level Nikos currently is, though, is IMHO incompatible with being paid to do it. ChrisA From p.johnson125 at gmail.com Tue Oct 8 10:53:42 2013 From: p.johnson125 at gmail.com (Pat Johnson) Date: Tue, 8 Oct 2013 07:53:42 -0700 (PDT) Subject: JUST GOT HACKED In-Reply-To: References: Message-ID: <96e1af71-3ba5-4c99-926b-321dbabe97ac@googlegroups.com> >From what I gather he was viewing files uploaded to the ftp folder and found this warning.html file contained within... So my take on it is, someone just uploaded it and this guy is freaking out making a buffoon out of himself. From arunraja.jaga at gmail.com Tue Oct 1 07:17:31 2013 From: arunraja.jaga at gmail.com (arunraja.jaga at gmail.com) Date: Tue, 1 Oct 2013 04:17:31 -0700 (PDT) Subject: Converting image into string and obtain the string in json Message-ID: How to insert an image in mysql and retrieve the image to pass it as a string to json file. From breamoreboy at yahoo.co.uk Tue Oct 1 09:15:43 2013 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Tue, 01 Oct 2013 14:15:43 +0100 Subject: Converting image into string and obtain the string in json In-Reply-To: References: Message-ID: On 01/10/2013 12:17, arunraja.jaga at gmail.com wrote: > How to insert an image in mysql and retrieve the image to pass it as a string to json file. > You write some code, try to run it and if it fails and you don't understand why, post a code snippet that fails, what you expected to happen, what actually happened and the full traceback if applicable, quoting your OS and Python version. -- Cheers. Mark Lawrence From robin at reportlab.com Tue Oct 1 12:41:59 2013 From: robin at reportlab.com (Robin Becker) Date: Tue, 01 Oct 2013 17:41:59 +0100 Subject: PyDoc_STRVAR error in msvc compile Message-ID: <524AFB57.9090500@chamonix.reportlab.co.uk> I'm trying to port C code that compiles under 2.7 to 3.3; the intention is to make things work under both. I used to use raw strings for the module documentation, but my porting guide suggests that I should be using PyDoc_STRVAR. That works for me using linux, but I'm getting errors under windows compiles My usage looks like this PyDoc_STRVAR(__DOC__, "Helper extension module for xxx.\n\ \n\ Interface summary:\n\ \n\ import _xxx\n\ ......\n" #ifdef ZZZZ " ZZZZ stuff\n" #endif "\n\ .........\n\ "); but it seems that the MSVC compiler for python-2.7 cannot hack this and I get messages like this > _xxx.c > C:\code\hg-repos\test_ext\xxx\_xxx.c(66) : error C2121: '#' : invalid character : possibly the > result of a macro expansion > C:\code\hg-repos\test_ext\xxx\_xxx.c(66) : error C2146: syntax error : missing ';' before ident > ifier 'ifdef' > C:\code\hg-repos\test_ext\xxx\_xxx.c(66) : error C2143: syntax error : missing '{' before 'cons > tant' > C:\code\hg-repos\test_ext\xxx\_xxx.c(66) : error C2059: syntax error : '' > C:\code\hg-repos\test_ext\xxx\_xxx.c(66) : error C2121: '#' : invalid character : possibly the > result of a macro expansion > error: command '"C:\Program Files\Microsoft Visual Studio 9.0\VC\BIN\cl.exe"' failed with exit status 2 line 66 is the closing paranthesis. Anyone got any ideas? Previously I used code like this static char moduleDoc ="Helper extension module for xxx.\n\ \n\ Interface summary:\n\ \n\ import _xxx\n\ ......\n" #ifdef ZZZZ " ZZZZ stuff\n" #endif "\n\ .........\n\ "; but the registration and usage differs quite a lot between python2 & python3 so I thought to switch to the newer mechanism. -- Robin Becker From python at mrabarnett.plus.com Tue Oct 1 13:26:15 2013 From: python at mrabarnett.plus.com (MRAB) Date: Tue, 01 Oct 2013 18:26:15 +0100 Subject: PyDoc_STRVAR error in msvc compile In-Reply-To: <524AFB57.9090500@chamonix.reportlab.co.uk> References: <524AFB57.9090500@chamonix.reportlab.co.uk> Message-ID: <524B05B7.4060906@mrabarnett.plus.com> On 01/10/2013 17:41, Robin Becker wrote: > I'm trying to port C code that compiles under 2.7 to 3.3; the intention is to > make things work under both. > > I used to use raw strings for the module documentation, but my porting guide > suggests that I should be using PyDoc_STRVAR. That works for me using linux, but > I'm getting errors under windows compiles > > My usage looks like this > > > PyDoc_STRVAR(__DOC__, > "Helper extension module for xxx.\n\ > \n\ > Interface summary:\n\ > \n\ > import _xxx\n\ > ......\n" > #ifdef ZZZZ > " ZZZZ stuff\n" > #endif > "\n\ > .........\n\ > "); > > > but it seems that the MSVC compiler for python-2.7 cannot hack this and I get > messages like this > >> _xxx.c >> C:\code\hg-repos\test_ext\xxx\_xxx.c(66) : error C2121: '#' : invalid character : possibly the >> result of a macro expansion >> C:\code\hg-repos\test_ext\xxx\_xxx.c(66) : error C2146: syntax error : missing ';' before ident >> ifier 'ifdef' >> C:\code\hg-repos\test_ext\xxx\_xxx.c(66) : error C2143: syntax error : missing '{' before 'cons >> tant' >> C:\code\hg-repos\test_ext\xxx\_xxx.c(66) : error C2059: syntax error : '' >> C:\code\hg-repos\test_ext\xxx\_xxx.c(66) : error C2121: '#' : invalid character : possibly the >> result of a macro expansion >> error: command '"C:\Program Files\Microsoft Visual Studio 9.0\VC\BIN\cl.exe"' failed with exit status 2 > > line 66 is the closing paranthesis. > > Anyone got any ideas? > > Previously I used code like this > > static char moduleDoc ="Helper extension module for xxx.\n\ > \n\ > Interface summary:\n\ > \n\ > import _xxx\n\ > ......\n" > #ifdef ZZZZ > " ZZZZ stuff\n" > #endif > "\n\ > .........\n\ > "; > > but the registration and usage differs quite a lot between python2 & python3 so > I thought to switch to the newer mechanism. > I've tried it in a minimal console program, and it seems to work for me. From robin at reportlab.com Wed Oct 2 05:00:09 2013 From: robin at reportlab.com (Robin Becker) Date: Wed, 02 Oct 2013 10:00:09 +0100 Subject: PyDoc_STRVAR error in msvc compile In-Reply-To: <524B05B7.4060906@mrabarnett.plus.com> References: <524AFB57.9090500@chamonix.reportlab.co.uk> <524B05B7.4060906@mrabarnett.plus.com> Message-ID: <524BE099.8040004@chamonix.reportlab.co.uk> On 01/10/2013 18:26, MRAB wrote: > On 01/10/2013 17:41, Robin Becker wrote: .......... >> > I've tried it in a minimal console program, and it seems to work for me. > thanks for the test. I thought this might be an issue with the macro call argument being spread out over several lines, but since your try works I'll dig deeper. -- Robin Becker From robin at reportlab.com Wed Oct 2 05:00:09 2013 From: robin at reportlab.com (Robin Becker) Date: Wed, 02 Oct 2013 10:00:09 +0100 Subject: PyDoc_STRVAR error in msvc compile In-Reply-To: <524B05B7.4060906@mrabarnett.plus.com> References: <524AFB57.9090500@chamonix.reportlab.co.uk> <524B05B7.4060906@mrabarnett.plus.com> Message-ID: <524BE099.8040004@chamonix.reportlab.co.uk> On 01/10/2013 18:26, MRAB wrote: > On 01/10/2013 17:41, Robin Becker wrote: .......... >> > I've tried it in a minimal console program, and it seems to work for me. > thanks for the test. I thought this might be an issue with the macro call argument being spread out over several lines, but since your try works I'll dig deeper. -- Robin Becker From robin at reportlab.com Wed Oct 2 06:01:33 2013 From: robin at reportlab.com (Robin Becker) Date: Wed, 02 Oct 2013 11:01:33 +0100 Subject: PyDoc_STRVAR error in msvc compile In-Reply-To: <524BE099.8040004@chamonix.reportlab.co.uk> References: <524AFB57.9090500@chamonix.reportlab.co.uk> <524B05B7.4060906@mrabarnett.plus.com> <524BE099.8040004@chamonix.reportlab.co.uk> Message-ID: <524BEEFD.50202@chamonix.reportlab.co.uk> On 02/10/2013 10:00, Robin Becker wrote: > On 01/10/2013 18:26, MRAB wrote: >> On 01/10/2013 17:41, Robin Becker wrote: > .......... >>> >> I've tried it in a minimal console program, and it seems to work for me. >> > thanks for the test. I thought this might be an issue with the macro call > argument being spread out over several lines, but since your try works I'll dig > deeper. I used -E to get the pre-processor output and it shows my string constant ends up with the #ifdef inside it eg ......\n" #ifdef ZZZZ " ZZZZ stuff\n" #endif "\n\.... I looked for any obvious reason why that should happen, but cannot find anything. if I remove the internal #ifdef and replace with a macro containing the conditional string or "" then MSVC 9 seems to be ok with it. -- Robin Becker From robin at reportlab.com Wed Oct 2 06:01:33 2013 From: robin at reportlab.com (Robin Becker) Date: Wed, 02 Oct 2013 11:01:33 +0100 Subject: PyDoc_STRVAR error in msvc compile In-Reply-To: <524BE099.8040004@chamonix.reportlab.co.uk> References: <524AFB57.9090500@chamonix.reportlab.co.uk> <524B05B7.4060906@mrabarnett.plus.com> <524BE099.8040004@chamonix.reportlab.co.uk> Message-ID: <524BEEFD.50202@chamonix.reportlab.co.uk> On 02/10/2013 10:00, Robin Becker wrote: > On 01/10/2013 18:26, MRAB wrote: >> On 01/10/2013 17:41, Robin Becker wrote: > .......... >>> >> I've tried it in a minimal console program, and it seems to work for me. >> > thanks for the test. I thought this might be an issue with the macro call > argument being spread out over several lines, but since your try works I'll dig > deeper. I used -E to get the pre-processor output and it shows my string constant ends up with the #ifdef inside it eg ......\n" #ifdef ZZZZ " ZZZZ stuff\n" #endif "\n\.... I looked for any obvious reason why that should happen, but cannot find anything. if I remove the internal #ifdef and replace with a macro containing the conditional string or "" then MSVC 9 seems to be ok with it. -- Robin Becker From davea at davea.name Wed Oct 2 06:49:47 2013 From: davea at davea.name (Dave Angel) Date: Wed, 2 Oct 2013 10:49:47 +0000 (UTC) Subject: PyDoc_STRVAR error in msvc compile References: <524AFB57.9090500@chamonix.reportlab.co.uk> <524B05B7.4060906@mrabarnett.plus.com> <524BE099.8040004@chamonix.reportlab.co.uk> <524BEEFD.50202@chamonix.reportlab.co.uk> Message-ID: On 2/10/2013 06:01, Robin Becker wrote: > On 02/10/2013 10:00, Robin Becker wrote: >> On 01/10/2013 18:26, MRAB wrote: >>> On 01/10/2013 17:41, Robin Becker wrote: >> .......... >>>> >>> I've tried it in a minimal console program, and it seems to work for me. >>> >> thanks for the test. I thought this might be an issue with the macro call >> argument being spread out over several lines, but since your try works I'll dig >> deeper. > I used -E to get the pre-processor output and it shows my string constant ends > up with the #ifdef inside it eg > > > ......\n" #ifdef ZZZZ " ZZZZ stuff\n" #endif "\n\.... > > I looked for any obvious reason why that should happen, but cannot find anything. > > if I remove the internal #ifdef and replace with a macro containing the > conditional string or "" then MSVC 9 seems to be ok with it. > MSVC and other compilers do not not see eye to eye on the preprocessor semantics. I no longer use MSVC so I can't experiment. I can only try to recall extensive manipulation two decades ago. I believe it does the logic of "backslash at the end of line" first. So if there are any spaces or tabs after those backslashes (which might have been lost when you pasted it here), fix them first. Then I think it looks for macro definitions, where the # must be the first non-whitespace of the line. Then it expands such macros, and I think MSVC is unusual in that it expands them multiple times, so a macro expansion can result in another macro invocation. I'm not sure where quotes fit in here. Your original message code doesn't match the expansion you show with -E, so i suspect your "..." eliding hid something significant. -- DaveA From robin at reportlab.com Wed Oct 2 07:28:31 2013 From: robin at reportlab.com (Robin Becker) Date: Wed, 02 Oct 2013 12:28:31 +0100 Subject: PyDoc_STRVAR error in msvc compile In-Reply-To: References: <524AFB57.9090500@chamonix.reportlab.co.uk> <524B05B7.4060906@mrabarnett.plus.com> <524BE099.8040004@chamonix.reportlab.co.uk> <524BEEFD.50202@chamonix.reportlab.co.uk> Message-ID: <524C035F.6030000@chamonix.reportlab.co.uk> On 02/10/2013 11:49, Dave Angel wrote: >> conditional string or "" then MSVC 9 seems to be ok with it. >> > > MSVC and other compilers do not not see eye to eye on the preprocessor > semantics. I no longer use MSVC so I can't experiment. I can only try > to recall extensive manipulation two decades ago. > > I believe it does the logic of "backslash at the end of line" first. So > if there are any spaces or tabs after those backslashes (which might > have been lost when you pasted it here), fix them first. > > Then I think it looks for macro definitions, where the # must be the > first non-whitespace of the line. Then it expands such macros, and I > think MSVC is unusual in that it expands them multiple times, so a macro > expansion can result in another macro invocation. > > I'm not sure where quotes fit in here. > > Your original message code doesn't match the expansion you show with -E, > so i suspect your "..." eliding hid something significant. The actual is this code from _renderPM.c https://bitbucket.org/rptlab/reportlab/src/fa65fe72b6c2aaecb7747bf14884adb996d8e87f/src/rl_addons/renderPM/_renderPM.c?at=default PyDoc_STRVAR(__DOC__, "Helper extension module for renderPM.\n\ \n\ Interface summary:\n\ \n\ import _renderPM\n\ gstate(width,height[,depth=3,bg=0xffffff]) #create an initialised graphics state\n\ makeT1Font(fontName,pfbPath,names[,reader]) #make a T1 font\n\ delCache() #delete all T1 font info\n\ pil2pict(cols,rows,datastr,palette) hreturn PICT version of im as bytes\n" #ifdef RENDERPM_FT " ft_get_face(fontName) --> ft_face instance\n" #endif "\n\ _libart_version # base library version string\n\ _version # module version string\n\ "); when I run that through the pre-processor I get (all on a single line) static char __DOC__[] = "Helper extension module for renderPM.\n\nInterface summary:\n\n import _renderPM\n gstate(width,height[,depth=3,bg=0xffffff]) #create an initialised graphics state\n makeT1Font(fontName,pfbPath,names[,reader]) #make a T1 font\n delCache() #delete all T1 font info\n pil2pict(cols,rows,datastr,palette) hreturn PICT version of im as bytes\n" #ifdef 1 " ft_get_face(fontName) --> ft_face instance\n" #endif "\n _libart_version # base library version string\n _version # module version string\n"; I tried a couple of variations of \ at the end of the line preceding #ifdef etc etc, but nothing seemed to work. The source is properly DOS formatted (according to vim) so it's not a simple line ending issue and I don't have any extra spaces at the end of the lines etc etc. -- Robin Becker From robin at reportlab.com Wed Oct 2 07:28:31 2013 From: robin at reportlab.com (Robin Becker) Date: Wed, 02 Oct 2013 12:28:31 +0100 Subject: PyDoc_STRVAR error in msvc compile In-Reply-To: References: <524AFB57.9090500@chamonix.reportlab.co.uk> <524B05B7.4060906@mrabarnett.plus.com> <524BE099.8040004@chamonix.reportlab.co.uk> <524BEEFD.50202@chamonix.reportlab.co.uk> Message-ID: <524C035F.6030000@chamonix.reportlab.co.uk> On 02/10/2013 11:49, Dave Angel wrote: >> conditional string or "" then MSVC 9 seems to be ok with it. >> > > MSVC and other compilers do not not see eye to eye on the preprocessor > semantics. I no longer use MSVC so I can't experiment. I can only try > to recall extensive manipulation two decades ago. > > I believe it does the logic of "backslash at the end of line" first. So > if there are any spaces or tabs after those backslashes (which might > have been lost when you pasted it here), fix them first. > > Then I think it looks for macro definitions, where the # must be the > first non-whitespace of the line. Then it expands such macros, and I > think MSVC is unusual in that it expands them multiple times, so a macro > expansion can result in another macro invocation. > > I'm not sure where quotes fit in here. > > Your original message code doesn't match the expansion you show with -E, > so i suspect your "..." eliding hid something significant. The actual is this code from _renderPM.c https://bitbucket.org/rptlab/reportlab/src/fa65fe72b6c2aaecb7747bf14884adb996d8e87f/src/rl_addons/renderPM/_renderPM.c?at=default PyDoc_STRVAR(__DOC__, "Helper extension module for renderPM.\n\ \n\ Interface summary:\n\ \n\ import _renderPM\n\ gstate(width,height[,depth=3,bg=0xffffff]) #create an initialised graphics state\n\ makeT1Font(fontName,pfbPath,names[,reader]) #make a T1 font\n\ delCache() #delete all T1 font info\n\ pil2pict(cols,rows,datastr,palette) hreturn PICT version of im as bytes\n" #ifdef RENDERPM_FT " ft_get_face(fontName) --> ft_face instance\n" #endif "\n\ _libart_version # base library version string\n\ _version # module version string\n\ "); when I run that through the pre-processor I get (all on a single line) static char __DOC__[] = "Helper extension module for renderPM.\n\nInterface summary:\n\n import _renderPM\n gstate(width,height[,depth=3,bg=0xffffff]) #create an initialised graphics state\n makeT1Font(fontName,pfbPath,names[,reader]) #make a T1 font\n delCache() #delete all T1 font info\n pil2pict(cols,rows,datastr,palette) hreturn PICT version of im as bytes\n" #ifdef 1 " ft_get_face(fontName) --> ft_face instance\n" #endif "\n _libart_version # base library version string\n _version # module version string\n"; I tried a couple of variations of \ at the end of the line preceding #ifdef etc etc, but nothing seemed to work. The source is properly DOS formatted (according to vim) so it's not a simple line ending issue and I don't have any extra spaces at the end of the lines etc etc. -- Robin Becker From davea at davea.name Wed Oct 2 08:05:02 2013 From: davea at davea.name (Dave Angel) Date: Wed, 2 Oct 2013 12:05:02 +0000 (UTC) Subject: PyDoc_STRVAR error in msvc compile References: <524AFB57.9090500@chamonix.reportlab.co.uk> <524B05B7.4060906@mrabarnett.plus.com> <524BE099.8040004@chamonix.reportlab.co.uk> <524BEEFD.50202@chamonix.reportlab.co.uk> <524C035F.6030000@chamonix.reportlab.co.uk> Message-ID: On 2/10/2013 07:28, Robin Becker wrote: > The actual is this code from _renderPM.c > > https://bitbucket.org/rptlab/reportlab/src/fa65fe72b6c2aaecb7747bf14884adb996d8e87f/src/rl_addons/renderPM/_renderPM.c?at=default > > PyDoc_STRVAR(__DOC__, > "Helper extension module for renderPM.\n\ > \n\ > Interface summary:\n\ > \n\ > import _renderPM\n\ > gstate(width,height[,depth=3,bg=0xffffff]) #create an initialised > graphics state\n\ > makeT1Font(fontName,pfbPath,names[,reader]) #make a T1 font\n\ > delCache() #delete all T1 font info\n\ > pil2pict(cols,rows,datastr,palette) hreturn PICT version of im as bytes\n" > #ifdef RENDERPM_FT > " ft_get_face(fontName) --> ft_face instance\n" > #endif > "\n\ > _libart_version # base library version string\n\ > _version # module version string\n\ > "); > > when I run that through the pre-processor I get (all on a single line) > > > static char __DOC__[] = "Helper extension module for renderPM.\n\nInterface > summary:\n\n import _renderPM\n gstate(width,height[,depth=3,bg=0xffffff]) > #create an initialised graphics state\n > makeT1Font(fontName,pfbPath,names[,reader]) #make a T1 font\n delCache() #delete > all T1 font info\n pil2pict(cols,rows,datastr,palette) hreturn PICT version of > im as bytes\n" #ifdef 1 " ft_get_face(fontName) --> ft_face instance\n" > #endif "\n _libart_version # base library version string\n _version # module > version string\n"; > > > > I tried a couple of variations of \ at the end of the line preceding #ifdef etc > etc, but nothing seemed to work. The source is properly DOS formatted (according > to vim) so it's not a simple line ending issue and I don't have any extra spaces > at the end of the lines etc etc. Unfortunately, bitbucket doesn't properly support highlighting either, so I had to copy/paste it into an editor to check for extra spaces. That's apparently not your problem. What I didn't understand before is that PyDoc_STRVAR is a macro, not a function. And inside the macro's parameters, you're trying to define an #ifdef. i don't think Microsoft supports that. If I'm right, you need to separate out the conditional string concatenation from the macro expansion. it's been too long for me even to remember the correct way to do that. There are some legal tricks you can use. Maybe search the internet for "preprocessor stringizing". -- DaveA From robin at reportlab.com Wed Oct 2 09:46:36 2013 From: robin at reportlab.com (Robin Becker) Date: Wed, 02 Oct 2013 14:46:36 +0100 Subject: PyDoc_STRVAR error in msvc compile In-Reply-To: References: <524AFB57.9090500@chamonix.reportlab.co.uk> <524B05B7.4060906@mrabarnett.plus.com> <524BE099.8040004@chamonix.reportlab.co.uk> <524BEEFD.50202@chamonix.reportlab.co.uk> <524C035F.6030000@chamonix.reportlab.co.uk> Message-ID: <524C23BC.6020806@chamonix.reportlab.co.uk> On 02/10/2013 13:05, Dave Angel wrote: > On 2/10/2013 07:28, Robin Becker wrote: > > >> The actual is this code from _renderPM.c >> >> https://bitbucket.org/rptlab/reportlab/src/fa65fe72b6c2aaecb7747bf14884adb996d8e87f/src/rl_addons/renderPM/_renderPM.c?at=default >> ........ >> at the end of the lines etc etc. > > Unfortunately, bitbucket doesn't properly support highlighting either, > so I had to copy/paste it into an editor to check for extra spaces. > That's apparently not your problem. > > What I didn't understand before is that PyDoc_STRVAR is a macro, not a > function. And inside the macro's parameters, you're trying to define an > #ifdef. i don't think Microsoft supports that. > > If I'm right, you need to separate out the conditional string > concatenation from the macro expansion. it's been too long for me even > to remember the correct way to do that. There are some legal tricks you > can use. Maybe search the internet for "preprocessor stringizing". . that's what I have done; it seems to work OK for MSVC and I'll have to check later if it breaks GCC or if I need to do token pasting or something. -- Robin Becker From robin at reportlab.com Wed Oct 2 09:46:36 2013 From: robin at reportlab.com (Robin Becker) Date: Wed, 02 Oct 2013 14:46:36 +0100 Subject: PyDoc_STRVAR error in msvc compile In-Reply-To: References: <524AFB57.9090500@chamonix.reportlab.co.uk> <524B05B7.4060906@mrabarnett.plus.com> <524BE099.8040004@chamonix.reportlab.co.uk> <524BEEFD.50202@chamonix.reportlab.co.uk> <524C035F.6030000@chamonix.reportlab.co.uk> Message-ID: <524C23BC.6020806@chamonix.reportlab.co.uk> On 02/10/2013 13:05, Dave Angel wrote: > On 2/10/2013 07:28, Robin Becker wrote: > > >> The actual is this code from _renderPM.c >> >> https://bitbucket.org/rptlab/reportlab/src/fa65fe72b6c2aaecb7747bf14884adb996d8e87f/src/rl_addons/renderPM/_renderPM.c?at=default >> ........ >> at the end of the lines etc etc. > > Unfortunately, bitbucket doesn't properly support highlighting either, > so I had to copy/paste it into an editor to check for extra spaces. > That's apparently not your problem. > > What I didn't understand before is that PyDoc_STRVAR is a macro, not a > function. And inside the macro's parameters, you're trying to define an > #ifdef. i don't think Microsoft supports that. > > If I'm right, you need to separate out the conditional string > concatenation from the macro expansion. it's been too long for me even > to remember the correct way to do that. There are some legal tricks you > can use. Maybe search the internet for "preprocessor stringizing". . that's what I have done; it seems to work OK for MSVC and I'll have to check later if it breaks GCC or if I need to do token pasting or something. -- Robin Becker From python at mrabarnett.plus.com Wed Oct 2 11:18:03 2013 From: python at mrabarnett.plus.com (MRAB) Date: Wed, 02 Oct 2013 16:18:03 +0100 Subject: PyDoc_STRVAR error in msvc compile In-Reply-To: <524C035F.6030000@chamonix.reportlab.co.uk> References: <524AFB57.9090500@chamonix.reportlab.co.uk> <524B05B7.4060906@mrabarnett.plus.com> <524BE099.8040004@chamonix.reportlab.co.uk> <524BEEFD.50202@chamonix.reportlab.co.uk> <524C035F.6030000@chamonix.reportlab.co.uk> Message-ID: <524C392B.3090003@mrabarnett.plus.com> On 02/10/2013 12:28, Robin Becker wrote: > On 02/10/2013 11:49, Dave Angel wrote: >>> conditional string or "" then MSVC 9 seems to be ok with it. >>> > >> MSVC and other compilers do not not see eye to eye on the preprocessor >> semantics. I no longer use MSVC so I can't experiment. I can only try >> to recall extensive manipulation two decades ago. >> >> I believe it does the logic of "backslash at the end of line" first. So >> if there are any spaces or tabs after those backslashes (which might >> have been lost when you pasted it here), fix them first. >> >> Then I think it looks for macro definitions, where the # must be the >> first non-whitespace of the line. Then it expands such macros, and I >> think MSVC is unusual in that it expands them multiple times, so a macro >> expansion can result in another macro invocation. >> >> I'm not sure where quotes fit in here. >> >> Your original message code doesn't match the expansion you show with -E, >> so i suspect your "..." eliding hid something significant. > > The actual is this code from _renderPM.c > > https://bitbucket.org/rptlab/reportlab/src/fa65fe72b6c2aaecb7747bf14884adb996d8e87f/src/rl_addons/renderPM/_renderPM.c?at=default > > PyDoc_STRVAR(__DOC__, > "Helper extension module for renderPM.\n\ > \n\ > Interface summary:\n\ > \n\ > import _renderPM\n\ > gstate(width,height[,depth=3,bg=0xffffff]) #create an initialised > graphics state\n\ > makeT1Font(fontName,pfbPath,names[,reader]) #make a T1 font\n\ > delCache() #delete all T1 font info\n\ > pil2pict(cols,rows,datastr,palette) hreturn PICT version of im as bytes\n" > #ifdef RENDERPM_FT > " ft_get_face(fontName) --> ft_face instance\n" > #endif > "\n\ > _libart_version # base library version string\n\ > _version # module version string\n\ > "); > > when I run that through the pre-processor I get (all on a single line) > > > static char __DOC__[] = "Helper extension module for renderPM.\n\nInterface > summary:\n\n import _renderPM\n gstate(width,height[,depth=3,bg=0xffffff]) > #create an initialised graphics state\n > makeT1Font(fontName,pfbPath,names[,reader]) #make a T1 font\n delCache() #delete > all T1 font info\n pil2pict(cols,rows,datastr,palette) hreturn PICT version of > im as bytes\n" #ifdef 1 " ft_get_face(fontName) --> ft_face instance\n" > #endif "\n _libart_version # base library version string\n _version # module > version string\n"; > > > > I tried a couple of variations of \ at the end of the line preceding #ifdef etc > etc, but nothing seemed to work. The source is properly DOS formatted (according > to vim) so it's not a simple line ending issue and I don't have any extra spaces > at the end of the lines etc etc. > I overlooked that PyDoc_STRVAR is a macro, so, no, it doesn't work for me either. From vasudevram at gmail.com Tue Oct 1 16:03:22 2013 From: vasudevram at gmail.com (vasudevram) Date: Tue, 1 Oct 2013 13:03:22 -0700 (PDT) Subject: Convert Microsoft Word files to PDF with DOCXtoPDF Message-ID: Hi list, I hope some people may find this useful. This post by me shows how to use DOCXtoPDF (a program I wrote recently) to convert the text in Microsoft Word files (that are in DOCX format) to PDF: Convert Microsoft Word files to PDF with DOCXtoPDF http://jugad2.blogspot.in/2013/10/convert-microsoft-word-files-to-pdf.html The prerequisites are also mentioned. They are: Python, Reportlab, my xtopdf toolkit and python-docx. ------------------------------------------------------- - Vasudev Ram Dancing Bison Enterprises Software training and consulting - Python, Linux, databases, open source ... http://www.dancingbison.com http://jugad2.blogspot.com ------------------------------------------------------- From devyncjohnson at gmail.com Tue Oct 1 16:36:01 2013 From: devyncjohnson at gmail.com (Devyn Collier Johnson) Date: Tue, 01 Oct 2013 16:36:01 -0400 Subject: Aloha! Check out the Betabots! Message-ID: <524B3231.90105@Gmail.com> Aloha Python Fans! Did you all miss me? I have been busy working on my Python-based chatbots called the Betabots. I mentioned this mailing list and thanked the Python mailing list in the documentation like I said I would. I just released v0.8a. Enjoy! Thanks for all the past help everyone. Mahalo, Devyn Collier Johnson DevynCJohnson at Gmail.com From tjreedy at udel.edu Tue Oct 1 17:30:41 2013 From: tjreedy at udel.edu (Terry Reedy) Date: Tue, 01 Oct 2013 17:30:41 -0400 Subject: Tail recursion to while iteration in 2 easy steps Message-ID: Part of the reason that Python does not do tail call optimization is that turning tail recursion into while iteration is almost trivial, once you know the secret of the two easy steps. Here it is. Assume that you have already done the work of turning a body recursive ('not tail recursive') form like def fact(n): return 1 if n <= 1 else n * fact(n-1) into a tail recursion like def fact(n, _fac=1): '''Return factorial for any count n. Users are not expected to override private parameter _fac. ''' if n <= 1: return _fac else: # optional return fact(n-1, n * _fac) (This conversion nearly requires adding an accumulator parameter, as done here. Turn this into while iteration with two easy steps. 1. If not already done, make if-else a statement, rather than an expression, with the recursive call in the if branch. If nothing else, just use 'not condition' to invert the condition. def fact(n, _fac=1): if n > 1: # not n <= 1 return fact(n-1, n * _fac) else: # optional return _fac While contrary to what is often published, this order makes logical sense. The recursive call means 'go to the top of this function with new bindings for the parameters'. So put it closest to the top. The base case means 'we are done, time to leave'. So put it at the bottom. 2. This order also makes the follow substeps work: 2a. Replace 'if' with 'while'. 2b. Replace the recursive call with multiple assignment, using the parameters as targets and the arguments as source. For our example: def fact(n, _fac=1): while n > 1: n, _fac = n-1, n * _fac else: return _fac The proof of correctness for this conversion might argue that the recursive form is equivalent to the following pseudo-Python: def fact(n, _fac=1): label top if n > 1: n, _fac = n-1, n * _fac goto top else: return _fac and that this is equivalent to the real Python with while. At this point, you call pull the initialization of the private parameter into the body, remove the else, and even add a value check. def fact(n): if n < 0 or n != int(n): raise ValueError('fact input {} is not a count'.format(n)) fac = 1 while n > 1: n, fac = n-1, n * fac return fac With care, the multiple-assignment statement can usually be turned into multiple assignment statements for greater efficiency. fac *= n n -= 1 But note that the reverse order would be a bug. So I would not do this, especially in more complicated situations, without having tests first. -- Terry Jan Reedy From rustompmody at gmail.com Wed Oct 2 01:16:08 2013 From: rustompmody at gmail.com (rusi) Date: Tue, 1 Oct 2013 22:16:08 -0700 (PDT) Subject: Tail recursion to while iteration in 2 easy steps In-Reply-To: References: Message-ID: <22277f09-f230-4cc9-914e-0b4b63fc0060@googlegroups.com> On Wednesday, October 2, 2013 3:00:41 AM UTC+5:30, Terry Reedy wrote: > Part of the reason that Python does not do tail call optimization is > that turning tail recursion into while iteration is almost trivial, once > you know the secret of the two easy steps. Here it is. What happens for mutual tail recursive code like this def isodd(x) : return False if x==0 else iseven(x-1) def iseven(x): return True if x==0 else isodd(x-1) ---------------- Notes: 1. I am not advocating writing odd/even like the above -- just giving a trivial example 2. General mutual structural (what you call 'body') recursion is more general than mutual tail recursion is more general than single tail recursion. 3. IOW tail recursion optimization is intermediate between the optimization you are showing and the maximally pessimistic implementation -- stack, activation record etc -- of programming languages 4. There is a whole spectrum of such optimizaitons -- 4a eg a single-call structural recursion example, does not need to push return address on the stack. It only needs to store the recursion depth: If zero jump to outside return add; if > 0 jump to internal return address 4b An example like quicksort in which one call is a tail call can be optimized with your optimization and the other, inner one with 4a above 5. Programming language implementations dont do optimizations like TR because these analyses are in themselves hard but because inter-procedural analysis per se is a headache. For python-like languages its a much bigger headache because the recursive name must be dynamically fetched for every call 6. [Personal pov] TR optimization is not really a big beef for me: As you can see, it does not figure in the "All things every programmer should learn from FP" list of mine http://blog.languager.org/2012/10/functional-programming-lost-booty.html 7. Pedagogically, understanding general recursion is far more important than exploiting tail recursion From alain at dpt-info.u-strasbg.fr Wed Oct 2 04:23:46 2013 From: alain at dpt-info.u-strasbg.fr (Alain Ketterlin) Date: Wed, 02 Oct 2013 10:23:46 +0200 Subject: Tail recursion to while iteration in 2 easy steps References: <22277f09-f230-4cc9-914e-0b4b63fc0060@googlegroups.com> Message-ID: <87d2noaxnx.fsf@dpt-info.u-strasbg.fr> rusi writes: > On Wednesday, October 2, 2013 3:00:41 AM UTC+5:30, Terry Reedy wrote: >> Part of the reason that Python does not do tail call optimization is >> that turning tail recursion into while iteration is almost trivial, once >> you know the secret of the two easy steps. Here it is. > > What happens for mutual tail recursive code like this > > def isodd(x) : return False if x==0 else iseven(x-1) > def iseven(x): return True if x==0 else isodd(x-1) It takes one step of inlining to make Terry's technique applicable. (Actually, out of curiosity, I tried this with gcc 4.6.3: the compiler does 16 levels of inlining, plus tail call optimization. The final code has no call.) -- Alain. From rustompmody at gmail.com Wed Oct 2 09:29:39 2013 From: rustompmody at gmail.com (rusi) Date: Wed, 2 Oct 2013 06:29:39 -0700 (PDT) Subject: Tail recursion to while iteration in 2 easy steps In-Reply-To: <87d2noaxnx.fsf@dpt-info.u-strasbg.fr> References: <22277f09-f230-4cc9-914e-0b4b63fc0060@googlegroups.com> <87d2noaxnx.fsf@dpt-info.u-strasbg.fr> Message-ID: On Wednesday, October 2, 2013 1:53:46 PM UTC+5:30, Alain Ketterlin wrote: > rusi writes: > > > > > On Wednesday, October 2, 2013 3:00:41 AM UTC+5:30, Terry Reedy wrote: > >> Part of the reason that Python does not do tail call optimization is > >> that turning tail recursion into while iteration is almost trivial, once > >> you know the secret of the two easy steps. Here it is. > > > > > What happens for mutual tail recursive code like this > > > > def isodd(x) : return False if x==0 else iseven(x-1) > > def iseven(x): return True if x==0 else isodd(x-1) > > > It takes one step of inlining to make Terry's technique applicable. Well I did say my example was trivial! For a more nontrivial case consider the delta function of an FSM. The cleanest way of doing it in C is to make one function with a goto label for each state and a goto for each transition The same thing can be done in a TR optimized language like scheme by making each state into a function and each transition into a TR-call > > > > (Actually, out of curiosity, I tried this with gcc 4.6.3: the compiler > does 16 levels of inlining, plus tail call optimization. The final code > has no call.) Good to know. From ganeshsahni07 at gmail.com Thu Oct 3 13:27:48 2013 From: ganeshsahni07 at gmail.com (Ravi Sahni) Date: Thu, 3 Oct 2013 22:57:48 +0530 Subject: Tail recursion to while iteration in 2 easy steps In-Reply-To: <22277f09-f230-4cc9-914e-0b4b63fc0060@googlegroups.com> References: <22277f09-f230-4cc9-914e-0b4b63fc0060@googlegroups.com> Message-ID: On Wed, Oct 2, 2013 at 10:46 AM, rusi wrote: > 4. There is a whole spectrum of such optimizaitons -- > 4a eg a single-call structural recursion example, does not need to push return address on the stack. It only needs to store the recursion depth: > > If zero jump to outside return add; if > 0 jump to internal return address > > 4b An example like quicksort in which one call is a tail call can be optimized with your optimization and the other, inner one with 4a above I am interested in studying more this 'whole spectrum of optimizations' Any further pointers? Thanks -- Ravi From rustompmody at gmail.com Fri Oct 4 06:52:30 2013 From: rustompmody at gmail.com (rusi) Date: Fri, 4 Oct 2013 03:52:30 -0700 (PDT) Subject: Tail recursion to while iteration in 2 easy steps In-Reply-To: References: <22277f09-f230-4cc9-914e-0b4b63fc0060@googlegroups.com> Message-ID: On Thursday, October 3, 2013 10:57:48 PM UTC+5:30, Ravi Sahni wrote: > On Wed, Oct 2, 2013 at 10:46 AM, rusi wrote: > > 4. There is a whole spectrum of such optimizaitons -- > > 4a eg a single-call structural recursion example, does not need to push return address on the stack. It only needs to store the recursion depth: > > > > If zero jump to outside return add; if > 0 jump to internal return address > > > > 4b An example like quicksort in which one call is a tail call can be optimized with your optimization and the other, inner one with 4a above > > > I am interested in studying more this 'whole spectrum of optimizations' > Any further pointers? Mmm? Bummer? There was a book having these -- at least 5-6 different optimizations of which tail-call was the most obvious. Maybe author was McGettrick dont remember for sure... Probably 20 years now! I'll try and re-remember them and post. From alain at dpt-info.u-strasbg.fr Wed Oct 2 04:17:45 2013 From: alain at dpt-info.u-strasbg.fr (Alain Ketterlin) Date: Wed, 02 Oct 2013 10:17:45 +0200 Subject: Tail recursion to while iteration in 2 easy steps References: Message-ID: <87had0axxy.fsf@dpt-info.u-strasbg.fr> Terry Reedy writes: > Part of the reason that Python does not do tail call optimization is > that turning tail recursion into while iteration is almost trivial, > once you know the secret of the two easy steps. Here it is. > > Assume that you have already done the work of turning a body recursive > ('not tail recursive') form like > > def fact(n): return 1 if n <= 1 else n * fact(n-1) > > into a tail recursion like [...] How do know that either "<=" or "*" didn't rebind the name "fact" to something else? I think that's the main reason why python cannot apply any procedural optimization (even things like inlining are impossible, or possible only under very conservative assumption, that make it worthless). -- Alain. P/S: don't take me wrong; your explanation is excellent (and very useful to python programmers). What I say is that it relies on assumptions that do not apply to python. From dreamingforward at gmail.com Wed Oct 2 14:59:49 2013 From: dreamingforward at gmail.com (Mark Janssen) Date: Wed, 2 Oct 2013 11:59:49 -0700 Subject: Tail recursion to while iteration in 2 easy steps In-Reply-To: <87had0axxy.fsf@dpt-info.u-strasbg.fr> References: <87had0axxy.fsf@dpt-info.u-strasbg.fr> Message-ID: >> def fact(n): return 1 if n <= 1 else n * fact(n-1) >> >> into a tail recursion like > [...] > > How do know that either "<=" or "*" didn't rebind the name "fact" to > something else? I think that's the main reason why python cannot apply > any procedural optimization (even things like inlining are impossible, > or possible only under very conservative assumption, that make it > worthless). It's called "operator precedence". -- MarkJ Tacoma, Washington From dreamingforward at gmail.com Wed Oct 2 17:05:17 2013 From: dreamingforward at gmail.com (Mark Janssen) Date: Wed, 2 Oct 2013 14:05:17 -0700 Subject: Tail recursion to while iteration in 2 easy steps In-Reply-To: <524C80B6.3010204@unistra.fr> References: <87had0axxy.fsf@dpt-info.u-strasbg.fr> <524C80B6.3010204@unistra.fr> Message-ID: On Wed, Oct 2, 2013 at 1:23 PM, Alain Ketterlin wrote: > On 10/02/2013 08:59 PM, Mark Janssen wrote: >>>> >>>> def fact(n): return 1 if n <= 1 else n * fact(n-1) >>> >>> How do know that either "<=" or "*" didn't rebind the name "fact" to >>> something else? I think that's the main reason why python cannot apply >>> any procedural optimization >> >> It's called "operator precedence". > > Operator precedence is totally irrelevant here, you misunderstand what > "bind" means. > > Imagine that you call fact(x) where x is an instance of the following class: > > class Strange: > ... > def __le__(dummy): > global fact > fact = someotherfun # this is "binding" > return false > > i.e., executing "n<=1" calls __le__, which rebinds the name "fact" to > someting else. Then, there is no recursive call at all. At the time > fact(x-1) is executed, "fact" is not the same function any more. > > You cannot prevent this in python. No, but you can't prevent a lot of bad moves in python. What you just did there is a total bonehead ("strange"?) of an idea. -- MarkJ Tacoma, Washington From alain at dpt-info.u-strasbg.fr Fri Oct 4 05:49:26 2013 From: alain at dpt-info.u-strasbg.fr (Alain Ketterlin) Date: Fri, 04 Oct 2013 11:49:26 +0200 Subject: Tail recursion to while iteration in 2 easy steps References: <87had0axxy.fsf@dpt-info.u-strasbg.fr> <524C80B6.3010204@unistra.fr> Message-ID: <87li292wnt.fsf@dpt-info.u-strasbg.fr> Mark Janssen writes: >>>>> def fact(n): return 1 if n <= 1 else n * fact(n-1) >> class Strange: >> ... >> def __le__(dummy): >> global fact >> fact = someotherfun # this is "binding" >> return false >> You cannot prevent this in python. > No, but you can't prevent a lot of bad moves in python. What you just > did there is a total bonehead ("strange"?) of an idea. I think allowing rebinding of function names is extremely strange, even though it's easier to implement. I tend to agree with you on the quality of the idea, but optimization has to take this into account (and give up, usually). -- Alain. From steve+comp.lang.python at pearwood.info Fri Oct 4 06:51:08 2013 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: 04 Oct 2013 10:51:08 GMT Subject: Tail recursion to while iteration in 2 easy steps References: <87had0axxy.fsf@dpt-info.u-strasbg.fr> <524C80B6.3010204@unistra.fr> <87li292wnt.fsf@dpt-info.u-strasbg.fr> Message-ID: <524e9d9c$0$29984$c3e8da3$5496439d@news.astraweb.com> On Fri, 04 Oct 2013 11:49:26 +0200, Alain Ketterlin wrote: > I think allowing rebinding of function names is extremely strange, It's not, it's quite common. Functions in Python are first-class values, and we can do things like this: from somelibrary import somethingwithalonglongname as shortname def inorder(tree, op=print): # Walk the tree in infix order, doing op to each node. process(tree.left, op) op(tree.payload) process(tree.right, op) _len = len def len(obj): do_something_first() return _len(obj) Now, the first two aren't the same sort of function-rebinding that you're talking about, or that were shown in your post, but the third is. What is unusual though is rebinding a function from within itself: def func(arg): global func do_this(arg) def func(arg): do_that(arg) but it's legal and it sometimes can be useful, although it counts as "clever code", possibly "too clever". -- Steven From tjreedy at udel.edu Fri Oct 4 18:32:22 2013 From: tjreedy at udel.edu (Terry Reedy) Date: Fri, 04 Oct 2013 18:32:22 -0400 Subject: Tail recursion to while iteration in 2 easy steps In-Reply-To: <87li292wnt.fsf@dpt-info.u-strasbg.fr> References: <87had0axxy.fsf@dpt-info.u-strasbg.fr> <524C80B6.3010204@unistra.fr> <87li292wnt.fsf@dpt-info.u-strasbg.fr> Message-ID: On 10/4/2013 5:49 AM, Alain Ketterlin wrote: > I think allowing rebinding of function names is extremely strange, Steven already countered the 'is extremely strange' part by showing that such rebinding is common, generally useful, and only occasionally dodgy and a candidate for being blocked. I want to consider here what it would mean to concretely implement the abstract notion 'disallow rebinding of function names' and show what would be behind calling the idea 'not feasible'. 1. A 'name' is not a 'function name' unless the name is bound, at runtime, to a 'function'. 2. A 'function' in Python is not just one class but any callable object -- with with a __call__ methods. That comprises an unbounded set. 3. Python does not mandate how namespaces are implemented. CPython uses both dicts and, for function local namespaces, internal C arrays. So 'names' in code can become either string keys for dicts or integer indexes for arrays. 4. Rebinding can be explicit ('='), implicit ('import', 'class', 'def'), *or hidden* (globals('a') = 1; ob.__dict__('a') = 1). The 'hidden' methods are intentional as they are sometimes needed*. In CPython, these forms remain different in the byte code, but it could be otherwise. The point is that is may or may not be possible for the interpreter to even recognize a 'rebinding' in order to apply any rebinding blocking rule. * There is no trick (that I know of) for hidden rebinding of function locals and nonlocals (short of using ctypes), but I am not sure that this is a language guarantee. However, I an sure that the absence is intentional. > even though it's easier to implement. No kidding. -- Terry Jan Reedy From alain at dpt-info.u-strasbg.fr Mon Oct 7 13:15:35 2013 From: alain at dpt-info.u-strasbg.fr (Alain Ketterlin) Date: Mon, 07 Oct 2013 19:15:35 +0200 Subject: Tail recursion to while iteration in 2 easy steps References: <87had0axxy.fsf@dpt-info.u-strasbg.fr> <524C80B6.3010204@unistra.fr> <87li292wnt.fsf@dpt-info.u-strasbg.fr> Message-ID: <878uy52ea0.fsf@dpt-info.u-strasbg.fr> Terry Reedy writes: > On 10/4/2013 5:49 AM, Alain Ketterlin wrote: > >> I think allowing rebinding of function names is extremely strange, > > Steven already countered the 'is extremely strange' part by showing > that such rebinding is common, generally useful, and only occasionally > dodgy and a candidate for being blocked. I was talking about rebinding a "function name", not about rebinding a name to a function. Steve's message was pretty clear on this: iirc, his first two cases were "binding a new name to an existing callable", the last case (rebinding len) was in line with what I meant (except his code "saved" the original function). My example was: the code of "fact" contains a call to "fact", which is not guaranteed to be bound to the function it appears in. And this prevents any kind of optimization. > I want to consider here what it would mean to concretely implement the > abstract notion 'disallow rebinding of function names' and show what > would be behind calling the idea 'not feasible'. Again, I'm more concerned about the function than about the name. And the fact that "disallow rebinding of function names" is not feasible in Python is a design choice, not an essential characteristic of every programming language. That's fine. My point was: you can't at the same time have full dynamicity *and* procedural optimizations (like tail call opt). Everybody should be clear about the trade-off. > 1. A 'name' is not a 'function name' unless the name is bound, at > runtime, to a 'function'. > > 2. A 'function' in Python is not just one class but any callable > object -- with with a __call__ methods. That comprises an unbounded > set. Right. Then when you do: def myfunc(...): ... myfunc is bound to an callable object. In my example, I was doing the equivalent to rebinding myfunc, losing the last reference to that piece of code: myfunc = somethingelse BTW, does the original callable object have a ref counter? Is it garbage collected in that case? If not, would it be considered a bug? > 3. Python does not mandate how namespaces are implemented. CPython > uses both dicts and, for function local namespaces, internal C arrays. > So 'names' in code can become either string keys for dicts or integer > indexes for arrays. Well, yes, but that's an implementation detail, no? > 4. Rebinding can be explicit ('='), implicit ('import', 'class', > def'), *or hidden* (globals('a') = 1; ob.__dict__('a') = 1). The > hidden' methods are intentional as they are sometimes needed*. In > CPython, these forms remain different in the byte code, but it could > be otherwise. The point is that is may or may not be possible for the > interpreter to even recognize a 'rebinding' in order to apply any > rebinding blocking rule. Sure (that's exactly why I said it is easier to implement). If you were to disallow rebinding of global function names, you would need a proper notion of global scope and various categories of names, something almost all compiled languages have. > * There is no trick (that I know of) for hidden rebinding of function > locals and nonlocals (short of using ctypes), but I am not sure that > this is a language guarantee. However, I an sure that the absence is > intentional. > >> even though it's easier to implement. > > No kidding. (See above). -- Alain. From antoon.pardon at rece.vub.ac.be Mon Oct 7 13:57:36 2013 From: antoon.pardon at rece.vub.ac.be (Antoon Pardon) Date: Mon, 07 Oct 2013 19:57:36 +0200 Subject: Tail recursion to while iteration in 2 easy steps In-Reply-To: <878uy52ea0.fsf@dpt-info.u-strasbg.fr> References: <87had0axxy.fsf@dpt-info.u-strasbg.fr> <524C80B6.3010204@unistra.fr> <87li292wnt.fsf@dpt-info.u-strasbg.fr> <878uy52ea0.fsf@dpt-info.u-strasbg.fr> Message-ID: <5252F610.9040403@rece.vub.ac.be> Op 07-10-13 19:15, Alain Ketterlin schreef: >> I want to consider here what it would mean to concretely implement the >> abstract notion 'disallow rebinding of function names' and show what >> would be behind calling the idea 'not feasible'. > > Again, I'm more concerned about the function than about the name. > > And the fact that "disallow rebinding of function names" is not feasible > in Python is a design choice, not an essential characteristic of every > programming language. > > That's fine. My point was: you can't at the same time have full > dynamicity *and* procedural optimizations (like tail call opt). > Everybody should be clear about the trade-off. Your wrong. Full dynamics is not in contradiction with tail call optimisation. Scheme has already done it for years. You can rebind names to other functions in scheme and scheme still has working tail call optimisatiosn. -- Antoon Pardon From python at mrabarnett.plus.com Mon Oct 7 17:39:40 2013 From: python at mrabarnett.plus.com (MRAB) Date: Mon, 07 Oct 2013 22:39:40 +0100 Subject: Tail recursion to while iteration in 2 easy steps In-Reply-To: <5252F610.9040403@rece.vub.ac.be> References: <87had0axxy.fsf@dpt-info.u-strasbg.fr> <524C80B6.3010204@unistra.fr> <87li292wnt.fsf@dpt-info.u-strasbg.fr> <878uy52ea0.fsf@dpt-info.u-strasbg.fr> <5252F610.9040403@rece.vub.ac.be> Message-ID: <52532A1C.4090503@mrabarnett.plus.com> On 07/10/2013 18:57, Antoon Pardon wrote: > Op 07-10-13 19:15, Alain Ketterlin schreef: >>> I want to consider here what it would mean to concretely >>> implement the abstract notion 'disallow rebinding of function >>> names' and show what would be behind calling the idea 'not >>> feasible'. >> >> Again, I'm more concerned about the function than about the name. >> >> And the fact that "disallow rebinding of function names" is not >> feasible in Python is a design choice, not an essential >> characteristic of every programming language. >> >> That's fine. My point was: you can't at the same time have full >> dynamicity *and* procedural optimizations (like tail call opt). >> Everybody should be clear about the trade-off. > > Your wrong. Full dynamics is not in contradiction with tail call > optimisation. Scheme has already done it for years. You can rebind > names to other functions in scheme and scheme still has working tail > call optimisatiosn. > Consider this code: def fact(n, acc=1): if n <= 1: return acc return fact(n-1, n*acc) It compiles to this: >>> dis.dis(fact) 2 0 LOAD_FAST 0 (n) 3 LOAD_CONST 1 (1) 6 COMPARE_OP 1 (<=) 9 POP_JUMP_IF_FALSE 16 3 12 LOAD_FAST 1 (acc) 15 RETURN_VALUE 4 >> 16 LOAD_GLOBAL 0 (fact) 19 LOAD_FAST 0 (n) 22 LOAD_CONST 1 (1) 25 BINARY_SUBTRACT 26 LOAD_FAST 0 (n) 29 LOAD_FAST 1 (acc) 32 BINARY_MULTIPLY 33 CALL_FUNCTION 2 (2 positional, 0 keyword pair) 36 RETURN_VALUE I think that CALL_FUNCTION immediately followed by RETURN_VALUE could be tail-call optimised by dropping the caller's stack frame provided that (like in this case) the callee doesn't refer to any name in the callers stack frame (nonlocal). You could also consider replacing the caller's stack frame with a smaller pseudo-frame, perhaps compressing multiple pseudo-frames or repeated sequences of pseudo-frames into a single pseudo-frame, so that it could still generate the same traceback as before (or an compressed traceback like what was discussed on python-ideas in the threads "Compressing excepthook output" and "Idea: Compressing the stack on the fly"). From dreamingforward at gmail.com Mon Oct 7 18:47:26 2013 From: dreamingforward at gmail.com (Mark Janssen) Date: Mon, 7 Oct 2013 15:47:26 -0700 Subject: Tail recursion to while iteration in 2 easy steps In-Reply-To: <5252F610.9040403@rece.vub.ac.be> References: <87had0axxy.fsf@dpt-info.u-strasbg.fr> <524C80B6.3010204@unistra.fr> <87li292wnt.fsf@dpt-info.u-strasbg.fr> <878uy52ea0.fsf@dpt-info.u-strasbg.fr> <5252F610.9040403@rece.vub.ac.be> Message-ID: >> That's fine. My point was: you can't at the same time have full >> dynamicity *and* procedural optimizations (like tail call opt). >> Everybody should be clear about the trade-off. > > Your wrong. Full dynamics is not in contradiction with tail call > optimisation. Scheme has already done it for years. You can rebind > names to other functions in scheme and scheme still has working > tail call optimisatiosn. Yeah, and this is where two models of computation have been conflated, creating magical effects, confusing everybody. I challenge you to get down to the machine code in scheme and formally describe how it's doing both. -- MarkJ Tacoma, Washington From steve+comp.lang.python at pearwood.info Mon Oct 7 19:50:48 2013 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: 07 Oct 2013 23:50:48 GMT Subject: Tail recursion to while iteration in 2 easy steps References: <87had0axxy.fsf@dpt-info.u-strasbg.fr> <524C80B6.3010204@unistra.fr> <87li292wnt.fsf@dpt-info.u-strasbg.fr> <878uy52ea0.fsf@dpt-info.u-strasbg.fr> <5252F610.9040403@rece.vub.ac.be> Message-ID: <525348d7$0$29984$c3e8da3$5496439d@news.astraweb.com> On Mon, 07 Oct 2013 15:47:26 -0700, Mark Janssen wrote: > I challenge you to get > down to the machine code in scheme and formally describe how it's doing > both. For which machine? Or are you assuming that there's only one machine code that runs on all computing devices? Frankly, asking somebody to *formally* describe a machine code implementation strikes me as confused. Normally formal descriptions are given in terms of abstract operations, often high level operations, sometimes *very* high level, and rarely in terms of low-level "flip this bit, copy this byte" machine code operations. I'm not sure how one would be expected to generate a formal description of a machine code implementation. But even putting that aside, even if somebody wrote such a description, it would be reductionism gone mad. What possible light on the problem would be shined by a long, long list of machine code operations, even if written using assembly mnemonics? Far more useful would be a high-level description of Scheme's programming model. If names can be rebound on the fly, how does Scheme even tell whether something is a recursive call or not? def foo(arg): do stuff here foo(arg-1) # how does Scheme know that this is the same foo? -- Steven From dreamingforward at gmail.com Mon Oct 7 20:16:35 2013 From: dreamingforward at gmail.com (Mark Janssen) Date: Mon, 7 Oct 2013 17:16:35 -0700 Subject: Tail recursion to while iteration in 2 easy steps In-Reply-To: <525348d7$0$29984$c3e8da3$5496439d@news.astraweb.com> References: <87had0axxy.fsf@dpt-info.u-strasbg.fr> <524C80B6.3010204@unistra.fr> <87li292wnt.fsf@dpt-info.u-strasbg.fr> <878uy52ea0.fsf@dpt-info.u-strasbg.fr> <5252F610.9040403@rece.vub.ac.be> <525348d7$0$29984$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Mon, Oct 7, 2013 at 4:50 PM, Steven D'Aprano wrote: > On Mon, 07 Oct 2013 15:47:26 -0700, Mark Janssen wrote: >> I challenge you to get >> down to the machine code in scheme and formally describe how it's doing >> both. > > For which machine? Right, I should stop assuming a modern implementation of vonNeumann architecture (even though that, too, is ambiguous) since I'm talking about theory, but yet it is relevant. My demarcation point for arguments between "the scheme way" and other procedural languages (which, apart from Pascal variants, I blithely all "the C way") gets down to differing models of computation which shouldn't get conflated, even though everyone thinks and lumps it all as "computation". They simply can't get *practically* translated between one and the other, even though they are *theoretically* translated between each other all the time. Humans, of course know how to translate, but that doesn't count from the pov of computer *science*. > Frankly, asking somebody to *formally* describe a machine code > implementation strikes me as confused. Normally formal descriptions are > given in terms of abstract operations, often high level operations, > sometimes *very* high level, and rarely in terms of low-level "flip this > bit, copy this byte" machine code operations. I'm not sure how one would > be expected to generate a formal description of a machine code > implementation. It's like this: there *should* be one-to-one mappings between the various high-level constructs to the machine code, varying only between different chips (that is the purpose of the compiler after all), yet for some operations, in languages like scheme, well... I cannot say what happens... hence my challenge. > But even putting that aside, even if somebody wrote such a description, > it would be reductionism gone mad. What possible light on the problem > would be shined by a long, long list of machine code operations, even if > written using assembly mnemonics? Only that you've got a consistent, stable (and therefore, formalizable) translation from your language to the machine. That's all. Everything else is magic. Do you know that the Warren Abstraction Engine used to power the predicate logic in Prolog into machien code for a VonNeumann machine is so complex, no one has understood it (or perhaps even verified it)? One hardly knows where these things originate. But here it gets into dark arts best not entered into too deeply. It will turn you mad, like that guy in the movie "pi". -- MarkJ Tacoma, Washington From dreamingforward at gmail.com Mon Oct 7 20:24:10 2013 From: dreamingforward at gmail.com (Mark Janssen) Date: Mon, 7 Oct 2013 17:24:10 -0700 Subject: Tail recursion to while iteration in 2 easy steps In-Reply-To: References: <87had0axxy.fsf@dpt-info.u-strasbg.fr> <524C80B6.3010204@unistra.fr> <87li292wnt.fsf@dpt-info.u-strasbg.fr> <878uy52ea0.fsf@dpt-info.u-strasbg.fr> <5252F610.9040403@rece.vub.ac.be> <525348d7$0$29984$c3e8da3$5496439d@news.astraweb.com> Message-ID: > Only that you've got a consistent, stable (and therefore, > formalizable) translation from your language to the machine. That's > all. Everything else is magic. Do you know that the Warren > Abstraction Engine used to power the predicate logic in Prolog into > machien code for a VonNeumann machine is so complex, no one has > understood it (or perhaps even verified it)? Sorry, I mean the Warren Abstraction Machine (or WAM). I refer you to www.cvc.uab.es/shared/teach/a25002/wambook.pdf. Now, one can easily argue that I've gone too far to say "no one has understood it" (obviously), so it's very little tongue-in-cheek, but really, when one tries to pretend that one model of computation can be substituted for another (arguing *for* the Church-Turing thesis), they are getting into troubling territory (it is only a thesis, remember).... -- MarkJ Tacoma, Washington From rustompmody at gmail.com Mon Oct 7 23:17:45 2013 From: rustompmody at gmail.com (rusi) Date: Mon, 7 Oct 2013 20:17:45 -0700 (PDT) Subject: Formal-ity and the Church-Turing thesis In-Reply-To: References: <87had0axxy.fsf@dpt-info.u-strasbg.fr> <524C80B6.3010204@unistra.fr> <87li292wnt.fsf@dpt-info.u-strasbg.fr> <878uy52ea0.fsf@dpt-info.u-strasbg.fr> <5252F610.9040403@rece.vub.ac.be> <525348d7$0$29984$c3e8da3$5496439d@news.astraweb.com> Message-ID: <48454d8d-19be-49e4-a63e-9718067e6417@googlegroups.com> On Tuesday, October 8, 2013 5:54:10 AM UTC+5:30, zipher wrote: > Now, one can easily argue that I've gone too far to say "no one has > understood it" (obviously), so it's very little tongue-in-cheek, but > really, when one tries to pretend that one model of computation can be > substituted for another (arguing *for* the Church-Turing thesis), they > are getting into troubling territory (it is only a thesis, > remember).... The CT thesis is scientific and provable in one sense and vague/philosophical in another. The Science: Turing computability and lambda-computability are equivalent. The proofs just consist of writing interpreters for one in terms of the other. The philosophy: *ALL* computational models are turing equivalent (and therefore lambda-equivalent) or weaker. The Idea (note not proof) is that for equivalence one can write pair-interpreters like above. For the 'weaker' case, (eg DFA and TMs) one proves that TMs can interpret DFAs and disproves the possibility of the other direction. This must remain an idea (aka thesis) and not a proof because one cannot conceive of all possible computational models. It is hard science however for all the models that anyone has so far come up with. Now there are caveats to this which I wont go into for now. As for: > I challenge you to get down to the machine code in scheme and formally > describe how it's doing both. I can only say how ironic it sounds to someone who is familiar with the history of our field: Turing was not a computer scientist (the term did not exist then) but a mathematician. And his major contribution was to create a form of argument so much more rigorous than what erstwhile mathematicians were used to that he was justified in calling that math as a machine. The irony is that today's generation assumes that 'some-machine' implies its something like 'Intel-machine'. To get out of this confusion ask yourself: Is it finite or infinite? If the TM were finite it would be a DFA If the Intel-machine (and like) were infinite they would need to exist in a different universe. And so when you understand that TMs are just a kind of mathematical rewrite system (as is ? calculus as are context free grammars as is school arithmetic etc etc) you will not find the equivalence so surprising From ganeshsahni07 at gmail.com Tue Oct 8 01:16:50 2013 From: ganeshsahni07 at gmail.com (Ravi Sahni) Date: Tue, 8 Oct 2013 10:46:50 +0530 Subject: Formal-ity and the Church-Turing thesis In-Reply-To: <48454d8d-19be-49e4-a63e-9718067e6417@googlegroups.com> References: <87had0axxy.fsf@dpt-info.u-strasbg.fr> <524C80B6.3010204@unistra.fr> <87li292wnt.fsf@dpt-info.u-strasbg.fr> <878uy52ea0.fsf@dpt-info.u-strasbg.fr> <5252F610.9040403@rece.vub.ac.be> <525348d7$0$29984$c3e8da3$5496439d@news.astraweb.com> <48454d8d-19be-49e4-a63e-9718067e6417@googlegroups.com> Message-ID: On Tue, Oct 8, 2013 at 8:47 AM, rusi wrote: > I can only say how ironic it sounds to someone who is familiar with the history of our field: > Turing was not a computer scientist (the term did not exist then) but a mathematician. And his major contribution was to create a form of argument so much more rigorous than what erstwhile mathematicians were used to that he was justified in calling that math as a machine. > > The irony is that today's generation assumes that 'some-machine' implies its something like 'Intel-machine'. > To get out of this confusion ask yourself: Is it finite or infinite? > If the TM were finite it would be a DFA > If the Intel-machine (and like) were infinite they would need to exist in a different universe. With due respect Sir, you saying that Turing machine not a machine? Very confusion Sir!!! > > And so when you understand that TMs are just a kind of mathematical rewrite system (as is ? calculus as are context free grammars as is school arithmetic etc etc) you will not find the equivalence so surprising -- Ravi From dreamingforward at gmail.com Tue Oct 8 01:19:11 2013 From: dreamingforward at gmail.com (Mark Janssen) Date: Mon, 7 Oct 2013 22:19:11 -0700 Subject: Formal-ity and the Church-Turing thesis In-Reply-To: <48454d8d-19be-49e4-a63e-9718067e6417@googlegroups.com> References: <87had0axxy.fsf@dpt-info.u-strasbg.fr> <524C80B6.3010204@unistra.fr> <87li292wnt.fsf@dpt-info.u-strasbg.fr> <878uy52ea0.fsf@dpt-info.u-strasbg.fr> <5252F610.9040403@rece.vub.ac.be> <525348d7$0$29984$c3e8da3$5496439d@news.astraweb.com> <48454d8d-19be-49e4-a63e-9718067e6417@googlegroups.com> Message-ID: > On Tuesday, October 8, 2013 5:54:10 AM UTC+5:30, zipher wrote: >> Now, one can easily argue that I've gone too far to say "no one has >> understood it" (obviously), so it's very little tongue-in-cheek, but >> really, when one tries to pretend that one model of computation can be >> substituted for another (arguing *for* the Church-Turing thesis), they >> are getting into troubling territory (it is only a thesis, >> remember).... > > The CT thesis is scientific and provable in one sense and vague/philosophical in another. > The Science: Turing computability and lambda-computability are equivalent. > The proofs just consist of writing interpreters for one in terms of the other. Ah, good, a fellow theoretician. Now it's nice that you use language that makes it seem quite clear, but understand that there's a hidden, subconscious, *cultural* encoding to your *statement*. The use of the term "equivalent", for example. Equivalent for the programmer, or for the machine? (etc., et cetera), and further: "writing interpreters for one in terms of the other", but again, this will change depending on your pragmatic requirements. To the theorist, you've accomplished something, but then that is a self-serving kind of accomplishment. To the programmer, operating under different requirements, you haven't accomplished anything. I don't have an infinite stack to implement lambda calculus, but I can treat my computer's memory as a TM and *practically* infinite and only rarely hit against the limits of physicality. This is just being "respectful"... ;^) (For the purposes of discussion, if I make a word in CamelCase, I am referring to a page on the WikiWikiWeb with the same name: http://c2.com/cgi/wiki?WikiWikiWeb.) > The philosophy: *ALL* computational models are turing equivalent (and therefore lambda-equivalent) or weaker. > The Idea (note not proof) is that for equivalence one can write pair-interpreters like above. For the 'weaker' case, (eg DFA and TMs) one proves that TMs can interpret DFAs and disproves the possibility of the other direction. > > This must remain an idea (aka thesis) and not a proof because one cannot conceive of all possible computational models. Why not? I can "conceive" of all possible integer numbers even if I never "pictured" them. Is there not an inductive way to conceive of and define computation? I mean, I observe that the field seems to define several ModelsOfComputation. Intuitively I see two primary domains > It is hard science however for all the models that anyone has so far come up with. And what of "interactive computation"? > As for: > >> I challenge you to get down to the machine code in scheme and formally >> describe how it's doing both. > > I can only say how ironic it sounds to someone who is familiar with the history of our field: > Turing was not a computer scientist (the term did not exist then) but a mathematician. And his major contribution was to create a form of argument so much more rigorous than what erstwhile mathematicians were used to that he was justified in calling that math as a machine. Hmm, I'm wondering if my use of the word "formally" is confusing you. In mathematics, this word has a subtly differing meaning, I think, than in computer science. Turing was "justified in calling that math as a machine" because he was using a definition (the translation table + finite dictionary) such that it remained perfectly deterministic. And here, again, one can easily gets mixed up using the same lexicon across two different domains: that of math and that of CS. I advise you to look at the dialog at ConfusedComputerScience. > The irony is that today's generation assumes that 'some-machine' implies its something like 'Intel-machine'. > To get out of this confusion ask yourself: Is it finite or infinite? But this only gets us out of the confusion for the mathematicians. For the programmer and perhaps even the computer scientist (the one's coming from physics), it is something different. > If the TM were finite it would be a DFA But this is not a useful formalism. Any particular Program implements a DFA, even as it runs on a TM. The issue of whether than TM is finite or not can be dismissed because a simple calculation can usually suffice, or at least establish a range "usefulness" so as not to "run out of memory". > If the Intel-machine (and like) were infinite they would need to exist in a different universe. Ha, yeah. Let us dismiss with that. > And so when you understand that TMs are just a kind of mathematical rewrite system (as is ? calculus as are context free grammars as is school arithmetic etc etc) you will not find the equivalence so surprising It's not that it's surprising, it's that it's *practically* a problem. The translation between one PL and another which assumes a different model of computation can get intractible. Maybe that makes sense.... MarkJ Tacoma, Washington From rustompmody at gmail.com Tue Oct 8 01:44:34 2013 From: rustompmody at gmail.com (rusi) Date: Mon, 7 Oct 2013 22:44:34 -0700 (PDT) Subject: Formal-ity and the Church-Turing thesis In-Reply-To: References: <87had0axxy.fsf@dpt-info.u-strasbg.fr> <524C80B6.3010204@unistra.fr> <87li292wnt.fsf@dpt-info.u-strasbg.fr> <878uy52ea0.fsf@dpt-info.u-strasbg.fr> <5252F610.9040403@rece.vub.ac.be> <525348d7$0$29984$c3e8da3$5496439d@news.astraweb.com> <48454d8d-19be-49e4-a63e-9718067e6417@googlegroups.com> Message-ID: <016233e8-6763-4c0b-8e05-d28bc6a8bef4@googlegroups.com> On Tuesday, October 8, 2013 10:46:50 AM UTC+5:30, Ravi Sahni wrote: > With due respect Sir, you saying that Turing machine not a machine? > Very confusion Sir!!! Thanks Ravi for the 'due respect' though it is a bit out of place on a list like this :-) Thanks even more for the 'very confusion'. I can tell you being a teacher for 25 years that the most terrifying thing is the Dunning Kruger effect -- students who are utterly clueless and dont have a frigging clue about that. Ive seen PhDs in CS ask questions in compiler-related degree projects that they would not ask if they really understood the halting problem. [Or were you suggestig that I am the confused? :-) ] So yes, its a big thing to be confused and to accept that. To explain at length will be too long and OT (off-topic) for this list. I'll just give you a link and you tell me what you make of it: http://sloan.stanford.edu/mousesite/Secondary/Whorfframe2.html [mainly concentrate on the first section] From breamoreboy at yahoo.co.uk Tue Oct 8 02:43:50 2013 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Tue, 08 Oct 2013 07:43:50 +0100 Subject: Formal-ity and the Church-Turing thesis In-Reply-To: <016233e8-6763-4c0b-8e05-d28bc6a8bef4@googlegroups.com> References: <87had0axxy.fsf@dpt-info.u-strasbg.fr> <524C80B6.3010204@unistra.fr> <87li292wnt.fsf@dpt-info.u-strasbg.fr> <878uy52ea0.fsf@dpt-info.u-strasbg.fr> <5252F610.9040403@rece.vub.ac.be> <525348d7$0$29984$c3e8da3$5496439d@news.astraweb.com> <48454d8d-19be-49e4-a63e-9718067e6417@googlegroups.com> <016233e8-6763-4c0b-8e05-d28bc6a8bef4@googlegroups.com> Message-ID: On 08/10/2013 06:44, rusi wrote: > On Tuesday, October 8, 2013 10:46:50 AM UTC+5:30, Ravi Sahni wrote: >> With due respect Sir, you saying that Turing machine not a machine? >> Very confusion Sir!!! > > Thanks Ravi for the 'due respect' though it is a bit out of place on a list like this :-) > With due respect Sir I'd like to point out that this appears to have very little to do (directly) with Python, so to go completely off topic I'll point out that my nephew is currently working on the film about the life of said Alan Turing :) -- Roses are red, Violets are blue, Most poems rhyme, But this one doesn't. Mark Lawrence From ganeshsahni07 at gmail.com Tue Oct 8 09:01:21 2013 From: ganeshsahni07 at gmail.com (Ravi Sahni) Date: Tue, 8 Oct 2013 18:31:21 +0530 Subject: Formal-ity and the Church-Turing thesis In-Reply-To: <016233e8-6763-4c0b-8e05-d28bc6a8bef4@googlegroups.com> References: <87had0axxy.fsf@dpt-info.u-strasbg.fr> <524C80B6.3010204@unistra.fr> <87li292wnt.fsf@dpt-info.u-strasbg.fr> <878uy52ea0.fsf@dpt-info.u-strasbg.fr> <5252F610.9040403@rece.vub.ac.be> <525348d7$0$29984$c3e8da3$5496439d@news.astraweb.com> <48454d8d-19be-49e4-a63e-9718067e6417@googlegroups.com> <016233e8-6763-4c0b-8e05-d28bc6a8bef4@googlegroups.com> Message-ID: On Tue, Oct 8, 2013 at 11:14 AM, rusi wrote: > To explain at length will be too long and OT (off-topic) for this list. > I'll just give you a link and you tell me what you make of it: > http://sloan.stanford.edu/mousesite/Secondary/Whorfframe2.html I am trying to read link. Very new idea: Buildings can catch fire by wrong boards!! Later part difficult for me to read. (My English not powerful --please excuse.) I will make my fullest efforts to read on your recommend but I not clear the connection with computers, programming, computer science and so on. Also this Mr. Mark Lawrence question. -- Ravi From rustompmody at gmail.com Tue Oct 8 09:33:35 2013 From: rustompmody at gmail.com (rusi) Date: Tue, 8 Oct 2013 06:33:35 -0700 (PDT) Subject: Formal-ity and the Church-Turing thesis In-Reply-To: References: <87had0axxy.fsf@dpt-info.u-strasbg.fr> <524C80B6.3010204@unistra.fr> <87li292wnt.fsf@dpt-info.u-strasbg.fr> <878uy52ea0.fsf@dpt-info.u-strasbg.fr> <5252F610.9040403@rece.vub.ac.be> <525348d7$0$29984$c3e8da3$5496439d@news.astraweb.com> <48454d8d-19be-49e4-a63e-9718067e6417@googlegroups.com> <016233e8-6763-4c0b-8e05-d28bc6a8bef4@googlegroups.com> Message-ID: <378e97f5-e93e-4be1-a6da-92c785a10ddc@googlegroups.com> On Tuesday, October 8, 2013 6:31:21 PM UTC+5:30, Ravi Sahni wrote: > On Tue, Oct 8, 2013 at 11:14 AM, rusi wrote: > > To explain at length will be too long and OT (off-topic) for this list. > > I'll just give you a link and you tell me what you make of it: > > http://sloan.stanford.edu/mousesite/Secondary/Whorfframe2.html > > > I am trying to read link. Very new idea: Buildings can catch fire by > wrong boards!! > > Later part difficult for me to read. (My English not powerful --please excuse.) > I will make my fullest efforts to read on your recommend Hell No! I only asked you to read the first page! [And 'Mr. Mark' will scold] > but I not > clear the connection with computers, programming, computer science and > so on. Also this Mr. Mark Lawrence question. Once you get that buildings can catch fire by wrong terminology you should get that: - the term 'Turing machine' can make people think its a machine even though its a mathematical formalism - the term '?-calculus' (partly due to the word calculus and partly due to the greek lambda) makes people think its mathematics even though its a computational framework From steve at pearwood.info Tue Oct 8 03:50:14 2013 From: steve at pearwood.info (Steven D'Aprano) Date: 08 Oct 2013 07:50:14 GMT Subject: Formal-ity and the Church-Turing thesis References: <87had0axxy.fsf@dpt-info.u-strasbg.fr> <524C80B6.3010204@unistra.fr> <87li292wnt.fsf@dpt-info.u-strasbg.fr> <878uy52ea0.fsf@dpt-info.u-strasbg.fr> <5252F610.9040403@rece.vub.ac.be> <525348d7$0$29984$c3e8da3$5496439d@news.astraweb.com> <48454d8d-19be-49e4-a63e-9718067e6417@googlegroups.com> Message-ID: <5253b935$0$29976$c3e8da3$5496439d@news.astraweb.com> On Tue, 08 Oct 2013 10:46:50 +0530, Ravi Sahni wrote: > On Tue, Oct 8, 2013 at 8:47 AM, rusi wrote: >> I can only say how ironic it sounds to someone who is familiar with the >> history of our field: Turing was not a computer scientist (the term did >> not exist then) but a mathematician. And his major contribution was to >> create a form of argument so much more rigorous than what erstwhile >> mathematicians were used to that he was justified in calling that math >> as a machine. >> >> The irony is that today's generation assumes that 'some-machine' >> implies its something like 'Intel-machine'. To get out of this >> confusion ask yourself: Is it finite or infinite? If the TM were finite >> it would be a DFA If the Intel-machine (and like) were infinite they >> would need to exist in a different universe. > > With due respect Sir, you saying that Turing machine not a machine? Very > confusion Sir!!! The mathematical ideal Turing Machine has an infinitely long tape, equivalent to infinite memory, and may take an unbounded amount of time to complete the computation. Since no *actual* physical machine can be infinitely big, and in practice there are strict limits on how long we are willing to wait for a computation to complete, in the *literal* sense, Turing Machines are not *actual* machines. They are a mathematical abstraction. But in practice, we can wave our hands and ignore this fact, and consider only not-quite-Turing Machines with finite amounts of tape, and note that they are equivalent to physical machines with finite amounts of memory. One could even build such a finite Turing Machine, although of course it would be very slow. Or one can simulate it in software. So in that sense, computers are Turing Machines. Anything a physical computing device can compute, a Turing Machine could too. The converse is not true though: a Turing Machine with infinite tape can compute things where a real physical device would run out of memory, although it might take longer than anyone is willing to wait. -- Steven From ganeshsahni07 at gmail.com Tue Oct 8 08:46:01 2013 From: ganeshsahni07 at gmail.com (Ravi Sahni) Date: Tue, 8 Oct 2013 18:16:01 +0530 Subject: Formal-ity and the Church-Turing thesis In-Reply-To: <5253b935$0$29976$c3e8da3$5496439d@news.astraweb.com> References: <87had0axxy.fsf@dpt-info.u-strasbg.fr> <524C80B6.3010204@unistra.fr> <87li292wnt.fsf@dpt-info.u-strasbg.fr> <878uy52ea0.fsf@dpt-info.u-strasbg.fr> <5252F610.9040403@rece.vub.ac.be> <525348d7$0$29984$c3e8da3$5496439d@news.astraweb.com> <48454d8d-19be-49e4-a63e-9718067e6417@googlegroups.com> <5253b935$0$29976$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Tue, Oct 8, 2013 at 1:20 PM, Steven D'Aprano wrote: > On Tue, 08 Oct 2013 10:46:50 +0530, Ravi Sahni wrote: > >> On Tue, Oct 8, 2013 at 8:47 AM, rusi wrote: >>> I can only say how ironic it sounds to someone who is familiar with the >>> history of our field: Turing was not a computer scientist (the term did >>> not exist then) but a mathematician. And his major contribution was to >>> create a form of argument so much more rigorous than what erstwhile >>> mathematicians were used to that he was justified in calling that math >>> as a machine. >>> >>> The irony is that today's generation assumes that 'some-machine' >>> implies its something like 'Intel-machine'. To get out of this >>> confusion ask yourself: Is it finite or infinite? If the TM were finite >>> it would be a DFA If the Intel-machine (and like) were infinite they >>> would need to exist in a different universe. >> >> With due respect Sir, you saying that Turing machine not a machine? Very >> confusion Sir!!! > > The mathematical ideal Turing Machine has an infinitely long tape, > equivalent to infinite memory, and may take an unbounded amount of time > to complete the computation. Since no *actual* physical machine can be > infinitely big, and in practice there are strict limits on how long we > are willing to wait for a computation to complete, in the *literal* > sense, Turing Machines are not *actual* machines. They are a mathematical > abstraction. > > But in practice, we can wave our hands and ignore this fact, and consider > only not-quite-Turing Machines with finite amounts of tape, and note that > they are equivalent to physical machines with finite amounts of memory. > One could even build such a finite Turing Machine, although of course it > would be very slow. Or one can simulate it in software. > > So in that sense, computers are Turing Machines. Anything a physical > computing device can compute, a Turing Machine could too. The converse is > not true though: a Turing Machine with infinite tape can compute things > where a real physical device would run out of memory, although it might > take longer than anyone is willing to wait. Thanks Sir the detailed explanation. You are offering me many thoughts inside few words so I will need some time to meditate upon the same. Presently Sir, I wish to ask single question: What you mean "wave our hands"?? Thanks -- Ravi From steve+comp.lang.python at pearwood.info Tue Oct 8 09:11:49 2013 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: 08 Oct 2013 13:11:49 GMT Subject: Formal-ity and the Church-Turing thesis References: <87had0axxy.fsf@dpt-info.u-strasbg.fr> <524C80B6.3010204@unistra.fr> <87li292wnt.fsf@dpt-info.u-strasbg.fr> <878uy52ea0.fsf@dpt-info.u-strasbg.fr> <5252F610.9040403@rece.vub.ac.be> <525348d7$0$29984$c3e8da3$5496439d@news.astraweb.com> <48454d8d-19be-49e4-a63e-9718067e6417@googlegroups.com> <5253b935$0$29976$c3e8da3$5496439d@news.astraweb.com> Message-ID: <52540495$0$29984$c3e8da3$5496439d@news.astraweb.com> On Tue, 08 Oct 2013 18:16:01 +0530, Ravi Sahni wrote: >> So in that sense, computers are Turing Machines. Anything a physical >> computing device can compute, a Turing Machine could too. The converse >> is not true though: a Turing Machine with infinite tape can compute >> things where a real physical device would run out of memory, although >> it might take longer than anyone is willing to wait. > > Thanks Sir the detailed explanation. You are offering me many thoughts > inside few words so I will need some time to meditate upon the same. > > Presently Sir, I wish to ask single question: What you mean "wave our > hands"?? It is an idiom very common in Australia. (It may not be well known in the rest of the English-speaking world.) It means to figuratively flap one's hands around in the air while skipping over technical details or complications. For example, we often talk about "hand-wavy estimates" for how long a job will take: "my hand-wavy estimate is it will take two days" is little better than a guess. -- Steven From robertkday at gmail.com Tue Oct 8 09:25:37 2013 From: robertkday at gmail.com (Robert Day) Date: Tue, 08 Oct 2013 14:25:37 +0100 Subject: Formal-ity and the Church-Turing thesis In-Reply-To: <52540495$0$29984$c3e8da3$5496439d@news.astraweb.com> References: <87had0axxy.fsf@dpt-info.u-strasbg.fr> <524C80B6.3010204@unistra.fr> <87li292wnt.fsf@dpt-info.u-strasbg.fr> <878uy52ea0.fsf@dpt-info.u-strasbg.fr> <5252F610.9040403@rece.vub.ac.be> <525348d7$0$29984$c3e8da3$5496439d@news.astraweb.com> <48454d8d-19be-49e4-a63e-9718067e6417@googlegroups.com> <5253b935$0$29976$c3e8da3$5496439d@news.astraweb.com> <52540495$0$29984$c3e8da3$5496439d@news.astraweb.com> Message-ID: <525407D1.1030204@gmail.com> On 08/10/13 14:11, Steven D'Aprano wrote: > On Tue, 08 Oct 2013 18:16:01 +0530, Ravi Sahni wrote: > >> >> Presently Sir, I wish to ask single question: What you mean "wave our >> hands"?? > It is an idiom very common in Australia. (It may not be well known in the > rest of the English-speaking world.) It means to figuratively flap one's > hands around in the air while skipping over technical details or > complications. It's known elsewhere as well (though mostly in technical circles) - it's in the Jargon File as http://www.catb.org/jargon/html/H/handwave.html. -------------- next part -------------- An HTML attachment was scrubbed... URL: From rosuav at gmail.com Tue Oct 8 17:36:58 2013 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 9 Oct 2013 08:36:58 +1100 Subject: Formal-ity and the Church-Turing thesis In-Reply-To: <52540495$0$29984$c3e8da3$5496439d@news.astraweb.com> References: <87had0axxy.fsf@dpt-info.u-strasbg.fr> <524C80B6.3010204@unistra.fr> <87li292wnt.fsf@dpt-info.u-strasbg.fr> <878uy52ea0.fsf@dpt-info.u-strasbg.fr> <5252F610.9040403@rece.vub.ac.be> <525348d7$0$29984$c3e8da3$5496439d@news.astraweb.com> <48454d8d-19be-49e4-a63e-9718067e6417@googlegroups.com> <5253b935$0$29976$c3e8da3$5496439d@news.astraweb.com> <52540495$0$29984$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Wed, Oct 9, 2013 at 12:11 AM, Steven D'Aprano wrote: > On Tue, 08 Oct 2013 18:16:01 +0530, Ravi Sahni wrote: > >>> So in that sense, computers are Turing Machines. Anything a physical >>> computing device can compute, a Turing Machine could too. The converse >>> is not true though: a Turing Machine with infinite tape can compute >>> things where a real physical device would run out of memory, although >>> it might take longer than anyone is willing to wait. >> >> Thanks Sir the detailed explanation. You are offering me many thoughts >> inside few words so I will need some time to meditate upon the same. >> >> Presently Sir, I wish to ask single question: What you mean "wave our >> hands"?? > > It is an idiom very common in Australia. (It may not be well known in the > rest of the English-speaking world.) It means to figuratively flap one's > hands around in the air while skipping over technical details or > complications. For example, we often talk about "hand-wavy estimates" for > how long a job will take: "my hand-wavy estimate is it will take two > days" is little better than a guess. A derivative of the term has gone mainstream, too: http://tvtropes.org/pmwiki/pmwiki.php/Main/HandWave The term is commonly used when moving to a higher level of abstraction - we all know a computer doesn't have a soul, can't "feel", and is ultimately just executing code and crunching numbers, but we handwave that (eg) the computer "thought" that this program was a risk, and that's why it quarantined it. When you're trying to explain to some user that he can't email .EXE files around, it's easier to take the slightly-inaccurate but simple explanation, hence the handwaves. ChrisA From rustompmody at gmail.com Tue Oct 8 02:01:40 2013 From: rustompmody at gmail.com (rusi) Date: Mon, 7 Oct 2013 23:01:40 -0700 (PDT) Subject: Formal-ity and the Church-Turing thesis In-Reply-To: References: <87had0axxy.fsf@dpt-info.u-strasbg.fr> <524C80B6.3010204@unistra.fr> <87li292wnt.fsf@dpt-info.u-strasbg.fr> <878uy52ea0.fsf@dpt-info.u-strasbg.fr> <5252F610.9040403@rece.vub.ac.be> <525348d7$0$29984$c3e8da3$5496439d@news.astraweb.com> <48454d8d-19be-49e4-a63e-9718067e6417@googlegroups.com> Message-ID: <9526d1fb-a3eb-4b6d-96f5-df872f462dfe@googlegroups.com> On Tuesday, October 8, 2013 10:49:11 AM UTC+5:30, zipher wrote: > I don't have an infinite stack to implement > lambda calculus, but... And then > But this is not a useful formalism. Any particular Program implements > a DFA, even as it runs on a TM. The issue of whether than TM is > finite or not can be dismissed because a simple calculation can > usually suffice, or at least establish a range "usefulness" so as not > to "run out of memory". Having it both ways aren't you? From dreamingforward at gmail.com Tue Oct 8 13:39:47 2013 From: dreamingforward at gmail.com (Mark Janssen) Date: Tue, 8 Oct 2013 10:39:47 -0700 Subject: Formal-ity and the Church-Turing thesis In-Reply-To: <9526d1fb-a3eb-4b6d-96f5-df872f462dfe@googlegroups.com> References: <87had0axxy.fsf@dpt-info.u-strasbg.fr> <524C80B6.3010204@unistra.fr> <87li292wnt.fsf@dpt-info.u-strasbg.fr> <878uy52ea0.fsf@dpt-info.u-strasbg.fr> <5252F610.9040403@rece.vub.ac.be> <525348d7$0$29984$c3e8da3$5496439d@news.astraweb.com> <48454d8d-19be-49e4-a63e-9718067e6417@googlegroups.com> <9526d1fb-a3eb-4b6d-96f5-df872f462dfe@googlegroups.com> Message-ID: >> I don't have an infinite stack to implement >> lambda calculus, but... > > And then > >> But this is not a useful formalism. Any particular Program implements >> a DFA, even as it runs on a TM. The issue of whether than TM is >> finite or not can be dismissed because a simple calculation can >> usually suffice, or at least establish a range "usefulness" so as not >> to "run out of memory". > > Having it both ways aren't you? I'm just speaking from programmer experience and the fact that most machines are VonNeumann architecture. Being that as it is, maxing out the stack simply happens, and I don't dare do any non-simple recursion, but otherwise, practically speaking, I can calculate my memory usage that may grow on the heap so that is effectively a non-issue. This may not be an important distinction for computing, the "art" (Hello ultimate lambda friends), but it is significant for the computing, the science. MarkJ Tacoma, Washington From steve+comp.lang.python at pearwood.info Mon Oct 7 22:36:06 2013 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: 08 Oct 2013 02:36:06 GMT Subject: Tail recursion to while iteration in 2 easy steps References: <87had0axxy.fsf@dpt-info.u-strasbg.fr> <524C80B6.3010204@unistra.fr> <87li292wnt.fsf@dpt-info.u-strasbg.fr> <878uy52ea0.fsf@dpt-info.u-strasbg.fr> <5252F610.9040403@rece.vub.ac.be> <525348d7$0$29984$c3e8da3$5496439d@news.astraweb.com> Message-ID: <52536f96$0$29984$c3e8da3$5496439d@news.astraweb.com> On Mon, 07 Oct 2013 17:16:35 -0700, Mark Janssen wrote: > It's like this: there *should* be one-to-one mappings between the > various high-level constructs to the machine code, varying only between > different chips (that is the purpose of the compiler after all), yet for > some operations, in languages like scheme, well... I cannot say what > happens... hence my challenge. > >> But even putting that aside, even if somebody wrote such a description, >> it would be reductionism gone mad. What possible light on the problem >> would be shined by a long, long list of machine code operations, even >> if written using assembly mnemonics? > > Only that you've got a consistent, stable (and therefore, formalizable) > translation from your language to the machine. You are mistaken to think that there is a single, one-to-one, mapping between high-level code and machine code. In practice, only if you use the exact same source code, the exact same compiler, the exact same version of that compiler, with the exact same compiler options, and the exact same version of the language and all its libraries, then and only then will you will get the same machine code. And even that is not guaranteed. Take, for example, the single high-level operation: sort(alist) What machine code will be executed? Obviously that will depend on the sort algorithm used. There are *dozens*. Here are just a few: http://en.wikipedia.org/wiki/Category:Sorting_algorithms Now sorting is pretty high level, but the same principle applies to even simple operations like "multiply two numbers". There are often multiple algorithms for performing the operation, and even a single algorithm can often be implemented in slightly different ways. Expecting all compilers to generate the same machine code is simply naive. We can use Python to discuss this, since Python includes a compiler. It generates byte code, which just means machine code for a virtual machine instead of a hardware machine, but the principle is the same. Python even has a disassembler, for taking those raw byte codes and turning them into human-readable pseudo-assembly for the Python Virtual Machine. Here is some "machine code" corresponding to the high-level instructions: x = 23 y = 42 z = x + y del x, y py> code = compile('x = 23; y = 42; z = x + y; del x, y', '', 'exec') py> code.co_code 'd\x00\x00Z\x00\x00d\x01\x00Z\x01\x00e\x00\x00e\x01\x00\x17Z\x02\x00[\x00 \x00[\x01\x00d\x02\x00S' Translated into "assembly": py> from dis import dis py> dis(code) 1 0 LOAD_CONST 0 (23) 3 STORE_NAME 0 (x) 6 LOAD_CONST 1 (42) 9 STORE_NAME 1 (y) 12 LOAD_NAME 0 (x) 15 LOAD_NAME 1 (y) 18 BINARY_ADD 19 STORE_NAME 2 (z) 22 DELETE_NAME 0 (x) 25 DELETE_NAME 1 (y) 28 LOAD_CONST 2 (None) 31 RETURN_VALUE You should be able to see that there are all sorts of changes that the compiler could have made, which would have lead to different "machine code" but with the exact same behaviour. This particular compiler emits code for x=23; y=42 in the order that they were given, but that's not actually required in this case. The compiler might have reversed the order, generating something like: 0 LOAD_CONST 1 (42) 3 STORE_NAME 1 (y) 6 LOAD_CONST 0 (23) 9 STORE_NAME 0 (x) or even: 0 LOAD_CONST 1 (42) 3 LOAD_CONST 0 (23) 6 STORE_NAME 1 (y) 9 STORE_NAME 0 (x) without changing the behaviour of the code. Or it might have even optimized the entire thing to: 0 LOAD_CONST 0 (65) 3 STORE_NAME 0 (z) since x and y are temporary variables and a smart compiler could perform the computation at compile time and throw them away. (Nitpicks about "what if x and y already existed?" aside.) CPython even does this sort of optimization, although in a more limited fashion: py> dis(compile('x = 23 + 42', '', 'exec')) 1 0 LOAD_CONST 3 (65) 3 STORE_NAME 0 (x) 6 LOAD_CONST 2 (None) 9 RETURN_VALUE This is called keyhole optimization. It's not beyond possibility for a smarter compiler to look beyond the keyhole and make optimizations based on the whole program. So you can see that there is no one-to-one correspondence from high-level source code to low-level machine code, even for a single machine. The same is even more true for languages such as C, Fortran, Pascal, Lisp, Scheme, Haskell, Java, etc. where people have spent years or decades working on compiler technology. Compilers differ in the quality and efficiency of the machine code they emit and the choices they make about translating source code to machine code. > That's all. Everything > else is magic. Do you know that the Warren Abstraction Engine used to > power the predicate logic in Prolog into machien code for a VonNeumann > machine is so complex, no one has understood it That's nonsense. In your next post, you even supply a link to a book describing how the WAM works with detailed step-by-step instructions for writing your own. For those reading, here it is: www.cvc.uab.es/shared/teach/a25002/wambook.pdf Prolog is not some dark black magic that nobody understands. There are multiple implementations of Prolog-like logic languages for Python: http://pyke.sourceforge.net/ https://github.com/logpy/logpy https://github.com/enriquepablo/nl/ http://code.google.com/p/fuxi/ https://sites.google.com/site/pydatalog/ to say nothing of similar, more advanced languages like Mercury. Just because *you personally* don't understand something, don't think that nobody else does. -- Steven From dreamingforward at gmail.com Mon Oct 7 23:27:13 2013 From: dreamingforward at gmail.com (Mark Janssen) Date: Mon, 7 Oct 2013 20:27:13 -0700 Subject: Tail recursion to while iteration in 2 easy steps In-Reply-To: <52536f96$0$29984$c3e8da3$5496439d@news.astraweb.com> References: <87had0axxy.fsf@dpt-info.u-strasbg.fr> <524C80B6.3010204@unistra.fr> <87li292wnt.fsf@dpt-info.u-strasbg.fr> <878uy52ea0.fsf@dpt-info.u-strasbg.fr> <5252F610.9040403@rece.vub.ac.be> <525348d7$0$29984$c3e8da3$5496439d@news.astraweb.com> <52536f96$0$29984$c3e8da3$5496439d@news.astraweb.com> Message-ID: >>> But even putting that aside, even if somebody wrote such a description, >>> it would be reductionism gone mad. What possible light on the problem >>> would be shined by a long, long list of machine code operations, even >>> if written using assembly mnemonics? >> >> Only that you've got a consistent, stable (and therefore, formalizable) >> translation from your language to the machine. > > You are mistaken to think that there is a single, one-to-one, mapping > between high-level code and machine code. It's not mistaken. Given a stable and formalized language definition, there should only be continued optimization of the lexical and procedural constructs into better machine code. In the case of an "interpreted" language like Python (which I'll define as a language which includes a layer of indirection between the user and the machine, encouraging the nice benefits of interactivity), such optimization isn't really apropos, because it's not the purpose of python to be optimal to the machine as much as "optimal to the programmer". In any case, while such optimization can continue over time, they generally create new compiler releases to indicate such changes. The one-to-one mapping is held by the compiler. Such determinism *defines* the machine, otherwise you might as well get rid of the notion of computer *science*. All else is error, akin to cosmic rays or magic. Unless the source code changes, all else remaining equal, the machine code is supposed to be the same, no matter how many times it is compiled. >[Only if you use the exact source, compiler, switches, etc]] will the output be the same. > And even that is not guaranteed. Oh, and what would cause such non-determinism? > Take, for example, the single high-level operation: > > sort(alist) > > What machine code will be executed? Obviously that will depend on the > sort algorithm used. There are *dozens*. Here are just a few: Well, since you didn't specify your programming language, you're then merely stating an English construct. As such, there can be no single mapping from English into the machine, which is why there are so many different languages and experiments that map your [English] concepts into source code. > Now sorting is pretty high level, but the same principle applies to even > simple operations like "multiply two numbers". There are often multiple > algorithms for performing the operation, and even a single algorithm can > often be implemented in slightly different ways. Expecting all compilers > to generate the same machine code is simply naive. You are both over-simplifying and complexifying things at once. Pick one. -- MarkJ Tacoma, Washington From steve at pearwood.info Tue Oct 8 05:22:44 2013 From: steve at pearwood.info (Steven D'Aprano) Date: 08 Oct 2013 09:22:44 GMT Subject: Tail recursion to while iteration in 2 easy steps References: <87had0axxy.fsf@dpt-info.u-strasbg.fr> <524C80B6.3010204@unistra.fr> <87li292wnt.fsf@dpt-info.u-strasbg.fr> <878uy52ea0.fsf@dpt-info.u-strasbg.fr> <5252F610.9040403@rece.vub.ac.be> <525348d7$0$29984$c3e8da3$5496439d@news.astraweb.com> <52536f96$0$29984$c3e8da3$5496439d@news.astraweb.com> Message-ID: <5253cee4$0$29976$c3e8da3$5496439d@news.astraweb.com> On Mon, 07 Oct 2013 20:27:13 -0700, Mark Janssen wrote: >>>> But even putting that aside, even if somebody wrote such a >>>> description, it would be reductionism gone mad. What possible light >>>> on the problem would be shined by a long, long list of machine code >>>> operations, even if written using assembly mnemonics? >>> >>> Only that you've got a consistent, stable (and therefore, >>> formalizable) translation from your language to the machine. >> >> You are mistaken to think that there is a single, one-to-one, mapping >> between high-level code and machine code. > > It's not mistaken. I'm afraid it is. Reality trumps your theory. gcc, clang, Microsoft Visual Studio, and all the many, many other C compilers do not generate identical machine code even when targeting the same hardware. This is a fact. It's not even the case that there is One True Way to implement a particular program on a given hardware device and compilers merely are buggy for doing something else. There are often different ways to implement it which are equally good, the only difference being personal preference. > Given a stable and formalized language definition, > there should only be continued optimization of the lexical and > procedural constructs into better machine code. Better than what? "Continued" optimization? When you say "lexical and procedural constructs", do you mean "source code"? > In the case of an > "interpreted" language like Python (which I'll define as a language > which includes a layer of indirection between the user and the machine, Irrelevant. In the case of Python, there is a machine. The fact that it is a VM rather than a physical machine is irrelevant. A machine is a machine -- we could be talking about a Lisp Machine, a Forth Machine, a x86 processor, an Motorola 68000, an Atom processor, one of those old Russian mainframes that used three-state trits instead of two-state bits, or even Babbage's Analytical Engine. Besides, most modern CPUs don't execute machine code directly, they run the machine code in a virtual machine implemented in hardware. So the difference between Python and x86 machine code is just a matter of degree. > encouraging the nice benefits of interactivity), such optimization isn't > really apropos, because it's not the purpose of python to be optimal to > the machine as much as "optimal to the programmer". In any case, while > such optimization can continue over time, they generally create new > compiler releases to indicate such changes. The one-to-one mapping is > held by the compiler. > > Such determinism *defines* the machine, otherwise you might as well get > rid of the notion of computer *science*. All else is error, akin to > cosmic rays or magic. Unless the source code changes, all else > remaining equal, the machine code is supposed to be the same, no matter > how many times it is compiled. That is akin to saying that there is *only one* way to measure the speed of light (say), standing in exactly the same place, using exactly the same equipment, using precisely the same measurement techniques, and that if we allow alternative methods for measuring the speed of light, physics is no longer a science. >>[Only if you use the exact source, compiler, switches, etc]] will the >>output be the same. >> And even that is not guaranteed. > > Oh, and what would cause such non-determinism? The compiler-writer, of course. A compiler is software, and is written by a person, who can program it to do anything the writer wants. If the writer wants the compiler to be non-deterministic, it can be. Some viruses use a similar technique to try to avoid virus scanners. They encrypt the payload, which is functionally equivalent to randomizing it (except it can be reversed if you have the key) so as to defeat virus scanners. A more whimsical example: perhaps a mischievous compiler writer included something like this in her compiler: when compiling integer multiplication, INT * INT: if today is Tuesday: emit machine code that does multiplication using repeated addition otherwise: emit machine code that does multiplication using ADD and SHIFT Both implementations of multiplication are perfectly valid. There may be a performance difference, or there may not be. Since no sensible programming language is going to specify the *detailed* machine code implementation of its high-level operations, such a mischievous compiler would still be valid. >> Take, for example, the single high-level operation: >> >> sort(alist) >> >> What machine code will be executed? Obviously that will depend on the >> sort algorithm used. There are *dozens*. Here are just a few: > > Well, since you didn't specify your programming language, you're then > merely stating an English construct. What difference does it make? But if it will make you feel better, I'm specifying Hypertalk. You've probably never heard of it, but regardless, it exists, and it has a sort command, and the high-level language does not specify which of many sort algorithms is to be used. > As such, there can be no single > mapping from English into the machine, which is why there are so many > different languages and experiments that map your [English] concepts > into source code. And there is no single mapping from source code to machine code either. -- Steven From charleshixsn at earthlink.net Wed Oct 9 18:45:05 2013 From: charleshixsn at earthlink.net (Charles Hixson) Date: Wed, 09 Oct 2013 15:45:05 -0700 Subject: Tail recursion to while iteration in 2 easy steps In-Reply-To: <5253cee4$0$29976$c3e8da3$5496439d@news.astraweb.com> References: <87had0axxy.fsf@dpt-info.u-strasbg.fr> <524C80B6.3010204@unistra.fr> <87li292wnt.fsf@dpt-info.u-strasbg.fr> <878uy52ea0.fsf@dpt-info.u-strasbg.fr> <5252F610.9040403@rece.vub.ac.be> <525348d7$0$29984$c3e8da3$5496439d@news.astraweb.com> <52536f96$0$29984$c3e8da3$5496439d@news.astraweb.com> <5253cee4$0$29976$c3e8da3$5496439d@news.astraweb.com> Message-ID: <5255DC71.7040808@earthlink.net> On 10/08/2013 02:22 AM, Steven D'Aprano wrote: > On Mon, 07 Oct 2013 20:27:13 -0700, Mark Janssen wrote: > >>>>> But even putting that aside, even if somebody wrote such a >>>>> description, it would be reductionism gone mad. What possible light >>>>> on the problem would be shined by a long, long list of machine code >>>>> operations, even if written using assembly mnemonics? >>>> Only that you've got a consistent, stable (and therefore, >>>> formalizable) translation from your language to the machine. >>> You are mistaken to think that there is a single, one-to-one, mapping >>> between high-level code and machine code. >> It's not mistaken. > I'm afraid it is. Reality trumps your theory. gcc, clang, Microsoft > Visual Studio, and all the many, many other C compilers do not generate > identical machine code even when targeting the same hardware. This is a > fact. It's not even the case that there is One True Way to implement a > particular program on a given hardware device and compilers merely are > buggy for doing something else. There are often different ways to > implement it which are equally good, the only difference being personal > preference. > > >> Given a stable and formalized language definition, >> there should only be continued optimization of the lexical and >> procedural constructs into better machine code. > Better than what? "Continued" optimization? When you say "lexical and > procedural constructs", do you mean "source code"? > > >> In the case of an >> "interpreted" language like Python (which I'll define as a language >> which includes a layer of indirection between the user and the machine, > Irrelevant. In the case of Python, there is a machine. The fact that it > is a VM rather than a physical machine is irrelevant. A machine is a > machine -- we could be talking about a Lisp Machine, a Forth Machine, a > x86 processor, an Motorola 68000, an Atom processor, one of those old > Russian mainframes that used three-state trits instead of two-state bits, > or even Babbage's Analytical Engine. > > Besides, most modern CPUs don't execute machine code directly, they run > the machine code in a virtual machine implemented in hardware. So the > difference between Python and x86 machine code is just a matter of degree. > > > >> encouraging the nice benefits of interactivity), such optimization isn't >> really apropos, because it's not the purpose of python to be optimal to >> the machine as much as "optimal to the programmer". In any case, while >> such optimization can continue over time, they generally create new >> compiler releases to indicate such changes. The one-to-one mapping is >> held by the compiler. >> >> Such determinism *defines* the machine, otherwise you might as well get >> rid of the notion of computer *science*. All else is error, akin to >> cosmic rays or magic. Unless the source code changes, all else >> remaining equal, the machine code is supposed to be the same, no matter >> how many times it is compiled. > That is akin to saying that there is *only one* way to measure the speed > of light (say), standing in exactly the same place, using exactly the > same equipment, using precisely the same measurement techniques, and that > if we allow alternative methods for measuring the speed of light, physics > is no longer a science. > > >>> [Only if you use the exact source, compiler, switches, etc]] will the >>> output be the same. >>> And even that is not guaranteed. >> Oh, and what would cause such non-determinism? > The compiler-writer, of course. A compiler is software, and is written by > a person, who can program it to do anything the writer wants. If the > writer wants the compiler to be non-deterministic, it can be. > > Some viruses use a similar technique to try to avoid virus scanners. They > encrypt the payload, which is functionally equivalent to randomizing it > (except it can be reversed if you have the key) so as to defeat virus > scanners. > > A more whimsical example: perhaps a mischievous compiler writer included > something like this in her compiler: > > > when compiling integer multiplication, INT * INT: > if today is Tuesday: > emit machine code that does multiplication using repeated addition > otherwise: > emit machine code that does multiplication using ADD and SHIFT > > > Both implementations of multiplication are perfectly valid. There may be > a performance difference, or there may not be. Since no sensible > programming language is going to specify the *detailed* machine code > implementation of its high-level operations, such a mischievous compiler > would still be valid. > > >>> Take, for example, the single high-level operation: >>> >>> sort(alist) >>> >>> What machine code will be executed? Obviously that will depend on the >>> sort algorithm used. There are *dozens*. Here are just a few: >> Well, since you didn't specify your programming language, you're then >> merely stating an English construct. > What difference does it make? But if it will make you feel better, I'm > specifying Hypertalk. You've probably never heard of it, but regardless, > it exists, and it has a sort command, and the high-level language does > not specify which of many sort algorithms is to be used. > > > >> As such, there can be no single >> mapping from English into the machine, which is why there are so many >> different languages and experiments that map your [English] concepts >> into source code. > And there is no single mapping from PROGRAMMING LANGUAGE HERE> source code to machine code either. I would assert that Python is not inherently a virtual machine language. Originally, IIRC, it was believed that LISP couldn't be compiled. Also, you could implement that virtual machine as a hardware machine. (Also, of course, on modern hardware assembly language is run on a virtual machine, implemented by an underneath microcode layer.) You can reasonably say that an implementation of Python is done in terms of a virtual machine. (Usually I don't bother about this kind of nit-pick, but in this discussion it seems apropos.) -- Charles Hixson From piet at vanoostrum.org Mon Oct 7 22:46:53 2013 From: piet at vanoostrum.org (Piet van Oostrum) Date: Mon, 07 Oct 2013 22:46:53 -0400 Subject: Tail recursion to while iteration in 2 easy steps References: <87had0axxy.fsf@dpt-info.u-strasbg.fr> <524C80B6.3010204@unistra.fr> <87li292wnt.fsf@dpt-info.u-strasbg.fr> <878uy52ea0.fsf@dpt-info.u-strasbg.fr> <5252F610.9040403@rece.vub.ac.be> <525348d7$0$29984$c3e8da3$5496439d@news.astraweb.com> Message-ID: Steven D'Aprano writes: > Far more useful would be a high-level description of Scheme's programming > model. If names can be rebound on the fly, how does Scheme even tell > whether something is a recursive call or not? Maybe it doesn't have to tell. If you do tail call optimization there is no need to do tail recursion optimization. -- Piet van Oostrum WWW: http://pietvanoostrum.com/ PGP key: [8DAE142BE17999C4] From jpiitula at ling.helsinki.fi Tue Oct 8 03:25:05 2013 From: jpiitula at ling.helsinki.fi (Jussi Piitulainen) Date: 08 Oct 2013 10:25:05 +0300 Subject: Tail recursion to while iteration in 2 easy steps References: <87had0axxy.fsf@dpt-info.u-strasbg.fr> <524C80B6.3010204@unistra.fr> <87li292wnt.fsf@dpt-info.u-strasbg.fr> <878uy52ea0.fsf@dpt-info.u-strasbg.fr> <5252F610.9040403@rece.vub.ac.be> <525348d7$0$29984$c3e8da3$5496439d@news.astraweb.com> Message-ID: Steven D'Aprano writes: > Far more useful would be a high-level description of Scheme's > programming model. If names can be rebound on the fly, how does > Scheme even tell whether something is a recursive call or not? > > def foo(arg): > do stuff here > foo(arg-1) # how does Scheme know that this is the same foo? In general, it doesn't know. It just calls whatever function is bound to foo. It does know that the call is in a tail position. If the compiler has access to all code that can possibly change the value of foo, it can know simply by proving that there is no such assignment statement in the code. This can happen if the compiler is told to assume that it has the whole program. It often happens in a local scope. Module systems create such local scopes for unexported variables, and even for exported variables by forbidding assignments outside. (I'm not sure if your question was rhetorical or if you were looking for this information.) From antoon.pardon at rece.vub.ac.be Tue Oct 8 05:18:35 2013 From: antoon.pardon at rece.vub.ac.be (Antoon Pardon) Date: Tue, 08 Oct 2013 11:18:35 +0200 Subject: Tail recursion to while iteration in 2 easy steps In-Reply-To: <525348d7$0$29984$c3e8da3$5496439d@news.astraweb.com> References: <87had0axxy.fsf@dpt-info.u-strasbg.fr> <524C80B6.3010204@unistra.fr> <87li292wnt.fsf@dpt-info.u-strasbg.fr> <878uy52ea0.fsf@dpt-info.u-strasbg.fr> <5252F610.9040403@rece.vub.ac.be> <525348d7$0$29984$c3e8da3$5496439d@news.astraweb.com> Message-ID: <5253CDEB.40106@rece.vub.ac.be> Op 08-10-13 01:50, Steven D'Aprano schreef: > On Mon, 07 Oct 2013 15:47:26 -0700, Mark Janssen wrote: > >> I challenge you to get >> down to the machine code in scheme and formally describe how it's doing >> both. > > For which machine? > > Or are you assuming that there's only one machine code that runs on all > computing devices? > > > Frankly, asking somebody to *formally* describe a machine code > implementation strikes me as confused. Normally formal descriptions are > given in terms of abstract operations, often high level operations, > sometimes *very* high level, and rarely in terms of low-level "flip this > bit, copy this byte" machine code operations. I'm not sure how one would > be expected to generate a formal description of a machine code > implementation. > > But even putting that aside, even if somebody wrote such a description, > it would be reductionism gone mad. What possible light on the problem > would be shined by a long, long list of machine code operations, even if > written using assembly mnemonics? > > Far more useful would be a high-level description of Scheme's programming > model. If names can be rebound on the fly, how does Scheme even tell > whether something is a recursive call or not? > > def foo(arg): > do stuff here > foo(arg-1) # how does Scheme know that this is the same foo? It doesn't and it doesn't need to. tail call optimisation is not limited to recursive functions. All tail calls can be optimised, recurisive call and others. -- Antoon Pardon From piet at vanoostrum.org Mon Oct 7 22:45:17 2013 From: piet at vanoostrum.org (Piet van Oostrum) Date: Mon, 07 Oct 2013 22:45:17 -0400 Subject: Tail recursion to while iteration in 2 easy steps References: <87had0axxy.fsf@dpt-info.u-strasbg.fr> <524C80B6.3010204@unistra.fr> <87li292wnt.fsf@dpt-info.u-strasbg.fr> <878uy52ea0.fsf@dpt-info.u-strasbg.fr> <5252F610.9040403@rece.vub.ac.be> Message-ID: Mark Janssen writes: > Yeah, and this is where two models of computation have been conflated, > creating magical effects, confusing everybody. I challenge you to get > down to the machine code in scheme and formally describe how it's > doing both. Which two models of computation are you talking about? And what magica; effects? AFAIK there is no magic in computer science, although every sufficiently advanced ... -- Piet van Oostrum WWW: http://pietvanoostrum.com/ PGP key: [8DAE142BE17999C4] From dreamingforward at gmail.com Mon Oct 7 23:34:26 2013 From: dreamingforward at gmail.com (Mark Janssen) Date: Mon, 7 Oct 2013 20:34:26 -0700 Subject: Tail recursion to while iteration in 2 easy steps In-Reply-To: References: <87had0axxy.fsf@dpt-info.u-strasbg.fr> <524C80B6.3010204@unistra.fr> <87li292wnt.fsf@dpt-info.u-strasbg.fr> <878uy52ea0.fsf@dpt-info.u-strasbg.fr> <5252F610.9040403@rece.vub.ac.be> Message-ID: >> Yeah, and this is where two models of computation have been conflated, >> creating magical effects, confusing everybody. I challenge you to get >> down to the machine code in scheme and formally describe how it's >> doing both. > > Which two models of computation are you talking about? And what magica; effects? Well, I delineate all computation involving predicates (like lambda calculus) between those using digital logic (like C). These realms of computation are so different, they are akin to mixing the complex numbers with the real. Yet hardly anyone points it out (I've concluded that hardly anyone has ever noticed -- the Church-Turing thesis has lulled the whole field into a shortcut in thinking which actually doesn't pan out in practice). > AFAIK there is no magic in computer science, although every sufficiently advanced ... Ha! That's very good. I'm glad you catch the spirit of my rant. "Any sufficiently advanced compiler can be substituted with magic to the neophyte without a change in output." A mini Liskov substitution. -- MarkJ Tacoma, Washington From tjreedy at udel.edu Mon Oct 7 16:42:40 2013 From: tjreedy at udel.edu (Terry Reedy) Date: Mon, 07 Oct 2013 16:42:40 -0400 Subject: Tail recursion to while iteration in 2 easy steps In-Reply-To: <878uy52ea0.fsf@dpt-info.u-strasbg.fr> References: <87had0axxy.fsf@dpt-info.u-strasbg.fr> <524C80B6.3010204@unistra.fr> <87li292wnt.fsf@dpt-info.u-strasbg.fr> <878uy52ea0.fsf@dpt-info.u-strasbg.fr> Message-ID: On 10/7/2013 1:15 PM, Alain Ketterlin wrote: > Terry Reedy writes: >> 3. Python does not mandate how namespaces are implemented. CPython >> uses both dicts and, for function local namespaces, internal C arrays. >> So 'names' in code can become either string keys for dicts or integer >> indexes for arrays. > > Well, yes, but that's an implementation detail, no? That is why I switched from 'Python' to 'CPython'. But I note the following: in 2.x, 'from mod import *' in a function meant that the compile time mapping of name to index could not be used and that a fallback to dict was necessary. So another implementation might take the easier path and always use a dict for function locals. In 3.x, such import are limited to module scope so that functions can always use an array and indexes for function locals. So other implementations can take the hint and do the same without needing a dict fallback. In other words, 3.x changed the language to facilitate the implementation detail. -- Terry Jan Reedy From random832 at fastmail.us Mon Oct 7 17:19:04 2013 From: random832 at fastmail.us (random832 at fastmail.us) Date: Mon, 07 Oct 2013 17:19:04 -0400 Subject: Tail recursion to while iteration in 2 easy steps In-Reply-To: <878uy52ea0.fsf@dpt-info.u-strasbg.fr> References: <87had0axxy.fsf@dpt-info.u-strasbg.fr> <524C80B6.3010204@unistra.fr> <87li292wnt.fsf@dpt-info.u-strasbg.fr> <878uy52ea0.fsf@dpt-info.u-strasbg.fr> Message-ID: <1381180744.4571.31155577.1631C8DD@webmail.messagingengine.com> On Mon, Oct 7, 2013, at 13:15, Alain Ketterlin wrote: > That's fine. My point was: you can't at the same time have full > dynamicity *and* procedural optimizations (like tail call opt). > Everybody should be clear about the trade-off. Let's be clear about what optimizations we are talking about. Tail call optimization, itself, doesn't care _what_ is being called. It can just as easily mean "erase its own stack frame and replace it with that of another function" as "reassign the arguments and jump to the top of this function". Some people have introduced the idea of _further_ optimizations, transforming "near" tail recursion (i.e. return self()+1) into tail recursion, and _that_ depends on knowing the identity of the function (though arguably that could be accounted for at the cost of including dead code for the path that assumes it may have been changed), but tail call optimization itself does not. From piet at vanoostrum.org Mon Oct 7 18:03:56 2013 From: piet at vanoostrum.org (Piet van Oostrum) Date: Mon, 07 Oct 2013 18:03:56 -0400 Subject: Tail recursion to while iteration in 2 easy steps References: <87had0axxy.fsf@dpt-info.u-strasbg.fr> <524C80B6.3010204@unistra.fr> <87li292wnt.fsf@dpt-info.u-strasbg.fr> <878uy52ea0.fsf@dpt-info.u-strasbg.fr> Message-ID: Alain Ketterlin writes: > BTW, does the original callable object have a ref counter? Is it garbage > collected in that case? If not, would it be considered a bug? In CPython ALL objects have ref counters. -- Piet van Oostrum WWW: http://pietvanoostrum.com/ PGP key: [8DAE142BE17999C4] From alain at dpt-info.u-strasbg.fr Tue Oct 8 04:41:41 2013 From: alain at dpt-info.u-strasbg.fr (Alain Ketterlin) Date: Tue, 08 Oct 2013 10:41:41 +0200 Subject: Tail recursion to while iteration in 2 easy steps References: <87had0axxy.fsf@dpt-info.u-strasbg.fr> <524C80B6.3010204@unistra.fr> <87li292wnt.fsf@dpt-info.u-strasbg.fr> <878uy52ea0.fsf@dpt-info.u-strasbg.fr> Message-ID: <874n8s2lyy.fsf@dpt-info.u-strasbg.fr> random832 at fastmail.us writes: > On Mon, Oct 7, 2013, at 13:15, Alain Ketterlin wrote: >> That's fine. My point was: you can't at the same time have full >> dynamicity *and* procedural optimizations (like tail call opt). >> Everybody should be clear about the trade-off. > > Let's be clear about what optimizations we are talking about. Tail call > optimization, itself, doesn't care _what_ is being called. It can just > as easily mean "erase its own stack frame and replace it with that of > another function" as "reassign the arguments and jump to the top of this > function". Some people have introduced the idea of _further_ > optimizations, transforming "near" tail recursion (i.e. return self()+1) > into tail recursion, and _that_ depends on knowing the identity of the > function (though arguably that could be accounted for at the cost of > including dead code for the path that assumes it may have been changed), > but tail call optimization itself does not. You're right, thanks for the clarification. -- Alain. From alain at dpt-info.u-strasbg.fr Tue Oct 8 04:56:20 2013 From: alain at dpt-info.u-strasbg.fr (Alain Ketterlin) Date: Tue, 08 Oct 2013 10:56:20 +0200 Subject: Tail recursion to while iteration in 2 easy steps References: <87had0axxy.fsf@dpt-info.u-strasbg.fr> <524C80B6.3010204@unistra.fr> <87li292wnt.fsf@dpt-info.u-strasbg.fr> <878uy52ea0.fsf@dpt-info.u-strasbg.fr> Message-ID: <87zjqk16q3.fsf@dpt-info.u-strasbg.fr> Antoon Pardon writes: > Op 07-10-13 19:15, Alain Ketterlin schreef: [...] >> That's fine. My point was: you can't at the same time have full >> dynamicity *and* procedural optimizations (like tail call opt). >> Everybody should be clear about the trade-off. > > Your wrong. Full dynamics is not in contradiction with tail call > optimisation. Scheme has already done it for years. You can rebind > names to other functions in scheme and scheme still has working > tail call optimisatiosn. See http://en.wikipedia.org/wiki/Scheme_%28programming_language%29#Lexical_scope (first sentence, about variable bindings). -- Alain. From jpiitula at ling.helsinki.fi Tue Oct 8 05:49:29 2013 From: jpiitula at ling.helsinki.fi (Jussi Piitulainen) Date: 08 Oct 2013 12:49:29 +0300 Subject: Tail recursion to while iteration in 2 easy steps References: <87had0axxy.fsf@dpt-info.u-strasbg.fr> <524C80B6.3010204@unistra.fr> <87li292wnt.fsf@dpt-info.u-strasbg.fr> <878uy52ea0.fsf@dpt-info.u-strasbg.fr> <87zjqk16q3.fsf@dpt-info.u-strasbg.fr> Message-ID: Alain Ketterlin writes: > Antoon Pardon writes: > > > Op 07-10-13 19:15, Alain Ketterlin schreef: > > [...] > >> That's fine. My point was: you can't at the same time have full > >> dynamicity *and* procedural optimizations (like tail call opt). > >> Everybody should be clear about the trade-off. > > > > Your wrong. Full dynamics is not in contradiction with tail call > > optimisation. Scheme has already done it for years. You can rebind > > names to other functions in scheme and scheme still has working > > tail call optimisatiosn. > > See > http://en.wikipedia.org/wiki/Scheme_%28programming_language%29#Lexical_scope > > (first sentence, about variable bindings). # ... Scheme is lexically scoped: all possible variable bindings in a # program unit can be analyzed by reading the text of the program unit # without consideration of the contexts in which it may be called ... The actual procedure to be called is still not known at compile time, in general. It can be a parameter. It needn't even be the value of any explicit variable (needn't be "bound to a name"). def call(f, a): ... return f(a) # tail call ... def wev(...): ... return (fs if c(k) else gs)[k](a) # tail call ... In the Scheme reports, a variable is said to be bound to a location, which is lexically apparent to the language processor; the value is stored in that location, and assignment to the variable means storing a new value in that location. It works like Python or Java; Python just has a different way of talking about how it works - binding names directly to values in a namespace, and rebinding to different values. However, Scheme processors know that the local variables are not accessible from anywhere else but the local code, so there are more opportunities for compile-time analysis. They can optimize many of those locations away, for example. From tjreedy at udel.edu Wed Oct 2 18:17:06 2013 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 02 Oct 2013 18:17:06 -0400 Subject: Tail recursion to while iteration in 2 easy steps In-Reply-To: <87had0axxy.fsf@dpt-info.u-strasbg.fr> References: <87had0axxy.fsf@dpt-info.u-strasbg.fr> Message-ID: On 10/2/2013 4:17 AM, Alain Ketterlin wrote: > Terry Reedy writes: > >> Part of the reason that Python does not do tail call optimization is >> that turning tail recursion into while iteration is almost trivial, >> once you know the secret of the two easy steps. Here it is. >> >> Assume that you have already done the work of turning a body recursive >> ('not tail recursive') form like >> >> def fact(n): return 1 if n <= 1 else n * fact(n-1) As I said in response to randomxxx, even this 0th step (recursion to recursion transformation) requires assumptions or carefully reasoning about the properties of the operations. >> into a tail recursion like > [...] > > How do know that either "<=" or "*" didn't rebind the name "fact" to > something else? I think that's the main reason why python cannot apply > any procedural optimization (even things like inlining are impossible, > or possible only under very conservative assumption, that make it > worthless). > > -- Alain. > > P/S: don't take me wrong; your explanation is excellent (and very useful > to python programmers). What I say is that it relies on assumptions that > do not apply to python. Program transformations (usually intended to be optimizations), whether automatic or manual, are infamous for being buggy in not always being correct because of hidden assumptions that are not always true. CPython core developers have be very conservative about what tranformations they put into the compiler. (1,2,3) can always be compiled as a constant, and so it is. [1,2,3] might or might not be a constant, depending on the context, and no attempt is made to analyze that. -- Terry Jan Reedy From steve+comp.lang.python at pearwood.info Wed Oct 2 21:24:10 2013 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: 03 Oct 2013 01:24:10 GMT Subject: Tail recursion to while iteration in 2 easy steps References: <87had0axxy.fsf@dpt-info.u-strasbg.fr> Message-ID: <524cc73a$0$29984$c3e8da3$5496439d@news.astraweb.com> On Wed, 02 Oct 2013 18:17:06 -0400, Terry Reedy wrote: > CPython core developers have be very conservative about what > tranformations they put into the compiler. (1,2,3) can always be > compiled as a constant, and so it is. [1,2,3] might or might not be a > constant, depending on the context, and no attempt is made to analyze > that. The first sentence of this is correct. The next two don't quite make sense to me, since I don't understand what you mean by "constant" in this context. I *think* you might be referring to the LOAD_CONST byte-code, which in Python 3.3 understands tuples like (1, 2, 3), but not lists. So a literal (1, 2, 3) gets created at compile-time with a single LOAD_CONST call: py> from dis import dis py> dis(compile("x = (1, 2, 3)", '', 'exec')) 1 0 LOAD_CONST 4 ((1, 2, 3)) 3 STORE_NAME 0 (x) 6 LOAD_CONST 3 (None) 9 RETURN_VALUE while a literal [1, 2, 3] does not: py> dis(compile("x = [1, 2, 3]", '', 'exec')) 1 0 LOAD_CONST 0 (1) 3 LOAD_CONST 1 (2) 6 LOAD_CONST 2 (3) 9 BUILD_LIST 3 12 STORE_NAME 0 (x) 15 LOAD_CONST 3 (None) 18 RETURN_VALUE But I don't think this is a necessary language limitation. Both (1, 2, 3) and [1, 2, 3] are known at compile time: the first cannot be anything other than a tuple of three ints, and the second a list of three ints. It seems to me that an implementation might provide a single byte-code to build list literals, perhaps even LOAD_CONST itself. The byte-codes used by the Python VM are not part of the language definition, and are subject to change without warning. And in fact, if we go all the way back to Python 1.5, even tuple literals weren't handled by a single byte-code, they were assembled at runtime like lists still are: [steve at ando ~]$ python1.5 Python 1.5.2 (#1, Aug 27 2012, 09:09:18) [GCC 4.1.2 20080704 (Red Hat 4.1.2-52)] on linux2 Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam >>> from dis import dis >>> dis(compile("x = (1, 2, 3)", '', 'exec')) 0 SET_LINENO 0 3 SET_LINENO 1 6 LOAD_CONST 0 (1) 9 LOAD_CONST 1 (2) 12 LOAD_CONST 2 (3) 15 BUILD_TUPLE 3 18 STORE_NAME 0 (x) 21 LOAD_CONST 3 (None) 24 RETURN_VALUE -- Steven From davea at davea.name Wed Oct 2 21:39:24 2013 From: davea at davea.name (Dave Angel) Date: Thu, 3 Oct 2013 01:39:24 +0000 (UTC) Subject: Tail recursion to while iteration in 2 easy steps References: <87had0axxy.fsf@dpt-info.u-strasbg.fr> <524cc73a$0$29984$c3e8da3$5496439d@news.astraweb.com> Message-ID: On 2/10/2013 21:24, Steven D'Aprano wrote: > On Wed, 02 Oct 2013 18:17:06 -0400, Terry Reedy wrote: > >> CPython core developers have be very conservative about what >> tranformations they put into the compiler. (1,2,3) can always be >> compiled as a constant, and so it is. [1,2,3] might or might not be a >> constant, depending on the context, and no attempt is made to analyze >> that. > > The first sentence of this is correct. The next two don't quite make > sense to me, since I don't understand what you mean by "constant" in this > context. I *think* you might be referring to the LOAD_CONST byte-code, > which in Python 3.3 understands tuples like (1, 2, 3), but not lists. So > a literal (1, 2, 3) gets created at compile-time with a single LOAD_CONST > call: > > py> from dis import dis > py> dis(compile("x = (1, 2, 3)", '', 'exec')) > 1 0 LOAD_CONST 4 ((1, 2, 3)) > 3 STORE_NAME 0 (x) > 6 LOAD_CONST 3 (None) > 9 RETURN_VALUE > > > while a literal [1, 2, 3] does not: > > The difference is that a tuple can be reused, so it makes sense for the comiler to produce it as a const. (Much like the interning of small integers) The list, however, would always have to be copied from the compile-time object. So that object itself would be a phantom, used only as the template with which the list is to be made. -- DaveA From python at mrabarnett.plus.com Wed Oct 2 21:46:53 2013 From: python at mrabarnett.plus.com (MRAB) Date: Thu, 03 Oct 2013 02:46:53 +0100 Subject: Tail recursion to while iteration in 2 easy steps In-Reply-To: References: <87had0axxy.fsf@dpt-info.u-strasbg.fr> <524cc73a$0$29984$c3e8da3$5496439d@news.astraweb.com> Message-ID: <524CCC8D.6020700@mrabarnett.plus.com> On 03/10/2013 02:39, Dave Angel wrote: > On 2/10/2013 21:24, Steven D'Aprano wrote: > >> On Wed, 02 Oct 2013 18:17:06 -0400, Terry Reedy wrote: >> >>> CPython core developers have be very conservative about what >>> tranformations they put into the compiler. (1,2,3) can always be >>> compiled as a constant, and so it is. [1,2,3] might or might not be a >>> constant, depending on the context, and no attempt is made to analyze >>> that. >> >> The first sentence of this is correct. The next two don't quite make >> sense to me, since I don't understand what you mean by "constant" in this >> context. I *think* you might be referring to the LOAD_CONST byte-code, >> which in Python 3.3 understands tuples like (1, 2, 3), but not lists. So >> a literal (1, 2, 3) gets created at compile-time with a single LOAD_CONST >> call: >> >> py> from dis import dis >> py> dis(compile("x = (1, 2, 3)", '', 'exec')) >> 1 0 LOAD_CONST 4 ((1, 2, 3)) >> 3 STORE_NAME 0 (x) >> 6 LOAD_CONST 3 (None) >> 9 RETURN_VALUE >> >> >> while a literal [1, 2, 3] does not: >> >> > > The difference is that a tuple can be reused, so it makes sense for the > comiler to produce it as a const. (Much like the interning of small > integers) The list, however, would always have to be copied from the > compile-time object. So that object itself would be a phantom, used > only as the template with which the list is to be made. > The key point here is that the tuple is immutable, including its items. From tjreedy at udel.edu Wed Oct 2 22:41:00 2013 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 02 Oct 2013 22:41:00 -0400 Subject: Tail recursion to while iteration in 2 easy steps In-Reply-To: <524CCC8D.6020700@mrabarnett.plus.com> References: <87had0axxy.fsf@dpt-info.u-strasbg.fr> <524cc73a$0$29984$c3e8da3$5496439d@news.astraweb.com> <524CCC8D.6020700@mrabarnett.plus.com> Message-ID: On 10/2/2013 9:46 PM, MRAB wrote: > On 03/10/2013 02:39, Dave Angel wrote: >> On 2/10/2013 21:24, Steven D'Aprano wrote: >> >>> On Wed, 02 Oct 2013 18:17:06 -0400, Terry Reedy wrote: >>> >>>> CPython core developers have be very conservative about what >>>> tranformations they put into the compiler. (1,2,3) can always be >>>> compiled as a constant, and so it is. [1,2,3] might or might not be a >>>> constant, depending on the context, and no attempt is made to analyze >>>> that. To be specific: in for i in [1,2,3]: print i it looks like it might be save to pre-compile the list and just use it when the code is run. But if the above is in a function object whose code object is ever introspected, the list object could be accessed and mutated. Letting the compiler know that it can do the optimization is one reason to use tuples in situations like the above. >>> The first sentence of this is correct. The next two don't quite make >>> sense to me, since I don't understand what you mean by "constant" in >>> this context. >>> I *think* you might be referring to the LOAD_CONST byte-code, I am referring to constant-value objects included in the code object. >>> def f(): return (1,2,3) >>> f.__code__.co_consts (None, 1, 2, 3, (1, 2, 3)) None is present as the default return, even if not needed for a particular function. Every literal is also tossed in, whether needed or not. >>> which in Python 3.3 understands tuples like (1, 2, 3), but not lists. The byte-code does not understand anything about types. LOAD_CONST n simply loads the (n+1)st object in .co_consts onto the top of the stack. >>> S9 a literal (1, 2, 3) gets created at compile-time with a single >>> LOAD_CONST call: >>> >>> py> from dis import dis >>> py> dis(compile("x = (1, 2, 3)", '', 'exec')) >>> 1 0 LOAD_CONST 4 ((1, 2, 3)) >>> 3 STORE_NAME 0 (x) >>> 6 LOAD_CONST 3 (None) >>> 9 RETURN_VALUE >>> >>> >>> while a literal [1, 2, 3] does not: >>> >> >> The difference is that a tuple can be reused, so it makes sense for the >> comiler to produce it as a const. (Much like the interning of small >> integers) The list, however, would always have to be copied from the >> compile-time object. So that object itself would be a phantom, used >> only as the template with which the list is to be made. >> > The key point here is that the tuple is immutable, including its items. The items of the tuple I gave as an examples are all constant. If they were not, the tuple would not be a constant for the purpose of compile-time creation. -- Terry Jan Reedy From random832 at fastmail.us Thu Oct 3 10:14:24 2013 From: random832 at fastmail.us (random832 at fastmail.us) Date: Thu, 03 Oct 2013 10:14:24 -0400 Subject: Tail recursion to while iteration in 2 easy steps In-Reply-To: <524CCC8D.6020700@mrabarnett.plus.com> References: <87had0axxy.fsf@dpt-info.u-strasbg.fr> <524cc73a$0$29984$c3e8da3$5496439d@news.astraweb.com> <524CCC8D.6020700@mrabarnett.plus.com> Message-ID: <1380809664.12405.29567505.3B72428D@webmail.messagingengine.com> On Wed, Oct 2, 2013, at 21:46, MRAB wrote: > > The difference is that a tuple can be reused, so it makes sense for the > > comiler to produce it as a const. (Much like the interning of small > > integers) The list, however, would always have to be copied from the > > compile-time object. So that object itself would be a phantom, used > > only as the template with which the list is to be made. > > > The key point here is that the tuple is immutable, including its items. Hey, while we're on the subject, can we talk about frozen(set|dict) literals again? I really don't understand why this discussion fizzles out whenever it's brought up on python-ideas. From ben+python at benfinney.id.au Thu Oct 3 20:18:49 2013 From: ben+python at benfinney.id.au (Ben Finney) Date: Fri, 04 Oct 2013 10:18:49 +1000 Subject: Literal syntax for frozenset, frozendict (was: Tail recursion to while iteration in 2 easy steps) References: <87had0axxy.fsf@dpt-info.u-strasbg.fr> <524cc73a$0$29984$c3e8da3$5496439d@news.astraweb.com> <524CCC8D.6020700@mrabarnett.plus.com> <1380809664.12405.29567505.3B72428D@webmail.messagingengine.com> Message-ID: <7wtxgxq45y.fsf_-_@benfinney.id.au> random832 at fastmail.us writes: > Hey, while we're on the subject, can we talk about frozen(set|dict) > literals again? I really don't understand why this discussion fizzles > out whenever it's brought up on python-ideas. Can you start us off by searching for previous threads discussing it, and summarise the arguments here? -- \ ?If you ever catch on fire, try to avoid seeing yourself in the | `\ mirror, because I bet that's what REALLY throws you into a | _o__) panic.? ?Jack Handey | Ben Finney From ethan at stoneleaf.us Thu Oct 3 21:31:00 2013 From: ethan at stoneleaf.us (Ethan Furman) Date: Thu, 03 Oct 2013 18:31:00 -0700 Subject: Literal syntax for frozenset, frozendict In-Reply-To: <7wtxgxq45y.fsf_-_@benfinney.id.au> References: <87had0axxy.fsf@dpt-info.u-strasbg.fr> <524cc73a$0$29984$c3e8da3$5496439d@news.astraweb.com> <524CCC8D.6020700@mrabarnett.plus.com> <1380809664.12405.29567505.3B72428D@webmail.messagingengine.com> <7wtxgxq45y.fsf_-_@benfinney.id.au> Message-ID: <524E1A54.10203@stoneleaf.us> On 10/03/2013 05:18 PM, Ben Finney wrote: > random832 at fastmail.us writes: > >> Hey, while we're on the subject, can we talk about frozen(set|dict) >> literals again? I really don't understand why this discussion fizzles >> out whenever it's brought up on python-ideas. > > Can you start us off by searching for previous threads discussing it, > and summarise the arguments here? And then start a new thread. :) -- ~Ethan~ From steve+comp.lang.python at pearwood.info Thu Oct 3 21:30:19 2013 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: 04 Oct 2013 01:30:19 GMT Subject: Tail recursion to while iteration in 2 easy steps References: <87had0axxy.fsf@dpt-info.u-strasbg.fr> <524cc73a$0$29984$c3e8da3$5496439d@news.astraweb.com> <524CCC8D.6020700@mrabarnett.plus.com> Message-ID: <524e1a2a$0$29984$c3e8da3$5496439d@news.astraweb.com> On Wed, 02 Oct 2013 22:41:00 -0400, Terry Reedy wrote: > I am referring to constant-value objects included in the code object. > >>> def f(): return (1,2,3) > > >>> f.__code__.co_consts > (None, 1, 2, 3, (1, 2, 3)) Okay, now that's more clear. I didn't understand what you meant before. So long as we understand we're talking about a CPython implementation detail. > None is present as the default return, even if not needed for a > particular function. Every literal is also tossed in, whether needed or > not. > >>>> which in Python 3.3 understands tuples like (1, 2, 3), but not lists. > > The byte-code does not understand anything about types. LOAD_CONST n > simply loads the (n+1)st object in .co_consts onto the top of the stack. Right, this is more clear to me now. As I understand it, the contents of code objects are implementation details, not required for implementations. For example, IronPython provides a co_consts attribute, but it only contains None. Jython doesn't provide a co_consts attribute at all. So while it's interesting to discuss what CPython does, we should not be fooled into thinking that this is guaranteed by every Python. I can imagine a Python implementation that compiles constants into some opaque object like __closure__ or co_code. In that case, it could treat the list in "for i in [1, 2, 3]: ..." as a constant too, since there is no fear that some other object could reach into the opaque object and change it. Of course, that would likely be a lot of effort for very little benefit. The compiler would have to be smart enough to see that the list was never modified or returned. Seems like a lot of trouble to go to just to save creating a small list. More likely would be implementations that didn't re-use constants, than implementations that aggressively re-used everything possible. -- Steven From steve+comp.lang.python at pearwood.info Wed Oct 2 22:34:15 2013 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: 03 Oct 2013 02:34:15 GMT Subject: Tail recursion to while iteration in 2 easy steps References: <87had0axxy.fsf@dpt-info.u-strasbg.fr> <524cc73a$0$29984$c3e8da3$5496439d@news.astraweb.com> Message-ID: <524cd7a7$0$29984$c3e8da3$5496439d@news.astraweb.com> On Thu, 03 Oct 2013 02:46:53 +0100, MRAB wrote: > On 03/10/2013 02:39, Dave Angel wrote: >> On 2/10/2013 21:24, Steven D'Aprano wrote: >> >>> On Wed, 02 Oct 2013 18:17:06 -0400, Terry Reedy wrote: >>> >>>> CPython core developers have be very conservative about what >>>> tranformations they put into the compiler. (1,2,3) can always be >>>> compiled as a constant, and so it is. [1,2,3] might or might not be a >>>> constant, depending on the context, and no attempt is made to analyze >>>> that. >>> >>> The first sentence of this is correct. The next two don't quite make >>> sense to me, since I don't understand what you mean by "constant" in >>> this context. I *think* you might be referring to the LOAD_CONST >>> byte-code, which in Python 3.3 understands tuples like (1, 2, 3), but >>> not lists. So a literal (1, 2, 3) gets created at compile-time with a >>> single LOAD_CONST call: >>> >>> py> from dis import dis >>> py> dis(compile("x = (1, 2, 3)", '', 'exec')) >>> 1 0 LOAD_CONST 4 ((1, 2, 3)) >>> 3 STORE_NAME 0 (x) >>> 6 LOAD_CONST 3 (None) >>> 9 RETURN_VALUE >>> >>> >>> while a literal [1, 2, 3] does not: >>> >>> >>> >> The difference is that a tuple can be reused, so it makes sense for the >> comiler to produce it as a const. (Much like the interning of small >> integers) The list, however, would always have to be copied from the >> compile-time object. So that object itself would be a phantom, used >> only as the template with which the list is to be made. >> > The key point here is that the tuple is immutable, including its items. You are both assuming that LOAD_CONST will re-use the same tuple (1, 2, 3) in multiple places. But that's not the case, as a simple test will show you: # Python 3.3 py> def f(): ... a = (1, 2, 3) ... b = (1, 2, 3) ... return a is b ... py> f() # Are the tuples the same object? False py> from dis import dis py> dis(f) 2 0 LOAD_CONST 4 ((1, 2, 3)) 3 STORE_FAST 0 (a) 3 6 LOAD_CONST 5 ((1, 2, 3)) 9 STORE_FAST 1 (b) 4 12 LOAD_FAST 0 (a) 15 LOAD_FAST 1 (b) 18 COMPARE_OP 8 (is) 21 RETURN_VALUE So even though both a and b are created by the same LOAD_CONST byte-code, the object is not re-used (although it could be!) and two distinct tuples are created. Re-use of objects (caching or interning) is an implementation optimization, not a language feature. An implementation might choose to cache ints, tuples, strings, floats, or none of them at all. That in no way affects whether the LOAD_CONST byte-code is used. In fact, LOAD_CONST may behave differently inside functions than outside: in CPython, functions will cache some literals in the function attribute __code__.co_consts, and LOAD_CONST may use that, while outside of a function, only small ints and identifier-like strings are cached but very little else. Other implementations may do differently -- I'm not sure whether __code__ is a public language feature or an implementation feature, but what goes into co_consts certainly isn't fixed. IronPython 2.6 doesn't appear to put anything in co_consts except None. And of course, *all of this is subject to change*, since it is not language semantics but implementation details. If HyperPython8.5 aggressively caches every tuple it can, while SimplePython1.1 uses BUILD_TUPLE rather than LOAD_CONST, both are still perfectly compliant Python implementations. (HyperPython probably will require a huge amount of memory, and SimplePython will probably be slow, but those are quality of implementation issues.) Aside: a sufficiently smart optimizing compiler could optimize function f above to either LOAD_CONST (True) RETURN_VALUE or LOAD_CONST (False) RETURN_VALUE and still be a correct Python implementation. Since Python the language doesn't specify when, if ever, objects should be cached, it could even randomly choose between those two options at compile time, or even at runtime, and still be correct. -- Steven From rosuav at gmail.com Thu Oct 3 00:14:27 2013 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 3 Oct 2013 14:14:27 +1000 Subject: Tail recursion to while iteration in 2 easy steps In-Reply-To: <524cd7a7$0$29984$c3e8da3$5496439d@news.astraweb.com> References: <87had0axxy.fsf@dpt-info.u-strasbg.fr> <524cc73a$0$29984$c3e8da3$5496439d@news.astraweb.com> <524cd7a7$0$29984$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Thu, Oct 3, 2013 at 12:34 PM, Steven D'Aprano wrote: > py> def f(): > ... a = (1, 2, 3) > ... b = (1, 2, 3) > ... return a is b > ... > py> f() # Are the tuples the same object? > False That just means the compiler doesn't detect reuse of the same tuple. But compare: >>> def f(): return (1,2,3) >>> f() is f() True Every time the function's called, it returns the same tuple (which obviously can't be done with lists). And of course if that would be dangerous, it's not done: >>> def f(): return (1,[2],3) >>> f()[1].append("Hello") >>> f() (1, [2], 3) >>> import dis >>> dis.dis(f) 2 0 LOAD_CONST 1 (1) 3 LOAD_CONST 2 (2) 6 BUILD_LIST 1 9 LOAD_CONST 3 (3) 12 BUILD_TUPLE 3 15 RETURN_VALUE ChrisA From random832 at fastmail.us Thu Oct 3 10:16:29 2013 From: random832 at fastmail.us (random832 at fastmail.us) Date: Thu, 03 Oct 2013 10:16:29 -0400 Subject: Tail recursion to while iteration in 2 easy steps In-Reply-To: <524cd7a7$0$29984$c3e8da3$5496439d@news.astraweb.com> References: <87had0axxy.fsf@dpt-info.u-strasbg.fr> <524cc73a$0$29984$c3e8da3$5496439d@news.astraweb.com> <524cd7a7$0$29984$c3e8da3$5496439d@news.astraweb.com> Message-ID: <1380809789.13619.29567925.0B0098F2@webmail.messagingengine.com> On Wed, Oct 2, 2013, at 22:34, Steven D'Aprano wrote: > You are both assuming that LOAD_CONST will re-use the same tuple > (1, 2, 3) in multiple places. But that's not the case, as a simple test > will show you: >>> def f(): ... return (1, 2, 3) >>> f() is f() True It does, in fact, re-use it when it is _the same LOAD_CONST instruction_. From tjreedy at udel.edu Thu Oct 3 15:04:21 2013 From: tjreedy at udel.edu (Terry Reedy) Date: Thu, 03 Oct 2013 15:04:21 -0400 Subject: Tail recursion to while iteration in 2 easy steps In-Reply-To: <524cd7a7$0$29984$c3e8da3$5496439d@news.astraweb.com> References: <87had0axxy.fsf@dpt-info.u-strasbg.fr> <524cc73a$0$29984$c3e8da3$5496439d@news.astraweb.com> <524cd7a7$0$29984$c3e8da3$5496439d@news.astraweb.com> Message-ID: On 10/2/2013 10:34 PM, Steven D'Aprano wrote: > You are both assuming that LOAD_CONST will re-use the same tuple > (1, 2, 3) in multiple places. No I did not. To save tuple creation time, a pre-compiled tuple is reused when its display expression is re-executed. If I had been interested in multiple occurrences of the same display, I would have tested. >>> def f(): a = 1,'a',3333, 'bbb'; x = 1,'a',3333, 'bbb' b = 1,'a',3333, 'bbb' c = 'a' d = 3333 + 3333 >>> f.__code__.co_consts (None, 1, 'a', 3333, 'bbb', (1, 'a', 3333, 'bbb'), (1, 'a', 3333, 'bbb'), (1, 'a', 3333, 'bbb'), 6666) Empirically, ints and strings are checked for prior occurrence in co_consts before being added. I suspect None is too, but will not assume. How is the issue of multiple occurrences of constants relevant to my topic statement? Let me quote it, with misspellings corrected. "CPython core developers have been very conservative about what transformations they put into the compiler." [misspellings corrected] Aha! Your example and that above reinforce this statement. Equal tuples are not necessarily identical and cannot necessarily be substituted for each other in all code. >>> (1, 2) == (1.0, 2.0) True But replacing (1.0, 2.0) with (1, 2), by only storing the latter, would not be valid without some possibly tricky context analysis. The same is true for equal numbers, and the optimizer pays attention. >>> def g(): a = 1 b = 1.0 >>> g.__code__.co_consts (None, 1, 1.0) For numbers, the proper check is relatively easy: for item in const_list: if type(x) is type(item) and x == item: break # identical item already in working list else: const_list.append(x) Writing a valid recursive function to do the same for tuples, and proving its validity to enough other core developers to make it accepted, is much harder and hardly seems worthwhile. It would probably be easier to compare the parsed AST subtrees for the displays rather than the objects created from them. --- > py> def f(): > ... a = (1, 2, 3) > ... b = (1, 2, 3) [snip] > So even though both a and b are created by the same LOAD_CONST byte-code, I am not sure what you mean by 'created'. LOAD_CONST puts the address of an object in co_consts on the top of the virtual machine stack. > the object is not re-used (although it could be!) It can be reused, in this case, because the constant displays are identical, as defined above. > and two distinct tuples are created. Because it is not easy to make the compiler see that only one is needed. -- Terry Jan Reedy From tjreedy at udel.edu Wed Oct 2 23:06:37 2013 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 02 Oct 2013 23:06:37 -0400 Subject: Tail recursion to while iteration in 2 easy steps In-Reply-To: <524cc73a$0$29984$c3e8da3$5496439d@news.astraweb.com> References: <87had0axxy.fsf@dpt-info.u-strasbg.fr> <524cc73a$0$29984$c3e8da3$5496439d@news.astraweb.com> Message-ID: On 10/2/2013 9:24 PM, Steven D'Aprano wrote: > On Wed, 02 Oct 2013 18:17:06 -0400, Terry Reedy wrote: > >> CPython core developers have be very conservative about what >> tranformations they put into the compiler. (1,2,3) can always be >> compiled as a constant, and so it is. [1,2,3] might or might not be a >> constant, depending on the context, and no attempt is made to analyze >> that. > > The first sentence of this is correct. The next two don't quite make > sense to me, since I don't understand what you mean by "constant" in this > context. I *think* you might be referring to the LOAD_CONST byte-code, > which in Python 3.3 understands tuples like (1, 2, 3), but not lists. So > a literal (1, 2, 3) gets created at compile-time with a single LOAD_CONST > call: Answered in another response. > py> from dis import dis > py> dis(compile("x = (1, 2, 3)", '', 'exec')) > 1 0 LOAD_CONST 4 ((1, 2, 3)) > 3 STORE_NAME 0 (x) > 6 LOAD_CONST 3 (None) > 9 RETURN_VALUE > > > while a literal [1, 2, 3] does not: > > > py> dis(compile("x = [1, 2, 3]", '', 'exec')) > 1 0 LOAD_CONST 0 (1) > 3 LOAD_CONST 1 (2) > 6 LOAD_CONST 2 (3) > 9 BUILD_LIST 3 > 12 STORE_NAME 0 (x) > 15 LOAD_CONST 3 (None) > 18 RETURN_VALUE > > > But I don't think this is a necessary language limitation. Both (1, 2, 3) > and [1, 2, 3] are known at compile time: the first cannot be anything > other than a tuple of three ints, and the second a list of three ints. Given introspectable code objects, the list must be built as runtime from the three ints to guarantee that. > seems to me that an implementation might provide a single byte-code to > build list literals, perhaps even LOAD_CONST itself. There are list displays, but not list literals. The distinction is important. The BUILD_LIST byte code is used above. LOAD_CONST 4 (1,2,3) BUILD_LIST_FROM_TUPLE_CONSTANT would be possible for the special case but hardly worthwhile. > The byte-codes used by the Python VM are not part of the language definition, which is why I specified CPython as the context, with 'current' as the default. > and are subject to change without warning. > > And in fact, if we go all the way back to Python 1.5, even tuple literals > weren't handled by a single byte-code, they were assembled at runtime > like lists still are: > > [steve at ando ~]$ python1.5 > Python 1.5.2 (#1, Aug 27 2012, 09:09:18) [GCC 4.1.2 20080704 (Red Hat > 4.1.2-52)] on linux2 > Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam >>>> from dis import dis >>>> dis(compile("x = (1, 2, 3)", '', 'exec')) > 0 SET_LINENO 0 > > 3 SET_LINENO 1 > 6 LOAD_CONST 0 (1) > 9 LOAD_CONST 1 (2) > 12 LOAD_CONST 2 (3) > 15 BUILD_TUPLE 3 > 18 STORE_NAME 0 (x) > 21 LOAD_CONST 3 (None) > 24 RETURN_VALUE Extending pre-complilation to tuples with nested constant tuples is even more recent. I 3.3.2, we have >>> f.__code__.co_consts (None, 1, 2, 3, (1, 2, 3)) >>> def f(): return ((1,2,3), (4,5)) >>> f.__code__.co_consts (None, 1, 2, 3, 4, 5, (1, 2, 3), (4, 5), ((1, 2, 3), (4, 5))) but I am sure if you go back you can find versions that lack the last item. -- The language is as conservative about mandating optimizations as the implementation is about doing them. I consider making None, False, True be un-rebindable keynames to be an optimization. This is not even for the other singletons Ellipsis and NotImplemented. I cannot think of too much else. Tuple constant optimization is not mandated. It would be as out of character for the language to require tail-recursion optimization as for CPython to do it. -- Terry Jan Reedy From duncan.booth at invalid.invalid Thu Oct 3 10:52:41 2013 From: duncan.booth at invalid.invalid (Duncan Booth) Date: 3 Oct 2013 14:52:41 GMT Subject: Tail recursion to while iteration in 2 easy steps References: <87had0axxy.fsf@dpt-info.u-strasbg.fr> Message-ID: Alain Ketterlin wrote: > Terry Reedy writes: > >> Part of the reason that Python does not do tail call optimization is >> that turning tail recursion into while iteration is almost trivial, >> once you know the secret of the two easy steps. Here it is. >> >> Assume that you have already done the work of turning a body recursive >> ('not tail recursive') form like >> >> def fact(n): return 1 if n <= 1 else n * fact(n-1) >> >> into a tail recursion like > [...] > > How do know that either "<=" or "*" didn't rebind the name "fact" to > something else? I think that's the main reason why python cannot apply > any procedural optimization (even things like inlining are impossible, > or possible only under very conservative assumption, that make it > worthless). > That isn't actually sufficient reason. It isn't hard to imagine adding a TAIL_CALL opcode to the interpreter that checks whether the function to be called is the same as the current function and if it is just updates the arguments and jumps to the start of the code block. If the function doesn't match it would simply fall through to doing the same as the current CALL_FUNCTION opcode. There is an issue that you would lose stack frames in any traceback. Also it means code for this modified Python wouldn't run on other non-modified interpreters, but it is at least theoretically possible without breaking Python's assumptions. -- Duncan Booth http://kupuguy.blogspot.com From neilc at norwich.edu Thu Oct 3 12:03:58 2013 From: neilc at norwich.edu (Neil Cerutti) Date: 3 Oct 2013 16:03:58 GMT Subject: Tail recursion to while iteration in 2 easy steps References: <87had0axxy.fsf@dpt-info.u-strasbg.fr> Message-ID: On 2013-10-03, Duncan Booth wrote: >> How do know that either "<=" or "*" didn't rebind the name >> "fact" to something else? I think that's the main reason why >> python cannot apply any procedural optimization (even things >> like inlining are impossible, or possible only under very >> conservative assumption, that make it worthless). > > That isn't actually sufficient reason. > > It isn't hard to imagine adding a TAIL_CALL opcode to the > interpreter that checks whether the function to be called is > the same as the current function and if it is just updates the > arguments and jumps to the start of the code block. Tail call optimization doesn't involve verification that the function is calling itself; you just have to verfify that the call is in tail position. The current frame would be removed from the stack frame and replaced with the one that results from calling the function. > There is an issue that you would lose stack frames in any > traceback. I don't think that's a major issue. Frames that got replaced would quite uninteresting. > Also it means code for this modified Python wouldn't run on > other non-modified interpreters, but it is at least > theoretically possible without breaking Python's assumptions. In any case it's so easy to implement yourself I'm not sure there's any point. -- Neil Cerutti From duncan.booth at invalid.invalid Fri Oct 4 06:16:26 2013 From: duncan.booth at invalid.invalid (Duncan Booth) Date: 4 Oct 2013 10:16:26 GMT Subject: Tail recursion to while iteration in 2 easy steps References: <87had0axxy.fsf@dpt-info.u-strasbg.fr> Message-ID: Neil Cerutti wrote: > On 2013-10-03, Duncan Booth wrote: >> It isn't hard to imagine adding a TAIL_CALL opcode to the >> interpreter that checks whether the function to be called is >> the same as the current function and if it is just updates the >> arguments and jumps to the start of the code block. > > Tail call optimization doesn't involve verification that the > function is calling itself; you just have to verfify that the > call is in tail position. You misunderstood me. As usually implemented tail call recursion doesn't require verifying that the function is calling itself, but in Python the function could be rebound so a check would be a necessary protection against this unlikely situation. Consider this Python: @some_decorator def fact(n, acc=1): if n <= 1: return acc return fact(n-1, n*acc) Is that tail recursion or not? You cannot tell whether or not that is tail recursion unless you know the definition of 'some_decorator' and even then the answer may vary from run to run. -- Duncan Booth http://kupuguy.blogspot.com From ian.g.kelly at gmail.com Fri Oct 4 06:41:28 2013 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Fri, 4 Oct 2013 04:41:28 -0600 Subject: Tail recursion to while iteration in 2 easy steps In-Reply-To: References: <87had0axxy.fsf@dpt-info.u-strasbg.fr> Message-ID: On Fri, Oct 4, 2013 at 4:16 AM, Duncan Booth wrote: > Neil Cerutti wrote: >> On 2013-10-03, Duncan Booth wrote: >>> It isn't hard to imagine adding a TAIL_CALL opcode to the >>> interpreter that checks whether the function to be called is >>> the same as the current function and if it is just updates the >>> arguments and jumps to the start of the code block. >> >> Tail call optimization doesn't involve verification that the >> function is calling itself; you just have to verfify that the >> call is in tail position. > > You misunderstood me. As usually implemented tail call recursion doesn't > require verifying that the function is calling itself, but in Python the > function could be rebound so a check would be a necessary protection > against this unlikely situation. Consider this Python: > > @some_decorator > def fact(n, acc=1): > if n <= 1: > return acc > return fact(n-1, n*acc) > > Is that tail recursion or not? > > You cannot tell whether or not that is tail recursion unless you know the > definition of 'some_decorator' and even then the answer may vary from run > to run. There is no doubt that it's a tail call. Whether it is recursion is irrelevant to optimizing it. The reason we talk about "tail call recursion" specifically is because the recursive case is the one that makes the optimization worthwhile, not because the recursion is necessary to perform the optimization. It is possible that fact is recursive but that some_decorator adds additional stack frames that are not tail calls and not optimizable. If this were the case, then the recursion would still eventually hit the stack limit, but there would still be benefit in optimizing the "fact" frames, as it would increase the allowable depth before the stack limit is reached. So I see no reason to check the identity of the called function before optimizing the tail call. From ian.g.kelly at gmail.com Fri Oct 4 06:46:20 2013 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Fri, 4 Oct 2013 04:46:20 -0600 Subject: Tail recursion to while iteration in 2 easy steps In-Reply-To: References: <87had0axxy.fsf@dpt-info.u-strasbg.fr> Message-ID: On Fri, Oct 4, 2013 at 4:41 AM, Ian Kelly wrote: > There is no doubt that it's a tail call. Whether it is recursion is > irrelevant to optimizing it. The reason we talk about "tail call > recursion" specifically is because the recursive case is the one that > makes the optimization worthwhile, not because the recursion is > necessary to perform the optimization. > > It is possible that fact is recursive but that some_decorator adds > additional stack frames that are not tail calls and not optimizable. > If this were the case, then the recursion would still eventually hit > the stack limit, but there would still be benefit in optimizing the > "fact" frames, as it would increase the allowable depth before the > stack limit is reached. So I see no reason to check the identity of > the called function before optimizing the tail call. On the other hand, if you start optimizing every tail call and not just the recursive functions, then I can see where that could start to get problematic for debugging -- as arbitrary functions get removed from the stack traces just because they happened to end in tail calls. From tjreedy at udel.edu Fri Oct 4 17:14:55 2013 From: tjreedy at udel.edu (Terry Reedy) Date: Fri, 04 Oct 2013 17:14:55 -0400 Subject: Tail recursion to while iteration in 2 easy steps In-Reply-To: References: <87had0axxy.fsf@dpt-info.u-strasbg.fr> Message-ID: On 10/4/2013 6:46 AM, Ian Kelly wrote: > On the other hand, if you start optimizing every tail call and not > just the recursive functions, then I can see where that could start to > get problematic for debugging -- as arbitrary functions get removed > from the stack traces just because they happened to end in tail calls. The idea of CPython space-optimizing tail calls when the call is made has been suggested on python-ideas. Guido verified that it is technically possible with the current bytecode interpreter but rejected it because it would arbitrarily mess up stack traces. -- Terry Jan Reedy From antoon.pardon at rece.vub.ac.be Sat Oct 5 03:39:40 2013 From: antoon.pardon at rece.vub.ac.be (Antoon Pardon) Date: Sat, 05 Oct 2013 09:39:40 +0200 Subject: Tail recursion to while iteration in 2 easy steps In-Reply-To: References: <87had0axxy.fsf@dpt-info.u-strasbg.fr> Message-ID: <524FC23C.70303@rece.vub.ac.be> Op 04-10-13 23:14, Terry Reedy schreef: > On 10/4/2013 6:46 AM, Ian Kelly wrote: > >> On the other hand, if you start optimizing every tail call and not >> just the recursive functions, then I can see where that could start to >> get problematic for debugging -- as arbitrary functions get removed >> from the stack traces just because they happened to end in tail calls. > > The idea of CPython space-optimizing tail calls when the call is made > has been suggested on python-ideas. Guido verified that it is > technically possible with the current bytecode interpreter but rejected > it because it would arbitrarily mess up stack traces. What does this mean? Does it mean that a naive implementation would arbitrarily mess up stack traces and he wasn't interested in investigating more sophisticated implementations? Does it mean he just didn't like the idea a stack trace wouldn't be a 100% represenatation of the active call history? Does it mean he investigated more sphisticated implementations but found them to have serious short comings that couldn't be remedied easily? -- Antoon Pardon From random832 at fastmail.us Mon Oct 7 17:27:18 2013 From: random832 at fastmail.us (random832 at fastmail.us) Date: Mon, 07 Oct 2013 17:27:18 -0400 Subject: Tail recursion to while iteration in 2 easy steps In-Reply-To: <524FC23C.70303@rece.vub.ac.be> References: <87had0axxy.fsf@dpt-info.u-strasbg.fr> <524FC23C.70303@rece.vub.ac.be> Message-ID: <1381181238.8467.31158469.2DEAAC88@webmail.messagingengine.com> On Sat, Oct 5, 2013, at 3:39, Antoon Pardon wrote: > What does this mean? > > Does it mean that a naive implementation would arbitrarily mess up > stack traces and he wasn't interested in investigating more > sophisticated implementations? > > Does it mean he just didn't like the idea a stack trace wouldn't be a > 100% represenatation of the active call history? > > Does it mean he investigated more sphisticated implementations but found > them to have serious short comings that couldn't be remedied easily? The entire point of tail call optimization requires not keeping the intervening stack frames around, in _any_ form, so as to allow arbitrarily deep recursion without ever having the possibility of a stack overflow. An implementation which reduced but did not eliminate the space used per call would not be worthwhile because it would not enable the recursive functional programming patterns that mandatory tail call optimization allows. You could require that an "optimizable tail call" be made explicit. Actually, I suspect it might be possible to do this now, by abusing exception handling as a control flow mechanism. From antoon.pardon at rece.vub.ac.be Tue Oct 8 05:43:33 2013 From: antoon.pardon at rece.vub.ac.be (Antoon Pardon) Date: Tue, 08 Oct 2013 11:43:33 +0200 Subject: Tail recursion to while iteration in 2 easy steps In-Reply-To: <1381181238.8467.31158469.2DEAAC88@webmail.messagingengine.com> References: <87had0axxy.fsf@dpt-info.u-strasbg.fr> <524FC23C.70303@rece.vub.ac.be> <1381181238.8467.31158469.2DEAAC88@webmail.messagingengine.com> Message-ID: <5253D3C5.1010903@rece.vub.ac.be> Op 07-10-13 23:27, random832 at fastmail.us schreef: > On Sat, Oct 5, 2013, at 3:39, Antoon Pardon wrote: >> What does this mean? >> >> Does it mean that a naive implementation would arbitrarily mess up >> stack traces and he wasn't interested in investigating more >> sophisticated implementations? >> >> Does it mean he just didn't like the idea a stack trace wouldn't be a >> 100% represenatation of the active call history? >> >> Does it mean he investigated more sphisticated implementations but found >> them to have serious short comings that couldn't be remedied easily? > > The entire point of tail call optimization requires not keeping the > intervening stack frames around, in _any_ form, so as to allow > arbitrarily deep recursion without ever having the possibility of a > stack overflow. An implementation which reduced but did not eliminate > the space used per call would not be worthwhile because it would not > enable the recursive functional programming patterns that mandatory tail > call optimization allows. So? What about an implementation that would keep its stackframes normally until it deteced recursion had occured. From then on it would only keep what it already had plus the four top stackframes (assuming it are all tail calls for the moment). This would allow for a stacktrace of the last four calls and essentially doesn't need any space per call from then on. -- Antoon Pardon From jpiitula at ling.helsinki.fi Tue Oct 8 03:11:13 2013 From: jpiitula at ling.helsinki.fi (Jussi Piitulainen) Date: 08 Oct 2013 10:11:13 +0300 Subject: Tail recursion to while iteration in 2 easy steps References: <87had0axxy.fsf@dpt-info.u-strasbg.fr> <524FC23C.70303@rece.vub.ac.be> Message-ID: random832 at fastmail.us writes: > The entire point of tail call optimization requires not keeping the > intervening stack frames around, in _any_ form, so as to allow > arbitrarily deep recursion without ever having the possibility of a > stack overflow. An implementation which reduced but did not > eliminate the space used per call would not be worthwhile because it > would not enable the recursive functional programming patterns that > mandatory tail call optimization allows. > > You could require that an "optimizable tail call" be made explicit. > Actually, I suspect it might be possible to do this now, by abusing > exception handling as a control flow mechanism. Python code already marks many of the functionally relevant tail calls with 'return'. It just wants to retain the trace. Another keyword could be used to indicate that the programmer does not want a stack frame retained. It's probably too much to suggest 'please return', but how about 'goto return'? A tail call is a 'goto that passes arguments', and I think 'goto' is a keyword already. (Actually I just wanted to suggest 'please return'. Not seriously.) From duncan.booth at invalid.invalid Fri Oct 4 07:16:44 2013 From: duncan.booth at invalid.invalid (Duncan Booth) Date: 4 Oct 2013 11:16:44 GMT Subject: Tail recursion to while iteration in 2 easy steps References: <87had0axxy.fsf@dpt-info.u-strasbg.fr> Message-ID: Ian Kelly wrote: > On Fri, Oct 4, 2013 at 4:41 AM, Ian Kelly wrote: >> There is no doubt that it's a tail call. Whether it is recursion is >> irrelevant to optimizing it. The reason we talk about "tail call >> recursion" specifically is because the recursive case is the one that >> makes the optimization worthwhile, not because the recursion is >> necessary to perform the optimization. >> >> It is possible that fact is recursive but that some_decorator adds >> additional stack frames that are not tail calls and not optimizable. >> If this were the case, then the recursion would still eventually hit >> the stack limit, but there would still be benefit in optimizing the >> "fact" frames, as it would increase the allowable depth before the >> stack limit is reached. So I see no reason to check the identity of >> the called function before optimizing the tail call. > > On the other hand, if you start optimizing every tail call and not > just the recursive functions, then I can see where that could start to > get problematic for debugging -- as arbitrary functions get removed > from the stack traces just because they happened to end in tail calls. Quite so. Losing some stack frames in the traceback because tail recursion was optimised is probably no big deal. Losing arbitrary stack frames because of a more widespread tail call optimisation would not IMHO fit with the spirit of Python except possibly as an optional optimisation that was off by default. -- Duncan Booth http://kupuguy.blogspot.com From jpiitula at ling.helsinki.fi Fri Oct 4 07:11:07 2013 From: jpiitula at ling.helsinki.fi (Jussi Piitulainen) Date: 04 Oct 2013 14:11:07 +0300 Subject: Tail recursion to while iteration in 2 easy steps References: <87had0axxy.fsf@dpt-info.u-strasbg.fr> Message-ID: Duncan Booth writes: > Neil Cerutti wrote: > > On 2013-10-03, Duncan Booth wrote: > >> It isn't hard to imagine adding a TAIL_CALL opcode to the > >> interpreter that checks whether the function to be called is > >> the same as the current function and if it is just updates the > >> arguments and jumps to the start of the code block. > > > > Tail call optimization doesn't involve verification that the > > function is calling itself; you just have to verfify that the > > call is in tail position. > > You misunderstood me. As usually implemented tail call recursion > doesn't require verifying that the function is calling itself, but > in Python the function could be rebound so a check would be a > necessary protection against this unlikely situation. Consider this > Python: > > @some_decorator > def fact(n, acc=1): > if n <= 1: > return acc > return fact(n-1, n*acc) > > Is that tail recursion or not? > > You cannot tell whether or not that is tail recursion unless you > know the definition of 'some_decorator' and even then the answer may > vary from run to run. Ignoring the decorator, fact(n-1, n*acc) is in a tail position because it follows the keyword return. It doesn't matter what function fact is at the time: the remaining action in the caller is to return. Tail positions, with respect to enclosing code, are defined syntactically. From random832 at fastmail.us Wed Oct 2 08:31:25 2013 From: random832 at fastmail.us (random832 at fastmail.us) Date: Wed, 02 Oct 2013 08:31:25 -0400 Subject: Tail recursion to while iteration in 2 easy steps In-Reply-To: References: Message-ID: <1380717085.6985.29063445.6AD889D5@webmail.messagingengine.com> On Tue, Oct 1, 2013, at 17:30, Terry Reedy wrote: > Part of the reason that Python does not do tail call optimization is > that turning tail recursion into while iteration is almost trivial, once > you know the secret of the two easy steps. Here it is. That should be a reason it _does_ do it - saying people should rewrite their functions with loops means declaring that Python is not really a multi-paradigm programming language but rather rejects functional programming styles in favor of imperative ones. From tjreedy at udel.edu Wed Oct 2 17:33:27 2013 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 02 Oct 2013 17:33:27 -0400 Subject: Tail recursion to while iteration in 2 easy steps In-Reply-To: <1380717085.6985.29063445.6AD889D5@webmail.messagingengine.com> References: <1380717085.6985.29063445.6AD889D5@webmail.messagingengine.com> Message-ID: On 10/2/2013 8:31 AM, random832 at fastmail.us wrote: > On Tue, Oct 1, 2013, at 17:30, Terry Reedy wrote: >> Part of the reason that Python does not do tail call optimization is >> that turning tail recursion into while iteration is almost trivial, once >> you know the secret of the two easy steps. Here it is. > > That should be a reason it _does_ do it - saying people should rewrite > their functions with loops means declaring that Python is not really a > multi-paradigm programming language but rather rejects functional > programming styles in favor of imperative ones. It is true that Python does not encourage the particular functional style that is encouraged by auto optimization of tail recursion. A different functional style would often use reduce (or fold) instead. Some other points I left out in a post of medium length yet brief for the topic. 1. If one starts with body recursion, as is typical, one must consider commutativity (possibly associativity) of the 'inner' operator in any conversion. 2. Instead of converting to tail recursion, one might convert to while iteration directly. 3. One often 'polishes' the while form in a way that cannot be done automatically. 4. While loops are actually rare in idiomatic Python code. In Python, for loops are the standard way to linearly process a collection. The final version I gave for a factorial while loop, def fact_while(n): if n < 0 or n != int(n): raise ValueError('fact input {} is not a count'.format(n)) fac = 1 while n > 1: fac *= n n -= 1 return fac should better be written with a for loop: def fact_for(n): if n < 0 or n != int(n): raise ValueError('fact input {} is not a count'.format(n)) fac = 1: for i in range(2, n): fac *= n When the input to a function is an iterable instead of n, the iterable should be part of the for loop source expression. For loops are integrated with Python's iterator protocol in much the same way that recursion is integrated with list first:rest pattern matching in some functional languages. It is true that Python encourages the use of for loops and for clauses in comprehensions (a functional construct). 5. Conversion of apparent recursion to iteration assumes that the function really is intended to be recursive. This assumption is the basis for replacing the recursive call with assignment and an implied internal goto. The programmer can determine that this semantic change is correct; the compiler should not assume that. (Because of Python's late name-binding semantics, recursive *intent* is better expressed in Python with iterative syntax than function call syntax. ) -- Terry Jan Reedy From random832 at fastmail.us Thu Oct 3 10:09:25 2013 From: random832 at fastmail.us (random832 at fastmail.us) Date: Thu, 03 Oct 2013 10:09:25 -0400 Subject: Tail recursion to while iteration in 2 easy steps In-Reply-To: References: <1380717085.6985.29063445.6AD889D5@webmail.messagingengine.com> Message-ID: <1380809365.10255.29564313.66DA359E@webmail.messagingengine.com> On Wed, Oct 2, 2013, at 17:33, Terry Reedy wrote: > 5. Conversion of apparent recursion to iteration assumes that the > function really is intended to be recursive. This assumption is the > basis for replacing the recursive call with assignment and an implied > internal goto. The programmer can determine that this semantic change is > correct; the compiler should not assume that. (Because of Python's late > name-binding semantics, recursive *intent* is better expressed in Python > with iterative syntax than function call syntax. ) Speaking of assumptions, I would almost say that we should make the assumption that operators (other than the __i family, and setitem/setattr/etc) are not intended to have visible side effects. This would open a _huge_ field of potential optimizations - including that this would no longer be a semantic change (since relying on one of the operators being allowed to change the binding of fact would no longer be guaranteed). From steve+comp.lang.python at pearwood.info Thu Oct 3 22:05:00 2013 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: 04 Oct 2013 02:05:00 GMT Subject: Tail recursion to while iteration in 2 easy steps References: <1380717085.6985.29063445.6AD889D5@webmail.messagingengine.com> Message-ID: <524e224b$0$29984$c3e8da3$5496439d@news.astraweb.com> On Thu, 03 Oct 2013 10:09:25 -0400, random832 wrote: > Speaking of assumptions, I would almost say that we should make the > assumption that operators (other than the __i family, and > setitem/setattr/etc) are not intended to have visible side effects. This > would open a _huge_ field of potential optimizations - including that > this would no longer be a semantic change (since relying on one of the > operators being allowed to change the binding of fact would no longer be > guaranteed). I like the idea of such optimizations, but I'm afraid that your last sentence seems a bit screwy to me. You seem to be saying, if we make this major semantic change to Python, we can then retroactively declare that it's not a semantic change at all, since under the new rules, it's no different from the new rules. Anyway... I think that it's something worth investigating, but it's not as straight forward as you might hope. There almost certainly is code out in the world that uses operator overloading for DSLs. For instance, I've played around something vaguely like this DSL: chain = Node('spam') chain >> 'eggs' chain >> 'ham' chain.head <= 'cheese' where I read >> as appending and <= as inserting. I was never quite happy with the syntax, so my experiments never went anywhere, but I expect that some people, somewhere, have. This is a legitimate way to use Python, and changing the semantics to prohibit it would be a Bad Thing. However, I can imagine something like a __future__ directive that enables, or disables, such optimizations on a per-module basis. In Python 3, it would have to be disabled by default. Python 4000 could make the optimizations enabled by default and use the __future__ machinery to disable it. -- Steven From dreamingforward at gmail.com Wed Oct 2 17:39:40 2013 From: dreamingforward at gmail.com (Mark Janssen) Date: Wed, 2 Oct 2013 14:39:40 -0700 Subject: Tail recursion to while iteration in 2 easy steps In-Reply-To: <1380717085.6985.29063445.6AD889D5@webmail.messagingengine.com> References: <1380717085.6985.29063445.6AD889D5@webmail.messagingengine.com> Message-ID: >> Part of the reason that Python does not do tail call optimization is >> that turning tail recursion into while iteration is almost trivial, once >> you know the secret of the two easy steps. Here it is. > > That should be a reason it _does_ do it - saying people should rewrite > their functions with loops means declaring that Python is not really a > multi-paradigm programming language but rather rejects functional > programming styles in favor of imperative ones. Yes, but that's fine. A PL language that includes every programming paradigm would be a total mess, if even possible. Python has functional programming where it does not conflict with its overall design. The only place I find that this is not the case is with lambda, but that is now adequately fixed with the addition of the ternary operator. -- MarkJ Tacoma, Washington From dihedral88888 at gmail.com Fri Oct 4 05:13:07 2013 From: dihedral88888 at gmail.com (88888 Dihedral) Date: Fri, 4 Oct 2013 02:13:07 -0700 (PDT) Subject: Tail recursion to while iteration in 2 easy steps In-Reply-To: References: <1380717085.6985.29063445.6AD889D5@webmail.messagingengine.com> Message-ID: <163e6437-2a2c-4af0-a549-1be6a164ed11@googlegroups.com> On Thursday, October 3, 2013 5:33:27 AM UTC+8, Terry Reedy wrote: > On 10/2/2013 8:31 AM, random832 at fastmail.us wrote: > > > On Tue, Oct 1, 2013, at 17:30, Terry Reedy wrote: > > >> Part of the reason that Python does not do tail call optimization is > > >> that turning tail recursion into while iteration is almost trivial, once > > >> you know the secret of the two easy steps. Here it is. > > > > > > That should be a reason it _does_ do it - saying people should rewrite > > > their functions with loops means declaring that Python is not really a > > > multi-paradigm programming language but rather rejects functional > > > programming styles in favor of imperative ones. > > > > It is true that Python does not encourage the particular functional > > style that is encouraged by auto optimization of tail recursion. A > > different functional style would often use reduce (or fold) instead. > > > > Some other points I left out in a post of medium length yet brief for > > the topic. > > > > 1. If one starts with body recursion, as is typical, one must consider > > commutativity (possibly associativity) of the 'inner' operator in any > > conversion. > > > > 2. Instead of converting to tail recursion, one might convert to while > > iteration directly. > > > > 3. One often 'polishes' the while form in a way that cannot be done > > automatically. > > > > 4. While loops are actually rare in idiomatic Python code. In Python, > > for loops are the standard way to linearly process a collection. The > > final version I gave for a factorial while loop, > > > > def fact_while(n): > > if n < 0 or n != int(n): > > raise ValueError('fact input {} is not a count'.format(n)) > > fac = 1 > > while n > 1: > > fac *= n > > n -= 1 > > return fac > > > > should better be written with a for loop: > As I pointed out before, an accelerated version without the limit of the stack depth for computing facotrials can be obtained by storing a list of products of primes first. Of course integer divisions are required to transform the to stack depth problem into the size of the 32-64 bit heap space. From steve+comp.lang.python at pearwood.info Wed Oct 2 09:32:29 2013 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: 02 Oct 2013 13:32:29 GMT Subject: Tail recursion to while iteration in 2 easy steps References: Message-ID: <524c206d$0$29984$c3e8da3$5496439d@news.astraweb.com> On Wed, 02 Oct 2013 08:31:25 -0400, random832 wrote: > On Tue, Oct 1, 2013, at 17:30, Terry Reedy wrote: >> Part of the reason that Python does not do tail call optimization is >> that turning tail recursion into while iteration is almost trivial, >> once you know the secret of the two easy steps. Here it is. > > That should be a reason it _does_ do it - saying people should rewrite > their functions with loops means declaring that Python is not really a > multi-paradigm programming language but rather rejects functional > programming styles in favor of imperative ones. Python is not as aggressively functional as (say) Haskell, but it is surely an exaggeration to suggest that the failure to include tail call optimization means that Python "rejects" functional programming styles. You can still write you code in a functional style, it just won't be as heavily optimized. -- Steven From random832 at fastmail.us Wed Oct 2 10:04:49 2013 From: random832 at fastmail.us (random832 at fastmail.us) Date: Wed, 02 Oct 2013 10:04:49 -0400 Subject: Tail recursion to while iteration in 2 easy steps In-Reply-To: <524c206d$0$29984$c3e8da3$5496439d@news.astraweb.com> References: <524c206d$0$29984$c3e8da3$5496439d@news.astraweb.com> Message-ID: <1380722689.16480.29101897.4955AB79@webmail.messagingengine.com> On Wed, Oct 2, 2013, at 9:32, Steven D'Aprano wrote: > Python is not as aggressively functional as (say) Haskell, but it is > surely an exaggeration to suggest that the failure to include tail call > optimization means that Python "rejects" functional programming styles. > You can still write you code in a functional style, it just won't be as > heavily optimized. IMO, tail call optimization is essential to writing in a functional style, since otherwise you end up with a stack overflow error on any input above a trivial size. From steve+comp.lang.python at pearwood.info Wed Oct 2 15:13:57 2013 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: 02 Oct 2013 19:13:57 GMT Subject: Tail recursion to while iteration in 2 easy steps References: <524c206d$0$29984$c3e8da3$5496439d@news.astraweb.com> Message-ID: <524c7075$0$29984$c3e8da3$5496439d@news.astraweb.com> On Wed, 02 Oct 2013 10:04:49 -0400, random832 wrote: > On Wed, Oct 2, 2013, at 9:32, Steven D'Aprano wrote: >> Python is not as aggressively functional as (say) Haskell, but it is >> surely an exaggeration to suggest that the failure to include tail call >> optimization means that Python "rejects" functional programming styles. >> You can still write you code in a functional style, it just won't be as >> heavily optimized. > > IMO, tail call optimization is essential to writing in a functional > style, since otherwise you end up with a stack overflow error on any > input above a trivial size. Hardly essential. Here's some functional code that doesn't rely on tail- call optimization for efficiency: result = map(some_function, some_iterator) In Python 3 that even uses lazy evaluation, for free. Or you can increase the amount of memory available in the stack. Another alternative would be for the compiler to convert your recursive code into iterative code. Well, that wouldn't work for Python. Not all functional code is recursive, and not all recursive functional code is tail-recursive. So while Python's lack of tail-call optimization is a failure of Best Practice functional *implementation*, it doesn't prevent you from writing in a functional *style*. Ultimately, Python does not pretend to be a pure-functional language. If you want Haskell, you know where to get it. Python steals a few good ideas from functional programming, like comprehensions and lazy-evaluated iterators, provides a few functional programming constructs like map, reduce, and filter, and gives you first-class functions as values. You can write code in a functional style in Python, but don't mistake that for Python being a functional language. -- Steven From cbf123 at mail.usask.ca Tue Oct 1 18:54:00 2013 From: cbf123 at mail.usask.ca (Chris Friesen) Date: Tue, 01 Oct 2013 16:54:00 -0600 Subject: python function parameters, debugging, comments, etc. Message-ID: <524B5288.1050709@mail.usask.ca> I've got a fair bit of programming experience (mostly kernel/POSIX stuff in C). I'm fairly new to python though, and was hoping for some advice. Given the fact that function parameters do not specify types, when you're looking at someone else's code how the heck do you know what is expected for a given argument? (Especially in a nontrivial system where the parameter is just passed on to some other function and may not be evaluated for several nested function calls.) Is the recommendation to have comments for each function describing the expected args? I was trying to debug some stuff that someone else wrote. It turned out that the problem was in code like this: def rebuild_instance(self, context, instance, image, ...) request_spec = scheduler_utils.build_request_spec(context, image, [instance]) ...stuff... other_function(...,image,...) where build_request_spec looks like: def build_request_spec(ctxt, image, instances): ...etc... and it took me a while to realize that rebuild_instance() was being passed the image ID (basically just a string), and other_function() was expecting the image ID, but build_request_spec() was expecting the actual image dictionary. It also took me a while to realize that that build_request_spec() was expecting a list of instances, while rebuild_instance() was passing in a single instance. That one is already fixed in the above code. So what's the recommended way of dealing with stuff like this in larger projects with many developers? Thanks, Chris From joel.goldstick at gmail.com Tue Oct 1 19:03:11 2013 From: joel.goldstick at gmail.com (Joel Goldstick) Date: Tue, 1 Oct 2013 19:03:11 -0400 Subject: python function parameters, debugging, comments, etc. In-Reply-To: <524B5288.1050709@mail.usask.ca> References: <524B5288.1050709@mail.usask.ca> Message-ID: On Tue, Oct 1, 2013 at 6:54 PM, Chris Friesen wrote: > > I've got a fair bit of programming experience (mostly kernel/POSIX stuff > in C). I'm fairly new to python though, and was hoping for some advice. > > Given the fact that function parameters do not specify types, when you're > looking at someone else's code how the heck do you know what is expected > for a given argument? (Especially in a nontrivial system where the > parameter is just passed on to some other function and may not be evaluated > for several nested function calls.) > > Is the recommendation to have comments for each function describing the > expected args? > > I was trying to debug some stuff that someone else wrote. It turned out > that the problem was in code like this: > > > > def rebuild_instance(self, context, instance, image, ...) > request_spec = scheduler_utils.build_request_spec(context, image, > [instance]) > ...stuff... > other_function(...,image,...) > > > where build_request_spec looks like: > > def build_request_spec(ctxt, image, instances): > ...etc... > > > and it took me a while to realize that rebuild_instance() was being passed > the image ID (basically just a string), and other_function() was expecting > the image ID, but build_request_spec() was expecting the actual image > dictionary. > > It also took me a while to realize that that build_request_spec() was > expecting a list of instances, while rebuild_instance() was passing in a > single instance. That one is already fixed in the above code. > > > So what's the recommended way of dealing with stuff like this in larger > projects with many developers? > > Thanks, > Chris > -- > https://mail.python.org/mailman/listinfo/python-list > One way is to require docstrings (the triple quoted text immediately following the def line) on your project and set out requirements for how to describe the function/method arguments there. There is a tool call pydoc that collects docstrings and makes great documentation. -- Joel Goldstick http://joelgoldstick.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From sg552 at hotmail.co.uk Tue Oct 1 19:45:55 2013 From: sg552 at hotmail.co.uk (Rotwang) Date: Wed, 02 Oct 2013 00:45:55 +0100 Subject: python function parameters, debugging, comments, etc. In-Reply-To: References: Message-ID: On 01/10/2013 23:54, Chris Friesen wrote: > > I've got a fair bit of programming experience (mostly kernel/POSIX stuff in C). I'm fairly new to python though, and was hoping for some advice. > > Given the fact that function parameters do not specify types, when you're looking at someone else's code how the heck do you know what is expected for a given argument? (Especially in a nontrivial system where the parameter is just passed on to some other function and may not be evaluated for several nested function calls.) > > Is the recommendation to have comments for each function describing the expected args? > > [...] In the Python community, one of the programming styles that is encouraged is "duck-typing". What this means is that rather than writing functions that check whether arguments passed to that function are of a specific type, the function should simply use any methods of those arguments it requires; that way the function will still work if passed an argument whose type is a custom type defined by the user which has the right interface so that the function body still makes sense (if it quacks like a duck, then the function might as well treat it like a duck). If a user passes an argument which doesn't have the right methods then the function will fail, but the traceback that the interpreter provides will often have enough information to make it clear why it failed. (see http://docs.python.org/3/glossary.html#term-duck-typing ) So the upside of duck-typing is clear. But as you've already discovered, so is the downside: Python's dynamic nature means that there's no way for the interpreter to know what kind of arguments a function will accept, and so a user of any function relies on the function having clear documentation. There are several ways to document a function; apart from comments, functions also have docstrings, which will be displayed, along with the function's signature, when you call help(function). A docstring is a string literal which occurs as the first statement of a function definition, like this: def foo(x, y = 2): '''This function takes an argument x, which should be iterable, and a function y, which should be a numeric type. It does nothing.''' pass If I call help(foo), I get this: Help on function foo in module __main__: foo(x, y=2) This function takes an argument x, which should be iterable, and a function y, which should be a numeric type. It does nothing. In Python 3.0 and later, functions can also have annotations; they have no semantics in the language itself but third-party modules can use them if they choose to do so. They look like this: def foo(x: str, y: int = 2, z: 'Hello' = None) -> tuple: return a, b, c For more about annotations, see here: http://www.python.org/dev/peps/pep-3107/ So the short answer is that Python gives you several methods for making it clear what kind of arguments the functions you define should be passed, but unfortunately you'll likely encounter functions written by people who made no use of those methods. On the plus side, Python's exception reporting is good, so if in doubt just try using a function in the interactive interpreter and see what happens (with the usual caveats about using untrusted code, obviously). From oscar.j.benjamin at gmail.com Wed Oct 2 06:15:03 2013 From: oscar.j.benjamin at gmail.com (Oscar Benjamin) Date: Wed, 2 Oct 2013 11:15:03 +0100 Subject: python function parameters, debugging, comments, etc. In-Reply-To: References: Message-ID: On 2 October 2013 00:45, Rotwang wrote: > > So the upside of duck-typing is clear. But as you've already discovered, so > is the downside: Python's dynamic nature means that there's no way for the > interpreter to know what kind of arguments a function will accept, and so a > user of any function relies on the function having clear documentation. It is still necessary to document the arguments of functions in explicitly typed languages. Knowing that you need a list of strings does not mean that you know what the function expects of the values of the strings and what it will try to do with them. When you see something like int atoi (const char * str); you know that it takes a string and returns an integer. However the function name does not clearly indicate any purpose. What kind of string should I pass in? Is the returned value an error code or a value generated from the string (it's actually both). Even if you know that the function parses strings representing integers there are still many different formats for representing numbers as strings. Should the string be in a locale-dependent format? What kind of text encoding is it using (utf-8 maybe)? Should the characters represent an integer in decimal format or hex, octal, binary or something else? Inspecting types can be a quick way to gain information about the meaning of arguments and return values but it is not something that you should be relying on to replace documentation. Oscar From sg552 at hotmail.co.uk Wed Oct 2 09:52:31 2013 From: sg552 at hotmail.co.uk (Rotwang) Date: Wed, 02 Oct 2013 14:52:31 +0100 Subject: python function parameters, debugging, comments, etc. In-Reply-To: References: Message-ID: On 02/10/2013 11:15, Oscar Benjamin wrote: > On 2 October 2013 00:45, Rotwang wrote: >> >> So the upside of duck-typing is clear. But as you've already discovered, so >> is the downside: Python's dynamic nature means that there's no way for the >> interpreter to know what kind of arguments a function will accept, and so a >> user of any function relies on the function having clear documentation. > > It is still necessary to document the arguments of functions in > explicitly typed languages. Knowing that you need a list of strings > does not mean that you know what the function expects of the values of > the strings and what it will try to do with them. > > [...] Well, yes. I didn't intend to suggest otherwise. From tjreedy at udel.edu Tue Oct 1 19:56:41 2013 From: tjreedy at udel.edu (Terry Reedy) Date: Tue, 01 Oct 2013 19:56:41 -0400 Subject: python function parameters, debugging, comments, etc. In-Reply-To: <524B5288.1050709@mail.usask.ca> References: <524B5288.1050709@mail.usask.ca> Message-ID: On 10/1/2013 6:54 PM, Chris Friesen wrote: > Given the fact that function parameters do not specify types, when you're looking at someone else's code how the heck do you know what is expected for a given argument? (Especially in a nontrivial system where the parameter is just passed on to some other function and may not be evaluated for several nested function calls.) > > Is the recommendation to have comments for each function describing the expected args? Here is an example from the stdlib. >>> print(int.__doc__) int(x=0) -> integer int(x, base=10) -> integer Convert a number or string to an integer, or return 0 if no arguments are given. If x is a number, return x.__int__(). For floating point numbers, this truncates towards zero. If x is not a number or if base is given, then x must be a string, bytes, or bytearray instance representing an integer literal in the given base. The literal can be preceded by '+' or '-' and be surrounded by whitespace. The base defaults to 10. Valid bases are 0 and 2-36. Base 0 means to interpret the base from the string as an integer literal. >>> int('0b100', base=0) 4 help(int) prints the above plus the int methods. Functions coded in Python should not have the signature in the doc string because help() can get it from the function object itself. >>> def f(a, b=3, *, c='abc'): '''return (a + b) / len(c) a and b (default 3) are numbers. c must be a sequence (default 'abc').''' >>> help(f) Help on function f in module __main__: f(a, b=3, *, c='abc') return (a + b) / len(c) a and b (default 3) are numbers. c must be a sequence (default 'abc'). >>> -- Terry Jan Reedy From ben+python at benfinney.id.au Tue Oct 1 21:51:09 2013 From: ben+python at benfinney.id.au (Ben Finney) Date: Wed, 02 Oct 2013 11:51:09 +1000 Subject: python function parameters, debugging, comments, etc. References: <524B5288.1050709@mail.usask.ca> Message-ID: <7w61tgsanm.fsf@benfinney.id.au> Chris Friesen writes: > I've got a fair bit of programming experience (mostly kernel/POSIX > stuff in C). I'm fairly new to python though, and was hoping for some > advice. Welcome! Thanks for taking the practice of programming seriously enough to seek improvement. > Is the recommendation to have comments for each function describing > the expected args? Not quite; we don't use comments for that purpose, but docstrings . Python docstrings allow documenting the API of modules, classes, and functions, in such a way that the documentation is right there in the source code *and* is available at run-time (with the ?__doc__? magic attribute). PEP 256 has links to the various specifications that give detailed information on guidelines for writing good docstrings, and for how good docstrings will be used by other tools. > So what's the recommended way of dealing with stuff like this in > larger projects with many developers? Choose community-endorsed standards ? the PEPs mentioned above, referenced from PEP 256 ? and discuss the benefits of them with your team. -- \ ?I think it would be a good idea.? ?Mohandas Gandhi (when asked | `\ what he thought of Western civilization) | _o__) | Ben Finney From andriy.kornatskyy at live.com Wed Oct 2 03:08:09 2013 From: andriy.kornatskyy at live.com (Andriy Kornatskyy) Date: Wed, 2 Oct 2013 10:08:09 +0300 Subject: How to manage Git or Mercurial repositories Message-ID: Managing version control repositories can be a challenge in multi-user environment especially when simplification of user collaboration is your goal. There are usually two primary concerns while considering enterprise deployment for version control repositories: access control and safety of your data. Both are not directly addressed by version control itself, thus a sort of security facade is necessary. Read more here: http://mindref.blogspot.com/2013/10/how-to-manage-git-or-mercurial.html Thanks. Andriy Kornatskyy From michael at stroeder.com Tue Oct 15 04:10:19 2013 From: michael at stroeder.com (=?UTF-8?B?TWljaGFlbCBTdHLDtmRlcg==?=) Date: Tue, 15 Oct 2013 10:10:19 +0200 Subject: How to manage Git or Mercurial repositories In-Reply-To: References: Message-ID: Andriy Kornatskyy wrote: > Managing version control repositories can be a challenge in multi-user environment especially when simplification of user collaboration is your goal. There are usually two primary concerns while considering enterprise deployment for version control repositories: access control and safety of your data. Both are not directly addressed by version control itself, thus a sort of security facade is necessary. Read more here: > > http://mindref.blogspot.com/2013/10/how-to-manage-git-or-mercurial.html This seems like simple advertising to me. Ciao, Michael. From seotaewong40 at gmail.com Wed Oct 2 05:36:11 2013 From: seotaewong40 at gmail.com (Tae Wong) Date: Wed, 2 Oct 2013 18:36:11 +0900 Subject: hg.python.org: Server unresponsive and timeout Message-ID: This post is irrelevant from using Python; so it's an Internet server problem. When you try to connect to hg.python.org, the connection takes forever. How to fix this server issue? From ben+python at benfinney.id.au Wed Oct 2 06:20:42 2013 From: ben+python at benfinney.id.au (Ben Finney) Date: Wed, 02 Oct 2013 20:20:42 +1000 Subject: hg.python.org: Server unresponsive and timeout References: Message-ID: <7wk3hwq8hx.fsf@benfinney.id.au> Tae Wong writes: > When you try to connect to hg.python.org, the connection takes > forever. (Note that this is the community discussion forum for the Python langauge; it is perhaps not the best forum to report problems with python.org infrastructure.) What client are you using to connect? Is there an exact command that we can try, to see whether the problem is reproducible? > How to fix this server issue? The monitor site says that ?hg.python.org? is operational when I check, so whoever investigates the problem will need more detail on what you're trying to do and what exactly isn't working. -- \ ?The fact that I have no remedy for all the sorrows of the | `\ world is no reason for my accepting yours. It simply supports | _o__) the strong probability that yours is a fake.? ?Henry L. Mencken | Ben Finney From tjreedy at udel.edu Wed Oct 2 18:25:14 2013 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 02 Oct 2013 18:25:14 -0400 Subject: hg.python.org: Server unresponsive and timeout In-Reply-To: References: Message-ID: On 10/2/2013 5:36 AM, Tae Wong wrote: > This post is irrelevant from using Python; so it's an Internet server problem. > > When you try to connect to hg.python.org, the connection takes forever. I believe hg.python.org is on a different machine than python.org. It has occasionally been down, but works fine with Firefox. Sometimes connections fail even if both endpoints are working. If you have a particular persistent problem with hg.python.org, I suggest talking to your ISP. Cloning the CPython repository may take an hour or two if that is what you are trying to do. -- Terry Jan Reedy From ben+python at benfinney.id.au Wed Oct 2 19:07:53 2013 From: ben+python at benfinney.id.au (Ben Finney) Date: Thu, 03 Oct 2013 09:07:53 +1000 Subject: hg.python.org: Server unresponsive and timeout References: Message-ID: <7w8uybqnjq.fsf@benfinney.id.au> Terry Reedy writes: > If you have a particular persistent problem with hg.python.org, I > suggest talking to your ISP. If a reproducible issue is discovered, to whom should it be reported? The Python bug tracker doesn't seem to allow reports of fault in the python.org infrastructure. -- \ ?Politics is not the art of the possible. It consists in | `\ choosing between the disastrous and the unpalatable.? ?John | _o__) Kenneth Galbraith, 1962-03-02 | Ben Finney From tjreedy at udel.edu Wed Oct 2 21:58:50 2013 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 02 Oct 2013 21:58:50 -0400 Subject: hg.python.org: Server unresponsive and timeout In-Reply-To: <7w8uybqnjq.fsf@benfinney.id.au> References: <7w8uybqnjq.fsf@benfinney.id.au> Message-ID: On 10/2/2013 7:07 PM, Ben Finney wrote: > Terry Reedy writes: > >> If you have a particular persistent problem with hg.python.org, I >> suggest talking to your ISP. > > If a reproducible issue is discovered, to whom should it be reported? > The Python bug tracker doesn't seem to allow reports of fault in the > python.org infrastructure. Report it here to verify that the problem is not limited to one computer and it will be seen by someone who also posts to pydev. http://www.downforeveryoneorjustme.com/ will check for you. I found it by searching 'down for everyone'. -- Terry Jan Reedy From oscar.j.benjamin at gmail.com Fri Oct 4 05:46:27 2013 From: oscar.j.benjamin at gmail.com (Oscar Benjamin) Date: Fri, 4 Oct 2013 10:46:27 +0100 Subject: hg.python.org: Server unresponsive and timeout In-Reply-To: References: Message-ID: On 2 October 2013 23:25, Terry Reedy wrote: > On 10/2/2013 5:36 AM, Tae Wong wrote: >> >> This post is irrelevant from using Python; so it's an Internet server >> problem. >> >> When you try to connect to hg.python.org, the connection takes forever. > > > I believe hg.python.org is on a different machine than python.org. It has > occasionally been down, but works fine with Firefox. Sometimes connections > fail even if both endpoints are working. If you have a particular persistent > problem with hg.python.org, I suggest talking to your ISP. > > Cloning the CPython repository may take an hour or two if that is what you > are trying to do. It takes 15 minutes to clone with hg from hg.python.org on my machine. It takes 4 minutes on the same machine to clone the semi-official github mirror using git: 'git clone https://github.com/python/cpython.git' Those times are obviously affected by the fact that I have a very good internet connection at work: neither of the above operations comes close to the maximum bandwidth that I can access from some servers. Perhaps if there are problems with hg.python.org you might want to try the github mirror though. Oscar From subhabangalore at gmail.com Wed Oct 2 06:04:16 2013 From: subhabangalore at gmail.com (subhabangalore at gmail.com) Date: Wed, 2 Oct 2013 03:04:16 -0700 (PDT) Subject: Lowest Value in List Message-ID: Dear Group, I am trying to work out a solution to the following problem in Python. The Problem: Suppose I have three lists. Each list is having 10 elements in ascending order. I have to construct one list having 10 elements which are of the lowest value among these 30 elements present in the three given lists. The Solution: I tried to address the issue in the following ways: a) I took three lists, like, list1=[1,2,3,4,5,6,7,8,9,10] list2=[0,1,2,3,4,5,6,7,8,9] list3=[-5,-4,-3,-2,-1,0,1,2,3,4] I tried to make sum and convert them as set to drop the repeating elements: set_sum=set(list1+list2+list3) set([0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, -1, -5, -4, -3, -2]) In the next step I tried to convert it back to list as, list_set=list(set_sum) gave the value as, [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, -1, -5, -4, -3, -2] Now, I imported heapq as, import heapq and took the result as, result=heapq.nsmallest(10,list_set) it gave as, [-5, -4, -3, -2, -1, 0, 1, 2, 3, 4] b) I am thinking to work out another approach. I am taking the lists again as, list1=[1,2,3,4,5,6,7,8,9,10] list2=[0,1,2,3,4,5,6,7,8,9] list3=[-5,-4,-3,-2,-1,0,1,2,3,4] as they are in ascending order, I am trying to take first four/five elements of each list,like, list1_4=list1[:4] >>> list2_4=list2[:4] >>> list3_4=list3[:4] Now, I am trying to add them as, list11=list1_4+list2_4+list3_4 thus, giving us the result [1, 2, 3, 4, 0, 1, 2, 3, -5, -4, -3, -2] Now, we are trying to sort the list of the set of the sum as, sort_sum=sorted(list(set(list11))) giving us the required result as, [-5, -4, -3, -2, 0, 1, 2, 3, 4] If by taking the value of each list portion as 4 gives as less number of elements in final value, as we are making set to avoid repeating numbers, we increase element count by one or two and if final result becomes more than 10 we take first ten. Are these approaches fine. Or should we think some other way. If any learned member of the group can kindly let me know how to solve I would be helpful enough. Thanking in Advance, Subhabrata. From steve at pearwood.info Wed Oct 2 06:10:34 2013 From: steve at pearwood.info (Steven D'Aprano) Date: 02 Oct 2013 10:10:34 GMT Subject: Lowest Value in List References: Message-ID: <524bf119$0$2865$c3e8da3$76491128@news.astraweb.com> On Wed, 02 Oct 2013 03:04:16 -0700, subhabangalore wrote: > Dear Group, > > I am trying to work out a solution to the following problem in Python. > > The Problem: > Suppose I have three lists. > Each list is having 10 elements in ascending order. I have to construct > one list having 10 elements which are of the lowest value among these 30 > elements present in the three given lists. If they have to be the lowest *unique* values, the easiest way is to build a set from all three lists, then sort, and take a slice of only the first 10: sorted(set(alist + blist + clist))[:10] If you don't want unique values, but want to keep duplicates, then drop the call to set: sorted(alist + blist + clist)[:10] > The Solution: > > I tried to address the issue in the following ways: Thank you for posting your attempts to solve this problem! You had the right idea, you just did a little bit too much work. -- Steven From antoon.pardon at rece.vub.ac.be Wed Oct 2 06:42:06 2013 From: antoon.pardon at rece.vub.ac.be (Antoon Pardon) Date: Wed, 02 Oct 2013 12:42:06 +0200 Subject: Lowest Value in List In-Reply-To: References: Message-ID: <524BF87E.3050601@rece.vub.ac.be> Op 02-10-13 12:04, subhabangalore at gmail.com schreef: > Dear Group, > > I am trying to work out a solution to the following problem in Python. > > The Problem: > Suppose I have three lists. > Each list is having 10 elements in ascending order. > I have to construct one list having 10 elements which are of the lowest value among these 30 elements present in the three given lists. > > The Solution: > > I tried to address the issue in the following ways: > > a) I took three lists, like, > list1=[1,2,3,4,5,6,7,8,9,10] > list2=[0,1,2,3,4,5,6,7,8,9] > list3=[-5,-4,-3,-2,-1,0,1,2,3,4] > > I tried to make sum and convert them as set to drop the repeating elements: > set_sum=set(list1+list2+list3) > set([0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, -1, -5, -4, -3, -2]) > > In the next step I tried to convert it back to list as, > list_set=list(set_sum) > gave the value as, > [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, -1, -5, -4, -3, -2] > > Now, I imported heapq as, > import heapq > > and took the result as, > result=heapq.nsmallest(10,list_set) > it gave as, > [-5, -4, -3, -2, -1, 0, 1, 2, 3, 4] > > b) I am thinking to work out another approach. > I am taking the lists again as, > > list1=[1,2,3,4,5,6,7,8,9,10] > list2=[0,1,2,3,4,5,6,7,8,9] > list3=[-5,-4,-3,-2,-1,0,1,2,3,4] > > as they are in ascending order, I am trying to take first four/five elements of each list,like, > > list1_4=list1[:4] >>>> list2_4=list2[:4] >>>> list3_4=list3[:4] > > Now, I am trying to add them as, > > list11=list1_4+list2_4+list3_4 > > thus, giving us the result > > [1, 2, 3, 4, 0, 1, 2, 3, -5, -4, -3, -2] > > Now, we are trying to sort the list of the set of the sum as, > > sort_sum=sorted(list(set(list11))) > > giving us the required result as, > > [-5, -4, -3, -2, 0, 1, 2, 3, 4] > > If by taking the value of each list portion as 4 gives as less number of > elements in final value, as we are making set to avoid repeating numbers, > we increase element count by one or two and if final result becomes more > than 10 we take first ten. > > Are these approaches fine. Or should we think some other way. > > If any learned member of the group can kindly let me know how to solve I would be helpful enough. You may consider a merge phase from the merge sort. Something like the following: (Pseudo code; not tested) iters = [iter(list1), iter(list2), iter(list3)] heads = [(itr.next() for itr in Iters] index, value = find_smallest_from(heads) # This function finds the smalles value and returns it with its index last = value result = [value] heads[index] = iters[index].next() while len(results) < 10: index, value = find_smallest_from(heads) if value == last: continue last = value result.append(value) heads[index] = iters[index].next() From dvghana at gmail.com Wed Oct 2 08:56:12 2013 From: dvghana at gmail.com (dvghana at gmail.com) Date: Wed, 2 Oct 2013 05:56:12 -0700 (PDT) Subject: Lowest Value in List In-Reply-To: References: Message-ID: <584b31e4-646c-42eb-9b21-9ea5ad4c1b03@googlegroups.com> On Wednesday, October 2, 2013 10:04:16 AM UTC, subhaba... at gmail.com wrote: > Dear Group, > > > > I am trying to work out a solution to the following problem in Python. > > > > The Problem: > > Suppose I have three lists. > > Each list is having 10 elements in ascending order. > > I have to construct one list having 10 elements which are of the lowest value among these 30 elements present in the three given lists. > > > > The Solution: > > > > I tried to address the issue in the following ways: > > > > a) I took three lists, like, > > list1=[1,2,3,4,5,6,7,8,9,10] > > list2=[0,1,2,3,4,5,6,7,8,9] > > list3=[-5,-4,-3,-2,-1,0,1,2,3,4] > > > > I tried to make sum and convert them as set to drop the repeating elements: > > set_sum=set(list1+list2+list3) > > set([0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, -1, -5, -4, -3, -2]) > > > > In the next step I tried to convert it back to list as, > > list_set=list(set_sum) > > gave the value as, > > [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, -1, -5, -4, -3, -2] > > > > Now, I imported heapq as, > > import heapq > > > > and took the result as, > > result=heapq.nsmallest(10,list_set) > > it gave as, > > [-5, -4, -3, -2, -1, 0, 1, 2, 3, 4] > > > > b) I am thinking to work out another approach. > > I am taking the lists again as, > > > > list1=[1,2,3,4,5,6,7,8,9,10] > > list2=[0,1,2,3,4,5,6,7,8,9] > > list3=[-5,-4,-3,-2,-1,0,1,2,3,4] > > > > as they are in ascending order, I am trying to take first four/five elements of each list,like, > > > > list1_4=list1[:4] > > >>> list2_4=list2[:4] > > >>> list3_4=list3[:4] > > > > Now, I am trying to add them as, > > > > list11=list1_4+list2_4+list3_4 > > > > thus, giving us the result > > > > [1, 2, 3, 4, 0, 1, 2, 3, -5, -4, -3, -2] > > > > Now, we are trying to sort the list of the set of the sum as, > > > > sort_sum=sorted(list(set(list11))) > > > > giving us the required result as, > > > > [-5, -4, -3, -2, 0, 1, 2, 3, 4] > > > > If by taking the value of each list portion as 4 gives as less number of elements in final value, as we are making set to avoid repeating numbers, we increase element count by one or two and if final result becomes more than 10 we take first ten. > > > > Are these approaches fine. Or should we think some other way. > > > > If any learned member of the group can kindly let me know how to solve I would be helpful enough. > > > > Thanking in Advance, > > Subhabrata. PS: I'm learning python (or any programming language) for the first time so I'm pretty sure you don't have to take my word for it but this is what I've got: list1 = [1,2,3,4,5,6,7,8,9,10] list2 = [1,2,5,8,9,10,12,15,16,17] list3 = [-1,-2,-3,8,20,30,40,50,60,17] def smallestTen(a,b,c): ultimatelist = a + b + c for i in ultimatelist: return sorted(set(ultimatelist))[:10] print (smallestTen(list1, list2, list3)) From ganeshsahni07 at gmail.com Wed Oct 2 10:05:04 2013 From: ganeshsahni07 at gmail.com (Ravi Sahni) Date: Wed, 2 Oct 2013 19:35:04 +0530 Subject: Lowest Value in List In-Reply-To: References: Message-ID: On Wed, Oct 2, 2013 at 3:34 PM, wrote: > Dear Group, > > I am trying to work out a solution to the following problem in Python. > > The Problem: > Suppose I have three lists. > Each list is having 10 elements in ascending order. > I have to construct one list having 10 elements which are of the lowest value among these 30 elements present in the three given lists. > > The Solution: > > I tried to address the issue in the following ways: > > a) I took three lists, like, > list1=[1,2,3,4,5,6,7,8,9,10] > list2=[0,1,2,3,4,5,6,7,8,9] > list3=[-5,-4,-3,-2,-1,0,1,2,3,4] > > I tried to make sum and convert them as set to drop the repeating elements: > set_sum=set(list1+list2+list3) > set([0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, -1, -5, -4, -3, -2]) > > In the next step I tried to convert it back to list as, > list_set=list(set_sum) > gave the value as, > [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, -1, -5, -4, -3, -2] > > Now, I imported heapq as, > import heapq > > and took the result as, > result=heapq.nsmallest(10,list_set) > it gave as, > [-5, -4, -3, -2, -1, 0, 1, 2, 3, 4] > > b) I am thinking to work out another approach. > I am taking the lists again as, > > list1=[1,2,3,4,5,6,7,8,9,10] > list2=[0,1,2,3,4,5,6,7,8,9] > list3=[-5,-4,-3,-2,-1,0,1,2,3,4] > > as they are in ascending order, I am trying to take first four/five elements of each list,like, > > list1_4=list1[:4] >>>> list2_4=list2[:4] >>>> list3_4=list3[:4] > > Now, I am trying to add them as, > > list11=list1_4+list2_4+list3_4 > > thus, giving us the result > > [1, 2, 3, 4, 0, 1, 2, 3, -5, -4, -3, -2] > > Now, we are trying to sort the list of the set of the sum as, > > sort_sum=sorted(list(set(list11))) > > giving us the required result as, > > [-5, -4, -3, -2, 0, 1, 2, 3, 4] > > If by taking the value of each list portion as 4 gives as less number of elements in final value, as we are making set to avoid repeating numbers, we increase element count by one or two and if final result becomes more than 10 we take first ten. > > Are these approaches fine. Or should we think some other way. > > If any learned member of the group can kindly let me know how to solve I would be helpful enough. > > Thanking in Advance, > Subhabrata. > > > -- > https://mail.python.org/mailman/listinfo/python-list [Disclaimer: Beginner myself] The heapq module has merge Since the lists are already sorted what's wrong with just this? list(merge(list1, list2, list3))[:10] -- - Ravi From __peter__ at web.de Thu Oct 3 07:12:26 2013 From: __peter__ at web.de (Peter Otten) Date: Thu, 03 Oct 2013 13:12:26 +0200 Subject: Lowest Value in List References: Message-ID: subhabangalore at gmail.com wrote: > Dear Group, > > I am trying to work out a solution to the following problem in Python. > > The Problem: > Suppose I have three lists. > Each list is having 10 elements in ascending order. > I have to construct one list having 10 elements which are of the lowest > value among these 30 elements present in the three given lists. > > The Solution: > > I tried to address the issue in the following ways: > > a) I took three lists, like, > list1=[1,2,3,4,5,6,7,8,9,10] > list2=[0,1,2,3,4,5,6,7,8,9] > list3=[-5,-4,-3,-2,-1,0,1,2,3,4] > > I tried to make sum and convert them as set to drop the repeating > elements: set_sum=set(list1+list2+list3) > set([0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, -1, -5, -4, -3, -2]) > > In the next step I tried to convert it back to list as, > list_set=list(set_sum) > gave the value as, > [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, -1, -5, -4, -3, -2] > > Now, I imported heapq as, > import heapq > > and took the result as, > result=heapq.nsmallest(10,list_set) > it gave as, > [-5, -4, -3, -2, -1, 0, 1, 2, 3, 4] > > b) I am thinking to work out another approach. > I am taking the lists again as, > > list1=[1,2,3,4,5,6,7,8,9,10] > list2=[0,1,2,3,4,5,6,7,8,9] > list3=[-5,-4,-3,-2,-1,0,1,2,3,4] > > as they are in ascending order, I am trying to take first four/five > elements of each list,like, > > list1_4=list1[:4] >>>> list2_4=list2[:4] >>>> list3_4=list3[:4] > > Now, I am trying to add them as, > > list11=list1_4+list2_4+list3_4 > > thus, giving us the result > > [1, 2, 3, 4, 0, 1, 2, 3, -5, -4, -3, -2] > > Now, we are trying to sort the list of the set of the sum as, > > sort_sum=sorted(list(set(list11))) > > giving us the required result as, > > [-5, -4, -3, -2, 0, 1, 2, 3, 4] > > If by taking the value of each list portion as 4 gives as less number of > elements in final value, as we are making set to avoid repeating numbers, > we increase element count by one or two and if final result becomes more > than 10 we take first ten. > > Are these approaches fine. Or should we think some other way. > > If any learned member of the group can kindly let me know how to solve I > would be helpful enough. A bit late to the show here's my take. You could separate your problem into three simpler ones: (1) combine multiple sequences into one big sequence (2) filter out duplicate items (3) find the largest items (1) is covered by the stdlib: items = itertools.chain.from_iterable([list1, list2, list3]) (2) is easy assuming the items are hashable: def unique(items): seen = set() for item in items: if item not in seen: seen.add(item) yield item items = unique(items) (3) is also covered by the stdlib: largest = heapq.nlargest(3, items) This approach has one disadvantage: the `seen` set in unique() may grow indefinitely if the sequence passed to it is "long" and has an unlimited number of distinct duplicates. So here's an alternative using a heap and a set both limited by the length of the result: import heapq def unique_nlargest(n, items): items = iter(items) heap = [] seen = set() for item in items: if item not in seen: seen.add(item) heapq.heappush(heap, item) if len(heap) > n: max_discard = heapq.heappop(heap) seen.remove(max_discard) break for item in items: if item > max_discard and item not in seen: max_discard = heapq.heappushpop(heap, item) seen.remove(max_discard) return heap if __name__ == "__main__": print(unique_nlargest(3, [1,2,3,4,5,4,3,2,1,6,2,7])) I did not test it, so there may be bugs, but the idea behind the code is simple: you can remove from the set all items that are below the minimum item in the heap. Thus both lengths can never grow beyond n (or n+1 in my actual implementation). From vignesh.harikrishna at gmail.com Wed Oct 2 06:44:24 2013 From: vignesh.harikrishna at gmail.com (JonDoe297) Date: Wed, 2 Oct 2013 03:44:24 -0700 (PDT) Subject: Efficency help for a Calculator Program Message-ID: <77bde84a-b0ce-4061-bb2e-e6cd23f4282c@googlegroups.com> You may remember me from this : https://groups.google.com/forum/#!topic/comp.lang.python/PIkUno3avkw I need help to increase the efficiency of this code : global repeat repeat=1 def main(): c=int(raw_input("How many numbers do you want to work? (Min. 2 Max. 3) ")) if c==2: x=int(raw_input("Enter the first number to be worked ")) y=int(raw_input("Enter the second number to be worked ")) elif c==3: x=int(raw_input("Enter the first number to be worked ")) y=int(raw_input("Enter the second number to be worked ")) z=int(raw_input("Enter the third number to be worked ")) else: print "Invalid input.";raw_input("Press to close this window");exit() p=int(raw_input("Do you want to divide, subtract, add or multiply these numbers? (1=divide, 2=subtract, 3=add, 4=multiply) ")) if p==1 and c==2: print "The result is : ",x/y repeat=int(raw_input("Do you want to calculate for more numbers? Yes=1 No=2 ")) if repeat==1: main() elif p==1 and c==3: print "The result is : ",x/y/z repeat=int(raw_input("Do you want to calculate for more numbers? Yes=1 No=2 ")) if repeat==1: main() elif p==2 and c==2: print "The result is : ",x-y repeat=int(raw_input("Do you want to calculate for more numbers? Yes=1 No=2 ")) if repeat==1: main() elif p==2 and c==3: print "The result is : ",x-y-z repeat=int(raw_input("Do you want to calculate for more numbers? Yes=1 No=2 ")) if repeat==1: main() elif p==3 and c==2: print "The result is : ",x+y repeat=int(raw_input("Do you want to calculate for more numbers? Yes=1 No=2 ")) if repeat==1: main() elif p==3 and c==3: print "The result is : ",x+y+z repeat=int(raw_input("Do you want to calculate for more numbers? Yes=1 No=2 ")) if repeat==1: main() elif p==4 and c==2: print "The result is : ",x*y repeat=int(raw_input("Do you want to calculate for more numbers? Yes=1 No=2 ")) if repeat==1: main() elif p==4 and c==3: print "The result is : "+str(x*y*z) repeat=int(raw_input("Do you want to calculate for more numbers? Yes=1 No=2 ")) if repeat==1: main() else: repeat=int(raw_input("Invalid Input. Please read instructions properly. Would you like to try again? Yes=1 No=2 ")) if repeat==1: main() else: exit() main() Is there any way to make it smaller? It does it's job, but I want it to look smaller, more efficient. From rosuav at gmail.com Wed Oct 2 07:01:03 2013 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 2 Oct 2013 21:01:03 +1000 Subject: Efficency help for a Calculator Program In-Reply-To: <77bde84a-b0ce-4061-bb2e-e6cd23f4282c@googlegroups.com> References: <77bde84a-b0ce-4061-bb2e-e6cd23f4282c@googlegroups.com> Message-ID: On Wed, Oct 2, 2013 at 8:44 PM, JonDoe297 wrote: > Is there any way to make it smaller? It does it's job, but I want it to look smaller, more efficient. Yes, it is, but let me first clarify something: "Smaller" and "more efficient" are two quite different concepts. Efficiency doesn't matter to your code here, so what you're looking for is smaller, clearer code. Which is a good thing to be doing :) At top level, the 'global' declaration doesn't do anything. You may as well not bother with it. If you change your recursive main() function into a while loop, you'll be able to combine all your common code very easily. I won't do the whole job for you, but consider this structure: repeat=1 while repeat==1: # get inputs # calculate and produce output repeat=int(raw_input("Do you want to do more? ")) And if you need your error state to have a different prompt, you can use 'continue' to skip the bottom of the loop. Hope that helps! ChrisA From vignesh.harikrishna at gmail.com Wed Oct 2 09:43:51 2013 From: vignesh.harikrishna at gmail.com (JonDoe297) Date: Wed, 2 Oct 2013 06:43:51 -0700 (PDT) Subject: Efficency help for a Calculator Program In-Reply-To: References: <77bde84a-b0ce-4061-bb2e-e6cd23f4282c@googlegroups.com> Message-ID: <53e42327-366c-42ea-b111-0e98aefc8cef@googlegroups.com> On Wednesday, October 2, 2013 4:31:03 PM UTC+5:30, Chris Angelico wrote: > On Wed, Oct 2, 2013 at 8:44 PM, JonDoe297 wrote: > > > Is there any way to make it smaller? It does it's job, but I want it to look smaller, more efficient. > > > > Yes, it is, but let me first clarify something: "Smaller" and "more > > efficient" are two quite different concepts. Efficiency doesn't matter > > to your code here, so what you're looking for is smaller, clearer > > code. Which is a good thing to be doing :) > > > > At top level, the 'global' declaration doesn't do anything. You may as > > well not bother with it. > > > > If you change your recursive main() function into a while loop, you'll > > be able to combine all your common code very easily. I won't do the > > whole job for you, but consider this structure: > > > > repeat=1 > > while repeat==1: > > # get inputs > > # calculate and produce output > > repeat=int(raw_input("Do you want to do more? ")) > > > > And if you need your error state to have a different prompt, you can > > use 'continue' to skip the bottom of the loop. > > > > Hope that helps! > > > > ChrisA Thanks a lot again Chris! You understood what I couldn't convey, perfectly! I'll use your suggestions ;) From rosuav at gmail.com Wed Oct 2 20:25:47 2013 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 3 Oct 2013 10:25:47 +1000 Subject: Efficency help for a Calculator Program In-Reply-To: References: <77bde84a-b0ce-4061-bb2e-e6cd23f4282c@googlegroups.com> Message-ID: On Thu, Oct 3, 2013 at 9:47 AM, Dennis Lee Bieber wrote: > try: > numItems = int(raw_input("\n\nHow many values? ")) > except: #naked exception is not really good programming > print "Invalid input, exiting..." > sys.exit(1) Please don't _ever_ advocate this programming style! Wrapping something in a try/except that emits a generic message and terminates is a bad idea - the default behaviour, if you simply let the exception happen, is to emit a very useful message and terminate. Never test for any error condition you're not prepared to handle, as the BOFH advised his boss. ChrisA From rosuav at gmail.com Fri Oct 4 02:45:51 2013 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 4 Oct 2013 16:45:51 +1000 Subject: Efficency help for a Calculator Program In-Reply-To: References: <77bde84a-b0ce-4061-bb2e-e6cd23f4282c@googlegroups.com> Message-ID: On Fri, Oct 4, 2013 at 9:15 AM, Dennis Lee Bieber wrote: > On Thu, 3 Oct 2013 10:25:47 +1000, Chris Angelico > declaimed the following: > >>On Thu, Oct 3, 2013 at 9:47 AM, Dennis Lee Bieber wrote: >>> try: >>> numItems = int(raw_input("\n\nHow many values? ")) >>> except: #naked exception is not really good programming >>> print "Invalid input, exiting..." >>> sys.exit(1) >> >>Please don't _ever_ advocate this programming style! Wrapping >>something in a try/except that emits a generic message and terminates >>is a bad idea - the default behaviour, if you simply let the exception >>happen, is to emit a very useful message and terminate. Never test for >>any error condition you're not prepared to handle, as the BOFH advised >>his boss. >> > Note: I DID include a comment that this was NOT good style. You mentioned that bare except is a problem; I'm more looking at the fact that the except clause simply writes a message and terminates. They're two separate issues, both bad style. I know _you_ know it's bad style; but someone reading over this needs to be aware that this shouldn't normally be done. ChrisA From nikos.gr33k at gmail.com Wed Oct 2 08:20:00 2013 From: nikos.gr33k at gmail.com (=?UTF-8?B?zp3Or866zr/Pgg==?=) Date: Wed, 02 Oct 2013 15:20:00 +0300 Subject: Can arbitrary code run in a server if someone's know just the MySQL password? Message-ID: Tim delaney said: "Because there's no chance with the brilliance you display that there could be any possibility of login details being kept in plaintext in your database. And of course your database is so well locked down that no attacker with a login to it could then execute arbitrary code on your system. And there's also zero chance that your personal account login details are also available in plaintext somewhere that you're unaware of." ========== Is it possible for someone that knows the MYSQL password of a server to run arbitrary code on a linux server? Okey he uses the password and he gain access to the databases, then what? MySQL is a database server how can he run run arbitrary shell commands by using MySQL? If yes, can you give an example please? Also, is there a chance for my account's password to be retrieved on some why due to MySQL access or perhaps by utilizing my own python code? I'm just trying to figure out how the upload of that .html file happened to '/home/nikos/public_html'. I need a theory and Zero Piraeus to answer too. Please, serious replies only, i won't answer to ironic comments or jokes. From antoon.pardon at rece.vub.ac.be Wed Oct 2 08:37:58 2013 From: antoon.pardon at rece.vub.ac.be (Antoon Pardon) Date: Wed, 02 Oct 2013 14:37:58 +0200 Subject: Can arbitrary code run in a server if someone's know just the MySQL password? In-Reply-To: References: Message-ID: <524C13A6.9060404@rece.vub.ac.be> Op 02-10-13 14:20, ????? schreef: > Tim delaney said: > > "Because there's no chance with the brilliance you display that there > could be any possibility of login details being kept in plaintext in > your database. > > And of course your database is so well locked down that no attacker with > a login to it could then execute arbitrary code on your system. > > And there's also zero chance that your personal account login details > are also available in plaintext somewhere that you're unaware of." > ========== > > Is it possible for someone that knows the MYSQL password of a server to > run arbitrary code on a linux server? > > Okey he uses the password and he gain access to the databases, then > what? MySQL is a database server how can he run run arbitrary shell > commands by using MySQL? > > If yes, can you give an example please? > > Also, is there a chance for my account's password to be retrieved on > some why due to MySQL access or perhaps by utilizing my own python code? > > I'm just trying to figure out how the upload of that .html file happened > to '/home/nikos/public_html'. I need a theory and Zero Piraeus to answer > too. > > Please, serious replies only, i won't answer to ironic comments or jokes. You are not asking a python question. This is a python list. Not a Nikos advise board. Find a list where your question is more appropiate. -- Antoon Pardon From feedthetroll at gmx.de Wed Oct 2 08:38:40 2013 From: feedthetroll at gmx.de (feedthetroll at gmx.de) Date: Wed, 2 Oct 2013 05:38:40 -0700 (PDT) Subject: Can arbitrary code run in a server if someone's know just the MySQL password? In-Reply-To: References: Message-ID: <82d41351-a5b7-46ff-98af-2576127ec02c@googlegroups.com> Am Mittwoch, 2. Oktober 2013 14:20:00 UTC+2 schrieb Ferrous Cranus: > ... > Is it possible for someone that knows the MYSQL password of a server to > run arbitrary code on a linux server? > ... > If yes, can you give an example please? http://lmgtfy.com/?q=mysql+shell+escape > Please, serious replies only, i won't answer to ironic comments or jokes. Please only questions about python. This not a mysql or security list. PLONK! (Hey Thunderbird has a very useful new feature. Ignore thread.) From python.list at tim.thechases.com Wed Oct 2 09:21:43 2013 From: python.list at tim.thechases.com (Tim Chase) Date: Wed, 2 Oct 2013 08:21:43 -0500 Subject: Killing threads with TB (was: Can arbitrary code run in a server if someone's know just the MySQL password?) In-Reply-To: <82d41351-a5b7-46ff-98af-2576127ec02c@googlegroups.com> References: <82d41351-a5b7-46ff-98af-2576127ec02c@googlegroups.com> Message-ID: <20131002082143.6707f019@bigbox.christie.dr> On 2013-10-02 05:38, feedthetroll at gmx.de wrote: > (Hey Thunderbird has a very useful new feature. Ignore thread.) Unfortunately, as of when I last tested it, it only works in the newsgroup part of TB, not the mail portion of TB. Sadly, Claws-Mail (my current mailer) doesn't have a native kill-thread functionality, but it does support external message filters, so I threw together a kill-thread filter in Python (bringing this back on-topic) which duplicates the TB functionality that I missed. -tkc From tjreedy at udel.edu Wed Oct 2 18:34:04 2013 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 02 Oct 2013 18:34:04 -0400 Subject: Killing threads with TB In-Reply-To: <20131002082143.6707f019@bigbox.christie.dr> References: <82d41351-a5b7-46ff-98af-2576127ec02c@googlegroups.com> <20131002082143.6707f019@bigbox.christie.dr> Message-ID: On 10/2/2013 9:21 AM, Tim Chase wrote: > On 2013-10-02 05:38, feedthetroll at gmx.de wrote: >> (Hey Thunderbird has a very useful new feature. Ignore thread.) > > Unfortunately, as of when I last tested it, it only works in the > newsgroup part of TB, not the mail portion of TB. One can read python-list as news.gmane.org newsgroup gmane.comp.python.general. -- Terry Jan Reedy From breamoreboy at yahoo.co.uk Wed Oct 2 18:48:31 2013 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Wed, 02 Oct 2013 23:48:31 +0100 Subject: Killing threads with TB In-Reply-To: References: <82d41351-a5b7-46ff-98af-2576127ec02c@googlegroups.com> <20131002082143.6707f019@bigbox.christie.dr> Message-ID: On 02/10/2013 23:34, Terry Reedy wrote: > On 10/2/2013 9:21 AM, Tim Chase wrote: >> On 2013-10-02 05:38, feedthetroll at gmx.de wrote: >>> (Hey Thunderbird has a very useful new feature. Ignore thread.) >> >> Unfortunately, as of when I last tested it, it only works in the >> newsgroup part of TB, not the mail portion of TB. > > One can read python-list as news.gmane.org newsgroup > gmane.comp.python.general. > You can also read hundreds of other Python lists at gmane.comp.python. -- Roses are red, Violets are blue, Most poems rhyme, But this one doesn't. Mark Lawrence From steve+comp.lang.python at pearwood.info Wed Oct 2 09:25:58 2013 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: 02 Oct 2013 13:25:58 GMT Subject: Can arbitrary code run in a server if someone's know just the MySQL password? References: Message-ID: <524c1ee6$0$29984$c3e8da3$5496439d@news.astraweb.com> On Wed, 02 Oct 2013 15:20:00 +0300, ????? wrote: > Is it possible for someone that knows the MYSQL password of a server to > run arbitrary code on a linux server? Yes, it is possible. > Okey he uses the password and he gain access to the databases, then > what? MySQL is a database server how can he run run arbitrary shell > commands by using MySQL? > > If yes, can you give an example please? Google for "run arbitrary shell commands MySQL". If you don't understand them, go find a beginner's forum where you can learn about MySQL, this is not it. https://duckduckgo.com/html/?q=run+arbitrary+shell+commands+MySQL https://www.google.com.au/search?q=run+arbitrary+shell+commands -- Steven From nikos.gr33k at gmail.com Wed Oct 2 09:41:40 2013 From: nikos.gr33k at gmail.com (=?UTF-8?B?zp3Or866zr/Pgg==?=) Date: Wed, 02 Oct 2013 16:41:40 +0300 Subject: Can arbitrary code run in a server if someone's know just the MySQL password? In-Reply-To: <524c1ee6$0$29984$c3e8da3$5496439d@news.astraweb.com> References: <524c1ee6$0$29984$c3e8da3$5496439d@news.astraweb.com> Message-ID: ???? 2/10/2013 4:25 ??, ?/? Steven D'Aprano ??????: > On Wed, 02 Oct 2013 15:20:00 +0300, ????? wrote: > >> Is it possible for someone that knows the MYSQL password of a server to >> run arbitrary code on a linux server? > > Yes, it is possible. Is that what might have happened and someone managed to upload the .html file in '~/home/nikos/www/' ? Can you think of any other way? From ned at nedbatchelder.com Wed Oct 2 09:58:17 2013 From: ned at nedbatchelder.com (Ned Batchelder) Date: Wed, 02 Oct 2013 09:58:17 -0400 Subject: Can arbitrary code run in a server if someone's know just the MySQL password? In-Reply-To: References: <524c1ee6$0$29984$c3e8da3$5496439d@news.astraweb.com> Message-ID: <524C2679.2010304@nedbatchelder.com> On 10/2/13 9:41 AM, ????? wrote: > ???? 2/10/2013 4:25 ??, ?/? Steven D'Aprano ??????: >> On Wed, 02 Oct 2013 15:20:00 +0300, ????? wrote: >> >>> Is it possible for someone that knows the MYSQL password of a server to >>> run arbitrary code on a linux server? >> >> Yes, it is possible. > > Is that what might have happened and someone managed to upload the > .html file in '~/home/nikos/www/' ? > > Can you think of any other way? > As others have said in this thread, this is not a Python topic. Find another forum for this question. Do not ask it here again. You've said that you can improve. Show us by not asking non-Python questions here. --Ned. From alister.ware at ntlworld.com Wed Oct 2 10:34:21 2013 From: alister.ware at ntlworld.com (Alister) Date: Wed, 02 Oct 2013 14:34:21 GMT Subject: Can arbitrary code run in a server if someone's know just the MySQL password? References: <524c1ee6$0$29984$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Wed, 02 Oct 2013 16:41:40 +0300, ????? wrote: > ???? 2/10/2013 4:25 ??, ?/? Steven D'Aprano ??????: >> On Wed, 02 Oct 2013 15:20:00 +0300, ????? wrote: >> >>> Is it possible for someone that knows the MYSQL password of a server >>> to run arbitrary code on a linux server? >> >> Yes, it is possible. > > Is that what might have happened and someone managed to upload the .html > file in '~/home/nikos/www/' ? > > Can you think of any other way? There are many other ways (i am not a hacker so i would not know whre to start) Against my better judgement I am going to give some advise (more to protect your customers than you) 1) tie down access to your server, nothing should be accessable from the internet unless absolutly necessary. certainly your database should not be accessible and this should be blocked in multiple ways (protection in depth) you should close down any un-necessary services. shut your firewall to all trafffix except http & https (ports 80 ,443) unless absolutely necessary. set your database accounts to only allow log in from localhost & and any explicit IP addresses that must have access & please google for further advise on server security & post questions in a suitable forum (not here) as many have said, security is not our area of expertise & this is the wrong place to ask. when correctly secured knowing your username & password should not be enough to allow access to your server. -- I'm not under the alkafluence of inkahol that some thinkle peep I am. It's just the drunker I sit here the longer I get. From ganeshsahni07 at gmail.com Wed Oct 2 11:13:19 2013 From: ganeshsahni07 at gmail.com (Ravi Sahni) Date: Wed, 2 Oct 2013 20:43:19 +0530 Subject: Can arbitrary code run in a server if someone's know just the MySQL password? In-Reply-To: References: <524c1ee6$0$29984$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Wed, Oct 2, 2013 at 8:04 PM, Alister wrote: > On Wed, 02 Oct 2013 16:41:40 +0300, ????? wrote: > >> ???? 2/10/2013 4:25 ??, ?/? Steven D'Aprano ??????: >>> On Wed, 02 Oct 2013 15:20:00 +0300, ????? wrote: >>> >>>> Is it possible for someone that knows the MYSQL password of a server >>>> to run arbitrary code on a linux server? >>> >>> Yes, it is possible. >> >> Is that what might have happened and someone managed to upload the .html >> file in '~/home/nikos/www/' ? >> >> Can you think of any other way? > > > There are many other ways (i am not a hacker so i would not know whre to > start) > Against my better judgement I am going to give some advise (more to > protect your customers than you) > > 1) tie down access to your server, nothing should be accessable from the > internet unless absolutly necessary. > certainly your database should not be accessible and this should be > blocked in multiple ways (protection in depth) > > you should close down any un-necessary services. > shut your firewall to all trafffix except http & https (ports 80 ,443) > unless absolutely necessary. > set your database accounts to only allow log in from localhost & and any > explicit IP addresses that must have access > > & please google for further advise on server security & post questions in > a suitable forum (not here) > > as many have said, security is not our area of expertise & this is the > wrong place to ask. > > when correctly secured knowing your username & password should not be > enough to allow access to your server. Thank you Alister for ansering the needs of needy persons. I am also needy. Please be kind to me as well: There is poverty and injustice in the world. Why?? I NEED to know People suffer and die. How come? I MUST know And there are morons... Why?? PLEASE TELL -- Ravi From nikos.gr33k at gmail.com Wed Oct 2 13:06:24 2013 From: nikos.gr33k at gmail.com (=?UTF-8?B?zp3Or866zr/PgiDOkc66zrXOvs+Mz4DOv8+FzrvOv8+C?=) Date: Wed, 02 Oct 2013 20:06:24 +0300 Subject: Can arbitrary code run in a server if someone's know just the MySQL password? In-Reply-To: References: <524c1ee6$0$29984$c3e8da3$5496439d@news.astraweb.com> Message-ID: ???? 2/10/2013 6:13 ??, ?/? Ravi Sahni ??????: > On Wed, Oct 2, 2013 at 8:04 PM, Alister wrote: >> On Wed, 02 Oct 2013 16:41:40 +0300, ????? wrote: >> >>> ???? 2/10/2013 4:25 ??, ?/? Steven D'Aprano ??????: >>>> On Wed, 02 Oct 2013 15:20:00 +0300, ????? wrote: >>>> >>>>> Is it possible for someone that knows the MYSQL password of a server >>>>> to run arbitrary code on a linux server? >>>> >>>> Yes, it is possible. >>> >>> Is that what might have happened and someone managed to upload the .html >>> file in '~/home/nikos/www/' ? >>> >>> Can you think of any other way? >> >> >> There are many other ways (i am not a hacker so i would not know whre to >> start) >> Against my better judgement I am going to give some advise (more to >> protect your customers than you) >> >> 1) tie down access to your server, nothing should be accessable from the >> internet unless absolutly necessary. >> certainly your database should not be accessible and this should be >> blocked in multiple ways (protection in depth) >> >> you should close down any un-necessary services. >> shut your firewall to all trafffix except http & https (ports 80 ,443) >> unless absolutely necessary. >> set your database accounts to only allow log in from localhost & and any >> explicit IP addresses that must have access >> >> & please google for further advise on server security & post questions in >> a suitable forum (not here) >> >> as many have said, security is not our area of expertise & this is the >> wrong place to ask. >> >> when correctly secured knowing your username & password should not be >> enough to allow access to your server. > > > Thank you Alister for ansering the needs of needy persons. > I am also needy. Please be kind to me as well: > > There is poverty and injustice in the world. Why?? I NEED to know > People suffer and die. How come? I MUST know > And there are morons... Why?? PLEASE TELL You are failing trying to mimic me. I have a reason when i ask because i did explanation for some matter. As for morons, yes they are lots of them in this world, including you trying to make fun out of this by impersonating me. You fail also as acting as a newbie, while you are a regular here. -- What is now proved was at first only imagined! & WebHost From nikos.gr33k at gmail.com Wed Oct 2 10:46:08 2013 From: nikos.gr33k at gmail.com (=?UTF-8?B?zp3Or866zr/Pgg==?=) Date: Wed, 02 Oct 2013 17:46:08 +0300 Subject: Can arbitrary code run in a server if someone's know just the MySQL password? In-Reply-To: References: <524c1ee6$0$29984$c3e8da3$5496439d@news.astraweb.com> Message-ID: ???? 2/10/2013 4:58 ??, ?/? Ned Batchelder ??????: > On 10/2/13 9:41 AM, ????? wrote: >> ???? 2/10/2013 4:25 ??, ?/? Steven D'Aprano ??????: >>> On Wed, 02 Oct 2013 15:20:00 +0300, ????? wrote: >>> >>>> Is it possible for someone that knows the MYSQL password of a server to >>>> run arbitrary code on a linux server? >>> >>> Yes, it is possible. >> >> Is that what might have happened and someone managed to upload the >> .html file in '~/home/nikos/www/' ? >> >> Can you think of any other way? >> > > As others have said in this thread, this is not a Python topic. Find > another forum for this question. Do not ask it here again. > > You've said that you can improve. Show us by not asking non-Python > questions here. > > --Ned. But i need to know what happened and how this .html file got uploaded. This is not a python question, but this happened from this pythons NG. And perhaps my python code was being utilized fo this upload to happen. I must know. -- *What is now proved was once only imagined!* From ishish at domhain.de Wed Oct 2 10:55:16 2013 From: ishish at domhain.de (ishish) Date: Wed, 02 Oct 2013 15:55:16 +0100 Subject: Can arbitrary code run in a server if someone's know just the MySQL =?UTF-8?Q?password=3F?= In-Reply-To: References: <524c1ee6$0$29984$c3e8da3$5496439d@news.astraweb.com> Message-ID: <6f05bfcd19123c760a6170e33964897e@home.minuskel.de> Am 02.10.2013 15:46, schrieb ?????: > But i need to know what happened and how this .html file got > uploaded. > This is not a python question, but this happened from this pythons > NG. ... ... Who says that?? From ned at nedbatchelder.com Wed Oct 2 11:15:48 2013 From: ned at nedbatchelder.com (Ned Batchelder) Date: Wed, 02 Oct 2013 11:15:48 -0400 Subject: Can arbitrary code run in a server if someone's know just the MySQL password? In-Reply-To: References: <524c1ee6$0$29984$c3e8da3$5496439d@news.astraweb.com> Message-ID: <524C38A4.5020808@nedbatchelder.com> On 10/2/13 10:46 AM, ????? wrote: > ???? 2/10/2013 4:58 ??, ?/? Ned Batchelder ??????: >> On 10/2/13 9:41 AM, ????? wrote: >>> ???? 2/10/2013 4:25 ??, ?/? Steven D'Aprano ??????: >>>> On Wed, 02 Oct 2013 15:20:00 +0300, ????? wrote: >>>> >>>>> Is it possible for someone that knows the MYSQL password of a >>>>> server to >>>>> run arbitrary code on a linux server? >>>> >>>> Yes, it is possible. >>> >>> Is that what might have happened and someone managed to upload the >>> .html file in '~/home/nikos/www/' ? >>> >>> Can you think of any other way? >>> >> >> As others have said in this thread, this is not a Python topic. Find >> another forum for this question. Do not ask it here again. >> >> You've said that you can improve. Show us by not asking non-Python >> questions here. >> >> --Ned. > But i need to know what happened and how this .html file got uploaded. > This is not a python question, but this happened from this pythons NG. > And perhaps my python code was being utilized fo this upload to happen. > > I must know. > This is not a topic for Python-List. We don't have answers for you, and you won't get answers to this question here. If you persist in asking about it here, don't be surprised when people get angry with you. This is anti-social behavior. I know you are upset about your server being compromised. I'm sorry about that, but it isn't on-topic here. There are other places you can get help with your question. --Ned. From denismfmcmahon at gmail.com Wed Oct 2 12:02:28 2013 From: denismfmcmahon at gmail.com (Denis McMahon) Date: Wed, 2 Oct 2013 16:02:28 +0000 (UTC) Subject: Can arbitrary code run in a server if someone's know just the MySQL password? References: <524c1ee6$0$29984$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Wed, 02 Oct 2013 17:46:08 +0300, ????? wrote: > But i need to know what happened and how this .html file got uploaded. The html file started out in an editor on on another machine, and was created by someone typing at the keyboard. It was then saved to hard disk as a file. The other machine then read the file into memory, and then sent it as a byte stream to the tcp/ip stack, where it was broken down down into packets which travelled across the tcp/ip network onto your server. Your server then re-assembled the packets into a byte stream which filled a block of memory, and then wrote the contents of that block of memory to disc as a file. (This explanation may contain some assumptions.) -- Denis McMahon, denismfmcmahon at gmail.com From ethan at stoneleaf.us Wed Oct 2 12:59:11 2013 From: ethan at stoneleaf.us (Ethan Furman) Date: Wed, 02 Oct 2013 09:59:11 -0700 Subject: Can arbitrary code run in a server if someone's know just the MySQL password? In-Reply-To: References: <524c1ee6$0$29984$c3e8da3$5496439d@news.astraweb.com> Message-ID: <524C50DF.1050406@stoneleaf.us> On 10/02/2013 07:46 AM, ????? wrote: > ???? 2/10/2013 4:58 ??, ?/? Ned Batchelder ??????: >> >> As others have said in this thread, this is not a Python topic. Find >> another forum for this question. Do not ask it here again. >> >> You've said that you can improve. Show us by not asking non-Python >> questions here. > > I must know. *plonk* From steve+comp.lang.python at pearwood.info Wed Oct 2 13:39:00 2013 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: 02 Oct 2013 17:39:00 GMT Subject: Can arbitrary code run in a server if someone's know just the MySQL password? References: <524c1ee6$0$29984$c3e8da3$5496439d@news.astraweb.com> Message-ID: <524c5a34$0$29984$c3e8da3$5496439d@news.astraweb.com> On Wed, 02 Oct 2013 16:41:40 +0300, ????? wrote: > ???? 2/10/2013 4:25 ??, ?/? Steven D'Aprano ??????: >> On Wed, 02 Oct 2013 15:20:00 +0300, ????? wrote: >> >>> Is it possible for someone that knows the MYSQL password of a server >>> to run arbitrary code on a linux server? >> >> Yes, it is possible. > > Is that what might have happened and someone managed to upload the .html > file in '~/home/nikos/www/' ? How the hell should I know? I am not a MySQL expert, and this is not a MySQL forum. Nikos, you embarrass me. I have gone out on a limb for you, and this is how you thank me? You said you were improving, and yet here you go completely ignoring the links I sent you, and continuing to ask off-topic questions here. Thanks for kicking me in the guts. I will remember this next time you ask a question. -- Steven From nikos.gr33k at gmail.com Wed Oct 2 14:02:40 2013 From: nikos.gr33k at gmail.com (=?UTF-8?B?zp3Or866zr/PgiDOkc67zrXOvs+Mz4DOv8+FzrvOv8+C?=) Date: Wed, 02 Oct 2013 21:02:40 +0300 Subject: Can arbitrary code run in a server if someone's know just the MySQL password? In-Reply-To: <524c5a34$0$29984$c3e8da3$5496439d@news.astraweb.com> References: <524c1ee6$0$29984$c3e8da3$5496439d@news.astraweb.com> <524c5a34$0$29984$c3e8da3$5496439d@news.astraweb.com> Message-ID: ???? 2/10/2013 8:39 ??, ?/? Steven D'Aprano ??????: > On Wed, 02 Oct 2013 16:41:40 +0300, ????? wrote: > >> ???? 2/10/2013 4:25 ??, ?/? Steven D'Aprano ??????: >>> On Wed, 02 Oct 2013 15:20:00 +0300, ????? wrote: >>> >>>> Is it possible for someone that knows the MYSQL password of a server >>>> to run arbitrary code on a linux server? >>> >>> Yes, it is possible. >> >> Is that what might have happened and someone managed to upload the .html >> file in '~/home/nikos/www/' ? > > How the hell should I know? I am not a MySQL expert, and this is not a > MySQL forum. > > Nikos, you embarrass me. I have gone out on a limb for you, and this is > how you thank me? You said you were improving, and yet here you go > completely ignoring the links I sent you, and continuing to ask off-topic > questions here. > > Thanks for kicking me in the guts. I will remember this next time you ask > a question. > > I just asked your opinion at this. But i okey i will stop since this is not going us anywhere. Neither will i replay to any more insulting comments. -- What is now proved was at first only imagined! & WebHost From michi.schwarz at gmail.com Wed Oct 2 11:38:21 2013 From: michi.schwarz at gmail.com (Michael Schwarz) Date: Wed, 2 Oct 2013 17:38:21 +0200 Subject: Running code from source that includes extension modules Message-ID: Hi I've just started looking into distutils because I need to write an extension module in C (for performance reasons) and distutils seems to be the most straight-forward way. I've had success building a C file into a Python extension module using "python setup.py build" but I am wondering what the recommended way for using that module during development is. While writing Python code I'm used to just run the code from the source directory. But the built extension module's .so of course does not just end up on sys.path magically. So how do I run my code so it will find the built extension module? Do I pass the output directory on the command line manually or is there some other solution? I would like to still be able to run the code from the source directory as I'm using PyCharm to edit and debug the code. Many thanks! Michael -------------- next part -------------- An HTML attachment was scrubbed... URL: From gvanem at yahoo.no Wed Oct 2 13:15:31 2013 From: gvanem at yahoo.no (Gisle Vanem) Date: Wed, 2 Oct 2013 19:15:31 +0200 Subject: Running code from source that includes extension modules References: Message-ID: "Michael Schwarz" wrote: > So how do I run my code so it will find the built extension module? Do I > pass the output directory on the command line manually or is there some > other solution? I would like to still be able to run the code from the > source directory as I'm using PyCharm to edit and debug the code. Doesn't Python on Linux (I assume that since you mentioned the module's .so) support having current-dir '.' in $PYTHONPATH? Works fine on Windows. Check with "python -v script.py | grep ". --gv From michi.schwarz at gmail.com Wed Oct 2 14:42:46 2013 From: michi.schwarz at gmail.com (Michael Schwarz) Date: Wed, 2 Oct 2013 20:42:46 +0200 Subject: Running code from source that includes extension modules In-Reply-To: References: Message-ID: <6015BCF4-3215-4446-B6B2-5E0ECCDAF238@gmail.com> On 2013-W40-3, at 19:15, "Gisle Vanem" wrote: > "Michael Schwarz" wrote: > >> So how do I run my code so it will find the built extension module? Do I >> pass the output directory on the command line manually or is there some >> other solution? I would like to still be able to run the code from the >> source directory as I'm using PyCharm to edit and debug the code. > > Doesn't Python on Linux (I assume that since you mentioned the module's .so) > support having current-dir '.' in $PYTHONPATH? Works fine on Windows. I'm running OS X 10.8 and Python 3.2, sorry I didn't mention it. But I assume the differences to Linux are minimal. The current directory is included in sys.path, otherwise I wouldn't be able to import modules in the same directory. But the problem is that the built extension module is in a subdirectory of the "build" directory: $ find -name '*.so' ./build/lib.macosx-10.8-x86_64-3.2/_foo.so And so I can't import it without manually adding that directory to sys.path. I'm convinced, someone on this list can shout at me, telling me that I got it completely backwards and that there's a straightforward and intuitive way to develop extension modules! Michael -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 2227 bytes Desc: not available URL: From stefan_ml at behnel.de Wed Oct 2 15:15:04 2013 From: stefan_ml at behnel.de (Stefan Behnel) Date: Wed, 02 Oct 2013 21:15:04 +0200 Subject: Running code from source that includes extension modules In-Reply-To: References: Message-ID: Michael Schwarz, 02.10.2013 17:38: > I've just started looking into distutils because I need to write an > extension module in C (for performance reasons) and distutils seems to be > the most straight-forward way. > > I've had success building a C file into a Python extension module using > "python setup.py build" but I am wondering what the recommended way for > using that module during development is. While writing Python code I'm used > to just run the code from the source directory. But the built extension > module's .so of course does not just end up on sys.path magically. > > So how do I run my code so it will find the built extension module? Do I > pass the output directory on the command line manually or is there some > other solution? I would like to still be able to run the code from the > source directory as I'm using PyCharm to edit and debug the code. You can run python setup.py build_ext -i That will build your extension module and install it right into your package structure. BTW, if you use Cython instead of plain C, you can use pyximport to get on-the-fly extension module builds during development. Stefan From michi.schwarz at gmail.com Wed Oct 2 18:28:18 2013 From: michi.schwarz at gmail.com (Michael Schwarz) Date: Thu, 3 Oct 2013 00:28:18 +0200 Subject: Running code from source that includes extension modules In-Reply-To: References: Message-ID: On 2013-W40-3, at 21:15, Stefan Behnel wrote: > Michael Schwarz, 02.10.2013 17:38: >> I've just started looking into distutils because I need to write an >> extension module in C (for performance reasons) and distutils seems to be >> the most straight-forward way. >> >> I've had success building a C file into a Python extension module using >> "python setup.py build" but I am wondering what the recommended way for >> using that module during development is. While writing Python code I'm used >> to just run the code from the source directory. But the built extension >> module's .so of course does not just end up on sys.path magically. >> >> So how do I run my code so it will find the built extension module? Do I >> pass the output directory on the command line manually or is there some >> other solution? I would like to still be able to run the code from the >> source directory as I'm using PyCharm to edit and debug the code. > > You can run > > python setup.py build_ext -i > > That will build your extension module and install it right into your > package structure. This is really very much what I was looking for! I've set up PyCharm to run this command (by configuring it as an "external tool", maybe there's a simpler way), before running the actual application. \o/ > BTW, if you use Cython instead of plain C, you can use pyximport to get > on-the-fly extension module builds during development. I will look into that too, that sounds very convenient. But am I right, that to use Cython the non-Python code needs to be written in the Cython language, which means I can't just copy&past C code into it? For my current project, this is exactly what I do, because the C code I use already existed. Thanks Michael -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 2227 bytes Desc: not available URL: From oscar.j.benjamin at gmail.com Thu Oct 3 05:01:44 2013 From: oscar.j.benjamin at gmail.com (Oscar Benjamin) Date: Thu, 3 Oct 2013 10:01:44 +0100 Subject: Running code from source that includes extension modules In-Reply-To: References: Message-ID: On 2 October 2013 23:28, Michael Schwarz wrote: > > I will look into that too, that sounds very convenient. But am I right, that to use Cython the non-Python code needs to be written in the Cython language, which means I can't just copy&past C code into it? For my current project, this is exactly what I do, because the C code I use already existed. It's better than that. Don't copy/paste your code. Just declare it in Cython and you can call straight into the existing C functions cutting out most of the boilerplate involved in making C code accessible to Python: http://docs.cython.org/src/userguide/external_C_code.html You'll sometimes need a short Cython wrapper function to convert from Python types to corresponding C types. But this is about 5 lines of easy to read Cython code vs maybe 30 lines of hard to follow C code. Having written CPython extension modules both by hand and using Cython I strongly recommend to use Cython. Oscar From tripsvt at gmail.com Wed Oct 2 13:01:16 2013 From: tripsvt at gmail.com (tripsvt at gmail.com) Date: Wed, 2 Oct 2013 10:01:16 -0700 (PDT) Subject: Rounding off Values of dicts (in a list) to 2 decimal points Message-ID: <0aa3c507-a570-4d39-823d-106ba63b0b70@googlegroups.com> am trying to round off values in a dict to 2 decimal points but have been unsuccessful so far. The input I have is like this: y = [{'a': 80.0, 'b': 0.0786235, 'c': 10.0, 'd': 10.6742903}, {'a': 80.73246, 'b': 0.0, 'c': 10.780323, 'd': 10.0}, {'a': 80.7239, 'b': 0.7823640, 'c': 10.0, 'd': 10.0}, {'a': 80.7802313217234, 'b': 0.0, 'c': 10.0, 'd': 10.9762304}] I want to round off all the values to two decimal points using the ceil function. Here's what I have: def roundingVals_toTwoDeci(): global y for d in y: for k, v in d.items(): v = ceil(v*100)/100.0 return roundingVals_toTwoDeci() But it is not working - I am still getting the old values. From skip at pobox.com Wed Oct 2 13:13:08 2013 From: skip at pobox.com (Skip Montanaro) Date: Wed, 2 Oct 2013 12:13:08 -0500 Subject: Rounding off Values of dicts (in a list) to 2 decimal points In-Reply-To: <0aa3c507-a570-4d39-823d-106ba63b0b70@googlegroups.com> References: <0aa3c507-a570-4d39-823d-106ba63b0b70@googlegroups.com> Message-ID: > def roundingVals_toTwoDeci(): > global y > for d in y: > for k, v in d.items(): > v = ceil(v*100)/100.0 > return > roundingVals_toTwoDeci() > > > > But it is not working - I am still getting the old values. You're not assigning the rounded value back into d. After assigning to v try this: d[k] = v Skip From jpiitula at ling.helsinki.fi Wed Oct 2 13:22:03 2013 From: jpiitula at ling.helsinki.fi (Jussi Piitulainen) Date: 02 Oct 2013 20:22:03 +0300 Subject: Rounding off Values of dicts (in a list) to 2 decimal points References: <0aa3c507-a570-4d39-823d-106ba63b0b70@googlegroups.com> Message-ID: tripsvt at gmail.com writes: > am trying to round off values in a dict to 2 decimal points but > have been unsuccessful so far. The input I have is like this: > > y = [{'a': 80.0, 'b': 0.0786235, 'c': 10.0, 'd': 10.6742903}, > {'a': 80.73246, 'b': 0.0, 'c': 10.780323, 'd': 10.0}, {'a': > 80.7239, 'b': 0.7823640, 'c': 10.0, 'd': 10.0}, {'a': > 80.7802313217234, 'b': 0.0, 'c': 10.0, 'd': 10.9762304}] > > I want to round off all the values to two decimal points using the > ceil function. Here's what I have: > > def roundingVals_toTwoDeci(): > global y > for d in y: > for k, v in d.items(): > v = ceil(v*100)/100.0 > return > roundingVals_toTwoDeci() > > But it is not working - I am still getting the old values. You are assigning to a local variable, v. Instead, store the new values back to the dict like this: d[k] = ceil(v*100)/100.0 And you don't need to declare y global. It would only be needed if you assigned directly to it, as in y = ... (usually not a good idea). The rounding may not work the way you expect, because float values are stored in binary. You may need a decimal type, or you may need to format the output when printing instead. From joel.goldstick at gmail.com Wed Oct 2 13:19:54 2013 From: joel.goldstick at gmail.com (Joel Goldstick) Date: Wed, 2 Oct 2013 13:19:54 -0400 Subject: Rounding off Values of dicts (in a list) to 2 decimal points In-Reply-To: <0aa3c507-a570-4d39-823d-106ba63b0b70@googlegroups.com> References: <0aa3c507-a570-4d39-823d-106ba63b0b70@googlegroups.com> Message-ID: On Wed, Oct 2, 2013 at 1:01 PM, wrote: > am trying to round off values in a dict to 2 decimal points but have been unsuccessful so far. The input I have is like this: > > > y = [{'a': 80.0, 'b': 0.0786235, 'c': 10.0, 'd': 10.6742903}, {'a': 80.73246, 'b': 0.0, 'c': 10.780323, 'd': 10.0}, {'a': 80.7239, 'b': 0.7823640, 'c': 10.0, 'd': 10.0}, {'a': 80.7802313217234, 'b': 0.0, 'c': 10.0, 'd': 10.9762304}] > > > > I want to round off all the values to two decimal points using the ceil function. Here's what I have: This is a snippet of what you have I am guessing. There is no print statement so you won't be able to see the results. Its best if you include your complete code (if its short) or an example that actually shows the problem.. > > > def roundingVals_toTwoDeci(): > global y > for d in y: > for k, v in d.items(): > v = ceil(v*100)/100.0 > return > roundingVals_toTwoDeci() > > That being said, you should pass y as a parameter to your function. Using globals is always a bad idea. That's another discussion entirely, but you should google why globals are a bad idea to learn more. Your code does a calculation to create a value you call v. You should put a print statement below that to see what v has become. Your inner loop rewrites v for each loop. It actually re-writes it twice i think -- once when it iterates, and once when it calculates. So you need to fix that. Also I think you need to interate using d.interitems() That's a start. Come back with the code you actually wrote and the results it showed you. Af > > But it is not working - I am still getting the old values. > -- > https://mail.python.org/mailman/listinfo/python-list -- Joel Goldstick http://joelgoldstick.com From neilc at norwich.edu Wed Oct 2 13:32:14 2013 From: neilc at norwich.edu (Neil Cerutti) Date: 2 Oct 2013 17:32:14 GMT Subject: Rounding off Values of dicts (in a list) to 2 decimal points References: <0aa3c507-a570-4d39-823d-106ba63b0b70@googlegroups.com> Message-ID: On 2013-10-02, tripsvt at gmail.com wrote: > am trying to round off values in a dict to 2 decimal points > but have been unsuccessful so far. The input I have is like > this: > > y = [{'a': 80.0, 'b': 0.0786235, 'c': 10.0, 'd': 10.6742903}, {'a': 80.73246, 'b': 0.0, 'c': 10.780323, 'd': 10.0}, {'a': 80.7239, 'b': 0.7823640, 'c': 10.0, 'd': 10.0}, {'a': 80.7802313217234, 'b': 0.0, 'c': 10.0, 'd': 10.9762304}] > > I want to round off all the values to two decimal points using > the ceil function. Here's what I have: I recommend using the builtin function round instead of math.ceil. math.ceil doesn't do what is normally thought of as rounding. In addition, it supports rounding to different numbers of decimal places. > def roundingVals_toTwoDeci(): > global y You are hopefully* making modifications to y's object, but not rebinding y, so you don't need this global statement. > for d in y: > for k, v in d.items(): > v = ceil(v*100)/100.0 [*] You're binding v to a new float object here, but not modifying y. Thus, this code will have no effect on y. You need to assign to y[k] here instead. for k, v in d.items(): y[k] = round(v, 2) > return Bare returns are not usual at the end of Python functions. Just let the function end; it returns None either way. Only return when you've got an interesting value to return, or when you need to end execution of the function early. -- Neil Cerutti From tripsvt at gmail.com Thu Oct 3 13:06:18 2013 From: tripsvt at gmail.com (tripsvt at gmail.com) Date: Thu, 3 Oct 2013 10:06:18 -0700 (PDT) Subject: Rounding off Values of dicts (in a list) to 2 decimal points In-Reply-To: <0aa3c507-a570-4d39-823d-106ba63b0b70@googlegroups.com> References: <0aa3c507-a570-4d39-823d-106ba63b0b70@googlegroups.com> Message-ID: <23564349-7dd1-4d7a-9bfb-8483ddf3335e@googlegroups.com> On Wednesday, October 2, 2013 10:01:16 AM UTC-7, tri... at gmail.com wrote: > am trying to round off values in a dict to 2 decimal points but have been unsuccessful so far. The input I have is like this: > > > > > > y = [{'a': 80.0, 'b': 0.0786235, 'c': 10.0, 'd': 10.6742903}, {'a': 80.73246, 'b': 0.0, 'c': 10.780323, 'd': 10.0}, {'a': 80.7239, 'b': 0.7823640, 'c': 10.0, 'd': 10.0}, {'a': 80.7802313217234, 'b': 0.0, 'c': 10.0, 'd': 10.9762304}] > > > > > > > > I want to round off all the values to two decimal points using the ceil function. Here's what I have: > > > > > > def roundingVals_toTwoDeci(): > > global y > > for d in y: > > for k, v in d.items(): > > v = ceil(v*100)/100.0 > > return > > roundingVals_toTwoDeci() > > > > > > > > But it is not working - I am still getting the old values. ____________________________________ I am not sure what's going on but here's the current scenario: I get the values with 2 decimal places as I originally required. When I do json.dumps(), it works fine. The goal is to send them to a URL and so I do a urlencode. When I decode the urlencoded string, it gives me the same goodold 2 decimal places. But, for some reason, at the URL, when I check, it no longer limits the values to 2 decimal places, but shows values like 9.10003677694312. What's going on. Here's the code that I have: class LessPrecise(float): def __repr__(self): return str(self) def roundingVals_toTwoDeci(y): for d in y: for k, v in d.iteritems(): d[k] = LessPrecise(round(v, 2)) return roundingVals_toTwoDeci(y) j = json.dumps(y) print j //At this point, print j gives me [{"a": 80.0, "b": 0.0, "c": 10.0, "d": 10.0}, {"a": 100.0, "b": 0.0, "c": 0.0, "d": 0.0}, {"a": 80.0, "b": 0.0, "c": 10.0, "d": 10.0}, {"a": 90.0, "b": 0.0, "c": 0.0, "d": 10.0}] //then I do, params = urllib.urlencode({'thekey': j}) //I then decode params and print it and it gives me thekey=[{"a": 80.0, "b": 0.0, "c": 10.0, "d": 10.0}, {"a": 100.0, "b": 0.0, "c": 0.0, "d": 0.0}, {"a": 80.0, "b": 0.0, "c": 10.0, "d": 10.0}, {"a": 90.0, "b": 0.0, "c": 0.0, "d": 10.0}] However, at the URL, the values show up as 90.000043278694123 From __peter__ at web.de Thu Oct 3 13:41:36 2013 From: __peter__ at web.de (Peter Otten) Date: Thu, 03 Oct 2013 19:41:36 +0200 Subject: Rounding off Values of dicts (in a list) to 2 decimal points References: <0aa3c507-a570-4d39-823d-106ba63b0b70@googlegroups.com> <23564349-7dd1-4d7a-9bfb-8483ddf3335e@googlegroups.com> Message-ID: tripsvt at gmail.com wrote: > On Wednesday, October 2, 2013 10:01:16 AM UTC-7, tri... at gmail.com wrote: >> am trying to round off values in a dict to 2 decimal points but have been >> unsuccessful so far. The input I have is like this: >> >> >> >> >> >> y = [{'a': 80.0, 'b': 0.0786235, 'c': 10.0, 'd': 10.6742903}, {'a': >> 80.73246, 'b': 0.0, 'c': 10.780323, 'd': 10.0}, {'a': 80.7239, 'b': >> 0.7823640, 'c': 10.0, 'd': 10.0}, {'a': 80.7802313217234, 'b': 0.0, >> 'c': 10.0, 'd': 10.9762304}] >> >> >> >> >> >> >> >> I want to round off all the values to two decimal points using the ceil >> function. Here's what I have: >> >> >> >> >> >> def roundingVals_toTwoDeci(): >> >> global y >> >> for d in y: >> >> for k, v in d.items(): >> >> v = ceil(v*100)/100.0 >> >> return >> >> roundingVals_toTwoDeci() >> >> >> >> >> >> >> >> But it is not working - I am still getting the old values. > ____________________________________ > > I am not sure what's going on but here's the current scenario: I get the > values with 2 decimal places as I originally required. When I do > json.dumps(), it works fine. The goal is to send them to a URL and so I do > a urlencode. When I decode the urlencoded string, it gives me the same > goodold 2 decimal places. But, for some reason, at the URL, when I check, > it no longer limits the values to 2 decimal places, but shows values like > 9.10003677694312. What's going on. Here's the code that I have: > > class LessPrecise(float): > def __repr__(self): > return str(self) > > def roundingVals_toTwoDeci(y): > for d in y: > for k, v in d.iteritems(): > d[k] = LessPrecise(round(v, 2)) > return That should only process the first dict in the list, due to a misplaced return. > roundingVals_toTwoDeci(y) > j = json.dumps(y) > print j > > //At this point, print j gives me > > [{"a": 80.0, "b": 0.0, "c": 10.0, "d": 10.0}, {"a": 100.0, "b": 0.0, "c": > [{0.0, "d": 0.0}, {"a": > 80.0, "b": 0.0, "c": 10.0, "d": 10.0}, {"a": 90.0, "b": 0.0, "c": 0.0, > "d": 10.0}] > > //then I do, > params = urllib.urlencode({'thekey': j}) > > //I then decode params and print it and it gives me > > thekey=[{"a": 80.0, "b": 0.0, "c": 10.0, "d": 10.0}, {"a": 100.0, "b": > 0.0, "c": 0.0, "d": 0.0}, {"a": 80.0, "b": 0.0, "c": 10.0, "d": 10.0}, > {"a": 90.0, "b": 0.0, "c": 0.0, "d": 10.0}] > > However, at the URL, the values show up as 90.000043278694123 Can you give the actual code, including the decoding part? Preferably you'd put both encoding and decoding into one small self-contained demo script. From neilc at norwich.edu Thu Oct 3 14:03:17 2013 From: neilc at norwich.edu (Neil Cerutti) Date: 3 Oct 2013 18:03:17 GMT Subject: Rounding off Values of dicts (in a list) to 2 decimal points References: <0aa3c507-a570-4d39-823d-106ba63b0b70@googlegroups.com> <23564349-7dd1-4d7a-9bfb-8483ddf3335e@googlegroups.com> Message-ID: On 2013-10-03, tripsvt at gmail.com wrote: > thekey=[{"a": 80.0, "b": 0.0, "c": 10.0, "d": 10.0}, {"a": > 100.0, "b": 0.0, "c": 0.0, "d": 0.0}, {"a": 80.0, "b": 0.0, > "c": 10.0, "d": 10.0}, {"a": 90.0, "b": 0.0, "c": 0.0, "d": > 10.0}] > > However, at the URL, the values show up as 90.000043278694123 You'll need to convert them to strings yourself before submitting them, by using % formatting or str.format. -- Neil Cerutti From tripsvt at gmail.com Thu Oct 3 14:17:54 2013 From: tripsvt at gmail.com (tripsvt at gmail.com) Date: Thu, 3 Oct 2013 11:17:54 -0700 (PDT) Subject: Rounding off Values of dicts (in a list) to 2 decimal points In-Reply-To: References: <0aa3c507-a570-4d39-823d-106ba63b0b70@googlegroups.com> <23564349-7dd1-4d7a-9bfb-8483ddf3335e@googlegroups.com> Message-ID: <58286399-5eb3-40f8-9d28-e6036451fbf9@googlegroups.com> On Thursday, October 3, 2013 11:03:17 AM UTC-7, Neil Cerutti wrote: > On 2013-10-03, tripsvt at gmail.com wrote: > > > thekey=[{"a": 80.0, "b": 0.0, "c": 10.0, "d": 10.0}, {"a": > > > 100.0, "b": 0.0, "c": 0.0, "d": 0.0}, {"a": 80.0, "b": 0.0, > > > "c": 10.0, "d": 10.0}, {"a": 90.0, "b": 0.0, "c": 0.0, "d": > > > 10.0}] > > > > > > However, at the URL, the values show up as 90.000043278694123 > > > > You'll need to convert them to strings yourself before submitting > > them, by using % formatting or str.format. > > > > -- > > Neil Cerutti I thought the class 'LessPrecise' converts them to strings. But even when I try doing it directly without the class at all, as in str(round(v, 2)), it gives all the expected values (as in {"a": "10.1", "b": "3.4", etc.}) but at the URL, it gives all the decimal places - 10.78324783923783 From josiah.carlson at gmail.com Wed Oct 2 13:53:16 2013 From: josiah.carlson at gmail.com (Josiah Carlson) Date: Wed, 2 Oct 2013 10:53:16 -0700 Subject: ANN: rom 0.21 - Redis object mapper for Python Message-ID: Hey everyone, As time progresses, so does my Redis object mapper. The "rom" package is a Redis object mapper for Python. It sports an interface similar to Django's ORM, SQLAlchemy + Elixir, or Appengine's datastore. The changelog for recent releases can be seen below my signature. You can find the package at: https://www.github.com/josiahcarlson/rom https://pypi.python.org/pypi/rom And docs can be found at: http://pythonhosted.org/rom/ Please CC me on any replies if you have any questions or comments. Thank you, - Josiah #----------------------------------- 0.21 ------------------------------------ [fixed] upload for rom 0.20 was missing new columns.py, now fixed #----------------------------------- 0.20 ------------------------------------ [changed] Added exception when performing .all(), .execute(), or .count() on query objects that have had no filters or attribute ordering provided. This addresses issue #12. [changed] Moved column definitions to their own module, shouldn't affect any normal uses of rom. [added] For users of Redis 2.6 and later, there is a beta Lua-enabled writing option that allows for multiple unique columns on models. In some cases, this may improve performance when writing many entities very quickly. [added] The ability to reload an entity from Redis, optionally discarding any modifications to the object itself. Check out the documentation for Model.refresh(), Session.refresh(), and Session.refresh_all() [added] Tests for the newly changed/added features. [changed] Tests no longer use flushdb() - all test models/indexes/etc. are prefixed with RomTest, and we find/delete such keys before and after any tests are run. Now anyone can reasonably run the test suite. #----------------------------------- 0.19 ------------------------------------ [fixed] Thanks to a bug report by https://github.com/MickeyKim , was notified of a bug when using unique indexes, which is now fixed and has a testcase. #----------------------------------- 0.18 ------------------------------------ [fixed] Thanks to a bug report by https://github.com/MickeyKim , was notified and received an interim patch for a bug that could cause deleted entities to be resurrected on session.commit() or session.flush() . This has now been fixed and a testcase has been added. -------------- next part -------------- An HTML attachment was scrubbed... URL: From alan.spence at ntlworld.com Wed Oct 2 15:22:50 2013 From: alan.spence at ntlworld.com (Alan Spence) Date: Wed, 2 Oct 2013 20:22:50 +0100 Subject: Trolls, Nikos, Ferrous Cranus Message-ID: <5B2AC3D5-4F87-4158-88B4-404E1C328196@ntlworld.com> Steven D'Aprano once wrote: > Definitely troll, and we got well and truly had. > > Shame on us -- he practically *screamed* "I am a troll" in his user-name. > "Ferrous Cranus" ("Iron Skull") is the name of a particular subspecies of > troll in Mike Reed's well known compendium: > > http://redwing.hutman.net/~mreed/war...rouscranus.htm > > > How many days was he posting here, pretending to be dumber than a box of > hammers? How many hours have we collectively wasted, reading and replying > to his posts? > > I wouldn't even trust that he is Greek. Quite likely he uses a non- > English attribution line to hint that English is not his first language, > so his victims will think that there is a language barrier. "Maybe he > doesn't understand me. If I answer again, in a slightly different way, > perhaps it will be more clear." Why did you start to engage again? Pity the link which I originally pointed out is no longer valid. Alan From rouslank at msn.com Wed Oct 2 17:05:32 2013 From: rouslank at msn.com (Rouslan Korneychuk) Date: Wed, 02 Oct 2013 17:05:32 -0400 Subject: Hyper-spacial ray-tracer Message-ID: I have been working on something I thought was interesting and I wanted to know what other people think. It's a ray-tracing library than can work with any number of spacial dimensions greater than two. It's a Python package that uses Pygame. The project and a screenshot are at: https://github.com/Rouslan/NTracer For those not familiar with the concept of hyper-space: a simple example of a three-dimensional object is a cube. A two-dimensional analogue is a square. With one dimension, it would be a line (and with zero dimensions, a point). Although our universe only has three spacial dimensions (ignore theoretical physics for a moment), there is actually no reason why it can't be any other number, and so you can go the other way. A four-dimensional analogue of a cube is a tesseract, and when generalized for any number of dimensions it's called a hypercube. Of course, it's really hard to imagine anything with more than three dimensions, which is precisely why I wrote this library. The screenshot in the link shows a three-dimensional cross-section of a six-dimensional hypercube at a particular angle. So far, all the library can draw is a scene with one hypercube (although you can position the camera anywhere you want), but I'm planning to add support for complex scenes where you can put various kinds of shapes with arbitrary transformations and materials (color and opacity at least). From rouslank at msn.com Fri Oct 4 20:17:52 2013 From: rouslank at msn.com (Rouslan Korneychuk) Date: Fri, 04 Oct 2013 20:17:52 -0400 Subject: Hyper-spacial ray-tracer In-Reply-To: References: Message-ID: On 10/04/2013 04:23 PM, Tony the Tiger wrote: > On Wed, 02 Oct 2013 17:05:32 -0400, Rouslan Korneychuk wrote: > >> game > > Sorry, but that sounds awful. I hate games. > This... isn't a game or even related to gaming. Is it because of the use of Pygame that you thought it was. I use Pygame because it's a wrapper for SDL, which gives you cross-platform graphics, input and even thread support, and because the additional drawing and font modules are useful for prototyping and implementing user-interfaces for navigating higher-dimensional space. The point of this was to explore the concept of hyperspace, which is a mathematical curiosity and also has relevance in theoretical physics. One idea I had for this was to simulate some sort of 3D scene involving physics (probably in another program, such as Blender), take the resulting coordinates of the geometry at every time interval and plot it as one 4D static scene. Every pair of connected vertexes would be extruded from one instant in time, to the next, so each object is a continuous 4D extrusion. When viewing with your local XYZ axes aligned with the global XYZ axes, you would see one instant of the scene as normal. Moving along the fourth axis, which I'll call T, will let you see the same, earlier or later in time, but if you rotate parallel to the T axis, you will effectively replace one of X, Y or Z with T. In essence you will turn the time axis into a spacial axis and the spacial axis into a time axis. Looking at a scene with space and time lumped into one 4D space might help in trying to better understand time, why it's different, and its relationship with space. I was also wondering about general relativity. I'm not going to go into too much detail, but basically: if an object with synchronized clocks on either end of it, passes by a static observer while traveling near the speed of light, to the outside observer, the object will appear shorter and the clocks will appear desynchronized, and from the object's perspective, it is the outside observer that becomes distorted this way. I was wondering if this seemingly strange effect is actually the natural consequence of a simple geometric transformation, such as rotation into the time axis. From rosuav at gmail.com Fri Oct 4 21:41:18 2013 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 5 Oct 2013 11:41:18 +1000 Subject: Hyper-spacial ray-tracer In-Reply-To: References: Message-ID: On Sat, Oct 5, 2013 at 10:17 AM, Rouslan Korneychuk wrote: > The point of this was to explore the concept of hyperspace, which is a > mathematical curiosity and also has relevance in theoretical physics. I don't have any actual use-case for what you've done, but it sure sounds cool! Having worked with 3D ray-tracing (with POV-Ray), I'm slightly in awe of the possibility of going to ten dimensions... yup, cool! ChrisA From rouslank at msn.com Fri Oct 4 23:00:48 2013 From: rouslank at msn.com (Rouslan Korneychuk) Date: Fri, 04 Oct 2013 23:00:48 -0400 Subject: Hyper-spacial ray-tracer In-Reply-To: References: Message-ID: On 10/04/2013 09:41 PM, Chris Angelico wrote: > On Sat, Oct 5, 2013 at 10:17 AM, Rouslan Korneychuk wrote: >> The point of this was to explore the concept of hyperspace, which is a >> mathematical curiosity and also has relevance in theoretical physics. > > I don't have any actual use-case for what you've done, but it sure > sounds cool! Having worked with 3D ray-tracing (with POV-Ray), I'm > slightly in awe of the possibility of going to ten dimensions... yup, > cool! > Thanks. For a while, I was worried nobody else thought it was interesting. It's funny that you say that about ten dimensions, considering I was thinking I should add scroll bars to the example script so the controls don't get cut off when going to 100 dimensions. From dihedral88888 at gmail.com Sat Oct 5 02:13:09 2013 From: dihedral88888 at gmail.com (88888 Dihedral) Date: Fri, 4 Oct 2013 23:13:09 -0700 (PDT) Subject: Hyper-spacial ray-tracer In-Reply-To: References: Message-ID: On Saturday, October 5, 2013 8:17:52 AM UTC+8, Rouslan Korneychuk wrote: > On 10/04/2013 04:23 PM, Tony the Tiger wrote: > > > On Wed, 02 Oct 2013 17:05:32 -0400, Rouslan Korneychuk wrote: > > > > > >> game > > > > > > Sorry, but that sounds awful. I hate games. > > > > > > > This... isn't a game or even related to gaming. Is it because of the use > > of Pygame that you thought it was. I use Pygame because it's a wrapper > > for SDL, which gives you cross-platform graphics, input and even thread > > support, and because the additional drawing and font modules are useful > > for prototyping and implementing user-interfaces for navigating > > higher-dimensional space. > > > > The point of this was to explore the concept of hyperspace, which is a > > mathematical curiosity and also has relevance in theoretical physics. > > > > One idea I had for this was to simulate some sort of 3D scene involving > > physics (probably in another program, such as Blender), take the > > resulting coordinates of the geometry at every time interval and plot it > > as one 4D static scene. Every pair of connected vertexes would be > > extruded from one instant in time, to the next, so each object is a > > continuous 4D extrusion. When viewing with your local XYZ axes aligned > > with the global XYZ axes, you would see one instant of the scene as > > normal. Moving along the fourth axis, which I'll call T, will let you > > see the same, earlier or later in time, but if you rotate parallel to > > the T axis, you will effectively replace one of X, Y or Z with T. In > > essence you will turn the time axis into a spacial axis and the spacial > > axis into a time axis. > > > > Looking at a scene with space and time lumped into one 4D space might > > help in trying to better understand time, why it's different, and its > > relationship with space. > > > > I was also wondering about general relativity. I'm not going to go into > > too much detail, but basically: if an object with synchronized clocks on > > either end of it, passes by a static observer while traveling near the > > speed of light, to the outside observer, the object will appear shorter > > and the clocks will appear desynchronized, and from the object's > > perspective, it is the outside observer that becomes distorted this way. > > I was wondering if this seemingly strange effect is actually the natural > > consequence of a simple geometric transformation, such as rotation into > > the time axis. Use the synchronous digital logics with a globbal clock by iterators of various actions for this kind of projects in Python. Please check myHDL and Python. auto- From ppearson at nowhere.invalid Sat Oct 5 11:26:28 2013 From: ppearson at nowhere.invalid (Peter Pearson) Date: 5 Oct 2013 15:26:28 GMT Subject: [OT] Re: Hyper-spacial ray-tracer References: Message-ID: On Fri, 04 Oct 2013 20:17:52 -0400, Rouslan Korneychuk wrote: [snip] > I was also wondering about general relativity. I'm not going to go into > too much detail, but basically: if an object with synchronized clocks on > either end of it, passes by a static observer while traveling near the > speed of light, to the outside observer, the object will appear shorter [snip] That's special relativity, not general relativity. Python is very sensitive to that distinction. -- To email me, substitute nowhere->spamcop, invalid->net. From rouslank at msn.com Sat Oct 5 14:14:28 2013 From: rouslank at msn.com (Rouslan Korneychuk) Date: Sat, 05 Oct 2013 14:14:28 -0400 Subject: [OT] Re: Hyper-spacial ray-tracer In-Reply-To: References: Message-ID: <8GY3u.12965$I35.533@fx01.iad> On 10/05/2013 11:26 AM, Peter Pearson wrote: > On Fri, 04 Oct 2013 20:17:52 -0400, Rouslan Korneychuk wrote: > [snip] >> I was also wondering about general relativity. I'm not going to go into >> too much detail, but basically: if an object with synchronized clocks on >> either end of it, passes by a static observer while traveling near the >> speed of light, to the outside observer, the object will appear shorter > [snip] > > That's special relativity, not general relativity. Python is > very sensitive to that distinction. > whoops From phil at riverbankcomputing.com Wed Oct 2 21:37:34 2013 From: phil at riverbankcomputing.com (Phil Thompson) Date: Thu, 03 Oct 2013 02:37:34 +0100 Subject: ANN: PyQt5 v5.1 Released Message-ID: PyQt5 v5.1 has been released and is available from http://www.riverbankcomputing.com/software/pyqt/download5. PyQt5 is a comprehensive set of bindings for v5 of Digia's Qt cross-platform application framework. It supports Python v3, v2.7 and v2.6. The highlights of this release include full support for Qt v5.1, the QtSensors and QtSerialPort modules, and bindings for OpenGL v2.0 and OpenGL ES/2. Windows installers are provided which contain everything needed for PyQt5 development (including Qt, Qt Designer, QScintilla, and MySQL, PostgreSQL, SQLite and ODBC drivers) except Python itself. Installers are provided for the 32 and 64 bit versions of Python v3.3. PyQt5 is implemented as a set of 22 extension modules including support for: - non-GUI infrastructure including event loops, threads, i18n, user and application settings, mapped files and shared memory - GUI infrastructure including window system integration, event handling, 2D graphics, basic imaging, fonts, OpenGL - a comprehensive set of desktop widgets - WebKit - full integration with Quick2 and QML allowing new Quick items to be implemented in Python and created in QML - event driven network programming - multimedia including cameras, audio and radios - sensors including accelerometers, altimeters, compasses, gyroscopes, magnetometers, and light, pressure, proximity, rotation and temperature sensors - serial ports - SQL - printing - DBus - XPath, XQuery, XSLT and XML Schema validation - a help system for creating and viewing searchable documentation - unit testing of GUI applications. From artyprog at gmail.com Thu Oct 3 03:48:47 2013 From: artyprog at gmail.com (Salvatore DI DIO) Date: Thu, 3 Oct 2013 00:48:47 -0700 (PDT) Subject: Nodebox(v1) on the web via RapydScript Message-ID: <41edf166-c5e5-4e1c-a387-ff485fa27aa3@googlegroups.com> Hello, Nodebox is a program in the spirit of Processing but for Python. The first version runs only on MAC. Tom, the creator has partly ported it to Javascript. But many of you dislike Javascript. The solution was to use a translator, Python -> Javascript Of the both two greats solutions Brython / RapydScript, I've choosen RapydScript (Brython and RapydScript does not achieve the same goal) You can see a preview of 'Nodebox on the Web' namely 'RapydBox' here : http://salvatore.pythonanywhere.com/RapydBox Regards From lightaiyee at gmail.com Thu Oct 3 12:01:18 2013 From: lightaiyee at gmail.com (JL) Date: Thu, 3 Oct 2013 09:01:18 -0700 (PDT) Subject: Multiple scripts versus single multi-threaded script Message-ID: What is the difference between running multiple python scripts and a single multi-threaded script? May I know what are the pros and cons of each approach? Right now, my preference is to run multiple separate python scripts because it is simpler. From rosuav at gmail.com Thu Oct 3 12:42:04 2013 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 4 Oct 2013 02:42:04 +1000 Subject: Multiple scripts versus single multi-threaded script In-Reply-To: References: Message-ID: On Fri, Oct 4, 2013 at 2:01 AM, JL wrote: > What is the difference between running multiple python scripts and a single multi-threaded script? May I know what are the pros and cons of each approach? Right now, my preference is to run multiple separate python scripts because it is simpler. (Caveat: The below is based on CPython. If you're using IronPython, Jython, or some other implementation, some details may be a little different.) Multiple threads can share state easily by simply referencing each other's variables, but the cost of that is that they'll never actually execute simultaneously. If you want your scripts to run in parallel on multiple CPUs/cores, you need multiple processes. But if you're doing something I/O bound (like servicing sockets), threads work just fine. As to using separate scripts versus the multiprocessing module, that's purely a matter of what looks cleanest. Do whatever suits your code. ChrisA From roy at panix.com Thu Oct 3 12:41:51 2013 From: roy at panix.com (Roy Smith) Date: Thu, 03 Oct 2013 12:41:51 -0400 Subject: Multiple scripts versus single multi-threaded script References: Message-ID: In article , JL wrote: > What is the difference between running multiple python scripts and a single > multi-threaded script? May I know what are the pros and cons of each > approach? Right now, my preference is to run multiple separate python scripts > because it is simpler. First, let's take a step back and think about multi-threading vs. multi-processing in general (i.e. in any language). Threads are lighter-weight. That means it's faster to start a new thread (compared to starting a new process), and a thread consumes fewer system resources than a process. If you have lots of short-lived tasks to run, this can be significant. If each task will run for a long time and do a lot of computation, the cost of startup becomes less of an issue because it's amortized over the longer run time. Threads can communicate with each other in ways that processes can't. For example, file descriptors are shared by all the threads in a process, so one thread can open a file (or accept a network connection), then hand the descriptor off to another thread for processing. Threads also make it easy to share large amounts of data because they all have access to the same memory. You can do this between processes with shared memory segments, but it's more work to set up. The downside to threads is that all of of this sharing makes them much more complicated to use properly. You have to be aware of how all the threads are interacting, and mediate access to shared resources. If you do that wrong, you get memory corruption, deadlocks, and all sorts of (extremely) difficult to debug problems. A lot of the really hairy problems (i.e. things like one thread continuing to use memory which another thread has freed) are solved by using a high-level language like Python which handles all the memory allocation for you, but you can still get deadlocks and data corruption. So, the full answer to your question is very complicated. However, if you're looking for a short answer, I'd say just keep doing what you're doing using multiple processes and don't get into threading. From rosuav at gmail.com Thu Oct 3 12:50:00 2013 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 4 Oct 2013 02:50:00 +1000 Subject: Multiple scripts versus single multi-threaded script In-Reply-To: References: Message-ID: On Fri, Oct 4, 2013 at 2:41 AM, Roy Smith wrote: > The downside to threads is that all of of this sharing makes them much > more complicated to use properly. You have to be aware of how all the > threads are interacting, and mediate access to shared resources. If you > do that wrong, you get memory corruption, deadlocks, and all sorts of > (extremely) difficult to debug problems. A lot of the really hairy > problems (i.e. things like one thread continuing to use memory which > another thread has freed) are solved by using a high-level language like > Python which handles all the memory allocation for you, but you can > still get deadlocks and data corruption. With CPython, you don't have any headaches like that; you have one very simple protection, a Global Interpreter Lock (GIL), which guarantees that no two threads will execute Python code simultaneously. No corruption, no deadlocks, no hairy problems. ChrisA From davea at davea.name Thu Oct 3 14:40:13 2013 From: davea at davea.name (Dave Angel) Date: Thu, 3 Oct 2013 18:40:13 +0000 (UTC) Subject: Multiple scripts versus single multi-threaded script References: Message-ID: On 3/10/2013 12:50, Chris Angelico wrote: > On Fri, Oct 4, 2013 at 2:41 AM, Roy Smith wrote: >> The downside to threads is that all of of this sharing makes them much >> more complicated to use properly. You have to be aware of how all the >> threads are interacting, and mediate access to shared resources. If you >> do that wrong, you get memory corruption, deadlocks, and all sorts of >> (extremely) difficult to debug problems. A lot of the really hairy >> problems (i.e. things like one thread continuing to use memory which >> another thread has freed) are solved by using a high-level language like >> Python which handles all the memory allocation for you, but you can >> still get deadlocks and data corruption. > > With CPython, you don't have any headaches like that; you have one > very simple protection, a Global Interpreter Lock (GIL), which > guarantees that no two threads will execute Python code > simultaneously. No corruption, no deadlocks, no hairy problems. > > ChrisA The GIL takes care of the gut-level interpreter issues like reference counts for shared objects. But it does not avoid deadlock or hairy problems. I'll just show one, trivial, problem, but many others exist. If two threads process the same global variable as follows, myglobal = myglobal + 1 Then you have no guarantee that the value will really get incremented twice. Presumably there's a mutex/critsection function in the threading module that can make this safe, but once you use it in two different places, you raise the possibility of deadlock. On the other hand, if you're careful to have the thread use only data that is unique to that thread, then it would seem to be safe. However, you still have the same risk if you call some library that wasn't written to be thread safe. I'll assume that print() and suchlike are safe, but some third party library could well use the equivalent of a global variable in an unsafe way. -- DaveA From roy at panix.com Thu Oct 3 14:28:32 2013 From: roy at panix.com (Roy Smith) Date: Thu, 03 Oct 2013 14:28:32 -0400 Subject: Multiple scripts versus single multi-threaded script References: Message-ID: In article , Chris Angelico wrote: > On Fri, Oct 4, 2013 at 2:41 AM, Roy Smith wrote: > > The downside to threads is that all of of this sharing makes them much > > more complicated to use properly. You have to be aware of how all the > > threads are interacting, and mediate access to shared resources. If you > > do that wrong, you get memory corruption, deadlocks, and all sorts of > > (extremely) difficult to debug problems. A lot of the really hairy > > problems (i.e. things like one thread continuing to use memory which > > another thread has freed) are solved by using a high-level language like > > Python which handles all the memory allocation for you, but you can > > still get deadlocks and data corruption. > > With CPython, you don't have any headaches like that; you have one > very simple protection, a Global Interpreter Lock (GIL), which > guarantees that no two threads will execute Python code > simultaneously. No corruption, no deadlocks, no hairy problems. > > ChrisA Well, the GIL certainly eliminates a whole range of problems, but it's still possible to write code that deadlocks. All that's really needed is for two threads to try to acquire the same two resources, in different orders. I'm running the following code right now. It appears to be doing a pretty good imitation of a deadlock. Any similarity to current political events is purely intentional. import threading import time lock1 = threading.Lock() lock2 = threading.Lock() class House(threading.Thread): def run(self): print "House starting..." lock1.acquire() time.sleep(1) lock2.acquire() print "House running" lock2.release() lock1.release() class Senate(threading.Thread): def run(self): print "Senate starting..." lock2.acquire() time.sleep(1) lock1.acquire() print "Senate running" lock1.release() lock2.release() h = House() s = Senate() h.start() s.start() Similarly, I can have data corruption. I can't get memory corruption in the way you can get in a C/C++ program, but I can certainly have one thread produce data for another thread to consume, and then (incorrectly) continue to mutate that data after it relinquishes ownership. Let's say I have a Queue. A producer thread pushes work units onto the Queue and a consumer thread pulls them off the other end. If my producer thread does something like: work = {'id': 1, 'data': "The Larch"} my_queue.put(work) work['id'] = 3 I've got a race condition where the consumer thread may get an id of either 1 or 3, depending on exactly when it reads the data from its end of the queue (more precisely, exactly when it uses that data). Here's a somewhat different example of data corruption between threads: import threading import random import sys sketch = "The Dead Parrot" class T1(threading.Thread): def run(self): current_sketch = str(sketch) while 1: if sketch != current_sketch: print "Blimey, it's changed!" return class T2(threading.Thread): def run(self): sketches = ["Piranah Brothers", "Spanish Enquisition", "Lumberjack"] while 1: global sketch sketch = random.choice(sketches) t1 = T1() t2 = T2() t2.daemon = True t1.start() t2.start() t1.join() sys.exit() From rosuav at gmail.com Thu Oct 3 14:36:25 2013 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 4 Oct 2013 04:36:25 +1000 Subject: Multiple scripts versus single multi-threaded script In-Reply-To: References: Message-ID: On Fri, Oct 4, 2013 at 4:28 AM, Roy Smith wrote: > Well, the GIL certainly eliminates a whole range of problems, but it's > still possible to write code that deadlocks. All that's really needed > is for two threads to try to acquire the same two resources, in > different orders. I'm running the following code right now. It appears > to be doing a pretty good imitation of a deadlock. Any similarity to > current political events is purely intentional. Right. Sorry, I meant that the GIL protects you from all that happening in the lower level code (even lower than the Senate, here), but yes, you can get deadlocks as soon as you acquire locks. That's nothing to do with threading, you can have the same issues with databases, file systems, or anything else that lets you lock something. It's a LOT easier to deal with deadlocks or data corruption that occurs in pure Python code than in C, since Python has awesome introspection facilities... and you're guaranteed that corrupt data is still valid Python objects. As to your corrupt data example, though, I'd advocate a very simple system of object ownership: as soon as the object has been put on the queue, it's "owned" by the recipient and shouldn't be mutated by anyone else. That kind of system generally isn't hard to maintain. ChrisA From roy at panix.com Thu Oct 3 15:53:38 2013 From: roy at panix.com (Roy Smith) Date: Thu, 03 Oct 2013 15:53:38 -0400 Subject: Multiple scripts versus single multi-threaded script References: Message-ID: In article , Chris Angelico wrote: > As to your corrupt data example, though, I'd advocate a very simple > system of object ownership: as soon as the object has been put on the > queue, it's "owned" by the recipient and shouldn't be mutated by > anyone else. Well, sure. I agree with you that threading in Python is about a zillion times easier to manage than threading in C/C++, but there are still things you need to think about when using threading in Python which you don't need to think about if you're not using threading at all. Transfer of ownership when you put something on a queue is one of those things. So, I think my original statement: > if you're looking for a short answer, I'd say just keep doing what > you're doing using multiple processes and don't get into threading. is still good advice for somebody who isn't sure they need threads. On the other hand, for somebody who is interested in learning about threads, Python is a great platform to learn because you get to experiment with the basic high-level concepts without getting bogged down in pthreads minutiae. And, as Chris pointed out, if you get it wrong, at least you've still got valid Python objects to puzzle over, not a smoking pile of bits on the floor. From rosuav at gmail.com Thu Oct 3 18:22:36 2013 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 4 Oct 2013 08:22:36 +1000 Subject: Multiple scripts versus single multi-threaded script In-Reply-To: References: Message-ID: On Fri, Oct 4, 2013 at 5:53 AM, Roy Smith wrote: > So, I think my original statement: > >> if you're looking for a short answer, I'd say just keep doing what >> you're doing using multiple processes and don't get into threading. > > is still good advice for somebody who isn't sure they need threads. > > On the other hand, for somebody who is interested in learning about > threads, Python is a great platform to learn because you get to > experiment with the basic high-level concepts without getting bogged > down in pthreads minutiae. And, as Chris pointed out, if you get it > wrong, at least you've still got valid Python objects to puzzle over, > not a smoking pile of bits on the floor. Agree wholeheartedly to both halves. I was just explaining a similar concept to my brother last night, with regard to network/database request handling: 1) The simplest code starts, executes, and finishes, with no threads, fork(), or other confusions.or shared state or anything. Execution can be completely predicted by eyeballing the source code. You can pretend that you have a dedicated CPU core that does nothing but run your program. 2) Threaded code adds a measure of complexity that you have to get your head around. Now you need to concern yourself with preemption, multiple threads doing things in different orders, locking, shared state, etc, etc. But you can still pretend that the execution of one job will happen as a single "thing", top down, with predictable intermediate state, if you like. (Python's threading and multiprocess modules both follow this style, they just have different levels of shared state.) 3) Asynchronous code adds significantly more "get your head around" complexity, since you now have to retain state for multiple jobs/requests in the same thread. You can't use local variables to keep track of where you're up to. Most likely, your code will do some tiny thing, update the state object for that request, fire off an asynchronous request of your own (maybe to the hard disk, with a callback when the data's read/written), and then return, back to some main loop. Now imagine you have a database written in style #1, and you have to drag it, kicking and screaming, into the 21st century. Oh look, it's easy! All you have to do is start multiple threads doing the same job! And then you'll have some problems with simultaneous edits, so you put some big fat locks all over the place to prevent two threads from doing the same thing at the same time. Even if one of those threads was handling something interactive and might hold its lock for some number of minutes. Suboptimal design, maybe, but hey, it works right? That's what my brother has to deal with every day, as a user of said database... :| ChrisA From jeremy at jeremysanders.net Fri Oct 4 04:02:42 2013 From: jeremy at jeremysanders.net (Jeremy Sanders) Date: Fri, 04 Oct 2013 10:02:42 +0200 Subject: Multiple scripts versus single multi-threaded script References: Message-ID: Roy Smith wrote: > Threads are lighter-weight. That means it's faster to start a new > thread (compared to starting a new process), and a thread consumes fewer > system resources than a process. If you have lots of short-lived tasks > to run, this can be significant. If each task will run for a long time > and do a lot of computation, the cost of startup becomes less of an > issue because it's amortized over the longer run time. This might be true on Windows, but I think on Linux process overheads are pretty similar to threads, e.g. http://stackoverflow.com/questions/807506/threads-vs-processes-in-linux Combined with the lack of a GIL-conflict, processes can be pretty efficient. Jeremy From invalid at invalid.invalid Fri Oct 4 12:38:00 2013 From: invalid at invalid.invalid (Grant Edwards) Date: Fri, 4 Oct 2013 16:38:00 +0000 (UTC) Subject: Multiple scripts versus single multi-threaded script References: Message-ID: On 2013-10-03, Roy Smith wrote: > Threads are lighter-weight. That means it's faster to start a new > thread (compared to starting a new process), and a thread consumes > fewer system resources than a process. That's true, but the extent to which it's true varies considerably from one OS to another. Starting processes is typically very cheap on Unix systems. On Linux a thread and a process are actually both started by the same system call, and the only significant difference is how some of the new page descriptors are set up (they're copy-on-write instead of shared). On other OSes, starting a process is _way_ more expensive/slow than starting a thread. That was very true for VMS, so one suspects it might also be true for its stepchild MS-Window. -- Grant Edwards grant.b.edwards Yow! RELATIVES!! at gmail.com From ckaynor at zindagigames.com Thu Oct 3 11:58:44 2013 From: ckaynor at zindagigames.com (Chris Kaynor) Date: Thu, 3 Oct 2013 08:58:44 -0700 Subject: [Python-Dev] summing integer and class In-Reply-To: <524D74EE.90308@oracle.com> References: <524D74EE.90308@oracle.com> Message-ID: This list is for development OF Python, not for development in python. For that reason, I will redirect this to python-list as well. My actual answer is below. On Thu, Oct 3, 2013 at 6:45 AM, Igor Vasilyev wrote: > Hi. > > Example test.py: > > class A(): > def __add__(self, var): > print("I'm in A class") > return 5 > a = A() > a+1 > 1+a > Execution: > python test.py > I'm in A class > Traceback (most recent call last): > File "../../test.py", line 7, in > 1+a > TypeError: unsupported operand type(s) for +: 'int' and 'instance' > > > So adding integer to class works fine, but adding class to integer fails. > I could not understand why it happens. In objects/abstact.c we have the > following function: > Based on the code you provided, you are only overloading the __add__ operator, which is only called when an "A" is added to something else, not when something is added to an "A". You can also override the __radd__ method to perform the swapped addition. See http://docs.python.org/2/reference/datamodel.html#object.__radd__ for the documentation (it is just below the entry on __add__). Note that for many simple cases, you could define just a single function, which then is defined as both the __add__ and __radd__ operator. For example, you could modify your "A" sample class to look like: class A(): def __add__(self, var): print("I'm in A") return 5 __radd__ = __add__ Which will produce: >>> a = A() >>> a + 1 I'm in A 5 >>> 1 + a I'm in A 5 Chris -------------- next part -------------- An HTML attachment was scrubbed... URL: From l.mohanphy at gmail.com Thu Oct 3 12:11:30 2013 From: l.mohanphy at gmail.com (Mohan L) Date: Thu, 3 Oct 2013 21:41:30 +0530 Subject: compare two list of dictionaries Message-ID: Dear All, I have two list of dictionaries like below: In the below dictionaries the value of ip can be either hostname or ip address. output1=[ {'count': 3 , 'ip': 'xxx.xx.xxx.1'}, {'count': 4, 'ip': 'xxx.xx.xxx.2'}, {'count': 8, 'ip': 'xxx.xx.xxx.3'}, {'count': 10, 'ip': 'xxx.xx.xxx.4'}, {'count': 212, 'ip': 'hostname1'}, {'count': 27, 'ip': 'hostname2'}, {'count': 513, 'ip': 'hostname3'}, {'count': 98, 'ip': 'hostname4'}, {'count': 1, 'ip': 'hostname10'}, {'count': 2, 'ip': 'hostname8'}, {'count': 3, 'ip': 'xxx.xx.xxx.11'}, {'count': 90, 'ip': 'xxx.xx.xxx.12'}, {'count': 12, 'ip': 'xxx.xx.xxx.13'}, {'count': 21, 'ip': 'xxx.xx.xxx.14'}, {'count': 54, 'ip': 'xxx.xx.xxx.15'}, {'count': 34, 'ip': 'xxx.xx.xxx.16'}, {'count': 11, 'ip': 'xxx.xx.xxx.17'}, {'count': 2, 'ip': 'xxx.xx.xxx.18'}, {'count': 19, 'ip': 'xxx.xx.xxx.19'}, {'count': 21, 'ip': 'xxx.xx.xxx.20'}, {'count': 25, 'ip': 'xxx.xx.xxx.21'}, {'count': 31, 'ip': 'xxx.xx.xxx.22'}, {'count': 43, 'ip': 'xxx.xx.xxx.23'}, {'count': 46, 'ip': 'xxx.xx.xxx.24'}, {'count': 80, 'ip': 'xxx.xx.xxx.25'}, {'count': 91, 'ip': 'xxx.xx.xxx.26'}, {'count': 90, 'ip': 'xxx.xx.xxx.27'}, {'count': 10, 'ip': 'xxx.xx.xxx.28'}, {'count': 3, 'ip': 'xxx.xx.xxx.29'}] In the below dictionaries have either hostname or ip or both. output2=( {'hostname': 'INNCHN01', 'ip_addr': 'xxx.xx.xxx.11'}, {'hostname': 'HYDRHC02', 'ip_addr': 'xxx.xx.xxx.12'}, {'hostname': 'INNCHN03', 'ip_addr': 'xxx.xx.xxx.13'}, {'hostname': 'MUMRHC01', 'ip_addr': 'xxx.xx.xxx.14'}, {'hostname': 'n/a', 'ip_addr': 'xxx.xx.xxx.15'}, {'hostname': 'INNCHN05', 'ip_addr': 'xxx.xx.xxx.16'}, {'hostname': 'hostname1', 'ip_addr': 'n/a'}, {'hostname': 'hostname2', 'ip_addr': 'n/a'}, {'hostname': 'hostname10', 'ip_addr': ''}, {'hostname': 'hostname8', 'ip_addr': ''}, {'hostname': 'hostname200', 'ip_addr': 'xxx.xx.xxx.200'}, {'hostname': 'hostname300', 'ip_addr': 'xxx.xx.xxx.400'}, ) trying to get the following difference from the above dictionary 1). compare the value of 'ip' in output1 dictionary with either 'hostname' and 'ip_addr' output2 dictionary and print their intersection. Tried below code: for doc in output1: for row in output2: if((row["hostname"] == doc["ip"]) or (row["ip_addr"] == doc["ip"])): print doc["ip"],doc["count"] *output:* hostname1 212 hostname2 27 hostname10 1 hostname8 2 xxx.xx.xxx.11 3 xxx.xx.xxx.12 90 xxx.xx.xxx.13 12 xxx.xx.xxx.14 21 xxx.xx.xxx.15 54 xxx.xx.xxx.16 34 2). need to print the below output if the value of 'ip' in output1 dictionary is not there in in output2 dictionary(ip/hostname which is there in output1 and not there in output2): xxx.xx.xxx.1 3 xxx.xx.xxx.2 4 xxx.xx.xxx.3 8 xxx.xx.xxx.4 10 hostname3 513 hostname4 98 xxx.xx.xxx.17 11 xxx.xx.xxx.18 2 xxx.xx.xxx.19 19 xxx.xx.xxx.20 21 xxx.xx.xxx.21 25 xxx.xx.xxx.22 31 xxx.xx.xxx.23 43 xxx.xx.xxx.24 46 xxx.xx.xxx.25 80 xxx.xx.xxx.26 91 xxx.xx.xxx.27 90 xxx.xx.xxx.28 10 xxx.xx.xxx.29 3 3). Ip address with is there only in output2 dictionary. xxx.xx.xxx.200 xxx.xx.xxx.400 Any help would be really appreciated. Thank you Thanks Mohan L -------------- next part -------------- An HTML attachment was scrubbed... URL: From python at mrabarnett.plus.com Thu Oct 3 14:44:27 2013 From: python at mrabarnett.plus.com (MRAB) Date: Thu, 03 Oct 2013 19:44:27 +0100 Subject: compare two list of dictionaries In-Reply-To: References: Message-ID: <524DBB0B.3080008@mrabarnett.plus.com> On 03/10/2013 17:11, Mohan L wrote: > Dear All, > > I have two list of dictionaries like below: > > In the below dictionaries the value of ip can be either hostname or ip > address. > > output1=[ > {'count': 3 , 'ip': 'xxx.xx.xxx.1'}, > {'count': 4, 'ip': 'xxx.xx.xxx.2'}, > {'count': 8, 'ip': 'xxx.xx.xxx.3'}, > {'count': 10, 'ip': 'xxx.xx.xxx.4'}, > {'count': 212, 'ip': 'hostname1'}, > {'count': 27, 'ip': 'hostname2'}, > {'count': 513, 'ip': 'hostname3'}, > {'count': 98, 'ip': 'hostname4'}, > {'count': 1, 'ip': 'hostname10'}, > {'count': 2, 'ip': 'hostname8'}, > {'count': 3, 'ip': 'xxx.xx.xxx.11'}, > {'count': 90, 'ip': 'xxx.xx.xxx.12'}, > {'count': 12, 'ip': 'xxx.xx.xxx.13'}, > {'count': 21, 'ip': 'xxx.xx.xxx.14'}, > {'count': 54, 'ip': 'xxx.xx.xxx.15'}, > {'count': 34, 'ip': 'xxx.xx.xxx.16'}, > {'count': 11, 'ip': 'xxx.xx.xxx.17'}, > {'count': 2, 'ip': 'xxx.xx.xxx.18'}, > {'count': 19, 'ip': 'xxx.xx.xxx.19'}, > {'count': 21, 'ip': 'xxx.xx.xxx.20'}, > {'count': 25, 'ip': 'xxx.xx.xxx.21'}, > {'count': 31, 'ip': 'xxx.xx.xxx.22'}, > {'count': 43, 'ip': 'xxx.xx.xxx.23'}, > {'count': 46, 'ip': 'xxx.xx.xxx.24'}, > {'count': 80, 'ip': 'xxx.xx.xxx.25'}, > {'count': 91, 'ip': 'xxx.xx.xxx.26'}, > {'count': 90, 'ip': 'xxx.xx.xxx.27'}, > {'count': 10, 'ip': 'xxx.xx.xxx.28'}, > {'count': 3, 'ip': 'xxx.xx.xxx.29'}] > > > In the below dictionaries have either hostname or ip or both. > > output2=( > > {'hostname': 'INNCHN01', 'ip_addr': 'xxx.xx.xxx.11'}, > {'hostname': 'HYDRHC02', 'ip_addr': 'xxx.xx.xxx.12'}, > {'hostname': 'INNCHN03', 'ip_addr': 'xxx.xx.xxx.13'}, > {'hostname': 'MUMRHC01', 'ip_addr': 'xxx.xx.xxx.14'}, > {'hostname': 'n/a', 'ip_addr': 'xxx.xx.xxx.15'}, > {'hostname': 'INNCHN05', 'ip_addr': 'xxx.xx.xxx.16'}, > {'hostname': 'hostname1', 'ip_addr': 'n/a'}, > {'hostname': 'hostname2', 'ip_addr': 'n/a'}, > {'hostname': 'hostname10', 'ip_addr': ''}, > {'hostname': 'hostname8', 'ip_addr': ''}, > {'hostname': 'hostname200', 'ip_addr': 'xxx.xx.xxx.200'}, > {'hostname': 'hostname300', 'ip_addr': 'xxx.xx.xxx.400'}, > > ) > > trying to get the following difference from the above dictionary > > 1). compare the value of 'ip' in output1 dictionary with either > 'hostname' and 'ip_addr' output2 dictionary and print their > intersection. Tried below code: > > > for doc in output1: > for row in output2: > if((row["hostname"] == doc["ip"]) or (row["ip_addr"] == > doc["ip"])): > print doc["ip"],doc["count"] > > *output:* > hostname1 212 > hostname2 27 > hostname10 1 > hostname8 2 > xxx.xx.xxx.11 3 > xxx.xx.xxx.12 90 > xxx.xx.xxx.13 12 > xxx.xx.xxx.14 21 > xxx.xx.xxx.15 54 > xxx.xx.xxx.16 34 > 1. Create a dict from output1 in which the key is the ip and the value is the count. 2. Create a set from output2 containing all the hostnames and ip_addrs. 3. Get the intersection of the keys of the dict with the set. 4. Print the entries of the dict for each member of the intersection. > 2). need to print the below output if the value of 'ip' in output1 > dictionary is not there in in output2 dictionary(ip/hostname which is > there in output1 and not there in output2): > > xxx.xx.xxx.1 3 > xxx.xx.xxx.2 4 > xxx.xx.xxx.3 8 > xxx.xx.xxx.4 10 > hostname3 513 > hostname4 98 > xxx.xx.xxx.17 11 > xxx.xx.xxx.18 2 > xxx.xx.xxx.19 19 > xxx.xx.xxx.20 21 > xxx.xx.xxx.21 25 > xxx.xx.xxx.22 31 > xxx.xx.xxx.23 43 > xxx.xx.xxx.24 46 > xxx.xx.xxx.25 80 > xxx.xx.xxx.26 91 > xxx.xx.xxx.27 90 > xxx.xx.xxx.28 10 > xxx.xx.xxx.29 3 > 1. Get the difference between the keys of the dict and the intersection. 2. Print the entries of the dict for each member of the difference. > 3). Ip address with is there only in output2 dictionary. > > xxx.xx.xxx.200 > xxx.xx.xxx.400 > 1. Create a set from output2 containing all the ip_addrs. 2. Get the difference between the set and the keys of the dict created from output1. > Any help would be really appreciated. Thank you > From l.mohanphy at gmail.com Thu Oct 3 22:29:26 2013 From: l.mohanphy at gmail.com (Mohan L) Date: Fri, 4 Oct 2013 07:59:26 +0530 Subject: compare two list of dictionaries In-Reply-To: <524DBB0B.3080008@mrabarnett.plus.com> References: <524DBB0B.3080008@mrabarnett.plus.com> Message-ID: On Fri, Oct 4, 2013 at 12:14 AM, MRAB wrote: > On 03/10/2013 17:11, Mohan L wrote: > >> Dear All, >> >> I have two list of dictionaries like below: >> >> In the below dictionaries the value of ip can be either hostname or ip >> address. >> >> output1=[ >> {'count': 3 , 'ip': 'xxx.xx.xxx.1'}, >> {'count': 4, 'ip': 'xxx.xx.xxx.2'}, >> {'count': 8, 'ip': 'xxx.xx.xxx.3'}, >> {'count': 10, 'ip': 'xxx.xx.xxx.4'}, >> {'count': 212, 'ip': 'hostname1'}, >> {'count': 27, 'ip': 'hostname2'}, >> {'count': 513, 'ip': 'hostname3'}, >> {'count': 98, 'ip': 'hostname4'}, >> {'count': 1, 'ip': 'hostname10'}, >> {'count': 2, 'ip': 'hostname8'}, >> {'count': 3, 'ip': 'xxx.xx.xxx.11'}, >> {'count': 90, 'ip': 'xxx.xx.xxx.12'}, >> {'count': 12, 'ip': 'xxx.xx.xxx.13'}, >> {'count': 21, 'ip': 'xxx.xx.xxx.14'}, >> {'count': 54, 'ip': 'xxx.xx.xxx.15'}, >> {'count': 34, 'ip': 'xxx.xx.xxx.16'}, >> {'count': 11, 'ip': 'xxx.xx.xxx.17'}, >> {'count': 2, 'ip': 'xxx.xx.xxx.18'}, >> {'count': 19, 'ip': 'xxx.xx.xxx.19'}, >> {'count': 21, 'ip': 'xxx.xx.xxx.20'}, >> {'count': 25, 'ip': 'xxx.xx.xxx.21'}, >> {'count': 31, 'ip': 'xxx.xx.xxx.22'}, >> {'count': 43, 'ip': 'xxx.xx.xxx.23'}, >> {'count': 46, 'ip': 'xxx.xx.xxx.24'}, >> {'count': 80, 'ip': 'xxx.xx.xxx.25'}, >> {'count': 91, 'ip': 'xxx.xx.xxx.26'}, >> {'count': 90, 'ip': 'xxx.xx.xxx.27'}, >> {'count': 10, 'ip': 'xxx.xx.xxx.28'}, >> {'count': 3, 'ip': 'xxx.xx.xxx.29'}] >> >> >> In the below dictionaries have either hostname or ip or both. >> >> output2=( >> >> {'hostname': 'INNCHN01', 'ip_addr': 'xxx.xx.xxx.11'}, >> {'hostname': 'HYDRHC02', 'ip_addr': 'xxx.xx.xxx.12'}, >> {'hostname': 'INNCHN03', 'ip_addr': 'xxx.xx.xxx.13'}, >> {'hostname': 'MUMRHC01', 'ip_addr': 'xxx.xx.xxx.14'}, >> {'hostname': 'n/a', 'ip_addr': 'xxx.xx.xxx.15'}, >> {'hostname': 'INNCHN05', 'ip_addr': 'xxx.xx.xxx.16'}, >> {'hostname': 'hostname1', 'ip_addr': 'n/a'}, >> {'hostname': 'hostname2', 'ip_addr': 'n/a'}, >> {'hostname': 'hostname10', 'ip_addr': ''}, >> {'hostname': 'hostname8', 'ip_addr': ''}, >> {'hostname': 'hostname200', 'ip_addr': 'xxx.xx.xxx.200'}, >> {'hostname': 'hostname300', 'ip_addr': 'xxx.xx.xxx.400'}, >> >> ) >> >> trying to get the following difference from the above dictionary >> >> 1). compare the value of 'ip' in output1 dictionary with either >> 'hostname' and 'ip_addr' output2 dictionary and print their >> intersection. Tried below code: >> >> >> for doc in output1: >> for row in output2: >> if((row["hostname"] == doc["ip"]) or (row["ip_addr"] == >> doc["ip"])): >> print doc["ip"],doc["count"] >> >> *output:* >> >> hostname1 212 >> hostname2 27 >> hostname10 1 >> hostname8 2 >> xxx.xx.xxx.11 3 >> xxx.xx.xxx.12 90 >> xxx.xx.xxx.13 12 >> xxx.xx.xxx.14 21 >> xxx.xx.xxx.15 54 >> xxx.xx.xxx.16 34 >> >> 1. Create a dict from output1 in which the key is the ip and the value > is the count. > > 2. Create a set from output2 containing all the hostnames and ip_addrs. > > 3. Get the intersection of the keys of the dict with the set. > > 4. Print the entries of the dict for each member of the intersection. > > > 2). need to print the below output if the value of 'ip' in output1 >> dictionary is not there in in output2 dictionary(ip/hostname which is >> there in output1 and not there in output2): >> >> xxx.xx.xxx.1 3 >> xxx.xx.xxx.2 4 >> xxx.xx.xxx.3 8 >> xxx.xx.xxx.4 10 >> hostname3 513 >> hostname4 98 >> xxx.xx.xxx.17 11 >> xxx.xx.xxx.18 2 >> xxx.xx.xxx.19 19 >> xxx.xx.xxx.20 21 >> xxx.xx.xxx.21 25 >> xxx.xx.xxx.22 31 >> xxx.xx.xxx.23 43 >> xxx.xx.xxx.24 46 >> xxx.xx.xxx.25 80 >> xxx.xx.xxx.26 91 >> xxx.xx.xxx.27 90 >> xxx.xx.xxx.28 10 >> xxx.xx.xxx.29 3 >> >> 1. Get the difference between the keys of the dict and the intersection. > > 2. Print the entries of the dict for each member of the difference. #!/bin/env python import sys output1=[ {'count': 3 , 'ip': 'xxx.xx.xxx.1'}, {'count': 4, 'ip': 'xxx.xx.xxx.2'}, {'count': 8, 'ip': 'xxx.xx.xxx.3'}, {'count': 10, 'ip': 'xxx.xx.xxx.4'}, {'count': 212, 'ip': 'hostname1'}, {'count': 27, 'ip': 'hostname2'}, {'count': 513, 'ip': 'hostname3'}, {'count': 98, 'ip': 'hostname4'}, {'count': 1, 'ip': 'hostname10'}, {'count': 2, 'ip': 'hostname8'}, {'count': 3, 'ip': 'xxx.xx.xxx.11'}, {'count': 90, 'ip': 'xxx.xx.xxx.12'}, {'count': 12, 'ip': 'xxx.xx.xxx.13'}, {'count': 21, 'ip': 'xxx.xx.xxx.14'}, {'count': 54, 'ip': 'xxx.xx.xxx.15'}, {'count': 34, 'ip': 'xxx.xx.xxx.16'}, {'count': 11, 'ip': 'xxx.xx.xxx.17'}, {'count': 2, 'ip': 'xxx.xx.xxx.18'}, {'count': 19, 'ip': 'xxx.xx.xxx.19'}, {'count': 21, 'ip': 'xxx.xx.xxx.20'}, {'count': 25, 'ip': 'xxx.xx.xxx.21'}, {'count': 31, 'ip': 'xxx.xx.xxx.22'}, {'count': 43, 'ip': 'xxx.xx.xxx.23'}, {'count': 46, 'ip': 'xxx.xx.xxx.24'}, {'count': 80, 'ip': 'xxx.xx.xxx.25'}, {'count': 91, 'ip': 'xxx.xx.xxx.26'}, {'count': 90, 'ip': 'xxx.xx.xxx.27'}, {'count': 10, 'ip': 'xxx.xx.xxx.28'}, {'count': 3, 'ip': 'xxx.xx.xxx.29'}] output2=(('INNCHN01','xxx.xx.xxx.11'), ('HYDRHC02', 'xxx.xx.xxx.12'), ('INNCHN03','xxx.xx.xxx.13'), ('MUMRHC01','xxx.xx.xxx.14'), ('n/a','xxx.xx.xxx.15'), ('INNCHN05','xxx.xx.xxx.16'), ('hostname1','n/a'), ('hostname2','n/a'), ('hostname10',''), ('hostname8',''), ('hostname200','xxx.xx.xxx.200'), ('hostname300','xxx.xx.xxx.400'), ) ## 1). ## Create a dict from output1 in which the key is the ip and the value is the count. mongodb_data={} for doc in output1: mongodb_data.update({doc['ip']:doc['count']}) ## Create a set from output2 containing all the hostnames and ip_addrs. all_hostname_ip_set=set(list(sum(output2, ()))) ## Get the intersection of the keys of the dict with the set. key_set=set(mongodb_data.keys()) int_keys=key_set & all_hostname_ip_set # Print the entries of the dict for each member of the intersection. print "-------------------intersection-----------------------------" for key in int_keys: print key,mongodb_data[key] ## 2). ## Get the difference between the keys of the dict and the intersection. deff_keys=key_set - all_hostname_ip_set ## Print the entries of the dict for each member of the difference. print "-------------------difference-------------------------------" for key in deff_keys: print key,mongodb_data[key] $ ./demo.py -------------------intersection----------------------------- xxx.xx.xxx.11 3 xxx.xx.xxx.12 90 xxx.xx.xxx.13 12 xxx.xx.xxx.14 21 xxx.xx.xxx.15 54 xxx.xx.xxx.16 34 hostname2 27 hostname1 212 hostname10 1 hostname8 2 -------------------difference------------------------------- xxx.xx.xxx.29 3 xxx.xx.xxx.28 10 xxx.xx.xxx.17 11 xxx.xx.xxx.18 2 xxx.xx.xxx.19 19 xxx.xx.xxx.23 43 xxx.xx.xxx.22 31 xxx.xx.xxx.25 80 xxx.xx.xxx.24 46 xxx.xx.xxx.27 90 xxx.xx.xxx.26 91 xxx.xx.xxx.2 4 xxx.xx.xxx.21 25 hostname3 513 hostname4 98 xxx.xx.xxx.4 10 xxx.xx.xxx.20 21 xxx.xx.xxx.1 3 xxx.xx.xxx.3 8 > > > 3). Ip address with is there only in output2 dictionary. >> >> xxx.xx.xxx.200 >> xxx.xx.xxx.400 >> >> 1. Create a set from output2 containing all the ip_addrs > 2. Get the difference between the set and the keys of the dict created > from output1. I have one problem here. I want to compare the output2 (hostname or ip address) with output1 'ip' values (i.e., either hostname or ip address of output2 does not matching the ip dict key in output1 then I want to print either hostname or ip address from output2). Any help? Thanks for your time. Thanks Mohan L -------------- next part -------------- An HTML attachment was scrubbed... URL: From python at mrabarnett.plus.com Fri Oct 4 12:41:36 2013 From: python at mrabarnett.plus.com (MRAB) Date: Fri, 04 Oct 2013 17:41:36 +0100 Subject: compare two list of dictionaries In-Reply-To: References: <524DBB0B.3080008@mrabarnett.plus.com> Message-ID: <524EEFC0.9050407@mrabarnett.plus.com> On 04/10/2013 03:29, Mohan L wrote: [snip] > > output1=[ > {'count': 3 , 'ip': 'xxx.xx.xxx.1'}, > {'count': 4, 'ip': 'xxx.xx.xxx.2'}, > {'count': 8, 'ip': 'xxx.xx.xxx.3'}, > {'count': 10, 'ip': 'xxx.xx.xxx.4'}, > {'count': 212, 'ip': 'hostname1'}, > {'count': 27, 'ip': 'hostname2'}, > {'count': 513, 'ip': 'hostname3'}, > {'count': 98, 'ip': 'hostname4'}, > {'count': 1, 'ip': 'hostname10'}, > {'count': 2, 'ip': 'hostname8'}, > {'count': 3, 'ip': 'xxx.xx.xxx.11'}, > {'count': 90, 'ip': 'xxx.xx.xxx.12'}, > {'count': 12, 'ip': 'xxx.xx.xxx.13'}, > {'count': 21, 'ip': 'xxx.xx.xxx.14'}, > {'count': 54, 'ip': 'xxx.xx.xxx.15'}, > {'count': 34, 'ip': 'xxx.xx.xxx.16'}, > {'count': 11, 'ip': 'xxx.xx.xxx.17'}, > {'count': 2, 'ip': 'xxx.xx.xxx.18'}, > {'count': 19, 'ip': 'xxx.xx.xxx.19'}, > {'count': 21, 'ip': 'xxx.xx.xxx.20'}, > {'count': 25, 'ip': 'xxx.xx.xxx.21'}, > {'count': 31, 'ip': 'xxx.xx.xxx.22'}, > {'count': 43, 'ip': 'xxx.xx.xxx.23'}, > {'count': 46, 'ip': 'xxx.xx.xxx.24'}, > {'count': 80, 'ip': 'xxx.xx.xxx.25'}, > {'count': 91, 'ip': 'xxx.xx.xxx.26'}, > {'count': 90, 'ip': 'xxx.xx.xxx.27'}, > {'count': 10, 'ip': 'xxx.xx.xxx.28'}, > {'count': 3, 'ip': 'xxx.xx.xxx.29'}] > > > > output2=(('INNCHN01','xxx.xx.xxx.11'), > ('HYDRHC02', 'xxx.xx.xxx.12'), > ('INNCHN03','xxx.xx.xxx.13'), > ('MUMRHC01','xxx.xx.xxx.14'), > ('n/a','xxx.xx.xxx.15'), > ('INNCHN05','xxx.xx.xxx.16'), > ('hostname1','n/a'), > ('hostname2','n/a'), > ('hostname10',''), > ('hostname8',''), > ('hostname200','xxx.xx.xxx.200'), > ('hostname300','xxx.xx.xxx.400'), > ) > > ## 1). > ## Create a dict from output1 in which the key is the ip and the value > is the count. > mongodb_data={} > for doc in output1: > mongodb_data.update({doc['ip']:doc['count']}) > A simpler way is: mongodb_data = {} for doc in output1: mongodb_data[doc['ip']] = doc['count'] > ## Create a set from output2 containing all the hostnames and ip_addrs. > all_hostname_ip_set=set(list(sum(output2, ()))) > > ## Get the intersection of the keys of the dict with the set. > key_set=set(mongodb_data.keys()) > int_keys=key_set & all_hostname_ip_set > # Print the entries of the dict for each member of the intersection. > print "-------------------intersection-----------------------------" > for key in int_keys: > print key,mongodb_data[key] > > ## 2). > ## Get the difference between the keys of the dict and the intersection. > deff_keys=key_set - all_hostname_ip_set > ## Print the entries of the dict for each member of the difference. > print "-------------------difference-------------------------------" > for key in deff_keys: > print key,mongodb_data[key] > > > $ ./demo.py > > -------------------intersection----------------------------- > xxx.xx.xxx.11 3 > xxx.xx.xxx.12 90 > xxx.xx.xxx.13 12 > xxx.xx.xxx.14 21 > xxx.xx.xxx.15 54 > xxx.xx.xxx.16 34 > hostname2 27 > hostname1 212 > hostname10 1 > hostname8 2 > -------------------difference------------------------------- > xxx.xx.xxx.29 3 > xxx.xx.xxx.28 10 > xxx.xx.xxx.17 11 > xxx.xx.xxx.18 2 > xxx.xx.xxx.19 19 > xxx.xx.xxx.23 43 > xxx.xx.xxx.22 31 > xxx.xx.xxx.25 80 > xxx.xx.xxx.24 46 > xxx.xx.xxx.27 90 > xxx.xx.xxx.26 91 > xxx.xx.xxx.2 4 > xxx.xx.xxx.21 25 > hostname3 513 > hostname4 98 > xxx.xx.xxx.4 10 > xxx.xx.xxx.20 21 > xxx.xx.xxx.1 3 > xxx.xx.xxx.3 8 > > > > 3). Ip address with is there only in output2 dictionary. > > xxx.xx.xxx.200 > xxx.xx.xxx.400 > > 1. Create a set from output2 containing all the ip_addrs > > > 2. Get the difference between the set and the keys of the dict created > from output1. > > > I have one problem here. I want to compare the output2 (hostname or ip > address) with output1 'ip' values (i.e., either hostname or ip address > of output2 does not matching the ip dict key in output1 then I want to > print either hostname or ip address from output2). > You already have the output1 'ip' values (the keys of mongodb_data) and the entries of output2 (all_hostname_ip_set). Just find the set difference. From toby at tobiah.org Fri Oct 4 15:22:28 2013 From: toby at tobiah.org (Tobiah) Date: Fri, 04 Oct 2013 12:22:28 -0700 Subject: compare two list of dictionaries In-Reply-To: References: Message-ID: <524F1574.4030900@tobiah.org> On 10/03/2013 09:11 AM, Mohan L wrote: > Dear All, > > I have two list of dictionaries like below: > > In the below dictionaries the value of ip can be either hostname or ip address. > > output1=[ > {'count': 3 , 'ip': 'xxx.xx.xxx.1'}, > {'count': 4, 'ip': 'xxx.xx.xxx.2'}, > {'count': 8, 'ip': 'xxx.xx.xxx.3'}, > {'count': 10, 'ip': 'xxx.xx.xxx.4'}, > {'count': 212, 'ip': 'hostname1'}, > {'count': 27, 'ip': 'hostname2'}, > {'count': 513, 'ip': 'hostname3'}, > {'count': 98, 'ip': 'hostname4'}, > {'count': 1, 'ip': 'hostname10'}, > {'count': 2, 'ip': 'hostname8'}, > {'count': 3, 'ip': 'xxx.xx.xxx.11'}, > {'count': 90, 'ip': 'xxx.xx.xxx.12'}, > {'count': 12, 'ip': 'xxx.xx.xxx.13'}, > {'count': 21, 'ip': 'xxx.xx.xxx.14'}, > {'count': 54, 'ip': 'xxx.xx.xxx.15'}, > {'count': 34, 'ip': 'xxx.xx.xxx.16'}, > {'count': 11, 'ip': 'xxx.xx.xxx.17'}, > {'count': 2, 'ip': 'xxx.xx.xxx.18'}, > {'count': 19, 'ip': 'xxx.xx.xxx.19'}, > {'count': 21, 'ip': 'xxx.xx.xxx.20'}, > {'count': 25, 'ip': 'xxx.xx.xxx.21'}, > {'count': 31, 'ip': 'xxx.xx.xxx.22'}, > {'count': 43, 'ip': 'xxx.xx.xxx.23'}, > {'count': 46, 'ip': 'xxx.xx.xxx.24'}, > {'count': 80, 'ip': 'xxx.xx.xxx.25'}, > {'count': 91, 'ip': 'xxx.xx.xxx.26'}, > {'count': 90, 'ip': 'xxx.xx.xxx.27'}, > {'count': 10, 'ip': 'xxx.xx.xxx.28'}, > {'count': 3, 'ip': 'xxx.xx.xxx.29'}] > > > In the below dictionaries have either hostname or ip or both. > > output2=( > > {'hostname': 'INNCHN01', 'ip_addr': 'xxx.xx.xxx.11'}, > {'hostname': 'HYDRHC02', 'ip_addr': 'xxx.xx.xxx.12'}, > {'hostname': 'INNCHN03', 'ip_addr': 'xxx.xx.xxx.13'}, > {'hostname': 'MUMRHC01', 'ip_addr': 'xxx.xx.xxx.14'}, > {'hostname': 'n/a', 'ip_addr': 'xxx.xx.xxx.15'}, > {'hostname': 'INNCHN05', 'ip_addr': 'xxx.xx.xxx.16'}, > {'hostname': 'hostname1', 'ip_addr': 'n/a'}, > {'hostname': 'hostname2', 'ip_addr': 'n/a'}, > {'hostname': 'hostname10', 'ip_addr': ''}, > {'hostname': 'hostname8', 'ip_addr': ''}, > {'hostname': 'hostname200', 'ip_addr': 'xxx.xx.xxx.200'}, > {'hostname': 'hostname300', 'ip_addr': 'xxx.xx.xxx.400'}, > > ) > > trying to get the following difference from the above dictionary > > 1). compare the value of 'ip' in output1 dictionary with either 'hostname' and 'ip_addr' output2 dictionary and print their > intersection. Tried below code: > > > for doc in output1: > for row in output2: > if((row["hostname"] == doc["ip"]) or (row["ip_addr"] == doc["ip"])): > print doc["ip"],doc["count"] > > *output:* > hostname1 212 > hostname2 27 > hostname10 1 > hostname8 2 > xxx.xx.xxx.11 3 > xxx.xx.xxx.12 90 > xxx.xx.xxx.13 12 > xxx.xx.xxx.14 21 > xxx.xx.xxx.15 54 > xxx.xx.xxx.16 34 > > 2). need to print the below output if the value of 'ip' in output1 dictionary is not there in in output2 dictionary(ip/hostname > which is there in output1 and not there in output2): > > xxx.xx.xxx.1 3 > xxx.xx.xxx.2 4 > xxx.xx.xxx.3 8 > xxx.xx.xxx.4 10 > hostname3 513 > hostname4 98 > xxx.xx.xxx.17 11 > xxx.xx.xxx.18 2 > xxx.xx.xxx.19 19 > xxx.xx.xxx.20 21 > xxx.xx.xxx.21 25 > xxx.xx.xxx.22 31 > xxx.xx.xxx.23 43 > xxx.xx.xxx.24 46 > xxx.xx.xxx.25 80 > xxx.xx.xxx.26 91 > xxx.xx.xxx.27 90 > xxx.xx.xxx.28 10 > xxx.xx.xxx.29 3 > > 3). Ip address with is there only in output2 dictionary. > > xxx.xx.xxx.200 > xxx.xx.xxx.400 > > Any help would be really appreciated. Thank you > > Thanks > Mohan L There is a bit of work that can be consolidated to help both case 2 and 3. Then both cases are a little more straightforward: ################################################## import itertools inside_out = {x['ip']: x for x in output1} set1 = set(inside_out.keys()) ### CASE 2 ### set2 = set(itertools.chain.from_iterable([x.values() for x in output2])) for name in set1 - set2: print name, inside_out[name]['count'] ### CASE 3 ### set2 = set([x['ip_addr'] for x in output2]) print "\n".join(set2 - set1) From tester.testerus at gmail.com Thu Oct 3 12:12:48 2013 From: tester.testerus at gmail.com (macker) Date: Thu, 3 Oct 2013 09:12:48 -0700 (PDT) Subject: feature requests Message-ID: <6782f295-1885-4114-aea8-d785480f3489@googlegroups.com> Hi, hope this is the right group for this: I miss two basic (IMO) features in parallel processing: 1. make `threading.Thread.start()` return `self` I'd like to be able to `workers = [Thread(params).start() for params in whatever]`. Right now, it's 5 ugly, menial lines: workers = [] for params in whatever: thread = threading.Thread(params) thread.start() workers.append(thread) 2. make multiprocessing pools (incl. ThreadPool) limit the size of their internal queues As it is now, the queue will greedily consume its entire input, and if the input is large and the pool workers are slow in consuming it, this blows up RAM. I'd like to be able to `pool = Pool(4, max_qsize=1000)`. Same with the output queue (finished tasks). Or does anyone know of a way to achieve this? From rosuav at gmail.com Thu Oct 3 12:21:13 2013 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 4 Oct 2013 02:21:13 +1000 Subject: feature requests In-Reply-To: <6782f295-1885-4114-aea8-d785480f3489@googlegroups.com> References: <6782f295-1885-4114-aea8-d785480f3489@googlegroups.com> Message-ID: On Fri, Oct 4, 2013 at 2:12 AM, macker wrote: > I'd like to be able to `workers = [Thread(params).start() for params in whatever]`. Right now, it's 5 ugly, menial lines: > > workers = [] > for params in whatever: > thread = threading.Thread(params) > thread.start() > workers.append(thread) You could shorten this by iterating twice, if that helps: workers = [Thread(params).start() for params in whatever] for thrd in workers: thrd.start() ChrisA From python.list at tim.thechases.com Thu Oct 3 12:42:05 2013 From: python.list at tim.thechases.com (Tim Chase) Date: Thu, 3 Oct 2013 11:42:05 -0500 Subject: feature requests In-Reply-To: References: <6782f295-1885-4114-aea8-d785480f3489@googlegroups.com> Message-ID: <20131003114205.7f2cff96@bigbox.christie.dr> On 2013-10-04 02:21, Chris Angelico wrote: > > workers = [] > > for params in whatever: > > thread = threading.Thread(params) > > thread.start() > > workers.append(thread) > > You could shorten this by iterating twice, if that helps: > > workers = [Thread(params).start() for params in whatever] > for thrd in workers: thrd.start() Do you mean workers = [Thread(params) for params in whatever] for thrd in workers: thrd.start() ? ("Thread(params)" vs. "Thread(params).start()" in your list comp) -tkc From rosuav at gmail.com Thu Oct 3 12:42:44 2013 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 4 Oct 2013 02:42:44 +1000 Subject: feature requests In-Reply-To: <20131003114205.7f2cff96@bigbox.christie.dr> References: <6782f295-1885-4114-aea8-d785480f3489@googlegroups.com> <20131003114205.7f2cff96@bigbox.christie.dr> Message-ID: On Fri, Oct 4, 2013 at 2:42 AM, Tim Chase wrote: > Do you mean > > workers = [Thread(params) for params in whatever] > for thrd in workers: thrd.start() > > ? ("Thread(params)" vs. "Thread(params).start()" in your list comp) Whoops, copy/paste fail. Yes, that's what I meant. Thanks for catching! ChrisA From ethan at stoneleaf.us Thu Oct 3 13:01:02 2013 From: ethan at stoneleaf.us (Ethan Furman) Date: Thu, 03 Oct 2013 10:01:02 -0700 Subject: feature requests In-Reply-To: <6782f295-1885-4114-aea8-d785480f3489@googlegroups.com> References: <6782f295-1885-4114-aea8-d785480f3489@googlegroups.com> Message-ID: <524DA2CE.603@stoneleaf.us> On 10/03/2013 09:12 AM, macker wrote: > Hi, hope this is the right group for this: > > I miss two basic (IMO) features in parallel processing: > > 1. make `threading.Thread.start()` return `self` > > I'd like to be able to `workers = [Thread(params).start() for params in whatever]`. Right now, it's 5 ugly, menial lines: > > workers = [] > for params in whatever: > thread = threading.Thread(params) > thread.start() > workers.append(thread) Ugly, menial lines are a clue that a function to hide it could be useful. > 2. make multiprocessing pools (incl. ThreadPool) limit the size of their internal queues > > As it is now, the queue will greedily consume its entire input, and if the input is large and the pool workers are slow in consuming it, this blows up RAM. I'd like to be able to `pool = Pool(4, max_qsize=1000)`. Same with the output queue (finished tasks). Have you verified that this is a problem in Python? > Or does anyone know of a way to achieve this? You could try subclassing. -- ~Ethan~ From tester.testerus at gmail.com Sat Oct 5 08:49:33 2013 From: tester.testerus at gmail.com (macker) Date: Sat, 5 Oct 2013 05:49:33 -0700 (PDT) Subject: feature requests In-Reply-To: References: <6782f295-1885-4114-aea8-d785480f3489@googlegroups.com> Message-ID: > > Ugly, menial lines are a clue that a function to hide it could be useful. Or a clue to add a trivial change elsewhere (hint for Ethan: `return self` at the end of `Thread.start()`). > Have you verified that this is a problem in Python? ? > You could try subclassing. I could try many things. What this thread is about is trying to fix it on stdlib level, so that people don't have to reinvent the wheel every time. Thanks to Chris for his suggestion. Ethan, please stay away from this thread. -macker > > > > -- > > ~Ethan~ From ethan at stoneleaf.us Sat Oct 5 11:58:02 2013 From: ethan at stoneleaf.us (Ethan Furman) Date: Sat, 05 Oct 2013 08:58:02 -0700 Subject: feature requests In-Reply-To: References: <6782f295-1885-4114-aea8-d785480f3489@googlegroups.com> Message-ID: <5250370A.3010103@stoneleaf.us> On 10/05/2013 05:49 AM, macker wrote: >> >> Ugly, menial lines are a clue that a function to hide it could be useful. > > Or a clue to add a trivial change elsewhere (hint for Ethan: `return self` at the end of `Thread.start()`). I'm aware that would solve your issue. I'm also aware that Python rarely does a 'return self' at the end of methods. Since that probably isn't going to change, a helper function is probably your best way forward. >> Have you verified that this is a problem in Python? > > ? You stated it "would blow up RAM" -- have you actually tested this, or are you making assumptions based on experience from other languages, or assumptions based on nothing at all? >> You could try subclassing. > > I could try many things. What this thread is about is trying to fix it on stdlib level, so that people don't have to reinvent the wheel every time. Did you really expect your idea to just sail through with no opposition, no counter-ideas, no reasons why it might not, or would not, work? > Thanks to Chris for his suggestion. Ethan, please stay away from this thread. Wow, you're rude. -- ~Ethan~ From tjreedy at udel.edu Sat Oct 5 17:56:03 2013 From: tjreedy at udel.edu (Terry Reedy) Date: Sat, 05 Oct 2013 17:56:03 -0400 Subject: feature requests In-Reply-To: <5250370A.3010103@stoneleaf.us> References: <6782f295-1885-4114-aea8-d785480f3489@googlegroups.com> <5250370A.3010103@stoneleaf.us> Message-ID: On 10/5/2013 11:58 AM, Ethan Furman wrote: > On 10/05/2013 05:49 AM, macker wrote: >>> >>> Ugly, menial lines are a clue that a function to hide it could be >>> useful. >> >> Or a clue to add a trivial change elsewhere (hint for Ethan: `return >> self` at the end of `Thread.start()`). > > I'm aware that would solve your issue. I'm also aware that Python > rarely does a 'return self' at the end of methods. Not returning self is a basic design principle of Python since its beginning. (I am not aware of any exceptions and would regard one as possibly a mistake.) Guido is aware that not doing so prevents chaining of mutation methods. He thinks it very important that people know and remember the difference between a method that mutates self and one that does not. Otherwise, one could write 'b = a.sort()' and not know (remember) that b is just an alias for a. He must have seen this type of error, especially in beginner code, in other languages before designing Python. > Since that probably isn't going to change, as it would only make things worse. Note that some mutation methods also return something useful other than default None. Examples are mylist.pop() and iterator.__next__ (usually accessed by next(iterator)*. So it is impossible for all mutation methods to just 'return self'. * iterator.__next__ is a generalized specialization of list.pop. It can only return the 'first' item, but can do so with any iterable, including those that are not ordered and those that represent virtual rather than concrete collections. -- Terry Jan Reedy From luolee.me at gmail.com Thu Oct 3 13:37:20 2013 From: luolee.me at gmail.com (=?UTF-8?B?5p2O5rSb?=) Date: Fri, 4 Oct 2013 01:37:20 +0800 Subject: Why didn't my threads exit correctly ? Message-ID: Hi list, I write an example script using threading as follow. It look like hang when the list l_ip is empty. And any suggestion with debug over the threading in Python ? 1 #!/usr/bin/env python 2 # -*- coding: utf-8 -*- 3 import re 4 import os 5 import threading 6 from Queue import Queue 7 from time import sleep 8 9 l_ip = [] 10 l_result = [] 11 result = re.compile(r"[1-3] received") 12 13 class ping(threading.Thread): 14 """ """ 15 def __init__(self, l_ip, l_result): 16 threading.Thread.__init__(self) 17 self.l_ip = l_ip 18 #self.l_result = l_result 19 20 def run(self): 21 """ """ 22 while True: 23 try: 24 ip = self.l_ip.pop() 25 except IndexError as e: 26 print e 27 break 28 ping_out = os.popen(''.join(['ping -q -c3 ',ip]), 'r') 29 print 'Ping ip:%s' % ip 30 while True: 31 line = ping_out.readline() 32 if not line: break 33 if result.findall(line): 34 l_result.append(ip) 35 break 36 37 queue = Queue() 38 39 for i in range(1,110): 40 l_ip.append(''.join(['192.168.1.', str(i)])) 41 for i in xrange(10): 42 t = ping(l_ip, l_result) 43 t.start() 44 queue.put(t) 45 queue.join() 46 print "Result will go here." 47 for i in l_result: 48 print 'IP %s is OK' % i -- All the best! http://luolee.me -------------- next part -------------- An HTML attachment was scrubbed... URL: From python at mrabarnett.plus.com Thu Oct 3 14:25:46 2013 From: python at mrabarnett.plus.com (MRAB) Date: Thu, 03 Oct 2013 19:25:46 +0100 Subject: Why didn't my threads exit correctly ? In-Reply-To: References: Message-ID: <524DB6AA.4080704@mrabarnett.plus.com> On 03/10/2013 18:37, ?? wrote: > Hi list, > I write an example script using threading as follow. > It look like hang when the list l_ip is empty. And any suggestion with > debug over the threading in Python ? > > 1 #!/usr/bin/env python > 2 # -*- coding: utf-8 -*- > 3 import re > 4 import os > 5 import threading > 6 from Queue import Queue > 7 from time import sleep > 8 > 9 l_ip = [] > 10 l_result = [] > 11 result = re.compile(r"[1-3] received") > 12 > 13 class ping(threading.Thread): > 14 """ """ > 15 def __init__(self, l_ip, l_result): > 16 threading.Thread.__init__(self) > 17 self.l_ip = l_ip > 18 #self.l_result = l_result > 19 > 20 def run(self): > 21 """ """ > 22 while True: > 23 try: > 24 ip = self.l_ip.pop() > 25 except IndexError as e: > 26 print e > 27 break > 28 ping_out = os.popen(''.join(['ping -q -c3 ',ip]), 'r') > 29 print 'Ping ip:%s' % ip > 30 while True: > 31 line = ping_out.readline() > 32 if not line: break > 33 if result.findall(line): > 34 l_result.append(ip) > 35 break > 36 > 37 queue = Queue() > 38 > 39 for i in range(1,110): > 40 l_ip.append(''.join(['192.168.1.', str(i)])) > 41 for i in xrange(10): > 42 t = ping(l_ip, l_result) > 43 t.start() > 44 queue.put(t) > 45 queue.join() > 46 print "Result will go here." > 47 for i in l_result: > 48 print 'IP %s is OK' % i > queue.join() will block until the queue is empty, which is never is! You're putting the workers in the queue, whereas the normal way of doing it is to put them into a list, the inputs into a queue, and the outputs into another queue. The workers then 'get' from the input queue, do some processing, and 'put' to the output queue. From luolee.me at gmail.com Fri Oct 4 12:41:13 2013 From: luolee.me at gmail.com (=?UTF-8?B?5p2O5rSb?=) Date: Sat, 5 Oct 2013 00:41:13 +0800 Subject: Why didn't my threads exit correctly ? In-Reply-To: <524DB6AA.4080704@mrabarnett.plus.com> References: <524DB6AA.4080704@mrabarnett.plus.com> Message-ID: Thanks for your reply, MRAB. I've seen that the child thread has been stoped, it just died in the queue.If I want the queue be empty, I must use queue.get() to dequeue and clean it, but I didn't do that. I've implement the thread using List now, thanks again. On Fri, Oct 4, 2013 at 2:25 AM, MRAB wrote: > On 03/10/2013 18:37, ?? wrote: > >> Hi list, >> I write an example script using threading as follow. >> It look like hang when the list l_ip is empty. And any suggestion with >> debug over the threading in Python ? >> >> 1 #!/usr/bin/env python >> 2 # -*- coding: utf-8 -*- >> 3 import re >> 4 import os >> 5 import threading >> 6 from Queue import Queue >> 7 from time import sleep >> 8 >> 9 l_ip = [] >> 10 l_result = [] >> 11 result = re.compile(r"[1-3] received") >> 12 >> 13 class ping(threading.Thread): >> 14 """ """ >> 15 def __init__(self, l_ip, l_result): >> 16 threading.Thread.__init__(**self) >> 17 self.l_ip = l_ip >> 18 #self.l_result = l_result >> 19 >> 20 def run(self): >> 21 """ """ >> 22 while True: >> 23 try: >> 24 ip = self.l_ip.pop() >> 25 except IndexError as e: >> 26 print e >> 27 break >> 28 ping_out = os.popen(''.join(['ping -q -c3 ',ip]), 'r') >> 29 print 'Ping ip:%s' % ip >> 30 while True: >> 31 line = ping_out.readline() >> 32 if not line: break >> 33 if result.findall(line): >> 34 l_result.append(ip) >> 35 break >> 36 >> 37 queue = Queue() >> 38 >> 39 for i in range(1,110): >> 40 l_ip.append(''.join(['192.168.**1.', str(i)])) >> 41 for i in xrange(10): >> 42 t = ping(l_ip, l_result) >> 43 t.start() >> 44 queue.put(t) >> 45 queue.join() >> 46 print "Result will go here." >> 47 for i in l_result: >> 48 print 'IP %s is OK' % i >> >> queue.join() will block until the queue is empty, which is never is! > > You're putting the workers in the queue, whereas the normal way of > doing it is to put them into a list, the inputs into a queue, and the > outputs into another queue. The workers then 'get' from the input > queue, do some processing, and 'put' to the output queue. > > -- > https://mail.python.org/**mailman/listinfo/python-list > -- All the best! http://luolee.me -------------- next part -------------- An HTML attachment was scrubbed... URL: From piet at vanoostrum.org Sat Oct 5 15:22:17 2013 From: piet at vanoostrum.org (Piet van Oostrum) Date: Sat, 05 Oct 2013 15:22:17 -0400 Subject: Why didn't my threads exit correctly ? References: <524DB6AA.4080704@mrabarnett.plus.com> Message-ID: ?? writes: > Thanks for your reply, MRAB. > > I've seen that the child thread has been stoped, it just died in the queue.If I want the queue be > empty, I must use queue.get() to dequeue and clean it, but I didn't do that. > I've implement the thread using List now, thanks again. Also make sure that l_ip and l_result are Queues instead of lists, as MRAB suggested. -- Piet van Oostrum WWW: http://pietvanoostrum.com/ PGP key: [8DAE142BE17999C4] From jshrager at gmail.com Thu Oct 3 13:42:39 2013 From: jshrager at gmail.com (jshrager at gmail.com) Date: Thu, 3 Oct 2013 10:42:39 -0700 (PDT) Subject: ipy %run noob confusion Message-ID: I have some rather complex code that works perfectly well if I paste it in by hand to ipython, but if I use %run it can't find some of the libraries, but others it can. The confusion seems to have to do with mathplotlib. I get it in stream by: %pylab osx and do a bunch of stuff interactively that works just fine, for example: clf() But I want it to run on a %run, but %pylab is (apparently) not allowed from a %run script, and importing matplotlib explicitly doesn't work...I mean, it imports, but then clf() is only defined in the module, not interactively. More confusing, if I do all the setup interactively, and the try to just run my script, again, clf() [etc] don't work (don't appear to exist), even though I can do them interactively. There seems to be some sort of scoping problem ... or, put more correctly, my problem is that I don't seem to understand the scoping, like, are %run eval'ed in some closed context that doesn't work the same way as ipython interactive? Is there any way to really do what I mean, which is: Please just read in commands from that script (short of getting out and passing my script through stdin to ipython?) Thanks! From tjreedy at udel.edu Thu Oct 3 15:26:18 2013 From: tjreedy at udel.edu (Terry Reedy) Date: Thu, 03 Oct 2013 15:26:18 -0400 Subject: ipy %run noob confusion In-Reply-To: References: Message-ID: On 10/3/2013 1:42 PM, jshrager at gmail.com wrote: > I have some rather complex code that works perfectly well if I paste it in by hand to ipython, but if I use %run it can't find some of the libraries, but others it can. Ipython is a separate product built on top of Python. If no answer here, look for an ipython-specific list or discussion group. -- Terry Jan Reedy From breamoreboy at yahoo.co.uk Thu Oct 3 15:34:40 2013 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Thu, 03 Oct 2013 20:34:40 +0100 Subject: ipy %run noob confusion In-Reply-To: References: Message-ID: On 03/10/2013 20:26, Terry Reedy wrote: > On 10/3/2013 1:42 PM, jshrager at gmail.com wrote: >> I have some rather complex code that works perfectly well if I paste >> it in by hand to ipython, but if I use %run it can't find some of the >> libraries, but others it can. > > Ipython is a separate product built on top of Python. If no answer here, > look for an ipython-specific list or discussion group. > Such as news.gmane.org/gmane.comp.python.ipython.user -- Roses are red, Violets are blue, Most poems rhyme, But this one doesn't. Mark Lawrence From oscar.j.benjamin at gmail.com Fri Oct 4 05:36:21 2013 From: oscar.j.benjamin at gmail.com (Oscar Benjamin) Date: Fri, 4 Oct 2013 10:36:21 +0100 Subject: ipy %run noob confusion In-Reply-To: References: Message-ID: On 3 October 2013 18:42, wrote: > I have some rather complex code that works perfectly well if I paste it in by hand to ipython, but if I use %run it can't find some of the libraries, but others it can. The confusion seems to have to do with mathplotlib. I get it in stream by: > > %pylab osx > > and do a bunch of stuff interactively that works just fine, for example: > > clf() > > But I want it to run on a %run, but %pylab is (apparently) not allowed from a %run script, and importing matplotlib explicitly doesn't work...I mean, it imports, but then clf() is only defined in the module, not interactively. The % commands are ipython magic commands. They are not valid Python syntax so you can't use them in a Python script. In a Python script you would use 'from pylab import *' for (roughly) the same effect: $ ipython Python 2.7.5 (default, May 15 2013, 22:43:36) [MSC v.1500 32 bit (Intel)] Type "copyright", "credits" or "license" for more information. IPython 0.13.2 -- An enhanced Interactive Python. ? -> Introduction and overview of IPython's features. %quickref -> Quick reference. help -> Python's own help system. object? -> Details about 'object', use 'object??' for extra details. In [1]: from pylab import * In [2]: clf Out[2]: > More confusing, if I do all the setup interactively, and the try to just run my script, again, clf() [etc] don't work (don't appear to exist), even though I can do them interactively. When you use %run it runs the script "externally". This is basically the same as typing 'python myscript.py' in the system terminal. In that case the script needs to import everything it wants to use. > There seems to be some sort of scoping problem ... or, put more correctly, my problem is that I don't seem to understand the scoping, like, are %run eval'ed in some closed context that doesn't work the same way as ipython interactive? Is there any way to really do what I mean, which is: Please just read in commands from that script (short of getting out and passing my script through stdin to ipython?) I'm not sure if I understand what you mean but I usually %edit the script and closing the editor seems to just run the commands as if I typed them directly in. If you really want this kind of semi-interactive Matlab-style approach I suggest having a look at the Spyder IDE. Personally though I think it's bad to work this way in Python (and in Matlab) and I discourage my students from doing this. The interactive interpreter modes are great for testing short snippets of code or introspecting modules etc. However any real code should go in a real script. Using %edit for convenience while you write the script is fine but make sure that what you're creating is a real Python script that you can run normally with 'python myscript.py'. Spyder is also good for doing this. Otherwise all of your work, computation and plots are a mess and it becomes impossible to trace back to exactly how you produced everything to check your work or to fix it when it becomes apparent that you've screwed up. Oscar From jshrager at gmail.com Fri Oct 4 10:16:48 2013 From: jshrager at gmail.com (Jeff Shrager) Date: Fri, 4 Oct 2013 07:16:48 -0700 Subject: ipy %run noob confusion In-Reply-To: References: Message-ID: Thank you. This is extremely helpful. The key that I was missing is that it's running them outside of the ipy context. I also discovered that if you call the script .ipy instead of .py, it actually does more or less what I was expecting -- that is, it allows magic commands, and I got the thing working as desired through this means. I do appreciate the concern that my script should be "real" python, and I would aim for that eventually, but while I'm experimenting, it's useful to be able to have the interactive and script work the same way. I'm happy to "real-ify" it later. BTW, I generally work the opposite way than %edit: I code in emacs, and have a shell that's running ipy. Actually ipy is bad at this because it sends all sorts of terminal crap that I can probably get rid of somehow, but working this way has many advantages, not the least of which is that you don't have to fire up editors over and over (my editor context is primary), and I get an automatic log of my whole session in the shell buffer. This mode has served me well using regular python, because I didn't have the .ipy v. .py problem, but when I started into ipyton, things no longer worked. But your guidance (and others) have help deconfuse me on this. Thanks again! -------------- next part -------------- An HTML attachment was scrubbed... URL: From anthra.norell at bluewin.ch Fri Oct 4 03:38:41 2013 From: anthra.norell at bluewin.ch (F.R.) Date: Fri, 04 Oct 2013 09:38:41 +0200 Subject: Where does MySQLdb put inserted data? Message-ID: <524E7081.3080308@bluewin.ch> Hi, As of late clipboard pasting into a terminal sometimes fails (a known bug, apparently), I use MySQLdb to access MySQL tables. In general this works just fine. But now I fail filling a new table. The table exists. "mysql>EXPLAIN new_table;" explains and "root at blackbox-one:/# sudo/find / -name 'new_table*'" finds "/var/lib/mysql/fr/new_table.frm". So I do "cursor.executemany ('insert into new_table values (%s)' % format, data)". No error occurs and "cursor.execute ('select * from new_table;')" returns the number of records read, and "cursor.fetchall ()" returns all new records. All looks fine, but "mysql>SELECT * FROM new_table;" produces an "Empty set" and "sudo find / -name 'new_table*" still finds only the format file, same as before. Could it have to do with COMMIT. I believe I am using ISAM tables (default?) and those don't recognize undo commands, right?. Anyway, an experimental "cursor.execute ('COMMIT')" didn't make a difference. It looks like MySQLdb puts the data into a cache and that cache should be saved either by the OS or by me. Strange thing is that this is one freak incident in an almost daily routine going back years and involving thousands of access operations in and out acting instantaneously. I seem to remember a similar case some time ago and it also involved a new empty table. Thanks for hints Frederic mysql> select version() -> ; +-------------------------+ | version() | +-------------------------+ | 5.5.31-0ubuntu0.12.04.1 | +-------------------------+ 1 row in set (0.00 sec) From steve+comp.lang.python at pearwood.info Fri Oct 4 04:34:22 2013 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: 04 Oct 2013 08:34:22 GMT Subject: Where does MySQLdb put inserted data? References: Message-ID: <524e7d8e$0$29984$c3e8da3$5496439d@news.astraweb.com> On Fri, 04 Oct 2013 09:38:41 +0200, F.R. wrote: > Hi, > As of late clipboard pasting into a terminal sometimes fails (a > known bug, apparently), I use MySQLdb to access MySQL tables. [...] You appear to have posted to the wrong list. This is a Python list, not MySQL. Nothing in your question appears to be about Python. If I am mistaken, please re-word your question explain what Python code you are having trouble with, what you tried, what you expected, and what it actually did. -- Steven From anthra.norell at bluewin.ch Fri Oct 4 06:05:17 2013 From: anthra.norell at bluewin.ch (F.R.) Date: Fri, 04 Oct 2013 12:05:17 +0200 Subject: Where does MySQLdb put inserted data? In-Reply-To: <524E7081.3080308@bluewin.ch> References: <524E7081.3080308@bluewin.ch> Message-ID: <524E92DD.9040501@bluewin.ch> On 10/04/2013 09:38 AM, F.R. wrote: > Hi, > As of late clipboard pasting into a terminal sometimes fails (a > known bug, apparently), I use MySQLdb to access MySQL tables. In > general this works just fine. But now I fail filling a new table. The > table exists. "mysql>EXPLAIN new_table;" explains and > "root at blackbox-one:/# sudo/find / -name 'new_table*'" finds > "/var/lib/mysql/fr/new_table.frm". So I do "cursor.executemany > ('insert into new_table values (%s)' % format, data)". No error occurs > and "cursor.execute ('select * from new_table;')" returns the number > of records read, and "cursor.fetchall ()" returns all new records. All > looks fine, but "mysql>SELECT * FROM new_table;" produces an "Empty > set" and "sudo find / -name 'new_table*" still finds only the format > file, same as before. > Could it have to do with COMMIT. I believe I am using ISAM tables > (default?) and those don't recognize undo commands, right?. Anyway, an > experimental "cursor.execute ('COMMIT')" didn't make a difference. It > looks like MySQLdb puts the data into a cache and that cache should be > saved either by the OS or by me. Strange thing is that this is one > freak incident in an almost daily routine going back years and > involving thousands of access operations in and out acting > instantaneously. I seem to remember a similar case some time ago and > it also involved a new empty table. > > Thanks for hints > > Frederic > > > > mysql> select version() > -> ; > +-------------------------+ > | version() | > +-------------------------+ > | 5.5.31-0ubuntu0.12.04.1 | > +-------------------------+ > 1 row in set (0.00 sec) > Thank you Chris, thank you Steven, The suggestion to switch to PostgreSQL isn't lost on me. I have it installed, but have been putting off the change, apprehensive of getting slowed down by many annoying side effects for some time to come. This may be the moment . . . Off list? MySQL is. MySQLdb is not. Before I know which of the two is the culprit, I don't know whether I'm off list or not and take the risk, prepared to beg pardon if I am. Frederic From rosuav at gmail.com Fri Oct 4 06:11:32 2013 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 4 Oct 2013 20:11:32 +1000 Subject: Where does MySQLdb put inserted data? In-Reply-To: <524E92DD.9040501@bluewin.ch> References: <524E7081.3080308@bluewin.ch> <524E92DD.9040501@bluewin.ch> Message-ID: On Fri, Oct 4, 2013 at 8:05 PM, F.R. wrote: > Off list? MySQL is. MySQLdb is not. Before I know which of the two is > the culprit, I don't know whether I'm off list or not and take the risk, > prepared to beg pardon if I am. > Just to clarify: Off-topic means discussing stuff that isn't about Python; off-list means sending private emails, not to python-list at python.org / comp.lang.python. You're uncertain as to whether you're off-topic or not, but you're definitely on-list; and my previous mail to you was off-list, so people here are going to be a little confused, as they lack context. (I merely suggested that switching to PostgreSQL would quite probably be a worthwhile time investment.) ChrisA From anthra.norell at bluewin.ch Fri Oct 4 06:45:18 2013 From: anthra.norell at bluewin.ch (F.R.) Date: Fri, 04 Oct 2013 12:45:18 +0200 Subject: Where does MySQLdb put inserted data? In-Reply-To: References: <524E7081.3080308@bluewin.ch> <524E92DD.9040501@bluewin.ch> Message-ID: <524E9C3E.8010104@bluewin.ch> On 10/04/2013 12:11 PM, Chris Angelico wrote: > On Fri, Oct 4, 2013 at 8:05 PM, F.R. wrote: >> Off list? MySQL is. MySQLdb is not. Before I know which of the two is >> the culprit, I don't know whether I'm off list or not and take the risk, >> prepared to beg pardon if I am. >> > Just to clarify: Off-topic means discussing stuff that isn't about > Python; off-list means sending private emails, not to > python-list at python.org / comp.lang.python. You're uncertain as to > whether you're off-topic or not, but you're definitely on-list; and my > previous mail to you was off-list, so people here are going to be a > little confused, as they lack context. (I merely suggested that > switching to PostgreSQL would quite probably be a worthwhile time > investment.) > > ChrisA I shall switch and give you credit for the impulse in addition to the terminological clarification on being off something or other . . . Frederic From anthra.norell at bluewin.ch Sat Oct 5 04:39:03 2013 From: anthra.norell at bluewin.ch (F.R.) Date: Sat, 05 Oct 2013 10:39:03 +0200 Subject: Where does MySQLdb put inserted data? In-Reply-To: <8mhu49dobl4dg99am35dcri2qegp0ssbrc@4ax.com> References: <524E7081.3080308@bluewin.ch> <8mhu49dobl4dg99am35dcri2qegp0ssbrc@4ax.com> Message-ID: <524FD027.6040202@bluewin.ch> On 10/05/2013 12:55 AM, Dennis Lee Bieber wrote: > On Fri, 04 Oct 2013 09:38:41 +0200, "F.R." > declaimed the following: > > > > MySQLdb, as with all DB-API compliant adapters, does NOT do > "auto-commit" -- you MUST execute a con.commit() after any query (sequence) > that modifies data. Without it, closing the connection will invoke a > ROLLBACK operation, removing any attempted changes. That's it! It works! Thank you sooo much. A miracle how I could go without commits for years and never have missing data. Anyway, another lesson learned . . . Thanks Frederic From breamoreboy at yahoo.co.uk Sat Oct 5 07:39:16 2013 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Sat, 05 Oct 2013 12:39:16 +0100 Subject: Where does MySQLdb put inserted data? In-Reply-To: <524FD027.6040202@bluewin.ch> References: <524E7081.3080308@bluewin.ch> <8mhu49dobl4dg99am35dcri2qegp0ssbrc@4ax.com> <524FD027.6040202@bluewin.ch> Message-ID: On 05/10/2013 09:39, F.R. wrote: > On 10/05/2013 12:55 AM, Dennis Lee Bieber wrote: >> On Fri, 04 Oct 2013 09:38:41 +0200, "F.R." >> declaimed the following: >> >> >> >> MySQLdb, as with all DB-API compliant adapters, does NOT do >> "auto-commit" -- you MUST execute a con.commit() after any query >> (sequence) >> that modifies data. Without it, closing the connection will invoke a >> ROLLBACK operation, removing any attempted changes. > > That's it! It works! Thank you sooo much. A miracle how I could go > without commits for years and never have missing data. Anyway, another > lesson learned . . . > > Thanks > > Frederic > Reminds me of the people who've never had an accident but have seen thousands in their rear view mirror. Perhaps the driver here http://www.bbc.co.uk/news/uk-england-hampshire-24367601 ? -- Roses are red, Violets are blue, Most poems rhyme, But this one doesn't. Mark Lawrence From dpalao.python at gmail.com Fri Oct 4 05:30:02 2013 From: dpalao.python at gmail.com (David Palao) Date: Fri, 4 Oct 2013 11:30:02 +0200 Subject: howto check programs and C libraries Message-ID: Hello, I'm in charge of preparing a computer room for the practices of "introduction to programming". One of the tasks is checking that from all the computers in the room one can execute some programs and link (and compile) against some libraries. My first idea was using Autotools (or cmake), but as I'm a big fan of python, I was thinking how to do that with python, and I don't have a clear solution yet. I know that distutils includes the distutils.command.config module, and I think it could do the job (when properly subclassed). Do you have a better idea? Thanks in advance. Regards, DPalao From oscar.j.benjamin at gmail.com Fri Oct 4 05:40:24 2013 From: oscar.j.benjamin at gmail.com (Oscar Benjamin) Date: Fri, 4 Oct 2013 10:40:24 +0100 Subject: howto check programs and C libraries In-Reply-To: References: Message-ID: On 4 October 2013 10:30, David Palao wrote: > Hello, > I'm in charge of preparing a computer room for the practices of > "introduction to programming". > One of the tasks is checking that from all the computers in the room > one can execute some programs and link (and compile) against some > libraries. > My first idea was using Autotools (or cmake), but as I'm a big fan of > python, I was thinking how to do that with python, and I don't have a > clear solution yet. > I know that distutils includes the distutils.command.config module, > and I think it could do the job (when properly subclassed). > Do you have a better idea? I'm not very experienced with it but perhaps waf would be better for you: https://code.google.com/p/waf/ Oscar From davea at davea.name Fri Oct 4 07:15:21 2013 From: davea at davea.name (Dave Angel) Date: Fri, 4 Oct 2013 11:15:21 +0000 (UTC) Subject: howto check programs and C libraries References: Message-ID: On 4/10/2013 05:30, David Palao wrote: > Hello, > I'm in charge of preparing a computer room for the practices of > "introduction to programming". > One of the tasks is checking that from all the computers in the room > one can execute some programs and link (and compile) against some > libraries. > My first idea was using Autotools (or cmake), but as I'm a big fan of > python, I was thinking how to do that with python, and I don't have a > clear solution yet. > I know that distutils includes the distutils.command.config module, > and I think it could do the job (when properly subclassed). > Do you have a better idea? I don't have any specific tool in mind, but I'd suggest that you really need to run the tool on only one of the machines, and run a separate tool on them all that assures that they are identical, presumably made that way by wiping each and copying from a master That assumes the hardware is close enough to identical to make this practical. -- DaveA From ganeshsahni07 at gmail.com Fri Oct 4 07:57:23 2013 From: ganeshsahni07 at gmail.com (Ravi Sahni) Date: Fri, 4 Oct 2013 17:27:23 +0530 Subject: howto check programs and C libraries In-Reply-To: References: Message-ID: On Fri, Oct 4, 2013 at 3:00 PM, David Palao wrote: > Hello, > I'm in charge of preparing a computer room for the practices of > "introduction to programming". > One of the tasks is checking that from all the computers in the room > one can execute some programs and link (and compile) against some > libraries. > My first idea was using Autotools (or cmake), but as I'm a big fan of > python, I was thinking how to do that with python, and I don't have a > clear solution yet. > I know that distutils includes the distutils.command.config module, > and I think it could do the job (when properly subclassed). > Do you have a better idea? I have ruby on rails friends who speak of capistrano and puppet. google puppet python gives me : http://stackful-dev.com/cuisine-the-lightweight-chefpuppet-alternative If you find it good I shall be interested to know. -- Ravi From markotaht at gmail.com Fri Oct 4 06:52:02 2013 From: markotaht at gmail.com (markotaht at gmail.com) Date: Fri, 4 Oct 2013 03:52:02 -0700 (PDT) Subject: Image manipulation Message-ID: <67342b91-2ca5-46e9-b887-cda31fe96697@googlegroups.com> Is there a way using the python 3.3.2 whidout any additional downloaded moduls, to get a pixels RGB value? From gary.herron at islandtraining.com Fri Oct 4 12:14:57 2013 From: gary.herron at islandtraining.com (Gary Herron) Date: Fri, 04 Oct 2013 09:14:57 -0700 Subject: Image manipulation In-Reply-To: <67342b91-2ca5-46e9-b887-cda31fe96697@googlegroups.com> References: <67342b91-2ca5-46e9-b887-cda31fe96697@googlegroups.com> Message-ID: <524EE981.9060807@islandtraining.com> On 10/04/2013 03:52 AM, markotaht at gmail.com wrote: > Is there a way using the python 3.3.2 whidout any additional downloaded moduls, to get a pixels RGB value? No (I guess). If you want a better answer, then you'll have to give us a *much* better question. Get a pixel from *what*? (screen? image on disk? image on web page? printed image on paper? ...?) On what operating system? Using what kind of a graphics/imaging hardware/software? How are said pixels being written? Please understand how broad the term pixel is and how little information your question has provided. Try again with a full description of what you want, and we'll try to provide a useful answer. Gary Herron From markotaht at gmail.com Sun Oct 6 09:19:48 2013 From: markotaht at gmail.com (markotaht at gmail.com) Date: Sun, 6 Oct 2013 06:19:48 -0700 (PDT) Subject: Image manipulation In-Reply-To: <67342b91-2ca5-46e9-b887-cda31fe96697@googlegroups.com> References: <67342b91-2ca5-46e9-b887-cda31fe96697@googlegroups.com> Message-ID: Image consists of pixels. Each pixel has its RGBA values. In python whidout any extra downloadable modules, is there a way to get those values. I know PIG has it but i hav Python 3.3.2 so PIG wont do. In java under swingx there is a command called .getRGB() and whit some manipulation to that value i can get the R G B and A. Is there something similar in python? From kwpolska at gmail.com Sun Oct 6 09:24:35 2013 From: kwpolska at gmail.com (=?UTF-8?B?Q2hyaXMg4oCcS3dwb2xza2HigJ0gV2Fycmljaw==?=) Date: Sun, 6 Oct 2013 15:24:35 +0200 Subject: Image manipulation In-Reply-To: References: <67342b91-2ca5-46e9-b887-cda31fe96697@googlegroups.com> Message-ID: On Sun, Oct 6, 2013 at 3:19 PM, wrote: > Image consists of pixels. Each pixel has its RGBA values. In python whidout any extra downloadable modules, is there a way to get those values. I know PIG has it but i hav Python 3.3.2 so PIG wont do. PIG? Did you mean PIL? In that case, go use Pillow, the Python 3-compatible fork. -- Chris ?Kwpolska? Warrick PGP: 5EAAEA16 stop html mail | always bottom-post | only UTF-8 makes sense From markotaht at gmail.com Tue Oct 8 10:15:46 2013 From: markotaht at gmail.com (markotaht at gmail.com) Date: Tue, 8 Oct 2013 07:15:46 -0700 (PDT) Subject: Image manipulation In-Reply-To: <67342b91-2ca5-46e9-b887-cda31fe96697@googlegroups.com> References: <67342b91-2ca5-46e9-b887-cda31fe96697@googlegroups.com> Message-ID: First helpful advice i have gotten from this forum From steve+comp.lang.python at pearwood.info Tue Oct 8 10:26:33 2013 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: 08 Oct 2013 14:26:33 GMT Subject: Image manipulation References: <67342b91-2ca5-46e9-b887-cda31fe96697@googlegroups.com> Message-ID: <52541618$0$29984$c3e8da3$5496439d@news.astraweb.com> On Tue, 08 Oct 2013 07:15:46 -0700, markotaht wrote: > First helpful advice i have gotten from this forum If you insist on dropping cryptic comments with bad spelling, incoherent sentences, and a complete lack of any context, it might be the last advice you get too. Please help us to help you. We are not mind readers, we cannot read your mind and magically understand what you are talking about. Please include content when replying to an posts. Please take the time to try to explain your questions, with proper grammar and syntax and spelling. We will make allowances if English is not your native language, but we won't make allowances if you are just being lazy. Please show small code snippets that demonstrate the problem. You should read this site: even though it is written for Java, the basic ideas hold for Python as well. http://sscce.org Remember that we are volunteers and we are not being paid to help you. The harder you make it for us to understand your posts, the less likely we are to solve your problem. -- Steven From markotaht at gmail.com Tue Oct 8 10:55:06 2013 From: markotaht at gmail.com (markotaht at gmail.com) Date: Tue, 8 Oct 2013 07:55:06 -0700 (PDT) Subject: Image manipulation In-Reply-To: <52541618$0$29984$c3e8da3$5496439d@news.astraweb.com> References: <67342b91-2ca5-46e9-b887-cda31fe96697@googlegroups.com> <52541618$0$29984$c3e8da3$5496439d@news.astraweb.com> Message-ID: <76c0f0ac-1135-4860-85f1-abbba09b54bc@googlegroups.com> teisip?ev, 8. oktoober 2013 17:26.33 UTC+3 kirjutas Steven D'Aprano: > On Tue, 08 Oct 2013 07:15:46 -0700, markotaht wrote: > > > > > First helpful advice i have gotten from this forum > > > > If you insist on dropping cryptic comments with bad spelling, incoherent > > sentences, and a complete lack of any context, it might be the last > > advice you get too. > > > > Please help us to help you. We are not mind readers, we cannot read your > > mind and magically understand what you are talking about. Please include > > content when replying to an posts. Please take the time to try to explain > > your questions, with proper grammar and syntax and spelling. We will make > > allowances if English is not your native language, but we won't make > > allowances if you are just being lazy. > > > > Please show small code snippets that demonstrate the problem. You should > > read this site: even though it is written for Java, the basic ideas hold > > for Python as well. > > > > http://sscce.org > > > > > > Remember that we are volunteers and we are not being paid to help you. > > The harder you make it for us to understand your posts, the less likely > > we are to solve your problem. > > > > > > -- > > Steven Well english isnt my native language, and there are things i just dont know how to explain in any language. And i cant give all the codes i am using, since there are loads of pieces i do not own, but i know the people who made them. From ben+python at benfinney.id.au Tue Oct 8 17:05:51 2013 From: ben+python at benfinney.id.au (Ben Finney) Date: Wed, 09 Oct 2013 08:05:51 +1100 Subject: How to help us to help you (was: Image manipulation) References: <67342b91-2ca5-46e9-b887-cda31fe96697@googlegroups.com> <52541618$0$29984$c3e8da3$5496439d@news.astraweb.com> <76c0f0ac-1135-4860-85f1-abbba09b54bc@googlegroups.com> Message-ID: <7wd2nfpj68.fsf_-_@benfinney.id.au> markotaht at gmail.com writes: > Well english isnt my native language, and there are things i just dont > know how to explain in any language. I sympathise with attempting to explain things in a foreign language. Sorry that you have that difficulty here. But the rest of Steven's advice is sound: you need to help us to help you. One way to do that: Please don't use Google Groups to post here, it mangles your replies in ways that make it difficult for you to be understood . > And i cant give all the codes i am using, since there are loads of > pieces i do not own, but i know the people who made them. So you need to present examples that are short, self-contained, correct, compilable . That may entail you writing the examples for the purpose of posting them here; but do run them yourself before posting! You'll often learn about the problem without needing our input if you do it that way. And if you don't, then you have a tool that will definitely get you better responses here. -- \ ?The problem with television is that the people must sit and | `\ keep their eyes glued on a screen: the average American family | _o__) hasn't time for it.? ?_The New York Times_, 1939 | Ben Finney From markotaht at gmail.com Tue Oct 8 10:57:25 2013 From: markotaht at gmail.com (markotaht at gmail.com) Date: Tue, 8 Oct 2013 07:57:25 -0700 (PDT) Subject: Image manipulation In-Reply-To: <67342b91-2ca5-46e9-b887-cda31fe96697@googlegroups.com> References: <67342b91-2ca5-46e9-b887-cda31fe96697@googlegroups.com> Message-ID: I rembembered a bit too late, are there any good tutorials for Pillow? ImageTk.PhotoImage() keeps giving me error that there isnt such a method. From steve+comp.lang.python at pearwood.info Fri Oct 4 09:56:14 2013 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: 04 Oct 2013 13:56:14 GMT Subject: API for custom Unicode error handlers Message-ID: <524ec8fe$0$29984$c3e8da3$5496439d@news.astraweb.com> I have some custom Unicode error handlers, and I'm looking for advice on the right API for dealing with them. I have a module containing custom Unicode error handlers. For example: # Python 3 import unicodedata def namereplace_errors(exc): c = exc.object[exc.start] try: name = unicodedata.name(c) except (KeyError, ValueError): n = ord(c) if n <= 0xFFFF: replace = "\\u%04x" else: assert n <= 0x10FFFF replace = "\\U%08x" replace = replace % n else: replace = "\\N{%s}" % name return replace, exc.start + 1 Before I can use the error handler, I need to register it using this: import codecs codecs.register_error('namereplace', namereplace_errors) And now: py> 'abc\u04F1'.encode('ascii', 'namereplace') b'abc\\N{CYRILLIC SMALL LETTER U WITH DIAERESIS}' Now, my question: Should the module holding the error handlers automatically register them? In other words, if I do: import error_handlers just importing it will have the side-effect of registering the error handlers. Normally, I dislike imports that have side-effects of this sort, but I'm not sure that the alternative is better, that is, to put responsibility on the caller to register some, or all, of the handlers: import error_handlers error_handlers.register(error_handlers.namereplace_errors) error_handlers.register_all() As far as I know, there is no way to find out what error handlers are registered, and no way to deregister one after it has been registered. Which API would you prefer if you were using this module? -- Steven From rosuav at gmail.com Fri Oct 4 13:22:48 2013 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 5 Oct 2013 03:22:48 +1000 Subject: API for custom Unicode error handlers In-Reply-To: <524ec8fe$0$29984$c3e8da3$5496439d@news.astraweb.com> References: <524ec8fe$0$29984$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Fri, Oct 4, 2013 at 11:56 PM, Steven D'Aprano wrote: > Should the module holding the error handlers automatically register them? > In other words, if I do: > > import error_handlers > > just importing it will have the side-effect of registering the error > handlers. Normally, I dislike imports that have side-effects of this > sort, but I'm not sure that the alternative is better, that is, to put > responsibility on the caller to register some, or all, of the handlers: > > import error_handlers > error_handlers.register(error_handlers.namereplace_errors) > error_handlers.register_all() Caveat: I don't actually use codecs much, so I don't know the specifics. I'd be quite happy with importing having a side-effect here. If you import a module that implements a numeric type, it should immediately register itself with the Numeric ABC, right? This is IMO equivalent to that. > As far as I know, there is no way to find out what error handlers are > registered, and no way to deregister one after it has been registered. The only risk that I see is of an accidental collision. Having a codec registered that you don't use can't hurt (afaik). Is there any mechanism for detecting a name collision? If not, I wouldn't worry about it. ChrisA From storchaka at gmail.com Fri Oct 4 15:08:50 2013 From: storchaka at gmail.com (Serhiy Storchaka) Date: Fri, 04 Oct 2013 22:08:50 +0300 Subject: API for custom Unicode error handlers In-Reply-To: References: <524ec8fe$0$29984$c3e8da3$5496439d@news.astraweb.com> Message-ID: 04.10.13 20:22, Chris Angelico ???????(??): > I'd be quite happy with importing having a side-effect here. If you > import a module that implements a numeric type, it should immediately > register itself with the Numeric ABC, right? This is IMO equivalent to > that. There is a difference. You can't use a numeric type without importing a module, but you can use error handler registered outside of your module. This leads to subtle bugs. Let the A module imports error_handlers and uses error handle. The module B uses error handle but doesn't import error_handlers. C.py imports A and B and all works. D.py imports B and A and fails. From ethan at stoneleaf.us Fri Oct 4 14:05:45 2013 From: ethan at stoneleaf.us (Ethan Furman) Date: Fri, 04 Oct 2013 11:05:45 -0700 Subject: API for custom Unicode error handlers In-Reply-To: <524ec8fe$0$29984$c3e8da3$5496439d@news.astraweb.com> References: <524ec8fe$0$29984$c3e8da3$5496439d@news.astraweb.com> Message-ID: <524F0379.6000500@stoneleaf.us> On 10/04/2013 06:56 AM, Steven D'Aprano wrote: > > Should the module holding the error handlers automatically register them? I think it should. Registration only needs to happen once, the module is useless without being registered, no threads nor processes are being started, and the only reason to import the module is to get the functionality... isn't it? What about help(), sphynx (sp?), or other introspection tools? This sounds similar to cgitb -- another module which you only import if you want the html'ized traceback, and yet it requires a separate cgitb.enable() call... I change my mind, it shouldn't. Throw in a .enable() function and call it good. :) -- ~Ethan~ From storchaka at gmail.com Fri Oct 4 15:35:47 2013 From: storchaka at gmail.com (Serhiy Storchaka) Date: Fri, 04 Oct 2013 22:35:47 +0300 Subject: API for custom Unicode error handlers In-Reply-To: <524ec8fe$0$29984$c3e8da3$5496439d@news.astraweb.com> References: <524ec8fe$0$29984$c3e8da3$5496439d@news.astraweb.com> Message-ID: 04.10.13 16:56, Steven D'Aprano ???????(??): > I have some custom Unicode error handlers, and I'm looking for advice on > the right API for dealing with them. > > I have a module containing custom Unicode error handlers. For example: > > # Python 3 > import unicodedata > def namereplace_errors(exc): > c = exc.object[exc.start] > try: > name = unicodedata.name(c) > except (KeyError, ValueError): > n = ord(c) > if n <= 0xFFFF: > replace = "\\u%04x" > else: > assert n <= 0x10FFFF > replace = "\\U%08x" > replace = replace % n > else: > replace = "\\N{%s}" % name > return replace, exc.start + 1 I'm planning to built this error handler in 3.4 (see http://comments.gmane.org/gmane.comp.python.ideas/21296). Actually Python implementation should looks like: def namereplace_errors(exc): if not isinstance(exc, UnicodeEncodeError): raise exc replace = [] for c in exc.object[exc.start:exc.end]: try: replace.append(r'\N{%s}' % unicodedata.name(c)) except KeyError: n = ord(c) if n < 0x100: replace.append(r'\x%02x' % n) elif n < 0x10000: replace.append(r'\u%04x' % n) else: replace.append(r'\U%08x' % n) return ''.join(replace), exc.end > Now, my question: > > Should the module holding the error handlers automatically register them? This question interesting me too. From tjreedy at udel.edu Fri Oct 4 18:44:44 2013 From: tjreedy at udel.edu (Terry Reedy) Date: Fri, 04 Oct 2013 18:44:44 -0400 Subject: API for custom Unicode error handlers In-Reply-To: References: <524ec8fe$0$29984$c3e8da3$5496439d@news.astraweb.com> Message-ID: On 10/4/2013 3:35 PM, Serhiy Storchaka wrote: > 04.10.13 16:56, Steven D'Aprano ???????(??): >> I have some custom Unicode error handlers, and I'm looking for advice on >> the right API for dealing with them. > I'm planning to built this error handler in 3.4 (see > http://comments.gmane.org/gmane.comp.python.ideas/21296). >> Should the module holding the error handlers automatically register them? > > This question interesting me too. I did not respond on the p-i thread, but +1 for 'namereplace' also. Like others, I would prefer auto-register unless that creates a problem. If it is a problem, perhaps the registry mechanism needs improvement. On the other hand, it is it built-in, it will be pre-registered. -- Terry Jan Reedy From john.r.moser at gmail.com Fri Oct 4 15:18:53 2013 From: john.r.moser at gmail.com (john.r.moser at gmail.com) Date: Fri, 4 Oct 2013 12:18:53 -0700 (PDT) Subject: Help wanted: Writing a PMI Project Management system Message-ID: <8983ac56-811b-4d1b-962a-8280c89a732a@googlegroups.com> I'm looking for anyone who has an interest in project management; workable Python design and programming skills; and wants to code for an open source Project Management system. Having used Redmine, Launchpad, Trak, OpenProj, and so on, I've found there's no good PM tools. Microsoft Project and Oracle PM are even kind of crap. The issue I have is that none of these are really in line with modern PM. The PMBOK and various practice standards outline something very different from what's available. For example: Redmine has the concept of Projects and Subprojects; but it doesn't have the concept of a Program, and instead follows the common open source model by which a "Project" is all on-going effort of a deliverable. Calling "Mozilla Firefox" a project is technically incorrect. "Mozilla Firefox" is the deliverable of a Program (called Mozilla Firefox), an ongoing effort to meet a business need (in this case, the "business need" is the collective desire to produce a Web browser). Mozilla Firefox 18 is a Project; Mozilla Firefox 19 is a Project; and so on. More Projects will be added to the Program; long-standing bugs in Mozilla Firefox are Program Issues, and a particular Project (i.e. Mozilla Firefox 21) may include as a requirement "Correct and close bug #55995" or such. I've been trying to write a technically correct Project Management software system that adheres to the PMBOK 5e (released January 2013), but I am a crappy software designer. I can code, some, but I don't know Python well enough; I don't know how to accomplish some design aspects (like modularity--the ability to extend via drop-in plug-ins instead of by tightly integrated code) and I don't know much about software design at all anyway. I know how to translate procedural concepts into code in any language I can code in to the best of my understanding of the language--some people think this makes them good programmers; they create absolute garbage, and I have no intent of being that guy. I'll code, but I'll code based on the design of somebody actually competent, thanks. So I guess what I want is to find people who are: * Interested in Project Management as a concept * Skilled in application design and Python * Interested in putting these two things together and contributing to a program to develop a technically correct Project Management software suite In open source software development, the primary compensation is self-enrichment: Developers are attracted by the chance to learn something new, to enhance their skills, to contribute to something they feel is worthwhile, or to work on something they simply enjoy working on. Thus I need to find people who go, "That looks interesting." Where do I find? >From a planning perspective, the initial goal is to create the four main modules: * Project Management * Program Management * Portfolio Management * Work Breakdown Structure Projects are a temporary effort to produce a deliverable. Programs are an ongoing effort to address a business or organizational need, and may include other programs and projects. Portfolios are collections of Projects, Programs, and Portfolios under the control of an organization. A business may have its Portfolio, containing its Marketing and MIS Portfolios, which each contain Programs (for Marketing or MIS) that contain Projects and other Programs. A department may even have sub-portfolios: the MIS Portfolio may contain Networking Portfolio, Programming Portfolio, and Systems Portfolio for the three departments under MIS. Each Project has a Work Breakdown Structure. Every element is a deliverable, so something that must actually exist. The Project is itself the sole top-level element, for example: 1. Work Breakdown Structure Module The process of Decomposition is used to break down the project into manageable work. Work is "Manageable" when it can be understood; breaking it down further gives no gain and causes increased management overhead, so is wasteful. There are many decomposition strategies such as top-down, bottom-up, brainstorm, and fishbone; they are usually combined (I favor top-down, then bottom-up). The first level of decomposition can go in any direction. It is often recommended as decomposition by phase (Design, Implementation, Testing...) or by major deliverables (Drive System, Engine, Wheels, Suspension...), or combination means. 1. Work Breakdown Structure Module 1.1 Project Management 1.2 Design 1.3 Implementation 1.4 Testing 1.5 Release These are further broken down. 1. Work Breakdown Structure Module 1.1 Project Management 1.1.1 Work Breakdown Structure 1.1.2 Meetings 1.2 Design 1.2.1 Document Schema 1.2.2 UI 1.2.2.1 Outline View 1.2.2.2 Table View 1.2.2.3 Graphical View 1.3 Implementation 1.3.1 Back-End 1.3.1.1 Storage Class 1.3.1.2 Data Rendering 1.3.2 Front-End 1.3.2.1 Views 1.3.2.1.1 Outline View 1.3.2.1.2 Table View 1.3.2.1.3 Graphical View 1.4 Testing 1.5 Release 1.5.1 Git Branch Merge 1.5.2 Git Tag etc. The reason I want these features as the earliest target is because then it becomes a useful planning tool. A Portfolio can be created with a Program to track each Project--the implementation of each module, further integration, code clean-up efforts, etc.--and the work can be broken down in the WBS module. That means the WBS module can be used to plan the Issue module; then the Issue module can be used to track any Project Issues in further modules. Eventually I want extension beyond a blunt PMI-focused PM tool into something that's targeted at development. Interesting features would be the separate tracking of Git repositories, repository management, and the association of Git branches with Projects and WBS Work Packages. Anyone interested? From galeomaga at gmail.com Sat Oct 5 03:38:51 2013 From: galeomaga at gmail.com (galeomaga at gmail.com) Date: Sat, 5 Oct 2013 00:38:51 -0700 (PDT) Subject: How to streamingly read text file and display whenever updated text Message-ID: <04ee91f9-1cbf-4364-bca3-da25aa4db45f@googlegroups.com> #!/usr/bin/python import time f = open('/home/martin/Downloads/a.txt') while 1: for line in f: print line; time.sleep(1); From galeomaga at gmail.com Sat Oct 5 03:54:39 2013 From: galeomaga at gmail.com (galeomaga at gmail.com) Date: Sat, 5 Oct 2013 00:54:39 -0700 (PDT) Subject: How to streamingly read text file and display whenever updated text In-Reply-To: <04ee91f9-1cbf-4364-bca3-da25aa4db45f@googlegroups.com> References: <04ee91f9-1cbf-4364-bca3-da25aa4db45f@googlegroups.com> Message-ID: <07aa5feb-8971-405e-bae8-887486e99127@googlegroups.com> gale... at gmail.com? 2013?10?5????UTC+8??3?38?51???? > #!/usr/bin/python > > import time > > f = open('/home/martin/Downloads/a.txt') > > while 1: > > for line in f: > > print line; > > time.sleep(1); if __name__ == '__main__': logfile = open("/home/martin/Downloads/a.txt","r"); while True: line = logfile.readline(); if not line: print line; time.sleep(1); this also failed From breamoreboy at yahoo.co.uk Sat Oct 5 04:12:55 2013 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Sat, 05 Oct 2013 09:12:55 +0100 Subject: How to streamingly read text file and display whenever updated text In-Reply-To: <07aa5feb-8971-405e-bae8-887486e99127@googlegroups.com> References: <04ee91f9-1cbf-4364-bca3-da25aa4db45f@googlegroups.com> <07aa5feb-8971-405e-bae8-887486e99127@googlegroups.com> Message-ID: On 05/10/2013 08:54, galeomaga at gmail.com wrote: > > if __name__ == '__main__': > logfile = open("/home/martin/Downloads/a.txt","r"); > while True: > line = logfile.readline(); > if not line: > print line; > time.sleep(1); > > this also failed > Usually please state your OS and Python versions, what you expected to happen, what actually happened and the full traceback if applicable. In this case I'd hazard a guess that as you're trying to print something that evaluates to false you're not likely to see much. You can also remove the semicolons as they're simply not needed. -- Roses are red, Violets are blue, Most poems rhyme, But this one doesn't. Mark Lawrence From james.harris.1 at gmail.com Sat Oct 5 04:06:32 2013 From: james.harris.1 at gmail.com (James Harris) Date: Sat, 5 Oct 2013 09:06:32 +0100 Subject: How to streamingly read text file and display whenever updated text References: <04ee91f9-1cbf-4364-bca3-da25aa4db45f@googlegroups.com> Message-ID: wrote in message news:04ee91f9-1cbf-4364-bca3-da25aa4db45f at googlegroups.com... > > > #!/usr/bin/python > import time > f = open('/home/martin/Downloads/a.txt') Looks like you are on Unix so you can do this from the shell tail -F /home/martin/Downloads/a.txt James From breamoreboy at yahoo.co.uk Sat Oct 5 04:14:48 2013 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Sat, 05 Oct 2013 09:14:48 +0100 Subject: How to streamingly read text file and display whenever updated text In-Reply-To: References: <04ee91f9-1cbf-4364-bca3-da25aa4db45f@googlegroups.com> Message-ID: On 05/10/2013 09:06, James Harris wrote: > wrote in message > news:04ee91f9-1cbf-4364-bca3-da25aa4db45f at googlegroups.com... >> >> >> #!/usr/bin/python >> import time >> f = open('/home/martin/Downloads/a.txt') > > Looks like you are on Unix so you can do this from the shell > > tail -F /home/martin/Downloads/a.txt > > James > > Tail also works on Windows if you've unxutils installed :) -- Roses are red, Violets are blue, Most poems rhyme, But this one doesn't. Mark Lawrence From nobody at nowhere.com Sat Oct 5 06:25:15 2013 From: nobody at nowhere.com (Nobody) Date: Sat, 05 Oct 2013 11:25:15 +0100 Subject: How to streamingly read text file and display whenever updated text References: <04ee91f9-1cbf-4364-bca3-da25aa4db45f@googlegroups.com> Message-ID: On Sat, 05 Oct 2013 00:38:51 -0700, galeomaga wrote: > #!/usr/bin/python > import time > f = open('/home/martin/Downloads/a.txt') > while 1: > for line in f: > print line; > time.sleep(1); So you're trying to implement "tail -f"? First, check that "tail -f" actually works for your particular use case. If the process writing the file uses buffered output, data will only actually be appended to the file when the buffer is full. You can't read what isn't there. And if the process creates a new file with the same name, rather than appending to the existing file, you'll still be reading the old file. You would need to open the file again to read the new file. From j.j.molenaar at gmail.com Sat Oct 5 07:02:05 2013 From: j.j.molenaar at gmail.com (Joost Molenaar) Date: Sat, 5 Oct 2013 13:02:05 +0200 Subject: How to streamingly read text file and display whenever updated text In-Reply-To: <04ee91f9-1cbf-4364-bca3-da25aa4db45f@googlegroups.com> References: <04ee91f9-1cbf-4364-bca3-da25aa4db45f@googlegroups.com> Message-ID: A bit of googling found me this: http://www.linux-support.com/cms/implementation-of-tail-in-python/ import time import sys def tail_f(file): interval = 1.0 while True: where = file.tell() line = file.readline() if not line: time.sleep(interval) file.seek(where) else: yield line From breamoreboy at yahoo.co.uk Sat Oct 5 07:08:18 2013 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Sat, 05 Oct 2013 12:08:18 +0100 Subject: How to streamingly read text file and display whenever updated text In-Reply-To: References: <04ee91f9-1cbf-4364-bca3-da25aa4db45f@googlegroups.com> Message-ID: On 05/10/2013 12:02, Joost Molenaar wrote: > A bit of googling found me this: > http://www.linux-support.com/cms/implementation-of-tail-in-python/ > > import time > import sys > > def tail_f(file): > interval = 1.0 > while True: > where = file.tell() > line = file.readline() > if not line: > time.sleep(interval) > file.seek(where) > else: > yield line > In future could you please quote some context so that it's easier for us mere mortals to follow the thread, thanks in anticipation. -- Roses are red, Violets are blue, Most poems rhyme, But this one doesn't. Mark Lawrence From galeomaga at gmail.com Sat Oct 5 23:17:32 2013 From: galeomaga at gmail.com (galeomaga at gmail.com) Date: Sat, 5 Oct 2013 20:17:32 -0700 (PDT) Subject: How to streamingly read text file and display whenever updated text In-Reply-To: References: <04ee91f9-1cbf-4364-bca3-da25aa4db45f@googlegroups.com> Message-ID: Joost Molenaar? 2013?10?5????UTC+8??7?02?05???? > A bit of googling found me this: > > http://www.linux-support.com/cms/implementation-of-tail-in-python/ > > > > import time > > import sys > > > > def tail_f(file): > > interval = 1.0 > > while True: > > where = file.tell() > > line = file.readline() > > if not line: > > time.sleep(interval) > > file.seek(where) > > else: > > yield line After tried many times, updated text file is not shown, it only print text at the first time. #!/usr/bin/python import time import sys import thread def tail_f(filehandler): interval = 1.0 while True: try: line = filehandler.readline() where = filehandler.tell() if not line: time.sleep(interval) filehandler.seek(where) else: yield line except: print "tail_f error" def readfile(systemname): try: filehandler = open("/home/martin/Downloads/a.txt","r"); while 1: #for line in tail_f(filehandler): # print line try: interval = 1.0 line = filehandler.readline() where = filehandler.tell() if not line: time.sleep(interval) filehandler.seek(where) print where else: print line except: print "tail_f error" except: print "Error: readfile" if __name__ == '__main__': try: thread.start_new_thread( readfile, ("Thread-1", ) ) except: print "Error: unable to start thread" while 1: pass From rosuav at gmail.com Sat Oct 5 23:28:02 2013 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 6 Oct 2013 14:28:02 +1100 Subject: How to streamingly read text file and display whenever updated text In-Reply-To: References: <04ee91f9-1cbf-4364-bca3-da25aa4db45f@googlegroups.com> Message-ID: On Sun, Oct 6, 2013 at 2:17 PM, wrote: > After tried many times, updated text file is not shown, it only print text at the first time. The implementation of tail has a lot of little oddities to deal with edge cases. Why not simply use it? A while ago, I wanted to make a system that would tail a bunch of logs on a bunch of computers, and display it all to me in a single unified view. Rather than write something that opened a whole lot of files and monitored them, I simply forked a 'tail' process for each file and reacted to its stdout. It was way WAY easier than dealing with everything that could possibly happen (log rotation, etc, etc, etc) - not that it'd be impossible to deal with, but it's a waste of time reinventing this particular wheel. Build on top of what's already there, save yourself the trouble. ChrisA From steve+comp.lang.python at pearwood.info Sun Oct 6 00:06:47 2013 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: 06 Oct 2013 04:06:47 GMT Subject: How to streamingly read text file and display whenever updated text References: <04ee91f9-1cbf-4364-bca3-da25aa4db45f@googlegroups.com> Message-ID: <5250e1d6$0$29984$c3e8da3$5496439d@news.astraweb.com> On Sat, 05 Oct 2013 20:17:32 -0700, galeomaga wrote: > if __name__ == '__main__': > try: > thread.start_new_thread( readfile, ("Thread-1", ) ) > except: > print "Error: unable to start thread" Why not? If you can't start a thread, you have a problem with your code. How do you expect to debug this problem? "I find it amusing when novice programmers believe their main job is preventing programs from crashing. More experienced programmers realize that correct code is great, code that crashes could use improvement, but incorrect code that doesn?t crash is a horrible nightmare." -- Chris Smith http://cdsmith.wordpress.com/2011/01/09/an-old-article-i-wrote/ -- Steven From galeomaga at gmail.com Sun Oct 6 04:49:25 2013 From: galeomaga at gmail.com (galeomaga at gmail.com) Date: Sun, 6 Oct 2013 01:49:25 -0700 (PDT) Subject: How to streamingly read text file and display whenever updated text In-Reply-To: <5250e1d6$0$29984$c3e8da3$5496439d@news.astraweb.com> References: <04ee91f9-1cbf-4364-bca3-da25aa4db45f@googlegroups.com> <5250e1d6$0$29984$c3e8da3$5496439d@news.astraweb.com> Message-ID: I can start thread and no exception error print, and I do not know how to use tail in python script I need to cope with MySQL in python later as all file path stored in it, it is to monitor all text files From breamoreboy at yahoo.co.uk Sun Oct 6 06:36:32 2013 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Sun, 06 Oct 2013 11:36:32 +0100 Subject: How to streamingly read text file and display whenever updated text In-Reply-To: <5250e1d6$0$29984$c3e8da3$5496439d@news.astraweb.com> References: <04ee91f9-1cbf-4364-bca3-da25aa4db45f@googlegroups.com> <5250e1d6$0$29984$c3e8da3$5496439d@news.astraweb.com> Message-ID: On 06/10/2013 05:06, Steven D'Aprano wrote: > On Sat, 05 Oct 2013 20:17:32 -0700, galeomaga wrote: > > >> if __name__ == '__main__': >> try: >> thread.start_new_thread( readfile, ("Thread-1", ) ) >> except: >> print "Error: unable to start thread" > > > Why not? If you can't start a thread, you have a problem with your code. > How do you expect to debug this problem? > > > "I find it amusing when novice programmers believe their main job is > preventing programs from crashing. More experienced programmers realize > that correct code is great, code that crashes could use improvement, but > incorrect code that doesn?t crash is a horrible nightmare." > -- Chris Smith > > http://cdsmith.wordpress.com/2011/01/09/an-old-article-i-wrote/ > > Roughly translated for the benefit of newbies remove the try/except :) Also note that a bare except is extremely bad practice, e.g. you can't stop rogue programs with a CTRL-C -- Roses are red, Violets are blue, Most poems rhyme, But this one doesn't. Mark Lawrence From rosuav at gmail.com Sun Oct 6 07:03:40 2013 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 6 Oct 2013 22:03:40 +1100 Subject: How to streamingly read text file and display whenever updated text In-Reply-To: References: <04ee91f9-1cbf-4364-bca3-da25aa4db45f@googlegroups.com> <5250e1d6$0$29984$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Sun, Oct 6, 2013 at 9:36 PM, Mark Lawrence wrote: > Also note that a bare except is extremely bad practice, e.g. you can't stop > rogue programs with a CTRL-C Or to be more accurate, a Ctrl-C will cause a jump to your except clause. Since, in this instance, that's going to emit and die, Ctrl-C will still stop the program. But yes, catching KeyboardInterrupt usually isn't your intention. ChrisA From breamoreboy at yahoo.co.uk Sun Oct 6 07:13:01 2013 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Sun, 06 Oct 2013 12:13:01 +0100 Subject: How to streamingly read text file and display whenever updated text In-Reply-To: References: <04ee91f9-1cbf-4364-bca3-da25aa4db45f@googlegroups.com> <5250e1d6$0$29984$c3e8da3$5496439d@news.astraweb.com> Message-ID: On 06/10/2013 12:03, Chris Angelico wrote: > On Sun, Oct 6, 2013 at 9:36 PM, Mark Lawrence wrote: >> Also note that a bare except is extremely bad practice, e.g. you can't stop >> rogue programs with a CTRL-C > > Or to be more accurate, a Ctrl-C will cause a jump to your except > clause. Since, in this instance, that's going to emit and die, Ctrl-C > will still stop the program. But yes, catching KeyboardInterrupt > usually isn't your intention. > > ChrisA > Good point, but at least this time I typed "rogue" correctly, unlike on the tutor mailing list :) -- Roses are red, Violets are blue, Most poems rhyme, But this one doesn't. Mark Lawrence From rosuav at gmail.com Sun Oct 6 07:15:12 2013 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 6 Oct 2013 22:15:12 +1100 Subject: How to streamingly read text file and display whenever updated text In-Reply-To: References: <04ee91f9-1cbf-4364-bca3-da25aa4db45f@googlegroups.com> <5250e1d6$0$29984$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Sun, Oct 6, 2013 at 10:13 PM, Mark Lawrence wrote: > Good point, but at least this time I typed "rogue" correctly, unlike on the > tutor mailing list :) Obligatory TVTropes link. http://tvtropes.org/pmwiki/pmwiki.php/Main/RougeAnglesOfSatin ChrisA From breamoreboy at yahoo.co.uk Sun Oct 6 07:31:28 2013 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Sun, 06 Oct 2013 12:31:28 +0100 Subject: How to streamingly read text file and display whenever updated text In-Reply-To: References: <04ee91f9-1cbf-4364-bca3-da25aa4db45f@googlegroups.com> <5250e1d6$0$29984$c3e8da3$5496439d@news.astraweb.com> Message-ID: On 06/10/2013 12:15, Chris Angelico wrote: > On Sun, Oct 6, 2013 at 10:13 PM, Mark Lawrence wrote: >> Good point, but at least this time I typed "rogue" correctly, unlike on the >> tutor mailing list :) > > Obligatory TVTropes link. > > http://tvtropes.org/pmwiki/pmwiki.php/Main/RougeAnglesOfSatin > > ChrisA > How do I know that you're not trying to send me to a rouge site? -- Roses are red, Violets are blue, Most poems rhyme, But this one doesn't. Mark Lawrence From rosuav at gmail.com Sun Oct 6 07:44:03 2013 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 6 Oct 2013 22:44:03 +1100 Subject: How to streamingly read text file and display whenever updated text In-Reply-To: References: <04ee91f9-1cbf-4364-bca3-da25aa4db45f@googlegroups.com> <5250e1d6$0$29984$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Sun, Oct 6, 2013 at 10:31 PM, Mark Lawrence wrote: > > How do I know that you're not trying to send me to a rouge site? > I assure you, the background color is most distinctly white. They probably contract with Google for their white pixel supply: http://www.google.com.au/technology/pigeonrank.html This is, however, quite distinctly off-topic for this list... though Google does use Python extensively, and until not long ago had GvR on their payroll. ChrisA From galeomaga at gmail.com Sun Oct 6 21:54:14 2013 From: galeomaga at gmail.com (galeomaga at gmail.com) Date: Sun, 6 Oct 2013 18:54:14 -0700 (PDT) Subject: How to streamingly read text file and display whenever updated text In-Reply-To: References: <04ee91f9-1cbf-4364-bca3-da25aa4db45f@googlegroups.com> <5250e1d6$0$29984$c3e8da3$5496439d@news.astraweb.com> Message-ID: <65edfb89-52a7-4da3-85af-cc7c45601175@googlegroups.com> https://docs.google.com/file/d/0B2D69u2pweEvelh1T25ra19oZEU/edit?usp=sharing no matter call tail directly in python or using the script of tail all failed it seems it can not read next line From andipersti at gmail.com Mon Oct 7 05:52:25 2013 From: andipersti at gmail.com (Andreas Perstinger) Date: Mon, 07 Oct 2013 11:52:25 +0200 Subject: How to streamingly read text file and display whenever updated text In-Reply-To: <65edfb89-52a7-4da3-85af-cc7c45601175@googlegroups.com> References: <04ee91f9-1cbf-4364-bca3-da25aa4db45f@googlegroups.com> <5250e1d6$0$29984$c3e8da3$5496439d@news.astraweb.com> <65edfb89-52a7-4da3-85af-cc7c45601175@googlegroups.com> Message-ID: <52528459.8050305@gmail.com> On 07.10.2013 03:54, galeomaga at gmail.com wrote: > https://docs.google.com/file/d/0B2D69u2pweEvelh1T25ra19oZEU/edit?usp=sharing > For the readers who don't bother clicking on the link above: It's a short video where the OP demonstrates how her/his usage of tail doesn't work. > no matter call tail directly in python or using the script of tail > all failed > it seems it can not read next line In your video you use gedit to write some file and "tail -f " to follow it. But "tail -f" will follow the file descriptor. Usually, editors like gedit won't save your changes to the original file but create a new temporary file and rename it later to the original file name after deleting the original one. Thus tail will follow an already deleted file. See also this blog post: http://tech.shantanugoel.com/2009/12/23/continuous-monitor-tail-fails.html For your example you will have to use "tail -F " which will follow the file name. Alternatively you could write a simple script to simulate a continously growing file like import time for i in range(1000): with open("test.txt", "a") as f: f.write(str(i) + '\n') time.sleep(1) which should work with "tail -f". Bye, Andreas From nikos.gr33k at gmail.com Sat Oct 5 09:38:14 2013 From: nikos.gr33k at gmail.com (=?UTF-8?B?zp3Or866zr/PgiDOkc67zrXOvs+Mz4DOv8+FzrvOv8+C?=) Date: Sat, 05 Oct 2013 16:38:14 +0300 Subject: Select fails when cookie tried to get a numeric value Message-ID: # initialize cookie cookie = cookies.SimpleCookie( os.environ.get('HTTP_COOKIE') ) cookie.load( cookie ) vip = cookie.get('ID') ....... ....... # if browser cookie does not exist, set it vip = random.randrange(0, 10000) cookie['ID'] = vip cookie['ID']['path'] = '/' # first time visitor on this page, create new record cur.execute('''INSERT INTO visitors (counterID, cookieID, host, city, useros, browser, ref, lastvisit) VALUES (%s, %s, %s, %s, %s, %s, %s, %s)''', (cID, vip, host, city, useros, browser, ref, lastvisit) ) ======================================= The above code i wrote gives me the following error: [Sat Oct 05 13:33:24 2013] [error] [client 108.162.229.114] File "/home/nikos/public_html/cgi-bin/metrites.py", line 209, in [Sat Oct 05 13:33:24 2013] [error] [client 108.162.229.114] cur.execute('''SELECT * FROM visitors WHERE counterID = %s and cookieID = %s''', (cID, vip) ) [Sat Oct 05 13:33:24 2013] [error] [client 108.162.229.114] File "/usr/local/bin/python/lib/python3.3/site-packages/pymysql/cursors.py", line 100, in execute [Sat Oct 05 13:33:24 2013] [error] [client 108.162.229.114] escaped_args = tuple(conn.escape(arg) for arg in args) [Sat Oct 05 13:33:24 2013] [error] [client 108.162.229.114] File "/usr/local/bin/python/lib/python3.3/site-packages/pymysql/cursors.py", line 100, in [Sat Oct 05 13:33:24 2013] [error] [client 108.162.229.114] escaped_args = tuple(conn.escape(arg) for arg in args) [Sat Oct 05 13:33:24 2013] [error] [client 108.162.229.114] File "/usr/local/bin/python/lib/python3.3/site-packages/pymysql/connections.py", line 650, in escape [Sat Oct 05 13:33:24 2013] [error] [client 108.162.229.114] return escape_item(obj, self.charset) [Sat Oct 05 13:33:24 2013] [error] [client 108.162.229.114] File "/usr/local/bin/python/lib/python3.3/site-packages/pymysql/converters.py", line 31, in escape_item [Sat Oct 05 13:33:24 2013] [error] [client 108.162.229.114] encoder = encoders[type(val)] [Sat Oct 05 13:33:24 2013] [error] [client 108.162.229.114] KeyError: What is the nature of the error? ??? I'll iam trying to do is to give a cookie a random number. From ned at nedbatchelder.com Sat Oct 5 09:53:11 2013 From: ned at nedbatchelder.com (Ned Batchelder) Date: Sat, 05 Oct 2013 09:53:11 -0400 Subject: Select fails when cookie tried to get a numeric value In-Reply-To: References: Message-ID: <525019C7.3020004@nedbatchelder.com> On 10/5/13 9:38 AM, ????? ??????????? wrote: > # initialize cookie > cookie = cookies.SimpleCookie( os.environ.get('HTTP_COOKIE') ) > cookie.load( cookie ) > vip = cookie.get('ID') > > ....... > ....... > > # if browser cookie does not exist, set it > vip = random.randrange(0, 10000) > cookie['ID'] = vip > cookie['ID']['path'] = '/' > > # first time visitor on this page, create new record > cur.execute('''INSERT INTO visitors (counterID, cookieID, host, city, > useros, browser, ref, lastvisit) VALUES (%s, %s, %s, %s, %s, %s, %s, > %s)''', (cID, vip, host, city, useros, browser, ref, lastvisit) ) > ======================================= > The above code i wrote gives me the following error: > > > [Sat Oct 05 13:33:24 2013] [error] [client 108.162.229.114] File > "/home/nikos/public_html/cgi-bin/metrites.py", line 209, in > [Sat Oct 05 13:33:24 2013] [error] [client 108.162.229.114] > cur.execute('''SELECT * FROM visitors WHERE counterID = %s and > cookieID = %s''', (cID, vip) ) > [Sat Oct 05 13:33:24 2013] [error] [client 108.162.229.114] File > "/usr/local/bin/python/lib/python3.3/site-packages/pymysql/cursors.py", line > 100, in execute > [Sat Oct 05 13:33:24 2013] [error] [client 108.162.229.114] > escaped_args = tuple(conn.escape(arg) for arg in args) > [Sat Oct 05 13:33:24 2013] [error] [client 108.162.229.114] File > "/usr/local/bin/python/lib/python3.3/site-packages/pymysql/cursors.py", line > 100, in > [Sat Oct 05 13:33:24 2013] [error] [client 108.162.229.114] > escaped_args = tuple(conn.escape(arg) for arg in args) > [Sat Oct 05 13:33:24 2013] [error] [client 108.162.229.114] File > "/usr/local/bin/python/lib/python3.3/site-packages/pymysql/connections.py", > line 650, in escape > [Sat Oct 05 13:33:24 2013] [error] [client 108.162.229.114] return > escape_item(obj, self.charset) > [Sat Oct 05 13:33:24 2013] [error] [client 108.162.229.114] File > "/usr/local/bin/python/lib/python3.3/site-packages/pymysql/converters.py", > line 31, in escape_item > [Sat Oct 05 13:33:24 2013] [error] [client 108.162.229.114] encoder = > encoders[type(val)] > [Sat Oct 05 13:33:24 2013] [error] [client 108.162.229.114] KeyError: > > > What is the nature of the error? > ??? > > I'll iam trying to do is to give a cookie a random number. An important skill to master is reading the traceback for clues. It shows you the lines of code that are involved in the failure. If you read up the stack from the bottom, you can see that bottom four stack frames are in the pymysql package. But the fifth frame is in your code, at metrites.py, like 209, executing a select SQL statement. That's the line to focus on. You haven't shown us that code, but that is where the problem is. From reading the bottom-most frame, you can see that the problem is that "val" is an http.cookies.Morsel object. This means you probably tried to use a cookie object as data in your SQL query, and MySQL doesn't know what to do with that object. You'll have to use a more primitive piece of data in your query. --Ned. From rosuav at gmail.com Sat Oct 5 09:59:47 2013 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 5 Oct 2013 23:59:47 +1000 Subject: Select fails when cookie tried to get a numeric value In-Reply-To: References: Message-ID: On Sat, Oct 5, 2013 at 11:38 PM, ????? ??????????? wrote: > cur.execute('''INSERT INTO visitors (counterID, cookieID, host, city, > useros, browser, ref, lastvisit) VALUES (%s, %s, %s, %s, %s, %s, %s, %s)''', > (cID, vip, host, city, useros, browser, ref, lastvisit) ) > ======================================= > The above code i wrote gives me the following error: > > > [Sat Oct 05 13:33:24 2013] [error] [client 108.162.229.114] File > "/home/nikos/public_html/cgi-bin/metrites.py", line 209, in > [Sat Oct 05 13:33:24 2013] [error] [client 108.162.229.114] > cur.execute('''SELECT * FROM visitors WHERE counterID = %s and cookieID = > %s''', (cID, vip) ) No, I don't think it does give you that error! Nikos, you've been around this group a good while; why can you not learn to read an exception traceback? Find line 209, which (as Ned said) is the one to focus on, and look at it. If you can't figure it out yourself, at the VERY least do us all a courtesy and show us the actual code that's having the problem! ChrisA From nikos.gr33k at gmail.com Sat Oct 5 10:24:17 2013 From: nikos.gr33k at gmail.com (=?UTF-8?B?zp3Or866zr/PgiDOkc67zrXOvs+Mz4DOv8+FzrvOv8+C?=) Date: Sat, 05 Oct 2013 17:24:17 +0300 Subject: Select fails when cookie tried to get a numeric value In-Reply-To: References: Message-ID: ???? 5/10/2013 4:59 ??, ?/? Chris Angelico ??????: > On Sat, Oct 5, 2013 at 11:38 PM, ????? ??????????? > wrote: >> cur.execute('''INSERT INTO visitors (counterID, cookieID, host, city, >> useros, browser, ref, lastvisit) VALUES (%s, %s, %s, %s, %s, %s, %s, %s)''', >> (cID, vip, host, city, useros, browser, ref, lastvisit) ) >> ======================================= >> The above code i wrote gives me the following error: >> >> >> [Sat Oct 05 13:33:24 2013] [error] [client 108.162.229.114] File >> "/home/nikos/public_html/cgi-bin/metrites.py", line 209, in >> [Sat Oct 05 13:33:24 2013] [error] [client 108.162.229.114] >> cur.execute('''SELECT * FROM visitors WHERE counterID = %s and cookieID = >> %s''', (cID, vip) ) > > No, I don't think it does give you that error! Nikos, you've been > around this group a good while; why can you not learn to read an > exception traceback? Find line 209, which (as Ned said) is the one to > focus on, and look at it. If you can't figure it out yourself, at the > VERY least do us all a courtesy and show us the actual code that's > having the problem! But i have given you the line that produces the error: # initialize cookie cookie = cookies.SimpleCookie( os.environ.get('HTTP_COOKIE') ) cookie.load( cookie ) cookieID = cookie.get('ID') # if browser cookie does not exist, set it if not cookieID: message = "??? ?? ??? ??? ???? ??? ?? ????, ??? ?? ????, ??? ?? ??????! ?? ????? ????? ? ??????? ??????????!!" cookie['ID'] = random.randrange(0, 10000) cookie['ID']['path'] = '/' cookie['ID']['expires'] = 60*60*24*365 #this cookie will expire in a month For some reason i think CookieID nevers gets inserted itnot the database that's why mysql's select statemnt fails. When i print CookieID i was under the impression i would see a random number like '5369' but instead it display the follwong. Set-Cookie: ID="Set-Cookie: ID=5369" The mysql filed CookieID is of datatype's int(5) so it cannto store this value. If iam correct and thi is trully the problem then how can i just get the random number part out the whole string? -- What is now proved was at first only imagined! & WebHost From rosuav at gmail.com Sat Oct 5 10:28:09 2013 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 6 Oct 2013 00:28:09 +1000 Subject: Select fails when cookie tried to get a numeric value In-Reply-To: References: Message-ID: On Sun, Oct 6, 2013 at 12:24 AM, ????? ??????????? wrote: > But i have given you the line that produces the error: The statement you quoted is an INSERT. The traceback quotes a SELECT. These are not the same line of code. You still have not shown us the actual line from the traceback. ChrisA From nikos.gr33k at gmail.com Sat Oct 5 10:28:47 2013 From: nikos.gr33k at gmail.com (=?UTF-8?B?zp3Or866zr/PgiDOkc67zrXOvs+Mz4DOv8+FzrvOv8+C?=) Date: Sat, 05 Oct 2013 17:28:47 +0300 Subject: Select fails when cookie tried to get a numeric value In-Reply-To: References: Message-ID: ???? 5/10/2013 5:24 ??, ?/? ????? ??????????? ??????: > ???? 5/10/2013 4:59 ??, ?/? Chris Angelico ??????: >> On Sat, Oct 5, 2013 at 11:38 PM, ????? ??????????? >> wrote: >>> cur.execute('''INSERT INTO visitors (counterID, cookieID, host, city, >>> useros, browser, ref, lastvisit) VALUES (%s, %s, %s, %s, %s, %s, %s, >>> %s)''', >>> (cID, vip, host, city, useros, browser, ref, lastvisit) ) >>> ======================================= >>> The above code i wrote gives me the following error: >>> >>> >>> [Sat Oct 05 13:33:24 2013] [error] [client 108.162.229.114] File >>> "/home/nikos/public_html/cgi-bin/metrites.py", line 209, in >>> [Sat Oct 05 13:33:24 2013] [error] [client 108.162.229.114] >>> cur.execute('''SELECT * FROM visitors WHERE counterID = %s and >>> cookieID = >>> %s''', (cID, vip) ) >> >> No, I don't think it does give you that error! Nikos, you've been >> around this group a good while; why can you not learn to read an >> exception traceback? Find line 209, which (as Ned said) is the one to >> focus on, and look at it. If you can't figure it out yourself, at the >> VERY least do us all a courtesy and show us the actual code that's >> having the problem! > > But i have given you the line that produces the error: > > # initialize cookie > cookie = cookies.SimpleCookie( os.environ.get('HTTP_COOKIE') ) > cookie.load( cookie ) > cookieID = cookie.get('ID') > > # if browser cookie does not exist, set it > if not cookieID: > message = "??? ?? ??? ??? ???? ??? ?? ????, ??? ?? ????, ??? ?? > ??????! ?? ????? ????? ? ??????? ??????????!!" > cookie['ID'] = random.randrange(0, 10000) > cookie['ID']['path'] = '/' > cookie['ID']['expires'] = 60*60*24*365 #this cookie will expire > in a month > > > > For some reason i think CookieID nevers gets inserted itnot the database > that's why mysql's select statemnt fails. > > When i print CookieID i was under the impression i would see a random > number like '5369' but instead it display the follwong. > > Set-Cookie: ID="Set-Cookie: ID=5369" > > The mysql filed CookieID is of datatype's int(5) so it cannto store this > value. > > If iam correct and thi is trully the problem then how can i just get the > random number part out the whole string? The line is that is failign is any line of any insert update, sleect staments that thats cookieID invilded like: [Sat Oct 05 14:26:24 2013] [error] [client 108.162.229.114] File "/home/nikos/public_html/cgi-bin/metrites.py", line 219, in [Sat Oct 05 14:26:24 2013] [error] [client 108.162.229.114] cur.execute('''SELECT * FROM visitors WHERE counterID = %s and cookieID = %s''', (cID, cookieID) ) -- What is now proved was at first only imagined! & WebHost From nikos.gr33k at gmail.com Sat Oct 5 10:30:53 2013 From: nikos.gr33k at gmail.com (=?UTF-8?B?zp3Or866zr/PgiDOkc67zrXOvs+Mz4DOv8+FzrvOv8+C?=) Date: Sat, 05 Oct 2013 17:30:53 +0300 Subject: Select fails when cookie tried to get a numeric value In-Reply-To: References: Message-ID: ???? 5/10/2013 5:28 ??, ?/? Chris Angelico ??????: > On Sun, Oct 6, 2013 at 12:24 AM, ????? ??????????? > wrote: >> But i have given you the line that produces the error: > > The statement you quoted is an INSERT. The traceback quotes a SELECT. > These are not the same line of code. You still have not shown us the > actual line from the traceback. > > ChrisA > Every mysql statemtns that involved cookieID fails. in this example this: # find the visitor record for the (saved) cID and current host cur.execute('''SELECT * FROM visitors WHERE counterID = %s and cookieID = %s''', (cID, cookieID) ) data = cur.fetchone() #cookieID is unique -- What is now proved was at first only imagined! & WebHost From ned at nedbatchelder.com Sat Oct 5 10:43:51 2013 From: ned at nedbatchelder.com (Ned Batchelder) Date: Sat, 05 Oct 2013 10:43:51 -0400 Subject: Select fails when cookie tried to get a numeric value In-Reply-To: References: Message-ID: <525025A7.3050707@nedbatchelder.com> On 10/5/13 10:30 AM, ????? ??????????? wrote: > ???? 5/10/2013 5:28 ??, ?/? Chris Angelico ??????: >> On Sun, Oct 6, 2013 at 12:24 AM, ????? ??????????? >> wrote: >>> But i have given you the line that produces the error: >> >> The statement you quoted is an INSERT. The traceback quotes a SELECT. >> These are not the same line of code. You still have not shown us the >> actual line from the traceback. >> >> ChrisA >> > Every mysql statemtns that involved cookieID fails. > > in this example this: > > # find the visitor record for the (saved) cID and current host > cur.execute('''SELECT * FROM visitors WHERE counterID = %s and > cookieID = %s''', (cID, cookieID) ) > data = cur.fetchone() #cookieID is unique > Nikos, slow down. Don't post three emails before someone has a chance to respond. To get help, you have to show the code that goes along with the traceback. Your subject line even says, "select fails", so you know it is a select statement in the traceback. You have to show us *that code*, and more than one line. You've shown the line here, but we don't know what cID and cookieID are, so we can't help yet. Saying "every mysql statement that involves cookieID fails" isn't enough. Show us the code containing the line that actually is failing in that traceback. Include enough of the code that we can figure out what is going on. You've said that you can do better here. Please try to. --Ned. From nikos.gr33k at gmail.com Sat Oct 5 10:47:54 2013 From: nikos.gr33k at gmail.com (=?UTF-8?B?zp3Or866zr/PgiDOkc67zrXOvs+Mz4DOv8+FzrvOv8+C?=) Date: Sat, 05 Oct 2013 17:47:54 +0300 Subject: Select fails when cookie tried to get a numeric value In-Reply-To: References: Message-ID: ???? 5/10/2013 5:43 ??, ?/? Ned Batchelder ??????: > On 10/5/13 10:30 AM, ????? ??????????? wrote: >> ???? 5/10/2013 5:28 ??, ?/? Chris Angelico ??????: >>> On Sun, Oct 6, 2013 at 12:24 AM, ????? ??????????? >>> wrote: >>>> But i have given you the line that produces the error: >>> >>> The statement you quoted is an INSERT. The traceback quotes a SELECT. >>> These are not the same line of code. You still have not shown us the >>> actual line from the traceback. >>> >>> ChrisA >>> >> Every mysql statemtns that involved cookieID fails. >> >> in this example this: >> >> # find the visitor record for the (saved) cID and current host >> cur.execute('''SELECT * FROM visitors WHERE counterID = %s and >> cookieID = %s''', (cID, cookieID) ) >> data = cur.fetchone() #cookieID is unique >> > > Nikos, slow down. Don't post three emails before someone has a chance > to respond. > > To get help, you have to show the code that goes along with the > traceback. Your subject line even says, "select fails", so you know it > is a select statement in the traceback. You have to show us *that > code*, and more than one line. You've shown the line here, but we don't > know what cID and cookieID are, so we can't help yet. > > Saying "every mysql statement that involves cookieID fails" isn't > enough. Show us the code containing the line that actually is failing > in that traceback. Include enough of the code that we can figure out > what is going on. > > You've said that you can do better here. Please try to. > > --Ned. Before we get to the part that iam actually try to insert, select releveant records the mysql database we need to make sure that CookieID is a number. # initialize cookie cookie = cookies.SimpleCookie( os.environ.get('HTTP_COOKIE') ) cookie.load( cookie ) cookieID = cookie.get('ID') # if browser cookie does not exist, set it if not cookieID: cookie['ID'] = random.randrange(0, 10000) cookie['ID']['path'] = '/' cookie['ID']['expires'] = 60*60*24*365 #this cookie will expire in a month cookieID = cookie.get('ID') print( cookie ) In the above code i try to retrive the cookie form the visitor's browser and if it does nto exist i create one. For some reason i think CookieID nevers gets inserted itnot the database that's why mysql's select statemnt fails. When i print CookieID i was under the impression i would see a random number like '5369' but instead it display the follwong. Set-Cookie: ID="Set-Cookie: ID=5369" The mysql filed CookieID is of datatype's int(5) so it cannto store this value. If iam correct and thi is trully the problem then how can i just get the random number part out the whole string? Do you see something wrong? Why cookie['ID'] retuned this string back and not just the number? If its not a number then it will not be selected or inserted properl into/from MySQL. From z at etiol.net Sat Oct 5 10:44:18 2013 From: z at etiol.net (Zero Piraeus) Date: Sat, 5 Oct 2013 11:44:18 -0300 Subject: Select fails when cookie tried to get a numeric value In-Reply-To: References: Message-ID: <20131005144418.GA2743@piedra> : On Sat, Oct 05, 2013 at 05:30:53PM +0300, ????? ??????????? wrote: > Every mysql statemtns that involved cookieID fails. > > in this example this: > > # find the visitor record for the (saved) cID and current host > cur.execute('''SELECT * FROM visitors WHERE counterID = %s and > cookieID = %s''', (cID, cookieID) ) > data = cur.fetchone() #cookieID is unique If every cur.execute() invocation that you try to pass cookieID to fails, that suggests you can't pass cookieID to cur.execute() ... perhaps because it's the wrong type. The use of '%s' string interpolation suggests that cur.execute() is expecting a string. Is cookieID a string? If not, is it some kind of object that contains a string value within it? Maybe it has some kind of attribute you could pass, like cookieID.value or similar? That may or may not be correct, but it's the kind of mental process you should be going through to solve your problem. -[]z. -- Zero Piraeus: inter caetera http://etiol.net/pubkey.asc From python at mrabarnett.plus.com Sat Oct 5 12:51:27 2013 From: python at mrabarnett.plus.com (MRAB) Date: Sat, 05 Oct 2013 17:51:27 +0100 Subject: Select fails when cookie tried to get a numeric value In-Reply-To: <20131005144418.GA2743@piedra> References: <20131005144418.GA2743@piedra> Message-ID: <5250438F.2020900@mrabarnett.plus.com> On 05/10/2013 15:44, Zero Piraeus wrote: > On Sat, Oct 05, 2013 at 05:30:53PM +0300, ????? ??????????? wrote: >> Every mysql statemtns that involved cookieID fails. >> >> in this example this: >> >> # find the visitor record for the (saved) cID and current host >> cur.execute('''SELECT * FROM visitors WHERE counterID = %s and >> cookieID = %s''', (cID, cookieID) ) >> data = cur.fetchone() #cookieID is unique > > If every cur.execute() invocation that you try to pass cookieID to > fails, that suggests you can't pass cookieID to cur.execute() ... > perhaps because it's the wrong type. > > The use of '%s' string interpolation suggests that cur.execute() is > expecting a string. Is cookieID a string? If not, is it some kind of > object that contains a string value within it? Maybe it has some kind > of attribute you could pass, like cookieID.value or similar? > MySQL string interpolation uses "%s" as the placeholder, but the value doesn't have to be a string. > That may or may not be correct, but it's the kind of mental process > you should be going through to solve your problem. > From andipersti at gmail.com Sat Oct 5 12:56:41 2013 From: andipersti at gmail.com (Andreas Perstinger) Date: Sat, 05 Oct 2013 18:56:41 +0200 Subject: Select fails when cookie tried to get a numeric value In-Reply-To: References: Message-ID: <525044C9.1080502@gmail.com> On 05.10.2013 16:24, ????? ??????????? wrote: > # initialize cookie > cookie = cookies.SimpleCookie( os.environ.get('HTTP_COOKIE') ) > cookie.load( cookie ) Watch: >>> cookie1 = cookies.SimpleCookie('ID=42') >>> cookie1.load(cookie1) >>> print(cookie1) Set-Cookie: ID="Set-Cookie: ID=42" >>> cookie1.get('ID').value 'Set-Cookie: ID=42' And now watch this: >>> cookie2 = cookies.SimpleCookie('ID=42') >>> print(cookie2) Set-Cookie: ID=42 >>> cookie2.get('ID').value '42' Explanation: http://docs.python.org/3/library/http.cookies.html#http.cookies.BaseCookie.load >>> c = cookies.SimpleCookie('ID=42') >>> isinstance(c, dict) True >>> c.items() dict_items([('ID', )]) Bye, Andreas From nikos.gr33k at gmail.com Sat Oct 5 13:38:28 2013 From: nikos.gr33k at gmail.com (=?UTF-8?B?zp3Or866zr/PgiDOkc67zrXOvs+Mz4DOv8+FzrvOv8+C?=) Date: Sat, 05 Oct 2013 20:38:28 +0300 Subject: Select fails when cookie tried to get a numeric value In-Reply-To: References: Message-ID: ???? 5/10/2013 7:56 ??, ?/? Andreas Perstinger ??????: > On 05.10.2013 16:24, ????? ??????????? wrote: >> # initialize cookie >> cookie = cookies.SimpleCookie( os.environ.get('HTTP_COOKIE') ) >> cookie.load( cookie ) > > Watch: > > >>> cookie1 = cookies.SimpleCookie('ID=42') > >>> cookie1.load(cookie1) > >>> print(cookie1) > Set-Cookie: ID="Set-Cookie: ID=42" > >>> cookie1.get('ID').value > 'Set-Cookie: ID=42' > > And now watch this: > > >>> cookie2 = cookies.SimpleCookie('ID=42') > >>> print(cookie2) > Set-Cookie: ID=42 > >>> cookie2.get('ID').value > '42' > > Explanation: > > http://docs.python.org/3/library/http.cookies.html#http.cookies.BaseCookie.load > > > >>> c = cookies.SimpleCookie('ID=42') > >>> isinstance(c, dict) > True > >>> c.items() > dict_items([('ID', )]) > > Bye, Andreas Thank you very much Andreas, it was this strnage behaviour that got me stuch for hours. Now value gets returned properly. -- What is now proved was at first only imagined! & WebHost From rdr.vladimir at gmail.com Sat Oct 5 10:35:58 2013 From: rdr.vladimir at gmail.com (Benjamin Rovny) Date: Sat, 5 Oct 2013 09:35:58 -0500 Subject: Select fails when cookie tried to get a numeric value In-Reply-To: References: Message-ID: On Oct 5, 2013 8:42 AM, "????? ???????????" wrote: > > # initialize cookie > cookie = cookies.SimpleCookie( os.environ.get('HTTP_COOKIE') ) > cookie.load( cookie ) > vip = cookie.get('ID') > > ....... > ....... > > # if browser cookie does not exist, set it > vip = random.randrange(0, 10000) > cookie['ID'] = vip > cookie['ID']['path'] = '/' Problem here? Randrange returns an integer here, which you are then treating like a dictionary (hence "Key Error"). That's the extent of my knowledge though, I don't know about the "cookie" module. > > # first time visitor on this page, create new record > cur.execute('''INSERT INTO visitors (counterID, cookieID, host, city, useros, browser, ref, lastvisit) VALUES (%s, %s, %s, %s, %s, %s, %s, %s)''', (cID, vip, host, city, useros, browser, ref, lastvisit) ) > ======================================= > The above code i wrote gives me the following error: > > > [Sat Oct 05 13:33:24 2013] [error] [client 108.162.229.114] File "/home/nikos/public_html/cgi-bin/metrites.py", line 209, in > [Sat Oct 05 13:33:24 2013] [error] [client 108.162.229.114] cur.execute('''SELECT * FROM visitors WHERE counterID = %s and cookieID = %s''', (cID, vip) ) > [Sat Oct 05 13:33:24 2013] [error] [client 108.162.229.114] File "/usr/local/bin/python/lib/python3.3/site-packages/pymysql/cursors.py", line 100, in execute > [Sat Oct 05 13:33:24 2013] [error] [client 108.162.229.114] escaped_args = tuple(conn.escape(arg) for arg in args) > [Sat Oct 05 13:33:24 2013] [error] [client 108.162.229.114] File "/usr/local/bin/python/lib/python3.3/site-packages/pymysql/cursors.py", line 100, in > [Sat Oct 05 13:33:24 2013] [error] [client 108.162.229.114] escaped_args = tuple(conn.escape(arg) for arg in args) > [Sat Oct 05 13:33:24 2013] [error] [client 108.162.229.114] File "/usr/local/bin/python/lib/python3.3/site-packages/pymysql/connections.py", line 650, in escape > [Sat Oct 05 13:33:24 2013] [error] [client 108.162.229.114] return escape_item(obj, self.charset) > [Sat Oct 05 13:33:24 2013] [error] [client 108.162.229.114] File "/usr/local/bin/python/lib/python3.3/site-packages/pymysql/converters.py", line 31, in escape_item > [Sat Oct 05 13:33:24 2013] [error] [client 108.162.229.114] encoder = encoders[type(val)] > [Sat Oct 05 13:33:24 2013] [error] [client 108.162.229.114] KeyError: > > What is the nature of the error? > ??? > > I'll iam trying to do is to give a cookie a random number. > -- > https://mail.python.org/mailman/listinfo/python-list -------------- next part -------------- An HTML attachment was scrubbed... URL: From nikos.gr33k at gmail.com Sat Oct 5 10:40:23 2013 From: nikos.gr33k at gmail.com (=?UTF-8?B?zp3Or866zr/PgiDOkc67zrXOvs+Mz4DOv8+FzrvOv8+C?=) Date: Sat, 05 Oct 2013 17:40:23 +0300 Subject: Select fails when cookie tried to get a numeric value In-Reply-To: References: Message-ID: ???? 5/10/2013 4:53 ??, ?/? Ned Batchelder ??????: > From reading the bottom-most frame, you can see that the problem is > that "val" is an http.cookies.Morsel object. This means you probably > tried to use a cookie object as data in your SQL query, and MySQL > doesn't know what to do with that object. You'll have to use a more > primitive piece of data in your query. # initialize cookie cookie = cookies.SimpleCookie( os.environ.get('HTTP_COOKIE') ) cookie.load( cookie ) cookieID = cookie.get('ID') # if browser cookie does not exist, set it if not cookieID: cookie['ID'] = random.randrange(0, 10000) cookie['ID']['path'] = '/' cookie['ID']['expires'] = 60*60*24*365 #this cookie will expire in a month cookieID = cookie.get('ID') print( cookie ) In the above code i try to retrive the cookie form the visitor's browser and if it does nto exist i create one. For some reason i think CookieID nevers gets inserted itnot the database that's why mysql's select statemnt fails. When i print CookieID i was under the impression i would see a random number like '5369' but instead it display the follwong. Set-Cookie: ID="Set-Cookie: ID=5369" The mysql filed CookieID is of datatype's int(5) so it cannto store this value. If iam correct and thi is trully the problem then how can i just get the random number part out the whole string? Do you see something wrong? Why cookie['ID'] retuned this string back and not just the number? From ned at nedbatchelder.com Sat Oct 5 10:54:25 2013 From: ned at nedbatchelder.com (Ned Batchelder) Date: Sat, 05 Oct 2013 10:54:25 -0400 Subject: Select fails when cookie tried to get a numeric value In-Reply-To: References: Message-ID: <52502821.3090509@nedbatchelder.com> On 10/5/13 10:40 AM, ????? ??????????? wrote: > ???? 5/10/2013 4:53 ??, ?/? Ned Batchelder ??????: > >> From reading the bottom-most frame, you can see that the problem is >> that "val" is an http.cookies.Morsel object. This means you probably >> tried to use a cookie object as data in your SQL query, and MySQL >> doesn't know what to do with that object. You'll have to use a more >> primitive piece of data in your query. > > # initialize cookie > cookie = cookies.SimpleCookie( os.environ.get('HTTP_COOKIE') ) > cookie.load( cookie ) > cookieID = cookie.get('ID') > > # if browser cookie does not exist, set it > if not cookieID: > cookie['ID'] = random.randrange(0, 10000) > cookie['ID']['path'] = '/' > cookie['ID']['expires'] = 60*60*24*365 #this cookie will > expire in a month > cookieID = cookie.get('ID') > print( cookie ) > > > In the above code i try to retrive the cookie form the visitor's > browser and if it does nto exist i create one. > > > > For some reason i think CookieID nevers gets inserted itnot the > database that's why mysql's select statemnt fails. > > When i print CookieID i was under the impression i would see a random > number like '5369' but instead it display the follwong. > > Set-Cookie: ID="Set-Cookie: ID=5369" > > The mysql filed CookieID is of datatype's int(5) so it cannto store > this value. > > If iam correct and thi is trully the problem then how can i just get > the random number part out the whole string? > > Do you see something wrong? > Why cookie['ID'] retuned this string back and not just the number? > > Nikos: stop sending so many emails. You've asked this question 3 times now in this thread. You aren't even giving people a chance to respond. If you want immediate feedback, use IRC. Email takes a little longer. --Ned. From ned at nedbatchelder.com Sat Oct 5 11:12:42 2013 From: ned at nedbatchelder.com (Ned Batchelder) Date: Sat, 05 Oct 2013 11:12:42 -0400 Subject: Select fails when cookie tried to get a numeric value In-Reply-To: References: Message-ID: <52502C6A.6070900@nedbatchelder.com> On 10/5/13 10:40 AM, ????? ??????????? wrote: > ???? 5/10/2013 4:53 ??, ?/? Ned Batchelder ??????: > >> From reading the bottom-most frame, you can see that the problem is >> that "val" is an http.cookies.Morsel object. This means you probably >> tried to use a cookie object as data in your SQL query, and MySQL >> doesn't know what to do with that object. You'll have to use a more >> primitive piece of data in your query. > > # initialize cookie > cookie = cookies.SimpleCookie( os.environ.get('HTTP_COOKIE') ) > cookie.load( cookie ) > cookieID = cookie.get('ID') > > # if browser cookie does not exist, set it > if not cookieID: > cookie['ID'] = random.randrange(0, 10000) > cookie['ID']['path'] = '/' > cookie['ID']['expires'] = 60*60*24*365 #this cookie will > expire in a month > cookieID = cookie.get('ID') > print( cookie ) > > > In the above code i try to retrive the cookie form the visitor's > browser and if it does nto exist i create one. > > > > For some reason i think CookieID nevers gets inserted itnot the > database that's why mysql's select statemnt fails. > > When i print CookieID i was under the impression i would see a random > number like '5369' but instead it display the follwong. > > Set-Cookie: ID="Set-Cookie: ID=5369" > > The mysql filed CookieID is of datatype's int(5) so it cannto store > this value. > > If iam correct and thi is trully the problem then how can i just get > the random number part out the whole string? > > Do you see something wrong? > Why cookie['ID'] retuned this string back and not just the number? > > Thanks for being patient. Where you have this: cookieID = cookie.get('ID') you actually want this: cookieID = cookie.get('ID').value --Ned. From z at etiol.net Sat Oct 5 11:06:49 2013 From: z at etiol.net (Zero Piraeus) Date: Sat, 5 Oct 2013 12:06:49 -0300 Subject: Select fails when cookie tried to get a numeric value In-Reply-To: References: Message-ID: <20131005150649.GA3393@piedra> : On Sat, Oct 05, 2013 at 05:40:23PM +0300, ????? ??????????? wrote: > When i print CookieID i was under the impression i would see a > random number like '5369' but instead it display the follwong. > > Set-Cookie: ID="Set-Cookie: ID=5369" On Sat, Oct 05, 2013 at 05:47:54PM +0300, ????? ??????????? wrote: > When i print CookieID i was under the impression i would see a > random number like '5369' but instead it display the follwong. > > Set-Cookie: ID="Set-Cookie: ID=5369" Please don't give identical or near-identical replies to multiple messages in the thread; other members of the list are either reading all of your posts or none of them, so repeating yourself like this is only going to irritate whoever is reading. Since printing cookieID doesn't produce the output you expect, the obvious next step is to look up the documentation for whatever kind of object it is. You can find out its type with type(cookieID) ... and then once you know that type (let's say for the sake of argument it's a Biscuit object), find out about that type of object's attributes either by googling for the docs or at the interpreter with help(Biscuit) As previously mentioned, there's likely to be some kind of 'value' attribute that will return just the number you want. -[]z. -- Zero Piraeus: post scriptum http://etiol.net/pubkey.asc From nikos.gr33k at gmail.com Sat Oct 5 11:31:42 2013 From: nikos.gr33k at gmail.com (=?UTF-8?B?zp3Or866zr/PgiDOkc67zrXOvs+Mz4DOv8+FzrvOv8+C?=) Date: Sat, 05 Oct 2013 18:31:42 +0300 Subject: Select fails when cookie tried to get a numeric value In-Reply-To: References: Message-ID: ???? 5/10/2013 6:12 ??, ?/? Ned Batchelder ??????: > On 10/5/13 10:40 AM, ????? ??????????? wrote: >> ???? 5/10/2013 4:53 ??, ?/? Ned Batchelder ??????: >> >>> From reading the bottom-most frame, you can see that the problem is >>> that "val" is an http.cookies.Morsel object. This means you probably >>> tried to use a cookie object as data in your SQL query, and MySQL >>> doesn't know what to do with that object. You'll have to use a more >>> primitive piece of data in your query. >> >> # initialize cookie >> cookie = cookies.SimpleCookie( os.environ.get('HTTP_COOKIE') ) >> cookie.load( cookie ) >> cookieID = cookie.get('ID') >> >> # if browser cookie does not exist, set it >> if not cookieID: >> cookie['ID'] = random.randrange(0, 10000) >> cookie['ID']['path'] = '/' >> cookie['ID']['expires'] = 60*60*24*365 #this cookie will >> expire in a month >> cookieID = cookie.get('ID') >> print( cookie ) >> >> >> In the above code i try to retrive the cookie form the visitor's >> browser and if it does nto exist i create one. >> >> >> >> For some reason i think CookieID nevers gets inserted itnot the >> database that's why mysql's select statemnt fails. >> >> When i print CookieID i was under the impression i would see a random >> number like '5369' but instead it display the follwong. >> >> Set-Cookie: ID="Set-Cookie: ID=5369" >> >> The mysql filed CookieID is of datatype's int(5) so it cannto store >> this value. >> >> If iam correct and thi is trully the problem then how can i just get >> the random number part out the whole string? >> >> Do you see something wrong? >> Why cookie['ID'] retuned this string back and not just the number? >> >> > > Thanks for being patient. Where you have this: > > cookieID = cookie.get('ID') > > you actually want this: > > cookieID = cookie.get('ID').value > > --Ned. Thank you Ned, indeed '.value' needed to just print the number. Now i have it like this: # connect to database con = pymysql.connect( db = 'nikos_metrites', user = 'nikos_root', passwd = 't1abhp2r!', charset = 'utf8', host = 'localhost' ) cur = con.cursor() # initialize cookie and retrieve cookie from clients broswer cookie = cookies.SimpleCookie( os.environ.get('HTTP_COOKIE') ) cookie.load( cookie ) # if browser cookie does not exist, set it if not cookie.get('ID'): cookie['ID'] = random.randrange(0, 10000) cookie['ID']['path'] = '/' cookie['ID']['expires'] = 60*60*24*365 #this cookie will expire in a month print( cookie ) cookieID = cookie['ID'].value # if browser cookie exist, just retrieve it else: cookieID = cookie.get('ID').value and it does not output an error, but for some reason the inserting and selecting never happens. here si the releveant code: if cookieID != 1977 and re.search( r'(msn|gator|amazon|yandex|reverse|who|cloudflare|fetch|barracuda|spider|google|crawl|pingdom)', host ) is None: try: # find the visitor record for the (saved) cID and current host cur.execute('''SELECT * FROM visitors WHERE counterID = %s and cookieID = %s''', (cID, cookieID) ) data = cur.fetchone() #cookieID is unique if not data: # first time visitor on this page, create new record cur.execute('''INSERT INTO visitors (counterID, cookieID, host, city, useros, browser, ref, lastvisit) VALUES (%s, %s, %s, %s, %s, %s, %s, %s)''', (cID, cookieID, host, city, useros, browser, ref, lastvisit) ) else: # found the page, save its primary key for later use vID = data[0] # UPDATE record using retrieved vID cur.execute('''UPDATE visitors SET host = %s, city = %s, useros = %s, browser = %s, ref= %s, hits = hits + 1, lastvisit = %s WHERE counterID = %s and cookieID = %s''', (host, city, useros, browser, ref, lastvisit, vID, cookieID) ) except pymysql.ProgrammingError as e: print( repr(e) ) sys.exit(0) ===================== Any ideas as to what i shoudld try? the statemnt don't return any error back though and the cookieID is indeed now a proper number so i see no reason as to why they fail. -- What is now proved was at first only imagined! & WebHost From andipersti at gmail.com Sat Oct 5 11:47:58 2013 From: andipersti at gmail.com (Andreas Perstinger) Date: Sat, 05 Oct 2013 17:47:58 +0200 Subject: Select fails when cookie tried to get a numeric value In-Reply-To: References: Message-ID: <525034AE.60001@gmail.com> On 05.10.2013 17:31, ????? ??????????? wrote: > Now i have it like this: > > # connect to database > con = pymysql.connect( db = 'nikos_metrites', user = 'nikos_root', > passwd = 't1abhp2r!', charset = 'utf8', host = 'localhost' ) Just to be sure: That's not your real password, is it? Bye, Andreas From rosuav at gmail.com Sat Oct 5 18:36:21 2013 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 6 Oct 2013 09:36:21 +1100 Subject: Select fails when cookie tried to get a numeric value In-Reply-To: References: Message-ID: On Sun, Oct 6, 2013 at 1:31 AM, ????? ??????????? wrote: > # find the visitor record for the (saved) cID and current > host > cur.execute('''SELECT * FROM visitors WHERE counterID = %s > and cookieID = %s''', (cID, cookieID) ) > > data = cur.fetchone() #cookieID is unique > > if not data: > > # first time visitor on this page, create new record > cur.execute('''INSERT INTO visitors (counterID, > cookieID, host, city, useros, browser, ref, lastvisit) VALUES (%s, %s, %s, > %s, %s, %s, %s, %s)''', (cID, cookieID, host, city, useros, browser, ref, > lastvisit) ) > else: > # found the page, save its primary key for later use > vID = data[0] > # UPDATE record using retrieved vID > cur.execute('''UPDATE visitors SET host = %s, city = > %s, useros = %s, browser = %s, ref= %s, hits = hits + 1, lastvisit = %s > > WHERE counterID = %s and cookieID = %s''', (host, city, useros, browser, > ref, lastvisit, vID, cookieID) ) Do you understand the expression "race condition", and how it applies to the above code? If not, you MUST read up on that before relying on code like this, and as that's not a Python question, I recommend Google and Wikipedia.[1] ChrisA [1] Yes, I know they're not primary sources. For something like this, tertiary sources are going to do him fine. From nikos.gr33k at gmail.com Sat Oct 5 11:52:47 2013 From: nikos.gr33k at gmail.com (=?UTF-8?B?zp3Or866zr/PgiDOkc67zrXOvs+Mz4DOv8+FzrvOv8+C?=) Date: Sat, 05 Oct 2013 18:52:47 +0300 Subject: Select fails when cookie tried to get a numeric value In-Reply-To: References: Message-ID: ???? 5/10/2013 6:12 ??, ?/? Ned Batchelder ??????: > On 10/5/13 10:40 AM, ????? ??????????? wrote: >> ???? 5/10/2013 4:53 ??, ?/? Ned Batchelder ??????: >> >>> From reading the bottom-most frame, you can see that the problem is >>> that "val" is an http.cookies.Morsel object. This means you probably >>> tried to use a cookie object as data in your SQL query, and MySQL >>> doesn't know what to do with that object. You'll have to use a more >>> primitive piece of data in your query. >> >> # initialize cookie >> cookie = cookies.SimpleCookie( os.environ.get('HTTP_COOKIE') ) >> cookie.load( cookie ) >> cookieID = cookie.get('ID') >> >> # if browser cookie does not exist, set it >> if not cookieID: >> cookie['ID'] = random.randrange(0, 10000) >> cookie['ID']['path'] = '/' >> cookie['ID']['expires'] = 60*60*24*365 #this cookie will >> expire in a month >> cookieID = cookie.get('ID') >> print( cookie ) >> >> >> In the above code i try to retrive the cookie form the visitor's >> browser and if it does nto exist i create one. >> >> >> >> For some reason i think CookieID nevers gets inserted itnot the >> database that's why mysql's select statemnt fails. >> >> When i print CookieID i was under the impression i would see a random >> number like '5369' but instead it display the follwong. >> >> Set-Cookie: ID="Set-Cookie: ID=5369" >> >> The mysql filed CookieID is of datatype's int(5) so it cannto store >> this value. >> >> If iam correct and thi is trully the problem then how can i just get >> the random number part out the whole string? >> >> Do you see something wrong? >> Why cookie['ID'] retuned this string back and not just the number? >> >> > > Thanks for being patient. Where you have this: > > cookieID = cookie.get('ID') > > you actually want this: > > cookieID = cookie.get('ID').value > > --Ned. [Sat Oct 05 15:51:02 2013] [error] [client 108.162.229.114] File "/home/nikos/public_html/cgi-bin/metrites.py", line 84, in [Sat Oct 05 15:51:02 2013] [error] [client 108.162.229.114] cookieID = cookie.get('ID').value [Sat Oct 05 15:51:02 2013] [error] [client 108.162.229.114] AttributeError: 'NoneType' object has no attribute 'value' -- What is now proved was at first only imagined! & WebHost From ned at nedbatchelder.com Sat Oct 5 12:08:22 2013 From: ned at nedbatchelder.com (Ned Batchelder) Date: Sat, 05 Oct 2013 12:08:22 -0400 Subject: Select fails when cookie tried to get a numeric value In-Reply-To: References: Message-ID: <52503976.1080709@nedbatchelder.com> On 10/5/13 11:52 AM, ????? ??????????? wrote: > ???? 5/10/2013 6:12 ??, ?/? Ned Batchelder ??????: >> On 10/5/13 10:40 AM, ????? ??????????? wrote: >>> ???? 5/10/2013 4:53 ??, ?/? Ned Batchelder ??????: >>> >>>> From reading the bottom-most frame, you can see that the problem is >>>> that "val" is an http.cookies.Morsel object. This means you probably >>>> tried to use a cookie object as data in your SQL query, and MySQL >>>> doesn't know what to do with that object. You'll have to use a more >>>> primitive piece of data in your query. >>> >>> # initialize cookie >>> cookie = cookies.SimpleCookie( os.environ.get('HTTP_COOKIE') ) >>> cookie.load( cookie ) >>> cookieID = cookie.get('ID') >>> >>> # if browser cookie does not exist, set it >>> if not cookieID: >>> cookie['ID'] = random.randrange(0, 10000) >>> cookie['ID']['path'] = '/' >>> cookie['ID']['expires'] = 60*60*24*365 #this cookie will >>> expire in a month >>> cookieID = cookie.get('ID') >>> print( cookie ) >>> >>> >>> In the above code i try to retrive the cookie form the visitor's >>> browser and if it does nto exist i create one. >>> >>> >>> >>> For some reason i think CookieID nevers gets inserted itnot the >>> database that's why mysql's select statemnt fails. >>> >>> When i print CookieID i was under the impression i would see a random >>> number like '5369' but instead it display the follwong. >>> >>> Set-Cookie: ID="Set-Cookie: ID=5369" >>> >>> The mysql filed CookieID is of datatype's int(5) so it cannto store >>> this value. >>> >>> If iam correct and thi is trully the problem then how can i just get >>> the random number part out the whole string? >>> >>> Do you see something wrong? >>> Why cookie['ID'] retuned this string back and not just the number? >>> >>> >> >> Thanks for being patient. Where you have this: >> >> cookieID = cookie.get('ID') >> >> you actually want this: >> >> cookieID = cookie.get('ID').value >> >> --Ned. > > > [Sat Oct 05 15:51:02 2013] [error] [client 108.162.229.114] File > "/home/nikos/public_html/cgi-bin/metrites.py", line 84, in > [Sat Oct 05 15:51:02 2013] [error] [client 108.162.229.114] cookieID = > cookie.get('ID').value > [Sat Oct 05 15:51:02 2013] [error] [client 108.162.229.114] > AttributeError: 'NoneType' object has no attribute 'value' > Nikos: you know enough to understand what is going on here. This list will not serve you well if you take every error message and paste it into an email without trying to get to the bottom of it yourself. At the very least, a Google search on, "AttributeError: 'NoneType' object has no attribute 'value'" will find you some answers. I've said it before, I'll say it again: slow down. --Ned. From nikos.gr33k at gmail.com Sat Oct 5 12:14:00 2013 From: nikos.gr33k at gmail.com (=?UTF-8?B?zp3Or866zr/PgiDOkc67zrXOvs+Mz4DOv8+FzrvOv8+C?=) Date: Sat, 05 Oct 2013 19:14:00 +0300 Subject: Select fails when cookie tried to get a numeric value In-Reply-To: References: Message-ID: ???? 5/10/2013 7:08 ??, ?/? Ned Batchelder ??????: > > On 10/5/13 11:52 AM, ????? ??????????? wrote: >> ???? 5/10/2013 6:12 ??, ?/? Ned Batchelder ??????: >>> On 10/5/13 10:40 AM, ????? ??????????? wrote: >>>> ???? 5/10/2013 4:53 ??, ?/? Ned Batchelder ??????: >>>> >>>>> From reading the bottom-most frame, you can see that the problem is >>>>> that "val" is an http.cookies.Morsel object. This means you probably >>>>> tried to use a cookie object as data in your SQL query, and MySQL >>>>> doesn't know what to do with that object. You'll have to use a more >>>>> primitive piece of data in your query. >>>> >>>> # initialize cookie >>>> cookie = cookies.SimpleCookie( os.environ.get('HTTP_COOKIE') ) >>>> cookie.load( cookie ) >>>> cookieID = cookie.get('ID') >>>> >>>> # if browser cookie does not exist, set it >>>> if not cookieID: >>>> cookie['ID'] = random.randrange(0, 10000) >>>> cookie['ID']['path'] = '/' >>>> cookie['ID']['expires'] = 60*60*24*365 #this cookie will >>>> expire in a month >>>> cookieID = cookie.get('ID') >>>> print( cookie ) >>>> >>>> >>>> In the above code i try to retrive the cookie form the visitor's >>>> browser and if it does nto exist i create one. >>>> >>>> >>>> >>>> For some reason i think CookieID nevers gets inserted itnot the >>>> database that's why mysql's select statemnt fails. >>>> >>>> When i print CookieID i was under the impression i would see a random >>>> number like '5369' but instead it display the follwong. >>>> >>>> Set-Cookie: ID="Set-Cookie: ID=5369" >>>> >>>> The mysql filed CookieID is of datatype's int(5) so it cannto store >>>> this value. >>>> >>>> If iam correct and thi is trully the problem then how can i just get >>>> the random number part out the whole string? >>>> >>>> Do you see something wrong? >>>> Why cookie['ID'] retuned this string back and not just the number? >>>> >>>> >>> >>> Thanks for being patient. Where you have this: >>> >>> cookieID = cookie.get('ID') >>> >>> you actually want this: >>> >>> cookieID = cookie.get('ID').value >>> >>> --Ned. >> >> >> [Sat Oct 05 15:51:02 2013] [error] [client 108.162.229.114] File >> "/home/nikos/public_html/cgi-bin/metrites.py", line 84, in >> [Sat Oct 05 15:51:02 2013] [error] [client 108.162.229.114] cookieID = >> cookie.get('ID').value >> [Sat Oct 05 15:51:02 2013] [error] [client 108.162.229.114] >> AttributeError: 'NoneType' object has no attribute 'value' >> > > Nikos: you know enough to understand what is going on here. > > This list will not serve you well if you take every error message and > paste it into an email without trying to get to the bottom of it > yourself. At the very least, a Google search on, "AttributeError: > 'NoneType' object has no attribute 'value'" will find you some answers. > > I've said it before, I'll say it again: slow down. > > --Ned. cookieID = cookie.get('ID').value is not returning what you said it will return and if cookie.get('ID') doenst exist it returns the error AttributeError: 'NoneType' object has no attribute 'value' These are 2 problem. value aint being returned thw ehole Set-Cookie: ID=some_number is being returned instead as tou cna see at http://superhost.gr/ and the second problem is that if the cookie dosnt exist i get the error of: AttributeError: 'NoneType' object has no attribute 'value' whne this line is tryign to be executed: cookieID = cookie.get('ID').value How can i deal with thse 2 problems? -- What is now proved was at first only imagined! & WebHost From nikos.gr33k at gmail.com Sat Oct 5 12:17:12 2013 From: nikos.gr33k at gmail.com (=?UTF-8?B?zp3Or866zr/PgiDOkc67zrXOvs+Mz4DOv8+FzrvOv8+C?=) Date: Sat, 05 Oct 2013 19:17:12 +0300 Subject: Select fails when cookie tried to get a numeric value In-Reply-To: References: Message-ID: ???? 5/10/2013 7:14 ??, ?/? ????? ??????????? ??????: > ???? 5/10/2013 7:08 ??, ?/? Ned Batchelder ??????: >> >> On 10/5/13 11:52 AM, ????? ??????????? wrote: >>> ???? 5/10/2013 6:12 ??, ?/? Ned Batchelder ??????: >>>> On 10/5/13 10:40 AM, ????? ??????????? wrote: >>>>> ???? 5/10/2013 4:53 ??, ?/? Ned Batchelder ??????: >>>>> >>>>>> From reading the bottom-most frame, you can see that the problem is >>>>>> that "val" is an http.cookies.Morsel object. This means you probably >>>>>> tried to use a cookie object as data in your SQL query, and MySQL >>>>>> doesn't know what to do with that object. You'll have to use a more >>>>>> primitive piece of data in your query. >>>>> >>>>> # initialize cookie >>>>> cookie = cookies.SimpleCookie( os.environ.get('HTTP_COOKIE') ) >>>>> cookie.load( cookie ) >>>>> cookieID = cookie.get('ID') >>>>> >>>>> # if browser cookie does not exist, set it >>>>> if not cookieID: >>>>> cookie['ID'] = random.randrange(0, 10000) >>>>> cookie['ID']['path'] = '/' >>>>> cookie['ID']['expires'] = 60*60*24*365 #this cookie will >>>>> expire in a month >>>>> cookieID = cookie.get('ID') >>>>> print( cookie ) >>>>> >>>>> >>>>> In the above code i try to retrive the cookie form the visitor's >>>>> browser and if it does nto exist i create one. >>>>> >>>>> >>>>> >>>>> For some reason i think CookieID nevers gets inserted itnot the >>>>> database that's why mysql's select statemnt fails. >>>>> >>>>> When i print CookieID i was under the impression i would see a random >>>>> number like '5369' but instead it display the follwong. >>>>> >>>>> Set-Cookie: ID="Set-Cookie: ID=5369" >>>>> >>>>> The mysql filed CookieID is of datatype's int(5) so it cannto store >>>>> this value. >>>>> >>>>> If iam correct and thi is trully the problem then how can i just get >>>>> the random number part out the whole string? >>>>> >>>>> Do you see something wrong? >>>>> Why cookie['ID'] retuned this string back and not just the number? >>>>> >>>>> >>>> >>>> Thanks for being patient. Where you have this: >>>> >>>> cookieID = cookie.get('ID') >>>> >>>> you actually want this: >>>> >>>> cookieID = cookie.get('ID').value >>>> >>>> --Ned. >>> >>> >>> [Sat Oct 05 15:51:02 2013] [error] [client 108.162.229.114] File >>> "/home/nikos/public_html/cgi-bin/metrites.py", line 84, in >>> [Sat Oct 05 15:51:02 2013] [error] [client 108.162.229.114] cookieID = >>> cookie.get('ID').value >>> [Sat Oct 05 15:51:02 2013] [error] [client 108.162.229.114] >>> AttributeError: 'NoneType' object has no attribute 'value' >>> >> >> Nikos: you know enough to understand what is going on here. >> >> This list will not serve you well if you take every error message and >> paste it into an email without trying to get to the bottom of it >> yourself. At the very least, a Google search on, "AttributeError: >> 'NoneType' object has no attribute 'value'" will find you some answers. >> >> I've said it before, I'll say it again: slow down. >> >> --Ned. > > > cookieID = cookie.get('ID').value > > is not returning what you said it will return > > and if cookie.get('ID') doenst exist it returns the error > AttributeError: 'NoneType' object has no attribute 'value' > > These are 2 problem. > > value aint being returned thw ehole Set-Cookie: ID=some_number is being > returned instead as tou cna see at http://superhost.gr/ > > and the second problem is > > that if the cookie dosnt exist i get the error of: AttributeError: > 'NoneType' object has no attribute 'value' > > whne this line is tryign to be executed: > cookieID = cookie.get('ID').value > > How can i deal with thse 2 problems? > The best solution i cna think of is put the whole thing into a try: block try: cookieID = cookie.get('ID').value except: cookie['ID'] = random.randrange(0, 10000) cookie['ID']['path'] = '/' print( cookie ) cookieID = cookie['ID'].value print( '''Content-type: text/html; charset=utf-8\n''' ) print( cookieID ) sys.exit(0) That will avoid the NoneType errot but: that still print out: Set-Cookie: ID=7413 instead of just the number -- What is now proved was at first only imagined! & WebHost From ned at nedbatchelder.com Sat Oct 5 12:42:21 2013 From: ned at nedbatchelder.com (Ned Batchelder) Date: Sat, 05 Oct 2013 12:42:21 -0400 Subject: Select fails when cookie tried to get a numeric value In-Reply-To: References: Message-ID: <5250416D.3090705@nedbatchelder.com> On 10/5/13 12:17 PM, ????? ??????????? wrote: > ???? 5/10/2013 7:14 ??, ?/? ????? ??????????? ??????: >> ???? 5/10/2013 7:08 ??, ?/? Ned Batchelder ??????: >>> >>> On 10/5/13 11:52 AM, ????? ??????????? wrote: >>>> ???? 5/10/2013 6:12 ??, ?/? Ned Batchelder ??????: >>>>> On 10/5/13 10:40 AM, ????? ??????????? wrote: >>>>>> ???? 5/10/2013 4:53 ??, ?/? Ned Batchelder ??????: >>>>>> >>>>>>> From reading the bottom-most frame, you can see that the >>>>>>> problem is >>>>>>> that "val" is an http.cookies.Morsel object. This means you >>>>>>> probably >>>>>>> tried to use a cookie object as data in your SQL query, and MySQL >>>>>>> doesn't know what to do with that object. You'll have to use a >>>>>>> more >>>>>>> primitive piece of data in your query. >>>>>> >>>>>> # initialize cookie >>>>>> cookie = cookies.SimpleCookie( os.environ.get('HTTP_COOKIE') ) >>>>>> cookie.load( cookie ) >>>>>> cookieID = cookie.get('ID') >>>>>> >>>>>> # if browser cookie does not exist, set it >>>>>> if not cookieID: >>>>>> cookie['ID'] = random.randrange(0, 10000) >>>>>> cookie['ID']['path'] = '/' >>>>>> cookie['ID']['expires'] = 60*60*24*365 #this cookie will >>>>>> expire in a month >>>>>> cookieID = cookie.get('ID') >>>>>> print( cookie ) >>>>>> >>>>>> >>>>>> In the above code i try to retrive the cookie form the visitor's >>>>>> browser and if it does nto exist i create one. >>>>>> >>>>>> >>>>>> >>>>>> For some reason i think CookieID nevers gets inserted itnot the >>>>>> database that's why mysql's select statemnt fails. >>>>>> >>>>>> When i print CookieID i was under the impression i would see a >>>>>> random >>>>>> number like '5369' but instead it display the follwong. >>>>>> >>>>>> Set-Cookie: ID="Set-Cookie: ID=5369" >>>>>> >>>>>> The mysql filed CookieID is of datatype's int(5) so it cannto store >>>>>> this value. >>>>>> >>>>>> If iam correct and thi is trully the problem then how can i just get >>>>>> the random number part out the whole string? >>>>>> >>>>>> Do you see something wrong? >>>>>> Why cookie['ID'] retuned this string back and not just the number? >>>>>> >>>>>> >>>>> >>>>> Thanks for being patient. Where you have this: >>>>> >>>>> cookieID = cookie.get('ID') >>>>> >>>>> you actually want this: >>>>> >>>>> cookieID = cookie.get('ID').value >>>>> >>>>> --Ned. >>>> >>>> >>>> [Sat Oct 05 15:51:02 2013] [error] [client 108.162.229.114] File >>>> "/home/nikos/public_html/cgi-bin/metrites.py", line 84, in >>>> [Sat Oct 05 15:51:02 2013] [error] [client 108.162.229.114] cookieID = >>>> cookie.get('ID').value >>>> [Sat Oct 05 15:51:02 2013] [error] [client 108.162.229.114] >>>> AttributeError: 'NoneType' object has no attribute 'value' >>>> >>> >>> Nikos: you know enough to understand what is going on here. >>> >>> This list will not serve you well if you take every error message and >>> paste it into an email without trying to get to the bottom of it >>> yourself. At the very least, a Google search on, "AttributeError: >>> 'NoneType' object has no attribute 'value'" will find you some answers. >>> >>> I've said it before, I'll say it again: slow down. >>> >>> --Ned. >> >> >> cookieID = cookie.get('ID').value >> >> is not returning what you said it will return >> >> and if cookie.get('ID') doenst exist it returns the error >> AttributeError: 'NoneType' object has no attribute 'value' >> >> These are 2 problem. >> >> value aint being returned thw ehole Set-Cookie: ID=some_number is being >> returned instead as tou cna see at http://superhost.gr/ >> >> and the second problem is >> >> that if the cookie dosnt exist i get the error of: AttributeError: >> 'NoneType' object has no attribute 'value' >> >> whne this line is tryign to be executed: >> cookieID = cookie.get('ID').value >> >> How can i deal with thse 2 problems? >> > The best solution i cna think of is put the whole thing into a try: block > > try: > cookieID = cookie.get('ID').value > except: > cookie['ID'] = random.randrange(0, 10000) > cookie['ID']['path'] = '/' > print( cookie ) > cookieID = cookie['ID'].value > > print( '''Content-type: text/html; charset=utf-8\n''' ) > > print( cookieID ) > sys.exit(0) > > That will avoid the NoneType errot but: > > that still print out: > Set-Cookie: ID=7413 > > instead of just the number > Nikos, you are now answering your own emails. You are going too fast. Slow down, think through a solution before writing another email. And seriously, consider IRC, you will be able to have a conversation with someone. The email pace doesn't suit you. A better solution is to check to see if you got None: if cookie.get('ID') is None: # make a new cookie.... --Ned. From nikos.gr33k at gmail.com Sat Oct 5 12:53:05 2013 From: nikos.gr33k at gmail.com (=?UTF-8?B?zp3Or866zr/PgiDOkc67zrXOvs+Mz4DOv8+FzrvOv8+C?=) Date: Sat, 05 Oct 2013 19:53:05 +0300 Subject: Select fails when cookie tried to get a numeric value In-Reply-To: References: Message-ID: ???? 5/10/2013 7:42 ??, ?/? Ned Batchelder ??????: > > A better solution is to check to see if you got None: > > if cookie.get('ID') is None: > # make a new cookie.... I have tried everythign even wgat you suggested right now: here is is: # initialize cookie and retrieve cookie from clients broswer cookie = cookies.SimpleCookie( os.environ.get('HTTP_COOKIE') ) cookie.load( cookie ) if cookie.get('ID') is not None: cookieID = cookie['ID'].value else: x = random.randrange(0, 10000) cookie['ID'] = x cookie['ID']['path'] = '/' print( cookie ) cookieID = x print( '''Content-type: text/html; charset=utf-8\n''' ) print( cookieID ) For some reason althogh the cookie does exist in my browser the returnd value of cookieI D = cookie['ID'].value is always: Set-Cookie: ID=7482 instead of just the number. But why? isnt value to return just the number? You can see the result of the above code yourself at the top left when you visit: http://superhost.gr it always retursn the whole string instead of just the number... why can we isolate th damn number only? in the else i manages to isolate it but not when i try to retrive it. From nikos.gr33k at gmail.com Sat Oct 5 11:50:00 2013 From: nikos.gr33k at gmail.com (=?UTF-8?B?zp3Or866zr/PgiDOkc67zrXOvs+Mz4DOv8+FzrvOv8+C?=) Date: Sat, 05 Oct 2013 18:50:00 +0300 Subject: Select fails when cookie tried to get a numeric value In-Reply-To: References: Message-ID: ???? 5/10/2013 6:06 ??, ?/? Zero Piraeus ??????: > : > > On Sat, Oct 05, 2013 at 05:40:23PM +0300, ????? ??????????? wrote: >> When i print CookieID i was under the impression i would see a >> random number like '5369' but instead it display the follwong. >> >> Set-Cookie: ID="Set-Cookie: ID=5369" > > On Sat, Oct 05, 2013 at 05:47:54PM +0300, ????? ??????????? wrote: >> When i print CookieID i was under the impression i would see a >> random number like '5369' but instead it display the follwong. >> >> Set-Cookie: ID="Set-Cookie: ID=5369" > > Please don't give identical or near-identical replies to multiple > messages in the thread; other members of the list are either reading all > of your posts or none of them, so repeating yourself like this is only > going to irritate whoever is reading. > > Since printing cookieID doesn't produce the output you expect, the > obvious next step is to look up the documentation for whatever kind of > object it is. You can find out its type with > > type(cookieID) > > ... and then once you know that type (let's say for the sake of argument > it's a Biscuit object), find out about that type of object's attributes > either by googling for the docs or at the interpreter with > > help(Biscuit) > > As previously mentioned, there's likely to be some kind of 'value' > attribute that will return just the number you want. nikos at superhost.gr [~/www/cgi-bin]# python Python 3.3.2 (default, Aug 26 2013, 06:41:42) [GCC 4.4.7 20120313 (Red Hat 4.4.7-3)] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import os, random >>> from http import cookies >>> cookie = cookies.SimpleCookie( os.environ.get('HTTP_COOKIE') ) cookie.load( cookie ) >>> cookieID = cookie.get('ID').value Traceback (most recent call last): File "", line 1, in AttributeError: 'NoneType' object has no attribute 'value' And if you go to my webpage http://superhost.gr at the top corner you see that allthough i use this code to get the value of the retrieved cookie or set the value if ti do # initialize cookie and retrieve cookie from clients broswer cookie = cookies.SimpleCookie( os.environ.get('HTTP_COOKIE') ) cookie.load( cookie ) cookieID = cookie.get('ID').value # if browser cookie does not exist, set it if not cookieID: cookie['ID'] = random.randrange(0, 10000) cookie['ID']['path'] = '/' cookie['ID']['expires'] = 60*60*24*365 #this cookie will expire in a month print( cookie ) cookieID = cookie['ID'].value print( '''Content-type: text/html; charset=utf-8\n''' ) print( cookieID ) sys.exit(0) The output is: Set-Cookie: ID=1376 But how is this possible since we applied the .value attribute in the cookie? From tjreedy at udel.edu Sat Oct 5 18:06:44 2013 From: tjreedy at udel.edu (Terry Reedy) Date: Sat, 05 Oct 2013 18:06:44 -0400 Subject: Select fails when cookie tried to get a numeric value In-Reply-To: References: Message-ID: On 10/5/2013 9:38 AM, ????? ??????????? wrote: > # initialize cookie > cookie = cookies.SimpleCookie( os.environ.get('HTTP_COOKIE') ) > cookie.load( cookie ) > vip = cookie.get('ID') > > ....... > ....... > > # if browser cookie does not exist, set it > vip = random.randrange(0, 10000) > cookie['ID'] = vip > cookie['ID']['path'] = '/' > > # first time visitor on this page, create new record > cur.execute('''INSERT INTO visitors (counterID, cookieID, host, city, > useros, browser, ref, lastvisit) VALUES (%s, %s, %s, %s, %s, %s, %s, > %s)''', (cID, vip, host, city, useros, browser, ref, lastvisit) ) > ======================================= > The above code i wrote gives me the following error: > > > [Sat Oct 05 13:33:24 2013] [error] [client 108.162.229.114] File > "/home/nikos/public_html/cgi-bin/metrites.py", line 209, in The prefix added to every line of the traceback by the logging process [Sat Oct 05 13:33:24 2013] [error] [client 108.162.229.114] makes the traceback much harder to read. I suggest removing it before posting. This is easy with a global replace with nothing. If you control the logging and can have the prefix printed just once, even better. -- Terry Jan Reedy From denismfmcmahon at gmail.com Sat Oct 5 19:36:14 2013 From: denismfmcmahon at gmail.com (Denis McMahon) Date: Sat, 5 Oct 2013 23:36:14 +0000 (UTC) Subject: Select fails when cookie tried to get a numeric value References: Message-ID: On Sat, 05 Oct 2013 16:38:14 +0300, ????? ??????????? wrote: > [my cookie code is fucked] Hi Nick I had never used python for www before yesterday. I had never used python before about 6 months ago. In the last 24 hours I have installed mod-wsgi on my apache web server, and written test code that enables me to see the result of all environment data, parse the get string, parse post data, set and parse cookies, and save session data to a file specific to the cookie data. There may be better ways to do some of it, but it works. Here is a link to the text of my python file: http://www.sined.co.uk/tmp/index.py.txt Once you have read through it (there are comments) and understood how it handles cookies and session data, you will realise that you can add and modify any session data just by adding relevant items to and reading them from the session data dictionary. The session data stays on the server, the cookie simply identifies the session (and the session data file), and hopefully is unique to the user. Don't ask me any questions about why I did something the way I did it. I did it that way because it works. If anyone knows a different way that also works and maybe works better, feel free to discuss it. But Nick, don't you dare suggest any change that doesn't work because you think it looks better. If you don't understand a module function that I've used, read the python documentation for it. If you think you have a question that is not covered by the above statements, feel free to ask me, here, why I wrote a given line number or group of line numbers the way I did. However, see the notes above - the answer will probably be "because it works that way". If you quote the whole file or even large chunks of it here, I am finished with trying to help you, ever! -- Denis McMahon, denismfmcmahon at gmail.com From nikos.gr33k at gmail.com Sun Oct 6 01:20:35 2013 From: nikos.gr33k at gmail.com (=?UTF-8?B?zp3Or866zr/PgiDOkc67zrXOvs+Mz4DOv8+FzrvOv8+C?=) Date: Sun, 06 Oct 2013 08:20:35 +0300 Subject: Select fails when cookie tried to get a numeric value In-Reply-To: References: Message-ID: ???? 6/10/2013 2:36 ??, ?/? Denis McMahon ??????: > On Sat, 05 Oct 2013 16:38:14 +0300, ????? ??????????? wrote: > >> [my cookie code is fucked] > > Hi Nick > > I had never used python for www before yesterday. I had never used python > before about 6 months ago. > > In the last 24 hours I have installed mod-wsgi on my apache web server, > and written test code that enables me to see the result of all > environment data, parse the get string, parse post data, set and parse > cookies, and save session data to a file specific to the cookie data. > > There may be better ways to do some of it, but it works. > > Here is a link to the text of my python file: > > http://www.sined.co.uk/tmp/index.py.txt > > Once you have read through it (there are comments) and understood how it > handles cookies and session data, you will realise that you can add and > modify any session data just by adding relevant items to and reading them > from the session data dictionary. The session data stays on the server, > the cookie simply identifies the session (and the session data file), and > hopefully is unique to the user. > > Don't ask me any questions about why I did something the way I did it. I > did it that way because it works. If anyone knows a different way that > also works and maybe works better, feel free to discuss it. But Nick, > don't you dare suggest any change that doesn't work because you think it > looks better. > > If you don't understand a module function that I've used, read the python > documentation for it. > > If you think you have a question that is not covered by the above > statements, feel free to ask me, here, why I wrote a given line number or > group of line numbers the way I did. However, see the notes above - the > answer will probably be "because it works that way". > > If you quote the whole file or even large chunks of it here, I am > finished with trying to help you, ever! Thank you Denis, i didn't knew about sessions up until i saw you post. Your code is very advanced for me to read but i will try to "decode it" I though that if we want to read something from our visitor, something we want, we have to save it to his browser a cookie, that we later retrieve within our python script's code. What "sessions" do more compared to just using cookies? i use 'cgi', but i noticed you used 'mod-wsgi'. I want to ask you why you chose the latter? Is the latter better than the former? Is it faster? Does it bahave the same way? I my code works works with cgi, which it does, will the same code works with mod-wcgi? -- What is now proved was at first only imagined! & WebHost From denismfmcmahon at gmail.com Sun Oct 6 10:57:46 2013 From: denismfmcmahon at gmail.com (Denis McMahon) Date: Sun, 6 Oct 2013 14:57:46 +0000 (UTC) Subject: Select fails when cookie tried to get a numeric value References: Message-ID: On Sun, 06 Oct 2013 08:20:35 +0300, ????? ??????????? wrote: > Thank you Denis, i didn't knew about sessions up until i saw you post. > Your code is very advanced for me to read but i will try to "decode it" > I though that if we want to read something from our visitor, something > we want, we have to save it to his browser a cookie, that we later > retrieve within our python script's code. > > What "sessions" do more compared to just using cookies? The concept of a session is that you preserve data in the server between multiple page requests from the same client. You do this by issuing a session id to the client in a cookie, and using the session id as a key to identify that user's data in the server. I use the session id as a filename in the tmp files dir on the server, and in the file I store a pickled dictionary that contains all the session data for the user. It should be just as easy to do this using a database with three fields, the session id, the last used time, and a binary object (or a string) to hold the pickled data. You could use json instead of pickle, that just creates a string. The big advantage of a session is that you keep the data on the server, and only send the session id to the browser. This means that the user can not manipulate the session data. If you don't understand why this is important, you need to stop coding websites until you do understand why this is important. As the importance of this is not a python issue, it is not appropriate to discuss here. > i use 'cgi', but i noticed you used 'mod-wsgi'. > I want to ask you why you chose the latter? Is the latter better than > the former? It was a choice between mod-python and mod-wsgi. I chose mod-wsgi. I think I felt that it was better supported in apache than mod-python, and it seemed to be better thought of in various forums. > Is it faster? Does it bahave the same way? I have no idea. I prefer using specific apache modules for server side scripting where possible, rather than cgi, as I find that they are generally less cpu and memory intensive than using the cgi interface. Although my compiled fortran, ada, basic, pascal and c modules still use mod-cgi.[1] > I my code works works with cgi, which it does, will the same code works > with mod-wcgi? I have no idea. Also, what is mod-wcgi anyway? [1] I got bored one weekend and decided to write basic form handling in several of the languages I knew just to prove I could do it. -- Denis McMahon, denismfmcmahon at gmail.com From piet at vanoostrum.org Sun Oct 6 22:16:49 2013 From: piet at vanoostrum.org (Piet van Oostrum) Date: Sun, 06 Oct 2013 22:16:49 -0400 Subject: Select fails when cookie tried to get a numeric value References: Message-ID: ????? ??????????? writes: > i use 'cgi', but i noticed you used 'mod-wsgi'. > I want to ask you why you chose the latter? Is the latter better than > the former? > > Is it faster? Does it bahave the same way? > I my code works works with cgi, which it does, will the same code works > with mod-wcgi? You probably mean mod-wsgi. It is faster than CGI because it doesn't have to start a new Python interpreter for each request. And the protocol it uses is different so you should rewrite your scripts if you want to use WSGI. There are bridges to use your CGI script within WSGI, but it needs care; you cannot use it for every CGI script - your script has to work in a certain way (like being careful with initializing everything everytime it runs). That said, you can use both at the same time in a web server: some CGI scripts and some WSGI, and convert them one after one if you think it is necessary. For example a script that is seldom used can stay as CGI script. And last but not least: not every Apache installation has mod-wsgi enabled. I think most hosting providers don't. -- Piet van Oostrum WWW: http://pietvanoostrum.com/ PGP key: [8DAE142BE17999C4] From nikos.gr33k at gmail.com Sat Oct 5 15:16:46 2013 From: nikos.gr33k at gmail.com (=?UTF-8?B?zp3Or866zr/PgiDOkc67zrXOvs+Mz4DOv8+FzrvOv8+C?=) Date: Sat, 05 Oct 2013 22:16:46 +0300 Subject: Database statements via python but database left intact Message-ID: Excuse me for asking again today, but i see no error in the following code, yes no isertion or update happens into the database: try: # locate the ID of the page's URL cur.execute('''SELECT ID FROM counters WHERE url = %s''', page ) data = cur.fetchone() #URL is unique, so should only be one if not data: #first time for page; primary key is automatic, hit is defaulted cur.execute('''INSERT INTO counters (url) VALUES (%s)''', page ) cID = cur.lastrowid #get the primary key value of the new added record else: #found the page, save primary key and use it to issue hit UPDATE cID = data[0] cur.execute('''UPDATE counters SET hits = hits + 1 WHERE ID = %s''', cID ) When this code runs i check instantly my database via PHPMyAdmin and i see that it was left intact. From nikos.gr33k at gmail.com Sat Oct 5 15:27:49 2013 From: nikos.gr33k at gmail.com (=?UTF-8?B?zp3Or866zr/PgiDOkc67zrXOvs+Mz4DOv8+FzrvOv8+C?=) Date: Sat, 05 Oct 2013 22:27:49 +0300 Subject: Database statements via python but database left intact In-Reply-To: References: Message-ID: Actually the whole code is this: # ================================================================================================================= # DATABASE INSERTS - # ================================================================================================================= if cookieID != 1977 and re.search( r'(msn|gator|amazon|yandex|reverse|who|cloudflare|fetch|barracuda|spider|google|crawl|pingdom)', host ) is None: try: # locate the ID of the page's URL cur.execute('''SELECT ID FROM counters WHERE url = %s''', page ) data = cur.fetchone() #URL is unique, so should only be one if not data: #first time for page; primary key is automatic, hit is defaulted cur.execute('''INSERT INTO counters (url) VALUES (%s)''', page ) cID = cur.lastrowid #get the primary key value of the new added record else: #found the page, save primary key and use it to issue hit UPDATE cID = data[0] cur.execute('''UPDATE counters SET hits = hits + 1 WHERE ID = %s''', cID ) # find the visitor record for the (saved) cID and Cookie cur.execute('''SELECT * FROM visitors WHERE counterID = %s and cookieID = %s''', (cID, cookieID) ) data = cur.fetchone() #cookieID is unique if not data: # first time visitor on this page, create new record cur.execute('''INSERT INTO visitors (counterID, cookieID, host, city, useros, browser, ref, lastvisit) VALUES (%s, %s, %s, %s, %s, %s, %s, %s)''', (cID, cookieID, host, city, useros, browser, ref, lastvisit) ) else: # found the page, save its primary key for later use vID = data[0] # UPDATE record using retrieved vID cur.execute('''UPDATE visitors SET host = %s, city = %s, useros = %s, browser = %s, ref= %s, hits = hits + 1, lastvisit = %s WHERE counterID = %s and cookieID = %s''', (host, city, useros, browser, ref, lastvisit, vID, cookieID) ) except pymysql.ProgrammingError as e: print( repr(e) ) sys.exit(0) ======================== If at some point an error is made does that mean that no update/insertion happens? PEhats that is why iam seeing no entries at all into my database tables? From z at etiol.net Sat Oct 5 15:29:55 2013 From: z at etiol.net (Zero Piraeus) Date: Sat, 5 Oct 2013 16:29:55 -0300 Subject: Database statements via python but database left intact In-Reply-To: References: Message-ID: <20131005192955.GA26771@piedra> : On Sat, Oct 05, 2013 at 10:16:46PM +0300, ????? ??????????? wrote: > Excuse me for asking again today, but i see no error in the > following code, yes no isertion or update happens into the database: > > [...] > > When this code runs i check instantly my database via PHPMyAdmin and > i see that it was left intact. Are you sure that you're committing your changes (either by having autocommit set or using an explicit con.commit() call)? http://geert.vanderkelen.org/dont-forget-the-commit-in-mysql/ -[]z. -- Zero Piraeus: inter caetera http://etiol.net/pubkey.asc From nikos.gr33k at gmail.com Sat Oct 5 15:36:19 2013 From: nikos.gr33k at gmail.com (=?UTF-8?B?zp3Or866zr/PgiDOkc67zrXOvs+Mz4DOv8+FzrvOv8+C?=) Date: Sat, 05 Oct 2013 22:36:19 +0300 Subject: Database statements via python but database left intact In-Reply-To: References: Message-ID: ???? 5/10/2013 10:29 ??, ?/? Zero Piraeus ??????: > : > > On Sat, Oct 05, 2013 at 10:16:46PM +0300, ????? ??????????? wrote: >> Excuse me for asking again today, but i see no error in the >> following code, yes no isertion or update happens into the database: >> >> [...] >> >> When this code runs i check instantly my database via PHPMyAdmin and >> i see that it was left intact. > > Are you sure that you're committing your changes (either by having > autocommit set or using an explicit con.commit() call)? > > http://geert.vanderkelen.org/dont-forget-the-commit-in-mysql/ I dont think that is the issue, because up until now i never used commit and all transaction were successfully were happening. -- What is now proved was at first only imagined! & WebHost From ian.g.kelly at gmail.com Sat Oct 5 16:31:04 2013 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Sat, 5 Oct 2013 14:31:04 -0600 Subject: Database statements via python but database left intact In-Reply-To: References: Message-ID: On Sat, Oct 5, 2013 at 1:36 PM, ????? ??????????? wrote: > ???? 5/10/2013 10:29 ??, ?/? Zero Piraeus ??????: >> >> : >> >> On Sat, Oct 05, 2013 at 10:16:46PM +0300, ????? ??????????? wrote: >>> >>> Excuse me for asking again today, but i see no error in the >>> following code, yes no isertion or update happens into the database: >>> >>> [...] >>> >>> >>> When this code runs i check instantly my database via PHPMyAdmin and >>> i see that it was left intact. >> >> >> Are you sure that you're committing your changes (either by having >> autocommit set or using an explicit con.commit() call)? >> >> http://geert.vanderkelen.org/dont-forget-the-commit-in-mysql/ > > > > I dont think that is the issue, because up until now i never used commit and > all transaction were successfully were happening. Well, have you changed anything in your database configuration? Whether MySQL uses transactions or not depends on which storage engine is being used. I suggest running a test insert with and without commit to check whether you actually need it or not. Also, are you certain that the code is actually being run? Perhaps the if condition is evaluating as false and the whole block is being skipped, or perhaps the code is silently exiting for some reason before it ever gets to this point. Do some debugging to determine what exactly is being executed. From awilliam at whitemice.org Sun Oct 6 11:52:11 2013 From: awilliam at whitemice.org (Adam Tauno Williams) Date: Sun, 06 Oct 2013 11:52:11 -0400 Subject: Database statements via python but database left intact In-Reply-To: References: Message-ID: >>> Are you sure that you're committing your changes (either by having >>> autocommit set or using an explicit con.commit() call)? >>> http://geert.vanderkelen.org/dont-forget-the-commit-in-mysql/ >> I dont think that is the issue, because up until now i never used >commit and >> all transaction were successfully were happening. Depending on autocommit is a bug [when does commit happen then? consistency is a real problem]. Code should always explicitly ate least COMMIT or ROLLBACK if not explicitly BEGIN. Not to mention how much easier it makes it to read the code and understand the units of work. >Well, have you changed anything in your database configuration? a big downside of autocommit - backend changes can break you app -- Adam Tauno Williams From nikos.gr33k at gmail.com Sun Oct 6 12:49:26 2013 From: nikos.gr33k at gmail.com (=?UTF-8?B?zp3Or866zr/PgiDOkc67zrXOvs+Mz4DOv8+FzrvOv8+C?=) Date: Sun, 06 Oct 2013 19:49:26 +0300 Subject: Database statements via python but database left intact In-Reply-To: References: Message-ID: ???? 6/10/2013 6:52 ??, ?/? Adam Tauno Williams ??????: >>>> Are you sure that you're committing your changes (either by having >>>> autocommit set or using an explicit con.commit() call)? >>>> http://geert.vanderkelen.org/dont-forget-the-commit-in-mysql/ >>> I dont think that is the issue, because up until now i never used >> commit and >>> all transaction were successfully were happening. > > Depending on autocommit is a bug [when does commit happen then? consistency is a real problem]. Code should always explicitly ate least COMMIT or ROLLBACK if not explicitly BEGIN. Not to mention how much easier it makes it to read the code and understand the units of work. > >> Well, have you changed anything in your database configuration? > > a big downside of autocommit - backend changes can break you app > > > try: # locate the ID of the page's URL cur.execute('''SELECT ID FROM counters WHERE url = %s''', page ) data = cur.fetchone() #URL is unique, so should only be one if not data: #first time for page; primary key is automatic, hit is defaulted cur.execute('''INSERT INTO counters (url) VALUES (%s)''', page ) cID = cur.lastrowid #get the primary key value of the new added record else: #found the page, save primary key and use it to issue hit UPDATE cID = data[0] cur.execute('''UPDATE counters SET hits = hits + 1 WHERE ID = %s''', cID ) # find the visitor record for the (saved) cID and Cookie cur.execute('''SELECT * FROM visitors WHERE counterID = %s and cookieID = %s''', (cID, cookieID) ) data = cur.fetchone() #cookieID is unique if not data: # first time visitor on this page, create new record cur.execute('''INSERT INTO visitors (counterID, cookieID, host, city, useros, browser, ref, lastvisit) VALUES (%s, %s, %s, %s, %s, %s, %s, %s)''', (cID, cookieID, host, city, useros, browser, ref, lastvisit) ) else: # found the page, save its primary key for later use vID = data[0] # UPDATE record using retrieved vID cur.execute('''UPDATE visitors SET host = %s, city = %s, useros = %s, browser = %s, ref= %s, hits = hits + 1, lastvisit = %s WHERE counterID = %s and cookieID = %s''', (host, city, useros, browser, ref, lastvisit, vID, cookieID) ) con.commit() except pymysql.ProgrammingError as e: con.rollback() print( repr(e) ) sys.exit(0) Before is qw your post i have chnaged it to this. rollback() is correct where i placed it, i hope con.commit() is also correct too. -- What is now proved was at first only imagined! & WebHost From nikos.gr33k at gmail.com Sat Oct 5 17:02:14 2013 From: nikos.gr33k at gmail.com (=?UTF-8?B?zp3Or866zr/PgiDOkc67zrXOvs+Mz4DOv8+FzrvOv8+C?=) Date: Sun, 06 Oct 2013 00:02:14 +0300 Subject: Database statements via python but database left intact In-Reply-To: References: Message-ID: ???? 5/10/2013 11:31 ??, ?/? Ian Kelly ??????: > Well, have you changed anything in your database configuration? > Whether MySQL uses transactions or not depends on which storage engine > is being used. I suggest running a test insert with and without > commit to check whether you actually need it or not. I cannot beleive it! I have deleted the database and tables and recreted it. By default it seems to use the InnoDB Engine which wouldn't let anythign get inserted/updated. I then deleted the database recretaed and used at the end fo my create table statements the: create table counters ( ID integer(5) not null auto_increment primary key, URL varchar(100) not null, hits integer(5) not null default 1, unique index (URL) )ENGINE = MYISAM; After that all mysql queries executed(inserted/updated) properly! I neved had though of than an engine type could make so much mess. MyISAM is the way to go then for my web development? Why InnoDB failed to execute the queries? From ned at nedbatchelder.com Sat Oct 5 17:39:55 2013 From: ned at nedbatchelder.com (Ned Batchelder) Date: Sat, 05 Oct 2013 17:39:55 -0400 Subject: Database statements via python but database left intact In-Reply-To: References: Message-ID: <5250872B.4070803@nedbatchelder.com> On 10/5/13 5:02 PM, ????? ??????????? wrote: > ???? 5/10/2013 11:31 ??, ?/? Ian Kelly ??????: >> Well, have you changed anything in your database configuration? >> Whether MySQL uses transactions or not depends on which storage engine >> is being used. I suggest running a test insert with and without >> commit to check whether you actually need it or not. > > > I cannot beleive it! > > I have deleted the database and tables and recreted it. > By default it seems to use the InnoDB Engine which wouldn't let > anythign get inserted/updated. > > I then deleted the database recretaed and used at the end fo my create > table statements the: > > create table counters > ( > ID integer(5) not null auto_increment primary key, > URL varchar(100) not null, > hits integer(5) not null default 1, > unique index (URL) > )ENGINE = MYISAM; > > After that all mysql queries executed(inserted/updated) properly! > > I neved had though of than an engine type could make so much mess. > MyISAM is the way to go then for my web development? > Why InnoDB failed to execute the queries? Now is a good time to go read about transactions, and committing, and the difference between MyISAM and InnoDB. Please don't ask more about it here. --Ned. From rosuav at gmail.com Sat Oct 5 18:51:23 2013 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 6 Oct 2013 09:51:23 +1100 Subject: Database statements via python but database left intact In-Reply-To: <5250872B.4070803@nedbatchelder.com> References: <5250872B.4070803@nedbatchelder.com> Message-ID: On Sun, Oct 6, 2013 at 8:39 AM, Ned Batchelder wrote: > Now is a good time to go read about transactions, and committing, and the > difference between MyISAM and InnoDB. Please don't ask more about it here. It's because of threads like this that I would really like Python to nudge people towards something stronger than MySQL. Would it kill Python to incorporate PostgreSQL bindings automatically? ChrisA From ben+python at benfinney.id.au Sat Oct 5 21:05:35 2013 From: ben+python at benfinney.id.au (Ben Finney) Date: Sun, 06 Oct 2013 12:05:35 +1100 Subject: Database engine bindings for Python (was: Database statements via python but database left intact) References: <5250872B.4070803@nedbatchelder.com> Message-ID: <7wpprjp5sw.fsf_-_@benfinney.id.au> Chris Angelico writes: > It's because of threads like this that I would really like Python to > nudge people towards something stronger than MySQL. Would it kill > Python to incorporate PostgreSQL bindings automatically? I'm not sure what would count as ?kill Python?. It would certainly make the release management of Python needlessly dependent on the release cycle of an independent project. The Python bindings for MySQL or PostgreSQL, or even SQLite, are tied to extension libraries for the specific database engine. With SQLite this is not a problem for Python's release management, because Python's release includes the entire SQLite database engine. That code is quite small, so this is deemed a good trade. With a separately-installed, far more complex database engine like MySQL or PostgreSQL, the Python bindings will only work if they are compiled against the correct client library. That client library is part of the database engine code release, not Python. So placing that library in Python's standard library would tie the release of Python's standard library to the version of the database engine. I sympathise with the desire to deprecate MySQL and encourage superior solutions. But your proposed solution would only make Python release management far more burdensome for an unclear benefit. -- \ ?I tell you the truth: some standing here will not taste death | `\ before they see the Son of Man coming in his kingdom.? ?Jesus | _o__) Christ, c. 30 CE, as quoted in Matthew 16:28 | Ben Finney From rosuav at gmail.com Sat Oct 5 21:19:13 2013 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 6 Oct 2013 12:19:13 +1100 Subject: Database engine bindings for Python (was: Database statements via python but database left intact) In-Reply-To: <7wpprjp5sw.fsf_-_@benfinney.id.au> References: <5250872B.4070803@nedbatchelder.com> <7wpprjp5sw.fsf_-_@benfinney.id.au> Message-ID: On Sun, Oct 6, 2013 at 12:05 PM, Ben Finney wrote: > The Python bindings for MySQL or PostgreSQL, or even SQLite, are tied to > extension libraries for the specific database engine. > > With SQLite this is not a problem for Python's release management, > because Python's release includes the entire SQLite database engine. > That code is quite small, so this is deemed a good trade. > > With a separately-installed, far more complex database engine like MySQL > or PostgreSQL, the Python bindings will only work if they are compiled > against the correct client library. Hmm. I see what you mean. Of course, that doesn't bind Python to a specific server version, as one version of the client can talk to any earlier and many later versions of server, but it is an issue. It'd probably be unreasonable to package libpq with Windows installations, but with Linux builds, it should be easy enough to link against whatever libpq happens to be around. If that's unsuited to your server version, it's going to be broken for any other libpq-based apps too. (With package managers like apt, same thing - link against whatever version can be retrieved easily.) The alternative is a pure-Python implementation of the PostgreSQL wire protocol. That would most likely be smaller (if the problem is the size cost of incorporating all of libpq), but would tie Python's pgsql module to a specific protocol version (which doesn't change very often). I don't know if one exists already or not, but it ought to be possible to port the Pike module [1] to Python, if GPL/LGPL/MPL is compatible with the Python licensing. ChrisA [1] http://pike.lysator.liu.se/generated/manual/modref/ex/predef_3A_3A/Sql/pgsql.html From dan at tombstonezero.net Sun Oct 6 02:22:54 2013 From: dan at tombstonezero.net (Dan Sommers) Date: Sun, 6 Oct 2013 06:22:54 +0000 (UTC) Subject: Database engine bindings for Python (was: Database statements via python but database left intact) References: < 5250872B.4070803@nedbatchelder.com> < CAPTjJmqXraZzu3UJO6p_D4LnYEE3zbMBrOX3gdgkLELrMP_xkQ@mail.gmail.com> < 7wpprjp5sw.fsf_-_@benfinney.id.au> Message-ID: On Sun, 06 Oct 2013 12:19:13 +1100, Chris Angelico wrote: > On Sun, Oct 6, 2013 at 12:05 PM, Ben Finney wrote: [ ... ] >> With a separately-installed, far more complex database engine like >> MySQL or PostgreSQL, the Python bindings will only work if they are >> compiled against the correct client library. [ ... ] > The alternative is a pure-Python implementation of the PostgreSQL wire > protocol ... I don't know if one exists already or not ... The PostgreSQL Python wiki [0] says that three such implementations do exist. Some time ago, before psycopg worked with Python3, I tried pg8000, and I could access my local databases, but I never got further than that. - Dan [0] http://wiki.postgresql.org/wiki/Python From kwpolska at gmail.com Sun Oct 6 05:10:53 2013 From: kwpolska at gmail.com (=?UTF-8?B?Q2hyaXMg4oCcS3dwb2xza2HigJ0gV2Fycmljaw==?=) Date: Sun, 6 Oct 2013 11:10:53 +0200 Subject: Database engine bindings for Python (was: Database statements via python but database left intact) In-Reply-To: References: Message-ID: Reposting what I said in the other thread: On Sun, Oct 6, 2013 at 12:51 AM, Chris Angelico wrote: > On Sun, Oct 6, 2013 at 8:39 AM, Ned Batchelder wrote: >> Now is a good time to go read about transactions, and committing, and the >> difference between MyISAM and InnoDB. Please don't ask more about it here. > > It's because of threads like this that I would really like Python to > nudge people towards something stronger than MySQL. Would it kill > Python to incorporate PostgreSQL bindings automatically? It would require Postgres around people?s (or at least packagers?) systems, and it often gets messy when we have such requirements. Psycopg2, the most popular binding, is licensed under LGPL3 + Zope (or such, there is a little mess here) which MAY pose a problem (IANAL though). Also, Postgres is much harder to configure than MySQL, especially if you have no experience or an asshole OS. Moreover, the stdlib is where packages come to die. So, instead of this, maybe we should work on getting psycopg2 to the top result on Googling ?python sql?, or even ?python mysql? with an anti-MySQL ad? (like vim was doing some time ago on Googling ?emacs?) We should also educate people on how PostgreSQL works with a nice, human-friendly tutorial. Especially in some non-standard things and things that differ between PostgreSQL and MySQL ? like how to make an auto-incrementing ID field (use sequences), or how PostgreSQL arrays work, among others. The wiki (that nobody reads anyways) could also use some marketing fixes. -- Chris ?Kwpolska? Warrick PGP: 5EAAEA16 stop html mail | always bottom-post | only UTF-8 makes sense From rosuav at gmail.com Sun Oct 6 09:37:55 2013 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 7 Oct 2013 00:37:55 +1100 Subject: Database engine bindings for Python (was: Database statements via python but database left intact) In-Reply-To: References: Message-ID: On Sun, Oct 6, 2013 at 8:10 PM, Chris ?Kwpolska? Warrick wrote: > It would require Postgres around people?s (or at least packagers?) > systems, and it often gets messy when we have such requirements. I would hope that an absence of libpq could simply result in a courteous exception when the module's imported, but maybe that'd be hard to implement. > Also, Postgres is much harder to configure than MySQL, > especially if you have no experience or an asshole OS. You can get a ready-to-go Postgres under Debian by simply apt-getting it. The default config might not give you optimum performance, but it'll work just fine. Most people shouldn't need to dig into the configs of _any_ database before getting the app going - out-of-the-box settings should be fine for early development, even deployment if you're not doing a lot of traffic. > Moreover, the stdlib is where packages come to die. Fair point. That is an issue. > We should also educate people on how PostgreSQL works with a nice, > human-friendly tutorial. Especially in some non-standard things and > things that differ between PostgreSQL and MySQL ? like how to make an > auto-incrementing ID field (use sequences), or how PostgreSQL arrays > work, among others. The wiki (that nobody reads anyways) could also > use some marketing fixes. Maybe! Possibly go a bit further and say "How-to: Python and Databasing", which could mention SQLite (great for something tiny), PostgreSQL (great for concurrency / multi-user), and "Other databases can also be used, with similar or identical APIs - check out PyPI for a module for your favorite database engine". I guess the above paragraph is sentencing [1] me to write the article, now... ChrisA [1] if you'll pardon a terrible pun From roy at panix.com Sun Oct 6 10:05:28 2013 From: roy at panix.com (Roy Smith) Date: Sun, 06 Oct 2013 10:05:28 -0400 Subject: Database engine bindings for Python (was: Database statements via python but database left intact) References: Message-ID: In article , Chris Angelico wrote: > I would hope that an absence of libpq could simply result in a > courteous exception when the module's imported, but maybe that'd be > hard to implement. It works fine. I've had this in production for a while: # Psycopg2 is only needed for a single hacky lookup which only happens # in the station details admin tool. Installing it has compicated # dependencies, so don't worry if it's not there. try: import psycopg2 except ImportError: psycopg2 = None [...] if psycopg2 is None: logger.warning("psycopg2 not installed") return None That being said, I agree that Postgres support does not belong in the core product. Nor does MySQL, or Oracle, or SqlServer, or MongoDB, or CouchDB, or Cassandra, or a zillion other databases. There's a few reasons. One (as several people have already mentioned), it bulks up, and complicates, the release process. The other is that once something is in the core distribution, it become the de-facto "right way" to do something, whether it's the best or not. >From my own experience, I resisted trying nose for quite a while because unittest was baked in and it was "good enough". Ditto for requests vs. urllib. And finally, with something like a database driver, you really don't want your release schedule to be tied to the language. If the postgres folks come out with a new database feature (or bug fix) which requires a change to the driver, they should be able to release the new driver on their own schedule. From kwpolska at gmail.com Sun Oct 6 05:05:24 2013 From: kwpolska at gmail.com (=?UTF-8?B?Q2hyaXMg4oCcS3dwb2xza2HigJ0gV2Fycmljaw==?=) Date: Sun, 6 Oct 2013 11:05:24 +0200 Subject: Database statements via python but database left intact In-Reply-To: References: <5250872B.4070803@nedbatchelder.com> Message-ID: On Sun, Oct 6, 2013 at 12:51 AM, Chris Angelico wrote: > On Sun, Oct 6, 2013 at 8:39 AM, Ned Batchelder wrote: >> Now is a good time to go read about transactions, and committing, and the >> difference between MyISAM and InnoDB. Please don't ask more about it here. > > It's because of threads like this that I would really like Python to > nudge people towards something stronger than MySQL. Would it kill > Python to incorporate PostgreSQL bindings automatically? It would require Postgres around people?s (or at least packagers?) systems, and it often gets messy when we have such requirements. Psycopg2, the most popular binding, is licensed under LGPL3 + Zope (or such, there is a little mess here) which MAY pose a problem (IANAL though). Also, Postgres is much harder to configure than MySQL, especially if you have no experience or an asshole OS. Moreover, the stdlib is where packages come to die. So, instead of this, maybe we should work on getting psycopg2 to the top result on Googling ?python sql?, or even ?python mysql? with an anti-MySQL ad? (like vim was doing some time ago on Googling ?emacs?) We should also educate people on how PostgreSQL works with a nice, human-friendly tutorial. Especially in some non-standard things and things that differ between PostgreSQL and MySQL ? like how to make an auto-incrementing ID field (use sequences), or how PostgreSQL arrays work, among others. The wiki (that nobody reads anyways) could also use some marketing fixes. -- Chris ?Kwpolska? Warrick PGP: 5EAAEA16 stop html mail | always bottom-post | only UTF-8 makes sense From rustompmody at gmail.com Sun Oct 6 07:36:00 2013 From: rustompmody at gmail.com (rusi) Date: Sun, 6 Oct 2013 04:36:00 -0700 (PDT) Subject: Database statements via python but database left intact In-Reply-To: References: <5250872B.4070803@nedbatchelder.com> Message-ID: <32e44117-b873-4c8e-9cd0-4f15ad729c96@googlegroups.com> On Sunday, October 6, 2013 2:35:24 PM UTC+5:30, Chris ?Kwpolska? Warrick wrote: > So, instead of this, maybe we should work on getting psycopg2 to the > top result on Googling ?python sql?, or even ?python mysql? with an > anti-MySQL ad? (like vim was doing some time ago on Googling ?emacs?) Do you have any accessible data about this? Reasons I ask: 1. The decreasing popularity of emacs wrt vi seems out of proportion to the actual functionality 2. The downward emacs-curve is all the more striking considering the reverse situation some 15-20 years ago From kwpolska at gmail.com Sun Oct 6 08:01:26 2013 From: kwpolska at gmail.com (=?UTF-8?B?Q2hyaXMg4oCcS3dwb2xza2HigJ0gV2Fycmljaw==?=) Date: Sun, 6 Oct 2013 14:01:26 +0200 Subject: Database statements via python but database left intact In-Reply-To: <32e44117-b873-4c8e-9cd0-4f15ad729c96@googlegroups.com> References: <5250872B.4070803@nedbatchelder.com> <32e44117-b873-4c8e-9cd0-4f15ad729c96@googlegroups.com> Message-ID: On Sun, Oct 6, 2013 at 1:36 PM, rusi wrote: > On Sunday, October 6, 2013 2:35:24 PM UTC+5:30, Chris ?Kwpolska? Warrick wrote: >> So, instead of this, maybe we should work on getting psycopg2 to the >> top result on Googling ?python sql?, or even ?python mysql? with an >> anti-MySQL ad? (like vim was doing some time ago on Googling ?emacs?) > > Do you have any accessible data about this? > Reasons I ask: > 1. The decreasing popularity of emacs wrt vi seems out of proportion to the actual functionality > 2. The downward emacs-curve is all the more striking considering the reverse situation some 15-20 years ago I have a screenshot: https://dl.dropboxusercontent.com/u/1933476/screenshots/emacs.png ? Dropbox claims it was taken at around 2011-03-19T11:32:24Z. Earlier today, the exact same ad appeared while searching for ?vim?, but not ?emacs? (why bother when you are the first hit for this query anyways?). Now, for statistics, how many hits it got, or whatnot ? go ask the Vim developers. -- Chris ?Kwpolska? Warrick PGP: 5EAAEA16 stop html mail | always bottom-post | only UTF-8 makes sense From z at etiol.net Sat Oct 5 17:45:13 2013 From: z at etiol.net (Zero Piraeus) Date: Sat, 5 Oct 2013 18:45:13 -0300 Subject: Database statements via python but database left intact In-Reply-To: References: Message-ID: <20131005214513.GA8934@piedra> : On Sun, Oct 06, 2013 at 12:02:14AM +0300, ????? ??????????? wrote: > I neved had though of than an engine type could make so much mess. > MyISAM is the way to go then for my web development? > Why InnoDB failed to execute the queries? Because you didn't commit. MyISAM doesn't support transactions, so when you use it as the engine, your mistake happens to go unpunished. Note that this is a weakness of MyISAM cancelling out a failure in your code; it does *not* mean that MyISAM is better suited to your task. By the way, Ned's right. At this point, this is no longer a Python issue, and is off-topic for discussion here. -[]z. -- Zero Piraeus: vive ut vivas http://etiol.net/pubkey.asc From awilliam at whitemice.org Sun Oct 6 11:55:49 2013 From: awilliam at whitemice.org (Adam Tauno Williams) Date: Sun, 06 Oct 2013 11:55:49 -0400 Subject: Database statements via python but database left intact In-Reply-To: References: Message-ID: <4480c4b3-e6f4-4585-b759-0406e147465f@email.android.com> >I neved had though of than an engine type could make so much mess. Because your app and how it is written is broken. >MyISAM is the way to go then for my web development? >Why InnoDB failed to execute the queries? No, nothing failed. Your app is broken. You are depending on auto commit - and that is a back end implementation detail. DO NOT USE AUTOCOMMIT. The newer engine is expecting you to do things the right way. The old engine was sloppy and does serialization wrong - the reason there is a new engine. -- Adam Tauno Williams From nikos.gr33k at gmail.com Sun Oct 6 11:57:43 2013 From: nikos.gr33k at gmail.com (=?UTF-8?B?zp3Or866zr/PgiDOkc67zrXOvs+Mz4DOv8+FzrvOv8+C?=) Date: Sun, 06 Oct 2013 18:57:43 +0300 Subject: Database statements via python but database left intact In-Reply-To: References: Message-ID: ???? 6/10/2013 12:45 ??, ?/? Zero Piraeus ??????: > : > > On Sun, Oct 06, 2013 at 12:02:14AM +0300, ????? ??????????? wrote: >> I neved had though of than an engine type could make so much mess. >> MyISAM is the way to go then for my web development? >> Why InnoDB failed to execute the queries? > > Because you didn't commit. MyISAM doesn't support transactions, so when > you use it as the engine, your mistake happens to go unpunished. > > Note that this is a weakness of MyISAM cancelling out a failure in your > code; it does *not* mean that MyISAM is better suited to your task. > > By the way, Ned's right. At this point, this is no longer a Python > issue, and is off-topic for discussion here. > > -[]z. > Thanks Zero, i will use con.commit() from now and on because yesterdays i lost 3 hours trying to identify what was wrong with my MySQL statements in python and it turned out to be for no good reason. -- What is now proved was at first only imagined! & WebHost From piet at vanoostrum.org Sun Oct 6 17:40:19 2013 From: piet at vanoostrum.org (Piet van Oostrum) Date: Sun, 06 Oct 2013 17:40:19 -0400 Subject: Database statements via python but database left intact References: Message-ID: ????? ??????????? writes: > i will use con.commit() from now and on because yesterdays i lost 3 > hours trying to identify what was wrong with my MySQL statements in > python and it turned out to be for no good reason. That shows how important it is to study the software that you are using, and also to study the underlying computer science concepts. -- Piet van Oostrum WWW: http://pietvanoostrum.com/ PGP key: [8DAE142BE17999C4] From hjgreenberg at gmail.com Sat Oct 5 21:08:08 2013 From: hjgreenberg at gmail.com (Harvey Greenberg) Date: Sat, 5 Oct 2013 18:08:08 -0700 (PDT) Subject: how to read list from file Message-ID: I am looping as for L in file.readlines(), where file is csv. L is a list of 3 items, eg, [{'a':1, 'b':2}, [1,2,3], 10] Note that the first item is a dir and 2nd is a list, so parsing with split doesn't work. Is there a way to convert L, which is a string, to the list of 3 items I want? From python.list at tim.thechases.com Sat Oct 5 21:24:39 2013 From: python.list at tim.thechases.com (Tim Chase) Date: Sat, 5 Oct 2013 20:24:39 -0500 Subject: how to read list from file In-Reply-To: References: Message-ID: <20131005202439.155b6911@bigbox.christie.dr> On 2013-10-05 18:08, Harvey Greenberg wrote: > I am looping as for L in file.readlines(), where file is csv. > > L is a list of 3 items, eg, [{'a':1, 'b':2}, [1,2,3], 10] Note that > the first item is a dir and 2nd is a list, so parsing with split > doesn't work. Is there a way to convert L, which is a string, to > the list of 3 items I want? sounds like you want ast.literal_eval(): Python 2.7.3 (default, Jan 2 2013, 13:56:14) [GCC 4.7.2] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> s = "[{'a':1, 'b':2}, [1,2,3], 10]" >>> import ast >>> print repr(ast.literal_eval(s)) [{'a': 1, 'b': 2}, [1, 2, 3], 10] -tkc From roy at panix.com Sat Oct 5 21:34:00 2013 From: roy at panix.com (Roy Smith) Date: Sat, 05 Oct 2013 21:34:00 -0400 Subject: how to read list from file References: Message-ID: In article , Harvey Greenberg wrote: > I am looping as for L in file.readlines(), where file is csv. > > L is a list of 3 items, eg, [{'a':1, 'b':2}, [1,2,3], 10] Note that the first > item is a dir and 2nd is a list, so parsing with split doesn't work. Is > there a way to convert L, which is a string, to the list of 3 items I want? I hate to recommend it (since it's bad practice for a number of legitimate reasons), but passing your string to eval() will get you what you want. It's also very close to being valid JSON syntax, the only difference being the use of single instead of double quotes. You might want to just turn it into JSON by substituting the right kind of quotes. json.loads("[{'a':1, 'b':2}, [1,2,3], 10]".replace("'", '"')) [{u'a': 1, u'b': 2}, [1, 2, 3], 10] From roy at panix.com Sat Oct 5 21:35:31 2013 From: roy at panix.com (Roy Smith) Date: Sat, 05 Oct 2013 21:35:31 -0400 Subject: how to read list from file References: Message-ID: In article , Tim Chase wrote: > sounds like you want ast.literal_eval(): This sounds like a better idea than either of my earlier suggestions! From hjgreenberg at gmail.com Sun Oct 6 12:41:33 2013 From: hjgreenberg at gmail.com (Harvey Greenberg) Date: Sun, 6 Oct 2013 09:41:33 -0700 (PDT) Subject: how to read list from file In-Reply-To: References: Message-ID: <61d4be5e-5483-4e1c-9b85-8dac7d6d5ea2@googlegroups.com> On Saturday, October 5, 2013 7:24:39 PM UTC-6, Tim Chase wrote: > On 2013-10-05 18:08, Harvey Greenberg wrote: > > > I am looping as for L in file.readlines(), where file is csv. > > > > > > L is a list of 3 items, eg, [{'a':1, 'b':2}, [1,2,3], 10] Note that > > > the first item is a dir and 2nd is a list, so parsing with split > > > doesn't work. Is there a way to convert L, which is a string, to > > > the list of 3 items I want? > > > > sounds like you want ast.literal_eval(): > > > > Python 2.7.3 (default, Jan 2 2013, 13:56:14) > > [GCC 4.7.2] on linux2 > > Type "help", "copyright", "credits" or "license" for more > > information. > > >>> s = "[{'a':1, 'b':2}, [1,2,3], 10]" > > >>> import ast > > >>> print repr(ast.literal_eval(s)) > > [{'a': 1, 'b': 2}, [1, 2, 3], 10] > > > > -tkc that didn't work. printing it looks like the list because it's the input, but try printing len(repr(ast.literal_eval(s))). It should give 3, but it gives 72 (number of chars). From hjgreenberg at gmail.com Sun Oct 6 12:46:35 2013 From: hjgreenberg at gmail.com (Harvey Greenberg) Date: Sun, 6 Oct 2013 09:46:35 -0700 (PDT) Subject: how to read list from file In-Reply-To: <61d4be5e-5483-4e1c-9b85-8dac7d6d5ea2@googlegroups.com> References: <61d4be5e-5483-4e1c-9b85-8dac7d6d5ea2@googlegroups.com> Message-ID: <6b202854-c905-48f3-a335-8fde2a8e8158@googlegroups.com> On Sunday, October 6, 2013 10:41:33 AM UTC-6, Harvey Greenberg wrote: > On Saturday, October 5, 2013 7:24:39 PM UTC-6, Tim Chase wrote: > > > On 2013-10-05 18:08, Harvey Greenberg wrote: > > > > > > > I am looping as for L in file.readlines(), where file is csv. > > > > > > > > > > > > > > L is a list of 3 items, eg, [{'a':1, 'b':2}, [1,2,3], 10] Note that > > > > > > > the first item is a dir and 2nd is a list, so parsing with split > > > > > > > doesn't work. Is there a way to convert L, which is a string, to > > > > > > > the list of 3 items I want? > > > > > > > > > > > > sounds like you want ast.literal_eval(): > > > > > > > > > > > > Python 2.7.3 (default, Jan 2 2013, 13:56:14) > > > > > > [GCC 4.7.2] on linux2 > > > > > > Type "help", "copyright", "credits" or "license" for more > > > > > > information. > > > > > > >>> s = "[{'a':1, 'b':2}, [1,2,3], 10]" > > > > > > >>> import ast > > > > > > >>> print repr(ast.literal_eval(s)) > > > > > > [{'a': 1, 'b': 2}, [1, 2, 3], 10] > > > > > > > > > > > > -tkc > > > > that didn't work. printing it looks like the list because it's the input, but try printing len(repr(ast.literal_eval(s))). It should give 3, but it gives 72 (number of chars). None of the responses worked; after import json, I used: for line in inputFile.readlines(): L = json.loads(line.replace("","")) print L, len(L) I get error. I probably misunderstood how to implement these suggestions, but I wrote a list to a csv file whose members have lists. I now want to read them (in another program) and end up with the origianl list. From ganeshsahni07 at gmail.com Sun Oct 6 12:50:01 2013 From: ganeshsahni07 at gmail.com (Ravi Sahni) Date: Sun, 6 Oct 2013 22:20:01 +0530 Subject: how to read list from file In-Reply-To: <61d4be5e-5483-4e1c-9b85-8dac7d6d5ea2@googlegroups.com> References: <61d4be5e-5483-4e1c-9b85-8dac7d6d5ea2@googlegroups.com> Message-ID: On Sun, Oct 6, 2013 at 10:11 PM, Harvey Greenberg wrote: > On Saturday, October 5, 2013 7:24:39 PM UTC-6, Tim Chase wrote: >> Python 2.7.3 (default, Jan 2 2013, 13:56:14) >> [GCC 4.7.2] on linux2 >> Type "help", "copyright", "credits" or "license" for more >> information. >> >>> s = "[{'a':1, 'b':2}, [1,2,3], 10]" >> >>> import ast >> >>> print repr(ast.literal_eval(s)) >> [{'a': 1, 'b': 2}, [1, 2, 3], 10] >> >> >> >> -tkc > > that didn't work. printing it looks like the list because it's the input, but try printing len(repr(ast.literal_eval(s))). It should give 3, but it gives 72 (number of chars). Please to remove the repr and try again? Thank you! -- Ravi From jpiitula at ling.helsinki.fi Mon Oct 7 02:04:16 2013 From: jpiitula at ling.helsinki.fi (Jussi Piitulainen) Date: 07 Oct 2013 09:04:16 +0300 Subject: how to read list from file References: <61d4be5e-5483-4e1c-9b85-8dac7d6d5ea2@googlegroups.com> Message-ID: Harvey Greenberg writes: > On Saturday, October 5, 2013 7:24:39 PM UTC-6, Tim Chase wrote: > > >>> s = "[{'a':1, 'b':2}, [1,2,3], 10]" > > >>> import ast > > >>> print repr(ast.literal_eval(s)) > > [{'a': 1, 'b': 2}, [1, 2, 3], 10] > > that didn't work. printing it looks like the list because it's the > input, but try printing len(repr(ast.literal_eval(s))). It should > give 3, but it gives 72 (number of chars). Not sure what the "print repr( )" is meant to achieve here, but I think you should be able to see through it: >>> ast.literal_eval(s) [{'a': 1, 'b': 2}, [1, 2, 3], 10] >>> len(ast.literal_eval(s)) 3 >>> From tjreedy at udel.edu Sun Oct 6 02:24:14 2013 From: tjreedy at udel.edu (Terry Reedy) Date: Sun, 06 Oct 2013 02:24:14 -0400 Subject: how to read list from file In-Reply-To: References: Message-ID: On 10/5/2013 9:08 PM, Harvey Greenberg wrote: > I am looping as for L in file.readlines(), where file is csv. I believe 'for L in file:' does the same, more efficiently, even in 2.7. -- Terry Jan Reedy From hjgreenberg at gmail.com Sun Oct 6 12:57:13 2013 From: hjgreenberg at gmail.com (Harvey Greenberg) Date: Sun, 6 Oct 2013 09:57:13 -0700 (PDT) Subject: how to read list from file In-Reply-To: References: Message-ID: <236e2bfa-94a2-4c6b-a6e7-7105370375fc@googlegroups.com> On Saturday, October 5, 2013 7:08:08 PM UTC-6, Harvey Greenberg wrote: > I am looping as for L in file.readlines(), where file is csv. > > > > L is a list of 3 items, eg, [{'a':1, 'b':2}, [1,2,3], 10] Note that the first item is a dir and 2nd is a list, so parsing with split doesn't work. Is there a way to convert L, which is a string, to the list of 3 items I want? Yay!!!! It worked. Thanks! From breamoreboy at yahoo.co.uk Sun Oct 6 13:24:21 2013 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Sun, 06 Oct 2013 18:24:21 +0100 Subject: how to read list from file In-Reply-To: <236e2bfa-94a2-4c6b-a6e7-7105370375fc@googlegroups.com> References: <236e2bfa-94a2-4c6b-a6e7-7105370375fc@googlegroups.com> Message-ID: On 06/10/2013 17:57, Harvey Greenberg wrote: > On Saturday, October 5, 2013 7:08:08 PM UTC-6, Harvey Greenberg wrote: >> I am looping as for L in file.readlines(), where file is csv. >> >> >> >> L is a list of 3 items, eg, [{'a':1, 'b':2}, [1,2,3], 10] Note that the first item is a dir and 2nd is a list, so parsing with split doesn't work. Is there a way to convert L, which is a string, to the list of 3 items I want? > > Yay!!!! It worked. Thanks! > Very pleased to know, but if you need to post again would you be kind enough to read this first, thanks https://wiki.python.org/moin/GoogleGroupsPython -- Roses are red, Violets are blue, Most poems rhyme, But this one doesn't. Mark Lawrence From ganeshsahni07 at gmail.com Sun Oct 6 13:38:43 2013 From: ganeshsahni07 at gmail.com (Ravi Sahni) Date: Sun, 6 Oct 2013 23:08:43 +0530 Subject: how to read list from file In-Reply-To: <236e2bfa-94a2-4c6b-a6e7-7105370375fc@googlegroups.com> References: <236e2bfa-94a2-4c6b-a6e7-7105370375fc@googlegroups.com> Message-ID: On Sun, Oct 6, 2013 at 10:27 PM, Harvey Greenberg wrote: > On Saturday, October 5, 2013 7:08:08 PM UTC-6, Harvey Greenberg wrote: >> I am looping as for L in file.readlines(), where file is csv. >> >> >> >> L is a list of 3 items, eg, [{'a':1, 'b':2}, [1,2,3], 10] Note that the first item is a dir and 2nd is a list, so parsing with split doesn't work. Is there a way to convert L, which is a string, to the list of 3 items I want? > > Yay!!!! It worked. Thanks! Which method working? Literal_eval method? JSON method? Some third method? [I am newbie so interested. Please to excuse!!] -- Ravi From john_ladasky at sbcglobal.net Sun Oct 6 00:04:25 2013 From: john_ladasky at sbcglobal.net (John Ladasky) Date: Sat, 5 Oct 2013 21:04:25 -0700 (PDT) Subject: Variable arguments (*args, **kwargs): seeking elegance Message-ID: <5783f76c-5565-4089-a6d5-accb0bf329c0@googlegroups.com> Hi folks, I'm trying to make some of Python class definitions behave like the ones I find in professional packages, such as Matplotlib. A Matplotlib class can often have a very large number of arguments -- some of which may be optional, some of which will assume default values if the user does not override them, etc. I have working code which does this kind of thing. I define required arguments and their default values as a class attribute, in an OrderedDict, so that I can match up defaults, in order, with *args. I'm using set.issuperset() to see if an argument passed in **kwargs conflicts with one which was passed in *args. I use set.isdisjoint() to look for arguments in **kwargs which are not expected by the class definition, raising an error if such arguments are found. Even though my code works, I'm finding it to be a bit clunky. And now, I'm writing a new class which has subclasses, and so actually keeps the "extra" kwargs instead of raising an error... This is causing me to re-evaluate my original code. It also leads me to ask: is there a CLEAN and BROADLY-APPLICABLE way for handling the *args/**kwargs/default values shuffle that I can study? Or is this sort of thing too idiosyncratic for there to be a general method? Thanks for any pointers! From __peter__ at web.de Sun Oct 6 03:25:51 2013 From: __peter__ at web.de (Peter Otten) Date: Sun, 06 Oct 2013 09:25:51 +0200 Subject: Variable arguments (*args, **kwargs): seeking elegance References: <5783f76c-5565-4089-a6d5-accb0bf329c0@googlegroups.com> Message-ID: John Ladasky wrote: > Hi folks, > > I'm trying to make some of Python class definitions behave like the ones I > find in professional packages, such as Matplotlib. A Matplotlib class can > often have a very large number of arguments -- some of which may be > optional, some of which will assume default values if the user does not > override them, etc. Personally, I'd rather not copy that kind of interface. > I have working code which does this kind of thing. I define required > arguments and their default values as a class attribute, in an > OrderedDict, so that I can match up defaults, in order, with *args. I'm > using set.issuperset() to see if an argument passed in **kwargs conflicts > with one which was passed in *args. I use set.isdisjoint() to look for > arguments in **kwargs which are not expected by the class definition, > raising an error if such arguments are found. Why do you rely on a homebrew solution instead of actually calling the function or initializer? > Even though my code works, I'm finding it to be a bit clunky. And now, > I'm writing a new class which has subclasses, and so actually keeps the > "extra" kwargs instead of raising an error... This is causing me to > re-evaluate my original code. > > It also leads me to ask: is there a CLEAN and BROADLY-APPLICABLE way for > handling the *args/**kwargs/default values shuffle that I can study? Or > is this sort of thing too idiosyncratic for there to be a general method? > > Thanks for any pointers! inspect.getcallargs() may be worth a look. From peter.cacioppi at gmail.com Sun Oct 6 17:34:57 2013 From: peter.cacioppi at gmail.com (Peter Cacioppi) Date: Sun, 6 Oct 2013 14:34:57 -0700 (PDT) Subject: Variable arguments (*args, **kwargs): seeking elegance In-Reply-To: <5783f76c-5565-4089-a6d5-accb0bf329c0@googlegroups.com> References: <5783f76c-5565-4089-a6d5-accb0bf329c0@googlegroups.com> Message-ID: On Saturday, October 5, 2013 9:04:25 PM UTC-7, John Ladasky wrote: > Hi folks, > > > > I'm trying to make some of Python class definitions behave like the ones I find in professional packages, such as Matplotlib. A Matplotlib class can often have a very large number of arguments -- some of which may be optional, some of which will assume default values if the user does not override them, etc. > > > > I have working code which does this kind of thing. I define required arguments and their default values as a class attribute, in an OrderedDict, so that I can match up defaults, in order, with *args. I'm using set.issuperset() to see if an argument passed in **kwargs conflicts with one which was passed in *args. I use set.isdisjoint() to look for arguments in **kwargs which are not expected by the class definition, raising an error if such arguments are found. > > > > Even though my code works, I'm finding it to be a bit clunky. And now, I'm writing a new class which has subclasses, and so actually keeps the "extra" kwargs instead of raising an error... This is causing me to re-evaluate my original code. > > > > It also leads me to ask: is there a CLEAN and BROADLY-APPLICABLE way for handling the *args/**kwargs/default values shuffle that I can study? Or is this sort of thing too idiosyncratic for there to be a general method? > > > > Thanks for any pointers! Elegance is a matter of taste, but here is one pattern. I tend to think that a very long argument lists are either the result of poor design or an indication that readability would benefit from some sort of dedicated, featherweight "parameter" or "builder" object. The builder object is mutable and copied by any functions that consume it. To my mind, a nice pattern can be as follows. --> Class A is a worker class --> Class B is a worker-builder (or worker-parameter). --> You build B first -->--> usually by first calling a constructor with few to no arguments and then by setting specific properties of B. --> You pass B to the constructor of A, which copies the data over to control the mutability of A. --> A has a getter code that returns a copy of it's saved, private B data, so that you can "remember" later on how it was built. The other point, perhaps more Pythonic, idea here is to exploit this language feature of Python 3 to force argument naming. This would be nice if typical usage involved many possible arguments but only a small number of passed arguments in the typical usage. http://stackoverflow.com/questions/2965271/forced-naming-of-parameters-in-python From steve+comp.lang.python at pearwood.info Sun Oct 6 22:43:18 2013 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: 07 Oct 2013 02:43:18 GMT Subject: Variable arguments (*args, **kwargs): seeking elegance References: <5783f76c-5565-4089-a6d5-accb0bf329c0@googlegroups.com> Message-ID: <52521fc6$0$29984$c3e8da3$5496439d@news.astraweb.com> On Sat, 05 Oct 2013 21:04:25 -0700, John Ladasky wrote: > Hi folks, > > I'm trying to make some of Python class definitions behave like the ones > I find in professional packages, such as Matplotlib. A Matplotlib class > can often have a very large number of arguments -- some of which may be > optional, some of which will assume default values if the user does not > override them, etc. What makes Matplotlib so professional? Assuming that "professional" packages necessarily do the right thing is an unsafe assumption. Many packages have *lousy* interfaces. They often get away with it because of the "sunk cost" fallacy -- if you've spent $3000 on a licence for CrapLib, you're likely to stick through the pain of learning its crap interface rather than admit you wasted $3000. Or the package is twenty years old, and remains compatible with interfaces that wouldn't be accepted now, but that's what the user community have learned and they don't want to learn anything new. Or backwards compatibility requires them to keep the old interface. I have not used mathplotlib enough to judge its interface, but see below. > I have working code which does this kind of thing. I define required > arguments and their default values as a class attribute, in an > OrderedDict, so that I can match up defaults, in order, with *args. I'm > using set.issuperset() to see if an argument passed in **kwargs > conflicts with one which was passed in *args. I use set.isdisjoint() > to look for arguments in **kwargs which are not expected by the class > definition, raising an error if such arguments are found. The cleanest way is: class Spam: def __init__( self, arg, required_arg, another_required_arg, arg_with_default=None, another_optional_arg=42, and_a_third="this is the default", ): and so on, for however many arguments your class wants. Then, when you call it: s = Spam(23, "something", another_optional_arg="oops, missed one") Python will automatically: [quote] match up defaults, in order, ... see if an argument conflicts with one ... look for arguments ... which are not expected... raising an error if such arguments are found [end quote] Why re-invent the wheel? Python already checks all these things for you, and probably much more efficiently than you do. What benefit are you getting from manually managing the arguments? When you have a big, complex set of arguments, you should have a single point of truth, one class or function or method that knows what args are expected and uses Python's argument-handling to handle them. Other classes and functions which are thin (or even not-so-thin) wrappers around that class shouldn't concern themselves with the details of what's in *args and **kwargs, they should just pass them on untouched. There are two main uses for *args: 1) Thin wrappers, where you just collect all the args and pass them on, without caring what name they eventually get assigned to: class MySubclass(MyClass): def spam(self, *args): print("calling MySubclass") super(MySubclass, self).spam(*args) 2) Collecting arbitrary, homogeneous arguments for processing, where the arguments don't get assigned to names, e.g.: def mysort(*args): return sorted(args) mysort(2, 5, 4, 7, 1) => [1, 2, 4, 5, 7] Using *args and then manually matching up each argument with a name just duplicates what Python already does. > Even though my code works, I'm finding it to be a bit clunky. And now, > I'm writing a new class which has subclasses, and so actually keeps the > "extra" kwargs instead of raising an error... This is causing me to > re-evaluate my original code. > > It also leads me to ask: is there a CLEAN and BROADLY-APPLICABLE way for > handling the *args/**kwargs/default values shuffle that I can study? Yes. Don't do it :-) It is sometimes useful to collect extra keyword arguments, handle them in the subclass, then throw them away before passing them on: class MySubclass(MyClass): def spam(self, *args, **kwargs): reverse = kwargs.pop('reverse', False) msg = "calling MySubclass" if reverse: msg = msg[::-1] print(msg) super(MySubclass, self).spam(*args, **kwargs) kwargs is also handy for implementing keyword-only arguments in Python 2 (in Python 3 it isn't needed). But in that case, you don't have to worry about matching up keyword args by position, since position is normally irrelevant. Python's basic named argument handling should cover nearly all the code you want to write, in my opinion. -- Steven From skip at pobox.com Mon Oct 7 09:52:44 2013 From: skip at pobox.com (Skip Montanaro) Date: Mon, 7 Oct 2013 08:52:44 -0500 Subject: Variable arguments (*args, **kwargs): seeking elegance In-Reply-To: <52521fc6$0$29984$c3e8da3$5496439d@news.astraweb.com> References: <5783f76c-5565-4089-a6d5-accb0bf329c0@googlegroups.com> <52521fc6$0$29984$c3e8da3$5496439d@news.astraweb.com> Message-ID: > What makes Matplotlib so professional? > > Assuming that "professional" packages necessarily do the right thing is > an unsafe assumption. Many packages have *lousy* interfaces. Not that it's a complete explanation for matplotlib's interfaces, but it did start out as a Python-based replacement for MATLAB. I seem to recall that John Hunter started the project because the lab he worked in as a postdoc only had a single MATLAB license, so it wasn't always available when he needed it. Skip From john_ladasky at sbcglobal.net Mon Oct 7 12:26:51 2013 From: john_ladasky at sbcglobal.net (John Ladasky) Date: Mon, 7 Oct 2013 09:26:51 -0700 (PDT) Subject: Variable arguments (*args, **kwargs): seeking elegance In-Reply-To: <5783f76c-5565-4089-a6d5-accb0bf329c0@googlegroups.com> References: <5783f76c-5565-4089-a6d5-accb0bf329c0@googlegroups.com> Message-ID: <1d56b6ac-e3ee-4a5e-8d58-dfdaa998b7cc@googlegroups.com> Thanks, everyone, for your replies. Perhaps I have complicated things unnecessarily? I was just trying to do some error-checking on the arguments supplied to the class constructor. Perhaps Python already implements automatically what I am trying to accomplish manually? I'll tinker around with some minimal code, try to provoke some errors, and see what I get. Here is one more detail which may be relevant. The base class for the family of classes I am developing is a numpy.ndarray. The numpy.ndarray is a C extension type (and if I understand correctly, that means it is immutable by ordinary Python methods). Subclassing ndarray can get a bit complicated (see http://docs.scipy.org/doc/numpy/user/basics.subclassing.html). The ndarray.__init__ method is inaccessible, instead one overrides ndarray.__new__. Making further subclasses of a subclassed numpy.ndarray, each of which may have their own arguments, is what I am trying to accomplish while adhering to the "DRY" principle. From john_ladasky at sbcglobal.net Mon Oct 7 12:42:43 2013 From: john_ladasky at sbcglobal.net (John Ladasky) Date: Mon, 7 Oct 2013 09:42:43 -0700 (PDT) Subject: Variable arguments (*args, **kwargs): seeking elegance In-Reply-To: <1d56b6ac-e3ee-4a5e-8d58-dfdaa998b7cc@googlegroups.com> References: <5783f76c-5565-4089-a6d5-accb0bf329c0@googlegroups.com> <1d56b6ac-e3ee-4a5e-8d58-dfdaa998b7cc@googlegroups.com> Message-ID: On Monday, October 7, 2013 9:26:51 AM UTC-7, I wrote: > Here is one more detail which may be relevant. The base class for the family of classes I am developing is a numpy.ndarray. The numpy.ndarray is a C extension type (and if I understand correctly, that means it is immutable by ordinary Python methods). Subclassing ndarray can get a bit complicated (see http://docs.scipy.org/doc/numpy/user/basics.subclassing.html). I've just been reading the above page, which is pretty new. It supersedes a now-defunct page, http://www.scipy.org/Subclasses. The unusual subclassing needs of an ndarray apparently arise, not from the fact that an ndarray is a C extension type, but because of numpy's special view casting and slicing requirements. While I don't believe this is highly relevant to the args/kwargs issues I have here, I thought that I should correct my earlier remark. From steve+comp.lang.python at pearwood.info Mon Oct 7 17:13:10 2013 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: 07 Oct 2013 21:13:10 GMT Subject: Variable arguments (*args, **kwargs): seeking elegance References: <5783f76c-5565-4089-a6d5-accb0bf329c0@googlegroups.com> <1d56b6ac-e3ee-4a5e-8d58-dfdaa998b7cc@googlegroups.com> Message-ID: <525323e5$0$29984$c3e8da3$5496439d@news.astraweb.com> On Mon, 07 Oct 2013 09:26:51 -0700, John Ladasky wrote: > Thanks, everyone, for your replies. Perhaps I have complicated things > unnecessarily? I was just trying to do some error-checking on the > arguments supplied to the class constructor. Perhaps Python already > implements automatically what I am trying to accomplish manually? I'll > tinker around with some minimal code, try to provoke some errors, and > see what I get. It's really hard to make definitive judgements without actually seeing your code and understanding your use-case. I can only suggest that, you *may* be complicating things unnecessarily. On the other hand, there's always the chance that your requirements are sufficiently unusual that you have done exactly what needs to be done. But I suspect even in this case, there may be a more elegant way to solve the problem of "I'm finding it to be a bit clunky", to quote your original post. Clunky code can sometimes be smoothed out by refactoring the complexity by use of decorators. Can you post an example of your code? One thought -- often, people turn to subclassing as the only tool in their toolbox. Have you considered that it may be easier/better to work with delegation and composition instead? > Here is one more detail which may be relevant. The base class for the > family of classes I am developing is a numpy.ndarray. The numpy.ndarray > is a C extension type (and if I understand correctly, that means it is > immutable by ordinary Python methods). Subclassing ndarray can get a > bit complicated (see > http://docs.scipy.org/doc/numpy/user/basics.subclassing.html). The > ndarray.__init__ method is inaccessible, instead one overrides > ndarray.__new__. Don't forget ndarray.__array_finalize__, __array_wrap__ and __array_prepare__. I am not an expert on numpy, but reading that page just makes me think they're doing it all wrong, adding far too much complication. (I've written code like that myself, but thank goodness I've had the sense to throw it away and start again...). I'm trying to give them the benefit of the doubt, but I've never liked the amount of DWIM cleverness in numpy, and I think they would have been *much* better off having a clean separation between the three ways of creating an array: - the normal Python __new__ and __init__ mechanism - creating a view into an array - templating instead of conflating the three into a single mechanism. I suspect that the fundamental confusion comes about because numpy doesn't have a clean distinction between views into an array, and actual arrays. Although I must admit I've not done more than dip my toe into numpy, so you should take my criticisms with a generous pinch of salt. > Making further subclasses of a subclassed numpy.ndarray, each of which > may have their own arguments, is what I am trying to accomplish while > adhering to the "DRY" principle. The usual way of doing this is to accept only keyword arguments for any additional args: class Base: def __new__(cls, doc, grumpy, happy, sleepy, bashful, sneezy, dopey): ... class Subclass(Base): def __new__(cls, *args, **kwargs): # grab the additional arguments sneaky = kwargs.pop('sneaky', True) # optional grabby = kwargs.pop('grabby') # mandatory touchy = kwargs.pop('touchy') feely = kwargs.pop('feely') instance = super(Subclass, cls).__new__(cls, *args, **kwargs) # process additional arguments instance.apply_extras(sneaky, grabby, touchy, feely) return instance # In Python 3, I can do this to make it even cleaner: class Subclass(Base): def __new__(cls, *args, sneaky=True, grabby, touchy, feely, **kwargs): instance = super(Subclass, cls).__new__(cls, *args, **kwargs) # process additional arguments instance.apply_extras(sneaky, grabby, touchy, feely) return instance In general, you should aim to use either __new__ or __init__ but not both, although that's not a hard law, just a guideline. Can you adapt this pattern to ndarray? -- Steven From john_ladasky at sbcglobal.net Mon Oct 7 17:49:06 2013 From: john_ladasky at sbcglobal.net (John Ladasky) Date: Mon, 7 Oct 2013 14:49:06 -0700 (PDT) Subject: Variable arguments (*args, **kwargs): seeking elegance In-Reply-To: <525323e5$0$29984$c3e8da3$5496439d@news.astraweb.com> References: <5783f76c-5565-4089-a6d5-accb0bf329c0@googlegroups.com> <1d56b6ac-e3ee-4a5e-8d58-dfdaa998b7cc@googlegroups.com> <525323e5$0$29984$c3e8da3$5496439d@news.astraweb.com> Message-ID: Wow, Steven, that was a great, detailed reply. I hope you will forgive me for shortcutting to the end, because I've been hacking away for a few hours and came to this very conclusion: On Monday, October 7, 2013 2:13:10 PM UTC-7, Steven D'Aprano wrote: > In general, you should aim to use either __new__ or __init__ but not > both, although that's not a hard law, just a guideline. My problems were solved by adhering to using only __new__ in my ndarray subclasses, and avoiding __init__. (If I used both methods, my arguments were passed to the object twice, once through each method. That's weird! It messed me up! And I'm not sure what purpose it serves.) The __new__ methods of my subclasses now call super().__new__ to handle the attributes and error checking which are common to all the classes, then handle the subclass-specific variables. One wrinkle that I had to comprehend was that super().__new__ would be returning me a half-baked object on which I had to do more work. I'm used to __init__, of course, which works on self. OK, as for some other points: > Don't forget ndarray.__array_finalize__, __array_wrap__ and > __array_prepare__. I handle __array_finalize__ in my base class. Also __reduce_ex__ and __setstate__, so that I can pickle and unpickle my array objects (which is necessary for multiprocessing work). I haven't had time to deal with __array_wrap__ or __array_prepare__ yet, but so far my downstream code is working without these methods (crossing fingers). > I am not an expert on numpy, but reading that page just makes me think > they're doing it all wrong, adding far too much complication. (I've > written code like that myself, but thank goodness I've had the sense to > throw it away and start again...). I'm trying to give them the benefit of > the doubt, but I've never liked the amount of DWIM cleverness in numpy, > and I think they would have been *much* better off having a clean > separation between the three ways of creating an array: > > - the normal Python __new__ and __init__ mechanism > - creating a view into an array > - templating > > instead of conflating the three into a single mechanism. I agree, I always find it complicated to wrap my head around these complexities. But I simply can't live without numpy! And finally: > sneaky = kwargs.pop('sneaky', True) # optional I don't know whether to be excited or embarrassed that I can still learn things about the basics of Python... I've never used the optional argument of dict.pop(). Cool! Thanks. From peter.cacioppi at gmail.com Mon Oct 7 17:12:52 2013 From: peter.cacioppi at gmail.com (Peter Cacioppi) Date: Mon, 7 Oct 2013 14:12:52 -0700 (PDT) Subject: Variable arguments (*args, **kwargs): seeking elegance In-Reply-To: <5783f76c-5565-4089-a6d5-accb0bf329c0@googlegroups.com> References: <5783f76c-5565-4089-a6d5-accb0bf329c0@googlegroups.com> Message-ID: <71efb122-0eed-4180-bd3f-10749cf7f60d@googlegroups.com> On Saturday, October 5, 2013 9:04:25 PM UTC-7, John Ladasky wrote: > Hi folks, > > > > I'm trying to make some of Python class definitions behave like the ones I find in professional packages, such as Matplotlib. A Matplotlib class can often have a very large number of arguments -- some of which may be optional, some of which will assume default values if the user does not override them, etc. > > > > I have working code which does this kind of thing. I define required arguments and their default values as a class attribute, in an OrderedDict, so that I can match up defaults, in order, with *args. I'm using set.issuperset() to see if an argument passed in **kwargs conflicts with one which was passed in *args. I use set.isdisjoint() to look for arguments in **kwargs which are not expected by the class definition, raising an error if such arguments are found. > > > > Even though my code works, I'm finding it to be a bit clunky. And now, I'm writing a new class which has subclasses, and so actually keeps the "extra" kwargs instead of raising an error... This is causing me to re-evaluate my original code. > > > > It also leads me to ask: is there a CLEAN and BROADLY-APPLICABLE way for handling the *args/**kwargs/default values shuffle that I can study? Or is this sort of thing too idiosyncratic for there to be a general method? > > > > Thanks for any pointers! "Subclassing ndarray can get a bit complicated" Another software pattern idea is "encapsulate don't inherit". When a class is really messy to subclass, start fresh with a new class that wraps the messy class. Create redirect methods for whatever is needed, then subclass from the class you created. In fact, I'd go so far as to say you should only subclass from classes that were designed with subclassing in mind. If you find yourself bending over backwards to make subclassing work, it means you should be wrapping and redirecting instead. This is perhaps more true in C#/Java than Python, but still something to think about. From peter.cacioppi at gmail.com Tue Oct 8 04:30:19 2013 From: peter.cacioppi at gmail.com (Peter Cacioppi) Date: Tue, 8 Oct 2013 01:30:19 -0700 (PDT) Subject: Variable arguments (*args, **kwargs): seeking elegance In-Reply-To: <5783f76c-5565-4089-a6d5-accb0bf329c0@googlegroups.com> References: <5783f76c-5565-4089-a6d5-accb0bf329c0@googlegroups.com> Message-ID: <923d2c56-6216-4fc1-b8ae-bae518a09518@googlegroups.com> On Saturday, October 5, 2013 9:04:25 PM UTC-7, John Ladasky wrote: > Hi folks, > > > > I'm trying to make some of Python class definitions behave like the ones I find in professional packages, such as Matplotlib. A Matplotlib class can often have a very large number of arguments -- some of which may be optional, some of which will assume default values if the user does not override them, etc. > > > > I have working code which does this kind of thing. I define required arguments and their default values as a class attribute, in an OrderedDict, so that I can match up defaults, in order, with *args. I'm using set.issuperset() to see if an argument passed in **kwargs conflicts with one which was passed in *args. I use set.isdisjoint() to look for arguments in **kwargs which are not expected by the class definition, raising an error if such arguments are found. > > > > Even though my code works, I'm finding it to be a bit clunky. And now, I'm writing a new class which has subclasses, and so actually keeps the "extra" kwargs instead of raising an error... This is causing me to re-evaluate my original code. > > > > It also leads me to ask: is there a CLEAN and BROADLY-APPLICABLE way for handling the *args/**kwargs/default values shuffle that I can study? Or is this sort of thing too idiosyncratic for there to be a general method? > > > > Thanks for any pointers! "One thought -- often, people turn to subclassing as the only tool in their toolbox. Have you considered that it may be easier/better to work with delegation and composition instead? " Double like. Subclassing is awesome when it is used properly ... which usually means used cautiously. Delegation/composition just doesn't result in the some sort of weird gotchas. From markotaht at gmail.com Sun Oct 6 09:10:16 2013 From: markotaht at gmail.com (markotaht at gmail.com) Date: Sun, 6 Oct 2013 06:10:16 -0700 (PDT) Subject: Odd-length string Message-ID: <0f1b6cd6-3025-4bdc-8f80-73a51a7ed241@googlegroups.com> print("Key-" + str(v?ti) + ": " + str(unhexlify("".join(tulemus)))) IM getting this error on this line. This is the print line of a decryption program. I wanti it to convert the tulemus which is in HEX to ASCII so i could read it. I could use online translators for the line, but since i have 255 lines of the HEX codes, it would be higly unefficient. How to i fix the Oddlenght string? From roy at panix.com Sun Oct 6 09:46:03 2013 From: roy at panix.com (Roy Smith) Date: Sun, 06 Oct 2013 09:46:03 -0400 Subject: Odd-length string References: <0f1b6cd6-3025-4bdc-8f80-73a51a7ed241@googlegroups.com> Message-ID: In article <0f1b6cd6-3025-4bdc-8f80-73a51a7ed241 at googlegroups.com>, markotaht at gmail.com wrote: > print("Key-" + str(v?ti) + ": " + str(unhexlify("".join(tulemus)))) > > IM getting this error on this line. This is the print line of a decryption > program. I wanti it to convert the tulemus which is in HEX to ASCII so i > could read it. I could use online translators for the line, but since i have > 255 lines of the HEX codes, it would be higly unefficient. > > How to i fix the Oddlenght string? Some basic advice on asking for help: 1) Don't tell us what the error was, show us. Run your program, then cut-and-paste the entire stack trace into your post. 2) Tell us what version of Python you're using. I'm assume one of the Python 3 versions, since you put ()'s after print, but don't make people guess. 3) Reduce the problem down to the smallest possible amount of code. Do you get the error when you do: print(str(unhexlify("".join(tulemus)))) what about print(unhexlify("".join(tulemus))) or print("".join(tulemus)) or print(str(v?ti)) every little thing you can hack away and still generate the error gets you (and us!) one step closer to understanding what's happening. From __peter__ at web.de Sun Oct 6 10:31:11 2013 From: __peter__ at web.de (Peter Otten) Date: Sun, 06 Oct 2013 16:31:11 +0200 Subject: Odd-length string References: <0f1b6cd6-3025-4bdc-8f80-73a51a7ed241@googlegroups.com> Message-ID: markotaht at gmail.com wrote: > print("Key-" + str(v?ti) + ": " + str(unhexlify("".join(tulemus)))) > > IM getting this error on this line. This is the print line of a decryption > program. I wanti it to convert the tulemus which is in HEX to ASCII so i > could read it. I could use online translators for the line, but since i > have 255 lines of the HEX codes, it would be higly unefficient. > > How to i fix the Oddlenght string? My crystal ball says: strip off the newline byte: >>> unhexlify(tulemus) Traceback (most recent call last): File "", line 1, in binascii.Error: Odd-length string >>> unhexlify(tulemus.rstrip(b"\n")) b'whatever' From markotaht at gmail.com Sun Oct 6 10:37:13 2013 From: markotaht at gmail.com (markotaht at gmail.com) Date: Sun, 6 Oct 2013 07:37:13 -0700 (PDT) Subject: Odd-length string In-Reply-To: <0f1b6cd6-3025-4bdc-8f80-73a51a7ed241@googlegroups.com> References: <0f1b6cd6-3025-4bdc-8f80-73a51a7ed241@googlegroups.com> Message-ID: I fixed this problem but encountered new problem. Problem was that some parts that came throug my decryption were 00 or 0 the first symbol so the program didnt show them. NEw problem is : Traceback (most recent call last): File "C:\Users\Marko\Desktop\hacker.org\XOR cypher.py", line 35, in print("Key-" + str(v?ti) + ": " + str("".join(tulemus2))) TypeError: sequence item 0: expected str instance, bytes found If i take away the join command i get this: Key-00000000: [b'u', b'o', b'\x00', b'\x1d', b' ', b'|', b'N', b'\x0f', b'9', b'j', b'K', b'J', b'&', b'#', b'A', b'K', b'5', b'k', b'_', b'\x1e', b',', b'j', b'\x0c', b'\x08', b'i', b'(', b'\x06', b'\\', b'r', b'3', b'\x1f', b'V', b's', b'9', b'\x1d'] the Key-00000000 is the key im using to decrypt the code. everything else is generated byt the decrytion process and the unhexlify command. So my guess is, the join command cant handle the b"u" type of format. how can i get rid of the b. Or does anyone have a better idea how to translate HEX into ASCII and sort out the lines that make sense From breamoreboy at yahoo.co.uk Sun Oct 6 10:56:05 2013 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Sun, 06 Oct 2013 15:56:05 +0100 Subject: Odd-length string In-Reply-To: References: <0f1b6cd6-3025-4bdc-8f80-73a51a7ed241@googlegroups.com> Message-ID: On 06/10/2013 15:37, markotaht at gmail.com wrote: > I fixed this problem but encountered new problem. Problem was that some parts that came throug my decryption were 00 or 0 the first symbol so the program didnt show them. > > NEw problem is : Traceback (most recent call last): > File "C:\Users\Marko\Desktop\hacker.org\XOR cypher.py", line 35, in > print("Key-" + str(v?ti) + ": " + str("".join(tulemus2))) > TypeError: sequence item 0: expected str instance, bytes found > > If i take away the join command i get this: Key-00000000: [b'u', b'o', b'\x00', b'\x1d', b' ', b'|', b'N', b'\x0f', b'9', b'j', b'K', b'J', b'&', b'#', b'A', b'K', b'5', b'k', b'_', b'\x1e', b',', b'j', b'\x0c', b'\x08', b'i', b'(', b'\x06', b'\\', b'r', b'3', b'\x1f', b'V', b's', b'9', b'\x1d'] > > the Key-00000000 is the key im using to decrypt the code. everything else is generated byt the decrytion process and the unhexlify command. So my guess is, the join command cant handle the b"u" type of format. how can i get rid of the b. > > Or does anyone have a better idea how to translate HEX into ASCII and sort out the lines that make sense > Will you please be kind enough to quote your replies in context. -- Roses are red, Violets are blue, Most poems rhyme, But this one doesn't. Mark Lawrence From tjreedy at udel.edu Sun Oct 6 15:50:38 2013 From: tjreedy at udel.edu (Terry Reedy) Date: Sun, 06 Oct 2013 15:50:38 -0400 Subject: Odd-length string In-Reply-To: References: <0f1b6cd6-3025-4bdc-8f80-73a51a7ed241@googlegroups.com> Message-ID: On 10/6/2013 10:37 AM, markotaht at gmail.com wrote: > NEw problem is : Traceback (most recent call last): > File "C:\Users\Marko\Desktop\hacker.org\XOR cypher.py", line 35, in > print("Key-" + str(v?ti) + ": " + str("".join(tulemus2))) > TypeError: sequence item 0: expected str instance, bytes found If ''.join(iterable_of_strings) executes, the result is a string and str would not be needed. If you try "".join(tulemus2) by itself, you will see the same error. > If i take away the join command i get this: Key-00000000: [b'u', b'o', b'\x00', b'\x1d', b' ', b'|', b'N', b'\x0f', b'9', b'j', b'K', b'J', b'&', b'#', b'A', b'K', b'5', b'k', b'_', b'\x1e', b',', b'j', b'\x0c', b'\x08', b'i', b'(', b'\x06', b'\\', b'r', b'3', b'\x1f', b'V', b's', b'9', b'\x1d'] Since tulemus2 is an iterable_of_bytes, you must join with bytes b'' and call str on the result. So just add the b prefix. What Roy tried to say to you earlier is that when you get an error in a expression, and you do not understand *exactly* where in the expression the error is, pull the expression apart and test the pieces individually. This sometimes includes splitting print(expression) # into s=expression print(s) -- Terry Jan Reedy From piet at vanoostrum.org Sun Oct 6 20:59:24 2013 From: piet at vanoostrum.org (Piet van Oostrum) Date: Sun, 06 Oct 2013 20:59:24 -0400 Subject: Odd-length string References: <0f1b6cd6-3025-4bdc-8f80-73a51a7ed241@googlegroups.com> Message-ID: markotaht at gmail.com writes: > I fixed this problem but encountered new problem. Problem was that some parts that came throug my decryption were 00 or 0 the first symbol so the program didnt show them. > > NEw problem is : Traceback (most recent call last): > File "C:\Users\Marko\Desktop\hacker.org\XOR cypher.py", line 35, in > print("Key-" + str(v?ti) + ": " + str("".join(tulemus2))) > TypeError: sequence item 0: expected str instance, bytes found > > If i take away the join command i get this: Key-00000000: [b'u', b'o', b'\x00', b'\x1d', b' ', b'|', b'N', b'\x0f', b'9', b'j', b'K', b'J', b'&', b'#', b'A', b'K', b'5', b'k', b'_', b'\x1e', b',', b'j', b'\x0c', b'\x08', b'i', b'(', b'\x06', b'\\', b'r', b'3', b'\x1f', b'V', b's', b'9', b'\x1d'] > > the Key-00000000 is the key im using to decrypt the code. everything else is generated byt the decrytion process and the unhexlify command. So my guess is, the join command cant handle the b"u" type of format. how can i get rid of the b. Use b''.join(tulemus2), and then to convert it to a string you have to specify the encoding, which should be 'ascii', as you say it is ASCII. str(b''.join(tulemus2), 'ascii') or b''.join(tulemus2).decode('ascii') But note: If your tulemus2 contains bytes > 127 the this will fail as it is then not ASCII. > > Or does anyone have a better idea how to translate HEX into ASCII and > sort out the lines that make sense You can use base64.b16decode, but it needs a byte string as input. For the rest is is the same as unhexlify. And both will give you a byte string, because there is no way to translate a hex string to a character string without specifying an encoding (if there is hex > 7F it is no longer ASCII). -- Piet van Oostrum WWW: http://pietvanoostrum.com/ PGP key: [8DAE142BE17999C4] From marco.buttu at gmail.com Sun Oct 6 14:17:33 2013 From: marco.buttu at gmail.com (Marco Buttu) Date: Sun, 06 Oct 2013 20:17:33 +0200 Subject: Overriding of the type.__call__() method in a metaclass Message-ID: Hi all, I have a question about class creation and the __call__ method. I have the following metaclass: >>> class FooMeta(type): ... def __call__(metacls, name, bases, namespace): ... print("FooMeta.__call__()") From what I undestood, at the end of the class statement happens something like this: >>> def __call__(metacls, name, bases, namespace): ... print("FooMeta.__call__()") ... >>> FooMeta = type('FooMeta', (type,), {'__call__': __call__}) The call to the metaclass type causes the call to type.__call__(), so that's happened is: >>> FooMeta = type.__call__(type, 'FooMeta', (type,), {'__call__': __call__}) Now I expected the output `FooMeta.__call__()` from the following Foo class creation: >>> class Foo(metaclass=FooMeta): ... pass because I thought at the end of the class Foo suite this should have been happened: >>> Foo = FooMeta.__call__(FooMeta, 'Foo', (), {}) FooMeta.__call__() but I thought wrong: >>> class FooMeta(type): ... def __call__(metacls, name, bases, namespace): ... print("FooMeta.__call__()") ... >>> class Foo(metaclass=FooMeta): ... pass ... >>> How come? Is it because the first argument of metaclass.__call__() is always type or I am thinking something wrong? Thanks in advance, Marco -- Marco From __peter__ at web.de Sun Oct 6 15:04:20 2013 From: __peter__ at web.de (Peter Otten) Date: Sun, 06 Oct 2013 21:04:20 +0200 Subject: Overriding of the type.__call__() method in a metaclass References: Message-ID: Marco Buttu wrote: > Hi all, I have a question about class creation and the __call__ method. > I have the following metaclass: > > >>> class FooMeta(type): > ... def __call__(metacls, name, bases, namespace): > ... print("FooMeta.__call__()") > > > From what I undestood, at the end of the class statement happens > something like this: > > >>> def __call__(metacls, name, bases, namespace): > ... print("FooMeta.__call__()") > ... > >>> FooMeta = type('FooMeta', (type,), {'__call__': __call__}) > > The call to the metaclass type causes the call to type.__call__(), so > that's happened is: > > >>> FooMeta = type.__call__(type, 'FooMeta', (type,), {'__call__': > __call__}) > > Now I expected the output `FooMeta.__call__()` from the following Foo > class creation: > > >>> class Foo(metaclass=FooMeta): > ... pass > > because I thought at the end of the class Foo suite this should have > been happened: > > >>> Foo = FooMeta.__call__(FooMeta, 'Foo', (), {}) > FooMeta.__call__() > > but I thought wrong: > > >>> class FooMeta(type): > ... def __call__(metacls, name, bases, namespace): > ... print("FooMeta.__call__()") > ... > >>> class Foo(metaclass=FooMeta): > ... pass > ... > >>> > > How come? Is it because the first argument of metaclass.__call__() is > always type or I am thinking something wrong? > Thanks in advance, Marco Forget about metaclasses for the moment and ask yourself what happens when a regular class class A: def __init__(...): ... def __call__(...): ... is "called": a = A(...) # invokes __init__() a(...) # invokes __call__() The metaclass is just the class of a class, i. e. the Foo object is an instance of FooMeta, so making Foo invokes (__new__() and) __init__(), and calling Foo invokes FooMeta.__call__(): >>> class FooMeta(type): ... def __call__(self, *args): print("__call__%r" % (args,)) ... >>> class Foo(metaclass=FooMeta): pass ... >>> Foo() __call__() If you follow that logic you can easily see that for FooMeta to invoke your custom __call__() method you'd have to define it in FooMeta's metaclass: >>> class FooMetaMeta(type): ... def __call__(*args): print(args) ... >>> class FooMeta(metaclass=FooMetaMeta): ... pass ... >>> class Foo(metaclass=FooMeta): ... pass ... (, 'Foo', (), {'__module__': '__main__', '__qualname__': 'Foo'}) >>> Foo is None True So, yes, it's turtles all the way down... From steve+comp.lang.python at pearwood.info Sun Oct 6 22:27:21 2013 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: 07 Oct 2013 02:27:21 GMT Subject: Overriding of the type.__call__() method in a metaclass References: Message-ID: <52521c08$0$29984$c3e8da3$5496439d@news.astraweb.com> On Sun, 06 Oct 2013 20:17:33 +0200, Marco Buttu wrote: > Hi all, I have a question about class creation and the __call__ method. > I have the following metaclass: > > >>> class FooMeta(type): > ... def __call__(metacls, name, bases, namespace): > ... print("FooMeta.__call__()") At this point, FooMeta is nothing special, it's just an ordinary class. Calling it "FooMeta" doesn't make it special. Like every ordinary class, __call__ will only be called when an *instance* is called. Since you have no FooMeta instances yet, FooMeta.__call__ won't be called. > From what I undestood, at the end of the class statement happens > something like this: > > >>> def __call__(metacls, name, bases, namespace): > ... print("FooMeta.__call__()") > ... > >>> FooMeta = type('FooMeta', (type,), {'__call__': __call__}) You're referring to the class statement ("class FooMeta(type): ...") being syntactic sugar for the above direct call to type. Correct. Again, this applies to *all* classes, not just metaclasses. > The call to the metaclass type causes the call to type.__call__(), so > that's happened is: > > >>> FooMeta = type.__call__(type, 'FooMeta', (type,), {'__call__': > __call__}) Yes but no... Your code snippet is correct. The "class FooMeta(type)..." statement is syntactic sugar for type.__call__(...). But your description is incorrect. This doesn't occur when you "call the metaclass type", not in the sense you mean. You don't have a "metaclass type" yet, except for type itself. You have a class *called* FooMeta, but it hasn't been called. It can't be called yet, because it hasn't yet been created! You're still executing the "class FooMeta..." statement, creating FooMeta. So at the point FooMeta is created, the only metaclass involved is type itself. Hence, it is only type.__call__ which is involved, not FooMeta.__call__. FooMeta.__call__ is used when you call an instance of FooMeta: class Foo(metaclass=FooMeta): ... obj = Foo() Here, Foo is an instance of FooMeta, so calling Foo calls FooMeta.__call__. What I think you are looking for is FooMeta.__new__, which gets called when the instance is created. What's the instance of FooMeta again? It's class Foo. So at the end of "class Foo(metaclass=FooMeta)..." the FooMeta.__new__ method is called to create Foo. Then, once Foo is created, instantiating it using "obj = Foo()" calls FooMeta.__call__. > Now I expected the output `FooMeta.__call__()` from the following Foo > class creation: > > >>> class Foo(metaclass=FooMeta): > ... pass No, not at the class creation. Metaclass.__new__ is called when you create an instance; the instance here is class Foo, so FooMeta.__new__ will be called. > because I thought at the end of the class Foo suite this should have > been happened: > > >>> Foo = FooMeta.__call__(FooMeta, 'Foo', (), {}) > FooMeta.__call__() Go back to the rule for class creation: class Spam(bases): is syntactic sugar for: type('Spam', bases, namespace) used to instantiate the new instance, Spam. If a metaclass is given, it is used instead of type. So you'll have: Meta('Spam', bases, namespace) and Meta.__new__ will be called to create the instance. Note: you don't need to use a *class* as metaclass! Bizarre but true: any callable object will do, so long as it matches the expected signature. Watch this: py> class Wot(metaclass=lambda name, bases, namespace: 42): ... a = 12 ... def __len__(self): ... return 9999 ... py> Wot 42 But I digress. In your case, you are using a subclass of type as your metaclass, and it is creating a new instance of FooMeta. When a new instance is created, FooMeta.__new__ is called. To get the effect you are after, you can: 1) Use FooMeta.__new__ instead of __call__; 2) Use a metaclass of the metaclass, FooMetaMeta.__call__; or 3) Use a function that takes the same signature as type. -- Steven From marco.buttu at gmail.com Mon Oct 7 01:22:02 2013 From: marco.buttu at gmail.com (Marco Buttu) Date: Mon, 07 Oct 2013 07:22:02 +0200 Subject: Overriding of the type.__call__() method in a metaclass References: <52521c08$0$29984$c3e8da3$5496439d@news.astraweb.com> Message-ID: On 10/07/2013 04:27 AM, Steven D'Aprano wrote: > On Sun, 06 Oct 2013 20:17:33 +0200, Marco Buttu wrote: >> > >> > >>> class FooMeta(type): >> >... def __call__(metacls, name, bases, namespace): >> >... print("FooMeta.__call__()") ... >> > From what I undestood, at the end of the class statement... >> > >> > >>> def __call__(metacls, name, bases, namespace): >> >... print("FooMeta.__call__()") >> >... >> > >>> FooMeta = type('FooMeta', (type,), {'__call__': __call__}) > > Your code snippet is correct. The "class FooMeta(type)..." statement is > syntactic sugar for type.__call__(...). But your description is > incorrect. This doesn't occur when you "call the metaclass type"... Oh damn! Your are right :) Thanks to you and Peter. Now (I hope...) it should be clear for me: >>> class FooMeta(type): ... def __call__(meta, name, bases, namespace): ... print('FooMeta.__call__()') ... >>> class InstanceOfFooMeta(type, metaclass=FooMeta): ... pass ... >>> class Foo(metaclass=InstanceOfFooMeta): ... pass ... FooMeta.__call__() I try to summarize the execution flow. The metaclass type creates FooMeta: >>> class FooMeta(type): ... def __call__(meta, name, bases, namespace): ... print('FooMeta.__call__()') ... This means at the end of the suite: FooMeta = type('FooMeta', (type,), {...}) So Python calls type. But type is an instance of type itself, so: FooMeta = type.__call__(type, 'FooMeta', (type,), {...}) At this point FooMeta is created. The next step is: >>> class InstanceOfFooMeta(type, metaclass=FooMeta): ... pass This causes: InstanceOfFooMeta= FooMeta('InstanceOfFooMeta', (type,), {...}) Python is calling FooMeta, so it is calling an instance of type, so the code above becomes: InstanceOfFooMeta = type.__call__(FooMeta, 'InstanceOfMeta', \ (type,), {...}) Finally: >>> class Foo(metaclass=InstanceOfFooMeta): ... pass ... FooMeta.__call__() In fact at the end of the suite of the class statement, Python calls an instance of FooMeta: Foo = InstanceOfFooMeta('Foo', (), {...}) so, definitively: Foo = FooMeta.__call__(InstanceOfFooMeta, 'Foo', (), {...}) Foo is None, but never mind. I just wanted to clarify me the class creation process. Thanks again and congratulations for your PEP, it is written very very well -- Marco Buttu From markotaht at gmail.com Sun Oct 6 15:07:19 2013 From: markotaht at gmail.com (markotaht at gmail.com) Date: Sun, 6 Oct 2013 12:07:19 -0700 (PDT) Subject: HEX to ASCII Message-ID: problem is : Traceback (most recent call last): File "C:\Users\Marko\Desktop\hacker.org\XOR cypher.py", line 35, in print("Key-" + str(v?ti) + ": " + str("".join(tulemus2))) TypeError: sequence item 0: expected str instance, bytes found If i take away the join command i get this: Key-00000000: [b'u', b'o', b'\x00', b'\x1d', b' ', b'|', b'N', b'\x0f', b'9', b'j', b'K', b'J', b'&', b'#', b'A', b'K', b'5', b'k', b'_', b'\x1e', b',', b'j', b'\x0c', b'\x08', b'i', b'(', b'\x06', b'\\', b'r', b'3', b'\x1f', b'V', b's', b'9', b'\x1d'] the Key-00000000 is the key im using to decrypt the code. everything else is generated by the decrytion process and the unhexlify command. So my guess is, the join command cant handle the b"u" type of format. how can i get rid of the b. Or does anyone have a better idea how to translate HEX into ASCII and sort out the lines that make sense From __peter__ at web.de Sun Oct 6 15:31:16 2013 From: __peter__ at web.de (Peter Otten) Date: Sun, 06 Oct 2013 21:31:16 +0200 Subject: HEX to ASCII References: Message-ID: markotaht at gmail.com wrote: > problem is : Traceback (most recent call last): > File "C:\Users\Marko\Desktop\hacker.org\XOR cypher.py", line 35, in > > print("Key-" + str(v?ti) + ": " + str("".join(tulemus2))) > TypeError: sequence item 0: expected str instance, bytes found > > If i take away the join command i get this: > Key-00000000: [b'u', b'o', b'\x00', b'\x1d', b' ', b'|', b'N', b'\x0f', > b'9', b'j', b'K', b'J', b'&', b'#', b'A', b'K', b'5', b'k', b'_', > b'\x1e', b',', b'j', b'\x0c', b'\x08', b'i', b'(', b'\x06', b'\\', b'r', > b'3', b'\x1f', b'V', b's', b'9', b'\x1d'] > > the Key-00000000 is the key im using to decrypt the code. everything else > is generated by the decrytion process and the unhexlify command. So my > guess is, the join command cant handle the b"u" type of format. how can i > get rid of the b. On the contrary, you need one more b prefix: >>> tulemus2 = [b'u', b'o', ...] >>> b"".join(tulemus2) b'uo\x00\x1d |N\x0f9jKJ&#AK5k_\x1e,j\x0c\x08i(\x06\\r3\x1fVs9\x1d' > Or does anyone have a better idea how to translate HEX into ASCII and sort > out the lines that make sense That is very likely, but you have to be specific about the input data and what "makes sense" as the output. If you start with a hexdump of binary data you already have the human- readable form, and binascii.unhexlify() will convert it back to the "unreadable" binary form. From python at mrabarnett.plus.com Sun Oct 6 15:36:47 2013 From: python at mrabarnett.plus.com (MRAB) Date: Sun, 06 Oct 2013 20:36:47 +0100 Subject: HEX to ASCII In-Reply-To: References: Message-ID: <5251BBCF.8060402@mrabarnett.plus.com> On 06/10/2013 20:07, markotaht at gmail.com wrote: > problem is : Traceback (most recent call last): > File "C:\Users\Marko\Desktop\hacker.org\XOR cypher.py", line 35, in > print("Key-" + str(v?ti) + ": " + str("".join(tulemus2))) > TypeError: sequence item 0: expected str instance, bytes found > > If i take away the join command i get this: > Key-00000000: [b'u', b'o', b'\x00', b'\x1d', b' ', b'|', b'N', b'\x0f', b'9', b'j', b'K', b'J', b'&', b'#', b'A', b'K', b'5', b'k', b'_', b'\x1e', b',', b'j', b'\x0c', b'\x08', b'i', b'(', b'\x06', b'\\', b'r', b'3', b'\x1f', b'V', b's', b'9', b'\x1d'] > > the Key-00000000 is the key im using to decrypt the code. everything else is generated by the decrytion process and the unhexlify command. So my guess is, the join command cant handle the b"u" type of format. how can i get rid of the b. > > Or does anyone have a better idea how to translate HEX into ASCII and sort out the lines that make sense > (I'm assuming you're using Python 3) tulemus2 is a list of bytestrings (bytes), "" is a Unicode string (str). You can't mix them. Try b"".join(tulemus2) instead. From piet at vanoostrum.org Sun Oct 6 21:27:44 2013 From: piet at vanoostrum.org (Piet van Oostrum) Date: Sun, 06 Oct 2013 21:27:44 -0400 Subject: HEX to ASCII References: Message-ID: markotaht at gmail.com writes: > problem is : Traceback (most recent call last): > File "C:\Users\Marko\Desktop\hacker.org\XOR cypher.py", line 35, in > print("Key-" + str(v?ti) + ": " + str("".join(tulemus2))) > TypeError: sequence item 0: expected str instance, bytes found > > If i take away the join command i get this: > Key-00000000: [b'u', b'o', b'\x00', b'\x1d', b' ', b'|', b'N', b'\x0f', b'9', b'j', b'K', b'J', b'&', b'#', b'A', b'K', b'5', b'k', b'_', b'\x1e', b',', b'j', b'\x0c', b'\x08', b'i', b'(', b'\x06', b'\\', b'r', b'3', b'\x1f', b'V', b's', b'9', b'\x1d'] > > the Key-00000000 is the key im using to decrypt the code. everything else is generated by the decrytion process and the unhexlify command. So my guess is, the join command cant handle the b"u" type of format. how can i get rid of the b. > > Or does anyone have a better idea how to translate HEX into ASCII and sort out the lines that make sense Why do you post the same question twice under different subjects? -- Piet van Oostrum WWW: http://pietvanoostrum.com/ PGP key: [8DAE142BE17999C4] From markotaht at gmail.com Mon Oct 7 09:51:29 2013 From: markotaht at gmail.com (markotaht at gmail.com) Date: Mon, 7 Oct 2013 06:51:29 -0700 (PDT) Subject: HEX to ASCII In-Reply-To: References: Message-ID: esmasp?ev, 7. oktoober 2013 4:27.44 UTC+3 kirjutas Piet van Oostrum: > markotaht at gmail.com writes: > > > > > problem is : Traceback (most recent call last): > > > File "C:\Users\Marko\Desktop\hacker.org\XOR cypher.py", line 35, in > > > print("Key-" + str(v?ti) + ": " + str("".join(tulemus2))) > > > TypeError: sequence item 0: expected str instance, bytes found > > > > > > If i take away the join command i get this: > > > Key-00000000: [b'u', b'o', b'\x00', b'\x1d', b' ', b'|', b'N', b'\x0f', b'9', b'j', b'K', b'J', b'&', b'#', b'A', b'K', b'5', b'k', b'_', b'\x1e', b',', b'j', b'\x0c', b'\x08', b'i', b'(', b'\x06', b'\\', b'r', b'3', b'\x1f', b'V', b's', b'9', b'\x1d'] > > > > > > the Key-00000000 is the key im using to decrypt the code. everything else is generated by the decrytion process and the unhexlify command. So my guess is, the join command cant handle the b"u" type of format. how can i get rid of the b. > > > > > > Or does anyone have a better idea how to translate HEX into ASCII and sort out the lines that make sense > > > > Why do you post the same question twice under different subjects? > > -- Because i was told so, cause the subject change so it cant be under there anymore. This is the code i came up with: from teisendaja import * from operator import * import binascii teisendus = teisendus() kood = input("Kood: ") key = input("V?ti: ") chunksize = 2 vastus = [teisendus.teisendus3(16,2,kood[i: (i + chunksize)]) for i in range(0, len(kood),chunksize)] vastus = ["0"*(8-len(x)) + x for x in vastus] #key = teisendus.teisendus3(10,2,int(key)) getBin = lambda x, n: x >= 0 and str(bin(x))[2:].zfill(n) or "-" + str(bin(x))[3:].zfill(n) def dekr?pteeria(vastus, key): XOR = [] tulemus = [] for i in range(len(vastus)): for j in range(8): XOR.append(str(ixor(int(vastus[i][j]), int(key[j])))) tulemus.append("".join(XOR)) key = "".join(XOR) XOR = [] return tulemus tulemus2= [] if key == "": for i in range(256): v?ti = getBin(i,8) tulemus = [teisendus.teisendus3(2,16,i) for i in dekr?pteeria(vastus, v?ti)] tulemus = ["0"*(2-len(x)) + x for x in tulemus] # for j in range(len(tulemus)): tulemus2.append(binascii.unhexlify("".join(tulemus))) print("Key-" + str(v?ti) + ": " + str(tulemus2)) tulemus2 = [] #tulemus = [teisendus.teisendus3(2,16,i) for i in dekr?pteeria(vastus, key)] #print(":".join(tulemus)) #751a6f1d3d5c3241365321016c05620a7e5e34413246660461412e5a2e412c49254a24 Although this is quite ugly atm. But it serves me well, until i reach the unhexlify part. The number and lette r string at the wery end is the mesage im trying to decypher. From piet at vanoostrum.org Mon Oct 7 11:52:21 2013 From: piet at vanoostrum.org (Piet van Oostrum) Date: Mon, 07 Oct 2013 11:52:21 -0400 Subject: HEX to ASCII References: Message-ID: markotaht at gmail.com writes: > This is the code i came up with: > from teisendaja import * > from operator import * > import binascii > > teisendus = teisendus() > kood = input("Kood: ") > key = input("V?ti: ") > > chunksize = 2 > vastus = [teisendus.teisendus3(16,2,kood[i: (i + chunksize)]) for i in range(0, len(kood),chunksize)] > vastus = ["0"*(8-len(x)) + x for x in vastus] > #key = teisendus.teisendus3(10,2,int(key)) > getBin = lambda x, n: x >= 0 and str(bin(x))[2:].zfill(n) or "-" + str(bin(x))[3:].zfill(n) Instead of boolean and expr1 or expr2 in current Python you can better write: expr1 if boolean else expr2. and I think using def getBin(x, n):would be more clear. But I have another observation: You use getBin only for positive ints, so the whole '-' case isn't necessary. Actually it would be damaging, as the rest of the code assumes that the key that results from getBin is 8 characters long, whereas in the negative case it would be 9 characters. Also you use int(key[j]) and if key[0] == '-' this would give an error. If you just want to get 8 binary digits for an int using format would be simpler: getBin = lambda x, n: '{0:={1}b}'.format(x, n) or getBin = lambda x, n: '{0:=0{1}b}'.format(x, n) if you want also negatives (but this would give you 7 or eight binary digits, not always 8.) > def dekr?pteeria(vastus, key): > XOR = [] > tulemus = [] > for i in range(len(vastus)): > for j in range(8): > XOR.append(str(ixor(int(vastus[i][j]), int(key[j])))) > tulemus.append("".join(XOR)) > key = "".join(XOR) > XOR = [] > return tulemus You can use list comprehension: def dekr?pteeria(vastus, key): tulemus = [] for i in range(len(vastus)): XOR = [(str(ixor(int(vastus[i][j]), int(key[j])))) for j in range(8)] tulemus.append("".join(XOR)) key = "".join(XOR) return tulemus and then because you only use "".join(XOR), not XOR itself: def dekr?pteeria(vastus, key): tulemus = [] for i in range(len(vastus)): XOR = "".join([(str(ixor(int(vastus[i][j]), int(key[j])))) for j in range(8)]) tulemus.append(XOR)) key = XOR return tulemus and then you could rewrite this also to use a list comprehension for tulemus, but that may make it in a too big one liner. Also note that you always use int() on the elements of key and vastus, so it might be simpler to store these as int arrays (lists) instead of strings > > tulemus2= [] > if key == "": > for i in range(256): > v?ti = getBin(i,8) > tulemus = [teisendus.teisendus3(2,16,i) for i in dekr?pteeria(vastus, v?ti)] > tulemus = ["0"*(2-len(x)) + x for x in tulemus] Look at the zfill method for the above 2 line Probably tulemus = [teisendus.teisendus3(2,16,i).zfill(2) for i in dekr?pteeria(vastus, v?ti)] will do the same > # for j in range(len(tulemus)): > tulemus2.append(binascii.unhexlify("".join(tulemus))) > print("Key-" + str(v?ti) + ": " + str(tulemus2)) > tulemus2 = [] > #tulemus = [teisendus.teisendus3(2,16,i) for i in dekr?pteeria(vastus, key)] > #print(":".join(tulemus)) > > #751a6f1d3d5c3241365321016c05620a7e5e34413246660461412e5a2e412c49254a24 > > Although this is quite ugly atm. But it serves me well, until i reach the unhexlify part. The number and lette r string at the wery end is the mesage im trying to decypher. The result of unhexlify is a byte string. So tulemus2 is a list of byte strings, which you can joint with xxx = b''.join(tulemus2), and then you have another byte string. If you are sure this is ASCII (which means all bytes are < 128), the you can convert it to a string with str(xxx, 'ascii') or xxx.decode('ascii'). If there are bytes > 127 then you have to know which encoding it is to be able to make a string out of it. Is this some known encryption method? If so why not use a standard solution for it? If it is a home brew encryption: are you sure it is safe? Most home brew solutions in encryption are not. -- Piet van Oostrum WWW: http://pietvanoostrum.com/ PGP key: [8DAE142BE17999C4] From markotaht at gmail.com Tue Oct 8 04:12:49 2013 From: markotaht at gmail.com (markotaht at gmail.com) Date: Tue, 8 Oct 2013 01:12:49 -0700 (PDT) Subject: HEX to ASCII In-Reply-To: References: Message-ID: <6f306c49-65a1-4c78-bbd8-b67ba5a3a11a@googlegroups.com> esmasp?ev, 7. oktoober 2013 18:52.21 UTC+3 kirjutas Piet van Oostrum: > markotaht at gmail.com writes: > > > > > This is the code i came up with: > > > from teisendaja import * > > > from operator import * > > > import binascii > > > > > > teisendus = teisendus() > > > kood = input("Kood: ") > > > key = input("V?ti: ") > > > > > > chunksize = 2 > > > vastus = [teisendus.teisendus3(16,2,kood[i: (i + chunksize)]) for i in range(0, len(kood),chunksize)] > > > vastus = ["0"*(8-len(x)) + x for x in vastus] > > > #key = teisendus.teisendus3(10,2,int(key)) > > > getBin = lambda x, n: x >= 0 and str(bin(x))[2:].zfill(n) or "-" + str(bin(x))[3:].zfill(n) > > > > Instead of boolean and expr1 or expr2 in current Python you can better write: > > expr1 if boolean else expr2. > > and I think using def getBin(x, n):would be more clear. > > > > But I have another observation: You use getBin only for positive ints, so the whole '-' case isn't necessary. Actually it would be damaging, as the rest of the code assumes that the key that results from getBin is 8 characters long, whereas in the negative case it would be 9 characters. Also you use int(key[j]) and if key[0] == '-' this would give an error. > > If you just want to get 8 binary digits for an int using format would be simpler: > > > > getBin = lambda x, n: '{0:={1}b}'.format(x, n) > > or getBin = lambda x, n: '{0:=0{1}b}'.format(x, n) if you want also negatives (but this would give you 7 or eight binary digits, not always 8.) > > > > > > > def dekr?pteeria(vastus, key): > > > XOR = [] > > > tulemus = [] > > > for i in range(len(vastus)): > > > for j in range(8): > > > XOR.append(str(ixor(int(vastus[i][j]), int(key[j])))) > > > tulemus.append("".join(XOR)) > > > key = "".join(XOR) > > > XOR = [] > > > return tulemus > > > > You can use list comprehension: > > > > def dekr?pteeria(vastus, key): > > tulemus = [] > > for i in range(len(vastus)): > > XOR = [(str(ixor(int(vastus[i][j]), int(key[j])))) for j in range(8)] > > tulemus.append("".join(XOR)) > > key = "".join(XOR) > > return tulemus > > > > and then because you only use "".join(XOR), not XOR itself: > > > > def dekr?pteeria(vastus, key): > > tulemus = [] > > for i in range(len(vastus)): > > XOR = "".join([(str(ixor(int(vastus[i][j]), int(key[j])))) for j in range(8)]) > > tulemus.append(XOR)) > > key = XOR > > return tulemus > > > > and then you could rewrite this also to use a list comprehension for tulemus, but that may make it in a too big one liner. > > > > Also note that you always use int() on the elements of key and vastus, so it might be simpler to store these as int arrays (lists) instead of strings > > > > > > tulemus2= [] > > > if key == "": > > > for i in range(256): > > > v?ti = getBin(i,8) > > > tulemus = [teisendus.teisendus3(2,16,i) for i in dekr?pteeria(vastus, v?ti)] > > > tulemus = ["0"*(2-len(x)) + x for x in tulemus] > > > > Look at the zfill method for the above 2 line > > > > Probably > > tulemus = [teisendus.teisendus3(2,16,i).zfill(2) for i in dekr?pteeria(vastus, v?ti)] > > will do the same > > > > > # for j in range(len(tulemus)): > > > tulemus2.append(binascii.unhexlify("".join(tulemus))) > > > print("Key-" + str(v?ti) + ": " + str(tulemus2)) > > > tulemus2 = [] > > > #tulemus = [teisendus.teisendus3(2,16,i) for i in dekr?pteeria(vastus, key)] > > > #print(":".join(tulemus)) > > > > > > #751a6f1d3d5c3241365321016c05620a7e5e34413246660461412e5a2e412c49254a24 > > > > > > Although this is quite ugly atm. But it serves me well, until i reach the unhexlify part. The number and lette r string at the wery end is the mesage im trying to decypher. > > > > The result of unhexlify is a byte string. So tulemus2 is a list of byte strings, which you can joint with xxx = b''.join(tulemus2), and then you have another byte string. If you are sure this is ASCII (which means all bytes are < 128), the you can convert it to a string with str(xxx, 'ascii') or xxx.decode('ascii'). If there are bytes > 127 then you have to know which encoding it is to be able to make a string out of it. > > > > Is this some known encryption method? If so why not use a standard solution for it? If it is a home brew encryption: are you sure it is safe? Most home brew solutions in encryption are not. > > -- > > Piet van Oostrum > > WWW: http://pietvanoostrum.com/ > > PGP key: [8DAE142BE17999 Are you familira with the page called hacker.org? THere are tons of challenges. And this is one of the cypher challenges called Feedback cypher. The end result should be in the format of "The antswer is ....." Or something like that. The problem is that the key is unknown. So i try all the 255 keys there are and then il look for the line that makes sense. From markotaht at gmail.com Mon Oct 7 09:54:12 2013 From: markotaht at gmail.com (markotaht at gmail.com) Date: Mon, 7 Oct 2013 06:54:12 -0700 (PDT) Subject: HEX to ASCII In-Reply-To: References: Message-ID: <0d9d19ae-8cba-4429-9777-8631df9f6bab@googlegroups.com> I forgot to tell. The teisendaja module that i have imported, is a number converter that allow to convert numbers from one base to another. i mostly use it for HEX to BIN and vice versa, but it supports other bases too. From breamoreboy at yahoo.co.uk Mon Oct 7 10:16:29 2013 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Mon, 07 Oct 2013 15:16:29 +0100 Subject: HEX to ASCII In-Reply-To: <0d9d19ae-8cba-4429-9777-8631df9f6bab@googlegroups.com> References: <0d9d19ae-8cba-4429-9777-8631df9f6bab@googlegroups.com> Message-ID: On 07/10/2013 14:54, markotaht at gmail.com wrote: > I forgot to tell. The teisendaja module that i have imported, is a number converter that allow to convert numbers from one base to another. i mostly use it for HEX to BIN and vice versa, but it supports other bases too. > That's nice to know, but what has it got to do with the market price of oranges in Timbuktu? Or to put it another way, you're forcing volunteers to go and find your original message as once again you don't quote any context. Please make life easier for everybody, including yourself, by quoting something from the original. Thanks in anticipation. -- Roses are red, Violets are blue, Most poems rhyme, But this one doesn't. Mark Lawrence From markotaht at gmail.com Tue Oct 8 04:13:17 2013 From: markotaht at gmail.com (markotaht at gmail.com) Date: Tue, 8 Oct 2013 01:13:17 -0700 (PDT) Subject: HEX to ASCII In-Reply-To: References: <0d9d19ae-8cba-4429-9777-8631df9f6bab@googlegroups.com> Message-ID: esmasp?ev, 7. oktoober 2013 17:16.29 UTC+3 kirjutas Mark Lawrence: > On 07/10/2013 14:54, markotaht at gmail.com wrote: > > > I forgot to tell. The teisendaja module that i have imported, is a number converter that allow to convert numbers from one base to another. i mostly use it for HEX to BIN and vice versa, but it supports other bases too. > > > > > > > That's nice to know, but what has it got to do with the market price of > > oranges in Timbuktu? Or to put it another way, you're forcing > > volunteers to go and find your original message as once again you don't > > quote any context. Please make life easier for everybody, including > > yourself, by quoting something from the original. > > > > Thanks in anticipation. > > > > -- > > Roses are red, > > Violets are blue, > > Most poems rhyme, > > But this one doesn't. > > > > Mark Lawrence teisendaja module doesent matter. Only thing that matters about is that it returns a string with the converted number. From breamoreboy at yahoo.co.uk Tue Oct 8 04:25:06 2013 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Tue, 08 Oct 2013 09:25:06 +0100 Subject: HEX to ASCII In-Reply-To: References: <0d9d19ae-8cba-4429-9777-8631df9f6bab@googlegroups.com> Message-ID: On 08/10/2013 09:13, markotaht at gmail.com wrote: > esmasp?ev, 7. oktoober 2013 17:16.29 UTC+3 kirjutas Mark Lawrence: >> On 07/10/2013 14:54, markotaht at gmail.com wrote: >> >>> I forgot to tell. The teisendaja module that i have imported, is a number converter that allow to convert numbers from one base to another. i mostly use it for HEX to BIN and vice versa, but it supports other bases too. >> >> That's nice to know, but what has it got to do with the market price of >> oranges in Timbuktu? Or to put it another way, you're forcing >> volunteers to go and find your original message as once again you don't >> quote any context. Please make life easier for everybody, including >> yourself, by quoting something from the original. >> Thanks in anticipation. >> >> -- >> >> Roses are red, >> Violets are blue, >> Most poems rhyme, >> But this one doesn't. >> Mark Lawrence > teisendaja module doesent matter. Only thing that matters about is that it returns a string with the converted number. > Actually another thing that matters is finding a technology that doesn't spread superfluous newlines throughout emails. I've snipped them above. Please take note of this https://wiki.python.org/moin/GoogleGroupsPython. Thanks in aniticipation. -- Roses are red, Violets are blue, Most poems rhyme, But this one doesn't. Mark Lawrence From jabba.laci at gmail.com Sun Oct 6 17:25:03 2013 From: jabba.laci at gmail.com (Jabba Laci) Date: Sun, 6 Oct 2013 23:25:03 +0200 Subject: nginx config: different projects in different directories Message-ID: Hi, I'm playing with Flask and I would like to try it in production environment too. I managed to bring Flask together with uwsgi and nginx. My Flask application is available at the address localhost:81 . I would like to add several applications and I want them to be available under different URLs. For instance, if I have two projects called "hello" and "world", I want to access them as localhost:81/hello/ and localhost:81/world/ . The problem is I can't figure out how to configure nginx for this. Here is my current setup: * The project "hello" is in this directory: /home/jabba/public_pyapps/hello/ * Its nginx entry: server { listen 81; server_name localhost; charset utf-8; client_max_body_size 75M; location / { try_files $uri @yourapplication; } location @yourapplication { include uwsgi_params; uwsgi_pass unix:/home/jabba/public_pyapps/hello/hello_uwsgi.sock; } } It's available at localhost:81 . Questions: (1) How to make it available under localhost:81/hello/ instead? (2) If I add a new application (e.g. "world"), how to add it to nginx? Thanks, Laszlo From roy at panix.com Sun Oct 6 18:31:45 2013 From: roy at panix.com (Roy Smith) Date: Sun, 06 Oct 2013 18:31:45 -0400 Subject: nginx config: different projects in different directories References: Message-ID: In article , Jabba Laci wrote: > The problem is I can't > figure out how to configure nginx for this. This is a python mailing list. You would do better asking nginx questions on an nginx mailing list. From rhjacksoniii at gmail.com Sun Oct 6 18:47:38 2013 From: rhjacksoniii at gmail.com (Robert Jackson) Date: Sun, 6 Oct 2013 18:47:38 -0400 Subject: Strange extra f added to bytes object Message-ID: I am very new to python so I'll apologize up front if this is some boneheaded thing. I am using python and pyserial to talk to an embedded pic processor in a piece of scientific equipment. I sometimes find the when I construct the bytes object to write it adds an extra f to the first byte. For example if I have b'\x03\x66\x02\x01\xaa\xbb' it evaluates to b'\x03f\x02\x01\xaa\xbb', which doesn't even seem valid. Can anyone shine some light this? -------------- next part -------------- An HTML attachment was scrubbed... URL: From ppearson at nowhere.invalid Sun Oct 6 20:32:34 2013 From: ppearson at nowhere.invalid (Peter Pearson) Date: 7 Oct 2013 00:32:34 GMT Subject: Strange extra f added to bytes object References: Message-ID: On Sun, 6 Oct 2013 18:47:38 -0400, Robert Jackson wrote: > --089e0160b7be912b9e04e81a52b2 > Content-Type: text/plain; charset=ISO-8859-1 > > I am very new to python [snip] Welcome. > . . . I sometimes find the > when I construct the bytes object to write it adds an extra f to the first > byte. > > For example if I have b'\x03\x66\x02\x01\xaa\xbb' it evaluates > to b'\x03f\x02\x01\xaa\xbb', which doesn't even seem valid. > > Can anyone shine some light this? "f" is the same as \x66; nothing has been changed. -- To email me, substitute nowhere->spamcop, invalid->net. From joel.goldstick at gmail.com Sun Oct 6 20:39:19 2013 From: joel.goldstick at gmail.com (Joel Goldstick) Date: Sun, 6 Oct 2013 20:39:19 -0400 Subject: Strange extra f added to bytes object In-Reply-To: References: Message-ID: On Sun, Oct 6, 2013 at 8:32 PM, Peter Pearson wrote: > On Sun, 6 Oct 2013 18:47:38 -0400, Robert Jackson wrote: >> --089e0160b7be912b9e04e81a52b2 >> Content-Type: text/plain; charset=ISO-8859-1 >> >> I am very new to python > [snip] > > Welcome. > > >> . . . I sometimes find the >> when I construct the bytes object to write it adds an extra f to the first >> byte. >> >> For example if I have b'\x03\x66\x02\x01\xaa\xbb' it evaluates >> to b'\x03f\x02\x01\xaa\xbb', which doesn't even seem valid. >> >> Can anyone shine some light this? > > "f" is the same as \x66; nothing has been changed. really? I would expect that \x66 = 0110 0110 and f = 1111 > > -- > To email me, substitute nowhere->spamcop, invalid->net. > -- > https://mail.python.org/mailman/listinfo/python-list -- Joel Goldstick http://joelgoldstick.com From ian.g.kelly at gmail.com Sun Oct 6 20:44:19 2013 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Sun, 6 Oct 2013 18:44:19 -0600 Subject: Strange extra f added to bytes object In-Reply-To: References: Message-ID: On Sun, Oct 6, 2013 at 6:39 PM, Joel Goldstick wrote: >> "f" is the same as \x66; nothing has been changed. > > really? I would expect that \x66 = 0110 0110 and f = 1111 The f here is the ASCII character f, not the hex digit f: >>> bin(ord(b'f')) '0b1100110' From python at mrabarnett.plus.com Sun Oct 6 20:38:15 2013 From: python at mrabarnett.plus.com (MRAB) Date: Mon, 07 Oct 2013 01:38:15 +0100 Subject: Strange extra f added to bytes object In-Reply-To: References: Message-ID: <52520277.7000205@mrabarnett.plus.com> On 06/10/2013 23:47, Robert Jackson wrote: > I am very new to python so I'll apologize up front if this is some > boneheaded thing. I am using python and pyserial to talk to an embedded > pic processor in a piece of scientific equipment. I sometimes find the > when I construct the bytes object to write it adds an extra f to the > first byte. > > For example if I have b'\x03\x66\x02\x01\xaa\xbb' it evaluates > to b'\x03f\x02\x01\xaa\xbb', which doesn't even seem valid. > > Can anyone shine some light this? > >>> b'\x66' == b'f' True Python always prints a bytestring the same way. It doesn't 'remember' how it was originally created. Another example: >>> b'\x68\x65\x6c\x6c\x6f\x20\x77\x6f\x72\x6c\x64' b'hello world' From joel.goldstick at gmail.com Sun Oct 6 20:41:52 2013 From: joel.goldstick at gmail.com (Joel Goldstick) Date: Sun, 6 Oct 2013 20:41:52 -0400 Subject: Strange extra f added to bytes object In-Reply-To: <52520277.7000205@mrabarnett.plus.com> References: <52520277.7000205@mrabarnett.plus.com> Message-ID: On Sun, Oct 6, 2013 at 8:38 PM, MRAB wrote: > On 06/10/2013 23:47, Robert Jackson wrote: >> >> I am very new to python so I'll apologize up front if this is some >> boneheaded thing. I am using python and pyserial to talk to an embedded >> pic processor in a piece of scientific equipment. I sometimes find the >> when I construct the bytes object to write it adds an extra f to the >> first byte. >> >> For example if I have b'\x03\x66\x02\x01\xaa\xbb' it evaluates >> to b'\x03f\x02\x01\xaa\xbb', which doesn't even seem valid. >> >> Can anyone shine some light this? >> >>>> b'\x66' == b'f' > True > > Python always prints a bytestring the same way. It doesn't 'remember' > how it was originally created. > > Another example: > >>>> b'\x68\x65\x6c\x6c\x6f\x20\x77\x6f\x72\x6c\x64' > b'hello world' ah, now I see ascii 'f' = \x66 that's a double take! > > -- > https://mail.python.org/mailman/listinfo/python-list -- Joel Goldstick http://joelgoldstick.com From ned at nedbatchelder.com Sun Oct 6 20:39:39 2013 From: ned at nedbatchelder.com (Ned Batchelder) Date: Sun, 06 Oct 2013 20:39:39 -0400 Subject: Strange extra f added to bytes object In-Reply-To: References: Message-ID: <525202CB.3090704@nedbatchelder.com> On 10/6/13 6:47 PM, Robert Jackson wrote: > I am very new to python so I'll apologize up front if this is some > boneheaded thing. I am using python and pyserial to talk to an > embedded pic processor in a piece of scientific equipment. I > sometimes find the when I construct the bytes object to write it adds > an extra f to the first byte. > > For example if I have b'\x03\x66\x02\x01\xaa\xbb' it evaluates > to b'\x03f\x02\x01\xaa\xbb', which doesn't even seem valid. > > Can anyone shine some light this? > > b'\x66' == b'f' . The hex for "f" is 66. The f isn't inserted, it's the second byte of your string. When Python displays a string, is uses the ASCII character if it can, and a hex escape if it can't. When you use a hex value that is a valid ASCII character, it will display the character. --Ned. -------------- next part -------------- An HTML attachment was scrubbed... URL: From tjreedy at udel.edu Sun Oct 6 20:46:20 2013 From: tjreedy at udel.edu (Terry Reedy) Date: Sun, 06 Oct 2013 20:46:20 -0400 Subject: Strange extra f added to bytes object In-Reply-To: References: Message-ID: On 10/6/2013 6:47 PM, Robert Jackson wrote: > I am very new to python so I'll apologize up front if this is some Welcome to a mostly great language. > boneheaded thing. I am using python and pyserial to talk to an embedded > pic processor in a piece of scientific equipment. I sometimes find the > when I construct the bytes object to write it adds an extra f to the > first byte. > > For example if I have b'\x03\x66\x02\x01\xaa\xbb' it evaluates > to b'\x03f\x02\x01\xaa\xbb', which doesn't even seem valid. Python (or at least cpython) uses ascii chars to display bytes when possible. This is often helpful, but not always ;-0. >>> b'\x66' == b'f' True When you have b'\x03\x66\x02\x01\xaa\xbb', which is the same as b'\x03f\x02\x01\xaa\xbb', the latter is used for display. So you do not really have a problem. Some various thing you can do to get various printouts. >>> b=b'\x03f\x02\x01\xaa\xbb' >>> list(b) [3, 102, 2, 1, 170, 187] >>> ' '.join('%x' % (c,) for c in b) '3 66 2 1 aa bb' >>> ''.join('\\x%x' % (c,) for c in b) '\\x3\\x66\\x2\\x1\\xaa\\xbb' 21 chars; each \\ represents one \ char, -- Terry Jan Reedy From steve+comp.lang.python at pearwood.info Sun Oct 6 21:07:31 2013 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: 07 Oct 2013 01:07:31 GMT Subject: Strange extra f added to bytes object References: Message-ID: <52520953$0$29984$c3e8da3$5496439d@news.astraweb.com> On Sun, 06 Oct 2013 20:39:39 -0400, Ned Batchelder wrote: > When Python displays a string, is uses A byte string. > the ASCII character if it can, and a hex escape if it can't. When you > use a hex value that is a valid ASCII character, it will display the > character. Obviously for Python 2 that behaviour can't change, but I am saddened that the opportunity to fix the display of byte strings in Python 3 wasn't taken. In my opinion, it would have been much better if byte strings were always shown in hex. (They could also have a separate method for showing them in ASCII, if necessary, but even that is only one call to decode() away.) Displaying a *byte* string using ASCII by default just continues the confusion that many people have, that the character "f" is necessarily the same as the byte 0x66. *And* it leads to the OP's error, wondering why his byte-stream of 0x66... displays with a strange 'f'. If you ask me, including ASCII in the printable representation of byte strings in Python 3 is buggy by design :-( -- Steven From rhjacksoniii at gmail.com Sun Oct 6 21:38:27 2013 From: rhjacksoniii at gmail.com (Robert Jackson) Date: Sun, 6 Oct 2013 21:38:27 -0400 Subject: Strange extra f added to bytes object In-Reply-To: <52520953$0$29984$c3e8da3$5496439d@news.astraweb.com> References: <52520953$0$29984$c3e8da3$5496439d@news.astraweb.com> Message-ID: Thank you all. It was unfortunate that it was f since I thought it was some strange mistaken hex nibble. All very clear and helpful. On Sun, Oct 6, 2013 at 9:07 PM, Steven D'Aprano < steve+comp.lang.python at pearwood.info> wrote: > On Sun, 06 Oct 2013 20:39:39 -0400, Ned Batchelder wrote: > > > When Python displays a string, is uses > > A byte string. > > > the ASCII character if it can, and a hex escape if it can't. When you > > use a hex value that is a valid ASCII character, it will display the > > character. > > Obviously for Python 2 that behaviour can't change, but I am saddened > that the opportunity to fix the display of byte strings in Python 3 > wasn't taken. In my opinion, it would have been much better if byte > strings were always shown in hex. (They could also have a separate method > for showing them in ASCII, if necessary, but even that is only one call > to decode() away.) > > Displaying a *byte* string using ASCII by default just continues the > confusion that many people have, that the character "f" is necessarily > the same as the byte 0x66. *And* it leads to the OP's error, wondering > why his byte-stream of 0x66... displays with a strange 'f'. > > If you ask me, including ASCII in the printable representation of byte > strings in Python 3 is buggy by design :-( > > > -- > Steven > -- > https://mail.python.org/mailman/listinfo/python-list > -------------- next part -------------- An HTML attachment was scrubbed... URL: From toby at tobiah.org Mon Oct 7 14:08:17 2013 From: toby at tobiah.org (Tobiah) Date: Mon, 07 Oct 2013 11:08:17 -0700 Subject: Mysql's mysql module Message-ID: I just noticed this: http://dev.mysql.com/doc/connector-python/en/index.html What are the thoughts on this vs. the MySQLdb with which I'm familiar? I also noticed MySQLdb2. I was wondering whether to use this version on a new project that is likely to take some months to develop. Thanks, Tobiah From skip at pobox.com Mon Oct 7 15:01:06 2013 From: skip at pobox.com (Skip Montanaro) Date: Mon, 7 Oct 2013 14:01:06 -0500 Subject: Mysql's mysql module In-Reply-To: References: Message-ID: On Mon, Oct 7, 2013 at 1:08 PM, Tobiah wrote: > I just noticed this: > > > http://dev.mysql.com/doc/connector-python/en/index.html * Does it adhere to the Python database API? http://www.python.org/dev/peps/pep-0249/ * Is source available? * Does it have a reasonable open source license? These questions come immediately to mind because at work we briefly considered, then rejected, a similar offering from the Sybase folks for connecting to (big surprise) Sybase. We never needed to ask the first and third questions, because the answer to the second was, "no", and they only offered a version built against Python 2.6. Since we use Python 2.4 and 2.7, that was an immediate nonstarter. Skip From duncan.booth at invalid.invalid Mon Oct 7 15:31:47 2013 From: duncan.booth at invalid.invalid (Duncan Booth) Date: 7 Oct 2013 19:31:47 GMT Subject: Mysql's mysql module References: Message-ID: Skip Montanaro wrote: > On Mon, Oct 7, 2013 at 1:08 PM, Tobiah wrote: >> I just noticed this: >> >> >> http://dev.mysql.com/doc/connector-python/en/index.html > > * Does it adhere to the Python database API? > http://www.python.org/dev/peps/pep-0249/ > > * Is source available? > > * Does it have a reasonable open source license? > > These questions come immediately to mind because at work we briefly > considered, then rejected, a similar offering from the Sybase folks > for connecting to (big surprise) Sybase. We never needed to ask the > first and third questions, because the answer to the second was, "no", > and they only offered a version built against Python 2.6. Since we use > Python 2.4 and 2.7, that was an immediate nonstarter. > > Skip Based on a quick look at the link given, I think the answers to questions 1 and 3 are yes and no respectively. No idea about #2. -- Duncan Booth http://kupuguy.blogspot.com From kwpolska at gmail.com Mon Oct 7 15:54:38 2013 From: kwpolska at gmail.com (=?UTF-8?B?Q2hyaXMg4oCcS3dwb2xza2HigJ0gV2Fycmljaw==?=) Date: Mon, 7 Oct 2013 21:54:38 +0200 Subject: Mysql's mysql module In-Reply-To: References: Message-ID: On Oct 7, 2013 9:36 PM, "Duncan Booth" wrote: > > Skip Montanaro wrote: > > > On Mon, Oct 7, 2013 at 1:08 PM, Tobiah wrote: > >> I just noticed this: > >> > >> > >> http://dev.mysql.com/doc/connector-python/en/index.html > > > > * Does it adhere to the Python database API? > > http://www.python.org/dev/peps/pep-0249/ > > > > * Is source available? > > > > * Does it have a reasonable open source license? > > > > These questions come immediately to mind because at work we briefly > > considered, then rejected, a similar offering from the Sybase folks > > for connecting to (big surprise) Sybase. We never needed to ask the > > first and third questions, because the answer to the second was, "no", > > and they only offered a version built against Python 2.6. Since we use > > Python 2.4 and 2.7, that was an immediate nonstarter. > > > > Skip > > Based on a quick look at the link given, I think the answers to questions 1 > and 3 are yes and no respectively. No idea about #2. Number two is "yes", unless there are deceptive statements on that page. > > -- > Duncan Booth http://kupuguy.blogspot.com > -- > https://mail.python.org/mailman/listinfo/python-list -------------- next part -------------- An HTML attachment was scrubbed... URL: From overhaalsgang_24_bob at me.com Mon Oct 7 15:12:40 2013 From: overhaalsgang_24_bob at me.com (BobAalsma) Date: Mon, 7 Oct 2013 12:12:40 -0700 (PDT) Subject: Newbie: installation difficulties [webapp2 / babel] Message-ID: Hi, I'm following webapp2 documentation (release 2.1). I made a mistake in following the text. I typed "pip install babel" and this led to errors in the installation. As that user is not in sudo list, I changed users, typed "sudo pip install babel" and everything seemed right. Further on, the manual says "$ pybabel ...." and when I use that command, the response is "-bash: pybabel: command not found" (in both user environments). I'd like to solve this and learn at the same time ;) I'm on OS X 10.8.5, python 2.7.5. I found a "babel" in /Library/Python/2.7/site-packages. I found a "babel" in the response to "python -c "help('modules')" | grep babel". >From which I conclude that babel is on the machine and is known to python. However, I gather from the webapp2 documentation that babel should also be known to the shell. I don't yet know how to check this, nor how to repair. find /usr -name '*babel*' and find /bin -name '*babel*' return no values, but find /Library -name '*babel*' does (as expected). I've tried to find similar situations in documentation, forums, Google but found nothing helpful. From joel.goldstick at gmail.com Mon Oct 7 15:31:52 2013 From: joel.goldstick at gmail.com (Joel Goldstick) Date: Mon, 7 Oct 2013 15:31:52 -0400 Subject: Newbie: installation difficulties [webapp2 / babel] In-Reply-To: References: Message-ID: On Mon, Oct 7, 2013 at 3:12 PM, BobAalsma wrote: > Hi, > > I'm following webapp2 documentation (release 2.1). > > I made a mistake in following the text. > I typed "pip install babel" and this led to errors in the installation. > As that user is not in sudo list, I changed users, typed "sudo pip install babel" and everything seemed right. > > Further on, the manual says "$ pybabel ...." and when I use that command, the response is "-bash: pybabel: command not found" (in both user environments). > > I'd like to solve this and learn at the same time ;) > > I'm on OS X 10.8.5, python 2.7.5. > > I found a "babel" in /Library/Python/2.7/site-packages. > I found a "babel" in the response to "python -c "help('modules')" | grep babel". > From which I conclude that babel is on the machine and is known to python. > > However, I gather from the webapp2 documentation that babel should also be known to the shell. > I don't yet know how to check this, nor how to repair. > > find /usr -name '*babel*' > and > find /bin -name '*babel*' > return no values, but > find /Library -name '*babel*' > does (as expected). > > I've tried to find similar situations in documentation, forums, Google but found nothing helpful. > -- > https://mail.python.org/mailman/listinfo/python-list If you can run $python babel then I think you need to set babel to be executable to run it without invoking python first. -- Joel Goldstick http://joelgoldstick.com From overhaalsgang_24_bob at me.com Mon Oct 7 16:00:58 2013 From: overhaalsgang_24_bob at me.com (BobAalsma) Date: Mon, 7 Oct 2013 13:00:58 -0700 (PDT) Subject: Newbie: installation difficulties [webapp2 / babel] In-Reply-To: References: Message-ID: Well Joel, umm, I'm not sure if I understand you correctly. $ python babel /Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python: can't open file 'babel': [Errno 2] No such file or directory And $ python Python 2.7.5 (v2.7.5:ab05e7dd2788, May 13 2013, 13:18:45) [GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> import babel >>> Does this help? Bob From joel.goldstick at gmail.com Mon Oct 7 16:18:50 2013 From: joel.goldstick at gmail.com (Joel Goldstick) Date: Mon, 7 Oct 2013 16:18:50 -0400 Subject: Newbie: installation difficulties [webapp2 / babel] In-Reply-To: References: Message-ID: On Mon, Oct 7, 2013 at 4:00 PM, BobAalsma wrote: > Well Joel, umm, I'm not sure if I understand you correctly. > > $ python babel > /Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python: can't open file 'babel': [Errno 2] No such file or directory > > And > > > $ python > Python 2.7.5 (v2.7.5:ab05e7dd2788, May 13 2013, 13:18:45) > [GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin > Type "help", "copyright", "credits" or "license" for more information. >>>> import babel >>>> > > Does this help? > > Bob > -- > https://mail.python.org/mailman/listinfo/python-list Bob, I'm not a pro at getting python stuff to run on Apple, but you might learn something here: http://docs.python.org/2/using/mac.html#using-python-on-a-macintosh >From the looks of it, babel doesn't seem to be in a directory where the mac looks for executables. You may want to cd yourself over to where babel is and try to run it. If that works, you have to figure out how to get babel in your executable path. I can't help you there. -- Joel Goldstick http://joelgoldstick.com From rosuav at gmail.com Mon Oct 7 17:30:03 2013 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 8 Oct 2013 08:30:03 +1100 Subject: Newbie: installation difficulties [webapp2 / babel] In-Reply-To: References: Message-ID: On Tue, Oct 8, 2013 at 7:00 AM, BobAalsma wrote: > Well Joel, umm, I'm not sure if I understand you correctly. > > $ python babel > /Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python: can't open file 'babel': [Errno 2] No such file or directory If the file's called babel.py, you have to invoke it that way - Python won't add an implicit file extension. Not sure if that helps your problem or not - I don't know babel, nor Mac OS. Try: $ python babel.py ChrisA From Chandru_Rajendran at infosys.com Tue Oct 8 01:34:55 2013 From: Chandru_Rajendran at infosys.com (Chandru Rajendran) Date: Tue, 8 Oct 2013 05:34:55 +0000 Subject: Multiprocessing and Multithreading Message-ID: Hi all, Please give me an idea about Multiprocessing and Multithreading. Thanks & Regards, Chandru **************** CAUTION - Disclaimer ***************** This e-mail contains PRIVILEGED AND CONFIDENTIAL INFORMATION intended solely for the use of the addressee(s). If you are not the intended recipient, please notify the sender by e-mail and delete the original message. Further, you are not to copy, disclose, or distribute this e-mail or its contents to any other person and any such actions are unlawful. This e-mail may contain viruses. Infosys has taken every reasonable precaution to minimize this risk, but is not liable for any damage you may sustain as a result of any virus in this e-mail. You should carry out your own virus checks before opening the e-mail or attachment. Infosys reserves the right to monitor and review the content of all messages sent to or from this e-mail address. Messages sent to or from this e-mail address may be stored on the Infosys e-mail system. ***INFOSYS******** End of Disclaimer ********INFOSYS*** -------------- next part -------------- An HTML attachment was scrubbed... URL: From tjreedy at udel.edu Tue Oct 8 03:16:25 2013 From: tjreedy at udel.edu (Terry Reedy) Date: Tue, 08 Oct 2013 03:16:25 -0400 Subject: Multiprocessing and Multithreading In-Reply-To: References: Message-ID: On 10/8/2013 1:34 AM, Chandru Rajendran wrote: > Please give me an idea about Multiprocessing and Multithreading. Please give us some idea of what you know and what you actually want to know. > **************** CAUTION - Disclaimer ***************** > This e-mail contains PRIVILEGED AND CONFIDENTIAL INFORMATION intended solely > for the use of the addressee(s). If you are not the intended recipient, please > notify the sender by e-mail and delete the original message. Further, you are not > to copy, disclose, or distribute this e-mail or its contents to any other person and > any such actions are unlawful. This e-mail may contain viruses. Infosys has taken > every reasonable precaution to minimize this risk, but is not liable for any damage > you may sustain as a result of any virus in this e-mail. You should carry out your > own virus checks before opening the e-mail or attachment. Infosys reserves the > right to monitor and review the content of all messages sent to or from this e-mail > address. Messages sent to or from this e-mail address may be stored on the > Infosys e-mail system. > ***INFOSYS******** End of Disclaimer ********INFOSYS*** And please try to avoid this bulky disclaimer that is a lie. -- Terry Jan Reedy From breamoreboy at yahoo.co.uk Tue Oct 8 03:21:51 2013 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Tue, 08 Oct 2013 08:21:51 +0100 Subject: Multiprocessing and Multithreading In-Reply-To: References: Message-ID: On 08/10/2013 06:34, Chandru Rajendran wrote: > Hi all, > > Please give me an idea about Multiprocessing and Multithreading. > > Thanks & Regards, > > Chandru > I'll assume that you're a newbie so I'll keep it simple. Multiprocessing is about more than one process and multithreading is about more than one thread. If you want Python specifics you could start here http://docs.python.org/3/library/multiprocessing.html or here http://docs.python.org/3/library/threading.html, both of which may have been found by using your favourite search engine. -- Roses are red, Violets are blue, Most poems rhyme, But this one doesn't. Mark Lawrence From invalid at invalid.invalid Tue Oct 8 10:05:58 2013 From: invalid at invalid.invalid (Grant Edwards) Date: Tue, 8 Oct 2013 14:05:58 +0000 (UTC) Subject: Multiprocessing and Multithreading References: Message-ID: On 2013-10-08, Mark Lawrence wrote: > On 08/10/2013 06:34, Chandru Rajendran wrote: >> Hi all, >> >> Please give me an idea about Multiprocessing and Multithreading. >> >> Thanks & Regards, >> >> Chandru >> > > I'll assume that you're a newbie so I'll keep it simple. > Multiprocessing is about more than one process and multithreading is > about more than one thread. I doubt a newbie knows the difference between a thread and a process. Threads share all memory and global variabls. They can communicate with each other through global variables. Processes are completely isolated from each other and much communicate with each other through mechanisms provided by the OS (e.g. sockets, mailboxes, pipes, files). -- Grant Edwards grant.b.edwards Yow! Are the STEWED PRUNES at still in the HAIR DRYER? gmail.com From rosuav at gmail.com Tue Oct 8 02:17:21 2013 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 8 Oct 2013 17:17:21 +1100 Subject: Code golf challenge: XKCD 936 passwords Message-ID: Who's up for some fun? Implement an XKCD-936-compliant password generator in Python 3, in less code than this: print(*__import__("random").sample(open("/usr/share/dict/words").read().split("\n"),4)) Second challenge: Use it for generating all your passwords :) [1] https://en.wikipedia.org/wiki/Code_golf [2] http://xkcd.com/936/ ChrisA From sprucebondera at gmail.com Tue Oct 8 02:45:39 2013 From: sprucebondera at gmail.com (sprucebondera at gmail.com) Date: Mon, 7 Oct 2013 23:45:39 -0700 (PDT) Subject: Code golf challenge: XKCD 936 passwords In-Reply-To: References: Message-ID: <68365e43-498f-4ad2-bac3-6a02938159c7@googlegroups.com> On Monday, October 7, 2013 8:17:21 PM UTC-10, Chris Angelico wrote: > Who's up for some fun? Implement an XKCD-936-compliant password > > generator in Python 3, in less code than this: > > > > print(*__import__("random").sample(open("/usr/share/dict/words").read().split("\n"),4)) > > > > Second challenge: Use it for generating all your passwords :) > > > > [1] https://en.wikipedia.org/wiki/Code_golf > > [2] http://xkcd.com/936/ > > > > ChrisA Well, here's a start: import random as r print(*r.sample(open("/usr/share/dict/words").readlines(),4)) Shaves off 6 characters. From sprucebondera at gmail.com Tue Oct 8 02:48:01 2013 From: sprucebondera at gmail.com (sprucebondera at gmail.com) Date: Mon, 7 Oct 2013 23:48:01 -0700 (PDT) Subject: Code golf challenge: XKCD 936 passwords In-Reply-To: <68365e43-498f-4ad2-bac3-6a02938159c7@googlegroups.com> References: <68365e43-498f-4ad2-bac3-6a02938159c7@googlegroups.com> Message-ID: On Monday, October 7, 2013 8:45:39 PM UTC-10, spruce... at gmail.com wrote: > On Monday, October 7, 2013 8:17:21 PM UTC-10, Chris Angelico wrote: > > > Who's up for some fun? Implement an XKCD-936-compliant password > > > > > > generator in Python 3, in less code than this: > > > > > > > > > > > > print(*__import__("random").sample(open("/usr/share/dict/words").read().split("\n"),4)) > > > > > > > > > > > > Second challenge: Use it for generating all your passwords :) > > > > > > > > > > > > [1] https://en.wikipedia.org/wiki/Code_golf > > > > > > [2] http://xkcd.com/936/ > > > > > > > > > > > > ChrisA > > > > Well, here's a start: > > > > import random as r > > print(*r.sample(open("/usr/share/dict/words").readlines(),4)) > > > > Shaves off 6 characters. And if we were actually trying then that filename should just be "/w". Would get rid of another 19 chars. From rosuav at gmail.com Tue Oct 8 02:52:03 2013 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 8 Oct 2013 17:52:03 +1100 Subject: Code golf challenge: XKCD 936 passwords In-Reply-To: References: <68365e43-498f-4ad2-bac3-6a02938159c7@googlegroups.com> Message-ID: On Tue, Oct 8, 2013 at 5:48 PM, wrote: > And if we were actually trying then that filename should just be "/w". Would get rid of another 19 chars. I'm working this on the assumption that the dictionary file already exists (that's where it is on my Debian Linux systems, for instance) and shouldn't be moved :) ChrisA From square.steve at gmail.com Tue Oct 8 03:02:25 2013 From: square.steve at gmail.com (Steve Simmons) Date: Tue, 08 Oct 2013 08:02:25 +0100 Subject: Code golf challenge: XKCD 936 passwords In-Reply-To: References: <68365e43-498f-4ad2-bac3-6a02938159c7@googlegroups.com> Message-ID: <842d8116-fa7b-4b8f-8a8b-425bd5d3cd5d@email.android.com> Chris Angelico wrote: >On Tue, Oct 8, 2013 at 5:48 PM, wrote: >> And if we were actually trying then that filename should just be >"/w". Would get rid of another 19 chars. > >I'm working this on the assumption that the dictionary file already >exists (that's where it is on my Debian Linux systems, for instance) >and shouldn't be moved :) > >ChrisA >-- >https://mail.python.org/mailman/listinfo/python-list Typical MUD Master - making up rules as you go along :-) Sent from a Galaxy far far away -------------- next part -------------- An HTML attachment was scrubbed... URL: From rosuav at gmail.com Tue Oct 8 03:13:39 2013 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 8 Oct 2013 18:13:39 +1100 Subject: Code golf challenge: XKCD 936 passwords In-Reply-To: <842d8116-fa7b-4b8f-8a8b-425bd5d3cd5d@email.android.com> References: <68365e43-498f-4ad2-bac3-6a02938159c7@googlegroups.com> <842d8116-fa7b-4b8f-8a8b-425bd5d3cd5d@email.android.com> Message-ID: On Tue, Oct 8, 2013 at 6:02 PM, Steve Simmons wrote: > Typical MUD Master - making up rules as you go along :-) Totally. Under the auspices of Rule Zero: http://tvtropes.org/pmwiki/pmwiki.php/Main/RuleOfFun :) ChrisA From breamoreboy at yahoo.co.uk Tue Oct 8 03:48:17 2013 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Tue, 08 Oct 2013 08:48:17 +0100 Subject: Code golf challenge: XKCD 936 passwords In-Reply-To: References: <68365e43-498f-4ad2-bac3-6a02938159c7@googlegroups.com> Message-ID: On 08/10/2013 07:48, sprucebondera at gmail.com wrote: > On Monday, October 7, 2013 8:45:39 PM UTC-10, spruce... at gmail.com wrote: >> On Monday, October 7, 2013 8:17:21 PM UTC-10, Chris Angelico wrote: >> >>> Who's up for some fun? Implement an XKCD-936-compliant password >>> generator in Python 3, in less code than this: >>> print(*__import__("random").sample(open("/usr/share/dict/words").read().split("\n"),4)) >> >>> Second challenge: Use it for generating all your passwords :) >> >>> [1] https://en.wikipedia.org/wiki/Code_golf >>> [2] http://xkcd.com/936/ >> >>> ChrisA >> >> Well, here's a start: >> >> import random as r >> print(*r.sample(open("/usr/share/dict/words").readlines(),4)) >> Shaves off 6 characters. > > And if we were actually trying then that filename should just be "/w". Would get rid of another 19 chars. > Very impressive, you've saved a total of 25 characters on one line and added too many lines to count to your emails, which I've snipped. Please read and digest this https://wiki.python.org/moin/GoogleGroupsPython, thanks in anticipation. -- Roses are red, Violets are blue, Most poems rhyme, But this one doesn't. Mark Lawrence From roy at panix.com Tue Oct 8 08:33:48 2013 From: roy at panix.com (Roy Smith) Date: Tue, 08 Oct 2013 08:33:48 -0400 Subject: Code golf challenge: XKCD 936 passwords References: <68365e43-498f-4ad2-bac3-6a02938159c7@googlegroups.com> Message-ID: In article , Chris Angelico wrote: > On Tue, Oct 8, 2013 at 5:48 PM, wrote: > > And if we were actually trying then that filename should just be "/w". > > Would get rid of another 19 chars. > > I'm working this on the assumption that the dictionary file already > exists (that's where it is on my Debian Linux systems, for instance) > and shouldn't be moved :) > > ChrisA In the old days, it used to be /usr/dict/words. Port Python to v6, and save another 6 characters :-) From denismfmcmahon at gmail.com Tue Oct 8 11:36:50 2013 From: denismfmcmahon at gmail.com (Denis McMahon) Date: Tue, 8 Oct 2013 15:36:50 +0000 (UTC) Subject: Code golf challenge: XKCD 936 passwords References: <68365e43-498f-4ad2-bac3-6a02938159c7@googlegroups.com> Message-ID: On Tue, 08 Oct 2013 08:33:48 -0400, Roy Smith wrote: > In article , > Chris Angelico wrote: > >> On Tue, Oct 8, 2013 at 5:48 PM, wrote: >> > And if we were actually trying then that filename should just be >> > "/w". >> > Would get rid of another 19 chars. >> >> I'm working this on the assumption that the dictionary file already >> exists (that's where it is on my Debian Linux systems, for instance) >> and shouldn't be moved :) > In the old days, it used to be /usr/dict/words. Port Python to v6, and > save another 6 characters :-) Doesn't matter where it is, a link to it exists at "/w" now ;) -- Denis McMahon, denismfmcmahon at gmail.com From python.list at tim.thechases.com Tue Oct 8 12:07:32 2013 From: python.list at tim.thechases.com (Tim Chase) Date: Tue, 8 Oct 2013 11:07:32 -0500 Subject: Code golf challenge: XKCD 936 passwords In-Reply-To: References: <68365e43-498f-4ad2-bac3-6a02938159c7@googlegroups.com> Message-ID: <20131008110732.35c8be78@bigbox.christie.dr> On 2013-10-08 15:36, Denis McMahon wrote: > On Tue, 08 Oct 2013 08:33:48 -0400, Roy Smith wrote: > > In the old days, it used to be /usr/dict/words. Port Python to > > v6, and save another 6 characters :-) > > Doesn't matter where it is, a link to it exists at "/w" now ;) You prodigal...wasting a "/". I just symlinked it from my current working directory so it exists at "w". ;-) -tkc From toby at tobiah.org Tue Oct 8 13:38:33 2013 From: toby at tobiah.org (Tobiah) Date: Tue, 08 Oct 2013 10:38:33 -0700 Subject: Code golf challenge: XKCD 936 passwords In-Reply-To: References: <68365e43-498f-4ad2-bac3-6a02938159c7@googlegroups.com> Message-ID: On 10/08/2013 09:07 AM, Tim Chase wrote: > On 2013-10-08 15:36, Denis McMahon wrote: >> On Tue, 08 Oct 2013 08:33:48 -0400, Roy Smith wrote: >>> In the old days, it used to be /usr/dict/words. Port Python to >>> v6, and save another 6 characters :-) >> >> Doesn't matter where it is, a link to it exists at "/w" now ;) > > You prodigal...wasting a "/". I just symlinked it from my current > working directory so it exists at "w". ;-) > > -tkc > > Yeah, but that's a lot of pixels! Link it to "'" in the current directory. From random832 at fastmail.us Tue Oct 8 11:47:56 2013 From: random832 at fastmail.us (random832 at fastmail.us) Date: Tue, 08 Oct 2013 11:47:56 -0400 Subject: Code golf challenge: XKCD 936 passwords In-Reply-To: <68365e43-498f-4ad2-bac3-6a02938159c7@googlegroups.com> References: <68365e43-498f-4ad2-bac3-6a02938159c7@googlegroups.com> Message-ID: <1381247276.21780.31496737.65DEAA07@webmail.messagingengine.com> On Tue, Oct 8, 2013, at 2:45, sprucebondera at gmail.com wrote: > On Monday, October 7, 2013 8:17:21 PM UTC-10, Chris Angelico wrote: > > print(*__import__("random").sample(open("/usr/share/dict/words").read().split("\n"),4)) # 87 > > import random as r > print(*r.sample(open("/usr/share/dict/words").readlines(),4)) # 80 How about this? My version is also portable to systems with different file locations, and localizable to different language dictionaries (Some assembly required). import sys,random print(*map(str.strip,random.sample(list(sys.stdin),4))) # 73 Importing random as r doesn't actually save anything, since " as r" is the same five characters you saved from the one use of it. From sprucebondera at gmail.com Tue Oct 8 16:27:45 2013 From: sprucebondera at gmail.com (sprucebondera at gmail.com) Date: Tue, 8 Oct 2013 13:27:45 -0700 (PDT) Subject: Code golf challenge: XKCD 936 passwords In-Reply-To: References: <68365e43-498f-4ad2-bac3-6a02938159c7@googlegroups.com> Message-ID: <49d2ed27-660d-4a42-bf1d-f8117aeb002e@googlegroups.com> On Tuesday, October 8, 2013 5:47:56 AM UTC-10, rand... at fastmail.us wrote: > Importing random as r doesn't actually save anything, since " as r" is > the same five characters you saved from the one use of it. I realize, it just looks nicer than the original __import__, and since it doesn't add any characters... The optimization was using readlines. From rosuav at gmail.com Tue Oct 8 16:35:24 2013 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 9 Oct 2013 07:35:24 +1100 Subject: Code golf challenge: XKCD 936 passwords In-Reply-To: <49d2ed27-660d-4a42-bf1d-f8117aeb002e@googlegroups.com> References: <68365e43-498f-4ad2-bac3-6a02938159c7@googlegroups.com> <49d2ed27-660d-4a42-bf1d-f8117aeb002e@googlegroups.com> Message-ID: On Wed, Oct 9, 2013 at 7:27 AM, wrote: > On Tuesday, October 8, 2013 5:47:56 AM UTC-10, rand... at fastmail.us wrote: >> Importing random as r doesn't actually save anything, since " as r" is >> the same five characters you saved from the one use of it. > > I realize, it just looks nicer than the original __import__, and since it doesn't add any characters... > The optimization was using readlines. Are you aware that readlines keeps the \n at the end of each line, though? Looks a lot less clean in its output that way. That's why I used .split() in the first place. ChrisA From roy at panix.com Tue Oct 8 21:38:12 2013 From: roy at panix.com (Roy Smith) Date: Tue, 08 Oct 2013 21:38:12 -0400 Subject: Code golf challenge: XKCD 936 passwords References: <68365e43-498f-4ad2-bac3-6a02938159c7@googlegroups.com> Message-ID: On Monday, October 7, 2013 8:17:21 PM UTC-10, Chris Angelico wrote: > > print(*__import__("random").sample(open("/usr/share/dict/words").read().spli > > t("\n"),4)) In article <68365e43-498f-4ad2-bac3-6a02938159c7 at googlegroups.com>, sprucebondera at gmail.com wrote: > import random as r > print(*r.sample(open("/usr/share/dict/words").readlines(),4)) > > Shaves off 6 characters. If you're willing to accept a sub-optimal random number generator: # Python-2, sorry import os print list(set(open('/usr/share/dict/words')))[os.getpid():][:4] I'll also point out that on my OSX box, I could have used /usr/share/dict/web2 and saved one more character :-) ? From rosuav at gmail.com Tue Oct 8 22:10:02 2013 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 9 Oct 2013 13:10:02 +1100 Subject: Code golf challenge: XKCD 936 passwords In-Reply-To: References: <68365e43-498f-4ad2-bac3-6a02938159c7@googlegroups.com> Message-ID: On Wed, Oct 9, 2013 at 12:38 PM, Roy Smith wrote: > If you're willing to accept a sub-optimal random number generator: > > # Python-2, sorry > import os > print list(set(open('/usr/share/dict/words')))[os.getpid():][:4] So that steps by your pid? That's going to drastically reduce the entropy in the result, which is kinda the point of XKCD 936. I'll call that one dubious... though it's still better than Rob's entry. (But thanks Rob, I do like that one :) ) ChrisA From nick.cash at npcinternational.com Wed Oct 9 12:07:58 2013 From: nick.cash at npcinternational.com (Nick Cash) Date: Wed, 9 Oct 2013 16:07:58 +0000 Subject: Code golf challenge: XKCD 936 passwords In-Reply-To: References: <68365e43-498f-4ad2-bac3-6a02938159c7@googlegroups.com> Message-ID: <1f3da76035e64263af9a6165c3cc8dd3@BY2PR06MB233.namprd06.prod.outlook.com> >> # Python-2, sorry >> import os >> print list(set(open('/usr/share/dict/words')))[os.getpid():][:4] > So that steps by your pid? Not really. It seems to rely on list(set(...)) kinda randomizing order... which is definitely not safe without hash randomization. But this brings up an interesting concept... if we can assume Python 2.7 with the -R flag, or Python 3.3+, I think we do get true pseudo-randomization out of list(set(...))... which means we can trim some! print(list(set(open('/usr/share/dict/words')))[:4]) No module imports needed! Although it's not as pretty as the original post, but neither was Roy's. -Nick Cash From roy at panix.com Wed Oct 9 20:29:00 2013 From: roy at panix.com (Roy Smith) Date: Wed, 09 Oct 2013 20:29:00 -0400 Subject: Code golf challenge: XKCD 936 passwords References: <68365e43-498f-4ad2-bac3-6a02938159c7@googlegroups.com> Message-ID: In article , Nick Cash wrote: > >> # Python-2, sorry > >> import os > >> print list(set(open('/usr/share/dict/words')))[os.getpid():][:4] > > > So that steps by your pid? > > Not really. It seems to rely on list(set(...)) kinda randomizing order... > which is definitely not safe without hash randomization. Exactly. I *did* preface this with: >>> If you're willing to accept a sub-optimal random number generator: [Nick, again] > But this brings up an interesting concept... if we can assume Python 2.7 with > the -R flag, or Python 3.3+, I think we do get true pseudo-randomization out > of list(set(...))... which means we can trim some! > > print(list(set(open('/usr/share/dict/words')))[:4]) Excellent! I didn't know about -R before this, so not only has this been fun, it's been educational too! From roy at panix.com Tue Oct 8 22:52:36 2013 From: roy at panix.com (Roy Smith) Date: Tue, 08 Oct 2013 22:52:36 -0400 Subject: Code golf challenge: XKCD 936 passwords References: <68365e43-498f-4ad2-bac3-6a02938159c7@googlegroups.com> Message-ID: In article , Chris Angelico wrote: > On Wed, Oct 9, 2013 at 12:38 PM, Roy Smith wrote: > > If you're willing to accept a sub-optimal random number generator: > > > > # Python-2, sorry > > import os > > print list(set(open('/usr/share/dict/words')))[os.getpid():][:4] > > So that steps by your pid? That's going to drastically reduce the > entropy in the result, Well, I did say it was a sub-optimal random number generator :-) I've been experimenting with os.urandom() as an entropy source, but can't get the (source code) character count down far enough. From toby at tobiah.org Tue Oct 8 13:58:18 2013 From: toby at tobiah.org (Tobiah) Date: Tue, 08 Oct 2013 10:58:18 -0700 Subject: Code golf challenge: XKCD 936 passwords In-Reply-To: References: Message-ID: <_IX4u.69249$yx6.3797@fx26.iad> On 10/07/2013 11:17 PM, Chris Angelico wrote: > Who's up for some fun? Implement an XKCD-936-compliant password > generator in Python 3, in less code than this: > > print(*__import__("random").sample(open("/usr/share/dict/words").read().split("\n"),4)) > > Second challenge: Use it for generating all your passwords :) > > [1] https://en.wikipedia.org/wiki/Code_golf > [2] http://xkcd.com/936/ > > ChrisA > So how about finding the last word that starts with each lower case letter of the alphabet in turn: azures bywords czars ... Tobiah From tim at thechases.com Tue Oct 8 12:08:20 2013 From: tim at thechases.com (Tim Chase) Date: Tue, 8 Oct 2013 11:08:20 -0500 Subject: Code golf challenge: XKCD 936 passwords In-Reply-To: References: Message-ID: <20131008110820.0c612adb@bigbox.christie.dr> On 2013-10-08 17:17, Chris Angelico wrote: > Who's up for some fun? Implement an XKCD-936-compliant password > generator in Python 3, in less code than this: > > print(*__import__("random").sample(open("/usr/share/dict/words").read().split("\n"),4)) > > Second challenge: Use it for generating all your passwords :) UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 9104: ordinal not in range(128) -tkc From rkd at rkd.me.uk Tue Oct 8 18:27:51 2013 From: rkd at rkd.me.uk (Rob Day) Date: Tue, 08 Oct 2013 23:27:51 +0100 Subject: Code golf challenge: XKCD 936 passwords In-Reply-To: References: Message-ID: <525486E7.6040909@rkd.me.uk> On 08/10/13 07:17, Chris Angelico wrote: > Who's up for some fun? Implement an XKCD-936-compliant password > generator in Python 3, in less code than this: > > print(*__import__("random").sample(open("/usr/share/dict/words").read().split("\n"),4)) > > print("imploring epsilon decamp graveyard's") # Chosen by fair random sampling, guaranteed to be random Comments don't count as code, right? ;) From random832 at fastmail.us Wed Oct 9 16:22:31 2013 From: random832 at fastmail.us (random832 at fastmail.us) Date: Wed, 09 Oct 2013 16:22:31 -0400 Subject: Code golf challenge: XKCD 936 passwords In-Reply-To: <525486E7.6040909@rkd.me.uk> References: <525486E7.6040909@rkd.me.uk> Message-ID: <1381350151.4210.32080629.2CDD1959@webmail.messagingengine.com> On Tue, Oct 8, 2013, at 18:27, Rob Day wrote: > On 08/10/13 07:17, Chris Angelico wrote: > > Who's up for some fun? Implement an XKCD-936-compliant password > > generator in Python 3, in less code than this: > > > > print(*__import__("random").sample(open("/usr/share/dict/words").read().split("\n"),4)) > > > > > print("imploring epsilon decamp graveyard's") > # Chosen by fair random sampling, guaranteed to be random Of course, the choice of dictionary makes all the difference in the world. print('kayo wide sitz keen') # Chosen by fair random sampling among SOWPODS words of four or fewer letters. # This is a set of 6870 words, XKCD Std. #936 requires a set of at least 2048. # This password's 50.98 bits of entropy exceed the standard's recommendations by 6.98. This can be further improved to the standard's recommended entropy level (but not strictly the same method) by selecting only the first one from this set, and selecting the remaining four words from the set of 1416 words of 3 letters or less, for 44.14 bits of entropy: print('axis ar cha tam') Or, keeping with the spirit of the standard, mix the set of 1416 words 3 letters or less with 632 randomly selected words from the four-letter set: print('pal alp govs deb') From steve at pearwood.info Wed Oct 9 01:24:12 2013 From: steve at pearwood.info (Steven D'Aprano) Date: 09 Oct 2013 05:24:12 GMT Subject: Code golf challenge: XKCD 936 passwords References: Message-ID: <5254e87b$0$29976$c3e8da3$5496439d@news.astraweb.com> On Tue, 08 Oct 2013 23:27:51 +0100, Rob Day wrote: > print("imploring epsilon decamp graveyard's") # Chosen by fair random > sampling, guaranteed to be random > > Comments don't count as code, right? ;) "cat cat cat cat..." That's the trouble with random, you can never quite tell. -- Steven From sam at giraffetech.biz Tue Oct 8 02:33:31 2013 From: sam at giraffetech.biz (Sam Giraffe) Date: Mon, 7 Oct 2013 23:33:31 -0700 Subject: Re for Apache log file format Message-ID: Hi, I am trying to split up the re pattern for Apache log file format and seem to be having some trouble in getting Python to understand multi-line pattern: #!/usr/bin/python import re #this is a single line string = '192.168.122.3 - - [29/Sep/2013:03:52:33 -0700] "GET / HTTP/1.0" 302 276 "-" "check_http/v1.4.16 (nagios-plugins 1.4.16)"' #trying to break up the pattern match for easy to read code pattern = re.compile(r'(?P\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})\s+' r'(?P\-)\s+' r'(?P\-)\s+' r'(?P\[(.*?)\])\s+' r'(?P\"(.*?)\")\s+' r'(?P\d{3})\s+' r'(?P\d+)\s+' r'(?P\"\")\s+' r'(?P\((.*?)\))') match = re.search(pattern, string) if match: print match.group('ip') else: print 'not found' The python interpreter is skipping to the 'math = re.search' and then the 'if' statement right after it looks at the , instead of moving onto and so on. mybox:~ user$ python -m pdb /Users/user/Documents/Python/apache.py > /Users/user/Documents/Python/apache.py(3)() -> import re (Pdb) n > /Users/user/Documents/Python/apache.py(5)() -> string = '192.168.122.3 - - [29/Sep/2013:03:52:33 -0700] "GET / HTTP/1.0" 302 276 "-" "check_http/v1.4.16 (nagios-plugins 1.4.16)"' (Pdb) n > /Users/user/Documents/Python/apache.py(7)() -> pattern = re.compile(r'(?P\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})\s+' (Pdb) n > /Users/user/Documents/Python/apache.py(17)() -> match = re.search(pattern, string) (Pdb) Thank you. -------------- next part -------------- An HTML attachment was scrubbed... URL: From andipersti at gmail.com Tue Oct 8 06:23:20 2013 From: andipersti at gmail.com (Andreas Perstinger) Date: Tue, 08 Oct 2013 12:23:20 +0200 Subject: Re for Apache log file format In-Reply-To: References: Message-ID: <5253DD18.9020601@gmail.com> On 08.10.2013 08:33, Sam Giraffe wrote: > #this is a single line > string = '192.168.122.3 - - [29/Sep/2013:03:52:33 -0700] "GET / HTTP/1.0" > 302 276 "-" "check_http/v1.4.16 (nagios-plugins 1.4.16)"' > > #trying to break up the pattern match for easy to read code > pattern = re.compile(r'(?P\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})\s+' > r'(?P\-)\s+' > r'(?P\-)\s+' > r'(?P\[(.*?)\])\s+' > r'(?P\"(.*?)\")\s+' > r'(?P\d{3})\s+' > r'(?P\d+)\s+' > r'(?P\"\")\s+' > r'(?P\((.*?)\))') [SNIP] > The python interpreter is skipping to the 'math = re.search' and then the > 'if' statement right after it looks at the , instead of moving onto > and so on. I'm not sure if I understand your problem, but your regex pattern only matches up to the size. When you look for the referrer, the pattern expects two quotes but in your string you have "-" (quote, dash, quote). Thus there is no match (i.e. "match" is None) and the if-statement will print "not found". Bye, Andreas From neilc at norwich.edu Tue Oct 8 08:50:22 2013 From: neilc at norwich.edu (Neil Cerutti) Date: 8 Oct 2013 12:50:22 GMT Subject: Re for Apache log file format References: Message-ID: On 2013-10-08, Sam Giraffe wrote: > > Hi, > > I am trying to split up the re pattern for Apache log file format and seem > to be having some trouble in getting Python to understand multi-line > pattern: > > #!/usr/bin/python > > import re > > #this is a single line > string = '192.168.122.3 - - [29/Sep/2013:03:52:33 -0700] "GET / HTTP/1.0" > 302 276 "-" "check_http/v1.4.16 (nagios-plugins 1.4.16)"' > > #trying to break up the pattern match for easy to read code > pattern = re.compile(r'(?P\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})\s+' > r'(?P\-)\s+' > r'(?P\-)\s+' > r'(?P\[(.*?)\])\s+' > r'(?P\"(.*?)\")\s+' > r'(?P\d{3})\s+' > r'(?P\d+)\s+' > r'(?P\"\")\s+' > r'(?P\((.*?)\))') I recommend using the re.VERBOSE flag when explicating an re. It'll make your life incrementally easier. pattern = re.compile( r"""(?P\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})\s+ (?P\-)\s+ (?P\-)\s+ (?P\[(.*?)\])\s+ # You can even insert comments. (?P\"(.*?)\")\s+ (?P\d{3})\s+ (?P\d+)\s+ (?P\"\")\s+ (?P\((.*?)\))""", re.VERBOSE) -- Neil Cerutti From denismfmcmahon at gmail.com Tue Oct 8 11:48:38 2013 From: denismfmcmahon at gmail.com (Denis McMahon) Date: Tue, 8 Oct 2013 15:48:38 +0000 (UTC) Subject: Re for Apache log file format References: Message-ID: On Mon, 07 Oct 2013 23:33:31 -0700, Sam Giraffe wrote: > I am trying to split up the re pattern for Apache log file format and > seem to be having some trouble in getting Python to understand > multi-line pattern: Aiui apache log format uses space as delimiter, encapsulates strings in '"' characters, and uses '-' as an empty field. So I think every element should match: (\S+|"[^"]+"|-) and there should be \s+ between elements. -- Denis McMahon, denismfmcmahon at gmail.com From skip at pobox.com Tue Oct 8 11:59:51 2013 From: skip at pobox.com (Skip Montanaro) Date: Tue, 8 Oct 2013 10:59:51 -0500 Subject: Re for Apache log file format In-Reply-To: References: Message-ID: > Aiui apache log format uses space as delimiter, encapsulates strings in > '"' characters, and uses '-' as an empty field. Specifying the field delimiter as a space, you might be able to use the csv module to read these. I haven't done any Apache log file work since long before the csv module was available, but it just might work. Skip From cs at zip.com.au Tue Oct 8 18:17:44 2013 From: cs at zip.com.au (Cameron Simpson) Date: Wed, 9 Oct 2013 09:17:44 +1100 Subject: Re for Apache log file format In-Reply-To: References: Message-ID: <20131008221744.GA24902@cskk.homeip.net> On 08Oct2013 10:59, Skip Montanaro wrote: | > Aiui apache log format uses space as delimiter, encapsulates strings in | > '"' characters, and uses '-' as an empty field. | | Specifying the field delimiter as a space, you might be able to use | the csv module to read these. I haven't done any Apache log file work | since long before the csv module was available, but it just might | work. You can definitely do this. I pull things out of apache log files using awk in exactly this fashion. It does rely on each of the "real" fields having a fixed number of "words" in it. You just stick the fields back together again. And also in Python. I've got a merge-apache-logs script to read multiple logs, presumed in time order, and produce a single output stream for passing to log analysis tools: https://bitbucket.org/cameron_simpson/css/src/tip/bin/merge-apache-logs It is a bit of a hack, but useful. It has an "aptime" function to pull and parse the time field from the line which starts like this: def aptime(logline, zones, defaultZone): ''' Compute a datetime object from the supplied Apache log line. `defaultZone` is the timezone to use if it cannot be deduced. ''' fields = logline.split() if len(fields) < 5: ##warning("bad log line: %s", logline) return None dt = None tzinfo = None # try for desired "[DD/Mon/YYYY:HH:MM:SS +hhmm]" format humantime, tzinfo = fields[3], fields[4] if len(humantime) == 21 \ and humantime.startswith('[') \ and tzinfo.endswith(']'): try: dt = datetime.strptime(humantime, "[%d/%b/%Y:%H:%M:%S") except ValueError, e: dt = None if dt is None: tzinfo = None else: tzinfo = tzinfo[:-1] and proceeeds otherwise (we have a few different log formats in play, alas). So regexpas are not your only choice here, and possibly not even the best choice. Cheers, -- Cameron Simpson This is not a bug. It's just the way it works, and makes perfect sense. - Tom Christiansen I like that line. I hope my boss falls for it. - Chaim Frenkel From piet at vanoostrum.org Wed Oct 9 13:33:14 2013 From: piet at vanoostrum.org (Piet van Oostrum) Date: Wed, 09 Oct 2013 13:33:14 -0400 Subject: Re for Apache log file format References: Message-ID: Sam Giraffe writes: > Hi, > > I am trying to split up the re pattern for Apache log file format and seem to be having some > trouble in getting Python to understand multi-line pattern: > > #!/usr/bin/python > > import re > > #this is a single line > string = '192.168.122.3 - - [29/Sep/2013:03:52:33 -0700] "GET / HTTP/1.0" 302 276 "-" "check_http/ > v1.4.16 (nagios-plugins 1.4.16)"' > > #trying to break up the pattern match for easy to read code > pattern = re.compile(r'(?P\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})\s+' > ???????????????????? r'(?P\-)\s+' > ???????????????????? r'(?P\-)\s+' > ???????????????????? r'(?P\[(.*?)\])\s+' > ???????????????????? r'(?P\"(.*?)\")\s+' > ???????????????????? r'(?P\d{3})\s+' > ???????????????????? r'(?P\d+)\s+' > ???????????????????? r'(?P\"\")\s+' > ???????????????????? r'(?P\((.*?)\))') > > match = re.search(pattern, string) > > if match: > ??? print match.group('ip') > else: > ??? print 'not found' > > The python interpreter is skipping to the 'math = re.search' and then the 'if' statement right > after it looks at the , instead of moving onto and so on. Although you have written the regexp as a sequence of lines, in reality it is a single string, and therefore pdb will do only a single step, and not go into its "parts", which really are not parts. > > mybox:~ user$ python -m pdb /Users/user/Documents/Python/apache.py >> /Users/user/Documents/Python/apache.py(3)() > -> import re > (Pdb) n >> /Users/user/Documents/Python/apache.py(5)() > -> string = '192.168.122.3 - - [29/Sep/2013:03:52:33 -0700] "GET / HTTP/1.0" 302 276 "-" > "check_http/v1.4.16 (nagios-plugins 1.4.16)"' > (Pdb) n >> /Users/user/Documents/Python/apache.py(7)() > -> pattern = re.compile(r'(?P\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})\s+' > (Pdb) n >> /Users/user/Documents/Python/apache.py(17)() > -> match = re.search(pattern, string) > (Pdb) Also as Andreas has noted the r'(?P\"\")\s+' part is wrong. It should probably be r'(?P\".*?\")\s+' And the r'(?P\((.*?)\))') will also not match as there is text outside the (). Should probably also be r'(?P\".*?\")') or something like it. -- Piet van Oostrum WWW: http://pietvanoostrum.com/ PGP key: [8DAE142BE17999C4] From nikos.gr33k at gmail.com Tue Oct 8 05:15:46 2013 From: nikos.gr33k at gmail.com (=?UTF-8?B?zp3Or866zr/PgiDOkc67zrXOvs+Mz4DOv8+FzrvOv8+C?=) Date: Tue, 08 Oct 2013 12:15:46 +0300 Subject: inserting or updating appropriately Message-ID: Hello, i'am trying to insert a new record or update an existing one in case counterID(stands for the page's URL) and cookieID(random number) is the same: try: # if first time for webpage; create new record( primary key is automatic, hit is defaulted ), if page exists then update record cur.execute('''INSERT INTO counters (url) VALUES (%s) ON DUPLICATE KEY UPDATE hits = hits + 1''', page ) # get the primary key value of the new added record cID = cur.lastrowid # if first time visitor on this page, create new record, if visitor exists then update record cur.execute('''INSERT INTO visitors (counterID, cookieID, host, city, useros, browser, ref, lastvisit) VALUES (%s, %s, %s, %s, %s, %s, %s, %s) ON DUPLICATE KEY UPDATE cookieID = %s, host = %s, city = %s, useros = %s, browser = %s, ref = %s, hits = hits + 1, lastvisit = %s WHERE counterID = %s and cookieID = %s''', (cID, cookieID, host, city, useros, browser, ref, lastvisit, cookieID, host, city, useros, browser, ref, lastvisit, cID, cookieID) ) ============= Error is: ProgrammingError(ProgrammingError(1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'WHERE counterID = 1 and cookieID = '3815'' at line 3"),) i notticed that if i remove the WHERE clause in the last execute it works but then its not updating properly. Can this happen in 1-statemnt with the ON DUPLICATE KEY INVOLVED WITHOUT BREAKING IT IN IN 2-STATEMNTS? THANKS. From nikos.gr33k at gmail.com Tue Oct 8 05:20:33 2013 From: nikos.gr33k at gmail.com (=?UTF-8?B?zp3Or866zr/PgiDOkc67zrXOvs+Mz4DOv8+FzrvOv8+C?=) Date: Tue, 08 Oct 2013 12:20:33 +0300 Subject: inserting or updating appropriately In-Reply-To: References: Message-ID: ???? 8/10/2013 12:15 ??, ?/? ????? ??????????? ??????: > Hello, i'am trying to insert a new record or update an existing one in > case counterID(stands for the page's URL) and cookieID(random number) is > the same: > > try: > # if first time for webpage; create new record( primary key is > automatic, hit is defaulted ), if page exists then update record > cur.execute('''INSERT INTO counters (url) VALUES (%s) ON > DUPLICATE KEY UPDATE hits = hits + 1''', page ) > # get the primary key value of the new added record > cID = cur.lastrowid > > # if first time visitor on this page, create new record, if > visitor exists then update record > cur.execute('''INSERT INTO visitors (counterID, cookieID, host, > city, useros, browser, ref, lastvisit) VALUES (%s, %s, %s, %s, %s, %s, > %s, %s) > ON DUPLICATE KEY UPDATE cookieID = %s, host = > %s, city = %s, useros = %s, browser = %s, ref = %s, hits = hits + 1, > lastvisit = %s > WHERE counterID = %s and cookieID = %s''', > (cID, cookieID, host, city, useros, browser, > ref, lastvisit, cookieID, host, city, useros, browser, ref, lastvisit, > cID, cookieID) ) > ============= > > Error is: ProgrammingError(ProgrammingError(1064, "You have an error in > your SQL syntax; check the manual that corresponds to your MySQL server > version for the right syntax to use near 'WHERE counterID = 1 and > cookieID = '3815'' at line 3"),) > > i notticed that if i remove the WHERE clause in the last execute it > works but then its not updating properly. > > Can this happen in 1-statemnt with the ON DUPLICATE KEY INVOLVED WITHOUT > BREAKING IT IN IN 2-STATEMNTS? > > THANKS. Actually what i want is this effect in cur.execute statement: # if first time visitor on this page, create new record, if visitor exists then update record cur.execute('''INSERT INTO visitors (counterID, cookieID, host, city, useros, browser, ref, lastvisit) VALUES (%s, %s, %s, %s, %s, %s, %s, %s)''', (cID, cookieID, host, city, useros, browser, ref, lastvisit) cur.execute('''UPDATE visitors SET cookieID = %s, host = %s, city = %s, useros = %s, browser = %s, ref = %s, hits = hits + 1, lastvisit = %s WHERE counterID = %s and cookieID = %s''', (cookieID, host, city, useros, browser, ref, lastvisit, cID, cookieID) ) -- What is now proved was at first only imagined! & WebHost From ben+python at benfinney.id.au Tue Oct 8 05:29:58 2013 From: ben+python at benfinney.id.au (Ben Finney) Date: Tue, 08 Oct 2013 20:29:58 +1100 Subject: inserting or updating appropriately References: Message-ID: <7whacsp0tl.fsf@benfinney.id.au> ????? ??????????? writes: > Error is: ProgrammingError(ProgrammingError(1064, "You have an error > in your SQL syntax; check the manual that corresponds to your MySQL > server version for the right syntax to use near 'WHERE counterID = 1 > and cookieID = '3815'' at line 3"),) This is an error from the database server. It has nothing to do with Python, as you already know from previous discussions here. Please do not ask questions about usage of the database server here any more. -- \ ?Holy knit one purl two, Batman!? ?Robin | `\ | _o__) | Ben Finney From nikos.gr33k at gmail.com Tue Oct 8 06:04:34 2013 From: nikos.gr33k at gmail.com (=?UTF-8?B?zp3Or866zr/PgiDOkc67zrXOvs+Mz4DOv8+FzrvOv8+C?=) Date: Tue, 08 Oct 2013 13:04:34 +0300 Subject: Cookie gets changed when hit comes from a referrer Message-ID: # initialize cookie and retrieve cookie from clients browser cookie = cookies.SimpleCookie( os.environ.get('HTTP_COOKIE') ) if cookie.get('ID') is not None: cookieID = cookie['ID'].value else: cookieID = random.randrange(0, 9999) cookie['ID'] = cookieID cookie['ID']['path'] = '/' print( cookie ) =========== =========== I use this code to retrive or set a cookie to the visitor's browser if present and identify him bu it. All work well except the situation where the user visits my webpage by clicking a backlink on another wbpage. Then for some reason the cookieID changes to another value thus a new entry appears into the database when insert happens. What cna i do about that? From ian.g.kelly at gmail.com Tue Oct 8 07:08:06 2013 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Tue, 8 Oct 2013 05:08:06 -0600 Subject: Cookie gets changed when hit comes from a referrer In-Reply-To: References: Message-ID: On Tue, Oct 8, 2013 at 4:04 AM, ????? ??????????? wrote: > I use this code to retrive or set a cookie to the visitor's browser if > present and identify him bu it. > > All work well except the situation where the user visits my webpage by > clicking a backlink on another wbpage. > > Then for some reason the cookieID changes to another value thus a new entry > appears into the database when insert happens. > > What cna i do about that? This question is really about HTTP, not Python, so you'd have better luck asking elsewhere. The most likely possibility is that the domain doesn't match. For example, the cookie is set for the domain www.foo.com, and the other webpage is linking to foo.com. Another possibility is that the cookie is expiring because the browser session was terminated, not because of anything to do with the other webpage. Or it could simply be a bug or unusual setting in whatever browser you're using to test it. From nikos.gr33k at gmail.com Tue Oct 8 10:18:12 2013 From: nikos.gr33k at gmail.com (=?UTF-8?B?zp3Or866zr/PgiDOkc67zrXOvs+Mz4DOv8+FzrvOv8+C?=) Date: Tue, 08 Oct 2013 17:18:12 +0300 Subject: Cookie gets changed when hit comes from a referrer In-Reply-To: References: Message-ID: ???? 8/10/2013 2:08 ??, ?/? Ian Kelly ??????: > On Tue, Oct 8, 2013 at 4:04 AM, ????? ??????????? wrote: >> I use this code to retrive or set a cookie to the visitor's browser if >> present and identify him bu it. >> >> All work well except the situation where the user visits my webpage by >> clicking a backlink on another wbpage. >> >> Then for some reason the cookieID changes to another value thus a new entry >> appears into the database when insert happens. >> >> What cna i do about that? > > This question is really about HTTP, not Python, so you'd have better > luck asking elsewhere. The most likely possibility is that the domain > doesn't match. For example, the cookie is set for the domain > www.foo.com, and the other webpage is linking to foo.com. Another > possibility is that the cookie is expiring because the browser session > was terminated, not because of anything to do with the other webpage. > Or it could simply be a bug or unusual setting in whatever browser > you're using to test it. > When i direct hit http://superhost.gr the cookie remains the same it is not lost. Also i have set: ookie['ID']['expires'] = 60*60*24*365 #this cookie will expire in a year but that didnt also help much because the cookie is also changing when the hit comes through a referrer. So, i cannot se the cookie down to its feet my whole insert or update procedure breaks and i have duplicate entried for the same hostnames. Where shoudl i rely to identify a visitor? I was relying on tis hostname(although i know that after a router reset) it changes, and then couple days ago i was thiking of relying to a cookie that i would/set retrive from the vistirs browser, but if it changes all the time i cannot evan rely to that. -- What is now proved was at first only imagined! & WebHost From ian.g.kelly at gmail.com Wed Oct 9 00:53:53 2013 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Tue, 8 Oct 2013 22:53:53 -0600 Subject: Cookie gets changed when hit comes from a referrer In-Reply-To: References: Message-ID: On Tue, Oct 8, 2013 at 8:18 AM, ????? ??????????? wrote: > Also i have set: > ookie['ID']['expires'] = 60*60*24*365 #this cookie will expire in > a year The Expires attribute takes a date. If you're passing an interval in seconds then you should use the Max-Age attribute instead. That said, I think I misunderstood the problem initially. You are saying that when the user is on another site, and they press the browser's Back button to return to your page, your host is not recording a visit from the cookie you've given them? This is probably happening because the browser is not actually sending a request to your web server when it navigates back, unless the user specifically requests a refresh. Otherwise, most browsers will just use the cached page already in memory in this situation. As far as the server is concerned, nothing has happened. From nikos.gr33k at gmail.com Wed Oct 9 02:47:29 2013 From: nikos.gr33k at gmail.com (=?UTF-8?B?zp3Or866zr/PgiDOkc67zrXOvs+Mz4DOv8+FzrvOv8+C?=) Date: Wed, 09 Oct 2013 09:47:29 +0300 Subject: Cookie gets changed when hit comes from a referrer In-Reply-To: References: Message-ID: ???? 9/10/2013 7:53 ??, ?/? Ian Kelly ??????: > On Tue, Oct 8, 2013 at 8:18 AM, ????? ??????????? wrote: >> Also i have set: >> ookie['ID']['expires'] = 60*60*24*365 #this cookie will expire in >> a year > > The Expires attribute takes a date. If you're passing an interval in > seconds then you should use the Max-Age attribute instead. > > That said, I think I misunderstood the problem initially. You are > saying that when the user is on another site, and they press the > browser's Back button to return to your page, your host is not > recording a visit from the cookie you've given them? This is probably > happening because the browser is not actually sending a request to > your web server when it navigates back, unless the user specifically > requests a refresh. Otherwise, most browsers will just use the cached > page already in memory in this situation. As far as the server is > concerned, nothing has happened. > No i dont mean that. When a user hits my link on another website, for exmaple they are on ypsilandio.gr and they hit the link of superhost.gr then a new entry with a new cookie is appearing into my visitors table! Where is the old cookie that was saved in my browser so it will get retrieved? I use chrome and i notice that when a visitor comes to my webpage form a referrer link the cookie's ID is always set to a new random value. I have no idea why. Why would it metter from where you sre coming from? The cookie ust have beeen present in the visitor's browser, shouldnt it? -- What is now proved was at first only imagined! & WebHost From ben+python at benfinney.id.au Wed Oct 9 03:12:21 2013 From: ben+python at benfinney.id.au (Ben Finney) Date: Wed, 09 Oct 2013 18:12:21 +1100 Subject: Cookie gets changed when hit comes from a referrer References: Message-ID: <7w8uy2q5nu.fsf@benfinney.id.au> ????? ??????????? writes: > When a user hits my link on another website, for exmaple they are on > ypsilandio.gr and they hit the link of superhost.gr then a new entry > with a new cookie is appearing into my visitors table! > > Where is the old cookie that was saved in my browser so it will get > retrieved? I use chrome and i notice that when a visitor comes to my > webpage form a referrer link the cookie's ID is always set to a new > random value. None of this has to do with Python. Please do not ask this Python doscussion forum to educate you on how HTTP operates. -- \ ?How many people here have telekenetic powers? Raise my hand.? | `\ ?Emo Philips | _o__) | Ben Finney From nikos.gr33k at gmail.com Tue Oct 8 10:36:50 2013 From: nikos.gr33k at gmail.com (=?UTF-8?B?zp3Or866zr/PgiDOkc67zrXOvs+Mz4DOv8+FzrvOv8+C?=) Date: Tue, 08 Oct 2013 17:36:50 +0300 Subject: Cookie gets changed when hit comes from a referrer In-Reply-To: References: Message-ID: ???? 8/10/2013 2:08 ??, ?/? Ian Kelly ??????: > This question is really about HTTP, not Python, so you'd have better > luck asking elsewhere. The most likely possibility is that the domain > doesn't match. For example, the cookie is set for the domain > www.foo.com, and the other webpage is linking to foo.com. I think this is the problem but iam not sure entirely how you mean. Can you please explain it a bit more? Shall i change cookie['ID']['path'] = '/' to something else so that never happens? -- What is now proved was at first only imagined! & WebHost From denismfmcmahon at gmail.com Tue Oct 8 11:55:01 2013 From: denismfmcmahon at gmail.com (Denis McMahon) Date: Tue, 8 Oct 2013 15:55:01 +0000 (UTC) Subject: Cookie gets changed when hit comes from a referrer References: Message-ID: On Tue, 08 Oct 2013 13:04:34 +0300, ????? ??????????? wrote: > I use this code to retrive or set a cookie to the visitor's browser if > present and identify him bu it. You are aware that using cookies to track a user who doesn't want to be tracked won't work, because he'll just tell his browser to not use cookies, aren't you. Nick, if a user doesn't want to be tracked, you can't track them. The user controls all the data their machine sends to you. This means that they can manipulate it. Nothing you can do will prevent this. -- Denis McMahon, denismfmcmahon at gmail.com From nikos.gr33k at gmail.com Tue Oct 8 12:04:37 2013 From: nikos.gr33k at gmail.com (=?UTF-8?B?zp3Or866zr/PgiDOkc67zrXOvs+Mz4DOv8+FzrvOv8+C?=) Date: Tue, 08 Oct 2013 19:04:37 +0300 Subject: Cookie gets changed when hit comes from a referrer In-Reply-To: References: Message-ID: ???? 8/10/2013 6:55 ??, ?/? Denis McMahon ??????: > On Tue, 08 Oct 2013 13:04:34 +0300, ????? ??????????? wrote: > >> I use this code to retrive or set a cookie to the visitor's browser if >> present and identify him bu it. > > You are aware that using cookies to track a user who doesn't want to be > tracked won't work, because he'll just tell his browser to not use > cookies, aren't you. > > Nick, if a user doesn't want to be tracked, you can't track them. The > user controls all the data their machine sends to you. This means that > they can manipulate it. Nothing you can do will prevent this. > Yes iam aware of that, but its the best trcking method i can think of. Tracking just the hostname is not accurate since with every router restart, that info is changing. Tracking the visitor by settign a cookie to its browser is not perfect/accurate since he can manipulate its broswer data or flush the cookies but this is the best one can do after having people register on the webiste. Or perhaps trying to identify the cookie + hostname is even better. Can you help me with this particuler problem please? -- What is now proved was at first only imagined! & WebHost From joel.goldstick at gmail.com Tue Oct 8 13:10:52 2013 From: joel.goldstick at gmail.com (Joel Goldstick) Date: Tue, 8 Oct 2013 13:10:52 -0400 Subject: Cookie gets changed when hit comes from a referrer In-Reply-To: References: Message-ID: On Tue, Oct 8, 2013 at 12:04 PM, ????? ??????????? wrote: > ???? 8/10/2013 6:55 ??, ?/? Denis McMahon ??????: > >> On Tue, 08 Oct 2013 13:04:34 +0300, ????? ??????????? wrote: >> >>> I use this code to retrive or set a cookie to the visitor's browser if >>> present and identify him bu it. >> >> Browser cookies have been defined and around for a very long time. If you google "browser cookie tutorial" you can learn how they work -- probably within an hour!. This will help you find your solution The first poster pointed out that www.example.com and example.com can be considered different domains. You can make a cookie work for both but you need to understand cookies to learn how. This is off topic, ... again! > https://mail.python.org/mailman/listinfo/python-list -- Joel Goldstick http://joelgoldstick.com From denismfmcmahon at gmail.com Tue Oct 8 15:29:44 2013 From: denismfmcmahon at gmail.com (Denis McMahon) Date: Tue, 8 Oct 2013 19:29:44 +0000 (UTC) Subject: Cookie gets changed when hit comes from a referrer References: Message-ID: On Tue, 08 Oct 2013 19:04:37 +0300, ????? ??????????? wrote: > Can you help me with this particuler problem please? Unfortunately I can't, because I am unable to reproduce the problem you describe. When I load my test page in the browser, then replace it with something else by entering an address in the address bar and pressing return, then use the back link followed by the reload one, I am back at my test page with the original cookie value. Of course, this is using my cookie etc code and mechanisms, and not yours .... Now, either it's an issue in your python implementation of cookie handling which isn't happening in my implementation, or it's something to do with the way that your system passes data around (cgi) that doesn't happen in mine (mod_wsgi), or it's happening in the browser you're testing in, but not in my browser. Have you checked the cookie jar in the browser to see what value the cookie has? Is that the value you think it should have? Note that checking the cookie jar is a browser topic, not a python topic, so if you don't know how to do that you're going to have to find the right place to ask, WHICH IS NOT HERE! Ideally you need to check what the server thinks it's setting the cooking to, what the browser thinks it received as the cookie, and what the server gets back afterwards to work out where the error is happening. -- Denis McMahon, denismfmcmahon at gmail.com From nikos.gr33k at gmail.com Tue Oct 8 18:52:44 2013 From: nikos.gr33k at gmail.com (=?UTF-8?B?zp3Or866zr/PgiDOkc67zrXOvs+Mz4DOv8+FzrvOv8+C?=) Date: Wed, 09 Oct 2013 01:52:44 +0300 Subject: Cookie gets changed when hit comes from a referrer In-Reply-To: References: Message-ID: ???? 8/10/2013 10:29 ??, ?/? Denis McMahon ??????: > On Tue, 08 Oct 2013 19:04:37 +0300, ????? ??????????? wrote: > >> Can you help me with this particuler problem please? > > Unfortunately I can't, because I am unable to reproduce the problem you > describe. > > When I load my test page in the browser, then replace it with something > else by entering an address in the address bar and pressing return, then > use the back link followed by the reload one, I am back at my test page > with the original cookie value. > > Of course, this is using my cookie etc code and mechanisms, and not > yours .... > > Now, either it's an issue in your python implementation of cookie > handling which isn't happening in my implementation, or it's something to > do with the way that your system passes data around (cgi) that doesn't > happen in mine (mod_wsgi), or it's happening in the browser you're > testing in, but not in my browser. > > Have you checked the cookie jar in the browser to see what value the > cookie has? Is that the value you think it should have? Note that > checking the cookie jar is a browser topic, not a python topic, so if you > don't know how to do that you're going to have to find the right place to > ask, WHICH IS NOT HERE! > > Ideally you need to check what the server thinks it's setting the cooking > to, what the browser thinks it received as the cookie, and what the > server gets back afterwards to work out where the error is happening. Is there something i can try to isolate the problem and make it work? By whole counters project is based on cookie handling now.... -- What is now proved was at first only imagined! & WebHost From ned at nedbatchelder.com Tue Oct 8 19:57:33 2013 From: ned at nedbatchelder.com (Ned Batchelder) Date: Tue, 08 Oct 2013 19:57:33 -0400 Subject: Cookie gets changed when hit comes from a referrer In-Reply-To: References: Message-ID: <52549BED.7030504@nedbatchelder.com> On 10/8/13 6:52 PM, ????? ??????????? wrote: > ???? 8/10/2013 10:29 ??, ?/? Denis McMahon ??????: >> On Tue, 08 Oct 2013 19:04:37 +0300, ????? ??????????? wrote: >> >>> Can you help me with this particuler problem please? >> >> Unfortunately I can't, because I am unable to reproduce the problem you >> describe. >> >> When I load my test page in the browser, then replace it with something >> else by entering an address in the address bar and pressing return, then >> use the back link followed by the reload one, I am back at my test page >> with the original cookie value. >> >> Of course, this is using my cookie etc code and mechanisms, and not >> yours .... >> >> Now, either it's an issue in your python implementation of cookie >> handling which isn't happening in my implementation, or it's >> something to >> do with the way that your system passes data around (cgi) that doesn't >> happen in mine (mod_wsgi), or it's happening in the browser you're >> testing in, but not in my browser. >> >> Have you checked the cookie jar in the browser to see what value the >> cookie has? Is that the value you think it should have? Note that >> checking the cookie jar is a browser topic, not a python topic, so if >> you >> don't know how to do that you're going to have to find the right >> place to >> ask, WHICH IS NOT HERE! >> >> Ideally you need to check what the server thinks it's setting the >> cooking >> to, what the browser thinks it received as the cookie, and what the >> server gets back afterwards to work out where the error is happening. > > Is there something i can try to isolate the problem and make it work? > By whole counters project is based on cookie handling now.... > > Nikos, as a few people have mentioned already, this is no longer a Python question. You need to find other avenues of support. --Ned. From steve+comp.lang.python at pearwood.info Tue Oct 8 21:33:25 2013 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: 09 Oct 2013 01:33:25 GMT Subject: Cookie gets changed when hit comes from a referrer References: Message-ID: <5254b264$0$29984$c3e8da3$5496439d@news.astraweb.com> On Wed, 09 Oct 2013 01:52:44 +0300, ????? ??????????? wrote: > Is there something i can try to isolate the problem and make it work? Of course there is. That is part of the job of the developer: hard work trying dozens, maybe hundreds of different things until you isolate the problem. There are no shortcuts, no magic button you can push to immediately identify the source of the problem. If you are not willing to spend hours, maybe days or weeks, working on this problem, then you should hire a programmer who is, and stop fooling yourself that you are a professional developer. An amateur who programs for fun can just give up when a problem becomes too difficult and isn't fun any more. A professional has to keep going. Start by identifying which browsers this occurs on. You should test using at least Firefox, Internet Explorer, Safari, Chrome and Opera, for as many different versions as you can find. You should also test with less common browsers such as Konqueror, Epiphany, lynx, links and others. See if there is a pattern in which ones behave as you expect and which ones don't. You should also test with and without cookies enabled, ad-blockers, and similar. Maybe you can replicate the problem if (say) the user accepts the first cookie, then rejects it when they click Back. If this only occurs with a single version of a single browser with cookies enabled and no ad blocker, you should report it as a bug to the browser developers. Make sure you give them enough detail to replicate the problem. If it's an old version, they'll probably say Won't Fix, and you'll just have to accept that your cookie handling code won't work for some percentage of visitors. Have you checked that the server is setting the cookie values you expect? Have you checked the value of the cookie in the browser? If you don't know how to do these things, this site will teach you everything you need: https://duckduckgo.com/ Follow the links until you reach enlightenment. There are *thousands* of pages on debugging programming problems. If you find it is broken with *all* of the above browsers, then you should suspect a bug in your Python code. In that case, since other people have failed to reproduce the reported problem, you are obviously doing something different than what you are telling us you are doing. Only in this case should you come back here to ask for help with your Python code. Before you do, read this, and follow the instructions: http://www.sscce.org/ If you are not willing to do these things, then stop pretending to be a professional developer, and admit that you are only programming for fun. There is no shame in this -- not everyone is cut out to be a professional programmer, just as not everybody makes a good doctor or taxi driver or carpenter. > By whole counters project is based on cookie handling now.... If you cannot solve the cookie problem, maybe you should reconsider the decision to rely on cookies. -- Steven From nikos.gr33k at gmail.com Wed Oct 9 04:24:35 2013 From: nikos.gr33k at gmail.com (=?UTF-8?B?zp3Or866zr/PgiDOkc67zrXOvs+Mz4DOv8+FzrvOv8+C?=) Date: Wed, 09 Oct 2013 11:24:35 +0300 Subject: Cookie gets changed when hit comes from a referrer In-Reply-To: <5254b264$0$29984$c3e8da3$5496439d@news.astraweb.com> References: <5254b264$0$29984$c3e8da3$5496439d@news.astraweb.com> Message-ID: ???? 9/10/2013 4:33 ??, ?/? Steven D'Aprano ??????: > On Wed, 09 Oct 2013 01:52:44 +0300, ????? ??????????? wrote: > >> Is there something i can try to isolate the problem and make it work? > > Of course there is. That is part of the job of the developer: hard work > trying dozens, maybe hundreds of different things until you isolate the > problem. There are no shortcuts, no magic button you can push to > immediately identify the source of the problem. > > If you are not willing to spend hours, maybe days or weeks, working on > this problem, then you should hire a programmer who is, and stop fooling > yourself that you are a professional developer. An amateur who programs > for fun can just give up when a problem becomes too difficult and isn't > fun any more. A professional has to keep going. > > Start by identifying which browsers this occurs on. You should test using > at least Firefox, Internet Explorer, Safari, Chrome and Opera, for as > many different versions as you can find. You should also test with less > common browsers such as Konqueror, Epiphany, lynx, links and others. See > if there is a pattern in which ones behave as you expect and which ones > don't. > > You should also test with and without cookies enabled, ad-blockers, and > similar. Maybe you can replicate the problem if (say) the user accepts > the first cookie, then rejects it when they click Back. > > If this only occurs with a single version of a single browser with > cookies enabled and no ad blocker, you should report it as a bug to the > browser developers. Make sure you give them enough detail to replicate > the problem. If it's an old version, they'll probably say Won't Fix, and > you'll just have to accept that your cookie handling code won't work for > some percentage of visitors. > > Have you checked that the server is setting the cookie values you expect? > Have you checked the value of the cookie in the browser? If you don't > know how to do these things, this site will teach you everything you need: > > https://duckduckgo.com/ > > Follow the links until you reach enlightenment. There are *thousands* of > pages on debugging programming problems. > > If you find it is broken with *all* of the above browsers, then you > should suspect a bug in your Python code. In that case, since other > people have failed to reproduce the reported problem, you are obviously > doing something different than what you are telling us you are doing. > Only in this case should you come back here to ask for help with your > Python code. Before you do, read this, and follow the instructions: > > http://www.sscce.org/ > > If you are not willing to do these things, then stop pretending to be a > professional developer, and admit that you are only programming for fun. > There is no shame in this -- not everyone is cut out to be a professional > programmer, just as not everybody makes a good doctor or taxi driver or > carpenter. > > >> By whole counters project is based on cookie handling now.... > > If you cannot solve the cookie problem, maybe you should reconsider the > decision to rely on cookies. > > > I managed t overcome it like this: cur.execute('''UPDATE visitors SET cookieID = %s, host = %s, city = %s, useros = %s, browser = %s, ref = %s, hits = hits + 1, lastvisit = %s WHERE counterID = %s and host = %s''', (cookieID, host, city, useros, browser, ref, lastvisit, cID, host) ) if not cur.rowcount: # if first time visitor on this page, create new record, if visitor exists then update record cur.execute('''INSERT INTO visitors (counterID, cookieID, host, city, useros, browser, ref, lastvisit) VALUES (%s, %s, %s, %s, %s, %s, %s, %s) ON DUPLICATE KEY UPDATE host = %s, city = %s, useros = %s, browser = %s, ref = %s, hits = hits + 1, lastvisit = %s''', (cID, cookieID, host, city, useros, browser, ref, lastvisit, host, city, useros, browser, ref, lastvisit) ) But thats a not clear way to handle the cookie because i involve host to help me identify its recorde since when my website hit comes from areferrer. i also tried adding the domain when i set the cookie but this didnt helped me at all: # initialize cookie and retrieve cookie from clients browser cookie = cookies.SimpleCookie( os.environ['HTTP_COOKIE'] ) if cookie.get('ID') is not None: cookieID = cookie['ID'].value else: cookieID = random.randrange(0, 9999) cookie['ID'] = cookieID cookie['ID']['domain'] = ".superhost.gr" cookie['ID']['path'] = '/' cookie["ID"]["expires"] = 60*60*24*365 # this cookie will expire in a year i read some links from duckduckgo but that didnt help me solve this. Please someone esle try to reproduce the problem by just using cgi and not mod_wsgi. ps. Really why duckduckgo and not google.com ? From breamoreboy at yahoo.co.uk Wed Oct 9 04:44:15 2013 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Wed, 09 Oct 2013 09:44:15 +0100 Subject: Cookie gets changed when hit comes from a referrer In-Reply-To: References: <5254b264$0$29984$c3e8da3$5496439d@news.astraweb.com> Message-ID: On 09/10/2013 09:24, ????? ??????????? wrote: You have been told repeatedly that your questions have nothing to do with Python, e.g. Ben Finney just over an hour ago "None of this has to do with Python. Please do not ask this Python doscussion forum to educate you on how HTTP operates." Please be courteous enough to take your questions to an appropriate forum. -- Roses are red, Violets are blue, Most poems rhyme, But this one doesn't. Mark Lawrence From denismfmcmahon at gmail.com Wed Oct 9 10:45:33 2013 From: denismfmcmahon at gmail.com (Denis McMahon) Date: Wed, 9 Oct 2013 14:45:33 +0000 (UTC) Subject: Cookie gets changed when hit comes from a referrer References: <5254b264$0$29984$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Wed, 09 Oct 2013 11:24:35 +0300, ????? ??????????? wrote: > Please someone esle try to reproduce the problem by just using cgi and > not mod_wsgi. I have no intention of reconfiguring my web server just to prove that your code isn't working. We already know that your code isn't working. -- Denis McMahon, denismfmcmahon at gmail.com From piet at vanoostrum.org Wed Oct 9 14:36:56 2013 From: piet at vanoostrum.org (Piet van Oostrum) Date: Wed, 09 Oct 2013 14:36:56 -0400 Subject: Cookie gets changed when hit comes from a referrer References: <5254b264$0$29984$c3e8da3$5496439d@news.astraweb.com> Message-ID: ????? ??????????? writes: > > # initialize cookie and retrieve cookie from clients browser > cookie = cookies.SimpleCookie( os.environ['HTTP_COOKIE'] ) > > if cookie.get('ID') is not None: > cookieID = cookie['ID'].value > else: > cookieID = random.randrange(0, 9999) > cookie['ID'] = cookieID > cookie['ID']['domain'] = ".superhost.gr" > cookie['ID']['path'] = '/' > cookie["ID"]["expires"] = 60*60*24*365 # this cookie will expire in a year > As Ian already has told you (but apparently you didn't pay attention to), your expires is wrong. So if your cookies disappear you should get this right first. from datetime import datetime, timedelta expiretime = datetime.utcnow() + timedelta(days=365) cookie["ID"]["expires"] = expiretime.strftime("%a, %d %b %Y %H:%M:%S GMT") -- Piet van Oostrum WWW: http://pietvanoostrum.com/ PGP key: [8DAE142BE17999C4] From nikos.gr33k at gmail.com Wed Oct 9 17:29:38 2013 From: nikos.gr33k at gmail.com (=?UTF-8?B?zp3Or866zr/PgiDOkc67zrXOvs+Mz4DOv8+FzrvOv8+C?=) Date: Thu, 10 Oct 2013 00:29:38 +0300 Subject: Cookie gets changed when hit comes from a referrer In-Reply-To: References: <5254b264$0$29984$c3e8da3$5496439d@news.astraweb.com> Message-ID: ???? 9/10/2013 9:36 ??, ?/? Piet van Oostrum ??????: > ????? ??????????? writes: > >> >> # initialize cookie and retrieve cookie from clients browser >> cookie = cookies.SimpleCookie( os.environ['HTTP_COOKIE'] ) >> >> if cookie.get('ID') is not None: >> cookieID = cookie['ID'].value >> else: >> cookieID = random.randrange(0, 9999) >> cookie['ID'] = cookieID >> cookie['ID']['domain'] = ".superhost.gr" >> cookie['ID']['path'] = '/' >> cookie["ID"]["expires"] = 60*60*24*365 # this cookie will expire in a year >> > As Ian already has told you (but apparently you didn't pay attention to), your expires is wrong. So if your cookies disappear you should get this right first. > > from datetime import datetime, timedelta > expiretime = datetime.utcnow() + timedelta(days=365) > > cookie["ID"]["expires"] = expiretime.strftime("%a, %d %b %Y %H:%M:%S GMT") > Expire is not the issue here, as i have it is working with no problem. when i print the cookie expiration time is calculated properly. Something else is going worng. From joel.goldstick at gmail.com Wed Oct 9 18:03:09 2013 From: joel.goldstick at gmail.com (Joel Goldstick) Date: Wed, 9 Oct 2013 18:03:09 -0400 Subject: Cookie gets changed when hit comes from a referrer In-Reply-To: References: <5254b264$0$29984$c3e8da3$5496439d@news.astraweb.com> Message-ID: > > Expire is not the issue here, as i have it is working with no problem. > when i print the cookie expiration time is calculated properly. > Something else is going worng. > Indeed! > -- > https://mail.python.org/mailman/listinfo/python-list -- Joel Goldstick http://joelgoldstick.com From breamoreboy at yahoo.co.uk Wed Oct 9 18:26:25 2013 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Wed, 09 Oct 2013 23:26:25 +0100 Subject: Cookie gets changed when hit comes from a referrer In-Reply-To: References: <5254b264$0$29984$c3e8da3$5496439d@news.astraweb.com> Message-ID: On 09/10/2013 23:03, Joel Goldstick wrote: >> >> Expire is not the issue here, as i have it is working with no problem. >> when i print the cookie expiration time is calculated properly. > > >> Something else is going worng. >> > Indeed! > Well explained here http://en.wikipedia.org/wiki/User_error -- Roses are red, Violets are blue, Most poems rhyme, But this one doesn't. Mark Lawrence From breamoreboy at yahoo.co.uk Wed Oct 9 03:00:16 2013 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Wed, 09 Oct 2013 08:00:16 +0100 Subject: Cookie gets changed when hit comes from a referrer In-Reply-To: References: Message-ID: On 08/10/2013 23:52, ????? ??????????? wrote: > > Is there something i can try to isolate the problem and make it work? > As you are the problem why not try solitary confinement? :) -- Roses are red, Violets are blue, Most poems rhyme, But this one doesn't. Mark Lawrence From denismfmcmahon at gmail.com Wed Oct 9 10:43:03 2013 From: denismfmcmahon at gmail.com (Denis McMahon) Date: Wed, 9 Oct 2013 14:43:03 +0000 (UTC) Subject: Cookie gets changed when hit comes from a referrer References: Message-ID: On Wed, 09 Oct 2013 01:52:44 +0300, ????? ??????????? wrote: > ???? 8/10/2013 10:29 ??, ?/? Denis McMahon ??????: >> Have you checked the cookie jar in the browser to see what value the >> cookie has? Is that the value you think it should have? Note that >> checking the cookie jar is a browser topic, not a python topic, so if >> you don't know how to do that you're going to have to find the right >> place to ask, WHICH IS NOT HERE! >> Ideally you need to check what the server thinks it's setting the >> cooking to, what the browser thinks it received as the cookie, and what >> the server gets back afterwards to work out where the error is >> happening. > Is there something i can try to isolate the problem and make it work? > By whole counters project is based on cookie handling now.... See those last two paragraphs there that you quoted. You should have read them. -- Denis McMahon, denismfmcmahon at gmail.com From nikos.gr33k at gmail.com Wed Oct 9 11:00:28 2013 From: nikos.gr33k at gmail.com (=?UTF-8?B?zp3Or866zr/PgiDOkc67zrXOvs+Mz4DOv8+FzrvOv8+C?=) Date: Wed, 09 Oct 2013 18:00:28 +0300 Subject: Cookie gets changed when hit comes from a referrer In-Reply-To: References: Message-ID: ???? 9/10/2013 5:43 ??, ?/? Denis McMahon ??????: > On Wed, 09 Oct 2013 01:52:44 +0300, ????? ??????????? wrote: > >> ???? 8/10/2013 10:29 ??, ?/? Denis McMahon ??????: > >>> Have you checked the cookie jar in the browser to see what value the >>> cookie has? Is that the value you think it should have? Note that >>> checking the cookie jar is a browser topic, not a python topic, so if >>> you don't know how to do that you're going to have to find the right >>> place to ask, WHICH IS NOT HERE! > >>> Ideally you need to check what the server thinks it's setting the >>> cooking to, what the browser thinks it received as the cookie, and what >>> the server gets back afterwards to work out where the error is >>> happening. > >> Is there something i can try to isolate the problem and make it work? >> By whole counters project is based on cookie handling now.... > > See those last two paragraphs there that you quoted. You should have read > them. > ok so then tell me where i should ask this. -- What is now proved was at first only imagined! & WebHost From joel.goldstick at gmail.com Wed Oct 9 11:20:07 2013 From: joel.goldstick at gmail.com (Joel Goldstick) Date: Wed, 9 Oct 2013 11:20:07 -0400 Subject: Cookie gets changed when hit comes from a referrer In-Reply-To: References: Message-ID: On Wed, Oct 9, 2013 at 11:00 AM, ????? ??????????? wrote: > ???? 9/10/2013 5:43 ??, ?/? Denis McMahon ??????: > >> On Wed, 09 Oct 2013 01:52:44 +0300, ????? ??????????? wrote: >> >>> ???? 8/10/2013 10:29 ??, ?/? Denis McMahon ??????: >> >> >>>> Have you checked the cookie jar in the browser to see what value the >>>> cookie has? Is that the value you think it should have? Note that >>>> checking the cookie jar is a browser topic, not a python topic, so if >>>> you don't know how to do that you're going to have to find the right >>>> place to ask, WHICH IS NOT HERE! >> >> >>>> Ideally you need to check what the server thinks it's setting the >>>> cooking to, what the browser thinks it received as the cookie, and what >>>> the server gets back afterwards to work out where the error is >>>> happening. >> >> >>> Is there something i can try to isolate the problem and make it work? >>> By whole counters project is based on cookie handling now.... >> >> >> See those last two paragraphs there that you quoted. You should have read >> them. >> > ok so then tell me where i should ask this. > I already told you where to learn about cookies. Try that first. People will be nicer to your questions if you show you are willing to come prepared. As to where to ask -- perhaps there is an http mailing list. Your code can't work if you don't know how cookies work. -- Joel Goldstick http://joelgoldstick.com From breamoreboy at yahoo.co.uk Wed Oct 9 11:39:34 2013 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Wed, 09 Oct 2013 16:39:34 +0100 Subject: Cookie gets changed when hit comes from a referrer In-Reply-To: References: Message-ID: On 09/10/2013 16:00, ????? ??????????? wrote: > ok so then tell me where i should ask this. > Google, bing, duckduckgo, ask, yahoo ... -- Roses are red, Violets are blue, Most poems rhyme, But this one doesn't. Mark Lawrence From denismfmcmahon at gmail.com Wed Oct 9 14:06:05 2013 From: denismfmcmahon at gmail.com (Denis McMahon) Date: Wed, 9 Oct 2013 18:06:05 +0000 (UTC) Subject: Cookie gets changed when hit comes from a referrer References: Message-ID: On Wed, 09 Oct 2013 18:00:28 +0300, ????? ??????????? wrote: > ???? 9/10/2013 5:43 ??, ?/? Denis McMahon ??????: >> On Wed, 09 Oct 2013 01:52:44 +0300, ????? ??????????? wrote: >> >>> ???? 8/10/2013 10:29 ??, ?/? Denis McMahon ??????: >> >>>> Have you checked the cookie jar in the browser to see what value the >>>> cookie has? Is that the value you think it should have? Note that >>>> checking the cookie jar is a browser topic, not a python topic, so if >>>> you don't know how to do that you're going to have to find the right >>>> place to ask, WHICH IS NOT HERE! >>>> Ideally you need to check what the server thinks it's setting the >>>> cooking to, what the browser thinks it received as the cookie, and >>>> what the server gets back afterwards to work out where the error is >>>> happening. >>> Is there something i can try to isolate the problem and make it work? >>> By whole counters project is based on cookie handling now.... >> See those last two paragraphs there that you quoted. You should have >> read them. > ok so then tell me where i should ask this. In those two paragraphs I have told you what you need to do to isolate where your problem is occurring. If you don't know how to do that, then I commend to you the wealth of information that may be discovered using search engines. However, as we keep telling you, this group is not the right place to ask questions such as: a) How do I see what's in my web browser's cookie jar? b) How do I make my web server log the cookies it sends and receives? For (a) you want information about your browser, not about python. For (b) you want information about your web server, not about python. Find the relevant forums and ask in them. -- Denis McMahon, denismfmcmahon at gmail.com From breamoreboy at yahoo.co.uk Wed Oct 9 14:28:06 2013 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Wed, 09 Oct 2013 19:28:06 +0100 Subject: Cookie gets changed when hit comes from a referrer In-Reply-To: References: Message-ID: On 09/10/2013 19:06, Denis McMahon wrote: > > Find the relevant forums and ask in them. > Why am I thinking of this http://en.wikipedia.org/wiki/There%27s_a_Hole_in_My_Bucket ? -- Roses are red, Violets are blue, Most poems rhyme, But this one doesn't. Mark Lawrence From python.list at tim.thechases.com Wed Oct 9 15:26:12 2013 From: python.list at tim.thechases.com (Tim Chase) Date: Wed, 9 Oct 2013 14:26:12 -0500 Subject: Cookie gets changed when hit comes from a referrer In-Reply-To: References: Message-ID: <20131009142612.2407385a@bigbox.christie.dr> On 2013-10-09 19:28, Mark Lawrence wrote: > On 09/10/2013 19:06, Denis McMahon wrote: >> Find the relevant forums and ask in them. > > Why am I thinking of this > http://en.wikipedia.org/wiki/There%27s_a_Hole_in_My_Bucket ? There's a bug in my program, dear newsgroup, dear newsgroup, There's a bug in my program, dear newsgroup a bug. Then fix it, dear Nikos, dear Nikos, dear Nikos, Then fix it, dear Nikos, dear Nikos, fix it. With what shall I fix it, dear newsgroup, dear newsgroup, With what shall I fix it, dear newsgroup, with what? The traceback, dear Nikos, dear Nikos, dear Nikos, The traceback, dear Nikos, dear Nikos, traceback. The problem's not Python, dear newsgroup, dear newsgroup, The problem's not Python, dear newsgroup, what now? Try Google, dear Nikos, dear Nikos, dear Nikos, Try Google, dear Nikos, dear Nikos, Google. But what shall I query, dear newsgroup, dear newsgroup, But what shall I query, dear newsgroup, but what? Some keywords, dear Nikos, dear Nikos, dear Nikos, Some keywords, dear Nikos, dear Nikos, keywords. I know it's off-topic, dear newsgroup, dear newsgroup, I know it's off-topic, dear newsgroup, but help... I can see the similarity ;-) -tkc From breamoreboy at yahoo.co.uk Wed Oct 9 15:40:21 2013 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Wed, 09 Oct 2013 20:40:21 +0100 Subject: Cookie gets changed when hit comes from a referrer In-Reply-To: <20131009142612.2407385a@bigbox.christie.dr> References: <20131009142612.2407385a@bigbox.christie.dr> Message-ID: On 09/10/2013 20:26, Tim Chase wrote: > On 2013-10-09 19:28, Mark Lawrence wrote: >> On 09/10/2013 19:06, Denis McMahon wrote: >>> Find the relevant forums and ask in them. >> >> Why am I thinking of this >> http://en.wikipedia.org/wiki/There%27s_a_Hole_in_My_Bucket ? > > There's a bug in my program, dear newsgroup, dear newsgroup, > There's a bug in my program, dear newsgroup a bug. > > Then fix it, dear Nikos, dear Nikos, dear Nikos, > Then fix it, dear Nikos, dear Nikos, fix it. > > With what shall I fix it, dear newsgroup, dear newsgroup, > With what shall I fix it, dear newsgroup, with what? > > The traceback, dear Nikos, dear Nikos, dear Nikos, > The traceback, dear Nikos, dear Nikos, traceback. > > The problem's not Python, dear newsgroup, dear newsgroup, > The problem's not Python, dear newsgroup, what now? > > Try Google, dear Nikos, dear Nikos, dear Nikos, > Try Google, dear Nikos, dear Nikos, Google. > > But what shall I query, dear newsgroup, dear newsgroup, > But what shall I query, dear newsgroup, but what? > > Some keywords, dear Nikos, dear Nikos, dear Nikos, > Some keywords, dear Nikos, dear Nikos, keywords. > > I know it's off-topic, dear newsgroup, dear newsgroup, > I know it's off-topic, dear newsgroup, but help... > > > I can see the similarity ;-) > > -tkc > Great minds think alike? :) -- Roses are red, Violets are blue, Most poems rhyme, But this one doesn't. Mark Lawrence From steve+comp.lang.python at pearwood.info Wed Oct 9 19:48:12 2013 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: 09 Oct 2013 23:48:12 GMT Subject: Cookie gets changed when hit comes from a referrer References: Message-ID: <5255eb3c$0$29984$c3e8da3$5496439d@news.astraweb.com> On Wed, 09 Oct 2013 18:06:05 +0000, Denis McMahon wrote: > Find the relevant forums and ask in them. In fairness to Nikos, that may not be an easy thing to do. I for one have *no idea* where to find an appropriate forum to learn about these sorts of web basics. comp.protocol.http doesn't exist :-) I'm pretty sure that everyone here has been in this situation. We've pretty much all asked for recommendations "where is a good place to learn about X?", for some definition of X. That is a reasonable question, and "just google it" is *not* a reasonable answer, because search engines are not good at that sort of value judgement. Human beings are. And even the best of us have had days were we simply cannot come up with good search terms that find us what we need. So, for the benefit of anyone, not just Nikos, who wants to learn about how browsers connect to web sites and how to run a web server, does anyone have any recommendation for tutorials, mailing lists, web forums or books which are suitable? Preferably things you have used yourself, and can personally vouch for being useful. I'm pretty sure that *somebody* here has been in the position of needing to learn about running a website, and can point Nikos in the right direction (away from here). How did you learn? And if nobody is able, or willing, to answer? I've been in that position too, asking for help that nobody was able to give. It sucks, and you move on and do your best. -- Steven From rosuav at gmail.com Wed Oct 9 20:18:55 2013 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 10 Oct 2013 11:18:55 +1100 Subject: Cookie gets changed when hit comes from a referrer In-Reply-To: <5255eb3c$0$29984$c3e8da3$5496439d@news.astraweb.com> References: <5255eb3c$0$29984$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Thu, Oct 10, 2013 at 10:48 AM, Steven D'Aprano wrote: > So, for the benefit of anyone, not just Nikos, who wants to learn about > how browsers connect to web sites and how to run a web server, does > anyone have any recommendation for tutorials, mailing lists, web forums > or books which are suitable? Preferably things you have used yourself, > and can personally vouch for being useful. Personally, what I find most useful is network-level tracing - if I want to know what my web server's doing, I'll telnet to it (or, more likely, use my MUD client) and look at exactly what it's sending; and if I want to know what a browser's doing, I'll do the same (since my MUD clients allow me to run them "backwards", listening rather than connecting). But that isn't for everyone, I'm aware of that. ChrisA From denismfmcmahon at gmail.com Wed Oct 9 20:31:06 2013 From: denismfmcmahon at gmail.com (Denis McMahon) Date: Thu, 10 Oct 2013 00:31:06 +0000 (UTC) Subject: Cookie gets changed when hit comes from a referrer References: <5255eb3c$0$29984$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Wed, 09 Oct 2013 23:48:12 +0000, Steven D'Aprano wrote: > On Wed, 09 Oct 2013 18:06:05 +0000, Denis McMahon wrote: > >> Find the relevant forums and ask in them. > > In fairness to Nikos, that may not be an easy thing to do. I for one > have *no idea* where to find an appropriate forum to learn about these > sorts of web basics. comp.protocol.http doesn't exist :-) If Nikos wants to write programs that communicate using internet protocols, Nikos really needs to learn where internet protocols are defined, how to read and interpret those protocol definitions, and how to check that the data he's sending or receiving is the data that he thinks he's sending or receiving. Just as, if Nikos wants to generate web pages using HTML markup, Nikos needs to learn where HTML markup is defined. There's no easy path to this knowledge. For some of us, it's been a 30 or more year journey so far and we're still learning (I'm into year 36, having started at the age of 14 and turning 51 in seven weeks). If Nikos can't even figure out the right queries to feed into a search engine to get started on such matters as looking at the cookie jar in a browser or enabling cookie logging on a server, then he probably shouldn't be trying to code at this level. -- Denis McMahon, denismfmcmahon at gmail.com From steve+comp.lang.python at pearwood.info Wed Oct 9 21:10:19 2013 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: 10 Oct 2013 01:10:19 GMT Subject: Cookie gets changed when hit comes from a referrer References: <5255eb3c$0$29984$c3e8da3$5496439d@news.astraweb.com> Message-ID: <5255fe7b$0$29984$c3e8da3$5496439d@news.astraweb.com> On Thu, 10 Oct 2013 00:31:06 +0000, Denis McMahon wrote: > On Wed, 09 Oct 2013 23:48:12 +0000, Steven D'Aprano wrote: > >> On Wed, 09 Oct 2013 18:06:05 +0000, Denis McMahon wrote: >> >>> Find the relevant forums and ask in them. >> >> In fairness to Nikos, that may not be an easy thing to do. I for one >> have *no idea* where to find an appropriate forum to learn about these >> sorts of web basics. comp.protocol.http doesn't exist :-) > > If Nikos wants to write programs that communicate using internet > protocols, Nikos really needs to learn where internet protocols are > defined, how to read and interpret those protocol definitions, and how > to check that the data he's sending or receiving is the data that he > thinks he's sending or receiving. You can't seriously mean that everyone who runs a website has to become skilled at reading and interpreting the RFCs for "internet protocols". Which protocols? All the way down to TCP/IP? > Just as, if Nikos wants to generate web pages using HTML markup, Nikos > needs to learn where HTML markup is defined. > > There's no easy path to this knowledge. For some of us, it's been a 30 > or more year journey so far and we're still learning (I'm into year 36, > having started at the age of 14 and turning 51 in seven weeks). > > If Nikos can't even figure out the right queries to feed into a search > engine to get started on such matters as looking at the cookie jar in a > browser or enabling cookie logging on a server, then he probably > shouldn't be trying to code at this level. Nikos Nikos Nikos... and what about me? If I asked you for a few pointers about a good place to learn about running a web site, would you tell me to fuck off too? I wonder what you are doing here, if you are so unwilling to share your hard-earned knowledge with others as you seem in this post. This attitude is not the Denis McMahon I'm used to. Honestly, your response seems to me to be just a more verbose way of saying "RTFM n00b", and about as helpful. Speaking for myself, I don't want this forum to be the sort of place where that is considered an appropriate response, no matter how politely the dismissal is dressed up. I'm here to help people, and yes, that even means Nikos. To give back to the community that helped me (and continues to help me). In my opinion, if we're going to tell people to RTFM the least we can do is point them at the right manual, and not just dismiss them by telling them to google it. I don't think that's too much to ask. (On the other hand, it's okay to say "I don't know of any good forums for learning this stuff. Sorry mate, I can't help.") I have no objection to encouraging people to read the fine manual, and I don't intend to be Nikos' (or anyone else's) unpaid full-time help desk and troubleshooter. But I do think it is simply unfair to treat him more harshly than we would others in the same position. If *anyone else* asked for help on these sorts of network and browser questions, we'd give them more constructive pointers than just "google it". Nikos, are you reading this? This is what happens when you behave like a royal pain in the arse and annoy people. They stop wanting to help you. Be told. Learn from this. Don't repeat this mistake in the next forum. If you learn nothing else, learn that lesson. -- Steven From rosuav at gmail.com Wed Oct 9 21:28:31 2013 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 10 Oct 2013 12:28:31 +1100 Subject: Cookie gets changed when hit comes from a referrer In-Reply-To: <5255fe7b$0$29984$c3e8da3$5496439d@news.astraweb.com> References: <5255eb3c$0$29984$c3e8da3$5496439d@news.astraweb.com> <5255fe7b$0$29984$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Thu, Oct 10, 2013 at 12:10 PM, Steven D'Aprano wrote: > On Thu, 10 Oct 2013 00:31:06 +0000, Denis McMahon wrote: > >> On Wed, 09 Oct 2013 23:48:12 +0000, Steven D'Aprano wrote: >> >>> On Wed, 09 Oct 2013 18:06:05 +0000, Denis McMahon wrote: >>> >>>> Find the relevant forums and ask in them. >>> >>> In fairness to Nikos, that may not be an easy thing to do. I for one >>> have *no idea* where to find an appropriate forum to learn about these >>> sorts of web basics. comp.protocol.http doesn't exist :-) >> >> If Nikos wants to write programs that communicate using internet >> protocols, Nikos really needs to learn where internet protocols are >> defined, how to read and interpret those protocol definitions, and how >> to check that the data he's sending or receiving is the data that he >> thinks he's sending or receiving. > > You can't seriously mean that everyone who runs a website has to become > skilled at reading and interpreting the RFCs for "internet protocols". > Which protocols? All the way down to TCP/IP? Certainly not; but if he's run into trouble with cookies, I think it's not too much to ask him to read: http://en.wikipedia.org/wiki/HTTP_cookie It has a lot of text, but quite a bit of it is likely to be of use. Then there's links down the bottom to an RFC, to Mozilla, and so on, which could also be of use. Half an hour spent reading there will pay good dividends. > ... I do think it is simply unfair to treat him more > harshly than we would others in the same position. If *anyone else* asked > for help on these sorts of network and browser questions, we'd give them > more constructive pointers than just "google it". > > Nikos, are you reading this? This is what happens when you behave like a > royal pain in the arse and annoy people. They stop wanting to help you. > Be told. Learn from this. Don't repeat this mistake in the next forum. If > you learn nothing else, learn that lesson. Actually no, it's not unfair to treat him more harshly. What's unfair is that he hasn't been banned, killfiled by everyone, or in some other way completely cut off from assistance. It's just as unfair as the mercy God shows to us. We're being extremely unjust in helping him... he hasn't earned anything, he hasn't merited help, he's not paying us. It's a gift, it's free service, and it's a privilege. Nikos, rejoice in the unfair and inconceivably merciful state of this forum, and as Steven says, learn from it and don't make the same mistake elsewhere. (I've made that mistake myself, too. I've made myself a stench in the nostrils of certain communities. It's not something you can't come back from; it's part of life, part of learning.) ChrisA From breamoreboy at yahoo.co.uk Thu Oct 10 01:21:57 2013 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Thu, 10 Oct 2013 06:21:57 +0100 Subject: Cookie gets changed when hit comes from a referrer In-Reply-To: References: <5255eb3c$0$29984$c3e8da3$5496439d@news.astraweb.com> <5255fe7b$0$29984$c3e8da3$5496439d@news.astraweb.com> Message-ID: On 10/10/2013 02:28, Chris Angelico wrote: > > Half an hour spent reading there will pay > good dividends. > That's been sadly lacking in all of these threads. With responses coming back faster than a ball on the centre court at Wimbledon, it's hardly surprising that progress has been conspicious by its absence. -- Roses are red, Violets are blue, Most poems rhyme, But this one doesn't. Mark Lawrence From rustompmody at gmail.com Thu Oct 10 01:36:54 2013 From: rustompmody at gmail.com (rusi) Date: Wed, 9 Oct 2013 22:36:54 -0700 (PDT) Subject: Cookie gets changed when hit comes from a referrer In-Reply-To: <5255fe7b$0$29984$c3e8da3$5496439d@news.astraweb.com> References: <5255eb3c$0$29984$c3e8da3$5496439d@news.astraweb.com> <5255fe7b$0$29984$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Thursday, October 10, 2013 6:40:19 AM UTC+5:30, Steven D'Aprano wrote: > > I have no objection to encouraging people to read the fine manual, and I > don't intend to be Nikos' (or anyone else's) unpaid full-time help desk > and troubleshooter. But I do think it is simply unfair to treat him more > harshly than we would others in the same position. If *anyone else* asked > for help on these sorts of network and browser questions, we'd give them > more constructive pointers than just "google it". https://mail.python.org/pipermail/python-list/2013-October/657221.html https://mail.python.org/pipermail/python-list/2013-October/657034.html From breamoreboy at yahoo.co.uk Thu Oct 10 01:56:17 2013 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Thu, 10 Oct 2013 06:56:17 +0100 Subject: Cookie gets changed when hit comes from a referrer In-Reply-To: References: <5255eb3c$0$29984$c3e8da3$5496439d@news.astraweb.com> <5255fe7b$0$29984$c3e8da3$5496439d@news.astraweb.com> Message-ID: On 10/10/2013 06:36, rusi wrote: > On Thursday, October 10, 2013 6:40:19 AM UTC+5:30, Steven D'Aprano wrote: >> >> I have no objection to encouraging people to read the fine manual, and I >> don't intend to be Nikos' (or anyone else's) unpaid full-time help desk >> and troubleshooter. But I do think it is simply unfair to treat him more >> harshly than we would others in the same position. If *anyone else* asked >> for help on these sorts of network and browser questions, we'd give them >> more constructive pointers than just "google it". > > https://mail.python.org/pipermail/python-list/2013-October/657221.html > > https://mail.python.org/pipermail/python-list/2013-October/657034.html > I don't understand you point, please explain. -- Roses are red, Violets are blue, Most poems rhyme, But this one doesn't. Mark Lawrence From ben+python at benfinney.id.au Thu Oct 10 02:01:25 2013 From: ben+python at benfinney.id.au (Ben Finney) Date: Thu, 10 Oct 2013 17:01:25 +1100 Subject: Cookie gets changed when hit comes from a referrer References: <5255eb3c$0$29984$c3e8da3$5496439d@news.astraweb.com> <5255fe7b$0$29984$c3e8da3$5496439d@news.astraweb.com> Message-ID: <7wwqlloea2.fsf@benfinney.id.au> rusi writes: > On Thursday, October 10, 2013 6:40:19 AM UTC+5:30, Steven D'Aprano wrote: > > If *anyone else* asked for help on these sorts of network and > > browser questions, we'd give them more constructive pointers than > > just "google it". Someone who has so consistently demonstrated their unwillingness to learn, and their persistent arrogance in flooding this forum with demands for assistance, is quite deserving of curt ?go and find out for yourself, leave this forum alone? responses. We are a, and should remain, an open and tolerant community. But the persistently disrespectful actions of Nikos threaten the forum's value and usefulness for everyone else. There are limits to how much disruption and obstinacy this community should tolerate from a given individual. > https://mail.python.org/pipermail/python-list/2013-October/657221.html > https://mail.python.org/pipermail/python-list/2013-October/657034.html Both of which got quite helpful responses: a brief, polite referral to seek a specific discussion forum where their requests would be on topic. -- \ ?If nature has made any one thing less susceptible than all | `\ others of exclusive property, it is the action of the thinking | _o__) power called an idea? ?Thomas Jefferson | Ben Finney From steve at pearwood.info Thu Oct 10 03:18:10 2013 From: steve at pearwood.info (Steven D'Aprano) Date: 10 Oct 2013 07:18:10 GMT Subject: Cookie gets changed when hit comes from a referrer References: <5255eb3c$0$29984$c3e8da3$5496439d@news.astraweb.com> <5255fe7b$0$29984$c3e8da3$5496439d@news.astraweb.com> Message-ID: <525654b2$0$29887$c3e8da3$5496439d@news.astraweb.com> On Wed, 09 Oct 2013 22:36:54 -0700, rusi wrote: > On Thursday, October 10, 2013 6:40:19 AM UTC+5:30, Steven D'Aprano > wrote: >> >> I have no objection to encouraging people to read the fine manual, and >> I don't intend to be Nikos' (or anyone else's) unpaid full-time help >> desk and troubleshooter. But I do think it is simply unfair to treat >> him more harshly than we would others in the same position. If *anyone >> else* asked for help on these sorts of network and browser questions, >> we'd give them more constructive pointers than just "google it". > > https://mail.python.org/pipermail/python-list/2013-October/657221.html That's a good example of exactly the sort of thing I'm talking about. Joel responded with "have you checked the pysvn mailing list" and gave a URL to that list. That is a good, helpful response, given that we can't be expected to know everything about every arbitrary package that might use Python. > https://mail.python.org/pipermail/python-list/2013-October/657034.html And even this got a response suggesting the poster look for an nginx mailing list, which while less helpful than it could have been, was still a concrete, helpful response: not "RTFM", or "just google it", but "that's a problem with nginx, you need an nginx forum, not a Python one". -- Steven From steve at pearwood.info Thu Oct 10 03:20:07 2013 From: steve at pearwood.info (Steven D'Aprano) Date: 10 Oct 2013 07:20:07 GMT Subject: Cookie gets changed when hit comes from a referrer References: <5255eb3c$0$29984$c3e8da3$5496439d@news.astraweb.com> <5255fe7b$0$29984$c3e8da3$5496439d@news.astraweb.com> Message-ID: <52565527$0$29887$c3e8da3$5496439d@news.astraweb.com> On Thu, 10 Oct 2013 17:01:25 +1100, Ben Finney wrote: > There are limits to how much disruption and obstinacy this community > should tolerate from a given individual. I think we are in violent agreement :-) It's possible we disagree about where to draw the line, and whether persistent cluelessness counts as wilful disruption, but otherwise I'm not disagreeing with you. -- Steven From antoon.pardon at rece.vub.ac.be Thu Oct 10 04:50:25 2013 From: antoon.pardon at rece.vub.ac.be (Antoon Pardon) Date: Thu, 10 Oct 2013 10:50:25 +0200 Subject: Cookie gets changed when hit comes from a referrer In-Reply-To: <5255fe7b$0$29984$c3e8da3$5496439d@news.astraweb.com> References: <5255eb3c$0$29984$c3e8da3$5496439d@news.astraweb.com> <5255fe7b$0$29984$c3e8da3$5496439d@news.astraweb.com> Message-ID: <52566A51.4070702@rece.vub.ac.be> Op 10-10-13 03:10, Steven D'Aprano schreef: > On Thu, 10 Oct 2013 00:31:06 +0000, Denis McMahon wrote: >> >> If Nikos can't even figure out the right queries to feed into a search >> engine to get started on such matters as looking at the cookie jar in a >> browser or enabling cookie logging on a server, then he probably >> shouldn't be trying to code at this level. > > Nikos Nikos Nikos... and what about me? If I asked you for a few pointers > about a good place to learn about running a web site, would you tell me > to fuck off too? I wonder what you are doing here, if you are so > unwilling to share your hard-earned knowledge with others as you seem in > this post. This attitude is not the Denis McMahon I'm used to. A year ago no. Now yes! Because you are behaving like an enabler. Nikos is the prime cause of the disruptions here and you are asking we should treat him as if he is a well behaving regular that comes with the occasional off topic question. You seem unable or unwilling to draw a line in how far we should go in enduring Nikos's anti-social behaviour. You seem unable to grasp that how we treat people depends on their behaviour in the past. > Honestly, your response seems to me to be just a more verbose way of > saying "RTFM n00b", and about as helpful. Speaking for myself, I don't > want this forum to be the sort of place where that is considered an > appropriate response, no matter how politely the dismissal is dressed up. But you are cooperating very hard in producing just that. By ignoring the reasons people react harshly to Nikos and pretending such a reaction is a reflection on how they would treat newbies in general you are being extremely unfair. > I'm here to help people, and yes, that even means Nikos. To give back to > the community that helped me (and continues to help me). In my opinion, > if we're going to tell people to RTFM the least we can do is point them > at the right manual, and not just dismiss them by telling them to google > it. I don't think that's too much to ask. And how far are you willing to go with this help? Right now you are helping Nikos in his anti-social behaviour. Is that your idea of giving back to the community? Annoying a significant part of it in your quest to help people? > (On the other hand, it's okay to say "I don't know of any good forums for > learning this stuff. Sorry mate, I can't help.") > > I have no objection to encouraging people to read the fine manual, and I > don't intend to be Nikos' (or anyone else's) unpaid full-time help desk > and troubleshooter. But I do think it is simply unfair to treat him more > harshly than we would others in the same position. If *anyone else* asked > for help on these sorts of network and browser questions, we'd give them > more constructive pointers than just "google it". We don't. The history peope have on this forum is part of the position they are in. If someone has abused the hospitality of an environment, then there is nothing wrong if that environment starts reacting hostile. That is the *situation* Nikos is in now. So stop pretending any regular asking an occasional off topic question would be the same situation. -- Antoon Pardon From denismfmcmahon at gmail.com Thu Oct 10 09:42:16 2013 From: denismfmcmahon at gmail.com (Denis McMahon) Date: Thu, 10 Oct 2013 13:42:16 +0000 (UTC) Subject: Cookie gets changed when hit comes from a referrer References: <5255eb3c$0$29984$c3e8da3$5496439d@news.astraweb.com> <5255fe7b$0$29984$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Thu, 10 Oct 2013 01:10:19 +0000, Steven D'Aprano wrote: >> If Nikos wants to write programs that communicate using internet >> protocols, Nikos really needs to learn where internet protocols are >> defined, how to read and interpret those protocol definitions, and how >> to check that the data he's sending or receiving is the data that he >> thinks he's sending or receiving. > > You can't seriously mean that everyone who runs a website has to become > skilled at reading and interpreting the RFCs for "internet protocols". > Which protocols? All the way down to TCP/IP? I'm not talking about running a website, I'm talking about writing code that uses specific protocols to transfer data. If Nikos is writing code that uses http, then he needs to understand http. His previous questions relating to his wish to spoof ip addresses suggests that in his case, he needs to understand tcp/ip as well. I'm not suggesting that every web admin needs to know these, but anyone who is trying to talk at the protocol level needs to understand the protocol, yes. > Nikos Nikos Nikos... and what about me? If I asked you for a few > pointers about a good place to learn about running a web site, would you > tell me to fuck off too? I wonder what you are doing here, if you are so > unwilling to share your hard-earned knowledge with others as you seem in > this post. This attitude is not the Denis McMahon I'm used to. I'm not unwilling, but this forum is not the place for tcp/ip 101, or http 101, or smtp 101, or dns 101, or geolocation 101, despite Nkos' attempts to turn it into one. This forum is a place for python coding. > Nikos, are you reading this? This is what happens when you behave like a > royal pain in the arse and annoy people. They stop wanting to help you. > Be told. Learn from this. Don't repeat this mistake in the next forum. > If you learn nothing else, learn that lesson. Yes, this is exactly the issue. I am so pissed off with trying to help Nikos and being told things like "Your solution is crap because you use too many lines, even though it works, unlike my [Nikos's] single line effort which I [Nikos] think looks aesthetically wonderful and which must therefore be the superior solution even though it doesn't work". I went so far as to set up mod-wsgi on my server simply to try and understand his cookie issues - before that I hadn't used python on my apache server except at the basic cgi level. I managed to get a functional cookie implementation up and running within a few hours, primarily by reading the relevant api documentation, and looking up a few examples on slashdot and similar. Given that Nikos presents as being a professional coder developing client facing facing python code for a hosting provider, his inability to do something similar, indeed his inability to locate relevant sources of information, is frankly quite astounding, and I am in agreement with the many others on this forum who regularly and frequently voice the opinion that Nikos, specifically, has no business coding anything on a web server. -- Denis McMahon, denismfmcmahon at gmail.com From ben+python at benfinney.id.au Wed Oct 9 20:43:50 2013 From: ben+python at benfinney.id.au (Ben Finney) Date: Thu, 10 Oct 2013 11:43:50 +1100 Subject: Cookie gets changed when hit comes from a referrer References: <5255eb3c$0$29984$c3e8da3$5496439d@news.astraweb.com> Message-ID: <7w1u3uoszd.fsf@benfinney.id.au> Steven D'Aprano writes: > So, for the benefit of anyone, not just Nikos, who wants to learn > about how browsers connect to web sites and how to run a web server, > does anyone have any recommendation for tutorials, mailing lists, web > forums or books which are suitable? Preferably things you have used > yourself, and can personally vouch for being useful. I learned a lot from reading the documents at the W3C, since they defined the standards we all use . They are very well written and explain a lot about the architecture of the Web. > I'm pretty sure that *somebody* here has been in the position of > needing to learn about running a website, and can point Nikos in the > right direction (away from here). How did you learn? Experiment, observation, and reference to standards. > And if nobody is able, or willing, to answer? I've been in that > position too, asking for help that nobody was able to give. It sucks, > and you move on and do your best. Right. What you don't do is disrupt a discussion forum by bombarding it with off-topic requests; that's a quick way to get a community turn against you. -- \ ?I have yet to see any problem, however complicated, which, | `\ when you looked at it in the right way, did not become still | _o__) more complicated.? ?Paul Anderson | Ben Finney From roy at panix.com Wed Oct 9 20:58:54 2013 From: roy at panix.com (Roy Smith) Date: Wed, 09 Oct 2013 20:58:54 -0400 Subject: Cookie gets changed when hit comes from a referrer References: <5255eb3c$0$29984$c3e8da3$5496439d@news.astraweb.com> Message-ID: In article <5255eb3c$0$29984$c3e8da3$5496439d at news.astraweb.com>, Steven D'Aprano wrote: > So, for the benefit of anyone, not just Nikos, who wants to learn about > how browsers connect to web sites and how to run a web server, does > anyone have any recommendation for tutorials, mailing lists, web forums > or books which are suitable? Preferably things you have used yourself, > and can personally vouch for being useful. In the way of tools, tcpdump (or wireshark, or whatever packet sniffer you have handy) is invaluable for seeing what's really going on. Also, curl is an amazing tool for sending arbitrary requests to a HTTP server. It's got a zillion options. You'll want to learn what most of them do. The combination of curl and tcpdump is a pretty potent weapon for poking at servers. An interesting alternative to tcpdump/curl is Google Chrome. Open up the developer tools (Command-Option-I on the Mac), click the network tab, and type a request into the address bar. It will capture the outgoing request and the response you get back, and let you dig into them in as much detail as you want. Very handy, at least for GET requests. I imagine any decent browser has similar functionality. For general background, I would start with https://en.wikipedia.org/wiki/Http and keep following links from there until you have read the whole wiki or gotten tired, whichever comes first. stackoverflow has become a really good forum for asking all sorts of programming-related questions (although, its sister site, serverfault, is more appropriate for networking questions). I find the U/I confusing, and often find the rules (i.e. what's allowed and what's not) kind of silly, but a lot of really smart people hang out there. If you want a question answered, go where the smart people are. > I'm pretty sure that *somebody* here has been in the position of needing > to learn about running a website, and can point Nikos in the right > direction (away from here). I guarantee the folks on either stackoverflow or serverfault won't put up with the kind of bullcrap that has been pervading this group as of late. If I was using a program, foo, and had a problem, the first thing I would do is google for "foo mailing list" and see if I can find one. You don't always find one, and sometimes the list is dead, but it's a good place to start. > How did you learn? Run. Fall down. Get up. Start running again. Repeat as needed. From mail at timgolden.me.uk Thu Oct 10 03:44:16 2013 From: mail at timgolden.me.uk (Tim Golden) Date: Thu, 10 Oct 2013 08:44:16 +0100 Subject: Learning about HTTP [was: Cookie gets changed when hit comes from a referrer] In-Reply-To: <5255eb3c$0$29984$c3e8da3$5496439d@news.astraweb.com> References: <5255eb3c$0$29984$c3e8da3$5496439d@news.astraweb.com> Message-ID: <52565AD0.9040603@timgolden.me.uk> On 10/10/2013 00:48, Steven D'Aprano wrote: > So, for the benefit of anyone, not just Nikos, who wants to learn about > how browsers connect to web sites and how to run a web server, does > anyone have any recommendation for tutorials, mailing lists, web forums > or books which are suitable? Preferably things you have used yourself, > and can personally vouch for being useful. Not that it's a free resource, but I found this O'Reilly pocket guide useful as a starter. It's not too expensive either. http://shop.oreilly.com/product/9781565928626.do I bought it for my Dad who's a retired civil engineer now happily engaged in setting up small websites for Parish groups and the like. Exactly the kind of thing where he can experiment with a bit of wackiness without people minding too much if things go wrong... (And he does have a test rig at home). As clearly evidenced by the wide range of reactions one sees in answer to "What's the best book...?" questions here, different people have wildly different learning styles. For some, going to the RFCs is *exactly* what they find interesting & illuminating. For others, a step-by-step with screenshots is more the thing, etc. TJG From denismfmcmahon at gmail.com Wed Oct 9 14:06:36 2013 From: denismfmcmahon at gmail.com (Denis McMahon) Date: Wed, 9 Oct 2013 18:06:36 +0000 (UTC) Subject: Cookie gets changed when hit comes from a referrer References: Message-ID: On Wed, 09 Oct 2013 18:00:28 +0300, ????? ??????????? wrote: > ???? 9/10/2013 5:43 ??, ?/? Denis McMahon ??????: >> On Wed, 09 Oct 2013 01:52:44 +0300, ????? ??????????? wrote: >> >>> ???? 8/10/2013 10:29 ??, ?/? Denis McMahon ??????: >> >>>> Have you checked the cookie jar in the browser to see what value the >>>> cookie has? Is that the value you think it should have? Note that >>>> checking the cookie jar is a browser topic, not a python topic, so if >>>> you don't know how to do that you're going to have to find the right >>>> place to ask, WHICH IS NOT HERE! >>>> Ideally you need to check what the server thinks it's setting the >>>> cooking to, what the browser thinks it received as the cookie, and >>>> what the server gets back afterwards to work out where the error is >>>> happening. >>> Is there something i can try to isolate the problem and make it work? >>> By whole counters project is based on cookie handling now.... >> See those last two paragraphs there that you quoted. You should have >> read them. > ok so then tell me where i should ask this. In those two paragraphs I have told you what you need to do to isolate where your problem is occurring. If you don't know how to do that, then I commend to you the wealth of information that may be discovered using search engines. However, as we keep telling you, this group is not the right place to ask questions such as: a) How do I see what's in my web browser's cookie jar? b) How do I make my web server log the cookies it sends and receives? For (a) you want information about your browser, not about python. For (b) you want information about your web server, not about python. Find the relevant forums and ask in them. -- Denis McMahon, denismfmcmahon at gmail.com From nikos.gr33k at gmail.com Tue Oct 8 13:30:48 2013 From: nikos.gr33k at gmail.com (=?UTF-8?B?zp3Or866zr/PgiDOkc67zrXOvs+Mz4DOv8+FzrvOv8+C?=) Date: Tue, 08 Oct 2013 20:30:48 +0300 Subject: Cookie gets changed when hit comes from a referrer In-Reply-To: References: Message-ID: Is there any better way to identif a previous visitor? i tried cookies which failed for me for the reason i opened this thread and host like follows: # try to locate the visitor cur.execute('''SELECT * FROM visitors WHERE counterID = %s and host = %s''', (cID, host) ) data = cur.fetchone() if not data: # if first time visitor on this page, create new record cur.execute('''INSERT INTO visitors (counterID, host, city, useros, browser, ref, lastvisit) VALUES (%s, %s, %s, %s, %s, %s, %s)''', (cID, host, city, useros, browser, ref, lastvisit) ) else: # since visitor exists just update his record cur.execute('''UPDATE visitors SET city = %s, useros = %s, browser = %s, ref = %s, hits = hits + 1, lastvisit = %s''', (city, useros, browser, ref, lastvisit) ) ======= Please tell me if you can think fo something else. From joel.goldstick at gmail.com Tue Oct 8 13:47:14 2013 From: joel.goldstick at gmail.com (Joel Goldstick) Date: Tue, 8 Oct 2013 13:47:14 -0400 Subject: Cookie gets changed when hit comes from a referrer In-Reply-To: References: Message-ID: On Tue, Oct 8, 2013 at 1:30 PM, ????? ??????????? wrote: > Is there any better way to identif a previous visitor? i tried cookies which > failed for me for the reason i opened this thread and host like follows: > > # try to locate the visitor > cur.execute('''SELECT * FROM visitors WHERE counterID = %s > and host = %s''', (cID, host) ) > data = cur.fetchone() > > if not data: > # if first time visitor on this page, create new > record > cur.execute('''INSERT INTO visitors (counterID, > host, city, useros, browser, ref, lastvisit) VALUES (%s, %s, %s, %s, %s, %s, > %s)''', > (cID, host, city, useros, > browser, ref, lastvisit) ) > else: > # since visitor exists just update his record > cur.execute('''UPDATE visitors SET city = %s, useros > = %s, browser = %s, ref = %s, hits = hits + 1, lastvisit = %s''', (city, > useros, browser, ref, lastvisit) ) > ======= > > Please tell me if you can think fo something else. Yes! there is a very simple and comprehensive way to learn about your visitors. Use Google Analytics. Its free, you only need a google account to open an analytics account. They give you a small bit of javascript that you copy and past to your pages. If you are using a template to create your pages, this is easy, since you just add google code to the template. > > -- > https://mail.python.org/mailman/listinfo/python-list -- Joel Goldstick http://joelgoldstick.com From steve+comp.lang.python at pearwood.info Tue Oct 8 21:24:10 2013 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: 09 Oct 2013 01:24:10 GMT Subject: Cookie gets changed when hit comes from a referrer References: Message-ID: <5254b039$0$29984$c3e8da3$5496439d@news.astraweb.com> On Tue, 08 Oct 2013 13:47:14 -0400, Joel Goldstick wrote: > Yes! there is a very simple and comprehensive way to learn about your > visitors. Use Google Analytics. Its free, you only need a google > account to open an analytics account. http://images.wikia.com/darkheresy/images/0/0c/Its_a_trap.jpg There is a very simple and comprehensive way for Google to learn about your visitors: Google Analytics. -- Steven From marco.buttu at gmail.com Tue Oct 8 06:13:48 2013 From: marco.buttu at gmail.com (Marco Buttu) Date: Tue, 08 Oct 2013 12:13:48 +0200 Subject: class-private names and the Zen of Python Message-ID: In the following case: >>> class Foo: ... _Foo__a = 100 ... __a = 33 ... >>> Foo._Foo__a 33 I think this behavior, for a user who does not know the convention, could be a surprise. Should be raising an exception (in order to inform the user the transformation of the name __a have been replaced an existing name) a possible --explicit-- alternative? Another question is: where is the place in which this transformation occurs? Is it at the parser level, before the dictionary attribute is gave as argument to the metaclass? I looked at the documentation: http://docs.python.org/3/reference/lexical_analysis.html http://docs.python.org/3/reference/expressions.html#atom-identifiers but it is not clear when this transformation happens. -- Marco Buttu From tjreedy at udel.edu Tue Oct 8 06:36:37 2013 From: tjreedy at udel.edu (Terry Reedy) Date: Tue, 08 Oct 2013 06:36:37 -0400 Subject: class-private names and the Zen of Python In-Reply-To: References: Message-ID: On 10/8/2013 6:13 AM, Marco Buttu wrote: > In the following case: > > >>> class Foo: > ... _Foo__a = 100 > ... __a = 33 > ... > >>> Foo._Foo__a > 33 > > I think this behavior, for a user who does not know the convention, > could be a surprise. No one qualified to use such names would do such a thing , so there is no need to worry about it or do anything about it. -- Terry Jan Reedy From marco.buttu at gmail.com Tue Oct 8 06:47:17 2013 From: marco.buttu at gmail.com (Marco Buttu) Date: Tue, 08 Oct 2013 12:47:17 +0200 Subject: class-private names and the Zen of Python References: Message-ID: <5253E2B5.4060905@gmail.com> On 10/08/2013 12:36 PM, Terry Reedy wrote: > On 10/8/2013 6:13 AM, Marco Buttu wrote: >> In the following case: >> >> >>> class Foo: >> ... _Foo__a = 100 >> ... __a = 33 >> ... >> >>> Foo._Foo__a >> 33 >> >> I think this behavior, for a user who does not know the convention, >> could be a surprise. > > No one qualified to use such names would do such a thing , so there is > no need to worry about it or do anything about it. Is this transformation performed by the parser, before to call the metaclass? -- Marco Buttu From ned at nedbatchelder.com Tue Oct 8 07:07:53 2013 From: ned at nedbatchelder.com (Ned Batchelder) Date: Tue, 08 Oct 2013 07:07:53 -0400 Subject: class-private names and the Zen of Python In-Reply-To: References: Message-ID: <5253E789.6040800@nedbatchelder.com> On 10/8/13 6:13 AM, Marco Buttu wrote: > In the following case: > > >>> class Foo: > ... _Foo__a = 100 > ... __a = 33 > ... > >>> Foo._Foo__a > 33 > > I think this behavior, for a user who does not know the convention, > could be a surprise. Should be raising an exception (in order to > inform the user the transformation of the name __a have been replaced > an existing name) a possible --explicit-- alternative? You also get a "problem" if you do this: >>> class Foo: ... a = 100 ... a = 33 ... >>> Foo.a 33 Or for that matter: >>> a = 100 >>> a = 33 >>> a 33 There are lots of ways to change what value a name refers to, it's not an error to reassign names. Also, as Terry mentions, no one has ever assigned the two names you show, so why try to warn about it? --Ned. From marco.buttu at gmail.com Tue Oct 8 07:15:17 2013 From: marco.buttu at gmail.com (Marco Buttu) Date: Tue, 08 Oct 2013 13:15:17 +0200 Subject: class-private names and the Zen of Python References: Message-ID: <5253E945.5030608@gmail.com> On 10/08/2013 01:07 PM, Ned Batchelder wrote: > On 10/8/13 6:13 AM, Marco Buttu wrote: >> >> >>> class Foo: >> ... _Foo__a = 100 >> ... __a = 33 >> ... >> >>> Foo._Foo__a >> 33 >> ... > You also get a "problem" if you do this: > > >>> class Foo: > ... a = 100 > ... a = 33 > ... > >>> Foo.a > 33 But this does not happen under the hood, it is explicit > Also, as Terry mentions, no one has ever assigned the two names you > show, Sincerely, I can not now if someone has assigned (or will assegne) in such way... -- Marco Buttu From ned at nedbatchelder.com Tue Oct 8 12:20:40 2013 From: ned at nedbatchelder.com (Ned Batchelder) Date: Tue, 08 Oct 2013 12:20:40 -0400 Subject: class-private names and the Zen of Python In-Reply-To: <5253E945.5030608@gmail.com> References: <5253E945.5030608@gmail.com> Message-ID: <525430D8.5090500@nedbatchelder.com> On 10/8/13 7:15 AM, Marco Buttu wrote: > >> Also, as Terry mentions, no one has ever assigned the two names you >> show, > > Sincerely, I can not now if someone has assigned (or will assegne) in > such way... If you explain more about what you are building, and where this crops up as a problem, we can help come up with a solution. --Ned. From steve+comp.lang.python at pearwood.info Tue Oct 8 09:24:10 2013 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: 08 Oct 2013 13:24:10 GMT Subject: class-private names and the Zen of Python References: Message-ID: <52540779$0$29984$c3e8da3$5496439d@news.astraweb.com> On Tue, 08 Oct 2013 12:13:48 +0200, Marco Buttu wrote: > In the following case: > > >>> class Foo: > ... _Foo__a = 100 > ... __a = 33 > ... > >>> Foo._Foo__a > 33 > > I think this behavior, for a user who does not know the convention, > could be a surprise. Yes, you are correct. It surprised me, and I've been using Python for more than 15 years, and I know the convention of double-underscore name- mangling. > Should be raising an exception (in order to inform > the user the transformation of the name __a have been replaced an > existing name) a possible --explicit-- alternative? No, I don't think so. That would slow down class creation, for no real benefit. Except for the name-mangling part, this is no different from: class Spam: x = 23 x = 42 If anything, something like PyLint or PyChecker could warn about it. But the language itself is fine like it is. > Another question is: where is the place in which this transformation > occurs? Is it at the parser level, before the dictionary attribute is > gave as argument to the metaclass? Good question! I don't have a full answer, but I have a part answer: it occurs before the metaclass sees the namespace: py> class Meta(type): ... def __new__(meta, name, bases, namespace): ... print(namespace) ... return super().__new__(meta, name, bases, namespace) ... py> py> class Test(metaclass=Meta): ... __test = 'foo' ... {'__module__': '__main__', '_Test__test': 'foo', '__qualname__': 'Test'} so I think it is done by the parser. -- Steven From oscar.j.benjamin at gmail.com Tue Oct 8 19:00:14 2013 From: oscar.j.benjamin at gmail.com (Oscar Benjamin) Date: Wed, 9 Oct 2013 00:00:14 +0100 Subject: class-private names and the Zen of Python In-Reply-To: <52540779$0$29984$c3e8da3$5496439d@news.astraweb.com> References: <52540779$0$29984$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Oct 8, 2013 2:26 PM, "Steven D'Aprano" < steve+comp.lang.python at pearwood.info> wrote: > > On Tue, 08 Oct 2013 12:13:48 +0200, Marco Buttu wrote: > > > Another question is: where is the place in which this transformation > > occurs? Is it at the parser level, before the dictionary attribute is > > gave as argument to the metaclass? > > Good question! > > I don't have a full answer, but I have a part answer: it occurs before > the metaclass sees the namespace: > I thought it was at the parser level and applied to assignments at class level and attribute assignments anywhere within a class body. I'm pretty sure there's no way to control the behaviour from Python code if that's what the metaclass question is getting at. Oscar -------------- next part -------------- An HTML attachment was scrubbed... URL: From marco.buttu at gmail.com Wed Oct 9 01:55:45 2013 From: marco.buttu at gmail.com (Marco Buttu) Date: Wed, 09 Oct 2013 07:55:45 +0200 Subject: class-private names and the Zen of Python References: <52540779$0$29984$c3e8da3$5496439d@news.astraweb.com> Message-ID: On 10/09/2013 01:00 AM, Oscar Benjamin wrote: > > On Tue, 08 Oct 2013 12:13:48 +0200, Marco Buttu wrote: > > > > > Another question is: where is the place in which this transformation > > > occurs? Is it at the parser level, before the dictionary attribute is > > > gave as argument to the metaclass? > > I thought it was at the parser level and applied to assignments at class > level and attribute assignments anywhere within a class body. > > I'm pretty sure there's no way to control the behaviour from Python code > if that's what the metaclass question is getting at. > > Oscar Thanks for the clear and exhaustive answers (from you and Steven). I was looking for a way to change the default behavior of this transformation (just for playing a bit with metaclasses, nothing serious), with snippets of code like the one of Steven (by rewriting __new__), but now is clear there is no way to do this with metaclasses. Thanks again -- Marco Buttu From charleshixsn at earthlink.net Wed Oct 9 15:34:46 2013 From: charleshixsn at earthlink.net (Charles Hixson) Date: Wed, 09 Oct 2013 12:34:46 -0700 Subject: class-private names and the Zen of Python In-Reply-To: <52540779$0$29984$c3e8da3$5496439d@news.astraweb.com> References: <52540779$0$29984$c3e8da3$5496439d@news.astraweb.com> Message-ID: <5255AFD6.3010004@earthlink.net> On 10/08/2013 06:24 AM, Steven D'Aprano wrote: > On Tue, 08 Oct 2013 12:13:48 +0200, Marco Buttu wrote: > >> In the following case: >> >> >>> class Foo: >> ... _Foo__a = 100 >> ... __a = 33 >> ... >> >>> Foo._Foo__a >> 33 >> >> I think this behavior, for a user who does not know the convention, >> could be a surprise. > Yes, you are correct. It surprised me, and I've been using Python for > more than 15 years, and I know the convention of double-underscore name- > mangling. > > >> Should be raising an exception (in order to inform >> the user the transformation of the name __a have been replaced an >> existing name) a possible --explicit-- alternative? > No, I don't think so. That would slow down class creation, for no real > benefit. Except for the name-mangling part, this is no different from: > > class Spam: > x = 23 > x = 42 > > If anything, something like PyLint or PyChecker could warn about it. But > the language itself is fine like it is. > > >> Another question is: where is the place in which this transformation >> occurs? Is it at the parser level, before the dictionary attribute is >> gave as argument to the metaclass? > Good question! > > I don't have a full answer, but I have a part answer: it occurs before > the metaclass sees the namespace: > > py> class Meta(type): > ... def __new__(meta, name, bases, namespace): > ... print(namespace) > ... return super().__new__(meta, name, bases, namespace) > ... > py> > py> class Test(metaclass=Meta): > ... __test = 'foo' > ... > {'__module__': '__main__', '_Test__test': 'foo', '__qualname__': 'Test'} > > > so I think it is done by the parser. Unless one wanted to add an error checking mode to the interpreter, not a totally bad idea, I agree that this is a job for external tools. But perhaps that same external tool should be referenced in the documentation. As it is I'm not clear that PyLint and/or PyChecker are kept current with the compiler/interpreter version, and I rather suspect that they aren't. (This won't normally show up, as changes that they would catch happen quite rarely, but...) OTOH, neither one really fits in as, say, an included module...they're more like idle, which now that I look does have a "check module" run option. Perhaps that invokes one of them. I note that Idle, itself, is barely mentioned in the documentation. Perhaps there needs to be a "tools" section. -- Charles Hixson From antoon.pardon at rece.vub.ac.be Tue Oct 8 08:20:20 2013 From: antoon.pardon at rece.vub.ac.be (Antoon Pardon) Date: Tue, 08 Oct 2013 14:20:20 +0200 Subject: parsing email from stdin Message-ID: <5253F884.7020306@rece.vub.ac.be> I want to do some postprocessing on messages from a particular mailbox. So I use getmail which will fetch the messages and feed them to stdin of my program. As I don't know what encoding these messages will be in, I thought it would be prudent to read stdin as binary data. Using python 3.3 on a debian box I have the following code. #!/usr/bin/python3 import sys from email import message_from_file sys.stdin = sys.stdin.detach() msg = message_from_file(sys.stdin) which gives me the following trace back File "/home/apardon/.getmail/verdeler", line 7, in msg = message_from_file(sys.stdin) File "/usr/lib/python3.3/email/__init__.py", line 56, in message_from_file return Parser(*args, **kws).parse(fp) File "/usr/lib/python3.3/email/parser.py", line 58, in parse feedparser.feed(data) File "/usr/lib/python3.3/email/feedparser.py", line 167, in feed self._input.push(data) File "/usr/lib/python3.3/email/feedparser.py", line 100, in push data, self._partial = self._partial + data, '' TypeError: Can't convert 'bytes' object to str implicitly)) which seems to be rather odd. The following header are in the msg: Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 8bit So why doesn't the email parser lookup the charset and use that for converting to string type? What is the canonical way to parse an email message from stdin? -- Antoon Pardon From andipersti at gmail.com Tue Oct 8 10:24:52 2013 From: andipersti at gmail.com (Andreas Perstinger) Date: Tue, 08 Oct 2013 16:24:52 +0200 Subject: parsing email from stdin In-Reply-To: <5253F884.7020306@rece.vub.ac.be> References: <5253F884.7020306@rece.vub.ac.be> Message-ID: <525415B4.1030208@gmail.com> On 08.10.2013 14:20, Antoon Pardon wrote: > As I don't know what encoding these messages will be in, I thought it > would be prudent to read stdin as binary data. > > Using python 3.3 on a debian box I have the following code. > > #!/usr/bin/python3 > > import sys > from email import message_from_file > > sys.stdin = sys.stdin.detach() > msg = message_from_file(sys.stdin) Looking at the docs, I've found there is also "message_from_binary_file" which works for me with your code. http://docs.python.org/3/library/email.parser.html#email.message_from_binary_file Bye, Andreas From antoon.pardon at rece.vub.ac.be Tue Oct 8 11:25:40 2013 From: antoon.pardon at rece.vub.ac.be (Antoon Pardon) Date: Tue, 08 Oct 2013 17:25:40 +0200 Subject: parsing email from stdin In-Reply-To: <525415B4.1030208@gmail.com> References: <5253F884.7020306@rece.vub.ac.be> <525415B4.1030208@gmail.com> Message-ID: <525423F4.7000902@rece.vub.ac.be> Op 08-10-13 16:24, Andreas Perstinger schreef: > On 08.10.2013 14:20, Antoon Pardon wrote: >> As I don't know what encoding these messages will be in, I thought it >> would be prudent to read stdin as binary data. >> >> Using python 3.3 on a debian box I have the following code. >> >> #!/usr/bin/python3 >> >> import sys >> from email import message_from_file >> >> sys.stdin = sys.stdin.detach() >> msg = message_from_file(sys.stdin) > > Looking at the docs, I've found there is also "message_from_binary_file" > which works for me with your code. > > http://docs.python.org/3/library/email.parser.html#email.message_from_binary_file > I can't try that out right now, but I had a look at the code and the ByteParser that is mentioned their looks like this: class BytesFeedParser(FeedParser): """Like FeedParser, but feed accepts bytes.""" def feed(self, data): super().feed(data.decode('ascii', 'surrogateescape')) Somehow I doubt that trying to decode my utf-8 stream as if it was ascii will work. -- Antoon Pardon From andipersti at gmail.com Tue Oct 8 13:11:59 2013 From: andipersti at gmail.com (Andreas Perstinger) Date: Tue, 08 Oct 2013 19:11:59 +0200 Subject: parsing email from stdin In-Reply-To: <525423F4.7000902@rece.vub.ac.be> References: <5253F884.7020306@rece.vub.ac.be> <525415B4.1030208@gmail.com> <525423F4.7000902@rece.vub.ac.be> Message-ID: <52543CDF.5050505@gmail.com> On 08.10.2013 17:25, Antoon Pardon wrote: > Op 08-10-13 16:24, Andreas Perstinger schreef: >> Looking at the docs, I've found there is also "message_from_binary_file" >> which works for me with your code. >> >> http://docs.python.org/3/library/email.parser.html#email.message_from_binary_file >> > > I can't try that out right now, but I had a look at the code and the > ByteParser that is mentioned their looks like this: > > class BytesFeedParser(FeedParser): > """Like FeedParser, but feed accepts bytes.""" > > def feed(self, data): > super().feed(data.decode('ascii', 'surrogateescape')) > > > Somehow I doubt that trying to decode my utf-8 stream as if it was > ascii will work. Actually it does work: $ cat testmail.txt From: "someone" To: "me" Subject: something Content-Type: text/plain; charset="UTF-8"; Content-Transfer-Encoding: 8bit foo bar A????? baz $ file testmail.txt testmail.txt: news or mail, UTF-8 Unicode text $ cat foo.py #!/usr/bin/python3 import sys from email import message_from_binary_file sys.stdin = sys.stdin.detach() msg = message_from_binary_file(sys.stdin) print("from: ", msg['From']) print("to: ", msg['To']) print("subject: ", msg['Subject']) print("body: ", msg.get_payload()) $ ./foo.py < testmail.txt from: "someone" to: "me" subject: something body: foo bar A????? baz Bye, Andreas From steve+comp.lang.python at pearwood.info Tue Oct 8 09:52:03 2013 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: 08 Oct 2013 13:52:03 GMT Subject: Encoding of surrogate code points to UTF-8 Message-ID: <52540e03$0$29984$c3e8da3$5496439d@news.astraweb.com> I think this is a bug in Python's UTF-8 handling, but I'm not sure. If I've read the Unicode FAQs correctly, you cannot encode *lone* surrogate code points into UTF-8: http://www.unicode.org/faq/utf_bom.html#utf8-5 Sure enough, using Python 3.3: py> surr = '\udc80' py> surr.encode('utf-8') Traceback (most recent call last): File "", line 1, in UnicodeEncodeError: 'utf-8' codec can't encode character '\udc80' in position 0: surrogates not allowed But reading the previous entry in the FAQs: http://www.unicode.org/faq/utf_bom.html#utf8-4 I interpret this as meaning that I should be able to encode valid pairs of surrogates. So if I find a code point that encodes to a surrogate pair in UTF-16: py> c = '\N{LINEAR B SYLLABLE B038 E}' py> surr_pair = c.encode('utf-16be') py> print(surr_pair) b'\xd8\x00\xdc\x01' and then use those same values as the code points, I ought to be able to encode to UTF-8, as if it were the same \N{LINEAR B SYLLABLE B038 E} code point. But I can't: py> s = '\ud800\udc01' py> s.encode('utf-8') Traceback (most recent call last): File "", line 1, in UnicodeEncodeError: 'utf-8' codec can't encode character '\ud800' in position 0: surrogates not allowed Have I misunderstood? I think that Python is being too strict about rejecting surrogate code points. It should only reject lone surrogates, or invalid pairs, not valid pairs. Have I misunderstood the Unicode FAQs, or is this a bug in Python's handling of UTF-8? -- Steven From neilc at norwich.edu Tue Oct 8 11:14:33 2013 From: neilc at norwich.edu (Neil Cerutti) Date: 8 Oct 2013 15:14:33 GMT Subject: Encoding of surrogate code points to UTF-8 References: <52540e03$0$29984$c3e8da3$5496439d@news.astraweb.com> Message-ID: On 2013-10-08, Steven D'Aprano wrote: > py> c = '\N{LINEAR B SYLLABLE B038 E}' > py> surr_pair = c.encode('utf-16be') > py> print(surr_pair) > b'\xd8\x00\xdc\x01' > > and then use those same values as the code points, I ought to be able to > encode to UTF-8, as if it were the same \N{LINEAR B SYLLABLE B038 E} code > point. But I can't: > > py> s = '\ud800\udc01' > py> s.encode('utf-8') > Traceback (most recent call last): > File "", line 1, in > UnicodeEncodeError: 'utf-8' codec can't encode character '\ud800' in > position 0: surrogates not allowed > > Have I misunderstood? I think that Python is being too strict > about rejecting surrogate code points. It should only reject > lone surrogates, or invalid pairs, not valid pairs. Have I > misunderstood the Unicode FAQs, or is this a bug in Python's > handling of UTF-8? >From RFC 3629: The definition of UTF-8 prohibits encoding character numbers between U+D800 and U+DFFF, which are reserved for use with the UTF-16 encoding form (as surrogate pairs) and do not directly represent characters. When encoding in UTF-8 from UTF-16 data, it is necessary to first decode the UTF-16 data to obtain character numbers, which are then encoded in UTF-8 as described above. This contrasts with CESU-8 [CESU-8], which is a UTF-8-like encoding that is not meant for use on the Internet. CESU-8 operates similarly to UTF-8 but encodes the UTF-16 code values (16-bit quantities) instead of the character number (code point). This leads to different results for character numbers above 0xFFFF; the CESU-8 encoding of those characters is NOT valid UTF-8. The Wikipedia article points out: Whether an actual application should [refuse to encode these character numbers] is debatable, as it makes it impossible to store invalid UTF-16 (that is, UTF-16 with unpaired surrogate halves) in a UTF-8 string. This is necessary to store unchecked UTF-16 such as Windows filenames as UTF-8. It is also incompatible with CESU encoding (described below). So Python's interpretation is conformant, though not without some disadvantages. In any case, "\ud800\udc01" isn't a valid unicode string. In a perfect world it would automatically get converted to '\u00010001' without intervention. -- Neil Cerutti From neilc at norwich.edu Tue Oct 8 11:54:30 2013 From: neilc at norwich.edu (Neil Cerutti) Date: 8 Oct 2013 15:54:30 GMT Subject: Encoding of surrogate code points to UTF-8 References: <52540e03$0$29984$c3e8da3$5496439d@news.astraweb.com> Message-ID: On 2013-10-08, Neil Cerutti wrote: > In any case, "\ud800\udc01" isn't a valid unicode string. In a > perfect world it would automatically get converted to > '\u00010001' without intervention. This last paragraph is erroneous. I must have had a typo in my testing. -- Neil Cerutti From steve+comp.lang.python at pearwood.info Tue Oct 8 18:30:41 2013 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: 08 Oct 2013 22:30:41 GMT Subject: Encoding of surrogate code points to UTF-8 References: <52540e03$0$29984$c3e8da3$5496439d@news.astraweb.com> Message-ID: <52548791$0$29984$c3e8da3$5496439d@news.astraweb.com> On Tue, 08 Oct 2013 15:14:33 +0000, Neil Cerutti wrote: > In any case, "\ud800\udc01" isn't a valid unicode string. I don't think this is correct. Can you show me where the standard says that Unicode strings[1] may not contain surrogates? I think that is a critical point, and the FAQ conflates *encoded strings* (i.e. bytes using one of the UTCs) with *Unicode strings*. The string you give above is is a Unicode string containing two code points, the surrogates U+D800 U+DC01, which as far as I am concerned is a legal string (subject to somebody pointing me to a definitive source that proves it is not). However, it *may or may not* be encodable to bytes using UTF-8, -16 or -32. Just as there are byte sequences that cannot be generated by the UTFs, possibly there are code point sequences that cannot be converted to bytes using the UTFs. > In a perfect > world it would automatically get converted to '\u00010001' without > intervention. I certainly hope not, because Unicode string != UTF-16. This is equivalent to saying: When encoding the sequence of code points '\ud800\udc01' to UTF-8 bytes, you should get the same result as if you treated the sequence of code points as if it were bytes, decoded it using UTF-16, and then encoded using UTF-8. That would be a horrible, horrible design, since it privileges UTF-16 in a completely inappropriate way. I *really* hope I am wrong, but I fear that is my interpretation of the FAQ. [1] Sequences of Unicode code points. -- Steven From tjreedy at udel.edu Tue Oct 8 21:28:25 2013 From: tjreedy at udel.edu (Terry Reedy) Date: Tue, 08 Oct 2013 21:28:25 -0400 Subject: Encoding of surrogate code points to UTF-8 In-Reply-To: <52548791$0$29984$c3e8da3$5496439d@news.astraweb.com> References: <52540e03$0$29984$c3e8da3$5496439d@news.astraweb.com> <52548791$0$29984$c3e8da3$5496439d@news.astraweb.com> Message-ID: On 10/8/2013 6:30 PM, Steven D'Aprano wrote: > On Tue, 08 Oct 2013 15:14:33 +0000, Neil Cerutti wrote: > >> In any case, "\ud800\udc01" isn't a valid unicode string. > > I don't think this is correct. Can you show me where the standard says > that Unicode strings[1] may not contain surrogates? I think that is a see below. > critical point, and the FAQ conflates *encoded strings* (i.e. bytes using > one of the UTCs) with *Unicode strings*. > > The string you give above is is a Unicode string containing two code > points, the surrogates U+D800 U+DC01, which as far as I am concerned is a > legal string (subject to somebody pointing me to a definitive source that > proves it is not). However, it *may or may not* be encodable to bytes > using UTF-8, -16 or -32. From chapter two of the standard. "Plain text is a pure sequence of character codes; plain Unicode-encoded text is therefore a sequence of Unicode character codes." http://www.unicode.org/versions/Unicode6.2.0/ch02.pdf#G13708 "All three encoding forms can be used to represent the full range of encoded characters in the Unicode Standard; ... Each of the three Unicode encoding forms can be efficiently transformed into eith er of the other two without any loss of data." "Surrogates Area. The Surrogates Area contains only surrogate code points and no encoded characters. See Section 16.6, Surrogates Area, for more detail." Before utf-16, the surrogates area was, I believe, part of the Private Use Area (which now starts where surrogates end). I think it would have been better if they were no longer called code points, but simply utf-16 code units. > Just as there are byte sequences that cannot be generated by the UTFs, > possibly there are code point sequences that cannot be converted to bytes > using the UTFs. True, but not to the point. You switched from sequences of characters (unicode text), which is what both I and Neil are talking about, to sequences of codepoints which is a larger set when you include the non-character surrogate 'code points' that are not allowed in unicode text. http://www.unicode.org/versions/Unicode6.2.0/ch03.pdf#G7404 "The Unicode Standard supports three character encoding forms: UTF-32, UTF-16, and UTF-8. Each encoding form maps the Unicode code points U+0000..U+D7FF and U+E000..U+10FFFF to unique code unit sequences." > [1] Sequences of Unicode code points. This is not the Standard's definition of 'unicode text'. It is also not its definition of 'unicode string'. "D80 Unicode string: A code unit sequence containing code units of a particular Unicode encoding form." In other words, a Unicode string is a utf encoding of unicode text. The FSR adaptively uses a subset of possible sequences from all three, though only one utf is used for any particular string. -- D79 says what I claimed before: "The mapping of the set of Unicode scalar values to the set of code unit sequences for a Unicode encoding form is one-to-one." -- Terry Jan Reedy From steve at pearwood.info Wed Oct 9 02:20:05 2013 From: steve at pearwood.info (Steven D'Aprano) Date: 09 Oct 2013 06:20:05 GMT Subject: Encoding of surrogate code points to UTF-8 References: <52540e03$0$29984$c3e8da3$5496439d@news.astraweb.com> <52548791$0$29984$c3e8da3$5496439d@news.astraweb.com> Message-ID: <5254f594$0$29976$c3e8da3$5496439d@news.astraweb.com> On Tue, 08 Oct 2013 21:28:25 -0400, Terry Reedy wrote: > On 10/8/2013 6:30 PM, Steven D'Aprano wrote: >> On Tue, 08 Oct 2013 15:14:33 +0000, Neil Cerutti wrote: >> >>> In any case, "\ud800\udc01" isn't a valid unicode string. >> >> I don't think this is correct. Can you show me where the standard says >> that Unicode strings[1] may not contain surrogates? I think that is a > > see below. > >> critical point, and the FAQ conflates *encoded strings* (i.e. bytes >> using one of the UTCs) with *Unicode strings*. >> >> The string you give above is is a Unicode string containing two code >> points, the surrogates U+D800 U+DC01, which as far as I am concerned is >> a legal string (subject to somebody pointing me to a definitive source >> that proves it is not). However, it *may or may not* be encodable to >> bytes using UTF-8, -16 or -32. > > From chapter two of the standard. > > "Plain text is a pure sequence of character codes; plain Unicode-encoded > text is therefore a sequence of Unicode character codes." Also there are many valid non-characters in Unicode, including 66 explicitly defined non-characters, plus the many surrogates. So defining Unicode strings in terms of characters is less than helpful, since it excludes a whole bunch of strings which aren't "text" since they include non-characters. Also, "character" in the context of Unicode is ambiguous, due to normalization and decomposition: a single character can have up to four distinct forms. http://www.macchiato.com/unicode/nfc-faq *Code points* are rigorously defined, not characters, which is why I have tried very hard to only refer to code points and bytes, not characters. > http://www.unicode.org/versions/Unicode6.2.0/ch02.pdf#G13708 "All three > encoding forms can be used to represent the full range of encoded > characters in the Unicode Standard; ... Each of the three Unicode > encoding forms can be efficiently transformed into eith er of the other > two without any loss of data." This merely says "encodings encode characters". We know that encodings can also encode non-characters, at least *some* non-characters. The question is, can they encode surrogates? > "Surrogates Area. The Surrogates Area contains only surrogate code > points and no encoded characters. See Section 16.6, Surrogates Area, for > more detail." > > Before utf-16, the surrogates area was, I believe, part of the Private > Use Area (which now starts where surrogates end). I think it would have > been better if they were no longer called code points, but simply utf-16 > code units. Private Use is irrelevant, since strings certainly can contain Private Use code-points, and UTF encodings can encode them. >> Just as there are byte sequences that cannot be generated by the UTFs, >> possibly there are code point sequences that cannot be converted to >> bytes using the UTFs. > > True, but not to the point. You switched from sequences of characters > (unicode text), which is what both I and Neil are talking about, to > sequences of codepoints which is a larger set when you include the > non-character surrogate 'code points' that are not allowed in unicode > text. I never mentioned sequences of characters. I've always talked about code points. > http://www.unicode.org/versions/Unicode6.2.0/ch03.pdf#G7404 > > "The Unicode Standard supports three character encoding forms: UTF-32, > UTF-16, and UTF-8. Each encoding form maps the Unicode code points > U+0000..U+D7FF and U+E000..U+10FFFF to unique code unit sequences." Ah! Now we're getting somewhere! I think you've hit the nail on the head: the three UTF forms explicitly exclude the surrogates. So I think we now have an answer: Surrogate code points can exist in Unicode strings, but cannot be encoded to bytes using the standard UTF-8, UTF-16 and UTF-32 encodings. There may be other encodings, or error handlers, which are capable of handling surrogates, but they aren't UTF-8. So I think this answers my question. (I reserve the right to change my mind after reading more of the standard.) Thank you to everyone who replied. -- Steven From wxjmfauth at gmail.com Wed Oct 9 04:22:54 2013 From: wxjmfauth at gmail.com (wxjmfauth at gmail.com) Date: Wed, 9 Oct 2013 01:22:54 -0700 (PDT) Subject: Encoding of surrogate code points to UTF-8 In-Reply-To: <5254f594$0$29976$c3e8da3$5496439d@news.astraweb.com> References: <52540e03$0$29984$c3e8da3$5496439d@news.astraweb.com> <52548791$0$29984$c3e8da3$5496439d@news.astraweb.com> <5254f594$0$29976$c3e8da3$5496439d@news.astraweb.com> Message-ID: <4b728b1a-cc37-4541-80a2-68335f1d5e5f@googlegroups.com> Le mercredi 9 octobre 2013 08:20:05 UTC+2, Steven D'Aprano a ?crit?: > > > > http://www.unicode.org/versions/Unicode6.2.0/ch02.pdf#G13708 "All three > > > encoding forms can be used to represent the full range of encoded > > > characters in the Unicode Standard; ... Each of the three Unicode > > > encoding forms can be efficiently transformed into eith er of the other > > > two without any loss of data." > > Yes, and what Unicode.org does not say is that these coding schemes (like any coding scheme) should be used in an exclusive way. Probably, because it is too obvious to understand. jmf From ned at nedbatchelder.com Wed Oct 9 06:22:20 2013 From: ned at nedbatchelder.com (Ned Batchelder) Date: Wed, 09 Oct 2013 06:22:20 -0400 Subject: Encoding of surrogate code points to UTF-8 In-Reply-To: <4b728b1a-cc37-4541-80a2-68335f1d5e5f@googlegroups.com> References: <52540e03$0$29984$c3e8da3$5496439d@news.astraweb.com> <52548791$0$29984$c3e8da3$5496439d@news.astraweb.com> <5254f594$0$29976$c3e8da3$5496439d@news.astraweb.com> <4b728b1a-cc37-4541-80a2-68335f1d5e5f@googlegroups.com> Message-ID: <52552E5C.4010700@nedbatchelder.com> On 10/9/13 4:22 AM, wxjmfauth at gmail.com wrote: > Le mercredi 9 octobre 2013 08:20:05 UTC+2, Steven D'Aprano a ?crit : >> >>> http://www.unicode.org/versions/Unicode6.2.0/ch02.pdf#G13708 "All three >>> encoding forms can be used to represent the full range of encoded >>> characters in the Unicode Standard; ... Each of the three Unicode >>> encoding forms can be efficiently transformed into eith er of the other >>> two without any loss of data." >> > Yes, > > and what Unicode.org does not say is that these coding > schemes (like any coding scheme) should be used in an > exclusive way. Can you clarify what you mean by "in an exclusive way"? --Ned. > Probably, because it is too obvious to understand. > > jmf > > From neilc at norwich.edu Wed Oct 9 08:55:47 2013 From: neilc at norwich.edu (Neil Cerutti) Date: 9 Oct 2013 12:55:47 GMT Subject: Encoding of surrogate code points to UTF-8 References: <52540e03$0$29984$c3e8da3$5496439d@news.astraweb.com> <52548791$0$29984$c3e8da3$5496439d@news.astraweb.com> <5254f594$0$29976$c3e8da3$5496439d@news.astraweb.com> <4b728b1a-cc37-4541-80a2-68335f1d5e5f@googlegroups.com> Message-ID: On 2013-10-09, Ned Batchelder wrote: > On 10/9/13 4:22 AM, wxjmfauth at gmail.com wrote: >> and what Unicode.org does not say is that these coding schemes >> (like any coding scheme) should be used in an exclusive way. > > Can you clarify what you mean by "in an exclusive way"? Ned, pay no attention to the person whalopping that dead horse. -- Neil Cerutti From petef4+usenet at gmail.com Tue Oct 8 11:23:57 2013 From: petef4+usenet at gmail.com (Pete Forman) Date: Tue, 08 Oct 2013 16:23:57 +0100 Subject: Encoding of surrogate code points to UTF-8 References: <52540e03$0$29984$c3e8da3$5496439d@news.astraweb.com> Message-ID: <86mwmjlraq.fsf@gmail.com> Steven D'Aprano writes: > I think this is a bug in Python's UTF-8 handling, but I'm not sure. [snip] > py> s = '\ud800\udc01' > py> s.encode('utf-8') > Traceback (most recent call last): > File "", line 1, in > UnicodeEncodeError: 'utf-8' codec can't encode character '\ud800' in > position 0: surrogates not allowed > > > Have I misunderstood? I think that Python is being too strict about > rejecting surrogate code points. It should only reject lone surrogates, > or invalid pairs, not valid pairs. Have I misunderstood the Unicode FAQs, > or is this a bug in Python's handling of UTF-8? http://www.unicode.org/versions/Unicode6.2.0/ch03.pdf D75 Surrogate pair: A representation for a single abstract character that consists of a sequence of two 16-bit code units, where the first value of the pair is a high-surrogate code unit and the second value is a low-surrogate code unit. * Surrogate pairs are used only in UTF-16. (See Section 3.9, Unicode EncodingForms.) * Isolated surrogate code units have no interpretation on their own. Certain other isolated code units in other encoding forms also have no interpretation on their own. For example, the isolated byte [\x80] has no interpretation in UTF-8; it can be used only as part of a multibyte sequence. (See Table 3-7). It could be argued that this line by itself should raise an error. That first bullet indicates that it is indeed illegal to use surrogate pairs in UTF-8 or UTF-32. -- Pete Forman From python at mrabarnett.plus.com Tue Oct 8 13:00:58 2013 From: python at mrabarnett.plus.com (MRAB) Date: Tue, 08 Oct 2013 18:00:58 +0100 Subject: Encoding of surrogate code points to UTF-8 In-Reply-To: <86mwmjlraq.fsf@gmail.com> References: <52540e03$0$29984$c3e8da3$5496439d@news.astraweb.com> <86mwmjlraq.fsf@gmail.com> Message-ID: <52543A4A.3070506@mrabarnett.plus.com> On 08/10/2013 16:23, Pete Forman wrote: > Steven D'Aprano writes: > >> I think this is a bug in Python's UTF-8 handling, but I'm not sure. > [snip] >> py> s = '\ud800\udc01' >> py> s.encode('utf-8') >> Traceback (most recent call last): >> File "", line 1, in >> UnicodeEncodeError: 'utf-8' codec can't encode character '\ud800' in >> position 0: surrogates not allowed >> >> >> Have I misunderstood? I think that Python is being too strict about >> rejecting surrogate code points. It should only reject lone surrogates, >> or invalid pairs, not valid pairs. Have I misunderstood the Unicode FAQs, >> or is this a bug in Python's handling of UTF-8? > > http://www.unicode.org/versions/Unicode6.2.0/ch03.pdf > > D75 Surrogate pair: A representation for a single abstract character > that consists of a sequence of two 16-bit code units, where the first > value of the pair is a high-surrogate code unit and the second value > is a low-surrogate code unit. > > * Surrogate pairs are used only in UTF-16. (See Section 3.9, Unicode > EncodingForms.) > > * Isolated surrogate code units have no interpretation on their own. > Certain other isolated code units in other encoding forms also have no > interpretation on their own. For example, the isolated byte [\x80] has > no interpretation in UTF-8; it can be used only as part of a multibyte > sequence. (See Table 3-7). It could be argued that this line by itself > should raise an error. > > > That first bullet indicates that it is indeed illegal to use surrogate > pairs in UTF-8 or UTF-32. > The only time you should get a surrogate pair in a Unicode string is in a narrow build, which doesn't exist in Python 3.3 and later. From wxjmfauth at gmail.com Tue Oct 8 14:24:58 2013 From: wxjmfauth at gmail.com (wxjmfauth at gmail.com) Date: Tue, 8 Oct 2013 11:24:58 -0700 (PDT) Subject: Encoding of surrogate code points to UTF-8 In-Reply-To: References: <52540e03$0$29984$c3e8da3$5496439d@news.astraweb.com> <86mwmjlraq.fsf@gmail.com> Message-ID: -------- >>> sys.version '3.3.2 (v3.3.2:d047928ae3f6, May 16 2013, 00:03:43) [MSC v.1600 32 bit (Intel)]' >>> '\ud800'.encode('utf-8') Traceback (most recent call last): File "", line 1, in UnicodeEncodeError: 'utf-8' codec can't encode character '\ud800' in position 0: surrogates not allowed >>> '\ud800'.encode('utf-32-be') b'\x00\x00\xd8\x00' >>> '\ud800'.encode('utf-32-le') b'\x00\xd8\x00\x00' >>> '\ud800'.encode('utf-32') b'\xff\xfe\x00\x00\x00\xd8\x00\x00' jmf From steve+comp.lang.python at pearwood.info Tue Oct 8 18:20:58 2013 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: 08 Oct 2013 22:20:58 GMT Subject: Encoding of surrogate code points to UTF-8 References: <52540e03$0$29984$c3e8da3$5496439d@news.astraweb.com> <86mwmjlraq.fsf@gmail.com> Message-ID: <52548549$0$29984$c3e8da3$5496439d@news.astraweb.com> On Tue, 08 Oct 2013 18:00:58 +0100, MRAB wrote: > The only time you should get a surrogate pair in a Unicode string is in > a narrow build, which doesn't exist in Python 3.3 and later. Incorrect. py> sys.version '3.3.0rc3 (default, Sep 27 2012, 18:44:58) \n[GCC 4.1.2 20080704 (Red Hat 4.1.2-52)]' py> s = '\ud800\udc01' py> print(len(s)) 2 py> import unicodedata as ud py> for c in s: ... print(ud.category(c)) ... Cs Cs s is a string containing two code points making up a surrogate pair. It is very frustrating that the Unicode FAQs don't always clearly distinguish between when they are talking about bytes and when they are talking about code points. This area about surrogates is one of places where they conflate the two. -- Steven From tjreedy at udel.edu Tue Oct 8 17:47:15 2013 From: tjreedy at udel.edu (Terry Reedy) Date: Tue, 08 Oct 2013 17:47:15 -0400 Subject: Encoding of surrogate code points to UTF-8 In-Reply-To: <52540e03$0$29984$c3e8da3$5496439d@news.astraweb.com> References: <52540e03$0$29984$c3e8da3$5496439d@news.astraweb.com> Message-ID: On 10/8/2013 9:52 AM, Steven D'Aprano wrote: > I think this is a bug in Python's UTF-8 handling, but I'm not sure. > > If I've read the Unicode FAQs correctly, you cannot encode *lone* > surrogate code points into UTF-8: > > http://www.unicode.org/faq/utf_bom.html#utf8-5 > > Sure enough, using Python 3.3: > > py> surr = '\udc80' I am pretty sure that if Python were being strict, that would raise an error, as the result is not a valid unicode string. Allowing the above or not was debated and laxness was allowed for at least the following practical reasons. 1. Python itself uses the invalid surrogate codepoints for surrogateescape error-handling. http://www.python.org/dev/peps/pep-0383/ 2. Invalid strings are needed for tests ;-) -- like the one you do next. 3. Invalid strings may be needed for interfacing with other C APIs. > py> surr.encode('utf-8') > Traceback (most recent call last): > File "", line 1, in > UnicodeEncodeError: 'utf-8' codec can't encode character '\udc80' in > position 0: surrogates not allowed Default strict encoding (utf-8 or otherwise) will only encode valid unicode strings. Encode invalid strings with surrogate codepoints with surrogateescape error handling. > But reading the previous entry in the FAQs: > > http://www.unicode.org/faq/utf_bom.html#utf8-4 > > I interpret this as meaning that I should be able to encode valid pairs > of surrogates. It says you should be able to 'convert' them, and that the result for utf-8 encoding must be a single 4-bytes code for the corresponding supplementary codepoint. > So if I find a code point that encodes to a surrogate pair > in UTF-16: > > py> c = '\N{LINEAR B SYLLABLE B038 E}' > py> surr_pair = c.encode('utf-16be') > py> print(surr_pair) > b'\xd8\x00\xdc\x01' > > and then use those same values as the code points, I ought to be able to > encode to UTF-8, as if it were the same \N{LINEAR B SYLLABLE B038 E} code > point. But I can't: > > py> s = '\ud800\udc01' This is now a string with two invalid codepoints instead of one ;-). As above, it would be rejected if Python were being strict. > py> s.encode('utf-8') > Traceback (most recent call last): > File "", line 1, in > UnicodeEncodeError: 'utf-8' codec can't encode character '\ud800' in > position 0: surrogates not allowed > > > Have I misunderstood? I think that Python is being too strict about > rejecting surrogate code points. No, it is being too lax about allowing them at all. I believe there is an issue on the tracker (maybe closed) about the doc for unicode escapes in string literals. Perhaps is should say more clearly that inserting surrogates is allowed but results in an invalid string that cannot be normally encoded. -- Terry Jan Reedy From tjreedy at udel.edu Tue Oct 8 18:17:01 2013 From: tjreedy at udel.edu (Terry Reedy) Date: Tue, 08 Oct 2013 18:17:01 -0400 Subject: Encoding of surrogate code points to UTF-8 In-Reply-To: References: <52540e03$0$29984$c3e8da3$5496439d@news.astraweb.com> Message-ID: On 10/8/2013 5:47 PM, Terry Reedy wrote: > On 10/8/2013 9:52 AM, Steven D'Aprano wrote: >> But reading the previous entry in the FAQs: >> >> http://www.unicode.org/faq/utf_bom.html#utf8-4 >> >> I interpret this as meaning that I should be able to encode valid pairs >> of surrogates. > > It says you should be able to 'convert' them, and that the result for > utf-8 encoding must be a single 4-bytes code for the corresponding > supplementary codepoint. To expand on this: The FAQ question is "How do I convert a UTF-16 surrogate pair such as to UTF-8?" utf-16 and utf-8 are both byte (or double byte) encodings of codepoints. Direct conversion would be 'transcoding', not encoding. Python has a few bytes transcoders and one string transcoder (rot_13), listed at the end of http://docs.python.org/3/library/codecs.html#python-specific-encodings But in general, one must decode bytes to string and encode back to bytes. >> So if I find a code point that encodes to a surrogate pair >> in UTF-16: >> >> py> c = '\N{LINEAR B SYLLABLE B038 E}' >> py> surr_pair = c.encode('utf-16be') >> py> print(surr_pair) >> b'\xd8\x00\xdc\x01' >> >> and then use those same values as the code points, I ought to be able to >> encode to UTF-8, as if it were the same \N{LINEAR B SYLLABLE B038 E} code >> point. I believe the utf encodings are defined as 1 to 1. If the above worked, utf-8 would not be. -- Terry Jan Reedy From kjakupak at gmail.com Tue Oct 8 10:28:18 2013 From: kjakupak at gmail.com (kjakupak at gmail.com) Date: Tue, 8 Oct 2013 07:28:18 -0700 (PDT) Subject: converting letters to numbers Message-ID: <26151b64-4f5e-4ee9-81ac-26679932f43d@googlegroups.com> I have to define a function add(c1, c2), where c1 and c2 are capital letters; the return value should be the sum (obtained by converting the letters to numbers, adding mod 26, then converting back to a capital letter). All I have so far is: def add(c1, c2): ord(c1) - ord('a') + 1 ord(c2) - ord('a') + 1 I know I need to use ord and chr, just not sure how. From robertkday at gmail.com Tue Oct 8 10:47:39 2013 From: robertkday at gmail.com (Robert Day) Date: Tue, 08 Oct 2013 15:47:39 +0100 Subject: converting letters to numbers In-Reply-To: <26151b64-4f5e-4ee9-81ac-26679932f43d@googlegroups.com> References: <26151b64-4f5e-4ee9-81ac-26679932f43d@googlegroups.com> Message-ID: <52541B0B.7020306@gmail.com> On 08/10/13 15:28, kjakupak at gmail.com wrote: > I have to define a function add(c1, c2), where c1 and c2 are capital letters; the return value should be the sum (obtained by converting the letters to numbers, adding mod 26, then converting back to a capital letter). > Can you give some expected outputs? For example, add('A', 'B') should presumably return 'C', and add('M', 'B') should presumably return 'O', but what about add('A', 'A') or add('Z', 'Z')? It feels like the only tricky bit is mapping letters to numbers (i.e. does A equal 1 or 0?), which you'd do by subtracting a fixed value from the result of chr. Once you've done that, you'd do the arithmetic to get a number between 1 and 26 (or 0 and 25), then add the same fixed value to that and call ord on the result. From kjakupak at gmail.com Tue Oct 8 10:51:24 2013 From: kjakupak at gmail.com (kjakupak at gmail.com) Date: Tue, 8 Oct 2013 07:51:24 -0700 (PDT) Subject: converting letters to numbers In-Reply-To: References: <26151b64-4f5e-4ee9-81ac-26679932f43d@googlegroups.com> Message-ID: <32293336-949a-448c-bc1c-a7ac5186fcb6@googlegroups.com> On Tuesday, October 8, 2013 10:47:39 AM UTC-4, Robert Day wrote: > On 08/10/13 15:28, kjakupak at gmail.com wrote: > > Can you give some expected outputs? For example, add('A', 'B') should > > presumably return 'C', and add('M', 'B') should presumably return 'O', > > but what about add('A', 'A') or add('Z', 'Z')? > > > > It feels like the only tricky bit is mapping letters to numbers (i.e. > > does A equal 1 or 0?), which you'd do by subtracting a fixed value from > > the result of chr. Once you've done that, you'd do the arithmetic to get > > a number between 1 and 26 (or 0 and 25), then add the same fixed value > > to that and call ord on the result. Expected output is add('C', 'E') returns 'G'; where 'C' and 'E' correspond to 2 and 4 respectively with sum 6, corresponding to 'G'. From joel.goldstick at gmail.com Tue Oct 8 11:01:12 2013 From: joel.goldstick at gmail.com (Joel Goldstick) Date: Tue, 8 Oct 2013 11:01:12 -0400 Subject: converting letters to numbers In-Reply-To: <32293336-949a-448c-bc1c-a7ac5186fcb6@googlegroups.com> References: <26151b64-4f5e-4ee9-81ac-26679932f43d@googlegroups.com> <32293336-949a-448c-bc1c-a7ac5186fcb6@googlegroups.com> Message-ID: You wrote this: def add(c1, c2): ord(c1) - ord('a') + 1 ord(c2) - ord('a') + 1 First of all, this looks like homework. People will help you with concepts here, but most frown on just providing answers. With that in mind look at this: >>> ord('A') 65 >>> ord('a') 97 >>> In your assignment you refer to Upper case letters. In your code you take the ordinal value of lower case 'a' -- Joel Goldstick http://joelgoldstick.com From random832 at fastmail.us Tue Oct 8 11:36:51 2013 From: random832 at fastmail.us (random832 at fastmail.us) Date: Tue, 08 Oct 2013 11:36:51 -0400 Subject: converting letters to numbers In-Reply-To: <26151b64-4f5e-4ee9-81ac-26679932f43d@googlegroups.com> References: <26151b64-4f5e-4ee9-81ac-26679932f43d@googlegroups.com> Message-ID: <1381246611.16408.31494445.07DDEE90@webmail.messagingengine.com> On Tue, Oct 8, 2013, at 10:28, kjakupak at gmail.com wrote: > I have to define a function add(c1, c2), where c1 and c2 are capital > letters; the return value should be the sum (obtained by converting the > letters to numbers, adding mod 26, then converting back to a capital > letter). > > All I have so far is: > > def add(c1, c2): > ord(c1) - ord('a') + 1 > ord(c2) - ord('a') + 1 > > I know I need to use ord and chr, just not sure how. Your description says capital letters, but 'a' is a lowercase letter. Does "mod 26" means A is 1, or is it 0? i.e., is A+A = B or is it A? What should your function do if the letter isn't a capital letter from the basic set of 26 English letters? From kjakupak at gmail.com Tue Oct 8 11:44:47 2013 From: kjakupak at gmail.com (kjakupak at gmail.com) Date: Tue, 8 Oct 2013 08:44:47 -0700 (PDT) Subject: converting letters to numbers In-Reply-To: References: <26151b64-4f5e-4ee9-81ac-26679932f43d@googlegroups.com> Message-ID: On Tuesday, October 8, 2013 11:36:51 AM UTC-4, rand... at fastmail.us wrote: > > > > Your description says capital letters, but 'a' is a lowercase letter. > > > > Does "mod 26" means A is 1, or is it 0? i.e., is A+A = B or is it A? > > > > What should your function do if the letter isn't a capital letter from > > the basic set of 26 English letters? A is 0. Transfer it to an uppercase letter if it's a letter, if it's not then an error. This isn't right, I know, just testing around def add(c1, c2): ans = '' for i in c1 + c2: ans += chr((((ord(i)-65))%26) + 65) return ans From random832 at fastmail.us Tue Oct 8 11:58:36 2013 From: random832 at fastmail.us (random832 at fastmail.us) Date: Tue, 08 Oct 2013 11:58:36 -0400 Subject: converting letters to numbers In-Reply-To: References: <26151b64-4f5e-4ee9-81ac-26679932f43d@googlegroups.com> Message-ID: <1381247916.26355.31503845.1B3F894F@webmail.messagingengine.com> On Tue, Oct 8, 2013, at 11:44, kjakupak at gmail.com wrote: > def add(c1, c2): > ans = '' This only makes sense if your answer is going to be multiple characters. > for i in c1 + c2: This line concatenates the strings together. > ans += chr((((ord(i)-65))%26) + 65) The way you are doing the modulus, this results in - well, let me illustrate: >>> add('','WXYZ[\]^_`abcde') 'WXYZABCDEFGHIJK' From timr at probo.com Sun Oct 13 23:13:32 2013 From: timr at probo.com (Tim Roberts) Date: Sun, 13 Oct 2013 20:13:32 -0700 Subject: converting letters to numbers References: <26151b64-4f5e-4ee9-81ac-26679932f43d@googlegroups.com> Message-ID: kjakupak at gmail.com wrote: > >Transfer it to an uppercase letter if it's a letter, if it's not then an error. >This isn't right, I know, just testing around > >def add(c1, c2): > ans = '' > for i in c1 + c2: > ans += chr((((ord(i)-65))%26) + 65) > return ans It's close. I think you're overthinking it. Take it step by step. Decode, process, encode. That means convert the inputs to integers, add the integers, convert the result back. def add(c1, c2): % Decode c1 = ord(c1) - 65 c2 = ord(c2) - 65 % Process i1 = (c1 + c2) % 26 % Encode return chr(i1+65) Or, as a one-liner: A = ord('A') def add(c1, c2): return chr((ord(c1)-A + ord(c2)-A) % 26 + A) -- Tim Roberts, timr at probo.com Providenza & Boekelheide, Inc. From steve at pearwood.info Mon Oct 14 01:02:36 2013 From: steve at pearwood.info (Steven D'Aprano) Date: 14 Oct 2013 05:02:36 GMT Subject: converting letters to numbers References: <26151b64-4f5e-4ee9-81ac-26679932f43d@googlegroups.com> Message-ID: <525b7aec$0$30000$c3e8da3$5496439d@news.astraweb.com> On Sun, 13 Oct 2013 20:13:32 -0700, Tim Roberts wrote: > def add(c1, c2): > % Decode > c1 = ord(c1) - 65 > c2 = ord(c2) - 65 > % Process > i1 = (c1 + c2) % 26 > % Encode > return chr(i1+65) Python uses # for comments, not %, as I'm sure you know. What language were you thinking off when you wrote the above? -- Steven From charleshixsn at earthlink.net Wed Oct 16 15:18:39 2013 From: charleshixsn at earthlink.net (Charles Hixson) Date: Wed, 16 Oct 2013 12:18:39 -0700 Subject: converting letters to numbers In-Reply-To: <525b7aec$0$30000$c3e8da3$5496439d@news.astraweb.com> References: <26151b64-4f5e-4ee9-81ac-26679932f43d@googlegroups.com> <525b7aec$0$30000$c3e8da3$5496439d@news.astraweb.com> Message-ID: <525EE68F.5080600@earthlink.net> On 10/13/2013 10:02 PM, Steven D'Aprano wrote: > On Sun, 13 Oct 2013 20:13:32 -0700, Tim Roberts wrote: > >> def add(c1, c2): >> % Decode >> c1 = ord(c1) - 65 >> c2 = ord(c2) - 65 >> % Process >> i1 = (c1 + c2) % 26 >> % Encode >> return chr(i1+65) > Python uses # for comments, not %, as I'm sure you know. What language > were you thinking off when you wrote the above? > > > IIRC Lisp uses % for comments, but it may need to be doubled. (It's been doubled in the examples I've seen, and I don't remember the syntax.) Perhaps Scheme has the same convention, but Scheme could be considered a part of the Lisp clade. -- Charles Hixson From piet at vanoostrum.org Wed Oct 16 16:25:21 2013 From: piet at vanoostrum.org (Piet van Oostrum) Date: Wed, 16 Oct 2013 16:25:21 -0400 Subject: converting letters to numbers References: <26151b64-4f5e-4ee9-81ac-26679932f43d@googlegroups.com> <525b7aec$0$30000$c3e8da3$5496439d@news.astraweb.com> Message-ID: Charles Hixson writes: > On 10/13/2013 10:02 PM, Steven D'Aprano wrote: >> On Sun, 13 Oct 2013 20:13:32 -0700, Tim Roberts wrote: >> >>> def add(c1, c2): >>> % Decode >>> c1 = ord(c1) - 65 >>> c2 = ord(c2) - 65 >>> % Process >>> i1 = (c1 + c2) % 26 >>> % Encode >>> return chr(i1+65) >> Python uses # for comments, not %, as I'm sure you know. What language >> were you thinking off when you wrote the above? >> >> >> > IIRC Lisp uses % for comments, but it may need to be doubled. (It's > been doubled in the examples I've seen, and I don't remember the > syntax.) > Perhaps Scheme has the same convention, but Scheme could be considered a > part of the Lisp clade. Lisp and scheme use semicolon (;). It wouldn't have been that difficult to look that up I think. -- Piet van Oostrum WWW: http://pietvanoostrum.com/ PGP key: [8DAE142BE17999C4] From sg552 at hotmail.co.uk Wed Oct 16 18:39:28 2013 From: sg552 at hotmail.co.uk (Rotwang) Date: Wed, 16 Oct 2013 23:39:28 +0100 Subject: converting letters to numbers In-Reply-To: <525b7aec$0$30000$c3e8da3$5496439d@news.astraweb.com> References: <26151b64-4f5e-4ee9-81ac-26679932f43d@googlegroups.com> <525b7aec$0$30000$c3e8da3$5496439d@news.astraweb.com> Message-ID: On 14/10/2013 06:02, Steven D'Aprano wrote: > On Sun, 13 Oct 2013 20:13:32 -0700, Tim Roberts wrote: > >> def add(c1, c2): >> % Decode >> c1 = ord(c1) - 65 >> c2 = ord(c2) - 65 >> % Process >> i1 = (c1 + c2) % 26 >> % Encode >> return chr(i1+65) > > Python uses # for comments, not %, as I'm sure you know. What language > were you thinking off when you wrote the above? Maybe TeX? From python at mrabarnett.plus.com Wed Oct 16 18:53:02 2013 From: python at mrabarnett.plus.com (MRAB) Date: Wed, 16 Oct 2013 23:53:02 +0100 Subject: converting letters to numbers In-Reply-To: References: <26151b64-4f5e-4ee9-81ac-26679932f43d@googlegroups.com> <525b7aec$0$30000$c3e8da3$5496439d@news.astraweb.com> Message-ID: <525F18CE.4080606@mrabarnett.plus.com> On 16/10/2013 23:39, Rotwang wrote: > On 14/10/2013 06:02, Steven D'Aprano wrote: >> On Sun, 13 Oct 2013 20:13:32 -0700, Tim Roberts wrote: >> >>> def add(c1, c2): >>> % Decode >>> c1 = ord(c1) - 65 >>> c2 = ord(c2) - 65 >>> % Process >>> i1 = (c1 + c2) % 26 >>> % Encode >>> return chr(i1+65) >> >> Python uses # for comments, not %, as I'm sure you know. What language >> were you thinking off when you wrote the above? > > Maybe TeX? > Or PostScript? From oscar.j.benjamin at gmail.com Wed Oct 16 20:21:49 2013 From: oscar.j.benjamin at gmail.com (Oscar Benjamin) Date: Thu, 17 Oct 2013 01:21:49 +0100 Subject: converting letters to numbers In-Reply-To: <525F18CE.4080606@mrabarnett.plus.com> References: <26151b64-4f5e-4ee9-81ac-26679932f43d@googlegroups.com> <525b7aec$0$30000$c3e8da3$5496439d@news.astraweb.com> <525F18CE.4080606@mrabarnett.plus.com> Message-ID: On Oct 16, 2013 11:54 PM, "MRAB" wrote: > > On 16/10/2013 23:39, Rotwang wrote: >> >> On 14/10/2013 06:02, Steven D'Aprano wrote: >>> >>> On Sun, 13 Oct 2013 20:13:32 -0700, Tim Roberts wrote: >>> >>>> def add(c1, c2): >>>> % Decode >>>> c1 = ord(c1) - 65 >>>> c2 = ord(c2) - 65 >>>> % Process >>>> i1 = (c1 + c2) % 26 >>>> % Encode >>>> return chr(i1+65) >>> >>> >>> Python uses # for comments, not %, as I'm sure you know. What language >>> were you thinking off when you wrote the above? >> >> >> Maybe TeX? >> > Or PostScript? My money's on matlab. -------------- next part -------------- An HTML attachment was scrubbed... URL: From timr at probo.com Sat Oct 19 19:31:14 2013 From: timr at probo.com (Tim Roberts) Date: Sat, 19 Oct 2013 16:31:14 -0700 Subject: converting letters to numbers References: <26151b64-4f5e-4ee9-81ac-26679932f43d@googlegroups.com> <525b7aec$0$30000$c3e8da3$5496439d@news.astraweb.com> Message-ID: Steven D'Aprano wrote: >On Sun, 13 Oct 2013 20:13:32 -0700, Tim Roberts wrote: > >> def add(c1, c2): >> % Decode >>... >Python uses # for comments, not %, as I'm sure you know. What language >were you thinking off when you wrote the above? Psssht, I know better than that. I've been reading through MATLAB code, which uses %, but I have CERTAINLY not written enough MATLAB to excuse that. -- Tim Roberts, timr at probo.com Providenza & Boekelheide, Inc. From rustompmody at gmail.com Sun Oct 20 04:33:38 2013 From: rustompmody at gmail.com (rusi) Date: Sun, 20 Oct 2013 01:33:38 -0700 (PDT) Subject: converting letters to numbers In-Reply-To: <525b7aec$0$30000$c3e8da3$5496439d@news.astraweb.com> References: <26151b64-4f5e-4ee9-81ac-26679932f43d@googlegroups.com> <525b7aec$0$30000$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Monday, October 14, 2013 10:32:36 AM UTC+5:30, Steven D'Aprano wrote: > On Sun, 13 Oct 2013 20:13:32 -0700, Tim Roberts wrote: > > > def add(c1, c2): > > % Decode > > c1 = ord(c1) - 65 > > c2 = ord(c2) - 65 > > % Process > > i1 = (c1 + c2) % 26 > > % Encode > > return chr(i1+65) > > Python uses # for comments, not %, as I'm sure you know. What language > were you thinking off when you wrote the above? Maybe Tim was putting in the percentage of CPU cycles (wetware cycles??) on Decode, Process and Encode. So we have the % but not the percent? From breamoreboy at yahoo.co.uk Tue Oct 8 12:05:33 2013 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Tue, 08 Oct 2013 17:05:33 +0100 Subject: converting letters to numbers In-Reply-To: <26151b64-4f5e-4ee9-81ac-26679932f43d@googlegroups.com> References: <26151b64-4f5e-4ee9-81ac-26679932f43d@googlegroups.com> Message-ID: On 08/10/2013 15:28, kjakupak at gmail.com wrote: > I have to define a function add(c1, c2), where c1 and c2 are capital letters; the return value should be the sum (obtained by converting the letters to numbers, adding mod 26, then converting back to a capital letter). > I'd say the requirement is lacking in that no encoding is specified. > All I have so far is: > > def add(c1, c2): > ord(c1) - ord('a') + 1 > ord(c2) - ord('a') + 1 > > I know I need to use ord and chr, just not sure how. > I'll further observe from your later replies that you're suffering from the highly contagious, highly virulent double line spacing disease. This is known to cause severe eye strain leading to blindness. In can be cured by purchasing medication here https://wiki.python.org/moin/GoogleGroupsPython -- Roses are red, Violets are blue, Most poems rhyme, But this one doesn't. Mark Lawrence From davea at davea.name Tue Oct 8 17:24:03 2013 From: davea at davea.name (Dave Angel) Date: Tue, 8 Oct 2013 21:24:03 +0000 (UTC) Subject: converting letters to numbers References: <26151b64-4f5e-4ee9-81ac-26679932f43d@googlegroups.com> Message-ID: On 8/10/2013 10:28, kjakupak at gmail.com wrote: > I have to define a function add(c1, c2), where c1 and c2 are capital letters; the return value should be the sum (obtained by converting the letters to numbers, adding mod 26, then converting back to a capital letter). > > All I have so far is: > > def add(c1, c2): > ord(c1) - ord('a') + 1 > ord(c2) - ord('a') + 1 > > I know I need to use ord and chr, just not sure how. Factor the problem into three functions. one function converts a one-character string into an int, or gives an exception if the character. isn't uppercase ASCII. Second function converts a small int into a string containing one uppercase ASCII letter, throwing an exception if negative or above 25. Third function takes two string arguents, throws an exception if either of them is not exactly one character in length. Then it calls the first function twice, adds the results, modulos it, and calls the second function, returning its return value. Which of these is giving you trouble? Notice you can use the first two functions to test each other. -- DaveA From josiah.carlson at gmail.com Tue Oct 8 21:36:04 2013 From: josiah.carlson at gmail.com (Josiah Carlson) Date: Tue, 8 Oct 2013 18:36:04 -0700 Subject: ANN: RPQueue 0.22 Message-ID: Hello everyone, For those of you who didn't know, if you are interested in a Redis-backed time and/or fifo-queue with priorities, retries, etc., to be used with Python, one exists and is mature: it's called RPQueue, and it seeks to simplify your life of task execution. The recent changelog entries are below my signature. The package has been around for two years next month, and does exactly what you expect it to do - no more, no less. You can find the package at: https://github.com/josiahcarlson/rpqueue/ https://pypi.python.org/pypi/rpqueue Please CC me on any replies if you have any questions or comments. Thank you, - Josiah #----------------------------------- 0.22 ------------------------------------ [fixed] setup.py-based installations. Ran into the bug myself :/ #----------------------------------- 0.21 ------------------------------------ [changed] where available, rpqueue will now use Lua to move delayed tasks from the time-based priority queue to fifo queues. This should reduce overhead in all cases, and should drastically improve performance for those that use large numbers of delayed tasks. [changed] the version number is now PEP 386/440 compliant. [added] this changelog that will document updates/changes/improvements in an easily referenced location. [fixed] thanks to https://github.com/dmaust , rpqueue added a missing 'redis' requirement. -------------- next part -------------- An HTML attachment was scrubbed... URL: From nagle at animats.com Wed Oct 9 01:28:44 2013 From: nagle at animats.com (John Nagle) Date: Tue, 08 Oct 2013 22:28:44 -0700 Subject: Applying 4x4 transformation to 3-element vector with numpy Message-ID: This is the basic transformation of 3D graphics. Take a 3D point, make it 4D by adding a 1 on the end, multiply by a transformation matrix to get a new 4-element vector, discard the last element. Is there some way to do that in numpy without adding the extra element and then discarding it? John Nagle From auriocus at gmx.de Wed Oct 9 01:36:17 2013 From: auriocus at gmx.de (Christian Gollwitzer) Date: Wed, 09 Oct 2013 07:36:17 +0200 Subject: Applying 4x4 transformation to 3-element vector with numpy In-Reply-To: References: Message-ID: Dear John, Am 09.10.13 07:28, schrieb John Nagle: > This is the basic transformation of 3D graphics. Take > a 3D point, make it 4D by adding a 1 on the end, multiply > by a transformation matrix to get a new 4-element vector, > discard the last element. > > Is there some way to do that in numpy without > adding the extra element and then discarding it? > if you can discard the last element, the matrix has a special structure: It is an affine transform, where the last row is unity, and it can be rewritten as A*x+b where A is the 3x3 upper left submatrix and b is the column vector. You can do this by simple slicing - with C as the 4x4 matrix it is something like dot(C[0:3, 0:3], x) + C[3, 0:3] (untested, you need to check if I got the indices right) *IF* however, your transform is perspective, then this is incorrect - you must divide the result vector by the last element before discarding it, if it is a 3D-point. For a 3D-vector (enhanced by a 0) you might still find a shortcut. Christian From nagle at animats.com Wed Oct 9 02:10:16 2013 From: nagle at animats.com (John Nagle) Date: Tue, 08 Oct 2013 23:10:16 -0700 Subject: Applying 4x4 transformation to 3-element vector with numpy In-Reply-To: References: Message-ID: On 10/8/2013 10:36 PM, Christian Gollwitzer wrote: > Dear John, > > Am 09.10.13 07:28, schrieb John Nagle: >> This is the basic transformation of 3D graphics. Take >> a 3D point, make it 4D by adding a 1 on the end, multiply >> by a transformation matrix to get a new 4-element vector, >> discard the last element. >> >> Is there some way to do that in numpy without >> adding the extra element and then discarding it? >> > > if you can discard the last element, the matrix has a special structure: > It is an affine transform, where the last row is unity, and it can be > rewritten as > > A*x+b > > where A is the 3x3 upper left submatrix and b is the column vector. You > can do this by simple slicing - with C as the 4x4 matrix it is something > like > > dot(C[0:3, 0:3], x) + C[3, 0:3] > > (untested, you need to check if I got the indices right) > > *IF* however, your transform is perspective, then this is incorrect - > you must divide the result vector by the last element before discarding > it, if it is a 3D-point. For a 3D-vector (enhanced by a 0) you might > still find a shortcut. I only need affine transformations. This is just moving the coordinate system of a point, not perspective rendering. I have to do this for a lot of points, and I'm hoping numpy has some way to do this without generating extra garbage on the way in and the way out. I've done this before in C++. John Nagle From nobody at nowhere.com Wed Oct 9 12:44:16 2013 From: nobody at nowhere.com (Nobody) Date: Wed, 09 Oct 2013 17:44:16 +0100 Subject: Applying 4x4 transformation to 3-element vector with numpy References: Message-ID: On Tue, 08 Oct 2013 23:10:16 -0700, John Nagle wrote: > I only need affine transformations. This is just moving > the coordinate system of a point, not perspective rendering. I have to do > this for a lot of points, and I'm hoping numpy has some way to do this > without generating extra garbage on the way in and the way out. In which case, Christian's answer is correct. For an affine transformation, the bottom row of the 4x4 matrix will be [0 0 0 1], so you have: [x'] [a b c d] [x] [ax+by+cz+d] [y'] = [e f g h] [y] = [ex+fy+gz+h] [z'] [i j k l] [z] [ix+jy+kz+l] [1 ] [0 0 0 1] [1] [1 ] => [x'] [ax+by+cz+d] [a b c] [x] [d] [y'] = [ex+fy+gz+h] = [e f g] [y] + [h] [z'] [ix+jy+kz+l] [i j k] [z] [l] IOW: xyz_ = m[:3,:3] * xyz + m[:3,3:] (where xyz is a column vector or 3xN array) From babmis307 at gmail.com Wed Oct 9 02:09:04 2013 From: babmis307 at gmail.com (bab mis) Date: Tue, 8 Oct 2013 23:09:04 -0700 (PDT) Subject: Can ay one help me on pysvn , i want to capture the log and status of checkout call ..... Message-ID: From breamoreboy at yahoo.co.uk Wed Oct 9 02:23:30 2013 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Wed, 09 Oct 2013 07:23:30 +0100 Subject: Can ay one help me on pysvn , i want to capture the log and status of checkout call ..... In-Reply-To: References: Message-ID: On 09/10/2013 07:09, bab mis wrote: > To repeat what I've said recently on the tutor mailing list, sorry but if you want us to do your work you'll have to send a suitably sized cheque to the PSF, where the size refers to the amount written in words and numbers and not the physical dimensions. -- Roses are red, Violets are blue, Most poems rhyme, But this one doesn't. Mark Lawrence From babmis307 at gmail.com Wed Oct 9 05:53:24 2013 From: babmis307 at gmail.com (bab mis) Date: Wed, 9 Oct 2013 02:53:24 -0700 (PDT) Subject: Can ay one help me on pysvn , i want to capture the log and status of checkout call ..... In-Reply-To: References: Message-ID: <7dd26e84-88df-4b4f-829b-f14a9819d5ce@googlegroups.com> On Wednesday, October 9, 2013 11:39:04 AM UTC+5:30, bab mis wrote: > Here is the code i am trying: 2 from pysvn import wc_status_kind 3 import pysvn 4 import os, os.path 6 import re 7 8 def createSVNClient(): 9 """Create a pysvn client, and setup some callback and options. 10 """ 11 12 def login(*args): 13 return True, 'root', 'pass', False 16 17 client = pysvn.Client() 18 client.set_interactive(True) 19 client.callback_get_login = login 20 return client 21 22 client = createSVNClient() 23 link = "http://demo.com/svn/trunk" 24 path = '/tmp/ux' 25 client.exception_style = 1 26 27 try: 28 revision = client.checkout(link, path, recurse=True) primary intention is revision = client.checkout(link, path, recurse=True) After this how can i check the exit code and capture the specific log that would have generated during this transaction of svn co command internally. From joel.goldstick at gmail.com Wed Oct 9 09:02:08 2013 From: joel.goldstick at gmail.com (Joel Goldstick) Date: Wed, 9 Oct 2013 09:02:08 -0400 Subject: Can ay one help me on pysvn , i want to capture the log and status of checkout call ..... In-Reply-To: <7dd26e84-88df-4b4f-829b-f14a9819d5ce@googlegroups.com> References: <7dd26e84-88df-4b4f-829b-f14a9819d5ce@googlegroups.com> Message-ID: On Wed, Oct 9, 2013 at 5:53 AM, bab mis wrote: > On Wednesday, October 9, 2013 11:39:04 AM UTC+5:30, bab mis wrote: >> > > Here is the code i am trying: > > > > 2 from pysvn import wc_status_kind > 3 import pysvn > 4 > import os, os.path > 6 import re > 7 > 8 def createSVNClient(): > 9 """Create a pysvn client, and setup some callback and options. > 10 """ > 11 > 12 def login(*args): > 13 return True, 'root', 'pass', False > 16 > 17 client = pysvn.Client() > 18 client.set_interactive(True) > 19 client.callback_get_login = login > 20 return client > 21 > 22 client = createSVNClient() > > 23 link = "http://demo.com/svn/trunk" > 24 path = '/tmp/ux' > 25 client.exception_style = 1 > 26 > 27 try: > 28 revision = client.checkout(link, path, recurse=True) > > > > > > primary intention is > > revision = client.checkout(link, path, recurse=True) > > After this how can i check the exit code and capture the specific log that would have generated during this transaction of svn co command internally. > -- > https://mail.python.org/mailman/listinfo/python-list have you checked the pysvn mailing list: http://pysvn.tigris.org/ds/viewForums.do -- Joel Goldstick http://joelgoldstick.com From markotaht at gmail.com Wed Oct 9 03:03:47 2013 From: markotaht at gmail.com (markotaht at gmail.com) Date: Wed, 9 Oct 2013 00:03:47 -0700 (PDT) Subject: Pygame with python 3.3.2 Message-ID: >From pygame tutorials i copied this example: import pygame class spritesheet(object): def __init__(self, filename): try: self.sheet = pygame.image.load(filename).convert() except pygame.error, message: print('Unable to load spritesheet image:', filename) raise SystemExit, message # Load a specific image from a specific rectangle def image_at(self, rectangle, colorkey = None): "Loads image from x,y,x+offset,y+offset" rect = pygame.Rect(rectangle) image = pygame.Surface(rect.size).convert() image.blit(self.sheet, (0, 0), rect) if colorkey is not None: if colorkey is -1: colorkey = image.get_at((0,0)) image.set_colorkey(colorkey, pygame.RLEACCEL) return image # Load a whole bunch of images and return them as a list def images_at(self, rects, colorkey = None): "Loads multiple images, supply a list of coordinates" return [self.image_at(rect, colorkey) for rect in rects] # Load a whole strip of images def load_strip(self, rect, image_count, colorkey = None): "Loads a strip of images and returns them as a list" tups = [(rect[0]+rect[2]*x, rect[1], rect[2], rect[3]) for x in range(image_count)] return self.images_at(tups, colorkey) When ever i run it i get syntax error on this line except pygame.error, message: There are no example to show what happens, because, this doesent even work, untile the error is fixed. My guess is, that this has something to do with the python versions. But im not very familiar with the changes so i dont know how to fix it. From breamoreboy at yahoo.co.uk Wed Oct 9 03:19:31 2013 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Wed, 09 Oct 2013 08:19:31 +0100 Subject: Pygame with python 3.3.2 In-Reply-To: References: Message-ID: On 09/10/2013 08:03, markotaht at gmail.com wrote: > From pygame tutorials i copied this example: > import pygame > > class spritesheet(object): > def __init__(self, filename): > try: > self.sheet = pygame.image.load(filename).convert() > except pygame.error, message: > print('Unable to load spritesheet image:', filename) > raise SystemExit, message > # Load a specific image from a specific rectangle > def image_at(self, rectangle, colorkey = None): > "Loads image from x,y,x+offset,y+offset" > rect = pygame.Rect(rectangle) > image = pygame.Surface(rect.size).convert() > image.blit(self.sheet, (0, 0), rect) > if colorkey is not None: > if colorkey is -1: > colorkey = image.get_at((0,0)) > image.set_colorkey(colorkey, pygame.RLEACCEL) > return image > # Load a whole bunch of images and return them as a list > def images_at(self, rects, colorkey = None): > "Loads multiple images, supply a list of coordinates" > return [self.image_at(rect, colorkey) for rect in rects] > # Load a whole strip of images > def load_strip(self, rect, image_count, colorkey = None): > "Loads a strip of images and returns them as a list" > tups = [(rect[0]+rect[2]*x, rect[1], rect[2], rect[3]) > for x in range(image_count)] > return self.images_at(tups, colorkey) > > When ever i run it i get syntax error on this line except pygame.error, message: > There are no example to show what happens, because, this doesent even work, untile the error is fixed. My guess is, that this has something to do with the python versions. But im not very familiar with the changes so i dont know how to fix it. > From http://docs.python.org/3.0/whatsnew/3.0.html "Change from except exc, var to except exc as var. See PEP 3110." You might also like to see this http://docs.python.org/3/howto/pyporting.html -- Roses are red, Violets are blue, Most poems rhyme, But this one doesn't. Mark Lawrence From jkn_gg at nicorp.f9.co.uk Wed Oct 9 07:36:17 2013 From: jkn_gg at nicorp.f9.co.uk (jkn) Date: Wed, 9 Oct 2013 04:36:17 -0700 (PDT) Subject: ePIPE exception received when other end can't send an RST - how come? Message-ID: <3a290052-8719-4142-b8d6-2eacd8e81f58@googlegroups.com> Hello there I am experimenting with a simple python script which establishes a TCP connection, over GPRS, to a server. It then periodically sends a small block of data (60 bytes or so) to the server. I then disconnect the GPRS antenna on this client machine (I am actually investigating the behaviour of an independant bit of C++ code; the python is really being used as a test bench). What I then see is that the number of bytes in the socket's output buffer builds up, and I then get a ePIPE exception (~SIGPIPE signal) in my script. Now my copy of Richard Steven's 'Unix Network programming' says: ?when a process writes to a socket that has received an RST, the SIGPIPE signal is sent to the process? My python code is very simple: something like: {{{ # setup gSock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) gSock.connect((IP_ADDR, IP_PORT)) p = MyPacket() while 1: try: gSock.send(p) except socket.error, e: # display exception except IOError, e # ePIPE exception encountered here - but why? time.sleep(15) }}} I am trying to understand how the ePIPE exception can occur, given my scenario, and whether it is due to my using Python, or might also be applicable to my C++ code. Surely, if I have disconnected the antenna, the server will have had no opportunity to send an RST? Is there another mechanism that might be causing the ePIPE? I'm running Python 2.7.4 under x86 Kubuntu Linux, and python 2.4 on an embedded ARM Linux. Thanks for any thoughts. J^n From js at globe.de Wed Oct 9 08:55:17 2013 From: js at globe.de (Schneider) Date: Wed, 09 Oct 2013 14:55:17 +0200 Subject: Good Python Book Message-ID: <52555235.3050100@globe.de> Hi List, I'm looking for a good advanced python book. Most books I looked at up to now are on beginners level. I don't need a reference (that's online) or a book explaining how to use the interpreter or how to use list comprehensions on the one side and skipping topics like decorators, metaclasses on the other side. any suggestions? bg, Johannes -- GLOBE Development GmbH K?nigsberger Strasse 260 48157 M?nsterGLOBE Development GmbH K?nigsberger Strasse 260 48157 M?nster 0251/5205 390 From kristof.leroux at gmail.com Thu Oct 10 08:40:30 2013 From: kristof.leroux at gmail.com (kristof leroux) Date: Thu, 10 Oct 2013 05:40:30 -0700 (PDT) Subject: Good Python Book In-Reply-To: References: Message-ID: <6af9809d-1fb1-4193-880a-acc009a38014@googlegroups.com> Python in Practice - Mark Summerfield On Wednesday, October 9, 2013 2:55:17 PM UTC+2, Schneider wrote: > Hi List, > > > > I'm looking for a good advanced python book. Most books I looked at up > > to now are on beginners level. > > I don't need a reference (that's online) or a book explaining how to use > > the interpreter or how to use list comprehensions on the one side and > > skipping topics like decorators, metaclasses on the other side. > > > > any suggestions? > > > > bg, > > Johannes > > > > -- > > GLOBE Development GmbH > > K?nigsberger Strasse 260 > > 48157 M?nsterGLOBE Development GmbH > > K?nigsberger Strasse 260 > > 48157 M?nster > > 0251/5205 390 From bouncingcats at gmail.com Thu Oct 10 09:16:24 2013 From: bouncingcats at gmail.com (David) Date: Fri, 11 Oct 2013 00:16:24 +1100 Subject: Good Python Book In-Reply-To: <52555235.3050100@globe.de> References: <52555235.3050100@globe.de> Message-ID: On 9 October 2013 23:55, Schneider wrote: > > I'm looking for a good advanced python book. Most books I looked at up to > now are on beginners level. > I don't need a reference (that's online) or a book explaining how to use the > interpreter or how to use list comprehensions on the one side and skipping > topics like decorators, metaclasses on the other side. > > any suggestions? https://wiki.python.org/moin/AdvancedBooks From markotaht at gmail.com Wed Oct 9 10:20:43 2013 From: markotaht at gmail.com (markotaht at gmail.com) Date: Wed, 9 Oct 2013 07:20:43 -0700 (PDT) Subject: =?ISO-8859-1?Q?=F6pca=F6_variable_refrenced_before_assignment?= Message-ID: <9610dace-0175-424a-b226-c401ce0f98b2@googlegroups.com> fail4 = "palgad.txt" f4 = open(fail4, "r") def koguarv_ridu failis(f): for i, l in enumerate(f): pass return i+1 def palgad(f4): palgad = 0 while True: f4r = f4.readline() if f4r == "": break palgad += int(f4r[f4r.find(";")+1:]) return palgad def kuu_keskmine(palgad, f): return palgad/koguarv_ridu_failis(f) print(kuu_keskmine(palgad(f4), f4)) Why does it give me local variable "i" referenced before assignment in koguarv_ridu_failis(f) on the return i+1 line But if i do directly koguarv_ridu_failis(f4) then i get correct antswer. From rosuav at gmail.com Wed Oct 9 10:41:57 2013 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 10 Oct 2013 01:41:57 +1100 Subject: =?ISO-8859-1?Q?Re=3A_=F6pca=F6_variable_refrenced_before_assignment?= In-Reply-To: <9610dace-0175-424a-b226-c401ce0f98b2@googlegroups.com> References: <9610dace-0175-424a-b226-c401ce0f98b2@googlegroups.com> Message-ID: On Thu, Oct 10, 2013 at 1:20 AM, wrote: > def koguarv_ridu failis(f): > for i, l in enumerate(f): > pass > return i+1 This will throw the exception you're seeing (by the way, it helps a LOT to actually copy and paste the full error, including the traceback - fortunately I can work this one out without) if the enumerate() doesn't yield any results. The whole loop gets skipped, nothing gets assigned to i. But the real question is: why are you not getting anything to enumerate? > def palgad(f4): > palgad = 0 > while True: > f4r = f4.readline() > if f4r == "": > break > palgad += int(f4r[f4r.find(";")+1:]) > return palgad > > def kuu_keskmine(palgad, f): > return palgad/koguarv_ridu_failis(f) And this would be why. Your first function is consuming the whole file (up to a blank line, but I'm guessing your file doesn't have any), and there's nothing left for ridu to read. But first, a word on naming. You've used the name palgad in four distinct ways: 1) The function introduced in 'def palgad(f4)' 2) A local variable inside #1, which accumulates the returned integer 3) A local variable inside keskmine, which happens to be passed the value that #1 returned 4) The file name, palgad.txt This is not a problem to the interpreter, as they're quite separate, but your first three senses are very confusing to a human. So. You have a major problem here in that you're calculating a number and then trying to divide it by the number of lines. There's a much MUCH simpler, cleaner, _and_ safer way to do that: just count up the lines at the same time as you calculate palgad. I'll let you do the specifics, but that's what I would advise you to explore :) Best of luck! ChrisA From jpiitula at ling.helsinki.fi Wed Oct 9 10:52:10 2013 From: jpiitula at ling.helsinki.fi (Jussi Piitulainen) Date: 09 Oct 2013 17:52:10 +0300 Subject: =?utf-8?b?UmU6IMO2cGNhw7YgdmFyaWE=?= =?utf-8?b?YmxlIHJlZnJlbmNlZCBi?= =?utf-8?b?ZWZvcmUgYXNzaWdubWVu?= =?utf-8?b?dA==?= References: <9610dace-0175-424a-b226-c401ce0f98b2@googlegroups.com> Message-ID: markotaht at gmail.com writes: > fail4 = "palgad.txt" > > f4 = open(fail4, "r") > > def koguarv_ridu failis(f): > for i, l in enumerate(f): > pass > return i+1 > > def palgad(f4): > palgad = 0 > while True: > f4r = f4.readline() > if f4r == "": > break > palgad += int(f4r[f4r.find(";")+1:]) > return palgad > > def kuu_keskmine(palgad, f): > return palgad/koguarv_ridu_failis(f) > > print(kuu_keskmine(palgad(f4), f4)) > > > Why does it give me local variable "i" referenced before assignment > in koguarv_ridu_failis(f) on the return i+1 line Because palgad(f4) consumed f, the loop in koguarv_ridu_failis is not executed even once. > But if i do directly koguarv_ridu_failis(f4) then i get correct > antswer. Try to do just koguarv_ridu_failis(f4) twice. You'll get the same error on the second attempt. From markotaht at gmail.com Wed Oct 9 11:15:16 2013 From: markotaht at gmail.com (markotaht at gmail.com) Date: Wed, 9 Oct 2013 08:15:16 -0700 (PDT) Subject: =?ISO-8859-1?Q?Re=3A_=F6pca=F6_variable_refrenced_before_assignment?= In-Reply-To: <9610dace-0175-424a-b226-c401ce0f98b2@googlegroups.com> References: <9610dace-0175-424a-b226-c401ce0f98b2@googlegroups.com> Message-ID: So i got it working, by saving palgad in a variable, before printing it and i count the lines into a global variable. Ty From breamoreboy at yahoo.co.uk Wed Oct 9 11:37:16 2013 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Wed, 09 Oct 2013 16:37:16 +0100 Subject: =?ISO-8859-1?Q?=F6pca=F6_variable_refrenced_before_a?= =?ISO-8859-1?Q?ssignment?= In-Reply-To: References: <9610dace-0175-424a-b226-c401ce0f98b2@googlegroups.com> Message-ID: On 09/10/2013 16:15, markotaht at gmail.com wrote: > So i got it working, by saving palgad in a variable, before printing it and i count the lines into a global variable. Ty > You are hereby placed in detention for one hour this evening. You will spend the whole hour writing repeatedly "I must remember to place things in context when replying to the Python main mailing list/news group". Do you understand this? -- Roses are red, Violets are blue, Most poems rhyme, But this one doesn't. Mark Lawrence From walterhurry at lavabit.com Wed Oct 9 10:41:53 2013 From: walterhurry at lavabit.com (Walter Hurry) Date: Wed, 9 Oct 2013 14:41:53 +0000 (UTC) Subject: UnicodeEncodeError: SOLVED Message-ID: Many thanks to those prepared to forgive my transgression in the 'Goodbye' thread. I mentioned there that I was puzzled by a UnicodeEncodeError, and said I would rise it as a separate thread. However, via this link, I was able to resolve the issue myself: http://stackoverflow.com/questions/3224268/python-unicode-encode-error Nevertheless, thanks again for the kind words. From steve+comp.lang.python at pearwood.info Wed Oct 9 21:47:52 2013 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: 10 Oct 2013 01:47:52 GMT Subject: UnicodeEncodeError: SOLVED References: Message-ID: <52560747$0$29984$c3e8da3$5496439d@news.astraweb.com> On Wed, 09 Oct 2013 14:41:53 +0000, Walter Hurry wrote: > Many thanks to those prepared to forgive my transgression in the > 'Goodbye' thread. I mentioned there that I was puzzled by a > UnicodeEncodeError, and said I would rise it as a separate thread. > > However, via this link, I was able to resolve the issue myself: > > http://stackoverflow.com/questions/3224268/python-unicode-encode-error I don't know what problem you had, and what your solution was, but the above link doesn't solve the problem, it just throws away data until the problem no longer appears, and never mind if it changes the semantics of the XML data. Instead of throwing away data, the right solution is likely to be, stop trying to deal with XML yourself, and use a proper UTF-8 compliant XML library. Or if you can't do that, at least open and read the XML file using UTF-8 in the first place. In Python 3, you can pass a codec to open. In Python 2, you can use codecs.open instead of the built-in open. -- Steven From walterhurry at lavabit.com Thu Oct 10 17:10:17 2013 From: walterhurry at lavabit.com (Walter Hurry) Date: Thu, 10 Oct 2013 21:10:17 +0000 (UTC) Subject: UnicodeEncodeError: SOLVED References: < 52560747$0$29984$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Thu, 10 Oct 2013 01:47:52 +0000, Steven D'Aprano wrote: > On Wed, 09 Oct 2013 14:41:53 +0000, Walter Hurry wrote: > >> Many thanks to those prepared to forgive my transgression in the >> 'Goodbye' thread. I mentioned there that I was puzzled by a >> UnicodeEncodeError, and said I would rise it as a separate thread. >> >> However, via this link, I was able to resolve the issue myself: >> >> http://stackoverflow.com/questions/3224268/python-unicode-encode-error > > I don't know what problem you had, and what your solution was, but the > above link doesn't solve the problem, it just throws away data until the > problem no longer appears, and never mind if it changes the semantics of > the XML data. > > Instead of throwing away data, the right solution is likely to be, stop > trying to deal with XML yourself, and use a proper UTF-8 compliant XML > library. > > Or if you can't do that, at least open and read the XML file using UTF-8 > in the first place. In Python 3, you can pass a codec to open. In Python > 2, you can use codecs.open instead of the built-in open. All true, but in *this* case, simply discarding the offending character was sufficient. Thanks anyway. From skip at pobox.com Wed Oct 9 11:15:49 2013 From: skip at pobox.com (Skip Montanaro) Date: Wed, 9 Oct 2013 10:15:49 -0500 Subject: datetime.timedelta.replace? Message-ID: Datetime objects have a replace method, but timedelta objects don't. If I take the diff of two datetimes and want to zero out the microseconds field, is there some way to do it more cleanly than this? delta = dt1 - dt2 zero_delta = datetime.timedelta(days=delta.days, seconds=delta.seconds) I guess that's not bad, but replace() seems cleaner (or at least more congruent with datetime objects). Skip From joshua at landau.ws Fri Oct 11 13:56:18 2013 From: joshua at landau.ws (Joshua Landau) Date: Fri, 11 Oct 2013 18:56:18 +0100 Subject: datetime.timedelta.replace? In-Reply-To: References: Message-ID: On 9 October 2013 16:15, Skip Montanaro wrote: > Datetime objects have a replace method, but timedelta objects don't. > If I take the diff of two datetimes and want to zero out the > microseconds field, is there some way to do it more cleanly than this? > > delta = dt1 - dt2 > zero_delta = datetime.timedelta(days=delta.days, seconds=delta.seconds) > > I guess that's not bad, but replace() seems cleaner (or at least more > congruent with datetime objects). Maybe one of delta - datetime.timedelta(0, 0, delta.microseconds) or delta - delta % datetime.timedelta(seconds=1) is clearer. From tspiegelman at amplify.com Wed Oct 9 11:37:39 2013 From: tspiegelman at amplify.com (tspiegelman at amplify.com) Date: Wed, 9 Oct 2013 08:37:39 -0700 (PDT) Subject: Receive packet using socket Message-ID: <11f1aa94-8d04-40da-a0da-48fcbb7e3e13@googlegroups.com> Hey all, I am trying to use socket to send / receive a packet (want to recreate some functionality of hping3 and port it to windows and mac as a tcp ping). I am having some problems with the recv functionality of socket. Below is the script I am using. I get an ack from the server (used wireshark to ensure it was working) when I run this, but the script doesn't see the ack for some reason and the script exits with this error or a timeout: Traceback (most recent call last): File "./tcpgcmtesttristedit.py", line 21, in s.recv(1024) socket.error: [Errno 104] Connection reset by peer Here is the script: import socket import time numberofpackets = int(raw_input("How many packets should be sent?\n> ")) hostname = 'mtalk.google.com' port = 5228 for i in range(numberofpackets): s = socket.socket(socket.AF_INET, socket.SOCK_STREAM) s.settimeout(4) s.connect((hostname, port)) s.send('data') start_time = time.time() s.recv(24) print time.time() - start_time s.close() Any help would be much appreciated. Thanks, Tom. From tspiegelman at amplify.com Wed Oct 9 12:07:07 2013 From: tspiegelman at amplify.com (tspiegelman at amplify.com) Date: Wed, 9 Oct 2013 09:07:07 -0700 (PDT) Subject: Receive packet using socket In-Reply-To: <11f1aa94-8d04-40da-a0da-48fcbb7e3e13@googlegroups.com> References: <11f1aa94-8d04-40da-a0da-48fcbb7e3e13@googlegroups.com> Message-ID: <55af799c-806b-4506-9002-10dd02d79408@googlegroups.com> BTW what I am trying to accomplish is easily done in hping3 using this command: hping3 mtalk.google.com -S -p 5228 I just want those same kind of results using python so I can make an exe out of it. On Wednesday, October 9, 2013 11:37:39 AM UTC-4, tspie... at amplify.com wrote: > Hey all, > > > > I am trying to use socket to send / receive a packet (want to recreate some functionality of hping3 and port it to windows and mac as a tcp ping). I am having some problems with the recv functionality of socket. Below is the script I am using. I get an ack from the server (used wireshark to ensure it was working) when I run this, but the script doesn't see the ack for some reason and the script exits with this error or a timeout: > > > > Traceback (most recent call last): > > File "./tcpgcmtesttristedit.py", line 21, in > > s.recv(1024) > > socket.error: [Errno 104] Connection reset by peer > > > > Here is the script: > > > > import socket > > import time > > > > numberofpackets = int(raw_input("How many packets should be sent?\n> ")) > > hostname = 'mtalk.google.com' > > port = 5228 > > > > for i in range(numberofpackets): > > s = socket.socket(socket.AF_INET, socket.SOCK_STREAM) > > s.settimeout(4) > > s.connect((hostname, port)) > > s.send('data') > > start_time = time.time() > > s.recv(24) > > print time.time() - start_time > > s.close() > > > > > > Any help would be much appreciated. Thanks, Tom. From nobody at nowhere.com Wed Oct 9 13:03:58 2013 From: nobody at nowhere.com (Nobody) Date: Wed, 09 Oct 2013 18:03:58 +0100 Subject: Receive packet using socket References: <11f1aa94-8d04-40da-a0da-48fcbb7e3e13@googlegroups.com> Message-ID: On Wed, 09 Oct 2013 08:37:39 -0700, tspiegelman wrote: > I am trying to use socket to send / receive a packet (want to recreate > some functionality of hping3 and port it to windows and mac as a tcp > ping). I am having some problems with the recv functionality of socket. > Below is the script I am using. I get an ack from the server (used > wireshark to ensure it was working) when I run this, but the script > doesn't see the ack for some reason and the script exits with this error > or a timeout: > s = socket.socket(socket.AF_INET, socket.SOCK_STREAM) If you're trying to send/receive individual packets, you need to use SOCK_RAW (which normally requires root/administrator privilege). > s.recv(24) Alternatively, if you use a normal TCP stream socket, the receiver must consume all data which is sent to it, and only close the socket after the writer has closed its end. Depending upon the application-layer protocol, this may involve some form of QUIT command, or a half-close using the .shutdown() method. From no-reply at thanks.com Tue Oct 22 00:01:34 2013 From: no-reply at thanks.com (john pierce) Date: Mon, 21 Oct 2013 23:01:34 -0500 Subject: Receive packet using socket Message-ID: <5265f89f$0$58729$c3e8da3$12bcf670@news.astraweb.com> Tom wrote: BTW what I am trying to accomplish is easily done in hping3 using this comm= and: hping3 mtalk.google.com -S -p 5228=20 I just want those same kind of results using python so I can make an exe ou= t of it. Hi Tom, Not sure if it's exactly what you're looking for, but I wrote a tcp syn scanner using raw sockets and another that uses scapy. You can take a look at http://www.arti-sec.com/article/spse-module-2-lesson-4-syn-scanner-python. You might take a look at scapy if you don't want to reinvent the wheel like I did. scapy is definitely worth a look if you want something powerfull and aren't interested in reinventing the wheel. Regarding your code/initial question, I think it may be that the server is resetting the connection when it receives the string 'Data' as opposed to something that looks like what it expects. --------------= Posted using GrabIt =---------------- ------= Binary Usenet downloading made easy =--------- -= Get GrabIt for free from http://www.shemes.com/ =- From marco.buttu at gmail.com Wed Oct 9 11:44:50 2013 From: marco.buttu at gmail.com (Marco Buttu) Date: Wed, 09 Oct 2013 17:44:50 +0200 Subject: super in Python 3 and variadic arguments Message-ID: Given this class: >>> class A: ... def afoo(*args): ... print(args) in Python 3 we can write the following class: >>> class B(A): ... def bfoo(*args): ... super(B, args[0]).afoo(*args[1:]) ... >>> B().bfoo(1, 2, 3) (<__main__.B object at 0x7f5b3bde48d0>, 1, 2, 3) without giving arguments to super, in this way: >>> class B(A): ... def bfoo(self, *args): ... super().afoo(*args) ... >>> B().bfoo(1, 2, 3) (<__main__.B object at 0x7f5b3bdea0d0>, 1, 2, 3) But it does not work in this case: >>> class B(A): ... def bfoo(*args): ... super().afoo(*args[1:]) ... >>> B().bfoo(1, 2, 3) Traceback (most recent call last): File "", line 1, in File "", line 3, in bfoo RuntimeError: super(): no arguments How come? -- Marco Buttu From ned at nedbatchelder.com Wed Oct 9 12:47:13 2013 From: ned at nedbatchelder.com (Ned Batchelder) Date: Wed, 09 Oct 2013 12:47:13 -0400 Subject: super in Python 3 and variadic arguments In-Reply-To: References: Message-ID: <52558891.4050607@nedbatchelder.com> On 10/9/13 11:44 AM, Marco Buttu wrote: > Given this class: > > >>> class A: > ... def afoo(*args): > ... print(args) > > in Python 3 we can write the following class: > > >>> class B(A): > ... def bfoo(*args): > ... super(B, args[0]).afoo(*args[1:]) > ... > >>> B().bfoo(1, 2, 3) > (<__main__.B object at 0x7f5b3bde48d0>, 1, 2, 3) > > > without giving arguments to super, in this way: > > >>> class B(A): > ... def bfoo(self, *args): > ... super().afoo(*args) > ... > >>> B().bfoo(1, 2, 3) > (<__main__.B object at 0x7f5b3bdea0d0>, 1, 2, 3) > > But it does not work in this case: > > >>> class B(A): > ... def bfoo(*args): > ... super().afoo(*args[1:]) > ... > >>> B().bfoo(1, 2, 3) > Traceback (most recent call last): > File "", line 1, in > File "", line 3, in bfoo > RuntimeError: super(): no arguments > > How come? The no-args super() call inspects the calling environment to determine the class and self. "self" is the first local name stored in frame.f_code.co_localsplus, but *args doesn't put "args" into that entry of the code object. Basically, super() is looking for the first regular argument in the function. --Ned. From marco.buttu at gmail.com Thu Oct 10 03:22:00 2013 From: marco.buttu at gmail.com (Marco Buttu) Date: Thu, 10 Oct 2013 09:22:00 +0200 Subject: super in Python 3 and variadic arguments References: Message-ID: <52565598.6000709@gmail.com> On 10/09/2013 06:47 PM, Ned Batchelder wrote: >> >>> class B(A): >> ... def bfoo(*args): >> ... super().afoo(*args[1:]) >> ... >> >>> B().bfoo(1, 2, 3) >> Traceback (most recent call last): >> File "", line 1, in >> File "", line 3, in bfoo >> RuntimeError: super(): no arguments >> >> How come? > > The no-args super() call inspects the calling environment to determine > the class and self. "self" is the first local name stored in > frame.f_code.co_localsplus, but *args doesn't put "args" into that entry > of the code object But is it a bug or the behavior we want? The first (implicit) argument is stored as expected as the first one in the args tuple, and the args tuple is inserted as expected in frame.f_locals: >>> import inspect >>> class B(A): ... def bfoo(*args): ... frame = inspect.currentframe() ... for obj, value in frame.f_locals.items(): ... print(obj, value, sep=' --> ') ... # super().afoo(*args[1:]) ... >>> B().bfoo(1, 2, 3) args --> (<__main__.B object at 0x7f28c960a590>, 1, 2, 3) frame --> So, why does not super use it? -- Marco Buttu From ned at nedbatchelder.com Thu Oct 10 07:04:38 2013 From: ned at nedbatchelder.com (Ned Batchelder) Date: Thu, 10 Oct 2013 07:04:38 -0400 Subject: super in Python 3 and variadic arguments In-Reply-To: <52565598.6000709@gmail.com> References: <52565598.6000709@gmail.com> Message-ID: <525689C6.9010302@nedbatchelder.com> On 10/10/13 3:22 AM, Marco Buttu wrote: > On 10/09/2013 06:47 PM, Ned Batchelder wrote: > >>> >>> class B(A): >>> ... def bfoo(*args): >>> ... super().afoo(*args[1:]) >>> ... >>> >>> B().bfoo(1, 2, 3) >>> Traceback (most recent call last): >>> File "", line 1, in >>> File "", line 3, in bfoo >>> RuntimeError: super(): no arguments >>> >>> How come? >> >> The no-args super() call inspects the calling environment to determine >> the class and self. "self" is the first local name stored in >> frame.f_code.co_localsplus, but *args doesn't put "args" into that entry >> of the code object > > But is it a bug or the behavior we want? The first (implicit) argument > is stored as expected as the first one in the args tuple, and the args > tuple is inserted as expected in frame.f_locals: > > >>> import inspect > >>> class B(A): > ... def bfoo(*args): > ... frame = inspect.currentframe() > ... for obj, value in frame.f_locals.items(): > ... print(obj, value, sep=' --> ') > ... # super().afoo(*args[1:]) > ... > >>> B().bfoo(1, 2, 3) > args --> (<__main__.B object at 0x7f28c960a590>, 1, 2, 3) > frame --> > > So, why does not super use it? > I haven't seen the discussion that decided the behavior of super(), but I'd guess that if you reported this as a bug, it would be closed as wontfix, because: 1) the use case you describe isn't something people actually write, 2) it would add to the complexity of super() to support it, and 3) there's a simple way to write your code that does work: class B(A): def bfoo(self, *args): super().afoo(*args) (though it's a bit odd to call afoo from bfoo.) Python has never claimed the kind of purity that makes everything work in a totally simple consistent way. super() with no args is a kind of hack to begin with. It involves a special case in the compiler (so that using the name "super" as a function call will act as if you had accessed the name "__class__" so that super can find it later), and inspecting the stack frame during execution. It's an interesting case of the Zen of Python. It violates one ("explicit is better than implicit"), but only because of another one ("practicality beats purity"). super(MyClass, self) in Python 2 is the kind of brain-bender that so many people get wrong at first, that it's helped plenty of people to do the arguments implicitly, even if there are oddball edge cases that it doesn't seem to handle properly. --Ned. From marco.buttu at gmail.com Thu Oct 10 08:54:10 2013 From: marco.buttu at gmail.com (Marco Buttu) Date: Thu, 10 Oct 2013 14:54:10 +0200 Subject: super in Python 3 and variadic arguments References: <52565598.6000709@gmail.com> Message-ID: On 10/10/2013 01:04 PM, Ned Batchelder wrote: > On 10/10/13 3:22 AM, Marco Buttu wrote: >> >>> import inspect >> >>> class B(A): >> ... def bfoo(*args): >> ... frame = inspect.currentframe() >> ... for obj, value in frame.f_locals.items(): >> ... print(obj, value, sep=' --> ') >> ... # super().afoo(*args[1:]) >> ... >> >>> B().bfoo(1, 2, 3) >> args --> (<__main__.B object at 0x7f28c960a590>, 1, 2, 3) >> frame --> >> >> So, why does not super use it? >> > > Python has never claimed the kind of purity that makes everything work > in a totally simple consistent way. super() with no args is a kind of > hack to begin with. It involves a special case in the compiler (so that > using the name "super" as a function call will act as if you had > accessed the name "__class__" so that super can find it later), and > inspecting the stack frame during execution. It seems reasonable > It's an interesting case of the Zen of Python. It violates one > ("explicit is better than implicit"), but only because of another one > ("practicality beats purity"). super(MyClass, self) in Python 2 is the > kind of brain-bender that so many people get wrong at first, that it's > helped plenty of people to do the arguments implicitly, even if there > are oddball edge cases that it doesn't seem to handle properly. > > --Ned. Thanks for the comprehensive answer ;) -- Marco Buttu From steve+comp.lang.python at pearwood.info Thu Oct 10 22:11:19 2013 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: 11 Oct 2013 02:11:19 GMT Subject: super in Python 3 and variadic arguments References: <52565598.6000709@gmail.com> Message-ID: <52575e47$0$29984$c3e8da3$5496439d@news.astraweb.com> On Thu, 10 Oct 2013 07:04:38 -0400, Ned Batchelder wrote: > super() with no args is a kind of hack to begin with. It involves a > special case in the compiler (so that using the name "super" as a > function call will act as if you had accessed the name "__class__" so > that super can find it later), and inspecting the stack frame during > execution. super() with no arguments is *completely* a hack[1], and one where GvR has said "Never again!" if I remember correctly. I don't think he regrets allowing the super compile-time magic, just that it really is magic and he doesn't want to make a habit of it. One of the side-effects of this being a hack is that this doesn't work: class X(Y): def method(self, arg): f = super f().method(arg) [1] Which is not necessarily a bad thing! -- Steven From ian.g.kelly at gmail.com Thu Oct 10 22:33:37 2013 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Thu, 10 Oct 2013 20:33:37 -0600 Subject: super in Python 3 and variadic arguments In-Reply-To: <52575e47$0$29984$c3e8da3$5496439d@news.astraweb.com> References: <52565598.6000709@gmail.com> <52575e47$0$29984$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Thu, Oct 10, 2013 at 8:11 PM, Steven D'Aprano wrote: > One of the side-effects of this being a hack is that this doesn't work: > > class X(Y): > def method(self, arg): > f = super > f().method(arg) Actually, that works just fine. The compiler sees that super is accessed within the method and creates the closure necessary to make it work. This does fail, however: f = super class X(Y): def method(self, arg): f().method(arg) From steve+comp.lang.python at pearwood.info Thu Oct 10 23:00:03 2013 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: 11 Oct 2013 03:00:03 GMT Subject: super in Python 3 and variadic arguments References: <52565598.6000709@gmail.com> <52575e47$0$29984$c3e8da3$5496439d@news.astraweb.com> Message-ID: <525769b2$0$29984$c3e8da3$5496439d@news.astraweb.com> On Thu, 10 Oct 2013 20:33:37 -0600, Ian Kelly wrote: > On Thu, Oct 10, 2013 at 8:11 PM, Steven D'Aprano > wrote: >> One of the side-effects of this being a hack is that this doesn't work: >> >> class X(Y): >> def method(self, arg): >> f = super >> f().method(arg) > > Actually, that works just fine. The compiler sees that super is > accessed within the method and creates the closure necessary to make it > work. This does fail, however: > > f = super > class X(Y): > def method(self, arg): > f().method(arg) Ah, that's the one! Thanks for the correction. I'll now go and write "I will always test my code snippets before posting" on the blackboard one hundred times. -- Steven From rosuav at gmail.com Fri Oct 11 02:08:45 2013 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 11 Oct 2013 17:08:45 +1100 Subject: super in Python 3 and variadic arguments In-Reply-To: <525769b2$0$29984$c3e8da3$5496439d@news.astraweb.com> References: <52565598.6000709@gmail.com> <52575e47$0$29984$c3e8da3$5496439d@news.astraweb.com> <525769b2$0$29984$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Fri, Oct 11, 2013 at 2:00 PM, Steven D'Aprano wrote: > I'll now go and write "I will always test my code snippets before > posting" on the blackboard one hundred times. print("I will always test my code snippets before posting\n"*100) ChrisA PS. Irony would be having a bug in that because I didn't test it. I almost didn't, but remembered just before hitting Send. From marco.buttu at gmail.com Fri Oct 11 02:17:45 2013 From: marco.buttu at gmail.com (Marco Buttu) Date: Fri, 11 Oct 2013 08:17:45 +0200 Subject: super in Python 3 and variadic arguments References: <52565598.6000709@gmail.com> <52575e47$0$29984$c3e8da3$5496439d@news.astraweb.com> Message-ID: <52579809.1040904@gmail.com> On 10/11/2013 04:33 AM, Ian Kelly wrote: > On Thu, Oct 10, 2013 at 8:11 PM, Steven D'Aprano >> >One of the side-effects of this being a hack is that this doesn't work: >> > >> >class X(Y): >> > def method(self, arg): >> > f = super >> > f().method(arg) > Actually, that works just fine. The compiler sees that super is > accessed within the method and creates the closure necessary to make > it work. This does fail, however: > > f = super > class X(Y): > def method(self, arg): > f().method(arg) Very interesting! Thanks :) -- Marco Buttu From marco.buttu at gmail.com Fri Oct 11 02:15:18 2013 From: marco.buttu at gmail.com (Marco Buttu) Date: Fri, 11 Oct 2013 08:15:18 +0200 Subject: super in Python 3 and variadic arguments References: <52565598.6000709@gmail.com> <52575e47$0$29984$c3e8da3$5496439d@news.astraweb.com> Message-ID: On 10/11/2013 04:11 AM, Steven D'Aprano wrote: > super() with no arguments is*completely* a hack[1], and one where GvR > has said "Never again!" if I remember correctly. I don't think he regrets > allowing the super compile-time magic, just that it really is magic and > he doesn't want to make a habit of it. > ... > [1] Which is not necessarily a bad thing! Thanks a lot for this anecdote :) -- Marco Buttu From real-not-anti-spam-address at apple-juice.co.uk Wed Oct 9 15:33:21 2013 From: real-not-anti-spam-address at apple-juice.co.uk (D.M. Procida) Date: Wed, 9 Oct 2013 20:33:21 +0100 Subject: PROPOSAL: PyCons in Africa Message-ID: <1lahqe8.1tnm9o4d2o04rN%real-not-anti-spam-address@apple-juice.co.uk> [reposted; the previous one didn't seem to make it out!] I have a written a first draft outlining a proposal for a PyCon in a sub-Saharan African nation where there has never been one. There's an email list for people interested in becoming involved in the idea: . I'm seeking support for the proposal, from all parties who'd need to be involved, from Python user groups in African cities where a PyCon might be viable, to people who'd want to take part and the wider Python community. If anyone would like to add themselves to the list of potential collaborators on this, please do so using GitHub: . If you can't do that, just ask me to add the information. There's a wiki on GitHub too: . I'm also seeking critical opinions on this, so if you think it's a poor idea, or poorly conceived, or that there would be better ways to achieve the aims in the proposal, or something key that's missing from it, I would like to hear why you think that, so I can address any problems. Thanks, Daniele From errol.anderson at gradient.com Wed Oct 9 17:47:54 2013 From: errol.anderson at gradient.com (Errol Anderson) Date: Wed, 9 Oct 2013 21:47:54 +0000 Subject: Can anyone help on conflicts between Python 2.5 and 2.7 Message-ID: <162dd5ac741c4aa9b1f50289bbfc67e1@BY2PR06MB172.namprd06.prod.outlook.com> I maintain a Delphi program, AAA, that runs Python 2.5 scripts using the PythonForDelphi (P4D)interface. I can install both Python 2.5 and Python 2.7 on my computer and AAA is unaffected. However one user of AAA uses another program, BBB, that requires Python 2.7. When they run AAA, an error is generated that suggests that AAA has been directed to the Python 2.7 libraries. The specific error is identified in Python27\lib\linecache.py line 127 with open(fullname, 'rU') as fp: as compared with Python25\lib\linecache.py line 128 fp = open(fullname, 'rU') It appears that the BBB program installs Python 2.7 to be the "default" Python version, although I do not know how this is done. Assuming BBB cannot be changed, I would like to know how I can modify AAA so that it ignores any "default" settings and simply runs Python 2.5. Regards Errol Anderson -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve+comp.lang.python at pearwood.info Wed Oct 9 21:31:31 2013 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: 10 Oct 2013 01:31:31 GMT Subject: Can anyone help on conflicts between Python 2.5 and 2.7 References: Message-ID: <52560373$0$29984$c3e8da3$5496439d@news.astraweb.com> Hi Errol, Happy to help, but first I have a brief note about house-keeping... this group is both a mailing list and a newsgroup on Usenet. A text newsgroup, so I'm afraid that HTML posts are frowned upon, because a large number of people reading this will see your message something like this: > xmlns:o="urn:schemas-microsoft-com:office:office" > xmlns:w="urn:schemas-microsoft-com:office:word" > xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" > xmlns="http://www.w3.org/TR/REC-html40"> > > >