From oscar.j.benjamin at gmail.com Mon Feb 1 07:42:52 2016 From: oscar.j.benjamin at gmail.com (Oscar Benjamin) Date: Mon, 1 Feb 2016 12:42:52 +0000 Subject: Error : 0x80070570 file or directory is corrupted or unavailable In-Reply-To: References: Message-ID: On 31 January 2016 at 07:26, archi dsouza wrote: > I was trying to install Python.exe in windows 8.1. But got error mention in > subject line. find attached log file. This particular error message has been reported here before: https://mail.python.org/pipermail/python-list/2015-September/697456.html It's not clear to me whether it is a Python problem or a disk problem or what. What version are you trying to install? Is it Python 3.5? Does it work if you try to install Python 3.4 instead? If so then this may be a bug in Python 3.5 on Windows and it needs to be reported so it can be fixed. -- Oscar From jason.swails at gmail.com Mon Feb 1 09:18:41 2016 From: jason.swails at gmail.com (Jason Swails) Date: Mon, 1 Feb 2016 09:18:41 -0500 Subject: ts.plot() pandas: No plot! In-Reply-To: References: Message-ID: On Sun, Jan 31, 2016 at 9:08 PM, Paulo da Silva < p_s_d_a_s_i_l_v_a_ns at netcabo.pt> wrote: > ?s 01:43 de 01-02-2016, Mark Lawrence escreveu: > > On 01/02/2016 00:46, Paulo da Silva wrote: > ... > > >> > > > > Is it as simple as adding a call to ts.show() ? > > > Thanks for the clue! > Not so simple however. > Needed to do > import matplotlib.pyplot as plt > plt.show() > ?What you saw ts.plot() return was the matplotlib artists (the things that will be drawn on whatever "canvas" is provided -- either saved to an image or drawn to a GUI widget). So whenever you see this kind of return value, you know you need to call the matplotlib.pyplot.show function in order to generate a canvas widget (with whatever backend you choose) and draw it. If you want to do this kind of interactive plotting (reminiscent, I've heard, of Matlab), I would highly recommend checking out IPython. You can use IPython's notebook or qtconsole and embed plots from matplotlib directly in the viewer. For example, try this: ipython qtconsole This opens up a window, then use the magic command "%matplotlib inline" to have all plots sent directly to the ipython console you are typing commands in. I've found that kind of workflow quite convenient for directly interacting with data. HTH, Jason From srkunze at mail.de Mon Feb 1 12:24:07 2016 From: srkunze at mail.de (Sven R. Kunze) Date: Mon, 1 Feb 2016 18:24:07 +0100 Subject: Heap Implementation In-Reply-To: References: <56AD3D83.2050308@mail.de> Message-ID: <56AF94B7.9050203@mail.de> On 31.01.2016 05:59, srinivas devaki wrote: > @Sven > actually you are not sweeping at all, as i remember from my last post > what i meant by sweeping is periodically deleting the elements which > were marked as popped items. Exactly. Maybe I didn't express myself well. Would you prefer the sweeping approach in terms of efficiency over how I implemented xheap currently? Without running some benchmarks, I have absolutely no feeling which approach is faster/more memory efficient etc. > kudos on that __setitem__ technique, > instead of using references to the items like in HeapDict, it is > brilliant of you to simply use __setitem__ Thanks. :) > On Sun, Jan 31, 2016 at 4:17 AM, Sven R. Kunze wrote: >> Hi again, >> >> as the topic of the old thread actually was fully discussed, I dare to open >> a new one. >> >> I finally managed to finish my heap implementation. You can find it at >> https://pypi.python.org/pypi/xheap + https://github.com/srkunze/xheap. >> >> I described my motivations and design decisions at >> http://srkunze.blogspot.com/2016/01/fast-object-oriented-heap-implementation.html >> >> @Cem >> You've been worried about a C implementation. I can assure you that I did >> not intend to rewrite the incredibly fast and well-tested heapq >> implementation. I just re-used it. >> >> I would really be grateful for your feedback as you have first-hand >> experience with heaps. >> >> @srinivas >> You might want to have a look at the removal implementation. Do you think it >> would be wiser/faster to switch for the sweeping approach? >> >> I plan to publish some benchmarks to compare heapq and xheap. >> >> @all >> What's the best/standardized tool in Python to perform benchmarking? Right >> now, I use a self-made combo of unittest.TestCase and time.time + proper >> formatting. >> >> Best, >> Sven >> >> >> PS: fixing some weird typos and added missing part. From vincent at vincentdavis.net Mon Feb 1 13:40:26 2016 From: vincent at vincentdavis.net (Vincent Davis) Date: Mon, 1 Feb 2016 11:40:26 -0700 Subject: x=something, y=somethinelse and z=crud all likely to fail - how do i wrap them up In-Reply-To: References: Message-ID: On Sat, Jan 30, 2016 at 9:58 PM, Veek. M wrote: > Is there some other nice way to wrap this stuff up? > I can't do: > try: > x= > y= > z= > except: > I happend to Have just been doing the something similar. You can put x,y,x in a list and loop over it. In my case a dict was better. See the example here. https://github.com/vincentdavis/USAC_data/blob/master/tools.py#L24 Vincent Davis 720-301-3003 From p_s_d_a_s_i_l_v_a_ns at netcabo.pt Mon Feb 1 14:22:08 2016 From: p_s_d_a_s_i_l_v_a_ns at netcabo.pt (Paulo da Silva) Date: Mon, 1 Feb 2016 19:22:08 +0000 Subject: ts.plot() pandas: No plot! References: Message-ID: ?s 14:18 de 01-02-2016, Jason Swails escreveu: > On Sun, Jan 31, 2016 at 9:08 PM, Paulo da Silva < > p_s_d_a_s_i_l_v_a_ns at netcabo.pt> wrote: > >> ?s 01:43 de 01-02-2016, Mark Lawrence escreveu: >>> On 01/02/2016 00:46, Paulo da Silva wrote: ... > > ?What you saw ts.plot() return was the matplotlib artists (the things that > will be drawn on whatever "canvas" is provided -- either saved to an image > or drawn to a GUI widget). So whenever you see this kind of return value, > you know you need to call the matplotlib.pyplot.show function in order to > generate a canvas widget (with whatever backend you choose) and draw it. > > If you want to do this kind of interactive plotting (reminiscent, I've > heard, of Matlab), I would highly recommend checking out IPython. You can > use IPython's notebook or qtconsole and embed plots from matplotlib > directly in the viewer. For example, try this: > > ipython qtconsole > > This opens up a window, then use the magic command "%matplotlib inline" to > have all plots sent directly to the ipython console you are typing commands > in. I've found that kind of workflow quite convenient for directly > interacting with data. Thank you Jason. This can be very usefull. From ryand.young at menloschool.org Mon Feb 1 14:30:51 2016 From: ryand.young at menloschool.org (Ryan Young) Date: Mon, 1 Feb 2016 11:30:51 -0800 Subject: Python Calculator Message-ID: I am new to Python but have known Java for a few years now. With python, so far, so good! I created a simple calculator to calculate the total cost of a meal. My variables were tip tax total and order. I am confused about how to put in a new 'order' because when i reset the order variable to a different number it still calculates the original value. I have sent you a picture to help you understand my problem. Thank you so much! Ryan Young From srkunze at mail.de Mon Feb 1 14:32:06 2016 From: srkunze at mail.de (Sven R. Kunze) Date: Mon, 1 Feb 2016 20:32:06 +0100 Subject: Heap Implementation In-Reply-To: <56ad6800$0$14486$c3e8da3@news.astraweb.com> References: <56ad6800$0$14486$c3e8da3@news.astraweb.com> Message-ID: <56AFB2B6.5070809@mail.de> On 31.01.2016 02:48, Steven D'Aprano wrote: > On Sunday 31 January 2016 09:47, Sven R. Kunze wrote: > >> @all >> What's the best/standardized tool in Python to perform benchmarking? > timeit Thanks, Steven. Maybe, I am doing it wrong but I get some weird results: >>> min(timeit.Timer('for _ in range(10000): heappop(h)', 'from heapq import heappop; h=list(range(10000000))').repeat(10, 1)), min(timeit.Timer('for _ in range(10000): h.pop()', 'from xheap import Heap; h=Heap(range(10000000))').repeat(10, 1)) (0.017267618000005314, 0.01615345600021101) >>> min(timeit.Timer('for _ in range(100000): heappop(h)', 'from heapq import heappop; h=list(range(10000000))').repeat(10, 1)), min(timeit.Timer('for _ in range(100000): h.pop()', 'from xheap import Heap; h=Heap(range(10000000))').repeat(10, 1)) (0.12321608699949138, 0.13042051299999002) >>> min(timeit.Timer('for _ in range(10000): heappop(h)', 'from heapq import heappop; h=list(range(1000000))').repeat(10, 1)), min(timeit.Timer('for _ in range(10000): h.pop()', 'from xheap import Heap; h=Heap(range(1000000))').repeat(10, 1)) (0.010081621999233903, 0.008791901999757101) >>> min(timeit.Timer('for _ in range(1000000): heappop(h)', 'from heapq import heappop; h=list(range(10000000))').repeat(10, 1)), min(timeit.Timer('for _ in range(1000000): h.pop()', 'from xheap import Heap; h=Heap(range(10000000))').repeat(10, 1)) (0.6218949679996513, 0.7172151949998806) How can it be that my wrapper is sometimes faster and sometimes slower than heapq? I wouldn't mind slower, but faster*? Best, Sven * that behavior is reproducible also for other combos and other machines. From zachary.ware+pylist at gmail.com Mon Feb 1 15:18:15 2016 From: zachary.ware+pylist at gmail.com (Zachary Ware) Date: Mon, 1 Feb 2016 14:18:15 -0600 Subject: Python Calculator In-Reply-To: References: Message-ID: Hi Ryan, On Mon, Feb 1, 2016 at 1:30 PM, Ryan Young wrote: > I am new to Python but have known Java for a few years now. With python, so > far, so good! I created a simple calculator to calculate the total cost of > a meal. My variables were tip tax total and order. I am confused about how > to put in a new 'order' because when i reset the order variable to a > different number it still calculates the original value. I have sent you a > picture to help you understand my problem. Thank you so much! Your picture didn't come through. This is a text-based mailing list gated to a Usenet group, many participants wouldn't receive an attached picture no matter how you tried to do it. Instead, just copy and paste the code you're running and the exact output you get into the body of a message, making sure that the formatting (especially indentation, which is significant in Python) is kept by your email client. The simplest step towards ensuring good formatting is to send your message in plain text mode (rather than HTML). With your code in hand, I'm sure someone here will be able to explain what's going on :) Regards, -- Zach From nathanladuca at gmail.com Mon Feb 1 17:39:58 2016 From: nathanladuca at gmail.com (nathanladuca at gmail.com) Date: Mon, 1 Feb 2016 14:39:58 -0800 (PST) Subject: Sr Python Developer job opening- SA, TX Message-ID: <1592f410-27dd-4a46-a09f-8a49c3833172@googlegroups.com> Let me know if interested... Day to Day: - Write and execute automated test scripts using a pre-defined framework - Writes positive and negative smoke and regression test scripts to test product functionality and integration with dependencies - Tests API's, user interfaces, web services and/or web applications - Writes performance, load, and stress tests utilizing a pre-defined framework - Participates in code reviews - Low to mid level of contribution to Openstack test suites, as applicable - Collaborates with other quality and development engineers to build, evolve, and maintain a scalable continuous build and deployment pipeline Required Skills: - Must know and work with Python - Must possess the ability to understand new concepts quickly, and apply them accurately through an evolving, dynamic environment - Strong knowledge of protocols, networking, and systems - Demonstrated intermediate knowledge of Unix shell scripting - Intermediate knowledge of a UI automation tool such as Selenium, QTP, or Silk - Experience working within an agile development process (Scrum, XP, Kanban, etc.) from the test design, test automation, and execution perspective Required Skills (continued): - Ability to analyze and translate requirements and development stories into test scripts Preferred Skills: - Strong understanding of software/testing methodologies such as TDD - Experience working in Cloud computing From mr.eightnoteight at gmail.com Mon Feb 1 19:48:05 2016 From: mr.eightnoteight at gmail.com (srinivas devaki) Date: Tue, 2 Feb 2016 06:18:05 +0530 Subject: Heap Implementation In-Reply-To: <56AF94B7.9050203@mail.de> References: <56AD3D83.2050308@mail.de> <56AF94B7.9050203@mail.de> Message-ID: On Feb 1, 2016 10:54 PM, "Sven R. Kunze" wrote: > > Maybe I didn't express myself well. Would you prefer the sweeping approach in terms of efficiency over how I implemented xheap currently? > complexity wise your approach is the best one of all that I have seen till now > Without running some benchmarks, I have absolutely no feeling which approach is faster/more memory efficient etc. > this is obviously memory efficient but I don't know whether this approach would be faster than previous approaches, with previous approaches there is no call back into Python code from C code for comparison. but this should be faster than HeapDict as HeapDict is directly using its own methods for heappush, heappop etc PS: if you have time, could you please review my pull request. From foxprone.r52 at gmail.com Mon Feb 1 21:19:57 2016 From: foxprone.r52 at gmail.com (Anup reni) Date: Tue, 2 Feb 2016 07:49:57 +0530 Subject: Data storing In-Reply-To: References: Message-ID: Hi I'm new to python and planning to make a web app which contains a large amount of data and involves comparing two data sets. Which is better to use in python, a Dictionary or Mysql? From steve+comp.lang.python at pearwood.info Mon Feb 1 23:43:52 2016 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Tue, 02 Feb 2016 15:43:52 +1100 Subject: Python Calculator References: Message-ID: <56b0340b$0$1524$c3e8da3$5496439d@news.astraweb.com> Hi Ryan, and welcome! On Tuesday 02 February 2016 06:30, Ryan Young wrote: > I am new to Python but have known Java for a few years now. With python, > so far, so good! I created a simple calculator to calculate the total cost > of a meal. My variables were tip tax total and order. I am confused about > how to put in a new 'order' because when i reset the order variable to a > different number it still calculates the original value. I have sent you a > picture to help you understand my problem. Thank you so much! I don't know what they taught you about Java programming, but Python programming involves typing text into a text editor or IDE, not editing pictures with Photoshop. So if you have a problem, the best way to get a solution is to post *code*, not pictures. Copy and paste the code, don't retype it from memory. For the best results, please read this: http://www.sscce.org/ before re-sending. Thanks, and good luck, -- Steve From steve+comp.lang.python at pearwood.info Tue Feb 2 00:38:11 2016 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Tue, 02 Feb 2016 16:38:11 +1100 Subject: Heap Implementation References: <56ad6800$0$14486$c3e8da3@news.astraweb.com> Message-ID: <56b040c5$0$1590$c3e8da3$5496439d@news.astraweb.com> On Tuesday 02 February 2016 06:32, Sven R. Kunze wrote: > On 31.01.2016 02:48, Steven D'Aprano wrote: >> On Sunday 31 January 2016 09:47, Sven R. Kunze wrote: >> >>> @all >>> What's the best/standardized tool in Python to perform benchmarking? >> timeit > Thanks, Steven. > > Maybe, I am doing it wrong but I get some weird results: You need to understand that in any modern desktop, server or laptop computer (embedded devices may be different) there is *constantly* a large amount of processing going on in the background. Your OS is constantly responding to network events, managing memory, skipping from one process to another, scheduling tasks; your anti-virus and other background programs are working; your window environment is watching the mouse, etc. So each time you run a test, it comes down to pure dumb luck how many other programs are busy at the same time. (You can, sometimes, influence that by quitting all other applications, unplugging the network cable, and keeping your hands off the keyboard and mouse while the test is running. But who does that?) All that adds noise to the measured times. It's not unusual for timings to differ by a factor of ten from one run to the next. The speed will also differ depending on processor cache misses, and many other factors that are effectively impossible to predict with any certainty. This makes timing measurements on modern systems an inherently noisy process. In effect, each measurement you take is made up of two components: * the actual time that the code would take if it had exclusive access to the machine with no other programs running, call it t; * and the noise added by the system, call it ?t. It is impossible to measure t alone, you always get t+?t. The aim in timing is to reduce the effect of the noise as much as possible. The way to do this with timeit is: - don't run extraneous code as part of your test (just measure what you care about, with as little scaffolding around it); - run that code snippet as many times as you can bear in a loop, *but* let timeit manage the loop; don't try doing it by hand; - only care about "best of N", where N is as large a number as you can stand; - averages, standard deviation, and other statistics are pretty much useless here, the only statistic that you care about is the minimum. Out of those four guidelines, you missed three of them: (1) Your test code includes scaffolding, the "for _ in range..." loop. You're timing how expensive it is to build a range object and loop over it. (2) You picked a tiny number for the number of loops: ten. Timeit defaults to one million, which is good for *really* fast and tiny snippets. I normally reduce that to 10000, but run a larger number of trials. If the timing from each trial is too small, I increase the number of loops, and if it takes too long (I am impatient) I decrease it, but never below 100. (3) You then use the repeat() method to calculate the "best of one", which is pretty much pointless. There is a timeit() method for "best of one", if that's ever useful. I normally run 5 or 7 trials, and report only the best (smallest). Here's your code: > >>> min(timeit.Timer('for _ in range(10000): heappop(h)', 'from heapq > import heappop; h=list(range(10000000))').repeat(10, 1)), > min(timeit.Timer('for _ in range(10000): h.pop()', 'from xheap import > Heap; h=Heap(range(10000000))').repeat(10, 1)) > (0.017267618000005314, 0.01615345600021101) Your code would be easier to read and understand if it were split over multiple lines. This is not Perl and there's no advantage to one-liners. Pulling your code apart and re-writing it in a way which I feel is more understandable: t1 = Timer( 'for _ in range(10000): heappop(h)', # code being tested 'from heapq import heappop; h=list(range(10000000))' # setup ) t2 = Timer( 'for _ in range(10000): h.pop()', 'from xheap import Heap; h=Heap(range(10000000))' ) min(t1.repeat(10, 1)) min(t2.repeat(10, 1)) Something like this will probably be less noisy: setup = """ from heapq import heappop from xheap import Heap a = list(range(10000000)) h = Heap(a) """ t1 = Timer("heappop(a)", setup) t2 = Timer("h.pop()", setup) # print best of 7 trials, where each trial runs the code snippet 10000 times print(min(t1.repeat(10000, 7))) print(min(t2.repeat(10000, 7))) The times printed will be in seconds per 10000 runs; divide it by 10 to get the time in *milliseconds* per run. Note that this will *not* eliminate all noise or variation from one timing test to the other, but it will (I hope!) reduce it. If you're still seeing a lot of variation, try turning the garbage collector off and quitting as many other applications as possible. If anyone else can suggest any other techniques for reducing noise, I'd love to hear about them. Also note that times generated by timeit with one version of Python may not be measuring the same thing as those using a different version. I believe that Python 3.4 and better will do a better job of measuring only the time the code snippet process is active, rather than wall-time. But that's also operating system dependent. > How can it be that my wrapper is sometimes faster and sometimes slower > than heapq? I wouldn't mind slower, but faster*? Maybe your code genuinely is "about as fast" as heapq, so the differences you're seeing are just due to noise in the system. -- Steve From dieter at handshake.de Tue Feb 2 03:29:38 2016 From: dieter at handshake.de (dieter) Date: Tue, 02 Feb 2016 09:29:38 +0100 Subject: Data storing References: Message-ID: <87mvrj7bwt.fsf@handshake.de> Anup reni writes: > Hi I'm new to python and planning to make a web app which contains a large > amount of data and involves comparing two data sets. Which is better to use > in python, a Dictionary or Mysql? Is it important for your data to be persistent (i.e. live across restarts of your web app)? If so (and I assume this is the case), then you need some form of persistence. "MySQL" comes with such persistency; with a Python "dict" you would need persistency in addition. For more guidance, a clearer description of "large amount" would be necessary. Python "dict"s usually are maintained in memory; data maintained there should not exceed the order of GB. In a "MySQL" database you can store larger amounts of data. Further considerations can be how you need to access the data sets. Python "dict"s support efficient access by a single key; "MySQL" supports various efficient access paths based on the indexes you define. From steve at pearwood.info Tue Feb 2 05:33:12 2016 From: steve at pearwood.info (Steven D'Aprano) Date: Tue, 02 Feb 2016 21:33:12 +1100 Subject: Data storing References: Message-ID: <56b085e9$0$1593$c3e8da3$5496439d@news.astraweb.com> On Tue, 2 Feb 2016 01:19 pm, Anup reni wrote: > Hi I'm new to python and planning to make a web app which contains a large > amount of data and involves comparing two data sets. Which is better to > use in python, a Dictionary or Mysql? "Hi, I'm planning to transport a large number of objects from one place to another. Which is better for me to use, a ship or an airplane?" :-) Well, it depends. How much data? You might think 100 MB is a lot of data, but Python won't. How are you planning to compare them? What sort of comparisons do you intend to do? If all you have is a set of key:value pairs, then dicts are the obvious choice. They are built-in and very fast, but unordered. They'll be good for millions or tens of millions of records, but they aren't persistent data storage. You will have to load them to and from disk each time the application starts. Perhaps the "shelve" module might help you with that. If you have many different relationships, and you want, or need, to use SQL commands to do some initial data processing, then you will certainly need a real database, not dicts. I would prefer Postgresql rather than MySQL, **especially** if you need to support non-ASCII (Unicode) data. But really, to decide which would be best for you, we would need to know a lot more about your application and its requirements. -- Steven From joel.goldstick at gmail.com Tue Feb 2 07:03:03 2016 From: joel.goldstick at gmail.com (Joel Goldstick) Date: Tue, 2 Feb 2016 07:03:03 -0500 Subject: Data storing In-Reply-To: <56b085e9$0$1593$c3e8da3$5496439d@news.astraweb.com> References: <56b085e9$0$1593$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Tue, Feb 2, 2016 at 5:33 AM, Steven D'Aprano wrote: > On Tue, 2 Feb 2016 01:19 pm, Anup reni wrote: > > > Hi I'm new to python and planning to make a web app which contains a > large > > amount of data and involves comparing two data sets. Which is better to > > use in python, a Dictionary or Mysql? > > "Hi, I'm planning to transport a large number of objects from one place to > another. Which is better for me to use, a ship or an airplane?" > > :-) > > Well, it depends. > > How much data? You might think 100 MB is a lot of data, but Python won't. > > How are you planning to compare them? What sort of comparisons do you > intend > to do? > > If all you have is a set of key:value pairs, then dicts are the obvious > choice. They are built-in and very fast, but unordered. They'll be good for > millions or tens of millions of records, but they aren't persistent data > storage. You will have to load them to and from disk each time the > application starts. Perhaps the "shelve" module might help you with that. > > If you have many different relationships, and you want, or need, to use SQL > commands to do some initial data processing, then you will certainly need a > real database, not dicts. > > I would prefer Postgresql rather than MySQL, **especially** if you need to > support non-ASCII (Unicode) data. > > But really, to decide which would be best for you, we would need to know a > lot more about your application and its requirements. > > > -- > Steven > > -- > https://mail.python.org/mailman/listinfo/python-list > Steven, in your case, how about the star trek transporter? Sorry, its early for me, and I want to be respectful, but questions like this seem to have a little flavor of "who would win? super man or mighty mouse?" -- Joel Goldstick http://joelgoldstick.com/stats/birthdays From Mohamed_Nedal at science.helwan.edu.eg Tue Feb 2 08:16:08 2016 From: Mohamed_Nedal at science.helwan.edu.eg (Mohamed Nedal) Date: Tue, 2 Feb 2016 13:16:08 +0000 Subject: Problem with 'requests' module Message-ID: Hello, I'm learning Python and I'm using Python 3.5.1 Shell to write some code using 'requests' module, but I get this error when I run the code: ImportError: No module named 'requests' Could you please tell me how to fix this issue? Sincerely, Mohamed Nedal MSc student in Solar Physics The Space Weather Monitoring Center (SWMC), Egypt Tel.: +2 - 0109 - 892 - 4860 From arsh.py at gmail.com Tue Feb 2 08:30:23 2016 From: arsh.py at gmail.com (arsh.py at gmail.com) Date: Tue, 2 Feb 2016 05:30:23 -0800 (PST) Subject: Behaviour of list comprehensions Message-ID: <4a9ac629-98fa-4643-af28-8d30b52bbc50@googlegroups.com> I am having some understandable behaviour from one of my function name week_graph_data() which call two functions those return a big tuple of tuples, But in function week_graph_data() line no. 30 does not work(returns no result in graph or error). I have check both functions are called in week_graph_data() individually and those run perfectly. (returning tuple of tuples) here is my code: pastebin.com/ck1uNu0U From steve at pearwood.info Tue Feb 2 08:58:22 2016 From: steve at pearwood.info (Steven D'Aprano) Date: Wed, 03 Feb 2016 00:58:22 +1100 Subject: Behaviour of list comprehensions References: <4a9ac629-98fa-4643-af28-8d30b52bbc50@googlegroups.com> Message-ID: <56b0b601$0$1621$c3e8da3$5496439d@news.astraweb.com> Hi Arsh, and welcome! On Wed, 3 Feb 2016 12:30 am, arsh.py at gmail.com wrote: > I am having some understandable behaviour from one of my function name > week_graph_data() which call two functions those return a big tuple of > tuples, But in function week_graph_data() line no. 30 does not > work(returns no result in graph or error). > > I have check both functions are called in week_graph_data() individually > and those run perfectly. (returning tuple of tuples) > > here is my code: pastebin.com/ck1uNu0U There's no need to waste your time, and other people's time, by making a pastebin. Since it's less than 30 lines, just include it in your email. (Be careful with Gmail that it doesn't kill the indentation. Just make sure you send plain text, with no "rich text" -- actually HTML -- formatting.) def week_list_func(): """ A function returning list of last 6 days in 3-tuple""" date_list = [] for i in xrange(7): d=date.today() - timedelta(i) t = d.year, d.month, d.day date_list.append(t) return reversed(date_list) def week_graph_data(): day_list_inbox = inbox_week() # returns big Tuple of tuples day_list_outbox = outbox_week() # Same as inbox_week() dates_inbox = tuple(a[:-7] for a in day_list_inbox) dates_outbox = tuple(a[:-7] for a in day_list_outbox) week_list = week_list_func() bar_chart = pygal.Bar() bar_chart.title = 'Weekly Email Analysis' bar_chart.x_labels = (list(week_list_func())) bar_chart.add('Received', [sum(t==i for t in dates_outbox) for i in list(week_list)] ) bar_chart.add('Sent', [sum(m==n for n in dates_inbox) for m in list(week_list)] ) week_chart = bar_chart.render(is_unicode = True) return bar_chart.render_to_file('ok_.svg') week_graph_data() Your problem is that week_list_func() returns an iterator, so once you convert it to a list the first time, it becomes empty. For example: py> values = [1, 2, 3, 4, 5, 6, 7] py> days = reversed(values) py> print days py> print list(days) [7, 6, 5, 4, 3, 2, 1] py> print list(days) [] Here, days is an iterator. Once you consume the iterator completely the first time, there's nothing left, so the second time you get an empty list. You can fix this by having week_list_func() return a list or tuple: # Version 1 def week_list_func(): """ A function returning list of last 6 days in 3-tuple""" date_list = [] for i in xrange(7): d=date.today() - timedelta(i) t = d.year, d.month, d.day date_list.append(t) return date_list[::-1] # reverse the list with a slice # Version 2 def week_list_func2(): """ A function returning list of last 6 days in 3-tuple""" date_list = [] for i in xrange(6, -1, -1): d=date.today() - timedelta(i) t = d.year, d.month, d.day date_list.append(t) return date_list Because this function returns an actual list, there is no need to call list() on the result. -- Steven From __peter__ at web.de Tue Feb 2 09:08:19 2016 From: __peter__ at web.de (Peter Otten) Date: Tue, 02 Feb 2016 15:08:19 +0100 Subject: Behaviour of list comprehensions References: <4a9ac629-98fa-4643-af28-8d30b52bbc50@googlegroups.com> Message-ID: arsh.py at gmail.com wrote: > I am having some understandable behaviour from one of my function name > week_graph_data() which call two functions those return a big tuple of > tuples, But in function week_graph_data() line no. 30 does not > work(returns no result in graph or error). > > I have check both functions are called in week_graph_data() individually > and those run perfectly. (returning tuple of tuples) > > here is my code: pastebin.com/ck1uNu0U > def week_list_func(): > """ A function returning list of last 6 days in 3-tuple""" > > date_list = [] > for i in xrange(7): > d=date.today() - timedelta(i) > t = d.year, d.month, d.day > date_list.append(t) > return reversed(date_list) reversed(date_list) returns an "iterator": >>> items = reversed(["a", "b", "c"]) >>> items This means that iteration will "consume" the items >>> list(items) ['c', 'b', 'a'] >>> list(items) [] and thus work only once. To allow for multiple iterations you can return a list instead: >>> items = ["a", "b", "c"] >>> items.reverse() >>> items ['c', 'b', 'a'] >>> list(items) ['c', 'b', 'a'] >>> list(items) ['c', 'b', 'a'] Note that list.reverse() is a mutating method and by convention returns None. Therefore return date_list.reverse() # wrong! will not work, you need two lines date_list.reverse() return date_list in your code. From oscar.j.benjamin at gmail.com Tue Feb 2 09:53:02 2016 From: oscar.j.benjamin at gmail.com (Oscar Benjamin) Date: Tue, 2 Feb 2016 14:53:02 +0000 Subject: Heap Implementation In-Reply-To: <56b040c5$0$1590$c3e8da3$5496439d@news.astraweb.com> References: <56ad6800$0$14486$c3e8da3@news.astraweb.com> <56b040c5$0$1590$c3e8da3$5496439d@news.astraweb.com> Message-ID: On 2 February 2016 at 05:38, Steven D'Aprano wrote: > > In effect, each measurement you take is made up of two components: > > * the actual time that the code would take if it had exclusive > access to the machine with no other programs running, call it t; > > * and the noise added by the system, call it ?t. > > It is impossible to measure t alone, you always get t+?t. The aim in timing > is to reduce the effect of the noise as much as possible. That's a reasonable model in some cases but not all. In particular the "actual time" part is itself not a constant but rather depends on all sorts of things like RAM caching etc. On the CPython side there could be all kinds of optimisations that would kick in at different times to change the "actual time". Likewise there are optimisations at OS and hardware levels that kick in at different times. > The way to do this with timeit is: > > - don't run extraneous code as part of your test (just measure what > you care about, with as little scaffolding around it); > > - run that code snippet as many times as you can bear in a loop, > *but* let timeit manage the loop; don't try doing it by hand; There are many situations where running something in a loop over and over triggers all kinds of optimisations. PyPy's jit warmup is an example but the effect can occur at the hardware level as well. Performance of code run repeatedly in a tight loop can be different to the same code called once or called sporadically in a long-running process. This is especially true of code that operates on a data structure. In real usage the data structure might be accessed rarely but a tight-profiling loop forces the entire structure into the CPU's caches. > - only care about "best of N", where N is as large a number as you > can stand; > > - averages, standard deviation, and other statistics are pretty > much useless here, the only statistic that you care about is > the minimum. This argument makes sense if you adopt the t+?t model described above but that model it is just a model. The minimum from micro-benchmarking in a tight-loop is not necessarily a more representative statistic than mean, median or whatever else. Standard deviation is actually useful for assessing the variability of time taken which can also be important and gives additional information about performance: I would like to minimise both the expected time taken and also the variability in that simultaneously if possible. You seem to be describing timeit's way of profiling which is reasonable but it's not the only way. -- Oscar From chris at simplistix.co.uk Tue Feb 2 10:44:20 2016 From: chris at simplistix.co.uk (Chris Withers) Date: Tue, 2 Feb 2016 15:44:20 +0000 Subject: testfixtures 4.8.0 Released! Message-ID: <56B0CED4.2020305@simplistix.co.uk> Hi All, I'm pleased to announce the release of testfixtures 4.8.0 featuring the following: - More succinct mocking with Replacer: For setUp usage: class MyTests(TestCase): def setUp(self): replace = Replacer() replace('x.y.z', Mock()) self.addCleanup(replace.restore) For one-shot context manager usage: with Replace('x.y.z', Mock()) as the_mock: ... Multi-replace context manager usage: with Replacer() as replace: dt = replace('x.y.z.datetime', test_datetime()) mock = replace('x.y.do_it', Mock()) - Two context managers for testing warnings: with ShouldWarn(UserWarning("foo")): warnings.warn('foo') with ShouldNotWarn(): ... code that should raise no warnings ... The package is on PyPI and a full list of all the links to docs, issue trackers and the like can be found here: https://github.com/Simplistix/testfixtures Any questions, please do ask on the Testing in Python list or on the Simplistix open source mailing list... cheers, Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk From paul.hermeneutic at gmail.com Tue Feb 2 12:00:27 2016 From: paul.hermeneutic at gmail.com (paul.hermeneutic at gmail.com) Date: Tue, 2 Feb 2016 10:00:27 -0700 Subject: psss...I want to move from Perl to Python In-Reply-To: References: <56AAB094.5050309@gmail.com> Message-ID: On Jan 29, 2016 6:51 AM, "Dennis Lee Bieber" wrote: > > http://www.barnesandnoble.com/w/perl-to-python-migration-martin-c-brown/1004847881?ean=9780201734881 Given that this was published in 2001, surely it is time for a second edition. From srkunze at mail.de Tue Feb 2 12:20:58 2016 From: srkunze at mail.de (Sven R. Kunze) Date: Tue, 2 Feb 2016 18:20:58 +0100 Subject: Heap Implementation In-Reply-To: References: <56AD3D83.2050308@mail.de> <56AF94B7.9050203@mail.de> Message-ID: <56B0E57A.8000803@mail.de> On 02.02.2016 01:48, srinivas devaki wrote: > > On Feb 1, 2016 10:54 PM, "Sven R. Kunze" > wrote: > > > > Maybe I didn't express myself well. Would you prefer the sweeping > approach in terms of efficiency over how I implemented xheap currently? > > > > complexity wise your approach is the best one of all that I have seen > till now > > > Without running some benchmarks, I have absolutely no feeling which > approach is faster/more memory efficient etc. > > > > this is obviously memory efficient but I don't know whether this > approach would be faster than previous approaches, with previous > approaches there is no call back into Python code from C code for > comparison. > but this should be faster than HeapDict as HeapDict is directly using > its own methods for heappush, heappop etc > Yes. So, it remains to be seen until I implemented the sweeping and compared them to each other. > PS: if you have time, could you please review my pull request. > Indeed, I am already thinking about it. :) Best, Sven From martin at linux-ip.net Tue Feb 2 16:04:15 2016 From: martin at linux-ip.net (Martin A. Brown) Date: Tue, 2 Feb 2016 13:04:15 -0800 Subject: psss...I want to move from Perl to Python In-Reply-To: References: <56AAB094.5050309@gmail.com> Message-ID: Hello, >http://www.barnesandnoble.com/w/perl-to-python-migration-martin-c-brown/1004847881?ean=9780201734881 > >Given that this was published in 2001, surely it is time for a >second edition. How many times do you think somebody migrates from Perl to Python?! ;) -Martin P.S. I was amused when I first discovered (about 15 years ago) Martin C. Brown, an author of Perl books. I am also amused to discover that he has written one on Python. Too many of us chaps named 'Martin Brown'. https://en.wikipedia.org/wiki/Radio_Active_(radio_series) the incompetent hospital-radio trained Martin Brown (Stevens) P.P.S. In case it is not utterly clear, I am not the above author. -- Martin A. Brown http://linux-ip.net/ From rantingrickjohnson at gmail.com Tue Feb 2 17:53:51 2016 From: rantingrickjohnson at gmail.com (Rick Johnson) Date: Tue, 2 Feb 2016 14:53:51 -0800 (PST) Subject: psss...I want to move from Perl to Python In-Reply-To: References: <2a8dc773-87a1-4ffd-8b8f-a77f2f6ff693@googlegroups.com> Message-ID: <13cae443-80cb-429b-a009-7311498d9174@googlegroups.com> On Sunday, January 31, 2016 at 9:15:38 PM UTC-6, Chris Angelico wrote: > That said, though, I would GREATLY prefer Rick to post > less provocatively. I'll admit that my tone can "sometimes" be acerbic, but it's not like i was attacking someone for *NO* justifiable reason, in fact, my response was not an attack at all, but merely, a plea. There is no doubt that a strong undercurrent of "militant anti-OOP" exists within this group (I know, because i've witnessed it for years), and those same militants have attempted to cast *ME* as a some sort of "religious OOP fanatic" -- but nothing could be further from the truth! Yes, (sigh) i've written a lot of Java code (quick, muster the militants!), and whilst I feel that the structural rigidity that java demands *does* encourage good programmers to write consistent code, it *does not* prevent poor programmers from writing garbage, nor does it allow me to fully express myself. Heck, Python's "multi-paradigm" nature, is IMHO, one of it's greatest strengths!!! We humans crave familiarity, because, those concepts that are familiar to us don't require any thought, no, we just wield them instinctively. However, we loath the unfamiliar, because, the unfamiliar requires us to expend mental effort *LEARNING* about the concept, *BEFORE* we can benefit from the fruits of it's power. Therefore, "the familiar" is like having a "slightly cool, diseased-donkey-piss beer" in your hand, and "the unfamiliar" is like having an "ice-cold, quality craft beer" way down in your basement refrigerator -- sometimes, we'd rather drink donkey-piss, than get off the couch. > But I'm not calling for his banning any more than I'd call > for Terry's, or my own, for that matter (I've posted > plenty of off-topic or otherwise useless posts). Chris, we don't always agree, but when we do, i must admit, it's always a *VERY* pleasurable experience. Thank you! :-) From rantingrickjohnson at gmail.com Tue Feb 2 18:12:40 2016 From: rantingrickjohnson at gmail.com (Rick Johnson) Date: Tue, 2 Feb 2016 15:12:40 -0800 (PST) Subject: psss...I want to move from Perl to Python In-Reply-To: References: Message-ID: <49845594-73c8-4360-b1f4-3492848123db@googlegroups.com> On Sunday, January 31, 2016 at 9:35:17 PM UTC-6, MRAB wrote: > In [jmf's] defence, he _was_ the one who drew attention to > the unexpected slowness of the FSR under certain > circumstances in the Windows build of Python 3.3, albeit > in a rather over-dramatized way. Thanks MRAB. You and Terry may be only "high ranking" members who are willing to defend those who cannot defend themselves. "jmf's contribution" is solid evidence of why we should never discourage anyone's participation in this community. Yes, jmf's posts are mostly repetitive, but we need to listen to all opinions and all complaints. Python cannot survive unless it maintains a solid community of people who (1) enjoy using the language, and (2) who know that others will listen when they voice a concern. Heck, i was banned from Python-ideas before i ever sent my first reply. Can anyone justify such tyranny? My "communal dignity" has been stolen from me! From russell.mccune at yahoo.com Tue Feb 2 21:49:04 2016 From: russell.mccune at yahoo.com (Russell McCune) Date: Tue, 2 Feb 2016 21:49:04 -0500 Subject: Error Message-ID: <174845.77155.bm@smtp229.mail.gq1.yahoo.com> Hi So I have installed python 3.5.1 (32-bit) for my windows 10 PC. I have also installed the IDE pycharm 5 to go along with it. Now my problem is when I run pycharm the 'Modify Setup' window for python keeps popping up everytime I try to do anything in pycharm. I don't know if i installed it incorrectly or what. What am I doing wrong?? Thank you Russell McCune From rantingrickjohnson at gmail.com Tue Feb 2 22:26:19 2016 From: rantingrickjohnson at gmail.com (Rick Johnson) Date: Tue, 2 Feb 2016 19:26:19 -0800 (PST) Subject: [STORY-TIME] THE BDFL AND HIS PYTHON PETTING ZOO Message-ID: <48762040-a7e0-434c-92e8-8a0969210e5b@googlegroups.com> ============================================================ [STORY-TIME] THE BDFL AND HIS PYTHON PETTING ZOO ============================================================ A long, long time a ago, in a sleepy little Scandinavian village, somewhere outside of Stockholm, a wee little boy was born, who would one day become known as: "the BDFL". And as all little boys love to do, he enjoyed frolicking around the countryside in his wee little wooden shoes, picking tulips for his mother, and taking rides on the arms of nearby windmills. (oh, how cute!) And as he grew older, he started to become interested in math and logic, and he became fascinated with the power of software, yes, and he *EVEN* dreamed of one day coding his own calculator app... BUT THAT GROUND BREAKING APP WOULD HAVE TO WAIT, FOR SOON, FATE WOULD INTERVENE! One day, when he was basking in the warm Scandinavian summer sun, surrounded by tall blades of grass blowing wistfully in the wind, a snake slithered close by, and began speaking "great things" into his ear. And the snake told him that everything in the programming world was completely wrong. And that the syntax had become nothing more than obnoxious line noise, and that, "religious paradigm fanaticism" was going to pollute, and eventually destroy, the pristine "code wilds". And he, being a wise lad, knew the snake was right! So he listened contently, and he learned. And he fell deeply in love with the snake, and the snake became his best, and *ONLY*, friend. And for many years, he, and the snake, developed a strong bond. And he swore that he would never leave, or harm the snake. And as the boy grew into early adulthood, a new and mysterious force began to shape his world, and this new force was called "idealism". And his idealism implored him to share the snake with the world, so that *ALL* the children of the world might become enlightened, by the wisdom of the snake. SO HE OPENED A LITTLE PETTING ZOO, AND HE DUBBED IT: "THE PSF". And many children came from the far and wide, and they would pet his snake, and they would play with his snake, and they would learn from his snake's philosophy. And the children became so happy with the his altruistic nature, that they affectionately referred to him as their "Benevolent Dictator For Life" (aka: BDFL). And so, the BDFL was happy. He was happy because the children who came to his petting zoo loved him, but most importantly, he was happy that they never questioned his "petting rules", or the philosophy of his snake. And he became an instant worldwide celebrity. And was interviewed by reporters, and documented in magazines. He even became worshiped, yes, like a god, which caused his pride to expand to new and dangerous levels. In fact, his pride became so great, that in one interview, he snarky dismissed the dangers of his snake's "wildly dynamic behavior", and claimed that, he would be fully content as a passenger in an airplane with his snake as the pilot... MOST PETTING-ZOO OWNERS THOUGHT HE WAS NUTS, BUT HE *REFUSED* TO RETRACT THE STATEMENT. As time went on, a small "inner group of disciples" began to develop around him. These disciples followed him everywhere. A band of loyal sycophants, or "Umpa Lumpas" if you will, who would blindly follow any demand he dictated, no matter how ridiculous. And he and his disciples became repulsed by the never ending hordes of children who would visit the petting zoo, they saw these "other children" as unclean, so they decided to create two holy places called "Python-Ideas" and "Python-Dev" (aka: the holy of holies), and they declared that, only those who are willing to lick the BDLF's boots, may enter these holy places. Soon after, the BDFL took permeate refuge in these holy places, and, from his rectory, he would sneer down at the children below, and if any child dared to question his motives, or broke one of his "petting rules", or even, did not goose-step to his political ideology, he would call for his minions to hurl furious rebukes at the visitor, and he would ensure the child was banned from ever entering his holy places. And although he could not prevent a child from entering the petting zoo commons, he would loose his minions, to pester and mock, the banned children, when they did visit. And for the first two years, this is how the BDFL managed his petting zoo. And although he was not happy, by most people's standards, he found peace within his "little refuge". ============================================================ BUT TROUBLE WAS SOON ON THE HORIZON... ============================================================ Near the end of the 2nd year, of running his petting zoo, the BDFL had become bored, because, even though the children of the world had never lost interest in his snake, most of them had lost interest in him, and since he was mostly absent from the petting zoo, he became something of myth, even a legend... He realized, that he had become nothing more than an amusement park owner, but what he *truly* craved was to reclaim the close bond that he and the snake once shared. Observing the pathetic state that his life had become, he became very angry, and he realized that, if *HE* could not have the snake, then by god, *NOBODY* would! But there was a big problem, he could not simply close the petting zoo, no, because if he did, not only would the world hate him, the snake would hate him as well. So he devised a plan, a wonderfully *evil* plan... later that night, while the snake was sleeping, he meticulously glued razors and nails to the skin of the snake, and when his evil deed was finished, he grinned and said: "Let them pet you now, my pretty!" And sure enough, the next day, many children's hands became injured when they "interfaced with the snake", and once word got out, that the snake was not "child friendly" anymore, the hordes of wide-eyed children, that once came in droves to play and learn from the snake, stopped visiting. The BDFL has gotten what he wanted, but not what he bargained for! Due to the rusty nails that were embedded into it's skin... The snake suffers a severe tetanus infection, and dies. Darkness again envelopes the "code wilds". And syntactic noise pollutes the quiet hedge rows. And the only consistency to be found... Was of the foolish kind. In the end, the BDFL realized that he got what he deserved. Due to his jealousy, his divisiveness, or maybe, just a lack of leadership skills, he had single handily destroyed what was once a vibrant community of like-minded individuals. His celebrity vanquished, and his petting zoo repossessed, he was left a bitter, and broken man. But what he lamented the most, was the destruction he perpetrated against his best friend, his *ONLY* friend, the snake. THE END From john_ladasky at sbcglobal.net Wed Feb 3 01:02:06 2016 From: john_ladasky at sbcglobal.net (John Ladasky) Date: Tue, 2 Feb 2016 22:02:06 -0800 (PST) Subject: [STORY-TIME] THE BDFL AND HIS PYTHON PETTING ZOO In-Reply-To: <48762040-a7e0-434c-92e8-8a0969210e5b@googlegroups.com> References: <48762040-a7e0-434c-92e8-8a0969210e5b@googlegroups.com> Message-ID: <5d31dd14-1a60-4f80-8889-d0616f404c70@googlegroups.com> Yeah, I know, I shouldn't answer this troll-bait. Rick, you don't like Python? Fine, go program in a language that you do like. There are many to choose from! Meanwhile, Python is satisfying many programmers' needs. From mafagafogigante at gmail.com Wed Feb 3 01:07:30 2016 From: mafagafogigante at gmail.com (Bernardo Sulzbach) Date: Wed, 3 Feb 2016 04:07:30 -0200 Subject: [STORY-TIME] THE BDFL AND HIS PYTHON PETTING ZOO In-Reply-To: <5d31dd14-1a60-4f80-8889-d0616f404c70@googlegroups.com> References: <48762040-a7e0-434c-92e8-8a0969210e5b@googlegroups.com> <5d31dd14-1a60-4f80-8889-d0616f404c70@googlegroups.com> Message-ID: I wonder if anyone besides Mr. Ladasky read through the Great Wall of Email. From salony.permanand at gmail.com Wed Feb 3 01:57:02 2016 From: salony.permanand at gmail.com (Salony Permanand) Date: Wed, 3 Feb 2016 12:27:02 +0530 Subject: problem in installing python Message-ID: sir, I downloaded different version of python but no one is installed on my pc because of same installation error each time having error code 2203. Please solve it... Thanking you From wolfgang.maier at biologie.uni-freiburg.de Wed Feb 3 03:27:41 2016 From: wolfgang.maier at biologie.uni-freiburg.de (Wolfgang Maier) Date: Wed, 3 Feb 2016 09:27:41 +0100 Subject: [STORY-TIME] THE BDFL AND HIS PYTHON PETTING ZOO In-Reply-To: <48762040-a7e0-434c-92e8-8a0969210e5b@googlegroups.com> References: <48762040-a7e0-434c-92e8-8a0969210e5b@googlegroups.com> Message-ID: On 03.02.2016 04:26, Rick Johnson wrote: > > ============================================================ > [STORY-TIME] THE BDFL AND HIS PYTHON PETTING ZOO > ============================================================ > > A long, long time a ago, in a sleepy little Scandinavian > village, somewhere outside of Stockholm, a wee little boy ... long long story Just one question: do you really think the Netherlands are a part of Scandinavia and that Stockholm is their capital? If so maybe you'll find this comforting: https://www.washingtonpost.com/news/monkey-cage/wp/2014/04/07/the-less-americans-know-about-ukraines-location-the-more-they-want-u-s-to-intervene/ From snailcoder at retrosite.invalid Wed Feb 3 03:38:51 2016 From: snailcoder at retrosite.invalid (Grobu) Date: Wed, 3 Feb 2016 09:38:51 +0100 Subject: [STORY-TIME] THE BDFL AND HIS PYTHON PETTING ZOO In-Reply-To: <48762040-a7e0-434c-92e8-8a0969210e5b@googlegroups.com> References: <48762040-a7e0-434c-92e8-8a0969210e5b@googlegroups.com> Message-ID: On 03/02/16 04:26, Rick Johnson wrote: [ ... ] > And many children came from the far and wide, and they > would pet his snake, and they would play with his snake Didn't know Pedobear had a biographer. From breamoreboy at yahoo.co.uk Wed Feb 3 03:49:27 2016 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Wed, 3 Feb 2016 08:49:27 +0000 Subject: Error In-Reply-To: <174845.77155.bm@smtp229.mail.gq1.yahoo.com> References: <174845.77155.bm@smtp229.mail.gq1.yahoo.com> Message-ID: On 03/02/2016 02:49, Russell McCune via Python-list wrote: > Hi > > So I have installed python 3.5.1 (32-bit) for my windows 10 PC. I have also installed the IDE pycharm 5 to go along with it. Now my problem is when I run pycharm the 'Modify Setup' window for python keeps popping up everytime I try to do anything in pycharm. I don't know if i installed it incorrectly or what. What am I doing wrong?? > > Thank you > Russell McCune > This has been asked and answered repeatedly in the past few weeks. Please search the archives for your answer. -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From oscar.j.benjamin at gmail.com Wed Feb 3 04:40:59 2016 From: oscar.j.benjamin at gmail.com (Oscar Benjamin) Date: Wed, 3 Feb 2016 09:40:59 +0000 Subject: Error In-Reply-To: References: <174845.77155.bm@smtp229.mail.gq1.yahoo.com> Message-ID: On 3 February 2016 at 08:49, Mark Lawrence wrote: > On 03/02/2016 02:49, Russell McCune via Python-list wrote: >> >> Hi >> >> So I have installed python 3.5.1 (32-bit) for my windows 10 PC. I have >> also installed the IDE pycharm 5 to go along with it. Now my problem is when >> I run pycharm the 'Modify Setup' window for python keeps popping up >> everytime I try to do anything in pycharm. I don't know if i installed it >> incorrectly or what. What am I doing wrong?? >> >> Thank you >> Russell McCune >> > > This has been asked and answered repeatedly in the past few weeks. Please > search the archives for your answer. Has it? I don't think I've seen any threads about installing PyCharm in Windows and I can't see anything in the past few weeks of archives. Can you provide a link please Mark? -- Oscar From oscar.j.benjamin at gmail.com Wed Feb 3 04:45:10 2016 From: oscar.j.benjamin at gmail.com (Oscar Benjamin) Date: Wed, 3 Feb 2016 09:45:10 +0000 Subject: problem in installing python In-Reply-To: References: Message-ID: On 3 February 2016 at 06:57, Salony Permanand wrote: > sir, > I downloaded different version of python but no one is installed on my pc > because of same installation error each time having error code 2203. > Please solve it... > Thanking you Hi Salony, I think this is a problem with Windows. Microsoft has some information for how to solve this: https://support.microsoft.com/en-gb/kb/2000547 If that doesn't work then ask here again but also tell us 1) which version of Python you're trying to install 2) where you downloaded it from 3) what operating system you are using -- Oscar From rosuav at gmail.com Wed Feb 3 04:59:28 2016 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 3 Feb 2016 20:59:28 +1100 Subject: Problem with 'requests' module In-Reply-To: References: Message-ID: On Wed, Feb 3, 2016 at 12:16 AM, Mohamed Nedal wrote: > > I'm learning Python and I'm using Python 3.5.1 Shell to write some code using 'requests' module, but I get this error when I run the code: ImportError: No module named 'requests' > > > Could you please tell me how to fix this issue? > The module you're looking for isn't part of the Python standard library - it's a third-party module. You'll need to install it using 'pip'. Exactly how you go about doing this depends on your platform and how you're using Python, but it's most likely going to involve running this shell command: python3 -m pip install requests ChrisA From eryksun at gmail.com Wed Feb 3 05:25:20 2016 From: eryksun at gmail.com (eryk sun) Date: Wed, 3 Feb 2016 04:25:20 -0600 Subject: problem in installing python In-Reply-To: References: Message-ID: On Wed, Feb 3, 2016 at 12:57 AM, Salony Permanand wrote: > > I downloaded different version of python but no one is installed on my pc > because of same installation error each time having error code 2203. 2203 may be a Windows Installer error [1]. If so, the error message has the following template: Database: [2]. Cannot open database file. System error [3]. Please provide the "System error", since it may help to determine why the installer can't open the file. Possibly an anti-virus program is interfering, in which case temporarily disabling it may solve the problem. [1]: https://msdn.microsoft.com/en-us/library/aa372835 From salony.permanand at gmail.com Wed Feb 3 05:57:14 2016 From: salony.permanand at gmail.com (Salony Permanand) Date: Wed, 3 Feb 2016 16:27:14 +0530 Subject: Fwd: problem in installing python In-Reply-To: References: Message-ID: Thankyou for consideration..I have solved my problem by changing name of temp files by "temp1" ---------- Forwarded message ---------- From: eryk sun Date: Wed, Feb 3, 2016 at 3:55 PM Subject: Re: problem in installing python To: python-list at python.org Cc: Salony Permanand On Wed, Feb 3, 2016 at 12:57 AM, Salony Permanand wrote: > > I downloaded different version of python but no one is installed on my pc > because of same installation error each time having error code 2203. 2203 may be a Windows Installer error [1]. If so, the error message has the following template: Database: [2]. Cannot open database file. System error [3]. Please provide the "System error", since it may help to determine why the installer can't open the file. Possibly an anti-virus program is interfering, in which case temporarily disabling it may solve the problem. [1]: https://msdn.microsoft.com/en-us/library/aa372835 From joel.goldstick at gmail.com Wed Feb 3 06:44:38 2016 From: joel.goldstick at gmail.com (Joel Goldstick) Date: Wed, 3 Feb 2016 06:44:38 -0500 Subject: Problem with 'requests' module In-Reply-To: References: Message-ID: On Wed, Feb 3, 2016 at 4:59 AM, Chris Angelico wrote: > On Wed, Feb 3, 2016 at 12:16 AM, Mohamed Nedal > wrote: > > > > I'm learning Python and I'm using Python 3.5.1 Shell to write some code > using 'requests' module, but I get this error when I run the code: > ImportError: No module named 'requests' > > > > > > Could you please tell me how to fix this issue? > > > > The module you're looking for isn't part of the Python standard > library - it's a third-party module. You'll need to install it using > 'pip'. Exactly how you go about doing this depends on your platform > and how you're using Python, but it's most likely going to involve > running this shell command: > > python3 -m pip install requests > > ChrisA > -- > https://mail.python.org/mailman/listinfo/python-list > Here is a link to the installation documents: http://docs.python-requests.org/en/latest/user/install/#install -- Joel Goldstick http://joelgoldstick.com/stats/birthdays From tjreedy at udel.edu Wed Feb 3 08:21:26 2016 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 3 Feb 2016 08:21:26 -0500 Subject: Error In-Reply-To: <174845.77155.bm@smtp229.mail.gq1.yahoo.com> References: <174845.77155.bm@smtp229.mail.gq1.yahoo.com> Message-ID: On 2/2/2016 9:49 PM, Russell McCune via Python-list wrote: > Hi > > So I have installed python 3.5.1 (32-bit) for my windows 10 PC. I > have also installed the IDE pycharm 5 to go along with it. Now my > problem is when I run pycharm the 'Modify Setup' window for python > keeps popping up everytime I try to do anything in pycharm. What happens if you run Python directly from All apps / Python 3.5? Did you try that before installing pycharm? This determines whether Pycharm is relevant to the issue. > I don't > know if i installed it incorrectly or what. What am I doing wrong?? -- Terry Jan Reedy From tjreedy at udel.edu Wed Feb 3 08:24:18 2016 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 3 Feb 2016 08:24:18 -0500 Subject: Error In-Reply-To: References: <174845.77155.bm@smtp229.mail.gq1.yahoo.com> Message-ID: On 2/3/2016 4:40 AM, Oscar Benjamin wrote: > On 3 February 2016 at 08:49, Mark Lawrence wrote: >> On 03/02/2016 02:49, Russell McCune via Python-list wrote: >>> >>> Hi >>> >>> So I have installed python 3.5.1 (32-bit) for my windows 10 PC. I have >>> also installed the IDE pycharm 5 to go along with it. Now my problem is when >>> I run pycharm the 'Modify Setup' window for python keeps popping up >>> everytime I try to do anything in pycharm. I don't know if i installed it >>> incorrectly or what. What am I doing wrong?? >>> >>> Thank you >>> Russell McCune >>> >> >> This has been asked and answered repeatedly in the past few weeks. Please >> search the archives for your answer. > > Has it? I don't think I've seen any threads about installing PyCharm > in Windows and I can't see anything in the past few weeks of archives. > Can you provide a link please Mark? I think Mark is assuming that the mention of pycharm is a red herring, because others have seen the behavior without installing pycharm. I separately suggested that Russel do the needed experiment on *his* system to determine the question. -- Terry Jan Reedy From breamoreboy at yahoo.co.uk Wed Feb 3 10:08:15 2016 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Wed, 3 Feb 2016 15:08:15 +0000 Subject: Mutant Registration: Implicit or Explicit Message-ID: Rather more interesting than RR or the RUE? http://inre.dundeemt.com/2016-02-03/mutant-registration-implicit-or-explicit/ -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From steve at pearwood.info Wed Feb 3 12:10:09 2016 From: steve at pearwood.info (Steven D'Aprano) Date: Thu, 04 Feb 2016 04:10:09 +1100 Subject: Mutant Registration: Implicit or Explicit References: Message-ID: <56b23473$0$1613$c3e8da3$5496439d@news.astraweb.com> On Thu, 4 Feb 2016 02:08 am, Mark Lawrence wrote: > Rather more interesting than RR or the RUE? > > http://inre.dundeemt.com/2016-02-03/mutant-registration-implicit-or-explicit/ > Hmmm. Well, it's an interesting blog post. The author says: "My question deals with the implicit nature of the mutation. Not that Python is somehow wrong, it is the fact that the function usage does not convey the mutation to the reader as pointedly as I want." We can't help what he wants (possibly a pony and a plastic rocket?), but the code is idiomatic and should be clear to any moderately experienced programmer of any language. The code is: toggle(mobject) which clearly doesn't return a value. (Or if it does, the return value is being ignored.) Hence, it is being called for its side-effects. Nearly every programming language has some form of procedure call that doesn't return a value but operates only by side-effect. In Pascal, you declare such routines as "procedure" instead of "function". In C, you declare them as functions with a return type of "void". In Python, you explicitly or implicitly return None. Perhaps some purely functional languages don't have such a thing, but they're not common. So regardless of what language(s) they are familiar with, most people should be able to correctly recognise toggle(mobject) as a procedure (of sorts). I can accept that the documentation for toggle() is insufficient. It should explicitly state that it modifies the flag in place. But since the author of the blog wrote the documentation, he has nobody to blame but himself. The author suggests returning the dict after mutating it, but that's possibly the worst of both worlds: the extra typing of functional programming plus unexpected side-effects: mobject = toggle(mobject) The experienced Pythonista will, on reading the source or documentation, realise that the binding is a waste of time, and rewrite that line as a pseudo-procedure call: toggle(mobject) while less experienced programmers (or those merely not paying attention) will write this: new_dict = toggle(old_dict) and then be painfully surprised that old_dict is modified. It looks like some sort of scary action-at-a-distance, where every time he modifies new_dict, old_dict gets modified too. But its not, since new_dict and old_dict are just two names for the one dict. In general, mutator functions and methods should not return the object they just mutated, unless there is a good reason to do so (e.g. decorators). -- Steven From srkunze at mail.de Wed Feb 3 15:32:01 2016 From: srkunze at mail.de (Sven R. Kunze) Date: Wed, 3 Feb 2016 21:32:01 +0100 Subject: Efficient Wrappers for Instance Methods Message-ID: <56B263C1.7030309@mail.de> Hi, as you might have noticed, I am working on https://github.com/srkunze/xheap right now. In order to make it even faster and closer to heapq's baseline performance, I wonder if there is a possibility of creating fast wrappers for functions. Please compare https://github.com/srkunze/xheap/blob/ca56ac55269ce0bc7b61d28ba9ceb41e9075476a/xheap.py#L29 https://github.com/srkunze/xheap/blob/ca56ac55269ce0bc7b61d28ba9ceb41e9075476a/xheap.py#L32 with https://github.com/srkunze/xheap/blob/ca56ac55269ce0bc7b61d28ba9ceb41e9075476a/xheap.py#L44 Why is it not possible to create a method from a function like I aliased replace with poppush? If I am not completely mistaken, it saves 1 stack frame, right? Best, Sven From mafagafogigante at gmail.com Wed Feb 3 15:40:58 2016 From: mafagafogigante at gmail.com (Bernardo Sulzbach) Date: Wed, 3 Feb 2016 18:40:58 -0200 Subject: Efficient Wrappers for Instance Methods In-Reply-To: <56B263C1.7030309@mail.de> References: <56B263C1.7030309@mail.de> Message-ID: I am not entirely sure about what your question is. Are you talking about the "heapreplace expected 2 arguments, got 1" you get if you set replace = heapreplace? From srkunze at mail.de Wed Feb 3 15:55:28 2016 From: srkunze at mail.de (Sven R. Kunze) Date: Wed, 3 Feb 2016 21:55:28 +0100 Subject: Efficient Wrappers for Instance Methods In-Reply-To: References: <56B263C1.7030309@mail.de> Message-ID: <56B26940.3090609@mail.de> On 03.02.2016 21:40, Bernardo Sulzbach wrote: > I am not entirely sure about what your question is. > > Are you talking about the "heapreplace expected 2 arguments, got 1" > you get if you set replace = heapreplace? Yes, I think so. I might ask differently: why do I need to write wrapper method when the function I am wrapping actually already has the same signature? Best, Sven From srkunze at mail.de Wed Feb 3 16:12:29 2016 From: srkunze at mail.de (Sven R. Kunze) Date: Wed, 3 Feb 2016 22:12:29 +0100 Subject: Efficient Wrappers for Instance Methods In-Reply-To: References: <56B263C1.7030309@mail.de> <56B26940.3090609@mail.de> Message-ID: <56B26D3D.6040703@mail.de> On 03.02.2016 22:06, Bernardo Sulzbach wrote: > I may say something wrong, but this is what I see going on: > > When you get "replace = heapreplace" you are creating a data attribute > called replace (you will access it by self.replace or > variable.replace) that is an alias for heapreplace. > > When you call x.replace(2) you are calling heapreplace(2), NOT > heapreplace(self, 2). It is exactly as you've described it. Question now is how can I circumvent/shortcut that? There are several proposals out there in the Web but none of them works. :-/ Best, Sven From mafagafogigante at gmail.com Wed Feb 3 16:14:47 2016 From: mafagafogigante at gmail.com (Bernardo Sulzbach) Date: Wed, 3 Feb 2016 19:14:47 -0200 Subject: Efficient Wrappers for Instance Methods In-Reply-To: <56B26D3D.6040703@mail.de> References: <56B263C1.7030309@mail.de> <56B26940.3090609@mail.de> <56B26D3D.6040703@mail.de> Message-ID: Thanks for quoting, for some reason my client always replies to the person and not the list (on this list only). I did what I could. I could show you a lambda function there, but it doesn't solve anything. If there is a way to avoid a wrapper, I don't know. From __peter__ at web.de Wed Feb 3 16:15:12 2016 From: __peter__ at web.de (Peter Otten) Date: Wed, 03 Feb 2016 22:15:12 +0100 Subject: Efficient Wrappers for Instance Methods References: <56B263C1.7030309@mail.de> Message-ID: Sven R. Kunze wrote: > Hi, > > as you might have noticed, I am working on > https://github.com/srkunze/xheap right now. > > In order to make it even faster and closer to heapq's baseline > performance, I wonder if there is a possibility of creating fast > wrappers for functions. > > > Please compare > > https://github.com/srkunze/xheap/blob/ca56ac55269ce0bc7b61d28ba9ceb41e9075476a/xheap.py#L29 > https://github.com/srkunze/xheap/blob/ca56ac55269ce0bc7b61d28ba9ceb41e9075476a/xheap.py#L32 > > with > > https://github.com/srkunze/xheap/blob/ca56ac55269ce0bc7b61d28ba9ceb41e9075476a/xheap.py#L44 > > > Why is it not possible to create a method from a function like I aliased > replace with poppush? The technical reason is that functions written in C don't implement the descriptor protocol. The bound method is created by invoking the __get__ method of the class attribute: >>> from heapq import heappush >>> class A: pass ... >>> def method(*args): print("invoked with", args) ... >>> a = A() >>> m = method.__get__(a) >>> m(42) invoked with (<__main__.A object at 0x7f07194c29b0>, 42) >>> n = heappush.__get__(a) Traceback (most recent call last): File "", line 1, in AttributeError: 'builtin_function_or_method' object has no attribute '__get__' > If I am not completely mistaken, it saves 1 stack frame, right? > > > Best, > Sven From larry.martell at gmail.com Wed Feb 3 16:19:20 2016 From: larry.martell at gmail.com (Larry Martell) Date: Wed, 3 Feb 2016 16:19:20 -0500 Subject: Emulating plotly's contour map color distribution Message-ID: I am using a javascript ploting package called plotly and I use that to generate a contour map. In that package you define a rangeMin, rangeMax, and binSize, and you give it a color map, e.g.: [["0", "rgb(0,0,0)"], ["0.3", "rgb(230,0,0)"], ["0.6", "rgb(255,210,0)"], ["1", "rgb(255,255,255)"]] This specifies the color you want at the endpoints of the range, and at the 1/3 and 2/3 points. Given the range and binSize plotly calculates the number of bins and distributes the color uniformly across the range, coloring the contour map appropriately given the data. I have another legacy map not generated with plotly and I want the colors of that to match the plotly map. In the legacy map I have the rangeMin and rangeMax and the binSize (although the binSize is not currently used). It currently determines the color for each value like this, in python: COLORS = ('#ff9', '#ff0', '#fc0', '#f90', '#f60', '#f00', '#930', '#800000') normal = (value - rangeMin) / (value_max - rangeMax) ordinal = int(normal * len(COLORS)) if ordinal == len(self.COLORS): ordinal -= 1 color = COLORS[ordinal] I am looking for some advice on how I can modify this to generate the same colors as plotly would, given the same data set and range and bin size. From __peter__ at web.de Wed Feb 3 16:19:35 2016 From: __peter__ at web.de (Peter Otten) Date: Wed, 03 Feb 2016 22:19:35 +0100 Subject: Efficient Wrappers for Instance Methods References: <56B263C1.7030309@mail.de> <56B26940.3090609@mail.de> <56B26D3D.6040703@mail.de> Message-ID: Sven R. Kunze wrote: > On 03.02.2016 22:06, Bernardo Sulzbach wrote: >> I may say something wrong, but this is what I see going on: >> >> When you get "replace = heapreplace" you are creating a data attribute >> called replace (you will access it by self.replace or >> variable.replace) that is an alias for heapreplace. >> >> When you call x.replace(2) you are calling heapreplace(2), NOT >> heapreplace(self, 2). > > It is exactly as you've described it. > > Question now is how can I circumvent/shortcut that? There are several > proposals out there in the Web but none of them works. :-/ You could try putting self.heappush = functools.partial(heapq.heappush, self) into the initializer. From srkunze at mail.de Wed Feb 3 16:33:31 2016 From: srkunze at mail.de (Sven R. Kunze) Date: Wed, 3 Feb 2016 22:33:31 +0100 Subject: Efficient Wrappers for Instance Methods In-Reply-To: References: <56B263C1.7030309@mail.de> <56B26940.3090609@mail.de> <56B26D3D.6040703@mail.de> Message-ID: <56B2722B.8000408@mail.de> On 03.02.2016 22:14, Bernardo Sulzbach wrote: > Thanks for quoting, for some reason my client always replies to the > person and not the list (on this list only). > > I did what I could. I could show you a lambda function there, but it > doesn't solve anything. If there is a way to avoid a wrapper, I don't > know. I appreciate every single reply. :) Best, Sven From mafagafogigante at gmail.com Wed Feb 3 16:34:09 2016 From: mafagafogigante at gmail.com (Bernardo Sulzbach) Date: Wed, 3 Feb 2016 19:34:09 -0200 Subject: Efficient Wrappers for Instance Methods In-Reply-To: <56B2722B.8000408@mail.de> References: <56B263C1.7030309@mail.de> <56B26940.3090609@mail.de> <56B26D3D.6040703@mail.de> <56B2722B.8000408@mail.de> Message-ID: Did Peter's suggestion work? From srkunze at mail.de Wed Feb 3 16:43:23 2016 From: srkunze at mail.de (Sven R. Kunze) Date: Wed, 3 Feb 2016 22:43:23 +0100 Subject: Efficient Wrappers for Instance Methods In-Reply-To: References: <56B263C1.7030309@mail.de> <56B26940.3090609@mail.de> <56B26D3D.6040703@mail.de> Message-ID: <56B2747B.6050209@mail.de> On 03.02.2016 22:19, Peter Otten wrote: > > You could try putting > > self.heappush = functools.partial(heapq.heappush, self) > > into the initializer. Actually a nice idea if there were no overhead of creating methods for all heap instances separately. I'll keep that in mind. :) From srkunze at mail.de Wed Feb 3 16:45:31 2016 From: srkunze at mail.de (Sven R. Kunze) Date: Wed, 3 Feb 2016 22:45:31 +0100 Subject: Efficient Wrappers for Instance Methods In-Reply-To: References: <56B263C1.7030309@mail.de> Message-ID: <56B274FB.7000000@mail.de> On 03.02.2016 22:15, Peter Otten wrote: > > The technical reason is that functions written in C don't implement the > descriptor protocol. The bound method is created by invoking the __get__ > method of the class attribute: Good to know. :-/ It's sad. These functions just look so method-like. Best, Sven From srkunze at mail.de Wed Feb 3 16:48:01 2016 From: srkunze at mail.de (Sven R. Kunze) Date: Wed, 3 Feb 2016 22:48:01 +0100 Subject: Efficient Wrappers for Instance Methods In-Reply-To: References: <56B263C1.7030309@mail.de> <56B26940.3090609@mail.de> <56B26D3D.6040703@mail.de> <56B2722B.8000408@mail.de> Message-ID: <56B27591.5090203@mail.de> On 03.02.2016 22:34, Bernardo Sulzbach wrote: > Did Peter's suggestion work? Somewhat for a single Heap class. However, it breaks inheritance. From ben+python at benfinney.id.au Wed Feb 3 16:50:59 2016 From: ben+python at benfinney.id.au (Ben Finney) Date: Thu, 04 Feb 2016 08:50:59 +1100 Subject: Reply to whom? (was: Efficient Wrappers for Instance Methods) References: <56B263C1.7030309@mail.de> <56B26940.3090609@mail.de> <56B26D3D.6040703@mail.de> Message-ID: <85h9hpcvjw.fsf_-_@benfinney.id.au> Bernardo Sulzbach writes: > Thanks for quoting, for some reason my client always replies to the > person and not the list (on this list only). You have a ?reply to sender? command in your mail client; every client has that. It seems that is what you used (it might just be named ?reply?). Such replies should not go to anyone else by default. You *should* also have a ?reply to list? command; if you don't, speak sternly to whomever makes the mail client you're using (?reply to list? has been around for decades), and until then switch to a better email client. When you reply, decide who should receive the response, and choose ?reply to sender? or ?reply to list? accordingly. (You will also have a ?reply to all? command. That's almost never appropriate in a forum like this.) -- \ ?The double standard that exempts religious activities from | `\ almost all standards of accountability should be dismantled | _o__) once and for all.? ?Daniel Dennett, 2010-01-12 | Ben Finney From taylorbarrie at hotmail.co.uk Wed Feb 3 16:55:59 2016 From: taylorbarrie at hotmail.co.uk (Barrie Taylor) Date: Wed, 3 Feb 2016 21:55:59 +0000 Subject: Install Error Message-ID: Hi, I am attempting to install and run Python3.5.1 on my Windows machine. After installation on launching I am presented the attached error message. It reads: 'The program can't start because api-ms-win-crt-runtime-l1-1-0.dll is missing from your computer. Try reinstalling the program to fix this problem.' Needless to say, I have tried uninstalling and reinstalling ad repairing the program a number of times to no avail. Any help with this issue would be greatly appreciated. Kind Regards, Barrie Taylor From syavosh at gmail.com Wed Feb 3 18:03:11 2016 From: syavosh at gmail.com (Syavosh Malek) Date: Thu, 4 Feb 2016 02:33:11 +0330 Subject: please help Message-ID: hi i install python 3.5.1 and found run time error see attach file and help me please From random832 at fastmail.com Wed Feb 3 18:47:38 2016 From: random832 at fastmail.com (Random832) Date: Wed, 03 Feb 2016 18:47:38 -0500 Subject: Efficient Wrappers for Instance Methods In-Reply-To: <56B2747B.6050209@mail.de> References: <56B263C1.7030309@mail.de> <56B26940.3090609@mail.de> <56B26D3D.6040703@mail.de> <56B2747B.6050209@mail.de> Message-ID: <1454543258.197018.511336178.74125AA7@webmail.messagingengine.com> On Wed, Feb 3, 2016, at 16:43, Sven R. Kunze wrote: > Actually a nice idea if there were no overhead of creating methods for > all heap instances separately. I'll keep that in mind. :) What about changing the class of the object to one which is inherited from its original class and has the method you want? What about reaching into the class and changing the method in the first place? Either may not be appropriate, of course, depending on your use case. From steve at pearwood.info Wed Feb 3 18:50:48 2016 From: steve at pearwood.info (Steven D'Aprano) Date: Thu, 04 Feb 2016 10:50:48 +1100 Subject: Reply to whom? (was: Efficient Wrappers for Instance Methods) References: <56B263C1.7030309@mail.de> <56B26940.3090609@mail.de> <56B26D3D.6040703@mail.de> Message-ID: <56b2925a$0$1585$c3e8da3$5496439d@news.astraweb.com> On Thu, 4 Feb 2016 08:50 am, Ben Finney wrote: > (You will also have a ?reply to all? command. That's almost never > appropriate in a forum like this.) If your mail client lacks Reply To List, use Reply All and then manually edit the To addresses of the new message. It honestly isn't hard, I've been doing it for years. It's four simple steps, and one complicated one: (1) Hit Reply All. You will[*] get a new email with headers (e.g.): To: The Sender CC: Mailing List (2) Edit your reply. This is the hard part, since you actually have to think about what you are doing. (3) Edit the CC header line, and CUT (not copy) the mailing list address. (4) Edit the To header line, delete the current contents, and PASTE the mailing list address. At this point you now have headers: To: Mailing List CC: (5) Hit Send. If your email software doesn't make this *trivially* easy, then your email software sucks. I don't care if it's on your mobile phone, or your microwave oven for that matter. If it is hard for you to change the To address, oh well, so sad, it sucks to be you. Perhaps you should wait until you can sit down at a desktop, laptop or even tablet before replying. If you (generic you) don't make the effort, and choose to send out the email with messed up To/CC headers, don't blame your lousy email client, or think that "Sorry, I'm on my phone" is an excuse. You're choosing to put your convenience ahead of that of the potentially thousands of others on the mailing list, so be honest about it: "I'm just being lazy. I won't adapt to the way everyone else works, so you can adapt to me and my crap tools." [*] With one exception: if the sender is sending via Yahoo, and possibly AOL (if there is anyone on AOL who knows how to send email). Yahoo and AOL do not interact well with mailing lists. As they say on the mailman mailing list, "Friends don't let friends use Yahoo email addresses." -- Steven From mafagafogigante at gmail.com Wed Feb 3 19:03:46 2016 From: mafagafogigante at gmail.com (Bernardo Sulzbach) Date: Wed, 3 Feb 2016 22:03:46 -0200 Subject: Reply to whom? (was: Efficient Wrappers for Instance Methods) In-Reply-To: <56b2925a$0$1585$c3e8da3$5496439d@news.astraweb.com> References: <56B263C1.7030309@mail.de> <56B26940.3090609@mail.de> <56B26D3D.6040703@mail.de> <56b2925a$0$1585$c3e8da3$5496439d@news.astraweb.com> Message-ID: This got a little big. I accidentally did not change the address from Sven's to the list and emailed him. Seconds later, he replied to the list quoting my entire message so that I wouldn't have to send it to the list too (after his reply, which would make it hard to understand). I thanked him for the favor. Then you two started going on about mail clients buttons and "here's a 'how to change recipient address'". From ben+python at benfinney.id.au Wed Feb 3 19:07:20 2016 From: ben+python at benfinney.id.au (Ben Finney) Date: Thu, 04 Feb 2016 11:07:20 +1100 Subject: Reply to whom? References: <56B263C1.7030309@mail.de> <56B26940.3090609@mail.de> <56B26D3D.6040703@mail.de> <56b2925a$0$1585$c3e8da3$5496439d@news.astraweb.com> Message-ID: <85d1sdcp8n.fsf@benfinney.id.au> Bernardo Sulzbach writes: > Then you two started going on about mail clients buttons Right. Thereby changing the subject of the evolving discussion. You're welcome :-) -- \ ?? one of the main causes of the fall of the Roman Empire was | `\ that, lacking zero, they had no way to indicate successful | _o__) termination of their C programs.? ?Robert Firth | Ben Finney From rosuav at gmail.com Wed Feb 3 19:29:03 2016 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 4 Feb 2016 11:29:03 +1100 Subject: Reply to whom? (was: Efficient Wrappers for Instance Methods) In-Reply-To: References: <56B263C1.7030309@mail.de> <56B26940.3090609@mail.de> <56B26D3D.6040703@mail.de> <56b2925a$0$1585$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Thu, Feb 4, 2016 at 11:03 AM, Bernardo Sulzbach wrote: > This got a little big. I accidentally did not change the address from > Sven's to the list and emailed him. Seconds later, he replied to the > list quoting my entire message so that I wouldn't have to send it to > the list too (after his reply, which would make it hard to > understand). I thanked him for the favor. > > Then you two started going on about mail clients buttons and "here's a > 'how to change recipient address'". You don't have to take it personally; sometimes, the best thing to do is to call someone out for the benefit of those watching from the sidelines. Steven's explanation of how to reply-list when you can't reply-list should be of value to everyone who uses Gmail, for instance. Incidentally, I tend to follow a simplified model: 1) Hit Reply-All 2) Edit your reply, yada yada, content 3) Remove the poster's name from To 3a) Also remove any previous individuals from Cc, if other people have been replying-all 4) Send So the list address ends up in Cc rather than To. Does this cause problems for anyone? I've been doing it for quite a while, and it's a lot more convenient than moving the address up to To; but I won't if it bothers folks. (Oh, and I sometimes do steps 2 and 3 in the other order, but only because Gmail lets me press Ctrl-Enter to send the post if the cursor's in the body, but for some reason won't let me do that if the cursor's in the To or Cc field. Dunno why.) ChrisA From mafagafogigante at gmail.com Wed Feb 3 19:43:44 2016 From: mafagafogigante at gmail.com (Bernardo Sulzbach) Date: Wed, 3 Feb 2016 22:43:44 -0200 Subject: Reply to whom? (was: Efficient Wrappers for Instance Methods) In-Reply-To: References: <56B263C1.7030309@mail.de> <56B26940.3090609@mail.de> <56B26D3D.6040703@mail.de> <56b2925a$0$1585$c3e8da3$5496439d@news.astraweb.com> Message-ID: For this list, and this list only (I likely could identify the issue by comparing raw messages, but I won't bother), when I hit 'r' I get the sender of the latest message. In this case, "rosuav at gmail.com". So I have to start typing "pyt" to change it to the list, double tab to body, type, then Ctrl + Enter to send it after 30 seconds. However, I am SO used to not changing the address (all work related communication is always just 'r', the same for Fedora, SQLite, Wikipedia, etc.) that I end up replying to the wrong address sometimes. From rosuav at gmail.com Wed Feb 3 19:59:54 2016 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 4 Feb 2016 11:59:54 +1100 Subject: Reply to whom? (was: Efficient Wrappers for Instance Methods) In-Reply-To: References: <56B263C1.7030309@mail.de> <56B26940.3090609@mail.de> <56B26D3D.6040703@mail.de> <56b2925a$0$1585$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Thu, Feb 4, 2016 at 11:43 AM, Bernardo Sulzbach wrote: > For this list, and this list only (I likely could identify the issue > by comparing raw messages, but I won't bother), when I hit 'r' I get > the sender of the latest message. In this case, "rosuav at gmail.com". So > I have to start typing "pyt" to change it to the list, double tab to > body, type, then Ctrl + Enter to send it after 30 seconds. > > However, I am SO used to not changing the address (all work related > communication is always just 'r', the same for Fedora, SQLite, > Wikipedia, etc.) that I end up replying to the wrong address > sometimes. Exactly the problem. I did the same thing quite a bit, until I started doing the "reply-all and remove" style as per Steven's description. The worst was crossing between the various Python lists, as they're easy to confuse - but sometimes I sent stuff about 19th century opera to this list, or stuff about computer programming to Savoynet... ChrisA From ben+python at benfinney.id.au Wed Feb 3 20:09:36 2016 From: ben+python at benfinney.id.au (Ben Finney) Date: Thu, 04 Feb 2016 12:09:36 +1100 Subject: Reply to whom? References: <56B263C1.7030309@mail.de> <56B26940.3090609@mail.de> <56B26D3D.6040703@mail.de> <56b2925a$0$1585$c3e8da3$5496439d@news.astraweb.com> Message-ID: <8560y5cmcv.fsf@benfinney.id.au> Bernardo Sulzbach writes: > For this list, and this list only (I likely could identify the issue > by comparing raw messages, but I won't bother), when I hit 'r' I get > the sender of the latest message. This list is correctly configured; a ?reply to sender? goes to the sender address. Some mailing lists are mis-configured to pretend that the list is the sender; this is an abuse of the mail system and it thwarts the correct behaviour of ?reply to sender?. > In this case, "rosuav at gmail.com". So I have to start typing "pyt" to > change it to the list, double tab to body, type, then Ctrl + Enter to > send it after 30 seconds. This is something to describe to the vendor of whichever mail program you are using. Until they correct their software, there are many other email clients you can use that do not misbehave this way, and correctly interact with mailing list technology. -- \ ?In economics, hope and faith coexist with great scientific | `\ pretension and also a deep desire for respectability.? ?John | _o__) Kenneth Galbraith, 1970-06-07 | Ben Finney From mafagafogigante at gmail.com Wed Feb 3 20:17:04 2016 From: mafagafogigante at gmail.com (Bernardo Sulzbach) Date: Wed, 3 Feb 2016 23:17:04 -0200 Subject: Reply to whom? In-Reply-To: <8560y5cmcv.fsf@benfinney.id.au> References: <56B263C1.7030309@mail.de> <56B26940.3090609@mail.de> <56B26D3D.6040703@mail.de> <56b2925a$0$1585$c3e8da3$5496439d@news.astraweb.com> <8560y5cmcv.fsf@benfinney.id.au> Message-ID: Mr. Finney, that would be Google itself. I have never bothered setting up a client such as Mutt and can't even name more than Outlook and Mutt. I must say that the way you put it makes sense. A "reply" to the last message targeting its sender (the person that wrote it) is very reasonable indeed. From ben+python at benfinney.id.au Wed Feb 3 22:18:03 2016 From: ben+python at benfinney.id.au (Ben Finney) Date: Thu, 04 Feb 2016 14:18:03 +1100 Subject: Reply to whom? References: <56B263C1.7030309@mail.de> <56B26940.3090609@mail.de> <56B26D3D.6040703@mail.de> <56b2925a$0$1585$c3e8da3$5496439d@news.astraweb.com> <8560y5cmcv.fsf@benfinney.id.au> Message-ID: <851t8tcges.fsf@benfinney.id.au> Bernardo Sulzbach writes: > Mr. Finney, that would be Google itself. Yep. You're the one who needs to interact with them because as a user of the program, it's you they're trying to please (more than me, at least). Can't work out how to interact with a corporation and get into correspondence about a bug in their software? That's probably a good sign you shouldn't be using their software :-) > I have never bothered setting up a client such as Mutt and can't even > name more than Outlook and Mutt. You'll know your needs more than me. Maybe you could start with Thunderbird (available in Debian as ?Icedove? because of trademark issues); that's a good all-round client that behaves well. > I must say that the way you put it makes sense. A "reply" to the last > message targeting its sender (the person that wrote it) is very > reasonable indeed. Thanks. This is a contentious issue, and some people want to mangle your messages;but that doesn't mean it's still open for debate ? one side is right :-) -- \ ?I cannot conceive that anybody will require multiplications at | `\ the rate of 40,000 or even 4,000 per hour ?? ?F. H. Wales, 1936 | _o__) | Ben Finney From mafagafogigante at gmail.com Wed Feb 3 22:23:26 2016 From: mafagafogigante at gmail.com (Bernardo Sulzbach) Date: Thu, 4 Feb 2016 01:23:26 -0200 Subject: Reply to whom? In-Reply-To: <851t8tcges.fsf@benfinney.id.au> References: <56B263C1.7030309@mail.de> <56B26940.3090609@mail.de> <56B26D3D.6040703@mail.de> <56b2925a$0$1585$c3e8da3$5496439d@news.astraweb.com> <8560y5cmcv.fsf@benfinney.id.au> <851t8tcges.fsf@benfinney.id.au> Message-ID: I see. I've bad experiences with Thunderbird in the past, but I will try a desktop client again. You **can** provide input to Google, they even say the love it. But I've never seen one company ignore user feedback as much as Google does. From random832 at fastmail.com Wed Feb 3 23:43:42 2016 From: random832 at fastmail.com (Random832) Date: Wed, 03 Feb 2016 23:43:42 -0500 Subject: Reply to whom? (was: Efficient Wrappers for Instance Methods) In-Reply-To: <85h9hpcvjw.fsf_-_@benfinney.id.au> References: <56B263C1.7030309@mail.de> <56B26940.3090609@mail.de> <56B26D3D.6040703@mail.de> <85h9hpcvjw.fsf_-_@benfinney.id.au> Message-ID: <1454561022.2123037.511515122.153B155A@webmail.messagingengine.com> On Wed, Feb 3, 2016, at 16:50, Ben Finney wrote: > (You will also have a ?reply to all? command. That's almost never > appropriate in a forum like this.) Why not? People reply all to messages I write all the time, and I find it somewhat useful since it separates replies to things I have said from discussions I'm not part of. From yanzhaoxun at greendh.com Thu Feb 4 00:03:33 2016 From: yanzhaoxun at greendh.com (=?UTF-8?B?6ZiO5YWG54+j?=) Date: Thu, 4 Feb 2016 13:03:33 +0800 (CST) Subject: =?UTF-8?B?aW1wb3J0IGNhbm5vdCBiZSB1c2VkIGluc2lkZSBldmFs?= Message-ID: <646074605.1229914.1454562213887.JavaMail.root@wmthree-9> a = input("tell me which py to execute: ?") print(a) print('import '+a) print(type('import'+a)) eval('print(a)') try: ? ? eval('import '+a) except Exception as e: ? ? print('Error: ', e) ##while True: ## ? ?pass @python 3.4.2 this code attempts to run a .py file that user specifies. eval() does fine with print() command but fails to call import command Is it a but or a taboo? Thanks Zhaoxun Yan ? ? ? ? ? Option Product Manager Gelin Dahua Futures Co.Ltd ?T: +86 10 56711783 From ben+python at benfinney.id.au Thu Feb 4 00:26:57 2016 From: ben+python at benfinney.id.au (Ben Finney) Date: Thu, 04 Feb 2016 16:26:57 +1100 Subject: Reply to whom? References: <56B263C1.7030309@mail.de> <56B26940.3090609@mail.de> <56B26D3D.6040703@mail.de> <85h9hpcvjw.fsf_-_@benfinney.id.au> <1454561022.2123037.511515122.153B155A@webmail.messagingengine.com> Message-ID: <85si19avvi.fsf@benfinney.id.au> Random832 writes: > On Wed, Feb 3, 2016, at 16:50, Ben Finney wrote: > > (You will also have a ?reply to all? command. That's almost never > > appropriate in a forum like this.) > > Why not? People reply all to messages I write all the time, and I find > it somewhat useful since it separates replies to things I have said > from discussions I'm not part of. That's nice for you that it coincides with what you find useful. For those who don't want messages to the forum duplicated in their inbox, the behaviour is obnoxious. In other words: Using ?reply to all? on a forum like this is inappropriate because you can't know what every participant wants, so your set of recipients should be conservative and err on the side of not sending a duplicate message to people who didn't ask for it. -- \ ?Don't worry about people stealing your ideas. If your ideas | `\ are any good, you'll have to ram them down people's throats.? | _o__) ?Howard Aiken | Ben Finney From nomail at invalid.com Thu Feb 4 03:03:35 2016 From: nomail at invalid.com (ast) Date: Thu, 4 Feb 2016 09:03:35 +0100 Subject: __bases__ attribute on classes not displayed by dir() command Message-ID: <56b305de$0$3063$426a74cc@news.free.fr> Hi I have a Carre class which inherit from a Rectangle class. Carre has a __bases__ tuple attribute which contains the classes which it inherit from. >>> Carre.__bases__ (,) and Rectangle only inherit from object, so: >>> Rectangle.__bases__ (,) Thats OK but if I am using dir to display all Carre's attributes and methods, __bases__ is not on the list. Why ? >>> dir(Carre) ['__class__', '__delattr__', '__dict__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__gt__', '__hash__', '__init__', '__iter__', '__le__', '__lt__', '__module__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '__weakref__', 'aire', 'count', 'long', 'perimetre', 'rotation'] From nomail at invalid.com Thu Feb 4 03:25:59 2016 From: nomail at invalid.com (ast) Date: Thu, 4 Feb 2016 09:25:59 +0100 Subject: Finding in which class an object's method comes from Message-ID: <56b30b1e$0$3637$426a74cc@news.free.fr> Hi Suppose we have: ClassC inherit from ClassB ClassB inherit from ClassA ClassA inherit from object Let's build an object: obj = ClassC() Let's invoke an obj method obj.funct() funct is first looked in ClassC, then if not found on ClassB, then ClassA then object But is there a command to know where funct is found ? Thx From rosuav at gmail.com Thu Feb 4 03:36:19 2016 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 4 Feb 2016 19:36:19 +1100 Subject: Finding in which class an object's method comes from In-Reply-To: <56b30b1e$0$3637$426a74cc@news.free.fr> References: <56b30b1e$0$3637$426a74cc@news.free.fr> Message-ID: On Thu, Feb 4, 2016 at 7:25 PM, ast wrote: > Hi > > Suppose we have: > > ClassC inherit from ClassB > ClassB inherit from ClassA > ClassA inherit from object > > Let's build an object: > > obj = ClassC() > > Let's invoke an obj method > > obj.funct() > > funct is first looked in ClassC, then if not found > on ClassB, then ClassA then object > > But is there a command to know where funct is > found ? You can see that by looking at the objects without calling them: >>> class A: ... def a(self): pass ... >>> class B(A): ... def b(self): pass ... >>> class C(B): ... def c(self): pass ... >>> obj = C() >>> obj.a > >>> obj.b > >>> obj.c > The information comes from the function's qualname: >>> obj.c.__func__.__qualname__ 'C.c' Is that what you're hoping for? ChrisA From nomail at invalid.com Thu Feb 4 03:43:09 2016 From: nomail at invalid.com (ast) Date: Thu, 4 Feb 2016 09:43:09 +0100 Subject: Finding in which class an object's method comes from In-Reply-To: References: <56b30b1e$0$3637$426a74cc@news.free.fr> Message-ID: <56b30f26$0$3350$426a74cc@news.free.fr> "Chris Angelico" a ?crit dans le message de news:mailman.43.1454574987.30993.python-list at python.org... > > Is that what you're hoping for? > yes, ty From rosuav at gmail.com Thu Feb 4 03:48:43 2016 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 4 Feb 2016 19:48:43 +1100 Subject: import cannot be used inside eval In-Reply-To: <646074605.1229914.1454562213887.JavaMail.root@wmthree-9> References: <646074605.1229914.1454562213887.JavaMail.root@wmthree-9> Message-ID: On Thu, Feb 4, 2016 at 4:03 PM, ??? wrote: > a = input("tell me which py to execute: ") > > print(a) > > print('import '+a) > > print(type('import'+a)) > > eval('print(a)') > > try: > > eval('import '+a) > > except Exception as e: > > print('Error: ', e) > > ##while True: > > ## pass > > @python 3.4.2 > > this code attempts to run a .py file that user specifies. > > eval() does fine with print() command > > but fails to call import command The eval() function evaluates an expression. It won't handle statements (if, for, while), and import is a statement. Possibly you want exec instead. ChrisA From ben+python at benfinney.id.au Thu Feb 4 03:49:49 2016 From: ben+python at benfinney.id.au (Ben Finney) Date: Thu, 04 Feb 2016 19:49:49 +1100 Subject: Finding in which class an object's method comes from References: <56b30b1e$0$3637$426a74cc@news.free.fr> Message-ID: <85h9hoc11u.fsf@benfinney.id.au> "ast" writes: > Let's invoke an obj method > > obj.funct() > > funct is first looked in ClassC, then if not found > on ClassB, then ClassA then object If it helps: What you describe is attribute lookup, and it happens whether or not you're going to call the attribute. In other words, you need only invoke ?foo.attr? to have Python look for attribute ?attr? among the class hierarchy as you describe. Calling that attribute is a distinct operation once the attribute is found. > But is there a command to know where funct is found ? That's been answered, but I'm curious to know what your program will do with that information? -- \ ?But Marge, what if we chose the wrong religion? Each week we | `\ just make God madder and madder.? ?Homer, _The Simpsons_ | _o__) | Ben Finney From nomail at invalid.com Thu Feb 4 03:54:02 2016 From: nomail at invalid.com (ast) Date: Thu, 4 Feb 2016 09:54:02 +0100 Subject: Finding in which class an object's method comes from In-Reply-To: References: <56b30b1e$0$3637$426a74cc@news.free.fr> Message-ID: <56b311b5$0$9219$426a74cc@news.free.fr> "Chris Angelico" a ?crit dans le message de news:mailman.43.1454574987.30993.python-list at python.org... > You can see that by looking at the objects without calling them: > >>>> class A: > ... def a(self): pass > ... >>>> class B(A): > ... def b(self): pass > ... >>>> class C(B): > ... def c(self): pass > ... >>>> obj = C() >>>> obj.a > > >>>> obj.b > > >>>> obj.c > > > > The information comes from the function's qualname: > >>>> obj.c.__func__.__qualname__ > 'C.c' > > Is that what you're hoping for? It is strange but I dont have the same result that you: (Python 3.4) >>> class A: def a(self):pass >>> class B(A): def b(self):pass >>> class C(B): def c(self):pass >>> obj = C() >>> obj.a > But with __qualname__ it works >>> obj.a.__func__.__qualname__ 'A.a' From gherron at digipen.edu Thu Feb 4 03:55:50 2016 From: gherron at digipen.edu (Gary Herron) Date: Thu, 4 Feb 2016 00:55:50 -0800 Subject: import cannot be used inside eval In-Reply-To: <646074605.1229914.1454562213887.JavaMail.root@wmthree-9> References: <646074605.1229914.1454562213887.JavaMail.root@wmthree-9> Message-ID: <56B31216.70007@digipen.edu> On 02/03/2016 09:03 PM, ??? wrote: > a = input("tell me which py to execute: ") > > print(a) > > print('import '+a) > > print(type('import'+a)) > > eval('print(a)') Eval is meant to evaluate Python expressions. The import is a statement, not an expression. Also, it's a bad idea to use eval like this, and it's a *really* bad idea to use eval with user supplied input. The user could inject *any* malicious code. Instead, use the importlib module to programmatically import a module. Gary Herron -- Dr. Gary Herron Department of Computer Science DigiPen Institute of Technology (425) 895-4418 From eryksun at gmail.com Thu Feb 4 03:56:46 2016 From: eryksun at gmail.com (eryk sun) Date: Thu, 4 Feb 2016 02:56:46 -0600 Subject: __bases__ attribute on classes not displayed by dir() command In-Reply-To: <56b305de$0$3063$426a74cc@news.free.fr> References: <56b305de$0$3063$426a74cc@news.free.fr> Message-ID: On Thu, Feb 4, 2016 at 2:03 AM, ast wrote: > but if I am using dir to display all Carre's attributes and methods, > __bases__ is not on the list. Why ? The __bases__ property is defined by the metaclass, "type". dir() of a class doesn't show attributes from the metaclass [1]. Because dir() is supplied primarily as a convenience for use at an interactive prompt, it tries to supply an interesting set of names more than it tries to supply a rigorously or consistently defined set of names, and its detailed behavior may change across releases. For example, metaclass attributes are not in the result list when the argument is a class. To see the metaclass attributes use dir(type(Carre)). [1]: https://docs.python.org/3/library/functions.html#dir From nomail at invalid.com Thu Feb 4 03:59:33 2016 From: nomail at invalid.com (ast) Date: Thu, 4 Feb 2016 09:59:33 +0100 Subject: Finding in which class an object's method comes from In-Reply-To: References: <56b30b1e$0$3637$426a74cc@news.free.fr> Message-ID: <56b312fe$0$669$426a74cc@news.free.fr> "Ben Finney" a ?crit dans le message de news:mailman.48.1454575803.30993.python-list at python.org... > > That's been answered, but I'm curious to know what your program will do > with that information? > Nothings, I was just wondering. From rosuav at gmail.com Thu Feb 4 04:00:14 2016 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 4 Feb 2016 20:00:14 +1100 Subject: Finding in which class an object's method comes from In-Reply-To: <56b311b5$0$9219$426a74cc@news.free.fr> References: <56b30b1e$0$3637$426a74cc@news.free.fr> <56b311b5$0$9219$426a74cc@news.free.fr> Message-ID: On Thu, Feb 4, 2016 at 7:54 PM, ast wrote: > It is strange but I dont have the same result that you: > (Python 3.4) > >>>> class A: > > def a(self):pass > >>>> class B(A): > def b(self):pass > >>>> class C(B): > def c(self):pass > >>>> obj = C() > >>>> obj.a > > > Curious. It appears to have changed between 3.4 and 3.5; my original copy/paste was from 3.6. ChrisA From nomail at invalid.com Thu Feb 4 04:11:40 2016 From: nomail at invalid.com (ast) Date: Thu, 4 Feb 2016 10:11:40 +0100 Subject: __bases__ attribute on classes not displayed by dir() command In-Reply-To: References: <56b305de$0$3063$426a74cc@news.free.fr> Message-ID: <56b315d5$0$19756$426a74cc@news.free.fr> "eryk sun" a ?crit dans le message de news:mailman.49.1454576255.30993.python-list at python.org... > On Thu, Feb 4, 2016 at 2:03 AM, ast wrote: >> but if I am using dir to display all Carre's attributes and methods, >> __bases__ is not on the list. Why ? > > The __bases__ property is defined by the metaclass, "type". dir() of a > class doesn't show attributes from the metaclass [1]. > Oh metaclass ! I am learning them at the moment, and have headaches From oscar.j.benjamin at gmail.com Thu Feb 4 04:30:52 2016 From: oscar.j.benjamin at gmail.com (Oscar Benjamin) Date: Thu, 4 Feb 2016 09:30:52 +0000 Subject: Install Error In-Reply-To: References: Message-ID: On 3 February 2016 at 21:55, Barrie Taylor wrote: > > I am attempting to install and run Python3.5.1 on my Windows machine. > > After installation on launching I am presented the attached error message. > It reads: > 'The program can't start because api-ms-win-crt-runtime-l1-1-0.dll is missing from your computer. Try reinstalling the program to fix this problem.' > Needless to say, I have tried uninstalling and reinstalling ad repairing the program a number of times to no avail. I don't know why it says to try re-installing. The problem is that Python 3.5 requires an update to your system runtimes from Microsoft. You can find more information about that here: https://support.microsoft.com/en-us/kb/2999226 -- Oscar From oscar.j.benjamin at gmail.com Thu Feb 4 04:32:48 2016 From: oscar.j.benjamin at gmail.com (Oscar Benjamin) Date: Thu, 4 Feb 2016 09:32:48 +0000 Subject: please help In-Reply-To: References: Message-ID: On 3 February 2016 at 23:03, Syavosh Malek wrote: > hi i install python 3.5.1 and found run time error > see attach file and help me please I'm afraid your attachment didn't arrive as this is a text-only mailing list. Can you include more information about the error? If it's that you're missing a dll called something like Api-ms-win-crt-runtime-l1-1-0.dll then you need to update your system runtimes from Microsoft. You can read about that here: https://support.microsoft.com/en-us/kb/2999226 -- Oscar From jfong at ms4.hinet.net Thu Feb 4 04:33:02 2016 From: jfong at ms4.hinet.net (jfong at ms4.hinet.net) Date: Thu, 4 Feb 2016 01:33:02 -0800 (PST) Subject: How this C function was called through ctypes this way? Message-ID: <2bcaa079-f7ec-4c1e-9d18-c607247eb239@googlegroups.com> Here is an example from "Python Cookbook, Third Edition(by David Beazley and Brian K. Jones)" Chapter 15.1. "Accessing C Code Using ctypes" ------------------- import ctypes ... # Try to locate the .so file in the same directory as this file ... _mod = ctypes.cdll.LoadLibrary(_path) ... ... # void avg(double *, int n) # Define a special type for the 'double *' argument class DoubleArrayType: def from_param(self, param): typename = type(param).__name__ if hasattr(self, 'from_' + typename): return getattr(self, 'from_' + typename)(param) elif isinstance(param, ctypes.Array): return param else: raise TypeError("Can't convert %s" % typename) ... # Cast from array.array objects def from_array(self, param): ... ... # Cast from lists def from_list(self, param): val = ((ctypes.c_double)*len(param))(*param) return val # Cast from a numpy array def from_ndarray(self, param): ... ... DoubleArray = DoubleArrayType() _avg = _mod.avg _avg.argtypes = (DoubleArray, ctypes.c_int) _avg.restype = ctypes.c_double def avg(values): return _avg(values, len(values)) avg([1,2,3]) ------------------ The followings was quoted from the book which explain how it does: "The DoubleArrayType class shows how to handle this situation. In this class, a single method from_param() is defined. The role of this method is to take a single parameter and narrow it down to a compatible ctypes object (a pointer to a ctypes.c_double, in the example). Within from_param(), you are free to do anything that you wish. In the solution, the typename of the parameter is extracted and used to dispatch to a more specialized method. For example, if a list is passed, the typename is list and a method from_list() is invoked." What confuse me are: (1) at line: _avg.argtypes = (DoubleArray, ctypes.c_int) The "DoubleArry" is an instance of the class "DoubleArrayType", Can it appear at where a type was expected? (2) How the method "from_param" was invoked? I can't see any mechanism to reach it from the "_avg(values, len(values))" call. Best Regards, Jach Fong From wolfgang.maier at biologie.uni-freiburg.de Thu Feb 4 04:37:18 2016 From: wolfgang.maier at biologie.uni-freiburg.de (Wolfgang Maier) Date: Thu, 4 Feb 2016 10:37:18 +0100 Subject: Finding in which class an object's method comes from In-Reply-To: References: <56b30b1e$0$3637$426a74cc@news.free.fr> <56b311b5$0$9219$426a74cc@news.free.fr> Message-ID: On 04.02.2016 10:00, Chris Angelico wrote: > On Thu, Feb 4, 2016 at 7:54 PM, ast wrote: >> It is strange but I dont have the same result that you: >> (Python 3.4) >> >>>>> class A: >> >> def a(self):pass >> >>>>> class B(A): >> def b(self):pass >> >>>>> class C(B): >> def c(self):pass >> >>>>> obj = C() >> >>>>> obj.a >> >> > > > Curious. It appears to have changed between 3.4 and 3.5; my original > copy/paste was from 3.6. > I think http://bugs.python.org/issue21389 causes the difference. I think the change was introduced with Python3.5. From mdelamo90 at gmail.com Thu Feb 4 04:45:19 2016 From: mdelamo90 at gmail.com (mdelamo90 at gmail.com) Date: Thu, 4 Feb 2016 01:45:19 -0800 (PST) Subject: Multiprocess videoplayer Message-ID: <6705fdb6-a5dd-4a2d-9dbc-04a557ec1026@googlegroups.com> I have coded a program with python and vlc that plays some videos, but whenever I try to play 3 videos at once, Windows closes the program, I'm guessing that the reason is that one process can't play 3 videos at once (but I don't really know). My trimmed program plays one video (trhough a global variable, os it's easy to change) and once you press the number 2 on the keyboard it put the data from a video on the queue to be played by another process. The new process starts and makes some prints, then it should start the new video and another one, so we should have two videos playing, one on the left part of the screen and the other on the right. I've done this on linear programming, but I can't make the new process to start the videos. I oame from a C/C++ background so I don't know if it's something I didn't fully grasp about the multiprocess on Python or something else that I'm doing wrong. Its huge for a miniexample (300 lines), but I didn't know how to make it shorter: # import external libraries import wx # 2.8 import vlc import pdb # import standard libraries import user import urllib import multiprocessing VIDEO1 = "Insert_your_first_video.mp4" VIDEO2 = "Insert_your_second_video.mp4" class MyOpener(urllib.FancyURLopener): version = "App/1.7" # doesn't work version = "Mozilla/4.0 (MSIE 6.0; Windows NT 5.0)2011-03-10 15:38:34" # works class Consumer(multiprocessing.Process): def __init__(self, task_queue): multiprocessing.Process.__init__(self) self.task_queue = task_queue def run(self): app = wx.App() proc_name = self.name print "waiting for queue" while True: # While queue not empy print self.task_queue.qsize() next_task = self.task_queue.get() global dual, midsection dual = True midsection = next_task.d player2 = Player("Dual PyVLC Player") player2.Centre() # show the player window centred and run the application print "parametro a " + str(next_task.a) print "parametro b " + str(next_task.b) print "parametro c " + str(next_task.c) print "parametro d " + str(next_task.d) # tasks.put(Task(media, dual, time, midsection)) player2.playFile(next_task.a,next_task.c,True) player2.playFile(VIDEO2,next_task.c,True) #player.vidplayer1.set_media(next_task.a) #player.vidplayer2.play() player2.Maximize(True) player2.OnFullscreen(None) player2.SetTransparent(255) player2.SetFocus() player2.Show() #player2.vidplayer1.set_title(next_task.a) '''player1.SetTransparent(0) player1.timer1.Start(fadetime) player1.set_amount(0)''' def playFile(self,moviefile,time,principal): # Creation self.Media = self.Instance.media_new_path(moviefile) self.SetTitle("Monkey") #self.SetIcon(wx.Icon('Monkey.ico', wx.BITMAP_TYPE_ICO)) if dual: if principal: #If we dont set any handle it starts in another window, maybe usefull for dual screens? self.vidplayer1.set_media(self.Media) #self.vlchandle = self.vidplayer1.get_xwindow() self.vidplayer1.set_hwnd(self.videopanel.GetHandle()) self.OnPlay(None,True) self.vidplayer1.set_time(time) else: self.vidplayer2.set_media(self.Media) #self.vlchandle = self.vidplayer2.get_xwindow() self.vidplayer2.set_hwnd(self.videopanel2.GetHandle()) self.OnPlay(None,False) self.vidplayer2.set_time(time) else: self.vidplayer1.set_media(self.Media) #self.vlchandle = self.vidplayer1.get_xwindow() self.vidplayer1.set_hwnd(self.videopanel.GetHandle()) self.OnPlay(None,True) self.vidplayer1.set_time(time) class Task(object): def __init__(self, a, b, c, d): self.a = a # can't be media, it's not pickable self.b = b # boolean self.c = c # Number (time) self.d = d # boolean def __str__(self): return '%s * %s' % (self.a, self.b, self.c, self.d) class Player(wx.Frame): """The main window has to deal with events. """ def __init__(self, title, OS="windows"): self.OS = OS wx.Frame.__init__(self, None, -1, title, pos=wx.DefaultPosition, size=(950, 500)) self.SetBackgroundColour(wx.BLACK) # Panels # The first panel holds the video/videos and it's all black self.videopanel = wx.Panel(self, -1) self.videopanel.SetBackgroundStyle(wx.BG_STYLE_COLOUR) self.videopanel.SetBackgroundColour(wx.BLACK) if dual: videopanel2 = wx.Panel(self, -1) self.videopanel2 = videopanel2 self.videopanel2.SetBackgroundStyle(wx.BG_STYLE_COLOUR) self.videopanel2.SetBackgroundColour(wx.BLACK) # Bind events self.videopanel.Bind(wx.EVT_KEY_DOWN, self.OnKeyDown) self.videopanel.Bind(wx.EVT_ENTER_WINDOW, self.OnMouseLeaveCtrlpanel) if dual: self.videopanel2.Bind(wx.EVT_ENTER_WINDOW, self.OnMouseLeaveCtrlpanel) # finally create the timer self.timer = wx.Timer(self) self.Bind(wx.EVT_TIMER, self.OnTimer, self.timer) # VLC player controls plugin_path = None if self.OS == "linux": plugin_path == "/usr/lib/vlc/plugins" elif self.OS == "windows": plugin_path == "C:\\Program Files (x86)\\VideoLAN\\VLC\\plugins" if plugin_path: self.Instance = vlc.Instance("--plugin-path %s" % plugin_path) else: self.Instance = vlc.Instance('--input-repeat=-1 --reset-config --no-video-title-show')# --start-time=10') #print "VLC Instance: %s" % self.Instance #We create instances of VLC, can we make them in the same Frame? self.vidplayer1 = self.Instance.media_player_new() self.vidplayer2 = self.Instance.media_player_new() def OnKeyDown(self, event): global dual,time,midsection,sizer1,To_pause keycode = event.GetKeyCode() print keycode if keycode == 50: # 2 if dual == False: # We change to dual video time = player.vidplayer1.get_time() media = player.vidplayer1.get_media() player.vidplayer1.stop() player.SetTransparent(0) tasks.put(Task(VIDEO1, dual, time, midsection)) else: # We change to mono if player2.vidplayer1.get_time() != -1: time = player2.vidplayer1.get_time() media = player2.vidplayer1.get_media() player2.vidplayer2.stop() player.vidplayer1.set_media(media) player1.vidplayer1.stop() player.vidplayer1.play() player.vidplayer1.set_time(time) sleep(0.8) player2.vidplayer1.stop() player2.Hide() if To_pause == False: player.vidplayer1.pause() player.Show(True) else: time = player3.vidplayer1.get_time() media = player3.vidplayer1.get_media() player2.vidplayer2.set_media(player3.vidplayer2.get_media()) player3.vidplayer2.stop() player.vidplayer1.set_media(media) player3.vidplayer1.stop() player.vidplayer1.play() player.vidplayer1.set_time(time) sleep(0.8) player3.vidplayer1.stop() player3.Hide() if To_pause == False: player.vidplayer1.pause() player.Show(True) dual = not dual def playFile(self,moviefile,time,principal): # Creation self.Media = self.Instance.media_new_path(moviefile) self.SetTitle("Monkey") #self.SetIcon(wx.Icon('Monkey.ico', wx.BITMAP_TYPE_ICO)) if dual: if principal: #If we dont set any handle it starts in another window, maybe usefull for dual screens? self.vidplayer1.set_media(self.Media) #self.vlchandle = self.vidplayer1.get_xwindow() self.vidplayer1.set_hwnd(self.videopanel.GetHandle()) self.OnPlay(None,True) self.vidplayer1.set_time(time) else: self.vidplayer2.set_media(self.Media) #self.vlchandle = self.vidplayer2.get_xwindow() self.vidplayer2.set_hwnd(self.videopanel2.GetHandle()) self.OnPlay(None,False) self.vidplayer2.set_time(time) else: self.vidplayer1.set_media(self.Media) #self.vlchandle = self.vidplayer1.get_xwindow() self.vidplayer1.set_hwnd(self.videopanel.GetHandle()) self.OnPlay(None,True) self.vidplayer1.set_time(time) # set the volume slider to the current volume #self.volslider.SetValue(self.vidplayer.audio_get_volume() / 2) def OnFullscreen(self,evt): self.Show() self.ShowFullScreen(not self.IsFullScreen()) def OnPlay(self, evt, principal): """Toggle the status to Play/Pause. If no file is loaded, open the dialog window. """ # check if there is a file to play, otherwise open a # wx.FileDialog to select a file if principal: if not self.vidplayer1.get_media(): self.OnOpen(None) else: # Try to launch the media, if this fails display an error message if self.vidplayer1.play() == -1: self.errorDialog("Unable to play.") else: #Set timer to callback every 500 ms timeslider moves every 0'5s self.timer.Start(500) else: if not self.vidplayer2.get_media(): self.OnOpen(None) else: # Try to launch the media, if this fails display an error message if self.vidplayer2.play() == -1: self.errorDialog("Unable to play.") else: #Set timer to callback every 500 ms timeslider moves every 0'5s self.timer.Start(500) def OnTimer(self, evt): """Update the time slider according to the current movie time. """ # update the time time = self.vidplayer1.get_time() time = self.vidplayer1.get_time() def OnMouseLeaveCtrlpanel(self,evt): #print "Mouse leave ctrlpanel" if self.IsFullScreen() and not self.showcontrols.GetValue(): self.ctrlpanel.Hide() self.GetSizer().Layout() def OnSetTime(self, evt): """Set the time according to the time sider. """ time = self.timeslider.GetValue() # vlc.MediaPlayer.set_time returns 0 if success, -1 otherwise if self.vidplayer1.set_time(time) == -1: self.errorDialog("Failed to set time") if __name__ == "__main__": global dual,midsection, tasks midsection = True tasks = multiprocessing.JoinableQueue() num_consumers = 2 consumers = [Consumer(tasks) for i in xrange(num_consumers)] for w in consumers: w.start() # Create a wx.App(), which handles the windowing system event loop app = wx.App() width, height = wx.GetDisplaySize() # Create the windows containing our media players dual = False # Inheritance and shit that? player1 = Player("Single PyVLC Player") player = Player("Single PyVLC Player") dual = True player.Centre() player.playFile(VIDEO1,0,True) player.Maximize(True) player.OnFullscreen(None) player.vidplayer1.play() #The machine can't play 3 vids at once... player.Show(True) dual = False app.MainLoop() From polyvertex at gmail.com Thu Feb 4 05:19:46 2016 From: polyvertex at gmail.com (Jean-Charles Lefebvre) Date: Thu, 4 Feb 2016 02:19:46 -0800 (PST) Subject: How a module is being marked as imported? Message-ID: <325bc056-0683-408b-85ba-6c3cd0f69d41@googlegroups.com> Hi all, The short version: How CPython marks a module as being fully imported, if it does, so that the same import statement ran from another C thread at the same time does not collide? Or, reversely, does not think the module is not already fully imported? The full version: I'm running CPython 3.5.1, embedded into a C++ application on Windows. The application is heavily multi-threaded so several C threads call some Python code at the same time (different Python modules), sharing interpreter's resources by acquiring/releasing the GIL frequently DURING the calls, at language boundaries. Sometimes (but always only once per application instance), a call to os.path.expandvars raises the AttributeError exception with message: module 'string' has no attribute 'ascii_letters'. It is raised by the ntpath.expandvars function (line 372). When I noticed the late import statement of the 'string' module at the line above, I thought that MAYBE, it could be because the interpreter is ran in an heavily multi-threaded environment and that the GIL acquiring/releasing occurred at a bad timing? Making me wonder how the import mechanism interacts with the GIL, if it does? From mdelamo90 at gmail.com Thu Feb 4 05:35:31 2016 From: mdelamo90 at gmail.com (mdelamo90 at gmail.com) Date: Thu, 4 Feb 2016 02:35:31 -0800 (PST) Subject: multiprocessing videoplayer Message-ID: I have coded a program with python and vlc that plays some videos, but whenever I try to play 3 videos at once, Windows closes the program, I'm guessing that the reason is that one process can't play 3 videos at once (but I don't really know). My trimmed program plays one video (trhough a global variable, os it's easy to change) and once you press the number 2 on the keyboard it put the data from a video on the queue to be played by another process. The new process starts and makes some prints, then it should start the new video and another one, so we should have two videos playing, one on the left part of the screen and the other on the right. I've done this on linear programming, but I can't make the new process to start the videos. I oame from a C/C++ background so I don't know if it's something I didn't fully grasp about the multiprocess on Python or something else that I'm doing wrong. Its huge for a miniexample (300 lines), but I didn't know how to make it shorter: # import external libraries import wx # 2.8 import vlc import pdb # import standard libraries import user import urllib import multiprocessing VIDEO1 = "Insert_your_first_video.mp4" VIDEO2 = "Insert_your_second_video.mp4" class MyOpener(urllib.FancyURLopener): version = "App/1.7" # doesn't work version = "Mozilla/4.0 (MSIE 6.0; Windows NT 5.0)2011-03-10 15:38:34" # works class Consumer(multiprocessing.Process): def __init__(self, task_queue): multiprocessing.Process.__init__(self) self.task_queue = task_queue def run(self): app = wx.App() proc_name = self.name print "waiting for queue" while True: # While queue not empy print self.task_queue.qsize() next_task = self.task_queue.get() global dual, midsection dual = True midsection = next_task.d player2 = Player("Dual PyVLC Player") player2.Centre() # show the player window centred and run the application print "parametro a " + str(next_task.a) print "parametro b " + str(next_task.b) print "parametro c " + str(next_task.c) print "parametro d " + str(next_task.d) # tasks.put(Task(media, dual, time, midsection)) player2.playFile(next_task.a,next_task.c,True) player2.playFile(VIDEO2,next_task.c,False) #player.vidplayer1.set_media(next_task.a) #player.vidplayer2.play() player2.Maximize(True) player2.OnFullscreen(None) player2.SetTransparent(255) player2.SetFocus() player2.Show() #player2.vidplayer1.set_title(next_task.a) '''player1.SetTransparent(0) player1.timer1.Start(fadetime) player1.set_amount(0)''' def playFile(self,moviefile,time,principal): # Creation self.Media = self.Instance.media_new_path(moviefile) self.SetTitle("Monkey") #self.SetIcon(wx.Icon('Monkey.ico', wx.BITMAP_TYPE_ICO)) if dual: if principal: #If we dont set any handle it starts in another window, maybe usefull for dual screens? self.vidplayer1.set_media(self.Media) #self.vlchandle = self.vidplayer1.get_xwindow() self.vidplayer1.set_hwnd(self.videopanel.GetHandle()) self.OnPlay(None,True) self.vidplayer1.set_time(time) else: self.vidplayer2.set_media(self.Media) #self.vlchandle = self.vidplayer2.get_xwindow() self.vidplayer2.set_hwnd(self.videopanel2.GetHandle()) self.OnPlay(None,False) self.vidplayer2.set_time(time) else: self.vidplayer1.set_media(self.Media) #self.vlchandle = self.vidplayer1.get_xwindow() self.vidplayer1.set_hwnd(self.videopanel.GetHandle()) self.OnPlay(None,True) self.vidplayer1.set_time(time) class Task(object): def __init__(self, a, b, c, d): self.a = a # can't be media, it's not pickable self.b = b # boolean self.c = c # Number (time) self.d = d # boolean def __str__(self): return '%s * %s' % (self.a, self.b, self.c, self.d) class Player(wx.Frame): """The main window has to deal with events. """ def __init__(self, title, OS="windows"): self.OS = OS wx.Frame.__init__(self, None, -1, title, pos=wx.DefaultPosition, size=(950, 500)) self.SetBackgroundColour(wx.BLACK) # Panels # The first panel holds the video/videos and it's all black self.videopanel = wx.Panel(self, -1) self.videopanel.SetBackgroundStyle(wx.BG_STYLE_COLOUR) self.videopanel.SetBackgroundColour(wx.BLACK) if dual: videopanel2 = wx.Panel(self, -1) self.videopanel2 = videopanel2 self.videopanel2.SetBackgroundStyle(wx.BG_STYLE_COLOUR) self.videopanel2.SetBackgroundColour(wx.BLACK) # Bind events self.videopanel.Bind(wx.EVT_KEY_DOWN, self.OnKeyDown) self.videopanel.Bind(wx.EVT_ENTER_WINDOW, self.OnMouseLeaveCtrlpanel) if dual: self.videopanel2.Bind(wx.EVT_ENTER_WINDOW, self.OnMouseLeaveCtrlpanel) # finally create the timer self.timer = wx.Timer(self) self.Bind(wx.EVT_TIMER, self.OnTimer, self.timer) # VLC player controls plugin_path = None if self.OS == "linux": plugin_path == "/usr/lib/vlc/plugins" elif self.OS == "windows": plugin_path == "C:\\Program Files (x86)\\VideoLAN\\VLC\\plugins" if plugin_path: self.Instance = vlc.Instance("--plugin-path %s" % plugin_path) else: self.Instance = vlc.Instance('--input-repeat=-1 --reset-config --no-video-title-show')# --start-time=10') #print "VLC Instance: %s" % self.Instance #We create instances of VLC, can we make them in the same Frame? self.vidplayer1 = self.Instance.media_player_new() self.vidplayer2 = self.Instance.media_player_new() def OnKeyDown(self, event): global dual,time,midsection,sizer1,To_pause keycode = event.GetKeyCode() print keycode if keycode == 50: # 2 if dual == False: # We change to dual video time = player.vidplayer1.get_time() media = player.vidplayer1.get_media() player.vidplayer1.stop() player.SetTransparent(0) tasks.put(Task(VIDEO1, dual, time, midsection)) else: # We change to mono if player2.vidplayer1.get_time() != -1: time = player2.vidplayer1.get_time() media = player2.vidplayer1.get_media() player2.vidplayer2.stop() player.vidplayer1.set_media(media) player1.vidplayer1.stop() player.vidplayer1.play() player.vidplayer1.set_time(time) sleep(0.8) player2.vidplayer1.stop() player2.Hide() if To_pause == False: player.vidplayer1.pause() player.Show(True) else: time = player3.vidplayer1.get_time() media = player3.vidplayer1.get_media() player2.vidplayer2.set_media(player3.vidplayer2.get_media()) player3.vidplayer2.stop() player.vidplayer1.set_media(media) player3.vidplayer1.stop() player.vidplayer1.play() player.vidplayer1.set_time(time) sleep(0.8) player3.vidplayer1.stop() player3.Hide() if To_pause == False: player.vidplayer1.pause() player.Show(True) dual = not dual def playFile(self,moviefile,time,principal): # Creation self.Media = self.Instance.media_new_path(moviefile) self.SetTitle("Monkey") #self.SetIcon(wx.Icon('Monkey.ico', wx.BITMAP_TYPE_ICO)) if dual: if principal: #If we dont set any handle it starts in another window, maybe usefull for dual screens? self.vidplayer1.set_media(self.Media) #self.vlchandle = self.vidplayer1.get_xwindow() self.vidplayer1.set_hwnd(self.videopanel.GetHandle()) self.OnPlay(None,True) self.vidplayer1.set_time(time) else: self.vidplayer2.set_media(self.Media) #self.vlchandle = self.vidplayer2.get_xwindow() self.vidplayer2.set_hwnd(self.videopanel2.GetHandle()) self.OnPlay(None,False) self.vidplayer2.set_time(time) else: self.vidplayer1.set_media(self.Media) #self.vlchandle = self.vidplayer1.get_xwindow() self.vidplayer1.set_hwnd(self.videopanel.GetHandle()) self.OnPlay(None,True) self.vidplayer1.set_time(time) # set the volume slider to the current volume #self.volslider.SetValue(self.vidplayer.audio_get_volume() / 2) def OnFullscreen(self,evt): self.Show() self.ShowFullScreen(not self.IsFullScreen()) def OnPlay(self, evt, principal): """Toggle the status to Play/Pause. If no file is loaded, open the dialog window. """ # check if there is a file to play, otherwise open a # wx.FileDialog to select a file if principal: if not self.vidplayer1.get_media(): self.OnOpen(None) else: # Try to launch the media, if this fails display an error message if self.vidplayer1.play() == -1: self.errorDialog("Unable to play.") else: #Set timer to callback every 500 ms timeslider moves every 0'5s self.timer.Start(500) else: if not self.vidplayer2.get_media(): self.OnOpen(None) else: # Try to launch the media, if this fails display an error message if self.vidplayer2.play() == -1: self.errorDialog("Unable to play.") else: #Set timer to callback every 500 ms timeslider moves every 0'5s self.timer.Start(500) def OnTimer(self, evt): """Update the time slider according to the current movie time. """ # update the time time = self.vidplayer1.get_time() time = self.vidplayer1.get_time() def OnMouseLeaveCtrlpanel(self,evt): #print "Mouse leave ctrlpanel" if self.IsFullScreen() and not self.showcontrols.GetValue(): self.ctrlpanel.Hide() self.GetSizer().Layout() def OnSetTime(self, evt): """Set the time according to the time sider. """ time = self.timeslider.GetValue() # vlc.MediaPlayer.set_time returns 0 if success, -1 otherwise if self.vidplayer1.set_time(time) == -1: self.errorDialog("Failed to set time") if __name__ == "__main__": global dual,midsection, tasks midsection = True tasks = multiprocessing.JoinableQueue() num_consumers = 2 consumers = [Consumer(tasks) for i in xrange(num_consumers)] for w in consumers: w.start() # Create a wx.App(), which handles the windowing system event loop app = wx.App() width, height = wx.GetDisplaySize() # Create the windows containing our media players dual = False # Inheritance and shit that? player1 = Player("Single PyVLC Player") player = Player("Single PyVLC Player") dual = True player.Centre() player.playFile(VIDEO1,0,True) player.Maximize(True) player.OnFullscreen(None) player.vidplayer1.play() #The machine can't play 3 vids at once... player.Show(True) dual = False app.MainLoop() From eryksun at gmail.com Thu Feb 4 06:33:59 2016 From: eryksun at gmail.com (eryk sun) Date: Thu, 4 Feb 2016 05:33:59 -0600 Subject: How this C function was called through ctypes this way? In-Reply-To: <2bcaa079-f7ec-4c1e-9d18-c607247eb239@googlegroups.com> References: <2bcaa079-f7ec-4c1e-9d18-c607247eb239@googlegroups.com> Message-ID: On Thu, Feb 4, 2016 at 3:33 AM, wrote: > > class DoubleArrayType: > def from_param(self, param): > > [snip] > > DoubleArray = DoubleArrayType() > _avg = _mod.avg > _avg.argtypes = (DoubleArray, ctypes.c_int) > > [snip] > > What confuse me are: > (1) at line: _avg.argtypes = (DoubleArray, ctypes.c_int) > The "DoubleArry" is an instance of the class "DoubleArrayType", > Can it appear at where a type was expected? ctypes generally (notwithstanding out parameters defined via paramflags) requires only that an object set in argtypes has a from_param callable to check and convert the corresponding argument. Usually classes are set in argtypes, so usually from_param is a classmethod. In this case the author chose to set an instance in argtypes and use an instance method. While unusual, this works all the same. > (2) How the method "from_param" was invoked? I can't see any > mechanism to reach it from the "_avg(values, len(values))" call. A ctypes function pointer is a callable implemented in C. Its tp_call slot function is PyCFuncPtr in Modules/_ctypes/_ctypes.c, which in turn calls _ctypes_callproc in Modules/_ctypes/callproc.c. Here's the snippet of code from _ctypes_callproc that's responsible for calling the from_param converter: converter = PyTuple_GET_ITEM(argtypes, i); v = PyObject_CallFunctionObjArgs(converter, arg, NULL); Note that "argtypes" in the above is a misnomer; it's actually the tuple of from_param converters. Each from_param return value is passed to ConvParam, which handles ctypes objects, integers, strings, and None. If the object isn't of those but has an _as_parameter_ attribute, ConvParam calls itself recursively using the _as_parameter_ value. > _mod = ctypes.cdll.LoadLibrary(_path) Calling ctypes.CDLL directly is preferable since it allows passing parameters such as "mode" and "use_errno". IMO, the ctypes.cdll and ctypes.windll loaders should be avoided in general, especially on Windows, since their attribute-based access (e.g. windll.user32) caches libraries, which in turn cache function-pointer attributes. You don't want function pointer instances being shared across unrelated packages. They may not use compatible prototypes and errcheck functions. Each package, module, or script should create private instances of CDLL, PyDLL, and WinDLL for a given shared library. From salony.permanand at gmail.com Thu Feb 4 07:22:32 2016 From: salony.permanand at gmail.com (Salony Permanand) Date: Thu, 4 Feb 2016 17:52:32 +0530 Subject: problem in installing python In-Reply-To: References: Message-ID: hello sir, During working on python I need urllib2 for my python version 2.7.11. Kindly provide me address from where to download it.. Thanking you. On Wed, Feb 3, 2016 at 4:27 PM, Salony Permanand wrote: > Thankyou for consideration..I have solved my problem by changing name of > temp files by "temp1" > > ---------- Forwarded message ---------- > From: eryk sun > Date: Wed, Feb 3, 2016 at 3:55 PM > Subject: Re: problem in installing python > To: python-list at python.org > Cc: Salony Permanand > > > On Wed, Feb 3, 2016 at 12:57 AM, Salony Permanand > wrote: > > > > I downloaded different version of python but no one is installed on my pc > > because of same installation error each time having error code 2203. > > 2203 may be a Windows Installer error [1]. If so, the error message > has the following template: > > Database: [2]. Cannot open database file. System error [3]. > > Please provide the "System error", since it may help to determine why > the installer can't open the file. Possibly an anti-virus program is > interfering, in which case temporarily disabling it may solve the > problem. > > [1]: https://msdn.microsoft.com/en-us/library/aa372835 > > From mdelamo90 at gmail.com Thu Feb 4 07:37:50 2016 From: mdelamo90 at gmail.com (mdelamo90 at gmail.com) Date: Thu, 4 Feb 2016 04:37:50 -0800 (PST) Subject: Non working Parallel videoplayer Message-ID: <75f9290f-dc92-4426-a42b-05b45a0d489b@googlegroups.com> I have coded a program with python and vlc that plays some videos, but whenever I try to play 3 videos at once, Windows closes the program, I'm guessing that the reason is that one process can't play 3 videos at once (but I don't really know). My trimmed program plays one video (trhough a global variable, os it's easy to change) and once you press the number 2 on the keyboard it put the data from a video on the queue to be played by another process. The new process starts and makes some prints, then it should start the new video and another one, so we should have two videos playing, one on the left part of the screen and the other on the right. I've done this on linear programming, but I can't make the new process to start the videos. I oame from a C/C++ background so I don't know if it's something I didn't fully grasp about the multiprocess on Python or something else that I'm doing wrong. Here is the linear working version (commented lines 206-220) and the parallel non-working version lines (221-225) Comment/Uncomment to toogle bewtween them. Its huge for a miniexample (383 lines), but I didn't know how to make it shorter (it comes from a 3000+ project): # import external libraries import wx # 2.8 import vlc import pdb from time import sleep # import standard libraries import user import urllib VIDEO1 = "Insert_your_first_video.mp4" VIDEO2 = "Insert_your_second_video.mp4" class MyOpener(urllib.FancyURLopener): version = "App/1.7" # doesn't work version = "Mozilla/4.0 (MSIE 6.0; Windows NT 5.0)2011-03-10 15:38:34" # works class Consumer(multiprocessing.Process): def __init__(self, task_queue): multiprocessing.Process.__init__(self) self.task_queue = task_queue def run(self): app = wx.App() proc_name = self.name print "waiting for queue" while True: # While queue not empy print self.task_queue.qsize() next_task = self.task_queue.get() global dual, midsection midsection = next_task.d dual = True player2 = Player("Dual PyVLC Player") player2.Centre() # show the player window centred and run the application print "parametro a " + str(next_task.a) print "parametro b " + str(next_task.b) print "parametro c " + str(next_task.c) print "parametro d " + str(next_task.d) # tasks.put(Task(media, dual, time, midsection)) player2.playFile(VIDEO1,next_task.c,True) player2.playFile(VIDEO2,next_task.c,False) #player.vidplayer1.set_media(next_task.a) #player.vidplayer2.play() player2.Maximize(True) player2.OnFullscreen(None) player2.SetTransparent(255) player2.SetFocus() player2.Show(True) #sleep(10000) #player2.vidplayer1.set_title(next_task.a) '''player1.SetTransparent(0) player1.timer1.Start(fadetime) player1.set_amount(0)''' def playFile(self,moviefile,time,principal): # Creation self.Media = self.Instance.media_new_path(moviefile) self.SetTitle("Monkey") #self.SetIcon(wx.Icon('Monkey.ico', wx.BITMAP_TYPE_ICO)) if dual: if principal: #If we dont set any handle it starts in another window, maybe usefull for dual screens? self.vidplayer1.set_media(self.Media) #self.vlchandle = self.vidplayer1.get_xwindow() self.vidplayer1.set_hwnd(self.videopanel.GetHandle()) self.OnPlay(None,True) self.vidplayer1.set_time(time) else: self.vidplayer2.set_media(self.Media) #self.vlchandle = self.vidplayer2.get_xwindow() self.vidplayer2.set_hwnd(self.videopanel2.GetHandle()) self.OnPlay(None,False) self.vidplayer2.set_time(time) else: self.vidplayer1.set_media(self.Media) #self.vlchandle = self.vidplayer1.get_xwindow() self.vidplayer1.set_hwnd(self.videopanel.GetHandle()) self.OnPlay(None,True) self.vidplayer1.set_time(time) class Task(object): def __init__(self, a, b, c, d): self.a = a # can't be media, it's not pickable self.b = b # boolean self.c = c # Number (time) self.d = d # boolean def __str__(self): return '%s * %s' % (self.a, self.b, self.c, self.d) class Player(wx.Frame): """The main window has to deal with events. """ def __init__(self, title, OS="windows"): self.OS = OS wx.Frame.__init__(self, None, -1, title, pos=wx.DefaultPosition, size=(950, 500)) self.SetBackgroundColour(wx.BLACK) # Panels # The first panel holds the video/videos and it's all black self.videopanel = wx.Panel(self, -1) self.videopanel.SetBackgroundColour(wx.BLACK) if dual: videopanel2 = wx.Panel(self, -1) self.videopanel2 = videopanel2 self.videopanel2.SetBackgroundColour(wx.BLACK) # The third panel holds controls ctrlpanel = wx.Panel(self, -1) self.ctrlpanel = ctrlpanel self.timeslider = wx.Slider(ctrlpanel, -1, 0, 0, 1000) self.timeslider.SetRange(0, 1000) '''nextvid = wx.Button(ctrlpanel, label=">") prevvid = wx.Button(ctrlpanel, label="<") pause = wx.Button(ctrlpanel, label="Pause") play = wx.Button(ctrlpanel, label="Play") stop = wx.Button(ctrlpanel, label="Stop") fullscreen = wx.Button(ctrlpanel, label="Fullscreen on/off") self.showcontrols = wx.CheckBox(ctrlpanel, -1, 'Show controls')''' # Bind controls to events #self.Bind(wx.EVT_BUTTON, self.on_full_screen, fullscreen) #self.Bind(wx.EVT_CHECKBOX, self.on_show_controls_checkbox, self.showcontrols) self.videopanel.Bind(wx.EVT_KEY_DOWN, self.OnKeyDown) # Give a pretty layout to the controls ctrlbox = wx.BoxSizer(wx.VERTICAL) box1 = wx.BoxSizer(wx.HORIZONTAL) box2 = wx.BoxSizer(wx.HORIZONTAL) '''box1.Add(self.timeslider, 1) # box2 contains some buttons box2.Add(play, flag=wx.RIGHT, border=5) box2.Add(pause) box2.Add(stop) box2.Add(nextvid) box2.Add(prevvid) box2.Add(fullscreen) box2.Add(self.showcontrols)''' box2.Add((-1, -1), 1) ctrlbox.Add(box1, 0, flag=wx.EXPAND | wx.BOTTOM, border=10) ctrlbox.Add(box2, 0, flag=wx.EXPAND) ctrlpanel.SetSizer(ctrlbox) self.videopanel.Bind(wx.EVT_ENTER_WINDOW, self.OnMouseLeaveCtrlpanel) if dual: self.videopanel2.Bind(wx.EVT_ENTER_WINDOW, self.OnMouseLeaveCtrlpanel) # Put everything together sizer2 = wx.BoxSizer(wx.VERTICAL) sizer1 = wx.BoxSizer(wx.HORIZONTAL) sizer1.Add(self.videopanel, 1, flag=wx.EXPAND) if dual: sizer1.Add(self.videopanel2, 1, flag=wx.EXPAND) sizer2.Add(sizer1, 1, flag=wx.EXPAND) sizer2.Add(ctrlpanel, 0, flag=wx.EXPAND | wx.BOTTOM | wx.TOP, border=1) self.SetSizer(sizer2) self.SetMinSize((550, 270)) # finally create the timer, which updates the timeslider self.timer = wx.Timer(self) #self.Bind(wx.EVT_TIMER, self.on_timer, self.timer) #self.show_controls(True) # VLC player controls plugin_path = None # Multiplatform if self.OS == "linux": plugin_path == "/usr/lib/vlc/plugins" elif self.OS == "windows": plugin_path == "C:\\Program Files (x86)\\VideoLAN\\VLC\\plugins" if plugin_path: self.Instance = vlc.Instance("--plugin-path %s" % plugin_path) else: self.Instance = vlc.Instance('--input-repeat=-1 --reset-config --no-video-title-show')# --start-time=10') #print "VLC Instance: %s" % self.Instance #We create instances of VLC, can we make them in the same Frame? self.vidplayer1 = self.Instance.media_player_new() self.vidplayer2 = self.Instance.media_player_new() def on_full_screen(self, evt): self.Show() self.ShowFullScreen(not self.IsFullScreen()) def OnKeyDown(self, event): global dual,time,midsection,sizer1,To_pause keycode = event.GetKeyCode() print keycode if keycode == 50: # 2 if dual == False: # We change to dual video '''dual = True time = player.vidplayer1.get_time() media = player.vidplayer1.get_media() # We make visible the background videoplayer player2.SetTransparent(255) # We stop the foreground videoplayer player1.vidplayer1.stop() player.vidplayer1.stop() # We create a dual video player and put the old video on the left player2.playFile(VIDEO1, 0, True) player2.playFile(VIDEO2, 0, False) # We make invisible the foreground videoplayer player.SetTransparent(0) player1.SetTransparent(0) player2.Show(True)''' time = player.vidplayer1.get_time() media = player.vidplayer1.get_media() player.vidplayer1.stop() player.SetTransparent(0) tasks.put(Task(VIDEO1, dual, time, midsection)) else: # We change to mono if player2.vidplayer1.get_time() != -1: time = player2.vidplayer1.get_time() media = player2.vidplayer1.get_media() player2.vidplayer2.stop() player.vidplayer1.set_media(media) player1.vidplayer1.stop() player.vidplayer1.play() player.vidplayer1.set_time(time) sleep(0.8) player2.vidplayer1.stop() player2.Hide() '''if To_pause == False: player.vidplayer1.pause()''' player.Show(True) else: time = player3.vidplayer1.get_time() media = player3.vidplayer1.get_media() player2.vidplayer2.set_media(player3.vidplayer2.get_media()) player3.vidplayer2.stop() player.vidplayer1.set_media(media) player3.vidplayer1.stop() player.vidplayer1.play() player.vidplayer1.set_time(time) sleep(0.8) player3.vidplayer1.stop() player3.Hide() '''if To_pause == False: player.vidplayer1.pause()''' player.Show(True) dual = not dual def playFile(self,moviefile,time,principal): # Creation self.Media = self.Instance.media_new_path(moviefile) self.SetTitle("Monkey") #self.SetIcon(wx.Icon('Monkey.ico', wx.BITMAP_TYPE_ICO)) if dual: if principal: #If we dont set any handle it starts in another window, maybe usefull for dual screens? self.vidplayer1.set_media(self.Media) #self.vlchandle = self.vidplayer1.get_xwindow() self.vidplayer1.set_hwnd(self.videopanel.GetHandle()) self.OnPlay(None,True) self.vidplayer1.set_time(time) else: self.vidplayer2.set_media(self.Media) #self.vlchandle = self.vidplayer2.get_xwindow() self.vidplayer2.set_hwnd(self.videopanel2.GetHandle()) self.OnPlay(None,False) self.vidplayer2.set_time(time) else: self.vidplayer1.set_media(self.Media) #self.vlchandle = self.vidplayer1.get_xwindow() self.vidplayer1.set_hwnd(self.videopanel.GetHandle()) self.OnPlay(None,True) self.vidplayer1.set_time(time) # set the volume slider to the current volume #self.volslider.SetValue(self.vidplayer.audio_get_volume() / 2) def OnFullscreen(self,evt): self.Show() self.ShowFullScreen(not self.IsFullScreen()) def OnPlay(self, evt, principal): """Toggle the status to Play/Pause. If no file is loaded, open the dialog window. """ # check if there is a file to play, otherwise open a # wx.FileDialog to select a file if principal: if not self.vidplayer1.get_media(): self.OnOpen(None) else: # Try to launch the media, if this fails display an error message if self.vidplayer1.play() == -1: self.errorDialog("Unable to play.") else: #Set timer to callback every 500 ms timeslider moves every 0'5s self.timer.Start(500) else: if not self.vidplayer2.get_media(): self.OnOpen(None) else: # Try to launch the media, if this fails display an error message if self.vidplayer2.play() == -1: self.errorDialog("Unable to play.") else: #Set timer to callback every 500 ms timeslider moves every 0'5s self.timer.Start(500) def OnTimer(self, evt): """Update the time slider according to the current movie time. """ # update the time time = self.vidplayer1.get_time() time = self.vidplayer1.get_time() def OnMouseLeaveCtrlpanel(self,evt): None def OnSetTime(self, evt): """Set the time according to the time sider. """ time = self.timeslider.GetValue() # vlc.MediaPlayer.set_time returns 0 if success, -1 otherwise if self.vidplayer1.set_time(time) == -1: self.errorDialog("Failed to set time") if __name__ == "__main__": global dual,midsection, tasks midsection = True tasks = multiprocessing.JoinableQueue() num_consumers = 2 consumers = [Consumer(tasks) for i in xrange(num_consumers)] for w in consumers: w.start() # Create a wx.App(), which handles the windowing system event loop app = wx.App() width, height = wx.GetDisplaySize() # Create the windows containing our media players dual = False player1 = Player("Single PyVLC Player") player = Player("Single PyVLC Player") dual = True player2 = Player("Dual PyVLC Player") player2.Centre() # show the player window centred and run the application player2.playFile(VIDEO1, 0, True) player2.playFile(VIDEO2, 0, False) player2.Maximize(True) player2.on_full_screen(None) player2.vidplayer1.stop() player2.vidplayer2.stop() # The machine can't play 3 vids at once... player2.Hide() player2.SetTransparent(0) player.Centre() player.playFile(VIDEO1,0,True) player.Maximize(True) player.OnFullscreen(None) player.vidplayer1.play() #The machine can't play 3 vids at once... player.Show(True) dual = False app.MainLoop() From prince100thomas at outlook.com Thu Feb 4 07:39:26 2016 From: prince100thomas at outlook.com (Prince Thomas) Date: Thu, 4 Feb 2016 12:39:26 +0000 Subject: =?utf-8?Q?setup_failed?= Message-ID: Hi I am an computer science engineer. I downloaded the python version 3.5.1.amd64 and just python 3.5.1. The problem is when I install the program setup is failed and showing 0*80070570-The file or directory is corrupted and unreadable. I install the newest visual c++ redist and still same. My os is win 8.1. Please help me out of this Sent from Windows Mail From mdelamo90 at gmail.com Thu Feb 4 07:44:24 2016 From: mdelamo90 at gmail.com (mdelamo90 at gmail.com) Date: Thu, 4 Feb 2016 04:44:24 -0800 (PST) Subject: Non working Parallel videoplayer In-Reply-To: <75f9290f-dc92-4426-a42b-05b45a0d489b@googlegroups.com> References: <75f9290f-dc92-4426-a42b-05b45a0d489b@googlegroups.com> Message-ID: <9da80d70-01e4-4228-85c4-7341696b13f2@googlegroups.com> I think it has to be with some functions that I think they are defined on the "forked" processes and maybe they don't, if I create a class on the new process it should have all the defined functions than the original right? From yossify at hotmobile.co.il Thu Feb 4 07:52:31 2016 From: yossify at hotmobile.co.il (Yossifoff Yossif) Date: Thu, 4 Feb 2016 12:52:31 +0000 Subject: .py file won't open in windows 7 Message-ID: Hallow, I try to open a .py file (attached), but what I get is a windows DOS window opening and closing in a couple of seconds. Ran repair of the program, nothing happened. I cannot see error messages and don't know where to look for ones. Would appreciate your piece of advice. [cid:image003.jpg at 01D15F5B.A41CF980] joseph Yossif Yossifoff Operations CS Core Hot Mobile Office +972539036945, Mobile +972532422649 yossify at hotmobile.co.il This message (including any attachments) is intended only for the use of the individual or entity to which it is addressed and may contain materials protected by copyright or information that is non-public, proprietary, privileged, confidential, and exempt from disclosure under applicable law or agreement. If you are not the intended recipient, you are hereby notified that any use, dissemination, distribution, or copying of this communication is strictly prohibited. If you have received this communication by error, notify the sender immediately and delete this message immediately. Thank you From mail at timgolden.me.uk Thu Feb 4 08:09:05 2016 From: mail at timgolden.me.uk (Tim Golden) Date: Thu, 4 Feb 2016 13:09:05 +0000 Subject: .py file won't open in windows 7 In-Reply-To: References: Message-ID: <56B34D71.1070308@timgolden.me.uk> On 04/02/2016 12:52, Yossifoff Yossif wrote: > Hallow, > I try to open a .py file (attached), but what I get is a windows DOS window opening and closing in a couple of seconds. Ran repair of the program, nothing happened. > I cannot see error messages and don't know where to look for ones. > Would appreciate your piece of advice. Attachments won't make it through to the list, Yossif. But your code is basically something like this: """ import sys def main(): # calculate stuff print(stuff) if __name__ == '__main__': sys.exit(main()) """ In that case, the program starts (in a console window), runs, prints the result, and then closes. You've got a few simple ways of seeing the output: * Run the program from the command line (start a console window in the directory where the code is and type "ss7calc.py") https://docs.python.org/3/faq/windows.html#how-do-i-run-a-python-program-under-windows * Instead of use sys.exit immediately, add input("Press enter...") to the end of your code: https://docs.python.org/3/library/functions.html?highlight=input#input eg: result = main() input("Press enter...") sys.exit(result) * set the PYTHONINSPECT environment variable for your user https://docs.python.org/3/using/cmdline.html#envvar-PYTHONINSPECT * Use a shebang line "#!python -i" (without the quotes) at the top of your program. https://docs.python.org/3/using/windows.html#shebang-lines These last two options will drop into the Python interpreter after your code has run. TJG From mail at timgolden.me.uk Thu Feb 4 08:18:27 2016 From: mail at timgolden.me.uk (Tim Golden) Date: Thu, 4 Feb 2016 13:18:27 +0000 Subject: .py file won't open in windows 7 In-Reply-To: <56B34D71.1070308@timgolden.me.uk> References: <56B34D71.1070308@timgolden.me.uk> Message-ID: <56B34FA3.4020007@timgolden.me.uk> On 04/02/2016 13:09, Tim Golden wrote: > On 04/02/2016 12:52, Yossifoff Yossif wrote: >> Hallow, >> I try to open a .py file (attached), but what I get is a windows DOS window opening and closing in a couple of seconds. Ran repair of the program, nothing happened. >> I cannot see error messages and don't know where to look for ones. >> Would appreciate your piece of advice. > > Attachments won't make it through to the list, Yossif. But your code is > basically something like this: > > """ > import sys > > def main(): > # calculate stuff > print(stuff) > > if __name__ == '__main__': > sys.exit(main()) > """ > > In that case, the program starts (in a console window), runs, prints the > result, and then closes. You've got a few simple ways of seeing the output: ... or run within a development environment. Python itself ships with IDLE: https://docs.python.org/3/library/idle.html but there are plenty of others: https://wiki.python.org/moin/IntegratedDevelopmentEnvironments TJG From yanzhaoxun at greendh.com Thu Feb 4 08:33:58 2016 From: yanzhaoxun at greendh.com (=?UTF-8?B?6ZiO5YWG54+j?=) Date: Thu, 4 Feb 2016 21:33:58 +0800 (CST) Subject: =?UTF-8?B?ZXZhbCggJ2ltcG9ydCBtYXRoJyAp?= Message-ID: <298169014.1288190.1454592838187.JavaMail.root@wmthree-11> Excuse me for the same problem in Python 3.4.2-32bit I just discovered that function does not necessarily take the string input and transfer it to a command to execute. So is there a problem with my assumption? Thanks Option Product Manager Gelin Dahua Futures Co.Ltd ?T: +86 10 56711783 From joel.goldstick at gmail.com Thu Feb 4 08:47:53 2016 From: joel.goldstick at gmail.com (Joel Goldstick) Date: Thu, 4 Feb 2016 08:47:53 -0500 Subject: problem in installing python In-Reply-To: References: Message-ID: On Thu, Feb 4, 2016 at 7:22 AM, Salony Permanand wrote: > hello sir, > During working on python I need urllib2 for my python version 2.7.11. > Kindly provide me address from where to download it.. > Thanking you. > > Hello Salony, Since this is a new question, its best to start a new thread with a new title. You may also want to drop the 'hello sir' greeting. Yes, mostly males here, but some very active women also contribute here. A simple hello will suffice As to urllib2, it is included in python so do this in your program: import urllib2 You may find that the third party library 'requests' is easier to use -- Joel Goldstick http://joelgoldstick.com/stats/birthdays From lutz.horn at outlook.de Thu Feb 4 09:05:33 2016 From: lutz.horn at outlook.de (Lutz Horn) Date: Thu, 4 Feb 2016 15:05:33 +0100 Subject: eval( 'import math' ) In-Reply-To: <298169014.1288190.1454592838187.JavaMail.root@wmthree-11> References: <298169014.1288190.1454592838187.JavaMail.root@wmthree-11> Message-ID: Hi, > I just discovered that function does not necessarily take the > string input and transfer it to a command to execute. Can you please show us the code you try to execute and tells what result you expect? Lutz From ian.g.kelly at gmail.com Thu Feb 4 09:30:14 2016 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Thu, 4 Feb 2016 07:30:14 -0700 Subject: eval( 'import math' ) In-Reply-To: <298169014.1288190.1454592838187.JavaMail.root@wmthree-11> References: <298169014.1288190.1454592838187.JavaMail.root@wmthree-11> Message-ID: On Thu, Feb 4, 2016 at 6:33 AM, ??? wrote: > Excuse me for the same problem in Python 3.4.2-32bit > > I just discovered that function does not necessarily take the > string input and transfer it to a command to execute. > > So is there a problem with my assumption? eval evaluates an expression, not a statement. For that, you would use exec. If you're just trying to import though, then you don't need it at all. Use the importlib.import_module function instead to import a module determined at runtime. This is more secure than eval or exec, which can cause any arbitrary Python code to be executed. From gemjack.pb at gmail.com Thu Feb 4 09:47:22 2016 From: gemjack.pb at gmail.com (gemjack.pb at gmail.com) Date: Thu, 4 Feb 2016 06:47:22 -0800 (PST) Subject: Tkinter problem: TclError> couldn't connect to display ":0 In-Reply-To: References: Message-ID: On Sunday, 29 December 2013 20:20:00 UTC, Michael Matveev wrote: > Hi, > I use live Debian on VM and trying to compile this code. > > > import Tkinter > > root = Tkinter.Tk() > > root.title("Fenster 1") > root.geometry("100x100") > > root.mainloop() > > > The shell gives out that kind of message: > > File "test.py", line 5, in > root = Tkinter.Tk() > File "/usr/lib/python2.7/lib-tk/Tkinter.py", line 1712, in __init__ > self.tk = _tkinter.create(screenName, baseName, className, interactive, wantobjects, useTk, sync, use) > _tkinter.TclError: couldn't connect to display ":0" > > > > thanks for helping out. > > greets. > Mike This fixed my problem with thkinter..... sudo cp ~/.Xauthority ~root/ From __peter__ at web.de Thu Feb 4 09:48:31 2016 From: __peter__ at web.de (Peter Otten) Date: Thu, 04 Feb 2016 15:48:31 +0100 Subject: eval( 'import math' ) References: <298169014.1288190.1454592838187.JavaMail.root@wmthree-11> Message-ID: ??? wrote: > Excuse me for the same problem in Python 3.4.2-32bit > > I just discovered that function does not necessarily take the > string input and transfer it to a command to execute. > > So is there a problem with my assumption? Python discriminates between statements and expressions. The eval function will only accept an expression. OK: >>> eval("1 + 1") 2 >>> eval("print(42)") # Python 3 only; in Python 2 print is a statement 42 >>> x = y = 1 >>> eval("x > y") False Not acceptable: >>> eval("1+1; 2+2") Traceback (most recent call last): File "", line 1, in File "", line 1 1+1; 2+2 ^ SyntaxError: invalid syntax >>> eval("import os") Traceback (most recent call last): File "", line 1, in File "", line 1 import os ^ SyntaxError: invalid syntax >>> eval("if x > y: print(42)") Traceback (most recent call last): File "", line 1, in File "", line 1 if x > y: print(42) ^ SyntaxError: invalid syntax To import a module dynamically either switch to exec() >>> exec("import os") >>> os or use the import_module() function: >>> import importlib >>> eval("importlib.import_module('os')") Of course you can use that function directly >>> importlib.import_module("os") and that's what you should do if your goal is to import a module rather than to run arbitrary Python code. From ian.g.kelly at gmail.com Thu Feb 4 10:25:10 2016 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Thu, 4 Feb 2016 08:25:10 -0700 Subject: Multiprocess videoplayer In-Reply-To: <6705fdb6-a5dd-4a2d-9dbc-04a557ec1026@googlegroups.com> References: <6705fdb6-a5dd-4a2d-9dbc-04a557ec1026@googlegroups.com> Message-ID: On Thu, Feb 4, 2016 at 2:45 AM, wrote: > I have coded a program with python and vlc that plays some videos, but whenever I try to play 3 videos at once, Windows closes the program, I'm guessing that the reason is that one process can't play 3 videos at once (but I don't really know). My suspicion would be that some uncaught exception is being raised and killing the whole process. Since this is a GUI program, are you running it with python.exe or pythonw.exe? Are you running it from the command line or launching it from the Windows GUI? If you're not running it from a command prompt window, try doing that in order to capture any exception output. From polyvertex at gmail.com Thu Feb 4 10:35:45 2016 From: polyvertex at gmail.com (Jean-Charles Lefebvre) Date: Thu, 4 Feb 2016 07:35:45 -0800 (PST) Subject: How a module is being marked as imported? In-Reply-To: <325bc056-0683-408b-85ba-6c3cd0f69d41@googlegroups.com> References: <325bc056-0683-408b-85ba-6c3cd0f69d41@googlegroups.com> Message-ID: So far, I've been advised to: 1/ Double-check that the GIL was correctly acquired 2/ Ensure there's no 'string' module in my project 3/ Manually pre-import commonly used standard modules at interpreter's init-time to avoid race conditions due to the multi-threaded nature of the running environment No problem found for 1/ & 2/ (double-checked). I tried 3/ before posting and could not reproduce the problem at all which is probably the patch I will apply due to the lack of a better solution. I guess I'll have to dig into __import__'s code and related. On Thursday, February 4, 2016 at 11:20:05 AM UTC+1, Jean-Charles Lefebvre wrote: > Hi all, > > The short version: How CPython marks a module as being fully imported, if it does, so that the same import statement ran from another C thread at the same time does not collide? Or, reversely, does not think the module is not already fully imported? > > The full version: I'm running CPython 3.5.1, embedded into a C++ application on Windows. The application is heavily multi-threaded so several C threads call some Python code at the same time (different Python modules), sharing interpreter's resources by acquiring/releasing the GIL frequently DURING the calls, at language boundaries. > > Sometimes (but always only once per application instance), a call to os.path.expandvars raises the AttributeError exception with message: module 'string' has no attribute 'ascii_letters'. It is raised by the ntpath.expandvars function (line 372). When I noticed the late import statement of the 'string' module at the line above, I thought that MAYBE, it could be because the interpreter is ran in an heavily multi-threaded environment and that the GIL acquiring/releasing occurred at a bad timing? Making me wonder how the import mechanism interacts with the GIL, if it does? From kevinjacobconway at gmail.com Thu Feb 4 11:11:09 2016 From: kevinjacobconway at gmail.com (Kevin Conway) Date: Thu, 04 Feb 2016 16:11:09 +0000 Subject: How a module is being marked as imported? In-Reply-To: References: <325bc056-0683-408b-85ba-6c3cd0f69d41@googlegroups.com> Message-ID: As an attempt to answer your original question, Python doesn't explicitly mark a module as done. It does keep imports cached in sys.modules, though. The behaviour you describe where later imports get the same module object is driven by that cache. There are cases, such as cyclical imports, where the object in sys.modules is empty for a period of time and acts as a placeholder until the code from that module is loaded. During that time, any usage of the module would result in a similar failure to what you have described. Additionally, if you are writing multithreaded code then acquiring the GIL may not be enough. There is another lock used for imports. On Thu, Feb 4, 2016, 09:43 Jean-Charles Lefebvre wrote: > So far, I've been advised to: > > 1/ Double-check that the GIL was correctly acquired > 2/ Ensure there's no 'string' module in my project > 3/ Manually pre-import commonly used standard modules at interpreter's > init-time to avoid race conditions due to the multi-threaded nature of the > running environment > > No problem found for 1/ & 2/ (double-checked). I tried 3/ before posting > and could not reproduce the problem at all which is probably the patch I > will apply due to the lack of a better solution. I guess I'll have to dig > into __import__'s code and related. > > > On Thursday, February 4, 2016 at 11:20:05 AM UTC+1, Jean-Charles Lefebvre > wrote: > > Hi all, > > > > The short version: How CPython marks a module as being fully imported, > if it does, so that the same import statement ran from another C thread at > the same time does not collide? Or, reversely, does not think the module is > not already fully imported? > > > > The full version: I'm running CPython 3.5.1, embedded into a C++ > application on Windows. The application is heavily multi-threaded so > several C threads call some Python code at the same time (different Python > modules), sharing interpreter's resources by acquiring/releasing the GIL > frequently DURING the calls, at language boundaries. > > > > Sometimes (but always only once per application instance), a call to > os.path.expandvars raises the AttributeError exception with message: module > 'string' has no attribute 'ascii_letters'. It is raised by the > ntpath.expandvars function (line 372). When I noticed the late import > statement of the 'string' module at the line above, I thought that MAYBE, > it could be because the interpreter is ran in an heavily multi-threaded > environment and that the GIL acquiring/releasing occurred at a bad timing? > Making me wonder how the import mechanism interacts with the GIL, if it > does? > -- > https://mail.python.org/mailman/listinfo/python-list > From srkunze at mail.de Thu Feb 4 11:18:21 2016 From: srkunze at mail.de (Sven R. Kunze) Date: Thu, 4 Feb 2016 17:18:21 +0100 Subject: Efficient Wrappers for Instance Methods In-Reply-To: <1454543258.197018.511336178.74125AA7@webmail.messagingengine.com> References: <56B263C1.7030309@mail.de> <56B26940.3090609@mail.de> <56B26D3D.6040703@mail.de> <56B2747B.6050209@mail.de> <1454543258.197018.511336178.74125AA7@webmail.messagingengine.com> Message-ID: <56B379CD.5090903@mail.de> On 04.02.2016 00:47, Random832 wrote: > On Wed, Feb 3, 2016, at 16:43, Sven R. Kunze wrote: >> Actually a nice idea if there were no overhead of creating methods for >> all heap instances separately. I'll keep that in mind. :) > What about changing the class of the object to one which is inherited > from its original class and has the method you want? What about reaching > into the class and changing the method in the first place? Either may > not be appropriate, of course, depending on your use case. There is no base class. Best, Sven From random832 at fastmail.com Thu Feb 4 13:35:32 2016 From: random832 at fastmail.com (Random832) Date: Thu, 04 Feb 2016 13:35:32 -0500 Subject: Efficient Wrappers for Instance Methods In-Reply-To: <56B379CD.5090903@mail.de> References: <56B263C1.7030309@mail.de> <56B26940.3090609@mail.de> <56B26D3D.6040703@mail.de> <56B2747B.6050209@mail.de> <1454543258.197018.511336178.74125AA7@webmail.messagingengine.com> <56B379CD.5090903@mail.de> Message-ID: <1454610932.523536.512150754.7911A33B@webmail.messagingengine.com> On Thu, Feb 4, 2016, at 11:18, Sven R. Kunze wrote: > On 04.02.2016 00:47, Random832 wrote: > > On Wed, Feb 3, 2016, at 16:43, Sven R. Kunze wrote: > >> Actually a nice idea if there were no overhead of creating methods for > >> all heap instances separately. I'll keep that in mind. :) > > What about changing the class of the object to one which is inherited > > from its original class and has the method you want? What about reaching > > into the class and changing the method in the first place? Either may > > not be appropriate, of course, depending on your use case. > > There is no base class. I meant something like... Class C: replace = heapreplace Cs = {} ... if not isinstance(x, C) T = type(x) cls = cache.get(T) if cls is None: cls = type('C_'+T.__name__, (C, T), {}) x.__class__ = cls (Of course, by dynamically reassigning __class__ and using the type constructor, this checks two of the three "crazy type system voodoo" boxes. I have no idea if it will work, or if I've made a mistake, or if you'll be able to understand it in six months.) From joel.goldstick at gmail.com Thu Feb 4 13:53:40 2016 From: joel.goldstick at gmail.com (Joel Goldstick) Date: Thu, 4 Feb 2016 13:53:40 -0500 Subject: Reply to whom? In-Reply-To: <85si19avvi.fsf@benfinney.id.au> References: <56B263C1.7030309@mail.de> <56B26940.3090609@mail.de> <56B26D3D.6040703@mail.de> <85h9hpcvjw.fsf_-_@benfinney.id.au> <1454561022.2123037.511515122.153B155A@webmail.messagingengine.com> <85si19avvi.fsf@benfinney.id.au> Message-ID: On Thu, Feb 4, 2016 at 12:26 AM, Ben Finney wrote: > Random832 writes: > > > On Wed, Feb 3, 2016, at 16:50, Ben Finney wrote: > > > (You will also have a ?reply to all? command. That's almost never > > > appropriate in a forum like this.) > > > > Why not? People reply all to messages I write all the time, and I find > > it somewhat useful since it separates replies to things I have said > > from discussions I'm not part of. > > That's nice for you that it coincides with what you find useful. For > those who don't want messages to the forum duplicated in their inbox, > the behaviour is obnoxious. > > In other words: Using ?reply to all? on a forum like this is > inappropriate because you can't know what every participant wants, so > your set of recipients should be conservative and err on the side of not > sending a duplicate message to people who didn't ask for it. > > -- > \ ?Don't worry about people stealing your ideas. If your ideas | > `\ are any good, you'll have to ram them down people's throats.? | > _o__) ?Howard Aiken | > Ben Finney > > -- > https://mail.python.org/mailman/listinfo/python-list > I use gmail and hit reply all and then (usually, unless i forget) delete all recipients except the group -- Joel Goldstick http://joelgoldstick.com/stats/birthdays From srkunze at mail.de Thu Feb 4 15:14:27 2016 From: srkunze at mail.de (Sven R. Kunze) Date: Thu, 4 Feb 2016 21:14:27 +0100 Subject: Efficient Wrappers for Instance Methods In-Reply-To: <1454610932.523536.512150754.7911A33B@webmail.messagingengine.com> References: <56B263C1.7030309@mail.de> <56B26940.3090609@mail.de> <56B26D3D.6040703@mail.de> <56B2747B.6050209@mail.de> <1454543258.197018.511336178.74125AA7@webmail.messagingengine.com> <56B379CD.5090903@mail.de> <1454610932.523536.512150754.7911A33B@webmail.messagingengine.com> Message-ID: <56B3B123.2060309@mail.de> On 04.02.2016 19:35, Random832 wrote: > On Thu, Feb 4, 2016, at 11:18, Sven R. Kunze wrote: >> On 04.02.2016 00:47, Random832 wrote: >>> On Wed, Feb 3, 2016, at 16:43, Sven R. Kunze wrote: >>>> Actually a nice idea if there were no overhead of creating methods for >>>> all heap instances separately. I'll keep that in mind. :) >>> What about changing the class of the object to one which is inherited >>> from its original class and has the method you want? What about reaching >>> into the class and changing the method in the first place? Either may >>> not be appropriate, of course, depending on your use case. >> There is no base class. > I meant something like... > > Class C: > replace = heapreplace > > Cs = {} > > ... > > if not isinstance(x, C) > T = type(x) > cls = cache.get(T) > if cls is None: > cls = type('C_'+T.__name__, (C, T), {}) > x.__class__ = cls > > (Of course, by dynamically reassigning __class__ and using the type > constructor, this checks two of the three "crazy type system voodoo" > boxes. I have no idea if it will work, or if I've made a mistake, or if > you'll be able to understand it in six months.) I think I agree with you that this might be a maintenance nightmare. ;) Best, Sven From mr.eightnoteight at gmail.com Thu Feb 4 15:50:01 2016 From: mr.eightnoteight at gmail.com (srinivas devaki) Date: Fri, 5 Feb 2016 02:20:01 +0530 Subject: _siftup and _siftdown implementation Message-ID: _siftdown function breaks out of the loop when the current pos has a valid parent. but _siftup function is not implemented in that fashion, if a valid subheap is given to the _siftup, it will bring down the root of sub heap and then again bring it up to its original place. I was wondering why it is so, is it just to make the code look simple??? Regards Srinivas Devaki Junior (3rd yr) student at Indian School of Mines,(IIT Dhanbad) Computer Science and Engineering Department ph: +91 9491 383 249 telegram_id: @eightnoteight From rosuav at gmail.com Thu Feb 4 16:57:52 2016 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 5 Feb 2016 08:57:52 +1100 Subject: problem in installing python In-Reply-To: References: Message-ID: On Thu, Feb 4, 2016 at 11:22 PM, Salony Permanand wrote: > During working on python I need urllib2 for my python version 2.7.11. > Kindly provide me address from where to download it.. > Thanking you. It should have come with Python. Try it - you should be able to just use it as-is. ChrisA From steve at pearwood.info Thu Feb 4 19:12:49 2016 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 05 Feb 2016 11:12:49 +1100 Subject: _siftup and _siftdown implementation References: Message-ID: <56b3e902$0$1613$c3e8da3$5496439d@news.astraweb.com> On Fri, 5 Feb 2016 07:50 am, srinivas devaki wrote: > _siftdown function breaks out of the loop when the current pos has a valid > parent. > > but _siftup function is not implemented in that fashion, if a valid > subheap is given to the _siftup, it will bring down the root of sub heap > and then again bring it up to its original place. > > I was wondering why it is so, is it just to make the code look simple??? Hi Srinivas, I'm sure that your question is obvious to you, but it's not obvious to us. Where are _siftup and _siftdown defined? Are they in your code? Somebody else's code? A library? Which library? What do they do? Where are they from? -- Steven From srkunze at mail.de Thu Feb 4 19:21:50 2016 From: srkunze at mail.de (Sven R. Kunze) Date: Fri, 5 Feb 2016 01:21:50 +0100 Subject: _siftup and _siftdown implementation In-Reply-To: <56b3e902$0$1613$c3e8da3$5496439d@news.astraweb.com> References: <56b3e902$0$1613$c3e8da3$5496439d@news.astraweb.com> Message-ID: <56B3EB1E.1050905@mail.de> On 05.02.2016 01:12, Steven D'Aprano wrote: > On Fri, 5 Feb 2016 07:50 am, srinivas devaki wrote: > >> _siftdown function breaks out of the loop when the current pos has a valid >> parent. >> >> but _siftup function is not implemented in that fashion, if a valid >> subheap is given to the _siftup, it will bring down the root of sub heap >> and then again bring it up to its original place. >> >> I was wondering why it is so, is it just to make the code look simple??? > Hi Srinivas, > > I'm sure that your question is obvious to you, but it's not obvious to us. > Where are _siftup and _siftdown defined? Are they in your code? Somebody > else's code? A library? Which library? What do they do? Where are they > from? The question originated here: https://github.com/srkunze/xheap/pull/1#discussion_r51770210 (btw, Steven, your email client somehow breaks my threading view in thunderbird. This reply appeared unconnected to Srinivas' post.) From mr.eightnoteight at gmail.com Thu Feb 4 20:26:12 2016 From: mr.eightnoteight at gmail.com (srinivas devaki) Date: Fri, 5 Feb 2016 06:56:12 +0530 Subject: _siftup and _siftdown implementation In-Reply-To: <56b3e902$0$1613$c3e8da3$5496439d@news.astraweb.com> References: <56b3e902$0$1613$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Feb 5, 2016 5:45 AM, "Steven D'Aprano" wrote: > > On Fri, 5 Feb 2016 07:50 am, srinivas devaki wrote: > > > _siftdown function breaks out of the loop when the current pos has a valid > > parent. > > > > but _siftup function is not implemented in that fashion, if a valid > > subheap is given to the _siftup, it will bring down the root of sub heap > > and then again bring it up to its original place. as I come to think of it again, it is not subheap, it actually heap cut at some level hope you get the idea from the usage of _siftup. so even though the `pos` children are valid the _siftup brings down the new element (i.e the element which is at first at `pos`) upto its leaf level and then again it is brought up by using _siftdown. why do the redundant work when it can simply breakout? > > > > I was wondering why it is so, is it just to make the code look simple??? > > Hi Srinivas, > > I'm sure that your question is obvious to you, but it's not obvious to us. > Where are _siftup and _siftdown defined? Are they in your code? Somebody > else's code? A library? Which library? What do they do? Where are they > from? _siftup and _siftdown are functions from python standard heapq module. PS: I do competitive programming, I use these modules every couple of days when compared to other modules. so didn't give much thought when posting to the mailing list. sorry for that. Regards Srinivas Devaki Junior (3rd yr) student at Indian School of Mines,(IIT Dhanbad) Computer Science and Engineering Department ph: +91 9491 383 249 telegram_id: @eightnoteight From jfong at ms4.hinet.net Fri Feb 5 01:57:51 2016 From: jfong at ms4.hinet.net (jfong at ms4.hinet.net) Date: Thu, 4 Feb 2016 22:57:51 -0800 (PST) Subject: How this C function was called through ctypes this way? In-Reply-To: References: <2bcaa079-f7ec-4c1e-9d18-c607247eb239@googlegroups.com> Message-ID: eryk sun at 2016/2/4 UTC+8 7:35:17PM wrote: > > _mod = ctypes.cdll.LoadLibrary(_path) > > Calling ctypes.CDLL directly is preferable since it allows passing > parameters such as "mode" and "use_errno". > > IMO, the ctypes.cdll and ctypes.windll loaders should be avoided in > general, especially on Windows, since their attribute-based access > (e.g. windll.user32) caches libraries, which in turn cache > function-pointer attributes. You don't want function pointer instances > being shared across unrelated packages. They may not use compatible > prototypes and errcheck functions. Each package, module, or script > should create private instances of CDLL, PyDLL, and WinDLL for a given > shared library. Thank you for your detail and deep explanation. I suppose the reason there are many cases using LoadLibrary() and attribute-based access is because it's the way the ctypes tutorial in Python document takes. Although both methods has been mentioned in the ctypes reference section, but no pros and cons was explained. --Jach From nomail at invalid.com Fri Feb 5 02:12:09 2016 From: nomail at invalid.com (ast) Date: Fri, 5 Feb 2016 08:12:09 +0100 Subject: metaclass Message-ID: <56b44b50$0$19767$426a74cc@news.free.fr> Hi I am looking the relationship between some classes from the enum module >>> from enum import EnumMeta, Enum >>> class Color(Enum): pass >>> type(EnumMeta) >>> EnumMeta.__bases__ (,) >>> so EnumMeta is a metaclass, it is an instance of type and inherit from type too. >>> type(Enum) >>> Enum.__bases__ (,) so Enum is an instance of EnumMeta and Enum inherit from object >>> type(Color) >>> Color.__bases__ (,) so Color is an instance of EnumMeta and inherit from Enum It is not obvious to me that Color is an instance of EnumMeta. Is it a python rule that if a class C inherit from a class which is an instance of a metaclass, then class C is an instance of the same metaclass too ? Or was it feasible to guess that ? From dieter at handshake.de Fri Feb 5 02:52:04 2016 From: dieter at handshake.de (dieter) Date: Fri, 05 Feb 2016 08:52:04 +0100 Subject: Finding in which class an object's method comes from References: <56b30b1e$0$3637$426a74cc@news.free.fr> Message-ID: <87io23a923.fsf@handshake.de> "ast" writes: > Suppose we have: > > ClassC inherit from ClassB > ClassB inherit from ClassA > ClassA inherit from object > > Let's build an object: > > obj = ClassC() > > Let's invoke an obj method > > obj.funct() > > funct is first looked in ClassC, then if not found > on ClassB, then ClassA then object In Python 2, I am using the following function to find out such information. from inspect import getmro def definedBy(name, class_): '''return *class_* base class defining *name*. *class_* may (now) also be an object. In this case, its class is used. ''' if not hasattr(class_, '__bases__'): class_ = class_.__class__ for cl in getmro(class_): if hasattr(cl,'__dict__'): if cl.__dict__.has_key(name): return cl elif hasattr(cl, name): return cl return None (Unlike other approaches reported in this thread) it not only works for methods but also for other attributes. I am using this for (interactive) debugging purposes: usually, I work with Zope/Plone which is a huge software stack and their it is handy to be able to quickly find out where something is defined in the code. From breamoreboy at yahoo.co.uk Fri Feb 5 03:07:57 2016 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Fri, 5 Feb 2016 08:07:57 +0000 Subject: Reply to whom? In-Reply-To: References: <56B263C1.7030309@mail.de> <56B26940.3090609@mail.de> <56B26D3D.6040703@mail.de> <56b2925a$0$1585$c3e8da3$5496439d@news.astraweb.com> <8560y5cmcv.fsf@benfinney.id.au> <851t8tcges.fsf@benfinney.id.au> Message-ID: On 04/02/2016 03:23, Bernardo Sulzbach wrote: > I see. I've bad experiences with Thunderbird in the past, but I will > try a desktop client again. > I've been using Thunderbird on Windows for years and never had a problem. I read all Python mailing list, blogs, or whatever via gmane, it's a piece of cake. -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From frank at chagford.com Fri Feb 5 03:28:56 2016 From: frank at chagford.com (Frank Millman) Date: Fri, 5 Feb 2016 10:28:56 +0200 Subject: Question about official API Message-ID: Hi all What is the rule for knowing if something is part of the official API? I have a queue.Queue(), I want to call q.join(), but I do not want it to block. Looking at dir(q), I find an attribute 'unfinished_tasks'. It is an integer, and it looks like the counter referred to in the documentation for 'task_done()'. I tried it out, and it does exactly what I want. However, it is not mentioned in the documentation. How do I know if it is safe to rely on this? Thanks Frank Millman From lutz.horn at outlook.de Fri Feb 5 03:33:04 2016 From: lutz.horn at outlook.de (Lutz Horn) Date: Fri, 5 Feb 2016 09:33:04 +0100 Subject: Question about official API In-Reply-To: References: Message-ID: Hi, > What is the rule for knowing if something is part of the official API? Look into https://docs.python.org/3/library/ Lutz From ben+python at benfinney.id.au Fri Feb 5 03:37:01 2016 From: ben+python at benfinney.id.au (Ben Finney) Date: Fri, 05 Feb 2016 19:37:01 +1100 Subject: Question about official API References: Message-ID: <85si17a6z6.fsf@benfinney.id.au> "Frank Millman" writes: > What is the rule for knowing if something is part of the official API? Part of what official API? Different libraries will have different rules about what is the official API. Some may not have official rules. For Python standard library modules, the official API is in the user documentation for each module. > However, it is not mentioned in the [standard library] documentation. Then it's not part of the official API (or the documentation has a bug which needs to be reported). -- \ ?The long-term solution to mountains of waste is not more | `\ landfill sites but fewer shopping centres.? ?Clive Hamilton, | _o__) _Affluenza_, 2005 | Ben Finney From frank at chagford.com Fri Feb 5 03:44:36 2016 From: frank at chagford.com (Frank Millman) Date: Fri, 5 Feb 2016 10:44:36 +0200 Subject: Question about official API In-Reply-To: References: Message-ID: > "Lutz Horn" wrote in message > news:BLU178-W1837247AF25E5755AF69EB9ED20 at phx.gbl... > Hi, > > > What is the rule for knowing if something is part of the official API? > > Look into https://docs.python.org/3/library/ > Thanks for the link, Lutz. Unfortunately I may have asked the wrong question. In my specific case, how do I know if it is safe to use the attribute 'unfinished_tasks' in the class queue.Queue? It could be that it is intended for use, but has been omitted from the documentation in error. Based on Ben's response I will raise an issue. At least that will settle the question one way or the other. Frank From frank at chagford.com Fri Feb 5 04:04:30 2016 From: frank at chagford.com (Frank Millman) Date: Fri, 5 Feb 2016 11:04:30 +0200 Subject: Question about official API In-Reply-To: References: Message-ID: "Frank Millman" wrote in message news:n91ndn$sc1$1 at ger.gmane.org... > > Thanks for the link, Lutz. Unfortunately I may have asked the wrong > question. > > In my specific case, how do I know if it is safe to use the attribute > 'unfinished_tasks' in the class queue.Queue? > > It could be that it is intended for use, but has been omitted from the > documentation in error. > > Based on Ben's response I will raise an issue. At least that will settle > the question one way or the other. > http://bugs.python.org/issue26294 From df at see.replyto.invalid Fri Feb 5 06:20:44 2016 From: df at see.replyto.invalid (Dave Farrance) Date: Fri, 05 Feb 2016 11:20:44 +0000 Subject: Tkinter problem: TclError> couldn't connect to display ":0 References: Message-ID: gemjack.pb at gmail.com wrote: >This fixed my problem with thkinter..... sudo cp ~/.Xauthority ~root/ Which means that you were creating a GUI window with Python as root, which is to be avoided if you can. If you can't avoid it and you're running it with sudo in a bash console, rather than a root console, then I'd suggest adding the line... XAUTHORITY=$HOME/.Xauthority ...to the root's .bashrc which avoids putting a specific user's xauthority file in the root directory. From reach.ram2020 at gmail.com Fri Feb 5 09:35:02 2016 From: reach.ram2020 at gmail.com (reach.ram2020 at gmail.com) Date: Fri, 5 Feb 2016 06:35:02 -0800 (PST) Subject: sharepoint python Message-ID: <9cfa88b1-432b-4be0-92e7-e9b661b3d24f@googlegroups.com> Hi Folks, Is there a python package available to check-in files from Unix to sharepoint? I hope current sharepoint package is used to read from sharepoint server. Thanks, Ramesh From srkunze at mail.de Fri Feb 5 09:42:29 2016 From: srkunze at mail.de (Sven R. Kunze) Date: Fri, 5 Feb 2016 15:42:29 +0100 Subject: _siftup and _siftdown implementation In-Reply-To: References: <56b3e902$0$1613$c3e8da3$5496439d@news.astraweb.com> Message-ID: <56B4B4D5.20804@mail.de> On 05.02.2016 02:26, srinivas devaki wrote: > as I come to think of it again, it is not subheap, it actually heap cut at > some level hope you get the idea from the usage of _siftup. so even though > the `pos` children are valid the _siftup brings down the new element (i.e > the element which is at first at `pos`) upto its leaf level and then again > it is brought up by using _siftdown. why do the redundant work when it can > simply breakout? The heapq module itself has a very extensive documentation inside. This is what it says for _siftup. I think this is sort of an optimization that works pretty well (cf. the numbers) for popping off the FIRST item: """ # The child indices of heap index pos are already heaps, and we want to make # a heap at index pos too. We do this by bubbling the smaller child of # pos up (and so on with that child's children, etc) until hitting a leaf, # then using _siftdown to move the oddball originally at index pos into place. # # We *could* break out of the loop as soon as we find a pos where newitem <= # both its children, but turns out that's not a good idea, and despite that # many books write the algorithm that way. During a heap pop, the last array # element is sifted in, and that tends to be large, so that comparing it # against values starting from the root usually doesn't pay (= usually doesn't # get us out of the loop early). See Knuth, Volume 3, where this is # explained and quantified in an exercise. # # Cutting the # of comparisons is important, since these routines have no # way to extract "the priority" from an array element, so that intelligence # is likely to be hiding in custom comparison methods, or in array elements # storing (priority, record) tuples. Comparisons are thus potentially # expensive. # # On random arrays of length 1000, making this change cut the number of # comparisons made by heapify() a little, and those made by exhaustive # heappop() a lot, in accord with theory. Here are typical results from 3 # runs (3 just to demonstrate how small the variance is): # # Compares needed by heapify Compares needed by 1000 heappops # -------------------------- -------------------------------- # 1837 cut to 1663 14996 cut to 8680 # 1855 cut to 1659 14966 cut to 8678 # 1847 cut to 1660 15024 cut to 8703 # # Building the heap by using heappush() 1000 times instead required # 2198, 2148, and 2219 compares: heapify() is more efficient, when # you can use it. # # The total compares needed by list.sort() on the same lists were 8627, # 8627, and 8632 (this should be compared to the sum of heapify() and # heappop() compares): list.sort() is (unsurprisingly!) more efficient # for sorting. """ What do you think about our use-case? > _siftup and _siftdown are functions from python standard heapq module. > > PS: I do competitive programming, I use these modules every couple of days > when compared to other modules. so didn't give much thought when posting to > the mailing list. sorry for that. Competitive programming? That sounds interesting. :) Best, Sven From mafagafogigante at gmail.com Fri Feb 5 09:48:18 2016 From: mafagafogigante at gmail.com (Bernardo Sulzbach) Date: Fri, 5 Feb 2016 12:48:18 -0200 Subject: _siftup and _siftdown implementation In-Reply-To: <56B4B4D5.20804@mail.de> References: <56b3e902$0$1613$c3e8da3$5496439d@news.astraweb.com> <56B4B4D5.20804@mail.de> Message-ID: <56B4B632.1030404@gmail.com> On 02/05/2016 12:42 PM, Sven R. Kunze wrote: >> >> PS: I do competitive programming, I use these modules every couple of >> days >> when compared to other modules. so didn't give much thought when >> posting to >> the mailing list. sorry for that. > > Competitive programming? That sounds interesting. :) > I wonder why you *can* use this amount of already done stuff in competitive programming. When I was into that you could use what the standard library of the language gave you and nothing else. From srkunze at mail.de Fri Feb 5 09:55:12 2016 From: srkunze at mail.de (Sven R. Kunze) Date: Fri, 5 Feb 2016 15:55:12 +0100 Subject: _siftup and _siftdown implementation In-Reply-To: <56B4B632.1030404@gmail.com> References: <56b3e902$0$1613$c3e8da3$5496439d@news.astraweb.com> <56B4B4D5.20804@mail.de> <56B4B632.1030404@gmail.com> Message-ID: <56B4B7D0.9090807@mail.de> On 05.02.2016 15:48, Bernardo Sulzbach wrote: > On 02/05/2016 12:42 PM, Sven R. Kunze wrote: >>> >>> PS: I do competitive programming, I use these modules every couple of >>> days >>> when compared to other modules. so didn't give much thought when >>> posting to >>> the mailing list. sorry for that. >> >> Competitive programming? That sounds interesting. :) >> > > I wonder why you *can* use this amount of already done stuff in > competitive programming. When I was into that you could use what the > standard library of the language gave you and nothing else. AFAICT, heapq is part of the standard lib. :) Best, Sven From mafagafogigante at gmail.com Fri Feb 5 09:59:39 2016 From: mafagafogigante at gmail.com (Bernardo Sulzbach) Date: Fri, 5 Feb 2016 12:59:39 -0200 Subject: _siftup and _siftdown implementation In-Reply-To: <56B4B7D0.9090807@mail.de> References: <56b3e902$0$1613$c3e8da3$5496439d@news.astraweb.com> <56B4B4D5.20804@mail.de> <56B4B632.1030404@gmail.com> <56B4B7D0.9090807@mail.de> Message-ID: <56B4B8DB.20509@gmail.com> On 02/05/2016 12:55 PM, Sven R. Kunze wrote: > On 05.02.2016 15:48, Bernardo Sulzbach wrote: >> On 02/05/2016 12:42 PM, Sven R. Kunze wrote: >>>> >>>> PS: I do competitive programming, I use these modules every couple of >>>> days >>>> when compared to other modules. so didn't give much thought when >>>> posting to >>>> the mailing list. sorry for that. >>> >>> Competitive programming? That sounds interesting. :) >>> >> >> I wonder why you *can* use this amount of already done stuff in >> competitive programming. When I was into that you could use what the >> standard library of the language gave you and nothing else. > > AFAICT, heapq is part of the standard lib. :) > Yes. I thought he was talking about XHEAP. However, rereading makes it look like he meant heapq indeed. From matt.mailinglists at gmail.com Fri Feb 5 10:16:51 2016 From: matt.mailinglists at gmail.com (Matt) Date: Fri, 5 Feb 2016 09:16:51 -0600 Subject: snmpset Message-ID: How do I do the equivalent of this in Python? snmpset -v 2c -c $community $ip .1.3.1.1.4.1.1.1.1.1.1.1.0 s test and snmpset -v 2c -c $community $ip .1.3.1.1.4.1.1.1.1.1.1.1.0 i 123 and snmpbulkget -v2c -c $community -m ALL $ip .1.3.1.1.4.1.1.1.1.1.1.1.1 and snmpget -v2c -c $community -m ALL $ip .1.3.1.1.4.1.1.1.1.1.1.1.2 From mr.eightnoteight at gmail.com Fri Feb 5 10:45:33 2016 From: mr.eightnoteight at gmail.com (srinivas devaki) Date: Fri, 5 Feb 2016 21:15:33 +0530 Subject: _siftup and _siftdown implementation In-Reply-To: <56B4B4D5.20804@mail.de> References: <56b3e902$0$1613$c3e8da3$5496439d@news.astraweb.com> <56B4B4D5.20804@mail.de> Message-ID: On Fri, Feb 5, 2016 at 8:12 PM, Sven R. Kunze wrote: > On 05.02.2016 02:26, srinivas devaki wrote: > What do you think about our use-case? > Oh, the logic is sound, every element that we have inserted has to be popped, We are spending some *extra* time in rearranging the elements only to be sure that we won't be spending more than this *extra* time when doing other operations, and our use-case isn't much different either, If by rearranging the elements in the heap(*subheap*) gets optimal for other operations like popping the root element(heap[0]) then obviously it is optimal for popping other elements (children of heap[0]). PS: @sven But don't yet merge the pull request, I could be wrong. as the heapq module already says that the variance is very small, let me write some tests(on more than 10**3 elements) and then get back here. -- Regards Srinivas Devaki Junior (3rd yr) student at Indian School of Mines,(IIT Dhanbad) Computer Science and Engineering Department ph: +91 9491 383 249 telegram_id: @eightnoteight From zachary.ware+pylist at gmail.com Fri Feb 5 11:10:51 2016 From: zachary.ware+pylist at gmail.com (Zachary Ware) Date: Fri, 5 Feb 2016 10:10:51 -0600 Subject: snmpset In-Reply-To: References: Message-ID: On Fri, Feb 5, 2016 at 9:16 AM, Matt wrote: > How do I do the equivalent of this in Python? > > snmpset -v 2c -c $community $ip .1.3.1.1.4.1.1.1.1.1.1.1.0 s test > > and > > snmpset -v 2c -c $community $ip .1.3.1.1.4.1.1.1.1.1.1.1.0 i 123 > > and > > snmpbulkget -v2c -c $community -m ALL $ip .1.3.1.1.4.1.1.1.1.1.1.1.1 > > and > > snmpget -v2c -c $community -m ALL $ip .1.3.1.1.4.1.1.1.1.1.1.1.2 I have had success with pysnmp (http://pysnmp.sourceforge.net/). It is far from a shining example of a Pythonic API and I don't envy you if you should need to subclass something to change its behavior, but it is effective. -- Zach From srkunze at mail.de Fri Feb 5 11:27:39 2016 From: srkunze at mail.de (Sven R. Kunze) Date: Fri, 5 Feb 2016 17:27:39 +0100 Subject: _siftup and _siftdown implementation In-Reply-To: References: <56b3e902$0$1613$c3e8da3$5496439d@news.astraweb.com> Message-ID: <56B4CD7B.70506@mail.de> Hi srinivas, I wrote this simple benchmark to measure comparisons: import random from xheapimport RemovalHeap class X(object): c =0 def __init__(self, x): self.x = x def __lt__(self, other): X.c +=1 return self.x < other.x n =100000 for jjin range(5): items = [X(i)for iin range(n)] random.shuffle(items) heap = RemovalHeap(items) random.shuffle(items) for i in items: heap.remove(i) print(X.c) X.c =0 old version: 430457 430810 430657 429971 430583 your pull request version: 426414 426045 425437 425528 425522 Can we do better here? Best, Sven From srkunze at mail.de Fri Feb 5 11:35:37 2016 From: srkunze at mail.de (Sven R. Kunze) Date: Fri, 5 Feb 2016 17:35:37 +0100 Subject: _siftup and _siftdown implementation In-Reply-To: <56B4CD7B.70506@mail.de> References: <56b3e902$0$1613$c3e8da3$5496439d@news.astraweb.com> <56B4CD7B.70506@mail.de> Message-ID: <56B4CF59.5020404@mail.de> again for the list: ################### import random from xheap import RemovalHeap class X(object): c = 0 def __init__(self, x): self.x = x def __lt__(self, other): X.c += 1 return self.x < other.x n = 100000 for jj in range(5): items = [X(i) for i in range(n)] random.shuffle(items) heap = RemovalHeap(items) random.shuffle(items) for i in items: heap.remove(i) print(X.c) X.c = 0 (note to myself: never copy PyCharm formatting strings to this list). On 05.02.2016 17:27, Sven R. Kunze wrote: > Hi srinivas, > > I wrote this simple benchmark to measure comparisons: > > import random > > from xheapimport RemovalHeap > > > class X(object): > c =0 def __init__(self, x): > self.x = x > def __lt__(self, other): > X.c +=1 return self.x < other.x > > n =100000 for jjin range(5): > items = [X(i)for iin range(n)] > random.shuffle(items) > heap = RemovalHeap(items) > > random.shuffle(items) > for i in items: > heap.remove(i) > > print(X.c) > X.c =0 > > > old version: > 430457 > 430810 > 430657 > 429971 > 430583 > > your pull request version: > 426414 > 426045 > 425437 > 425528 > 425522 > > > Can we do better here? > > Best, > Sven From joel.goldstick at gmail.com Fri Feb 5 12:12:58 2016 From: joel.goldstick at gmail.com (Joel Goldstick) Date: Fri, 5 Feb 2016 12:12:58 -0500 Subject: snmpset In-Reply-To: References: Message-ID: On Fri, Feb 5, 2016 at 11:10 AM, Zachary Ware wrote: > On Fri, Feb 5, 2016 at 9:16 AM, Matt wrote: > > How do I do the equivalent of this in Python? > > > > snmpset -v 2c -c $community $ip .1.3.1.1.4.1.1.1.1.1.1.1.0 s test > > > > and > > > > snmpset -v 2c -c $community $ip .1.3.1.1.4.1.1.1.1.1.1.1.0 i 123 > > > > and > > > > snmpbulkget -v2c -c $community -m ALL $ip .1.3.1.1.4.1.1.1.1.1.1.1.1 > > > > and > > > > snmpget -v2c -c $community -m ALL $ip .1.3.1.1.4.1.1.1.1.1.1.1.2 > > I have had success with pysnmp (http://pysnmp.sourceforge.net/). It > is far from a shining example of a Pythonic API and I don't envy you > if you should need to subclass something to change its behavior, but > it is effective. > > -- > Zach > -- > https://mail.python.org/mailman/listinfo/python-list > That page 404s for me -- Joel Goldstick http://joelgoldstick.com/stats/birthdays From joel.goldstick at gmail.com Fri Feb 5 12:14:16 2016 From: joel.goldstick at gmail.com (Joel Goldstick) Date: Fri, 5 Feb 2016 12:14:16 -0500 Subject: snmpset In-Reply-To: References: Message-ID: On Fri, Feb 5, 2016 at 12:12 PM, Joel Goldstick wrote: > > > On Fri, Feb 5, 2016 at 11:10 AM, Zachary Ware < > zachary.ware+pylist at gmail.com> wrote: > >> On Fri, Feb 5, 2016 at 9:16 AM, Matt wrote: >> > How do I do the equivalent of this in Python? >> > >> > snmpset -v 2c -c $community $ip .1.3.1.1.4.1.1.1.1.1.1.1.0 s test >> > >> > and >> > >> > snmpset -v 2c -c $community $ip .1.3.1.1.4.1.1.1.1.1.1.1.0 i 123 >> > >> > and >> > >> > snmpbulkget -v2c -c $community -m ALL $ip .1.3.1.1.4.1.1.1.1.1.1.1.1 >> > >> > and >> > >> > snmpget -v2c -c $community -m ALL $ip .1.3.1.1.4.1.1.1.1.1.1.1.2 >> >> I have had success with pysnmp (http://pysnmp.sourceforge.net/). It >> is far from a shining example of a Pythonic API and I don't envy you >> if you should need to subclass something to change its behavior, but >> it is effective. >> >> -- >> Zach >> -- >> https://mail.python.org/mailman/listinfo/python-list >> > > That page 404s for me. > Pardon me, looks like sourceforge is down > > > -- > Joel Goldstick > http://joelgoldstick.com/stats/birthdays > -- Joel Goldstick http://joelgoldstick.com/stats/birthdays From invalid at invalid.invalid Fri Feb 5 12:18:31 2016 From: invalid at invalid.invalid (Grant Edwards) Date: Fri, 5 Feb 2016 17:18:31 +0000 (UTC) Subject: snmpset References: Message-ID: On 2016-02-05, Joel Goldstick wrote: >> I have had success with pysnmp (http://pysnmp.sourceforge.net/). > > That page 404s for me Looks like sourceforge is suffering an outage of some kind. -- Grant Edwards grant.b.edwards Yow! The FALAFEL SANDWICH at lands on my HEAD and I gmail.com become a VEGETARIAN ... From zachary.ware+pylist at gmail.com Fri Feb 5 12:18:47 2016 From: zachary.ware+pylist at gmail.com (Zachary Ware) Date: Fri, 5 Feb 2016 11:18:47 -0600 Subject: snmpset In-Reply-To: References: Message-ID: On Fri, Feb 5, 2016 at 11:12 AM, Joel Goldstick wrote: > On Fri, Feb 5, 2016 at 11:10 AM, Zachary Ware > wrote: >> I have had success with pysnmp (http://pysnmp.sourceforge.net/). It > > That page 404s for me Hmm, it works for me (just tried again). Even Gmail's automatic linkification didn't kill it. -- Zach From mailman at hanez.org Fri Feb 5 12:20:27 2016 From: mailman at hanez.org (Johannes Findeisen) Date: Fri, 5 Feb 2016 18:20:27 +0100 Subject: snmpset In-Reply-To: References: Message-ID: <20160205182027.0612ecb4@phantom> On Fri, 5 Feb 2016 12:12:58 -0500 Joel Goldstick wrote: > On Fri, Feb 5, 2016 at 11:10 AM, Zachary Ware > wrote: > > > On Fri, Feb 5, 2016 at 9:16 AM, Matt wrote: > > > How do I do the equivalent of this in Python? > > > > > > snmpset -v 2c -c $community $ip .1.3.1.1.4.1.1.1.1.1.1.1.0 s test > > > > > > and > > > > > > snmpset -v 2c -c $community $ip .1.3.1.1.4.1.1.1.1.1.1.1.0 i 123 > > > > > > and > > > > > > snmpbulkget -v2c -c $community -m ALL $ip .1.3.1.1.4.1.1.1.1.1.1.1.1 > > > > > > and > > > > > > snmpget -v2c -c $community -m ALL $ip .1.3.1.1.4.1.1.1.1.1.1.1.2 > > > > I have had success with pysnmp (http://pysnmp.sourceforge.net/). It > > is far from a shining example of a Pythonic API and I don't envy you > > if you should need to subclass something to change its behavior, but > > it is effective. > > > > -- > > Zach > > -- > > https://mail.python.org/mailman/listinfo/python-list > > > > That page 404s for me SourceForge seems to have problems at all... The Homepage is down too. Johannes From mafagafogigante at gmail.com Fri Feb 5 12:21:18 2016 From: mafagafogigante at gmail.com (Bernardo Sulzbach) Date: Fri, 5 Feb 2016 15:21:18 -0200 Subject: snmpset In-Reply-To: References: Message-ID: <56B4DA0E.9010408@gmail.com> On 02/05/2016 03:18 PM, Grant Edwards wrote: > On 2016-02-05, Joel Goldstick wrote: > >>> I have had success with pysnmp (http://pysnmp.sourceforge.net/). >> >> That page 404s for me > > Looks like sourceforge is suffering an outage of some kind. > Agree, it does not work for me right now. From zachary.ware+pylist at gmail.com Fri Feb 5 12:24:34 2016 From: zachary.ware+pylist at gmail.com (Zachary Ware) Date: Fri, 5 Feb 2016 11:24:34 -0600 Subject: snmpset In-Reply-To: References: Message-ID: On Fri, Feb 5, 2016 at 11:14 AM, Joel Goldstick wrote: > On Fri, Feb 5, 2016 at 12:12 PM, Joel Goldstick > wrote: >> That page 404s for me. >> > > Pardon me, looks like sourceforge is down Ah, I guess caching fooled me when I rechecked. -- Zach From joel.goldstick at gmail.com Fri Feb 5 12:31:46 2016 From: joel.goldstick at gmail.com (Joel Goldstick) Date: Fri, 5 Feb 2016 12:31:46 -0500 Subject: sharepoint python In-Reply-To: <9cfa88b1-432b-4be0-92e7-e9b661b3d24f@googlegroups.com> References: <9cfa88b1-432b-4be0-92e7-e9b661b3d24f@googlegroups.com> Message-ID: On Fri, Feb 5, 2016 at 9:35 AM, wrote: > Hi Folks, > > Is there a python package available to check-in files from Unix to > sharepoint? > I hope current sharepoint package is used to read from sharepoint server. > > Thanks, > Ramesh > -- > https://mail.python.org/mailman/listinfo/python-list > Does this help? https://pypi.python.org/pypi/sharepoint/0.3.2 -- Joel Goldstick http://joelgoldstick.com/stats/birthdays From joel.goldstick at gmail.com Fri Feb 5 12:34:08 2016 From: joel.goldstick at gmail.com (Joel Goldstick) Date: Fri, 5 Feb 2016 12:34:08 -0500 Subject: sharepoint python In-Reply-To: References: <9cfa88b1-432b-4be0-92e7-e9b661b3d24f@googlegroups.com> Message-ID: On Fri, Feb 5, 2016 at 12:31 PM, Joel Goldstick wrote: > > > On Fri, Feb 5, 2016 at 9:35 AM, wrote: > >> Hi Folks, >> >> Is there a python package available to check-in files from Unix to >> sharepoint? >> I hope current sharepoint package is used to read from sharepoint server. >> >> Thanks, >> Ramesh >> -- >> https://mail.python.org/mailman/listinfo/python-list >> > > Does this help? https://pypi.python.org/pypi/sharepoint/0.3.2 > or this: http://stackoverflow.com/questions/23696705/how-to-upload-a-file-to-sharepoint-site-using-python-script > > > -- > Joel Goldstick > http://joelgoldstick.com/stats/birthdays > -- Joel Goldstick http://joelgoldstick.com/stats/birthdays From mr.eightnoteight at gmail.com Fri Feb 5 12:42:40 2016 From: mr.eightnoteight at gmail.com (srinivas devaki) Date: Fri, 5 Feb 2016 23:12:40 +0530 Subject: _siftup and _siftdown implementation In-Reply-To: <56B4CD7B.70506@mail.de> References: <56b3e902$0$1613$c3e8da3$5496439d@news.astraweb.com> <56B4CD7B.70506@mail.de> Message-ID: wow, that's great. you read a comment in the code, and you test it, to only find that it is indeed true, sounds ok, but feels great. :) Just experimenting a bit, I swaped the lines _siftdown and _siftup and something strange happened the number of comparisions in both the versions remained same. I'm attaching the files. do you have any idea why this happened? On Fri, Feb 5, 2016 at 9:57 PM, Sven R. Kunze wrote: > > Can we do better here? > I don't know, I have to read TAOP knuth article. -- Regards Srinivas Devaki Junior (3rd yr) student at Indian School of Mines,(IIT Dhanbad) Computer Science and Engineering Department ph: +91 9491 383 249 telegram_id: @eightnoteight From paul.hermeneutic at gmail.com Fri Feb 5 13:39:35 2016 From: paul.hermeneutic at gmail.com (paul.hermeneutic at gmail.com) Date: Fri, 5 Feb 2016 11:39:35 -0700 Subject: Daemon strategy Message-ID: It appears that python-deamon would be exactly what I need. Alas, appears not to run on Windows. If I am wrong about that, please tell me. To what tools should I turn? I am not eager to produce a "service" on Windows unless it cannot be avoided. From ben+python at benfinney.id.au Fri Feb 5 13:52:36 2016 From: ben+python at benfinney.id.au (Ben Finney) Date: Sat, 06 Feb 2016 05:52:36 +1100 Subject: Daemon strategy References: Message-ID: <85d1sb9eh7.fsf@benfinney.id.au> paul.hermeneutic at gmail.com writes: > It appears that python-deamon would be exactly what I need. Alas, > appears not to run on Windows. If I am wrong about that, please tell > me. You're correct that ?python-daemon? uses Unix facilities to create a well-behaved Unix daemon process. Since MS Windows lacks those facilities, ?python-daemon? can't use them. > To what tools should I turn? If what you need ? the support to create a Unix daemon process ? is available only on Unix by definition, it seems you'll need to deploy to Unix. > I am not eager to produce a "service" on Windows unless it cannot be > avoided. Agreed :-) -- \ ?Wrinkles should merely indicate where smiles have been.? ?Mark | `\ Twain, _Following the Equator_ | _o__) | Ben Finney From rgacote at appropriatesolutions.com Fri Feb 5 13:56:40 2016 From: rgacote at appropriatesolutions.com (Ray Cote) Date: Fri, 5 Feb 2016 13:56:40 -0500 Subject: Daemon strategy In-Reply-To: References: Message-ID: We?ve recently stopped building Windows services (Python or any other type) and started using the NSSM service manager. Takes a normal Windows application and runs it as a service. The NSSM service manager provides great command-line support for installing, configuring, and controlling Windows services. Has greatly reduced our development time and test time since we no longer need to test both app and service executables. Downside is that it is a separate tool to install. Upside is that it has features such as : - run-once at startup. - auto-restart failed services - setting service dependences (don?t start until another service is up and running) - set processor dependencies ?Ray On Fri, Feb 5, 2016 at 1:39 PM, wrote: > It appears that python-deamon would be exactly what I need. Alas, > appears not to run on Windows. If I am wrong about that, please tell > me. > > To what tools should I turn? > > I am not eager to produce a "service" on Windows unless it cannot be > avoided. > -- > https://mail.python.org/mailman/listinfo/python-list > -- Raymond Cote, President voice: +1.603.924.6079 email: rgacote at AppropriateSolutions.com skype: ray.cote From nhilterbrand at gmail.com Fri Feb 5 13:58:15 2016 From: nhilterbrand at gmail.com (Nathan Hilterbrand) Date: Fri, 5 Feb 2016 13:58:15 -0500 Subject: Daemon strategy In-Reply-To: References: Message-ID: <56B4F0C7.9060908@gmail.com> On 02/05/2016 01:39 PM, paul.hermeneutic at gmail.com wrote: > It appears that python-deamon would be exactly what I need. Alas, > appears not to run on Windows. If I am wrong about that, please tell > me. > > To what tools should I turn? > > I am not eager to produce a "service" on Windows unless it cannot be avoided. I am not 100% sure exactly what you want to do, but you might take a look at using 'pythonw.exe' instead of 'python.exe' YMMV Nathan From tuxtimo at gmail.com Fri Feb 5 13:58:42 2016 From: tuxtimo at gmail.com (Timo Furrer) Date: Fri, 5 Feb 2016 10:58:42 -0800 Subject: Daemon strategy In-Reply-To: <85d1sb9eh7.fsf@benfinney.id.au> References: <85d1sb9eh7.fsf@benfinney.id.au> Message-ID: As Ben already said .. either deploy to Unix systems or use subprocess.Popen and detach the process: from subprocess import Popenfrom win32process import DETACHED_PROCESS Popen(["YOURPROCESS"],creationflags=DETACHED_PROCESS,shell=True) On Fri, Feb 5, 2016 at 10:52 AM, Ben Finney wrote: > paul.hermeneutic at gmail.com writes: > > > It appears that python-deamon would be exactly what I need. Alas, > > appears not to run on Windows. If I am wrong about that, please tell > > me. > > You're correct that ?python-daemon? uses Unix facilities to create a > well-behaved Unix daemon process. > > Since MS Windows lacks those facilities, ?python-daemon? can't use them. > > > To what tools should I turn? > > If what you need ? the support to create a Unix daemon process ? is > available only on Unix by definition, it seems you'll need to deploy to > Unix. > > > I am not eager to produce a "service" on Windows unless it cannot be > > avoided. > > Agreed :-) > > -- > \ ?Wrinkles should merely indicate where smiles have been.? ?Mark | > `\ Twain, _Following the Equator_ | > _o__) | > Ben Finney > > -- > https://mail.python.org/mailman/listinfo/python-list > -- *Timo Furrer* https://github.com/timofurrer tuxtimo at gmail.com From paul.hermeneutic at gmail.com Fri Feb 5 14:36:14 2016 From: paul.hermeneutic at gmail.com (paul.hermeneutic at gmail.com) Date: Fri, 5 Feb 2016 12:36:14 -0700 Subject: Daemon strategy In-Reply-To: <85d1sb9eh7.fsf@benfinney.id.au> References: <85d1sb9eh7.fsf@benfinney.id.au> Message-ID: On Fri, Feb 5, 2016 at 11:52 AM, Ben Finney wrote: > paul.hermeneutic at gmail.com writes: > >> It appears that python-deamon would be exactly what I need. Alas, >> appears not to run on Windows. If I am wrong about that, please tell >> me. > > You're correct that ?python-daemon? uses Unix facilities to create a > well-behaved Unix daemon process. > > Since MS Windows lacks those facilities, ?python-daemon? can't use them. As you might imagine, I am not always able to specify which OS is deployed. That does not mean that I am not responsible for getting the work done. Perhaps you will tell me that what I want is not a daemon. BTW, I thought that pip would know that python-daemon would not run on my machine, but it had no complaints installing it. That gave me unmerited hope. I want to create a program that will run a list of command lines. Some of the tasks might take 42 milliseconds, but others might take 4.2 hours. I need to be able to: - list the tasks currently being run - kill a task that is currently being run - list the tasks that are not yet run - delete a task not yet run from the list - add tasks to the list The goal is to load the machine up to a specified percentage of CPU/memory/io bandwidth. I want to keep the machine loaded up to 90% of capacity. It is important to monitor more than just CPU utilization because the machine may already be over-committed on memory while CPU utilization is low. Adding more processes in such an environment just makes the system go slower. I realize that high-end schedulers like IBM/Tivoli, CA7, and BMC might do things like that. Those are not usually within budget. Is a daemon what I need? Any other suggestions? From lucan at NNN.it Fri Feb 5 14:49:32 2016 From: lucan at NNN.it (lucan) Date: Fri, 5 Feb 2016 20:49:32 +0100 Subject: realtime output and csv files Message-ID: I'm new of python adn I'm using it only to complete some experiments. I'm reading a series of data from various sensors and in the meantime I'm writing datas on a file. I would like to print output in realtime (or refresh it when I need) but the problem is that I'm writing on a file every x seconds (my timestep). Anyway from the moment that datas are scientific value is it correct to write on a file using str(temp) and separating with ","? I need a csv file to read it with some data analysis softwares. ... now = datetime.datetime.now() dayexperiment = str(now.day)+str(now.month)+str(now.year) myfilename ="myData"+dayexperiment+".dat" index = 0 count = 1 timestep = 10 with open(myfilename,'w') as f: f.write('#Index Time Value') while True: reading = ... ... temp=... print 'Temp{0:0.3F}*C'.format(temp) now = datetime.datetime.now() file.write('\n'+str(index)) f.write(str(index)+','+str(now.hour)+':'+str(now.minute)+', '+str(temp)) index +=1 time.sleep(timestep) From mafagafogigante at gmail.com Fri Feb 5 14:57:32 2016 From: mafagafogigante at gmail.com (Bernardo Sulzbach) Date: Fri, 5 Feb 2016 17:57:32 -0200 Subject: realtime output and csv files In-Reply-To: References: Message-ID: <56B4FEAC.4080706@gmail.com> On 02/05/2016 05:49 PM, lucan wrote: > Anyway from the moment that datas are scientific value is it correct to > write on a file using str(temp) and separating with ","? > I need a csv file to read it with some data analysis softwares. > What do you mean? What is "datas"? What do you mean by "correct"? CSVs is essentially text separated by commas, so you likely do not need any library to write it "Just separating with ','" should work if you are formatting them correctly. From shaunak.bangale at gmail.com Fri Feb 5 14:58:25 2016 From: shaunak.bangale at gmail.com (shaunak.bangale at gmail.com) Date: Fri, 5 Feb 2016 11:58:25 -0800 (PST) Subject: Exception handling for socket.error in Python 3.5/RStudio Message-ID: <1067e50c-bcbd-464e-8013-754bd66fda91@googlegroups.com> I am running this python script on R-studio. I have Python 3.5 installed on my system. count = 10 while (count > 0): try : # read line from file: print(file.readline()) # parse parse_json(file.readline()) count = count - 1 except socket.error as e print('Connection fail', e) print(traceback.format_exc()) # wait for user input to end # input("\n Press Enter to exit..."); # close the SSLSocket, will also close the underlying socket ssl_sock.close() The error I am getting is here: line 53 except socket.error as e ^ SyntaxError: invalid syntax I tried changing socket.error to ConnectionRefusedError. and still got the same error. Please tell me if the problem is with Rstudio, Python version or the syntax. TIA -Shaunak From nhilterbrand at gmail.com Fri Feb 5 15:07:49 2016 From: nhilterbrand at gmail.com (Nathan Hilterbrand) Date: Fri, 5 Feb 2016 15:07:49 -0500 Subject: Exception handling for socket.error in Python 3.5/RStudio In-Reply-To: <1067e50c-bcbd-464e-8013-754bd66fda91@googlegroups.com> References: <1067e50c-bcbd-464e-8013-754bd66fda91@googlegroups.com> Message-ID: On Feb 5, 2016 15:01, wrote: > > I am running this python script on R-studio. I have Python 3.5 installed on my system. > > count = 10 > while (count > 0): > try : > # read line from file: > print(file.readline()) > # parse > parse_json(file.readline()) > count = count - 1 > except socket.error as e > print('Connection fail', e) > print(traceback.format_exc()) > > # wait for user input to end > # input("\n Press Enter to exit..."); > # close the SSLSocket, will also close the underlying socket > ssl_sock.close() > The error I am getting is here: > > line 53 except socket.error as e ^ SyntaxError: invalid syntax > > I tried changing socket.error to ConnectionRefusedError. and still got the same error. > > Please tell me if the problem is with Rstudio, Python version or the syntax. > > TIA > -Shaunak > -- > https://mail.python.org/mailman/listinfo/python-list Looks like you are missing a colon after 'as e'... .... as e: might do the trick Nathan From martin at linux-ip.net Fri Feb 5 15:08:58 2016 From: martin at linux-ip.net (Martin A. Brown) Date: Fri, 5 Feb 2016 12:08:58 -0800 Subject: Exception handling for socket.error in Python 3.5/RStudio In-Reply-To: <1067e50c-bcbd-464e-8013-754bd66fda91@googlegroups.com> References: <1067e50c-bcbd-464e-8013-754bd66fda91@googlegroups.com> Message-ID: > except socket.error as e >line 53 except socket.error as e ^ SyntaxError: invalid syntax > >I tried changing socket.error to ConnectionRefusedError. and still >got the same error. >Please tell me if the problem is with Rstudio, Python version or >the syntax. Syntax. Your code has, unfortunately, suffered a colonectomy. When you transplant a colon, it is more likely to function properly again. For example: except socket.error as e: Good luck, -Martin -- Martin A. Brown http://linux-ip.net/ From shaunak.bangale at gmail.com Fri Feb 5 15:11:05 2016 From: shaunak.bangale at gmail.com (shaunak.bangale at gmail.com) Date: Fri, 5 Feb 2016 12:11:05 -0800 (PST) Subject: Exception handling for socket.error in Python 3.5/RStudio In-Reply-To: References: <1067e50c-bcbd-464e-8013-754bd66fda91@googlegroups.com> Message-ID: <824579b6-b0c5-4447-a123-3fc2b850369f@googlegroups.com> On Friday, February 5, 2016 at 1:08:11 PM UTC-7, Nathan Hilterbrand wrote: > On Feb 5, 2016 15:01, wrote: > > > > I am running this python script on R-studio. I have Python 3.5 installed > on my system. > > > > count = 10 > > while (count > 0): > > try : > > # read line from file: > > print(file.readline()) > > # parse > > parse_json(file.readline()) > > count = count - 1 > > except socket.error as e > > print('Connection fail', e) > > print(traceback.format_exc()) > > > > # wait for user input to end > > # input("\n Press Enter to exit..."); > > # close the SSLSocket, will also close the underlying socket > > ssl_sock.close() > > The error I am getting is here: > > > > line 53 except socket.error as e ^ SyntaxError: invalid syntax > > > > I tried changing socket.error to ConnectionRefusedError. and still got > the same error. > > > > Please tell me if the problem is with Rstudio, Python version or the > syntax. > > > > TIA > > -Shaunak > > -- > > https://mail.python.org/mailman/listinfo/python-list > > Looks like you are missing a colon after 'as e'... .... as e: might do > the trick > > Nathan Hi Nathan, Tried colon and a comma as well. Both did not work. From shaunak.bangale at gmail.com Fri Feb 5 15:17:04 2016 From: shaunak.bangale at gmail.com (shaunak.bangale at gmail.com) Date: Fri, 5 Feb 2016 12:17:04 -0800 (PST) Subject: Exception handling for socket.error in Python 3.5/RStudio In-Reply-To: <824579b6-b0c5-4447-a123-3fc2b850369f@googlegroups.com> References: <1067e50c-bcbd-464e-8013-754bd66fda91@googlegroups.com> <824579b6-b0c5-4447-a123-3fc2b850369f@googlegroups.com> Message-ID: <0f14bba2-1dbb-4c41-beda-992939b1a968@googlegroups.com> On Friday, February 5, 2016 at 1:11:19 PM UTC-7, shaunak... at gmail.com wrote: > On Friday, February 5, 2016 at 1:08:11 PM UTC-7, Nathan Hilterbrand wrote: > > On Feb 5, 2016 15:01, wrote: > > > > > > I am running this python script on R-studio. I have Python 3.5 installed > > on my system. > > > > > > count = 10 > > > while (count > 0): > > > try : > > > # read line from file: > > > print(file.readline()) > > > # parse > > > parse_json(file.readline()) > > > count = count - 1 > > > except socket.error as e > > > print('Connection fail', e) > > > print(traceback.format_exc()) > > > > > > # wait for user input to end > > > # input("\n Press Enter to exit..."); > > > # close the SSLSocket, will also close the underlying socket > > > ssl_sock.close() > > > The error I am getting is here: > > > > > > line 53 except socket.error as e ^ SyntaxError: invalid syntax > > > > > > I tried changing socket.error to ConnectionRefusedError. and still got > > the same error. > > > > > > Please tell me if the problem is with Rstudio, Python version or the > > syntax. > > > > > > TIA > > > -Shaunak > > > -- > > > https://mail.python.org/mailman/listinfo/python-list > > > > Looks like you are missing a colon after 'as e'... .... as e: might do > > the trick > > > > Nathan > > Hi Nathan, > > Tried colon and a comma as well. Both did not work. Of course I am new to Python. I am wondering if it has to do anything with the indentation. Putting my most recent code here, again: try : # read line from file: print(file.readline()) # parse parse_json(file.readline()) count = count - 1 except socket.error as e: print('Connection fail', e) print(traceback.format_exc()) From shaunak.bangale at gmail.com Fri Feb 5 15:22:29 2016 From: shaunak.bangale at gmail.com (shaunak.bangale at gmail.com) Date: Fri, 5 Feb 2016 12:22:29 -0800 (PST) Subject: Exception handling for socket.error in Python 3.5/RStudio In-Reply-To: References: <1067e50c-bcbd-464e-8013-754bd66fda91@googlegroups.com> Message-ID: <6e83a175-83e8-4759-88c1-076339c54d0a@googlegroups.com> On Friday, February 5, 2016 at 1:09:35 PM UTC-7, Martin A. Brown wrote: > > except socket.error as e > > >line 53 except socket.error as e ^ SyntaxError: invalid syntax > > > >I tried changing socket.error to ConnectionRefusedError. and still > >got the same error. > > >Please tell me if the problem is with Rstudio, Python version or > >the syntax. > > Syntax. > > Your code has, unfortunately, suffered a colonectomy. > > When you transplant a colon, it is more likely to function properly > again. For example: > > except socket.error as e: > > Good luck, > > -Martin > > -- > Martin A. Brown > http://linux-ip.net/ I was first running with a colon only. Later tried with a comma. But it didn't work. I got the same error. From martin at linux-ip.net Fri Feb 5 15:44:15 2016 From: martin at linux-ip.net (Martin A. Brown) Date: Fri, 5 Feb 2016 12:44:15 -0800 Subject: Exception handling for socket.error in Python 3.5/RStudio In-Reply-To: <1067e50c-bcbd-464e-8013-754bd66fda91@googlegroups.com> References: <1067e50c-bcbd-464e-8013-754bd66fda91@googlegroups.com> Message-ID: Hi there Shaunak, I saw your few replies to my (and Nathan's) quick identification of syntax error. More comments follow, here. >I am running this python script on R-studio. I have Python 3.5 installed on my system. > >count = 10 >while (count > 0): > try : > # read line from file: > print(file.readline()) > # parse > parse_json(file.readline()) > count = count - 1 > except socket.error as e > print('Connection fail', e) > print(traceback.format_exc()) > ># wait for user input to end ># input("\n Press Enter to exit..."); ># close the SSLSocket, will also close the underlying socket >ssl_sock.close() > >The error I am getting is here: > >line 53 except socket.error as e ^ SyntaxError: invalid syntax > >I tried changing socket.error to ConnectionRefusedError. and still got the same error. We were assuming that line 53 in your file is the part you pasted above. That clearly shows a syntax error (the missing colon). If, after fixing that error, you are still seeing errors, then the probable explanations are: * you are not executing the same file you are editing * there is a separate syntax error elsewhere in the file (you sent us only a fragment) Additional points: * While the word 'file' is not reserved in Python 3.x, it is in Python 2.x, so, just be careful when working with older Python versions. You could always change your variable name, but you do not need to. * When you catch the error in the above, you print the traceback information, but your loop will continue. Is that what you desired? I might suggest saving your work carefully and make sure that you are running the same code that you are working on. Then, if you are still experiencing syntax errors, study the lines that the interpreter is complaining about. And, of course, send the list an email. Best of luck, -Martin -- Martin A. Brown http://linux-ip.net/ From shaunak.bangale at gmail.com Fri Feb 5 15:50:58 2016 From: shaunak.bangale at gmail.com (Shaunak Bangale) Date: Fri, 5 Feb 2016 13:50:58 -0700 Subject: Exception handling for socket.error in Python 3.5/RStudio In-Reply-To: References: <1067e50c-bcbd-464e-8013-754bd66fda91@googlegroups.com> Message-ID: Hi Martin, Thanks for the detailed reply. I edited, saved and opened the file again. Still I am getting exactly the same error. Putting bigger chunk of code and the error again: # create socket s = socket.socket(socket.AF_INET) #create a SSL context with the recommended security settings for client sockets, including automatic certificate verification: context = ssl.create_default_context() # Alternatively, a customized context could be created: #context = ssl.SSLContext(ssl.PROTOCOL_SSLv23) #context.verify_mode = ssl.CERT_REQUIRED #context.check_hostname = True # Load a set of default CA certificates from default locations #context.load_default_certs() ssl_sock = context.wrap_socket(s, server_hostname ='firehose.flightaware.com ') print("Connecting...") ssl_sock.connect(('firehose.flightaware.com', 1501)) print("Connection succeeded") # send initialization command to server: ssl_sock.write(bytes(initiation_command, 'UTF-8')) # return a file object associated with the socket file = ssl_sock.makefile('r') # use "while True" for no limit in messages received count = 10 while (count > 0): try : # read line from file: print(file.readline()) # parse parse_json(file.readline()) count = count - 1 except socket.error as e: print('Connection fail', e) print(traceback.format_exc()) # wait for user input to end # input("\n Press Enter to exit..."); # close the SSLSocket, will also close the underlying socket ssl_sock.close() ---------- Error: except socket.error as e: ^ SyntaxError: invalid syntax TIA. On Fri, Feb 5, 2016 at 1:44 PM, Martin A. Brown wrote: > > Hi there Shaunak, > > I saw your few replies to my (and Nathan's) quick identification of > syntax error. More comments follow, here. > > >I am running this python script on R-studio. I have Python 3.5 installed > on my system. > > > >count = 10 > >while (count > 0): > > try : > > # read line from file: > > print(file.readline()) > > # parse > > parse_json(file.readline()) > > count = count - 1 > > except socket.error as e > > print('Connection fail', e) > > print(traceback.format_exc()) > > > ># wait for user input to end > ># input("\n Press Enter to exit..."); > ># close the SSLSocket, will also close the underlying socket > >ssl_sock.close() > > > >The error I am getting is here: > > > >line 53 except socket.error as e ^ SyntaxError: invalid syntax > > > >I tried changing socket.error to ConnectionRefusedError. and still got > the same error. > > We were assuming that line 53 in your file is the part you pasted > above. That clearly shows a syntax error (the missing colon). > > If, after fixing that error, you are still seeing errors, then the > probable explanations are: > > * you are not executing the same file you are editing > > * there is a separate syntax error elsewhere in the file (you sent > us only a fragment) > > Additional points: > > * While the word 'file' is not reserved in Python 3.x, it is in > Python 2.x, so, just be careful when working with older Python > versions. You could always change your variable name, but you > do not need to. > > * When you catch the error in the above, you print the traceback > information, but your loop will continue. Is that what you > desired? > > I might suggest saving your work carefully and make sure that you > are running the same code that you are working on. Then, if you > are still experiencing syntax errors, study the lines that the > interpreter is complaining about. And, of course, send the list an > email. > > Best of luck, > > -Martin > > -- > Martin A. Brown > http://linux-ip.net/ > From shaunak.bangale at gmail.com Fri Feb 5 16:00:26 2016 From: shaunak.bangale at gmail.com (Shaunak Bangale) Date: Fri, 5 Feb 2016 14:00:26 -0700 Subject: Exception handling for socket.error in Python 3.5/RStudio In-Reply-To: References: <1067e50c-bcbd-464e-8013-754bd66fda91@googlegroups.com> Message-ID: Hi Martin, Answering your questions below: On Fri, Feb 5, 2016 at 1:50 PM, Shaunak Bangale wrote: > Hi Martin, > > Thanks for the detailed reply. I edited, saved and opened the file again. > Still I am getting exactly the same error. > > Putting bigger chunk of code and the error again: > > > > # create socket > s = socket.socket(socket.AF_INET) > #create a SSL context with the recommended security settings for client > sockets, including automatic certificate verification: > context = ssl.create_default_context() > # Alternatively, a customized context could be created: > #context = ssl.SSLContext(ssl.PROTOCOL_SSLv23) > #context.verify_mode = ssl.CERT_REQUIRED > #context.check_hostname = True > # Load a set of default CA certificates from default locations > #context.load_default_certs() > > ssl_sock = context.wrap_socket(s, server_hostname =' > firehose.flightaware.com') > print("Connecting...") > ssl_sock.connect(('firehose.flightaware.com', 1501)) > print("Connection succeeded") > > # send initialization command to server: > ssl_sock.write(bytes(initiation_command, 'UTF-8')) > # return a file object associated with the socket > file = ssl_sock.makefile('r') > # use "while True" for no limit in messages received > count = 10 > while (count > 0): > try : > # read line from file: > print(file.readline()) > # parse > parse_json(file.readline()) > count = count - 1 > except socket.error as e: > print('Connection fail', e) > print(traceback.format_exc()) > > > # wait for user input to end > # input("\n Press Enter to exit..."); > # close the SSLSocket, will also close the underlying socket > ssl_sock.close() > > ---------- > > Error: > except socket.error as e: > ^ > SyntaxError: invalid syntax > > > TIA. > > > > > > On Fri, Feb 5, 2016 at 1:44 PM, Martin A. Brown > wrote: > >> >> Hi there Shaunak, >> >> I saw your few replies to my (and Nathan's) quick identification of >> syntax error. More comments follow, here. >> >> >I am running this python script on R-studio. I have Python 3.5 installed >> on my system. >> > >> >count = 10 >> >while (count > 0): >> > try : >> > # read line from file: >> > print(file.readline()) >> > # parse >> > parse_json(file.readline()) >> > count = count - 1 >> > except socket.error as e >> > print('Connection fail', e) >> > print(traceback.format_exc()) >> > >> ># wait for user input to end >> ># input("\n Press Enter to exit..."); >> ># close the SSLSocket, will also close the underlying socket >> >ssl_sock.close() >> > >> >The error I am getting is here: >> > >> >line 53 except socket.error as e ^ SyntaxError: invalid syntax >> > >> >I tried changing socket.error to ConnectionRefusedError. and still got >> the same error. >> >> We were assuming that line 53 in your file is the part you pasted >> above. That clearly shows a syntax error (the missing colon). >> >> If, after fixing that error, you are still seeing errors, then the >> probable explanations are: >> >> * you are not executing the same file you are editing >> >> * there is a separate syntax error elsewhere in the file (you sent >> us only a fragment) >> >> Additional points: >> >> * While the word 'file' is not reserved in Python 3.x, it is in >> Python 2.x, so, just be careful when working with older Python >> versions. You could always change your variable name, but you >> do not need to. >> >> But according to FlighAware, this code is supposed to work on the Python 3.X and I have Python 3.5 on my computer and I am hoping the same is being used by Rstudio. > * When you catch the error in the above, you print the traceback >> information, but your loop will continue. Is that what you >> desired? >> >> Yes, I want the loop to run 10 times. > I might suggest saving your work carefully and make sure that you >> are running the same code that you are working on. Then, if you >> are still experiencing syntax errors, study the lines that the >> interpreter is complaining about. And, of course, send the list an >> email. >> >> The same code is supposedly running on a mac machine- Rstudio. I am not sure if the issue is also with Windows 7- Rstudio- Python 3.5 combo. > Best of luck, >> >> -Martin >> >> -- >> Martin A. Brown >> http://linux-ip.net/ >> > > From rosuav at gmail.com Fri Feb 5 16:01:17 2016 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 6 Feb 2016 08:01:17 +1100 Subject: Exception handling for socket.error in Python 3.5/RStudio In-Reply-To: <1067e50c-bcbd-464e-8013-754bd66fda91@googlegroups.com> References: <1067e50c-bcbd-464e-8013-754bd66fda91@googlegroups.com> Message-ID: On Sat, Feb 6, 2016 at 6:58 AM, wrote: > I am running this python script on R-studio. I have Python 3.5 installed on my system. > Let's just try a quick smoke test. Run this script: import sys print(sys.version) input("Press Enter to exit...") That'll tell you a few things about how your system is set up. Most notably, if it doesn't say you're using Python 3.5, there's a problem. ChrisA From martin at linux-ip.net Fri Feb 5 16:01:22 2016 From: martin at linux-ip.net (Martin A. Brown) Date: Fri, 5 Feb 2016 13:01:22 -0800 Subject: Exception handling for socket.error in Python 3.5/RStudio In-Reply-To: References: <1067e50c-bcbd-464e-8013-754bd66fda91@googlegroups.com> Message-ID: Hi there, >Thanks for the detailed reply. I edited, saved and opened the file >again. Still I am getting exactly the same error. > >Putting bigger chunk of code and the error again: [snipped; thanks for the larger chunk] >Error: >except socket.error as e: > ^ >SyntaxError: invalid syntax I ran your code. I see this: $ python3 shaunak.bangale.py Connecting... Connection succeeded Traceback (most recent call last): File "shaunak.bangale.py", line 23, in ssl_sock.write(bytes(initiation_command, 'UTF-8')) NameError: name 'initiation_command' is not defined Strictly speaking, I don't think you are having a Python problem. * Are you absolutely certain you are (or your IDE is) executing the same code you are writing? * How would you be able to tell? Close your IDE. Run the code on the command-line. * How much time have you taken to work out what the interpreter is telling you? Good luck, -Martin -- Martin A. Brown http://linux-ip.net/ From mafagafogigante at gmail.com Fri Feb 5 16:08:40 2016 From: mafagafogigante at gmail.com (Bernardo Sulzbach) Date: Fri, 5 Feb 2016 19:08:40 -0200 Subject: Exception handling for socket.error in Python 3.5/RStudio In-Reply-To: References: <1067e50c-bcbd-464e-8013-754bd66fda91@googlegroups.com> Message-ID: <56B50F58.90901@gmail.com> On 02/05/2016 07:01 PM, Chris Angelico wrote: > On Sat, Feb 6, 2016 at 6:58 AM, wrote: >> I am running this python script on R-studio. I have Python 3.5 installed on my system. >> > > Let's just try a quick smoke test. Run this script: > > import sys > print(sys.version) > input("Press Enter to exit...") > > That'll tell you a few things about how your system is set up. Most > notably, if it doesn't say you're using Python 3.5, there's a problem. > > ChrisA > Is there? If he just got the minor version wrong it wouldn't be a problem. Unless RStudio requires 3.**5** for some reason. From lucan at NNN.it Fri Feb 5 16:09:12 2016 From: lucan at NNN.it (lucan) Date: Fri, 5 Feb 2016 22:09:12 +0100 Subject: realtime output and csv files References: Message-ID: > What do you mean? What is "datas"? What do you mean by "correct"? "datas" I mean the values for example temperature = 20.4 (so they are floating point) Index time temp 1 10:24 20.4 2 10:25 20.6 ... I wonder if this is correct "my way" to write a csv file: file.write('\n'+str(index)) f.write(str(index)+','+str(now.hour)+':'+str(now.minute)+','+str(temp)) Or if there is a better way to do that. > CSVs is essentially text separated by commas, so you likely do not need > any library to write it "Just separating with ','" should work if you > are formatting them correctly. ok. From mafagafogigante at gmail.com Fri Feb 5 16:13:09 2016 From: mafagafogigante at gmail.com (Bernardo Sulzbach) Date: Fri, 5 Feb 2016 19:13:09 -0200 Subject: realtime output and csv files In-Reply-To: References: Message-ID: <56B51065.2090104@gmail.com> On 02/05/2016 07:09 PM, lucan wrote: > >> What do you mean? What is "datas"? What do you mean by "correct"? > > "datas" I mean the values for example temperature = 20.4 (so they are > floating point) > > Index time temp > 1 10:24 20.4 > 2 10:25 20.6 > ... > > I wonder if this is correct "my way" to write a csv file: > > file.write('\n'+str(index)) > f.write(str(index)+','+str(now.hour)+':'+str(now.minute)+','+str(temp)) > > Or if there is a better way to do that. > If you got your newlines and headers (if the other program requires it) right, this should be all you need. From shaunak.bangale at gmail.com Fri Feb 5 16:15:50 2016 From: shaunak.bangale at gmail.com (Shaunak Bangale) Date: Fri, 5 Feb 2016 14:15:50 -0700 Subject: Exception handling for socket.error in Python 3.5/RStudio In-Reply-To: References: <1067e50c-bcbd-464e-8013-754bd66fda91@googlegroups.com> Message-ID: I do have the initiation command defined. Just that I am not allowed to make the username, pwd public. I am absolutely sure I am running the same code. Now opened the same file with Python 3.5 shell and I get following error: from _ssl import RAND_status, RAND_egd, RAND_add ImportError: cannot import name 'RAND_egd' I am new to coding and this code has been borrowed from an online source but I can see same code working on mac+Rstudio+python combo. Salute your patience. Sincerely, On Fri, Feb 5, 2016 at 2:01 PM, Martin A. Brown wrote: > > Hi there, > > >Thanks for the detailed reply. I edited, saved and opened the file > >again. Still I am getting exactly the same error. > > > >Putting bigger chunk of code and the error again: > > [snipped; thanks for the larger chunk] > > >Error: > >except socket.error as e: > > ^ > >SyntaxError: invalid syntax > > I ran your code. I see this: > > $ python3 shaunak.bangale.py > Connecting... > Connection succeeded > Traceback (most recent call last): > File "shaunak.bangale.py", line 23, in > ssl_sock.write(bytes(initiation_command, 'UTF-8')) > NameError: name 'initiation_command' is not defined > > Strictly speaking, I don't think you are having a Python problem. > > * Are you absolutely certain you are (or your IDE is) executing > the same code you are writing? > > * How would you be able to tell? Close your IDE. Run the code on > the command-line. > > * How much time have you taken to work out what the interpreter is > telling you? > > Good luck, > > -Martin > > -- > Martin A. Brown > http://linux-ip.net/ > > From shaunak.bangale at gmail.com Fri Feb 5 16:26:14 2016 From: shaunak.bangale at gmail.com (shaunak.bangale at gmail.com) Date: Fri, 5 Feb 2016 13:26:14 -0800 (PST) Subject: Exception handling for socket.error in Python 3.5/RStudio In-Reply-To: References: <1067e50c-bcbd-464e-8013-754bd66fda91@googlegroups.com> Message-ID: On Friday, February 5, 2016 at 2:09:11 PM UTC-7, Bernardo Sulzbach wrote: > On 02/05/2016 07:01 PM, Chris Angelico wrote: > > On Sat, Feb 6, 2016 at 6:58 AM, wrote: > >> I am running this python script on R-studio. I have Python 3.5 installed on my system. > >> > > > > Let's just try a quick smoke test. Run this script: > > > > import sys > > print(sys.version) > > input("Press Enter to exit...") > > > > That'll tell you a few things about how your system is set up. Most > > notably, if it doesn't say you're using Python 3.5, there's a problem. > > > > ChrisA > > > > Is there? If he just got the minor version wrong it wouldn't be a > problem. Unless RStudio requires 3.**5** for some reason. Hi Chris, Output: 3.5.0 (v3.5.0:374f501f4567, Sep 13 2015, 02:16:59) [MSC v.1900 32 bit (Intel)] Hi Martin, I do have the initiation command defined. Just that I am not allowed to make the username, pwd public. I am absolutely sure I am running the same code. Now opened the same file with Python 3.5 shell and I get following error: from _ssl import RAND_status, RAND_egd, RAND_add ImportError: cannot import name 'RAND_egd' I am new to coding and this code has been borrowed from an online source but I can see same code working on mac+Rstudio+python combo. Salute your patience. Sincerely, From mafagafogigante at gmail.com Fri Feb 5 16:37:19 2016 From: mafagafogigante at gmail.com (Bernardo Sulzbach) Date: Fri, 5 Feb 2016 19:37:19 -0200 Subject: Exception handling for socket.error in Python 3.5/RStudio In-Reply-To: References: <1067e50c-bcbd-464e-8013-754bd66fda91@googlegroups.com> Message-ID: <56B5160F.1030900@gmail.com> On 02/05/2016 07:26 PM, shaunak.bangale at gmail.com wrote: > > from _ssl import RAND_status, RAND_egd, RAND_add > ImportError: cannot import name 'RAND_egd' > I believe I've already seen this issue myself. It has to do with LibreSSL not having RAND_egd for some reason I can't recall. This seems to be the related ticket https://bugs.python.org/issue21356 On 02/05/2016 07:26 PM, shaunak.bangale at gmail.com wrote: > I am new to coding and this code has been borrowed from an online > source but I can see same code working on mac+Rstudio+python combo. Are you sure that the Python you talk about is 3.5? From joel.goldstick at gmail.com Fri Feb 5 16:43:15 2016 From: joel.goldstick at gmail.com (Joel Goldstick) Date: Fri, 5 Feb 2016 16:43:15 -0500 Subject: realtime output and csv files In-Reply-To: <56B51065.2090104@gmail.com> References: <56B51065.2090104@gmail.com> Message-ID: On Fri, Feb 5, 2016 at 4:13 PM, Bernardo Sulzbach wrote: > On 02/05/2016 07:09 PM, lucan wrote: > >> >> What do you mean? What is "datas"? What do you mean by "correct"? >>> >> >> "datas" I mean the values for example temperature = 20.4 (so they are >> floating point) >> >> Index time temp >> 1 10:24 20.4 >> 2 10:25 20.6 >> ... >> >> I wonder if this is correct "my way" to write a csv file: >> >> file.write('\n'+str(index)) >> f.write(str(index)+','+str(now.hour)+':'+str(now.minute)+','+str(temp)) >> > You might want to do this: f.write("%d, %2d:%2d, %.1f" % (index, now.hour, now.minute, temp)) In my test: >>> print("%d, %2d:%2d, %.1f" % (1,10,24,20.4)) 1, 10:24, 20.4 This uses the original python formatting method. There is a newer one as well > >> Or if there is a better way to do that. >> >> > If you got your newlines and headers (if the other program requires it) > right, this should be all you need. > -- > https://mail.python.org/mailman/listinfo/python-list > -- Joel Goldstick http://joelgoldstick.com/stats/birthdays From mafagafogigante at gmail.com Fri Feb 5 16:51:19 2016 From: mafagafogigante at gmail.com (Bernardo Sulzbach) Date: Fri, 5 Feb 2016 19:51:19 -0200 Subject: realtime output and csv files In-Reply-To: References: <56B51065.2090104@gmail.com> Message-ID: <56B51957.2030900@gmail.com> On 02/05/2016 07:43 PM, Joel Goldstick wrote: >>>> print("%d, %2d:%2d, %.1f" % (1,10,24,20.4)) > 1, 10:24, 20.4 Let us be more careful there. Although CSV has no formal specification (according to the IETF), *those spaces are not good*. It is **very unlikely** that they will cause issues, but 1,10:24,20.4 is *safer* and - to my eyes - better CSV. I wouldn't be surprised if a parser treated a value as text only because it has spaces on it. For OP, if you are going for this, I - personally - suggest sticking to "%d,%2d:%2d,%.1f". From shaunak.bangale at gmail.com Fri Feb 5 16:57:17 2016 From: shaunak.bangale at gmail.com (shaunak.bangale at gmail.com) Date: Fri, 5 Feb 2016 13:57:17 -0800 (PST) Subject: Exception handling for socket.error in Python 3.5/RStudio In-Reply-To: <1067e50c-bcbd-464e-8013-754bd66fda91@googlegroups.com> References: <1067e50c-bcbd-464e-8013-754bd66fda91@googlegroups.com> Message-ID: On Friday, February 5, 2016 at 12:58:37 PM UTC-7, shaunak... at gmail.com wrote: > I am running this python script on R-studio. I have Python 3.5 installed on my system. > > count = 10 > while (count > 0): > try : > # read line from file: > print(file.readline()) > # parse > parse_json(file.readline()) > count = count - 1 > except socket.error as e > print('Connection fail', e) > print(traceback.format_exc()) > > # wait for user input to end > # input("\n Press Enter to exit..."); > # close the SSLSocket, will also close the underlying socket > ssl_sock.close() > The error I am getting is here: > > line 53 except socket.error as e ^ SyntaxError: invalid syntax > > I tried changing socket.error to ConnectionRefusedError. and still got the same error. > > Please tell me if the problem is with Rstudio, Python version or the syntax. > > TIA > -Shaunak It is 3.5. I am looking at the link. From rosuav at gmail.com Fri Feb 5 16:57:53 2016 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 6 Feb 2016 08:57:53 +1100 Subject: Exception handling for socket.error in Python 3.5/RStudio In-Reply-To: <56B50F58.90901@gmail.com> References: <1067e50c-bcbd-464e-8013-754bd66fda91@googlegroups.com> <56B50F58.90901@gmail.com> Message-ID: On Sat, Feb 6, 2016 at 8:08 AM, Bernardo Sulzbach wrote: > On 02/05/2016 07:01 PM, Chris Angelico wrote: >> >> On Sat, Feb 6, 2016 at 6:58 AM, wrote: >>> >>> I am running this python script on R-studio. I have Python 3.5 installed >>> on my system. >>> >> >> Let's just try a quick smoke test. Run this script: >> >> import sys >> print(sys.version) >> input("Press Enter to exit...") >> >> That'll tell you a few things about how your system is set up. Most >> notably, if it doesn't say you're using Python 3.5, there's a problem. >> >> ChrisA >> > > Is there? If he just got the minor version wrong it wouldn't be a problem. > Unless RStudio requires 3.**5** for some reason. Stuff might work, but if you think you're using 3.5 and you're actually running under 3.4, there's a high probability that you have two Python installations and it's picking the wrong one. That, in turn, means problems with package installations and such, so it's a reasonable smoke test to try. (It's unlikely there'll be two different 3.5s installed.) And, of course, if it shows up that it's running under *2*.5, well, that would explain a lot :) Anyway, it is indeed what's expected - smoke test passed. ChrisA From joel.goldstick at gmail.com Fri Feb 5 17:12:36 2016 From: joel.goldstick at gmail.com (Joel Goldstick) Date: Fri, 5 Feb 2016 17:12:36 -0500 Subject: realtime output and csv files In-Reply-To: <56B51957.2030900@gmail.com> References: <56B51065.2090104@gmail.com> <56B51957.2030900@gmail.com> Message-ID: On Fri, Feb 5, 2016 at 4:51 PM, Bernardo Sulzbach wrote: > On 02/05/2016 07:43 PM, Joel Goldstick wrote: > >> print("%d, %2d:%2d, %.1f" % (1,10,24,20.4)) >>>>> >>>> 1, 10:24, 20.4 >> > > Let us be more careful there. Although CSV has no formal specification > (according to the IETF), *those spaces are not good*. > > It is **very unlikely** that they will cause issues, but 1,10:24,20.4 is > *safer* and - to my eyes - better CSV. > > I wouldn't be surprised if a parser treated a value as text only because > it has spaces on it. > > For OP, if you are going for this, I - personally - suggest sticking to > "%d,%2d:%2d,%.1f". Good point! > > -- > https://mail.python.org/mailman/listinfo/python-list > -- Joel Goldstick http://joelgoldstick.com/stats/birthdays From shaunak.bangale at gmail.com Fri Feb 5 17:35:06 2016 From: shaunak.bangale at gmail.com (shaunak.bangale at gmail.com) Date: Fri, 5 Feb 2016 14:35:06 -0800 (PST) Subject: Exception handling for socket.error in Python 3.5/RStudio In-Reply-To: <1067e50c-bcbd-464e-8013-754bd66fda91@googlegroups.com> References: <1067e50c-bcbd-464e-8013-754bd66fda91@googlegroups.com> Message-ID: On Friday, February 5, 2016 at 12:58:37 PM UTC-7, shaunak... at gmail.com wrote: > I am running this python script on R-studio. I have Python 3.5 installed on my system. > > count = 10 > while (count > 0): > try : > # read line from file: > print(file.readline()) > # parse > parse_json(file.readline()) > count = count - 1 > except socket.error as e > print('Connection fail', e) > print(traceback.format_exc()) > > # wait for user input to end > # input("\n Press Enter to exit..."); > # close the SSLSocket, will also close the underlying socket > ssl_sock.close() > The error I am getting is here: > > line 53 except socket.error as e ^ SyntaxError: invalid syntax > > I tried changing socket.error to ConnectionRefusedError. and still got the same error. > > Please tell me if the problem is with Rstudio, Python version or the syntax. > > TIA > -Shaunak Chris, That sounds legitimate. But I never installed second 3.X version and deleted previous 2.X version and using only 3.5 now. What will be the next test according to you? -Shaun From ben+python at benfinney.id.au Fri Feb 5 18:10:46 2016 From: ben+python at benfinney.id.au (Ben Finney) Date: Sat, 06 Feb 2016 10:10:46 +1100 Subject: Daemon strategy References: <85d1sb9eh7.fsf@benfinney.id.au> Message-ID: <858u2yah3d.fsf@benfinney.id.au> paul.hermeneutic at gmail.com writes: > On Fri, Feb 5, 2016 at 11:52 AM, Ben Finney wrote: > > Since MS Windows lacks those facilities, ?python-daemon? can't use > > them. > > As you might imagine, I am not always able to specify which OS is > deployed. That does not mean that I am not responsible for getting the > work done. Perhaps you will tell me that what I want is not a daemon. I'm telling you none of those. What I'm telling you is MS Windows does not support what is needed to make a Unix daemon. You may need to re-visit the requirements and negotiate something different ? a different deployment platform, or something which MS Windows can do which is less than a proper Unix daemon. > BTW, I thought that pip would know that python-daemon would not run on > my machine, but it had no complaints installing it. That gave me > unmerited hope. Sorry to learn that. The PyPI metadata for ?python-daemon? explicitly declares the supported OS limited to ?Operating System :: POSIX?, which MS Windows is not compatible with. If ?pip? does not honour that field and inform you of the incompatibility, you may want to report a bug to the ?pip? developers. -- \ ?My business is to teach my aspirations to conform themselves | `\ to fact, not to try and make facts harmonise with my | _o__) aspirations.? ?Thomas Henry Huxley, 1860-09-23 | Ben Finney From emile at fenx.com Fri Feb 5 18:54:45 2016 From: emile at fenx.com (Emile van Sebille) Date: Fri, 5 Feb 2016 15:54:45 -0800 Subject: How to resize an animated gif to fit the window In-Reply-To: <3216251b-bab1-4946-9d9a-17eb13e76726@googlegroups.com> References: <3216251b-bab1-4946-9d9a-17eb13e76726@googlegroups.com> Message-ID: Googling that finds https://cloud.google.com/appengine/docs/python/images/ which may be of some help. Emile On 1/29/2016 5:50 PM, kwekey at gmail.com wrote: > Hi, I am able to display animated gif using pyglet using below code, but I would like the image to stretch and fit the window as i resize the window. > Any one can help me? > > import pyglet > # pick an animated gif file you have in the working directory > > CODE: SELECT ALL > def animate(str): > ag_file = str > animation = pyglet.resource.animation(ag_file) > sprite = pyglet.sprite.Sprite(animation) > # create a window and set it to the image size > win = pyglet.window.Window(width=sprite.width, height=sprite.height) > > @win.event > def on_draw(): > win.clear() > sprite.draw() > > pyglet.app.run() > > animate("***.gif") > > > CODE: SELECT ALL > win = pyglet.window.Window(width=sprite.width, height=sprite.height) > > Above line will resize the window to fit the animated gif but i want it the other way, that is the gif to fit the windows size.. > From rosuav at gmail.com Fri Feb 5 21:04:19 2016 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 6 Feb 2016 13:04:19 +1100 Subject: Exception handling for socket.error in Python 3.5/RStudio In-Reply-To: References: <1067e50c-bcbd-464e-8013-754bd66fda91@googlegroups.com> Message-ID: On Sat, Feb 6, 2016 at 9:35 AM, wrote: > On Friday, February 5, 2016 at 12:58:37 PM UTC-7, shaunak... at gmail.com wrote: >> I am running this python script on R-studio. I have Python 3.5 installed on my system. >> >> count = 10 >> while (count > 0): >> try : >> # read line from file: >> print(file.readline()) >> # parse >> parse_json(file.readline()) >> count = count - 1 >> except socket.error as e >> print('Connection fail', e) >> print(traceback.format_exc()) >> >> # wait for user input to end >> # input("\n Press Enter to exit..."); >> # close the SSLSocket, will also close the underlying socket >> ssl_sock.close() >> The error I am getting is here: >> >> line 53 except socket.error as e ^ SyntaxError: invalid syntax >> >> I tried changing socket.error to ConnectionRefusedError. and still got the same error. >> >> Please tell me if the problem is with Rstudio, Python version or the syntax. >> >> TIA >> -Shaunak > > > Chris, > > That sounds legitimate. > But I never installed second 3.X version and deleted previous 2.X version and using only 3.5 now. What will be the next test according to you? (Please quote the person you're replying to, rather than simply quoting your own original post - it helps to provide context. Thanks!) The next thing to try would be to add the colon at the end of the "except" clause, as has already been suggested, and then to post the latest code (preferably all of it - if it looks like too much, it's worth shortening the code, rather than posting a snippet) and the full traceback. ChrisA From python.list at tim.thechases.com Fri Feb 5 23:27:15 2016 From: python.list at tim.thechases.com (Tim Chase) Date: Fri, 5 Feb 2016 22:27:15 -0600 Subject: realtime output and csv files In-Reply-To: <56B4FEAC.4080706@gmail.com> References: <56B4FEAC.4080706@gmail.com> Message-ID: <20160205222715.06320d38@bigbox.christie.dr> On 2016-02-05 17:57, Bernardo Sulzbach wrote: > CSVs is essentially text separated by commas, so you likely do not > need any library to write it "Just separating with ','" should work > if you are formatting them correctly. > https://mail.python.org/mailman/listinfo/python-list And even if you have things to escape or format correctly, the stdlib has a "csv" module that makes this trivially easy: data = [ (1, 3.14, "hello"), (2, 1.41, "goodbye"), ] from csv import writer with open("out.csv", "wb") as f: w = writer(f) w.writerows(data) # for row in data: # w.writerow(data) -tkc From mafagafogigante at gmail.com Fri Feb 5 23:53:13 2016 From: mafagafogigante at gmail.com (Bernardo Sulzbach) Date: Sat, 6 Feb 2016 02:53:13 -0200 Subject: realtime output and csv files In-Reply-To: <20160205222715.06320d38@bigbox.christie.dr> References: <56B4FEAC.4080706@gmail.com> <20160205222715.06320d38@bigbox.christie.dr> Message-ID: On Sat, Feb 6, 2016 at 2:27 AM, Tim Chase wrote: > On 2016-02-05 17:57, Bernardo Sulzbach wrote: >> CSVs is essentially text separated by commas, so you likely do not >> need any library to write it "Just separating with ','" should work >> if you are formatting them correctly. > >> https://mail.python.org/mailman/listinfo/python-list > And even if you have things to escape or format correctly, the stdlib > has a "csv" module that makes this trivially easy: > I supposed it had one. Obviously, I've never used it myself, otherwise I would be sure about its existence. Nice to know about it, although I assume that for many data transfers it will not be painless if you need several escapes, when CSV starts to become complicated it starts to fail because of quirky and naive parsers. From ian.g.kelly at gmail.com Sat Feb 6 02:16:40 2016 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Sat, 6 Feb 2016 00:16:40 -0700 Subject: Daemon strategy In-Reply-To: <858u2yah3d.fsf@benfinney.id.au> References: <85d1sb9eh7.fsf@benfinney.id.au> <858u2yah3d.fsf@benfinney.id.au> Message-ID: On Fri, Feb 5, 2016 at 4:10 PM, Ben Finney wrote: > Sorry to learn that. The PyPI metadata for ?python-daemon? > explicitly declares > the supported OS limited to ?Operating System :: POSIX?, which MS > Windows is not compatible with. Depends on the version: https://en.wikipedia.org/wiki/Windows_Services_for_UNIX Unfortunately that's no longer available for modern Windows. But it's not the only option either: https://en.wikipedia.org/wiki/POSIX#POSIX_for_Windows > If ?pip? does not honour that field and inform you of the > incompatibility, you may want to report a bug to the ?pip? developers. Linux and FreeBSD are also not POSIX-certified, even though they mostly comply. Should pip warn about those also? From frank at chagford.com Sat Feb 6 02:55:26 2016 From: frank at chagford.com (Frank Millman) Date: Sat, 6 Feb 2016 09:55:26 +0200 Subject: asyncio - how to stop background task cleanly Message-ID: Hi all It is easy enough to set up a task to run in the background every 10 seconds using asyncio - async def background_task(): while True: await perform_task() await asyncio.sleep(10) asyncio.ensure_future(background_task()) When shutting the main program down, I want to stop the task, but I cannot figure out how to stop it cleanly - i.e. wait until it has finished the current task and possibly performed some cleanup, before continuing. async def background_task(): await perform_setup() while condition: await perform_task() await asyncio.sleep(10) await perform_cleanup() Previously I would run the task in another thread, then set a flag to tell it to stop, then join() the thread which would block until the task had finished. I used threading.Event as the flag, which allows it to 'sleep' using wait() with a timeout value, but reacts instantly when set() is called, so it was ideal. Is there a way to achieve this using asyncio? Thanks Frank Millman From ben+python at benfinney.id.au Sat Feb 6 03:15:14 2016 From: ben+python at benfinney.id.au (Ben Finney) Date: Sat, 06 Feb 2016 19:15:14 +1100 Subject: Daemon strategy References: <85d1sb9eh7.fsf@benfinney.id.au> <858u2yah3d.fsf@benfinney.id.au> Message-ID: <854mdm9rvx.fsf@benfinney.id.au> Ian Kelly writes: > Depends on the version: https://en.wikipedia.org/wiki/Windows_Services_for_UNIX > https://en.wikipedia.org/wiki/POSIX#POSIX_for_Windows > > Linux and FreeBSD are also not POSIX-certified, even though they > mostly comply. Should pip warn about those also? You're implying that the PyPI trove category ?Operating System :: POSIX? includes MS Windows? Or that it excludes Linux and FreeBSD? Or that it's meaningless? -- \ ?Drop your trousers here for best results.? ?dry cleaner, | `\ Bangkok | _o__) | Ben Finney From marko at pacujo.net Sat Feb 6 03:39:37 2016 From: marko at pacujo.net (Marko Rauhamaa) Date: Sat, 06 Feb 2016 10:39:37 +0200 Subject: asyncio - how to stop background task cleanly References: Message-ID: <87lh6ys052.fsf@elektro.pacujo.net> "Frank Millman" : > When shutting the main program down, I want to stop the task, but I > cannot figure out how to stop it cleanly - i.e. wait until it has > finished the current task and possibly performed some cleanup, before > continuing. Here (and really, only here) is where asyncio shows its superiority over threads: you can multiplex. You should await asyncio.wait(..., return_when=asyncio.FIRST_COMPLETED) to deal with multiple alternative stimuli. In fact, since there is always a minimum of two alternative stimuli to await, you should only ever await asyncio.wait(). And, while viable, that's what makes every asyncio program ugly as hell. Marko From python.list at tim.thechases.com Sat Feb 6 07:45:53 2016 From: python.list at tim.thechases.com (Tim Chase) Date: Sat, 6 Feb 2016 06:45:53 -0600 Subject: realtime output and csv files In-Reply-To: References: <56B4FEAC.4080706@gmail.com> <20160205222715.06320d38@bigbox.christie.dr> Message-ID: <20160206064553.7fd622a5@bigbox.christie.dr> On 2016-02-06 02:53, Bernardo Sulzbach wrote: >> And even if you have things to escape or format correctly, the >> stdlib has a "csv" module that makes this trivially easy: >> > > I supposed it had one. Obviously, I've never used it myself, > otherwise I would be sure about its existence. Nice to know about > it, although I assume that for many data transfers it will not be > painless if you need several escapes, when CSV starts to become > complicated it starts to fail because of quirky and naive parsers. If you read up on the csv module's help, it has all manner of knobs to twiddle when it comes to the "dialect" of CSV: - delimiters: comma is the default, but you can specify others like tab or pipe) - quote character: double-quote is the default, but you can use single-quote or otherwise - escaping: doubling up the quote-character is the default but I think you can specify other schemes There are others, but that's what I have to tweak most often. Finally, there's a sniffer in the module, so even if you don't know the format, you can make an intelligent guess about it: with open('example.csv', 'rb') as csvfile: dialect = csv.Sniffer().sniff(csvfile.read(1024*4)) csvfile.seek(0) reader = csv.reader(csvfile, dialect) The only edge case I've hit regularly is that it doesn't (didn't? not sure if it's been fixed) do the right thing on CSV files containing a single column, as it would try to accept a common alphanumeric character as the delimiter instead of making the better assumption that it was a single column. -tkc From lucan at NNN.it Sat Feb 6 08:40:10 2016 From: lucan at NNN.it (lucan) Date: Sat, 6 Feb 2016 14:40:10 +0100 Subject: realtime output and csv files References: <56B51065.2090104@gmail.com> Message-ID: > I wouldn't be surprised if a parser treated a value as text only because > it has spaces on it. > > For OP, if you are going for this, I - personally - suggest sticking to > "%d,%2d:%2d,%.1f". you're rightin fact importing datas in spreadsheet I've had some problems. I'll follow this suggestion. tnks a lot! :) From jillzoe at hotmail.co.uk Sat Feb 6 08:40:35 2016 From: jillzoe at hotmail.co.uk (jill davies) Date: Sat, 6 Feb 2016 13:40:35 +0000 Subject: =?utf-8?Q?python?= Message-ID: I would like to know how to open your python launcher to start coding on windows 8 the python version I have installed is python 3.5.1 but it keeps giving me an instillation set up Sent from Windows Mail From frank at chagford.com Sat Feb 6 09:01:49 2016 From: frank at chagford.com (Frank Millman) Date: Sat, 6 Feb 2016 16:01:49 +0200 Subject: asyncio - how to stop background task cleanly In-Reply-To: <87lh6ys052.fsf@elektro.pacujo.net> References: <87lh6ys052.fsf@elektro.pacujo.net> Message-ID: "Marko Rauhamaa" wrote in message news:87lh6ys052.fsf at elektro.pacujo.net... > > "Frank Millman" : > > > When shutting the main program down, I want to stop the task, but I > > cannot figure out how to stop it cleanly - i.e. wait until it has > > finished the current task and possibly performed some cleanup, before > > continuing. > > Here (and really, only here) is where asyncio shows its superiority over > threads: you can multiplex. > > You should > > await asyncio.wait(..., return_when=asyncio.FIRST_COMPLETED) > > to deal with multiple alternative stimuli. > Thanks, Marko, that works very well. It took me a while to get it working, because I initiate shutting down the program from another thread. Eventually I figured out that I could put all my event loop shutdown procedures into a coroutine, and then call asyncio.run_coroutine_threadsafe() from the main thread. Now I just have one problem left. I will keep experimenting, but if someone gives me a hint in the meantime it will be appreciated. I run my background task like this - stop_task = False async def background_task(): while not stop_task: await perform_task() await asyncio.sleep(10) I stop the task by setting stop_task to True. It works, but it waits for the 10-second sleep to expire before it is actioned. With threading, I could set up a threading.Event(), call evt.wait(timeout=10) to run the loop, and evt.set() to stop it. It stopped instantly. Is there an equivalent in asyncio? Thanks Frank From breamoreboy at yahoo.co.uk Sat Feb 6 09:37:11 2016 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Sat, 6 Feb 2016 14:37:11 +0000 Subject: python In-Reply-To: References: Message-ID: On 06/02/2016 13:40, jill davies wrote: > > I would like to know how to open your python launcher to start coding on windows 8 the python version I have installed is python 3.5.1 but it keeps giving me an instillation set up > hello and welcome. Your question has been asked and answered repeatedly over the last few months so please search the archives for the solution. You might also like to consider giving a rather more specific subject for your next question. -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From mr.eightnoteight at gmail.com Sat Feb 6 09:46:36 2016 From: mr.eightnoteight at gmail.com (srinivas devaki) Date: Sat, 6 Feb 2016 20:16:36 +0530 Subject: _siftup and _siftdown implementation In-Reply-To: References: <56b3e902$0$1613$c3e8da3$5496439d@news.astraweb.com> <56B4CD7B.70506@mail.de> Message-ID: As the comments in the heapq module says, in most of the cases (probability 0.837 [from knuth vol3]) the the new inserted element whose initial location is end of the array, which means that it will be larger than the `pos` children. So the old version and new version code is same in these cases because in old version the comparison is taking place at line 129, in the new version comparison is taking place inside _siftdown. SO the total number of comparisons are not decreased by those cases the other type case is the swapped element is smaller than `pos` parents, in this case the old version does a comparison and then goto _sifdown but in the new version _siftdown will be executed and then it comes to _siftup, here in 50% of the cases(len(heap last level) == len(heap) // 2) the pos doesn't have a child, so no comparison. ================================================================================================ `pos` | probability | `old version comparisons` | `new version comparisions` ================================================================================================ last level | 0.5 | 1 | 0 last - 1 level | 0.25 | 1 | 1 lower levels | 0.25 | 1 | >=1 ================================================================================================ as the new version doesn't do a comparison if it is in the last level, the optimization is occurring in that place. Which makes the reason behind the heapq module's choice of _siftup code is not at all related to this cause. PS: please copy the table to some text editor, for better visualization. On Fri, Feb 5, 2016 at 11:12 PM, srinivas devaki wrote: > wow, that's great. > you read a comment in the code, and you test it, to only find that it > is indeed true, > sounds ok, but feels great. :) > > Just experimenting a bit, I swaped the lines _siftdown and _siftup and something > strange happened the number of comparisions in both the versions remained same. > I'm attaching the files. > > do you have any idea why this happened? > > On Fri, Feb 5, 2016 at 9:57 PM, Sven R. Kunze wrote: >> >> Can we do better here? >> > I don't know, I have to read TAOP knuth article. > > -- > Regards > Srinivas Devaki > Junior (3rd yr) student at Indian School of Mines,(IIT Dhanbad) > Computer Science and Engineering Department > ph: +91 9491 383 249 > telegram_id: @eightnoteight -- Regards Srinivas Devaki Junior (3rd yr) student at Indian School of Mines,(IIT Dhanbad) Computer Science and Engineering Department ph: +91 9491 383 249 telegram_id: @eightnoteight From joel.goldstick at gmail.com Sat Feb 6 10:12:07 2016 From: joel.goldstick at gmail.com (Joel Goldstick) Date: Sat, 6 Feb 2016 10:12:07 -0500 Subject: python In-Reply-To: References: Message-ID: On Sat, Feb 6, 2016 at 8:40 AM, jill davies wrote: > > > > > I would like to know how to open your python launcher to start coding on > windows 8 the python version I have installed is python 3.5.1 but it keeps > giving me an instillation set up > > > Sent from Windows Mail > -- > https://mail.python.org/mailman/listinfo/python-list Open up the command window type python You are now in the interactive python shell If you want to run a program you have saved in a file type: python my_file.py -- Joel Goldstick http://joelgoldstick.com/stats/birthdays From marko at pacujo.net Sat Feb 6 10:34:47 2016 From: marko at pacujo.net (Marko Rauhamaa) Date: Sat, 06 Feb 2016 17:34:47 +0200 Subject: asyncio - how to stop background task cleanly References: <87lh6ys052.fsf@elektro.pacujo.net> Message-ID: <87fux5svhk.fsf@elektro.pacujo.net> "Frank Millman" : > "Marko Rauhamaa" wrote in message news:87lh6ys052.fsf at elektro.pacujo.net... >> You should >> >> await asyncio.wait(..., return_when=asyncio.FIRST_COMPLETED) >> >> to deal with multiple alternative stimuli. >> > > Thanks, Marko, that works very well. > > [...] > > Now I just have one problem left. I will keep experimenting, but if > someone gives me a hint in the meantime it will be appreciated. > > I run my background task like this - > > stop_task = False > > async def background_task(): > while not stop_task: > await perform_task() > await asyncio.sleep(10) You should set up a separate asyncio.Event or asyncio.Queue to send "out-of-band" signals to your background_task: async def background_task(cancel_event): async def doze_off(): await asyncio.sleep(10) while True: await asyncio.wait( perform_task, cancel_event.wait, return_when=asyncio.FIRST_COMPETED) if cancel_event_is_set() break await asyncio.wait( doze_off, cancel_event.wait, return_when=asyncio.FIRST_COMPETED) if cancel_event_is_set() break That should be the idea, anyway. I didn't try it out. > I stop the task by setting stop_task to True. It works, but it waits for > the 10-second sleep to expire before it is actioned. > > With threading, I could set up a threading.Event(), call > evt.wait(timeout=10) to run the loop, and evt.set() to stop it. It > stopped instantly. > > Is there an equivalent in asyncio? Yes, you could simplify the above thusly: async def background_task(cancel_event): while True: await asyncio.wait( perform_task, cancel_event.wait, return_when=asyncio.FIRST_COMPETED) if cancel_event_is_set() break await asyncio.wait( cancel_event.wait, timeout=10, return_when=asyncio.FIRST_COMPETED) if cancel_event_is_set() break Marko From paul.hermeneutic at gmail.com Sat Feb 6 11:04:31 2016 From: paul.hermeneutic at gmail.com (paul.hermeneutic at gmail.com) Date: Sat, 6 Feb 2016 09:04:31 -0700 Subject: Daemon strategy In-Reply-To: <858u2yah3d.fsf@benfinney.id.au> References: <85d1sb9eh7.fsf@benfinney.id.au> <858u2yah3d.fsf@benfinney.id.au> Message-ID: On Fri, Feb 5, 2016 at 4:10 PM, Ben Finney wrote: > paul.hermeneutic at gmail.com writes: > >> On Fri, Feb 5, 2016 at 11:52 AM, Ben Finney wrote: >> > Since MS Windows lacks those facilities, ?python-daemon? can't use >> > them. >> >> As you might imagine, I am not always able to specify which OS is >> deployed. That does not mean that I am not responsible for getting the >> work done. Perhaps you will tell me that what I want is not a daemon. > > I'm telling you none of those. What I'm telling you is MS Windows does > not support what is needed to make a Unix daemon. > > You may need to re-visit the requirements and negotiate something > different ? a different deployment platform, or something which MS > Windows can do which is less than a proper Unix daemon. I fully understand that Windows is not a proper UNIX. It might be that UNIX is not in each and every aspect a proper Windows. If it is possible, I would like to create one tool to do this rather than multiple. Is there anything in Python that would help to accomplish this goal? From paul.hermeneutic at gmail.com Sat Feb 6 11:13:54 2016 From: paul.hermeneutic at gmail.com (paul.hermeneutic at gmail.com) Date: Sat, 6 Feb 2016 09:13:54 -0700 Subject: Daemon strategy In-Reply-To: References: <85d1sb9eh7.fsf@benfinney.id.au> Message-ID: > As Ben already said .. either deploy to Unix systems or use > subprocess.Popen and detach the process: > > from subprocess import Popenfrom win32process import DETACHED_PROCESS > Popen(["YOURPROCESS"],creationflags=DETACHED_PROCESS,shell=True) This sounds promising. What are the common methods to communicate with the subprocess? I need to query the status and change some settings for the running process? From ian.g.kelly at gmail.com Sat Feb 6 11:28:07 2016 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Sat, 6 Feb 2016 09:28:07 -0700 Subject: Daemon strategy In-Reply-To: <854mdm9rvx.fsf@benfinney.id.au> References: <85d1sb9eh7.fsf@benfinney.id.au> <858u2yah3d.fsf@benfinney.id.au> <854mdm9rvx.fsf@benfinney.id.au> Message-ID: On Sat, Feb 6, 2016 at 1:15 AM, Ben Finney wrote: > Ian Kelly writes: > >> Depends on the version: https://en.wikipedia.org/wiki/Windows_Services_for_UNIX >> https://en.wikipedia.org/wiki/POSIX#POSIX_for_Windows >> >> Linux and FreeBSD are also not POSIX-certified, even though they >> mostly comply. Should pip warn about those also? > > You're implying that the PyPI trove category ?Operating System :: POSIX? > includes MS Windows? Or that it excludes Linux and FreeBSD? Or that it's > meaningless? Mostly the latter. At least it's not very clear as to what OSes are sufficiently compliant to be considered supported. Anything that claims POSIX support might work in Cygwin, or it might not. Who knows? From alejod.a at hotmail.com Sat Feb 6 15:10:38 2016 From: alejod.a at hotmail.com (=?iso-8859-1?Q?D=E1nisa_Andrea_Alejo_Garc=EDa?=) Date: Sat, 6 Feb 2016 20:10:38 +0000 Subject: Issues with to run python Message-ID: Dear Technician, I am writing to you because I have downloaded Python,I have followed all the instructions, but for some reason I am unable to use it. I am sending to you a screen shot, so in that way you can have an idea of what is going on. I have already repaired and modified, but all the time I am back to the same screen, that shows the same different options that can appreciate on the screen shot that I am sending to you. Those options are : Modify Repair Uninstall I will like to have some feedback and further instructions if they are need it. Thanks for your time. Danisa Andrea Alejo Northeastern University Cellphone number: 617-431-0280 From alejod.a at hotmail.com Sat Feb 6 15:13:18 2016 From: alejod.a at hotmail.com (=?iso-8859-1?Q?D=E1nisa_Andrea_Alejo_Garc=EDa?=) Date: Sat, 6 Feb 2016 20:13:18 +0000 Subject: Issues with to run python In-Reply-To: References: Message-ID: Sent from my iPhone On Feb 6, 2016, at 3:10 PM, D?nisa Andrea Alejo Garc?a > wrote: Dear Technician, I am writing to you because I have downloaded Python,I have followed all the instructions, but for some reason I am unable to use it. I am sending to you a screen shot, so in that way you can have an idea of what is going on. I have already repaired and modified, but all the time I am back to the same screen, that shows the same different options that you can appreciate on the screen shot that I am sending to you as an attachment . Those options are : Modify Repair Uninstall I will like to have some feedback and further instructions if they are need it. Thanks for your time. Danisa Andrea Alejo Northeastern University Cellphone number: 617-431-0280 From marko at pacujo.net Sat Feb 6 15:37:06 2016 From: marko at pacujo.net (Marko Rauhamaa) Date: Sat, 06 Feb 2016 22:37:06 +0200 Subject: asyncio - how to stop background task cleanly References: <87lh6ys052.fsf@elektro.pacujo.net> <87fux5svhk.fsf@elektro.pacujo.net> Message-ID: <8737t5shhp.fsf@elektro.pacujo.net> Marko Rauhamaa : > async def background_task(cancel_event): > while True: > await asyncio.wait( > perform_task, cancel_event.wait, > return_when=asyncio.FIRST_COMPETED) > if cancel_event_is_set() > break > await asyncio.wait( > cancel_event.wait, timeout=10, > return_when=asyncio.FIRST_COMPETED) > if cancel_event_is_set() > break [Typo: cancel_event_is_set() ==> cancel_event.is_set().] Actually, cancellation is specially supported in asyncio () so this should do: async def background_task(): while True: await perform_task() await asyncio.sleep(10) Marko From rantingrickjohnson at gmail.com Sat Feb 6 15:54:12 2016 From: rantingrickjohnson at gmail.com (Rick Johnson) Date: Sat, 6 Feb 2016 12:54:12 -0800 (PST) Subject: [STORY-TIME] THE BDFL AND HIS PYTHON PETTING ZOO In-Reply-To: <5d31dd14-1a60-4f80-8889-d0616f404c70@googlegroups.com> References: <48762040-a7e0-434c-92e8-8a0969210e5b@googlegroups.com> <5d31dd14-1a60-4f80-8889-d0616f404c70@googlegroups.com> Message-ID: <4e902c58-3370-4adf-905d-74630a6eea90@googlegroups.com> On Wednesday, February 3, 2016 at 12:02:35 AM UTC-6, John Ladasky wrote: > Rick, you don't like Python? If i didn't like Python, then i would happily let it self- destruct, yes? The problem is, i *DO* like Python. Python2 was a great language, but python3 has been a disaster. Heck, even the BDFL has warned that Python4 cannot introduce as many backwards incompatible changes as python3. So obviously, he is admitting that Python3 was a disaster. From robertvstepp at gmail.com Sat Feb 6 16:22:22 2016 From: robertvstepp at gmail.com (boB Stepp) Date: Sat, 6 Feb 2016 15:22:22 -0600 Subject: Issues with to run python In-Reply-To: References: Message-ID: On Sat, Feb 6, 2016 at 2:10 PM, D?nisa Andrea Alejo Garc?a wrote: > I am writing to you because I have downloaded Python,I have followed all the instructions, but for some reason I am unable to use it. This is a very common question of late. If you search the list archives, you will probably find your answer. I'll just add that if you are using Win XP, you can only use <= Python 3.4.x. Win XP won't work with >= Python 3.5.x. But if by searching the archives you cannot answer your question, then you will need to provide additional information: What is your OS? Which Python version are you trying to install? What exact steps did you take? How did you attempt to start Python? Etc. > I am sending to you a screen shot, so in that way you can have an idea of what is going on. I have already repaired and modified, but all the time I am back to the same screen, that shows the same different options that can appreciate on the screen shot that I am sending to you. Those options are : > > Modify > > Repair > > Uninstall This is a plain text only list, so your screen shot attachment is useless here. boB From ben+python at benfinney.id.au Sat Feb 6 19:07:55 2016 From: ben+python at benfinney.id.au (Ben Finney) Date: Sun, 07 Feb 2016 11:07:55 +1100 Subject: Daemon strategy References: <85d1sb9eh7.fsf@benfinney.id.au> <858u2yah3d.fsf@benfinney.id.au> <854mdm9rvx.fsf@benfinney.id.au> Message-ID: <85zivd8js4.fsf@benfinney.id.au> Ian Kelly writes: > On Sat, Feb 6, 2016 at 1:15 AM, Ben Finney wrote: > > You're implying that the PyPI trove category ?Operating System :: POSIX? > > includes MS Windows? Or that it excludes Linux and FreeBSD? Or that it's > > meaningless? > > At least it's not very clear as to what OSes are sufficiently > compliant to be considered supported. Anything that claims POSIX > support might work in Cygwin, or it might not. Who knows? That the category is not perfectly reliable doesn't argue for ignoring it altogether. I'm saying that the information is available to ?pip? for it to help the use determine whether the package will be useful to them. And that the user, in this instance, has a good case to report a bug against ?pip? requesting that it should do that. -- \ ?Faith may be defined briefly as an illogical belief in the | `\ occurrence of the improbable.? ?Henry L. Mencken | _o__) | Ben Finney From donny6983 at yahoo.com Sat Feb 6 19:34:23 2016 From: donny6983 at yahoo.com (donald alsept) Date: Sun, 7 Feb 2016 00:34:23 +0000 (UTC) Subject: trouble installing python References: <1698609935.300641.1454805263360.JavaMail.yahoo.ref@mail.yahoo.com> Message-ID: <1698609935.300641.1454805263360.JavaMail.yahoo@mail.yahoo.com> Hello, I'm trying to install the 3.5.1 of Python and am running windows 7. I keep getting an error about api-ms-win-crt-runtime-|1-1-0.dll not being installed. Any advice on what is wrong? From paul.hermeneutic at gmail.com Sat Feb 6 19:38:14 2016 From: paul.hermeneutic at gmail.com (paul.hermeneutic at gmail.com) Date: Sat, 6 Feb 2016 17:38:14 -0700 Subject: Simple tkinter query about creating a tk,StringVar() failure Message-ID: I know this may be more suited to the tutor list. I tried to subscribe, but no response yet. Why does this interactive instantiation fail when it seems to work when run in a script? (py35-64) C:\src\pygui>python Python 3.5.1 (v3.5.1:37a07cee5969, Dec 6 2015, 01:54:25) [MSC v.1900 64 bit (AMD64)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> import tkinter as tk >>> name = tk.StringVar() Traceback (most recent call last): File "", line 1, in File "C:\Program Files\Python35\lib\tkinter\__init__.py", line 329, in __init__ Variable.__init__(self, master, value, name) File "C:\Program Files\Python35\lib\tkinter\__init__.py", line 233, in __init__ self._root = master._root() AttributeError: 'NoneType' object has no attribute '_root' From python at mrabarnett.plus.com Sat Feb 6 20:52:44 2016 From: python at mrabarnett.plus.com (MRAB) Date: Sun, 07 Feb 2016 01:52:44 +0000 Subject: Simple tkinter query about creating a tk,StringVar() failure In-Reply-To: Message-ID: On 2016-02-07 00:38:14, paul.hermeneutic at gmail.com wrote: >I know this may be more suited to the tutor list. I tried to >subscribe, but no response yet. > >Why does this interactive instantiation fail when it seems to work >when run in a script? > >(py35-64) C:\src\pygui>python >Python 3.5.1 (v3.5.1:37a07cee5969, Dec 6 2015, 01:54:25) [MSC v.1900 >64 bit (AMD64)] on win32 >Type "help", "copyright", "credits" or "license" for more information. >>>> import tkinter as tk >>>> name = tk.StringVar() >Traceback (most recent call last): > File "", line 1, in > File "C:\Program Files\Python35\lib\tkinter\__init__.py", line 329, >in __init__ > Variable.__init__(self, master, value, name) > File "C:\Program Files\Python35\lib\tkinter\__init__.py", line 233, >in __init__ > self._root = master._root() >AttributeError: 'NoneType' object has no attribute '_root' It's just down to the way that tkinter works. It's normally called within a GUI program which has windows and an event loop, and it's trying to get the root (top-level) window, but there isn't one. From robertvstepp at gmail.com Sat Feb 6 21:40:16 2016 From: robertvstepp at gmail.com (boB Stepp) Date: Sat, 6 Feb 2016 20:40:16 -0600 Subject: Simple tkinter query about creating a tk,StringVar() failure In-Reply-To: References: Message-ID: On Sat, Feb 6, 2016 at 6:38 PM, wrote: > Why does this interactive instantiation fail when it seems to work > when run in a script? You have to establish your root window first: Python 3.5.1 (v3.5.1:37a07cee5969, Dec 6 2015, 01:54:25) [MSC v.1900 64 bit (AMD64)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> import tkinter as tk >>> root = tk.Tk() # This will open an empty window. >>> name = tk.StringVar() >>> -- boB From dimva13 at gmail.com Sat Feb 6 21:47:54 2016 From: dimva13 at gmail.com (dimva13 at gmail.com) Date: Sat, 6 Feb 2016 18:47:54 -0800 (PST) Subject: Python's import situation has driven me to the brink of imsanity Message-ID: No, that's not a typo, it's the name of a package I created. :) The problems I have with python's import system are detailed in the README of my package here: https://github.com/vadimg/imsanity Basically, relative imports are broken if you like running scripts as executables, so the only real solution is modifying PYTHONPATH every time you switch projects, which is also not ideal. Some people suggest adding various boilerplate to the beginning of every file to fix this, ranging from the crazy stuff in PEP 0366 to the slightly more sane sys.path.append(os.path.realpath(os.path.join(os.path.dirname(__file__), path_to_root))), where path_to_root is some number of '../'s depending on the file's location. Neither can be used indiscriminately as a macro, which is annoying. Imsanity allows you to make imports usable (not ideal, but at least usable) for python projects without having to manage PYTHONPATHs or do whacky stuff like running files with python -m or put even whackier boilerplate at the top of every file. And all it requires is 'import imsanity' at the top of every file. You can put it in a macro or even just type it because it's short and easy to remember. My question is: is this crazy? Please tell me there's a better way and I just wasted my time creating this package. There's nothing I'd like to hear more. From rosuav at gmail.com Sat Feb 6 21:52:21 2016 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 7 Feb 2016 13:52:21 +1100 Subject: Python's import situation has driven me to the brink of imsanity In-Reply-To: References: Message-ID: On Sun, Feb 7, 2016 at 1:47 PM, wrote: > Imsanity allows you to make imports usable (not ideal, but at least usable) for python projects without having to manage PYTHONPATHs or do whacky stuff like running files with python -m or put even whackier boilerplate at the top of every file. And all it requires is 'import imsanity' at the top of every file. You can put it in a macro or even just type it because it's short and easy to remember. > > My question is: is this crazy? Please tell me there's a better way and I just wasted my time creating this package. There's nothing I'd like to hear more. Well, anything that makes you type "import imsanity" at the top of every script MUST be crazy. :) I don't know about the actual content/purpose though. Good luck with it! ChrisA From songofacandy at gmail.com Sat Feb 6 22:02:19 2016 From: songofacandy at gmail.com (INADA Naoki) Date: Sun, 7 Feb 2016 12:02:19 +0900 Subject: [STORY-TIME] THE BDFL AND HIS PYTHON PETTING ZOO In-Reply-To: <4e902c58-3370-4adf-905d-74630a6eea90@googlegroups.com> References: <48762040-a7e0-434c-92e8-8a0969210e5b@googlegroups.com> <5d31dd14-1a60-4f80-8889-d0616f404c70@googlegroups.com> <4e902c58-3370-4adf-905d-74630a6eea90@googlegroups.com> Message-ID: Python 3 is a disaster because of incompatibility with Python 2. Python 3 itself is not so bad. We can help disaster recovery by stop using Python 2 as possible. Let's stop using legacy Python! On Sun, Feb 7, 2016 at 5:54 AM, Rick Johnson wrote: > On Wednesday, February 3, 2016 at 12:02:35 AM UTC-6, John Ladasky wrote: > > > Rick, you don't like Python? > > If i didn't like Python, then i would happily let it self- > destruct, yes? The problem is, i *DO* like Python. Python2 > was a great language, but python3 has been a disaster. Heck, > even the BDFL has warned that Python4 cannot introduce as > many backwards incompatible changes as python3. So obviously, > he is admitting that Python3 was a disaster. > > > -- > https://mail.python.org/mailman/listinfo/python-list > -- INADA Naoki From rosuav at gmail.com Sat Feb 6 22:06:03 2016 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 7 Feb 2016 14:06:03 +1100 Subject: [STORY-TIME] THE BDFL AND HIS PYTHON PETTING ZOO In-Reply-To: References: <48762040-a7e0-434c-92e8-8a0969210e5b@googlegroups.com> <5d31dd14-1a60-4f80-8889-d0616f404c70@googlegroups.com> <4e902c58-3370-4adf-905d-74630a6eea90@googlegroups.com> Message-ID: On Sun, Feb 7, 2016 at 2:02 PM, INADA Naoki wrote: > Python 3 is a disaster because of incompatibility with Python 2. Python 3 > itself is not so bad. > We can help disaster recovery by stop using Python 2 as possible. > > Let's stop using legacy Python! > It's not even a disaster. It's fairly easy to write 2/3 compatible code; in fact, a lot of my students end up accidentally running their code under Py2 and only notice when they get strange errors from input() with non-integers, or maybe if they use print() and get an empty tuple displayed, or something like that. ChrisA From torriem at gmail.com Sat Feb 6 23:22:23 2016 From: torriem at gmail.com (Michael Torrie) Date: Sat, 6 Feb 2016 21:22:23 -0700 Subject: Daemon strategy In-Reply-To: References: <85d1sb9eh7.fsf@benfinney.id.au> <858u2yah3d.fsf@benfinney.id.au> Message-ID: <56B6C67F.1090402@gmail.com> On 02/06/2016 09:04 AM, paul.hermeneutic at gmail.com wrote: > On Fri, Feb 5, 2016 at 4:10 PM, Ben Finney wrote: >> paul.hermeneutic at gmail.com writes: >> >>> On Fri, Feb 5, 2016 at 11:52 AM, Ben Finney wrote: >>>> Since MS Windows lacks those facilities, ?python-daemon? can't use >>>> them. >>> >>> As you might imagine, I am not always able to specify which OS is >>> deployed. That does not mean that I am not responsible for getting the >>> work done. Perhaps you will tell me that what I want is not a daemon. >> >> I'm telling you none of those. What I'm telling you is MS Windows does >> not support what is needed to make a Unix daemon. >> >> You may need to re-visit the requirements and negotiate something >> different ? a different deployment platform, or something which MS >> Windows can do which is less than a proper Unix daemon. > > I fully understand that Windows is not a proper UNIX. It might be that > UNIX is not in each and every aspect a proper Windows. > > If it is possible, I would like to create one tool to do this rather > than multiple. Is there anything in Python that would help to > accomplish this goal? The fairly standard python-win32 package certainly allows you to make a proper win32 service. It's not part of the standard library, but you could ship it with your package. http://ryrobes.com/python/running-python-scripts-as-a-windows-service/ It's just not practical to have one tool that does such an OS-dependent thing. But it's not that hard to have specific code for Windows. From frank at chagford.com Sun Feb 7 00:27:19 2016 From: frank at chagford.com (Frank Millman) Date: Sun, 7 Feb 2016 07:27:19 +0200 Subject: asyncio - how to stop background task cleanly In-Reply-To: <8737t5shhp.fsf@elektro.pacujo.net> References: <87lh6ys052.fsf@elektro.pacujo.net> <87fux5svhk.fsf@elektro.pacujo.net> <8737t5shhp.fsf@elektro.pacujo.net> Message-ID: "Marko Rauhamaa" wrote in message news:8737t5shhp.fsf at elektro.pacujo.net... > > > Actually, cancellation is specially supported in asyncio ( https://docs.python.org/3/library/asyncio-task.html#asyncio.Task.cancel>) > so this should do: > > async def background_task(): > while True: > await perform_task() > await asyncio.sleep(10) > That's exactly what I needed - thanks, Marko async def background_task() try: while True: await perform_task() await asyncio.sleep(10) except asyncio.CancelledError: await perform_cleanup() At startup - task = asyncio.ensure_future(background_task()) At shutdown - task.cancel() await asyncio.wait([task]) Works perfectly - thanks again. Frank From steve+comp.lang.python at pearwood.info Sun Feb 7 00:59:48 2016 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Sun, 07 Feb 2016 16:59:48 +1100 Subject: [STORY-TIME] THE BDFL AND HIS PYTHON PETTING ZOO References: <48762040-a7e0-434c-92e8-8a0969210e5b@googlegroups.com> <5d31dd14-1a60-4f80-8889-d0616f404c70@googlegroups.com> <4e902c58-3370-4adf-905d-74630a6eea90@googlegroups.com> Message-ID: <56b6dd57$0$1614$c3e8da3$5496439d@news.astraweb.com> On Sunday 07 February 2016 14:02, INADA Naoki wrote: > Python 3 is a disaster because of incompatibility with Python 2. How is that a disaster? What is your criteria for deciding what is, and isn't, a disaster? According to TIOBE, Python's popularity continues to grow: http://www.tiobe.com/index.php/content/paperinfo/tpci/index.html and is in the top five, above PHP and Javascript. Likewise with IEEE Spectrum, that puts Python in the top 5 most popular languages for 2015, above C#, PHP and Javascript. http://spectrum.ieee.org/computing/software/the-2015-top-ten-programming- languages According to CodeEval, Python has *dominated* at #1 for the fourth year in a row: http://blog.codeeval.com/codeevalblog/2015 It continues to capture more of the scientific computing niche: http://www.johndcook.com/blog/2015/07/16/scientific-computing-in-python/ and Python's broader ecosystem continues to grow, with at least six powerful add-ons for speeding up Python: - cython - numba - theano - copperhead - parakeet - blaze and more. The future looks brighter than ever for Python. How it is a disaster? -- Steve From kevinjacobconway at gmail.com Sun Feb 7 01:34:45 2016 From: kevinjacobconway at gmail.com (Kevin Conway) Date: Sun, 07 Feb 2016 06:34:45 +0000 Subject: Python's import situation has driven me to the brink of imsanity In-Reply-To: References: Message-ID: > My question is: is this crazy? Please tell me there's a better way and I just wasted my time creating this package. There is a better way and you have wasted your time creating this package. I hear your problem statement as asking two questions. The first is: What is the right way to include executable content in my Python project? The second is: How do I expose executable content from a Python project? As to the first question, from your project README: > Say you have a python project (not a package), with the following structure: All Python code that you want to install and make available in any form, import or executable, _must_ be contained within a Python package. Organizing Python code in any way other than Python packages will result in the challenges you have described. The correct way to include executable content is to place the Python code within the package structure. It should not be put in other directories within the repository root. As to the second question, once all Python code is contained within a package that can be installed you can use setuptools entry points to expose the executable code. The setup() function from setuptools that is used to create setup.py files has an argument called 'entry_points' that allows you to expose executable content over the command line. See [1] and [2] for more details. Feel free to reach out to me off-list if you have a specific project you need advice on. The rules for organizing and packaging Python code aren't complex but they tend to cause new Python developers to stumble at first. A general rule I give everyone when talking about packaging or importing code: If you have to modify sys.path to makes something work then you have most certainly made a mistake. [1] https://pythonhosted.org/setuptools/setuptools.html#automatic-script-creation [2] http://python-packaging.readthedocs.org/en/latest/command-line-scripts.html#the-console-scripts-entry-point On Sat, Feb 6, 2016 at 8:54 PM Chris Angelico wrote: > On Sun, Feb 7, 2016 at 1:47 PM, wrote: > > Imsanity allows you to make imports usable (not ideal, but at least > usable) for python projects without having to manage PYTHONPATHs or do > whacky stuff like running files with python -m or put even whackier > boilerplate at the top of every file. And all it requires is 'import > imsanity' at the top of every file. You can put it in a macro or even just > type it because it's short and easy to remember. > > > > My question is: is this crazy? Please tell me there's a better way and I > just wasted my time creating this package. There's nothing I'd like to hear > more. > > Well, anything that makes you type "import imsanity" at the top of > every script MUST be crazy. :) I don't know about the actual > content/purpose though. Good luck with it! > > ChrisA > -- > https://mail.python.org/mailman/listinfo/python-list > From no.email at nospam.invalid Sun Feb 7 02:04:29 2016 From: no.email at nospam.invalid (Paul Rubin) Date: Sat, 06 Feb 2016 23:04:29 -0800 Subject: [STORY-TIME] THE BDFL AND HIS PYTHON PETTING ZOO References: <48762040-a7e0-434c-92e8-8a0969210e5b@googlegroups.com> <5d31dd14-1a60-4f80-8889-d0616f404c70@googlegroups.com> <4e902c58-3370-4adf-905d-74630a6eea90@googlegroups.com> <56b6dd57$0$1614$c3e8da3$5496439d@news.astraweb.com> Message-ID: <87mvrdovb6.fsf@jester.gateway.pace.com> Steven D'Aprano writes: > According to TIOBE, Python's popularity continues to grow: > http://www.tiobe.com/index.php/content/paperinfo/tpci/index.html I wonder how much of that growth is Python 3 and how much is Python 2. I'm amazed there's still so much C being done. I meet good programmers all the time these days, who have never touched C code. The crash of Objective-C is amusing ;-). It's surprising that R places at all, given the amount of math that its users need to know. I wonder what it means that Perl is booming. Hmm. From frank at chagford.com Sun Feb 7 02:10:36 2016 From: frank at chagford.com (Frank Millman) Date: Sun, 7 Feb 2016 09:10:36 +0200 Subject: asyncio - how to stop background task cleanly In-Reply-To: References: <87lh6ys052.fsf@elektro.pacujo.net> <87fux5svhk.fsf@elektro.pacujo.net> <8737t5shhp.fsf@elektro.pacujo.net> Message-ID: "Frank Millman" wrote in message news:n96kjr$mvl$1 at ger.gmane.org... > > "Marko Rauhamaa" wrote in message > news:8737t5shhp.fsf at elektro.pacujo.net... > > > Actually, cancellation is specially supported in asyncio ( > https://docs.python.org/3/library/asyncio-task.html#asyncio.Task.cancel>) > > so this should do: > > > > async def background_task(): > > while True: > > await perform_task() > > await asyncio.sleep(10) > > > > That's exactly what I needed - thanks, Marko > > async def background_task() > try: > while True: > await perform_task() > await asyncio.sleep(10) > except asyncio.CancelledError: > await perform_cleanup() > > At startup - > > task = asyncio.ensure_future(background_task()) > > At shutdown - > > task.cancel() > await asyncio.wait([task]) > > Works perfectly - thanks again. > Alas, I spoke too soon. I tried to simulate what would happen if the background task was busy with a task when it was cancelled - async def background_task() try: while True: print('start') time.sleep(2) print('done') await asyncio.sleep(10) except asyncio.CancelledError: print('cleanup') print('DONE') If I cancel after a pair of 'start/done' appear, the background task is in the 'asyncio.sleep' stage. The words 'cleanup' and 'DONE' appear instantly, and the program halts. If I cancel after 'start', but before 'done', the background task is executing a task. There is a delay of up to 2 seconds, then the words 'done', 'cleanup', and 'DONE' appear, but the program hangs. If I press Ctrl+C, I get a traceback from the threading module - line 1288, in _shutdown t.join() line 1054, in join self._wait_for_tstate_lock() line 1070, in _wait_for_tstate_lock KeyboardInterrupt So it is waiting for join() to complete. I will continue investigating, but will report it here to see if anyone can come up with an explanation/solution. Thanks Frank From rustompmody at gmail.com Sun Feb 7 02:31:56 2016 From: rustompmody at gmail.com (Rustom Mody) Date: Sat, 6 Feb 2016 23:31:56 -0800 (PST) Subject: [STORY-TIME] THE BDFL AND HIS PYTHON PETTING ZOO In-Reply-To: <87mvrdovb6.fsf@jester.gateway.pace.com> References: <48762040-a7e0-434c-92e8-8a0969210e5b@googlegroups.com> <5d31dd14-1a60-4f80-8889-d0616f404c70@googlegroups.com> <4e902c58-3370-4adf-905d-74630a6eea90@googlegroups.com> <56b6dd57$0$1614$c3e8da3$5496439d@news.astraweb.com> <87mvrdovb6.fsf@jester.gateway.pace.com> Message-ID: On Sunday, February 7, 2016 at 8:04:42 AM UTC+1, Paul Rubin wrote: > Steven D'Aprano writes: > > According to TIOBE, Python's popularity continues to grow: > > http://www.tiobe.com/index.php/content/paperinfo/tpci/index.html > > I wonder how much of that growth is Python 3 and how much is Python 2. > > I'm amazed there's still so much C being done. I meet good programmers > all the time these days, who have never touched C code. > > The crash of Objective-C is amusing ;-). > > It's surprising that R places at all, given the amount of math that its > users need to know. > > I wonder what it means that Perl is booming. Hmm. Nice points: Programming language popularity is generally voodoo. I am only surprised at your surprise at R. Data (science) is after all the hot subject A programmer moving into that field typically starts with python A statistician typically starts R [My voodoo of course :-) ] From auriocus at gmx.de Sun Feb 7 02:44:35 2016 From: auriocus at gmx.de (Christian Gollwitzer) Date: Sun, 7 Feb 2016 08:44:35 +0100 Subject: [STORY-TIME] THE BDFL AND HIS PYTHON PETTING ZOO In-Reply-To: <87mvrdovb6.fsf@jester.gateway.pace.com> References: <48762040-a7e0-434c-92e8-8a0969210e5b@googlegroups.com> <5d31dd14-1a60-4f80-8889-d0616f404c70@googlegroups.com> <4e902c58-3370-4adf-905d-74630a6eea90@googlegroups.com> <56b6dd57$0$1614$c3e8da3$5496439d@news.astraweb.com> <87mvrdovb6.fsf@jester.gateway.pace.com> Message-ID: Am 07.02.16 um 08:04 schrieb Paul Rubin: > Steven D'Aprano writes: >> According to TIOBE, Python's popularity continues to grow: >> http://www.tiobe.com/index.php/content/paperinfo/tpci/index.html > > I wonder how much of that growth is Python 3 and how much is Python 2. > > I'm amazed there's still so much C being done. I meet good programmers > all the time these days, who have never touched C code. This is easily explained; C is used a lot in the hardware and embedded field. Compiled C output is nearly as compact as handwritten assembly code, and arguably much more maintainable. C above C++ can be an artifact. The metric uses search engines to ask for "C -c++ -java -python ..." - I'm not sure they can really distinguish between C and C++ this way. Christian From no.email at nospam.invalid Sun Feb 7 02:51:32 2016 From: no.email at nospam.invalid (Paul Rubin) Date: Sat, 06 Feb 2016 23:51:32 -0800 Subject: [STORY-TIME] THE BDFL AND HIS PYTHON PETTING ZOO References: <48762040-a7e0-434c-92e8-8a0969210e5b@googlegroups.com> <5d31dd14-1a60-4f80-8889-d0616f404c70@googlegroups.com> <4e902c58-3370-4adf-905d-74630a6eea90@googlegroups.com> <56b6dd57$0$1614$c3e8da3$5496439d@news.astraweb.com> <87mvrdovb6.fsf@jester.gateway.pace.com> Message-ID: <87io21ot4r.fsf@jester.gateway.pace.com> Rustom Mody writes: > Data (science) is after all the hot subject > A programmer moving into that field typically starts with python > A statistician typically starts R There aren't THAT many statisticians out there, compared to programmers. From marko at pacujo.net Sun Feb 7 02:53:09 2016 From: marko at pacujo.net (Marko Rauhamaa) Date: Sun, 07 Feb 2016 09:53:09 +0200 Subject: asyncio - how to stop background task cleanly References: <87lh6ys052.fsf@elektro.pacujo.net> <87fux5svhk.fsf@elektro.pacujo.net> <8737t5shhp.fsf@elektro.pacujo.net> Message-ID: <87r3gpq7mi.fsf@elektro.pacujo.net> "Frank Millman" : > Alas, I spoke too soon. > > [...] > > If I press Ctrl+C, I get a traceback from the threading module - > > line 1288, in _shutdown > t.join() > line 1054, in join > self._wait_for_tstate_lock() > line 1070, in _wait_for_tstate_lock > KeyboardInterrupt > > So it is waiting for join() to complete. I will continue > investigating, but will report it here to see if anyone can come up > with an explanation/solution. I can't see your complete program, but here's mine, and it seems to be working: ======================================================================== #!/usr/bin/env python3 import asyncio, time def main(): loop = asyncio.get_event_loop() try: task = asyncio.async(background_task()) loop.run_until_complete(asyncio.wait([ task, terminator(task) ])) finally: loop.close() @asyncio.coroutine def terminator(task): yield from asyncio.sleep(5) task.cancel() yield from asyncio.wait([ task ]) @asyncio.coroutine def background_task(): try: while True: print('start') time.sleep(2) print('done') yield from asyncio.sleep(10) except asyncio.CancelledError: print('cleanup') print('DONE') if __name__ == '__main__': main() ======================================================================== (My Python is slightly older, so replace yield from ==> await @asyncio.coroutine ==> async asyncio.async ==> asyncio.ensure_future) Marko From rosuav at gmail.com Sun Feb 7 02:58:50 2016 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 7 Feb 2016 18:58:50 +1100 Subject: [STORY-TIME] THE BDFL AND HIS PYTHON PETTING ZOO In-Reply-To: <87io21ot4r.fsf@jester.gateway.pace.com> References: <48762040-a7e0-434c-92e8-8a0969210e5b@googlegroups.com> <5d31dd14-1a60-4f80-8889-d0616f404c70@googlegroups.com> <4e902c58-3370-4adf-905d-74630a6eea90@googlegroups.com> <56b6dd57$0$1614$c3e8da3$5496439d@news.astraweb.com> <87mvrdovb6.fsf@jester.gateway.pace.com> <87io21ot4r.fsf@jester.gateway.pace.com> Message-ID: On Sun, Feb 7, 2016 at 6:51 PM, Paul Rubin wrote: > Rustom Mody writes: >> Data (science) is after all the hot subject >> A programmer moving into that field typically starts with python >> A statistician typically starts R > > There aren't THAT many statisticians out there, compared to programmers. Would writing a script to figure out whether there are more statisticians or programmers be a statistician's job or a programmer's? ChrisA From dieter at handshake.de Sun Feb 7 03:23:04 2016 From: dieter at handshake.de (dieter) Date: Sun, 07 Feb 2016 09:23:04 +0100 Subject: Exception handling for socket.error in Python 3.5/RStudio References: <1067e50c-bcbd-464e-8013-754bd66fda91@googlegroups.com> Message-ID: <874mdl2al3.fsf@handshake.de> Shaunak Bangale writes: > ... > while (count > 0): > try : > # read line from file: > print(file.readline()) > # parse > parse_json(file.readline()) > count = count - 1 > except socket.error as e: > print('Connection fail', e) > print(traceback.format_exc()) > ... > Error: > except socket.error as e: > ^ > SyntaxError: invalid syntax Are you sure, that there is no invisible character at the end of that line? When I try code like the above (in PYthon 2.7), there is no "SyntaxError": >>> while(False): ... try : ... # read line from file: ... print(file.readline()) ... # parse ... parse_json(file.readline()) ... count = count - 1 ... except socket.error as e: ... print('Connection fail', e) ... print(traceback.format_exc()) ... >>> Another reason for your problem could be an older Python version. The "as" construct as part of the "except" clause is a more recent addition to Python. From frank at chagford.com Sun Feb 7 03:55:58 2016 From: frank at chagford.com (Frank Millman) Date: Sun, 7 Feb 2016 10:55:58 +0200 Subject: asyncio - how to stop background task cleanly In-Reply-To: <87r3gpq7mi.fsf@elektro.pacujo.net> References: <87lh6ys052.fsf@elektro.pacujo.net> <87fux5svhk.fsf@elektro.pacujo.net> <8737t5shhp.fsf@elektro.pacujo.net> <87r3gpq7mi.fsf@elektro.pacujo.net> Message-ID: "Marko Rauhamaa" wrote in message news:87r3gpq7mi.fsf at elektro.pacujo.net... > > I can't see your complete program, but here's mine, and it seems to be > working > Thanks, Marko, I really appreciate your assistance. I wanted to show you my complete program, but as it is quite long I distilled it down to its essence, and lo and behold it works! So now I just have to go through my program and find where it differs - I must have a bug somewhere. For the record, here is my stripped down version. The main difference from yours is that I want to run a genuine 'loop forever', and only shut it down on receipt of some external signal. I have never been able to get Ctrl+C to work properly on Windows, so I use a separate thread that simply waits for Enter. You will see that if you press Enter after 'done' appears, the program closes instantly, but if you press it in between 'start' and 'done', it waits for the task to complete before it closes. Frank ============================================================= import asyncio, time import threading def main(): loop = asyncio.get_event_loop() task = asyncio.async(background_task()) threading.Thread(target=stop, args=(loop, task)).start() loop.run_forever() @asyncio.coroutine def background_task(): try: while True: print('start') time.sleep(2) print('done') yield from asyncio.sleep(5) except asyncio.CancelledError: print('cleanup') print('DONE') @asyncio.coroutine def shutdown(loop, task): task.cancel() yield from asyncio.wait([task], loop=loop) loop.stop() def stop(loop, task): input('Press to stop\n') asyncio.run_coroutine_threadsafe(shutdown(loop, task), loop) if __name__ == '__main__': main() From marko at pacujo.net Sun Feb 7 04:27:09 2016 From: marko at pacujo.net (Marko Rauhamaa) Date: Sun, 07 Feb 2016 11:27:09 +0200 Subject: asyncio - how to stop background task cleanly References: <87lh6ys052.fsf@elektro.pacujo.net> <87fux5svhk.fsf@elektro.pacujo.net> <8737t5shhp.fsf@elektro.pacujo.net> <87r3gpq7mi.fsf@elektro.pacujo.net> Message-ID: <871t8orhua.fsf@elektro.pacujo.net> "Frank Millman" : > I have never been able to get Ctrl+C to work properly on Windows, so I > use a separate thread that simply waits for Enter. Now you are leaving my realm of expertise, as I haven't programmed for windows since Windows 1.0. Mixing threads, asyncio, input() and Windows seems like begging for trouble, though. Marko From breamoreboy at yahoo.co.uk Sun Feb 7 05:13:23 2016 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Sun, 7 Feb 2016 10:13:23 +0000 Subject: trouble installing python In-Reply-To: <1698609935.300641.1454805263360.JavaMail.yahoo@mail.yahoo.com> References: <1698609935.300641.1454805263360.JavaMail.yahoo.ref@mail.yahoo.com> <1698609935.300641.1454805263360.JavaMail.yahoo@mail.yahoo.com> Message-ID: On 07/02/2016 00:34, donald alsept via Python-list wrote: > Hello, > I'm trying to install the 3.5.1 of Python and am running windows 7. I keep getting an error about api-ms-win-crt-runtime-|1-1-0.dll not being installed. Any advice on what is wrong? > Hello and welcome. This question has been asked and ansered repeatedly in the last few weeks so please search the archives for the answer. -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From frank at chagford.com Sun Feb 7 05:20:12 2016 From: frank at chagford.com (Frank Millman) Date: Sun, 7 Feb 2016 12:20:12 +0200 Subject: asyncio - how to stop background task cleanly In-Reply-To: <871t8orhua.fsf@elektro.pacujo.net> References: <87lh6ys052.fsf@elektro.pacujo.net> <87fux5svhk.fsf@elektro.pacujo.net> <8737t5shhp.fsf@elektro.pacujo.net> <87r3gpq7mi.fsf@elektro.pacujo.net> <871t8orhua.fsf@elektro.pacujo.net> Message-ID: "Marko Rauhamaa" wrote in message news:871t8orhua.fsf at elektro.pacujo.net... > "Frank Millman" : > > > I have never been able to get Ctrl+C to work properly on Windows, so I > > use a separate thread that simply waits for Enter. > > Now you are leaving my realm of expertise, as I haven't programmed for > windows since Windows 1.0. Mixing threads, asyncio, input() and Windows > seems like begging for trouble, though. > Well, it is not such a big deal :-) The program I posted works cross-platform, so it will run on your linux box. No matter, you have been an enormous help, and I am very grateful. I found my bug, and my program is now running sweetly. Frank From cl at isbd.net Sun Feb 7 06:19:25 2016 From: cl at isbd.net (cl at isbd.net) Date: Sun, 7 Feb 2016 11:19:25 +0000 Subject: What's the best/neatest way to get Unicode data from a database into a grid cell? Message-ID: I'm using this as a starting point for creating a grid to view and edit some sqlite data:- http://www.salstat.com/news/linking-wxgrid-to-sqlite-database-in-python-an-example I can actually understand most of it which is a good start. However my database has quite a lot of Unicode data as there are French (and other) names with accents etc. What's the right way to handle this reasonably neatly? At the moment it traps an error at line 37:- self.SetCellValue(row_num, i, str(cells[i])) I realise why this fails (I think) but how should one program this so that:- 1 - the accented characters are displayed correctly in the grid cell 2 - One can edit the cell with accented characters (I'll get round how to get the accented characters back to the database later!) My system (xubuntu 15.10) is all UTF8 so accented characters are handled by the display, in terminals, etc. correctly. I'm currently using python 2.7 for this but would be quite happy to move to 3.4 if this handles UTF8 better (I seem to remember it does maybe). -- Chris Green ? From sivan at vitakka.co Sun Feb 7 06:43:50 2016 From: sivan at vitakka.co (Sivan Greenberg) Date: Sun, 7 Feb 2016 13:43:50 +0200 Subject: Python's import situation has driven me to the brink of imsanity In-Reply-To: References: Message-ID: in sake of providing another example, as I ran into the same problem and fixed it the very same way, one can also take a look at: (this is the main executable) https://github.com/sivang/navistore/blob/master/navistore/backend/nhttpserver.py Note the explicit imports in lines 19-22 to allow the executable finding modules within the OS instance's site-packages structure. Here's is Navistore's setup.py for reference (I actually use this project as a study project in a Python course for programmers I'm running): - https://github.com/sivang/navistore/blob/master/setup.py You may find this to be helpful: - https://pythonhosted.org/setuptools/setuptools.html#using-find-packages To make Navistore's main executable be avail at command line I used this: - http://python-packaging.readthedocs.org/en/latest/command-line-scripts.html On Sun, Feb 7, 2016 at 8:34 AM, Kevin Conway wrote: > > My question is: is this crazy? Please tell me there's a better way and I > just wasted my time creating this package. > > There is a better way and you have wasted your time creating this package. > > I hear your problem statement as asking two questions. The first is: What > is the right way to include executable content in my Python project? The > second is: How do I expose executable content from a Python project? > > As to the first question, from your project README: > > Say you have a python project (not a package), with the following > structure: > > All Python code that you want to install and make available in any form, > import or executable, _must_ be contained within a Python package. > Organizing Python code in any way other than Python packages will result in > the challenges you have described. The correct way to include executable > content is to place the Python code within the package structure. It should > not be put in other directories within the repository root. > > As to the second question, once all Python code is contained within a > package that can be installed you can use setuptools entry points to expose > the executable code. The setup() function from setuptools that is used to > create setup.py files has an argument called 'entry_points' that allows you > to expose executable content over the command line. See [1] and [2] for > more details. > > Feel free to reach out to me off-list if you have a specific project you > need advice on. The rules for organizing and packaging Python code aren't > complex but they tend to cause new Python developers to stumble at first. A > general rule I give everyone when talking about packaging or importing > code: If you have to modify sys.path to makes something work then you have > most certainly made a mistake. > > [1] > > https://pythonhosted.org/setuptools/setuptools.html#automatic-script-creation > [2] > > http://python-packaging.readthedocs.org/en/latest/command-line-scripts.html#the-console-scripts-entry-point > > > On Sat, Feb 6, 2016 at 8:54 PM Chris Angelico wrote: > > > On Sun, Feb 7, 2016 at 1:47 PM, wrote: > > > Imsanity allows you to make imports usable (not ideal, but at least > > usable) for python projects without having to manage PYTHONPATHs or do > > whacky stuff like running files with python -m or put even whackier > > boilerplate at the top of every file. And all it requires is 'import > > imsanity' at the top of every file. You can put it in a macro or even > just > > type it because it's short and easy to remember. > > > > > > My question is: is this crazy? Please tell me there's a better way and > I > > just wasted my time creating this package. There's nothing I'd like to > hear > > more. > > > > Well, anything that makes you type "import imsanity" at the top of > > every script MUST be crazy. :) I don't know about the actual > > content/purpose though. Good luck with it! > > > > ChrisA > > -- > > https://mail.python.org/mailman/listinfo/python-list > > > -- > https://mail.python.org/mailman/listinfo/python-list > -- Sivan Greenberg Co founder & CTO Vitakka Consulting From rosuav at gmail.com Sun Feb 7 06:56:07 2016 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 7 Feb 2016 22:56:07 +1100 Subject: What's the best/neatest way to get Unicode data from a database into a grid cell? In-Reply-To: References: Message-ID: On Sun, Feb 7, 2016 at 10:19 PM, wrote: > My system (xubuntu 15.10) is all UTF8 so accented characters are > handled by the display, in terminals, etc. correctly. I'm currently > using python 2.7 for this but would be quite happy to move to 3.4 if > this handles UTF8 better (I seem to remember it does maybe). As a general rule, yes, Python 3 handles Unicode somewhat better than Python 2 does. The main reason for this is that the native string type is now a Unicode string, instead of the native string type being a byte string; so "abcd" is not a string of bytes that happen to be encoded as UTF-8, it is actually a string of Unicode characters. For the most part, you'll be able to ignore character encodings and such, and just work with text. So the question then becomes: Under Python 3, can you use regular string objects for both the things you're working with? I can confirm that the inbuilt sqlite3 module works just fine with Unicode text; so all you need to do is try out your GUI code under Python 3. I've no idea how good wx support in Py3 is, so you might find you need to switch GUI toolkits to get everything working; but whether it's with wxWidgets, GTK, QT, or some other library, you should be able to put something together under Python 3 that "just works" as regards Unicode. Caveat: I haven't done any serious GUI programming using Python. (Note that "just works" can actually be a lot of hassle if you truly want to support *all* of Unicode. I've seen programs that don't behave correctly in the presence of combining characters, or right-to-left text, or zero-width characters, or non-BMP characters; but you can get far better support for the same amount of effort if you use Py3 and Unicode than if you try to do things manually under Py2.) ChrisA From cl at isbd.net Sun Feb 7 07:42:45 2016 From: cl at isbd.net (cl at isbd.net) Date: Sun, 7 Feb 2016 12:42:45 +0000 Subject: What's the best/neatest way to get Unicode data from a database into a grid cell? References: Message-ID: <5nqjoc-55e.ln1@esprimo.zbmc.eu> Chris Angelico wrote: > > So the question then becomes: Under Python 3, can you use regular > string objects for both the things you're working with? I can confirm > that the inbuilt sqlite3 module works just fine with Unicode text; so > all you need to do is try out your GUI code under Python 3. I've no > idea how good wx support in Py3 is, so you might find you need to > switch GUI toolkits to get everything working; but whether it's with > wxWidgets, GTK, QT, or some other library, you should be able to put > something together under Python 3 that "just works" as regards > Unicode. Caveat: I haven't done any serious GUI programming using > Python. > Sadly wxpython doesn't work at all in Python 3. There's a project called Phoenix to move it across to Python 3 but it's nowhere near stable yet, which is rather a pity. So the question is do I stay with 2.7 and live with the Unicode difficulties or do I try and move to another GUI that does work with Python 3. Are there any Python 3 GUIs that would be reasonably easy to move to? E.g. ones which have a grid object and which work in the same sort of way as wxpython in general? -- Chris Green ? From rosuav at gmail.com Sun Feb 7 08:11:26 2016 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 8 Feb 2016 00:11:26 +1100 Subject: What's the best/neatest way to get Unicode data from a database into a grid cell? In-Reply-To: <5nqjoc-55e.ln1@esprimo.zbmc.eu> References: <5nqjoc-55e.ln1@esprimo.zbmc.eu> Message-ID: On Sun, Feb 7, 2016 at 11:42 PM, wrote: > > Are there any Python 3 GUIs that would be reasonably easy to move to? > E.g. ones which have a grid object and which work in the same sort of > way as wxpython in general? Grid object? I'm not sure. But all of my GUI work of late has been with GTK, and I'm pretty happy with it. (Granted, that's with Pike, not Python, so the exact API is slightly different; but I know there is good support for Python 3.) You may also want to look into Tk / tkinter, although its Unicode support is (or was, last I heard) limited to the Basic Multilingual Plane - the 64K most commonly used characters. ChrisA From cl at isbd.net Sun Feb 7 08:59:59 2016 From: cl at isbd.net (cl at isbd.net) Date: Sun, 7 Feb 2016 13:59:59 +0000 Subject: What's the best/neatest way to get Unicode data from a database into a grid cell? References: <5nqjoc-55e.ln1@esprimo.zbmc.eu> Message-ID: Chris Angelico wrote: > On Sun, Feb 7, 2016 at 11:42 PM, wrote: > > > > Are there any Python 3 GUIs that would be reasonably easy to move to? > > E.g. ones which have a grid object and which work in the same sort of > > way as wxpython in general? > > Grid object? I'm not sure. But all of my GUI work of late has been > with GTK, and I'm pretty happy with it. (Granted, that's with Pike, > not Python, so the exact API is slightly different; but I know there > is good support for Python 3.) You may also want to look into Tk / > tkinter, although its Unicode support is (or was, last I heard) > limited to the Basic Multilingual Plane - the 64K most commonly used > characters. > Tkinter is certainly a possibility. Basic Unicode support will be fine, I'm only interested in handling European accented characters. -- Chris Green ? From dimva13 at gmail.com Sun Feb 7 09:13:57 2016 From: dimva13 at gmail.com (dimva13 at gmail.com) Date: Sun, 7 Feb 2016 06:13:57 -0800 (PST) Subject: Python's import situation has driven me to the brink of imsanity In-Reply-To: References: Message-ID: <51b52cf3-0b0a-451e-a232-d9a671f7cd20@googlegroups.com> I see that this would work once you've installed the package, but how do you develop it? Say you are working on a change that modifies both email.py and reports.py. Do you run setup.py every time you make a change in email.py? On Sunday, February 7, 2016 at 1:35:15 AM UTC-5, Kevin Conway wrote: > > My question is: is this crazy? Please tell me there's a better way and I > just wasted my time creating this package. > > There is a better way and you have wasted your time creating this package. > > I hear your problem statement as asking two questions. The first is: What > is the right way to include executable content in my Python project? The > second is: How do I expose executable content from a Python project? > > As to the first question, from your project README: > > Say you have a python project (not a package), with the following > structure: > > All Python code that you want to install and make available in any form, > import or executable, _must_ be contained within a Python package. > Organizing Python code in any way other than Python packages will result in > the challenges you have described. The correct way to include executable > content is to place the Python code within the package structure. It should > not be put in other directories within the repository root. > > As to the second question, once all Python code is contained within a > package that can be installed you can use setuptools entry points to expose > the executable code. The setup() function from setuptools that is used to > create setup.py files has an argument called 'entry_points' that allows you > to expose executable content over the command line. See [1] and [2] for > more details. > > Feel free to reach out to me off-list if you have a specific project you > need advice on. The rules for organizing and packaging Python code aren't > complex but they tend to cause new Python developers to stumble at first. A > general rule I give everyone when talking about packaging or importing > code: If you have to modify sys.path to makes something work then you have > most certainly made a mistake. > > [1] > https://pythonhosted.org/setuptools/setuptools.html#automatic-script-creation > [2] > http://python-packaging.readthedocs.org/en/latest/command-line-scripts.html#the-console-scripts-entry-point > > > On Sat, Feb 6, 2016 at 8:54 PM Chris Angelico wrote: > > > On Sun, Feb 7, 2016 at 1:47 PM, wrote: > > > Imsanity allows you to make imports usable (not ideal, but at least > > usable) for python projects without having to manage PYTHONPATHs or do > > whacky stuff like running files with python -m or put even whackier > > boilerplate at the top of every file. And all it requires is 'import > > imsanity' at the top of every file. You can put it in a macro or even just > > type it because it's short and easy to remember. > > > > > > My question is: is this crazy? Please tell me there's a better way and I > > just wasted my time creating this package. There's nothing I'd like to hear > > more. > > > > Well, anything that makes you type "import imsanity" at the top of > > every script MUST be crazy. :) I don't know about the actual > > content/purpose though. Good luck with it! > > > > ChrisA > > -- > > https://mail.python.org/mailman/listinfo/python-list > > From maillist at schwertberger.de Sun Feb 7 10:06:24 2016 From: maillist at schwertberger.de (Dietmar Schwertberger) Date: Sun, 7 Feb 2016 16:06:24 +0100 Subject: What's the best/neatest way to get Unicode data from a database into a grid cell? In-Reply-To: References: Message-ID: <56B75D70.3030504@schwertberger.de> On 07.02.2016 12:19, cl at isbd.net wrote: > However my database has quite a lot of Unicode data as there are > French (and other) names with accents etc. What's the right way to > handle this reasonably neatly? At the moment it traps an error at > line 37:- > self.SetCellValue(row_num, i, str(cells[i])) The unicode versions of wxPython should have no problems with handling unicode strings. The problem that you see here is that str(...) tries to convert your unicode data into a non-unicode string, which of course fails. Do something like: value = cells[i] if not isinstance(value, basestring): value = str(value) self.SetCellValue(row_num, i, value) Once you switch to Python 3 and Phoenix you have to modify this slightly, e.g. by adding this to the top of your code: try: basestring except: basestring = (bytes,str) Regards, Dietmar From kevinjacobconway at gmail.com Sun Feb 7 11:09:11 2016 From: kevinjacobconway at gmail.com (Kevin Conway) Date: Sun, 07 Feb 2016 16:09:11 +0000 Subject: Python's import situation has driven me to the brink of imsanity In-Reply-To: <51b52cf3-0b0a-451e-a232-d9a671f7cd20@googlegroups.com> References: <51b52cf3-0b0a-451e-a232-d9a671f7cd20@googlegroups.com> Message-ID: You can use 'setup.py develop' or 'pip install -e' to install your package in editable mode. It makes it so your local code is used. Modifications are seen immediately. On Sun, Feb 7, 2016, 08:16 wrote: > I see that this would work once you've installed the package, but how do > you develop it? Say you are working on a change that modifies both email.py > and reports.py. Do you run setup.py every time you make a change in > email.py? > > On Sunday, February 7, 2016 at 1:35:15 AM UTC-5, Kevin Conway wrote: > > > My question is: is this crazy? Please tell me there's a better way and > I > > just wasted my time creating this package. > > > > There is a better way and you have wasted your time creating this > package. > > > > I hear your problem statement as asking two questions. The first is: What > > is the right way to include executable content in my Python project? The > > second is: How do I expose executable content from a Python project? > > > > As to the first question, from your project README: > > > Say you have a python project (not a package), with the following > > structure: > > > > All Python code that you want to install and make available in any form, > > import or executable, _must_ be contained within a Python package. > > Organizing Python code in any way other than Python packages will result > in > > the challenges you have described. The correct way to include executable > > content is to place the Python code within the package structure. It > should > > not be put in other directories within the repository root. > > > > As to the second question, once all Python code is contained within a > > package that can be installed you can use setuptools entry points to > expose > > the executable code. The setup() function from setuptools that is used to > > create setup.py files has an argument called 'entry_points' that allows > you > > to expose executable content over the command line. See [1] and [2] for > > more details. > > > > Feel free to reach out to me off-list if you have a specific project you > > need advice on. The rules for organizing and packaging Python code aren't > > complex but they tend to cause new Python developers to stumble at > first. A > > general rule I give everyone when talking about packaging or importing > > code: If you have to modify sys.path to makes something work then you > have > > most certainly made a mistake. > > > > [1] > > > https://pythonhosted.org/setuptools/setuptools.html#automatic-script-creation > > [2] > > > http://python-packaging.readthedocs.org/en/latest/command-line-scripts.html#the-console-scripts-entry-point > > > > > > On Sat, Feb 6, 2016 at 8:54 PM Chris Angelico wrote: > > > > > On Sun, Feb 7, 2016 at 1:47 PM, wrote: > > > > Imsanity allows you to make imports usable (not ideal, but at least > > > usable) for python projects without having to manage PYTHONPATHs or do > > > whacky stuff like running files with python -m or put even whackier > > > boilerplate at the top of every file. And all it requires is 'import > > > imsanity' at the top of every file. You can put it in a macro or even > just > > > type it because it's short and easy to remember. > > > > > > > > My question is: is this crazy? Please tell me there's a better way > and I > > > just wasted my time creating this package. There's nothing I'd like to > hear > > > more. > > > > > > Well, anything that makes you type "import imsanity" at the top of > > > every script MUST be crazy. :) I don't know about the actual > > > content/purpose though. Good luck with it! > > > > > > ChrisA > > > -- > > > https://mail.python.org/mailman/listinfo/python-list > > > > > -- > https://mail.python.org/mailman/listinfo/python-list > From sivan at vitakka.co Sun Feb 7 11:15:26 2016 From: sivan at vitakka.co (Sivan Greenberg) Date: Sun, 7 Feb 2016 18:15:26 +0200 Subject: Python's import situation has driven me to the brink of imsanity In-Reply-To: References: <51b52cf3-0b0a-451e-a232-d9a671f7cd20@googlegroups.com> Message-ID: I if it's more than few files, I use 'develop' as Kevin noted in the note before: - http://www.ewencp.org/blog/a-brief-introduction-to-packaging-python/ Hope this helps.. -Sivan On Sun, Feb 7, 2016 at 6:09 PM, Kevin Conway wrote: > You can use 'setup.py develop' or 'pip install -e' to install your package > in editable mode. It makes it so your local code is used. Modifications are > seen immediately. > > On Sun, Feb 7, 2016, 08:16 wrote: > > > I see that this would work once you've installed the package, but how do > > you develop it? Say you are working on a change that modifies both > email.py > > and reports.py. Do you run setup.py every time you make a change in > > email.py? > > > > On Sunday, February 7, 2016 at 1:35:15 AM UTC-5, Kevin Conway wrote: > > > > My question is: is this crazy? Please tell me there's a better way > and > > I > > > just wasted my time creating this package. > > > > > > There is a better way and you have wasted your time creating this > > package. > > > > > > I hear your problem statement as asking two questions. The first is: > What > > > is the right way to include executable content in my Python project? > The > > > second is: How do I expose executable content from a Python project? > > > > > > As to the first question, from your project README: > > > > Say you have a python project (not a package), with the following > > > structure: > > > > > > All Python code that you want to install and make available in any > form, > > > import or executable, _must_ be contained within a Python package. > > > Organizing Python code in any way other than Python packages will > result > > in > > > the challenges you have described. The correct way to include > executable > > > content is to place the Python code within the package structure. It > > should > > > not be put in other directories within the repository root. > > > > > > As to the second question, once all Python code is contained within a > > > package that can be installed you can use setuptools entry points to > > expose > > > the executable code. The setup() function from setuptools that is used > to > > > create setup.py files has an argument called 'entry_points' that allows > > you > > > to expose executable content over the command line. See [1] and [2] for > > > more details. > > > > > > Feel free to reach out to me off-list if you have a specific project > you > > > need advice on. The rules for organizing and packaging Python code > aren't > > > complex but they tend to cause new Python developers to stumble at > > first. A > > > general rule I give everyone when talking about packaging or importing > > > code: If you have to modify sys.path to makes something work then you > > have > > > most certainly made a mistake. > > > > > > [1] > > > > > > https://pythonhosted.org/setuptools/setuptools.html#automatic-script-creation > > > [2] > > > > > > http://python-packaging.readthedocs.org/en/latest/command-line-scripts.html#the-console-scripts-entry-point > > > > > > > > > On Sat, Feb 6, 2016 at 8:54 PM Chris Angelico > wrote: > > > > > > > On Sun, Feb 7, 2016 at 1:47 PM, wrote: > > > > > Imsanity allows you to make imports usable (not ideal, but at least > > > > usable) for python projects without having to manage PYTHONPATHs or > do > > > > whacky stuff like running files with python -m or put even whackier > > > > boilerplate at the top of every file. And all it requires is 'import > > > > imsanity' at the top of every file. You can put it in a macro or even > > just > > > > type it because it's short and easy to remember. > > > > > > > > > > My question is: is this crazy? Please tell me there's a better way > > and I > > > > just wasted my time creating this package. There's nothing I'd like > to > > hear > > > > more. > > > > > > > > Well, anything that makes you type "import imsanity" at the top of > > > > every script MUST be crazy. :) I don't know about the actual > > > > content/purpose though. Good luck with it! > > > > > > > > ChrisA > > > > -- > > > > https://mail.python.org/mailman/listinfo/python-list > > > > > > > > -- > > https://mail.python.org/mailman/listinfo/python-list > > > -- > https://mail.python.org/mailman/listinfo/python-list > -- Sivan Greenberg Co founder & CTO Vitakka Consulting From vlastimil.brom at gmail.com Sun Feb 7 13:22:34 2016 From: vlastimil.brom at gmail.com (Vlastimil Brom) Date: Sun, 7 Feb 2016 19:22:34 +0100 Subject: What's the best/neatest way to get Unicode data from a database into a grid cell? In-Reply-To: References: Message-ID: 2016-02-07 12:19 GMT+01:00 : > I'm using this as a starting point for creating a grid to view and > edit some sqlite data:- > http://www.salstat.com/news/linking-wxgrid-to-sqlite-database-in-python-an-example > > I can actually understand most of it which is a good start. > > However my database has quite a lot of Unicode data as there are > French (and other) names with accents etc. What's the right way to > handle this reasonably neatly? At the moment it traps an error at > line 37:- > self.SetCellValue(row_num, i, str(cells[i])) > > I realise why this fails (I think) but how should one program this so > that:- > 1 - the accented characters are displayed correctly in the grid cell > 2 - One can edit the cell with accented characters > > (I'll get round how to get the accented characters back to the > database later!) > > My system (xubuntu 15.10) is all UTF8 so accented characters are > handled by the display, in terminals, etc. correctly. I'm currently > using python 2.7 for this but would be quite happy to move to 3.4 if > this handles UTF8 better (I seem to remember it does maybe). > > -- > Chris Green > ? > -- > https://mail.python.org/mailman/listinfo/python-list Hi, your code in http://www.salstat.com/news/linking-wxgrid-to-sqlite-database-in-python-an-example seems to work for me after small changes with both python 2.7 and 3.4 (using wx Phoenix) the changes I made are: - encoding declaration at the beginning of the file (mainly for py 2, if utf-8 is used): #! Python # -*- coding: utf-8 -*- - removing the str(...) call in the offending line you mentioned: self.SetCellValue(row_num, i, cells[i]) (what was the intent of the conversion to str?) - corrected event name (insted of EVT_GRID_CELL_CHANGE) self.Bind(gridlib.EVT_GRID_CELL_CHANGED, self.CellContentsChanged) - normal App() call (instead of the PySimpleApp) app = wx.App() I randomly tried some characters using Latin, Greek etc. characters (within BMP) as well as Gothic - beyond the FFFF range - the cell content seems to be saved and and newly loaded from the sqlite db on subsequent calls of the app. regards, vbr From mbrahimi02 at gmail.com Sun Feb 7 15:45:07 2016 From: mbrahimi02 at gmail.com (Malik Brahimi) Date: Sun, 7 Feb 2016 15:45:07 -0500 Subject: Handling Print Events Message-ID: Hello guys, So basically, a print job (.SPL file) is generated during local spooling as soon as a document is requested to print. I do NOT want to handle this event. I want to trigger a callback function as soon as a print job reaches a print server regardless of whether it begins to immediately print there or not. Essentially, even on a high volume network, I want to catch the even in which the print job reaches the print server, not the event in which the print job is initially created. Is there anyway to do that with Python? I would also like to get the JobId of said print job as this event is caught. http://stackoverflow.com/questions/35228080/print-servers-and-triggering-callbacks http://stackoverflow.com/questions/34519642/received-print-job-python Thanks, Malik From p_s_d_a_s_i_l_v_a_ns at netcabo.pt Sun Feb 7 16:46:58 2016 From: p_s_d_a_s_i_l_v_a_ns at netcabo.pt (Paulo da Silva) Date: Sun, 7 Feb 2016 21:46:58 +0000 Subject: A sets algorithm Message-ID: Hello! This may not be a strict python question, but ... Suppose I have already a class MyFile that has an efficient method (or operator) to compare two MyFile s for equality. What is the most efficient way to obtain all sets of equal files (of course each set must have more than one file - all single files are discarded)? Thanks for any suggestions. Paulo From rosuav at gmail.com Sun Feb 7 16:58:28 2016 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 8 Feb 2016 08:58:28 +1100 Subject: A sets algorithm In-Reply-To: References: Message-ID: On Mon, Feb 8, 2016 at 8:46 AM, Paulo da Silva wrote: > Hello! > > This may not be a strict python question, but ... > > Suppose I have already a class MyFile that has an efficient method (or > operator) to compare two MyFile s for equality. > > What is the most efficient way to obtain all sets of equal files (of > course each set must have more than one file - all single files are > discarded)? > > Thanks for any suggestions. Hash them in some way. This has two costs: 1) You need to figure out some hashing algorithm such that any two equal files have the same hash, and ideally, that unequal files will generally have unequal hashes. 2) Hash each file once, and then do your comparisons on the hashes, finally testing for actual equality only on those with the same hash. (The last step takes care of hash collisions - where different files happen to have the same hash - so ideally, you shouldn't have to do this often.) If your definition of "equal" among MyFiles is simply based on the file content, it's easy - just hash the content. But if there are ways for files to be considered equal without being bit-for-bit identical, you'll have to reflect that in the hash. For instance, if you consider files equal if they differ only in whitespace, then you'd need to convert all whitespace to a single space before hashing; or if you don't care about the order of lines in the file, you could either hash the lines separately and sum/xor the hashes, or sort the lines before hashing. But the rest of the job is pretty straight-forward. ChrisA From oscar.j.benjamin at gmail.com Sun Feb 7 17:03:16 2016 From: oscar.j.benjamin at gmail.com (Oscar Benjamin) Date: Sun, 7 Feb 2016 22:03:16 +0000 Subject: A sets algorithm In-Reply-To: References: Message-ID: On 7 Feb 2016 21:51, "Paulo da Silva" wrote: > > Hello! > > This may not be a strict python question, but ... > > Suppose I have already a class MyFile that has an efficient method (or > operator) to compare two MyFile s for equality. > > What is the most efficient way to obtain all sets of equal files (of > course each set must have more than one file - all single files are > discarded)? If you can make the MyFiles hashable then this is easily done with defaultdict(list). -- Oscar From python.list at tim.thechases.com Sun Feb 7 17:17:54 2016 From: python.list at tim.thechases.com (Tim Chase) Date: Sun, 7 Feb 2016 16:17:54 -0600 Subject: A sets algorithm In-Reply-To: References: Message-ID: <20160207161754.478f687e@bigbox.christie.dr> On 2016-02-07 21:46, Paulo da Silva wrote: > Suppose I have already a class MyFile that has an efficient method > (or operator) to compare two MyFile s for equality. > > What is the most efficient way to obtain all sets of equal files (of > course each set must have more than one file - all single files are > discarded)? If your MyFile grows a __hash__ method that is based off the uniqueness-qualities that you're comparing, you can use collections.Counter and filter the results: from collections import Counter interesting = [ my_file for my_file, count in Counter(generate_MyFile_objects()).iteritems() if count > 1 ] Which should be roughly O(n). It also has the advantage of iterating over your generator once. If you the MyFile objects can be unique but compare for equality (e.g. two files on the file-system that have the same SHA1 hash, but you want to know the file-names), you'd have to do a paired search which would have worse performance and would need to iterate over the data multiple times: all_files = list(generate_MyFile_objects()) interesting = [ (my_file1, my_file2) for i, my_file1 in enumerate(all_files, 1) for my_file2 in all_files[i:] if my_file1 == my_file2 ] -tkc From p_s_d_a_s_i_l_v_a_ns at netcabo.pt Sun Feb 7 19:05:16 2016 From: p_s_d_a_s_i_l_v_a_ns at netcabo.pt (Paulo da Silva) Date: Mon, 8 Feb 2016 00:05:16 +0000 Subject: A sets algorithm References: Message-ID: ?s 22:17 de 07-02-2016, Tim Chase escreveu: > On 2016-02-07 21:46, Paulo da Silva wrote: ... > > If you the MyFile objects can be unique but compare for equality > (e.g. two files on the file-system that have the same SHA1 hash, but > you want to know the file-names), you'd have to do a paired search > which would have worse performance and would need to iterate over the > data multiple times: > > all_files = list(generate_MyFile_objects()) > interesting = [ > (my_file1, my_file2) > for i, my_file1 > in enumerate(all_files, 1) > for my_file2 > in all_files[i:] > if my_file1 == my_file2 > ] > "my_file1 == my_file2" can be implemented into MyFile class taking advantage of caching sizes (if different files are different), hashes or even content (for small files) or file headers (first n bytes). However this seems to have a problem: all_files: a b c d e ... If a==b then comparing b with c,d,e is useless. May be using several steps with dict - sizes, then hashes for same sizes files, etc ... Another solution I thought of, could be defining some methods (I still don't know which ones) in MyFile so that I could use sets intersection. Would this one be a faster solution? Thanks From python.list at tim.thechases.com Sun Feb 7 19:20:50 2016 From: python.list at tim.thechases.com (Tim Chase) Date: Sun, 7 Feb 2016 18:20:50 -0600 Subject: A sets algorithm In-Reply-To: References: Message-ID: <20160207182050.03d1488a@bigbox.christie.dr> On 2016-02-08 00:05, Paulo da Silva wrote: > ?s 22:17 de 07-02-2016, Tim Chase escreveu: >> all_files = list(generate_MyFile_objects()) >> interesting = [ >> (my_file1, my_file2) >> for i, my_file1 >> in enumerate(all_files, 1) >> for my_file2 >> in all_files[i:] >> if my_file1 == my_file2 >> ] > > "my_file1 == my_file2" can be implemented into MyFile class taking > advantage of caching sizes (if different files are different), > hashes or even content (for small files) or file headers (first n > bytes). However this seems to have a problem: > all_files: a b c d e ... > If a==b then comparing b with c,d,e is useless. Depends on what the OP wants to have happen if more than one input file is equal. I.e., a == b == c. Does one just want "a has duplicates" (and optionally "and here's one of them"), or does one want "a == b", "a == c" and "b == c" in the output? > Another solution I thought of, could be defining some methods (I > still don't know which ones) in MyFile so that I could use sets > intersection. Would this one be a faster solution? Adding __hash__ would allow for the set operations, but would require (as ChrisA points out) knowing how to create a hash function that encompasses the information you want to compare. -tkc From dimva13 at gmail.com Sun Feb 7 19:38:43 2016 From: dimva13 at gmail.com (dimva13 at gmail.com) Date: Sun, 7 Feb 2016 16:38:43 -0800 (PST) Subject: Python's import situation has driven me to the brink of imsanity In-Reply-To: References: <51b52cf3-0b0a-451e-a232-d9a671f7cd20@googlegroups.com> Message-ID: <14095c33-b5d4-419f-aaf4-e6f293d0cbf9@googlegroups.com> Running python setup.py develop doesn't work, it gives me this error: error: invalid command 'develop' Running pip install -e . does work. This is somewhat reasonable, and I think may work for my purposes, but it has the following drawbacks: - you have to create a fake package with a fake version, and all the setup.py boilerplate - to get to your python files, you have to go through an additional subdirectory (ex: project_name/project_name/package.py) - you have to remember to run pip install -e . before you can use anything inside the project - you should be using a virtualenv (I am), otherwise things can get messy if you have these fake projects installed in a global location All in all, this seems like a hack - I'm creating a fake package with a fake version just so I can do imports sanely. Still less of a hack than imsanity, though. Thanks for the advice! I just wish it was better documented and easier to find. I have several years of professional python experience at 3 different companies, and I've never seen this solution anywhere - all the companies solved it by setting PYTHONPATH to the base directory. I'll try to spread the word. On Sunday, February 7, 2016 at 11:09:35 AM UTC-5, Kevin Conway wrote: > You can use 'setup.py develop' or 'pip install -e' to install your package > in editable mode. It makes it so your local code is used. Modifications are > seen immediately. > > On Sun, Feb 7, 2016, 08:16 wrote: > > > I see that this would work once you've installed the package, but how do > > you develop it? Say you are working on a change that modifies both email.py > > and reports.py. Do you run setup.py every time you make a change in > > email.py? > > > > On Sunday, February 7, 2016 at 1:35:15 AM UTC-5, Kevin Conway wrote: > > > > My question is: is this crazy? Please tell me there's a better way and > > I > > > just wasted my time creating this package. > > > > > > There is a better way and you have wasted your time creating this > > package. > > > > > > I hear your problem statement as asking two questions. The first is: What > > > is the right way to include executable content in my Python project? The > > > second is: How do I expose executable content from a Python project? > > > > > > As to the first question, from your project README: > > > > Say you have a python project (not a package), with the following > > > structure: > > > > > > All Python code that you want to install and make available in any form, > > > import or executable, _must_ be contained within a Python package. > > > Organizing Python code in any way other than Python packages will result > > in > > > the challenges you have described. The correct way to include executable > > > content is to place the Python code within the package structure. It > > should > > > not be put in other directories within the repository root. > > > > > > As to the second question, once all Python code is contained within a > > > package that can be installed you can use setuptools entry points to > > expose > > > the executable code. The setup() function from setuptools that is used to > > > create setup.py files has an argument called 'entry_points' that allows > > you > > > to expose executable content over the command line. See [1] and [2] for > > > more details. > > > > > > Feel free to reach out to me off-list if you have a specific project you > > > need advice on. The rules for organizing and packaging Python code aren't > > > complex but they tend to cause new Python developers to stumble at > > first. A > > > general rule I give everyone when talking about packaging or importing > > > code: If you have to modify sys.path to makes something work then you > > have > > > most certainly made a mistake. > > > > > > [1] > > > > > https://pythonhosted.org/setuptools/setuptools.html#automatic-script-creation > > > [2] > > > > > http://python-packaging.readthedocs.org/en/latest/command-line-scripts.html#the-console-scripts-entry-point > > > > > > > > > On Sat, Feb 6, 2016 at 8:54 PM Chris Angelico wrote: > > > > > > > On Sun, Feb 7, 2016 at 1:47 PM, wrote: > > > > > Imsanity allows you to make imports usable (not ideal, but at least > > > > usable) for python projects without having to manage PYTHONPATHs or do > > > > whacky stuff like running files with python -m or put even whackier > > > > boilerplate at the top of every file. And all it requires is 'import > > > > imsanity' at the top of every file. You can put it in a macro or even > > just > > > > type it because it's short and easy to remember. > > > > > > > > > > My question is: is this crazy? Please tell me there's a better way > > and I > > > > just wasted my time creating this package. There's nothing I'd like to > > hear > > > > more. > > > > > > > > Well, anything that makes you type "import imsanity" at the top of > > > > every script MUST be crazy. :) I don't know about the actual > > > > content/purpose though. Good luck with it! > > > > > > > > ChrisA > > > > -- > > > > https://mail.python.org/mailman/listinfo/python-list > > > > > > > > -- > > https://mail.python.org/mailman/listinfo/python-list > > From cfkaran2 at gmail.com Sun Feb 7 20:07:40 2016 From: cfkaran2 at gmail.com (Cem Karan) Date: Sun, 7 Feb 2016 20:07:40 -0500 Subject: A sets algorithm In-Reply-To: References: Message-ID: On Feb 7, 2016, at 4:46 PM, Paulo da Silva wrote: > Hello! > > This may not be a strict python question, but ... > > Suppose I have already a class MyFile that has an efficient method (or > operator) to compare two MyFile s for equality. > > What is the most efficient way to obtain all sets of equal files (of > course each set must have more than one file - all single files are > discarded)? > > Thanks for any suggestions. If you're after strict equality (every byte in a pair of files is identical), then here are a few heuristics that may help you: 1) Test for file length, without reading in the whole file. You can use os.path.getsize() to do this (I hope that this is a constant-time operation, but I haven't tested it). As Oscar Benjamin suggested, you can create a defaultdict(list) which will make it possible to gather lists of files of equal size. This should help you gather your potentially identical files quickly. 2) Once you have your dictionary from above, you can iterate its values, each of which will be a list. If a list has only one file in it, you know its unique, and you don't have to do any more work on it. If there are two files in the list, then you have several different options: a) Use Chris Angelico's suggestion and hash each of the files (use the standard library's 'hashlib' for this). Identical files will always have identical hashes, but there may be false positives, so you'll need to verify that files that have identical hashes are indeed identical. b) If your files tend to have sections that are very different (e.g., the first 32 bytes tend to be different), then you pretend that section of the file is its hash. You can then do the same trick as above. (the advantage of this is that you will read in a lot less data than if you have to hash the entire file). c) You may be able to do something clever by reading portions of each file. That is, use zip() combined with read(1024) to read each of the files in sections, while keeping hashes of the files. Or, maybe you'll be able to read portions of them and sort the list as you're reading. In either case, if any files are NOT identical, then you'll be able to stop work as soon as you figure this out, rather than having to read the entire file at once. The main purpose of these suggestions is to reduce the amount of reading you're doing. Storage tends to be slow, and any tricks that reduce the number of bytes you need to read in will be helpful to you. Good luck! Cem Karan From cfkaran2 at gmail.com Sun Feb 7 20:37:00 2016 From: cfkaran2 at gmail.com (Cem Karan) Date: Sun, 7 Feb 2016 20:37:00 -0500 Subject: Heap Implementation In-Reply-To: <56AD3D83.2050308@mail.de> References: <56AD3D83.2050308@mail.de> Message-ID: <7C522D08-9D73-48D2-A71D-F1D1D34C02A5@gmail.com> On Jan 30, 2016, at 5:47 PM, Sven R. Kunze wrote: > Hi again, > > as the topic of the old thread actually was fully discussed, I dare to open a new one. > > I finally managed to finish my heap implementation. You can find it at https://pypi.python.org/pypi/xheap + https://github.com/srkunze/xheap. > > I described my motivations and design decisions at http://srkunze.blogspot.com/2016/01/fast-object-oriented-heap-implementation.html > > @Cem > You've been worried about a C implementation. I can assure you that I did not intend to rewrite the incredibly fast and well-tested heapq implementation. I just re-used it. > > I would really be grateful for your feedback as you have first-hand experience with heaps. <> My apologies for not writing sooner, but work has been quite busy lately (and likely will be for some time to come). I read your approach, and it looks pretty good, but there may be one issue with it; how do you handle the same item being pushed into the heap more than once? In my simple simulator, I'll push the same object into my event queue multiple times in a row. The priority is the moment in the future when the object will be called. As a result, items don't have unique priorities. I know that there are methods of handling this from the client-side (tuples with unique counters come to mind), but if your library can handle it directly, then that could be useful to others as well. Thanks, Cem Karan From p_s_d_a_s_i_l_v_a_ns at netcabo.pt Sun Feb 7 21:22:53 2016 From: p_s_d_a_s_i_l_v_a_ns at netcabo.pt (Paulo da Silva) Date: Mon, 8 Feb 2016 02:22:53 +0000 Subject: A sets algorithm References: Message-ID: ?s 21:46 de 07-02-2016, Paulo da Silva escreveu: > Hello! > > This may not be a strict python question, but ... > > Suppose I have already a class MyFile that has an efficient method (or > operator) to compare two MyFile s for equality. > > What is the most efficient way to obtain all sets of equal files (of > course each set must have more than one file - all single files are > discarded)? > After reading all suggestions I decided to try first the defaultdict(list), as first suggested by Oscar, in several steps. First with sizes and then with other partial contents or/and "strong" hashes as suggested by Cem. Thank you very much to all who responded for all helpful suggestions. If I find something better I'll report here. Paulo From mr.eightnoteight at gmail.com Sun Feb 7 22:15:51 2016 From: mr.eightnoteight at gmail.com (srinivas devaki) Date: Mon, 8 Feb 2016 08:45:51 +0530 Subject: Heap Implementation In-Reply-To: <7C522D08-9D73-48D2-A71D-F1D1D34C02A5@gmail.com> References: <56AD3D83.2050308@mail.de> <7C522D08-9D73-48D2-A71D-F1D1D34C02A5@gmail.com> Message-ID: On Feb 8, 2016 7:07 AM, "Cem Karan" wrote: > > > > I know that there are methods of handling this from the client-side (tuples with unique counters come to mind), but if your library can handle it directly, then that could be useful to others as well. yeah it is a good idea to do at client side. but if it should be introduced as feature into the library, instead of tuples, we should just piggyback a single counter it to the self._indexes dict, or better make another self._counts dict which will be light and fast. and if you think again with this method you can easily subclass with just using self._counts dict in your subclass. but still I think it is good to introduce it as a feature in the library. Regards Srinivas Devaki Junior (3rd yr) student at Indian School of Mines,(IIT Dhanbad) Computer Science and Engineering Department ph: +91 9491 383 249 telegram_id: @eightnoteight From shaunak.bangale at gmail.com Mon Feb 8 03:04:24 2016 From: shaunak.bangale at gmail.com (shaunak.bangale at gmail.com) Date: Mon, 8 Feb 2016 00:04:24 -0800 (PST) Subject: Exception handling for socket.error in Python 3.5/RStudio In-Reply-To: References: <1067e50c-bcbd-464e-8013-754bd66fda91@googlegroups.com> Message-ID: <0e2fc62a-a4b8-4962-8737-b9cfdbafefc4@googlegroups.com> On Sunday, February 7, 2016 at 1:23:32 AM UTC-7, dieter wrote: > Shaunak Bangale writes: > > ... > > while (count > 0): > > try : > > # read line from file: > > print(file.readline()) > > # parse > > parse_json(file.readline()) > > count = count - 1 > > except socket.error as e: > > print('Connection fail', e) > > print(traceback.format_exc()) > > ... > > Error: > > except socket.error as e: > > ^ > > SyntaxError: invalid syntax > > Are you sure, that there is no invisible character at the end > of that line? > > When I try code like the above (in PYthon 2.7), there is no > "SyntaxError": > > >>> while(False): > ... try : > ... # read line from file: > ... print(file.readline()) > ... # parse > ... parse_json(file.readline()) > ... count = count - 1 > ... except socket.error as e: > ... print('Connection fail', e) > ... print(traceback.format_exc()) > ... > >>> > > > Another reason for your problem could be an older Python version. > The "as" construct as part of the "except" clause is a more recent > addition to Python. Hi Dieter, I typed that line again but didn't help. I am using Python 3.5. From mscir at yahoo.com Mon Feb 8 03:19:23 2016 From: mscir at yahoo.com (Mike S) Date: Mon, 8 Feb 2016 00:19:23 -0800 Subject: Install Error In-Reply-To: References: Message-ID: On 2/3/2016 1:55 PM, Barrie Taylor wrote: > Hi, > > I am attempting to install and run Python3.5.1 on my Windows machine. > > After installation on launching I am presented the attached error message. > It reads: > 'The program can't start because api-ms-win-crt-runtime-l1-1-0.dll is missing from your computer. Try reinstalling the program to fix this problem.' > Needless to say, I have tried uninstalling and reinstalling ad repairing the program a number of times to no avail. > > Any help with this issue would be greatly appreciated. > > Kind Regards, > > Barrie Taylor What OS? On my XP machine I got 3.4.4. to work, although I couldn't figure out how to get IPython notebook to work, so I'm focusing on getting that to work on my w7 and w10 boxes. Idk if you need Ipython. From mscir at yahoo.com Mon Feb 8 03:27:39 2016 From: mscir at yahoo.com (Mike S) Date: Mon, 8 Feb 2016 00:27:39 -0800 Subject: setup failed In-Reply-To: References: Message-ID: On 2/4/2016 4:39 AM, Prince Thomas wrote: > Hi > I am an computer science engineer. I downloaded the python version 3.5.1.amd64 and just python 3.5.1. > The problem is when I install the program setup is failed and showing 0*80070570-The file or directory is > corrupted and unreadable. I install the newest visual c++ redist and still same. My os is win 8.1. > Please help me out of this This sounds similar. Did you install the 32 bit or 64 bit version? If you installed the 32 bit version and don't object to using the 64bit version I would try installing that. https://mail.python.org/pipermail/python-list/2015-September/697456.html From vek.m1234 at gmail.com Mon Feb 8 04:17:02 2016 From: vek.m1234 at gmail.com (Veek. M) Date: Mon, 08 Feb 2016 14:47:02 +0530 Subject: coroutine, throw, yield, call-stack and exception handling Message-ID: **************************** Exceptions can be raised inside a coroutine using the throw( Exceptions raised in this manner will originate at the currently executing yield state-ment in the coroutine.A coroutine can elect to catch exceptions and handle them as appropriate. It is not safe to use throw() as an asynchronous signal to a coroutine?it should never be invoked from a separate execution thread or in a signal handler. **************************** What does Beazley mean by this: 'will originate at the currently executing yield state-ment in the coroutine' If he's throw'ing an exception surely it originates at the throw: def mycoroutine(): while len(n) > 2: n = (yield) throw('RuntimeError' "die!") ---------------------- Also: 'not safe to use throw() as an asynchronous signal to a coroutine? it should never be invoked from a separate execution thread or in a signal handler.' You can use throw within a coroutine to raise an exception. How would you use it as an async-sig to a coroutine.. eg: you have two threads 1. coroutine does except FooException: 2. throw(FooException, 'message') so moment 'throw' runs and an exception is raised.. it'll propagate within thread-2 to its parent etc - how is thread-1 affected? From vek.m1234 at gmail.com Mon Feb 8 04:34:55 2016 From: vek.m1234 at gmail.com (Veek. M) Date: Mon, 08 Feb 2016 15:04:55 +0530 Subject: coroutine, throw, yield, call-stack and exception handling References: Message-ID: Veek. M wrote: > **************************** > Exceptions can be raised inside a coroutine using the throw( > > Exceptions raised in this manner will originate at the currently > executing yield state-ment in the coroutine.A coroutine can elect to > catch exceptions and handle them as appropriate. It is not safe to use > throw() as an asynchronous signal to a coroutine?it should never be > invoked from a separate execution thread or in a signal handler. > **************************** > > What does Beazley mean by this: 'will originate at the currently > executing yield state-ment in the coroutine' > > If he's throw'ing an exception surely it originates at the throw: > > def mycoroutine(): > while len(n) > 2: > n = (yield) > > throw('RuntimeError' "die!") > ---------------------- > Also: 'not safe to use throw() as an asynchronous signal to a > coroutine? it should never be invoked from a separate execution thread > or in a signal handler.' > > You can use throw within a coroutine to raise an exception. > How would you use it as an async-sig to a coroutine.. > eg: you have two threads > 1. coroutine does except FooException: > 2. throw(FooException, 'message') > > so moment 'throw' runs and an exception is raised.. it'll propagate > within thread-2 to its parent etc - how is thread-1 affected? Veek. M wrote: > **************************** > Exceptions can be raised inside a coroutine using the throw( > > Exceptions raised in this manner will originate at the currently > executing yield state-ment in the coroutine.A coroutine can elect to > catch exceptions and handle them as appropriate. It is not safe to use > throw() as an asynchronous signal to a coroutine?it should never be > invoked from a separate execution thread or in a signal handler. > **************************** > > What does Beazley mean by this: 'will originate at the currently > executing yield state-ment in the coroutine' > > If he's throw'ing an exception surely it originates at the throw: > > def mycoroutine(): > while len(n) > 2: > n = (yield) > > throw('RuntimeError' "die!") > ---------------------- > Also: 'not safe to use throw() as an asynchronous signal to a > coroutine? it should never be invoked from a separate execution thread > or in a signal handler.' > > You can use throw within a coroutine to raise an exception. > How would you use it as an async-sig to a coroutine.. > eg: you have two threads > 1. coroutine does except FooException: > 2. throw(FooException, 'message') > > so moment 'throw' runs and an exception is raised.. it'll propagate > within thread-2 to its parent etc - how is thread-1 affected? Also this bit: *********************** If a coroutine returns values, some care is required if exceptions raised with throw() are being handled. If you raise an exception in a coroutine using throw(), the value passed to the next yield in the coroutine will be returned as the result of throw(). If you need this value and forget to save it, it will be lost. *********************** def coroutine(): while True: line = (yield result) throw(FooException) where is the question of a 'yield'? You'll exit the coroutine straight away.. From marco.kaulea at gmail.com Mon Feb 8 04:56:43 2016 From: marco.kaulea at gmail.com (Marco Kaulea) Date: Mon, 8 Feb 2016 10:56:43 +0100 Subject: Set Operations on Dicts Message-ID: Hi, In one talk (I think it was [1]) it was described that sets are basically dicts without the values. Therefor it should be easy to apply set operations on dicts, for example: {'a': 123, 'b': 456} & {'a'} => {'a': 123} {'a': 123, 'b': 456} - {'a'} => {'b': 456} This if currently not implemented. Is there a technical reason that this is difficult or are there a lot of corner cases that make it not worth the trouble? I have not spend a lot of time on this, I just needed a feature like that and was surprised that it did not exists. - Marco [1] https://www.youtube.com/watch?v=C4Kc8xzcA68 From cl at isbd.net Mon Feb 8 05:22:22 2016 From: cl at isbd.net (cl at isbd.net) Date: Mon, 8 Feb 2016 10:22:22 +0000 Subject: What's the best/neatest way to get Unicode data from a database into a grid cell? References: Message-ID: Dietmar Schwertberger wrote: > On 07.02.2016 12:19, cl at isbd.net wrote: > > However my database has quite a lot of Unicode data as there are > > French (and other) names with accents etc. What's the right way to > > handle this reasonably neatly? At the moment it traps an error at > > line 37:- > > self.SetCellValue(row_num, i, str(cells[i])) > The unicode versions of wxPython should have no problems with handling > unicode strings. > The problem that you see here is that str(...) tries to convert your > unicode data into a non-unicode string, which of course fails. > Do something like: > > value = cells[i] > if not isinstance(value, basestring): > value = str(value) > self.SetCellValue(row_num, i, value) > Thanks, worked perfectly! I had been trying to produce something similar in tkinter but grids are a bit clumsy in tkinter. This wx implementation will do me nicely. > Once you switch to Python 3 and Phoenix you have to modify this > slightly, e.g. by adding this to the top of your code: > > try: > basestring > except: > basestring = (bytes,str) > Everything else is 3 compatible so moving should be fairly painless if/when phoenix becomes available. -- Chris Green ? From cl at isbd.net Mon Feb 8 05:42:11 2016 From: cl at isbd.net (cl at isbd.net) Date: Mon, 8 Feb 2016 10:42:11 +0000 Subject: What's the best/neatest way to get Unicode data from a database into a grid cell? References: Message-ID: <318moc-ra4.ln1@esprimo.zbmc.eu> Vlastimil Brom wrote: > > Hi, > your code in > http://www.salstat.com/news/linking-wxgrid-to-sqlite-database-in-python-an-example > > seems to work for me after small changes with both python 2.7 and 3.4 > (using wx Phoenix) Where are you getting phoenix from? It's not in the Ubuntu repositories and when I search on Google I seem to go in ever decreasing circles and always end up finding wxPython for 2.7. Did you build it from source yourself (I can do that but only if it's necessary). > the changes I made are: > - encoding declaration at the beginning of the file (mainly for py 2, > if utf-8 is used): > > #! Python > # -*- coding: utf-8 -*- > > - removing the str(...) call in the offending line you mentioned: > self.SetCellValue(row_num, i, cells[i]) > (what was the intent of the conversion to str?) > I've no idea what the intent is/was, I just copied the code as I found it. I guess the cell value could be numeric. > - corrected event name (insted of EVT_GRID_CELL_CHANGE) > self.Bind(gridlib.EVT_GRID_CELL_CHANGED, self.CellContentsChanged) > > - normal App() call (instead of the PySimpleApp) > > app = wx.App() > > I randomly tried some characters using Latin, Greek etc. characters > (within BMP) as well as Gothic - beyond the FFFF range - the cell > content seems to be saved and and newly loaded from the sqlite db on > subsequent calls of the app. > Thanks for your help, very useful. -- Chris Green ? From list at qtrac.plus.com Mon Feb 8 05:50:03 2016 From: list at qtrac.plus.com (Mark Summerfield) Date: Mon, 8 Feb 2016 02:50:03 -0800 (PST) Subject: setup failed In-Reply-To: References: Message-ID: <7f3af18b-fcad-4da5-8b01-231c4c33bcc4@googlegroups.com> If you need 32-bit Python on Windows my advice is to install 3.4. If you need 32-bit and 64-bit Python on Windows, then I think it will only work with 3.4 (or older), but not with 3.5's new installer. I have tried installing 3.5.0 and 3.5.1 on several machines both 32- and 64-bit Windows. The 32-bit installer either doesn't work at all or won't work with two third-party packages I use: APSW and PyWin32. The 64-bit installer does seem to work on 64-bit machines, but in my particular case I need both 32- and 64-bit versions so use Python 3.4. Hopefully the problem will be fixed before 3.4's end of life:-) From rosuav at gmail.com Mon Feb 8 05:55:02 2016 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 8 Feb 2016 21:55:02 +1100 Subject: What's the best/neatest way to get Unicode data from a database into a grid cell? In-Reply-To: <318moc-ra4.ln1@esprimo.zbmc.eu> References: <318moc-ra4.ln1@esprimo.zbmc.eu> Message-ID: On Mon, Feb 8, 2016 at 9:42 PM, wrote: > Vlastimil Brom wrote: >> >> Hi, >> your code in >> http://www.salstat.com/news/linking-wxgrid-to-sqlite-database-in-python-an-example >> >> seems to work for me after small changes with both python 2.7 and 3.4 >> (using wx Phoenix) > > Where are you getting phoenix from? It's not in the Ubuntu > repositories and when I search on Google I seem to go in ever > decreasing circles and always end up finding wxPython for 2.7. > > Did you build it from source yourself (I can do that but only if it's > necessary). > > http://wiki.wxpython.org/ProjectPhoenix has links to the source code and to daily snapshots. I suspect the easiest way to get it will be from source. ChrisA From rosuav at gmail.com Mon Feb 8 05:57:29 2016 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 8 Feb 2016 21:57:29 +1100 Subject: What's the best/neatest way to get Unicode data from a database into a grid cell? In-Reply-To: References: Message-ID: On Mon, Feb 8, 2016 at 9:22 PM, wrote: >> Once you switch to Python 3 and Phoenix you have to modify this >> slightly, e.g. by adding this to the top of your code: >> >> try: >> basestring >> except: >> basestring = (bytes,str) >> > Everything else is 3 compatible so moving should be fairly painless > if/when phoenix becomes available. Small point: Even for code as small as this, I would be inclined to catch only the one exception you care about - in this case, NameError. I try to avoid having a bare except clause anywhere other than a "log and continue" or "react and reraise" kind of situation. ChrisA From list at qtrac.plus.com Mon Feb 8 06:02:01 2016 From: list at qtrac.plus.com (Mark Summerfield) Date: Mon, 8 Feb 2016 03:02:01 -0800 (PST) Subject: setup failed In-Reply-To: <7f3af18b-fcad-4da5-8b01-231c4c33bcc4@googlegroups.com> References: <7f3af18b-fcad-4da5-8b01-231c4c33bcc4@googlegroups.com> Message-ID: On Monday, February 8, 2016 at 10:50:19 AM UTC, Mark Summerfield wrote: > If you need 32-bit Python on Windows my advice is to install 3.4. If you need 32-bit and 64-bit Python on Windows, then I think it will only work with 3.4 (or older), but not with 3.5's new installer. > > I have tried installing 3.5.0 and 3.5.1 on several machines both 32- and 64-bit Windows. The 32-bit installer either doesn't work at all or won't work with two third-party packages I use: APSW and PyWin32. The 64-bit installer does seem to work on 64-bit machines, but in my particular case I need both 32- and 64-bit versions so use Python 3.4. Hopefully the problem will be fixed before 3.4's end of life:-) Ooops, of course you're already using the 64-bit installer. My advice is use Python 3.4 if you need to work on Windows: unlike the 3.5 installer I've never encountered a problem with the 3.4 (or 3.3) installers. From egarrulo at gmail.com Mon Feb 8 06:29:39 2016 From: egarrulo at gmail.com (egarrulo at gmail.com) Date: Mon, 8 Feb 2016 03:29:39 -0800 (PST) Subject: Changing logging level only for my code? Message-ID: <0fb0145c-2331-44ab-bf58-211aa2083ca6@googlegroups.com> I am using the "logging" module for my own package, but changing the level from "INFO" to "DEBUG" enables debugging statements from third-party libraries as well. How can I avoid them? Here is the code I am using: import logging logger = logging.getLogger(__name__) log_file = logging.FileHandler("activity.log", mode='w') log_file.setFormatter(logging.Formatter(fmt='%(levelname)s: %(message)s')) logger.addHandler(log_file) logging.basicConfig(level=logging.INFO) # or DEBUG Thank you. From cfkaran2 at gmail.com Mon Feb 8 06:47:57 2016 From: cfkaran2 at gmail.com (Cem Karan) Date: Mon, 8 Feb 2016 06:47:57 -0500 Subject: Heap Implementation In-Reply-To: References: <56AD3D83.2050308@mail.de> <7C522D08-9D73-48D2-A71D-F1D1D34C02A5@gmail.com> Message-ID: <185DAEA4-8728-4792-A3B7-7F6AC5A7F876@gmail.com> On Feb 7, 2016, at 10:15 PM, srinivas devaki wrote: > On Feb 8, 2016 7:07 AM, "Cem Karan" wrote: > > I know that there are methods of handling this from the client-side (tuples with unique counters come to mind), but if your library can handle it directly, then that could be useful to others as well. > > yeah it is a good idea to do at client side. > but if it should be introduced as feature into the library, instead of tuples, we should just piggyback a single counter it to the self._indexes dict, or better make another self._counts dict which will be light and fast. > and if you think again with this method you can easily subclass with just using self._counts dict in your subclass. but still I think it is good to introduce it as a feature in the library. > > Regards > Srinivas Devaki Just to be 100% sure, you do mean to use the counters as UUIDs, right? I don't mean that the elements in the heap get counted, I meant that the counter is a trick to separate different instances of (item, priority) pairs when you're pushing in the same item multiple times, but with different priorities. Thanks, Cem Karan From jeanmichel at sequans.com Mon Feb 8 07:07:48 2016 From: jeanmichel at sequans.com (jmp) Date: Mon, 08 Feb 2016 13:07:48 +0100 Subject: Changing logging level only for my code? In-Reply-To: <0fb0145c-2331-44ab-bf58-211aa2083ca6@googlegroups.com> References: <0fb0145c-2331-44ab-bf58-211aa2083ca6@googlegroups.com> Message-ID: On 02/08/2016 12:29 PM, egarrulo at gmail.com wrote: > I am using the "logging" module for my own package, but changing the level from "INFO" to "DEBUG" enables debugging statements from third-party libraries as well. How can I avoid them? Here is the code I am using: > > import logging > > logger = logging.getLogger(__name__) > log_file = logging.FileHandler("activity.log", mode='w') > log_file.setFormatter(logging.Formatter(fmt='%(levelname)s: %(message)s')) > logger.addHandler(log_file) > logging.basicConfig(level=logging.INFO) # or DEBUG > > Thank you. > Hi, basicConfig will configure the *root* logger, the parent of all loggers. You need to set it to DEBUG, otherwise no DEBUG logs will be displayed. However you can set levels to loggers independently, and you simply need to configure the third party logger to whatever level you want. logging.basicConfig(level=logging.DEBUG) logging.getLogger("yourthirdpartyloggername").setLevel(logging.INFO) JM From __peter__ at web.de Mon Feb 8 07:12:28 2016 From: __peter__ at web.de (Peter Otten) Date: Mon, 08 Feb 2016 13:12:28 +0100 Subject: Changing logging level only for my code? References: <0fb0145c-2331-44ab-bf58-211aa2083ca6@googlegroups.com> Message-ID: egarrulo at gmail.com wrote: > I am using the "logging" module for my own package, but changing the level > from "INFO" to "DEBUG" enables debugging statements from third-party > libraries as well. How can I avoid them? Here is the code I am using: > > import logging > > logger = logging.getLogger(__name__) > log_file = logging.FileHandler("activity.log", mode='w') > log_file.setFormatter(logging.Formatter(fmt='%(levelname)s: > %(message)s')) logger.addHandler(log_file) > logging.basicConfig(level=logging.INFO) # or DEBUG You can set the level for your logger, e. g. $ cat loglevels.py import logging logging.basicConfig( level=logging.INFO, filename="activity.log", format='%(levelname)s: %(message)s') mylogger = logging.getLogger(__name__) mylogger.setLevel(logging.DEBUG) mylogger.info("my info") mylogger.debug("my debug message") otherlogger = logging.getLogger("some.lib") otherlogger.info("some.lib info") otherlogger.debug("some.lib debug message") $ python3 loglevels.py $ cat activity.log INFO: my info DEBUG: my debug message INFO: some.lib info From jenswaelkens at gmail.com Mon Feb 8 07:15:18 2016 From: jenswaelkens at gmail.com (jenswaelkens at gmail.com) Date: Mon, 8 Feb 2016 04:15:18 -0800 (PST) Subject: [newbie] problem with geometry setting in Tkinter Message-ID: <31e9b382-bc20-4283-b17c-23a97c8b6083@googlegroups.com> I'm trying to set the geometry of my top window, but the size is unaffected. This is the code: #!/usr/bin/env python import Tkinter top=Tkinter.Tk() top.geometry=('900x460') top.update_idletasks() print (top.winfo_width()) print (top.winfo_height()) print (top.winfo_geometry()) top.mainloop() and this is the result when running the code: 200 200 200x200+1+584 Can anyone here tell me what I am doing wrong and how to change it? Thanks Jens From jussi.piitulainen at helsinki.fi Mon Feb 8 07:17:36 2016 From: jussi.piitulainen at helsinki.fi (Jussi Piitulainen) Date: Mon, 08 Feb 2016 14:17:36 +0200 Subject: Set Operations on Dicts References: Message-ID: Marco Kaulea writes: > In one talk (I think it was [1]) it was described that sets are > basically dicts without the values. Therefor it should be easy to > apply set operations on dicts, for example: > > {'a': 123, 'b': 456} & {'a'} => {'a': 123} > {'a': 123, 'b': 456} - {'a'} => {'b': 456} > > This if currently not implemented. Is there a technical reason that > this is difficult or are there a lot of corner cases that make it not > worth the trouble? I think nobody was quite willing to lay down the law on which dictionary would take precedence when they have keys in common but different values on those keys. Both ways make sense, and sometimes you want something like arithmetic done to combine the values on common keys. (A collection.Counter does some version of the latter, I think. Maybe the answer is to create a custom class that allows what you want?) But that's an interesting proposal to only allow sets as the second argument. Those particular cases may not be *too* difficult to express as comprehensions, though still quite a mouthful compared to your suggestion: { k:d[k] for k in d if k in s } # d & s { k:d[k] for k in d if k not in s } # d - s Perhaps there is already some more compact expression for these? Also, what would be the nicest current way to express a priority union of dicts? { k:(d if k in d else e)[k] for k in d.keys() | e.keys() } > I have not spend a lot of time on this, I just needed a feature like > that and was surprised that it did not exists. > > - Marco > > [1] https://www.youtube.com/watch?v=C4Kc8xzcA68 From stephane at wirtel.be Mon Feb 8 07:18:28 2016 From: stephane at wirtel.be (Stephane Wirtel) Date: Mon, 08 Feb 2016 13:18:28 +0100 Subject: New mailing for Python in Belgium Message-ID: Hello everyone, I would like to announce a new mailing list for the Belgian Python Users. This main goal of this mailing list is mainly for the Python Community of Belgium, Dutch, French and German. The main language is English, I don?t want to discuss about the languages, maybe in the future. So, here is the mailing list: belgium at python.org you can subscribe via this link: https://mail.python.org/mailman/listinfo/belgium Via this mailing list, you could discuss about: * Python in Belgium * Organise some events, meetups, AFPyro in Belgium * Create a great community. * Help for the PythonFOSDEM in Belgium * etc? Hope to see you in this ML. Thank you so much, Stephane From __peter__ at web.de Mon Feb 8 07:26:31 2016 From: __peter__ at web.de (Peter Otten) Date: Mon, 08 Feb 2016 13:26:31 +0100 Subject: [newbie] problem with geometry setting in Tkinter References: <31e9b382-bc20-4283-b17c-23a97c8b6083@googlegroups.com> Message-ID: jenswaelkens at gmail.com wrote: > I'm trying to set the geometry of my top window, but the size is > unaffected. > This is the code: > > #!/usr/bin/env python > import Tkinter > top=Tkinter.Tk() > top.geometry=('900x460') That's an assignment, but geometry() is a method that you have to invoke: top.geometry('900x460') > top.update_idletasks() > print (top.winfo_width()) > print (top.winfo_height()) > print (top.winfo_geometry()) > top.mainloop() > > and this is the result when running the code: > 200 > 200 > 200x200+1+584 > > > Can anyone here tell me what I am doing wrong and how to change it? > Thanks > > Jens From marco.kaulea at gmail.com Mon Feb 8 07:33:56 2016 From: marco.kaulea at gmail.com (Marco Kaulea) Date: Mon, 8 Feb 2016 13:33:56 +0100 Subject: Set Operations on Dicts In-Reply-To: References: Message-ID: On Mon, Feb 8, 2016 at 1:17 PM, Jussi Piitulainen < jussi.piitulainen at helsinki.fi> wrote: > I think nobody was quite willing to lay down the law on which dictionary > would take precedence when they have keys in common but different values > on those keys. Both ways make sense, and sometimes you want something > like arithmetic done to combine the values on common keys. > I don't expect this to be really useful. I only thought about using a set as a second argument. But that's an interesting proposal to only allow sets as the second > argument. Those particular cases may not be *too* difficult to express > as comprehensions, though still quite a mouthful compared to your > suggestion: > That is the restriction I had in mind. { k:d[k] for k in d if k in s } # d & s > > { k:d[k] for k in d if k not in s } # d - s > > That is basically what I did. But I expect this could be quite slow, since it has to take each value in s and perform a lookup in d. I would expect the pure set implementation to be more optimized. Also, what would be the nicest current way to express a priority union > of dicts? > > { k:(d if k in d else e)[k] for k in d.keys() | e.keys() } This seems like it might be useful for default configurations, as that is currently quite the hassle to do with `x = conf.get('x', fallback="default") From snailcoder at retrosite.invalid Mon Feb 8 07:47:53 2016 From: snailcoder at retrosite.invalid (Grobu) Date: Mon, 8 Feb 2016 13:47:53 +0100 Subject: Set Operations on Dicts In-Reply-To: References: Message-ID: You can use dictionary comprehension : Say : dict1 = {'a': 123, 'b': 456} set1 = {'a'} intersection : >>> { key:dict1[key] for key in dict1 if key in set1 } {'a': 123} difference : >>> { key:dict1[key] for key in dict1 if not key in set1 } {'b': 456} From m at funkyhat.org Mon Feb 8 08:32:36 2016 From: m at funkyhat.org (Matt Wheeler) Date: Mon, 8 Feb 2016 13:32:36 +0000 Subject: Set Operations on Dicts In-Reply-To: References: Message-ID: On 8 February 2016 at 12:17, Jussi Piitulainen wrote: > Also, what would be the nicest current way to express a priority union > of dicts? > > { k:(d if k in d else e)[k] for k in d.keys() | e.keys() } Since Python 3.5: {**e, **d} -- Matt Wheeler http://funkyh.at From jussi.piitulainen at helsinki.fi Mon Feb 8 09:02:48 2016 From: jussi.piitulainen at helsinki.fi (Jussi Piitulainen) Date: Mon, 08 Feb 2016 16:02:48 +0200 Subject: Set Operations on Dicts References: Message-ID: Matt Wheeler writes: > On 8 February 2016 at 12:17, Jussi Piitulainen wrote: >> Also, what would be the nicest current way to express a priority union >> of dicts? >> >> { k:(d if k in d else e)[k] for k in d.keys() | e.keys() } > > Since Python 3.5: {**e, **d} Thanks. I have considered this news briefly and I think I'll like it :) From jenswaelkens at gmail.com Mon Feb 8 09:34:59 2016 From: jenswaelkens at gmail.com (jenswaelkens at gmail.com) Date: Mon, 8 Feb 2016 06:34:59 -0800 (PST) Subject: [newbie] problem with geometry setting in Tkinter In-Reply-To: References: <31e9b382-bc20-4283-b17c-23a97c8b6083@googlegroups.com> Message-ID: <2c0abe95-fbea-4818-bf8d-0bb1261cf8b8@googlegroups.com> Op maandag 8 februari 2016 13:26:56 UTC+1 schreef Peter Otten: > jenswaelkens at gmail.com wrote: > > > I'm trying to set the geometry of my top window, but the size is > > unaffected. > > This is the code: > > > > #!/usr/bin/env python > > import Tkinter > > top=Tkinter.Tk() > > top.geometry=('900x460') > > That's an assignment, but geometry() is a method that you have to invoke: > > top.geometry('900x460') > > > top.update_idletasks() > > print (top.winfo_width()) > > print (top.winfo_height()) > > print (top.winfo_geometry()) > > top.mainloop() > > > > and this is the result when running the code: > > 200 > > 200 > > 200x200+1+584 > > > > > > Can anyone here tell me what I am doing wrong and how to change it? > > Thanks > > > > Jens thanks a lot for helping me out kind regards, Jens From random832 at fastmail.com Mon Feb 8 09:49:52 2016 From: random832 at fastmail.com (Random832) Date: Mon, 08 Feb 2016 09:49:52 -0500 Subject: A sets algorithm In-Reply-To: References: Message-ID: <1454942992.2532814.515120466.5DA4A683@webmail.messagingengine.com> On Sun, Feb 7, 2016, at 20:07, Cem Karan wrote: > a) Use Chris Angelico's suggestion and hash each of the files (use the standard library's 'hashlib' for this). Identical files will always have identical hashes, but there may be false positives, so you'll need to verify that files that have identical hashes are indeed identical. > b) If your files tend to have sections that are very different (e.g., the first 32 bytes tend to be different), then you pretend that section of the file is its hash. You can then do the same trick as above. (the advantage of this is that you will read in a lot less data than if you have to hash the entire file). > c) You may be able to do something clever by reading portions of each file. That is, use zip() combined with read(1024) to read each of the files in sections, while keeping hashes of the files. Or, maybe you'll be able to read portions of them and sort the list as you're reading. In either case, if any files are NOT identical, then you'll be able to stop work as soon as you figure this out, rather than having to read the entire file at once. > > The main purpose of these suggestions is to reduce the amount of reading > you're doing. hashing a file using a conventional hashing algorithm requires reading the whole file. Unless the files are very likely to be identical _until_ near the end, you're better off just reading the first N bytes of both files, then the next N bytes, etc, until you find somewhere they're different. The filecmp module may be useful for this. From random832 at fastmail.com Mon Feb 8 09:52:43 2016 From: random832 at fastmail.com (Random832) Date: Mon, 08 Feb 2016 09:52:43 -0500 Subject: Set Operations on Dicts In-Reply-To: References: Message-ID: <1454943163.2534518.515125218.0C782BB2@webmail.messagingengine.com> On Mon, Feb 8, 2016, at 08:32, Matt Wheeler wrote: > On 8 February 2016 at 12:17, Jussi Piitulainen > wrote: > > Also, what would be the nicest current way to express a priority union > > of dicts? > > > > { k:(d if k in d else e)[k] for k in d.keys() | e.keys() } > > Since Python 3.5: {**e, **d} And before that... dict(ChainMap(d, e)) From vlastimil.brom at gmail.com Mon Feb 8 10:00:22 2016 From: vlastimil.brom at gmail.com (Vlastimil Brom) Date: Mon, 8 Feb 2016 16:00:22 +0100 Subject: What's the best/neatest way to get Unicode data from a database into a grid cell? In-Reply-To: <318moc-ra4.ln1@esprimo.zbmc.eu> References: <318moc-ra4.ln1@esprimo.zbmc.eu> Message-ID: 2016-02-08 11:42 GMT+01:00 : > Vlastimil Brom wrote: >> >> Hi, >> your code in >> http://www.salstat.com/news/linking-wxgrid-to-sqlite-database-in-python-an-example >> >> seems to work for me after small changes with both python 2.7 and 3.4 >> (using wx Phoenix) > > Where are you getting phoenix from? It's not in the Ubuntu > repositories and when I search on Google I seem to go in ever > decreasing circles and always end up finding wxPython for 2.7. > > Did you build it from source yourself (I can do that but only if it's > necessary). > > ... Hi, I'm glad it helped a bit. However, I have no experiences with running wx phoenix on linux - I use it with windows (7) - where the mentioned snapshot builds are available (e.g. as wheel files installable with pip). http://wxpython.org/Phoenix/snapshot-builds/ I believe, building and installing on linux should be doable, but I can't supply any details, see e.g.: http://wxpython.org/Phoenix/docs/html/#linux-unix-etc the source files are available along with the snapshot builds: http://wxpython.org/Phoenix/snapshot-builds/ hth, vbr From jussi.piitulainen at helsinki.fi Mon Feb 8 10:05:56 2016 From: jussi.piitulainen at helsinki.fi (Jussi Piitulainen) Date: Mon, 08 Feb 2016 17:05:56 +0200 Subject: Set Operations on Dicts References: Message-ID: Random832 writes: > On Mon, Feb 8, 2016, at 08:32, Matt Wheeler wrote: >> On 8 February 2016 at 12:17, Jussi Piitulainen wrote: >> > Also, what would be the nicest current way to express a priority union >> > of dicts? >> > >> > { k:(d if k in d else e)[k] for k in d.keys() | e.keys() } >> >> Since Python 3.5: {**e, **d} > > And before that... dict(ChainMap(d, e)) New in version 3.3. Thanks, I didn't know of that either. Reading the docs, I think for the use I had in mind, ChainMap(d, e) itself would suffice. From rosuav at gmail.com Mon Feb 8 10:11:30 2016 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 9 Feb 2016 02:11:30 +1100 Subject: A sets algorithm In-Reply-To: <1454942992.2532814.515120466.5DA4A683@webmail.messagingengine.com> References: <1454942992.2532814.515120466.5DA4A683@webmail.messagingengine.com> Message-ID: On Tue, Feb 9, 2016 at 1:49 AM, Random832 wrote: > On Sun, Feb 7, 2016, at 20:07, Cem Karan wrote: >> a) Use Chris Angelico's suggestion and hash each of the files (use the standard library's 'hashlib' for this). Identical files will always have identical hashes, but there may be false positives, so you'll need to verify that files that have identical hashes are indeed identical. >> b) If your files tend to have sections that are very different (e.g., the first 32 bytes tend to be different), then you pretend that section of the file is its hash. You can then do the same trick as above. (the advantage of this is that you will read in a lot less data than if you have to hash the entire file). >> c) You may be able to do something clever by reading portions of each file. That is, use zip() combined with read(1024) to read each of the files in sections, while keeping hashes of the files. Or, maybe you'll be able to read portions of them and sort the list as you're reading. In either case, if any files are NOT identical, then you'll be able to stop work as soon as you figure this out, rather than having to read the entire file at once. >> >> The main purpose of these suggestions is to reduce the amount of reading >> you're doing. > > hashing a file using a conventional hashing algorithm requires reading > the whole file. Unless the files are very likely to be identical _until_ > near the end, you're better off just reading the first N bytes of both > files, then the next N bytes, etc, until you find somewhere they're > different. The filecmp module may be useful for this. That's fine for comparing one file against one other. He started out by saying he already had a way to compare files for equality. What he wants is a way to capitalize on that to find all the identical files in a group. A naive approach would simply compare every file against every other, for O(N*N) comparisons - but a hash lookup can make that O(N) on the files themselves, plus (I think) an O(N log N) hash comparison job, which has much lower constant factors. The key here is the hashing algorithm though. ChrisA From auriocus at gmx.de Mon Feb 8 10:11:50 2016 From: auriocus at gmx.de (Christian Gollwitzer) Date: Mon, 8 Feb 2016 16:11:50 +0100 Subject: [newbie] problem with geometry setting in Tkinter In-Reply-To: <2c0abe95-fbea-4818-bf8d-0bb1261cf8b8@googlegroups.com> References: <31e9b382-bc20-4283-b17c-23a97c8b6083@googlegroups.com> <2c0abe95-fbea-4818-bf8d-0bb1261cf8b8@googlegroups.com> Message-ID: Am 08.02.16 um 15:34 schrieb jenswaelkens at gmail.com: > Op maandag 8 februari 2016 13:26:56 UTC+1 schreef Peter Otten: >> jenswaelkens at gmail.com wrote: >> >>> I'm trying to set the geometry of my top window, but the size is >>> unaffected. >>> This is the code: >> >> top.geometry('900x460') >> > > thanks a lot for helping me out The answer is correct, however I think that in general it is wrong to set the geometry like this. grid and pack compute the necessary space for the window themselves, which adapts to the length of labels on buttons etc. In all GUI programs I've written, the only place to set window sizes in pixels have been canvas widgets, which do not have a native size derived from the content. Setting the size on the toplevel, and in absolute numbers, i.e. not computed from other sizes, is likely to lead to problems on different platforms, font sizes or theme changes. Christian From jason.swails at gmail.com Mon Feb 8 10:20:59 2016 From: jason.swails at gmail.com (Jason Swails) Date: Mon, 8 Feb 2016 10:20:59 -0500 Subject: [STORY-TIME] THE BDFL AND HIS PYTHON PETTING ZOO In-Reply-To: References: <48762040-a7e0-434c-92e8-8a0969210e5b@googlegroups.com> <5d31dd14-1a60-4f80-8889-d0616f404c70@googlegroups.com> <4e902c58-3370-4adf-905d-74630a6eea90@googlegroups.com> <56b6dd57$0$1614$c3e8da3$5496439d@news.astraweb.com> <87mvrdovb6.fsf@jester.gateway.pace.com> <87io21ot4r.fsf@jester.gateway.pace.com> Message-ID: On Sun, Feb 7, 2016 at 2:58 AM, Chris Angelico wrote: > > Would writing a script to figure out whether there are more > statisticians or programmers be a statistician's job or a > programmer's? > ?Yes. From cl at isbd.net Mon Feb 8 10:36:54 2016 From: cl at isbd.net (cl at isbd.net) Date: Mon, 8 Feb 2016 15:36:54 +0000 Subject: Confused by wxpython documentation Message-ID: I'm playing around with some existing code that uses wxpython. I've been trying to understand a basic bit about the import statement and so went to the beginning of the wxPython on line documents. Going from the top to the "Hello World Example" (can't give URL as the URL is the same for all the docs, http://www.wxpython.org/onlinedocs.php) the first thing I see is some C/C++ code. What's this about? How have I got to the underlying C/C++ implementation rather than the Python documentation? I realise the Python usage follows the underlying C/C++ very closely but seeing the C/C++ doesn't really help me write my Python too much. Looking more closely what I've actually been taken to is the wxWidgets documentation which, not unreasonably, is in C/C++. ... but where do I find the Python documentation for this? -- Chris Green ? From ian.g.kelly at gmail.com Mon Feb 8 10:40:23 2016 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Mon, 8 Feb 2016 08:40:23 -0700 Subject: [STORY-TIME] THE BDFL AND HIS PYTHON PETTING ZOO In-Reply-To: <4e902c58-3370-4adf-905d-74630a6eea90@googlegroups.com> References: <48762040-a7e0-434c-92e8-8a0969210e5b@googlegroups.com> <5d31dd14-1a60-4f80-8889-d0616f404c70@googlegroups.com> <4e902c58-3370-4adf-905d-74630a6eea90@googlegroups.com> Message-ID: On Sat, Feb 6, 2016 at 1:54 PM, Rick Johnson wrote: > On Wednesday, February 3, 2016 at 12:02:35 AM UTC-6, John Ladasky wrote: > >> Rick, you don't like Python? > > If i didn't like Python, then i would happily let it self- > destruct, yes? The problem is, i *DO* like Python. Python2 > was a great language, but python3 has been a disaster. Heck, > even the BDFL has warned that Python4 cannot introduce as > many backwards incompatible changes as python3. So obviously, > he is admitting that Python3 was a disaster. That's a complete non sequitur. "George Lucas notes that he can't pull the Darth Vader reveal again. So obviously, he is admitting that Empire Strikes Back was a disaster." Besides, you're forgetting that the whole point of having so many backwards incompatible changes in Python 3 in the first place was to get them out of the way and not have to do them further into the future. Python 4.0 has never been planned to be anything more than an incremental release like Python 2.0, and AIUI the only reason any of the core devs are even talking about Python 4 at this point in time is because Guido doesn't like "3.10" as a version number. From cl at isbd.net Mon Feb 8 10:44:25 2016 From: cl at isbd.net (cl at isbd.net) Date: Mon, 8 Feb 2016 15:44:25 +0000 Subject: A question about imports in wxpython Message-ID: I'm playing with some code that uses the wxpython grid. *Every* example I have seen starts with the imports:- import wx import wx.grid as Gridlib As Gridlib is exactly the same number of characters as wx.grid I really don't see the point. Am I missing something? -- Chris Green ? From random832 at fastmail.com Mon Feb 8 10:44:35 2016 From: random832 at fastmail.com (Random832) Date: Mon, 08 Feb 2016 10:44:35 -0500 Subject: [STORY-TIME] THE BDFL AND HIS PYTHON PETTING ZOO In-Reply-To: References: <48762040-a7e0-434c-92e8-8a0969210e5b@googlegroups.com> <5d31dd14-1a60-4f80-8889-d0616f404c70@googlegroups.com> <4e902c58-3370-4adf-905d-74630a6eea90@googlegroups.com> Message-ID: <1454946275.2545504.515184770.3F04D19C@webmail.messagingengine.com> On Mon, Feb 8, 2016, at 10:40, Ian Kelly wrote: > Besides, you're forgetting that the whole point of having so many > backwards incompatible changes in Python 3 in the first place was to > get them out of the way and not have to do them further into the > future. Python 4.0 has never been planned to be anything more than an > incremental release like Python 2.0, and AIUI the only reason any of > the core devs are even talking about Python 4 at this point in time is > because Guido doesn't like "3.10" as a version number. I still think we should just retroactively declare 3.5 to be python 5, and then keep going with python 6, 7, etc... From rosuav at gmail.com Mon Feb 8 10:46:19 2016 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 9 Feb 2016 02:46:19 +1100 Subject: [STORY-TIME] THE BDFL AND HIS PYTHON PETTING ZOO In-Reply-To: <1454946275.2545504.515184770.3F04D19C@webmail.messagingengine.com> References: <48762040-a7e0-434c-92e8-8a0969210e5b@googlegroups.com> <5d31dd14-1a60-4f80-8889-d0616f404c70@googlegroups.com> <4e902c58-3370-4adf-905d-74630a6eea90@googlegroups.com> <1454946275.2545504.515184770.3F04D19C@webmail.messagingengine.com> Message-ID: On Tue, Feb 9, 2016 at 2:44 AM, Random832 wrote: > On Mon, Feb 8, 2016, at 10:40, Ian Kelly wrote: >> Besides, you're forgetting that the whole point of having so many >> backwards incompatible changes in Python 3 in the first place was to >> get them out of the way and not have to do them further into the >> future. Python 4.0 has never been planned to be anything more than an >> incremental release like Python 2.0, and AIUI the only reason any of >> the core devs are even talking about Python 4 at this point in time is >> because Guido doesn't like "3.10" as a version number. > > I still think we should just retroactively declare 3.5 to be python 5, > and then keep going with python 6, 7, etc... http://dirtsimple.org/2004/12/python-is-not-java.html ChrisA From random832 at fastmail.com Mon Feb 8 10:53:02 2016 From: random832 at fastmail.com (Random832) Date: Mon, 08 Feb 2016 10:53:02 -0500 Subject: [STORY-TIME] THE BDFL AND HIS PYTHON PETTING ZOO In-Reply-To: References: <48762040-a7e0-434c-92e8-8a0969210e5b@googlegroups.com> <5d31dd14-1a60-4f80-8889-d0616f404c70@googlegroups.com> <4e902c58-3370-4adf-905d-74630a6eea90@googlegroups.com> <1454946275.2545504.515184770.3F04D19C@webmail.messagingengine.com> Message-ID: <1454946782.2548275.515193066.513B4862@webmail.messagingengine.com> On Mon, Feb 8, 2016, at 10:46, Chris Angelico wrote: > > I still think we should just retroactively declare 3.5 to be python 5, > > and then keep going with python 6, 7, etc... > > http://dirtsimple.org/2004/12/python-is-not-java.html Java's hardly the only, or even the first, project to drop a version number. (I think the first may actually have been GNU Emacs), and it's certainly not the only one with a release schedule that frequently increments the major version number. From ian.g.kelly at gmail.com Mon Feb 8 11:12:45 2016 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Mon, 8 Feb 2016 09:12:45 -0700 Subject: Set Operations on Dicts In-Reply-To: References: Message-ID: On Mon, Feb 8, 2016 at 5:47 AM, Grobu wrote: > You can use dictionary comprehension : > > Say : > dict1 = {'a': 123, 'b': 456} > set1 = {'a'} > > intersection : >>>> { key:dict1[key] for key in dict1 if key in set1 } > {'a': 123} > > difference : >>>> { key:dict1[key] for key in dict1 if not key in set1 } > {'b': 456} dict does already expose set-like views. How about: {k: d[k] for k in d.keys() & s} # d & s {k: d[k] for k in d.keys() - s} # d - s From ian.g.kelly at gmail.com Mon Feb 8 11:20:26 2016 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Mon, 8 Feb 2016 09:20:26 -0700 Subject: Confused by wxpython documentation In-Reply-To: References: Message-ID: On Mon, Feb 8, 2016 at 8:36 AM, wrote: > I'm playing around with some existing code that uses wxpython. I've > been trying to understand a basic bit about the import statement and > so went to the beginning of the wxPython on line documents. > > Going from the top to the "Hello World Example" (can't give URL as the > URL is the same for all the docs, http://www.wxpython.org/onlinedocs.php) > the first thing I see is some C/C++ code. What's this about? How have > I got to the underlying C/C++ implementation rather than the Python > documentation? > > I realise the Python usage follows the underlying C/C++ very closely > but seeing the C/C++ doesn't really help me write my Python too much. > > Looking more closely what I've actually been taken to is the wxWidgets > documentation which, not unreasonably, is in C/C++. > > ... but where do I find the Python documentation for this? wiki.wxpython.org was the site I used when I was developing wxPython. From ian.g.kelly at gmail.com Mon Feb 8 11:25:38 2016 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Mon, 8 Feb 2016 09:25:38 -0700 Subject: A question about imports in wxpython In-Reply-To: References: Message-ID: On Mon, Feb 8, 2016 at 8:44 AM, wrote: > I'm playing with some code that uses the wxpython grid. *Every* > example I have seen starts with the imports:- > > import wx > import wx.grid as Gridlib > > As Gridlib is exactly the same number of characters as wx.grid I > really don't see the point. Am I missing something? You're not missing anything. I've actually never seen that before (or at least never noticed). The first hit when searching "import wx.grid" is http://wxpython.org/Phoenix/docs/html/grid_overview.html which doesn't use the "as" (but I see some hits farther down that do). Probably the author of that code was just trying to save a dict lookup every time "wx.grid" is referenced (which has to look up "wx" in the globals and then "grid" as an attribute). Seems like an unnecessary micro-optimization to me. From cl at isbd.net Mon Feb 8 11:50:38 2016 From: cl at isbd.net (cl at isbd.net) Date: Mon, 8 Feb 2016 16:50:38 +0000 Subject: Confused by wxpython documentation References: Message-ID: Ian Kelly wrote: > On Mon, Feb 8, 2016 at 8:36 AM, wrote: > > I'm playing around with some existing code that uses wxpython. I've > > been trying to understand a basic bit about the import statement and > > so went to the beginning of the wxPython on line documents. > > > > Going from the top to the "Hello World Example" (can't give URL as the > > URL is the same for all the docs, http://www.wxpython.org/onlinedocs.php) > > the first thing I see is some C/C++ code. What's this about? How have > > I got to the underlying C/C++ implementation rather than the Python > > documentation? > > > > I realise the Python usage follows the underlying C/C++ very closely > > but seeing the C/C++ doesn't really help me write my Python too much. > > > > Looking more closely what I've actually been taken to is the wxWidgets > > documentation which, not unreasonably, is in C/C++. > > > > ... but where do I find the Python documentation for this? > > wiki.wxpython.org was the site I used when I was developing wxPython. Thanks, a good place to start, and it tells me why I get to the C/C++ docs for wxWidgets. -- Chris Green ? From kwpolska at gmail.com Mon Feb 8 12:48:59 2016 From: kwpolska at gmail.com (Chris Warrick) Date: Mon, 8 Feb 2016 18:48:59 +0100 Subject: [STORY-TIME] THE BDFL AND HIS PYTHON PETTING ZOO In-Reply-To: <1454946782.2548275.515193066.513B4862@webmail.messagingengine.com> References: <48762040-a7e0-434c-92e8-8a0969210e5b@googlegroups.com> <5d31dd14-1a60-4f80-8889-d0616f404c70@googlegroups.com> <4e902c58-3370-4adf-905d-74630a6eea90@googlegroups.com> <1454946275.2545504.515184770.3F04D19C@webmail.messagingengine.com> <1454946782.2548275.515193066.513B4862@webmail.messagingengine.com> Message-ID: On 8 February 2016 at 16:53, Random832 wrote: > On Mon, Feb 8, 2016, at 10:46, Chris Angelico wrote: >> > I still think we should just retroactively declare 3.5 to be python 5, >> > and then keep going with python 6, 7, etc... >> >> http://dirtsimple.org/2004/12/python-is-not-java.html > > Java's hardly the only, or even the first, project to drop a version > number. (I think the first may actually have been GNU Emacs), and it's > certainly not the only one with a release schedule that frequently > increments the major version number. > -- > https://mail.python.org/mailman/listinfo/python-list In fact, this was done by a very popular Python project two years ago. That project is pip, which went from 1.5.6 to 6.0, and is now at 8.0.2. And its best friend setuptools is up to version 20.0. -- Chris Warrick PGP: 5EAAEA16 From stephane at wirtel.be Mon Feb 8 13:34:43 2016 From: stephane at wirtel.be (Stephane Wirtel) Date: Mon, 8 Feb 2016 19:34:43 +0100 Subject: Syntax Highlightning for C API of CPython in VIM Message-ID: <20160208183443.GA27740@sg1> Hi everyone, With my talk "Exploring our Python Interpreter", I think this VIM plugin can be useful for the community. It's a syntax highlighter for the C API of CPython 3.5 and 3.6. I used Clang for the parsing and automatically generated the keywords for VIM. PyObject and the others typedefs of CPython will have the defined color of your favourite editor and it's the same for the enums, the typedefs, the functions and the macros. Where can you use this VIM plugin ? If you want to write a CPython extension or if you want to hack in the CPython code. Check this screenshot: http://i.imgur.com/0k13KOU.png Here is the repository: https://github.com/matrixise/cpython-vim-syntax Please, if you see some issues, tell me via an issue on Github. Thank you so much, Stephane -- St?phane Wirtel - http://wirtel.be - @matrixise From brendan.simon at etrix.com.au Mon Feb 8 15:59:27 2016 From: brendan.simon at etrix.com.au (Brendan Simon (eTRIX)) Date: Tue, 9 Feb 2016 07:59:27 +1100 Subject: Help using Thread (or other method) Message-ID: <56B901AF.5030602@etrix.com.au> Hi. Need some urgent help. I have a python app that uses `select` to wait for data from an arm embedded linux kernel interrupt (every second). The mainloop of the app then grabs data from some memory mapped area, processes it and then does a http post to a server. The problem is the http post can take multiple seconds to respond and blocks (using the requests module). So I've put the posting in a separate thread (using Thread module) and send data to it via a queue (using the Queue module). The http post can send multiple records by pulling multiple items off the queue. Sounds good so far, right? This seems to work ok, but I also need to grab data from some other serial devices (gps, weather station) to put in the post. Various sample apps on the web use Threads to do this, so I am doing the same. One of the threads uses `select` (see below), following by readlines() on the device. def __init__(self): select_timeout = 1 serial_timeout = 0.1 def wait_and_process(self): '''wait for data and process it.''' r = select.select([self.serial_dev], [], [], self.select_timeout) if not r[0]: #print("DEBUG: TIMEOUT: wait_and_process") return #print("DEBUG: Weather Data Captured") for s in self.serial_dev.readlines(): #print("INFO: Data: {}".format(s)) temperature = extract_temperature(s) Is using `select` here redundant? It's used to block the thread until the first character is ready to be read from the serial device? Then I use serial_dev.readlines() to read the stream of chars from the weather station. Is that ok? Does readline block until it sees an end-of-line char? i.e. does it only wake up the thread when it has a _complete string_ or a timeout of 0.1 seconds? or will the process block other threads from running while it is gathering the chars until the newline? I'm assuming the gathering of chars will happen in the background and allow my main thread to run, correct? My application mainloop (not the threads) needs to be high priority and always process as soon as it gets an interrupt. Is using select a good way of doing this? How can I ensure that no other threads are utilizing the CPU, etc? I'm worried about the GIL (I'm using CPython 2.7). Is there a better way of structuring this to ensure I process the interrupt and not get interrupt overrun? Is using select only in the mainloop, with multiple file descriptors, a better way of doing things, so that I can process the file descriptor of interest first, before any others if set? Is using the multiprocessing module a better option? Thanks for any advice, Brendan. From marko at pacujo.net Mon Feb 8 16:27:56 2016 From: marko at pacujo.net (Marko Rauhamaa) Date: Mon, 08 Feb 2016 23:27:56 +0200 Subject: Help using Thread (or other method) References: Message-ID: <87si12q4df.fsf@elektro.pacujo.net> "Brendan Simon (eTRIX)" : > Is using `select` here redundant? Yes, if you go the thread way, select() is usually redundant and just gets in the way. > Does readline block until it sees an end-of-line char? i.e. does it > only wake up the thread when it has a _complete string_ or a timeout > of 0.1 seconds? You probably don't want to mix 0.1-second timeouts with readline but let the readline thread do its job: call readline in an indefinite wait. You can then have Queue.get(timeout=...) do the timeout if necessary. > or will the process block other threads from running while it is > gathering the chars until the newline? It shouldn't, but I didn't try it out. > My application mainloop (not the threads) needs to be high priority > and always process as soon as it gets an interrupt. Is using select a > good way of doing this? How can I ensure that no other threads are > utilizing the CPU, etc? I'm worried about the GIL (I'm using CPython > 2.7). Here's what you can do: * Have your main, high-priority thread have an input Queue coming in. Have the peripheral threads put their notifications in that Queue. * Have each peripheral thread have an acknowledgement threading.Event object. Whenever a peripheral thread has put a notification in the Queue, have it wait for the main thread to set() the acknowledgement event before looking for more work. That prevents the peripheral threads from starving the CPU under any circumstances. > Is using select only in the mainloop, with multiple file descriptors, a > better way of doing things, so that I can process the file descriptor of > interest first, before any others if set? > > Is using the multiprocessing module a better option? I don't know enough of your exact problem to recommend anything. Both of those are viable options. However, readline() and select() don't mix well at all because readline() does buffered I/O and select() deals with raw file descriptors. If you *can* reduce all functionality to file descriptor operations, use os.read() and os.write() only. Then, you can use select.select() or select.epoll() and not bother with threads at all. However, then you'll have to implement HTTP processing on your own. (The same goes for 3.5's new async/await scheme: https://docs.python.or g/3/whatsnew/3.5.html?highlight=asyncio#pep-492-coroutines-with-async-a nd-await-syntax>.) Given the nature of your questions, you might want to stick with threads. Your problem will then be how to shut down those threads in the end... Marko From tdsimpson at gmail.com Mon Feb 8 16:45:24 2016 From: tdsimpson at gmail.com (MrPink) Date: Mon, 8 Feb 2016 13:45:24 -0800 (PST) Subject: Searching Sets (Lottery Results) Message-ID: <6afb7298-6da7-44f9-847e-9603554c2f09@googlegroups.com> This is a continuation of my pursuit to learn Python. I have been tinkering with this for a number of years and I am back at it again. I am stuck and need some guidance. This is related to other posts that I have made in the past. For example: Searching for Lottery drawing list of ticket match: https://groups.google.com/forum/#!topic/comp.lang.python/sHLPrfmY3q4 I have a text file with the results of lottery drawing like so: Draw Date WB1 WB2 WB3 WB4 WB5 PB PP 01/02/2016 42 15 06 05 29 10 2 12/30/2015 12 61 54 38 36 22 3 12/26/2015 65 40 44 59 27 20 2 12/23/2015 67 16 63 38 55 25 4 12/19/2015 30 68 59 41 28 10 2 12/16/2015 09 42 10 55 32 06 2 12/12/2015 62 02 30 19 14 22 2 12/09/2015 16 46 10 56 07 01 2 12/05/2015 47 33 68 27 13 13 2 12/02/2015 14 18 19 64 32 09 2 11/28/2015 47 02 66 67 06 02 3 11/25/2015 53 16 69 58 29 21 2 11/21/2015 37 57 47 50 52 21 3 11/18/2015 40 17 46 69 41 06 2 11/14/2015 66 37 22 14 45 05 3 11/11/2015 26 04 32 55 64 18 3 11/07/2015 50 53 07 16 25 15 2 11/04/2015 12 02 17 20 65 17 4 10/31/2015 09 47 20 25 68 07 2 10/28/2015 56 62 54 63 04 10 2 10/24/2015 20 31 56 64 60 02 3 10/21/2015 57 32 30 42 56 11 4 I load the lottery drawings into memory for searching with the following code although, it is incomplete. I am stuck and need some guidance. The set datatype seems to be the best for searching, but how best can I implement it? And I want the results to highlight the numbers that were matched. For example, if the white balls in the drawing are: "42 15 06 05 29" AND the numbers on the lottery ticket are: "06 15 32 42 56" THEN the display might look like: "06* 15* 32 42* 56" WHERE * signifies a match. ############################### from datetime import datetime class Powerball(object): """Summary of class here. Longer class information. . .Longer Attributes: fileName: File name to load drawing from. """ # class Constants _DATE_FORMAT = '%m/%d/%Y' # class variables fileName = 'pb.txt' # class initialization def __init__(self, pFileName): """Return a Powerball Object with a set of winning drawings. :param pFileName: """ self.fileName = pFileName # Open file and load drawing data sets. def readFileData(self): """File to open and load data from. """ f = open(self.fileName, 'r') # read the whole file into a list of lines. lines = f.readlines() f.close() # For each line in the list of lines. . . for line in lines[1:50]: # split the string on whitespace into a list of strings fields = line.split() d = datetime.strptime(fields[0], self._DATE_FORMAT).date() # Use list comprehension to create a frozenset. wb = frozenset(int(num_str) for num_str in fields[1:6]) pb = int(fields[6]) t = tuple([wb, pb, d]) # Store t into a data structure for searching later. . . # Not sure of best way to do this. . . p = Powerball("pb.txt") p.readFileData() ################################# From ian.g.kelly at gmail.com Mon Feb 8 16:56:55 2016 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Mon, 8 Feb 2016 14:56:55 -0700 Subject: coroutine, throw, yield, call-stack and exception handling In-Reply-To: References: Message-ID: On Mon, Feb 8, 2016 at 2:17 AM, Veek. M wrote: > **************************** > Exceptions can be raised inside a coroutine using the throw( > > Exceptions raised in this manner will originate at the currently > executing yield state-ment in the coroutine.A coroutine can elect to > catch exceptions and handle them as appropriate. It is not safe to use > throw() as an asynchronous signal to a coroutine?it should never be > invoked from a separate execution thread or in a signal handler. > **************************** > > What does Beazley mean by this: 'will originate at the currently > executing yield state-ment in the coroutine' > > If he's throw'ing an exception surely it originates at the throw: > > def mycoroutine(): > while len(n) > 2: > n = (yield) > > throw('RuntimeError' "die!") The "throw" is not called from inside the coroutine. It's a method of the generator object, and it's used by the calling code. It's similar to calling the send method, except that instead of passing a value to be returned by the yield expression, it passes an exception to be raised inside the coroutine at the yield expression. Example: def mycoroutine(): n = 0 while True: try: n = (yield n) except SomeException: n = 42 coro = mycoroutine() coro.next() for i in range(100): if i % 6 == 0: coro.send(i % 6) else: coro.throw(SomeException()) > Also this bit: > *********************** > If a coroutine returns values, some care is required if exceptions > raised with throw() are being handled. If you raise an exception in a > coroutine using throw(), the value passed to the next yield in the > coroutine will be returned as the result of throw(). If > you need this value and forget to save it, it will be lost. > *********************** > > def coroutine(): > while True: > line = (yield result) > > throw(FooException) > > where is the question of a 'yield'? You'll exit the coroutine straight > away.. Taking my example from above, after SomeException is caught, the next value yielded inside the coroutine will be the return value of the coro.throw() call. This may be surprising if you're only expecting coro.send() and not coro.throw() to return yielded values. From marko at pacujo.net Mon Feb 8 17:13:11 2016 From: marko at pacujo.net (Marko Rauhamaa) Date: Tue, 09 Feb 2016 00:13:11 +0200 Subject: Asyncio thought experiment Message-ID: <87oabqq2a0.fsf@elektro.pacujo.net> As I stated in an earlier post, a normal subroutine may turn out to be blocking. To make it well-behaved under asyncio, you then dutifully tag the subroutine with "async" and adorn the blocking statement with "await". Consequently, you put "await" in front of all calls to the subroutine and cascade the "async"s and "await"s all the way to the top level. Now what would prevent you from making *every* function an "async" and "await"ing *every* function call? Then, you would never fall victim to the cascading async/await. And if you did that, why bother sprinkling async's and await's everywhere? Why not make every single function call an await implicitly and every single subroutine an async? In fact, that's how everything works in multithreading: blocking statements don't need to be ornamented in any manner. Marko From ben+python at benfinney.id.au Mon Feb 8 18:32:59 2016 From: ben+python at benfinney.id.au (Ben Finney) Date: Tue, 09 Feb 2016 10:32:59 +1100 Subject: Set Operations on Dicts References: Message-ID: <85io1y93ro.fsf@benfinney.id.au> Marco Kaulea writes: > In one talk (I think it was [1]) it was described that sets are basically > dicts without the values. It seems an unhelpful thing to say about ?set?, I disagree with that characterisation. > Therefor it should be easy to apply set operations on dicts Yes, that's one reason I disagree; it leads to incorrect inferences like this :-) -- \ ?I went to court for a parking ticket; I pleaded insanity. I | `\ said ?Your Honour, who in their right mind parks in the passing | _o__) lane??? ?Steven Wright | Ben Finney From oscar.j.benjamin at gmail.com Mon Feb 8 18:51:55 2016 From: oscar.j.benjamin at gmail.com (Oscar Benjamin) Date: Mon, 8 Feb 2016 23:51:55 +0000 Subject: trouble installing python In-Reply-To: <1698609935.300641.1454805263360.JavaMail.yahoo@mail.yahoo.com> References: <1698609935.300641.1454805263360.JavaMail.yahoo.ref@mail.yahoo.com> <1698609935.300641.1454805263360.JavaMail.yahoo@mail.yahoo.com> Message-ID: On 7 Feb 2016 09:50, "donald alsept via Python-list" wrote: > > Hello, > I'm trying to install the 3.5.1 of Python and am running windows 7. I keep getting an error about api-ms-win-crt-runtime-|1-1-0.dll not being installed. Any advice on what is wrong? Hi Donald, you're seeing this problem because Python 3.5 on Windows 7 needs a runtime update. You can get this update from Microsoft: https://support.microsoft.com/en-us/kb/2999226 Alternatively you can install Python 3.4 instead since 3.4 doesn't need the update. -- Oscar From rosuav at gmail.com Mon Feb 8 19:04:56 2016 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 9 Feb 2016 11:04:56 +1100 Subject: Searching Sets (Lottery Results) In-Reply-To: <6afb7298-6da7-44f9-847e-9603554c2f09@googlegroups.com> References: <6afb7298-6da7-44f9-847e-9603554c2f09@googlegroups.com> Message-ID: On Tue, Feb 9, 2016 at 8:45 AM, MrPink wrote: > I load the lottery drawings into memory for searching with the following code although, it is incomplete. I am stuck and need some guidance. > > The set datatype seems to be the best for searching, but how best can I implement it? > > And I want the results to highlight the numbers that were matched. For example, if the white balls in the drawing are: > "42 15 06 05 29" > > AND the numbers on the lottery ticket are: > "06 15 32 42 56" > > THEN the display might look like: > "06* 15* 32 42* 56" > > WHERE * signifies a match. > This suggests that there is an order to the numbers on your ticket (you want to print them out in the same order), but not to the winning numbers, which are simply a set. The easiest way to handle that would be to iterate over your numbers, asking "if number in winning_numbers:", and printing out a "match" marker if it is or a "non-match" marker if it isn't. ChrisA From rosuav at gmail.com Mon Feb 8 19:17:50 2016 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 9 Feb 2016 11:17:50 +1100 Subject: Help using Thread (or other method) In-Reply-To: <56B901AF.5030602@etrix.com.au> References: <56B901AF.5030602@etrix.com.au> Message-ID: On Tue, Feb 9, 2016 at 7:59 AM, Brendan Simon (eTRIX) wrote: > My application mainloop (not the threads) needs to be high priority and > always process as soon as it gets an interrupt. Is using select a good > way of doing this? How can I ensure that no other threads are utilizing > the CPU, etc? I'm worried about the GIL (I'm using CPython 2.7). > First and foremost, don't worry about the GIL. It's almost never a problem, so wait until you have proof of its guilt before you start trying to fire it. In your situation, your quoted problem is that HTTP requests take time - in other words, that your code is blocking - so nothing's going to be spinning in the CPU. The most obvious solutions to your initial problem are (1) threads, and (2) async I/O. If there's only one part of your code that's ever at risk of problematic blocking, you might be able to do that part asynchronously and most of the rest of the code with simple blocking calls; what that'd mean is that you won't process HTTP responses until your code next "falls idle" in its primary loop (waiting for another interrupt), which is pretty much what you'd have with any other scheme anyway. You could look into a Python 3.5 async/await model, using a library like this (never used it, just found it on Google): https://github.com/KeepSafe/aiohttp Otherwise, threads are great. ChrisA From mr.eightnoteight at gmail.com Mon Feb 8 22:12:12 2016 From: mr.eightnoteight at gmail.com (srinivas devaki) Date: Tue, 9 Feb 2016 08:42:12 +0530 Subject: Heap Implementation In-Reply-To: <185DAEA4-8728-4792-A3B7-7F6AC5A7F876@gmail.com> References: <56AD3D83.2050308@mail.de> <7C522D08-9D73-48D2-A71D-F1D1D34C02A5@gmail.com> <185DAEA4-8728-4792-A3B7-7F6AC5A7F876@gmail.com> Message-ID: On Feb 8, 2016 5:17 PM, "Cem Karan" wrote: > > On Feb 7, 2016, at 10:15 PM, srinivas devaki wrote: > > On Feb 8, 2016 7:07 AM, "Cem Karan" wrote: > > > I know that there are methods of handling this from the client-side (tuples with unique counters come to mind), but if your library can handle it directly, then that could be useful to others as well. > > > > yeah it is a good idea to do at client side. > > but if it should be introduced as feature into the library, instead of tuples, we should just piggyback a single counter it to the self._indexes dict, or better make another self._counts dict which will be light and fast. > > and if you think again with this method you can easily subclass with just using self._counts dict in your subclass. but still I think it is good to introduce it as a feature in the library. > > > > Regards > > Srinivas Devaki > > I meant that the counter is a trick to separate different instances of (item, priority) pairs when you're pushing in the same item multiple times, but with different priorities. oh okay, I'm way too off. what you are asking for is a Priority Queue like feature. but the emphasis is on providing extra features to heap data structure. and xheap doesn't support having duplicate items. and if you want to insert same items with distinct priorities, you can provide the priority with key argument to the xheap. what xheap doesn't support is having same keys/priorities. So I got confused and proposed a method to have same keys. Regards Srinivas Devaki Junior (3rd yr) student at Indian School of Mines,(IIT Dhanbad) Computer Science and Engineering Department ph: +91 9491 383 249 telegram_id: @eightnoteight From steve+comp.lang.python at pearwood.info Mon Feb 8 23:13:07 2016 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Tue, 09 Feb 2016 15:13:07 +1100 Subject: A sets algorithm References: <1454942992.2532814.515120466.5DA4A683@webmail.messagingengine.com> Message-ID: <56b96755$0$2909$c3e8da3$76491128@news.astraweb.com> On Tuesday 09 February 2016 02:11, Chris Angelico wrote: > That's fine for comparing one file against one other. He started out > by saying he already had a way to compare files for equality. What he > wants is a way to capitalize on that to find all the identical files > in a group. A naive approach would simply compare every file against > every other, for O(N*N) comparisons - but a hash lookup can make that > O(N) on the files themselves, plus (I think) an O(N log N) hash > comparison job, which has much lower constant factors. You're mixing up N's there :-) In the first two (compare every file against every other file, and hash lookup), N stands for the number of files. But in the hash comparison, well, I'm not sure what you mean by that, unless you mean the calculation of the hash itself, which will be O(M) where M is the size of the file. Unfortunately, even using a hash, you still have to do O(N**2) work, or rather, O(N*M) where N is the number of files and M is their size. Assuming that duplicate files are relatively rare, the best way to do this is to collate files with the same size. Looking up the file size ought to be pretty fast. It is an IO operation, but it is approximately constant time in the sense that it doesn't actually depend on the size of the file. (If you're using a programming language or operating system where the only way to find out how big a file is to open it and count the bytes, this obviously won't work for you.) So let's collect files into a separate bucket for each unique file size: bysize = {} for file in files: bysize.setdefault(file.getsize(), []).append(file) In practice, you may find that nearly all files have different sizes, and most of the bucks have only a single file in them. Those files you can just ignore, as they must be unique. So instead of having to compare N files against each of the others, you are left with a much smaller number of buckets, each with (hopefully) only two or three files of the same size. You might have gone from N=1000000 to thirty buckets with five files each. And most of those files will probably differ on the very first byte, or at least within the first 16K of bytes, so it's hardly worth hashing them (especially if the hash algorithm is an expensive cryptographic hash, or if the file is large). I would delay the hash comparison as long as possible, something like this: def __hash__(self): if self._hash is None: self._hash = calculate_hash() # expensive, so we cache it return self._hash def __eq__(self, other): size = self.getsize() if size != other.getsize(): return False if self._hash and other._hash: # Both hashes are already done, so might as well use them. return ( hash(self) == hash(other) # in case of collisions... and self.read() == other.read() ) if size < MAXSIZE: return self.read(size) == other.read(size) else: if self.read(MAXSIZE) != other.read(MAXSIZE): return False else: return ( hash(self) == hash(other) # in case of collisions... and self.read() == other.read() ) If your hash is strong enough that collisions are vanishingly rare, you could ignore the `and self.read() == other.read()` check. -- Steve From mscir at yahoo.com Mon Feb 8 23:22:36 2016 From: mscir at yahoo.com (Mike S) Date: Mon, 8 Feb 2016 20:22:36 -0800 Subject: from scipy.linalg import _fblas ImportError: DLL load failed: The specified module could not be found. Message-ID: I have Python 3.4.4 installed on Windows 7, also IPython, scipy, numpy, statsmodels, and a lot of other modules, and am working through this tutorial http://www.analyticsvidhya.com/blog/2016/02/time-series-forecasting-codes-python/ In Ipython notebook I run this code from statsmodels.tsa.stattools import adfuller def test_stationarity(timeseries): #determining rolling statistics rolmean = pd.rolling_mean(timeseries, window=12) rolstd = pd.rolling_std(timeseries, window=12) #plot rolling statistics orig = plt.plot(timeseries, color='blue', label='Original') mean = plt.plot(rolmean, color='red', label='Rolling Mean') std = plt.plot(rolstd, color='black', label='Rolling Std') plt.legend(loc='best') plt.title('Rolling Mean and Standard Deviation') plt.show(block=false) #perform Dickey=Fuller test print('Results of Dickey-Fuller Test:') dftest = adfuller(timeseries, autolag='AIC') dfoutput = pd.Series(dftest[0:4], index=['Test Statistic','p-value','#Lags Used','Number of Observations Used']) for key, value in dftest[4].items(): dfoutput['Critical Value (%s)'%key]=value print(dfoutput) test_stationary(ts And see this output: --------------------------------------------------------------------------- ImportError Traceback (most recent call last) in () ----> 1 from statsmodels.tsa.stattools import adfuller 2 def test_stationarity(timeseries): 3 #determining rolling statistics 4 rolmean = pd.rolling_mean(timeseries, window=12) 5 rolstd = pd.rolling_std(timeseries, window=12) c:\python34\lib\site-packages\statsmodels\__init__.py in () 6 7 from warnings import simplefilter ----> 8 from .tools.sm_exceptions import (ConvergenceWarning, CacheWriteWarning, 9 IterationLimitWarning, InvalidTestWarning) 10 c:\python34\lib\site-packages\statsmodels\tools\__init__.py in () ----> 1 from .tools import add_constant, categorical c:\python34\lib\site-packages\statsmodels\tools\tools.py in () 6 import numpy.lib.recfunctions as nprf 7 import numpy.linalg as L ----> 8 from scipy.linalg import svdvals 9 from statsmodels.distributions import (ECDF, monotone_fn_inverter, 10 StepFunction) c:\python34\lib\site-packages\scipy\linalg\__init__.py in () 172 from .linalg_version import linalg_version as __version__ 173 --> 174 from .misc import * 175 from .basic import * 176 from .decomp import * c:\python34\lib\site-packages\scipy\linalg\misc.py in () 3 import numpy as np 4 from numpy.linalg import LinAlgError ----> 5 from .blas import get_blas_funcs 6 from .lapack import get_lapack_funcs 7 c:\python34\lib\site-packages\scipy\linalg\blas.py in () 153 import numpy as _np 154 --> 155 from scipy.linalg import _fblas 156 try: 157 from scipy.linalg import _cblas ImportError: DLL load failed: The specified module could not be found. Do I read this correctly to mean that the very last import statement is the one having the problem, "from scipy.linalg import _fblas" How do I troubleshoot this? I'm wondering if I have version conflict between two modules. Thanks, Mike From cfkaran2 at gmail.com Mon Feb 8 23:25:42 2016 From: cfkaran2 at gmail.com (Cem Karan) Date: Mon, 8 Feb 2016 23:25:42 -0500 Subject: Heap Implementation In-Reply-To: References: <56AD3D83.2050308@mail.de> <7C522D08-9D73-48D2-A71D-F1D1D34C02A5@gmail.com> <185DAEA4-8728-4792-A3B7-7F6AC5A7F876@gmail.com> Message-ID: <2308C08E-D2A1-4F16-800F-C2794D30F96B@gmail.com> On Feb 8, 2016, at 10:12 PM, srinivas devaki wrote: > > On Feb 8, 2016 5:17 PM, "Cem Karan" wrote: > > > > On Feb 7, 2016, at 10:15 PM, srinivas devaki wrote: > > > On Feb 8, 2016 7:07 AM, "Cem Karan" wrote: > > > > I know that there are methods of handling this from the client-side (tuples with unique counters come to mind), but if your library can handle it directly, then that could be useful to others as well. > > > > > > yeah it is a good idea to do at client side. > > > but if it should be introduced as feature into the library, instead of tuples, we should just piggyback a single counter it to the self._indexes dict, or better make another self._counts dict which will be light and fast. > > > and if you think again with this method you can easily subclass with just using self._counts dict in your subclass. but still I think it is good to introduce it as a feature in the library. > > > > > > Regards > > > Srinivas Devaki > > > > I meant that the counter is a trick to separate different instances of (item, priority) pairs when you're pushing in the same item multiple times, but with different priorities. > > oh okay, I'm way too off. > > what you are asking for is a Priority Queue like feature. > > but the emphasis is on providing extra features to heap data structure. > > and xheap doesn't support having duplicate items. > > and if you want to insert same items with distinct priorities, you can provide the priority with key argument to the xheap. what xheap doesn't support is having same keys/priorities. > So I got confused and proposed a method to have same keys. > > Regards > Srinivas Devaki No problem, that's what I thought happened. And you're right, I'm looking for a priority queue (not the only reason to use a heap, but a pretty important reason!) Thanks, Cem Karan From rosuav at gmail.com Mon Feb 8 23:27:40 2016 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 9 Feb 2016 15:27:40 +1100 Subject: A sets algorithm In-Reply-To: <56b96755$0$2909$c3e8da3$76491128@news.astraweb.com> References: <1454942992.2532814.515120466.5DA4A683@webmail.messagingengine.com> <56b96755$0$2909$c3e8da3$76491128@news.astraweb.com> Message-ID: On Tue, Feb 9, 2016 at 3:13 PM, Steven D'Aprano wrote: > On Tuesday 09 February 2016 02:11, Chris Angelico wrote: > >> That's fine for comparing one file against one other. He started out >> by saying he already had a way to compare files for equality. What he >> wants is a way to capitalize on that to find all the identical files >> in a group. A naive approach would simply compare every file against >> every other, for O(N*N) comparisons - but a hash lookup can make that >> O(N) on the files themselves, plus (I think) an O(N log N) hash >> comparison job, which has much lower constant factors. > > You're mixing up N's there :-) In the first two (compare every file against > every other file, and hash lookup), N stands for the number of files. But in > the hash comparison, well, I'm not sure what you mean by that, unless you > mean the calculation of the hash itself, which will be O(M) where M is the > size of the file. > > Unfortunately, even using a hash, you still have to do O(N**2) work, or > rather, O(N*M) where N is the number of files and M is their size. > My intention was that every N was "number of files being compared", but it's possible I made a mistake elsewhere. Assuming the hashes become integers too large for a bucket sort (which they certainly will, unless you want inordinate numbers of hash collisions), the code would look something like this: hash_to_filename = defaultdict(list) for fn in files: # Step 1: Hash every file. hash = calculate_hash(fn) # Step 2: Locate all pairs of files with identical hashes hash_to_filename[hash].append(fn) This loop executes once for each file, so calculate_hash() is called once for each file. The cost of that is O(N), or possibly O(N*M). If you're fairly sure the files are going to differ in size, you could use the file size *as* the hash - it's cheap to calculate (no M component whatsoever - just a stat call, and even that could be optimized away in some cases, eg if you're scanning a whole directory), but isn't cryptographically secure and ignores file content altogether. The second step involves one dictionary lookup or insertion for each file. That's an O(log N) operation, where N is the number of elements in the dictionary. So yes, it's not quite the same as the number of files (if every file exists exactly twice, this N will be half the other N), but it's broadly going to correspond. And an O(log N) operation performed N times has an overall cost of O(N log N). I might have something wrong here, but definitely I meant to have the Ns all mean the same thing, like X on a Magic: The Gathering card. :) ChrisA From greg.ewing at canterbury.ac.nz Mon Feb 8 23:48:58 2016 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Tue, 09 Feb 2016 17:48:58 +1300 Subject: A sets algorithm In-Reply-To: References: <1454942992.2532814.515120466.5DA4A683@webmail.messagingengine.com> <56b96755$0$2909$c3e8da3$76491128@news.astraweb.com> Message-ID: Chris Angelico wrote: > hash_to_filename = defaultdict(list) > for fn in files: > # Step 1: Hash every file. > hash = calculate_hash(fn) > > # Step 2: Locate all pairs of files with identical hashes > hash_to_filename[hash].append(fn) I think you can avoid hashing the files altogether. First divide the files into groups of the same size. Then for each group, open all the files at once, read them in parallel and compare them with each other. As soon as you find a difference, split the group into smaller groups. When a group gets down to just one file, you can stop reading that file. Assuming that most of the differing files differ near the beginning, this strategy means that you will hardly ever have to read a whole file. Hashing the files beforehand, in contrast, requires reading all of every file every time. -- Greg From snailcoder at retrosite.invalid Tue Feb 9 02:21:22 2016 From: snailcoder at retrosite.invalid (Grobu) Date: Tue, 9 Feb 2016 08:21:22 +0100 Subject: Set Operations on Dicts In-Reply-To: References: Message-ID: On 08/02/16 17:12, Ian Kelly wrote: > dict does already expose set-like views. How about: > > {k: d[k] for k in d.keys() & s} # d & s > {k: d[k] for k in d.keys() - s} # d - s > Interesting. But seemingly only applies to Python 3. From frank at chagford.com Tue Feb 9 02:33:49 2016 From: frank at chagford.com (Frank Millman) Date: Tue, 9 Feb 2016 09:33:49 +0200 Subject: asyncio and blocking - an update Message-ID: Hi all Some of you may have been following my attempts to modify my asyncio app so that it does not block when accessing the database. Here is an update. I came up with what felt like a good idea. Run the database handler in a separate thread, pass requests to it using a queue.Queue, and get it to pass results back using an asyncio.Queue. It works, but I had a vague sense that performance was a bit sluggish, so I tried the 'recommended' approach of using asyncio.run_in_executor() to execute database calls in a separate thread. It felt a bit faster. Now I have written a proper timing test, and the recommended approach is much faster. I am not 100% sure of the reason, but I think the problem is that, with my method, when the database tries to 'put' a row on the return queue, it has to use 'loop.call_soon_threadsafe()', and this seems to create a bottleneck. It would not matter so much if I used cur.fetchall(), and sent all the rows back in one 'put', but I really wanted to iterate over the cursor and 'put' a row at a time. The idea was that it would be truly asynchronous from end to end - the database handler would retrieve rows one at a time, and via the queue my app would process them one at a time, all without blocking. I can switch to fetchall(), but then there is no benefit over the recommended approach. Although fetchall() is non-blocking, once the rows are received as a list the function that processes them effectively does block. If that became a problem one could use an Asynchronous Iterator to process the list, but I have not had that need yet. Writing timing tests is tricky. It is possible that under some circumstances, with a heavy load, a large table, and a time-consuming function to process each row, the overhead of call_soon_threadsafe() would be minimised and my approach might be effective. For now, however, I will regretfully abandon my approach and stick with run_in_executor(). Frank Millman From breamoreboy at yahoo.co.uk Tue Feb 9 04:33:00 2016 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Tue, 9 Feb 2016 09:33:00 +0000 Subject: from scipy.linalg import _fblas ImportError: DLL load failed: The specified module could not be found. In-Reply-To: References: Message-ID: On 09/02/2016 04:22, Mike S via Python-list wrote: > I have Python 3.4.4 installed on Windows 7, also IPython, scipy, numpy, > statsmodels, and a lot of other modules, and am working through this > tutorial > http://www.analyticsvidhya.com/blog/2016/02/time-series-forecasting-codes-python/ > [snip bulk of code and traceback] > 154 > --> 155 from scipy.linalg import _fblas > 156 try: > 157 from scipy.linalg import _cblas > > ImportError: DLL load failed: The specified module could not be found. > > Do I read this correctly to mean that the very last import statement is > the one having the problem, > > "from scipy.linalg import _fblas" > > How do I troubleshoot this? I'm wondering if I have version conflict > between two modules. Alomost certainly, hopefully this link will help. http://stackoverflow.com/questions/21350153/error-importing-scipy-linalg-on-windows-python-3-3 > > Thanks, > Mike No problem :) -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From jenswaelkens at gmail.com Tue Feb 9 04:33:14 2016 From: jenswaelkens at gmail.com (jenswaelkens at gmail.com) Date: Tue, 9 Feb 2016 01:33:14 -0800 (PST) Subject: [newbie] how to create log files Message-ID: <5fb6c529-b5b1-495a-9f3f-26ef50d44fa0@googlegroups.com> When I run my Python scripts from the command prompt in Linux, I can make visible all kinds of information I want to check by using print statements e.g. print (top.winfo_width()), this is very useful when debugging. However, the final version of my program won't be run from the command line, but by simply clicking on its icon on the desktop. I'd like to have a log-file which gathers the output of all those print statements, so I still can check the same information I got before. Can anyone here tell me how this is accomplished the best way? thanks and kind regards, Jens From breamoreboy at yahoo.co.uk Tue Feb 9 04:40:42 2016 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Tue, 9 Feb 2016 09:40:42 +0000 Subject: Heap Implementation In-Reply-To: <2308C08E-D2A1-4F16-800F-C2794D30F96B@gmail.com> References: <56AD3D83.2050308@mail.de> <7C522D08-9D73-48D2-A71D-F1D1D34C02A5@gmail.com> <185DAEA4-8728-4792-A3B7-7F6AC5A7F876@gmail.com> <2308C08E-D2A1-4F16-800F-C2794D30F96B@gmail.com> Message-ID: On 09/02/2016 04:25, Cem Karan wrote: > > No problem, that's what I thought happened. And you're right, I'm looking for a priority queue (not the only reason to use a heap, but a pretty important reason!) > I'm assuming I've missed the explanation, so what is the problem again with https://docs.python.org/3/library/queue.html#queue.PriorityQueue or even https://docs.python.org/3/library/asyncio-queue.html#asyncio.PriorityQueue ? -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From marco.nawijn at colosso.nl Tue Feb 9 04:41:20 2016 From: marco.nawijn at colosso.nl (marco.nawijn at colosso.nl) Date: Tue, 9 Feb 2016 01:41:20 -0800 (PST) Subject: [newbie] how to create log files In-Reply-To: <5fb6c529-b5b1-495a-9f3f-26ef50d44fa0@googlegroups.com> References: <5fb6c529-b5b1-495a-9f3f-26ef50d44fa0@googlegroups.com> Message-ID: Hello Jens, Are you aware of Python's own logging facility? It is quite powerful and flexible. Python 2: https://docs.python.org/2/library/logging.html Python 3: https://docs.python.org/3/library/logging.html Marco From breamoreboy at yahoo.co.uk Tue Feb 9 04:46:36 2016 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Tue, 9 Feb 2016 09:46:36 +0000 Subject: [newbie] how to create log files In-Reply-To: <5fb6c529-b5b1-495a-9f3f-26ef50d44fa0@googlegroups.com> References: <5fb6c529-b5b1-495a-9f3f-26ef50d44fa0@googlegroups.com> Message-ID: On 09/02/2016 09:33, jenswaelkens at gmail.com wrote: Hello and welcome. > When I run my Python scripts from the command prompt in Linux, I can make visible all kinds of information I want to check by using print statements > e.g. print (top.winfo_width()), this is very useful when debugging. > However, the final version of my program won't be run from the command line, but by simply clicking on its icon on the desktop. I'd like to have > a log-file which gathers the output of all those print statements, so I still can check the same information I got before. > Can anyone here tell me how this is accomplished the best way? > > thanks and kind regards, > Jens > https://docs.python.org/3/library/logging.html -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From asdasdqeqe at gmail.com Tue Feb 9 05:49:27 2016 From: asdasdqeqe at gmail.com (asdasdqeqe at gmail.com) Date: Tue, 9 Feb 2016 02:49:27 -0800 (PST) Subject: newbie. how can i solve this looping problem? Message-ID: http://pastebin.com/Khrm3gHq for the code above, everytime it scraps off tweets and loads the next 13 tweets, it'll re-run through the previous scrapped tweets before recording the new ones. I'm up to 700 over tweets and it'll keep re-running the previous 700 before adding the final 13 to the list. I'm looking at a few thousand tweets so it'll slow down as the tweets get more and more. Any idea how i can stop it from going over the previous tweets and just add the last few? how can i get this line elem = WebDriverWait(driver,10).until(EC.visibility_of_element_located((By.CSS_SELECTOR,lastTweetCss))) to move on further in the code if it doesn't find anything after waiting for 10 sec? From cfkaran2 at gmail.com Tue Feb 9 06:44:01 2016 From: cfkaran2 at gmail.com (Cem Karan) Date: Tue, 9 Feb 2016 06:44:01 -0500 Subject: Heap Implementation In-Reply-To: References: <56AD3D83.2050308@mail.de> <7C522D08-9D73-48D2-A71D-F1D1D34C02A5@gmail.com> <185DAEA4-8728-4792-A3B7-7F6AC5A7F876@gmail.com> <2308C08E-D2A1-4F16-800F-C2794D30F96B@gmail.com> Message-ID: <65105210-BAE5-4CD1-8C8C-73FC51982901@gmail.com> On Feb 9, 2016, at 4:40 AM, Mark Lawrence wrote: > On 09/02/2016 04:25, Cem Karan wrote: >> >> No problem, that's what I thought happened. And you're right, I'm looking for a priority queue (not the only reason to use a heap, but a pretty important reason!) >> > > I'm assuming I've missed the explanation, so what is the problem again with https://docs.python.org/3/library/queue.html#queue.PriorityQueue or even https://docs.python.org/3/library/asyncio-queue.html#asyncio.PriorityQueue ? Efficiently changing the the priority of items already in the queue/deleting items in the queue (not the first item). This comes up a LOT in event-based simulators where it's easier to tentatively add an event knowing that you might need to delete it or change it later. Thanks, Cem Karan From vek.m1234 at gmail.com Tue Feb 9 06:55:56 2016 From: vek.m1234 at gmail.com (Veek. M) Date: Tue, 09 Feb 2016 17:25:56 +0530 Subject: coroutine, throw, yield, call-stack and exception handling References: Message-ID: Ian Kelly wrote: > On Mon, Feb 8, 2016 at 2:17 AM, Veek. M wrote: >> **************************** >> Exceptions can be raised inside a coroutine using the throw( >> >> Exceptions raised in this manner will originate at the currently >> executing yield state-ment in the coroutine.A coroutine can elect to >> catch exceptions and handle them as appropriate. It is not safe to >> use throw() as an asynchronous signal to a coroutine?it should never >> be invoked from a separate execution thread or in a signal handler. >> **************************** >> >> What does Beazley mean by this: 'will originate at the currently >> executing yield state-ment in the coroutine' >> >> If he's throw'ing an exception surely it originates at the throw: >> >> def mycoroutine(): >> while len(n) > 2: >> n = (yield) >> >> throw('RuntimeError' "die!") > > The "throw" is not called from inside the coroutine. It's a method of > the generator object, and it's used by the calling code. It's similar > to calling the send method, except that instead of passing a value to > be returned by the yield expression, it passes an exception to be > raised inside the coroutine at the yield expression. > > Example: > > def mycoroutine(): > n = 0 > while True: > try: > n = (yield n) > except SomeException: > n = 42 > > coro = mycoroutine() > coro.next() > for i in range(100): > if i % 6 == 0: > coro.send(i % 6) > else: > coro.throw(SomeException()) > > >> Also this bit: >> *********************** >> If a coroutine returns values, some care is required if exceptions >> raised with throw() are being handled. If you raise an exception in a >> coroutine using throw(), the value passed to the next yield in the >> coroutine will be returned as the result of throw(). If >> you need this value and forget to save it, it will be lost. >> *********************** >> >> def coroutine(): >> while True: >> line = (yield result) >> >> throw(FooException) >> >> where is the question of a 'yield'? You'll exit the coroutine >> straight away.. > > Taking my example from above, after SomeException is caught, the next > value yielded inside the coroutine will be the return value of the > coro.throw() call. This may be surprising if you're only expecting > coro.send() and not coro.throw() to return yielded values. Thanks, that made it abundantly clear :) From cl at isbd.net Tue Feb 9 06:57:49 2016 From: cl at isbd.net (cl at isbd.net) Date: Tue, 9 Feb 2016 11:57:49 +0000 Subject: How to trap this error (comes from sqlite3) Message-ID: I have the following code snippet populating a wxPython grid with data from a database. :- # # # populate grid with data # all = self.cur.execute("SELECT * from " + table + " ORDER by id ") for row in all: row_num = row[0] cells = row[1:] for col in range(len(cells)): if cells[col] != None and cells[col] != "null": xx = cells[col] if not isinstance(xx, basestring): xx = str(xx) print("row: ",row_num, "col: ", col, "value: ", xx) self.SetCellValue(row_num, col, xx) It works fine until it hits an invalid character in one of the columns. The print is just a temporary diagnostic. The output I get, when it hits an invalid character is:- ('row: ', 5814, 'col: ', 9, 'value: ', u'') ('row: ', 5814, 'col: ', 10, 'value: ', '10.5') ('row: ', 5814, 'col: ', 11, 'value: ', u'') ('row: ', 5814, 'col: ', 12, 'value: ', u' Fuel (with inter-tank tap open) is at about 10.5 - 11cm in the sight glass before setting out.') ('row: ', 6186, 'col: ', 0, 'value: ', '0') Traceback (most recent call last): File "/home/chris/bin/pg.py", line 100, in grid = Grid(frame, dbCon, table) File "/home/chris/bin/pg.py", line 52, in __init__ self.SetCellValue(row_num, col, xx) File "/usr/lib/python2.7/dist-packages/wx-3.0-gtk2/wx/grid.py", line 2016, in SetCellValue return _grid.Grid_SetCellValue(*args, **kwargs) sqlite3.OperationalError: Could not decode to UTF-8 column 'dayno' with text '?' It's absolutely right, there is a non-UTF character in the column. However I don't want to have to clean up the data, it would take rather a long time. How can I trap the error and just put a Null or zero in the datagrid? Where do I put the try:/except: ? -- Chris Green ? From __peter__ at web.de Tue Feb 9 07:39:42 2016 From: __peter__ at web.de (Peter Otten) Date: Tue, 09 Feb 2016 13:39:42 +0100 Subject: How to trap this error (comes from sqlite3) References: Message-ID: cl at isbd.net wrote: > I have the following code snippet populating a wxPython grid with data > from a database. :- > > # > # > # populate grid with data > # > all = self.cur.execute("SELECT * from " + table + " ORDER by id ") > for row in all: > row_num = row[0] > cells = row[1:] > for col in range(len(cells)): > if cells[col] != None and cells[col] != "null": > xx = cells[col] > if not isinstance(xx, basestring): > xx = str(xx) > print("row: ",row_num, "col: ", col, "value: ", xx) > > self.SetCellValue(row_num, col, xx) > > > It works fine until it hits an invalid character in one of the > columns. The print is just a temporary diagnostic. > > The output I get, when it hits an invalid character is:- > > ('row: ', 5814, 'col: ', 9, 'value: ', u'') > ('row: ', 5814, 'col: ', 10, 'value: ', '10.5') > ('row: ', 5814, 'col: ', 11, 'value: ', u'') > ('row: ', 5814, 'col: ', 12, 'value: ', u' Fuel (with inter-tank tap > open) is at about 10.5 - 11cm in the sight glass before setting out.') > ('row: ', 6186, 'col: ', 0, 'value: ', '0') Traceback (most recent > call last): > File "/home/chris/bin/pg.py", line 100, in > grid = Grid(frame, dbCon, table) > File "/home/chris/bin/pg.py", line 52, in __init__ > self.SetCellValue(row_num, col, xx) > File "/usr/lib/python2.7/dist-packages/wx-3.0-gtk2/wx/grid.py", line > 2016, in SetCellValue > return _grid.Grid_SetCellValue(*args, **kwargs) > sqlite3.OperationalError: Could not decode to UTF-8 column 'dayno' > with text '?' > > > It's absolutely right, there is a non-UTF character in the column. > However I don't want to have to clean up the data, it would take > rather a long time. How can I trap the error and just put a Null or > zero in the datagrid? > > Where do I put the try:/except: ? Handle the problem earlier on by setting a text_factory that ignores or replaces the offending bytes: db = sqlite3.connect(...) db.text_factory = lambda b: b.decode("utf-8", "replace") https://docs.python.org/2.7/library/sqlite3.html#sqlite3.Connection.text_factory https://docs.python.org/2/howto/unicode.html From breamoreboy at yahoo.co.uk Tue Feb 9 07:53:20 2016 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Tue, 9 Feb 2016 12:53:20 +0000 Subject: How to trap this error (comes from sqlite3) In-Reply-To: References: Message-ID: On 09/02/2016 11:57, cl at isbd.net wrote: > I have the following code snippet populating a wxPython grid with data > from a database. :- > > # > # > # populate grid with data > # > all = self.cur.execute("SELECT * from " + table + " ORDER by id ") > for row in all: > row_num = row[0] > cells = row[1:] > for col in range(len(cells)): > if cells[col] != None and cells[col] != "null": > xx = cells[col] > if not isinstance(xx, basestring): > xx = str(xx) > print("row: ",row_num, "col: ", col, "value: ", xx) The usual way of writing the above loop is:- for cell in cells: if cell != None and cell != "null": xx = cell if not isinstance(xx, basestring): xx = str(xx) > > self.SetCellValue(row_num, col, xx) > > > It works fine until it hits an invalid character in one of the > columns. The print is just a temporary diagnostic. > > The output I get, when it hits an invalid character is:- > > ('row: ', 5814, 'col: ', 9, 'value: ', u'') > ('row: ', 5814, 'col: ', 10, 'value: ', '10.5') > ('row: ', 5814, 'col: ', 11, 'value: ', u'') > ('row: ', 5814, 'col: ', 12, 'value: ', u' Fuel (with inter-tank tap open) is at about 10.5 - 11cm in the sight glass before setting out.') > ('row: ', 6186, 'col: ', 0, 'value: ', '0') > Traceback (most recent call last): > File "/home/chris/bin/pg.py", line 100, in > grid = Grid(frame, dbCon, table) > File "/home/chris/bin/pg.py", line 52, in __init__ > self.SetCellValue(row_num, col, xx) > File "/usr/lib/python2.7/dist-packages/wx-3.0-gtk2/wx/grid.py", line 2016, in SetCellValue > return _grid.Grid_SetCellValue(*args, **kwargs) > sqlite3.OperationalError: Could not decode to UTF-8 column 'dayno' with text '?' > > > It's absolutely right, there is a non-UTF character in the column. > However I don't want to have to clean up the data, it would take > rather a long time. How can I trap the error and just put a Null or > zero in the datagrid? > > Where do I put the try:/except: ? > The rule is always keep the try/except to the bare minimum hence. try: self.SetCellValue(row_num, col, xx) except sqlite3.OperationalError: doSomething() -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From joel.goldstick at gmail.com Tue Feb 9 08:49:18 2016 From: joel.goldstick at gmail.com (Joel Goldstick) Date: Tue, 9 Feb 2016 08:49:18 -0500 Subject: newbie. how can i solve this looping problem? In-Reply-To: References: Message-ID: On Tue, Feb 9, 2016 at 5:49 AM, wrote: > http://pastebin.com/Khrm3gHq > > for the code above, everytime it scraps off tweets and loads the next 13 > tweets, it'll re-run through the previous scrapped tweets before recording > the new ones. I'm up to 700 over tweets and it'll keep re-running the > previous 700 before adding the final 13 to the list. I'm looking at a few > thousand tweets so it'll slow down as the tweets get more and more. Any > idea how i can stop it from going over the previous tweets and just add the > last few? > > how can i get this line > elem = > WebDriverWait(driver,10).until(EC.visibility_of_element_located((By.CSS_SELECTOR,lastTweetCss))) > to move on further in the code if it doesn't find anything after waiting > for 10 sec? > -- > https://mail.python.org/mailman/listinfo/python-list > I can't answer your question, but to make it easier for someone who can, put your code in your question, edit out all the code that has nothing to do with your question, show what you get, and what you want. If you get a traceback, copy and paste it in the qeustion Good luck -- Joel Goldstick http://joelgoldstick.com/stats/birthdays From anitagoyal571 at gmail.com Tue Feb 9 08:55:10 2016 From: anitagoyal571 at gmail.com (anitagoyal571 at gmail.com) Date: Tue, 9 Feb 2016 05:55:10 -0800 (PST) Subject: First Program with Python! Message-ID: <59666b2f-1368-4652-9791-3b267e2fe634@googlegroups.com> Python is a very powerful high-level, object-oriented programming language.Python has a very easy-to-use and simple syntax, making it the perfect language for someone trying to learn computer programming for the first time. Python is an interpreted language. Interpreter is a program that converts the high-level program we write into low-level program that the computer understands. Click here to find the difference between interpreter and compiler. This tutorial is based on Python 3 and all the examples in this tutorial have been tested and verified in Python. Start learning Python from basics to advance levels here... https://goo.gl/hGzm6o From tdsimpson at gmail.com Tue Feb 9 09:04:44 2016 From: tdsimpson at gmail.com (MrPink) Date: Tue, 9 Feb 2016 06:04:44 -0800 (PST) Subject: Searching Sets (Lottery Results) In-Reply-To: References: <6afb7298-6da7-44f9-847e-9603554c2f09@googlegroups.com> Message-ID: <09c7f060-e01f-48ff-a02d-b5ba1f52f41a@googlegroups.com> On Monday, February 8, 2016 at 7:05:24 PM UTC-5, Chris Angelico wrote: > On Tue, Feb 9, 2016 at 8:45 AM, MrPink wrote: > > I load the lottery drawings into memory for searching with the following code although, it is incomplete. I am stuck and need some guidance. > > > > The set datatype seems to be the best for searching, but how best can I implement it? > > > > And I want the results to highlight the numbers that were matched. For example, if the white balls in the drawing are: > > "42 15 06 05 29" > > > > AND the numbers on the lottery ticket are: > > "06 15 32 42 56" > > > > THEN the display might look like: > > "06* 15* 32 42* 56" > > > > WHERE * signifies a match. > > > > This suggests that there is an order to the numbers on your ticket > (you want to print them out in the same order), but not to the winning > numbers, which are simply a set. The easiest way to handle that would > be to iterate over your numbers, asking "if number in > winning_numbers:", and printing out a "match" marker if it is or a > "non-match" marker if it isn't. > > ChrisA Thanks Chris. Very good point. I was just too deep in the weeds to see that simple solution. I was overthinking it. ;-) Sincerely, From asdasdqeqe at gmail.com Tue Feb 9 09:11:53 2016 From: asdasdqeqe at gmail.com (asdasdqeqe at gmail.com) Date: Tue, 9 Feb 2016 06:11:53 -0800 (PST) Subject: newbie. how can i solve this looping problem? In-Reply-To: References: Message-ID: http://pastebin.com/uQSW5iwZ here's the part of the code which I would like to change. I don't know how to get the following line to not "Timeout" and instead continue onwards to printTweet(driver) elem = WebDriverWait(driver,10).until(EC.visibility_of_element_located((By.CSS_SELECTOR,lastTweetCss))) From breamoreboy at yahoo.co.uk Tue Feb 9 09:27:20 2016 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Tue, 9 Feb 2016 14:27:20 +0000 Subject: Heap Implementation In-Reply-To: <65105210-BAE5-4CD1-8C8C-73FC51982901@gmail.com> References: <56AD3D83.2050308@mail.de> <7C522D08-9D73-48D2-A71D-F1D1D34C02A5@gmail.com> <185DAEA4-8728-4792-A3B7-7F6AC5A7F876@gmail.com> <2308C08E-D2A1-4F16-800F-C2794D30F96B@gmail.com> <65105210-BAE5-4CD1-8C8C-73FC51982901@gmail.com> Message-ID: On 09/02/2016 11:44, Cem Karan wrote: > > On Feb 9, 2016, at 4:40 AM, Mark Lawrence wrote: > >> On 09/02/2016 04:25, Cem Karan wrote: >>> >>> No problem, that's what I thought happened. And you're right, I'm looking for a priority queue (not the only reason to use a heap, but a pretty important reason!) >>> >> >> I'm assuming I've missed the explanation, so what is the problem again with https://docs.python.org/3/library/queue.html#queue.PriorityQueue or even https://docs.python.org/3/library/asyncio-queue.html#asyncio.PriorityQueue ? > > Efficiently changing the the priority of items already in the queue/deleting items in the queue (not the first item). This comes up a LOT in event-based simulators where it's easier to tentatively add an event knowing that you might need to delete it or change it later. > > Thanks, > Cem Karan > Thanks for that, but from the sounds of it sooner you than me :) -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From cl at isbd.net Tue Feb 9 09:31:25 2016 From: cl at isbd.net (cl at isbd.net) Date: Tue, 9 Feb 2016 14:31:25 +0000 Subject: How to trap this error (comes from sqlite3) References: Message-ID: Peter Otten <__peter__ at web.de> wrote: > cl at isbd.net wrote: > > > It's absolutely right, there is a non-UTF character in the column. > > However I don't want to have to clean up the data, it would take > > rather a long time. How can I trap the error and just put a Null or > > zero in the datagrid? > > > > Where do I put the try:/except: ? > > Handle the problem earlier on by setting a text_factory that ignores or > replaces the offending bytes: > > db = sqlite3.connect(...) > db.text_factory = lambda b: b.decode("utf-8", "replace") > > https://docs.python.org/2.7/library/sqlite3.html#sqlite3.Connection.text_factory > https://docs.python.org/2/howto/unicode.html > Brilliant, thank you, works perfectly and (IMHO) is about the neatest possible solution. -- Chris Green ? From shaunak.bangale at gmail.com Tue Feb 9 09:55:14 2016 From: shaunak.bangale at gmail.com (shaunak.bangale at gmail.com) Date: Tue, 9 Feb 2016 06:55:14 -0800 (PST) Subject: ImportError: cannot import name 'RAND_egd' Message-ID: <4c113094-db27-4b60-9140-eded08939ab0@googlegroups.com> Hi, I am trying to run a 60 lines Python code which is running on a mac machine but on windows machine, I am getting this error when I run on it on shell(open file and run module). I have Python 3.5 installed. from _ssl import RAND_status, RAND_egd, RAND_add ImportError: cannot import name 'RAND_egd' Form forums, I found that it is a common error but could not find a good solution that will work for me. One of the ways was to create scripts folder and putting easy_install.exe and then running easy_install pip but that gave me sytnax error. Please advise. Thanks in advance. Regards Shaunak From ian.g.kelly at gmail.com Tue Feb 9 10:14:15 2016 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Tue, 9 Feb 2016 08:14:15 -0700 Subject: Set Operations on Dicts In-Reply-To: References: Message-ID: On Tue, Feb 9, 2016 at 12:21 AM, Grobu wrote: > On 08/02/16 17:12, Ian Kelly wrote: > >> dict does already expose set-like views. How about: >> >> {k: d[k] for k in d.keys() & s} # d & s >> {k: d[k] for k in d.keys() - s} # d - s >> > Interesting. But seemingly only applies to Python 3. Substitute d.viewkeys() for d.keys() in Python 2.7. From fillmore_remove at hotmail.com Tue Feb 9 11:41:14 2016 From: fillmore_remove at hotmail.com (Fillmore) Date: Tue, 09 Feb 2016 11:41:14 -0500 Subject: Cygwin and Python3 Message-ID: Hi, I am having a hard time making my Cygwin run Python 3.5 (or Python 2.7 for that matter). The command will hang and nothing happens. A cursory search on the net reveals many possibilities, which might mean a lot of trial and error, which I would very much like to avoid. Any suggestions on how I can get cygwin and Python3.5 to play together like brother and sister? thanks From alvin.hacopian at gmail.com Tue Feb 9 14:29:22 2016 From: alvin.hacopian at gmail.com (alvin.hacopian at gmail.com) Date: Tue, 9 Feb 2016 11:29:22 -0800 (PST) Subject: Cygwin and Python3 In-Reply-To: References: Message-ID: <09fc58ce-8106-4266-8a3d-13b1dfcedc90@googlegroups.com> On Tuesday, February 9, 2016 at 8:41:43 AM UTC-8, Fillmore wrote: > Hi, I am having a hard time making my Cygwin run Python 3.5 (or Python 2.7 for that matter). > The command will hang and nothing happens. > > A cursory search on the net reveals many possibilities, which might mean a lot > of trial and error, which I would very much like to avoid. > > Any suggestions on how I can get cygwin and Python3.5 to play together like brother and sister? > > thanks Please see bellow: $ whereis python python: /usr/bin/python /usr/bin/python2.7-config /usr/bin/python3.2 /usr/bin/python3.2m-config /usr/lib/python2.6 /usr/lib/python2.7 /usr/lib/python3.2 /usr/local/bin/python /usr/local/bin/python2.7 /usr/local/lib/python2.7 /usr/include/python2.7 /usr/include/python3.2m /usr/share/man/man1/python.1.gz $ echo $PATH /usr/local/bin:/usr/bin:/cygdrive/c/Windows/system32:/cygdrive/c/Windows:/cygdrive/c/Windows/System32/Wbem:/cygdrive/c/Windows/System32/WindowsPowerShell/v1.0:/cygdrive/c/Program Files (x86)/Vim/vim73:/cygdrive/c/Program Files/WIDCOMM/Bluetooth Software:/cygdrive/c/Program Files/WIDCOMM/Bluetooth Software/syswow64:/cygdrive/c/Program Files (x86)/Skype/Phone:/cygdrive/c/opscode/chef/bin:/cygdrive/c/opscode/chef/embedded/bin:/opt/apache2/bin:/cygdrive/c/Program Files (x86)/QuickTime/QTSystem:/usr/sbin:/usr/lib/lapack $ ls -l /usr/bin/python rm /usr/bin/python $ ln -s /usr/bin/python /usr/bin/python3.2m.exe $ /usr/bin/python --version Python 3.2.5 $ pydoc modules From fillmore_remove at hotmail.com Tue Feb 9 15:19:38 2016 From: fillmore_remove at hotmail.com (Fillmore) Date: Tue, 9 Feb 2016 15:19:38 -0500 Subject: Cygwin and Python3 References: <09fc58ce-8106-4266-8a3d-13b1dfcedc90@googlegroups.com> Message-ID: On 2/9/2016 2:29 PM, alvin.hacopian at gmail.com wrote: > > $ ls -l /usr/bin/python > rm /usr/bin/python > > $ ln -s /usr/bin/python /usr/bin/python3.2m.exe > > $ /usr/bin/python --version > Python 3.2.5 > > $ pydoc modules > Still no luck (: ~ $ python --version Python 3.5.1 ~ $ python (..hangs indefinitely) ^C ~ $ pydoc modules -bash: pydoc: command not found ~ $ echo $PATH /usr/local/bin:/usr/bin:/cygdrive/c/Python27:/cygdrive/c/ Python27/Scripts:/cygdrive/c/Windows/system32:/cygdrive/ c/Windows:/cygdrive/c/Windows/System32/Wbem:/cygdrive/ c/Windows/System32/WindowsPowerShell/v1.0:/cygdrive/ c/Program Files (x86)/Common Files/Roxio Shared/OEM/ DLLShared:/cygdrive/c/Program Files (x86)/Common Files/ Roxio Shared/OEM/DLLShared:/cygdrive/c/Program Files (x86)/Common Files/Roxio Shared/OEM/12.0/ DLLShared:/cygdrive/c/Program Files (x86)/Roxio/OEM/ AudioCore:/cygdrive/c/unxutils/bin:/cygdrive/c/unxutils /usr/local/wbin:/cygdrive/c/strawberry/c/bin:/cygdrive/ c/strawberry/perl/site/bin:/cygdrive/c/strawberry/ perl/bin:/cygdrive/c/Program Files/Intel/WiFi/bin:/ cygdrive/c/Program Files/Common Files/Intel/ WirelessCommon:/cygdrive/c/Users/user/AppData/Local/ Programs/Python/Python35/Scripts:/cygdrive/c/Users/ user/AppData/Local/Programs/Python/Python35:%APPDATA% /Python/Scripts:/cygdrive/c/Program Files/Intel/WiFi/ bin:/cygdrive/c/Program Files/Common Files/Intel/ WirelessCommon From alvin.hacopian at gmail.com Tue Feb 9 15:30:37 2016 From: alvin.hacopian at gmail.com (alvin.hacopian at gmail.com) Date: Tue, 9 Feb 2016 12:30:37 -0800 (PST) Subject: Cygwin and Python3 In-Reply-To: References: <09fc58ce-8106-4266-8a3d-13b1dfcedc90@googlegroups.com> Message-ID: <98ba5ab8-726f-4939-8a88-b56df4701b05@googlegroups.com> On Tuesday, February 9, 2016 at 12:20:06 PM UTC-8, Fillmore wrote: > On 2/9/2016 2:29 PM, alvin.hacopian at gmail.com wrote: > > > > > > $ ls -l /usr/bin/python > > rm /usr/bin/python > > > > $ ln -s /usr/bin/python /usr/bin/python3.2m.exe > > > > $ /usr/bin/python --version > > Python 3.2.5 > > > > $ pydoc modules > > > > Still no luck (: > > ~ > $ python --version > Python 3.5.1 > > ~ > $ python > (..hangs indefinitely) > ^C > > ~ > $ pydoc modules > -bash: pydoc: command not found > > ~ > $ echo $PATH > /usr/local/bin:/usr/bin:/cygdrive/c/Python27:/cygdrive/c/ > Python27/Scripts:/cygdrive/c/Windows/system32:/cygdrive/ > c/Windows:/cygdrive/c/Windows/System32/Wbem:/cygdrive/ > c/Windows/System32/WindowsPowerShell/v1.0:/cygdrive/ > c/Program Files (x86)/Common Files/Roxio Shared/OEM/ > DLLShared:/cygdrive/c/Program Files (x86)/Common Files/ > Roxio Shared/OEM/DLLShared:/cygdrive/c/Program > Files (x86)/Common Files/Roxio Shared/OEM/12.0/ > DLLShared:/cygdrive/c/Program Files (x86)/Roxio/OEM/ > AudioCore:/cygdrive/c/unxutils/bin:/cygdrive/c/unxutils > /usr/local/wbin:/cygdrive/c/strawberry/c/bin:/cygdrive/ > c/strawberry/perl/site/bin:/cygdrive/c/strawberry/ > perl/bin:/cygdrive/c/Program Files/Intel/WiFi/bin:/ > cygdrive/c/Program Files/Common Files/Intel/ > WirelessCommon:/cygdrive/c/Users/user/AppData/Local/ > Programs/Python/Python35/Scripts:/cygdrive/c/Users/ > user/AppData/Local/Programs/Python/Python35:%APPDATA% > /Python/Scripts:/cygdrive/c/Program Files/Intel/WiFi/ > bin:/cygdrive/c/Program Files/Common Files/Intel/ > WirelessCommon When you run the cygwin installer you have the option of installing 2.7 and 3.2.5, by default it will install 2.7 and 3.2 together. After running the installer run whereis python and use the alternatives to change it or use python3 instead of python #!/usr/bin/python3 Hope this helps. From ian.g.kelly at gmail.com Tue Feb 9 15:32:22 2016 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Tue, 9 Feb 2016 13:32:22 -0700 Subject: ImportError: cannot import name 'RAND_egd' In-Reply-To: <4c113094-db27-4b60-9140-eded08939ab0@googlegroups.com> References: <4c113094-db27-4b60-9140-eded08939ab0@googlegroups.com> Message-ID: On Tue, Feb 9, 2016 at 7:55 AM, wrote: > Hi, > > I am trying to run a 60 lines Python code which is running on a mac machine but on windows machine, I am getting this error when I run on it on shell(open file and run module). I have Python 3.5 installed. > > from _ssl import RAND_status, RAND_egd, RAND_add > ImportError: cannot import name 'RAND_egd' Why are you importing these directly from the "_ssl" C module and not from the "ssl" wrapper module? Anything that starts with an _ should be considered a private implementation detail and shouldn't be relied upon. > Form forums, I found that it is a common error but could not find a good solution that will work for me. > > One of the ways was to create scripts folder and putting easy_install.exe and then running easy_install pip but that gave me sytnax error. > > Please advise. Thanks in advance. The ssl module in the standard library has this: try: from _ssl import RAND_egd except ImportError: # LibreSSL does not provide RAND_egd pass So it looks like you cannot depend on ssl.RAND_egd to be present. From fillmore_remove at hotmail.com Tue Feb 9 16:47:58 2016 From: fillmore_remove at hotmail.com (Fillmore) Date: Tue, 9 Feb 2016 16:47:58 -0500 Subject: Cygwin and Python3 References: <09fc58ce-8106-4266-8a3d-13b1dfcedc90@googlegroups.com> <98ba5ab8-726f-4939-8a88-b56df4701b05@googlegroups.com> Message-ID: On 2/9/2016 3:30 PM, alvin.hacopian at gmail.com wrote: > > When you run the cygwin installer you have the option of installing 2.7 > and 3.2.5, by default it will install 2.7 and 3.2 together. > After running the installer run whereis python and use the alternatives > to change it or use python3 instead of python #!/usr/bin/python3 > > Hope this helps. I see. I was trying to do it the Perl way. I simply linked the strawberry perl.exe from cygwin environemnt and it replaced the built in perl that sucked. OK. Backtrack. I'll try with a purely cygwin solution... Thank you From fillmore_remove at hotmail.com Tue Feb 9 17:36:20 2016 From: fillmore_remove at hotmail.com (Fillmore) Date: Tue, 9 Feb 2016 17:36:20 -0500 Subject: Cygwin and Python3 References: <09fc58ce-8106-4266-8a3d-13b1dfcedc90@googlegroups.com> <98ba5ab8-726f-4939-8a88-b56df4701b05@googlegroups.com> Message-ID: On 2/9/2016 4:47 PM, Fillmore wrote: > On 2/9/2016 3:30 PM, alvin.hacopian at gmail.com wrote: > >> >> When you run the cygwin installer you have the option of installing 2.7 > > and 3.2.5, by default it will install 2.7 and 3.2 together. > > After running the installer run whereis python and use the alternatives > > to change it or use python3 instead of python #!/usr/bin/python3 >> >> Hope this helps. > > > I see. I was trying to do it the Perl way. I simply linked the > strawberry perl.exe from cygwin environemnt and it replaced the built in > perl that sucked. > OK. Backtrack. I'll try with a purely cygwin solution... > > Thank you > > $ python --version Python 2.7.10 $ python3 --version Python 3.4.3 Thank you, Alvin From tim at akwebsoft.com Tue Feb 9 18:23:04 2016 From: tim at akwebsoft.com (Tim Johnson) Date: Tue, 9 Feb 2016 14:23:04 -0900 Subject: Importing two modules of same name Message-ID: <20160209232303.GC2583@mail.akwebsoft.com> Before proceding, let me state that this is to satisfy my curiousity, not to solve any problem I am having. Scenario : Web application developed at /some/dir/sites/flask/ If I have a package - let us call it app and in my /some/dir/sites/flask/app/__init__.py is the following: from config import config imports the config dictionary from /some/dir/sites/flask/config.py (the real-case scenario is M. Grinberg's tutorial on Flask). What if I wanted to add a module in the app package and call it from __init__.py That entails having two modules name config one at /some/dir/sites/flask/config.py and the other at /some/dir/sites/flask/app/config.py What would be the proper way to do this? (If proper at all :)) I realize that it may not be best practices. And is a practice that I avoided in the past. FYI: Platform - python 2.7 on Ubuntu 14.04. Experience: long-time python CGI programmer before retiring about 3 years ago. Thanks -- Tim http://www.akwebsoft.com, http://www.tj49.com From carl at oddbird.net Tue Feb 9 18:36:41 2016 From: carl at oddbird.net (Carl Meyer) Date: Tue, 9 Feb 2016 16:36:41 -0700 Subject: Importing two modules of same name In-Reply-To: <20160209232303.GC2583@mail.akwebsoft.com> References: <20160209232303.GC2583@mail.akwebsoft.com> Message-ID: <56BA7809.1000206@oddbird.net> Hi Tim, On 02/09/2016 04:23 PM, Tim Johnson wrote: > Before proceding, let me state that this is to satisfy my > curiousity, not to solve any problem I am having. > > Scenario : > Web application developed at /some/dir/sites/flask/ > > If I have a package - let us call it app and in my > /some/dir/sites/flask/app/__init__.py is the following: > > from config import config > > imports the config dictionary from /some/dir/sites/flask/config.py > > (the real-case scenario is M. Grinberg's tutorial on Flask). > > What if I wanted to add a module in the app package and call it from > __init__.py > > That entails having two modules name config > one at /some/dir/sites/flask/config.py > and the other at /some/dir/sites/flask/app/config.py > > What would be the proper way to do this? (If proper at all :)) I > realize that it may not be best practices. And is a practice that I > avoided in the past. The proper way to do this in Python 2.7 is to place `from __future__ import absolute_import` at the top of flask/app/__init__.py (maybe best at the top of every Python file in your project, to keep the behavior consistent). Once you have that future-import, `import config` will always import the top-level config.py. To import the "local" config.py, you'd either `from . import config` or `import app.config`. Python 3 behaves this way without the need for a future-import. If you omit the future-import in Python 2.7, `import config` will import the neighboring app/config.py by default, and there is no way to import the top-level config.py. Carl -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: OpenPGP digital signature URL: From mscir at yahoo.com Tue Feb 9 18:57:21 2016 From: mscir at yahoo.com (Mike S) Date: Tue, 9 Feb 2016 15:57:21 -0800 Subject: from scipy.linalg import _fblas ImportError: DLL load failed: The specified module could not be found. In-Reply-To: References: Message-ID: On 2/9/2016 1:33 AM, Mark Lawrence wrote: > On 09/02/2016 04:22, Mike S via Python-list wrote: >> I have Python 3.4.4 installed on Windows 7, also IPython, scipy, numpy, >> statsmodels, and a lot of other modules, and am working through this >> tutorial >> http://www.analyticsvidhya.com/blog/2016/02/time-series-forecasting-codes-python/ > > [snip bulk of code and traceback] > >> 154 >> --> 155 from scipy.linalg import _fblas >> 156 try: >> 157 from scipy.linalg import _cblas >> ImportError: DLL load failed: The specified module could not be found. >> Do I read this correctly to mean that the very last import statement is >> the one having the problem, >> "from scipy.linalg import _fblas" >> How do I troubleshoot this? I'm wondering if I have version conflict >> between two modules. > > Alomost certainly, hopefully this link will help. > http://stackoverflow.com/questions/21350153/error-importing-scipy-linalg-on-windows-python-3-3 >> >> Thanks, >> Mike > > No problem :) Mark, I uninstalled scipy, numpy and pandas, then installed this version scipy-0.15.1-win32-superpack-python3.4 I had previously installed this version scipy-0.16.1-win32-superpack-python3.4 That solved the conflict, you have great search skills, I tried but didn't find a resolution. Thanks Very Much! Mike From oscar.j.benjamin at gmail.com Tue Feb 9 19:31:52 2016 From: oscar.j.benjamin at gmail.com (Oscar Benjamin) Date: Wed, 10 Feb 2016 00:31:52 +0000 Subject: Python's import situation has driven me to the brink of imsanity In-Reply-To: <14095c33-b5d4-419f-aaf4-e6f293d0cbf9@googlegroups.com> References: <51b52cf3-0b0a-451e-a232-d9a671f7cd20@googlegroups.com> <14095c33-b5d4-419f-aaf4-e6f293d0cbf9@googlegroups.com> Message-ID: On 8 February 2016 at 00:38, wrote: > Running python setup.py develop doesn't work, it gives me this error: error: invalid command 'develop' This is presumably because your setup.py script uses distutils rather than setuptools: distutils doesn't have the develop command. > Running pip install -e . does work. That's because pip "injects setuptools" so that when you import distutils in your setup.py your actually importing a monkey-patched setuptools. You may as well import setuptools in your setup.py but either way the recommended invocation is "pip install -e .". -- Oscar From cfkaran2 at gmail.com Tue Feb 9 19:41:20 2016 From: cfkaran2 at gmail.com (Cem Karan) Date: Tue, 9 Feb 2016 19:41:20 -0500 Subject: Heap Implementation In-Reply-To: References: <56AD3D83.2050308@mail.de> <7C522D08-9D73-48D2-A71D-F1D1D34C02A5@gmail.com> <185DAEA4-8728-4792-A3B7-7F6AC5A7F876@gmail.com> <2308C08E-D2A1-4F16-800F-C2794D30F96B@gmail.com> <65105210-BAE5-4CD1-8C8C-73FC51982901@gmail.com> Message-ID: On Feb 9, 2016, at 9:27 AM, Mark Lawrence wrote: > On 09/02/2016 11:44, Cem Karan wrote: >> >> On Feb 9, 2016, at 4:40 AM, Mark Lawrence wrote: >> >>> On 09/02/2016 04:25, Cem Karan wrote: >>>> >>>> No problem, that's what I thought happened. And you're right, I'm looking for a priority queue (not the only reason to use a heap, but a pretty important reason!) >>>> >>> >>> I'm assuming I've missed the explanation, so what is the problem again with https://docs.python.org/3/library/queue.html#queue.PriorityQueue or even https://docs.python.org/3/library/asyncio-queue.html#asyncio.PriorityQueue ? >> >> Efficiently changing the the priority of items already in the queue/deleting items in the queue (not the first item). This comes up a LOT in event-based simulators where it's easier to tentatively add an event knowing that you might need to delete it or change it later. >> >> Thanks, >> Cem Karan >> > > Thanks for that, but from the sounds of it sooner you than me :) Eh, its not too bad once you figure out how to do it. It's easier in C though; you can use pointer tricks that let you find the element in constant time, and then removal will involve figuring out how to fix up your heap after you've removed the element. Thanks, Cem Karan From tim at akwebsoft.com Tue Feb 9 20:03:46 2016 From: tim at akwebsoft.com (Tim Johnson) Date: Tue, 9 Feb 2016 16:03:46 -0900 Subject: Importing two modules of same name In-Reply-To: <56BA7809.1000206@oddbird.net> References: <20160209232303.GC2583@mail.akwebsoft.com> <56BA7809.1000206@oddbird.net> Message-ID: <20160210010346.GD2583@mail.akwebsoft.com> * Carl Meyer [160209 15:28]: > Hi Tim, <...> > The proper way to do this in Python 2.7 is to place `from __future__ > import absolute_import` at the top of flask/app/__init__.py (maybe best > at the top of every Python file in your project, to keep the behavior > consistent). Once you have that future-import, `import config` will > always import the top-level config.py. To import the "local" config.py, > you'd either `from . import config` or `import app.config`. > > Python 3 behaves this way without the need for a future-import. > > If you omit the future-import in Python 2.7, `import config` will import > the neighboring app/config.py by default, and there is no way to import > the top-level config.py. Thanks for setting me straight Carl. I'm including the full package constructor (app/__init__.py) code for other's edification and further comment (if deemed necessary). Some commented annotation added ################################################################## from __future__ import absolute_import from flask import Flask from flask.ext.bootstrap import Bootstrap from flask.ext.mail import Mail from flask.ext.moment import Moment from flask.ext.sqlalchemy import SQLAlchemy # Import top-level config from config import config # Import same-level config avoiding name collision from . import config as cfg bootstrap = Bootstrap() mail = Mail() moment = Moment() db = SQLAlchemy() def create_app(config_name): app = Flask(__name__) app.config.from_object(config[config_name]) config[config_name].init_app(app) bootstrap.init_app(app) mail.init_app(app) moment.init_app(app) db.init_app(app) from .main import main as main_blueprint app.register_blueprint(main_blueprint) return app Cheers -- Tim http://www.akwebsoft.com, http://www.tj49.com From anthony at cajuntechie.org Tue Feb 9 20:26:13 2016 From: anthony at cajuntechie.org (Anthony Papillion) Date: Tue, 9 Feb 2016 19:26:13 -0600 Subject: There has to be a better way to split this string! Message-ID: <56BA91B5.5090400@cajuntechie.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 Hello Everyone, I am using datetime.now() to create a unique version of a filename. When the final file is named, it will look something like: myfile-2015-02-09-19-08-45-4223 Notice I'm replacing all of the "."'s, " "'s, and ":"'s returned by datetime.now() with "-"'s. I'm doing that using the following code but it's freaking ugly and I KNOW there is a better way to do it. I just can't seem to think of it right now. Can anyone help? What is the "right", or at least, less ugly, way to do this task? Here is the code I'm using: unprocessed_tag = str(datetime.datetime.now()) removed_spaces = unprocessed_tag.split(" ") intermediate_string = removed_spaces[0] + "-" + removed_spaces[1] removed_colons = intermediate_string.split(":") intermediate_string = removed_colons[0] + "-" + removed_colons[1] + "-" + removed_colons[2] removed_dots = intermediate_string.split(".") final_string = removed.dots[0] + "-" + removed_dots[1] return final_string Thanks! Anthony -----BEGIN PGP SIGNATURE----- iQIcBAEBCgAGBQJWupG0AAoJEAKK33RTsEsVVJEQAKZEk5jolpxa4plDvW6rY9ux YdXHxtBa3emLprVMJTAg6+NJLZaeBjulttWv2Vy1y2tE8j1prLqHVtzITdX5kVG5 z0VUZgLRYi9Ocfk5vz4lGsWEWwRtkUQrpodbXqCArerUkBqHEW/Bgs2PGq9FGbNP 6QCWLsNr1APkEl5q8P3QCM7x1z0nsQKsbVMdvNjfi2kzsvDIm6lXgl9uUFxOFesp fUkuiLFHFpbIRnnMke2mLahXWyN14QNmw6OOkqG963gOjecfC+2dNozAOSf42ul+ s2p9h3BiwijMvU/nvc/6jR3uFmoE3dIcGWsyYFNPWnPvrHUy0qHIgcavWDEo3+cw J+qOYSX3XlTIrGc5ZJKdJWs6FH7d1+WZ4GndSj316LAKq0XN46rq/ppJksStpUwU I7Gmk9mAnRt9uZy25xsx93jY8IiiOKDmI1cTclo/UKAYvQk8ib/allLnVZrssmsc 2hgugwBe6R966wavdrYl/yt1VlfwRdImRUL6pTNnCDMYYcJS0F7ATJ/dWV9iRD0h PiIiB6zB61EyKw1djBRQ0F8XqxNZK0qR0UMR5hPWJcHRp3y94bgl2tI0/aMXmFW+ jiwQ4ecNgqb9k2C6iok+9OXJ3YoqToVbXFS7/svXpS88giksS+Re+5RChxZGSWSd gXhYvKEHs+2l3Vuyhdsr =8uK+ -----END PGP SIGNATURE----- From mr.eightnoteight at gmail.com Tue Feb 9 20:27:14 2016 From: mr.eightnoteight at gmail.com (srinivas devaki) Date: Wed, 10 Feb 2016 06:57:14 +0530 Subject: Heap Implementation In-Reply-To: References: <56AD3D83.2050308@mail.de> <7C522D08-9D73-48D2-A71D-F1D1D34C02A5@gmail.com> <185DAEA4-8728-4792-A3B7-7F6AC5A7F876@gmail.com> <2308C08E-D2A1-4F16-800F-C2794D30F96B@gmail.com> <65105210-BAE5-4CD1-8C8C-73FC51982901@gmail.com> Message-ID: On Feb 10, 2016 6:11 AM, "Cem Karan" wrote: > > Eh, its not too bad once you figure out how to do it. It's easier in C though; you can use pointer tricks that let you find the element in constant time, and then removal will involve figuring out how to fix up your heap after you've removed the element. > If you can do it with C pointers then you can do it with python's references/mutable objects. :) in case of immutable objects, use a light mutable wrapper or better use list for performance. Regards Srinivas Devaki Junior (3rd yr) student at Indian School of Mines,(IIT Dhanbad) Computer Science and Engineering Department ph: +91 9491 383 249 telegram_id: @eightnoteight From rosuav at gmail.com Tue Feb 9 20:34:54 2016 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 10 Feb 2016 12:34:54 +1100 Subject: There has to be a better way to split this string! In-Reply-To: <56BA91B5.5090400@cajuntechie.org> References: <56BA91B5.5090400@cajuntechie.org> Message-ID: On Wed, Feb 10, 2016 at 12:26 PM, Anthony Papillion wrote: > I am using datetime.now() to create a unique version of a filename. First off, be aware that this won't make a unique file name. But if you're okay with that (deal with collisions somehow), then sure. > When the final file is named, it will look something like: > > myfile-2015-02-09-19-08-45-4223 > > Notice I'm replacing all of the "."'s, " "'s, and ":"'s returned by > datetime.now() with "-"'s. I'm doing that using the following code but > it's freaking ugly and I KNOW there is a better way to do it. I just > can't seem to think of it right now. Can anyone help? What is the > "right", or at least, less ugly, way to do this task? Instead of using str(), use strftime(): >>> now = datetime.datetime.now() >>> str(now) '2016-02-10 12:34:26.377701' >>> now.strftime("%Y-%m-%d-%H-%M-%S-%f") '2016-02-10-12-34-26-377701' You could instead use some other format string if you like. Here's your options: https://docs.python.org/3/library/datetime.html#strftime-strptime-behavior ChrisA From python.list at tim.thechases.com Tue Feb 9 20:39:47 2016 From: python.list at tim.thechases.com (Tim Chase) Date: Tue, 9 Feb 2016 19:39:47 -0600 Subject: There has to be a better way to split this string! In-Reply-To: <56BA91B5.5090400@cajuntechie.org> References: <56BA91B5.5090400@cajuntechie.org> Message-ID: <20160209193947.6d4bb59b@bigbox.christie.dr> On 2016-02-09 19:26, Anthony Papillion wrote: > myfile-2015-02-09-19-08-45-4223 > > Notice I'm replacing all of the "."'s, " "'s, and ":"'s returned by > datetime.now() with "-"'s. I'm doing that using the following code > but it's freaking ugly and I KNOW there is a better way to do it. I > just can't seem to think of it right now. Can anyone help? What is > the "right", or at least, less ugly, way to do this task? > > unprocessed_tag = str(datetime.datetime.now()) > removed_spaces = unprocessed_tag.split(" ") > intermediate_string = removed_spaces[0] + "-" + > removed_spaces[1] removed_colons = intermediate_string.split(":") > intermediate_string = removed_colons[0] + "-" + > removed_colons[1] > + "-" + removed_colons[2] > removed_dots = intermediate_string.split(".") > final_string = removed.dots[0] + "-" + removed_dots[1] Why not format it the way you want to begin with? >>> datetime.datetime.now().strftime("%Y-%m-%d-%H-%M-%S-%f") '2016-02-09-19-38-17-972532' -tkc From anthony at cajuntechie.org Tue Feb 9 20:40:21 2016 From: anthony at cajuntechie.org (Anthony Papillion) Date: Tue, 9 Feb 2016 19:40:21 -0600 Subject: [SOLVED] There has to be a better way to split this string! In-Reply-To: <56BA91B5.5090400@cajuntechie.org> References: <56BA91B5.5090400@cajuntechie.org> Message-ID: <56BA9505.40900@cajuntechie.org> On 02/09/2016 07:26 PM, Anthony Papillion wrote: > Hello Everyone, > > I am using datetime.now() to create a unique version of a filename. > When the final file is named, it will look something like: > > myfile-2015-02-09-19-08-45-4223 > > Notice I'm replacing all of the "."'s, " "'s, and ":"'s returned by > datetime.now() with "-"'s. I'm doing that using the following code but > it's freaking ugly and I KNOW there is a better way to do it. I just > can't seem to think of it right now. Can anyone help? What is the > "right", or at least, less ugly, way to do this task? Found the solution in strftime(). Exactly what I was looking for. From oscar.j.benjamin at gmail.com Tue Feb 9 20:45:42 2016 From: oscar.j.benjamin at gmail.com (Oscar Benjamin) Date: Wed, 10 Feb 2016 01:45:42 +0000 Subject: There has to be a better way to split this string! In-Reply-To: <56BA91B5.5090400@cajuntechie.org> References: <56BA91B5.5090400@cajuntechie.org> Message-ID: On 10 February 2016 at 01:26, Anthony Papillion wrote: > I am using datetime.now() to create a unique version of a filename. > When the final file is named, it will look something like: > > myfile-2015-02-09-19-08-45-4223 > > Notice I'm replacing all of the "."'s, " "'s, and ":"'s returned by > datetime.now() with "-"'s. I'm doing that using the following code but > it's freaking ugly and I KNOW there is a better way to do it. I just > can't seem to think of it right now. Can anyone help? What is the > "right", or at least, less ugly, way to do this task? > > Here is the code I'm using: > > > unprocessed_tag = str(datetime.datetime.now()) > removed_spaces = unprocessed_tag.split(" ") > intermediate_string = removed_spaces[0] + "-" + removed_spaces[1] > removed_colons = intermediate_string.split(":") > intermediate_string = removed_colons[0] + "-" + removed_colons[1] > + "-" + removed_colons[2] > removed_dots = intermediate_string.split(".") > final_string = removed.dots[0] + "-" + removed_dots[1] > > return final_string Chris' suggestion to use strftime is better but assuming you really needed to work with the default string then there are easier ways e.g.: >>> from datetime import datetime >>> str(datetime.now()).translate(str.maketrans(': .', '---')) '2016-02-10-01-44-54-244789' -- Oscar From ben+python at benfinney.id.au Tue Feb 9 20:47:16 2016 From: ben+python at benfinney.id.au (Ben Finney) Date: Wed, 10 Feb 2016 12:47:16 +1100 Subject: [SOLVED] There has to be a better way to split this string! References: <56BA91B5.5090400@cajuntechie.org> <56BA9505.40900@cajuntechie.org> Message-ID: <85mvr972vv.fsf@benfinney.id.au> Anthony Papillion writes: > On 02/09/2016 07:26 PM, Anthony Papillion wrote: > > I am using datetime.now() to create a unique version of a filename. > > [?] > > Found the solution in strftime(). Exactly what I was looking for. For the task of making a unique filename, you should also consider the ?tempfile? module in the standard library. -- \ ?I distrust those people who know so well what God wants them | `\ to do to their fellows, because it always coincides with their | _o__) own desires.? ?Susan Brownell Anthony, 1896 | Ben Finney From mr.eightnoteight at gmail.com Tue Feb 9 20:53:21 2016 From: mr.eightnoteight at gmail.com (srinivas devaki) Date: Wed, 10 Feb 2016 07:23:21 +0530 Subject: There has to be a better way to split this string! In-Reply-To: <56BA91B5.5090400@cajuntechie.org> References: <56BA91B5.5090400@cajuntechie.org> Message-ID: On Feb 10, 2016 6:56 AM, "Anthony Papillion" wrote: > > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA512 > > Hello Everyone, > > I am using datetime.now() to create a unique version of a filename. > When the final file is named, it will look something like: > > myfile-2015-02-09-19-08-45-4223 > You can easily do this(retrieving the tokens) using re module. In [33]: mat = re.search(r'(\S+)-(\d{4})-(\d{2})-(\d{2})-(\d{2})-(\d{2})-(\d{2})-(\d{4})', 'myfi-le-2015-02-09-19-08-45-4223') In [34]: mat Out[34]: <_sre.SRE_Match object; span=(0, 32), match='myfi-le-2015-02-09-19-08-45-4223'> In [35]: mat.groups() Out[35]: ('myfi-le', '2015', '02', '09', '19', '08', '45', '4223') In [36]: mat = re.search(r'(\S+)-(\d{4})-(\d{2})-(\d{2})-(\d{2})-(\d{2})-(\d{2})-(\d{4})', 'myfile-2015-02-09-19-08-45-4223') In [37]: mat Out[37]: <_sre.SRE_Match object; span=(0, 31), match='myfile-2015-02-09-19-08-45-4223'> In [38]: mat.groups() Out[38]: ('myfile', '2015', '02', '09', '19', '08', '45', '4223') if you don't want fiddle with regex you can use parse module( https://github.com/r1chardj0n3s/parse). but why use an external library when stdlib already provides it? :) Regards Srinivas Devaki Junior (3rd yr) student at Indian School of Mines,(IIT Dhanbad) Computer Science and Engineering Department ph: +91 9491 383 249 telegram_id: @eightnoteight From anthony at cajuntechie.org Tue Feb 9 20:58:26 2016 From: anthony at cajuntechie.org (Anthony Papillion) Date: Tue, 9 Feb 2016 19:58:26 -0600 Subject: [SOLVED] There has to be a better way to split this string! In-Reply-To: <85mvr972vv.fsf@benfinney.id.au> References: <56BA91B5.5090400@cajuntechie.org> <56BA9505.40900@cajuntechie.org> <85mvr972vv.fsf@benfinney.id.au> Message-ID: <56BA9942.9060904@cajuntechie.org> On 02/09/2016 07:47 PM, Ben Finney wrote: > Anthony Papillion writes: > >> On 02/09/2016 07:26 PM, Anthony Papillion wrote: >>> I am using datetime.now() to create a unique version of a filename. >>> [?] >> >> Found the solution in strftime(). Exactly what I was looking for. > > For the task of making a unique filename, you should also consider the > ?tempfile? module in the standard library. I looked at tempfile. Unfortunately, the filename has to be both 'unique' and 'identifiable' to the original. So if I am using mydog.jpg as the source and I am adding something unique to it, it has to still have mydog.jpg in the filename. Tempfile, I think, doesn't allow an easy way to do that. So I'm just adding the exact date and time which is unique enough for my purposes. Thanks for the pointer though. Anthony From ckaynor at zindagigames.com Tue Feb 9 21:02:27 2016 From: ckaynor at zindagigames.com (Chris Kaynor) Date: Tue, 9 Feb 2016 18:02:27 -0800 Subject: [SOLVED] There has to be a better way to split this string! In-Reply-To: <56BA9942.9060904@cajuntechie.org> References: <56BA91B5.5090400@cajuntechie.org> <56BA9505.40900@cajuntechie.org> <85mvr972vv.fsf@benfinney.id.au> <56BA9942.9060904@cajuntechie.org> Message-ID: On Tue, Feb 9, 2016 at 5:58 PM, Anthony Papillion wrote: > > On 02/09/2016 07:47 PM, Ben Finney wrote: > > Anthony Papillion writes: > > > >> On 02/09/2016 07:26 PM, Anthony Papillion wrote: > >>> I am using datetime.now() to create a unique version of a filename. > >>> [?] > >> > >> Found the solution in strftime(). Exactly what I was looking for. > > > > For the task of making a unique filename, you should also consider the > > ?tempfile? module in the standard library. > > I looked at tempfile. Unfortunately, the filename has to be both > 'unique' and 'identifiable' to the original. So if I am using mydog.jpg > as the source and I am adding something unique to it, it has to still > have mydog.jpg in the filename. Tempfile, I think, doesn't allow an easy > way to do that. So I'm just adding the exact date and time which is > unique enough for my purposes. Actually, it doe (untested, but I've used the feature in projects): tempfile.TemporaryFile(prefix="mydog", suffix=".jpg") Chris From mr.eightnoteight at gmail.com Tue Feb 9 21:04:02 2016 From: mr.eightnoteight at gmail.com (srinivas devaki) Date: Wed, 10 Feb 2016 07:34:02 +0530 Subject: There has to be a better way to split this string! In-Reply-To: References: <56BA91B5.5090400@cajuntechie.org> Message-ID: On Feb 10, 2016 7:23 AM, "srinivas devaki" wrote: > > > On Feb 10, 2016 6:56 AM, "Anthony Papillion" wrote: > > > > -----BEGIN PGP SIGNED MESSAGE----- > > Hash: SHA512 > > > > Hello Everyone, > > > > I am using datetime.now() to create a unique version of a filename. > > When the final file is named, it will look something like: > > > > myfile-2015-02-09-19-08-45-4223 > > > > You can easily do this(retrieving the tokens) using re module. > > In [33]: mat = re.search(r'(\S+)-(\d{4})-(\d{2})-(\d{2})-(\d{2})-(\d{2})-(\d{2})-(\d{4})', 'myfi-le-2015-02-09-19-08-45-4223') > > In [34]: mat > > Out[34]: <_sre.SRE_Match object; span=(0, 32), match='myfi-le-2015-02-09-19-08-45-4223'> > > In [35]: mat.groups() Out[35]: ('myfi-le', '2015', '02', '09', '19', '08', '45', '4223') > > In [36]: mat = re.search(r'(\S+)-(\d{4})-(\d{2})-(\d{2})-(\d{2})-(\d{2})-(\d{2})-(\d{4})', 'myfile-2015-02-09-19-08-45-4223') > In [37]: mat > > Out[37]: <_sre.SRE_Match object; span=(0, 31), match='myfile-2015-02-09-19-08-45-4223'> In [38]: mat.groups() > > Out[38]: ('myfile', '2015', '02', '09', '19', '08', '45', '4223') > > if you don't want fiddle with regex you can use parse module( https://github.com/r1chardj0n3s/parse). but why use an external library when stdlib already provides it? :) I'm a stupid. as soon as I saw strftime it looked like strptime and I assumed he is trying to extract the tokens and wrote that stupid/unrelated mail. PS: trying to read mailing list when you are half woke, is a bad idea and trying reply to it is even bad idea. Regards Srinivas Devaki Junior (3rd yr) student at Indian School of Mines,(IIT Dhanbad) Computer Science and Engineering Department ph: +91 9491 383 249 telegram_id: @eightnoteight From cfkaran2 at gmail.com Tue Feb 9 21:47:12 2016 From: cfkaran2 at gmail.com (Cem Karan) Date: Tue, 9 Feb 2016 21:47:12 -0500 Subject: Heap Implementation In-Reply-To: References: <56AD3D83.2050308@mail.de> <7C522D08-9D73-48D2-A71D-F1D1D34C02A5@gmail.com> <185DAEA4-8728-4792-A3B7-7F6AC5A7F876@gmail.com> <2308C08E-D2A1-4F16-800F-C2794D30F96B@gmail.com> <65105210-BAE5-4CD1-8C8C-73FC51982901@gmail.com> Message-ID: <0064EC13-09DA-4FC1-8BDE-B2AB9ACDFEB7@gmail.com> On Feb 9, 2016, at 8:27 PM, srinivas devaki wrote: > > > On Feb 10, 2016 6:11 AM, "Cem Karan" wrote: > > > > Eh, its not too bad once you figure out how to do it. It's easier in C though; you can use pointer tricks that let you find the element in constant time, and then removal will involve figuring out how to fix up your heap after you've removed the element. > > > > If you can do it with C pointers then you can do it with python's references/mutable objects. :) > in case of immutable objects, use a light mutable wrapper or better use list for performance. I should have been clearer; it's easier to UNDERSTAND in C, but you can implement it in either language. C will still be faster, but only because its compiled. It will also take a lot longer to code and ensure that it's correct, but that is the tradeoff. Thanks, Cem Karan From orgnut at yahoo.com Tue Feb 9 22:26:32 2016 From: orgnut at yahoo.com (Larry Hudson) Date: Tue, 9 Feb 2016 19:26:32 -0800 Subject: Cygwin and Python3 In-Reply-To: References: Message-ID: <3ZCdnZc2PpJ3MCfLnZ2dnUU7-RWdnZ2d@giganews.com> On 02/09/2016 08:41 AM, Fillmore wrote: > > Hi, I am having a hard time making my Cygwin run Python 3.5 (or Python 2.7 for that matter). > The command will hang and nothing happens. > Just curious... Since Python runs natively in Windows, why are you trying to run it with Cygwin? I'm not implying that you shouldn't, just offhand I don't see a reason for it. -=- Larry -=- From mscir at yahoo.com Tue Feb 9 22:39:32 2016 From: mscir at yahoo.com (Mike S) Date: Tue, 9 Feb 2016 19:39:32 -0800 Subject: Cygwin and Python3 In-Reply-To: <3ZCdnZc2PpJ3MCfLnZ2dnUU7-RWdnZ2d@giganews.com> References: <3ZCdnZc2PpJ3MCfLnZ2dnUU7-RWdnZ2d@giganews.com> Message-ID: On 2/9/2016 7:26 PM, Larry Hudson wrote: > On 02/09/2016 08:41 AM, Fillmore wrote: >> >> Hi, I am having a hard time making my Cygwin run Python 3.5 (or Python >> 2.7 for that matter). >> The command will hang and nothing happens. >> > > Just curious... > > Since Python runs natively in Windows, why are you trying to run it with > Cygwin? > I'm not implying that you shouldn't, just offhand I don't see a reason > for it. > > -=- Larry -=- > > Have you seen this? http://www.davidbaumgold.com/tutorials/set-up-python-windows/ From cs at zip.com.au Wed Feb 10 01:50:02 2016 From: cs at zip.com.au (Cameron Simpson) Date: Wed, 10 Feb 2016 17:50:02 +1100 Subject: There has to be a better way to split this string! In-Reply-To: References: Message-ID: <20160210065002.GA86307@cskk.homeip.net> On 10Feb2016 07:34, srinivas devaki wrote: >On Feb 10, 2016 7:23 AM, "srinivas devaki" >wrote: >> On Feb 10, 2016 6:56 AM, "Anthony Papillion" >wrote: >> > I am using datetime.now() to create a unique version of a filename. >> > When the final file is named, it will look something like: >> > myfile-2015-02-09-19-08-45-4223 >> >> You can easily do this(retrieving the tokens) using re module. [... complicated suggestion for the inverse problem ...] > >I'm a stupid. >as soon as I saw strftime it looked like strptime and I assumed he is >trying to extract the tokens and wrote that stupid/unrelated mail. > >PS: trying to read mailing list when you are half woke, is a bad idea and >trying reply to it is even bad idea. Regrettably, when one is half awake one is unable to realise what a bad idea it may be:-) Cheers, Cameron Simpson From dieter at handshake.de Wed Feb 10 02:54:42 2016 From: dieter at handshake.de (dieter) Date: Wed, 10 Feb 2016 08:54:42 +0100 Subject: Importing two modules of same name References: <20160209232303.GC2583@mail.akwebsoft.com> <56BA7809.1000206@oddbird.net> Message-ID: <87y4atngot.fsf@handshake.de> Carl Meyer writes: > ... > If you omit the future-import in Python 2.7, `import config` will import > the neighboring app/config.py by default, and there is no way to import > the top-level config.py. There is the "__import__" builtin function which allows to specify the "parent package" indirectly via its "globals" parameter. This way, you can import the "top-level" config (passing an empty "globals"). From shaunak.bangale at gmail.com Wed Feb 10 04:32:42 2016 From: shaunak.bangale at gmail.com (shaunak.bangale at gmail.com) Date: Wed, 10 Feb 2016 01:32:42 -0800 (PST) Subject: ImportError: cannot import name 'RAND_egd' In-Reply-To: References: <4c113094-db27-4b60-9140-eded08939ab0@googlegroups.com> Message-ID: <76905ae4-626d-4f36-9575-474ae54a21a9@googlegroups.com> On Tuesday, February 9, 2016 at 1:33:23 PM UTC-7, Ian wrote: > On Tue, Feb 9, 2016 at 7:55 AM, wrote: > > Hi, > > > > I am trying to run a 60 lines Python code which is running on a mac machine but on windows machine, I am getting this error when I run on it on shell(open file and run module). I have Python 3.5 installed. > > > > from _ssl import RAND_status, RAND_egd, RAND_add > > ImportError: cannot import name 'RAND_egd' > > Why are you importing these directly from the "_ssl" C module and not > from the "ssl" wrapper module? Anything that starts with an _ should > be considered a private implementation detail and shouldn't be relied > upon. > > > Form forums, I found that it is a common error but could not find a good solution that will work for me. > > > > One of the ways was to create scripts folder and putting easy_install.exe and then running easy_install pip but that gave me sytnax error. > > > > Please advise. Thanks in advance. > > The ssl module in the standard library has this: > > try: > from _ssl import RAND_egd > except ImportError: > # LibreSSL does not provide RAND_egd > pass > > So it looks like you cannot depend on ssl.RAND_egd to be present. Hi Ian, Thanks for your reply. I wasn't trying to import it from _ssl. That was part of the error. My code did not have RAND_egd. I think it was just about ssl package being missing. After installing analcondra distribution, it stopped throwing this particular error at least. From martinphillips at ladybridge.com Wed Feb 10 07:07:49 2016 From: martinphillips at ladybridge.com (Martin Phillips) Date: Wed, 10 Feb 2016 12:07:49 -0000 Subject: Copying void * string to Message-ID: <004a01d163fb$aa7d8930$ff789b90$@ladybridge.com> I am writing a Python wrapper to go around a C library. I have encountered a problem that I have been unable to resolve with countless web searches. Several functions in the C library return pointers to dynamically allocated w_char null terminated strings. I need to copy the string to a Python variable and call an existing library function that will free the dynamically allocate memory. My test code for this is def Test(fno, item): func = mylib. MyFunc func.restype = ct.c_void_p s = func(fno, item) result = s mylib.free(s) return result The problem is with the line that sets the result variable. I need this to make a copy of the dynamically allocated string, not the pointer to it. Thanks in advance. Martin From breamoreboy at yahoo.co.uk Wed Feb 10 08:05:33 2016 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Wed, 10 Feb 2016 13:05:33 +0000 Subject: Cygwin and Python3 In-Reply-To: References: <3ZCdnZc2PpJ3MCfLnZ2dnUU7-RWdnZ2d@giganews.com> Message-ID: On 10/02/2016 03:39, Mike S via Python-list wrote: > On 2/9/2016 7:26 PM, Larry Hudson wrote: >> On 02/09/2016 08:41 AM, Fillmore wrote: >>> >>> Hi, I am having a hard time making my Cygwin run Python 3.5 (or Python >>> 2.7 for that matter). >>> The command will hang and nothing happens. >>> >> >> Just curious... >> >> Since Python runs natively in Windows, why are you trying to run it with >> Cygwin? >> I'm not implying that you shouldn't, just offhand I don't see a reason >> for it. >> >> -=- Larry -=- >> >> > Have you seen this? > http://www.davidbaumgold.com/tutorials/set-up-python-windows/ > I have now, but I'm perfectly happy with the free versions of Visual Studio. -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From michael at stroeder.com Wed Feb 10 08:39:57 2016 From: michael at stroeder.com (=?UTF-8?Q?Michael_Str=c3=b6der?=) Date: Wed, 10 Feb 2016 14:39:57 +0100 Subject: pylint -> ImportError: No module named lazy_object_proxy Message-ID: HI! Hmm, I've used pylint before but my current installation gives me an ImportError: $ pylint Traceback (most recent call last): File "/usr/bin/pylint", line 3, in run_pylint() File "/usr/lib/python2.7/site-packages/pylint/__init__.py", line 22, in run_pylint from pylint.lint import Run File "/usr/lib/python2.7/site-packages/pylint/lint.py", line 44, in import astroid File "/usr/lib/python2.7/site-packages/astroid/__init__.py", line 54, in from astroid.nodes import * File "/usr/lib/python2.7/site-packages/astroid/nodes.py", line 39, in from astroid.node_classes import ( File "/usr/lib/python2.7/site-packages/astroid/node_classes.py", line 24, in import lazy_object_proxy ImportError: No module named lazy_object_proxy Can anybody here give me a hint what's missing? six, astroid and tk modules are installed. Any more dependencies? Ciao, Michael. From __peter__ at web.de Wed Feb 10 09:09:33 2016 From: __peter__ at web.de (Peter Otten) Date: Wed, 10 Feb 2016 15:09:33 +0100 Subject: pylint -> ImportError: No module named lazy_object_proxy References: Message-ID: Michael Str?der wrote: > HI! > > Hmm, I've used pylint before but my current installation gives me an > ImportError: > > $ pylint > Traceback (most recent call last): > File "/usr/bin/pylint", line 3, in > run_pylint() > File "/usr/lib/python2.7/site-packages/pylint/__init__.py", line 22, in > run_pylint > from pylint.lint import Run > File "/usr/lib/python2.7/site-packages/pylint/lint.py", line 44, in > > import astroid > File "/usr/lib/python2.7/site-packages/astroid/__init__.py", line 54, in > > from astroid.nodes import * > File "/usr/lib/python2.7/site-packages/astroid/nodes.py", line 39, in > > from astroid.node_classes import ( > File "/usr/lib/python2.7/site-packages/astroid/node_classes.py", line > 24, in > > import lazy_object_proxy > ImportError: No module named lazy_object_proxy > > Can anybody here give me a hint what's missing? > six, astroid and tk modules are installed. > Any more dependencies? How about the dependencies' dependencies? Grepping through the astroid source finds ./astroid/__pkginfo__.py:install_requires = ['six', 'lazy_object_proxy', 'wrapt'] But doesn't pip care of these? From ian.g.kelly at gmail.com Wed Feb 10 10:30:16 2016 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Wed, 10 Feb 2016 08:30:16 -0700 Subject: Copying void * string to In-Reply-To: <004a01d163fb$aa7d8930$ff789b90$@ladybridge.com> References: <004a01d163fb$aa7d8930$ff789b90$@ladybridge.com> Message-ID: On Wed, Feb 10, 2016 at 5:07 AM, Martin Phillips wrote: > I am writing a Python wrapper to go around a C library. I have encountered a problem that I have been unable to resolve with > countless web searches. > > > > Several functions in the C library return pointers to dynamically allocated w_char null terminated strings. I need to copy the > string to a Python variable and call an existing library function that will free the dynamically allocate memory. > > > > My test code for this is > > > > def Test(fno, item): > > func = mylib. MyFunc > > func.restype = ct.c_void_p > > s = func(fno, item) > > result = s > > mylib.free(s) > > return result > > > > The problem is with the line that sets the result variable. I need this to make a copy of the dynamically allocated string, not the > pointer to it. Does ctypes.wstring_at(s) do what you want? From andra.irina.vasile at gmail.com Wed Feb 10 11:31:19 2016 From: andra.irina.vasile at gmail.com (Andra-Irina Vasile) Date: Wed, 10 Feb 2016 17:31:19 +0100 Subject: Python Twitter Error Message-ID: Hello! Could you be so kind to help me with an issue? I have some problems with python when I am trying to use the twitter api... I have Python 3.5.1 (32 bit). I tried to reinstall the module, but I am still receiving the same errors... I attached in this email some screenshots with them. Thank you in advance! Best regards, Andra-Irina Vasile From joel.goldstick at gmail.com Wed Feb 10 11:50:55 2016 From: joel.goldstick at gmail.com (Joel Goldstick) Date: Wed, 10 Feb 2016 11:50:55 -0500 Subject: Python Twitter Error In-Reply-To: References: Message-ID: On Wed, Feb 10, 2016 at 11:31 AM, Andra-Irina Vasile < andra.irina.vasile at gmail.com> wrote: > Hello! > > Could you be so kind to help me with an issue? I have some problems with > python when I am trying to use the twitter api... I have Python 3.5.1 (32 > bit). I tried to reinstall the module, but I am still receiving the same > errors... I attached in this email some screenshots with them. > > Thank you in advance! > > Best regards, > Andra-Irina Vasile > -- > https://mail.python.org/mailman/listinfo/python-list > This list doesn't take attachments. Please copy and past the error in your question -- Joel Goldstick http://joelgoldstick.com/stats/birthdays From michael at stroeder.com Wed Feb 10 12:25:18 2016 From: michael at stroeder.com (=?UTF-8?Q?Michael_Str=c3=b6der?=) Date: Wed, 10 Feb 2016 18:25:18 +0100 Subject: pylint -> ImportError: No module named lazy_object_proxy In-Reply-To: References: Message-ID: Peter Otten wrote: > Michael Str?der wrote: > >> HI! >> >> Hmm, I've used pylint before but my current installation gives me an >> ImportError: >> >> $ pylint >> [..] >> ImportError: No module named lazy_object_proxy >> >> Can anybody here give me a hint what's missing? >> six, astroid and tk modules are installed. >> Any more dependencies? > > How about the dependencies' dependencies? Grepping through the astroid > source finds > > ../astroid/__pkginfo__.py:install_requires = ['six', 'lazy_object_proxy', > 'wrapt'] Ah, overlooked this. Thanks. > But doesn't pip care of these? Yes, likely, but... I've added new openSUSE packages python-lazy_object_proxy and python-wrapt which hopefully will appear in devel:languages:python repo soon. Ciao, Michael. From srkunze at mail.de Wed Feb 10 12:44:14 2016 From: srkunze at mail.de (Sven R. Kunze) Date: Wed, 10 Feb 2016 18:44:14 +0100 Subject: Asyncio thought experiment In-Reply-To: <87oabqq2a0.fsf@elektro.pacujo.net> References: <87oabqq2a0.fsf@elektro.pacujo.net> Message-ID: <56BB76EE.6050801@mail.de> On 08.02.2016 23:13, Marko Rauhamaa wrote: > As I stated in an earlier post, a normal subroutine may turn out to be > blocking. To make it well-behaved under asyncio, you then dutifully tag > the subroutine with "async" and adorn the blocking statement with > "await". Consequently, you put "await" in front of all calls to the > subroutine and cascade the "async"s and "await"s all the way to the top > level. > > Now what would prevent you from making *every* function an "async" and > "await"ing *every* function call? Then, you would never fall victim to > the cascading async/await. > > And if you did that, why bother sprinkling async's and await's > everywhere? Why not make every single function call an await implicitly > and every single subroutine an async? In fact, that's how everything > works in multithreading: blocking statements don't need to be ornamented > in any manner. So? :) Best, Sven From srkunze at mail.de Wed Feb 10 13:23:01 2016 From: srkunze at mail.de (Sven R. Kunze) Date: Wed, 10 Feb 2016 19:23:01 +0100 Subject: Heap Implementation In-Reply-To: <7C522D08-9D73-48D2-A71D-F1D1D34C02A5@gmail.com> References: <56AD3D83.2050308@mail.de> <7C522D08-9D73-48D2-A71D-F1D1D34C02A5@gmail.com> Message-ID: <56BB8005.8090708@mail.de> Hi Cem, On 08.02.2016 02:37, Cem Karan wrote: > My apologies for not writing sooner, but work has been quite busy lately (and likely will be for some time to come). no problem here. :) > I read your approach, and it looks pretty good, but there may be one issue with it; how do you handle the same item being pushed into the heap more than once? In my simple simulator, I'll push the same object into my event queue multiple times in a row. The priority is the moment in the future when the object will be called. As a result, items don't have unique priorities. I know that there are methods of handling this from the client-side (tuples with unique counters come to mind), but if your library can handle it directly, then that could be useful to others as well. I've pondered about that in the early design phase. I considered it a slowdown for my use-case without benefit. Why? Because I always push a fresh object ALTHOUGH it might be equal comparing attributes (priority, deadline, etc.). That's the reason why I need to ask again: why pushing the same item on a heap? Are we talking about function objects? If so, then your concern is valid. Would you accept a solution that would involve wrapping the function in another object carrying the priority? Would you prefer a wrapper that's defined by xheap itself so you can just use it? Best, Sven From sivan at vitakka.co Wed Feb 10 14:05:49 2016 From: sivan at vitakka.co (Sivan Greenberg) Date: Wed, 10 Feb 2016 21:05:49 +0200 Subject: Python's import situation has driven me to the brink of imsanity In-Reply-To: References: <51b52cf3-0b0a-451e-a232-d9a671f7cd20@googlegroups.com> <14095c33-b5d4-419f-aaf4-e6f293d0cbf9@googlegroups.com> Message-ID: not entirely on-topic here, but is distutils still being in active use "in the wild" ? -Sivan On Wed, Feb 10, 2016 at 2:31 AM, Oscar Benjamin wrote: > On 8 February 2016 at 00:38, wrote: > > Running python setup.py develop doesn't work, it gives me this error: > error: invalid command 'develop' > > This is presumably because your setup.py script uses distutils rather > than setuptools: distutils doesn't have the develop command. > > > Running pip install -e . does work. > > That's because pip "injects setuptools" so that when you import > distutils in your setup.py your actually importing a monkey-patched > setuptools. You may as well import setuptools in your setup.py but > either way the recommended invocation is "pip install -e .". > > -- > Oscar > -- > https://mail.python.org/mailman/listinfo/python-list > -- Sivan Greenberg Co founder & CTO Vitakka Consulting From tim at akwebsoft.com Wed Feb 10 14:26:48 2016 From: tim at akwebsoft.com (Tim Johnson) Date: Wed, 10 Feb 2016 10:26:48 -0900 Subject: Importing two modules of same name In-Reply-To: <87y4atngot.fsf@handshake.de> References: <20160209232303.GC2583@mail.akwebsoft.com> <56BA7809.1000206@oddbird.net> <87y4atngot.fsf@handshake.de> Message-ID: <20160210192648.GH2583@mail.akwebsoft.com> * dieter [160209 23:03]: > Carl Meyer writes: > > ... > > If you omit the future-import in Python 2.7, `import config` will import > > the neighboring app/config.py by default, and there is no way to import > > the top-level config.py. > > There is the "__import__" builtin function which allows to specify > the "parent package" indirectly via its "globals" parameter. This > way, you can import the "top-level" config (passing an empty "globals"). Thanks. I used __import__ as part of a custom load() function that I used in my own framework when I was developing CGI sites. I never used the globals parameter tho'. If I start -- Tim http://www.akwebsoft.com, http://www.tj49.com From breamoreboy at yahoo.co.uk Wed Feb 10 16:16:06 2016 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Wed, 10 Feb 2016 21:16:06 +0000 Subject: Python's import situation has driven me to the brink of imsanity In-Reply-To: References: <51b52cf3-0b0a-451e-a232-d9a671f7cd20@googlegroups.com> <14095c33-b5d4-419f-aaf4-e6f293d0cbf9@googlegroups.com> Message-ID: On 10/02/2016 19:05, Sivan Greenberg wrote: > not entirely on-topic here, but is distutils still being in active use "in > the wild" ? > > -Sivan > Given that there was distutils2 which took the same course as the Norwegian Blue, I would say no, distutils is not active. I'll happily stand corrected. As a slight aside, please don't top post, it's irritating, thanks :) -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From use.reply.to at INVALID.ADDRESS Wed Feb 10 16:26:25 2016 From: use.reply.to at INVALID.ADDRESS (Benoit Izac) Date: Wed, 10 Feb 2016 22:26:25 +0100 Subject: Cygwin and Python3 References: <3ZCdnZc2PpJ3MCfLnZ2dnUU7-RWdnZ2d@giganews.com> Message-ID: <87lh6sxnni.fsf@izac.org> Larry Hudson writes: >> Hi, I am having a hard time making my Cygwin run Python 3.5 (or >> Python 2.7 for that matter). >> The command will hang and nothing happens. > > Just curious... > > Since Python runs natively in Windows, why are you trying to run it > with Cygwin? I'm not implying that you shouldn't, just offhand I don't > see a reason for it. I do it because it's easier to install third party packages, those that need an external library to run. Cygwin come with a lot of lib* and lib*-devel that permit to just run `pip install xxx' if not already packaged. I gave a try on the native Windows version and Anaconda but there is at least one package that I could not run (and I loosed a lot of time to compile a bunch of libraries). Example of package: pyproj (proj4), openpyxl with lxml (libxml2, libxslt) and pillow (libjpeg, zlib, libtiff, ...), psycopg2 (libpq). -- Benoit Izac From tjreedy at udel.edu Wed Feb 10 16:59:41 2016 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 10 Feb 2016 16:59:41 -0500 Subject: Cygwin and Python3 In-Reply-To: <87lh6sxnni.fsf@izac.org> References: <3ZCdnZc2PpJ3MCfLnZ2dnUU7-RWdnZ2d@giganews.com> <87lh6sxnni.fsf@izac.org> Message-ID: On 2/10/2016 4:26 PM, Benoit Izac wrote: > Larry Hudson writes: >> Since Python runs natively in Windows, why are you trying to run it >> with Cygwin? I'm not implying that you shouldn't, just offhand I don't >> see a reason for it. > > I do it because it's easier to install third party packages, those that > need an external library to run. Cygwin come with a lot of lib* and > lib*-devel that permit to just run `pip install xxx' if not already > packaged. I gave a try on the native Windows version and Anaconda but > there is at least one package that I could not run (and I loosed > a lot of time to compile a bunch of libraries). > > Example of package: pyproj (proj4), openpyxl with lxml (libxml2, > libxslt) and pillow (libjpeg, zlib, libtiff, ...), psycopg2 (libpq). I belive these are all available at http://www.lfd.uci.edu/~gohlke/pythonlibs/ -- Terry Jan Reedy From joel.goldstick at gmail.com Wed Feb 10 18:00:12 2016 From: joel.goldstick at gmail.com (Joel Goldstick) Date: Wed, 10 Feb 2016 18:00:12 -0500 Subject: Python Twitter Error In-Reply-To: References: Message-ID: On Wed, Feb 10, 2016 at 5:12 PM, Andra-Irina Vasile < andra.irina.vasile at gmail.com> wrote: > Thank you for your quick answer! These are my errors... i just followed > the basic stepts to pip twitter and the other libraries... I just started > to learn about python..: > > AttributeError: module 'twitter' has no attribute 'Twitter' > > ImportError: no module named '_file_cache' > > Trimis de pe iPhone-ul meu > > Pe 10 feb. 2016, la 17:50, Joel Goldstick a > scris: > > > if you are using python 3.x you need to do pip3 to install the correct packages > > On Wed, Feb 10, 2016 at 11:31 AM, Andra-Irina Vasile < > andra.irina.vasile at gmail.com> wrote: > >> Hello! >> >> Could you be so kind to help me with an issue? I have some problems with >> python when I am trying to use the twitter api... I have Python 3.5.1 (32 >> bit). I tried to reinstall the module, but I am still receiving the same >> errors... I attached in this email some screenshots with them. >> >> Thank you in advance! >> >> Best regards, >> Andra-Irina Vasile >> -- >> https://mail.python.org/mailman/listinfo/python-list >> > > This list doesn't take attachments. Please copy and past the error in > your question > > > -- > Joel Goldstick > http://joelgoldstick.com/stats/birthdays > > -- Joel Goldstick http://joelgoldstick.com/stats/birthdays From mscir at yahoo.com Wed Feb 10 18:05:40 2016 From: mscir at yahoo.com (Mike S) Date: Wed, 10 Feb 2016 15:05:40 -0800 Subject: Cygwin and Python3 In-Reply-To: References: <3ZCdnZc2PpJ3MCfLnZ2dnUU7-RWdnZ2d@giganews.com> Message-ID: On 2/10/2016 5:05 AM, Mark Lawrence wrote: > On 10/02/2016 03:39, Mike S via Python-list wrote: >> On 2/9/2016 7:26 PM, Larry Hudson wrote: >>> On 02/09/2016 08:41 AM, Fillmore wrote: >>>> >>>> Hi, I am having a hard time making my Cygwin run Python 3.5 (or Python >>>> 2.7 for that matter). >>>> The command will hang and nothing happens. >>>> >>> >>> Just curious... >>> >>> Since Python runs natively in Windows, why are you trying to run it with >>> Cygwin? >>> I'm not implying that you shouldn't, just offhand I don't see a reason >>> for it. >>> >>> -=- Larry -=- >>> >>> >> Have you seen this? >> http://www.davidbaumgold.com/tutorials/set-up-python-windows/ >> > > I have now, but I'm perfectly happy with the free versions of Visual > Studio. I was referring to the procedure in general, and what looked like an important step to include. So it looks like the terminal can find ssh and git, but not python. That's understandable, since we didn't use Cygwin to install Python. To tell Cygwin how to find Python, run the following command: $ echo "PATH=\$PATH:/cygdrive/c/Python32" >> .bash_profile I don't see any references to VS on that page so I don't know what you're referring to. From israel at ravnalaska.net Wed Feb 10 18:33:56 2016 From: israel at ravnalaska.net (Israel Brewster) Date: Wed, 10 Feb 2016 14:33:56 -0900 Subject: Handling transactions in Python DBI module Message-ID: <92D3C964-0323-46EE-B770-B89E7E7E6D36@ravnalaska.net> I am working on implementing a Python DB API module, and am hoping I can get some help with figuring out the workflow of handling transactions. In my experience (primarily with psycopg2) the workflow goes like this: - When you open a connection (or is it when you get a cursor? I *think* it is on opening a connection), a new transaction is started - When you close a connection, an implicit ROLLBACK is performed - After issuing SQL statements that modify the database, you call commit() on the CONNECTION object, not the cursor. My primary confusion is that at least for the DB I am working on, to start/rollback/commit a transaction, you execute the appropriate SQL statement (the c library I'm using doesn't have any transactional commands, not that it should). However, to execute the statement, you need a cursor. So how is this *typically* handled? Does the connection object keep an internal cursor that it uses to manage transactions? I'm assuming, since it is called on the connection, not the cursor, that any COMMIT/ROLLBACK commands called affect all cursors on that connection. Is that correct? Or is this DB specific? Finally, how do other DB API modules, like psycopg2, ensure that ROLLBACK is called if the user never explicitly calls close()? Thanks for any assistance that can be provided. ----------------------------------------------- Israel Brewster Systems Analyst II Ravn Alaska 5245 Airport Industrial Rd Fairbanks, AK 99709 (907) 450-7293 ----------------------------------------------- From eryksun at gmail.com Wed Feb 10 19:23:24 2016 From: eryksun at gmail.com (eryk sun) Date: Wed, 10 Feb 2016 18:23:24 -0600 Subject: Copying void * string to In-Reply-To: <004a01d163fb$aa7d8930$ff789b90$@ladybridge.com> References: <004a01d163fb$aa7d8930$ff789b90$@ladybridge.com> Message-ID: On Wed, Feb 10, 2016 at 6:07 AM, Martin Phillips wrote: > > Several functions in the C library return pointers to dynamically allocated w_char null > terminated strings. I need to copy the string to a Python variable and call an existing > library function that will free the dynamically allocate memory. > > My test code for this is > > def Test(fno, item): > func = mylib. MyFunc > func.restype = ct.c_void_p > s = func(fno, item) > result = s > mylib.free(s) > return result > > The problem is with the line that sets the result variable. I need this to make a copy of > the dynamically allocated string, not the pointer to it. There are several options, but I think the simplest is to use a subclass of ctypes.c_wchar_p. subclasses of simple types don't get converted automatically. Copy the string using the "value" attribute. Then free() the pointer. But only copy and free the result if it isn't NULL (i.e. a false boolean value). import ctypes mylib = ctypes.CDLL('path/to/mylib') class MyLibError(Exception): pass class my_wchar_p(ctypes.c_wchar_p): pass mylib.MyFunc.restype = my_wchar_p def test(fno, item): s = mylib.MyFunc(fno, item) if s: result = s.value mylib.free(s) return result raise MyLibError('mylib.MyFunc returned NULL') From larry.martell at gmail.com Wed Feb 10 21:48:51 2016 From: larry.martell at gmail.com (Larry Martell) Date: Wed, 10 Feb 2016 21:48:51 -0500 Subject: RegExp help Message-ID: Given this string: >>> s = """|Type=Foo ... |Side=Left""" >>> print s |Type=Foo |Side=Left I can match with this: >>> m = re.search(r'^\|Type=(.*)$\n^\|Side=(.*)$',s,re.MULTILINE) >>> print m.group(0) |Type=Foo |Side=Left >>> print m.group(1) Foo >>> print m.group(2) Left But when I try and sub it doesn't work: >>> rn = re.sub(r'^\|Type=(.*)$^\|Side=(.*)$', r'|Side Type=\2 \1',s,re.MULTILINE) >>> print rn |Type=Foo |Side=Left What very stupid thing am I doing wrong? From python at mrabarnett.plus.com Wed Feb 10 22:00:05 2016 From: python at mrabarnett.plus.com (MRAB) Date: Thu, 11 Feb 2016 03:00:05 +0000 Subject: RegExp help In-Reply-To: References: Message-ID: <56BBF935.9050902@mrabarnett.plus.com> On 2016-02-11 02:48, Larry Martell wrote: > Given this string: > >>>> s = """|Type=Foo > ... |Side=Left""" >>>> print s > |Type=Foo > |Side=Left > > I can match with this: > >>>> m = re.search(r'^\|Type=(.*)$\n^\|Side=(.*)$',s,re.MULTILINE) >>>> print m.group(0) > |Type=Foo > |Side=Left >>>> print m.group(1) > Foo >>>> print m.group(2) > Left > > But when I try and sub it doesn't work: > >>>> rn = re.sub(r'^\|Type=(.*)$^\|Side=(.*)$', r'|Side Type=\2 \1',s,re.MULTILINE) >>>> print rn > |Type=Foo > |Side=Left > > What very stupid thing am I doing wrong? > The 4th argument of re.sub is the count. From larry.martell at gmail.com Wed Feb 10 22:09:51 2016 From: larry.martell at gmail.com (Larry Martell) Date: Wed, 10 Feb 2016 22:09:51 -0500 Subject: RegExp help In-Reply-To: <56BBF935.9050902@mrabarnett.plus.com> References: <56BBF935.9050902@mrabarnett.plus.com> Message-ID: On Wed, Feb 10, 2016 at 10:00 PM, MRAB wrote: > On 2016-02-11 02:48, Larry Martell wrote: >> >> Given this string: >> >>>>> s = """|Type=Foo >> >> ... |Side=Left""" >>>>> >>>>> print s >> >> |Type=Foo >> |Side=Left >> >> I can match with this: >> >>>>> m = re.search(r'^\|Type=(.*)$\n^\|Side=(.*)$',s,re.MULTILINE) >>>>> print m.group(0) >> >> |Type=Foo >> |Side=Left >>>>> >>>>> print m.group(1) >> >> Foo >>>>> >>>>> print m.group(2) >> >> Left >> >> But when I try and sub it doesn't work: >> >>>>> rn = re.sub(r'^\|Type=(.*)$^\|Side=(.*)$', r'|Side Type=\2 >>>>> \1',s,re.MULTILINE) >>>>> print rn >> >> |Type=Foo >> |Side=Left >> >> What very stupid thing am I doing wrong? >> > The 4th argument of re.sub is the count. Thanks. Turned out that this site is running 2.6 and that doesn't support the flags arg to sub. So I had to change it to: re.sub(r'\|Type=(.*)\n\|Side=(.*)', r'\|Side Type=\2 \1',s) From python at mrabarnett.plus.com Wed Feb 10 22:31:26 2016 From: python at mrabarnett.plus.com (MRAB) Date: Thu, 11 Feb 2016 03:31:26 +0000 Subject: RegExp help In-Reply-To: References: <56BBF935.9050902@mrabarnett.plus.com> Message-ID: <56BC008E.1030203@mrabarnett.plus.com> On 2016-02-11 03:09, Larry Martell wrote: > On Wed, Feb 10, 2016 at 10:00 PM, MRAB wrote: >> On 2016-02-11 02:48, Larry Martell wrote: >>> >>> Given this string: >>> >>>>>> s = """|Type=Foo >>> >>> ... |Side=Left""" >>>>>> >>>>>> print s >>> >>> |Type=Foo >>> |Side=Left >>> >>> I can match with this: >>> >>>>>> m = re.search(r'^\|Type=(.*)$\n^\|Side=(.*)$',s,re.MULTILINE) >>>>>> print m.group(0) >>> >>> |Type=Foo >>> |Side=Left >>>>>> >>>>>> print m.group(1) >>> >>> Foo >>>>>> >>>>>> print m.group(2) >>> >>> Left >>> >>> But when I try and sub it doesn't work: >>> >>>>>> rn = re.sub(r'^\|Type=(.*)$^\|Side=(.*)$', r'|Side Type=\2 >>>>>> \1',s,re.MULTILINE) >>>>>> print rn >>> >>> |Type=Foo >>> |Side=Left >>> >>> What very stupid thing am I doing wrong? >>> >> The 4th argument of re.sub is the count. > > > Thanks. Turned out that this site is running 2.6 and that doesn't > support the flags arg to sub. So I had to change it to: > > re.sub(r'\|Type=(.*)\n\|Side=(.*)', r'\|Side Type=\2 \1',s) > You could've used the inline flag "(?m)" in the pattern: rn = re.sub(r'(?m)^\|Type=(.*)$^\|Side=(.*)$', r'|Side Type=\2 \1',s) From frank at chagford.com Thu Feb 11 00:06:09 2016 From: frank at chagford.com (Frank Millman) Date: Thu, 11 Feb 2016 07:06:09 +0200 Subject: Handling transactions in Python DBI module In-Reply-To: <92D3C964-0323-46EE-B770-B89E7E7E6D36@ravnalaska.net> References: <92D3C964-0323-46EE-B770-B89E7E7E6D36@ravnalaska.net> Message-ID: "Israel Brewster" wrote in message news:92D3C964-0323-46EE-B770-B89E7E7E6D36 at ravnalaska.net... > I am working on implementing a Python DB API module, and am hoping I can > get some help with figuring out the workflow of handling transactions. In > my experience (primarily with > psycopg2) the workflow goes like this: > > - When you open a connection (or is it when you get a cursor? I *think* it > is on opening a connection), a new transaction is started > - When you close a connection, an implicit ROLLBACK is performed > - After issuing SQL statements that modify the database, you call commit() > on the CONNECTION object, not the cursor. > > My primary confusion is that at least for the DB I am working on, to > start/rollback/commit a transaction, you execute the appropriate SQL > statement (the c library I'm using doesn't > have any transactional commands, not that it should). However, to execute > the statement, you need a cursor. So how is this *typically* handled? Does > the connection object keep an > internal cursor that it uses to manage > transactions? > > I'm assuming, since it is called on the connection, not the cursor, that > any COMMIT/ROLLBACK commands called affect all cursors on that connection. > Is that correct? Or is this DB > specific? > > Finally, how do other DB API modules, like psycopg2, ensure that ROLLBACK > is called if the user never explicitly calls close()? Rather than try to answer your questions point-by-point, I will describe the results of some investigations I carried out into this subject a while ago. I currently support 3 databases, so I use 3 DB API modules - PostgreSQL/psycopg2, Sql Server/pyodbc, and sqlite3/sqlite3. The following applies specifically to psycopg2, but I applied the lessons learned to the other 2 as well, and have had no issues. A connection has 2 possible states - 'in transaction', or 'not in transaction'. When you create the connection it starts off as 'not'. When you call cur.execute(), it checks to see what state it is in. If the state is 'not', it silently issues a 'BEGIN TRANSACTION' before executing your statement. This applies for SELECT as well as other statements. All subsequent statements form part of the transaction, until you issue either conn.commit() or conn.rollback(). This performs the required action, and resets the state to 'not'. I learned the hard way that it is important to use conn.commit() and not cur.execute('commit'). Both succeed in committing, but the second does not reset the state, therefore the next statement does not trigger a 'BEGIN', with possible unfortunate side-effects. HTH Frank Millman From rosuav at gmail.com Thu Feb 11 00:14:10 2016 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 11 Feb 2016 16:14:10 +1100 Subject: Handling transactions in Python DBI module In-Reply-To: References: <92D3C964-0323-46EE-B770-B89E7E7E6D36@ravnalaska.net> Message-ID: On Thu, Feb 11, 2016 at 4:06 PM, Frank Millman wrote: > A connection has 2 possible states - 'in transaction', or 'not in > transaction'. When you create the connection it starts off as 'not'. > > When you call cur.execute(), it checks to see what state it is in. If the > state is 'not', it silently issues a 'BEGIN TRANSACTION' before executing > your statement. This applies for SELECT as well as other statements. > > All subsequent statements form part of the transaction, until you issue > either conn.commit() or conn.rollback(). This performs the required action, > and resets the state to 'not'. > > I learned the hard way that it is important to use conn.commit() and not > cur.execute('commit'). Both succeed in committing, but the second does not > reset the state, therefore the next statement does not trigger a 'BEGIN', > with possible unfortunate side-effects. When I advise my students on basic databasing concepts, I recommend this structure: conn = psycopg2.connect(...) with conn, conn.cursor() as cur: cur.execute(...) The transaction block should always start at the 'with' block and end when it exits. As long as you never nest them (including calling other database-using functions from inside that block), it's easy to reason about the database units of work - they always correspond perfectly to the code blocks. Personally, I'd much rather the structure were "with conn.transaction() as cur:", because I've never been able to adequately explain what a cursor is/does. It's also a bit weird that "with conn:" doesn't close the connection at the end (just closes the transaction within that connection). But I guess we don't need a "Python DB API 3.0". ChrisA From frank at chagford.com Thu Feb 11 00:28:47 2016 From: frank at chagford.com (Frank Millman) Date: Thu, 11 Feb 2016 07:28:47 +0200 Subject: Handling transactions in Python DBI module In-Reply-To: References: <92D3C964-0323-46EE-B770-B89E7E7E6D36@ravnalaska.net> Message-ID: "Chris Angelico" wrote in message news:CAPTjJmphJvtCKUB6Qr-vp_1epEWxBgQxmfKEPMOhQp3pAPGG+A at mail.gmail.com... > > When I advise my students on basic databasing concepts, I recommend > this structure: > > conn = psycopg2.connect(...) > > with conn, conn.cursor() as cur: > cur.execute(...) > Does this automatically issue a 'conn.commit()' on exit? I have found that this is important, for the following reason. If you issue a bare SELECT to PostgreSQL, it executes it without creating any locks. However, if it is inside a transaction, it does create a lock (I forget exactly which one). Because psycopg2 silently executes 'BEGIN TRANSACTION', your SELECTs always happen inside a transaction whether you specify it or not. If you do not issue a conn.commit(), the locks do not get cleared. Frank From rosuav at gmail.com Thu Feb 11 00:38:07 2016 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 11 Feb 2016 16:38:07 +1100 Subject: Handling transactions in Python DBI module In-Reply-To: References: <92D3C964-0323-46EE-B770-B89E7E7E6D36@ravnalaska.net> Message-ID: On Thu, Feb 11, 2016 at 4:28 PM, Frank Millman wrote: > "Chris Angelico" wrote in message > news:CAPTjJmphJvtCKUB6Qr-vp_1epEWxBgQxmfKEPMOhQp3pAPGG+A at mail.gmail.com... >> >> >> When I advise my students on basic databasing concepts, I recommend >> this structure: >> >> conn = psycopg2.connect(...) >> >> with conn, conn.cursor() as cur: >> cur.execute(...) >> > > Does this automatically issue a 'conn.commit()' on exit? If there was no exception raised, yes. If an exception starts to bubble across that boundary, the transaction's rolled back instead (and then the exception continues). > I have found that this is important, for the following reason. > > If you issue a bare SELECT to PostgreSQL, it executes it without creating > any locks. However, if it is inside a transaction, it does create a lock (I > forget exactly which one). That's not strictly true. A SELECT query always acquires some locks, albeit weak ones; but if you're not currently inside a transaction, you get an implicit transaction that gets immediately ended. So by the time you get to the next line of Python code, the locks have been released. > Because psycopg2 silently executes 'BEGIN TRANSACTION', your SELECTs always > happen inside a transaction whether you specify it or not. If you do not > issue a conn.commit(), the locks do not get cleared. Correct. And that's how you should _always_ work. When I started databasing seriously, it was with DB2 on OS/2, and if you didn't explicitly BEGIN TRANSACTION, one would be begun - and not ended. So the model was: Do your stuff, then commit/rollback. Then do more stuff. I never used BEGIN TRANSACTION unless I needed to add parameters to it (picking an isolation level or something). With Postgres, it's pretty similar, only you execute BEGIN TRANSACTION more explicitly. But that can be buried in the interface library, giving you a nice tidy system of "here, this is how we do some work... and now we're done". Which I strongly recommend. ChrisA From frank at chagford.com Thu Feb 11 00:45:16 2016 From: frank at chagford.com (Frank Millman) Date: Thu, 11 Feb 2016 07:45:16 +0200 Subject: asyncio and blocking - an update In-Reply-To: References: Message-ID: "Frank Millman" wrote in message news:n9c4p3$gmp$1 at ger.gmane.org... > > Some of you may have been following my attempts to modify my asyncio app > so that it does not block when accessing the database. Here is an update. > Here is an update to my update ... > I came up with what felt like a good idea. Run the database handler in a > separate thread, pass requests to it using a queue.Queue, and get it to > pass results back using an asyncio.Queue. > > It works, but I had a vague sense that performance was a bit sluggish, so > I tried the 'recommended' approach of using asyncio.run_in_executor() to > execute database calls in a separate thread. It felt a bit faster. > > Now I have written a proper timing test, and the recommended approach is > much faster. I am not 100% sure of the reason, but I think the problem is > that, with my method, when the database tries to 'put' a row on the return > queue, it has to use 'loop.call_soon_threadsafe()', and this seems to > create a bottleneck. > I have come up with a plan that seems to provide a solution. Instead of 'putting' one row at a time, let the database handler build up a block of rows, and then 'put' the block. I tried a block of 10, and it ran a lot faster. I increased it to 50, and it ran faster again. I tried 100 and there was not much improvement, so 50 seems like an optimum number. The speed is now only slightly slower than run_in_executor(), and it is more truly asynchronous. Frank From rosuav at gmail.com Thu Feb 11 00:57:49 2016 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 11 Feb 2016 16:57:49 +1100 Subject: asyncio and blocking - an update In-Reply-To: References: Message-ID: On Thu, Feb 11, 2016 at 4:45 PM, Frank Millman wrote: > I have come up with a plan that seems to provide a solution. > > Instead of 'putting' one row at a time, let the database handler build up a > block of rows, and then 'put' the block. > > I tried a block of 10, and it ran a lot faster. I increased it to 50, and it > ran faster again. I tried 100 and there was not much improvement, so 50 > seems like an optimum number. The speed is now only slightly slower than > run_in_executor(), and it is more truly asynchronous. Something worth checking would be real-world database performance metrics: what's time-to-first-row versus time-to-subsequent-rows? When you submit a query, the server first has to parse it and check for errors, then do all its optimization and stuff, and figure out an access plan. Then it goes and fetches stuff. If your query is a simple "select * from tablename" on a huge table, then it's entirely possible that you save a lot of time by fetching subsequent rows asynchronously; but if there's an ORDER BY that can't be resolved from an index (maybe involving a table join or a non-optimizable function call), the database might have to read everything from the disk before it can return a single row, so the time from first row to last row is a tiny fraction of the time to first row. What are your actual real-world queries like? Most likely, the database is reading rows in pages, and you'll have no way of predicting how many usable result rows are on any page. I wonder, is there any way you can actually operate this the other way around? Presumably the database is connected to you via a socket of some sort (TCP if you're going over a network, maybe a Unix socket for local connections), which you could select() on same as any other; if you can react to the arrival of more rows, that might be the easiest solution. That would probably require a dedicated async database library, though. ChrisA From frank at chagford.com Thu Feb 11 01:36:41 2016 From: frank at chagford.com (Frank Millman) Date: Thu, 11 Feb 2016 08:36:41 +0200 Subject: asyncio and blocking - an update In-Reply-To: References: Message-ID: "Chris Angelico" wrote in message news:CAPTjJmrVCkKAEevc9TW8FYYTnZgRUMPHectz+bD=DQRphXYTpw at mail.gmail.com... > > Something worth checking would be real-world database performance metrics [snip lots of valid questions] My approach is guided by something I read a long time ago, and I don't know how true it is, but it feels plausible. This is a rough paraphrase. Modern databases are highly optimised to execute a query and return the result as quickly as possible. A properly written database adaptor will work in conjunction with the database to optimise the retrieval of the result. Therefore the quickest way to get the result is to let the adaptor iterate over the cursor and let it figure out how best to achieve it. Obviously you still have to tune your query to make make sure it is efficient, using indexes etc. But there is no point in trying to second-guess the database adaptor in figuring out the quickest way to get the result. My theory rests on an assumption which may be faulty. I have assumed that, in order to execute a query using run_in_executor(), the way to get the result is to use cur.fetchall(). Maybe there are alternatives. However, based on that assumption, my theory contrasts the following two approaches - 1. In a separate thread, perform the following - cur.execute('SELECT ...') rows = cur.fetchall() return rows The awaiting function will perform the following - future = loop.run_in_executor('SELECT ...') await future rows = future.result() for row in rows: process row The SELECT will not block, because it is run in a separate thread. But it will return all the rows in a single list, and the calling function will block while it processes the rows, unless it takes the extra step of turning the list into an Asynchronous Iterator. 2. In a separate thread, perform the following - cur.execute('SELECT ...') for row in cur: build up block of 50 rows loop.call_soon_threadsafe(return_queue.put_nowait, block) The awaiting function will call the following - rows = AsyncCursor('SELECT ...') async for row in rows: process row AsyncCursor looks like this (abridged) - def __init__(self, sql, params): loop = asyncio.get_event_loop() self.return_queue = asyncio.Queue() request_queue.put((loop, sql, params, self.return_queue)) self.rows = [] async def __aiter__(self): return self async def __anext__(self): if self.rows: return self.rows.pop(0) self.rows = await self.return_queue.get() return self.rows.pop(0) Hope this makes sense. Frank From rosuav at gmail.com Thu Feb 11 02:07:55 2016 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 11 Feb 2016 18:07:55 +1100 Subject: asyncio and blocking - an update In-Reply-To: References: Message-ID: On Thu, Feb 11, 2016 at 5:36 PM, Frank Millman wrote: > "Chris Angelico" wrote in message > news:CAPTjJmrVCkKAEevc9TW8FYYTnZgRUMPHectz+bD=DQRphXYTpw at mail.gmail.com... >> >> >> Something worth checking would be real-world database performance metrics > > > [snip lots of valid questions] > > My approach is guided by something I read a long time ago, and I don't know > how true it is, but it feels plausible. This is a rough paraphrase. > > Modern databases are highly optimised to execute a query and return the > result as quickly as possible. A properly written database adaptor will work > in conjunction with the database to optimise the retrieval of the result. > Therefore the quickest way to get the result is to let the adaptor iterate > over the cursor and let it figure out how best to achieve it. > > Obviously you still have to tune your query to make make sure it is > efficient, using indexes etc. But there is no point in trying to > second-guess the database adaptor in figuring out the quickest way to get > the result. As far as that goes, it's sound. (It's pretty obvious that collecting all the rows into a list is going to take (at least) as long to give the first row as iteration would take to give the last row, simply because you could always implement one on top of the other, and iteration has flexibility that fetchall doesn't.) The only question is, what price are you paying for that? > 1. > future = loop.run_in_executor('SELECT ...') > await future > rows = future.result() > for row in rows: > process row > > The SELECT will not block, because it is run in a separate thread. But it > will return all the rows in a single list, and the calling function will > block while it processes the rows, unless it takes the extra step of turning > the list into an Asynchronous Iterator. This is beautifully simple. > 2. > rows = AsyncCursor('SELECT ...') > async for row in rows: > process row Also beautifully simple. But this one comes with much more complexity cost in your second thread and your AsyncCursor. So really, the question is: Is this complexity buying you enough performance that it's worthwhile? My questions about real-world stats are based on the flip side of your assumption - to quote it again: > Modern databases are highly optimised to execute a query and return the > result as quickly as possible. A properly written database adaptor will work > in conjunction with the database to optimise the retrieval of the result. > Therefore the quickest way to get the result is to let the adaptor iterate > over the cursor and let it figure out how best to achieve it. A properly-built database will optimize for two things: Time to first row, and time to query completion. (And other things, like memory usage, which don't directly affect this discussion.) In some cases, they'll be very different figures, and then you'll get a lot of benefit from iteration. In other cases, they'll be virtually the same - imagine a query that involves a number of tables and lots of aggregate functions, governed by a big GROUP BY that gathers them all up into, say, three rows, sorted by one of the aggregate functions (eg "show me these categories, sorted by the total value of sales per category"). How long does it take for the database to get the first row? It has to execute the entire query. How long to get the other two? Just return 'em from memory. So there's basically no benefit to this query of iteration above fetchall. Most queries will be somewhere in between, hence the question about real-world significance. If it costs you little to iterate, great! But if you're paying a high price, it's something to consider. ChrisA From blindanagram at nowhere.net Thu Feb 11 02:46:40 2016 From: blindanagram at nowhere.net (blindanagram at nowhere.net) Date: Thu, 11 Feb 2016 07:46:40 +0000 Subject: Cygwin and Python3 In-Reply-To: References: <3ZCdnZc2PpJ3MCfLnZ2dnUU7-RWdnZ2d@giganews.com> Message-ID: On 10/02/2016 23:05, Mike S wrote: > On 2/10/2016 5:05 AM, Mark Lawrence wrote: [snip] >>> Have you seen this? >>> http://www.davidbaumgold.com/tutorials/set-up-python-windows/ >>> >> >> I have now, but I'm perfectly happy with the free versions of Visual >> Studio. [snip] > I don't see any references to VS on that page so I don't know what > you're referring to. I suspect that Mark is reacting indirectly to the misleading implication on that page that it is necessary to install Cygwin if you want to develop Python code on Windows. From dieter at handshake.de Thu Feb 11 02:59:01 2016 From: dieter at handshake.de (dieter) Date: Thu, 11 Feb 2016 08:59:01 +0100 Subject: Handling transactions in Python DBI module References: <92D3C964-0323-46EE-B770-B89E7E7E6D36@ravnalaska.net> Message-ID: <87io1vlltm.fsf@handshake.de> Chris Angelico writes: > ... > When I advise my students on basic databasing concepts, I recommend > this structure: > > conn = psycopg2.connect(...) > > with conn, conn.cursor() as cur: > cur.execute(...) > > The transaction block should always start at the 'with' block and end > when it exits. As long as you never nest them (including calling other > database-using functions from inside that block), it's easy to reason > about the database units of work - they always correspond perfectly to > the code blocks. In my context (web applications), I strongly discourage this use - at least when "conn" does not handle subtransactions properly. In a web application, the main transaction should in general be controlled at the request level: a request should either succeed as a whole or have no side effects whatsoever. This prohibits local (somewhere deep in a component) transaction control. From dieter at handshake.de Thu Feb 11 03:07:41 2016 From: dieter at handshake.de (dieter) Date: Thu, 11 Feb 2016 09:07:41 +0100 Subject: Handling transactions in Python DBI module References: <92D3C964-0323-46EE-B770-B89E7E7E6D36@ravnalaska.net> Message-ID: <87egcjllf6.fsf@handshake.de> Israel Brewster writes: > I am working on implementing a Python DB API module, and am hoping I can get some help with figuring out the workflow of handling transactions. In my experience (primarily with psycopg2) the workflow goes like this: > > - When you open a connection (or is it when you get a cursor? I *think* it is on opening a connection), a new transaction is started All databases I have seen so far associate transaction control with the connection, not with the cursor -- this is important, as for some applications you need several independent cursors at the same time which nevertheless must belong to the same transaction. Your cursor api may give you "commit/rollback" -- but only as a convenience; those operations operate on the "connection", not the cursor. > .... > My primary confusion is that at least for the DB I am working on, to start/rollback/commit a transaction, you execute the appropriate SQL statement (the c library I'm using doesn't have any transactional commands, not that it should). However, to execute the statement, you need a cursor. So how is this *typically* handled? Does the connection object keep an internal cursor that it uses to manage transactions? When you open a connection, it is in an "inital mode". I have seen as "initial mode": * "no transaction mode": a transaction is automatically started when an SQL command is executed * "auto commit mode": each SQL command is run in its own transaction; Use "BEGIN" to enter explicite transaction control. From framstag at rus.uni-stuttgart.de Thu Feb 11 03:31:39 2016 From: framstag at rus.uni-stuttgart.de (Ulli Horlacher) Date: Thu, 11 Feb 2016 08:31:39 +0000 (UTC) Subject: tarfile : read from a socket? Message-ID: https://docs.python.org/2/library/tarfile.html says: tarfile.open(name=None, mode='r', fileobj=None, bufsize=10240, **kwargs) Return a TarFile object for the pathname name. (How) can I read a tar file from a (tcp) socket? I do not have a pathname but a socket object from socket.create_connection() -- Ullrich Horlacher Server und Virtualisierung Rechenzentrum IZUS/TIK E-Mail: horlacher at tik.uni-stuttgart.de Universitaet Stuttgart Tel: ++49-711-68565868 Allmandring 30a Fax: ++49-711-682357 70550 Stuttgart (Germany) WWW: http://www.tik.uni-stuttgart.de/ From rosuav at gmail.com Thu Feb 11 03:50:10 2016 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 11 Feb 2016 19:50:10 +1100 Subject: Handling transactions in Python DBI module In-Reply-To: <87io1vlltm.fsf@handshake.de> References: <92D3C964-0323-46EE-B770-B89E7E7E6D36@ravnalaska.net> <87io1vlltm.fsf@handshake.de> Message-ID: On Thu, Feb 11, 2016 at 6:59 PM, dieter wrote: > In my context (web applications), I strongly discourage this use - at least > when "conn" does not handle subtransactions properly. > > In a web application, the main transaction should in general be controlled > at the request level: a request should either succeed as a whole or have no > side effects whatsoever. This prohibits local (somewhere > deep in a component) transaction control. Hmm. I'm not 100% convinced that web apps should behave that way; but part of the simplicity comes from requiring that database-dependent code should not call other database-dependent code, as that would create a nested transaction. (That said, though, it's entirely possible that psycopg2 could handle a nested "with conn" as "SAVEPOINT" and either "RELEASE SAVEPOINT" or "ROLLBACK TO SAVEPOINT". But I wouldn't recommend depending on that without confirming it in the docs.) Bear in mind, the rule I gave was a broad and general rule for students to follow, not a hard-and-fast rule for all databasing. It's designed such that the student can learn the exceptions later on, but meanwhile, his code will be correct (if occasionally a little warped to avoid nesting transactions). ChrisA From frank at chagford.com Thu Feb 11 04:15:30 2016 From: frank at chagford.com (Frank Millman) Date: Thu, 11 Feb 2016 11:15:30 +0200 Subject: asyncio and blocking - an update In-Reply-To: References: Message-ID: "Chris Angelico" wrote in message news:CAPTjJmor8dMv2TDtq8RHQgWeSAaZgAmxK9gFth=OojhidWHs3w at mail.gmail.com... > > So really, the question is: Is this complexity buying you enough > performance that it's worthwhile? > Indeed, that is the question. Actually, in my case it is not quite the question. Firstly, although it took me a little while to get AsyncCursor working, it does not feel unduly complex, and actually feels quite light-weight. My tests show fairly consistently that my approach is slightly (5-10%) slower than run_in_executor(), so if that was the only issue I would not hesitate to abandon my approach. However, my concern is not to maximise database performance, but to ensure that in an asynchronous environment, one task does not block the others from responding. My tests simulate a number of tasks running concurrently and trying to access the database. Among other measurements, I track the time that each database access commences. As I expected, tasks run with 'run_in_executor' run sequentially, i.e. the next one only starts when the previous one has finished. This is not because the tasks themselves are sequential, but because 'fetchall()' is (I think) a blocking operation. Conversely, with my approach, all the tasks start within a short time of each other. Because I can process the rows as they are received, it seems to give each task a fairer time allocation. Not to mention that there are very likely to be other non-database tasks running concurrently, and they should also be more responsive. It would be quite difficult to simulate all of this, so I confess that I am relying on gut instinct at the moment. Frank From mscir at yahoo.com Thu Feb 11 04:21:22 2016 From: mscir at yahoo.com (Mike S) Date: Thu, 11 Feb 2016 01:21:22 -0800 Subject: Cygwin and Python3 In-Reply-To: References: <3ZCdnZc2PpJ3MCfLnZ2dnUU7-RWdnZ2d@giganews.com> Message-ID: On 2/10/2016 11:46 PM, blindanagram at nowhere.net wrote: > On 10/02/2016 23:05, Mike S wrote: >> On 2/10/2016 5:05 AM, Mark Lawrence wrote: > > [snip] >>>> Have you seen this? >>>> http://www.davidbaumgold.com/tutorials/set-up-python-windows/ >>>> >>> >>> I have now, but I'm perfectly happy with the free versions of Visual >>> Studio. > > [snip] >> I don't see any references to VS on that page so I don't know what >> you're referring to. > > I suspect that Mark is reacting indirectly to the misleading implication > on that page that it is necessary to install Cygwin if you want to > develop Python code on Windows. Thanks for explaining, I had no idea what that comment might be based on. From ahlusar.ahluwalia at gmail.com Thu Feb 11 04:52:06 2016 From: ahlusar.ahluwalia at gmail.com (kbtyo) Date: Thu, 11 Feb 2016 01:52:06 -0800 (PST) Subject: Suggestions for best practices when automating geocoding task Message-ID: <80396af0-13dc-4edc-8b4f-001e2c06d842@googlegroups.com> Good Morning, I welcome feedback and suggestions for libraries or resources in order to automate the following: 1. Given a directory of CSV files (each containing an address field) a. Read each CSV file b. Use address instance in row as part of a query and send request to external API in order to geocode address c. Write response to each row and return the updates file I have been wondering if using a series of decorators could be implemented. Moreover, for the request component, has anyone explored using Tornado or Twister to create a queue for requests? Thank you again for your feedback. From songofacandy at gmail.com Thu Feb 11 05:26:53 2016 From: songofacandy at gmail.com (INADA Naoki) Date: Thu, 11 Feb 2016 19:26:53 +0900 Subject: tarfile : read from a socket? In-Reply-To: References: Message-ID: ?ave you tried socket.makefile() method? From gandalf at shopzeus.com Thu Feb 11 05:39:25 2016 From: gandalf at shopzeus.com (=?UTF-8?Q?Nagy_L=c3=a1szl=c3=b3_Zsolt?=) Date: Thu, 11 Feb 2016 11:39:25 +0100 Subject: timedelta and float multiplication in python 2 Message-ID: <56BC64DD.4090908@shopzeus.com> I ran into a problem today where I had to determine the mean point between two datetimes. Here is an example: >>> start = datetime.datetime(2016,2,11) >>> stop = datetime.datetime.now() >>> mean = start + (stop-start)*0.5 Traceback (most recent call last): File "", line 1, in TypeError: unsupported operand type(s) for *: 'datetime.timedelta' and 'float' A workaround could be something like this: >>> start + datetime.timedelta(seconds=0.5*((stop-start).total_seconds())) datetime.datetime(2016, 2, 11, 5, 45, 45, 818009) Interesting thing is that total_seconds() already returns fractions of seconds, so it would be almost trivial to implement timedelta multiplication with floats. I have checked and it does work with Python 3. But it does not work with Python 2 - is there a good reason for this? Thanks, Laszlo From antoon.pardon at rece.vub.ac.be Thu Feb 11 05:47:24 2016 From: antoon.pardon at rece.vub.ac.be (Antoon Pardon) Date: Thu, 11 Feb 2016 11:47:24 +0100 Subject: tarfile : read from a socket? In-Reply-To: References: Message-ID: <56BC66BC.7060709@rece.vub.ac.be> On 02/11/2016 09:31 AM, Ulli Horlacher wrote: > https://docs.python.org/2/library/tarfile.html says: > > tarfile.open(name=None, mode='r', fileobj=None, bufsize=10240, **kwargs) > > Return a TarFile object for the pathname name. > > > (How) can I read a tar file from a (tcp) socket? > I do not have a pathname but a socket object from socket.create_connection # First you construct a file object with makefile. fo = socket.makefile() # Then you use the fileobj argument with tarfile.open. tarfile.open(mode='r', fileobj = fo) From rosuav at gmail.com Thu Feb 11 05:59:21 2016 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 11 Feb 2016 21:59:21 +1100 Subject: timedelta and float multiplication in python 2 In-Reply-To: <56BC64DD.4090908@shopzeus.com> References: <56BC64DD.4090908@shopzeus.com> Message-ID: On Thu, Feb 11, 2016 at 9:39 PM, Nagy L?szl? Zsolt wrote: > I have checked and it does work with Python 3. But it does not work with > Python 2 - is there a good reason for this? Mainly that Python 3 has had six years of development since Python 2.7, and Python 2 has been getting only bugfixes and security patches since then. I would recommend migrating to Python 3 if you want this feature. ChrisA From cfkaran2 at gmail.com Thu Feb 11 06:06:00 2016 From: cfkaran2 at gmail.com (Cem Karan) Date: Thu, 11 Feb 2016 06:06:00 -0500 Subject: Heap Implementation In-Reply-To: <56BB8005.8090708@mail.de> References: <56AD3D83.2050308@mail.de> <7C522D08-9D73-48D2-A71D-F1D1D34C02A5@gmail.com> <56BB8005.8090708@mail.de> Message-ID: On Feb 10, 2016, at 1:23 PM, "Sven R. Kunze" wrote: > Hi Cem, > > On 08.02.2016 02:37, Cem Karan wrote: >> My apologies for not writing sooner, but work has been quite busy lately (and likely will be for some time to come). > > no problem here. :) > >> I read your approach, and it looks pretty good, but there may be one issue with it; how do you handle the same item being pushed into the heap more than once? In my simple simulator, I'll push the same object into my event queue multiple times in a row. The priority is the moment in the future when the object will be called. As a result, items don't have unique priorities. I know that there are methods of handling this from the client-side (tuples with unique counters come to mind), but if your library can handle it directly, then that could be useful to others as well. > > I've pondered about that in the early design phase. I considered it a slowdown for my use-case without benefit. > > Why? Because I always push a fresh object ALTHOUGH it might be equal comparing attributes (priority, deadline, etc.). > > > That's the reason why I need to ask again: why pushing the same item on a heap? > > > Are we talking about function objects? If so, then your concern is valid. Would you accept a solution that would involve wrapping the function in another object carrying the priority? Would you prefer a wrapper that's defined by xheap itself so you can just use it? Yes. I use priority queues for event loops. The items I push in are callables (sometimes callbacks, sometimes objects with __call__()) and the priority is the simulation date that they should be called. I push the same item multiple times in a row because it will modify itself by the call (e.g., the location of an actor is calculated by its velocity and the date). There are certain calls that I tend to push in all at once because the math for calculating when the event should occur is somewhat expensive to calculate, and always returns multiple dates at once. That is also why deleting or changing events can be useful; I know that at least some of those events will be canceled in the future, which makes deleting useful. Note that it is also possible to cancel an event by marking it as cancelled, and then simply not executing it when you pop it off the queue, but I've found that there are a few cases in my simulations where the number of dead events that are in the queue exceeds the number of live events, which does have an impact on memory and operational speed (maintaining the heap invariant). There isn't much difference though, but I need FAST code to deal with size of my simulations (thousands to tens of thousands of actors, over hundreds of millions of simulations, which is why I finally had to give up on python and switch to pure C). Having a wrapper defined by xheap would be ideal; I suspect that I won't be the only one that needs to deal with this, so having it centrally located would be best. It may also make it possible for you to optimize xheap's behavior in some way. Thanks, Cem Karan From __peter__ at web.de Thu Feb 11 06:24:20 2016 From: __peter__ at web.de (Peter Otten) Date: Thu, 11 Feb 2016 12:24:20 +0100 Subject: timedelta and float multiplication in python 2 References: <56BC64DD.4090908@shopzeus.com> Message-ID: Nagy L?szl? Zsolt wrote: > I ran into a problem today where I had to determine the mean point > between two datetimes. Here is an example: > >>>> start = datetime.datetime(2016,2,11) >>>> stop = datetime.datetime.now() >>>> mean = start + (stop-start)*0.5 > Traceback (most recent call last): > File "", line 1, in > TypeError: unsupported operand type(s) for *: 'datetime.timedelta' and > 'float' > > A workaround could be something like this: > >>>> start + datetime.timedelta(seconds=0.5*((stop-start).total_seconds())) > datetime.datetime(2016, 2, 11, 5, 45, 45, 818009) How about mean = start + (stop - start) / 2 ? > Interesting thing is that total_seconds() already returns fractions of > seconds, so it would be almost trivial to implement timedelta > multiplication with floats. > > I have checked and it does work with Python 3. But it does not work with > Python 2 - is there a good reason for this? > > Thanks, > > Laszlo From framstag at rus.uni-stuttgart.de Thu Feb 11 07:53:09 2016 From: framstag at rus.uni-stuttgart.de (Ulli Horlacher) Date: Thu, 11 Feb 2016 12:53:09 +0000 (UTC) Subject: tarfile : read from a socket? References: Message-ID: Antoon Pardon wrote: > > (How) can I read a tar file from a (tcp) socket? > > I do not have a pathname but a socket object from socket.create_connection > > # First you construct a file object with makefile. > > fo = socket.makefile() > > # Then you use the fileobj argument with tarfile.open. > > tarfile.open(mode='r', fileobj = fo) I have: sock = socket.create_connection((server,port)) bs = kB64 taro = tarfile.open(fileobj=sock.makefile('w',kB64),mode='w') Traceback (most recent call last): (...) File "./fexit.py", line 1838, in sex_send taro = tarfile.open(fileobj=sock.makefile('w',kB64),mode='w') File "/usr/lib/python2.7/tarfile.py", line 1695, in open return cls.taropen(name, mode, fileobj, **kwargs) File "/usr/lib/python2.7/tarfile.py", line 1705, in taropen return cls(name, mode, fileobj, **kwargs) File "/usr/lib/python2.7/tarfile.py", line 1566, in __init__ self.offset = self.fileobj.tell() AttributeError: '_fileobject' object has no attribute 'tell' -- Ullrich Horlacher Server und Virtualisierung Rechenzentrum IZUS/TIK E-Mail: horlacher at tik.uni-stuttgart.de Universitaet Stuttgart Tel: ++49-711-68565868 Allmandring 30a Fax: ++49-711-682357 70550 Stuttgart (Germany) WWW: http://www.tik.uni-stuttgart.de/ From python at mrabarnett.plus.com Thu Feb 11 08:16:53 2016 From: python at mrabarnett.plus.com (MRAB) Date: Thu, 11 Feb 2016 13:16:53 +0000 Subject: tarfile : read from a socket? In-Reply-To: References: Message-ID: <56BC89C5.9060500@mrabarnett.plus.com> On 2016-02-11 12:53, Ulli Horlacher wrote: > Antoon Pardon wrote: > >> > (How) can I read a tar file from a (tcp) socket? >> > I do not have a pathname but a socket object from socket.create_connection >> >> # First you construct a file object with makefile. >> >> fo = socket.makefile() >> >> # Then you use the fileobj argument with tarfile.open. >> >> tarfile.open(mode='r', fileobj = fo) > > > I have: > > sock = socket.create_connection((server,port)) > bs = kB64 > taro = tarfile.open(fileobj=sock.makefile('w',kB64),mode='w') > > > > Traceback (most recent call last): > (...) > File "./fexit.py", line 1838, in sex_send > taro = tarfile.open(fileobj=sock.makefile('w',kB64),mode='w') > File "/usr/lib/python2.7/tarfile.py", line 1695, in open > return cls.taropen(name, mode, fileobj, **kwargs) > File "/usr/lib/python2.7/tarfile.py", line 1705, in taropen > return cls(name, mode, fileobj, **kwargs) > File "/usr/lib/python2.7/tarfile.py", line 1566, in __init__ > self.offset = self.fileobj.tell() > AttributeError: '_fileobject' object has no attribute 'tell' > I suppose you could write your own class to wrap the socket and provide the required methods. From rosuav at gmail.com Thu Feb 11 08:18:04 2016 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 12 Feb 2016 00:18:04 +1100 Subject: tarfile : read from a socket? In-Reply-To: References: Message-ID: On Thu, Feb 11, 2016 at 11:53 PM, Ulli Horlacher wrote: > I have: > > sock = socket.create_connection((server,port)) > bs = kB64 > taro = tarfile.open(fileobj=sock.makefile('w',kB64),mode='w') > > > > Traceback (most recent call last): > (...) > File "./fexit.py", line 1838, in sex_send > taro = tarfile.open(fileobj=sock.makefile('w',kB64),mode='w') > File "/usr/lib/python2.7/tarfile.py", line 1695, in open > return cls.taropen(name, mode, fileobj, **kwargs) > File "/usr/lib/python2.7/tarfile.py", line 1705, in taropen > return cls(name, mode, fileobj, **kwargs) > File "/usr/lib/python2.7/tarfile.py", line 1566, in __init__ > self.offset = self.fileobj.tell() > AttributeError: '_fileobject' object has no attribute 'tell' Sounds like tarfile needs a seekable file. How big is this file you're reading? Can you simply read the whole thing into memory, then use io.BytesIO? I had a quick glance at help(BytesIO) but didn't find a simple way to make a buffer that reads from an upstream file when it needs more content, but it should be possible to build one. ChrisA From breamoreboy at yahoo.co.uk Thu Feb 11 08:20:06 2016 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Thu, 11 Feb 2016 13:20:06 +0000 Subject: Cygwin and Python3 In-Reply-To: References: <3ZCdnZc2PpJ3MCfLnZ2dnUU7-RWdnZ2d@giganews.com> Message-ID: On 11/02/2016 07:46, blindanagram at nowhere.net wrote: > On 10/02/2016 23:05, Mike S wrote: >> On 2/10/2016 5:05 AM, Mark Lawrence wrote: > > [snip] >>>> Have you seen this? >>>> http://www.davidbaumgold.com/tutorials/set-up-python-windows/ >>>> >>> >>> I have now, but I'm perfectly happy with the free versions of Visual >>> Studio. > > [snip] >> I don't see any references to VS on that page so I don't know what >> you're referring to. > > I suspect that Mark is reacting indirectly to the misleading implication > on that page that it is necessary to install Cygwin if you want to > develop Python code on Windows. > Absolutely correct, marks out of ten, fifteen :) -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From framstag at rus.uni-stuttgart.de Thu Feb 11 08:25:26 2016 From: framstag at rus.uni-stuttgart.de (Ulli Horlacher) Date: Thu, 11 Feb 2016 13:25:26 +0000 (UTC) Subject: tarfile : read from a socket? References: Message-ID: Ulli Horlacher wrote: > I have: > > sock = socket.create_connection((server,port)) > bs = kB64 > taro = tarfile.open(fileobj=sock.makefile('w',kB64),mode='w') > > > > Traceback (most recent call last): > (...) > File "./fexit.py", line 1838, in sex_send > taro = tarfile.open(fileobj=sock.makefile('w',kB64),mode='w') > File "/usr/lib/python2.7/tarfile.py", line 1695, in open > return cls.taropen(name, mode, fileobj, **kwargs) > File "/usr/lib/python2.7/tarfile.py", line 1705, in taropen > return cls(name, mode, fileobj, **kwargs) > File "/usr/lib/python2.7/tarfile.py", line 1566, in __init__ > self.offset = self.fileobj.tell() > AttributeError: '_fileobject' object has no attribute 'tell' Reading the doc helps :-) https://docs.python.org/2/library/tarfile.html For special purposes, there is a second format for mode: 'filemode|[compression]'. tarfile.open() will return a TarFile object that processes its data as a stream of blocks. With taro = tarfile.open(fileobj=sock.makefile('w',kB64),mode='w|') I get no more error. -- Ullrich Horlacher Server und Virtualisierung Rechenzentrum IZUS/TIK E-Mail: horlacher at tik.uni-stuttgart.de Universitaet Stuttgart Tel: ++49-711-68565868 Allmandring 30a Fax: ++49-711-682357 70550 Stuttgart (Germany) WWW: http://www.tik.uni-stuttgart.de/ From framstag at rus.uni-stuttgart.de Thu Feb 11 08:34:45 2016 From: framstag at rus.uni-stuttgart.de (Ulli Horlacher) Date: Thu, 11 Feb 2016 13:34:45 +0000 (UTC) Subject: tarfile : read from a socket? References: Message-ID: Chris Angelico wrote: > Sounds like tarfile needs a seekable file. How big is this file you're > reading? No limits. It can be many TBs... The use case is: http://fex.rus.uni-stuttgart.de:8080/ -- Ullrich Horlacher Server und Virtualisierung Rechenzentrum IZUS/TIK E-Mail: horlacher at tik.uni-stuttgart.de Universitaet Stuttgart Tel: ++49-711-68565868 Allmandring 30a Fax: ++49-711-682357 70550 Stuttgart (Germany) WWW: http://www.tik.uni-stuttgart.de/ From cl at isbd.net Thu Feb 11 09:29:04 2016 From: cl at isbd.net (cl at isbd.net) Date: Thu, 11 Feb 2016 14:29:04 +0000 Subject: Does anyone here use wxGlade on Linux? Message-ID: I am trying out wxGlade on Linux, version 0.7.1 of wxGlade on xubuntu 15.10. I have already written something using wxPython directly so I have the basics (of my Python skills and the environment) OK I think. I am having a lot of trouble getting beyond the first hurdle of creating a trivial Python GUI with wxGlade. Some of the problem is no doubt that I'm unfamiliar with the interface but I seem to repeatedly get to a situation where the interface won't respond to mouse clicks (though the main menu items still work, I can Exit OK for instance). Is wxPython still buggy or is it really just down to my lack of familiarity with it? -- Chris Green ? From arjuns123 at gmail.com Thu Feb 11 10:12:06 2016 From: arjuns123 at gmail.com (Arjun Srivatsa) Date: Thu, 11 Feb 2016 07:12:06 -0800 (PST) Subject: Unable to insert data into MongoDB. Message-ID: Hi guys. I am basically transferring the data from PLC to PC (where the Python API runs) but I'm unable to insert into MongoDB thereafter. When I run the Python script on IDLE, the output is Hello World! Traceback (most recent call last): File "C:\Users\SRA2LO\Desktop\API.py", line 32, in s.bind((IP, PORT)) File "C:\Python27\lib\socket.py", line 228, in meth return getattr(self._sock,name)(*args) error: [Errno 10013] An attempt was made to access a socket in a way forbidden by its access permissions and when I change the IP of MongoDB server, it shows Hello World! Traceback (most recent call last): File "C:\Users\SRA2LO\Desktop\API.py", line 32, in s.bind((IP, PORT)) File "C:\Python27\lib\socket.py", line 228, in meth return getattr(self._sock,name)(*args) error: [Errno 10049] The requested address is not valid in its context. Could you please help me out? I have disabled the firewall as well. Here's the API I have written. #!/usr/bin/python import socket import socket from pymongo import MongoClient #from eve import Eve import datetime # Connection to server (PLC) on port 27017 server = socket.socket() host = "10.52.124.135" #IP of PLC port = 27017 BUFFER_SIZE = 1024 ############### server.connect((host, port)) print server.recv(1024) server.close #Connection to Client (Mongodb) on port 27017 IP = "127.0.0.1" PORT = 27017 BUFFER_SIZE = 1024 client = MongoClient('127.0.0.1', 27017) db = client.test_database s = socket.socket(socket.AF_INET, socket.SOCK_STREAM) s.bind((IP, PORT)) s.listen(1) #connections loop while True: conn, addr = s.accept() print 'Connection address:',addr try: # read loop while True: data = server.recv(BUFFER_SIZE) if not data: break conn.sendall(data) # send to MongoDB mongodoc = { "data": data, "date" : datetime.datetime.utcnow() } ABC = db.ABC ABC_id = ABC.insert_one(mongodoc).inserted_id finally: conn.close() From arjuns123 at gmail.com Thu Feb 11 11:07:35 2016 From: arjuns123 at gmail.com (Arjun Srivatsa) Date: Thu, 11 Feb 2016 08:07:35 -0800 (PST) Subject: Unable to insert data into MongoDB. In-Reply-To: References: Message-ID: <8265726c-a5af-4110-8dcd-33dccd5aecb3@googlegroups.com> Hello, I changed PORT = 27017 to PORT = 55555. I am not getting the error anymore. But I am still unable to insert the data into MongoDB. On Thursday, February 11, 2016 at 4:12:30 PM UTC+1, Arjun Srivatsa wrote: > Hi guys. I am basically transferring the data from PLC to PC (where the Python API runs) but I'm unable to insert into MongoDB thereafter. When I run the Python script on IDLE, the output is > > Hello World! > Traceback (most recent call last): File "C:\Users\SRA2LO\Desktop\API.py", line 32, in s.bind((IP, PORT)) File "C:\Python27\lib\socket.py", line 228, in meth return getattr(self._sock,name)(*args) error: [Errno 10013] An attempt was made to access a socket in a way forbidden by its access permissions > and when I change the IP of MongoDB server, it shows > > > Hello World! > Traceback (most recent call last): File "C:\Users\SRA2LO\Desktop\API.py", line 32, in s.bind((IP, PORT)) File "C:\Python27\lib\socket.py", line 228, in meth return getattr(self._sock,name)(*args) error: [Errno 10049] The requested address is not valid in its context. > > Could you please help me out? I have disabled the firewall as well. > > Here's the API I have written. > > #!/usr/bin/python > > import socket > import socket > from pymongo import MongoClient > #from eve import Eve > import datetime > > # Connection to server (PLC) on port 27017 > server = socket.socket() > host = "10.52.124.135" #IP of PLC > port = 27017 > BUFFER_SIZE = 1024 > ############### > > server.connect((host, port)) > print server.recv(1024) > > server.close > > #Connection to Client (Mongodb) on port 27017 > IP = "127.0.0.1" > PORT = 27017 > BUFFER_SIZE = 1024 > > client = MongoClient('127.0.0.1', 27017) > db = client.test_database > > s = socket.socket(socket.AF_INET, socket.SOCK_STREAM) > s.bind((IP, PORT)) > s.listen(1) > > #connections loop > while True: > conn, addr = s.accept() > print 'Connection address:',addr > try: > # read loop > while True: > data = server.recv(BUFFER_SIZE) > > if not data: break > conn.sendall(data) > > > # send to MongoDB > > mongodoc = { "data": data, "date" : datetime.datetime.utcnow() } > > > ABC = db.ABC > ABC_id = ABC.insert_one(mongodoc).inserted_id > > finally: > conn.close() From python at mrabarnett.plus.com Thu Feb 11 11:09:29 2016 From: python at mrabarnett.plus.com (MRAB) Date: Thu, 11 Feb 2016 16:09:29 +0000 Subject: Unable to insert data into MongoDB. In-Reply-To: References: Message-ID: <56BCB239.2020507@mrabarnett.plus.com> On 2016-02-11 15:12, Arjun Srivatsa wrote: > Hi guys. I am basically transferring the data from PLC to PC (where the Python API runs) but I'm unable to insert into MongoDB thereafter. When I run the Python script on IDLE, the output is > > Hello World! > Traceback (most recent call last): File "C:\Users\SRA2LO\Desktop\API.py", line 32, in s.bind((IP, PORT)) File "C:\Python27\lib\socket.py", line 228, in meth return getattr(self._sock,name)(*args) error: [Errno 10013] An attempt was made to access a socket in a way forbidden by its access permissions > and when I change the IP of MongoDB server, it shows > > > Hello World! > Traceback (most recent call last): File "C:\Users\SRA2LO\Desktop\API.py", line 32, in s.bind((IP, PORT)) File "C:\Python27\lib\socket.py", line 228, in meth return getattr(self._sock,name)(*args) error: [Errno 10049] The requested address is not valid in its context. > > Could you please help me out? I have disabled the firewall as well. > > Here's the API I have written. > > #!/usr/bin/python > > import socket > import socket > from pymongo import MongoClient > #from eve import Eve > import datetime > > # Connection to server (PLC) on port 27017 > server = socket.socket() > host = "10.52.124.135" #IP of PLC > port = 27017 > BUFFER_SIZE = 1024 > ############### > > server.connect((host, port)) > print server.recv(1024) > > server.close > > #Connection to Client (Mongodb) on port 27017 > IP = "127.0.0.1" > PORT = 27017 > BUFFER_SIZE = 1024 > > client = MongoClient('127.0.0.1', 27017) > db = client.test_database > > s = socket.socket(socket.AF_INET, socket.SOCK_STREAM) > s.bind((IP, PORT)) > s.listen(1) > > #connections loop > while True: > conn, addr = s.accept() > print 'Connection address:',addr > try: > # read loop > while True: > data = server.recv(BUFFER_SIZE) > > if not data: break > conn.sendall(data) > > > # send to MongoDB > > mongodoc = { "data": data, "date" : datetime.datetime.utcnow() } > > > ABC = db.ABC > ABC_id = ABC.insert_one(mongodoc).inserted_id > > finally: > conn.close() > I don't know whether it's relevant, but you didn't close the server socket. You have "server.close" instead of "server.close()". Also, the code as posted won't compile because the block after the "while True:" isn't indented. From fpm at u.washington.edu Thu Feb 11 11:41:11 2016 From: fpm at u.washington.edu (Frank Miles) Date: Thu, 11 Feb 2016 16:41:11 -0000 (UTC) Subject: Does anyone here use wxGlade on Linux? References: Message-ID: On Thu, 11 Feb 2016 14:29:04 +0000, cl wrote: > I am trying out wxGlade on Linux, version 0.7.1 of wxGlade on xubuntu > 15.10. > > I have already written something using wxPython directly so I have the > basics (of my Python skills and the environment) OK I think. > > I am having a lot of trouble getting beyond the first hurdle of > creating a trivial Python GUI with wxGlade. Some of the problem is no > doubt that I'm unfamiliar with the interface but I seem to repeatedly > get to a situation where the interface won't respond to mouse clicks > (though the main menu items still work, I can Exit OK for instance). > > Is wxPython still buggy or is it really just down to my lack of > familiarity with it? Sure, there are bugs in wxPython, but they are "minor". I haven't tried using wxGlade, but if it's anything like the glade I tried using long ago there are issues in getting the two working together. From framstag at rus.uni-stuttgart.de Thu Feb 11 11:41:43 2016 From: framstag at rus.uni-stuttgart.de (Ulli Horlacher) Date: Thu, 11 Feb 2016 16:41:43 +0000 (UTC) Subject: tarfile : read from a socket? References: Message-ID: Ulli Horlacher wrote: > With > > taro = tarfile.open(fileobj=sock.makefile('w',kB64),mode='w|') > > I get no more error. Of course, this is the writing client. Now I have a small problem with the reading client. This code works so far: sfo = sock.makefile('r') taro = tarfile.open(fileobj=sfo,mode='r|') taro.extractall(path=edir) But it does not writes anything to the terminal to inform the user. When I use: for member in taro.getmembers(): print('extracting "%s"' % member.name) taro.extract(member) I get the error: File "/usr/lib/python2.7/tarfile.py", line 556, in seek raise StreamError("seeking backwards is not allowed") Of course, a stream is not seekable. Any ideas? -- Ullrich Horlacher Server und Virtualisierung Rechenzentrum IZUS/TIK E-Mail: horlacher at tik.uni-stuttgart.de Universitaet Stuttgart Tel: ++49-711-68565868 Allmandring 30a Fax: ++49-711-682357 70550 Stuttgart (Germany) WWW: http://www.tik.uni-stuttgart.de/ From israel at ravnalaska.net Thu Feb 11 12:00:21 2016 From: israel at ravnalaska.net (Israel Brewster) Date: Thu, 11 Feb 2016 08:00:21 -0900 Subject: Handling transactions in Python DBI module In-Reply-To: References: <92D3C964-0323-46EE-B770-B89E7E7E6D36@ravnalaska.net> Message-ID: On Feb 10, 2016, at 8:14 PM, Chris Angelico wrote: > > On Thu, Feb 11, 2016 at 4:06 PM, Frank Millman wrote: >> A connection has 2 possible states - 'in transaction', or 'not in >> transaction'. When you create the connection it starts off as 'not'. >> >> When you call cur.execute(), it checks to see what state it is in. If the >> state is 'not', it silently issues a 'BEGIN TRANSACTION' before executing >> your statement. This applies for SELECT as well as other statements. >> >> All subsequent statements form part of the transaction, until you issue >> either conn.commit() or conn.rollback(). This performs the required action, >> and resets the state to 'not'. >> >> I learned the hard way that it is important to use conn.commit() and not >> cur.execute('commit'). Both succeed in committing, but the second does not >> reset the state, therefore the next statement does not trigger a 'BEGIN', >> with possible unfortunate side-effects. > > When I advise my students on basic databasing concepts, I recommend > this structure: > > conn = psycopg2.connect(...) > > with conn, conn.cursor() as cur: > cur.execute(...) And that is the structure I tend to use in my programs as well. I could, of course, roll the transaction control into that structure. However, that is a usage choice of the end user, whereas I am looking at the design of the connection/cursor itself. If I use psycopg, I get the transaction - even if I don't use a with block. > > The transaction block should always start at the 'with' block and end > when it exits. As long as you never nest them (including calling other > database-using functions from inside that block), it's easy to reason > about the database units of work - they always correspond perfectly to > the code blocks. > > Personally, I'd much rather the structure were "with > conn.transaction() as cur:", because I've never been able to > adequately explain what a cursor is/does. It's also a bit weird that > "with conn:" doesn't close the connection at the end (just closes the > transaction within that connection). But I guess we don't need a > "Python DB API 3.0". In my mind, cursors are simply query objects containing (potentially) result sets - so you could have two cursors, and loop through them something like "for result_1,result_2 in zip(cursor_1,cursor_2): ". Personally, I've never had a need for more than one cursor, but if you are working with large data sets, and need to work with multiple queries simultaneously without the overhead of loading the results into memory, I could see them being useful. Of course, someone else might have a completely different explanation :-) > > ChrisA > -- > https://mail.python.org/mailman/listinfo/python-list From israel at ravnalaska.net Thu Feb 11 12:05:31 2016 From: israel at ravnalaska.net (Israel Brewster) Date: Thu, 11 Feb 2016 08:05:31 -0900 Subject: Handling transactions in Python DBI module In-Reply-To: References: <92D3C964-0323-46EE-B770-B89E7E7E6D36@ravnalaska.net> Message-ID: On Feb 10, 2016, at 8:06 PM, Frank Millman wrote: > > "Israel Brewster" wrote in message news:92D3C964-0323-46EE-B770-B89E7E7E6D36 at ravnalaska.net... > >> I am working on implementing a Python DB API module, and am hoping I can get some help with figuring out the workflow of handling transactions. In my experience (primarily with >> psycopg2) the workflow goes like this: >> >> - When you open a connection (or is it when you get a cursor? I *think* it is on opening a connection), a new transaction is started >> - When you close a connection, an implicit ROLLBACK is performed >> - After issuing SQL statements that modify the database, you call commit() on the CONNECTION object, not the cursor. >> >> My primary confusion is that at least for the DB I am working on, to start/rollback/commit a transaction, you execute the appropriate SQL statement (the c library I'm using doesn't >> have any transactional commands, not that it should). However, to execute the statement, you need a cursor. So how is this *typically* handled? Does the connection object keep an > internal cursor that it uses to manage transactions? >> >> I'm assuming, since it is called on the connection, not the cursor, that any COMMIT/ROLLBACK commands called affect all cursors on that connection. Is that correct? Or is this DB >> specific? >> >> Finally, how do other DB API modules, like psycopg2, ensure that ROLLBACK is called if the user never explicitly calls close()? > > Rather than try to answer your questions point-by-point, I will describe the results of some investigations I carried out into this subject a while ago. > > I currently support 3 databases, so I use 3 DB API modules - PostgreSQL/psycopg2, Sql Server/pyodbc, and sqlite3/sqlite3. The following applies specifically to psycopg2, but I applied the lessons learned to the other 2 as well, and have had no issues. > > A connection has 2 possible states - 'in transaction', or 'not in transaction'. When you create the connection it starts off as 'not'. > > When you call cur.execute(), it checks to see what state it is in. If the state is 'not', it silently issues a 'BEGIN TRANSACTION' before executing your statement. This applies for SELECT as well as other statements. > > All subsequent statements form part of the transaction, until you issue either conn.commit() or conn.rollback(). This performs the required action, and resets the state to 'not'. > > I learned the hard way that it is important to use conn.commit() and not cur.execute('commit'). Both succeed in committing, but the second does not reset the state, therefore the next statement does not trigger a 'BEGIN', with possible unfortunate side-effects. Thanks - that is actually quite helpful. So the way I am looking at it now is that the connection would have an internal cursor as I suggested. From your response, I'll add a "state" flag as well. If the state flag is not set when execute is called on a cursor, the cursor itself will start a transaction and set the flag (this could happen from any cursor, though, so that could potentially cause a race condition, correct?). In any case, there is now a transaction open, until such a time as commit() or rollback() is called on the connection, or close is called, which executes a rollback(), using the connection's internal cursor. Hopefully that all sounds kosher. > > HTH > > Frank Millman > > > -- > https://mail.python.org/mailman/listinfo/python-list From python at mrabarnett.plus.com Thu Feb 11 12:10:16 2016 From: python at mrabarnett.plus.com (MRAB) Date: Thu, 11 Feb 2016 17:10:16 +0000 Subject: tarfile : read from a socket? In-Reply-To: References: Message-ID: <56BCC078.8030909@mrabarnett.plus.com> On 2016-02-11 16:41, Ulli Horlacher wrote: > Ulli Horlacher wrote: > >> With >> >> taro = tarfile.open(fileobj=sock.makefile('w',kB64),mode='w|') >> >> I get no more error. > > Of course, this is the writing client. > > Now I have a small problem with the reading client. > > This code works so far: > > sfo = sock.makefile('r') > taro = tarfile.open(fileobj=sfo,mode='r|') > taro.extractall(path=edir) > > But it does not writes anything to the terminal to inform the user. > > When I use: > > for member in taro.getmembers(): > print('extracting "%s"' % member.name) > taro.extract(member) > > I get the error: > > File "/usr/lib/python2.7/tarfile.py", line 556, in seek > raise StreamError("seeking backwards is not allowed") > > Of course, a stream is not seekable. > > Any ideas? > Try this: member = taro.next() while member is not None: print('extracting "%s"' % member.name) taro.extract(member) member = tar.next() From framstag at rus.uni-stuttgart.de Thu Feb 11 12:10:44 2016 From: framstag at rus.uni-stuttgart.de (Ulli Horlacher) Date: Thu, 11 Feb 2016 17:10:44 +0000 (UTC) Subject: modifying a standard module? (was: Re: tarfile : read from a socket?) References: Message-ID: Ulli Horlacher wrote: > This code works so far: > > sfo = sock.makefile('r') > taro = tarfile.open(fileobj=sfo,mode='r|') > taro.extractall(path=edir) > > But it does not writes anything to the terminal to inform the user. > > When I use: > > for member in taro.getmembers(): > print('extracting "%s"' % member.name) > taro.extract(member) > > I get the error: > > File "/usr/lib/python2.7/tarfile.py", line 556, in seek > raise StreamError("seeking backwards is not allowed") > > Of course, a stream is not seekable. > > Any ideas? As a hack, I modified the standard library module tarfile.py: root at diaspora:/usr/lib/python2.7# vv -d --- ./.versions/tarfile.py~1~ 2015-06-22 21:59:27.000000000 +0200 +++ tarfile.py 2016-02-11 18:01:50.185555952 +0100 @@ -2045,6 +2045,7 @@ directories.append(tarinfo) tarinfo = copy.copy(tarinfo) tarinfo.mode = 0700 + print('untar "%s"' % tarinfo.name) self.extract(tarinfo, path) # Reverse sort directories. This gives me exact the output I want :-) BUT I want to distribute my program and all others will not see the tar extracting information. Now my question: How can I substitute the standard module function tarfile.extractall() with my own function? -- Ullrich Horlacher Server und Virtualisierung Rechenzentrum IZUS/TIK E-Mail: horlacher at tik.uni-stuttgart.de Universitaet Stuttgart Tel: ++49-711-68565868 Allmandring 30a Fax: ++49-711-682357 70550 Stuttgart (Germany) WWW: http://www.tik.uni-stuttgart.de/ From __peter__ at web.de Thu Feb 11 12:18:55 2016 From: __peter__ at web.de (Peter Otten) Date: Thu, 11 Feb 2016 18:18:55 +0100 Subject: tarfile : read from a socket? References: Message-ID: Ulli Horlacher wrote: > Ulli Horlacher wrote: > >> With >> >> taro = tarfile.open(fileobj=sock.makefile('w',kB64),mode='w|') >> >> I get no more error. > > Of course, this is the writing client. > > Now I have a small problem with the reading client. > > This code works so far: > > sfo = sock.makefile('r') > taro = tarfile.open(fileobj=sfo,mode='r|') > taro.extractall(path=edir) > > But it does not writes anything to the terminal to inform the user. > > When I use: > > for member in taro.getmembers(): > print('extracting "%s"' % member.name) > taro.extract(member) > > I get the error: > > File "/usr/lib/python2.7/tarfile.py", line 556, in seek > raise StreamError("seeking backwards is not allowed") > > Of course, a stream is not seekable. > > Any ideas? A look into the source is often helpful ;) $ cat extract_from_stream.py import sys from tarfile import TarFile class MyTarFile(TarFile): def extract(self, member, path="."): print "extracting", member return TarFile.extract(self, member, path) tf = MyTarFile.open(fileobj=sys.stdin, mode="r|") tf.extractall() $ touch foo bar $ tar -cf archive.tar foo bar $ python extract_from_stream.py < archive.tar extracting extracting From lars at gustaebel.de Thu Feb 11 12:27:28 2016 From: lars at gustaebel.de (Lars =?iso-8859-1?Q?Gust=E4bel?=) Date: Thu, 11 Feb 2016 18:27:28 +0100 Subject: tarfile : read from a socket? In-Reply-To: References: Message-ID: <20160211172728.GA27743@axis.g33x.de> On Thu, Feb 11, 2016 at 04:41:43PM +0000, Ulli Horlacher wrote: > sfo = sock.makefile('r') > taro = tarfile.open(fileobj=sfo,mode='r|') > taro.extractall(path=edir) What about using an iterator? def myiter(tar): for t in tar: print "extracting", t.name yield t sfo = sock.makefile('r') taro = tarfile.open(fileobj=sfo,mode='r|') taro.extractall(members=myiter(taro),path=edir) Cheers, -- Lars Gust?bel lars at gustaebel.de From cl at isbd.net Thu Feb 11 12:27:53 2016 From: cl at isbd.net (cl at isbd.net) Date: Thu, 11 Feb 2016 17:27:53 +0000 Subject: Does anyone here use wxGlade on Linux? References: Message-ID: Frank Miles wrote: > On Thu, 11 Feb 2016 14:29:04 +0000, cl wrote: > > > I am trying out wxGlade on Linux, version 0.7.1 of wxGlade on xubuntu > > 15.10. > > > > I have already written something using wxPython directly so I have the > > basics (of my Python skills and the environment) OK I think. > > > > I am having a lot of trouble getting beyond the first hurdle of > > creating a trivial Python GUI with wxGlade. Some of the problem is no > > doubt that I'm unfamiliar with the interface but I seem to repeatedly > > get to a situation where the interface won't respond to mouse clicks > > (though the main menu items still work, I can Exit OK for instance). > > > > Is wxPython still buggy or is it really just down to my lack of > > familiarity with it? > > Sure, there are bugs in wxPython, but they are "minor". I haven't tried > using wxGlade, but if it's anything like the glade I tried using long ago > there are issues in getting the two working together. > Oops, I meant to say "Is wxGlade still buggy....". -- Chris Green ? From averettlanthony at gmail.com Thu Feb 11 13:02:02 2016 From: averettlanthony at gmail.com (anthony averett) Date: Thu, 11 Feb 2016 13:02:02 -0500 Subject: I cannot open IDLE Message-ID: I have downloaded python but I cannot open up idle. I really need this issue resolved for I have work do for one of my computer science class. From ltomassmail at gmail.com Thu Feb 11 13:07:21 2016 From: ltomassmail at gmail.com (ltomassmail at gmail.com) Date: Thu, 11 Feb 2016 10:07:21 -0800 (PST) Subject: Suggested datatype for getting latest information from log files Message-ID: <621f72dc-ad32-48e8-ad37-a2e7eb2d4bd8@googlegroups.com> I have timestamped log files I need to read through and keep track of the most upto date information. For example lets say we had a log file timeStamp,name,marblesHeld,timeNow,timeSinceLastEaten I need to keep track of every 'name' in this table, I don't want duplicate values so if values come in from a later timestamp that is different then that needs to get updated. For example if a later timestamp showed 'dave' with less marbles that should get updated. I thought a dictionary would be a good idea because of the key restrictions ensuring no duplicates, so the data would always update - However because they are unordered and I need to do some more processing on the data afterwards I'm having trouble. For example lets assume that once I have the most upto date values from dave,steve,jenny I wanted to do timeNow - timeSinceLastEaten to get an interval then write all the info together to some other database. Crucially order is important here. I don't know of a particular name will appear in the records or not, so it needs to created on the first instance and updated from then on. Could anyone suggest some good approaches or suggested data structures for this? I thought about trying to create an object for each 'name' then check if that object exists and update values within that object. However that seemed like a. overkill b. beyond my Python skills for the timeframe I have From jeanmichel at sequans.com Thu Feb 11 13:16:07 2016 From: jeanmichel at sequans.com (jmp) Date: Thu, 11 Feb 2016 19:16:07 +0100 Subject: Suggested datatype for getting latest information from log files In-Reply-To: <621f72dc-ad32-48e8-ad37-a2e7eb2d4bd8@googlegroups.com> References: <621f72dc-ad32-48e8-ad37-a2e7eb2d4bd8@googlegroups.com> Message-ID: On 02/11/2016 07:07 PM, ltomassmail at gmail.com wrote: > I thought a dictionary would be a good idea because of the key restrictions ensuring no duplicates, so the data would always update - However because they are unordered and I need to do some more processing on the data afterwards I'm having trouble. If it's your only concern about using dictionaries, then you may have a look at https://docs.python.org/2/library/collections.html#collections.OrderedDict JM From use.reply.to at INVALID.ADDRESS Thu Feb 11 13:17:29 2016 From: use.reply.to at INVALID.ADDRESS (Benoit Izac) Date: Thu, 11 Feb 2016 19:17:29 +0100 Subject: Cygwin and Python3 References: <3ZCdnZc2PpJ3MCfLnZ2dnUU7-RWdnZ2d@giganews.com> <87lh6sxnni.fsf@izac.org> Message-ID: <87mvr7xgau.fsf@izac.org> Terry Reedy writes: >>> Since Python runs natively in Windows, why are you trying to run it >>> with Cygwin? I'm not implying that you shouldn't, just offhand I don't >>> see a reason for it. >> >> I do it because it's easier to install third party packages, those that >> need an external library to run. Cygwin come with a lot of lib* and >> lib*-devel that permit to just run `pip install xxx' if not already >> packaged. I gave a try on the native Windows version and Anaconda but >> there is at least one package that I could not run (and I loosed >> a lot of time to compile a bunch of libraries). >> >> Example of package: pyproj (proj4), openpyxl with lxml (libxml2, >> libxslt) and pillow (libjpeg, zlib, libtiff, ...), psycopg2 (libpq). > > I belive these are all available at > http://www.lfd.uci.edu/~gohlke/pythonlibs/ How do you know when an upgrade is avaible? -- Benoit Izac From use.reply.to at INVALID.ADDRESS Thu Feb 11 13:18:44 2016 From: use.reply.to at INVALID.ADDRESS (Benoit Izac) Date: Thu, 11 Feb 2016 19:18:44 +0100 Subject: Cygwin and Python3 References: <3ZCdnZc2PpJ3MCfLnZ2dnUU7-RWdnZ2d@giganews.com> <87lh6sxnni.fsf@izac.org> Message-ID: <87k2mbxg8r.fsf@izac.org> Terry Reedy writes: >>> Since Python runs natively in Windows, why are you trying to run it >>> with Cygwin? I'm not implying that you shouldn't, just offhand I don't >>> see a reason for it. >> >> I do it because it's easier to install third party packages, those that >> need an external library to run. Cygwin come with a lot of lib* and >> lib*-devel that permit to just run `pip install xxx' if not already >> packaged. I gave a try on the native Windows version and Anaconda but >> there is at least one package that I could not run (and I loosed >> a lot of time to compile a bunch of libraries). >> >> Example of package: pyproj (proj4), openpyxl with lxml (libxml2, >> libxslt) and pillow (libjpeg, zlib, libtiff, ...), psycopg2 (libpq). > > I belive these are all available at > http://www.lfd.uci.edu/~gohlke/pythonlibs/ How do you know when an upgrade is available? -- Benoit Izac From ltomassmail at gmail.com Thu Feb 11 13:25:36 2016 From: ltomassmail at gmail.com (ltomassmail at gmail.com) Date: Thu, 11 Feb 2016 10:25:36 -0800 (PST) Subject: Suggested datatype for getting latest information from log files In-Reply-To: References: <621f72dc-ad32-48e8-ad37-a2e7eb2d4bd8@googlegroups.com> Message-ID: On Thursday, February 11, 2016 at 6:16:35 PM UTC, jmp wrote: > On 02/11/2016 07:07 PM, ltomassmail at gmail.com wrote: > > I thought a dictionary would be a good idea because of the key restrictions ensuring no duplicates, so the data would always update - However because they are unordered and I need to do some more processing on the data afterwards I'm having trouble. > > If it's your only concern about using dictionaries, then you may have a > look at > https://docs.python.org/2/library/collections.html#collections.OrderedDict > > JM I did look into that but I'm trying to do something like this which doesn't work - I guess I'm struggling a little with the implementation. fillinfo = {} fillInfo['name'] = OrderedDict('info1','info2','info3','info4','info5',) From ltomassmail at gmail.com Thu Feb 11 13:31:23 2016 From: ltomassmail at gmail.com (ltomassmail at gmail.com) Date: Thu, 11 Feb 2016 10:31:23 -0800 (PST) Subject: Suggested datatype for getting latest information from log files In-Reply-To: References: <621f72dc-ad32-48e8-ad37-a2e7eb2d4bd8@googlegroups.com> Message-ID: <80862362-7edb-403d-8854-7b17d3f69cc2@googlegroups.com> On Thursday, February 11, 2016 at 6:16:35 PM UTC, jmp wrote: > On 02/11/2016 07:07 PM, ltomassmail at gmail.com wrote: > > I thought a dictionary would be a good idea because of the key restrictions ensuring no duplicates, so the data would always update - However because they are unordered and I need to do some more processing on the data afterwards I'm having trouble. > > If it's your only concern about using dictionaries, then you may have a > look at > https://docs.python.org/2/library/collections.html#collections.OrderedDict > > JM I did look into this but struggling a little with the implementation, currently trying to do something like this which doesn't work: fillInfo = {} p = re.compile('PATTERN') with (open(path,'r')) as f: for row in f: m = p.search(row) if m == None: continue else: fillInfo[m.group(5)] = OrderedDict(m.group(1),m.group(2),m.group(3),m.group(4),m.group(6)) From martin at linux-ip.net Thu Feb 11 13:58:33 2016 From: martin at linux-ip.net (Martin A. Brown) Date: Thu, 11 Feb 2016 10:58:33 -0800 Subject: Suggested datatype for getting latest information from log files In-Reply-To: <621f72dc-ad32-48e8-ad37-a2e7eb2d4bd8@googlegroups.com> References: <621f72dc-ad32-48e8-ad37-a2e7eb2d4bd8@googlegroups.com> Message-ID: Greetings, >I have timestamped log files I need to read through and keep track >of the most upto date information. > >For example lets say we had a log file > >timeStamp,name,marblesHeld,timeNow,timeSinceLastEaten I do not quite understand the distinction between timeStamp and timeNow. >I need to keep track of every 'name' in this table, I don't want >duplicate values so if values come in from a later timestamp that >is different then that needs to get updated. For example if a later >timestamp showed 'dave' with less marbles that should get updated. > >I thought a dictionary would be a good idea because of the key >restrictions ensuring no duplicates, so the data would always >update - Yes. A dictionary seems reasonable. >However because they are unordered and I need to do some more >processing on the data afterwards I'm having trouble. Ordered how? For each name, you need to keep the stream of data ordered? This is what I'm assuming based on your problem description. If the order of names (dave, steve and jenny) is important, then you should look to OrderedDict as JM has suggested. I am inferring from your description that the order of events (along a timeline) is what is important, not the sequence of players to each other(, since that is already in the logfile). >For example lets assume that once I have the most upto date values >from dave,steve,jenny I wanted to do timeNow - timeSinceLastEaten >to get an interval then write all the info together to some other >database. Crucially order is important here. Again, it's not utterly clear what "order" means. If order of events for a single player is important, then see below. >I don't know of a particular name will appear in the records or >not, so it needs to created on the first instance and updated from >then on. Again, a dictionary is great for this. It seems that you could benefit, also from a list (to store an event and the time at which the event occurred). But, you don't want to store all of history, so you want to use a bounded length list. You may find a collections.deque useful here. >Could anyone suggest some good approaches or suggested data >structures for this? First, JM already pointed you to OrderedDict, which may help depending on exactly what you are trying to order. There are two other data structures in the collections module that may be helpful for you. I perceive the following (from your description). You have a set of names (players). You wish to store, for each name, a value (marblesHeld). You wish to store, for each name, a value (timeSinceLastEaten). I recommend learning how to use both: collections.defaultdict [0]: so you can dynamically create entries for new players in the marble game without checking if they already exist in the dictionary (very convenient!) collectionst.deque [1]: in this case, I'm suggesting using it as a bounded-length list; you keep adding stuff to it and after it stores X entries, the old ones will "fall off" Note, I fabricated players and data, but the bit that you are probably interested in is the interaction between the dictionary, whose keys are the names of the players, and whose values contain the deque capturing (the last 10 entries) of the users marble count and the time at which this occurred. mydeque = functools.partial(collections.deque, maxlen=10) record = collections.defaultdict(mydeque) Storing both the marble count and the time will allow you to calculate at any time later the duration since the user last had a marble count change. I don't understand how the eating fits into your problem, but maybe my code (below) will afford you an example of how to approach the problem with a few of Python's wonderfully convenient standard library data structures. Good luck, -Martin P.S. I just read your reply to JM, and it looks like you also are trying to figure out how to read the input data. Is it CSV? Could you simply use the csv module [2]? [0] https://docs.python.org/3/library/collections.html#collections.defaultdict [1] https://docs.python.org/3/library/collections.html#collections.deque [2] https://docs.python.org/3/library/csv.html #! /usr/bin/python3 import time import random import functools import collections import pprint players = ['Steve', 'Jenny', 'Dave', 'Samuel', 'Jerzy', 'Ellen'] mydeque = functools.partial(collections.deque, maxlen=10) def marblegame(rounds): record = collections.defaultdict(mydeque) for _ in range(rounds): now = time.time() who = random.choice(players) marbles = random.randint(0, 100) record[who].append((marbles, now)) for whom, marblehistory in record.items(): print(whom, end=": ") pprint.pprint(marblehistory) if __name__ == '__main__': import sys if len(sys.argv) > 1: count = int(sys.argv[1]) else: count = 30 marblegame(count) # -- end of file -- Martin A. Brown http://linux-ip.net/ From alvin.hacopian at gmail.com Thu Feb 11 15:40:41 2016 From: alvin.hacopian at gmail.com (alvin.hacopian at gmail.com) Date: Thu, 11 Feb 2016 12:40:41 -0800 (PST) Subject: Cygwin and Python3 In-Reply-To: <87k2mbxg8r.fsf@izac.org> References: <3ZCdnZc2PpJ3MCfLnZ2dnUU7-RWdnZ2d@giganews.com> <87lh6sxnni.fsf@izac.org> <87k2mbxg8r.fsf@izac.org> Message-ID: On Thursday, February 11, 2016 at 10:18:53 AM UTC-8, Benoit Izac wrote: > Terry Reedy writes: > > >>> Since Python runs natively in Windows, why are you trying to run it > >>> with Cygwin? I'm not implying that you shouldn't, just offhand I don't > >>> see a reason for it. > >> > >> I do it because it's easier to install third party packages, those that > >> need an external library to run. Cygwin come with a lot of lib* and > >> lib*-devel that permit to just run `pip install xxx' if not already > >> packaged. I gave a try on the native Windows version and Anaconda but > >> there is at least one package that I could not run (and I loosed > >> a lot of time to compile a bunch of libraries). > >> > >> Example of package: pyproj (proj4), openpyxl with lxml (libxml2, > >> libxslt) and pillow (libjpeg, zlib, libtiff, ...), psycopg2 (libpq). > > > > I belive these are all available at > > http://www.lfd.uci.edu/~gohlke/pythonlibs/ > > How do you know when an upgrade is available? > > -- > Benoit Izac This thread also seems to address windows lack of support with linux. Only if windows had a linux command line support instead of using power shell. Mac is not preferred OS and its just a matter of time since windows is obsolete. From random832 at fastmail.com Thu Feb 11 16:23:52 2016 From: random832 at fastmail.com (Random832) Date: Thu, 11 Feb 2016 16:23:52 -0500 Subject: tarfile : read from a socket? In-Reply-To: References: Message-ID: <1455225832.3581975.518819850.29179C13@webmail.messagingengine.com> On Thu, Feb 11, 2016, at 11:41, Ulli Horlacher wrote: > When I use: > > for member in taro.getmembers(): > print('extracting "%s"' % member.name) > taro.extract(member) > > I get the error: > > File "/usr/lib/python2.7/tarfile.py", line 556, in seek > raise StreamError("seeking backwards is not allowed") > > Of course, a stream is not seekable. > > Any ideas? Try this: while True: member = taro.next() if member is None: break print('extracting "%s"' % member.name) taro.extract(member) From framstag at rus.uni-stuttgart.de Thu Feb 11 18:24:01 2016 From: framstag at rus.uni-stuttgart.de (Ulli Horlacher) Date: Thu, 11 Feb 2016 23:24:01 +0000 (UTC) Subject: tarfile : secure extract? Message-ID: In https://docs.python.org/2/library/tarfile.html there is a warning: Never extract archives from untrusted sources without prior inspection. It is possible that files are created outside of path, e.g. members that have absolute filenames starting with "/" or filenames with two dots "..". My program has to extract tar archives from untrusted sources :-} So far, I ignore files with dangerous pathnames: for member in taro.getmembers(): file = member.name if match(r'^(?i)([a-z]:)?(\.\.)?[/\\]',file): print('ignoring "%s"' % file) else: print('extracting "%s"' % file) taro.extract(member) A better approach would be to rename such files while extracting. Is this possible? -- Ullrich Horlacher Server und Virtualisierung Rechenzentrum IZUS/TIK E-Mail: horlacher at tik.uni-stuttgart.de Universitaet Stuttgart Tel: ++49-711-68565868 Allmandring 30a Fax: ++49-711-682357 70550 Stuttgart (Germany) WWW: http://www.tik.uni-stuttgart.de/ From p_s_d_a_s_i_l_v_a_ns at netcabo.pt Thu Feb 11 19:31:14 2016 From: p_s_d_a_s_i_l_v_a_ns at netcabo.pt (Paulo da Silva) Date: Fri, 12 Feb 2016 00:31:14 +0000 Subject: Storing a big amount of path names Message-ID: Hi! What is the best (shortest memory usage) way to store lots of pathnames in memory where: 1. Path names are pathname=(dirname,filename) 2. There many different dirnames but much less than pathnames 3. dirnames have in general many chars The idea is to share the common dirnames. More realistically not only the pathnames are stored but objects each object being a MyFile containing self.name - getPathname(self) - other stuff class MyFile: __allfiles=[] def __init__(self,dirname,filename): self.dirname=dirname # But I want to share this with other files self.name=filename MyFile.__allfiles.append(self) ... def getPathname(self): return os.path.join(self.dirname,self.name) ... Thanks for any suggestion. Paulo From rosuav at gmail.com Thu Feb 11 19:39:32 2016 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 12 Feb 2016 11:39:32 +1100 Subject: Storing a big amount of path names In-Reply-To: References: Message-ID: On Fri, Feb 12, 2016 at 11:31 AM, Paulo da Silva wrote: > What is the best (shortest memory usage) way to store lots of pathnames > in memory where: > > 1. Path names are pathname=(dirname,filename) > 2. There many different dirnames but much less than pathnames > 3. dirnames have in general many chars > > The idea is to share the common dirnames. > > More realistically not only the pathnames are stored but objects each > object being a MyFile containing > self.name - > getPathname(self) - > other stuff Just store them in the most obvious way, and don't worry about memory usage. How many path names are you likely to have? A million? You can still afford to have 1KB pathnames and it'll take up no more than a gigabyte of RAM - and most computers throw around gigs of virtual memory like it's nothing. ChrisA From ben+python at benfinney.id.au Thu Feb 11 19:44:17 2016 From: ben+python at benfinney.id.au (Ben Finney) Date: Fri, 12 Feb 2016 11:44:17 +1100 Subject: Storing a big amount of path names References: Message-ID: <85egci7o66.fsf@benfinney.id.au> Paulo da Silva writes: > What is the best (shortest memory usage) way to store lots of > pathnames in memory I challenge the premise. Why is ?shortest memory usage? your criterion for ?best?, here? How have you determined that factors like ?easily understandable when reading?, or ?using standard Python idioms?, are less important? As for ?lots of pathnames?, how many are you expecting? Python's built-in container types are highly optimised for quite large amounts of data. Have you measured an implementation with normal built-in container types with your expected quantity of items, and confirmed that the performance is unacceptable? > Thanks for any suggestion. I would suggest that the assumption you have too much data for Python's built-in container types, is an assumption that should be rigorously tested because it is likely not true. -- \ ?We suffer primarily not from our vices or our weaknesses, but | `\ from our illusions.? ?Daniel J. Boorstin, historian, 1914?2004 | _o__) | Ben Finney From python.list at tim.thechases.com Thu Feb 11 20:13:32 2016 From: python.list at tim.thechases.com (Tim Chase) Date: Thu, 11 Feb 2016 19:13:32 -0600 Subject: Storing a big amount of path names In-Reply-To: References: Message-ID: <20160211191332.1ede13a5@bigbox.christie.dr> On 2016-02-12 00:31, Paulo da Silva wrote: > What is the best (shortest memory usage) way to store lots of > pathnames in memory where: > > 1. Path names are pathname=(dirname,filename) > 2. There many different dirnames but much less than pathnames > 3. dirnames have in general many chars > > The idea is to share the common dirnames. Well, you can create a dict that has dirname->list(filenames) which will reduce the dirname to a single instance. You could store that dict in the class, shared by all of the instances, though that starts to pick up a code-smell. But unless you're talking about an obscenely large number of dirnames & filenames, or a severely resource-limited machine, just use the default built-ins. If you start to push the boundaries of system resources, then I'd try the "anydbm" module or use the "shelve" module to marshal them out to disk. Finally, you *could* create an actual sqlite database on disk if size really does exceed reasonable system specs. -tkc From rgaddi at highlandtechnology.invalid Thu Feb 11 21:17:49 2016 From: rgaddi at highlandtechnology.invalid (Rob Gaddi) Date: Fri, 12 Feb 2016 02:17:49 -0000 (UTC) Subject: Storing a big amount of path names References: Message-ID: Tim Chase wrote: > On 2016-02-12 00:31, Paulo da Silva wrote: >> What is the best (shortest memory usage) way to store lots of >> pathnames in memory where: >> >> 1. Path names are pathname=(dirname,filename) >> 2. There many different dirnames but much less than pathnames >> 3. dirnames have in general many chars >> >> The idea is to share the common dirnames. > > Well, you can create a dict that has dirname->list(filenames) which > will reduce the dirname to a single instance. You could store that > dict in the class, shared by all of the instances, though that starts > to pick up a code-smell. > > But unless you're talking about an obscenely large number of > dirnames & filenames, or a severely resource-limited machine, just > use the default built-ins. If you start to push the boundaries of > system resources, then I'd try the "anydbm" module or use the > "shelve" module to marshal them out to disk. Finally, you *could* > create an actual sqlite database on disk if size really does exceed > reasonable system specs. > > -tkc > Probably more memory efficient to make a list of lists, and just declare that element[0] of each list is the dirname. That way you're not wasting memory on the unused entryies of the hashtable. But unless the OP has both a) plus of a million entries and b) let's say at least 20 filenames to each dirname, it's not worth doing. Now, if you do really have a million entries, one thing that would help with memory is setting __slots__ for MyFile rather than letting it create an instance dictionary for each one. -- Rob Gaddi, Highland Technology -- www.highlandtechnology.com Email address domain is currently out of order. See above to fix. From python at mrabarnett.plus.com Thu Feb 11 22:13:17 2016 From: python at mrabarnett.plus.com (MRAB) Date: Fri, 12 Feb 2016 03:13:17 +0000 Subject: Storing a big amount of path names In-Reply-To: References: Message-ID: <56BD4DCD.4080401@mrabarnett.plus.com> On 2016-02-12 00:31, Paulo da Silva wrote: > Hi! > > What is the best (shortest memory usage) way to store lots of pathnames > in memory where: > > 1. Path names are pathname=(dirname,filename) > 2. There many different dirnames but much less than pathnames > 3. dirnames have in general many chars > > The idea is to share the common dirnames. > > More realistically not only the pathnames are stored but objects each > object being a MyFile containing > self.name - > getPathname(self) - > other stuff > > class MyFile: > > __allfiles=[] > > def __init__(self,dirname,filename): > self.dirname=dirname # But I want to share this with other files > self.name=filename > MyFile.__allfiles.append(self) > ... > > def getPathname(self): > return os.path.join(self.dirname,self.name) > > ... > Apart from all of the other answers that have been given: >>> p1 = 'foo/bar' >>> p2 = 'foo/bar' >>> id(p1), id(p2) (982008930176, 982008930120) >>> d = {} >>> id(d.setdefault(p1, p1)) 982008930176 >>> id(d.setdefault(p2, p2)) 982008930176 The dict maps equal strings (dirnames) to the same string, so you won't have multiple copies. From rosuav at gmail.com Thu Feb 11 22:49:15 2016 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 12 Feb 2016 14:49:15 +1100 Subject: Storing a big amount of path names In-Reply-To: <56BD4DCD.4080401@mrabarnett.plus.com> References: <56BD4DCD.4080401@mrabarnett.plus.com> Message-ID: On Fri, Feb 12, 2016 at 2:13 PM, MRAB wrote: > Apart from all of the other answers that have been given: > >>>> p1 = 'foo/bar' >>>> p2 = 'foo/bar' >>>> id(p1), id(p2) > (982008930176, 982008930120) >>>> d = {} >>>> id(d.setdefault(p1, p1)) > 982008930176 >>>> id(d.setdefault(p2, p2)) > 982008930176 > > The dict maps equal strings (dirnames) to the same string, so you won't have > multiple copies. Simpler to let the language do that for you: >>> import sys >>> p1 = sys.intern('foo/bar') >>> p2 = sys.intern('foo/bar') >>> id(p1), id(p2) (139621017266528, 139621017266528) ChrisA From p_s_d_a_s_i_l_v_a_ns at netcabo.pt Thu Feb 11 23:15:52 2016 From: p_s_d_a_s_i_l_v_a_ns at netcabo.pt (Paulo da Silva) Date: Fri, 12 Feb 2016 04:15:52 +0000 Subject: Storing a big amount of path names References: <56BD4DCD.4080401@mrabarnett.plus.com> Message-ID: ?s 03:49 de 12-02-2016, Chris Angelico escreveu: > On Fri, Feb 12, 2016 at 2:13 PM, MRAB wrote: >> Apart from all of the other answers that have been given: >> ... > > Simpler to let the language do that for you: > >>>> import sys >>>> p1 = sys.intern('foo/bar') >>>> p2 = sys.intern('foo/bar') >>>> id(p1), id(p2) > (139621017266528, 139621017266528) > I didn't know about id or sys.intern :-) I need to look at them ... As I can understand I can do in MyFile class self.dirname=sys.intern(dirname) # dirname passed as arg to the __init__ and the character string doesn't get repeated. Is this correct? From rosuav at gmail.com Thu Feb 11 23:23:20 2016 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 12 Feb 2016 15:23:20 +1100 Subject: Storing a big amount of path names In-Reply-To: References: <56BD4DCD.4080401@mrabarnett.plus.com> Message-ID: On Fri, Feb 12, 2016 at 3:15 PM, Paulo da Silva wrote: > ?s 03:49 de 12-02-2016, Chris Angelico escreveu: >> On Fri, Feb 12, 2016 at 2:13 PM, MRAB wrote: >>> Apart from all of the other answers that have been given: >>> > ... >> >> Simpler to let the language do that for you: >> >>>>> import sys >>>>> p1 = sys.intern('foo/bar') >>>>> p2 = sys.intern('foo/bar') >>>>> id(p1), id(p2) >> (139621017266528, 139621017266528) >> > > I didn't know about id or sys.intern :-) > I need to look at them ... > > As I can understand I can do in MyFile class > > self.dirname=sys.intern(dirname) # dirname passed as arg to the __init__ > > and the character string doesn't get repeated. > Is this correct? Correct. Two equal strings, passed to sys.intern(), will come back as identical strings, which means they use the same memory. You can have a million references to the same string and it takes up no additional memory. But I reiterate: Don't even bother with this unless you know your program is running short of memory. Start by coding things in the simple and obvious way, and then fix problems only when you see them. ChrisA From p_s_d_a_s_i_l_v_a_ns at netcabo.pt Thu Feb 11 23:45:57 2016 From: p_s_d_a_s_i_l_v_a_ns at netcabo.pt (Paulo da Silva) Date: Fri, 12 Feb 2016 04:45:57 +0000 Subject: Storing a big amount of path names References: <56BD4DCD.4080401@mrabarnett.plus.com> Message-ID: ?s 04:23 de 12-02-2016, Chris Angelico escreveu: > On Fri, Feb 12, 2016 at 3:15 PM, Paulo da Silva > wrote: >> ?s 03:49 de 12-02-2016, Chris Angelico escreveu: >>> On Fri, Feb 12, 2016 at 2:13 PM, MRAB wrote: >>>> Apart from all of the other answers that have been given: >>>> >> ... >>> >>> Simpler to let the language do that for you: >>> >>>>>> import sys >>>>>> p1 = sys.intern('foo/bar') >>>>>> p2 = sys.intern('foo/bar') >>>>>> id(p1), id(p2) >>> (139621017266528, 139621017266528) >>> >> >> I didn't know about id or sys.intern :-) >> I need to look at them ... >> >> As I can understand I can do in MyFile class >> >> self.dirname=sys.intern(dirname) # dirname passed as arg to the __init__ >> >> and the character string doesn't get repeated. >> Is this correct? > > Correct. Two equal strings, passed to sys.intern(), will come back as > identical strings, which means they use the same memory. You can have > a million references to the same string and it takes up no additional > memory. I have being playing with this and found that it is not always true! For example: In [1]: def f(s): ...: print(id(sys.intern(s))) ...: In [2]: import sys In [3]: f("12345") 139805480756480 In [4]: f("12345") 139805480755640 In [5]: f("12345") 139805480756480 In [6]: f("12345") 139805480756480 In [7]: f("12345") 139805480750864 I think a dict, as MRAB suggested, is needed. At the end of the store process I may delete the dict. > > But I reiterate: Don't even bother with this unless you know your > program is running short of memory. Yes, it is. This is part of a previous post (sets of equal files) and I need lots of memory for performance reasons. I only have 2G in this computer. I already had implemented a solution. I used two dicts. One to map dirnames to an int handler and the other to map the handler to dir names. At the end I deleted the 1st. one because I only need to get the dirname from the handler. But I thought there should be a better choice. Thanks Paulo From rosuav at gmail.com Fri Feb 12 00:02:43 2016 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 12 Feb 2016 16:02:43 +1100 Subject: Storing a big amount of path names In-Reply-To: References: <56BD4DCD.4080401@mrabarnett.plus.com> Message-ID: On Fri, Feb 12, 2016 at 3:45 PM, Paulo da Silva wrote: >> Correct. Two equal strings, passed to sys.intern(), will come back as >> identical strings, which means they use the same memory. You can have >> a million references to the same string and it takes up no additional >> memory. > I have being playing with this and found that it is not always true! > For example: > > In [1]: def f(s): > ...: print(id(sys.intern(s))) > ...: > > In [2]: import sys > > In [3]: f("12345") > 139805480756480 > > In [4]: f("12345") > 139805480755640 > > In [5]: f("12345") > 139805480756480 > > In [6]: f("12345") > 139805480756480 > > In [7]: f("12345") > 139805480750864 > > I think a dict, as MRAB suggested, is needed. > At the end of the store process I may delete the dict. I'm not 100% sure of what's going on here, but my suspicion is that a string that isn't being used is allowed to be flushed from the dictionary. If you retain a reference to the string (not to its id, but to the string itself), you shouldn't see that change. By doing the dict yourself, you guarantee that ALL the strings will be retained, which can never be _less_ memory than interning them all, and can easily be _more_. >> But I reiterate: Don't even bother with this unless you know your >> program is running short of memory. > > Yes, it is. > This is part of a previous post (sets of equal files) and I need lots of > memory for performance reasons. I only have 2G in this computer. How many files, roughly? Do you ever look at the contents of the files? Most likely, you'll be dwarfing the files' names with their contents. Unless you actually have over two million unique files, each one with over a thousand characters in the name, you can't use all that 2GB with file names. If virtual memory is active, all that'll happen is that you dip into the swapper / page file a bit... and THAT is when you start looking at reducing memory usage. Don't bother optimizing until you need to, and even then, you measure first to see what part of the program actually needs to be optimized. > I already had implemented a solution. I used two dicts. One to map > dirnames to an int handler and the other to map the handler to dir > names. At the end I deleted the 1st. one because I only need to get the > dirname from the handler. But I thought there should be a better choice. If all your dir names are interned, their identities (approximately the values returned by id(), but not quite) will be those handlers for you, without any overhead and without any complexity. ChrisA From mentificium at gmail.com Fri Feb 12 00:21:47 2016 From: mentificium at gmail.com (mentificium at gmail.com) Date: Thu, 11 Feb 2016 21:21:47 -0800 (PST) Subject: First Program with Python! In-Reply-To: <59666b2f-1368-4652-9791-3b267e2fe634@googlegroups.com> References: <59666b2f-1368-4652-9791-3b267e2fe634@googlegroups.com> Message-ID: <01e32351-dcf4-4858-a8c9-fa1da5046d6f@googlegroups.com> On Tuesday, February 9, 2016 at 5:55:30 AM UTC-8, Anita Goyal wrote: > > Start learning Python from basics to advance levels here... > https://goo.gl/hGzm6o Python experts please translate the webserver "Ghost" Perlmind into Python. http://ai.neocities.org/P6AI_FAQ.html From p_s_d_a_s_i_l_v_a_ns at netcabo.pt Fri Feb 12 00:49:52 2016 From: p_s_d_a_s_i_l_v_a_ns at netcabo.pt (Paulo da Silva) Date: Fri, 12 Feb 2016 05:49:52 +0000 Subject: Storing a big amount of path names References: <56BD4DCD.4080401@mrabarnett.plus.com> Message-ID: ?s 05:02 de 12-02-2016, Chris Angelico escreveu: > On Fri, Feb 12, 2016 at 3:45 PM, Paulo da Silva > wrote: ... >> I think a dict, as MRAB suggested, is needed. >> At the end of the store process I may delete the dict. > > I'm not 100% sure of what's going on here, but my suspicion is that a > string that isn't being used is allowed to be flushed from the > dictionary. You are right. I have tried with a small class and it seems to work. Thanks. ... > > How many files, roughly? Do you ever look at the contents of the > files? Most likely, you'll be dwarfing the files' names with their > contents. Unless you actually have over two million unique files, each > one with over a thousand characters in the name, you can't use all > that 2GB with file names. That's not only the filenames. The more memory I have more expensive but faster algorithm I can implement. Thank you very much for your nice suggestion which also contributed to my Python knowledge. Thank you all who responded. Paulo From steve at pearwood.info Fri Feb 12 00:51:21 2016 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 12 Feb 2016 16:51:21 +1100 Subject: Storing a big amount of path names References: <56BD4DCD.4080401@mrabarnett.plus.com> Message-ID: <56bd72db$0$1615$c3e8da3$5496439d@news.astraweb.com> On Fri, 12 Feb 2016 04:02 pm, Chris Angelico wrote: > On Fri, Feb 12, 2016 at 3:45 PM, Paulo da Silva > wrote: >>> Correct. Two equal strings, passed to sys.intern(), will come back as >>> identical strings, which means they use the same memory. You can have >>> a million references to the same string and it takes up no additional >>> memory. >> I have being playing with this and found that it is not always true! It is true, but only for the lifetime of the string. Once the string is garbage collected, it is removed from the cache as well. If you then add the string again, you may not get the same id. py> mystr = "hello world" py> str2 = sys.intern(mystr) py> str3 = "hello world" py> mystr is str2 # same string object, as str2 is interned True py> mystr is str3 # not the same string object False But if we delete all references to the string objects, the intern cache is also flushed, and we may not get the same id: py> del str2, str3 py> id(mystr) # remember this ID number 3079482600 py> del mystr py> id(sys.intern("hello world")) # a new entry in the cache 3079227624 This is the behaviour you want: if a string is completely deleted, you don't want it remaining in the intern cache taking up memory. > I'm not 100% sure of what's going on here, but my suspicion is that a > string that isn't being used is allowed to be flushed from the > dictionary. If you retain a reference to the string (not to its id, > but to the string itself), you shouldn't see that change. By doing the > dict yourself, you guarantee that ALL the strings will be retained, > which can never be _less_ memory than interning them all, and can > easily be _more_. Yep. Back in the early days, interned strings were immortal and lasted forever. That wasted memory, and is no longer the case. -- Steven From mr.eightnoteight at gmail.com Fri Feb 12 01:16:41 2016 From: mr.eightnoteight at gmail.com (srinivas devaki) Date: Fri, 12 Feb 2016 11:46:41 +0530 Subject: Storing a big amount of path names In-Reply-To: References: Message-ID: On Feb 12, 2016 6:05 AM, "Paulo da Silva" wrote: > > Hi! > > What is the best (shortest memory usage) way to store lots of pathnames > in memory where: > > 1. Path names are pathname=(dirname,filename) > 2. There many different dirnames but much less than pathnames > 3. dirnames have in general many chars > > The idea is to share the common dirnames. > > More realistically not only the pathnames are stored but objects each > object being a MyFile containing > self.name - > getPathname(self) - > other stuff > > class MyFile: > > __allfiles=[] > > def __init__(self,dirname,filename): > self.dirname=dirname # But I want to share this with other files > self.name=filename > MyFile.__allfiles.append(self) > ... > > def getPathname(self): > return os.path.join(self.dirname,self.name) > what you want is Trie data structure, which won't use extra memory if the basepath of your strings are common. instead of having constructing a char Trie, try to make it as string Trie i.e each directory name is a node and all the files and folders are it's children, each node can be of two types a file and folder. if you come to think about it this is most intuitive way to represent the file structure in your program. you can extract the directory name from the file object by traversing it's parents. I hope this helps. Regards Srinivas Devaki Junior (3rd yr) student at Indian School of Mines,(IIT Dhanbad) Computer Science and Engineering Department ph: +91 9491 383 249 telegram_id: @eightnoteight From zubvit at gmail.com Fri Feb 12 01:23:02 2016 From: zubvit at gmail.com (zubvit at gmail.com) Date: Thu, 11 Feb 2016 22:23:02 -0800 (PST) Subject: Best programs written completly in Python In-Reply-To: References: Message-ID: On Sunday, August 5, 2007 at 1:14:38 PM UTC+3, Franz Steinh?usler wrote: > Hello NG, > > wWhat are the best programs in your opinion, written entirly > in pyhton, divided into categories like: > a) Games > b) Utilities/System > c) Office > d) Web/Newsreader/Mail/Browser > ... > > I don't want to start a long thread, if a site of such > an discussion already exists, a link will be enough. > > Many thanks in advance! > > -- > Franz Steinhaeusler ankiweb.net + ankisrs.net = amazing! From dieter at handshake.de Fri Feb 12 02:31:40 2016 From: dieter at handshake.de (dieter) Date: Fri, 12 Feb 2016 08:31:40 +0100 Subject: Handling transactions in Python DBI module References: <92D3C964-0323-46EE-B770-B89E7E7E6D36@ravnalaska.net> <87io1vlltm.fsf@handshake.de> Message-ID: <87a8n6v0yr.fsf@handshake.de> Chris Angelico writes: > On Thu, Feb 11, 2016 at 6:59 PM, dieter wrote: >> In my context (web applications), I strongly discourage this use - at least >> when "conn" does not handle subtransactions properly. >> >> In a web application, the main transaction should in general be controlled >> at the request level: a request should either succeed as a whole or have no >> side effects whatsoever. This prohibits local (somewhere >> deep in a component) transaction control. > > Hmm. I'm not 100% convinced that web apps should behave that way; In a web application, the user either sees a success or an error response (for his request). In case of an error, it would be fatal (in the general case) that persistent data would have been inconsistently modified (e.g. an purchase performed; without proper acknowledgent to the user). > but > part of the simplicity comes from requiring that database-dependent > code should not call other database-dependent code, as that would > create a nested transaction. The web applications I am working with are highly modular. Many of those modules manage persistent data in a database and operate on it independently. It would be fatal when a successful database interaction in one module would as a side effect commit database operations in another module. > (That said, though, it's entirely > possible that psycopg2 could handle a nested "with conn" as > "SAVEPOINT" and either "RELEASE SAVEPOINT" or "ROLLBACK TO SAVEPOINT". > But I wouldn't recommend depending on that without confirming it in > the docs.) For "psycopg2", I have done this. > Bear in mind, the rule I gave was a broad and general rule for > students to follow, not a hard-and-fast rule for all databasing. It's > designed such that the student can learn the exceptions later on, but > meanwhile, his code will be correct (if occasionally a little warped > to avoid nesting transactions). I want to stress that in the domain of web applications, local transaction control is not a good rule of thumb. At least there, transaction control should in general belong to the request framework (as this is the atomic user interaction level for web applications) and to local components. From john_ladasky at sbcglobal.net Fri Feb 12 02:51:22 2016 From: john_ladasky at sbcglobal.net (John Ladasky) Date: Thu, 11 Feb 2016 23:51:22 -0800 (PST) Subject: [STORY-TIME] THE BDFL AND HIS PYTHON PETTING ZOO In-Reply-To: <4e902c58-3370-4adf-905d-74630a6eea90@googlegroups.com> References: <48762040-a7e0-434c-92e8-8a0969210e5b@googlegroups.com> <5d31dd14-1a60-4f80-8889-d0616f404c70@googlegroups.com> <4e902c58-3370-4adf-905d-74630a6eea90@googlegroups.com> Message-ID: <60036746-78f7-4f05-a6d8-04e88b09c00f@googlegroups.com> On Saturday, February 6, 2016 at 12:54:41 PM UTC-8, Rick Johnson wrote: > On Wednesday, February 3, 2016 at 12:02:35 AM UTC-6, John Ladasky wrote: > > > Rick, you don't like Python? > > If i didn't like Python, then i would happily let it self- > destruct, yes? The problem is, i *DO* like Python. Python2 > was a great language, but python3 has been a disaster. Heck, > even the BDFL has warned that Python4 cannot introduce as > many backwards incompatible changes as python3. So obviously, > he is admitting that Python3 was a disaster. I had to wait until my favorite packages were ported (numpy, scipy, matplotlib, pandas). But once that happened, I moved from Py2 to Py3 years ago with scarcely a bump, bruise, or scratch. I like lazy evaluation. I think that Unicode handling is vastly improved (and yes, I'm fully aware that exactly two people in this newsgroup don't agree, they make sure we all know). I have encountered few surprises, and nothing that makes my job harder. To be sure, people who are entrenched in the Py2 way of doing things, with a lot of legacy code, have some work to do -- on their code, and possibly on their brains. Keep Py2 if you want it, then. You still have a few more years before the PSF stops maintaining it. If you really like it that much, why not maintain it yourself? From frank at chagford.com Fri Feb 12 03:17:34 2016 From: frank at chagford.com (Frank Millman) Date: Fri, 12 Feb 2016 10:17:34 +0200 Subject: asyncio and blocking - an update In-Reply-To: References: Message-ID: "Frank Millman" wrote in message news:n9hjfp$ad7$1 at ger.gmane.org... > > However, my concern is not to maximise database performance, but to ensure > that in an asynchronous environment, one task does not block the others > from responding. My tests simulate a number of tasks running concurrently > and trying to access the database. Among other measurements, I track the > time that each database access commences. As I expected, tasks run with > 'run_in_executor' run sequentially, i.e. the next one only starts when the > previous one has finished. This is not because the tasks themselves are > sequential, but because 'fetchall()' is (I think) a blocking operation. > Conversely, with my approach, all the tasks start within a short time of > each other. Because I can process the rows as they are received, it seems > to give each task a fairer time allocation. Not to mention that there are > very likely to be other non-database tasks running concurrently, and they > should also be more responsive. > > It would be quite difficult to simulate all of this, so I confess that I > am relying on gut instinct at the moment. > It seems that my gut instinct was correct. Up to now my timing tests have been run independently of my app, but now I have embedded them so that I can run the tests while I am logged in as a user. I run a task every 10 seconds that runs 25 concurrent tasks, each reading a database table of about 2000 rows. Using run_in_executor() and cur.fetchall(), I experience delays of up to 2 seconds while the task is active. Using my approach, the maximum I saw was about a tenth of a second, and that is because I was looking for it - a normal user would not notice. Obviously the task took longer, but I can live with that trade-off. As I mentioned before, I could be using run_in_executor() in a na?ve way, and there could be better approaches. But until someone points out a better way, I have nothing else to go on. Frank From rosuav at gmail.com Fri Feb 12 03:19:41 2016 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 12 Feb 2016 19:19:41 +1100 Subject: asyncio and blocking - an update In-Reply-To: References: Message-ID: On Fri, Feb 12, 2016 at 7:17 PM, Frank Millman wrote: > Using run_in_executor() and cur.fetchall(), I experience delays of up to 2 > seconds while the task is active. > > Using my approach, the maximum I saw was about a tenth of a second, and that > is because I was looking for it - a normal user would not notice. Obviously > the task took longer, but I can live with that trade-off. That looks like good, useful data! Cool. ChrisA From antoon.pardon at rece.vub.ac.be Fri Feb 12 03:35:40 2016 From: antoon.pardon at rece.vub.ac.be (Antoon Pardon) Date: Fri, 12 Feb 2016 09:35:40 +0100 Subject: tarfile : read from a socket? In-Reply-To: <20160211172728.GA27743@axis.g33x.de> References: <20160211172728.GA27743@axis.g33x.de> Message-ID: <56BD995C.1030600@rece.vub.ac.be> On 02/11/2016 06:27 PM, Lars Gust?bel wrote: > On Thu, Feb 11, 2016 at 04:41:43PM +0000, Ulli Horlacher wrote: >> sfo = sock.makefile('r') >> taro = tarfile.open(fileobj=sfo,mode='r|') >> taro.extractall(path=edir) > What about using an iterator? > > def myiter(tar): > for t in tar: > print "extracting", t.name > yield t > > sfo = sock.makefile('r') > taro = tarfile.open(fileobj=sfo,mode='r|') > taro.extractall(members=myiter(taro),path=edir) > > Cheers, The tarfile is already an iterator. Just do the following: for ti in taro: print "extracting", ti.name taro.extract(ti) From lars at gustaebel.de Fri Feb 12 03:40:38 2016 From: lars at gustaebel.de (Lars =?iso-8859-1?Q?Gust=E4bel?=) Date: Fri, 12 Feb 2016 09:40:38 +0100 Subject: tarfile : read from a socket? In-Reply-To: <56BD995C.1030600@rece.vub.ac.be> References: <20160211172728.GA27743@axis.g33x.de> <56BD995C.1030600@rece.vub.ac.be> Message-ID: <20160212084038.GA26255@axis.g33x.de> On Fri, Feb 12, 2016 at 09:35:40AM +0100, Antoon Pardon wrote: > On 02/11/2016 06:27 PM, Lars Gust?bel wrote: > > What about using an iterator? > > > > def myiter(tar): > > for t in tar: > > print "extracting", t.name > > yield t > > > > sfo = sock.makefile('r') > > taro = tarfile.open(fileobj=sfo,mode='r|') > > taro.extractall(members=myiter(taro),path=edir) > > > The tarfile is already an iterator. Just do the following: for ti in > taro: print "extracting", ti.name taro.extract(ti) The extractall() method does a little bit more than just extract(), i.e. setting directory mtimes, see https://docs.python.org/3.5/library/tarfile.html#tarfile.TarFile.extractall -- Lars Gust?bel lars at gustaebel.de From sh5501970 at gmail.com Fri Feb 12 04:08:05 2016 From: sh5501970 at gmail.com (Mohammed Zakria) Date: Fri, 12 Feb 2016 01:08:05 -0800 (PST) Subject: Python study Message-ID: <48b3e67b-09e9-4adb-8961-d0c8fdae0efe@googlegroups.com> hello thank you very much for this intersting programming languch now I finished study Fundamentals of python in microsift and now im asking about what course should study after that and in web or game or networking? thank you for your time mz From sh5501970 at gmail.com Fri Feb 12 04:46:46 2016 From: sh5501970 at gmail.com (Mohammed Zakria) Date: Fri, 12 Feb 2016 01:46:46 -0800 (PST) Subject: working Message-ID: <4a3a1a88-4776-4d05-b3ad-962c23d59399@googlegroups.com> hello i want to know the company that ican work as freelance python devloper From framstag at rus.uni-stuttgart.de Fri Feb 12 07:48:30 2016 From: framstag at rus.uni-stuttgart.de (Ulli Horlacher) Date: Fri, 12 Feb 2016 12:48:30 +0000 (UTC) Subject: tarfile : read from a socket? References: <20160211172728.GA27743@axis.g33x.de> <56BD995C.1030600@rece.vub.ac.be> Message-ID: Lars Gust?bel wrote: > On Fri, Feb 12, 2016 at 09:35:40AM +0100, Antoon Pardon wrote: > > On 02/11/2016 06:27 PM, Lars Gust?bel wrote: > > > What about using an iterator? > > > > > > def myiter(tar): > > > for t in tar: > > > print "extracting", t.name > > > yield t > > > > > > sfo = sock.makefile('r') > > > taro = tarfile.open(fileobj=sfo,mode='r|') > > > taro.extractall(members=myiter(taro),path=edir) > > > > > The tarfile is already an iterator. Just do the following: for ti in > > taro: print "extracting", ti.name taro.extract(ti) > > The extractall() method does a little bit more than just extract(), i.e. > setting directory mtimes, see > https://docs.python.org/3.5/library/tarfile.html#tarfile.TarFile.extractall This is an important hint! Thanks! -- Ullrich Horlacher Server und Virtualisierung Rechenzentrum IZUS/TIK E-Mail: horlacher at tik.uni-stuttgart.de Universitaet Stuttgart Tel: ++49-711-68565868 Allmandring 30a Fax: ++49-711-682357 70550 Stuttgart (Germany) WWW: http://www.tik.uni-stuttgart.de/ From mentificium at gmail.com Fri Feb 12 08:59:44 2016 From: mentificium at gmail.com (mentificium at gmail.com) Date: Fri, 12 Feb 2016 05:59:44 -0800 (PST) Subject: Best programs written completly in Python In-Reply-To: References: Message-ID: <4bc2a922-3ba9-44f0-8628-13ec06342264@googlegroups.com> On Sunday, August 5, 2007 at 3:14:38 AM UTC-7, Franz Steinh?usler wrote: > Hello NG, > > wWhat are the best programs in your opinion, written entirly > in pyhton, divided into categories like: Maybe such a list of categories should include a) Artificial Intelligence > a) Games > b) Utilities/System > c) Office > d) Web/Newsreader/Mail/Browser > ... > > I don't want to start a long thread, if a site of such > an discussion already exists, a link will be enough. > > Many thanks in advance! > > -- > Franz Steinhaeusler A winner-take-all genuine "killer app" for Python would be a port of the webserver "Ghost" Strong AI artificial intelligence being ported from Forth first into Perl5 and then the newly released Perl6. See http://dl.acm.org/citation.cfm?doid=307824.307853 on Mind.Forth AI; http://aihub.net/artificial-intelligence-lab-projects http://ai.neocities.org/perlmind.txt -- Download URL; http://ai.neocities.org/P6AI_FAQ.html -- Frequently Asked Questions; http://ai.neocities.org/P6AI_man.html -- Perl6 AI User Manual; http://ai.neocities.org/PMPJ.html -- Perl Mind Programming Journal; http://www.amazon.com/dp/B00FKJY1WY shows how the Perl AI reasons. May the best programming language win on the way to the Singularity! Respectfully submitted AiHasBeenSolved From oscar.j.benjamin at gmail.com Fri Feb 12 09:52:43 2016 From: oscar.j.benjamin at gmail.com (Oscar Benjamin) Date: Fri, 12 Feb 2016 14:52:43 +0000 Subject: I cannot open IDLE In-Reply-To: References: Message-ID: On 11 February 2016 at 18:02, anthony averett wrote: > I have downloaded python but I cannot open up idle. I really need this > issue resolved for I have work do for one of my computer science class. What operating system are you using? Windows? Which version? Have you installed Python and if so where did you get it from? Assuming you're on Windows and used the normal Python installer and that you installed Python (rather than simply downloading the installer) then there should be a menu item for IDLE somewhere in your programs menu. -- Oscar From pmiscml at gmail.com Fri Feb 12 10:43:38 2016 From: pmiscml at gmail.com (Paul Sokolovsky) Date: Fri, 12 Feb 2016 17:43:38 +0200 Subject: [ANN] MicroPython 1.6 Message-ID: <20160212174338.651accf5@x230> Hello, MicroPython is a lean and efficient Python implementation for microcontrollers, embedded, and mobile systems (which also runs just as fine on desktops, servers, and clouds). https://github.com/micropython/micropython https://github.com/micropython/micropython/releases/tag/v1.6 There're following major changes since 1.5: 1. LwIP module support for embedded TCP/IP networking. 2. IPv6 support in the Unix port. 3. Beta support for persistent bytecode (similar to CPython's .pyc) 4. 64-bit NaN boxing (improved floating-point performance if enabled). 5. Support for new official PyBoards PYBv1.1 and PYBLITEv1.0. 6. Long int constant folding during bytecode compilation (glad that CPython will catch up in that area thanks to FAT Python project). 7. There's a ongoing crowdfunding campaign to fund complete and well-maintained MicroPython port to ubiquitous ESP8266 WiFi SoC, and improve networking and IoT support in MicroPython in general: https://www.kickstarter.com/projects/214379695/micropython-on-the-esp8266-beautifully-easy-iot -- Best regards, Paul mailto:pmiscml at gmail.com From random832 at fastmail.com Fri Feb 12 11:01:13 2016 From: random832 at fastmail.com (Random832) Date: Fri, 12 Feb 2016 11:01:13 -0500 Subject: tarfile : secure extract? In-Reply-To: References: Message-ID: <1455292873.3836281.519527058.29D12FE3@webmail.messagingengine.com> On Thu, Feb 11, 2016, at 18:24, Ulli Horlacher wrote: > A better approach would be to rename such files while extracting. > Is this possible? What happens if you change member.name before extracting? From rgaddi at highlandtechnology.invalid Fri Feb 12 12:05:24 2016 From: rgaddi at highlandtechnology.invalid (Rob Gaddi) Date: Fri, 12 Feb 2016 17:05:24 -0000 (UTC) Subject: Storing a big amount of path names References: <56BD4DCD.4080401@mrabarnett.plus.com> Message-ID: Chris Angelico wrote: > Start by coding things in the > simple and obvious way, and then fix problems only when you see them. Is that statement available in 10 foot letters etched into stone? -- Rob Gaddi, Highland Technology -- www.highlandtechnology.com Email address domain is currently out of order. See above to fix. From rosuav at gmail.com Fri Feb 12 12:18:16 2016 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 13 Feb 2016 04:18:16 +1100 Subject: Storing a big amount of path names In-Reply-To: References: <56BD4DCD.4080401@mrabarnett.plus.com> Message-ID: On Sat, Feb 13, 2016 at 4:05 AM, Rob Gaddi wrote: > Chris Angelico wrote: > >> Start by coding things in the >> simple and obvious way, and then fix problems only when you see them. > > Is that statement available in 10 foot letters etched into stone? I actually had that built behind my house, at one point. Sadly, the letters sank until they were partly embedded into the ground, and what's left says, in the local language, "Go stick your head in a PHP", so it's lit up only for special celebrations. ChrisA From sohcahtoa82 at gmail.com Fri Feb 12 13:57:28 2016 From: sohcahtoa82 at gmail.com (sohcahtoa82 at gmail.com) Date: Fri, 12 Feb 2016 10:57:28 -0800 (PST) Subject: working In-Reply-To: <4a3a1a88-4776-4d05-b3ad-962c23d59399@googlegroups.com> References: <4a3a1a88-4776-4d05-b3ad-962c23d59399@googlegroups.com> Message-ID: <9428d1c1-70ee-4984-9ecc-ebbde0e579e4@googlegroups.com> On Friday, February 12, 2016 at 1:47:24 AM UTC-8, Mohammed Zakria wrote: > hello > i want to know the company that ican work as freelance python devloper There are some recruiters that read this mailing list and will send unsolicited e-mail about job openings, but they might pass right over you if you're not willing to spend the time to proofread your messages. From lars at gustaebel.de Fri Feb 12 14:21:25 2016 From: lars at gustaebel.de (Lars =?iso-8859-1?Q?Gust=E4bel?=) Date: Fri, 12 Feb 2016 20:21:25 +0100 Subject: tarfile : secure extract? In-Reply-To: References: Message-ID: <20160212192125.GA6690@axis.g33x.de> On Thu, Feb 11, 2016 at 11:24:01PM +0000, Ulli Horlacher wrote: > In https://docs.python.org/2/library/tarfile.html there is a warning: > > Never extract archives from untrusted sources without prior inspection. > It is possible that files are created outside of path, e.g. members that > have absolute filenames starting with "/" or filenames with two dots > "..". > > My program has to extract tar archives from untrusted sources :-} Read the discussion in this issue on why this might be a bad idea: http://bugs.python.org/issue21109 -- Lars Gust?bel lars at gustaebel.de From framstag at rus.uni-stuttgart.de Fri Feb 12 14:43:32 2016 From: framstag at rus.uni-stuttgart.de (Ulli Horlacher) Date: Fri, 12 Feb 2016 19:43:32 +0000 (UTC) Subject: tarfile : secure extract? References: Message-ID: Random832 wrote: > On Thu, Feb 11, 2016, at 18:24, Ulli Horlacher wrote: > > A better approach would be to rename such files while extracting. > > Is this possible? > > What happens if you change member.name before extracting? Ohh... such an easy solution! :-) -- Ullrich Horlacher Server und Virtualisierung Rechenzentrum IZUS/TIK E-Mail: horlacher at tik.uni-stuttgart.de Universitaet Stuttgart Tel: ++49-711-68565868 Allmandring 30a Fax: ++49-711-682357 70550 Stuttgart (Germany) WWW: http://www.tik.uni-stuttgart.de/ From sonimanas695 at gmail.com Fri Feb 12 15:16:49 2016 From: sonimanas695 at gmail.com (Manas Soni) Date: Fri, 12 Feb 2016 20:16:49 +0000 Subject: No subject Message-ID: <56be3d9f.c711c30a.e9bd9.fffff70f@mx.google.com> I have downloaded python and when I click on it, it asks me to repair which I do, it then says successful however when I click on it again it won?t let me on it Sent from Mail for Windows 10 From m at funkyhat.org Fri Feb 12 15:53:16 2016 From: m at funkyhat.org (Matt Wheeler) Date: Fri, 12 Feb 2016 20:53:16 +0000 Subject: modifying a standard module? (was: Re: tarfile : read from a socket?) In-Reply-To: References: Message-ID: On 11 February 2016 at 17:10, Ulli Horlacher wrote: > > Ulli Horlacher wrote: > As a hack, I modified the standard library module tarfile.py: > > root at diaspora:/usr/lib/python2.7# vv -d > --- ./.versions/tarfile.py~1~ 2015-06-22 21:59:27.000000000 +0200 > +++ tarfile.py 2016-02-11 18:01:50.185555952 +0100 > @@ -2045,6 +2045,7 @@ > directories.append(tarinfo) > tarinfo = copy.copy(tarinfo) > tarinfo.mode = 0700 > + print('untar "%s"' % tarinfo.name) > self.extract(tarinfo, path) > > # Reverse sort directories. > > > This gives me exact the output I want :-) > > BUT I want to distribute my program and all others will not see the tar > extracting information. > > Now my question: > > How can I substitute the standard module function tarfile.extractall() with > my own function? import tarfile def new_extractall(self, *args, **kwargs): print("I am a function. Woohoo!") tarfile.TarFile.extractall = new_extractall But bear in mind that that will change tarfile.extractall for every single module that imports it within the same python process. Is that really what you want? Is there a reason you can't subclass TarFile as others have suggested? Perhaps even this is enough: class NoisyTarFile(TarFile): """untested, sorry""" def extract(self, member, *args, **kwargs): print('extracting "%s"' % member.name) super(NoisyTarFile, self).extract(member, *args, **kwargs) As the very next step after your print in extractall is a call to extract anyway? If you must patch the standard library tarfile module then I would suggest patching it to have an extra, default False, argument to enable your printing behaviour, so you don't risk messing up anyone else's use of it. -- Matt Wheeler http://funkyh.at From breamoreboy at yahoo.co.uk Fri Feb 12 16:37:14 2016 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Fri, 12 Feb 2016 21:37:14 +0000 Subject: Storing a big amount of path names In-Reply-To: References: <56BD4DCD.4080401@mrabarnett.plus.com> Message-ID: On 12/02/2016 17:05, Rob Gaddi wrote: > Chris Angelico wrote: > >> Start by coding things in the >> simple and obvious way, and then fix problems only when you see them. > > Is that statement available in 10 foot letters etched into stone? > Hopefully not as that would be a waste, it should be made more obvious by using a red hot poker to engrave it onto every newbies' forehead. Even then some simply wouldn't take a blind bit of notice. -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From breamoreboy at yahoo.co.uk Fri Feb 12 16:39:27 2016 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Fri, 12 Feb 2016 21:39:27 +0000 Subject: (unknown) In-Reply-To: <56be3d9f.c711c30a.e9bd9.fffff70f@mx.google.com> References: <56be3d9f.c711c30a.e9bd9.fffff70f@mx.google.com> Message-ID: On 12/02/2016 20:16, Manas Soni wrote: > > I have downloaded python and when I click on it, it asks me to repair which I do, it then says successful however when I click on it again it won?t let me on it > Sent from Mail for Windows 10 > Please search the archives as this has been asked and answered repeatedly over the last few months. -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From ben+python at benfinney.id.au Fri Feb 12 16:49:24 2016 From: ben+python at benfinney.id.au (Ben Finney) Date: Sat, 13 Feb 2016 08:49:24 +1100 Subject: Storing a big amount of path names References: <56BD4DCD.4080401@mrabarnett.plus.com> Message-ID: <85a8n57g63.fsf@benfinney.id.au> Chris Angelico writes: > I actually had that built behind my house, at one point. Sadly, the > letters sank until they were partly embedded into the ground, and > what's left says, in the local language, "Go stick your head in a > PHP", so it's lit up only for special celebrations. Douglas Adams, you are sorely missed. -- \ ?The greatest tragedy in mankind's entire history may be the | `\ hijacking of morality by religion.? ?Arthur C. Clarke, 1991 | _o__) | Ben Finney From framstag at rus.uni-stuttgart.de Fri Feb 12 17:46:59 2016 From: framstag at rus.uni-stuttgart.de (Ulli Horlacher) Date: Fri, 12 Feb 2016 22:46:59 +0000 (UTC) Subject: modifying a standard module? References: Message-ID: Matt Wheeler wrote: > > How can I substitute the standard module function tarfile.extractall() with > > my own function? > > import tarfile > def new_extractall(self, *args, **kwargs): > print("I am a function. Woohoo!") > > tarfile.TarFile.extractall = new_extractall This is more easy than I could imagined :-) It is in my Python notes, now. > But bear in mind that that will change tarfile.extractall for every > single module that imports it within the same python process. Is that > really what you want? Yes. I have no own modules. Just one program file. > Is there a reason you can't subclass TarFile as others have suggested? The reason: I have no ideas on classes :-} Of course, I should have to learn about, but until now it was not necessary. The other solutions in this thread are sufficent for me. Meanwhile I have implemented the iterator function: taro.extractall(members=itar(taro),path=edir) def itar(tar): for ti in tar: # minimal protection against dangerous file names # see http://bugs.python.org/issue21109#msg215656 ti.name = subst(r'^(?i)([a-z]:)?(\.\.)?[/\\]','',ti.name) print('untar "%s"' % ti.name) yield ti Perfekt solution for me :-) Thanks to all. > If you must patch the standard library tarfile module then I would > suggest patching it to have an extra, default False, argument to > enable your printing behaviour, so you don't risk messing up anyone > else's use of it. Yes, a good idea. -- Ullrich Horlacher Server und Virtualisierung Rechenzentrum IZUS/TIK E-Mail: horlacher at tik.uni-stuttgart.de Universitaet Stuttgart Tel: ++49-711-68565868 Allmandring 30a Fax: ++49-711-682357 70550 Stuttgart (Germany) WWW: http://www.tik.uni-stuttgart.de/ From funkyhat at gmail.com Fri Feb 12 18:31:35 2016 From: funkyhat at gmail.com (Matt Wheeler) Date: Fri, 12 Feb 2016 23:31:35 +0000 Subject: Storing a big amount of path names In-Reply-To: References: <56BD4DCD.4080401@mrabarnett.plus.com> Message-ID: On 12 Feb 2016 21:37, "Mark Lawrence" wrote: > Hopefully not as that would be a waste, it should be made more obvious by using a red hot poker to engrave it onto every newbies' forehead. Even then some simply wouldn't take a blind bit of notice. Yes sorry about that, I think our aim was a little off with a few of the brandings. -- Matt Wheeler http://funkyh.at From no.email at nospam.invalid Fri Feb 12 19:21:30 2016 From: no.email at nospam.invalid (Paul Rubin) Date: Fri, 12 Feb 2016 16:21:30 -0800 Subject: modifying a standard module? References: Message-ID: <87bn7lcved.fsf@nightsong.com> Ulli Horlacher writes: >> tarfile.TarFile.extractall = new_extractall > > This is more easy than I could imagined :-) It is in my Python notes, > now. This is called "duck punching" or "monkey patching" and sometimes it's necessary, but it's something of an antipattern since the module could change under you between versions, and that sort of thing. If you have to do it, then fine; but if you have a choice, it's preferable to avoid that sort of thing. From joel.goldstick at gmail.com Fri Feb 12 20:43:45 2016 From: joel.goldstick at gmail.com (Joel Goldstick) Date: Fri, 12 Feb 2016 20:43:45 -0500 Subject: No subject In-Reply-To: <56be3d9f.c711c30a.e9bd9.fffff70f@mx.google.com> References: <56be3d9f.c711c30a.e9bd9.fffff70f@mx.google.com> Message-ID: On Fri, Feb 12, 2016 at 3:16 PM, Manas Soni wrote: > > I have downloaded python and when I click on it, it asks me to repair > which I do, it then says successful however when I click on it again it > won?t let me on it > Sent from Mail for Windows 10 > > -- > https://mail.python.org/mailman/listinfo/python-list > I don't do windows, so I can't give you an answer. To the group, this has come up so consistantly lately. Maybe a windows python user could explain why this is so confusing to newbies on windows? -- Joel Goldstick http://joelgoldstick.com/stats/birthdays From bob.martin at excite.com Sat Feb 13 03:58:02 2016 From: bob.martin at excite.com (Bob Martin) Date: Sat, 13 Feb 2016 08:58:02 GMT Subject: working References: <4a3a1a88-4776-4d05-b3ad-962c23d59399@googlegroups.com> <9428d1c1-70ee-4984-9ecc-ebbde0e579e4@googlegroups.com> Message-ID: in 753638 20160212 185728 sohcahtoa82 at gmail.com wrote: >On Friday, February 12, 2016 at 1:47:24 AM UTC-8, Mohammed Zakria wrote: >> hello >> i want to know the company that ican work as freelance python devloper > >There are some recruiters that read this mailing list and will send unsolicited e-mail about job ope >nings, but they might pass right over you if you're not willing to spend the time to proofread your >messages. The standard of his English might not be a problem but in a previous post he indicated that he is an absolute beginner. From irmen.NOSPAM at xs4all.nl Sat Feb 13 06:37:52 2016 From: irmen.NOSPAM at xs4all.nl (Irmen de Jong) Date: Sat, 13 Feb 2016 12:37:52 +0100 Subject: confusing installation on windows? was: Re: In-Reply-To: References: <56be3d9f.c711c30a.e9bd9.fffff70f@mx.google.com> Message-ID: <56bf158d$0$23725$e4fe514c@news.xs4all.nl> On 13-2-2016 2:43, Joel Goldstick wrote: > On Fri, Feb 12, 2016 at 3:16 PM, Manas Soni wrote: > >> >> I have downloaded python and when I click on it, it asks me to repair >> which I do, it then says successful however when I click on it again it >> won?t let me on it >> Sent from Mail for Windows 10 >> >> -- >> https://mail.python.org/mailman/listinfo/python-list >> > > I don't do windows, so I can't give you an answer. To the group, this has > come up so consistantly lately. Maybe a windows python user could explain > why this is so confusing to newbies on windows? > I think it could be because of the confusing start menu in recent windows versions. Perhaps they keep clicking the tile that shows the installer (as a 'recent document' or 'download'), rather than the new tile that would show the actual Python/idle/documentation icon. I can't check right now because here I have windows 7 (with a sane start menu) but will have a look at how it's presented when freshly installing a new python on my machine at office next week (which has windows 8...) Irmen From miragewebstudio12 at gmail.com Sat Feb 13 08:35:04 2016 From: miragewebstudio12 at gmail.com (MWS) Date: Sat, 13 Feb 2016 19:05:04 +0530 Subject: Python Not Working as expected on Win 8 and above. In-Reply-To: References: <56be3d9f.c711c30a.e9bd9.fffff70f@mx.google.com> Message-ID: <56BF3108.5050903@gmail.com> On Saturday 13 February 2016 07:13 AM, Joel Goldstick wrote: > On Fri, Feb 12, 2016 at 3:16 PM, Manas Soni wrote: > >> I have downloaded python and when I click on it, it asks me to repair >> which I do, it then says successful however when I click on it again it >> won?t let me on it >> Sent from Mail for Windows 10 >> >> -- >> https://mail.python.org/mailman/listinfo/python-list >> > I don't do windows, so I can't give you an answer. To the group, this has > come up so consistantly lately. Maybe a windows python user could explain > why this is so confusing to newbies on windows? > Hello, Just to add to the above discussion, i find that when my workplace updated from win 7 to win 8.1 with fresh install, i downloaded the official python 3.5 and installed it. Everything went well during installation, but, i couldn't find the default install python directory (maybe i didn't pay attention earlier), later, after some scratching my head and other intelligent thoughts and experiments i found it got installed in the users hidden appdata folder by default(confusing as i expected it to either get installed in a root folder or program files and also wondering why). I used pycharm for working on previous win version, so after installing the same again i selected the python.exe file as default interpreter. The big headache i found is that previously i could easily install packages from the pycharm interface, but currently the that thing is not working nor could i use it to run any of my previous apps(scipts) i created on win 7 (maybe not a python problem and i may find a workaround). Finally as i had to get my work done, i used a (trusted) portable version of python3.4 with preinstalled modules and that got me working. Wonder why is that so hard to work on win with python. I work on linux at home and i never faced such a problem on it because it is natively supported on it. Wonder when will microsoft understand that some programs need to be natively supported even if they are not made by them. I love windows for what they are but i hold linux even more dearer for what it is. George PS : The system has upgraded to Win 10 but not tested the python installation on it. From irmen.NOSPAM at xs4all.nl Sat Feb 13 10:29:52 2016 From: irmen.NOSPAM at xs4all.nl (Irmen de Jong) Date: Sat, 13 Feb 2016 16:29:52 +0100 Subject: made a simple rhythm sample mixer for fun (think Roland TR-909) Message-ID: <56bf4bee$0$23843$e4fe514c@news.xs4all.nl> Hi, For fun and to learn a bit from messing around a little with audio samples, I've created a rhythm sample mixer inspired by the Roland TR-909 drum machine. Basically you write rhythm sample patterns and it then sequences and mixes them into a output wav file or streams it to your speakers. Songs are built up from pattern sequences, where a pattern is built up from trigger bars for the used instruments. The tool also has a simple command line interface to edit patterns and play samples etc. As an example here is a fragment of one of the example songs; [song] patterns = pat1 pat2 pat3 pat2b pat3b outro [pattern.pat2] hihat2 = x...x... x...x... kick7 = x....... ........ kick9 = ........ x....... snare2 = ........ x....... You can find it here (also some examples): https://github.com/irmen/rhythmbox Python 3.x is required, also needs pyaudio if you want streaming. Other than that, I've found that the standard age-old audioop module may be limited, but is enough to build some basic sample mixing stuff with. I'm interested in what rhythm loops you can come up with! have fun, Irmen From oscar.j.benjamin at gmail.com Sat Feb 13 11:48:49 2016 From: oscar.j.benjamin at gmail.com (Oscar Benjamin) Date: Sat, 13 Feb 2016 16:48:49 +0000 Subject: (unknown) In-Reply-To: References: <56be3d9f.c711c30a.e9bd9.fffff70f@mx.google.com> Message-ID: On 12 February 2016 at 21:39, Mark Lawrence wrote: > On 12/02/2016 20:16, Manas Soni wrote: >> >> >> I have downloaded python and when I click on it, it asks me to repair >> which I do, it then says successful however when I click on it again it >> won?t let me on it >> Sent from Mail for Windows 10 >> > > Please search the archives as this has been asked and answered repeatedly > over the last few months. I've seen the "repair" question asked many times but I don't actually recall seeing a solution to this particular problem. What causes it and how do you solve it? Or can someone point to a thread about this that actually contains the solution rather than the many that look like this one did before I added this post? Does the error happen when running the installer or when trying to launch IDLE or something after install? There are a lot of questions like this about installing Python 3.5 on Windows and AFAICT they come under the following categories (have I missed anything?): 1) XP users - I think the 3.5.1 installer gives a proper error message for this now. 2) ms-api-xxxxx.dll - update C runtime from MS - the installer should probably give a proper error message for this too 3) error code 0x8023482348 (or some hex string) - some Windows problem with temporary files. 4) New user can't work out how to "run python" after installing. Should be able to find it in the programs menu I guess but perhaps it's not easy to find on newer Windows >= 8. 5) modify/repair/install - like this one. I don't know what the solution is. Perhaps there should be a wiki page listing the common problems and solutions so that people can be pointed to that instead of being told to search the archives. I'm not sure how you'd go about creating one of those and realistically it should be novice-friendly so it really needs screenshots of how it looks under different Windows versions and takes people right through to the point of say running a command in IDLE. Not being a Windows user it's not easy for me to provide the necessary level of detail for that. Also as concerns problem 2) presumably the installer could detect this and point users to the solution as it does for XP. -- Oscar From eryksun at gmail.com Sat Feb 13 12:16:28 2016 From: eryksun at gmail.com (eryk sun) Date: Sat, 13 Feb 2016 11:16:28 -0600 Subject: Python Not Working as expected on Win 8 and above. In-Reply-To: <56BF3108.5050903@gmail.com> References: <56be3d9f.c711c30a.e9bd9.fffff70f@mx.google.com> <56BF3108.5050903@gmail.com> Message-ID: On Sat, Feb 13, 2016 at 7:35 AM, MWS wrote: > couldn't find the default install python directory (maybe i didn't pay > attention earlier), later, after some scratching my head and other > intelligent thoughts and experiments i found it got installed in the users > hidden appdata folder by default(confusing as i expected it to either get > installed in a root folder or program files and also wondering why). The new installer uses the standard locations for per-user and per-machine installations, but you can change it to use whatever you want. The following are the default installation locations (using "shell:" links). per-user install shell:UserProgramFiles\Python\PythonXY[-32] per-machine install shell:ProgramFiles[x86]\PythonXY[-32] UserProgramFiles defaults to "shell:Local AppData\Programs". You can relocate it, but I'd only do that for a new account. From christopher_reimer at icloud.com Sat Feb 13 12:30:55 2016 From: christopher_reimer at icloud.com (Christopher Reimer) Date: Sat, 13 Feb 2016 09:30:55 -0800 Subject: Python Not Working as expected on Win 8 and above. In-Reply-To: <56BF3108.5050903@gmail.com> References: <56be3d9f.c711c30a.e9bd9.fffff70f@mx.google.com> <56BF3108.5050903@gmail.com> Message-ID: <56BF684F.2050507@icloud.com> On 2/13/2016 5:35 AM, MWS wrote: > Just to add to the above discussion, i find that when my workplace > updated from win 7 to win 8.1 with fresh install, i downloaded the > official python 3.5 and installed it. Everything went well during > installation, but, i couldn't find the default install python directory > (maybe i didn't pay attention earlier), later, after some scratching my > head and other intelligent thoughts and experiments i found it got > installed in the users hidden appdata folder by default(confusing as i > expected it to either get installed in a root folder or program files > and also wondering why). Based on my experience with installing Python 3.5.0, I ran into the same problem by clicking on the "Download Python 3.5.0" link on the Python download page and used the python-3.5.0.exe installer. Since nothing worked as I expected, I uninstalled that installer version of Python. Digging through the specific release section, I downloaded the Windows x86-64 executable installer (python-3.5.0-amd64.exe). That installer behaved exactly as I expected. In fact, this is the installer I've always used for previous versions. Besides the focus of the content files, the python-3.5.0.exe and python-3.5.0-amd64.exe installers don't behave exactly the same. Thanks, Chris R. From werotizy at freent.dd Sat Feb 13 12:58:06 2016 From: werotizy at freent.dd (Tom P) Date: Sat, 13 Feb 2016 18:58:06 +0100 Subject: problem with dateutil Message-ID: I am writing a program that has to deal with various date/time formats and convert these into timestamps. It looks as if dateutil.parser.parse should be able to handle about any format, but what I get is: datetimestr = '2012-10-22 11:22:33' print(dateutil.parser.parse(datetimestr)) result: datetime.datetime(2012, 10, 22, 11, 22, 33) However: datetimestr = '2012:10:22 11:22:33' print(dateutil.parser.parse(datetimestr)) result: datetime.datetime(2016, 2, 13, 11, 22, 33) In other words, it's getting the date wrong when colons are used to separate YYYY:MM:DD. Is there a way to include this as a valid format? From gherron at digipen.edu Sat Feb 13 13:13:06 2016 From: gherron at digipen.edu (Gary Herron) Date: Sat, 13 Feb 2016 10:13:06 -0800 Subject: problem with dateutil In-Reply-To: References: Message-ID: <56BF7232.8030808@digipen.edu> On 02/13/2016 09:58 AM, Tom P wrote: > I am writing a program that has to deal with various date/time formats > and convert these into timestamps. It looks as if > dateutil.parser.parse should be able to handle about any format, but > what I get is: > > datetimestr = '2012-10-22 11:22:33' > print(dateutil.parser.parse(datetimestr)) > result: datetime.datetime(2012, 10, 22, 11, 22, 33) > > However: > datetimestr = '2012:10:22 11:22:33' > print(dateutil.parser.parse(datetimestr)) > result: datetime.datetime(2016, 2, 13, 11, 22, 33) > > In other words, it's getting the date wrong when colons are used to > separate YYYY:MM:DD. Is there a way to include this as a valid format? > Yes, there is a way to specify your own format. Search the datetime documentation for datetime.strptime(date_string, format) Gary Herron -- Dr. Gary Herron Department of Computer Science DigiPen Institute of Technology (425) 895-4418 From p_s_d_a_s_i_l_v_a_ns at netcabo.pt Sat Feb 13 14:29:35 2016 From: p_s_d_a_s_i_l_v_a_ns at netcabo.pt (Paulo da Silva) Date: Sat, 13 Feb 2016 19:29:35 +0000 Subject: What is heating the memory here? hashlib? Message-ID: Hello all. I'm running in a very strange (for me at least) problem. def getHash(self): bfsz=File.blksz h=hashlib.sha256() hu=h.update with open(self.getPath(),'rb') as f: f.seek(File.hdrsz) # Skip header b=f.read(bfsz) while len(b)>0: hu(b) b=f.read(bfsz) fhash=h.digest() return fhash hdrsz is always 4K here. All files are greater than 4K. If I use a 40MB bfsz this tooks all my memory very quickly. After few hundreds of files it begins to swap ending up with the program being killed (BTW, I'm using linux kubuntu 14.04). If I reduce bfsz to 1MB it successfully completes my full test (~100000 files) reaching about 6GB of memory. If I reduce further bfsz to 16KB there is no noticeable memory taken!! I have tried the following code, but it didn't fix the problem: def getHash(self): bfsz=File.blksz h=hashlib.sha256() hu=h.update with open(self.getPath(),'rb') as f: husz=8192 f.seek(File.hdrsz) # Skip header b=f.read(bfsz) while len(b)>0: for i in range(0,len(b),husz): hu(b[i:i+husz]) b=f.read(bfsz) fhash=h.digest() return fhash What is wrong here?! Thanks for any help/comments. Paulo From mkondrashin at gmail.com Sat Feb 13 15:19:52 2016 From: mkondrashin at gmail.com (mkondrashin at gmail.com) Date: Sat, 13 Feb 2016 12:19:52 -0800 (PST) Subject: Storing a big amount of path names In-Reply-To: References: <56BD4DCD.4080401@mrabarnett.plus.com> Message-ID: <243825dd-451a-405e-ad41-855622e80b06@googlegroups.com> In my application I have used two approaches: 1. To store paths as a tree (as directories for a tree. 2. For long list of similar paths, to store difference of strings. Though this was c++/obj-c project, I can share a diff code with you if you drip me a line (mkondrashin & gmail , com) From werotizy at freent.dd Sat Feb 13 15:27:20 2016 From: werotizy at freent.dd (Tom P) Date: Sat, 13 Feb 2016 21:27:20 +0100 Subject: problem with dateutil In-Reply-To: References: Message-ID: On 02/13/2016 07:13 PM, Gary Herron wrote: > On 02/13/2016 09:58 AM, Tom P wrote: >> I am writing a program that has to deal with various date/time formats >> and convert these into timestamps. It looks as if >> dateutil.parser.parse should be able to handle about any format, but >> what I get is: >> >> datetimestr = '2012-10-22 11:22:33' >> print(dateutil.parser.parse(datetimestr)) >> result: datetime.datetime(2012, 10, 22, 11, 22, 33) >> >> However: >> datetimestr = '2012:10:22 11:22:33' >> print(dateutil.parser.parse(datetimestr)) >> result: datetime.datetime(2016, 2, 13, 11, 22, 33) >> >> In other words, it's getting the date wrong when colons are used to >> separate YYYY:MM:DD. Is there a way to include this as a valid format? >> > > Yes, there is a way to specify your own format. Search the datetime > documentation for > datetime.strptime(date_string, format) > > Gary Herron > Thanks. I started out with datetime.strptime but AFAICS that means I have to go through try/except for every conceivable format. Are you saying that I can't use dateutil.parser? From gherron at digipen.edu Sat Feb 13 15:45:39 2016 From: gherron at digipen.edu (Gary Herron) Date: Sat, 13 Feb 2016 12:45:39 -0800 Subject: problem with dateutil In-Reply-To: References: Message-ID: <56BF95F3.1000906@digipen.edu> On 02/13/2016 12:27 PM, Tom P wrote: > On 02/13/2016 07:13 PM, Gary Herron wrote: >> On 02/13/2016 09:58 AM, Tom P wrote: >>> I am writing a program that has to deal with various date/time formats >>> and convert these into timestamps. It looks as if >>> dateutil.parser.parse should be able to handle about any format, but >>> what I get is: >>> >>> datetimestr = '2012-10-22 11:22:33' >>> print(dateutil.parser.parse(datetimestr)) >>> result: datetime.datetime(2012, 10, 22, 11, 22, 33) >>> >>> However: >>> datetimestr = '2012:10:22 11:22:33' >>> print(dateutil.parser.parse(datetimestr)) >>> result: datetime.datetime(2016, 2, 13, 11, 22, 33) >>> >>> In other words, it's getting the date wrong when colons are used to >>> separate YYYY:MM:DD. Is there a way to include this as a valid format? >>> >> >> Yes, there is a way to specify your own format. Search the datetime >> documentation for >> datetime.strptime(date_string, format) >> >> Gary Herron >> > > Thanks. I started out with datetime.strptime but AFAICS that means I > have to go through try/except for every conceivable format. Are you > saying that I can't use dateutil.parser? Well now... If by "every conceivable format" you are including formats that the author of dateutil.parser did not conceive of, then of course you cannot use dateutil.parser. But you have the code for dateutil.parser -- perhaps you could modify it to accept whatever odd formats you care about. Gary Herron -- Dr. Gary Herron Department of Computer Science DigiPen Institute of Technology (425) 895-4418 From breamoreboy at yahoo.co.uk Sat Feb 13 16:01:01 2016 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Sat, 13 Feb 2016 21:01:01 +0000 Subject: problem with dateutil In-Reply-To: References: Message-ID: On 13/02/2016 17:58, Tom P wrote: > I am writing a program that has to deal with various date/time formats > and convert these into timestamps. It looks as if dateutil.parser.parse > should be able to handle about any format, but what I get is: > > datetimestr = '2012-10-22 11:22:33' > print(dateutil.parser.parse(datetimestr)) > result: datetime.datetime(2012, 10, 22, 11, 22, 33) > > However: > datetimestr = '2012:10:22 11:22:33' > print(dateutil.parser.parse(datetimestr)) > result: datetime.datetime(2016, 2, 13, 11, 22, 33) > > In other words, it's getting the date wrong when colons are used to > separate YYYY:MM:DD. Is there a way to include this as a valid format? > From http://labix.org/python-dateutil#head-a23e8ae0a661d77b89dfb3476f85b26f0b30349c parserinfo This parameter allows one to change how the string is parsed, by using a different parserinfo class instance. Using it you may, for example, intenationalize the parser strings, or make it ignore additional words. HTH. -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From p_s_d_a_s_i_l_v_a_ns at netcabo.pt Sat Feb 13 17:26:53 2016 From: p_s_d_a_s_i_l_v_a_ns at netcabo.pt (Paulo da Silva) Date: Sat, 13 Feb 2016 22:26:53 +0000 Subject: What is heating the memory here? hashlib? References: Message-ID: I meant eating! :-) From rosuav at gmail.com Sat Feb 13 17:45:00 2016 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 14 Feb 2016 09:45:00 +1100 Subject: What is heating the memory here? hashlib? In-Reply-To: References: Message-ID: On Sun, Feb 14, 2016 at 9:26 AM, Paulo da Silva wrote: > I meant eating! :-) Heh, "heating" works too - the more you use memory, the more it heats up :) I'm assuming this is inside "class File:" and you have class members for your constants like header size? There's no context for the name "File" otherwise. What happens if, after hashing each file (and returning from this function), you call gc.collect()? If that reduces your RAM usage, you have reference cycles somewhere. ChrisA From h.hodges at gmx.com Sat Feb 13 18:15:34 2016 From: h.hodges at gmx.com (Heaven Hodges) Date: Sat, 13 Feb 2016 15:15:34 -0800 Subject: Changing A Subscription Option Message-ID: <56BFB916.8010301@gmx.com> Hello. Is there a way to change my subscription option from digest to individual e-mails? I couldn't find any information on the list web page: https://mail.python.org/mailman/listinfo/python-list. I've tried filling out the subscription form again, but the result was an automated e-mail about someone trying to alter my subscription. I've also sent an e-mail to python-list-request at python.org, but I haven't received a response. -- Thanks, Heaven Hodges From mail at timgolden.me.uk Sat Feb 13 18:44:54 2016 From: mail at timgolden.me.uk (Tim Golden) Date: Sat, 13 Feb 2016 23:44:54 +0000 Subject: Changing A Subscription Option In-Reply-To: <56BFB916.8010301@gmx.com> References: <56BFB916.8010301@gmx.com> Message-ID: <56BFBFF6.4060908@timgolden.me.uk> On 13/02/2016 23:15, Heaven Hodges wrote: > Hello. Is there a way to change my subscription option from digest to > individual e-mails? I couldn't find any information on the list web > page: https://mail.python.org/mailman/listinfo/python-list. At the bottom of that page is a button to "Unsubscribe or edit options". That's the one you want. TJG From p_s_d_a_s_i_l_v_a_ns at netcabo.pt Sat Feb 13 20:44:55 2016 From: p_s_d_a_s_i_l_v_a_ns at netcabo.pt (Paulo da Silva) Date: Sun, 14 Feb 2016 01:44:55 +0000 Subject: What is heating the memory here? hashlib? References: Message-ID: ?s 22:45 de 13-02-2016, Chris Angelico escreveu: > On Sun, Feb 14, 2016 at 9:26 AM, Paulo da Silva > wrote: >> I meant eating! :-) > > Heh, "heating" works too - the more you use memory, the more it heats up :) :-) It is heating my head! ... > > What happens if, after hashing each file (and returning from this > function), you call gc.collect()? If that reduces your RAM usage, you > have reference cycles somewhere. > I have used gc and del. No luck. The most probable cause seems to be hashlib not correctly handling big buffers updates. I am working in a computer and testing in another. For the second part may be somehow I forgot to transfer the change to the other computer. Unlikely but possible. Anyway it is doing its job right now with bfsz=16KB (this takes a few hours). No memory leakages anymore. I'll address this problem in a near future, may be when I move to the new kubuntu 16.04 LTS. This will bring new SW releases and the problem may have been fixed. Thanks Paulo From rosuav at gmail.com Sat Feb 13 21:01:52 2016 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 14 Feb 2016 13:01:52 +1100 Subject: What is heating the memory here? hashlib? In-Reply-To: References: Message-ID: On Sun, Feb 14, 2016 at 12:44 PM, Paulo da Silva wrote: >> What happens if, after hashing each file (and returning from this >> function), you call gc.collect()? If that reduces your RAM usage, you >> have reference cycles somewhere. >> > I have used gc and del. No luck. > > The most probable cause seems to be hashlib not correctly handling big > buffers updates. I am working in a computer and testing in another. For > the second part may be somehow I forgot to transfer the change to the > other computer. Unlikely but possible. I'd like to see the problem boiled down to just the hashlib calls. Something like this: import hashlib data = b"*" * 4*1024*1024 lastdig = None while "simulating files": h = hashlib.sha256() hu = h.update for chunk in range(100): hu(data) dig = h.hexdigest() if lastdig is None: lastdig = dig print("Digest:",dig) else: if lastdig != dig: print("Digest fail!") Running this on my system (Python 3.6 on Debian Linux) produces a long-running process with stable memory usage, which is exactly what I'd expect. Even using different data doesn't change that: import hashlib import itertools byte = itertools.count() data = b"*" * 4*1024*1024 while "simulating files": h = hashlib.sha256() hu = h.update for chunk in range(100): hu(data + bytes([next(byte)&255])) dig = h.hexdigest() print("Digest:",dig) Somewhere between my code and yours is something that consumes all that memory. Can you neuter the actual disk reading (replacing it with constants, like this) and make a complete and shareable program that leaks all that memory? ChrisA From steve at pearwood.info Sat Feb 13 21:21:17 2016 From: steve at pearwood.info (Steven D'Aprano) Date: Sun, 14 Feb 2016 13:21:17 +1100 Subject: What is heating the memory here? hashlib? References: Message-ID: <56bfe49e$0$1587$c3e8da3$5496439d@news.astraweb.com> On Sun, 14 Feb 2016 06:29 am, Paulo da Silva wrote: > Hello all. > > I'm running in a very strange (for me at least) problem. > > def getHash(self): > bfsz=File.blksz > h=hashlib.sha256() > hu=h.update > with open(self.getPath(),'rb') as f: > f.seek(File.hdrsz) # Skip header > b=f.read(bfsz) > while len(b)>0: > hu(b) > b=f.read(bfsz) > fhash=h.digest() > return fhash This is a good, and tricky, question! Unfortunately, this sort of performance issue may depend on the specific details of your system. You can start by telling us what version of Python you are running. You've already said you're running on Kubuntu, which makes it Linux. Is that a 32-bit or 64-bit version? Next, let's see if we can simplify the code and make it runnable by anyone, in the spirit of http://www.sscce.org/ import hashlib K = 1024 M = 1024*K def get_hash(pathname, size): h = hashlib.sha256() with open(pathname, 'rb') as f: f.seek(4*K) b = f.read(size) while b: h.update(b) b = f.read(size) return h.digest() Does this simplified version demonstrate the same problem? What happens if you eliminate the actual hashing? def get_hash(pathname, size): with open(pathname, 'rb') as f: f.seek(4*K) b = f.read(size) while b: b = f.read(size) return "1234"*16 This may allow you to determine whether the problem lies in *reading* the files or *hashing* the files. Be warned: if you read from the same file over and over again, Linux will cache that file, and your tests will not reflect the behaviour when you read thousands of different files from disk rather than from memory cache. What sort of media are you reading from? - hard drive? - flash drive or USB stick? - solid state disk? - something else? They will all have different read characteristics. What happens when you call f.read(size)? By default, Python uses the following buffering strategy for binary files: * Binary files are buffered in fixed-size chunks; the size of the buffer is chosen using a heuristic trying to determine the underlying device's "block size" and falling back on `io.DEFAULT_BUFFER_SIZE`. On many systems, the buffer will typically be 4096 or 8192 bytes long. See help(open). That's your first clue that, perhaps, you should be reading in relatively small blocks, more like 4K than 4MB. Sure enough, a quick bit of googling shows that typically you should read from files in small-ish chunks, and that trying to read in large chunks is often counter-productive: https://duckduckgo.com/html/?q=file+read+buffer+size The first three links all talk about optimal sizes being measured in small multiples of 4K, not 40MB. You can try to increase the system buffer, by changing the "open" line to: with open(pathname, 'rb', buffering=40*M) as f: and see whether that helps. By the way, do you need a cryptographic checksum? sha256 is expensive to calculate. If all you are doing is trying to match files which could have the same content, you could use a cheaper hash, like md5 or even crc32. -- Steven From vek.m1234 at gmail.com Sat Feb 13 23:40:58 2016 From: vek.m1234 at gmail.com (Veek. M) Date: Sun, 14 Feb 2016 10:10:58 +0530 Subject: How do i instantiate a class_name passed via cmd line Message-ID: I'm writing a price parser. I need to do the equivalent of perl's $$var to instantiate a class where $car is the class_name. I'm passing 'Ebay' or 'Newegg' or 'Amazon' via cmd-line. I have a module named ebay.py and a class called Ebay (price parser). I do something like: \> main.py ebay motherboard and this does: module = __import__(module_name) but now i need to instantiate the class - right now I do: instance = module.Ebay(module_name, product) how do i replace the 'Ebay' bit with a variable so that I can load any class via cmd line. class Load(object): def __init__(self, module_name, product): try: module = __import__(module_name) instance = module.Ebay(module_name, product) except ImportError: print("Can't find module %s" % module_name) From rantingrickjohnson at gmail.com Sun Feb 14 00:17:03 2016 From: rantingrickjohnson at gmail.com (Rick Johnson) Date: Sat, 13 Feb 2016 21:17:03 -0800 (PST) Subject: I cannot open IDLE In-Reply-To: References: Message-ID: On Friday, February 12, 2016 at 7:06:45 AM UTC-6, anthony averett wrote: > I have downloaded python but I cannot open up idle. I really need this > issue resolved for I have work do for one of my computer science class. Sorry, but you failed to provide enough information. Here are a few possibilities: (1) Are you unaware of how to run the program, or need guidance as to it's location in the file hierarchy? or (2) Did you attempt to run the program, only to experience a crash or some sort of unexpected behavior? or (3) something else entirely... Please be more specific, and provide to us all the relevant specifications regarding your OS and Python versions. Thanks From cs at zip.com.au Sun Feb 14 00:20:06 2016 From: cs at zip.com.au (Cameron Simpson) Date: Sun, 14 Feb 2016 16:20:06 +1100 Subject: How do i instantiate a class_name passed via cmd line In-Reply-To: References: Message-ID: <20160214052006.GA28616@cskk.homeip.net> On 14Feb2016 10:10, Veek. M wrote: >I'm writing a price parser. I need to do the equivalent of perl's >$$var to instantiate a class where $car is the class_name. > >I'm passing 'Ebay' or 'Newegg' or 'Amazon' via cmd-line. I have a module >named ebay.py and a class called Ebay (price parser). I do something >like: > >\> main.py ebay motherboard > >and this does: > module = __import__(module_name) >but now i need to instantiate the class - right now I do: > instance = module.Ebay(module_name, product) > >how do i replace the 'Ebay' bit with a variable so that I can load any >class via cmd line. > >class Load(object): > def __init__(self, module_name, product): > try: > module = __import__(module_name) > instance = module.Ebay(module_name, product) > except ImportError: > print("Can't find module %s" % module_name) Ebay = geattr(module, 'Ebay') or klass = getattr(module, module_name.capitalize()) instance = klass(module_name, product) Cheers, Cameron Simpson From rantingrickjohnson at gmail.com Sun Feb 14 00:31:30 2016 From: rantingrickjohnson at gmail.com (Rick Johnson) Date: Sat, 13 Feb 2016 21:31:30 -0800 (PST) Subject: How do i instantiate a class_name passed via cmd line In-Reply-To: References: Message-ID: <7142528d-6465-4825-af4f-85d6c7640b16@googlegroups.com> On Saturday, February 13, 2016 at 10:41:20 PM UTC-6, Veek. M wrote: > how do i replace the 'Ebay' bit with a variable so that I > can load any class via cmd line. Is this what you're trying to do? (Python2.x code) >>> import Tkinter as tk >>> classNames = ["Button", "Label"] >>> root = tk.Tk() >>> for className in classNames: classN = getattr(tk, className) instanceN = classN(root, text=className) instanceN.pack() Note: You won't need to call "mainloop" when testing this code on the command line, only in a script or in the IDLE shell. >>> root.mainloop() From vek.m1234 at gmail.com Sun Feb 14 00:39:40 2016 From: vek.m1234 at gmail.com (Veek. M) Date: Sun, 14 Feb 2016 11:09:40 +0530 Subject: How do i instantiate a class_name passed via cmd line References: <7142528d-6465-4825-af4f-85d6c7640b16@googlegroups.com> Message-ID: Rick Johnson wrote: > On Saturday, February 13, 2016 at 10:41:20 PM UTC-6, Veek. M wrote: >> how do i replace the 'Ebay' bit with a variable so that I >> can load any class via cmd line. > > Is this what you're trying to do? > > (Python2.x code) >>>> import Tkinter as tk >>>> classNames = ["Button", "Label"] >>>> root = tk.Tk() >>>> for className in classNames: > classN = getattr(tk, className) > instanceN = classN(root, text=className) > instanceN.pack() > > Note: You won't need to call "mainloop" when testing this > code on the command line, only in a script or in the IDLE > shell. > >>>> root.mainloop() Nope - this is what i'm doing: class Foo(): pass x = 'Foo' How do i use 'x' to create an instance of class Foo? From greg.ewing at canterbury.ac.nz Sun Feb 14 00:41:54 2016 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Sun, 14 Feb 2016 18:41:54 +1300 Subject: How do i instantiate a class_name passed via cmd line In-Reply-To: References: Message-ID: Veek. M wrote: > I'm writing a price parser. I need to do the equivalent of perl's > $$var to instantiate a class where $car is the class_name. > > I'm passing 'Ebay' or 'Newegg' or 'Amazon' via cmd-line. I have a module > named ebay.py and a class called Ebay (price parser). I do something > like: > > \> main.py ebay motherboard > > and this does: > module = __import__(module_name) > but now i need to instantiate the class - right now I do: > instance = module.Ebay(module_name, product) > > how do i replace the 'Ebay' bit with a variable so that I can load any > class via cmd line. > > class Load(object): > def __init__(self, module_name, product): > try: > module = __import__(module_name) > instance = module.Ebay(module_name, product) > except ImportError: > print("Can't find module %s" % module_name) > Something like this should do it: instance = getattr(module, class_name)(module_name, product) If the class name is always the same as the module name with the first letter capitalized, you could use instance = getattr(module, module_name.capitalize())(module_name, product) -- Greg From vek.m1234 at gmail.com Sun Feb 14 00:51:08 2016 From: vek.m1234 at gmail.com (Veek. M) Date: Sun, 14 Feb 2016 11:21:08 +0530 Subject: How do i instantiate a class_name passed via cmd line References: Message-ID: Gregory Ewing wrote: > Veek. M wrote: >> I'm writing a price parser. I need to do the equivalent of perl's >> $$var to instantiate a class where $car is the class_name. >> >> I'm passing 'Ebay' or 'Newegg' or 'Amazon' via cmd-line. I have a >> module named ebay.py and a class called Ebay (price parser). I do >> something like: >> >> \> main.py ebay motherboard >> >> and this does: >> module = __import__(module_name) >> but now i need to instantiate the class - right now I do: >> instance = module.Ebay(module_name, product) >> >> how do i replace the 'Ebay' bit with a variable so that I can load >> any class via cmd line. >> >> class Load(object): >> def __init__(self, module_name, product): >> try: >> module = __import__(module_name) >> instance = module.Ebay(module_name, product) >> except ImportError: >> print("Can't find module %s" % module_name) >> > > Something like this should do it: > > instance = getattr(module, class_name)(module_name, product) > > If the class name is always the same as the module name with the > first letter capitalized, you could use > > instance = getattr(module, module_name.capitalize())(module_name, > product) > Ah! i see - clever! 'getattr' returns the class object with class_name=whatever and we can instantiate now that we have a class object - nice - thanks guys - the bell should have rung from Rick's example. From rantingrickjohnson at gmail.com Sun Feb 14 00:55:14 2016 From: rantingrickjohnson at gmail.com (Rick Johnson) Date: Sat, 13 Feb 2016 21:55:14 -0800 (PST) Subject: How do i instantiate a class_name passed via cmd line In-Reply-To: References: <7142528d-6465-4825-af4f-85d6c7640b16@googlegroups.com> Message-ID: On Saturday, February 13, 2016 at 11:39:56 PM UTC-6, Veek. M wrote: > Nope - this is what i'm doing: > > class Foo(): > pass > > x = 'Foo' > > How do i use 'x' to create an instance of class Foo? Use the builtin function `getattr` on the module that contains the class named "Foo". For example: module.Foo is equivelant to: getattr(module, "Foo") Here is an interactive session: py> import Tkinter py> Tkinter.Label py> getattr(Tkinter, "Label") From p_s_d_a_s_i_l_v_a_ns at netcabo.pt Sun Feb 14 02:04:36 2016 From: p_s_d_a_s_i_l_v_a_ns at netcabo.pt (Paulo da Silva) Date: Sun, 14 Feb 2016 07:04:36 +0000 Subject: What is heating the memory here? hashlib? References: Message-ID: I was unable to reproduce the situation using a simple program just walking through all files>4K, with or without the seek, and computing their shasums. Only some fluctuations of about 500MB in memory consumption. I'll look at this when I get more time, taking in consideration the suggestions here posted. For the time being, my work is done. With a small buffer size (16k) the results produced were correct and no memory was leaked! If I can find any explanation (if not embarrassing :-) ), I'll post it here. Thank you all. Paulo From songofacandy at gmail.com Sun Feb 14 04:49:51 2016 From: songofacandy at gmail.com (INADA Naoki) Date: Sun, 14 Feb 2016 18:49:51 +0900 Subject: What is heating the memory here? hashlib? In-Reply-To: References: Message-ID: tracemalloc module may help you to investigate leaks. 2016/02/14 ??4:05 "Paulo da Silva" : > I was unable to reproduce the situation using a simple program just > walking through all files>4K, with or without the seek, and computing > their shasums. > Only some fluctuations of about 500MB in memory consumption. > > I'll look at this when I get more time, taking in consideration the > suggestions here posted. > > For the time being, my work is done. With a small buffer size (16k) the > results produced were correct and no memory was leaked! > > If I can find any explanation (if not embarrassing :-) ), I'll post it > here. > > Thank you all. > Paulo > > -- > https://mail.python.org/mailman/listinfo/python-list > From antoon.pardon at rece.vub.ac.be Sun Feb 14 05:54:39 2016 From: antoon.pardon at rece.vub.ac.be (Antoon Pardon) Date: Sun, 14 Feb 2016 11:54:39 +0100 Subject: ftplib throws: IndexError: tuple index out of range Message-ID: <56C05CEF.5080304@rece.vub.ac.be> I have written a small backup program, that uses ftplib to make remote backups. However recentely the program starts to regularly raise IndexErrors, as far as I can see the problem is in socket.py Can anyone shed some light? This is the traceback: Traceback (most recent call last): File "/usr/local/lib/python-apps/rmtdump/pymain.py", line 93, in main Exit_Nr = process(sys.argv) File "/usr/local/lib/python-apps/rmtdump/rmtdump.py", line 228, in program process(path) File "/usr/local/lib/python-apps/rmtdump/rmtdump.py", line 170, in process of = rmt.open(rmtsep.join([rmt5lcl(rt), archive]), "wb") File "/usr/local/lib/python-apps/rmtdump/ftputil.py", line 198, in open return ftpfile(ftp, fn, mode, True) File "/usr/local/lib/python-apps/rmtdump/ftputil.py", line 69, in __init__ self.cnct = self.ftp.transfercmd("%s %s" % (cmd, rfn)) File "/usr/lib/python2.7/ftplib.py", line 376, in transfercmd return self.ntransfercmd(cmd, rest)[0] File "/usr/lib/python2.7/ftplib.py", line 693, in ntransfercmd conn, size = FTP.ntransfercmd(self, cmd, rest) File "/usr/lib/python2.7/ftplib.py", line 339, in ntransfercmd resp = self.sendcmd(cmd) File "/usr/lib/python2.7/ftplib.py", line 249, in sendcmd return self.getresp() File "/usr/lib/python2.7/ftplib.py", line 215, in getresp resp = self.getmultiline() File "/usr/lib/python2.7/ftplib.py", line 201, in getmultiline line = self.getline() File "/usr/lib/python2.7/ftplib.py", line 186, in getline line = self.file.readline(self.maxline + 1) File "/usr/lib/python2.7/socket.py", line 478, in readline if e.args[0] == EINTR: IndexError: tuple index out of range Locals by frame, innermost last Frame main in /usr/local/lib/python-apps/rmtdump/pymain.py at line 111 backtrace = pn = 'rmtdump' process = Frame program in /usr/local/lib/python-apps/rmtdump/rmtdump.py at line 261 aborted = True argv = ['/usr/local/sbin/rmtdump'] path = '/home' Frame process in /usr/local/lib/python-apps/rmtdump/rmtdump.py at line 190 b_rt = '/home' backupflag = 'r--' backupstamp = 1453021366 checkstamp = 1450374304 dirs = ['antoon', 'jenny', 'lost+found', 'photo'] entries = ['0-home', 'antoon', 'homelady', 'jenny', 'lost+found', 'photo', 'recover'] entry = 'recover' fileinfo = posix.stat_result(st_mode=16877, st_ino=2490369, st_dev=2058L, st_nlink=5, st_uid=0, st_gid=0, st_size=4 ... files = [] fqpn = '/home/recover' islink = ismount = last = 1455445804.979914 newfn = 'home/!2016-01-17 at 10_02_46-tbz' now = 1455445804 nr_of_tries = 0 of = None oldfn = 'home/!ENTRY-tbz' path = '/home' rt = '/home' stamp = '1450374304' start = 1455445804 timefn = '!2016-01-17 at 10_02_46-tbz' tryal = 1 update = True Frame open in /usr/local/lib/python-apps/rmtdump/ftputil.py at line 198 fn = 'home/!ENTRY-tbz' ftp = mode = 'wb' self = Frame __init__ in /usr/local/lib/python-apps/rmtdump/ftputil.py at line 69 bound = True ch = 'b' cmd = 'STOR' cn = count = 1 mode = 'wb' rfn = 'home/!ENTRY-tbz' self = None Frame transfercmd in /usr/lib/python2.7/ftplib.py at line 376 cmd = 'STOR home/!ENTRY-tbz' rest = None self = Frame ntransfercmd in /usr/lib/python2.7/ftplib.py at line 693 cmd = 'STOR home/!ENTRY-tbz' rest = None self = Frame ntransfercmd in /usr/lib/python2.7/ftplib.py at line 352 cmd = 'STOR home/!ENTRY-tbz' conn = host = '192.110.0.54' port = 64649 rest = None self = size = None Frame sendcmd in /usr/lib/python2.7/ftplib.py at line 249 cmd = 'STOR home/!ENTRY-tbz' self = Frame getresp in /usr/lib/python2.7/ftplib.py at line 215 self = Frame getmultiline in /usr/lib/python2.7/ftplib.py at line 201 self = Frame getline in /usr/lib/python2.7/ftplib.py at line 186 self = Frame readline in /usr/lib/python2.7/socket.py at line 478 buf = buf_len = 0 e = timeout() self = size = 8193 From stephane at wirtel.be Sun Feb 14 06:00:16 2016 From: stephane at wirtel.be (Stephane Wirtel) Date: Sun, 14 Feb 2016 12:00:16 +0100 Subject: ftplib throws: IndexError: tuple index out of range In-Reply-To: <56C05CEF.5080304@rece.vub.ac.be> References: <56C05CEF.5080304@rece.vub.ac.be> Message-ID: <20160214110016.GA32251@sg1> Hi Antoon, EINTR, is an error when there is an emited signal to your process. http://man7.org/linux/man-pages/man2/read.2.html Look for EINTR in this page On 02/14, Antoon Pardon wrote: >I have written a small backup program, that uses ftplib to make >remote backups. However recentely the program starts to regularly >raise IndexErrors, as far as I can see the problem is in socket.py >Can anyone shed some light? > >This is the traceback: > >Traceback (most recent call last): > File "/usr/local/lib/python-apps/rmtdump/pymain.py", line 93, in main > Exit_Nr = process(sys.argv) > File "/usr/local/lib/python-apps/rmtdump/rmtdump.py", line 228, in program > process(path) > File "/usr/local/lib/python-apps/rmtdump/rmtdump.py", line 170, in process > of = rmt.open(rmtsep.join([rmt5lcl(rt), archive]), "wb") > File "/usr/local/lib/python-apps/rmtdump/ftputil.py", line 198, in open > return ftpfile(ftp, fn, mode, True) > File "/usr/local/lib/python-apps/rmtdump/ftputil.py", line 69, in __init__ > self.cnct = self.ftp.transfercmd("%s %s" % (cmd, rfn)) > File "/usr/lib/python2.7/ftplib.py", line 376, in transfercmd > return self.ntransfercmd(cmd, rest)[0] > File "/usr/lib/python2.7/ftplib.py", line 693, in ntransfercmd > conn, size = FTP.ntransfercmd(self, cmd, rest) > File "/usr/lib/python2.7/ftplib.py", line 339, in ntransfercmd > resp = self.sendcmd(cmd) > File "/usr/lib/python2.7/ftplib.py", line 249, in sendcmd > return self.getresp() > File "/usr/lib/python2.7/ftplib.py", line 215, in getresp > resp = self.getmultiline() > File "/usr/lib/python2.7/ftplib.py", line 201, in getmultiline > line = self.getline() > File "/usr/lib/python2.7/ftplib.py", line 186, in getline > line = self.file.readline(self.maxline + 1) > File "/usr/lib/python2.7/socket.py", line 478, in readline > if e.args[0] == EINTR: >IndexError: tuple index out of range > >Locals by frame, innermost last > >Frame main in /usr/local/lib/python-apps/rmtdump/pymain.py at line 111 > backtrace = > pn = 'rmtdump' > process = > >Frame program in /usr/local/lib/python-apps/rmtdump/rmtdump.py at line 261 > aborted = True > argv = ['/usr/local/sbin/rmtdump'] > path = '/home' > >Frame process in /usr/local/lib/python-apps/rmtdump/rmtdump.py at line 190 > b_rt = '/home' > backupflag = 'r--' > backupstamp = 1453021366 > checkstamp = 1450374304 > dirs = ['antoon', 'jenny', 'lost+found', 'photo'] > entries = ['0-home', 'antoon', 'homelady', 'jenny', 'lost+found', 'photo', 'recover'] > entry = 'recover' > fileinfo = posix.stat_result(st_mode=16877, st_ino=2490369, st_dev=2058L, st_nlink=5, st_uid=0, st_gid=0, st_size=4 ... > files = [] > fqpn = '/home/recover' > islink = > ismount = > last = 1455445804.979914 > newfn = 'home/!2016-01-17 at 10_02_46-tbz' > now = 1455445804 > nr_of_tries = 0 > of = None > oldfn = 'home/!ENTRY-tbz' > path = '/home' > rt = '/home' > stamp = '1450374304' > start = 1455445804 > timefn = '!2016-01-17 at 10_02_46-tbz' > tryal = 1 > update = True > >Frame open in /usr/local/lib/python-apps/rmtdump/ftputil.py at line 198 > fn = 'home/!ENTRY-tbz' > ftp = > mode = 'wb' > self = > >Frame __init__ in /usr/local/lib/python-apps/rmtdump/ftputil.py at line 69 > bound = True > ch = 'b' > cmd = 'STOR' > cn = > count = 1 > mode = 'wb' > rfn = 'home/!ENTRY-tbz' > self = None > >Frame transfercmd in /usr/lib/python2.7/ftplib.py at line 376 > cmd = 'STOR home/!ENTRY-tbz' > rest = None > self = > >Frame ntransfercmd in /usr/lib/python2.7/ftplib.py at line 693 > cmd = 'STOR home/!ENTRY-tbz' > rest = None > self = > >Frame ntransfercmd in /usr/lib/python2.7/ftplib.py at line 352 > cmd = 'STOR home/!ENTRY-tbz' > conn = > host = '192.110.0.54' > port = 64649 > rest = None > self = > size = None > >Frame sendcmd in /usr/lib/python2.7/ftplib.py at line 249 > cmd = 'STOR home/!ENTRY-tbz' > self = > >Frame getresp in /usr/lib/python2.7/ftplib.py at line 215 > self = > >Frame getmultiline in /usr/lib/python2.7/ftplib.py at line 201 > self = > >Frame getline in /usr/lib/python2.7/ftplib.py at line 186 > self = > >Frame readline in /usr/lib/python2.7/socket.py at line 478 > buf = > buf_len = 0 > e = timeout() > self = > size = 8193 >-- >https://mail.python.org/mailman/listinfo/python-list -- St?phane Wirtel - http://wirtel.be - @matrixise From werotizy at freent.dd Sun Feb 14 07:26:00 2016 From: werotizy at freent.dd (Tom P) Date: Sun, 14 Feb 2016 13:26:00 +0100 Subject: problem with dateutil In-Reply-To: References: Message-ID: On 02/13/2016 09:45 PM, Gary Herron wrote: > On 02/13/2016 12:27 PM, Tom P wrote: >> On 02/13/2016 07:13 PM, Gary Herron wrote: >>> On 02/13/2016 09:58 AM, Tom P wrote: >>>> I am writing a program that has to deal with various date/time formats >>>> and convert these into timestamps. It looks as if >>>> dateutil.parser.parse should be able to handle about any format, but >>>> what I get is: >>>> >>>> datetimestr = '2012-10-22 11:22:33' >>>> print(dateutil.parser.parse(datetimestr)) >>>> result: datetime.datetime(2012, 10, 22, 11, 22, 33) >>>> >>>> However: >>>> datetimestr = '2012:10:22 11:22:33' >>>> print(dateutil.parser.parse(datetimestr)) >>>> result: datetime.datetime(2016, 2, 13, 11, 22, 33) >>>> >>>> In other words, it's getting the date wrong when colons are used to >>>> separate YYYY:MM:DD. Is there a way to include this as a valid format? >>>> >>> >>> Yes, there is a way to specify your own format. Search the datetime >>> documentation for >>> datetime.strptime(date_string, format) >>> >>> Gary Herron >>> >> >> Thanks. I started out with datetime.strptime but AFAICS that means I >> have to go through try/except for every conceivable format. Are you >> saying that I can't use dateutil.parser? > > Well now... If by "every conceivable format" you are including formats > that the author of dateutil.parser did not conceive of, then of course > you cannot use dateutil.parser. But you have the code for > dateutil.parser -- perhaps you could modify it to accept whatever odd > formats you care about. > > Gary Herron > > I had a look at the code for dateutil.parser. Have you looked at it? Meanwhile I'm living with try: dt = datetime.datetime.strptime(datetimestr, '%Y:%m:%d %H:%M:%S') except ValueError: dt = dateutil.parser.parse(datetimestr) unixtime = time.mktime(dt.timetuple()) From werotizy at freent.dd Sun Feb 14 07:27:51 2016 From: werotizy at freent.dd (Tom P) Date: Sun, 14 Feb 2016 13:27:51 +0100 Subject: problem with dateutil In-Reply-To: References: Message-ID: On 02/13/2016 10:01 PM, Mark Lawrence wrote: > On 13/02/2016 17:58, Tom P wrote: >> I am writing a program that has to deal with various date/time formats >> and convert these into timestamps. It looks as if dateutil.parser.parse >> should be able to handle about any format, but what I get is: >> >> datetimestr = '2012-10-22 11:22:33' >> print(dateutil.parser.parse(datetimestr)) >> result: datetime.datetime(2012, 10, 22, 11, 22, 33) >> >> However: >> datetimestr = '2012:10:22 11:22:33' >> print(dateutil.parser.parse(datetimestr)) >> result: datetime.datetime(2016, 2, 13, 11, 22, 33) >> >> In other words, it's getting the date wrong when colons are used to >> separate YYYY:MM:DD. Is there a way to include this as a valid format? >> > > From > http://labix.org/python-dateutil#head-a23e8ae0a661d77b89dfb3476f85b26f0b30349c > > > > parserinfo > This parameter allows one to change how the string is parsed, by > using a different parserinfo class instance. Using it you may, for > example, intenationalize the parser strings, or make it ignore > additional words. > > > HTH. > Thanks, let me look at that. From geoff.munn at gmail.com Sun Feb 14 08:39:20 2016 From: geoff.munn at gmail.com (Geoff Munn) Date: Sun, 14 Feb 2016 05:39:20 -0800 (PST) Subject: Syntax error (The Python Book) Linux User and Developer Bookazine Message-ID: Hi, Noob at the Python thing so here goes, I have copied a program to demonstrate control structures in Python but get a syntax error at line 31, isint = False. I'm using Python 2.7.6 and Linux Mint based around ubuntu14.04.1. I have pasted all the code below, #!/usr/bin/env python2 ''' We are going to write a program that will ask for the user to input an arbitary number of integers, store them in a collection, and then demonstrate how the collection would be used with various control structures ''' import sys # Used for the sys.exit function target_int=raw_input("How many integers?") ''' By now the variable target_int contains a string representation of whatever the user typed. We nee to try and convert that to an integer but be ready to deal with the error if it's not. Otherwise the program will crash ''' try: target_int=int(target_int) except ValueError: sys.exit("You must enter an integer") ints=list() # list to store the integers count = 0 # Track how many integers have been inputted # Keep asking for a number until we have reached the required number while count < target_int: new_int=raw_input("Please enter integer {0}:".format(count +1) isint = False try: new_int=int(new_int) # If the above succeeds then isint will #be set to true: isint = True except: print("You must enter an integer") ''' Only carry on if we have an integer. If not we will loop again. The == below is a comparision operator, a single = is an asignment operator ''' if isnit==True: ints.append(new_int) # Adds the integer to the collection count += 1 # Count is incremented by 1 # The for loop print ("Using a for loop") for values in ints: print (str(value)) # The while loop print ("Using a while loop") total=len(ints) # We already have the total from above but using len we can determine from the ints list. count = 0 while count < total: print (str(ints[count])) count += 1 From __peter__ at web.de Sun Feb 14 08:40:35 2016 From: __peter__ at web.de (Peter Otten) Date: Sun, 14 Feb 2016 14:40:35 +0100 Subject: ftplib throws: IndexError: tuple index out of range References: <56C05CEF.5080304@rece.vub.ac.be> Message-ID: Antoon Pardon wrote: > I have written a small backup program, that uses ftplib to make > remote backups. However recentely the program starts to regularly > raise IndexErrors, as far as I can see the problem is in socket.py > Can anyone shed some light? > > This is the traceback: [...] > File "/usr/lib/python2.7/socket.py", line 478, in readline > if e.args[0] == EINTR: > IndexError: tuple index out of range The offending line seems to be part of try: data = self._sock.recv(self._rbufsize) except error, e: if e.args[0] == EINTR: continue raise > Locals by frame, innermost last [...] > Frame readline in /usr/lib/python2.7/socket.py at line 478 > buf = > buf_len = 0 > e = timeout() > self = > size = 8193 It looks like the actual error is socket.timeout which is probably raised from somewhere inside the stdlib without args. I think this is a bug in socket.py; the error handler should special-case with either try: ... except timeout, e: raise except error, e: ... or try: ... except error, e: if e.args and e.args[0] == EINTR: continue raise You will still have to find the cause and handle the effects of the timeout. PS: How did you produce the overview over the local variables? That looks nice. From __peter__ at web.de Sun Feb 14 08:48:16 2016 From: __peter__ at web.de (Peter Otten) Date: Sun, 14 Feb 2016 14:48:16 +0100 Subject: How do i instantiate a class_name passed via cmd line References: Message-ID: Gregory Ewing wrote: > Something like this should do it: > > instance = getattr(module, class_name)(module_name, product) > > If the class name is always the same as the module name with the > first letter capitalized, you could use > > instance = getattr(module, module_name.capitalize())(module_name, > product) If the convention is that strict you could also expose the class under an alias: # in ebay.py Site = Ebay # in the client module module = importlib.import_module(module_name) instance = module.Site(product) From __peter__ at web.de Sun Feb 14 08:53:02 2016 From: __peter__ at web.de (Peter Otten) Date: Sun, 14 Feb 2016 14:53:02 +0100 Subject: Syntax error (The Python Book) Linux User and Developer Bookazine References: Message-ID: Geoff Munn wrote: > Noob at the Python thing so here goes, > > I have copied a program to demonstrate control structures in Python but > get a syntax error at line 31, isint = False. Often the actual syntax error in your code is in one of the lines preceding the one that Python is complaining about. > new_int=raw_input("Please enter integer {0}:".format(count +1) > isint = False Hint: Do the parentheses match? From rosuav at gmail.com Sun Feb 14 12:38:13 2016 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 15 Feb 2016 04:38:13 +1100 Subject: Syntax error (The Python Book) Linux User and Developer Bookazine In-Reply-To: References: Message-ID: On Mon, Feb 15, 2016 at 12:39 AM, Geoff Munn wrote: > Hi, > > Noob at the Python thing so here goes, > > I have copied a program to demonstrate control structures in Python but get a syntax error at line 31, isint = False. I'm using Python 2.7.6 and Linux Mint based around ubuntu14.04.1. I have pasted all the code below, > Peter's already explained your actual problem, so I'll make a few other comments about the code - starting with one about... comments. > ''' > By now the variable target_int contains a string representation of > whatever the user typed. We nee to try and convert that to an integer but > be ready to deal with the error if it's not. Otherwise the program will crash > ''' This isn't a comment. It's a triple-quoted string literal. Since a string literal as an expression on its own is legal and insignificant, this doesn't hurt much, but just be aware that these blocks aren't comments. (BTW, is the typo "nee to try" (s/be "need") from the original?) > try: > target_int=int(target_int) > except ValueError: > sys.exit("You must enter an integer") What does "crash" mean? In the case of Python, failing to check for this exception will result in a message printed to stderr and program termination. Instead, the exception is caught... and a message is printed to stderr and the program terminated. Is it really worth the effort? > while count < target_int: > new_int=raw_input("Please enter integer {0}:".format(count +1) > isint = False > try: > new_int=int(new_int) # If the above succeeds then isint will > #be set to true: isint = True > > except: > print("You must enter an integer") A bare except clause! Bad idea. Never do this. In the *extremely* rare cases when you actually do want to catch absolutely everything, you can spell it "except BaseException:", but most of the time, you want to catch one specific exception. > ''' > Only carry on if we have an integer. If not we will loop again. > The == below is a comparision operator, a single = is an asignment operator > ''' > if isnit==True: > ints.append(new_int) # Adds the integer to the collection > count += 1 # Count is incremented by 1 Despite your comments, isint is never set to true - and isnit is always going to be a NameError. Based on the number of typos here, I'm wondering if we can actually depend on the code that IS having trouble, which Peter mentioned as being a parenthesis count; maybe these are all transcription errors? > # The for loop > print ("Using a for loop") > for values in ints: > print (str(value)) > # The while loop > print ("Using a while loop") > total=len(ints) # We already have the total from above but using len we can determine from the ints list. > count = 0 > while count < total: > print (str(ints[count])) > count += 1 These two loops are inside your outer while loop - is that intentional? I strongly recommend not using the name 'count' for two completely different jobs in the same loop. Although the first one isn't actually used, which kinda makes it a bit pointless. This is not an example of Python best practice. It may be teaching you some things, but don't imitate its style. ChrisA From antoon.pardon at rece.vub.ac.be Sun Feb 14 13:10:16 2016 From: antoon.pardon at rece.vub.ac.be (Antoon Pardon) Date: Sun, 14 Feb 2016 19:10:16 +0100 Subject: ftplib throws: IndexError: tuple index out of range In-Reply-To: References: <56C05CEF.5080304@rece.vub.ac.be> Message-ID: <56C0C308.6000202@rece.vub.ac.be> Op 14-02-16 om 14:40 schreef Peter Otten: > Antoon Pardon wrote: > >> I have written a small backup program, that uses ftplib to make >> remote backups. However recentely the program starts to regularly >> raise IndexErrors, as far as I can see the problem is in socket.py >> Can anyone shed some light? >> >> This is the traceback: > > [...] > >> File "/usr/lib/python2.7/socket.py", line 478, in readline >> if e.args[0] == EINTR: >> IndexError: tuple index out of range > > The offending line seems to be part of > > try: > data = self._sock.recv(self._rbufsize) > except error, e: > if e.args[0] == EINTR: > continue > raise > > >> Locals by frame, innermost last > > [...] > >> Frame readline in /usr/lib/python2.7/socket.py at line 478 >> buf = >> buf_len = 0 >> e = timeout() >> self = >> size = 8193 > > It looks like the actual error is socket.timeout which is probably raised > from somewhere inside the stdlib without args. I think I know what is going on. I have my own timeout mechanism at work here, that works with signals and alarm. When the SIGALRM fires I just raise the socket.timeout without arguments, which then causes the problem. I defined my own timeout exception class and now raise that and the problem disappeared. Thanks for the extra pair of eyes. -- Antoon Pardon. From antoon.pardon at rece.vub.ac.be Sun Feb 14 13:23:57 2016 From: antoon.pardon at rece.vub.ac.be (Antoon Pardon) Date: Sun, 14 Feb 2016 19:23:57 +0100 Subject: ftplib throws: IndexError: tuple index out of range In-Reply-To: References: <56C05CEF.5080304@rece.vub.ac.be> Message-ID: <56C0C63D.4060908@rece.vub.ac.be> Op 14-02-16 om 14:40 schreef Peter Otten: > > PS: How did you produce the overview over the local variables? That looks > nice. > I started from this recipe: http://code.activestate.com/recipes/52215-get-more-information-from-tracebacks/ made it more to my liking and turned it into a module. So that instead of a program being run like this: def program(argv): ... if __name__ == '__main__': program(sys.argv) I start it now like this from pymain import main def program(argv): ... if __name__ == '__main__': main(program) pymain.main will then trap almost all exceptions and produces the overview over the local variables. -- Antoon. From ben+python at benfinney.id.au Sun Feb 14 16:46:03 2016 From: ben+python at benfinney.id.au (Ben Finney) Date: Mon, 15 Feb 2016 08:46:03 +1100 Subject: Make a unique filesystem path, without creating the file Message-ID: <85r3gf55k4.fsf@benfinney.id.au> Howdy all, How should a program generate a unique filesystem path and *not* create the filesystem entry? The ?tempfile.mktemp? function is strongly deprecated, and rightly so because it leaves the program vulnerable to insecure file creation. In some code (e.g. unit tests) I am calling ?tempfile.mktemp? to generate a unique path for a filesystem entry that I *do not want* to exist on the real filesystem. In this case the filesystem security concerns are irrelevant because there is no file. The deprecation of that function is a concern still, because I don't want code that makes every conscientious reader need to decide whether the code is a problem. Instead the code should avoid rightly-deprecated APIs. It is also prone to that API function disappearing at some point in the future, because it is explicitly and strongly deprecated. So I agree with the deprecation, but the library doesn't appear to provide a replacement. What standard library function should I be using to generate ?tempfile.mktemp?-like unique paths, and *not* ever create a real file by that path? -- \ ?If you have the facts on your side, pound the facts. If you | `\ have the law on your side, pound the law. If you have neither | _o__) on your side, pound the table.? ?anonymous | Ben Finney From PointedEars at web.de Sun Feb 14 17:48:25 2016 From: PointedEars at web.de (Thomas 'PointedEars' Lahn) Date: Sun, 14 Feb 2016 23:48:25 +0100 Subject: Make a unique filesystem path, without creating the file References: Message-ID: <6650790.gUWpFlgntZ@PointedEars.de> Ben Finney wrote: > How should a program generate a unique filesystem path and *not* create > the filesystem entry? The Python documentation suggests that it should not. > The ?tempfile.mktemp? function is strongly deprecated, and rightly so > > because it leaves the program vulnerable to insecure file creation. > > In some code (e.g. unit tests) I am calling ?tempfile.mktemp? to > generate a unique path for a filesystem entry that I *do not want* to > exist on the real filesystem. In this case the filesystem security > concerns are irrelevant because there is no file. I do not think that you have properly understood the problems with tmpfile.mktemp(). > [?] > It is also prone to that API function disappearing at some point in the > future, because it is explicitly and strongly deprecated. > > So I agree with the deprecation, but the library doesn't appear to > provide a replacement. | mktemp() usage can be replaced easily with NamedTemporaryFile(), passing | it the delete=False parameter: [example] > What standard library function should I be using to generate > ?tempfile.mktemp?-like unique paths, and *not* ever create a real file > by that path? I do not think it is possible to avoid the creation of a real file using the PSL; in fact, that a file is created appears to be precisely what fixes the problems with tempfile.mktemp() because then it cannot happen that someone else creates a file with the same name at the same time: | tempfile.NamedTemporaryFile(mode='w+b', buffering=None, encoding=None, | newline=None, suffix=None, prefix=None, dir=None, delete=True) | | This function operates exactly as TemporaryFile() does, except that the | file is guaranteed to have a visible name in the file system (on Unix, the | directory entry is not unlinked). [?] If delete is true (the default), the | file is deleted as soon as it is closed. [?] It is of course possible to generate a filename that is not currently used, but I am not aware of a PSL feature that does this, and if there were such a feature there would be the same problems with it as with mktemp(). -- PointedEars Twitter: @PointedEars2 Please do not cc me. / Bitte keine Kopien per E-Mail. From m at funkyhat.org Sun Feb 14 18:58:47 2016 From: m at funkyhat.org (Matt Wheeler) Date: Sun, 14 Feb 2016 23:58:47 +0000 Subject: Make a unique filesystem path, without creating the file In-Reply-To: References: <85r3gf55k4.fsf@benfinney.id.au> Message-ID: On 14 Feb 2016 21:46, "Ben Finney" wrote: > What standard library function should I be using to generate > ?tempfile.mktemp?-like unique paths, and *not* ever create a real file > by that path? Could you use tempfile.TemporaryDirectory and then just use a consistent name within that directory. It's guaranteed not to exist because the directory was only just created and only you can write to it? Has the added bonus of still being reasonably secure, to appease people like Mr PointedEars. (If you need multiple nonexistent paths in the same dir then perhaps use tempfile.NamedTemporaryFile with your newly created temp dir and an arbitrary suffix, and strip the suffix off to get the name you actually use.) -- Matt Wheeler http://funkyh.at From tjreedy at udel.edu Sun Feb 14 19:00:57 2016 From: tjreedy at udel.edu (Terry Reedy) Date: Sun, 14 Feb 2016 19:00:57 -0500 Subject: LIGO and Python Message-ID: On the PSF-Community list, a Python user points out that LIGO uses Python, with numpy and IPython, extensively. He linked to https://losc.ligo.org/s/events/GW150914/GW150914_tutorial.html which explains, with code, how the gravity wave signal was pulled out of the (publicly available) raw data. I found it an interesting read. -- Terry Jan Reedy From ben+python at benfinney.id.au Sun Feb 14 19:08:52 2016 From: ben+python at benfinney.id.au (Ben Finney) Date: Mon, 15 Feb 2016 11:08:52 +1100 Subject: Make a unique filesystem path, without creating the file References: <85r3gf55k4.fsf@benfinney.id.au> Message-ID: <85mvr26dij.fsf@benfinney.id.au> Matt Wheeler writes: > On 14 Feb 2016 21:46, "Ben Finney" wrote: > > What standard library function should I be using to generate > > ?tempfile.mktemp?-like unique paths, and *not* ever create a real > > file by that path? > > Could you use tempfile.TemporaryDirectory and then just use a > consistent name within that directory. That fails because it touches the filesystem. I want to avoid using a real file or a real directory. > It's guaranteed not to exist I am unconcerned with whether there is a real filesystem entry of that name; the goal entails having no filesystem activity for this. I want a valid unique filesystem path, without touching the filesystem. -- \ ?I believe our future depends powerfully on how well we | `\ understand this cosmos, in which we float like a mote of dust | _o__) in the morning sky.? ?Carl Sagan, _Cosmos_, 1980 | Ben Finney From sorsorday at gmail.com Sun Feb 14 19:17:44 2016 From: sorsorday at gmail.com (Herman) Date: Sun, 14 Feb 2016 16:17:44 -0800 Subject: How to properly override the default factory of defaultdict? Message-ID: I want to pass in the key to the default_factory of defaultdict and I found that defaultdict somehow can intercept my call to dict.__getitem__(self, key), so my class's __getitem__ have to catch a TypeError instead instead of KeyError. The following class is my code: class DefaultDictWithEnhancedFactory(defaultdict): """Just like the standard python collections.dict, but the default_factory takes the missing key as argument. Args: default_factory: A function that takes the missing key as the argument and return a value for the missing key. *a: arguments passing to the defaultdict constructor **kw: keyword arguments passing to the defaultdict constructor """ def __init__(self, default_factory, *a, **kw): defaultdict.__init__(self, default_factory, *a, **kw) def __getitem__(self, key): try: return dict.__getitem__(self, key) except KeyError: # Normally, you would expect this line to be # called for missing keys... return self.default_factory(key) except TypeError as ex: # However, this is actually getting called # because for some reason, defaultdict still # intercepts the __getitem__ call and raises: # TypeError: () takes exactly 1 argument (0 given) # So we have to catch that instead... if "lambda" in str(ex): return self.default_factory(key) From kaleywall93 at gmail.com Sun Feb 14 19:23:39 2016 From: kaleywall93 at gmail.com (Kaley Tucker) Date: Sun, 14 Feb 2016 16:23:39 -0800 (PST) Subject: Creating a List from a Loop Message-ID: <25e7431f-a3b1-4a0a-8bca-0d040d5af69b@googlegroups.com> I'm in an Intro to Python class and have an assignment where I need to implement a function calculating distance between two points, then create a loop to find points within a certain distance and create a list from those points. I was able to create the function, but I cant get my loop to create the list. Any help would be appreciated. I can't figure out the solution from my textbooks or other web sources. my current work: https://ghostbin.com/paste/qypvb From ben+python at benfinney.id.au Sun Feb 14 19:36:45 2016 From: ben+python at benfinney.id.au (Ben Finney) Date: Mon, 15 Feb 2016 11:36:45 +1100 Subject: How to properly override the default factory of defaultdict? References: Message-ID: <85io1q6c82.fsf@benfinney.id.au> Herman writes: > I want to pass in the key to the default_factory of defaultdict and I > found that defaultdict somehow can intercept my call to > dict.__getitem__(self, key), so my class's __getitem__ have to catch a > TypeError instead instead of KeyError. The following class is my code: I don't see an example of using that code, so that we can use it the same way you are. So I'll comment on some things that may not be relevant but nevertheless should be addressed: > class DefaultDictWithEnhancedFactory(defaultdict): > """Just like the standard python collections.dict, > but the default_factory takes the missing key as argument. Your doc string should have only a brief (about 50?60 characters) single line synopsis. If it's longer, you may be writing something too complex. (See PEP 257.) > Args: The class itself doesn't take arguments; its numerous methods do. I think you mean these descriptions to be in the ?__init__? method's doc string. > def __init__(self, default_factory, *a, **kw): > defaultdict.__init__(self, default_factory, *a, **kw) > > def __getitem__(self, key): > try: > return dict.__getitem__(self, key) you are using the inheritance hierarchy but thwarting it by not using ?super?. Instead:: super().__init__(self, default_factory, *a, **kw) and:: super().__getitem__(self, key) If you're not using Python 3 (and you should, for new code), ?super? is a little more complex. Migrating to Python 3 has this advantage among many others. -- \ "Those who will not reason, are bigots, those who cannot, are | `\ fools, and those who dare not, are slaves." ??Lord? George | _o__) Gordon Noel Byron | Ben Finney From rosuav at gmail.com Sun Feb 14 19:57:43 2016 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 15 Feb 2016 11:57:43 +1100 Subject: Creating a List from a Loop In-Reply-To: <25e7431f-a3b1-4a0a-8bca-0d040d5af69b@googlegroups.com> References: <25e7431f-a3b1-4a0a-8bca-0d040d5af69b@googlegroups.com> Message-ID: On Mon, Feb 15, 2016 at 11:23 AM, Kaley Tucker wrote: > I'm in an Intro to Python class and have an assignment where I need to implement a function calculating distance between two points, then create a loop to find points within a certain distance and create a list from those points. I was able to create the function, but I cant get my loop to create the list. Any help would be appreciated. I can't figure out the solution from my textbooks or other web sources. > > my current work: > > https://ghostbin.com/paste/qypvb Hi Kaley! Welcome to python-list; you're learning one of the best languages in the world. When you have questions like this, it's usually best to include your source code right in the body of the email, rather than including a link or an attachment. Attachments often don't survive, and links can break (plus, some people will simply refuse to click on them). Here's the code: import math blue_points = [[ 30 , 536254.99137 , 3659453.06343 ], [ 33 , 536721.584912 , 3659162.97207 ], [ 50 , 535807.099324 , 3659576.92825 ], [ 112 , 536827.131371 , 3657913.01245 ], [ 117 , 536473.254082 , 3659433.57702 ], [ 120 , 536196.9844 , 3658713.72722 ], [ 127 , 536387.547701 , 3658527.70015 ], [ 133 , 537397.838429 , 3659554.48657 ], [ 144 , 537715.931243 , 3658625.59997 ], [ 166 , 538367.648437 , 3658867.34288 ], [ 172 , 537112.662366 , 3657921.28957 ], [ 173 , 536418.315024 , 3658715.47946 ], [ 209 , 538096.28422 , 3658514.93514 ], [ 211 , 538077.87716 , 3658138.39337 ], [ 223 , 536220.396985 , 3659243.54161 ], [ 242 , 536102.087002 , 3658703.61054 ], [ 244 , 536968.755886 , 3659409.42857 ], [ 246 , 535996.903591 , 3658705.08691 ], [ 275 , 538078.165429 , 3659022.35547 ], [ 303 , 535999.885405 , 3658521.91524 ]] red_point = [1, 1073706.744,3658967.925] #points [ID, x coordinate, y coordinate] def identifyNeighbor(point, c_point): cid = c_point[0] cx = c_point[1] cy = c_point[2] bid = point[0] x = point[1] y = point [2] c_point = (cid, cx, cy) point = (bid, x, y) dx = cx - x dy = cy - y dsquared = dx**2 + dy**2 edistance = math.sqrt(dsquared) if edistance <= 536000: print "True" else: print "False" neighbor = [] def neighborloop(p): while p <= 18: p = p + 1 identifyNeighbor(blue_points[p], red_point) if identifyNeighbor == "True": neighbor.append(blue_points[p][0]) What you've done here is make a function which *prints* the word "True" or "False". Instead, you want to *return* something. Check your class textbook or other course materials; you should have been introduced to this concept. Once you figure that part out, you'll probably want to do the same thing in neighborloop too; don't use a global list, but instead, create a new list inside the function, and return that list. I'll let you explore the details on your own, rather than giving it all away (where's the fun in that!). Incidentally, your print statements indicate that you're using Python 2 here. I strongly recommend using Python 3 instead; the differences aren't huge, but it's much easier if you start on the modern side of the fence, instead of the backward-compatibility mode. There are a few small syntactic changes, but the most significant change involves Unicode handling, and trust me, it's a LOT easier if you learn that correctly from the beginning, rather than going through ASCII -> Extended ASCII -> DBCS -> Extended ASCII -> Unicode, the way I did! Nothing in your code actually calls any of these functions. Is that how you intend to do this? Any time you have a 'while' loop that counts upward, have a look to see if you can do this as a 'for' loop instead. Again, I'm assuming you've been taught about them already; if you haven't, ignore this recommendation, and just know that the way you're looping is wordy and unnecessary :) Have fun learning Python. It's a great language, and you seem to have most of it sorted out. If you have more trouble, don't hesitate to come back to this list, or to the python-tutor list if you think your problems are more at the tutorial level than general. All the best! ChrisA From rosuav at gmail.com Sun Feb 14 20:04:20 2016 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 15 Feb 2016 12:04:20 +1100 Subject: How to properly override the default factory of defaultdict? In-Reply-To: References: Message-ID: On Mon, Feb 15, 2016 at 11:17 AM, Herman wrote: > I want to pass in the key to the default_factory of defaultdict and I found > that defaultdict somehow can intercept my call to dict.__getitem__(self, > key), so my class's __getitem__ have to catch a TypeError instead instead > of KeyError. The following class is my code: Save yourself a lot of trouble, and just override __missing__: class DefaultDictWithEnhancedFactory(collections.defaultdict): def __missing__(self, key): return self.default_factory(key) ChrisA From dan at tombstonezero.net Sun Feb 14 20:07:00 2016 From: dan at tombstonezero.net (Dan Sommers) Date: Mon, 15 Feb 2016 01:07:00 -0000 (UTC) Subject: Make a unique filesystem path, without creating the file References: <85r3gf55k4.fsf@benfinney.id.au> Message-ID: On Mon, 15 Feb 2016 11:08:52 +1100, Ben Finney wrote: > I am unconcerned with whether there is a real filesystem entry of that > name; the goal entails having no filesystem activity for this. I want > a valid unique filesystem path, without touching the filesystem. That's an odd use case. If it's really just one valid filesystem path (your original post said *paths*, plural), then how about __file__? or os.__file__? From ben+python at benfinney.id.au Sun Feb 14 20:19:49 2016 From: ben+python at benfinney.id.au (Ben Finney) Date: Mon, 15 Feb 2016 12:19:49 +1100 Subject: Make a unique filesystem path, without creating the file References: <85r3gf55k4.fsf@benfinney.id.au> Message-ID: <85egce6a8a.fsf@benfinney.id.au> Dan Sommers writes: > On Mon, 15 Feb 2016 11:08:52 +1100, Ben Finney wrote: > > > I am unconcerned with whether there is a real filesystem entry of > > that name; the goal entails having no filesystem activity for this. > > I want a valid unique filesystem path, without touching the > > filesystem. > > That's an odd use case. It's very common to want filesystem paths divorced from accessing a filesystem entry. For example: test paths in a unit test. Filesystem access is orders of magnitude slower than accessing fake files in memory only, it is more complex and prone to irrelevant failures. So in such a test case filesystem access should be avoided as unnecessary. > If it's really just one valid filesystem path (your original post said > *paths*, plural), then how about __file__? or os.__file__? One valid filesystem path each time it's accessed. That is, behaviour equivalent to ?tempfile.mktemp?. My question is because the standard library clearly has this useful functionality implemented, but simultaneously warns strongly against its use. I'm looking for how to get at that functionality in a non-deprecated way, without re-implementing it myself. -- \ ?The most common way people give up their power is by thinking | `\ they don't have any.? ?Alice Walker | _o__) | Ben Finney From p_s_d_a_s_i_l_v_a_ns at netcabo.pt Sun Feb 14 21:21:23 2016 From: p_s_d_a_s_i_l_v_a_ns at netcabo.pt (Paulo da Silva) Date: Mon, 15 Feb 2016 02:21:23 +0000 Subject: What is heating the memory here? hashlib? References: Message-ID: ?s 07:04 de 14-02-2016, Paulo da Silva escreveu: > I was unable to reproduce the situation using a simple program just > walking through all files>4K, with or without the seek, and computing > their shasums. > Only some fluctuations of about 500MB in memory consumption. Today I gave another try to the program using 40MB bfsz on the same circumstances except for a previous reboot and, surprisingly, it worked pretty fine. The fluctuations in memory were of the same magnitude of those of the simple program. No swaps at all! Some history ... The 1st. time the problem occurred, I found an issue that I thought could cause that behavior. An equivalent statement for h=hashlib.sha256() was out of the files loop. I put it in the arguments parser because the user could choose the algorithm to use. And instead of testing the option for each file I put it there. Apart from the memory leakage hashlib seemed to work fine. After the "digest" I started feeding it with the contents of another file. 1. Is it possible that the memory exhaustion caused some sort of problem that left the system in a way to cause gc malfunction on the next runs? 2. The filesystem is btrfs. So, is it possible some "fight" among btrfs, gc and my program cause inability to gc free memory in time? This seems unlikely because I was only reading and the filesystem is mounted with noatime. However I don't know if btrfs takes some organization work during the readings. Anyway, I tried at least 3 times the failed tests one of which updating hashlib with 8KB chunks and another with 1MB bfsz. This last one ran until the end but used ~5GB swap. 3. There is another small change I made since then. Some (few) times hashlib was fed with empty data (zero length). That was fixed. So far I tried the program twice and it ran perfectly. When I need to run it in future, out of this confusion, and if the same problem occurs, I'll try to see things more carefully. Once more thank you all. Paulo From steve+comp.lang.python at pearwood.info Sun Feb 14 22:56:32 2016 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Mon, 15 Feb 2016 14:56:32 +1100 Subject: How to properly override the default factory of defaultdict? References: Message-ID: <56c14c74$0$1500$c3e8da3$5496439d@news.astraweb.com> On Monday 15 February 2016 11:17, Herman wrote: > I want to pass in the key to the default_factory of defaultdict Just use a regular dict and define __missing__: class MyDefaultDict(dict): def __missing__(self, key): return "We gotcha key %r right here!" % key If you want a per-instance __missing__, do something like this: class MyDefaultDict(dict): def __init__(self, factory): self._factory = factory def __missing__(self, key): return self._factory(self, key) d = MyDefaultDict(lambda self, key: ...) -- Steve From steve+comp.lang.python at pearwood.info Sun Feb 14 23:06:45 2016 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Mon, 15 Feb 2016 15:06:45 +1100 Subject: Make a unique filesystem path, without creating the file References: <85r3gf55k4.fsf@benfinney.id.au> Message-ID: <56c14ed7$0$11089$c3e8da3@news.astraweb.com> On Monday 15 February 2016 11:08, Ben Finney wrote: > I am unconcerned with whether there is a real filesystem entry of that > name; the goal entails having no filesystem activity for this. I want a > valid unique filesystem path, without touching the filesystem. Your phrasing is ambiguous. If you are unconcerned whether or not a file of that name exists, then just pick a name and use that: unique_path = /tmp/foo is guaranteed to be valid on POSIX systems and unique, and it may or may not exist. If you actually do care that /tmp/foo *doesn't* exist, then you have a problem: whatever name you pick *now* may no longer "not exist" a millisecond later. In general there's no way to create a valid pathname which doesn't exist *now* and is guaranteed to continue to not exist unless you touch the file system. But if you explain in more detail why you want this filename, perhaps we can come up with some ideas that will help. -- Steve From ben+python at benfinney.id.au Sun Feb 14 23:28:27 2016 From: ben+python at benfinney.id.au (Ben Finney) Date: Mon, 15 Feb 2016 15:28:27 +1100 Subject: Make a unique filesystem path, without creating the file References: <85r3gf55k4.fsf@benfinney.id.au> <56c14ed7$0$11089$c3e8da3@news.astraweb.com> Message-ID: <85a8n261hw.fsf@benfinney.id.au> Steven D'Aprano writes: > On Monday 15 February 2016 11:08, Ben Finney wrote: > > > I am unconcerned with whether there is a real filesystem entry of > > that name; the goal entails having no filesystem activity for this. > > I want a valid unique filesystem path, without touching the > > filesystem. > > Your phrasing is ambiguous. The existing behaviour of ?tempfile.mktemp? ? actually of its internal class ?tempfile._RandomNameSequence? ? is to generate unpredictable, unique, valid filesystem paths that are different each time. That's the behaviour I want, in a public API that exposes what ?tempfile? already has implemented, documented in a way that doesn't create a scare about security. > But if you explain in more detail why you want this filename, perhaps > we can come up with some ideas that will help. The behaviour is already implemented in the standard library. What I'm looking for is a way to use it (not re-implement it) that is public API and isn't scolded by the library documentation. -- \ ?Try adding ?as long as you don't breach the terms of service ? | `\ according to our sole judgement? to the end of any cloud | _o__) computing pitch.? ?Simon Phipps, 2010-12-11 | Ben Finney From martin at linux-ip.net Sun Feb 14 23:48:58 2016 From: martin at linux-ip.net (Martin A. Brown) Date: Sun, 14 Feb 2016 20:48:58 -0800 Subject: Make a unique filesystem path, without creating the file In-Reply-To: <85a8n261hw.fsf@benfinney.id.au> References: <85r3gf55k4.fsf@benfinney.id.au> <56c14ed7$0$11089$c3e8da3@news.astraweb.com> <85a8n261hw.fsf@benfinney.id.au> Message-ID: Good evening/morning Ben, >> > I am unconcerned with whether there is a real filesystem entry of >> > that name; the goal entails having no filesystem activity for this. >> > I want a valid unique filesystem path, without touching the >> > filesystem. >> >> Your phrasing is ambiguous. > >The existing behaviour of ?tempfile.mktemp? ? actually of its >internal class ?tempfile._RandomNameSequence? ? is to generate >unpredictable, unique, valid filesystem paths that are different >each time. > >That's the behaviour I want, in a public API that exposes what >?tempfile? already has implemented, documented in a way that >doesn't create a scare about security. If your code is not actually touching the filesystem, then it will not be affected by the race condition identified in the tempfile.mktemp() warning anyway. So, I'm unsure of your worry. >> But if you explain in more detail why you want this filename, perhaps >> we can come up with some ideas that will help. > >The behaviour is already implemented in the standard library. What >I'm looking for is a way to use it (not re-implement it) that is >public API and isn't scolded by the library documentation. I might also suggest the (bound) method _create_tmp() on class mailbox.Maildir, which achieves roughly the same goals, but for a permanent file. Of course, that particular method also touches the filesystem. The Maildir naming approach is based on the assumptions* that time is monotonically increasing, that system nodes never share the same name and that you don't need more than 1 uniquely named file per directory per millisecond. If so, then you can use the 9 or 10 lines of that method. Good luck, -Martin * I was tempted to joke about these two guarantees, but I think that undermines my basic message. To wit, you can probably rely on this naming technique about as much as you can rely on your system clock. I'll assume that you aren't naming all of your nodes 'franklin.p.gundersnip'. -- Martin A. Brown http://linux-ip.net/ From steve+comp.lang.python at pearwood.info Sun Feb 14 23:54:42 2016 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Mon, 15 Feb 2016 15:54:42 +1100 Subject: Make a unique filesystem path, without creating the file References: <85r3gf55k4.fsf@benfinney.id.au> Message-ID: <56c15a25$0$1622$c3e8da3$5496439d@news.astraweb.com> On Monday 15 February 2016 12:19, Ben Finney wrote: > One valid filesystem path each time it's accessed. That is, behaviour > equivalent to ?tempfile.mktemp?. > > My question is because the standard library clearly has this useful > functionality implemented, but simultaneously warns strongly against its > use. If you can absolutely guarantee that this string will never actually be used on a real filesystem, then go right ahead and use it. There's nothing wrong with (for instance) calling mktemp to generate *strings* that merely *look* like pathnames. If you want to guarantee that these faux pathnames can't leak out of your test suite and touch the file system, prepend an ASCII NUL to them. That will make it an illegal path on all file systems that I'm aware of. > I'm looking for how to get at that functionality in a non-deprecated > way, without re-implementing it myself. You probably can't, not if you want to future-proof your code against the day when tempfile.mktemp is removed. But you can simply fork that module, delete all the irrelevant bits, and make the mktemp function a private utility in your test suite. -- Steve From ben+python at benfinney.id.au Mon Feb 15 00:25:24 2016 From: ben+python at benfinney.id.au (Ben Finney) Date: Mon, 15 Feb 2016 16:25:24 +1100 Subject: Make a unique filesystem path, without creating the file References: <85r3gf55k4.fsf@benfinney.id.au> <56c15a25$0$1622$c3e8da3$5496439d@news.astraweb.com> Message-ID: <854mda5yuz.fsf@benfinney.id.au> Steven D'Aprano writes: > If you can absolutely guarantee that this string will never actually > be used on a real filesystem, then go right ahead and use it. I'm giving advice in examples in documentation. It's not enough to have some private usage that I know is good, I am looking for a standard API that when the reader looks it up will not be laden with big scary warnings. Currently I can write about the public API ?tempfile.mktemp? in documentation, but the conscientious reader will be correct to have concerns when the examples I give are sternly deprecated in the standard library documentation. Or I can write about the private API ?tempfile._RandomNameSequence? in the documentation, and the conscientious reader will be correct to have concerns about use of an undocumented private-use API. I'm looking for a way to give examples that use that standard library functionality, with an API that is both public and not discouraged. > > I'm looking for how to get at that functionality in a non-deprecated > > way, without re-implementing it myself. > > You probably can't, not if you want to future-proof your code against > the day when tempfile.mktemp is removed. That's disappointing. It is already implemented and well-tested, it is useful as is. Forking and duplicating it is poor practice if it can simply be used in a standard place. I have reported for this request. -- \ ?Nothing worth saying is inoffensive to everyone. Nothing worth | `\ saying will fail to make you enemies. And nothing worth saying | _o__) will not produce a confrontation.? ?Johann Hari, 2011 | Ben Finney From nigamreetesh84 at gmail.com Mon Feb 15 01:07:47 2016 From: nigamreetesh84 at gmail.com (reetesh nigam) Date: Sun, 14 Feb 2016 22:07:47 -0800 (PST) Subject: fetchall is taking much longer time while getting data from Sybase module in Python Message-ID: Hi All, I am retrieving data from Sybase database using Sybase module of Python. My query is not taking time however fecthall is taking longer time. Below is the test script : def run_query(db,query): ## Run query and resturn record result t1 = datetime.now() cursorObj = db.cursor() t2 = datetime.now() cursorObj.execute(query) t3 = datetime.now() import pdb pdb.set_trace() rowset = cursorObj.fetchall() t4 = datetime.now() cursorObj.close() print "Time taken to make cursor --%s"%(t2-t1) print "Time taken to execute query --%s"%(t3-t2) print "Time taken for fetchall--%s"%(t4-t3) return rowset Output: Time taken to make cursor --0:00:00.000037 Time taken to execute query --0:00:00.379443 Time taken for fetchall--0:00:14.739064 From frank at chagford.com Mon Feb 15 01:35:16 2016 From: frank at chagford.com (Frank Millman) Date: Mon, 15 Feb 2016 08:35:16 +0200 Subject: asyncio - run coroutine in the background Message-ID: Hi all Using asyncio, there are times when I want to execute a coroutine which is time-consuming. I do not need the result immediately, and I do not want to block the current task, so I want to run it in the background. run_in_executor() can run an arbitrary function in the background, but a coroutine needs an event loop. After some experimenting I came up with this - class BackgroundTask: async def run(self, coro, args, callback=None): loop = asyncio.get_event_loop() loop.run_in_executor(None, self.task_runner, coro, args, callback) def task_runner(self, coro, args, callback): loop = asyncio.new_event_loop() asyncio.set_event_loop(loop) fut = asyncio.ensure_future(coro(*args)) if callback is not None: fut.add_done_callback(callback) loop.run_until_complete(fut) loop.close() Usage - bg_task = BackgroundTask() args = (arg1, arg2 ...) callback = my_callback_function await bg_task.run(coro, args, callback) Although it 'awaits' bk_task.run(), it returns immediately, as it is simply waiting for run_in_executor() to be launched. Hope this is of some interest. Frank Millman From marko at pacujo.net Mon Feb 15 01:54:38 2016 From: marko at pacujo.net (Marko Rauhamaa) Date: Mon, 15 Feb 2016 08:54:38 +0200 Subject: asyncio - run coroutine in the background References: Message-ID: <8737sumpjl.fsf@elektro.pacujo.net> "Frank Millman" : > Using asyncio, there are times when I want to execute a coroutine which > is time-consuming. I do not need the result immediately, and I do not > want to block the current task, so I want to run it in the background. You can't run code "in the background" using asyncio. Coroutines perform cooperative multitasking in a single thread on a single CPU. Parallel processing requires the use of threads or, often preferably, processes. To put it in another way, never run time-consuming code in asyncio. Marko From frank at chagford.com Mon Feb 15 02:16:34 2016 From: frank at chagford.com (Frank Millman) Date: Mon, 15 Feb 2016 09:16:34 +0200 Subject: asyncio - run coroutine in the background In-Reply-To: <8737sumpjl.fsf@elektro.pacujo.net> References: <8737sumpjl.fsf@elektro.pacujo.net> Message-ID: "Marko Rauhamaa" wrote in message news:8737sumpjl.fsf at elektro.pacujo.net... > > "Frank Millman" : > > > Using asyncio, there are times when I want to execute a coroutine which > > is time-consuming. I do not need the result immediately, and I do not > > want to block the current task, so I want to run it in the background. > > You can't run code "in the background" using asyncio. Coroutines perform > cooperative multitasking in a single thread on a single CPU. > > Parallel processing requires the use of threads or, often preferably, > processes. > > To put it in another way, never run time-consuming code in asyncio. > No arguments there. I started with a task that ran quickly, but as I added stuff it started to slow down. The execution of the task involves calling some existing functions, which are themselves coroutines. As you have noted elsewhere, once you turn one function into a coroutine, all calls higher up the chain have to be coroutines as well. The benefit of my class is that it enables me to take the coroutine and run it in another thread, without having to re-engineer the whole thing. Hope this makes sense. Frank From marko at pacujo.net Mon Feb 15 02:34:35 2016 From: marko at pacujo.net (Marko Rauhamaa) Date: Mon, 15 Feb 2016 09:34:35 +0200 Subject: asyncio - run coroutine in the background References: <8737sumpjl.fsf@elektro.pacujo.net> Message-ID: <87si0ul94k.fsf@elektro.pacujo.net> "Frank Millman" : > The benefit of my class is that it enables me to take the coroutine > and run it in another thread, without having to re-engineer the whole > thing. > > Hope this makes sense. Sure. Marko From p_s_d_a_s_i_l_v_a_ns at netcabo.pt Mon Feb 15 02:38:06 2016 From: p_s_d_a_s_i_l_v_a_ns at netcabo.pt (Paulo da Silva) Date: Mon, 15 Feb 2016 07:38:06 +0000 Subject: What is heating the memory here? hashlib? References: Message-ID: ?s 09:49 de 14-02-2016, INADA Naoki escreveu: > tracemalloc module may help you to investigate leaks. > 2016/02/14 ??4:05 "Paulo da Silva" : > Thanks. I didn't know it! Paulo From no.email at nospam.invalid Mon Feb 15 02:39:07 2016 From: no.email at nospam.invalid (Paul Rubin) Date: Sun, 14 Feb 2016 23:39:07 -0800 Subject: asyncio - run coroutine in the background References: <8737sumpjl.fsf@elektro.pacujo.net> Message-ID: <87h9ha8lt0.fsf@jester.gateway.pace.com> "Frank Millman" writes: > The benefit of my class is that it enables me to take the coroutine > and run it in another thread, without having to re-engineer the whole > thing. Threads in Python don't get you parallelism either, of course. I haven't used async/await yet and it's looking painful. I've been wanting to read this: http://www.snarky.ca/how-the-heck-does-async-await-work-in-python-3-5 but I start to think it isn't all that great an approach to concurrency. From greg.ewing at canterbury.ac.nz Mon Feb 15 03:00:39 2016 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Mon, 15 Feb 2016 21:00:39 +1300 Subject: Make a unique filesystem path, without creating the file In-Reply-To: References: <85r3gf55k4.fsf@benfinney.id.au> Message-ID: Ben Finney wrote: > One valid filesystem path each time it's accessed. That is, behaviour > equivalent to ?tempfile.mktemp?. > > My question is because the standard library clearly has this useful > functionality implemented, but simultaneously warns strongly against its > use. But it *doesn't*, if your requirement is truly to not touch the filesystem at all, because tempfile.mktemp() *reads* the file system to make sure the name it's returning isn't in use. What's more, because you're *not* creating the file, mktemp() would be within its rights to return the same file name the second time you call it. If you want something that really doesn't go near the file system and/or is guaranteed to produce multiple different non-existing file names, you'll have to write it yourself. -- Greg From p_s_d_a_s_i_l_v_a_ns at netcabo.pt Mon Feb 15 03:05:46 2016 From: p_s_d_a_s_i_l_v_a_ns at netcabo.pt (Paulo da Silva) Date: Mon, 15 Feb 2016 08:05:46 +0000 Subject: What is heating the memory here? hashlib? References: <56bfe49e$0$1587$c3e8da3$5496439d@news.astraweb.com> Message-ID: ?s 02:21 de 14-02-2016, Steven D'Aprano escreveu: > On Sun, 14 Feb 2016 06:29 am, Paulo da Silva wrote: ... Thanks Steven for your advices. This is a small script to solve a specific problem. It will be used in future to solve other similar problems probably with small changes. When I found it eating memory and, what I thought was the 1st reason for that was fixed and it still ate the memory, I thought of something less obvious. After all it seems there is nothing wrong with it (see my other post). > That's your first clue that, perhaps, you should be reading in relatively > small blocks, more like 4K than 4MB. Sure enough, a quick bit of googling > shows that typically you should read from files in small-ish chunks, and > that trying to read in large chunks is often counter-productive: > > https://duckduckgo.com/html/?q=file+read+buffer+size > > The first three links all talk about optimal sizes being measured in small > multiples of 4K, not 40MB. > I didn't know about this! Most of my files are about ~>30MB. So I chose 40MB to avoid python loops. After all, python should be able to optimize those things. > You can try to increase the system buffer, by changing the "open" line to: > > with open(pathname, 'rb', buffering=40*M) as f: > This is another thing. One thing is the requested amount of data I want another is to choose de "really" buffer size. (I didn't know about this argument - thanks). ... > By the way, do you need a cryptographic checksum? sha256 is expensive to > calculate. If all you are doing is trying to match files which could have > the same content, you could use a cheaper hash, like md5 or even crc32. I don't know the probability of collision of each of them. The script has sha256 and md5 as options. When the failed execution I had chosen sha256. I didn't check if it takes much more time. A collision might cause data loss. So ... Thank you. Paulo From greg.ewing at canterbury.ac.nz Mon Feb 15 03:11:01 2016 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Mon, 15 Feb 2016 21:11:01 +1300 Subject: Make a unique filesystem path, without creating the file In-Reply-To: References: <85r3gf55k4.fsf@benfinney.id.au> <56c14ed7$0$11089$c3e8da3@news.astraweb.com> Message-ID: Ben Finney wrote: > The existing behaviour of ?tempfile.mktemp? ? actually of its internal > class ?tempfile._RandomNameSequence? ? is to generate unpredictable, > unique, valid filesystem paths that are different each time. But that's not documented behaviour, so even if mktemp() weren't marked as deprecated, you'd still be relying on undocumented and potentially changeable behaviour. > What I'm > looking for is a way to use it (not re-implement it) that is public API > and isn't scolded by the library documentation. Then you're looking for something that doesn't exist, I'm sorry to say, and it's unlikely you'll persuade anyone to make it exist. If you want to leverage stdlib functionality for this, I'd suggest something along the lines of: def fakefilename(dir, ext): return os.path.join(dir, str(uuid.uuid4())) + ext -- Greg From dfnsonfsduifb at gmx.de Mon Feb 15 03:12:48 2016 From: dfnsonfsduifb at gmx.de (Johannes Bauer) Date: Mon, 15 Feb 2016 09:12:48 +0100 Subject: What is heating the memory here? hashlib? In-Reply-To: References: Message-ID: On 15.02.2016 03:21, Paulo da Silva wrote: > So far I tried the program twice and it ran perfectly. I think you measured your RAM consumption wrong. Linux uses all free RAM as HDD cache. That's what is used in "buffers". That is, it's not "free", but it would be free if any process would sbrk(). My guess is that you only looked at the "free" number going down and concluded your program is eating your RAM. Which it wasn't. Cheers Johannes -- >> Wo hattest Du das Beben nochmal GENAU vorhergesagt? > Zumindest nicht ?ffentlich! Ah, der neueste und bis heute genialste Streich unsere gro?en Kosmologen: Die Geheim-Vorhersage. - Karl Kaos ?ber R?diger Thomas in dsa From frank at chagford.com Mon Feb 15 03:17:28 2016 From: frank at chagford.com (Frank Millman) Date: Mon, 15 Feb 2016 10:17:28 +0200 Subject: asyncio - run coroutine in the background In-Reply-To: <87h9ha8lt0.fsf@jester.gateway.pace.com> References: <8737sumpjl.fsf@elektro.pacujo.net> <87h9ha8lt0.fsf@jester.gateway.pace.com> Message-ID: "Paul Rubin" wrote in message news:87h9ha8lt0.fsf at jester.gateway.pace.com... > > "Frank Millman" writes: > > The benefit of my class is that it enables me to take the coroutine > > and run it in another thread, without having to re-engineer the whole > > thing. > > Threads in Python don't get you parallelism either, of course. > Agreed. My class does not alter the total time taken, but it does free up the original task to carry on with other work. run_in_executor() uses threads by default, but it does allow you to specify processes as an alternative. > I haven't used async/await yet and it's looking painful. I've been > wanting to read this: > > http://www.snarky.ca/how-the-heck-does-async-await-work-in-python-3-5 > > but I start to think it isn't all that great an approach to concurrency. > Thanks for that link. I had a quick scan, and it looks interesting, but some of it a bit above my head. I have bookmarked it, as I think that as my understanding increases, I will gain more from it on each re-read. Frank From mal at europython.eu Mon Feb 15 03:27:47 2016 From: mal at europython.eu (M.-A. Lemburg) Date: Mon, 15 Feb 2016 09:27:47 +0100 Subject: EuroPython 2016: Sending out the first gravitational waves Message-ID: <56C18C03.1000504@europython.eu> We are pleased to announce the launch of our all new EuroPython 2016 website. Over the last few weeks, we have been busy talking to sponsors and getting the website prepared for the launch. You may have heard about the recent direct observation of gravitational waves by the LIGO (Laser Interferometer Gravitational-wave Observatory). What you may not know is that Python helped in analyzing the data, so we now have two things to celebrate: 1. Python?s use in this phenomenal direct proof of Einstein?s prediction and 2. the launch of our 2016 edition of the EuroPython conference. So here it is: *** https://ep2016.europython.eu/ *** July 17 - 24 2016 Many thanks go to our launch sponsors who have signed up early to give us that extra boost in motivation to get the conference and it?s website set up. Meet our Launch Sponsors ------------------------ * Bilbao Ekintza * Intel * UPV/EHU * Udemy * Python Software Foundation * Blue Yonder * Jet Brains * Numberly * Flying Circus * Limejump * RedHat * Vzzual.com * Django-CMS * Riverbank PS: We?d like to thank the EuroPython Web WG for the web site improvements and our friends at Python Italia for making their code available. With gravitational regards, -- EuroPython 2016 Team http://ep2016.europython.eu/ http://www.europython-society.org/ From greg.ewing at canterbury.ac.nz Mon Feb 15 03:28:28 2016 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Mon, 15 Feb 2016 21:28:28 +1300 Subject: How to properly override the default factory of defaultdict? In-Reply-To: References: Message-ID: Herman wrote: > I want to pass in the key to the default_factory of defaultdict and I found > that defaultdict somehow can intercept my call to dict.__getitem__(self, > key), What's happening here is that defaultdict doesn't actually override __getitem__ at all. Instead, it overrides __missing__, which gets called by the standard dict's __getitem__ for a missing key. As Steven said, you don't need a defaultdict here at all, just a dict subclass that defines __missing__ the way you want. -- Greg From __peter__ at web.de Mon Feb 15 04:25:46 2016 From: __peter__ at web.de (Peter Otten) Date: Mon, 15 Feb 2016 10:25:46 +0100 Subject: fetchall is taking much longer time while getting data from Sybase module in Python References: Message-ID: reetesh nigam wrote: > Hi All, > > I am retrieving data from Sybase database using Sybase module of Python. > My query is not taking time however fecthall is taking longer time. > > Below is the test script : > > def run_query(db,query): ## Run query and resturn record result > t1 = datetime.now() > cursorObj = db.cursor() > t2 = datetime.now() > cursorObj.execute(query) > t3 = datetime.now() > import pdb > pdb.set_trace() > rowset = cursorObj.fetchall() > t4 = datetime.now() > cursorObj.close() > print "Time taken to make cursor --%s"%(t2-t1) > print "Time taken to execute query --%s"%(t3-t2) > print "Time taken for fetchall--%s"%(t4-t3) > return rowset > > Output: > Time taken to make cursor --0:00:00.000037 > Time taken to execute query --0:00:00.379443 > Time taken for fetchall--0:00:14.739064 fetchall() probably has to transfer a lot of rows. If you want to process them one at a time you can turn run_query into a generator def run_query(db, query): ... while True: row = cursor.fetchone() if row is None: break yield row ... A complete implementation that guarantees that the cursor is closed might look like this (untested): @contextlib.contextmanager def run_query(db, query): cursor = db.cursor() try: cursor.execute(query) yield iter(cursor.fetchone, None) finally: cursor.close() # use it with run_query(db, query) as rows: for row in rows: print row This is likely to *increase* the overall time taken, but should drastically reduce the time you have to wait for the first record to be printed, i. e. the latency. From __peter__ at web.de Mon Feb 15 05:04:13 2016 From: __peter__ at web.de (Peter Otten) Date: Mon, 15 Feb 2016 11:04:13 +0100 Subject: ftplib throws: IndexError: tuple index out of range References: <56C05CEF.5080304@rece.vub.ac.be> <56C0C308.6000202@rece.vub.ac.be> Message-ID: Antoon Pardon wrote: >> It looks like the actual error is socket.timeout which is probably raised >> from somewhere inside the stdlib without args. > > I think I know what is going on. I have my own timeout mechanism at work > here, that works with signals and alarm. When the SIGALRM fires I just > raise the socket.timeout without arguments, which then causes the problem. OK, then it's self-inflicted pain ;) From __peter__ at web.de Mon Feb 15 05:05:11 2016 From: __peter__ at web.de (Peter Otten) Date: Mon, 15 Feb 2016 11:05:11 +0100 Subject: ftplib throws: IndexError: tuple index out of range References: <56C05CEF.5080304@rece.vub.ac.be> <56C0C63D.4060908@rece.vub.ac.be> Message-ID: Antoon Pardon wrote: >> PS: How did you produce the overview over the local variables? That looks >> nice. > I started from this recipe: > http://code.activestate.com/recipes/52215-get-more-information-from-tracebacks/ > made it more to my liking and turned it into a module. So that instead of > a program being run like this: Ah, thanks! From arjuns123 at gmail.com Mon Feb 15 05:44:34 2016 From: arjuns123 at gmail.com (Arjun Srivatsa) Date: Mon, 15 Feb 2016 02:44:34 -0800 (PST) Subject: Unable to insert data into MongoDB. In-Reply-To: References: Message-ID: <8bb40e3f-57ce-4d5a-a426-6b56cebc8de7@googlegroups.com> I changed the port number from 27017 to 55555 in the code segment: IP = "127.0.0.1" PORT = 27017 BUFFER_SIZE = 1024 client = MongoClient('127.0.0.1', 27017) And my output on Python shell is: hello world! Connection address: ('127.0.0.1', 16951) Connection address: ('127.0.0.1', 16953) Connection address: ('127.0.0.1', 16957) Connection address: ('127.0.0.1', 16958) Connection address: ('127.0.0.1', 16959) Connection address: ('127.0.0.1', 16961) Connection address: ('127.0.0.1', 16962) Connection address: ('127.0.0.1', 16963) Connection address: ('127.0.0.1', 16964) Connection address: ('127.0.0.1', 16965) and it goes on and on... What does the above port numbers mean? And I can't still see the data being inserted into Database yet. And as you said, if I change server.close to server.close(), then I get a traceback: Traceback (most recent call last): File "C:\Users\SRA2LO\Desktop\API.py", line 41, in data = server.recv(BUFFER_SIZE) File "C:\Python27\lib\socket.py", line 174, in _dummy raise error(EBADF, 'Bad file descriptor') error: [Errno 9] Bad file descriptor On Thursday, February 11, 2016 at 5:09:53 PM UTC+1, MRAB wrote: > On 2016-02-11 15:12, Arjun Srivatsa wrote: > > Hi guys. I am basically transferring the data from PLC to PC (where the Python API runs) but I'm unable to insert into MongoDB thereafter. When I run the Python script on IDLE, the output is > > > > Hello World! > > Traceback (most recent call last): File "C:\Users\SRA2LO\Desktop\API.py", line 32, in s.bind((IP, PORT)) File "C:\Python27\lib\socket.py", line 228, in meth return getattr(self._sock,name)(*args) error: [Errno 10013] An attempt was made to access a socket in a way forbidden by its access permissions > > and when I change the IP of MongoDB server, it shows > > > > > > Hello World! > > Traceback (most recent call last): File "C:\Users\SRA2LO\Desktop\API.py", line 32, in s.bind((IP, PORT)) File "C:\Python27\lib\socket.py", line 228, in meth return getattr(self._sock,name)(*args) error: [Errno 10049] The requested address is not valid in its context. > > > > Could you please help me out? I have disabled the firewall as well. > > > > Here's the API I have written. > > > > #!/usr/bin/python > > > > import socket > > import socket > > from pymongo import MongoClient > > #from eve import Eve > > import datetime > > > > # Connection to server (PLC) on port 27017 > > server = socket.socket() > > host = "10.52.124.135" #IP of PLC > > port = 27017 > > BUFFER_SIZE = 1024 > > ############### > > > > server.connect((host, port)) > > print server.recv(1024) > > > > server.close > > > > #Connection to Client (Mongodb) on port 27017 > > IP = "127.0.0.1" > > PORT = 27017 > > BUFFER_SIZE = 1024 > > > > client = MongoClient('127.0.0.1', 27017) > > db = client.test_database > > > > s = socket.socket(socket.AF_INET, socket.SOCK_STREAM) > > s.bind((IP, PORT)) > > s.listen(1) > > > > #connections loop > > while True: > > conn, addr = s.accept() > > print 'Connection address:',addr > > try: > > # read loop > > while True: > > data = server.recv(BUFFER_SIZE) > > > > if not data: break > > conn.sendall(data) > > > > > > # send to MongoDB > > > > mongodoc = { "data": data, "date" : datetime.datetime.utcnow() } > > > > > > ABC = db.ABC > > ABC_id = ABC.insert_one(mongodoc).inserted_id > > > > finally: > > conn.close() > > > I don't know whether it's relevant, but you didn't close the server > socket. You have "server.close" instead of "server.close()". > > Also, the code as posted won't compile because the block after the > "while True:" isn't indented. From levarii at walla.com Mon Feb 15 05:57:42 2016 From: levarii at walla.com (izik43) Date: Mon, 15 Feb 2016 04:57:42 -0600 Subject: Changing sound volume References: Message-ID: i had had to play some wav files and the volume was very poor. i used "audacity" and used amplify effect to change the basic amplitude. From marko at pacujo.net Mon Feb 15 06:05:32 2016 From: marko at pacujo.net (Marko Rauhamaa) Date: Mon, 15 Feb 2016 13:05:32 +0200 Subject: asyncio - run coroutine in the background References: <8737sumpjl.fsf@elektro.pacujo.net> <87h9ha8lt0.fsf@jester.gateway.pace.com> Message-ID: <87povyp72b.fsf@elektro.pacujo.net> Paul Rubin : > Threads in Python don't get you parallelism either, of course. Ah, of course. Processes it is, then. Marko From __peter__ at web.de Mon Feb 15 06:36:17 2016 From: __peter__ at web.de (Peter Otten) Date: Mon, 15 Feb 2016 12:36:17 +0100 Subject: Unable to insert data into MongoDB. References: <8bb40e3f-57ce-4d5a-a426-6b56cebc8de7@googlegroups.com> Message-ID: Arjun Srivatsa wrote: > I changed the port number from 27017 to 55555 in the code segment: Instead of throwing arbitrary changes at your script in the hope that one works I recommend that you write two independent scripts, from scratch: (1) write_to_db.py: Write made-up data into the MongoDB on your local machine. (2) read_from_server.py: Read data from the server and print it to stdout. Reduce these scripts to the bare minimum. Debug them one ofter another. Once both scripts work to your satisfaction replace the sample data in script one with code from script two that reads actual data. From vek.m1234 at gmail.com Mon Feb 15 08:05:01 2016 From: vek.m1234 at gmail.com (Veek. M) Date: Mon, 15 Feb 2016 18:35:01 +0530 Subject: repr( open('/etc/motd', 'rt').read() ) Message-ID: When I do at the interpreter prompt, repr( open('/etc/motd', 'rt').read() ) i get # 1 #: "'\\nThe programs included with the Debian GNU/Linux system are free software;\\nthe exact distribution terms for each program are described in the\\nindividual files in /usr/share/doc/*/copyright.\\n\\nDebian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent\\npermitted by applicable law.\\n'" whereas if i just do: open('/etc/motd', 'rt').read() i get # 2 #: '\nThe programs included with the Debian GNU/Linux system are free software;\nthe exact distribution terms for each program are described in the\nindividual files in /usr/share/doc/*/copyright.\n\nDebian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent\npermitted by applicable law.\n' --------------- With # 2 # read returns a string that the interpreter displays by calling __str__ via print so newlines are converted to \n. What is happening with # 1 # (repr)? repr calls __repr__ which gives you bytes.. why does this result in \\n From emile at fenx.com Mon Feb 15 09:22:56 2016 From: emile at fenx.com (Emile van Sebille) Date: Mon, 15 Feb 2016 06:22:56 -0800 Subject: There has to be a better way to split this string! In-Reply-To: <20160210065002.GA86307@cskk.homeip.net> References: <20160210065002.GA86307@cskk.homeip.net> Message-ID: On 2/9/2016 10:50 PM, Cameron Simpson wrote: > On 10Feb2016 07:34, srinivas devaki wrote: >> PS: trying to read mailing list when you are half woke, is a bad idea and >> trying reply to it is even bad idea. > > Regrettably, when one is half awake one is unable to realise what a bad > idea it may be:-) Not to worry -- you find out quick enough. :) Emile From random832 at fastmail.com Mon Feb 15 09:47:22 2016 From: random832 at fastmail.com (Random832) Date: Mon, 15 Feb 2016 09:47:22 -0500 Subject: repr( open('/etc/motd', 'rt').read() ) In-Reply-To: References: Message-ID: <1455547642.1366484.521647634.5BB28D8A@webmail.messagingengine.com> On Mon, Feb 15, 2016, at 08:05, Veek. M wrote: > What is happening with # 1 # (repr)? > repr calls __repr__ which gives you bytes.. why does this result in \\n When you call a function that returns a string directly in the interpreter prompt (i.e. without print), it passes the result to repr, which means in this case repr has been called twice. From geoff.munn at gmail.com Mon Feb 15 09:56:12 2016 From: geoff.munn at gmail.com (Geoff Munn) Date: Mon, 15 Feb 2016 06:56:12 -0800 (PST) Subject: Syntax error (The Python Book) Linux User and Developer Bookazine In-Reply-To: References: Message-ID: On Sunday, 14 February 2016 13:39:52 UTC, Geoff Munn wrote: > Hi, > > Noob at the Python thing so here goes, > > I have copied a program to demonstrate control structures in Python but get a syntax error at line 31, isint = False. I'm using Python 2.7.6 and Linux Mint based around ubuntu14.04.1. I have pasted all the code below, > > > > #!/usr/bin/env python2 > > ''' > We are going to write a program that will ask for the user to input an arbitary > number of integers, store them in a collection, and then demonstrate how > the collection would be used with various control structures > ''' > > import sys # Used for the sys.exit function > > target_int=raw_input("How many integers?") > > ''' > By now the variable target_int contains a string representation of > whatever the user typed. We nee to try and convert that to an integer but > be ready to deal with the error if it's not. Otherwise the program will crash > ''' > > try: > target_int=int(target_int) > except ValueError: > sys.exit("You must enter an integer") > > ints=list() # list to store the integers > > count = 0 # Track how many integers have been inputted > > # Keep asking for a number until we have reached the required number > while count < target_int: > new_int=raw_input("Please enter integer {0}:".format(count +1) > isint = False > try: > new_int=int(new_int) # If the above succeeds then isint will > #be set to true: isint = True > > except: > print("You must enter an integer") > > ''' > Only carry on if we have an integer. If not we will loop again. > The == below is a comparision operator, a single = is an asignment operator > ''' > if isnit==True: > ints.append(new_int) # Adds the integer to the collection > count += 1 # Count is incremented by 1 > # The for loop > print ("Using a for loop") > for values in ints: > print (str(value)) > # The while loop > print ("Using a while loop") > total=len(ints) # We already have the total from above but using len we can determine from the ints list. > count = 0 > while count < total: > print (str(ints[count])) > count += 1 Thanks Peter and Chris, yes missed the parentheses by taking the error as being in line 31, DOH but a lesson learned. I have checked and checked the code I entered against the provided code and had to make some more changes to at least go through the first while loop but have given up on the rest of it. Given your comments do you think its worth persevering with this book or is there a better 'entry' into Python programming? From arjuns123 at gmail.com Mon Feb 15 10:00:22 2016 From: arjuns123 at gmail.com (Arjun Srivatsa) Date: Mon, 15 Feb 2016 07:00:22 -0800 (PST) Subject: Unable to insert data into MongoDB. In-Reply-To: References: <8bb40e3f-57ce-4d5a-a426-6b56cebc8de7@googlegroups.com> Message-ID: Hi Peter. Thank you for the reply. This is the read_server code: import socket from pymongo import MongoClient #import datetime import sys # Connection to server (PLC) on port 27017 host = "10.52.124.135" port = 27017 s = socket.socket(socket.AF_INET, socket.SOCK_STREAM) s.connect((host, port)) sys.stdout.write(s.recv(1024)) And the write_db code: from pymongo import MongoClient import datetime import socket import sys client = MongoClient('mongodb://localhost:27017/') db = client.test_database mongodoc = { "data": 'data', "date" : datetime.datetime.utcnow() } values = db.values values_id = values.insert_one(mongodoc).inserted_id So, both these scripts work independently. While, read_server shows the output of the actual data from PLC, write_db inserts the sample data into the MongoDB. I am not sure as to how to combine these both and get the desired output. On Monday, February 15, 2016 at 12:37:02 PM UTC+1, Peter Otten wrote: > Arjun Srivatsa wrote: > > > I changed the port number from 27017 to 55555 in the code segment: > > Instead of throwing arbitrary changes at your script in the hope that one > works I recommend that you write two independent scripts, from scratch: > > (1) write_to_db.py: > Write made-up data into the MongoDB on your local machine. > > (2) read_from_server.py: > Read data from the server and print it to stdout. > > Reduce these scripts to the bare minimum. Debug them one ofter another. > > Once both scripts work to your satisfaction replace the sample data in > script one with code from script two that reads actual data. From joel.goldstick at gmail.com Mon Feb 15 10:06:34 2016 From: joel.goldstick at gmail.com (Joel Goldstick) Date: Mon, 15 Feb 2016 10:06:34 -0500 Subject: Syntax error (The Python Book) Linux User and Developer Bookazine In-Reply-To: References: Message-ID: On Mon, Feb 15, 2016 at 9:56 AM, Geoff Munn wrote: > On Sunday, 14 February 2016 13:39:52 UTC, Geoff Munn wrote: > > Hi, > > > > Noob at the Python thing so here goes, > > > > I have copied a program to demonstrate control structures in Python but > get a syntax error at line 31, isint = False. I'm using Python 2.7.6 and > Linux Mint based around ubuntu14.04.1. I have pasted all the code below, > > > > > > > > #!/usr/bin/env python2 > > > > ''' > > We are going to write a program that will ask for the user to input an > arbitary > > number of integers, store them in a collection, and then demonstrate how > > the collection would be used with various control structures > > ''' > > > > import sys # Used for the sys.exit function > > > > target_int=raw_input("How many integers?") > > > > ''' > > By now the variable target_int contains a string representation of > > whatever the user typed. We nee to try and convert that to an integer but > > be ready to deal with the error if it's not. Otherwise the program will > crash > > ''' > > > > try: > > target_int=int(target_int) > > except ValueError: > > sys.exit("You must enter an integer") > > > > ints=list() # list to store the integers > > > > count = 0 # Track how many integers have been inputted > > > > # Keep asking for a number until we have reached the required number > > while count < target_int: > > new_int=raw_input("Please enter integer {0}:".format(count +1) > > isint = False > > try: > > new_int=int(new_int) # If the above succeeds then isint will > > #be set to true: isint = True > > > > except: > > print("You must enter an integer") > > > > ''' > > Only carry on if we have an integer. If not we will loop again. > > The == below is a comparision operator, a single = is an asignment > operator > > ''' > > if isnit==True: > > ints.append(new_int) # Adds the integer to the collection > > count += 1 # Count is incremented by 1 > > # The for loop > > print ("Using a for loop") > > for values in ints: > > print (str(value)) > > # The while loop > > print ("Using a while loop") > > total=len(ints) # We already have the total from above but using > len we can determine from the ints list. > > count = 0 > > while count < total: > > print (str(ints[count])) > > count += 1 > > Thanks Peter and Chris, yes missed the parentheses by taking the error as > being in line 31, DOH but a lesson learned. I have checked and checked the > code I entered against the provided code and had to make some more changes > to at least go through the first while loop but have given up on the rest > of it. Given your comments do you think its worth persevering with this > book or is there a better 'entry' into Python programming? > -- > https://mail.python.org/mailman/listinfo/python-list > Learn Python the Hard Way is pretty good some people say. Its online. Also Diving into Python is online written by the now offline Mark Pilgrim. -- Joel Goldstick http://joelgoldstick.com/stats/birthdays From arjuns123 at gmail.com Mon Feb 15 10:28:02 2016 From: arjuns123 at gmail.com (Arjun Srivatsa) Date: Mon, 15 Feb 2016 07:28:02 -0800 (PST) Subject: Unable to insert data into MongoDB. In-Reply-To: References: <8bb40e3f-57ce-4d5a-a426-6b56cebc8de7@googlegroups.com> Message-ID: <18fabe32-b500-4b53-b6eb-e5419c8e530a@googlegroups.com> Okay, I added with open('yourfile','w') as f: f.write(data) to the read_server code in which the Actual data is stored in a file on my desktop. Then, it must be possible to read this file in the Write_db script to insert the data. On Monday, February 15, 2016 at 4:00:37 PM UTC+1, Arjun Srivatsa wrote: > Hi Peter. > > Thank you for the reply. > > This is the read_server code: > > import socket > from pymongo import MongoClient > #import datetime > import sys > > # Connection to server (PLC) on port 27017 > host = "10.52.124.135" > port = 27017 > > s = socket.socket(socket.AF_INET, socket.SOCK_STREAM) > s.connect((host, port)) > sys.stdout.write(s.recv(1024)) > > > And the write_db code: > > from pymongo import MongoClient > import datetime > import socket > import sys > > client = MongoClient('mongodb://localhost:27017/') > db = client.test_database > > mongodoc = { "data": 'data', "date" : datetime.datetime.utcnow() } > values = db.values > values_id = values.insert_one(mongodoc).inserted_id > > > So, both these scripts work independently. While, read_server shows the output of the actual data from PLC, write_db inserts the sample data into the MongoDB. > > I am not sure as to how to combine these both and get the desired output. > > > > On Monday, February 15, 2016 at 12:37:02 PM UTC+1, Peter Otten wrote: > > Arjun Srivatsa wrote: > > > > > I changed the port number from 27017 to 55555 in the code segment: > > > > Instead of throwing arbitrary changes at your script in the hope that one > > works I recommend that you write two independent scripts, from scratch: > > > > (1) write_to_db.py: > > Write made-up data into the MongoDB on your local machine. > > > > (2) read_from_server.py: > > Read data from the server and print it to stdout. > > > > Reduce these scripts to the bare minimum. Debug them one ofter another. > > > > Once both scripts work to your satisfaction replace the sample data in > > script one with code from script two that reads actual data. From geoff.munn at gmail.com Mon Feb 15 10:33:44 2016 From: geoff.munn at gmail.com (Geoff Munn) Date: Mon, 15 Feb 2016 07:33:44 -0800 (PST) Subject: Syntax error (The Python Book) Linux User and Developer Bookazine In-Reply-To: References: Message-ID: <14f5a7c7-ff9b-4f9a-ba5b-5a624f253a7f@googlegroups.com> On Monday, 15 February 2016 15:07:03 UTC, Joel Goldstick wrote: > On Mon, Feb 15, 2016 at 9:56 AM, Geoff Munn wrote: > > > On Sunday, 14 February 2016 13:39:52 UTC, Geoff Munn wrote: > > > Hi, > > > > > > Noob at the Python thing so here goes, > > > > > > I have copied a program to demonstrate control structures in Python but > > get a syntax error at line 31, isint = False. I'm using Python 2.7.6 and > > Linux Mint based around ubuntu14.04.1. I have pasted all the code below, > > > > > > > > > > > > #!/usr/bin/env python2 > > > > > > ''' > > > We are going to write a program that will ask for the user to input an > > arbitary > > > number of integers, store them in a collection, and then demonstrate how > > > the collection would be used with various control structures > > > ''' > > > > > > import sys # Used for the sys.exit function > > > > > > target_int=raw_input("How many integers?") > > > > > > ''' > > > By now the variable target_int contains a string representation of > > > whatever the user typed. We nee to try and convert that to an integer but > > > be ready to deal with the error if it's not. Otherwise the program will > > crash > > > ''' > > > > > > try: > > > target_int=int(target_int) > > > except ValueError: > > > sys.exit("You must enter an integer") > > > > > > ints=list() # list to store the integers > > > > > > count = 0 # Track how many integers have been inputted > > > > > > # Keep asking for a number until we have reached the required number > > > while count < target_int: > > > new_int=raw_input("Please enter integer {0}:".format(count +1) > > > isint = False > > > try: > > > new_int=int(new_int) # If the above succeeds then isint will > > > #be set to true: isint = True > > > > > > except: > > > print("You must enter an integer") > > > > > > ''' > > > Only carry on if we have an integer. If not we will loop again. > > > The == below is a comparision operator, a single = is an asignment > > operator > > > ''' > > > if isnit==True: > > > ints.append(new_int) # Adds the integer to the collection > > > count += 1 # Count is incremented by 1 > > > # The for loop > > > print ("Using a for loop") > > > for values in ints: > > > print (str(value)) > > > # The while loop > > > print ("Using a while loop") > > > total=len(ints) # We already have the total from above but using > > len we can determine from the ints list. > > > count = 0 > > > while count < total: > > > print (str(ints[count])) > > > count += 1 > > > > Thanks Peter and Chris, yes missed the parentheses by taking the error as > > being in line 31, DOH but a lesson learned. I have checked and checked the > > code I entered against the provided code and had to make some more changes > > to at least go through the first while loop but have given up on the rest > > of it. Given your comments do you think its worth persevering with this > > book or is there a better 'entry' into Python programming? > > -- > > https://mail.python.org/mailman/listinfo/python-list > > > > > Learn Python the Hard Way is pretty good some people say. Its online. > Also Diving into Python is online written by the now offline Mark Pilgrim. > -- > Joel Goldstick > http://joelgoldstick.com/stats/birthdays Thanks Joel From invalid at invalid.invalid Mon Feb 15 10:46:47 2016 From: invalid at invalid.invalid (Grant Edwards) Date: Mon, 15 Feb 2016 15:46:47 +0000 (UTC) Subject: Make a unique filesystem path, without creating the file References: Message-ID: On 2016-02-14, Ben Finney wrote: > Howdy all, > > How should a program generate a unique filesystem path and *not* create > the filesystem entry? Short answer: you can't because it's the filesystem entry operation that is atomic and guarantees uniqueness. > [..] > What standard library function should I be using to generate > ?tempfile.mktemp?-like unique paths, and *not* ever create a real file > by that path? What's the point of creating a unique path if you don't want to create the file? -- Grant Edwards grant.b.edwards Yow! I'm rated PG-34!! at gmail.com From invalid at invalid.invalid Mon Feb 15 10:49:36 2016 From: invalid at invalid.invalid (Grant Edwards) Date: Mon, 15 Feb 2016 15:49:36 +0000 (UTC) Subject: Make a unique filesystem path, without creating the file References: <85r3gf55k4.fsf@benfinney.id.au> Message-ID: On 2016-02-15, Ben Finney wrote: > Dan Sommers writes: > >> On Mon, 15 Feb 2016 11:08:52 +1100, Ben Finney wrote: >> >> > I am unconcerned with whether there is a real filesystem entry of >> > that name; the goal entails having no filesystem activity for this. >> > I want a valid unique filesystem path, without touching the >> > filesystem. >> >> That's an odd use case. > > It's very common to want filesystem paths divorced from accessing a > filesystem entry. If the filesystem paths are not associated with a filesystem, what do you mean by "unique"? You want to make sure that path which doesn't exist in some filesystem is different from all other paths that don't exist in some filesystem? > For example: test paths in a unit test. Filesystem access is orders > of magnitude slower than accessing fake files in memory only, How is "fake files in memory" not a filesystem? -- Grant Edwards grant.b.edwards Yow! The Korean War must at have been fun. gmail.com From __peter__ at web.de Mon Feb 15 11:33:40 2016 From: __peter__ at web.de (Peter Otten) Date: Mon, 15 Feb 2016 17:33:40 +0100 Subject: Unable to insert data into MongoDB. References: <8bb40e3f-57ce-4d5a-a426-6b56cebc8de7@googlegroups.com> Message-ID: Arjun Srivatsa wrote: > Hi Peter. > > Thank you for the reply. > > This is the read_server code: > > import socket > from pymongo import MongoClient > #import datetime > import sys > > # Connection to server (PLC) on port 27017 > host = "10.52.124.135" > port = 27017 > > s = socket.socket(socket.AF_INET, socket.SOCK_STREAM) > s.connect((host, port)) > sys.stdout.write(s.recv(1024)) > > > And the write_db code: > > from pymongo import MongoClient > import datetime > import socket > import sys > > client = MongoClient('mongodb://localhost:27017/') > db = client.test_database > > mongodoc = { "data": 'data', "date" : datetime.datetime.utcnow() } > values = db.values > values_id = values.insert_one(mongodoc).inserted_id > > > So, both these scripts work independently. While, read_server shows the > output of the actual data from PLC, write_db inserts the sample data into > the MongoDB. > > I am not sure as to how to combine these both and get the desired output. What I mean is once you have working scripts connect_to_mongodb() while True: record = make_fake_data() insert_record_into_mongodb(record) and connect_to_server() while True: record = read_record_from_server() print(record) you can combine the code in a third script to connect_to_server() connect_to_mongodb() while True: record = read_record_from_server() insert_record_into_mongodb(record) and be fairly sure that the combination works, too. From p_s_d_a_s_i_l_v_a_ns at netcabo.pt Mon Feb 15 12:29:07 2016 From: p_s_d_a_s_i_l_v_a_ns at netcabo.pt (Paulo da Silva) Date: Mon, 15 Feb 2016 17:29:07 +0000 Subject: What is heating the memory here? hashlib? References: Message-ID: ?s 08:12 de 15-02-2016, Johannes Bauer escreveu: > On 15.02.2016 03:21, Paulo da Silva wrote: > >> So far I tried the program twice and it ran perfectly. > > I think you measured your RAM consumption wrong. > > Linux uses all free RAM as HDD cache. That's what is used in "buffers". > That is, it's not "free", but it would be free if any process would > sbrk(). My guess is that you only looked at the "free" number going down > and concluded your program is eating your RAM. Which it wasn't. > No, for sure. I monitored (using atop) free, cache and swap. In general, because I only have 2GB, freemem is almost always a few tens of MB. Remaining "free" memory is in Cache. When Cache goes low it begins to swap out. Paulo From tjreedy at udel.edu Mon Feb 15 15:03:21 2016 From: tjreedy at udel.edu (Terry Reedy) Date: Mon, 15 Feb 2016 15:03:21 -0500 Subject: repr( open('/etc/motd', 'rt').read() ) In-Reply-To: References: Message-ID: On 2/15/2016 8:05 AM, Veek. M wrote: > When I do at the interpreter prompt, > repr( open('/etc/motd', 'rt').read() ) > i get # 1 #: When posting questions here or at Stackoverflow or elsewhere, it is a really good idea to develop and post a 'minimal, complete, verifiable example' that demonstrates the behavior in question. In this case, the open and read calls are just noise. A string with a newline illustrates your question without distraction. >>> s = '\n' >>> len(s) 1 >>> len(str(s)) 1 >>> len(repr(s)) 4 >>> s '\n' >>> str(s) '\n' >>> repr(s) "'\\n'" >>> print(s) >>> print(str(s)) >>> print(repr(s)) '\n' >>> For this question, 'at the interpreter prompt' is essential, so leaving the >>> prompt is a good idea. I did the above with 3.5.1 also in IDLE and got exactly the same result, which should be the case. print('start') s='\n' print(s) print(str(s)) print(repr(s)) print('add repr') print(repr(s)) print(repr(str(s))) print(repr(repr(s))) print('end') duplicates the collective >>> responses seen above and demonstrates, as Random832 said, that '>>> expr' prints repr(expr). start '\n' add repr '\n' '\n' "'\\n'" end -- Terry Jan Reedy From orgnut at yahoo.com Mon Feb 15 16:17:23 2016 From: orgnut at yahoo.com (Larry Hudson) Date: Mon, 15 Feb 2016 13:17:23 -0800 Subject: Syntax error (The Python Book) Linux User and Developer Bookazine In-Reply-To: References: Message-ID: On 02/15/2016 07:06 AM, Joel Goldstick wrote: [snip a lot...] > > Learn Python the Hard Way is pretty good some people say. Its online. > Also Diving into Python is online written by the now offline Mark Pilgrim. > I have a couple of "Hard Way" books and personally, I don't like his style of teaching. Of course, take that as one person's opinion -- and as always, YMMV. :-) -=- Larry -=- From roel at roelschroeven.net Mon Feb 15 17:22:47 2016 From: roel at roelschroeven.net (Roel Schroeven) Date: Mon, 15 Feb 2016 23:22:47 +0100 Subject: Make a unique filesystem path, without creating the file In-Reply-To: <85r3gf55k4.fsf@benfinney.id.au> References: <85r3gf55k4.fsf@benfinney.id.au> Message-ID: Ben Finney schreef op 2016-02-14 22:46: > How should a program generate a unique filesystem path and *not* create > the filesystem entry? > ... > What standard library function should I be using to generate > ?tempfile.mktemp?-like unique paths, and *not* ever create a real file > by that path? Use uuid.uuid1()? -- The saddest aspect of life right now is that science gathers knowledge faster than society gathers wisdom. -- Isaac Asimov Roel Schroeven From cs at zip.com.au Mon Feb 15 18:05:39 2016 From: cs at zip.com.au (Cameron Simpson) Date: Tue, 16 Feb 2016 10:05:39 +1100 Subject: Make a unique filesystem path, without creating the file In-Reply-To: <85egce6a8a.fsf@benfinney.id.au> References: <85egce6a8a.fsf@benfinney.id.au> Message-ID: <20160215230539.GA42677@cskk.homeip.net> On 15Feb2016 12:19, Ben Finney wrote: >Dan Sommers writes: >> On Mon, 15 Feb 2016 11:08:52 +1100, Ben Finney wrote: >> > I am unconcerned with whether there is a real filesystem entry of >> > that name; the goal entails having no filesystem activity for this. >> > I want a valid unique filesystem path, without touching the >> > filesystem. >> >> That's an odd use case. > >It's very common to want filesystem paths divorced from accessing a >filesystem entry. > >For example: test paths in a unit test. Filesystem access is orders of >magnitude slower than accessing fake files in memory only, it is more >complex and prone to irrelevant failures. So in such a test case >filesystem access should be avoided as unnecessary. But.. then why a filesystem path at all in that case? Why use a filesystem as a reference at all? I've been watching this for a few days, and am struggling to understand your use case. The only modes I can imagine for such a thing (a generated but unused filename) are: checking that the name is syntactly valid, for whatever constrains you may have (but if you're calling an opaque mktemp-like function, is this feasible or remediable?) checking that the name generated does in fact not correspond to an existing file (which presumes that the target directory has no other users, which also implies that you don't need mktemp - a simple prefix+unused-ordinal will do) generating test paths using a real filesystem as a reference but not making a test file - I'm having trouble imagining how this can be useful generating test paths without using a real filesystem as a reference, but then you can't even use mktemp I think I can contrive your test case scenario using #3: filepath = mktemp(existing_dir_path) fp = InMemoryFileLikeClassWithBogusName(filepath) do I/O on fp ... but I don't see how it is useful to have a notion of a filepath at all in this case, and therefore I don't see why you would want a mktemp-like function available. Can you elaborate with a concrete example and its purpose which would work with a mktemp-ish official function? You say: >One valid filesystem path each time it's accessed. That is, behaviour >equivalent to ?tempfile.mktemp?. > >My question is because the standard library clearly has this useful >functionality implemented, but simultaneously warns strongly against its >use. > >I'm looking for how to get at that functionality in a non-deprecated >way, without re-implementing it myself. I think "the standard library clearly has this useful functionality implemented, but simultaneously warns strongly against its use" pretty much precludes this. I think you probably need to reimplement. However if your intent is never to use the path you can use something very simple (my personal habit is prefix+ordinal where that doesn't already exist - keep the last ordinal to arrange a distinct name next time). Cheers, Cameron Simpson From PointedEars at web.de Mon Feb 15 19:18:17 2016 From: PointedEars at web.de (Thomas 'PointedEars' Lahn) Date: Tue, 16 Feb 2016 01:18:17 +0100 Subject: Make a unique filesystem path, without creating the file References: <85r3gf55k4.fsf@benfinney.id.au> Message-ID: <2015485.VjBY4A5gp9@PointedEars.de> Gregory Ewing wrote: > Ben Finney wrote: >> One valid filesystem path each time it's accessed. That is, behaviour >> equivalent to ?tempfile.mktemp?. >> >> My question is because the standard library clearly has this useful >> functionality implemented, but simultaneously warns strongly against its >> use. > > But it *doesn't*, Yes, it does. > if your requirement is truly to not touch the filesystem at all, because > tempfile.mktemp() *reads* the file system to make sure the name it's > returning isn't in use. But there is a race condition occurring between the moment that the filesystem has been read and is being written to by another user. Hence the deprecation in favor of tempfile.mkstemp() which also *creates* the file instead, and the warning about the security hole if tempfile.mktemp() is used anyway. You can use tempfile.mktemp() only as long as it is irrelevant if a file with that name already exists, or exists later but was not created by you. -- PointedEars Twitter: @PointedEars2 Please do not cc me. / Bitte keine Kopien per E-Mail. From ben+python at benfinney.id.au Mon Feb 15 20:53:59 2016 From: ben+python at benfinney.id.au (Ben Finney) Date: Tue, 16 Feb 2016 12:53:59 +1100 Subject: Make a unique filesystem path, without creating the file References: <85r3gf55k4.fsf@benfinney.id.au> Message-ID: <85povx4dzc.fsf@benfinney.id.au> Roel Schroeven writes: > Use uuid.uuid1()? That has potential. A little counter-intuitive, for use in documentation about testing filesystem paths; but not frightening or dubious to the conscientious reader. I'll see whether that meets this use case, thank you. The bug report (to make a supported ?tempfile? API for generating filesystem paths only) remains, and fixing that would be the correct way to address this IMO. -- \ ?I used to be a proofreader for a skywriting company.? ?Steven | `\ Wright | _o__) | Ben Finney From rantingrickjohnson at gmail.com Mon Feb 15 21:02:10 2016 From: rantingrickjohnson at gmail.com (Rick Johnson) Date: Mon, 15 Feb 2016 18:02:10 -0800 (PST) Subject: [STORY-TIME] THE BDFL AND HIS PYTHON PETTING ZOO In-Reply-To: <60036746-78f7-4f05-a6d8-04e88b09c00f@googlegroups.com> References: <48762040-a7e0-434c-92e8-8a0969210e5b@googlegroups.com> <5d31dd14-1a60-4f80-8889-d0616f404c70@googlegroups.com> <4e902c58-3370-4adf-905d-74630a6eea90@googlegroups.com> <60036746-78f7-4f05-a6d8-04e88b09c00f@googlegroups.com> Message-ID: <032f0f66-34df-4325-98e0-1127025d2a94@googlegroups.com> On Friday, February 12, 2016 at 1:51:35 AM UTC-6, John Ladasky wrote: Reguarding a migration from Python2 to Pyhton3, John said: > I had to wait until my favorite packages were ported > (numpy, scipy, matplotlib, pandas). WxPython is not ported either, much to my chagrin. > But once that happened, I moved from Py2 to Py3 years ago > with scarcely a bump, bruise, or scratch. So you have no Python2.x code remaining in your repos? Are you telling us that you moved *EVERYTHING* to Python3? If so, i can't imagine how something like that would even be possible, hmm, unless of course, you don't have much code to move...? I could imagine that *IF* someone's entire Python repo consisted of a couple hundred (or less) small scripts, with few, or no, dependencies, such a migration would be relatively easy. But even though i've only been writing Python code for a handful of years, my repo consists of thousands of scripts, millions of lines of code, and many, *MANY* dependencies -- because after all, DRY is very important, yes? (That reminds me, i need to write a script to compile some statistics on these files...) > I like lazy evaluation. Well, it is a "Pythonic feature" no doubt. And I'll admit, many of the changes are "good changes". But they are not good enough to risk blowing up millions of lines of code, John. And what will i gain if i did? Nothing. There are *NO* new features to offer my users, and even the features that *WOULD* improve my coding experiences, are not enough to warrant the misery.Heck, almost all of these festures i could implement myself. What's easier: migrating millions of lines into a hostile environment just so i can gain a few new functionalities, or writing a few lines to bring the needed functionalities to *ME*? I don't need, nor do i want, *EVERYTHING* that Python3 brings to the table. > I think that Unicode handling is vastly improved (and yes, > I'm fully aware that exactly two people in this newsgroup > don't agree, they make sure we all know). I have > encountered few surprises, and nothing that makes my job > harder. Well John, not everyone is going to experience the same "level of onerous" when they port their code over to Python3. Perhaps your case is unique. And there is no way for myself, or anyone one else, to make an objective judgment, without looking at the breath and depth of the code you had to migrate. Perhaps you don't have a lot of dependencies. Perhaps you have not engineered "feature-rich libraries". Perhaps you have have not written code that is on the bleeding edge, or pushed your mental power to it's extremes. Perhaps your code style is to "play it safe". Perhaps you're scripts consist mostly of simple maintenance and utilities. So even if you "believe" that the migration from Python2.x to Python3.x is smooth, you're only able to make that judgment relative to own *PERSONAL* experience, with your own *PERSONAL* library. So even though there is only two people in this group who don't care for Pyhton Unicode handling (hey, your stats, not mine!), there is only *ONE* of you who can judge the level of onerous required to migrate *YOUR* code! Until you make the depth and breath of you repo publicly view-able, your opinion is just that -- an *OPINION*! As for me, I've written vast libraries over the top of wxPython and Tkinter. I've extended existing functionality, added new functionality, and created a more consistent and intuitive API than either library could have dreamed. I've spent many long hours debugging, testing, and perfecting these libraries, and so far, they work very nicely. Heck, my refactored Tkinter API, and IDLE editor, is leaps and bounds ahead of anything Python3 could manage to do. And it's a shame too, because these two libraries are among some of the oldest and most neglected of all. But now i'm faced with a tough choice: (1) I could attempt to migrate all these libraries, and risk mountains of subtle bugs echoing down the inheritance/dependency chains, or (2) I could leave the code where it is, and milk-it until the majority of OS's will no longer run Python2. JUST FYI: I'M GOING WITH CHOICE NUMBER TWO! Why? Because only a fool would risk "exception hell" on code that has been stable for a reasonable amount of time. Ever heard the expression: "if it ain't broke, don't fix it"? Well my code ain't broke, and until it becomes broken, my time would be better spent forging new paths, instead of flogging the old ones like dead, quadrupedaled, "beasts of burden"! > To be sure, people who are entrenched in the Py2 way of > doing things, with a lot of legacy code, have some work to > do -- on their code, and possibly on their brains. Your "veiled ad hominem" is not justified John.With that statement, you've just insulted well over half of the Python community -- and the "better half", i might add! You're talking about people who have been coding python, and many other languages, probably long before you were even born! I'm not as "curmudgeonly" as some folks around here, but i'm no "spring chicken" either. Hurling personal attack just because people don't gush over your beloved Pyhton3 is a low-blow. > Keep Py2 if you want it, then. You still have a few more > years before the PSF stops maintaining it. If you really > like it that much, why not maintain it yourself? I don't need Python3. And i reckon that by the time i do, something more interesting will come along, or, i'll create something more interesting myself. i've been drafting, and dreaming, of a new language spec for over a year now.. And the best thing about starting a new language, you can do anything you want... no dependencies! From nobody at nowhere.invalid Mon Feb 15 21:14:10 2016 From: nobody at nowhere.invalid (Nobody) Date: Tue, 16 Feb 2016 02:14:10 +0000 Subject: Make a unique filesystem path, without creating the file References: <85r3gf55k4.fsf@benfinney.id.au> <56c14ed7$0$11089$c3e8da3@news.astraweb.com> Message-ID: On Mon, 15 Feb 2016 15:28:27 +1100, Ben Finney wrote: > The behaviour is already implemented in the standard library. What I'm > looking for is a way to use it (not re-implement it) that is public API > and isn't scolded by the library documentation. So, basically you want (essentially) the exact behaviour of tempfile.mktemp(), except without any mention of the (genuine) risks that such a function presents? I suspect that you'll have to settle for either a) using that function and simply documenting the reasons why it isn't an issue in this particular case, or b) re-implementing it (so that you can choose to avoid mentioning the issue in its documentation). At the outside, you *might* have a third option: c) persuade the maintainers to tweak the documentation to further clarify that the risk arises from creating a file with the returned name, not from simply calling the function. But actually it's already fairly clear if you actually read it. If it's the bold-face "Warning:" and the red background that you don't like, I wouldn't expect those to go away either for mktemp() or for any other function with similar behaviour (i.e. something which someone *might* try to use to actually create temporary files). The simple fact that it might get used that way is enough to warrant a prominent warning. From rantingrickjohnson at gmail.com Mon Feb 15 21:26:01 2016 From: rantingrickjohnson at gmail.com (Rick Johnson) Date: Mon, 15 Feb 2016 18:26:01 -0800 (PST) Subject: Make a unique filesystem path, without creating the file In-Reply-To: <56c15a25$0$1622$c3e8da3$5496439d@news.astraweb.com> References: <85r3gf55k4.fsf@benfinney.id.au> <56c15a25$0$1622$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Sunday, February 14, 2016 at 10:55:11 PM UTC-6, Steven D'Aprano wrote: > If you want to guarantee that these faux pathnames can't > leak out of your test suite and touch the file system, > prepend an ASCII NUL to them. That will make it an illegal > path on all file systems that I'm aware of. Hmm, the unfounded fears in this thread are beginning to remind me of a famous Black Sabbath song. Finished with "py tempfile", 'cause it, couldn't help to, ease my mind. People think i'm insane, because, i want "faux paths", all the time. All day long i think of ways, but nothing seems to, satisfy. Think i'll loose my mind, if i don't, find a py-module to, pacify. CAN YOU HELP ME? MAKE "FAUX PATHS" TODAAAAY, OH YEAH... From rosuav at gmail.com Mon Feb 15 21:30:14 2016 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 16 Feb 2016 13:30:14 +1100 Subject: [STORY-TIME] THE BDFL AND HIS PYTHON PETTING ZOO In-Reply-To: <032f0f66-34df-4325-98e0-1127025d2a94@googlegroups.com> References: <48762040-a7e0-434c-92e8-8a0969210e5b@googlegroups.com> <5d31dd14-1a60-4f80-8889-d0616f404c70@googlegroups.com> <4e902c58-3370-4adf-905d-74630a6eea90@googlegroups.com> <60036746-78f7-4f05-a6d8-04e88b09c00f@googlegroups.com> <032f0f66-34df-4325-98e0-1127025d2a94@googlegroups.com> Message-ID: On Tue, Feb 16, 2016 at 1:02 PM, Rick Johnson wrote: >> But once that happened, I moved from Py2 to Py3 years ago >> with scarcely a bump, bruise, or scratch. > > So you have no Python2.x code remaining in your repos? Are > you telling us that you moved *EVERYTHING* to Python3? If > so, i can't imagine how something like that would even be > possible, hmm, unless of course, you don't have much code to > move...? I have one project that's bound to Py2 by its one dependency, and I've kept it Py3-compatible apart from the one block of code that sets that up. (That includes renaming a function that was originally called "await", even though that's not actually a keyword as of 3.6.) All the rest of my code will run on Python 3. However, there is still a lot of Python 2 code in my repos - it's the same code! Most of my stuff is simple enough that I keep it 2/3 compatible. Sometimes there's a block of code at the top to deal with {raw_,}input or a future directive or a changed import name, but the bulk of the code is unchanged. Of course, now that I go to actually *check*, it turns out that a lot of my code has accidentally not been cross-version compatible. I use the open(fn, encoding="...") form in quite a few places, so to make that work, I would need "from io import open" (which seems to be a safe no-op on Py3), and there are places where I use multi-arg versions of print, which will end up displaying tuples in Py2 if I don't slap in a __future__ directive. But that just means that I really truly have *moved* to Python 3, to the extent that I don't always even test my code on 2.7 any more. ChrisA From rantingrickjohnson at gmail.com Mon Feb 15 22:56:33 2016 From: rantingrickjohnson at gmail.com (Rick Johnson) Date: Mon, 15 Feb 2016 19:56:33 -0800 (PST) Subject: Changing sound volume In-Reply-To: References: Message-ID: On Monday, February 15, 2016 at 4:58:04 AM UTC-6, izik43 wrote: > i had had to play some wav files and the volume was very > poor. i used "audacity" and used amplify effect to change > the basic amplitude. Yes, you can achieve a more powerful punch that way, but it's not the same as what the OP needs. Heck, with that method, he just might blow-out his cheap monitor speakers! ;-) Besides, editing an audio file is not something you should do without practice. If you're not careful, you can destroy the sound quality by "clipping-off" the extreme tips of the waveform. Sure, distortion can sometimes add a desirable effect, but too much of a good thing, well, you know... In my early days as an "amateur sound guy", i destroyed some valuable live recordings due to my ignorance of clipping. Yeah, i didn't realize i was editing the *ACTUAL FILE*, and not a copy of it! Had to learn the hard way. :-'( When you're an audiophile, those kinds of screw-ups are devastating. Magical musical moments are a rarity. From john at gwinner.org Mon Feb 15 23:26:56 2016 From: john at gwinner.org (John D. Gwinner) Date: Tue, 16 Feb 2016 04:26:56 +0000 Subject: Installation error, compiling from source on Oracle Linux Message-ID: I'm installing an app that requires Carbon and some other Python 2.7 features. The version of Oracle Linux we're using comes with 2.6. I've read that it is not a good idea to directly update the O/S as it "may break things" so I'm doing make altinstall. I've downloaded Python-2.7.11 Downloaded zlib-1.2.8 Done ./configure --prefix=/root/Python-2.7.8 --with-libs=/usr/local/lib --disable-ipv6 However, I get an error while compiling. make altinstall gcc -pthread -c -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I. -IInclude -I./Include -DPy_BUILD_CORE -o Modules/python.o ./Modules/python.c In file included from Include/Python.h:58, from ./Modules/python.c:3: Include/pyport.h:256:13: error: #error "This platform's pyconfig.h needs to define PY_FORMAT_LONG_LONG" make: *** [Modules/python.o] Error 1 I CAN compile without zlib, but then pip gives an error. python2.7 get-pip.py Traceback (most recent call last): File "get-pip.py", line 19017, in main() File "get-pip.py", line 194, in main bootstrap(tmpdir=tmpdir) File "get-pip.py", line 82, in bootstrap import pip zipimport.ZipImportError: can't decompress data; zlib not available I'd use findRPM but that seems to be 2.7.8, not 2.7.11, and it seems reasonable, if I'm building this, to build the most recent version. Any ideas? I have web searched this; I found a bug that was closed in 2014, and I just got all new source *right now*. I'm doing a very vanilla install on Oracle Linux Server release 6.7 Thank you, == John == From nimbiotics at gmail.com Mon Feb 15 23:46:18 2016 From: nimbiotics at gmail.com (Mario R. Osorio) Date: Mon, 15 Feb 2016 20:46:18 -0800 (PST) Subject: Make a unique filesystem path, without creating the file In-Reply-To: References: Message-ID: I would create a RAM disk (http://www.cyberciti.biz/faq/howto-create-linux-ram-disk-filesystem/), generate all the path/files I want with any, or my own algorithm, run the tests, unmount it, destroy it, be happy ... Whats wrong with that?? AFAIK, RAM disks do not get logged, and even if they do, any "insecure" file created would also be gone. On Sunday, February 14, 2016 at 4:46:42 PM UTC-5, Ben Finney wrote: > Howdy all, > > How should a program generate a unique filesystem path and *not* create > the filesystem entry? > > The 'tempfile.mktemp' function is strongly deprecated, and rightly so > > because it leaves the program vulnerable to insecure file creation. > > In some code (e.g. unit tests) I am calling 'tempfile.mktemp' to > generate a unique path for a filesystem entry that I *do not want* to > exist on the real filesystem. In this case the filesystem security > concerns are irrelevant because there is no file. > > The deprecation of that function is a concern still, because I don't > want code that makes every conscientious reader need to decide whether > the code is a problem. Instead the code should avoid rightly-deprecated > APIs. > > It is also prone to that API function disappearing at some point in the > future, because it is explicitly and strongly deprecated. > > So I agree with the deprecation, but the library doesn't appear to > provide a replacement. > > What standard library function should I be using to generate > 'tempfile.mktemp'-like unique paths, and *not* ever create a real file > by that path? > > -- > \ "If you have the facts on your side, pound the facts. If you | > `\ have the law on your side, pound the law. If you have neither | > _o__) on your side, pound the table." --anonymous | > Ben Finney From ben+python at benfinney.id.au Tue Feb 16 00:03:47 2016 From: ben+python at benfinney.id.au (Ben Finney) Date: Tue, 16 Feb 2016 16:03:47 +1100 Subject: Make a unique filesystem path, without creating the file References: Message-ID: <85h9h94570.fsf@benfinney.id.au> "Mario R. Osorio" writes: > I would create a RAM disk > (http://www.cyberciti.biz/faq/howto-create-linux-ram-disk-filesystem/), > generate all the path/files I want with any, or my own algorithm, run > the tests, unmount it, destroy it, be happy ... Whats wrong with > that?? It is addressing the problem at a different level. I am not asking about writing a wrapper around the test suite, I am asking about an API to generate filesystem paths. Your solution is a fine response to a different question. -- \ ?Consider the daffodil. And while you're doing that, I'll be | `\ over here, looking through your stuff.? ?Jack Handey | _o__) | Ben Finney From john_ladasky at sbcglobal.net Tue Feb 16 00:24:13 2016 From: john_ladasky at sbcglobal.net (John Ladasky) Date: Mon, 15 Feb 2016 21:24:13 -0800 (PST) Subject: [STORY-TIME] THE BDFL AND HIS PYTHON PETTING ZOO In-Reply-To: <032f0f66-34df-4325-98e0-1127025d2a94@googlegroups.com> References: <48762040-a7e0-434c-92e8-8a0969210e5b@googlegroups.com> <5d31dd14-1a60-4f80-8889-d0616f404c70@googlegroups.com> <4e902c58-3370-4adf-905d-74630a6eea90@googlegroups.com> <60036746-78f7-4f05-a6d8-04e88b09c00f@googlegroups.com> <032f0f66-34df-4325-98e0-1127025d2a94@googlegroups.com> Message-ID: <31a618b2-a407-4723-9a4f-9b756fc93b0b@googlegroups.com> On Monday, February 15, 2016 at 6:02:24 PM UTC-8, Rick Johnson wrote: > On Friday, February 12, 2016 at 1:51:35 AM UTC-6, John Ladasky wrote: > > Reguarding a migration from Python2 to Pyhton3, John said: > > I had to wait until my favorite packages were ported > > (numpy, scipy, matplotlib, pandas). > > WxPython is not ported either, much to my chagrin. I was a big fan of WxPython myself. I waited for Phoenix for a while, then decided to learn PyQt5. Having been exposed to both GUI's ways of doing things, I'm not sure which I prefer. I never liked Tkinter. > > But once that happened, I moved from Py2 to Py3 years ago > > with scarcely a bump, bruise, or scratch. > > So you have no Python2.x code remaining in your repos? Are > you telling us that you moved *EVERYTHING* to Python3? If > so, i can't imagine how something like that would even be > possible, hmm, unless of course, you don't have much code to > move...? > > I could imagine that *IF* someone's entire Python repo > consisted of a couple hundred (or less) small scripts, with > few, or no, dependencies, such a migration would be > relatively easy. But even though i've only been writing > Python code for a handful of years, my repo consists of > thousands of scripts, millions of lines of code, and many, > *MANY* dependencies -- because after all, DRY is very > important, yes? It's true, I only have about 25,000 lines of code. My largest program suite is only about 10% of that. And indeed, it's now all in Py3. The 2to3 utility took care of most of my porting needs. Yes, I aspire to the DRY principle. I've written a few packages which extend Numpy's behavior in specific ways which I've re-used many times. I acknowledge that there's a vocal group of Py2 users who have decided that it's too much trouble to switch. The PSF has decided that its energies are better expended on growing Py3 than on maintaining Py2 for that diminishing user base. But no one owns Python. There is no law that states that Py2 adherents cannot maintain the language themselves, if they want it. Sheesh. Just do it. Make it yours. Or make something better if you want. You state: > I don't need Python3. And i reckon that by the time i do, > something more interesting will come along, or, i'll create > something more interesting myself. i've been drafting, and > dreaming, of a new language spec for over a year now.. And > the best thing about starting a new language, you can do > anything you want... no dependencies! If you have all the skills that you claim, you're a far better programmer than I. So -- exactly why are you complaining to people who are developing and/or using Py3? Go off and impress everyone. Become your own BDFL. Cheers. From wij at totalbb.net.tw Tue Feb 16 00:49:52 2016 From: wij at totalbb.net.tw (wij at totalbb.net.tw) Date: Mon, 15 Feb 2016 21:49:52 -0800 (PST) Subject: Project Xnod is created Message-ID: <96598158-628a-4377-be4c-626db4f90bc6@googlegroups.com> I am happy to announce that project Xnod is created. https://sourceforge.net/projects/systemnode/ Xnod tries to develop a more uniform language or scaffolding, or kind of program deployment to reduce efforts for nearly every programming tasks. Xnod is currently based on Xarg, and has to be a collabrated project. So, consider join this project in this initial stage helping ourselves by sharing your know-how. From rosuav at gmail.com Tue Feb 16 00:51:18 2016 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 16 Feb 2016 16:51:18 +1100 Subject: asyncio - run coroutine in the background In-Reply-To: <87h9ha8lt0.fsf@jester.gateway.pace.com> References: <8737sumpjl.fsf@elektro.pacujo.net> <87h9ha8lt0.fsf@jester.gateway.pace.com> Message-ID: On Mon, Feb 15, 2016 at 6:39 PM, Paul Rubin wrote: > "Frank Millman" writes: >> The benefit of my class is that it enables me to take the coroutine >> and run it in another thread, without having to re-engineer the whole >> thing. > > Threads in Python don't get you parallelism either, of course. > They can. The only limitation is that, in CPython (and some others), no two threads can concurrently be executing Python byte-code. The instant you drop into a C-implemented function, it can release the GIL and let another thread start running. Obviously this happens any time there's going to be a blocking API call (eg if one thread waits on a socket read, others can run), but it can also happen with computational work: import numpy import threading def thread1(): arr = numpy.zeros(100000000, dtype=numpy.int64) while True: print("1: %d" % arr[0]) arr += 1 arr = (arr * arr) % 142957 def thread2(): arr = numpy.zeros(100000000, dtype=numpy.int64) while True: print("2: %d" % arr[0]) arr += 2 arr = (arr * arr) % 142957 threading.Thread(target=thread1).start() thread2() This will happily keep two CPU cores occupied. Most of the work is being done inside Numpy, which releases the GIL before doing any work. So it's not strictly true that threading can't parallelise Python code (and as mentioned, it depends on your interpreter - Jython can, I believe, do true multithreading), but just that there are limitations on what can execute concurrently. ChrisA From ben+python at benfinney.id.au Tue Feb 16 00:56:06 2016 From: ben+python at benfinney.id.au (Ben Finney) Date: Tue, 16 Feb 2016 16:56:06 +1100 Subject: Make a unique filesystem path, without creating the file References: <85egce6a8a.fsf@benfinney.id.au> <20160215230539.GA42677@cskk.homeip.net> Message-ID: <851t8d42rt.fsf@benfinney.id.au> Cameron Simpson writes: > I've been watching this for a few days, and am struggling to > understand your use case. Yes, you're not alone. This surprises me, which is why I'm persisting. > Can you elaborate with a concrete example and its purpose which would > work with a mktemp-ish official function? An example:: import io import tempfile names = tempfile._get_candidate_names() def test_frobnicates_configured_spungfile(): """ ?foo? should frobnicate the configured spungfile. """ fake_file_path = os.path.join(tempfile.gettempdir(), names.next()) fake_file = io.BytesIO("Lorem ipsum, dolor sit amet".encode("utf-8")) patch_builtins_open( when_accessing_path=fake_file_path, provide_file=fake_file) system_under_test.config.spungfile_path = fake_file_path system_under_test.foo() assert_correctly_frobnicated(fake_file) So the test case creates a fake file, makes a valid filesystem path to associate with it, then patches the ?open? function so that it will return the fake file when that specific path is requested. Then the test case alters the system under test's configuration, giving it the generated filesystem path for an important file. The test case then calls the function about which the unit test is asserting behaviour, ?system_under_test.foo?. When that call returns, the test case asserts some properties of the fake file to ensure the system under test actually accessed that file. With a supported standard library API for this ? ?tempfile.makepath? for example ? the generation of the filesystem path would change from four separate function calls, one of which is a private API:: names = tempfile._get_candidate_names() fake_file_path = os.path.join(tempfile.gettempdir(), names.next()) to a simple public function call:: fake_file_path = tempfile.makepath() This whole thread began because I expected such an API would exist. > I don't see how it is useful to have a notion of a filepath at all > in this case, and therefore I don't see why you would want a > mktemp-like function available. Because the system under test expects to be dealing with a filesystem, including normal restrictions on filesystem paths. The filesystem path needs to be valid because the test case isn't making assertions about what the system does with invalid paths. A test case should be very narrow in what it asserts so that the failure's cause is as obvious as possible. The filesystem path needs to be unpredictable to make sure we're not using some hard-coded value; the test case asserts that the system under test will access whatever file is named in the configuration. The file object needs to be fake because the test case should not be prone to irrelevant failures when the real filesystem isn't behaving as expected; this test case makes assertions only about what ?system_under_test.foo? does internally, not what the filesystem does. The system library functionality should be providing this because it's *already implemented there* and well tested and maintained. It should be in a public non-deprecated API because merely generating filesystem paths is not a security risk. > But.. then why a filesystem path at all in that case? Because the system under test is expecting valid filesystem paths, and I have no good reason to violate that constraint. > Why use a filesystem as a reference at all? An actual running filesystem is irrelevant to this inquiry. I'm only wanting to use functionality, with the constraints I enumerated earlier (already implemented in the standard library), to generate filesystem paths. > The only modes I can imagine for such a thing (a generated but unused > filename) are: > > checking that the name is syntactly valid, for whatever constrains > you may have (but if you're calling an opaque mktemp-like function, is > this feasible or remediable?) Almost. I want the filesystem paths to be valid because the system under test expects them, it may perform its own validation, and I have no good reason to complicate the unit test by possibly supplying an invalid path when that's not relevant to the test case. > generating test paths without using a real filesystem as a reference, > but then you can't even use mktemp I hadn't realised the filesystem was accessed by ?tempfile.mktemp?, and I apologise for the complication that entails. I would prefer to access some standard public documented non-deprecated function that internally uses ?tempfile._get_candidate_names? and returns a new path each time. > I think "the standard library clearly has this useful functionality > implemented, but simultaneously warns strongly against its use" pretty > much precludes this. I hope to get that addressed with . -- \ ?Timid men prefer the calm of despotism to the boisterous sea | `\ of liberty.? ?Thomas Jefferson | _o__) | Ben Finney From steve+comp.lang.python at pearwood.info Tue Feb 16 01:43:41 2016 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Tue, 16 Feb 2016 17:43:41 +1100 Subject: repr( open('/etc/motd', 'rt').read() ) References: Message-ID: <56c2c520$0$2741$c3e8da3$76491128@news.astraweb.com> On Tuesday 16 February 2016 00:05, Veek. M wrote: > When I do at the interpreter prompt, > repr( open('/etc/motd', 'rt').read() ) Opening and reading MOTD is a needless distraction from your actual question. This demonstrates the issue most simply: # at the interactive interpreter py> s = "text\n" py> s # evaluate the string s 'text\n' py> repr(s) "'text\\n'" Evaluating the string `s` in the REPL (Read-Eval-Print Loop) displays the repr() of the string. Contrast that to using print directly: py> print s # note the blank line text py> print repr(s) 'text\n' So when you call print on a string, the string is printed in the most accurate way possible. When you call repr() on a string, it returns a new string containing the Python representation of a string. So the string: text plus newline (but without the indent) has a representation in Python of: 'text\n' so repr("text\n") == "'text\n'" If you call repr() twice, you get this string: py> print repr(repr("text\n")) "'text\\n'" That should look familiar: py> repr(s) "'text\\n'" So when you evaluate a string on its own, the REPL prints the repr() of the string. So if you evaluate repr(s), you see repr(repr(s)) printed. > With # 2 # read returns a string that the interpreter displays by > calling __str__ via print so newlines are converted to \n. No, that's incorrect. The REPL uses the repr() of the string. We can test this with a class that makes the difference between __str__ and __repr__ more obvious: py> class C(object): ... def __repr__(self): ... return "the repr" ... def __str__(self): ... return "the str" ... py> c = C() py> print(c) the str py> c the repr -- Steve From framstag at rus.uni-stuttgart.de Tue Feb 16 03:30:59 2016 From: framstag at rus.uni-stuttgart.de (Ulli Horlacher) Date: Tue, 16 Feb 2016 08:30:59 +0000 (UTC) Subject: extending PATH on Windows? Message-ID: I need to extend the PATH environment variable on Windows. So far, I use: system('setx PATH "%PATH%;'+bindir+'"') The problem: In a new process (cmd.exe) PATH contains a lot of double elements. As far as I have understood, Windows builds the PATH environment variable from a system component and a user component. With the setx command from above I have copied the system PATH into the user PATH component. Is there a better way to extend the PATH environment variable for the user? It must be persistent, not only for the current process. -- Ullrich Horlacher Server und Virtualisierung Rechenzentrum IZUS/TIK E-Mail: horlacher at tik.uni-stuttgart.de Universitaet Stuttgart Tel: ++49-711-68565868 Allmandring 30a Fax: ++49-711-682357 70550 Stuttgart (Germany) WWW: http://www.tik.uni-stuttgart.de/ From oscar.j.benjamin at gmail.com Tue Feb 16 03:35:40 2016 From: oscar.j.benjamin at gmail.com (Oscar Benjamin) Date: Tue, 16 Feb 2016 08:35:40 +0000 Subject: Make a unique filesystem path, without creating the file In-Reply-To: <851t8d42rt.fsf@benfinney.id.au> References: <85egce6a8a.fsf@benfinney.id.au> <20160215230539.GA42677@cskk.homeip.net> <851t8d42rt.fsf@benfinney.id.au> Message-ID: On 16 Feb 2016 05:57, "Ben Finney" wrote: > > Cameron Simpson writes: > > > I've been watching this for a few days, and am struggling to > > understand your use case. > > Yes, you're not alone. This surprises me, which is why I'm persisting. > > > Can you elaborate with a concrete example and its purpose which would > > work with a mktemp-ish official function? > > An example:: > > import io > import tempfile > names = tempfile._get_candidate_names() > > def test_frobnicates_configured_spungfile(): > """ ?foo? should frobnicate the configured spungfile. """ > > fake_file_path = os.path.join(tempfile.gettempdir(), names.next()) > fake_file = io.BytesIO("Lorem ipsum, dolor sit amet".encode("utf-8")) > > patch_builtins_open( > when_accessing_path=fake_file_path, > provide_file=fake_file) > > system_under_test.config.spungfile_path = fake_file_path > system_under_test.foo() > assert_correctly_frobnicated(fake_file) If you're going to patch open to return a fake file when asked to open fake_file_path why do you care whether there is a real file of that name? -- Oscar From jfong at ms4.hinet.net Tue Feb 16 03:39:22 2016 From: jfong at ms4.hinet.net (jfong at ms4.hinet.net) Date: Tue, 16 Feb 2016 00:39:22 -0800 (PST) Subject: Will file be closed automatically in a "for ... in open..." statement? Message-ID: I know with open('foo.txt') as f: ...do something... will close the file automatically when the "with" block ends. I also saw codes in a book: for line in open('foo.txt'): ...do something... but it didn't mention if the file will be closed automatically or not when the "for" block ends. Is there any document talking about this? and how to know if a file is in "open" or not? --Jach Fong From rosuav at gmail.com Tue Feb 16 04:16:01 2016 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 16 Feb 2016 20:16:01 +1100 Subject: Will file be closed automatically in a "for ... in open..." statement? In-Reply-To: References: Message-ID: On Tue, Feb 16, 2016 at 7:39 PM, wrote: > I know > > with open('foo.txt') as f: > ...do something... > > will close the file automatically when the "with" block ends. > > I also saw codes in a book: > > for line in open('foo.txt'): > ...do something... > > but it didn't mention if the file will be closed automatically or not when the "for" block ends. Is there any document talking about this? and how to know if a file is in "open" or not? > The file will be closed when the open file object is disposed of. That will happen at some point after there are no more references to it. You're guaranteed that it stays around for the entire duration of the 'for' loop (the loop keeps track of the thing it's iterating over), but exactly when after that is not guaranteed. In current versions of CPython, the garbage collector counts references, so the file will be closed immediately; but other Python interpreters, and future versions of CPython, may not behave the same way. So the file will *probably* be closed *reasonably* promptly, but unlike the "with" case, you have no guarantee that it'll be immediate. For small scripts, it probably won't even matter, though. You're unlikely to run out of file handles, and the only time it would matter is if you're opening, closing, and then reopening the file - for example: fn = input("Name a file to frobnosticate: ") with open(fn) as f: data = [] for line in f: data = frobnosticate(data, line) with open(fn, "w") as f: f.writelines(data) For this to work reliably, the file MUST be closed for reading before it's opened for writing. The context managers are important. But this is pretty unusual. Of course, since it's so little trouble to use the 'with' block, it's generally worth just using it everywhere. Why run the risk? :) ChrisA who often forgets to use 'with' anyway From cs at zip.com.au Tue Feb 16 04:24:05 2016 From: cs at zip.com.au (Cameron Simpson) Date: Tue, 16 Feb 2016 20:24:05 +1100 Subject: Will file be closed automatically in a "for ... in open..." statement? In-Reply-To: References: Message-ID: <20160216092405.GA8877@cskk.homeip.net> On 16Feb2016 00:39, jfong at ms4.hinet.net wrote: >I know > > with open('foo.txt') as f: > ...do something... > >will close the file automatically when the "with" block ends. Yes, because open is a context manager - they're great for reliably tidying up in the face of exceptions or "direct" departure from the block, such as a "return" statement. >I also saw codes in a book: > > for line in open('foo.txt'): > ...do something... > >but it didn't mention if the file will be closed automatically or not when the >"for" block ends. Is there any document talking about this? and how to know if >a file is in "open" or not? This does not reliably close the file. In CPython (the common implementation, and likely what you are using), objects are reference counted and when the interpreter notices their counter go to zero, the object's __del__ method is called before releasing the object's memory. For an open file, __del__ _does_ call close if the file is open. However, only reference counting Pythons will call __del__ promptly - other systems rely on garbage collectors to detect unused objects. In the for loop above, for interpreter obtains an iterator from the open file which returns lines of text. That iterator has a reference to the open file, and the for loop has a reference to the iterator. Therefore the file remains references while the loop runs. AT the end of the loop the iterator is discarded, reducing its references to zero. That in turn triggers releasing the open file, dropping its references to zero. In CPython, that in turn will fire the open file's __del__, which will close the file. In other Pythons, not necessarily that promptly. Also, there are plenty of ways to phrase this where the file reference doesn't go to zero. FOr example: f = open(...) for line in f: ... You can see that it is easy to forget to close the file here (especially if you have an exception or exit the function precipitously). Try to use the "with open(...) as f:" formulation when possible. It is much better. Cheers, Cameron Simpson From tjreedy at udel.edu Tue Feb 16 04:41:56 2016 From: tjreedy at udel.edu (Terry Reedy) Date: Tue, 16 Feb 2016 04:41:56 -0500 Subject: Will file be closed automatically in a "for ... in open..." statement? In-Reply-To: References: Message-ID: On 2/16/2016 3:39 AM, jfong at ms4.hinet.net wrote: > I know > > with open('foo.txt') as f: > ...do something... > > will close the file automatically when the "with" block ends. > > I also saw codes in a book: > > for line in open('foo.txt'): > ...do something... Some books were originally written before 'with context_manager' was added, or a little later, before it became the normal thing to do. -- Terry Jan Reedy From thorsten at thorstenkampe.de Tue Feb 16 07:19:32 2016 From: thorsten at thorstenkampe.de (Thorsten Kampe) Date: Tue, 16 Feb 2016 13:19:32 +0100 Subject: extending PATH on Windows? References: Message-ID: * Ulli Horlacher (Tue, 16 Feb 2016 08:30:59 +0000 (UTC)) > I need to extend the PATH environment variable on Windows. > > So far, I use: > > system('setx PATH "%PATH%;'+bindir+'"') > > The problem: In a new process (cmd.exe) PATH contains a lot of double > elements. As far as I have understood, Windows builds the PATH > environment variable from a system component and a user component. With > the setx command from above I have copied the system PATH into the user > PATH component. > > Is there a better way to extend the PATH environment variable for the user? > It must be persistent, not only for the current process. `os.system` should be `subprocess.call` on modern systems (actually `subprocess.run` if you have Python 3.5). Since `setx` doesn't seem to have unique and add options, you basically have two options: 1. Add the path component yourself into HKEY_CURRENT_USER and make sure it's not there already (pure Python). 2. a) use a shell that offers that capability with `set`: https://jpsoft.com/help/set.htm (TCC/LE is free) b) use a dedicated environment variable editor: http://www.rapidee.com/en/command-line Thorsten From framstag at rus.uni-stuttgart.de Tue Feb 16 07:38:44 2016 From: framstag at rus.uni-stuttgart.de (Ulli Horlacher) Date: Tue, 16 Feb 2016 12:38:44 +0000 (UTC) Subject: extending PATH on Windows? References: Message-ID: Thorsten Kampe wrote: > * Ulli Horlacher (Tue, 16 Feb 2016 08:30:59 +0000 (UTC)) > > I need to extend the PATH environment variable on Windows. > > 1. Add the path component yourself into HKEY_CURRENT_USER and make > sure it's not there already (pure Python). Preferred! What is HKEY_CURRENT_USER? Another environment variable? -- Ullrich Horlacher Server und Virtualisierung Rechenzentrum IZUS/TIK E-Mail: horlacher at tik.uni-stuttgart.de Universitaet Stuttgart Tel: ++49-711-68565868 Allmandring 30a Fax: ++49-711-682357 70550 Stuttgart (Germany) WWW: http://www.tik.uni-stuttgart.de/ From arjuns123 at gmail.com Tue Feb 16 07:41:57 2016 From: arjuns123 at gmail.com (Arjun Srivatsa) Date: Tue, 16 Feb 2016 04:41:57 -0800 (PST) Subject: Unable to insert data into MongoDB. In-Reply-To: References: <8bb40e3f-57ce-4d5a-a426-6b56cebc8de7@googlegroups.com> Message-ID: <147512d7-ace9-4944-be2f-3eecb0ad525f@googlegroups.com> Thanks a lot. Will implement that. Although I am able to do using just 2 scripts as well. On Monday, February 15, 2016 at 5:34:08 PM UTC+1, Peter Otten wrote: > Arjun Srivatsa wrote: > > > Hi Peter. > > > > Thank you for the reply. > > > > This is the read_server code: > > > > import socket > > from pymongo import MongoClient > > #import datetime > > import sys > > > > # Connection to server (PLC) on port 27017 > > host = "10.52.124.135" > > port = 27017 > > > > s = socket.socket(socket.AF_INET, socket.SOCK_STREAM) > > s.connect((host, port)) > > sys.stdout.write(s.recv(1024)) > > > > > > And the write_db code: > > > > from pymongo import MongoClient > > import datetime > > import socket > > import sys > > > > client = MongoClient('mongodb://localhost:27017/') > > db = client.test_database > > > > mongodoc = { "data": 'data', "date" : datetime.datetime.utcnow() } > > values = db.values > > values_id = values.insert_one(mongodoc).inserted_id > > > > > > So, both these scripts work independently. While, read_server shows the > > output of the actual data from PLC, write_db inserts the sample data into > > the MongoDB. > > > > I am not sure as to how to combine these both and get the desired output. > > What I mean is once you have working scripts > > connect_to_mongodb() > while True: > record = make_fake_data() > insert_record_into_mongodb(record) > > and > > connect_to_server() > while True: > record = read_record_from_server() > print(record) > > you can combine the code in a third script to > > connect_to_server() > connect_to_mongodb() > while True: > record = read_record_from_server() > insert_record_into_mongodb(record) > > and be fairly sure that the combination works, too. From mr.eightnoteight at gmail.com Tue Feb 16 07:46:45 2016 From: mr.eightnoteight at gmail.com (srinivas devaki) Date: Tue, 16 Feb 2016 18:16:45 +0530 Subject: Multiple Assignment a = b = c Message-ID: Hi, a = b = c as an assignment doesn't return anything, i ruled out a = b = c as chained assignment, like a = (b = c) SO i thought, a = b = c is resolved as a, b = [c, c] at-least i fixed in my mind that every assignment like operation in python is done with references and then the references are binded to the named variables. like globals()['a'] = result() but today i learned that this is not the case with great pain(7 hours of debugging.) class Mytest(object): def __init__(self, a): self.a = a def __getitem__(self, k): print('__getitem__', k) return self.a[k] def __setitem__(self, k, v): print('__setitem__', k, v) self.a[k] = v roots = Mytest([0, 1, 2, 3, 4, 5, 6, 7, 8]) a = 4 roots[4] = 6 a = roots[a] = roots[roots[a]] the above program's output is __setitem__ 4 6 __getitem__ 4 __getitem__ 6 __setitem__ 6 6 But the output that i expected is __setitem__ 4 6 __getitem__ 4 __getitem__ 6 __setitem__ 4 6 SO isn't it counter intuitive from all other python operations. like how we teach on how python performs a swap operation??? I just want to get a better idea around this. -- Regards Srinivas Devaki Junior (3rd yr) student at Indian School of Mines,(IIT Dhanbad) Computer Science and Engineering Department ph: +91 9491 383 249 telegram_id: @eightnoteight From srkunze at mail.de Tue Feb 16 08:05:34 2016 From: srkunze at mail.de (Sven R. Kunze) Date: Tue, 16 Feb 2016 14:05:34 +0100 Subject: Multiple Assignment a = b = c In-Reply-To: References: Message-ID: <56C31E9E.1060609@mail.de> Hi Srinivas, On 16.02.2016 13:46, srinivas devaki wrote: > Hi, > > a = b = c > > as an assignment doesn't return anything, i ruled out a = b = c as > chained assignment, like a = (b = c) > SO i thought, a = b = c is resolved as > a, b = [c, c] > > > at-least i fixed in my mind that every assignment like operation in > python is done with references and then the references are binded to > the named variables. > like globals()['a'] = result() > > but today i learned that this is not the case with great pain(7 hours > of debugging.) > > class Mytest(object): > def __init__(self, a): > self.a = a > def __getitem__(self, k): > print('__getitem__', k) > return self.a[k] > def __setitem__(self, k, v): > print('__setitem__', k, v) > self.a[k] = v > > roots = Mytest([0, 1, 2, 3, 4, 5, 6, 7, 8]) > a = 4 > roots[4] = 6 > a = roots[a] = roots[roots[a]] > > > the above program's output is > __setitem__ 4 6 > __getitem__ 4 > __getitem__ 6 > __setitem__ 6 6 > > > But the output that i expected is > __setitem__ 4 6 > __getitem__ 4 > __getitem__ 6 > __setitem__ 4 6 > > SO isn't it counter intuitive from all other python operations. > like how we teach on how python performs a swap operation??? > > I just want to get a better idea around this. I think the tuple assignment you showed basically nails it. First, the rhs is evaluated. Second, the lhs is evaluated from left to right. Completely wrong? Best, Sven From srkunze at mail.de Tue Feb 16 08:10:26 2016 From: srkunze at mail.de (Sven R. Kunze) Date: Tue, 16 Feb 2016 14:10:26 +0100 Subject: Multiple Assignment a = b = c In-Reply-To: <56C31E9E.1060609@mail.de> References: <56C31E9E.1060609@mail.de> Message-ID: <56C31FC2.2030206@mail.de> On 16.02.2016 14:05, Sven R. Kunze wrote: > Hi Srinivas, > > I think the tuple assignment you showed basically nails it. > > First, the rhs is evaluated. > Second, the lhs is evaluated from left to right. > > Completely wrong? > > Best, > Sven As you mentioned swapping. The following two statements do the same (as you suggested at the beginning). a,b=b,a=4,5 (a,b),(b,a)=(4,5),(4,5) Best, Sven From eryksun at gmail.com Tue Feb 16 08:17:24 2016 From: eryksun at gmail.com (eryk sun) Date: Tue, 16 Feb 2016 07:17:24 -0600 Subject: extending PATH on Windows? In-Reply-To: References: Message-ID: On Tue, Feb 16, 2016 at 2:30 AM, Ulli Horlacher wrote: > > So far, I use: > > system('setx PATH "%PATH%;'+bindir+'"') > > The problem: In a new process (cmd.exe) PATH contains a lot of double > elements. As far as I have understood, Windows builds the PATH > environment variable from a system component and a user component. With > the setx command from above I have copied the system PATH into the user > PATH component. setx broadcasts a WM_SETTINGCHANGE [1] message that notifies Explorer to reload its environment from the registry, so the user doesn't have to start a new session. It also decides whether to use REG_SZ or REG_EXPAND_SZ depending on the presence of mutliple "%" characters in the string. [1]: https://msdn.microsoft.com/en-us/library/ms725497 But as you note it's no good for extending an existing value, especially not for PATH or a value that references other "%variables%" that you want to remain unexpanded. To do this right, you have to at least use winreg to query the user's PATH value from the registry. But then you may as well replace setx completely. Here's a little something to get you started. import os import sys import types import ctypes user32 = ctypes.WinDLL('user32', use_last_error=True) try: import winreg except ImportError: import _winreg as winreg def extend_path(new_paths, persist=True): if isinstance(new_paths, getattr(types, 'StringTypes', str)): new_paths = [new_paths] new_paths = [os.path.abspath(p) for p in new_paths] paths = [p for p in os.environ.get('PATH', '').split(os.pathsep) if p] for p in new_paths: if p not in paths: paths.append(p) os.environ['PATH'] = os.pathsep.join(paths) if persist: _persist_path(new_paths) def _persist_path(new_paths): if sys.version_info[0] == 2: temp_paths = [] for p in new_paths: if isinstance(p, unicode): temp_paths.append(p) else: temp_paths.append(p.decode('mbcs')) new_paths = temp_paths with winreg.OpenKey(winreg.HKEY_CURRENT_USER, 'Environment', 0, winreg.KEY_QUERY_VALUE | winreg.KEY_SET_VALUE) as hkey: try: user_path, dtype = winreg.QueryValueEx(hkey, 'PATH') except WindowsError as e: ERROR_FILE_NOT_FOUND = 0x0002 if e.winerror != ERROR_FILE_NOT_FOUND: raise paths = [] else: if dtype in (winreg.REG_SZ, winreg.REG_EXPAND_SZ): paths = [p for p in user_path.split(os.pathsep) if p] else: paths = [] for p in new_paths: if p not in paths: paths.append(p) pathstr = os.pathsep.join(paths) if pathstr.count('%') < 2: dtype = winreg.REG_SZ else: dtype = winreg.REG_EXPAND_SZ winreg.SetValueEx(hkey, 'PATH', 0, dtype, pathstr) _broadcast_change(u'Environment') def _broadcast_change(lparam): HWND_BROADCAST = 0xFFFF WM_SETTINGCHANGE = 0x001A SMTO_ABORTIFHUNG = 0x0002 ERROR_TIMEOUT = 0x05B4 wparam = 0 if not user32.SendMessageTimeoutW( HWND_BROADCAST, WM_SETTINGCHANGE, wparam, ctypes.c_wchar_p(lparam), SMTO_ABORTIFHUNG, 1000, None): err = ctypes.get_last_error() if err != ERROR_TIMEOUT: raise ctypes.WinError(err) From kevinjacobconway at gmail.com Tue Feb 16 08:22:36 2016 From: kevinjacobconway at gmail.com (Kevin Conway) Date: Tue, 16 Feb 2016 13:22:36 +0000 Subject: asyncio - run coroutine in the background In-Reply-To: References: <8737sumpjl.fsf@elektro.pacujo.net> <87h9ha8lt0.fsf@jester.gateway.pace.com> Message-ID: If you're handling coroutines there is an asyncio facility for "background tasks". The ensure_future [1] will take a coroutine, attach it to a Task, and return a future to you that resolves when the coroutine is complete. The coroutine you schedule with that function will not cause your current coroutine to wait unless you await the future it returns. [1] https://docs.python.org/3/library/asyncio-task.html#asyncio.ensure_future On Mon, Feb 15, 2016, 23:53 Chris Angelico wrote: > On Mon, Feb 15, 2016 at 6:39 PM, Paul Rubin > wrote: > > "Frank Millman" writes: > >> The benefit of my class is that it enables me to take the coroutine > >> and run it in another thread, without having to re-engineer the whole > >> thing. > > > > Threads in Python don't get you parallelism either, of course. > > > > They can. The only limitation is that, in CPython (and some others), > no two threads can concurrently be executing Python byte-code. The > instant you drop into a C-implemented function, it can release the GIL > and let another thread start running. Obviously this happens any time > there's going to be a blocking API call (eg if one thread waits on a > socket read, others can run), but it can also happen with > computational work: > > import numpy > import threading > > def thread1(): > arr = numpy.zeros(100000000, dtype=numpy.int64) > while True: > print("1: %d" % arr[0]) > arr += 1 > arr = (arr * arr) % 142957 > > def thread2(): > arr = numpy.zeros(100000000, dtype=numpy.int64) > while True: > print("2: %d" % arr[0]) > arr += 2 > arr = (arr * arr) % 142957 > > threading.Thread(target=thread1).start() > thread2() > > This will happily keep two CPU cores occupied. Most of the work is > being done inside Numpy, which releases the GIL before doing any work. > So it's not strictly true that threading can't parallelise Python code > (and as mentioned, it depends on your interpreter - Jython can, I > believe, do true multithreading), but just that there are limitations > on what can execute concurrently. > > ChrisA > -- > https://mail.python.org/mailman/listinfo/python-list > From thorsten at thorstenkampe.de Tue Feb 16 08:38:42 2016 From: thorsten at thorstenkampe.de (Thorsten Kampe) Date: Tue, 16 Feb 2016 14:38:42 +0100 Subject: extending PATH on Windows? References: Message-ID: * Ulli Horlacher (Tue, 16 Feb 2016 12:38:44 +0000 (UTC)) > > Thorsten Kampe wrote: > > * Ulli Horlacher (Tue, 16 Feb 2016 08:30:59 +0000 (UTC)) > > > I need to extend the PATH environment variable on Windows. > > > > 1. Add the path component yourself into HKEY_CURRENT_USER and make > > sure it's not there already (pure Python). > > Preferred! > What is HKEY_CURRENT_USER? Another environment variable? It's a hive in the Windows registry and the equivalent of `~/.*` in Linux terms (HKEY_LOCAL_MACHINE[/Software] being the equivalent of `/etc`). The fact that you're asking indicates that you should read about that in advance. The task itself is definitely not that hard. Maybe someone has already asked at StackOverflow. But the devil's in the detail. Some things to consider - Python is not by default installed on Windows, so you have to use a way to run your script without (PyInstaller for instance). - by default there is no entry in HKCU, so you have to create it first (under HKEY_CURRENT_USER\Environment). - you need to create the correct type (REG_SZ, null-terminated string) - Windows paths are semicolon separated (not colon). - Windows only module for the Registry: https://docs.python.org/3/library/winreg.html Thorsten From thorsten at thorstenkampe.de Tue Feb 16 08:43:54 2016 From: thorsten at thorstenkampe.de (Thorsten Kampe) Date: Tue, 16 Feb 2016 14:43:54 +0100 Subject: extending PATH on Windows? References: Message-ID: * Ulli Horlacher (Tue, 16 Feb 2016 12:38:44 +0000 (UTC)) By the way: there is a script called `win_add2path.py` in your Python distribution which "is a simple script to add Python to the Windows search path. It modifies the current user (HKCU) tree of the registry.". That should do most of what you want. Thorsten From frank at chagford.com Tue Feb 16 08:52:01 2016 From: frank at chagford.com (Frank Millman) Date: Tue, 16 Feb 2016 15:52:01 +0200 Subject: asyncio - run coroutine in the background In-Reply-To: References: <8737sumpjl.fsf@elektro.pacujo.net> <87h9ha8lt0.fsf@jester.gateway.pace.com> Message-ID: "Kevin Conway" wrote in message news:CAKF=+dim8wzPRvm86_V2W5-XSopCcHvgm0Hy8r4xeHDYzy_P4w at mail.gmail.com... > > If you're handling coroutines there is an asyncio facility for "background > tasks". The ensure_future [1] will take a coroutine, attach it to a Task, > and return a future to you that resolves when the coroutine is complete. > The coroutine you schedule with that function will not cause your current > coroutine to wait unless you await the future it returns. > > [1] > https://docs.python.org/3/library/asyncio-task.html#asyncio.ensure_future > Thank you Kevin! That works perfectly, and is much neater than my effort. Frank From marko at pacujo.net Tue Feb 16 09:17:01 2016 From: marko at pacujo.net (Marko Rauhamaa) Date: Tue, 16 Feb 2016 16:17:01 +0200 Subject: asyncio - run coroutine in the background References: <8737sumpjl.fsf@elektro.pacujo.net> <87h9ha8lt0.fsf@jester.gateway.pace.com> Message-ID: <87d1rwpwo2.fsf@elektro.pacujo.net> Kevin Conway : > If you're handling coroutines there is an asyncio facility for > "background tasks". The ensure_future [1] will take a coroutine, > attach it to a Task, and return a future to you that resolves when the > coroutine is complete. Ok, yes, but those "background tasks" monopolize the CPU once they are scheduled to run. If your "background task" doesn't need a long time to run, just call the function in the foreground and be done with it. If it does consume time, you need to delegate it to a separate process so the other tasks remain responsive. Marko From frank at chagford.com Tue Feb 16 09:36:00 2016 From: frank at chagford.com (Frank Millman) Date: Tue, 16 Feb 2016 16:36:00 +0200 Subject: asyncio - run coroutine in the background In-Reply-To: <87d1rwpwo2.fsf@elektro.pacujo.net> References: <8737sumpjl.fsf@elektro.pacujo.net> <87h9ha8lt0.fsf@jester.gateway.pace.com> <87d1rwpwo2.fsf@elektro.pacujo.net> Message-ID: "Marko Rauhamaa" wrote in message news:87d1rwpwo2.fsf at elektro.pacujo.net... > > Kevin Conway : > > > If you're handling coroutines there is an asyncio facility for > > "background tasks". The ensure_future [1] will take a coroutine, > > attach it to a Task, and return a future to you that resolves when the > > coroutine is complete. > > Ok, yes, but those "background tasks" monopolize the CPU once they are > scheduled to run. > > If your "background task" doesn't need a long time to run, just call the > function in the foreground and be done with it. If it does consume time, > you need to delegate it to a separate process so the other tasks remain > responsive. > I will explain my situation - perhaps you can tell me if it makes sense. My background task does take a long time to run - about 10 seconds - but most of that time is spent waiting for database responses, which is handled in another thread. You could argue that the database thread should rather be handled by another process, and that is definitely an option if I find that response times are affected. So far my response times have been very good, even with database activity in the background. However, I have not simulated a large number of concurrent users. That could throw up the kinds of problem that you are concerned about. Frank From kevinjacobconway at gmail.com Tue Feb 16 09:54:04 2016 From: kevinjacobconway at gmail.com (Kevin Conway) Date: Tue, 16 Feb 2016 14:54:04 +0000 Subject: asyncio - run coroutine in the background In-Reply-To: References: <8737sumpjl.fsf@elektro.pacujo.net> <87h9ha8lt0.fsf@jester.gateway.pace.com> <87d1rwpwo2.fsf@elektro.pacujo.net> Message-ID: > Ok, yes, but those "background tasks" monopolize the CPU once they are scheduled to run. This is true if the coroutines are cpu bound. If that is the case then a coroutine is likely the wrong choice for that code to begin with. Coroutines, in asyncio land, are primarily designed for io bound work. > My background task does take a long time to run - about 10 seconds - but most of that time is spent waiting for database responses, which is handled in another thread. Something else to look into is an asyncio driver for your database connections. Threads aren't inherently harmful, but using them to achieve async networking when running asyncio is a definite code smell since that is precisely the problem asyncio is supposed to solve for. On Tue, Feb 16, 2016, 08:37 Frank Millman wrote: > "Marko Rauhamaa" wrote in message news:87d1rwpwo2.fsf at elektro.pacujo.net. > .. > > > > Kevin Conway : > > > > > If you're handling coroutines there is an asyncio facility for > > > "background tasks". The ensure_future [1] will take a coroutine, > > > attach it to a Task, and return a future to you that resolves when the > > > coroutine is complete. > > > > Ok, yes, but those "background tasks" monopolize the CPU once they are > > scheduled to run. > > > > If your "background task" doesn't need a long time to run, just call the > > function in the foreground and be done with it. If it does consume time, > > you need to delegate it to a separate process so the other tasks remain > > responsive. > > > > I will explain my situation - perhaps you can tell me if it makes sense. > > My background task does take a long time to run - about 10 seconds - but > most of that time is spent waiting for database responses, which is handled > in another thread. > > You could argue that the database thread should rather be handled by > another > process, and that is definitely an option if I find that response times are > affected. > > So far my response times have been very good, even with database activity > in > the background. However, I have not simulated a large number of concurrent > users. That could throw up the kinds of problem that you are concerned > about. > > Frank > > > -- > https://mail.python.org/mailman/listinfo/python-list > From steve at pearwood.info Tue Feb 16 10:16:23 2016 From: steve at pearwood.info (Steven D'Aprano) Date: Wed, 17 Feb 2016 02:16:23 +1100 Subject: Multiple Assignment a = b = c References: Message-ID: <56c33d48$0$1587$c3e8da3$5496439d@news.astraweb.com> On Tue, 16 Feb 2016 11:46 pm, srinivas devaki wrote: > Hi, > > a = b = c > > as an assignment doesn't return anything, i ruled out a = b = c as > chained assignment, like a = (b = c) > SO i thought, a = b = c is resolved as > a, b = [c, c] That is one way of thinking of it. A better way would be: a = c b = c except that isn't necessarily correct for complex assignments involving attribute access or item assignment. A better way is: _temp = c a = _temp b = _temp del _temp except the name "_temp" isn't actually used. > at-least i fixed in my mind that every assignment like operation in > python is done with references and then the references are binded to > the named variables. > like globals()['a'] = result() That's broadly correct. > but today i learned that this is not the case with great pain(7 hours > of debugging.) > > class Mytest(object): > def __init__(self, a): > self.a = a > def __getitem__(self, k): > print('__getitem__', k) > return self.a[k] > def __setitem__(self, k, v): > print('__setitem__', k, v) > self.a[k] = v > > roots = Mytest([0, 1, 2, 3, 4, 5, 6, 7, 8]) > a = 4 > roots[4] = 6 > a = roots[a] = roots[roots[a]] `roots[4] = 6` will give "__setitem__ 4 6", as you expect. On the right hand side, you have: roots[roots[a]] which evaluates `roots[a]` first, giving "__getitem__ 4". That returns 6, as you expect. So now you have `roots[6]`, which gives "__getitem__ 6", as you expect, and returns 6. The left hand side has: a = roots[a] = ... which becomes: a = roots[a] = 6 which behaves like: a = 6 roots[a] = 6 So you end up with: a = roots[6] = 6 which gives "__setitem__ 6 6", **not** "__setitem__ 4 6" like you expected. Here is a simpler demonstration: py> L = [0, 1, 2, 3, 4, 5, 6] py> a = L[a//100] = 500 py> print a 500 py> print L [0, 1, 2, 3, 4, 500, 6] Let's look at the byte-code generated by the statement: a = L[a] = x The exact byte-code used will depend on the version of Python you have, but for 2.7 it looks like this: py> from dis import dis py> code = compile("a = L[a] = x", "", "exec") py> dis(code) 1 0 LOAD_NAME 0 (x) 3 DUP_TOP 4 STORE_NAME 1 (a) 7 LOAD_NAME 2 (L) 10 LOAD_NAME 1 (a) 13 STORE_SUBSCR 14 LOAD_CONST 0 (None) 17 RETURN_VALUE Translated to English: - evaluate the expression `x` and push the result onto the stack; - duplicate the top value on the stack; - pop the top value off the stack and assign to name `a`; - evaluate the name `L`, and push the result onto the stack; - evaluate the name `a`, and push the result onto the stack; - call setattr with the top three items from the stack. > SO isn't it counter intuitive from all other python operations. > like how we teach on how python performs a swap operation??? No. Let's look at the equivalent swap: py> L = [10, 20, 30, 40, 50] py> a = 3 py> a, L[a] = L[a], a Traceback (most recent call last): File "", line 1, in IndexError: list assignment index out of range This is equivalent to: _temp1 = L[a] # 40 pushed onto the stack _temp2 = a # 3 pushed onto the stack a = _temp1 # 40 # rotate the stack, and pull the top item 40 L[a] = _temp2 # L[40] = 3 which obviously fails. Here's the byte-code: py> code = compile("a, L[a] = L[a], a", "", "exec") py> dis(code) 1 0 LOAD_NAME 0 (L) 3 LOAD_NAME 1 (a) 6 BINARY_SUBSCR 7 LOAD_NAME 1 (a) 10 ROT_TWO 11 STORE_NAME 1 (a) 14 LOAD_NAME 0 (L) 17 LOAD_NAME 1 (a) 20 STORE_SUBSCR 21 LOAD_CONST 0 (None) 24 RETURN_VALUE If you do the swap in the other order, it works: py> L = [10, 20, 30, 40, 50] py> a = 3 py> L[a], a = a, L[a] py> print a 40 py> print L [10, 20, 30, 3, 50] In all cases, the same rule applies: - evaluate the right hand side from left-most to right-most, pushing the values onto the stack; - perform assignments on the left hand side, from left-most to right-most. -- Steven From steve at pearwood.info Tue Feb 16 10:17:17 2016 From: steve at pearwood.info (Steven D'Aprano) Date: Wed, 17 Feb 2016 02:17:17 +1100 Subject: asyncio - run coroutine in the background References: <8737sumpjl.fsf@elektro.pacujo.net> <87h9ha8lt0.fsf@jester.gateway.pace.com> <87d1rwpwo2.fsf@elektro.pacujo.net> Message-ID: <56c33d7e$0$1587$c3e8da3$5496439d@news.astraweb.com> On Wed, 17 Feb 2016 01:17 am, Marko Rauhamaa wrote: > Ok, yes, but those "background tasks" monopolize the CPU once they are > scheduled to run. Can you show some code demonstrating this? -- Steven From frank at chagford.com Tue Feb 16 10:21:04 2016 From: frank at chagford.com (Frank Millman) Date: Tue, 16 Feb 2016 17:21:04 +0200 Subject: asyncio - run coroutine in the background In-Reply-To: References: <8737sumpjl.fsf@elektro.pacujo.net> <87h9ha8lt0.fsf@jester.gateway.pace.com> <87d1rwpwo2.fsf@elektro.pacujo.net> Message-ID: "Kevin Conway" wrote in message news:CAKF=+dhXZ=yax8STAWr_gjX3Tg8yUjPRJG-7yM2_bRV2Kxm3Jg at mail.gmail.com... > > > My background task does take a long time to run - about 10 seconds - but > > most of that time is spent waiting for database responses, which is > > handled > > in another thread. > > Something else to look into is an asyncio driver for your database > connections. Threads aren't inherently harmful, but using them to achieve > async networking when running asyncio is a definite code smell since that > is precisely the problem asyncio is supposed to solve for. > Maybe I have not explained very well. I am not using threads to achieve async networking. I am using asyncio in a client server environment, and it works very well. If a client request involves a database query, I use a thread to perform that so that it does not slow down the other users. I usually want the originating client to block until I have a response, so I use 'await'. However, occasionally the request takes some time, and it is not necessary for the client to wait for the response, so I want to unblock the client straight away, run the task in the background, and then notify the client when the task is complete. This is where your suggestion of 'ensure_future' does the job perfectly. I would love to drive the database asynchronously, but of the three databases I use, only psycopg2 seems to have asyncio support. As my home-grown solution (using queues) seems to be working well so far, I am sticking with that until I start to experience responsiveness issues. If that happens, my first line of attack will be to switch from threads to processes. Hope this makes sense. Frank From rosuav at gmail.com Tue Feb 16 10:28:46 2016 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 17 Feb 2016 02:28:46 +1100 Subject: asyncio - run coroutine in the background In-Reply-To: References: <8737sumpjl.fsf@elektro.pacujo.net> <87h9ha8lt0.fsf@jester.gateway.pace.com> <87d1rwpwo2.fsf@elektro.pacujo.net> Message-ID: On Wed, Feb 17, 2016 at 2:21 AM, Frank Millman wrote: > I would love to drive the database asynchronously, but of the three > databases I use, only psycopg2 seems to have asyncio support. As my > home-grown solution (using queues) seems to be working well so far, I am > sticking with that until I start to experience responsiveness issues. If > that happens, my first line of attack will be to switch from threads to > processes. And this is where we demonstrate divergent thought processes. *My* first line of attack if hybrid async/thread doesn't work would be to mandate a PostgreSQL backend, not to switch to hybrid async/process :) Is the added value of "you get three options of database back-end" worth the added cost of "but now my code is massively more complex"? ChrisA From frank at chagford.com Tue Feb 16 10:45:00 2016 From: frank at chagford.com (Frank Millman) Date: Tue, 16 Feb 2016 17:45:00 +0200 Subject: asyncio - run coroutine in the background In-Reply-To: References: <8737sumpjl.fsf@elektro.pacujo.net> <87h9ha8lt0.fsf@jester.gateway.pace.com> <87d1rwpwo2.fsf@elektro.pacujo.net> Message-ID: "Chris Angelico" wrote in message news:CAPTjJmqMiE4gROqNYVHwAhCn2mWQeYQxt5KvFivotrHQY-sNdw at mail.gmail.com... > > On Wed, Feb 17, 2016 at 2:21 AM, Frank Millman wrote: > > I would love to drive the database asynchronously, but of the three > > databases I use, only psycopg2 seems to have asyncio support. As my > > home-grown solution (using queues) seems to be working well so far, I am > > sticking with that until I start to experience responsiveness issues. If > > that happens, my first line of attack will be to switch from threads to > > processes. > > And this is where we demonstrate divergent thought processes. *My* > first line of attack if hybrid async/thread doesn't work would be to > mandate a PostgreSQL backend, not to switch to hybrid async/process :) > Is the added value of "you get three options of database back-end" > worth the added cost of "but now my code is massively more complex"? Then we will have to agree to diverge ;-) If I ever get my app off the ground, it will be an all-purpose, multi-company, multi-currency, multi-everything accounting/business system. There is a massive market out there, and a large percentage of that is Microsoft-only shops. I have no intention of cutting myself off from that market before I even start. I am very happy with my choice of 3 databases - 1. sqlite3 - ideal for demo purposes and for one-man businesses 2. Sql Server for those that insist on it 3. PostgreSQL for every one else, and my recommendation if asked Frank From mr.eightnoteight at gmail.com Tue Feb 16 11:00:21 2016 From: mr.eightnoteight at gmail.com (srinivas devaki) Date: Tue, 16 Feb 2016 21:30:21 +0530 Subject: Multiple Assignment a = b = c In-Reply-To: <56c33d48$0$1587$c3e8da3$5496439d@news.astraweb.com> References: <56c33d48$0$1587$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Tue, Feb 16, 2016 at 6:35 PM, Sven R. Kunze wrote: > > First, the rhs is evaluated. > Second, the lhs is evaluated from left to right. Great, I will remember these two lines :) On Tue, Feb 16, 2016 at 8:46 PM, Steven D'Aprano wrote: > _temp = c > a = _temp > b = _temp > del _temp > > > except the name "_temp" isn't actually used. > So it is like first right most expression is evaluated and then lhs is evaluated from left to right. > py> from dis import dis > py> code = compile("a = L[a] = x", "", "exec") > py> dis(code) > 1 0 LOAD_NAME 0 (x) > 3 DUP_TOP > 4 STORE_NAME 1 (a) > 7 LOAD_NAME 2 (L) > 10 LOAD_NAME 1 (a) > 13 STORE_SUBSCR > 14 LOAD_CONST 0 (None) > 17 RETURN_VALUE > > > Translated to English: > > - evaluate the expression `x` and push the result onto the stack; > > - duplicate the top value on the stack; > > - pop the top value off the stack and assign to name `a`; > > - evaluate the name `L`, and push the result onto the stack; > > - evaluate the name `a`, and push the result onto the stack; > > - call setattr with the top three items from the stack. > thank-you so much, for explaining how to find the underlying details. >> SO isn't it counter intuitive from all other python operations. >> like how we teach on how python performs a swap operation??? > > No. Let's look at the equivalent swap: > > In all cases, the same rule applies: > > - evaluate the right hand side from left-most to right-most, pushing the > values onto the stack; > > - perform assignments on the left hand side, from left-most to right-most. > uhh, i related it with swap because I was thinking variables are binded, like first of all for all lhs assignments get their references or names and then put the value of rhs in them. as `a` is a name, so the rhs reference is copied to the a `roots[a]` is a reference to an object, so it is initialized with the reference of rhs. anyway I got it, and all my further doubts are cleared from that compiled code. I tried some other examples and understood how it works. thanks a lot. -- Regards Srinivas Devaki Junior (3rd yr) student at Indian School of Mines,(IIT Dhanbad) Computer Science and Engineering Department ph: +91 9491 383 249 telegram_id: @eightnoteight From theohamilton at gmail.com Tue Feb 16 11:19:19 2016 From: theohamilton at gmail.com (Theo Hamilton) Date: Tue, 16 Feb 2016 16:19:19 +0000 Subject: [Glitch?] Python has just stopped working Message-ID: I woke up two days ago to find out that python literally won't work any more. I have looked everywhere, asked multiple Stack Overflow questions, and am ready to give up. Whenever I run python (3.5), I get the following message: Fatal Python error: Py_initialize: unable to load the file system codec ImportError: No module named 'encodings' Current thread 0x00002168 (most recent call first): If there's anything you know that I could do to fix this, then please tell me. I've tried uninstalling and reparing, so it's not those. Thanks! From marko at pacujo.net Tue Feb 16 12:12:12 2016 From: marko at pacujo.net (Marko Rauhamaa) Date: Tue, 16 Feb 2016 19:12:12 +0200 Subject: asyncio - run coroutine in the background References: <8737sumpjl.fsf@elektro.pacujo.net> <87h9ha8lt0.fsf@jester.gateway.pace.com> <87d1rwpwo2.fsf@elektro.pacujo.net> <56c33d7e$0$1587$c3e8da3$5496439d@news.astraweb.com> Message-ID: <87vb5ok2ab.fsf@elektro.pacujo.net> Steven D'Aprano : > On Wed, 17 Feb 2016 01:17 am, Marko Rauhamaa wrote: > >> Ok, yes, but those "background tasks" monopolize the CPU once they >> are scheduled to run. > > Can you show some code demonstrating this? Sure: ======================================================================== #!/usr/bin/env python3 import asyncio, time def main(): asyncio.get_event_loop().run_until_complete(asyncio.wait([ background_task(), looping_task() ])) @asyncio.coroutine def looping_task(): while True: yield from asyncio.sleep(1) print(int(time.time())) @asyncio.coroutine def background_task(): yield from asyncio.sleep(4) t = time.time() while time.time() - t < 10: pass if __name__ == '__main__': main() ======================================================================== which prints: 1455642629 1455642630 1455642631 1455642642 <============== gap 1455642643 1455642644 1455642645 Marko From marko at pacujo.net Tue Feb 16 12:13:05 2016 From: marko at pacujo.net (Marko Rauhamaa) Date: Tue, 16 Feb 2016 19:13:05 +0200 Subject: asyncio - run coroutine in the background References: <8737sumpjl.fsf@elektro.pacujo.net> <87h9ha8lt0.fsf@jester.gateway.pace.com> <87d1rwpwo2.fsf@elektro.pacujo.net> <56c33d7e$0$1587$c3e8da3$5496439d@news.astraweb.com> Message-ID: <87r3gck28u.fsf@elektro.pacujo.net> Steven D'Aprano : > On Wed, 17 Feb 2016 01:17 am, Marko Rauhamaa wrote: > >> Ok, yes, but those "background tasks" monopolize the CPU once they are >> scheduled to run. > > Can you show some code demonstrating this? Sure: ======================================================================== #!/usr/bin/env python3 import asyncio, time def main(): asyncio.get_event_loop().run_until_complete(asyncio.wait([ background_task(), looping_task() ])) @asyncio.coroutine def looping_task(): while True: yield from asyncio.sleep(1) print(int(time.time())) @asyncio.coroutine def background_task(): yield from asyncio.sleep(4) t = time.time() while time.time() - t < 10: pass if __name__ == '__main__': main() ======================================================================== which prints: 1455642629 1455642630 1455642631 1455642642 <============== gap 1455642643 1455642644 1455642645 Marko From marko at pacujo.net Tue Feb 16 12:14:18 2016 From: marko at pacujo.net (Marko Rauhamaa) Date: Tue, 16 Feb 2016 19:14:18 +0200 Subject: asyncio - run coroutine in the background References: <8737sumpjl.fsf@elektro.pacujo.net> <87h9ha8lt0.fsf@jester.gateway.pace.com> <87d1rwpwo2.fsf@elektro.pacujo.net> <56c33d7e$0$1587$c3e8da3$5496439d@news.astraweb.com> Message-ID: <87mvr0k26t.fsf@elektro.pacujo.net> Steven D'Aprano : > On Wed, 17 Feb 2016 01:17 am, Marko Rauhamaa wrote: > >> Ok, yes, but those "background tasks" monopolize the CPU once they are >> scheduled to run. > > Can you show some code demonstrating this? Sure: ======================================================================== #!/usr/bin/env python3 import asyncio, time def main(): asyncio.get_event_loop().run_until_complete(asyncio.wait([ background_task(), looping_task() ])) @asyncio.coroutine def looping_task(): while True: yield from asyncio.sleep(1) print(int(time.time())) @asyncio.coroutine def background_task(): yield from asyncio.sleep(4) t = time.time() while time.time() - t < 10: pass if __name__ == '__main__': main() ======================================================================== which prints: 1455642629 1455642630 1455642631 1455642642 <============== gap 1455642643 1455642644 1455642645 Marko From marko at pacujo.net Tue Feb 16 12:15:01 2016 From: marko at pacujo.net (Marko Rauhamaa) Date: Tue, 16 Feb 2016 19:15:01 +0200 Subject: asyncio - run coroutine in the background References: <8737sumpjl.fsf@elektro.pacujo.net> <87h9ha8lt0.fsf@jester.gateway.pace.com> <87d1rwpwo2.fsf@elektro.pacujo.net> <56c33d7e$0$1587$c3e8da3$5496439d@news.astraweb.com> Message-ID: <87io1ok25m.fsf@elektro.pacujo.net> Steven D'Aprano : > On Wed, 17 Feb 2016 01:17 am, Marko Rauhamaa wrote: > >> Ok, yes, but those "background tasks" monopolize the CPU once they are >> scheduled to run. > > Can you show some code demonstrating this? Sure: ======================================================================== #!/usr/bin/env python3 import asyncio, time def main(): asyncio.get_event_loop().run_until_complete(asyncio.wait([ background_task(), looping_task() ])) @asyncio.coroutine def looping_task(): while True: yield from asyncio.sleep(1) print(int(time.time())) @asyncio.coroutine def background_task(): yield from asyncio.sleep(4) t = time.time() while time.time() - t < 10: pass if __name__ == '__main__': main() ======================================================================== which prints: 1455642629 1455642630 1455642631 1455642642 1455642643 1455642644 1455642645 Marko From marko at pacujo.net Tue Feb 16 12:15:33 2016 From: marko at pacujo.net (Marko Rauhamaa) Date: Tue, 16 Feb 2016 19:15:33 +0200 Subject: asyncio - run coroutine in the background References: <8737sumpjl.fsf@elektro.pacujo.net> <87h9ha8lt0.fsf@jester.gateway.pace.com> <87d1rwpwo2.fsf@elektro.pacujo.net> <56c33d7e$0$1587$c3e8da3$5496439d@news.astraweb.com> <87io1ok25m.fsf@elektro.pacujo.net> Message-ID: <87egcck24q.fsf@elektro.pacujo.net> Marko Rauhamaa : > Sure: Sorry for the multiple copies. Marko From marko at pacujo.net Tue Feb 16 12:20:13 2016 From: marko at pacujo.net (Marko Rauhamaa) Date: Tue, 16 Feb 2016 19:20:13 +0200 Subject: asyncio - run coroutine in the background References: <8737sumpjl.fsf@elektro.pacujo.net> <87h9ha8lt0.fsf@jester.gateway.pace.com> <87d1rwpwo2.fsf@elektro.pacujo.net> Message-ID: <87r3gc608i.fsf@elektro.pacujo.net> "Frank Millman" : > I would love to drive the database asynchronously, but of the three > databases I use, only psycopg2 seems to have asyncio support. Yes, asyncio is at its infancy. There needs to be a moratorium on blocking I/O. Marko From steve at pearwood.info Tue Feb 16 12:36:50 2016 From: steve at pearwood.info (Steven D'Aprano) Date: Wed, 17 Feb 2016 04:36:50 +1100 Subject: Make a unique filesystem path, without creating the file References: <85egce6a8a.fsf@benfinney.id.au> <20160215230539.GA42677@cskk.homeip.net> Message-ID: <56c35e34$0$1600$c3e8da3$5496439d@news.astraweb.com> On Tue, 16 Feb 2016 04:56 pm, Ben Finney wrote: > An example:: > > import io > import tempfile > names = tempfile._get_candidate_names() I'm not sure that calling a private function of the tempfile module is better than calling a deprecated function. > def test_frobnicates_configured_spungfile(): > """ ?foo? should frobnicate the configured spungfile. """ > > fake_file_path = os.path.join(tempfile.gettempdir(), names.next()) At this point, you have a valid pathname, but no guarantee whether it refers to a real file on the file system or not. That's the whole problem with tempfile.makepath -- it can return a file name which is not in use, but by the time it returns to you, you cannot guarantee that it still doesn't exist. Now, since this is a test which doesn't actually open that file, it doesn't matter. There's no actual security vulnerability here. So your test doesn't actually require that the file is unique, or that it doesn't actually exist. (Which is good, because you can't guarantee that it doesn't exist.) So why not just pick a random bunch of characters? chars = list(string.ascii_letters) random.shuffle(chars) fake_file_path = ''.join(chars[:10]) > fake_file = io.BytesIO("Lorem ipsum, dolor sit > amet".encode("utf-8")) > > patch_builtins_open( > when_accessing_path=fake_file_path, > provide_file=fake_file) There's nothing apparent in this that requires that fake_file_path not actually exist, which is good since (as I've pointed out before) you cannot guarantee that it doesn't exist. One could just as easily, and just as correctly, write: patch_builtins_open( when_accessing_path='/foo/bar/baz', provide_file=fake_file) and regardless of whether /foo/bar/baz actually exists or not, you are guaranteed to get the fake file rather than the real file. So I question whether you actually need this tempfile.makepath function at all. *But* having questioned it, for the sake of the argument I'll assume you do need it, and continue accordingly. > system_under_test.config.spungfile_path = fake_file_path > system_under_test.foo() > assert_correctly_frobnicated(fake_file) > > So the test case creates a fake file, makes a valid filesystem path to > associate with it, then patches the ?open? function so that it will > return the fake file when that specific path is requested. > > Then the test case alters the system under test's configuration, giving > it the generated filesystem path for an important file. The test case > then calls the function about which the unit test is asserting > behaviour, ?system_under_test.foo?. When that call returns, the test > case asserts some properties of the fake file to ensure the system under > test actually accessed that file. Personally, I think it would be simpler and easier to understand if, instead of patching open, you allowed the test to read and write real files: file_path = '/tmp/spam' system_under_test.config.spungfile_path = file_path system_under_test.foo() assert_correctly_frobnicated(file_path) os.unlink(file_path) In practice, I'd want to only unlike the file if the test passes. If it fails, I'd want to look at the file to see why it wasn't frobnicated. I think that a correctly-working filesystem is a perfectly reasonable prerequisite for the test, just like a working CPU, memory, power supply, operating system and Python interpreter. You don't have to guard against every imaginable failure ("fixme: test may return invalid results if the speed of light changes by more than 0.0001%"), and you might as well take advantage of real files for debugging. But that's my opinion, and if you have another, that's your personal choice. > With a supported standard library API for this ? ?tempfile.makepath? for > example ? the generation of the filesystem path would change from four > separate function calls, one of which is a private API:: > > names = tempfile._get_candidate_names() > fake_file_path = os.path.join(tempfile.gettempdir(), names.next()) > > to a simple public function call:: > > fake_file_path = tempfile.makepath() Nobody doubts that your use of tempfile.makepath is legitimate for your use-case. But it is *not* legitimate for the tempfile module, and it is a mistake that it was added in the first place, hence the deprecation. Assuming that your test suite needs this function, your test library, or test suite, should provide that function, not tempfile. I believe it is unreasonable to expect the tempfile module to keep a function which is a security risk in the context of "temp files" just because it is useful for some completely unrelated use-cases. After all, your use of this doesn't actually have anything to do with temporary files. It is a mocked *permanent* file, not a real temporary one. > This whole thread began because I expected such an API would exist. > > >> I don't see how it is useful to have a notion of a filepath at all >> in this case, and therefore I don't see why you would want a >> mktemp-like function available. > > Because the system under test expects to be dealing with a filesystem, > including normal restrictions on filesystem paths. Yes, but the system doesn't try to enforce the filesystem's rules, does it? Apart from simple restrictions like "path must be a string", I wouldn't expect your system to make rules like: - file names must be 8 characters followed by a dot followed by 3 characters; - paths must not contain ASCII nulls; etc. That's for the file system to enforce. So you don't really know ahead of time what "normal restrictions" exist. (Are newlines allowed in filenames? ASCII null bytes? limited to 2**16 path components?) So your test can completely ignore any and all restrictions, and your monkeypatched open() will be perfectly happy to deal with them: fake_file_path = "" # Empty string. patch_builtins_open( when_accessing_path=fake_file_path, provide_file=fake_file) and your system shouldn't care. (Well, perhaps it will reject the empty string as a path. But it shouldn't reject anything else.) > The file object needs to be fake because the test case should not be > prone to irrelevant failures when the real filesystem isn't behaving as > expected; this test case makes assertions only about what > ?system_under_test.foo? does internally, not what the filesystem does. Since you're monkeypatching open(), the patched open() can define any path it likes as "fake". Admittedly it might be a bit scary to see things like: open("/dev/sda", "w").write("pwned!!!") in the test suite, but it's perfectly safe (assuming your test harness is working correctly to patch open). >> But.. then why a filesystem path at all in that case? > > Because the system under test is expecting valid filesystem paths, and I > have no good reason to violate that constraint. Since your test doesn't know what filesystem your code will be running on, you can't make any assumptions about what paths are valid or not valid. > Almost. I want the filesystem paths to be valid because the system under > test expects them, it may perform its own validation, If the system tries to validate paths, it is broken. That's how you get broken applications that insist that all file names must be located in C:\\My Documents. The application should allow the file system to validate paths. -- Steven From robin at reportlab.com Tue Feb 16 12:52:36 2016 From: robin at reportlab.com (Robin Becker) Date: Tue, 16 Feb 2016 17:52:36 +0000 Subject: asyncio - run coroutine in the background In-Reply-To: <87egcck24q.fsf@elektro.pacujo.net> References: <8737sumpjl.fsf@elektro.pacujo.net> <87h9ha8lt0.fsf@jester.gateway.pace.com> <87d1rwpwo2.fsf@elektro.pacujo.net> <56c33d7e$0$1587$c3e8da3$5496439d@news.astraweb.com> <87io1ok25m.fsf@elektro.pacujo.net> <87egcck24q.fsf@elektro.pacujo.net> Message-ID: <56C361E4.9030509@chamonix.reportlab.co.uk> On 16/02/2016 17:15, Marko Rauhamaa wrote: > Marko Rauhamaa : > >> Sure: > > Sorry for the multiple copies. > > > Marko > I thought perhaps background jobs were sending them :) -- Robin Becker From tjreedy at udel.edu Tue Feb 16 14:31:00 2016 From: tjreedy at udel.edu (Terry Reedy) Date: Tue, 16 Feb 2016 14:31:00 -0500 Subject: Multiple Assignment a = b = c In-Reply-To: References: Message-ID: On 2/16/2016 7:46 AM, srinivas devaki wrote: > Hi, > > a = b = c > > as an assignment doesn't return anything, i ruled out a = b = c as > chained assignment, like a = (b = c) > SO i thought, a = b = c is resolved as > a, b = [c, c] https://docs.python.org/3/reference/simple_stmts.html#assignment-statements "An assignment statement evaluates the expression list (remember that this can be a single expression or a comma-separated list, the latter yielding a tuple) and assigns the single resulting object to each of the target lists, from left to right." a = b = c is the same as tem = c; a = tem; b = tem. This does not work if tem is an iterator. >>> def g(): yield 1 yield 2 >>> a,b = c,d = g() Traceback (most recent call last): File "", line 1, in a,b = c,d = g() ValueError: not enough values to unpack (expected 2, got 0) >>> a, b (1, 2) >>> c,d Traceback (most recent call last): File "", line 1, in c,d NameError: name 'c' is not defined > at-least i fixed in my mind that every assignment like operation in > python is done with references and then the references are binded to > the named variables. > like globals()['a'] = result() > > but today i learned that this is not the case with great pain(7 hours > of debugging.) > > class Mytest(object): > def __init__(self, a): > self.a = a > def __getitem__(self, k): > print('__getitem__', k) > return self.a[k] > def __setitem__(self, k, v): > print('__setitem__', k, v) > self.a[k] = v > > roots = Mytest([0, 1, 2, 3, 4, 5, 6, 7, 8]) > a = 4 > roots[4] = 6 > a = roots[a] = roots[roots[a]] tem = roots[roots[a]] a = tem roots[a] = tem > the above program's output is > __setitem__ 4 6 > __getitem__ 4 > __getitem__ 6 > __setitem__ 6 6 -- Terry Jan Reedy From ben+python at benfinney.id.au Tue Feb 16 14:40:20 2016 From: ben+python at benfinney.id.au (Ben Finney) Date: Wed, 17 Feb 2016 06:40:20 +1100 Subject: Make a unique filesystem path, without creating the file References: <85egce6a8a.fsf@benfinney.id.au> <20160215230539.GA42677@cskk.homeip.net> <851t8d42rt.fsf@benfinney.id.au> Message-ID: <85twl830m3.fsf@benfinney.id.au> Oscar Benjamin writes: > If you're going to patch open to return a fake file when asked to open > fake_file_path why do you care whether there is a real file of that > name? I don't, and have been saying explicitly many times in this thread that I do not care whether the file exists. Somehow that is still not clear? -- \ ?Nothing exists except atoms and empty space; everything else | `\ is opinion.? ?Democritus, c. 460 BCE ? 370 BCE | _o__) | Ben Finney From ben+python at benfinney.id.au Tue Feb 16 14:48:14 2016 From: ben+python at benfinney.id.au (Ben Finney) Date: Wed, 17 Feb 2016 06:48:14 +1100 Subject: Make a unique filesystem path, without creating the file References: <85egce6a8a.fsf@benfinney.id.au> <20160215230539.GA42677@cskk.homeip.net> <56c35e34$0$1600$c3e8da3$5496439d@news.astraweb.com> Message-ID: <85povw308x.fsf@benfinney.id.au> Steven D'Aprano writes: > On Tue, 16 Feb 2016 04:56 pm, Ben Finney wrote: > > > names = tempfile._get_candidate_names() > > I'm not sure that calling a private function of the tempfile module is > better than calling a deprecated function. Agreed, which is why I'm seeking a public API that is not deprecated. > So why not just pick a random bunch of characters? > > chars = list(string.ascii_letters) > random.shuffle(chars) > fake_file_path = ''.join(chars[:10]) This (an equivalent) is already implemented, internally to ?tempfile? and tested and maintained and more robust than me re-inventing the wheel. > Yes, but the system doesn't try to enforce the filesystem's rules, > does it? The test case I'm writing should not be prone to failure if the system happens to perform some arbitrary validation of filesystem paths. ?tempfile? already knows how to generate filesystem paths, I want to use that and not have to get it right myself. > and your system shouldn't care. If it does, this test case should not fail. > Since your test doesn't know what filesystem your code will be running > on, you can't make any assumptions about what paths are valid or not > valid. That implies that ?tempfile._get_candidate_names? would generate paths that would potentially be invalid. Is that what you intend to imply? > > Almost. I want the filesystem paths to be valid because the system > > under test expects them, it may perform its own validation, > > If the system tries to validate paths, it is broken. This is ?you don't want what you say you want?, and seeing the justifications presented I don't agree. -- \ ?I must say that I find television very educational. The minute | `\ somebody turns it on, I go to the library and read a book.? | _o__) ?Groucho Marx | Ben Finney From imcchoudhury at outlook.com Tue Feb 16 14:55:52 2016 From: imcchoudhury at outlook.com (Chinmaya Choudhury) Date: Wed, 17 Feb 2016 01:25:52 +0530 Subject: I m facing some problem while opening the interpreter. how can I resolve the issue? Message-ID: Please guide me. #Chinmay Sent from Mail for Windows 10 From nikos.gr33k at gmail.com Tue Feb 16 15:02:18 2016 From: nikos.gr33k at gmail.com (=?ISO-8859-7?B?1v7t9OHyIMvh5O/w8eHq/PDv9evv8g==?=) Date: Tue, 16 Feb 2016 12:02:18 -0800 (PST) Subject: Weird and sparese cgi:error Message-ID: <813cd989-bc60-4d24-96ef-f20244d91362@googlegroups.com> Hello, I recentely changed VPS server and when i try to load my webiste iam receiving 500 error which i wasnt receiving in my old VPS server with the same exact cgi scripts. After looking at Apacher's error_log iam seeing the following output when i try to load scripts from cgi-bin directory. [Tue Feb 16 21:28:55.970302 2016] [cgi:error] [pid 17478] [client 180.76.15.28:21311] End of script output before headers: metrites.py [Tue Feb 16 21:29:31.193281 2016] [cgi:error] [pid 18802] [client 180.76.15.29:34155] End of script output before headers: metrites.py [Tue Feb 16 21:32:21.306627 2016] [cgi:error] [pid 17478] [client 188.138.124.110:40212] End of script output before headers: metrites.py [Tue Feb 16 21:37:21.128472 2016] [cgi:error] [pid 19731] [client 204.152.200.42:54376] End of script output before headers: metrites.py [Tue Feb 16 21:42:21.198207 2016] [cgi:error] [pid 17478] [client 184.75.210.226:57252] End of script output before headers: metrites.py [Tue Feb 16 21:47:21.228361 2016] [cgi:error] [pid 19731] [client 76.72.172.208:34730] End of script output before headers: metrites.py [Tue Feb 16 21:52:21.422066 2016] [cgi:error] [pid 18802] [client 85.17.156.11:52071] End of script output before headers: metrites.py Please i need your assistance to this. thank you.Prob From auriocus at gmx.de Tue Feb 16 15:13:29 2016 From: auriocus at gmx.de (Christian Gollwitzer) Date: Tue, 16 Feb 2016 21:13:29 +0100 Subject: [Glitch?] Python has just stopped working In-Reply-To: References: Message-ID: Am 16.02.16 um 17:19 schrieb Theo Hamilton: > I woke up two days ago to find out that python literally won't work any > more. I have looked everywhere, asked multiple Stack Overflow questions, > and am ready to give up. Whenever I run python (3.5), I get the following > message: > > Fatal Python error: Py_initialize: unable to load the file system codec > ImportError: No module named 'encodings' Can it be that you have just set a strange locale? What happens if you run it as LANG=C python ? Christian From breamoreboy at yahoo.co.uk Tue Feb 16 15:15:30 2016 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Tue, 16 Feb 2016 20:15:30 +0000 Subject: I m facing some problem while opening the interpreter. how can I resolve the issue? In-Reply-To: References: Message-ID: On 16/02/2016 19:55, Chinmaya Choudhury wrote: > Please guide me. > #Chinmay > > Sent from Mail for Windows 10 > Please read http://catb.org/~esr/faqs/smart-questions.html and possibly http://www.sscce.org/, then try asking again. -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From sonulaldeepak at gmail.com Tue Feb 16 15:31:11 2016 From: sonulaldeepak at gmail.com (SoNu KuMaR) Date: Wed, 17 Feb 2016 02:01:11 +0530 Subject: Not intalling Message-ID: I'm trying to install the latest version of python . First time it didn't install successfully ,i tried again now it have installed but not working. I'm sending screenshot and log file. I tried reinstallation again and again but result is same. From Alister.ware at ntlworld.com Tue Feb 16 15:33:37 2016 From: Alister.ware at ntlworld.com (Alister) Date: Tue, 16 Feb 2016 20:33:37 GMT Subject: I m facing some problem while opening the interpreter. how can I resolve the issue? References: Message-ID: On Wed, 17 Feb 2016 01:25:52 +0530, Chinmaya Choudhury wrote: > Please guide me. > #Chinmay > > Sent from Mail for Windows 10 open it correctly -- The temperature of the aqueous content of an unremittingly ogled culinary vessel will not achieve 100 degrees on the Celsius scale. From ikorot01 at gmail.com Tue Feb 16 15:41:05 2016 From: ikorot01 at gmail.com (Igor Korot) Date: Tue, 16 Feb 2016 15:41:05 -0500 Subject: I m facing some problem while opening the interpreter. how can I resolve the issue? In-Reply-To: References: Message-ID: On Tue, Feb 16, 2016 at 2:55 PM, Chinmaya Choudhury wrote: > Please guide me. > #Chinmay Dear Cousin Muscle, I have a serious trouble with Tom. Need you help at once, Jerry. (C) > > Sent from Mail for Windows 10 > > -- > https://mail.python.org/mailman/listinfo/python-list From eryksun at gmail.com Tue Feb 16 15:59:58 2016 From: eryksun at gmail.com (eryk sun) Date: Tue, 16 Feb 2016 14:59:58 -0600 Subject: [Glitch?] Python has just stopped working In-Reply-To: References: Message-ID: On Tue, Feb 16, 2016 at 10:19 AM, Theo Hamilton wrote: > Whenever I run python (3.5), I get the following message: > > Fatal Python error: Py_initialize: unable to load the file system codec > ImportError: No module named 'encodings' > > Current thread 0x00002168 (most recent call first): The interpreter can't find the standard library, which is a symptom of having PYTHONHOME set to some other directory: C:\>set PYTHONHOME=C:\ C:\>py -3.5 Fatal Python error: Py_Initialize: unable to load the file system codec ImportError: No module named 'encodings' Current thread 0x00000940 (most recent call first): Generally this environment variable is unnecessary for normal use and shouldn't be set permanently. From jcasale at activenetwerx.com Tue Feb 16 16:48:59 2016 From: jcasale at activenetwerx.com (Joseph L. Casale) Date: Tue, 16 Feb 2016 21:48:59 +0000 Subject: Passing data across callbacks in ThreadPoolExecutor Message-ID: What is the pattern for chaining execution of tasks with ThreadPoolExecutor? Callbacks is not an adequate facility as each task I have will generate new output. Thanks, jlc From ben+python at benfinney.id.au Tue Feb 16 17:27:20 2016 From: ben+python at benfinney.id.au (Ben Finney) Date: Wed, 17 Feb 2016 09:27:20 +1100 Subject: Not intalling References: Message-ID: <85lh6k2svr.fsf@benfinney.id.au> SoNu KuMaR writes: > I'm trying to install the latest version of python . First time it > didn't install successfully ,i tried again now it have installed but > not working. What exactly did you try? What details about the host can you describe so we know what may be peculiar to the problem? > I'm sending screenshot and log file. I tried reinstallation again and > again but result is same. You will need to copy and paste the actual text; graphic images are not suitable for this forum. -- \ ?The optimist thinks this is the best of all possible worlds. | `\ The pessimist fears it is true.? ?J. Robert Oppenheimer | _o__) | Ben Finney From alan at csail.mit.edu Tue Feb 16 19:24:20 2016 From: alan at csail.mit.edu (Alan Bawden) Date: Tue, 16 Feb 2016 19:24:20 -0500 Subject: Make a unique filesystem path, without creating the file References: <85egce6a8a.fsf@benfinney.id.au> <20160215230539.GA42677@cskk.homeip.net> Message-ID: Ben Finney writes: > Cameron Simpson writes: > >> I've been watching this for a few days, and am struggling to >> understand your use case. > > Yes, you're not alone. This surprises me, which is why I'm persisting. > >> Can you elaborate with a concrete example and its purpose which would >> work with a mktemp-ish official function? > > An example:: Let me present another example that might strike some as more straightforward. If I want to create a temporary file, I can call mkstemp(). If I want to create a temporary directory, I can call mkdtemp(). Suppose that instead of a file or a directory, I want a FIFO or a socket. A FIFO is created by passing a pathname to os.mkfifo(). A socket is created by passing a pathname to an AF_UNIX socket's bind() method. In both cases, the pathname must not name anything yet (not even a symbolic link), otherwise the call will fail. So in the FIFO case, I might write something like the following: def make_temp_fifo(mode=0o600): while True: path = tempfile.mktemp() try: os.mkfifo(path, mode=mode) except FileExistsError: pass else: return path mktemp() is convenient here, because I don't have to worry about whether I should be using "/tmp" or "/var/tmp" or "c:\temp", or whether the TMPDIR environment variable is set, or whether I have permission to create entries in those directories. It just gives me a pathname without making me think about the rest of that stuff. Yes, I have to defend against the possibility that somebody else creates something with the same name first, but as you can see, I did that, and it wasn't rocket science. So is there something wrong with the above code? Other than the fact that the documentation says something scary about mktemp()? It looks to me like mktemp() provides some real utility, packaged up in a way that is orthogonal to the type of file system entry I want to create, the permissions I want to give to that entry, and the mode I want use to open it. It looks like a useful, albeit low-level, primitive that it is perfectly reasonable for the tempfile module to supply. And yet the documentation condemns it as "deprecated", and tells me I should use mkstemp() instead. (As if that would be of any use in the situation above!) It looks like anxiety that some people might use mktemp() in a stupid way has caused an over-reaction. Let the documentation warn about the problem and point to prepackaged solutions in the common cases of making files and directories, but I see no good reason to deprecate this useful utility. -- Alan Bawden From jfong at ms4.hinet.net Tue Feb 16 23:04:14 2016 From: jfong at ms4.hinet.net (jfong at ms4.hinet.net) Date: Tue, 16 Feb 2016 20:04:14 -0800 (PST) Subject: Will file be closed automatically in a "for ... in open..." statement? In-Reply-To: References: Message-ID: <230cfde5-e6f2-45fe-9739-e56c5c7739ab@googlegroups.com> Thanks for these detailed explanation. Both statements will close file automatically sooner or later and, when considering the exceptions, "with" is better. Hope my understanding is right. But, just curious, how do you know the "for" will do it? I can't find any document about it from every sources I know. Very depressed:-( --Jach From rosuav at gmail.com Tue Feb 16 23:36:53 2016 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 17 Feb 2016 15:36:53 +1100 Subject: Will file be closed automatically in a "for ... in open..." statement? In-Reply-To: <230cfde5-e6f2-45fe-9739-e56c5c7739ab@googlegroups.com> References: <230cfde5-e6f2-45fe-9739-e56c5c7739ab@googlegroups.com> Message-ID: On Wed, Feb 17, 2016 at 3:04 PM, wrote: > Thanks for these detailed explanation. Both statements will close file automatically sooner or later and, when considering the exceptions, "with" is better. Hope my understanding is right. > > But, just curious, how do you know the "for" will do it? I can't find any document about it from every sources I know. Very depressed:-( > It's not the 'for' loop that does it. The for loop is kinda like this: _temp = open("foo.txt") _temp.read() # do stuff, do stuff _temp = None When you stop holding onto an object, Python can get rid of it. When that happens is not promised, though - and if you have a reference loop, it might hang around for a long time. But when a file object is disposed of, the underlying file will get closed. ChrisA From raspberryaether at e-s.invalid Tue Feb 16 23:42:25 2016 From: raspberryaether at e-s.invalid (Raspberry Aether) Date: Tue, 16 Feb 2016 23:42:25 -0500 Subject: Will file be closed automatically in a "for ... in open..." statement? In-Reply-To: <230cfde5-e6f2-45fe-9739-e56c5c7739ab@googlegroups.com> References: <230cfde5-e6f2-45fe-9739-e56c5c7739ab@googlegroups.com> Message-ID: On 02/16/2016 11:04 PM, jfong at ms4.hinet.net wrote: > Thanks for these detailed explanation. Both statements will close file automatically sooner or later and, when considering the exceptions, "with" is better. Hope my understanding is right. > > But, just curious, how do you know the "for" will do it? I can't find any document about it from every sources I know. Very depressed:-( > > --Jach > First-- IMO, don't depend on it. Instead, use something like: with open('foo.txt') as f: for line in f: pass # do something here It's one extra indent and one extra line, but it's cleaner. To answer your question, technically, it might not-- it really depends upon your implementation of Python. It just so happens that the most popular version of Python (CPython, the reference implementation) will garbage collect the file object right away. HOWEVER. The reason the "for" will PROBABLY result in file closure is because as soon as the for loop exits, there is no reason to hold onto the object returned by "open", so it is disposed. When file objects are disposed, they are closed. IMO, don't depend on this behaviour; it's bad form. From vek.m1234 at gmail.com Wed Feb 17 00:36:30 2016 From: vek.m1234 at gmail.com (Veek. M) Date: Wed, 17 Feb 2016 11:06:30 +0530 Subject: threading - Doug Hellman stdlib book, Timer() subclassing etc Message-ID: In Doug Hellman's book on the stdlib, he does: import threading import logging logging.basicConfig(level=logging.DEBUG, format=?(%(threadName)-10s) %(message)s?, ) class MyThreadWithArgs(threading.Thread): def __init__(self, group=None, target=None, name=None, args=(), kwargs=None, verbose=None): threading.Thread.__init__(self, group=group, target=target, name=name, verbose=verbose) self.args = args self.kwargs = kwargs return def run(self): logging.debug(?running with %s and %s?, self.args, self.kwargs) return for i in range(5): t = MyThreadWithArgs(args=(i,), kwargs={?a?:?A?, ?b?:?B?}) t.start() 1. Shouldn't def run() also include a call to the target function? 2. How does a call to a function_target result in a thread being created? Normally you'd have to call a function in pthreads (OS call) One can sort of figure that t.start() hides the actual OS call, but when we implement run().. somehow, magically there's no OS call? WTH! ?? Then in the Timer example in the next section, how is the whole delay/canecl bit implemented? We do t1.start so the 3 second counter starts ticking somewhere - where? And how does he cancel that? import threading import time import logging logging.basicConfig(level=logging.DEBUG, format=?(%(threadName)-10s) %(message)s?, ) def delayed(): logging.debug(?worker running?) return t1 = threading.Timer(3, delayed) t1.setName(?t1?) t2 = threading.Timer(3, delayed) t2.setName(?t2?) logging.debug(?starting timers?) t1.start() t2.start() logging.debug(?waiting before canceling %s?, t2.getName()) time.sleep(2) logging.debug(?canceling %s?, t2.getName()) t2.cancel() logging.debug(?done?) From steve+comp.lang.python at pearwood.info Wed Feb 17 00:51:31 2016 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Wed, 17 Feb 2016 16:51:31 +1100 Subject: Will file be closed automatically in a "for ... in open..." statement? References: <230cfde5-e6f2-45fe-9739-e56c5c7739ab@googlegroups.com> Message-ID: <56c40a67$0$11104$c3e8da3@news.astraweb.com> On Wednesday 17 February 2016 15:04, jfong at ms4.hinet.net wrote: > Thanks for these detailed explanation. Both statements will close file > automatically sooner or later and, when considering the exceptions, "with" > is better. Hope my understanding is right. > > But, just curious, how do you know the "for" will do it? I can't find any > document about it from every sources I know. Very depressed:-( This has nothing to do with "for". You would get exactly the same behaviour without "for": f = open("some file", "r") x = f.read(20) x = f.read(30) x = f.read() So long as the variable "f" is in-scope, the file will stay open. If the above code is in a function, "f" goes out of scope when the function returns. If the above code is at the top level of the module, "f" will stay in scope forever, or until you either delete the variable from the scope: del f or re-assign to something else: f = "hello world" At this point, after the function has exited, and the variable has completely gone out of scope, what happens? The garbage collector will: - reclaim the memory used by the object; - close the file. BUT there is no promise WHEN the file will be closed. It might be immediately, or it might be when the application shuts down. If you want the file to be closed immediately, you must: - use a with statement; - or explicitly call f.close() otherwise you are at the mercy of the interpreter, which will close the file whenever it wants. -- Steve From steve+comp.lang.python at pearwood.info Wed Feb 17 00:53:57 2016 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Wed, 17 Feb 2016 16:53:57 +1100 Subject: I m facing some problem while opening the interpreter. how can I resolve the issue? References: Message-ID: <56c40af5$0$11104$c3e8da3@news.astraweb.com> On Wednesday 17 February 2016 06:55, Chinmaya Choudhury wrote: > Please guide me. > #Chinmay > > Sent from Mail for Windows 10 How can we help you when we don't know what problem you have? Is the computer turned on? Is the mouse plugged in? Are you double-clicking the icon on the desktop? What happens? Do you get an error message or does the computer suddenly reboot? What does the error message say? -- Steve From rustompmody at gmail.com Wed Feb 17 03:51:25 2016 From: rustompmody at gmail.com (Rustom Mody) Date: Wed, 17 Feb 2016 00:51:25 -0800 (PST) Subject: Guido on python3 for beginners In-Reply-To: <31a618b2-a407-4723-9a4f-9b756fc93b0b@googlegroups.com> References: <48762040-a7e0-434c-92e8-8a0969210e5b@googlegroups.com> <5d31dd14-1a60-4f80-8889-d0616f404c70@googlegroups.com> <4e902c58-3370-4adf-905d-74630a6eea90@googlegroups.com> <60036746-78f7-4f05-a6d8-04e88b09c00f@googlegroups.com> <032f0f66-34df-4325-98e0-1127025d2a94@googlegroups.com> <31a618b2-a407-4723-9a4f-9b756fc93b0b@googlegroups.com> Message-ID: <0a920e92-5d31-4866-815b-cae7e201e4d8@googlegroups.com> On Tuesday, February 16, 2016 at 10:54:43 AM UTC+5:30, John Ladasky wrote: > On Monday, February 15, 2016 at 6:02:24 PM UTC-8, Rick Johnson wrote: > > I don't need Python3. And i reckon that by the time i do, > > something more interesting will come along, or, i'll create > > something more interesting myself. i've been drafting, and > > dreaming, of a new language spec for over a year now.. And > > the best thing about starting a new language, you can do > > anything you want... no dependencies! > > If you have all the skills that you claim, you're a far better programmer than I. So -- exactly why are you complaining to people who are developing and/or using Py3? Go off and impress everyone. Become your own BDFL. I hardly want to enter a 'who-can-fart-loudest' contest with Rick... I hope someone can help me find this link: There is some record that Guido has said that python3 is probably a bit harder on noobs than python2. Does anyone know/have that link? From mail at timgolden.me.uk Wed Feb 17 04:20:36 2016 From: mail at timgolden.me.uk (Tim Golden) Date: Wed, 17 Feb 2016 09:20:36 +0000 Subject: Fwd: [python-uk] URGENT: Volunteers for teacher collaboration please... :-) In-Reply-To: <56C42FA7.8000806@ntoll.org> References: <56C42FA7.8000806@ntoll.org> Message-ID: <56C43B64.8070008@timgolden.me.uk> [Forwarding an email from python-uk in the hope that UK-based Pythonistas might see it here who don't hang out there...] From: Nicholas H.Tollervey Reply-To: UK Python Users To: UK Python Users Hi Folks, I realise I must sound like a stuck record about this sort of thing - please accept my sincere apologies. TL;DR: I need volunteers from around the country to support a twilight meetup of teachers happening in various parts of the UK. It's not difficult and likely to be a lot of fun and will only take a few hours of your time in the early evening of a single day. I may be able to cover travel expenses. Please get in touch. More detail below... Computing at School (see: http://www.computingatschool.org.uk/), a grass roots movement of computing teachers in the UK would like to run a series of training courses for "Master Teachers" in MicroPython on the BBC micro:bit during March. These teachers would go on to act as the seed / catalyst for other teachers who require Python training during a series of training events over the summer. Put simply, this is an exercise in Python evangelism for teachers. Master teachers are those who have demonstrated a combination of deep subject knowledge and teaching skill. Put simply, they're the most senior teachers you can get. They're also the leaders in the field and what they say or do influences many hundreds of their colleagues. The idea is for the master teachers to get together with Python developers (that'd be *you*) for a few hours to work through MicroPython related educational resources. These events would happen at university based hubs around the country. As a Python developer you'll *get a BBC micro:bit* and be expected to offer advice, answer questions and demonstrate Python as needed. Honestly, it's not an onerous task and will only last a few hours in a "twilight" session (i.e. after work). The locations and proposed dates are as follows: London: 25th February Birmingham: 9th March Nottingham: 15th March Lancaster: 16th March Newcastle: 17th March Hertfordshire: 21st March Manchester: 23rd March Southampton: 23rd March It's easy for UK Python to be very London-centric. This is an opportunity for Pythonistas throughout the UK to step up and get involved. Why should you volunteer a few hours of your time to help teachers? Need you ask? Your help and influence will ultimately contribute to the education of the next generation of programmers - your future colleagues. It's a way to give back to the community by fostering the next generation of Pythonistas with the help of the CAS Master Teachers. It's also, from a moral point of view, simply a selfless and unambiguously good thing to do. If you're thinking "oh, they won't want me", then YOU ARE EXACTLY THE PERSON WE NEED! Your experience, perspective and knowledge is invaluable and teachers need to hear from you. Rest assured, this will not be a difficult or high-pressure activity. In fact, it's likely to be a lot of fun. Remember that awesome person who mentored you and/or gave you a step up? Now's your chance to be that person for a group of master teachers. If this is of interest to you, please get in touch ASAP and I can start to coordinate things with CAS. I'm going to put in a grant request to the PSF to see if we can cover travel costs for developers. But there's no guarantee this will come about. Best wishes, N. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 486 bytes Desc: not available URL: -------------- next part -------------- _______________________________________________ python-uk mailing list python-uk at python.org https://mail.python.org/mailman/listinfo/python-uk From anubhav1691 at gmail.com Wed Feb 17 04:26:27 2016 From: anubhav1691 at gmail.com (Anubhav Yadav) Date: Wed, 17 Feb 2016 14:56:27 +0530 Subject: [python-uk] URGENT: Volunteers for teacher collaboration please... :-) In-Reply-To: <56C43B64.8070008@timgolden.me.uk> References: <56C42FA7.8000806@ntoll.org> <56C43B64.8070008@timgolden.me.uk> Message-ID: I met @ntoll at an event in India. It was refreshing to see his passion for teaching kids about programming and python. I would have loved to be there, but I don't reside in the UK. Nevertheless, just replying to wish you guys all the best. On Wed, Feb 17, 2016 at 2:50 PM, Tim Golden wrote: > [Forwarding an email from python-uk in the hope that UK-based > Pythonistas might see it here who don't hang out there...] > > From: Nicholas H.Tollervey > Reply-To: UK Python Users > To: UK Python Users > > Hi Folks, > > I realise I must sound like a stuck record about this sort of thing - > please accept my sincere apologies. > > TL;DR: I need volunteers from around the country to support a twilight > meetup of teachers happening in various parts of the UK. It's not > difficult and likely to be a lot of fun and will only take a few hours > of your time in the early evening of a single day. I may be able to > cover travel expenses. Please get in touch. More detail below... > > Computing at School (see: http://www.computingatschool.org.uk/), a grass > roots movement of computing teachers in the UK would like to run a > series of training courses for "Master Teachers" in MicroPython on the > BBC micro:bit during March. These teachers would go on to act as the > seed / catalyst for other teachers who require Python training during a > series of training events over the summer. Put simply, this is an > exercise in Python evangelism for teachers. > > Master teachers are those who have demonstrated a combination of deep > subject knowledge and teaching skill. Put simply, they're the most > senior teachers you can get. They're also the leaders in the field and > what they say or do influences many hundreds of their colleagues. > > The idea is for the master teachers to get together with Python > developers (that'd be *you*) for a few hours to work through MicroPython > related educational resources. These events would happen at university > based hubs around the country. As a Python developer you'll *get a BBC > micro:bit* and be expected to offer advice, answer questions and > demonstrate Python as needed. Honestly, it's not an onerous task and > will only last a few hours in a "twilight" session (i.e. after work). > > The locations and proposed dates are as follows: > > London: 25th February > Birmingham: 9th March > Nottingham: 15th March > Lancaster: 16th March > Newcastle: 17th March > Hertfordshire: 21st March > Manchester: 23rd March > Southampton: 23rd March > > It's easy for UK Python to be very London-centric. This is an > opportunity for Pythonistas throughout the UK to step up and get involved. > > Why should you volunteer a few hours of your time to help teachers? Need > you ask? Your help and influence will ultimately contribute to the > education of the next generation of programmers - your future > colleagues. It's a way to give back to the community by fostering the > next generation of Pythonistas with the help of the CAS Master Teachers. > It's also, from a moral point of view, simply a selfless and > unambiguously good thing to do. > > If you're thinking "oh, they won't want me", then YOU ARE EXACTLY THE > PERSON WE NEED! Your experience, perspective and knowledge is invaluable > and teachers need to hear from you. Rest assured, this will not be a > difficult or high-pressure activity. In fact, it's likely to be a lot of > fun. > > Remember that awesome person who mentored you and/or gave you a step up? > Now's your chance to be that person for a group of master teachers. > > If this is of interest to you, please get in touch ASAP and I can start > to coordinate things with CAS. > > I'm going to put in a grant request to the PSF to see if we can cover > travel costs for developers. But there's no guarantee this will come about. > > Best wishes, > > N. > > > -- > https://mail.python.org/mailman/listinfo/python-list > > -- Regards, Anubhav Yadav KPIT Technologies, Pune. From oscar.j.benjamin at gmail.com Wed Feb 17 08:31:22 2016 From: oscar.j.benjamin at gmail.com (Oscar Benjamin) Date: Wed, 17 Feb 2016 13:31:22 +0000 Subject: Make a unique filesystem path, without creating the file In-Reply-To: <85twl830m3.fsf@benfinney.id.au> References: <85egce6a8a.fsf@benfinney.id.au> <20160215230539.GA42677@cskk.homeip.net> <851t8d42rt.fsf@benfinney.id.au> <85twl830m3.fsf@benfinney.id.au> Message-ID: On 16 February 2016 at 19:40, Ben Finney wrote: > Oscar Benjamin writes: > >> If you're going to patch open to return a fake file when asked to open >> fake_file_path why do you care whether there is a real file of that >> name? > > I don't, and have been saying explicitly many times in this thread that > I do not care whether the file exists. Somehow that is still not clear? Sorry Ben I misunderstood. I think I can see the source of confusion which is in your first message: """ In some code (e.g. unit tests) I am calling ?tempfile.mktemp? to generate a unique path for a filesystem entry that I *do not want* to exist on the real filesystem. """ I read that as meaning that it was important that the file did not exist. But you say that you don't care if the file actually exists in the filesystem or not and just want a unique path. What do you mean by unique here? The intention of mktemp is that the path is unique so that there would not exist a file of that name and if you opened it for writing you wouldn't be interfering with any existing file. Do you just mean a function that returns a different value each time it's called? How about this: count = 0 def unique_path(): global count count += 1 return os.path.join(tempfile.gettempdir(), str(count)) -- Oscar From jrmy.lnrd at gmail.com Wed Feb 17 08:47:53 2016 From: jrmy.lnrd at gmail.com (Jeremy Leonard) Date: Wed, 17 Feb 2016 05:47:53 -0800 (PST) Subject: Weird and sparese cgi:error In-Reply-To: <813cd989-bc60-4d24-96ef-f20244d91362@googlegroups.com> References: <813cd989-bc60-4d24-96ef-f20244d91362@googlegroups.com> Message-ID: On Tuesday, February 16, 2016 at 3:02:40 PM UTC-5, ?????? ??????????????? wrote: > Hello, > > I recentely changed VPS server and when i try to load my webiste iam receiving 500 error which i wasnt receiving in my old VPS server with the same exact cgi scripts. > > After looking at Apacher's error_log iam seeing the following output when i try to load scripts from cgi-bin directory. > > > [Tue Feb 16 21:28:55.970302 2016] [cgi:error] [pid 17478] [client 180.76.15.28:21311] End of script output before headers: metrites.py > [Tue Feb 16 21:29:31.193281 2016] [cgi:error] [pid 18802] [client 180.76.15.29:34155] End of script output before headers: metrites.py > [Tue Feb 16 21:32:21.306627 2016] [cgi:error] [pid 17478] [client 188.138.124.110:40212] End of script output before headers: metrites.py > [Tue Feb 16 21:37:21.128472 2016] [cgi:error] [pid 19731] [client 204.152.200.42:54376] End of script output before headers: metrites.py > [Tue Feb 16 21:42:21.198207 2016] [cgi:error] [pid 17478] [client 184.75.210.226:57252] End of script output before headers: metrites.py > [Tue Feb 16 21:47:21.228361 2016] [cgi:error] [pid 19731] [client 76.72.172.208:34730] End of script output before headers: metrites.py > [Tue Feb 16 21:52:21.422066 2016] [cgi:error] [pid 18802] [client 85.17.156.11:52071] End of script output before headers: metrites.py > > > Please i need your assistance to this. > thank you.Prob If you import the cgitb module and then put cgitb.enable() in the first line of your code this can help give you more descriptive error messages on the page that is producing the errors From rosuav at gmail.com Wed Feb 17 08:53:23 2016 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 18 Feb 2016 00:53:23 +1100 Subject: Weird and sparese cgi:error In-Reply-To: References: <813cd989-bc60-4d24-96ef-f20244d91362@googlegroups.com> Message-ID: On Thu, Feb 18, 2016 at 12:47 AM, Jeremy Leonard wrote: > On Tuesday, February 16, 2016 at 3:02:40 PM UTC-5, ?????? ??????????????? wrote: >> Hello, >> >> I recentely changed VPS server and when i try to load my webiste iam receiving 500 error which i wasnt receiving in my old VPS server with the same exact cgi scripts. >> >> After looking at Apacher's error_log iam seeing the following output when i try to load scripts from cgi-bin directory. >> >> >> [Tue Feb 16 21:28:55.970302 2016] [cgi:error] [pid 17478] [client 180.76.15.28:21311] End of script output before headers: metrites.py >> [Tue Feb 16 21:29:31.193281 2016] [cgi:error] [pid 18802] [client 180.76.15.29:34155] End of script output before headers: metrites.py >> [Tue Feb 16 21:32:21.306627 2016] [cgi:error] [pid 17478] [client 188.138.124.110:40212] End of script output before headers: metrites.py >> [Tue Feb 16 21:37:21.128472 2016] [cgi:error] [pid 19731] [client 204.152.200.42:54376] End of script output before headers: metrites.py >> [Tue Feb 16 21:42:21.198207 2016] [cgi:error] [pid 17478] [client 184.75.210.226:57252] End of script output before headers: metrites.py >> [Tue Feb 16 21:47:21.228361 2016] [cgi:error] [pid 19731] [client 76.72.172.208:34730] End of script output before headers: metrites.py >> [Tue Feb 16 21:52:21.422066 2016] [cgi:error] [pid 18802] [client 85.17.156.11:52071] End of script output before headers: metrites.py >> >> >> Please i need your assistance to this. >> thank you.Prob > > If you import the cgitb module and then put cgitb.enable() in the first line of your code this can help give you more descriptive error messages on the page that is producing the errors > And if that doesn't work, I'd look at the very beginning of code execution - can the Python interpreter even be found? What happens if you run one of the scripts from a shell? Maybe there's a shebang problem. ChrisA From ganesh1pal at gmail.com Wed Feb 17 08:58:51 2016 From: ganesh1pal at gmail.com (Ganesh Pal) Date: Wed, 17 Feb 2016 19:28:51 +0530 Subject: TypeError: not all arguments converted during string formatting Message-ID: Hi Team, Iam on python 2.6 and Linux , I had replaced print out, err ret with logging.info(out, err ,ret) in the below code . I am getting "TypeError: not all arguments converted during string formatting" error any quick suggestion try: out, err, ret = run(cmd, timeout=60) # New line added below logging.info(out, err ,ret) if ret != 0: logging.error("Can't run %s got %s (%d)!" % (cmd, err, ret)) raise Exception("Preparing cluster failed...Exiting !!!") except Exception as e: logging.exception("Failed to run %s got %s" % (cmd, e)) sys.exit("Preparing cluster failed") logging.info("Preparing Cluster.....Done !!!") (Pdb) c Traceback (most recent call last): File "/usr/local/lib/python2.6/logging/__init__.py", line 755, in emit File "/usr/local/lib/python2.6/logging/__init__.py", line 637, in format File "/usr/local/lib/python2.6/logging/__init__.py", line 425, in format File "/usr/local/lib/python2.6/logging/__init__.py", line 295, in getMessage TypeError: not all arguments converted during string formatting Regards, Ganesh From ganesh1pal at gmail.com Wed Feb 17 09:00:20 2016 From: ganesh1pal at gmail.com (Ganesh Pal) Date: Wed, 17 Feb 2016 19:30:20 +0530 Subject: TypeError: not all arguments converted during string formatting In-Reply-To: References: Message-ID: I think logging.info(out) works the problem is when I add logging.info(out,err,ret) ,may be there is a better way to supply this arguments On Wed, Feb 17, 2016 at 7:28 PM, Ganesh Pal wrote: > Hi Team, > > > Iam on python 2.6 and Linux , I had replaced print out, err ret with > logging.info(out, err ,ret) in the below code . I am getting > > "TypeError: not all arguments converted during string formatting" > error any quick suggestion > > > try: > out, err, ret = run(cmd, timeout=60) > # New line added below > logging.info(out, err ,ret) > if ret != 0: > logging.error("Can't run %s got %s (%d)!" % (cmd, err, ret)) > raise Exception("Preparing cluster failed...Exiting !!!") > except Exception as e: > logging.exception("Failed to run %s got %s" % (cmd, e)) > sys.exit("Preparing cluster failed") > logging.info("Preparing Cluster.....Done !!!") > > > (Pdb) c > Traceback (most recent call last): > File "/usr/local/lib/python2.6/logging/__init__.py", line 755, in emit > File "/usr/local/lib/python2.6/logging/__init__.py", line 637, in format > File "/usr/local/lib/python2.6/logging/__init__.py", line 425, in format > File "/usr/local/lib/python2.6/logging/__init__.py", line 295, in getMessage > TypeError: not all arguments converted during string formatting > > > Regards, > Ganesh From rosuav at gmail.com Wed Feb 17 09:02:01 2016 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 18 Feb 2016 01:02:01 +1100 Subject: TypeError: not all arguments converted during string formatting In-Reply-To: References: Message-ID: On Thu, Feb 18, 2016 at 12:58 AM, Ganesh Pal wrote: > Iam on python 2.6 and Linux , I had replaced print out, err ret with > logging.info(out, err ,ret) in the below code . I am getting > > "TypeError: not all arguments converted during string formatting" > error any quick suggestion > The print statement/function happily accepts multiple arguments, and will join them according to a set of predefined rules. The logging functions don't have those rules, so they take one message and some optional parameters. Try this, instead: logging.info("%r %r %r", out, err, ret) and then tweak the format string according to requirements. For a quick dump, this will serve you well. ChrisA From ian.g.kelly at gmail.com Wed Feb 17 11:24:31 2016 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Wed, 17 Feb 2016 09:24:31 -0700 Subject: Passing data across callbacks in ThreadPoolExecutor In-Reply-To: References: Message-ID: On Tue, Feb 16, 2016 at 2:48 PM, Joseph L. Casale wrote: > What is the pattern for chaining execution of tasks with ThreadPoolExecutor? > Callbacks is not an adequate facility as each task I have will generate new output. Can you specify in more detail what your use case is? If you don't mind having threads sitting around waiting, you can just submit each chained task at the start and have each task wait on the futures of its dependencies. The advantage of this is that it's easy to conceptualize the dependency graph of the tasks. The disadvantage is that it eats up extra threads. You'll probably want to increase the size of the thread pool to handle the waiting tasks (or use a separate ThreadPoolExecutor for each chained task). E.g.: def task2(input, f1): f1_result = f1.result() # Wait for f1 to be done. result = frobnicate(input, f1_result) return result f1 = executor.submit(task1, input) f2 = executor.submit(task2, input, f1) f1.add_done_callback(handle_task1_done) f2.add_done_callback(handle_task2_done) Otherwise, is there some reason you can't use multiple callbacks, one to handle the task's output and one to submit the chained task? E.g.: def chain_task2(input, f2): f2 = executor.submit(task2, input, f2.result()) f2.add_done_callback(handle_task2_done) f1 = executor.submit(task1, input) f1.add_done_callback(handle_task1_done) f1.add_done_callback(functools.partial(chain_task2, input)) From framstag at rus.uni-stuttgart.de Wed Feb 17 12:49:11 2016 From: framstag at rus.uni-stuttgart.de (Ulli Horlacher) Date: Wed, 17 Feb 2016 17:49:11 +0000 (UTC) Subject: extending PATH on Windows? References: Message-ID: Thorsten Kampe wrote: > By the way: there is a script called `win_add2path.py` in your Python > distribution I have "Python 2.7.11 (v2.7.11:6d1b6a68f775, Dec 5 2015, 20:32:19) [MSC v.1500 32 bit (Intel)] on win32" and there is no "win_add2path.py" But I found another solution: I need the extended PATH only for cmd.exe: users shall run my program (a compiled executable) without having to enter the full path. At startup cmd.exe runs a script which is defined by the registry variable AutoRun in "HKCU\Software\Microsoft\Command Processor" I set this variable with: rc = "HKCU\Software\Microsoft\Command Processor" ar = "%USERPROFILE%\Desktop\autorun.cmd" system('reg add "%s" /v AutoRun /d "%s"' % (rc,ar)) and write into autorun.cmd: set PATH=%PATH%;%USERPROFILE%\Desktop This command extends PATH only for this cmd.exe instance. This is all I need! -- Ullrich Horlacher Server und Virtualisierung Rechenzentrum IZUS/TIK E-Mail: horlacher at tik.uni-stuttgart.de Universitaet Stuttgart Tel: ++49-711-68565868 Allmandring 30a Fax: ++49-711-682357 70550 Stuttgart (Germany) WWW: http://www.tik.uni-stuttgart.de/ From g.rodola at gmail.com Wed Feb 17 12:53:52 2016 From: g.rodola at gmail.com (Giampaolo Rodola') Date: Wed, 17 Feb 2016 18:53:52 +0100 Subject: ANN: psutil 4.0.0 released - how to get "real" process memory and environ in Python Message-ID: Full blog post: http://grodola.blogspot.com/2016/02/psutil-4-real-process-memory-and-environ.html -- Giampaolo - http://grodola.blogspot.com From sushanthece at gmail.com Wed Feb 17 13:00:15 2016 From: sushanthece at gmail.com (Sushanth) Date: Wed, 17 Feb 2016 23:30:15 +0530 Subject: Reg: Data frame conversion Message-ID: ?i need to convert r data fr?ame to pandas dataframe and vise versa -- *Regards* *Sushanth* *ph : 91-9444583307* From gheskett at wdtv.com Wed Feb 17 13:21:49 2016 From: gheskett at wdtv.com (Gene Heskett) Date: Wed, 17 Feb 2016 13:21:49 -0500 Subject: Are there any pyvcp experts here? Message-ID: <201602171321.49262.gheskett@wdtv.com> Greetings; I am having difficulty with the resultant gui objects its making that are too tall. They fit in the except the resultant box is 40 to 50 pixels too tall, wasting valueable screen real estate, and I haven't found a way to give the a "40" command that seems to be acceptable. I know the list strips attachments, so whoever wants to tackle in, I'll put the screen snapshot (one pix is worth 10k words), and the .xml file that constructs in on my web page & send the link. Cheers, Gene Heskett -- "There are four boxes to be used in defense of liberty: soap, ballot, jury, and ammo. Please use in that order." -Ed Howdershelt (Author) Genes Web page From tony at vanderhoff.org Wed Feb 17 13:37:20 2016 From: tony at vanderhoff.org (Tony van der Hoff) Date: Wed, 17 Feb 2016 18:37:20 +0000 Subject: Reg: Data frame conversion In-Reply-To: References: Message-ID: <56C4BDE0.8060403@vanderhoff.org> On 17/02/16 18:00, Sushanth wrote: > ?i need to convert r data fr?ame to pandas dataframe and vise versa > Wow! How do you plan to do that? -- Tony van der Hoff | mailto:tony at vanderhoff.org Buckinghamshire, England | From eryksun at gmail.com Wed Feb 17 14:02:21 2016 From: eryksun at gmail.com (eryk sun) Date: Wed, 17 Feb 2016 13:02:21 -0600 Subject: extending PATH on Windows? In-Reply-To: References: Message-ID: On Wed, Feb 17, 2016 at 11:49 AM, Ulli Horlacher wrote: > At startup cmd.exe runs a script which is defined by the registry variable > AutoRun in "HKCU\Software\Microsoft\Command Processor" > > I set this variable with: > > rc = "HKCU\Software\Microsoft\Command Processor" > ar = "%USERPROFILE%\Desktop\autorun.cmd" > system('reg add "%s" /v AutoRun /d "%s"' % (rc,ar)) > > and write into autorun.cmd: > > set PATH=%PATH%;%USERPROFILE%\Desktop The AutoRun command (it's a command line, not a script path) gets run for every instance of cmd.exe, unless cmd is started with the /D option. This includes the CRT's system() function. Your batch file needs to set a sentinel variable such as YOUR_APP_AUTORUN that, if set, makes the batch file exit without extending PATH if it was already done. Also, you can't just overwrite a user's AutoRun command like that. If a command currently exists, you have to concatenate your command with the existing command using parentheses and the "&" operator, e.g. (previous_command) & (your_command). Use winreg for this. From wrong.address.1 at gmail.com Wed Feb 17 14:49:16 2016 From: wrong.address.1 at gmail.com (wrong.address.1 at gmail.com) Date: Wed, 17 Feb 2016 11:49:16 -0800 (PST) Subject: Considering migrating to Python from Visual Basic 6 for engineering applications Message-ID: <90cc50d2-1ce5-4588-9bfd-a49d439f00dd@googlegroups.com> I am mostly getting positive feedback for Python. It seems Python is used more for web based applications. Is it equally fine for creating stand-alone *.exe's? Can the same code be compiled to run on Linux or Android or web-based? Is it possible to create GUI elements with a good IDE? Can they be defined like in Visual Basic with given sizes, fonts, visible/invisible, etc.? Is it easy to do matrix operations in Python? Or do I need to write subroutines like in Visual Basic? Could someone kindly tell me advantages and disadvantages of Python? Or any better options? I have like 40-50 VB Forms and may be around 20000 lines of code. It will be a task to learn a new language and translate/re-write that code. Thanks for your responses. From sohcahtoa82 at gmail.com Wed Feb 17 15:25:51 2016 From: sohcahtoa82 at gmail.com (sohcahtoa82 at gmail.com) Date: Wed, 17 Feb 2016 12:25:51 -0800 (PST) Subject: Considering migrating to Python from Visual Basic 6 for engineering applications In-Reply-To: <90cc50d2-1ce5-4588-9bfd-a49d439f00dd@googlegroups.com> References: <90cc50d2-1ce5-4588-9bfd-a49d439f00dd@googlegroups.com> Message-ID: On Wednesday, February 17, 2016 at 11:49:44 AM UTC-8, wrong.a... at gmail.com wrote: > I am mostly getting positive feedback for Python. Good! > > It seems Python is used more for web based applications. Is it equally fine for creating stand-alone *.exe's? Can the same code be compiled to run on Linux or Android or web-based? Python is not traditionally compiled. It is interpreted. That said, there are utilities out there than can compile a Python script to a Windows executable. I personally have never used them, so I don't know about their limitations. As for Android, I've heard of frameworks for making Android apps with Python, but I don't know of any off the top of my head. > > Is it possible to create GUI elements with a good IDE? Can they be defined like in Visual Basic with given sizes, fonts, visible/invisible, etc.? I'm fairly certain there are GUI editors for Python. I don't use any because I don't write anything with a GUI. > > Is it easy to do matrix operations in Python? Or do I need to write subroutines like in Visual Basic? Check out the numpy and scipy packages. > > Could someone kindly tell me advantages and disadvantages of Python? Or any better options? I have like 40-50 VB Forms and may be around 20000 lines of code. It will be a task to learn a new language and translate/re-write that code. IMO, Python's greatest advantage is readability. Python's syntax just plain makes sense to me. It is just an easy language to work with. It's funny, really. I used to be a hardcore C/C++ fanatic. When I first started learning Python, I hated it. I thought it made things too easy and held your hand too much. Now, I love it because it makes things so easy and holds your hand. I spend less time worrying about memory allocation and pointer syntax and more time actually getting things done. I think Python's biggest disadvantage is performance. If you write 100% pure Python code, it can be a bit on the slower side. However, if you're doing heavy number crunching using numpy/scipy, they're mostly written in C and are quite fast. Basically, the way I see it, if speed of development is much more important than speed of execution, typically Python is a good choice. > > Thanks for your responses. I'm hoping someone else will be able to give a better response to Python on Android and the GUI editor. From framstag at rus.uni-stuttgart.de Wed Feb 17 15:29:37 2016 From: framstag at rus.uni-stuttgart.de (Ulli Horlacher) Date: Wed, 17 Feb 2016 20:29:37 +0000 (UTC) Subject: extending PATH on Windows? References: Message-ID: eryk sun wrote: > > At startup cmd.exe runs a script which is defined by the registry variable > > AutoRun in "HKCU\Software\Microsoft\Command Processor" > > > > I set this variable with: > > > > rc = "HKCU\Software\Microsoft\Command Processor" > > ar = "%USERPROFILE%\Desktop\autorun.cmd" > > system('reg add "%s" /v AutoRun /d "%s"' % (rc,ar)) > > > > and write into autorun.cmd: > > > > set PATH=%PATH%;%USERPROFILE%\Desktop > > The AutoRun command (it's a command line, not a script path) A script path is a legal command line, too. > gets run for every instance of cmd.exe Yes, this is the intended trick! > Your batch file needs to set a sentinel variable such as YOUR_APP_AUTORUN > that, if set, makes the batch file exit without extending PATH if it was > already done. Having "%USERPROFILE%\Desktop" twice in PATH is not a problem. > Also, you can't just overwrite a user's AutoRun command like that. If > a command currently exists, you have to concatenate your command with > the existing command using parentheses and the "&" operator, e.g. > (previous_command) & (your_command). Good hint, thanks! > Use winreg for this. system("reg ...") works for me and is easier :-) -- Ullrich Horlacher Server und Virtualisierung Rechenzentrum IZUS/TIK E-Mail: horlacher at tik.uni-stuttgart.de Universitaet Stuttgart Tel: ++49-711-68565868 Allmandring 30a Fax: ++49-711-682357 70550 Stuttgart (Germany) WWW: http://www.tik.uni-stuttgart.de/ From breamoreboy at yahoo.co.uk Wed Feb 17 15:41:06 2016 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Wed, 17 Feb 2016 20:41:06 +0000 Subject: Reg: Data frame conversion In-Reply-To: References: Message-ID: On 17/02/2016 18:00, Sushanth wrote: > ?i need to convert r data fr?ame to pandas dataframe and vise versa > It is very easy, just write some code. When you have finished, please show us your results. -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From breamoreboy at yahoo.co.uk Wed Feb 17 15:51:29 2016 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Wed, 17 Feb 2016 20:51:29 +0000 Subject: Python 3 in 2016 Message-ID: rr eat your heart out. https://hynek.me/articles/python3-2016/ -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From rgaddi at highlandtechnology.invalid Wed Feb 17 15:54:21 2016 From: rgaddi at highlandtechnology.invalid (Rob Gaddi) Date: Wed, 17 Feb 2016 20:54:21 -0000 (UTC) Subject: Considering migrating to Python from Visual Basic 6 for engineering applications References: <90cc50d2-1ce5-4588-9bfd-a49d439f00dd@googlegroups.com> Message-ID: wrong.address.1 at gmail.com wrote: > I am mostly getting positive feedback for Python. Welcome to the party. Learn to write Python 3. There is no reason whatsoever for someone picking up Python to learn the deprecated problems of Python 2. > It seems Python is used more for web based applications. Is it equally fine for creating stand-alone *.exe's? Can the same code be compiled to run on Linux or Android or web-based? Standalone EXEs are hit and miss. I've had best luck deploying just as Python scripts to machines with an expectation that the environment is already configured (i.e. Python already installed). You don't compile, you interpret. So long as you've got the interpreter on the target platform you're set. The EXE tools out there do so by bundling your plain-text source up with the interpreter environment. Packaging/distribution remains one of the real pain points in Python. > Is it possible to create GUI elements with a good IDE? Can they be defined like in Visual Basic with given sizes, fonts, visible/invisible, etc.? > Yep, and you've got your choice of GUI toolkits (which has ups and downs). Coming from VB you're probably used to graphical designers, in which case I'd point you to Qt Designer and PySide or PyQt. > Is it easy to do matrix operations in Python? Or do I need to write subroutines like in Visual Basic? Here's where Python's a godsend. NumPy will give you vectors, matrices, linear algebra, all manner of good stuff, all compiled to be blindingly fast. Depending on what you mean by engineering applications there's a ton of good stuff in SciPy too; as an EE I make pretty extensive use of SciPy's signal processing and optimization packages. Matplotlib is the 3rd piece of the puzzle and behold! you have graphs, as simple or complex as you'd like. > Could someone kindly tell me advantages and disadvantages of Python? Or any better options? I have like 40-50 VB Forms and may be around 20000 lines of code. It will be a task to learn a new language and translate/re-write that code. > It will be, and I'm not sure I'd recommend doing it. The last time I wrote any VB was Windows 3.1, and the .NET framework wasn't even a gleam in Paul Allen's ey. So I can't speak too authoratively on what you're leaving. But over the years I've probably tried to get work done in two dozen languages. None of them turn out correct code more quickly than Python; few even come close. -- Rob Gaddi, Highland Technology -- www.highlandtechnology.com Email address domain is currently out of order. See above to fix. From rgaddi at highlandtechnology.invalid Wed Feb 17 15:56:57 2016 From: rgaddi at highlandtechnology.invalid (Rob Gaddi) Date: Wed, 17 Feb 2016 20:56:57 -0000 (UTC) Subject: Reg: Data frame conversion References: Message-ID: Sushanth wrote: > ?i need to convert r data fr?ame to pandas dataframe and vise versa > Now, now, let's at least _try_ to help. http://lmgtfy.com/?q=convert+r+data+frame+to+pandas -- Rob Gaddi, Highland Technology -- www.highlandtechnology.com Email address domain is currently out of order. See above to fix. From rgacote at appropriatesolutions.com Wed Feb 17 16:13:52 2016 From: rgacote at appropriatesolutions.com (Ray Cote) Date: Wed, 17 Feb 2016 16:13:52 -0500 Subject: Considering migrating to Python from Visual Basic 6 for engineering applications In-Reply-To: References: <90cc50d2-1ce5-4588-9bfd-a49d439f00dd@googlegroups.com> Message-ID: On Wed, Feb 17, 2016 at 3:54 PM, Rob Gaddi < rgaddi at highlandtechnology.invalid> wrote: > > It seems Python is used more for web based applications. Is it equally > fine for creating stand-alone *.exe's? Can the same code be compiled to run > on Linux or Android or web-based? > > Standalone EXEs are hit and miss. I've had best luck deploying just as > Python scripts to machines with an expectation that the environment is > already configured (i.e. Python already installed). > > You don't compile, you interpret. So long as you've got the > interpreter on the target platform you're set. The EXE tools out there > do so by bundling your plain-text source up with the interpreter > environment. > > Packaging/distribution remains one of the real pain points in Python. > Python is used for much more then web based applications. The latest versions of pyInstaller (http://www.pyinstaller.org) are incredibly good at generating single-file binaries for both Windows and Linux. We?re using pyInstaller to distribute applications based on some fairly complex frameworks?including wxPython (http://www.wxpython.org) which is one option for building GUI applications. ?Ray -- Raymond Cote, President voice: +1.603.924.6079 email: rgacote at AppropriateSolutions.com skype: ray.cote From eryksun at gmail.com Wed Feb 17 16:17:46 2016 From: eryksun at gmail.com (eryk sun) Date: Wed, 17 Feb 2016 15:17:46 -0600 Subject: extending PATH on Windows? In-Reply-To: References: Message-ID: On Wed, Feb 17, 2016 at 2:29 PM, Ulli Horlacher wrote: > eryk sun wrote: >> > >> > set PATH=%PATH%;%USERPROFILE%\Desktop >> >> The AutoRun command (it's a command line, not a script path) > > A script path is a legal command line, too. If the registry value were just a script path, you'd have to modify your script to chain to the previous script, if any. Since it's a command line you can simply use the & operator to append another command. >> gets run for every instance of cmd.exe > > Yes, this is the intended trick! Do you really intend for your batch file to be run every time cmd.exe is executed, including every time that every program on the machine calls the CRT system() function? Why don't you just install a shortcut to a batch file that starts a command prompt with the extended PATH? >> Also, you can't just overwrite a user's AutoRun command like that. If >> a command currently exists, you have to concatenate your command with >> the existing command using parentheses and the "&" operator, e.g. >> (previous_command) & (your_command). > > Good hint, thanks! > >> Use winreg for this. > > system("reg ...") works for me and is easier :-) system('reg...') won't be able to do that unless you export the key to a .reg file and parse the existing AutoRun value. It would be simpler to use subprocess.check_output('reg query ...'), but simpler still and more reliable to just call QueryValueEx. From paul.hermeneutic at gmail.com Wed Feb 17 16:18:51 2016 From: paul.hermeneutic at gmail.com (paul.hermeneutic at gmail.com) Date: Wed, 17 Feb 2016 14:18:51 -0700 Subject: Considering migrating to Python from Visual Basic 6 for engineering applications In-Reply-To: <90cc50d2-1ce5-4588-9bfd-a49d439f00dd@googlegroups.com> References: <90cc50d2-1ce5-4588-9bfd-a49d439f00dd@googlegroups.com> Message-ID: On Wed, Feb 17, 2016 at 12:49 PM, wrote: > Could someone kindly tell me advantages and disadvantages of Python? Or any better options? I have like 40-50 VB Forms and may be around 20000 lines of code. It will be a task to learn a new language and translate/re-write that code. Why are you looking to change from VB? Too slow? Cannot run on anything but Windows? Buggy? I, like most people here, would encourage you to learn Python 3. However, there is something to be said for "if it ain't broke, don't fix it." From wrw at mac.com Wed Feb 17 16:27:59 2016 From: wrw at mac.com (William Ray Wing) Date: Wed, 17 Feb 2016 16:27:59 -0500 Subject: Considering migrating to Python from Visual Basic 6 for engineering applications In-Reply-To: <90cc50d2-1ce5-4588-9bfd-a49d439f00dd@googlegroups.com> References: <90cc50d2-1ce5-4588-9bfd-a49d439f00dd@googlegroups.com> Message-ID: > On Feb 17, 2016, at 2:49 PM, wrong.address.1 at gmail.com wrote: > > I am mostly getting positive feedback for Python. > I would be surprised if you weren?t. > It seems Python is used more for web based applications. Is it equally fine for creating stand-alone *.exe's? Can the same code be compiled to run on Linux or Android or web-based? > I?m not sure where you got that idea. Python has been and is being used for an extremely broad range of applications. At one extreme, the popular YouTube site is based on Python (and yes, that is a web application), at the other extreme much (most?) of the data analysis of the recent LIGO data leading up to the discovery of gravitational waves was done in Python, and numpy (a python numerical library); with the results displayed in matplotlib (a python library for scientific-engineering graphics). The basic python language and its libraries are completely cross-platform (Linux, OS-X, Windows, iOS, and Android). The same source code will run on all of them. However, because it is an Interpreted language, the step to a bundled, self-contained image is platform dependent. There python compliers (as opposed to interpreters) for some, but not all platforms, and there is a very active JiT compiler project that would fall somewhere in between. > Is it possible to create GUI elements with a good IDE? Can they be defined like in Visual Basic with given sizes, fonts, visible/invisible, etc.? > There are several GUI libraries for Python. You might check out TK (which comes built-in), but there are also libraries like wxPython, Kivi (which is completely cross platform), and PyQt. Note that these are the GUI libraries themselves, not IDE?s which tend to be independent. There are too many IDE?s to list here - check with Google. > Is it easy to do matrix operations in Python? Or do I need to write subroutines like in Visual Basic? Matrix (and similar operations) can be done in Python directly, but if you need maximum performance you should use the optimized vector routines available in numpy, these are typically pre-compiled with C or FORTRAN and squeeze out the maximum speed your hardware is capable of. (There is work being done to migrate them to GPUs, but is VERY hardware dependent at this point.) If you need to do scientific/engineering analysis, you should also check out scipy, which is built on top of it and is also vector optimized. > > Could someone kindly tell me advantages and disadvantages of Python? If you are completely satisfied with VB, there is no reason to change (other than the enjoyable exercise of learning a new and more powerful language). On the other hand, Python has often been referred to as ?executable pseudo code? - in other words, the english-like steps you might sketch out to outline the solution to a programming problem in another language is to all intents and purposes Python itself. This makes for very fast and productive code generation with typically fewer errors than are made in other languages. In addition, and as has been implied above, it is cross platform. The same python source code (with the appropriate libraries) will run on all the platforms mentioned above. One of the biggest productive features of Python is the fact that it is interpreted (in its most common instantiation). Thus, you can have two windows open side by side, with the source code in one and running code in the other. Changes in the source code can instantly be reflected in the output. > Or any better options? I have like 40-50 VB Forms and may be around 20000 lines of code. It will be a task to learn a new language and translate/re-write that code. > Not necessarily better, but at least worth looking into would be the Swift language, which shares much of Python?s modern structure, is now also open source, and has been ported to most flavors of Linux. A Window?s port is promised, but I don?t know how far along it is. It *is* a compiled language, so you would then be back in the edit, compile, link, and run loop as you debug your code. I?m sure you will get lots of other answers - Google each of the things I?ve mentioned and you will get a truck-load of info. You might start with Wikipedia and read there about Python and Swift. > Thanks for your responses. > -- > https://mail.python.org/mailman/listinfo/python-list From breamoreboy at yahoo.co.uk Wed Feb 17 17:08:26 2016 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Wed, 17 Feb 2016 22:08:26 +0000 Subject: How the heck does async/await work in Python 3.5 Message-ID: Seeing there is a lot of interest in asyncio recently I figured people might be interested in this http://www.snarky.ca/how-the-heck-does-async-await-work-in-python-3-5 -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From framstag at rus.uni-stuttgart.de Wed Feb 17 17:37:29 2016 From: framstag at rus.uni-stuttgart.de (Ulli Horlacher) Date: Wed, 17 Feb 2016 22:37:29 +0000 (UTC) Subject: extending PATH on Windows? References: Message-ID: eryk sun wrote: > >> The AutoRun command (it's a command line, not a script path) > > > > A script path is a legal command line, too. > > If the registry value were just a script path, you'd have to modify > your script to chain to the previous script, if any. Since it's a > command line you can simply use the & operator to append another > command. This is the better way, no question. I will change it. > Do you really intend for your batch file to be run every time cmd.exe > is executed, including every time that every program on the machine > calls the CRT system() function? Yes. It just extends PATH, it should not do any harm. > Why don't you just install a shortcut to a batch file that starts a > command prompt with the extended PATH? Hmm... how does a user starts this shortcut, when it is not found in PATH? > system('reg...') won't be able to do that unless you export the key to > a .reg file and parse the existing AutoRun value. It would be simpler > to use subprocess.check_output('reg query ...') I do it this way, already. > but simpler still and more reliable to just call QueryValueEx. I find it more complicated. -- Ullrich Horlacher Server und Virtualisierung Rechenzentrum IZUS/TIK E-Mail: horlacher at tik.uni-stuttgart.de Universitaet Stuttgart Tel: ++49-711-68565868 Allmandring 30a Fax: ++49-711-682357 70550 Stuttgart (Germany) WWW: http://www.tik.uni-stuttgart.de/ From jfong at ms4.hinet.net Wed Feb 17 20:29:17 2016 From: jfong at ms4.hinet.net (jfong at ms4.hinet.net) Date: Wed, 17 Feb 2016 17:29:17 -0800 (PST) Subject: Will file be closed automatically in a "for ... in open..." statement? In-Reply-To: References: Message-ID: The "for ... open ..." is definitely not a good design pattern. It opens a file at "for" block but leaves it closed somewhere in the sky. > The garbage collector will: > - reclaim the memory used by the object; > - close the file. I suppose (IMO) that the primitive idea of garbage collection doing the file-close affair is to make itself a final defense to prevent the possible disaster. (after all it's not of his business:-) But some coding may abuse this advantage and forget its hidden danger. It's better to drop this "for...open..." style completely and stick at "with". Thanks for all you gurus clarify my puzzle. I deeply appreciated it. --Jach From breamoreboy at yahoo.co.uk Wed Feb 17 21:05:07 2016 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Thu, 18 Feb 2016 02:05:07 +0000 Subject: Will file be closed automatically in a "for ... in open..." statement? In-Reply-To: References: Message-ID: On 18/02/2016 01:29, jfong at ms4.hinet.net wrote: > The "for ... open ..." is definitely not a good design pattern. It opens a file at "for" block but leaves it closed somewhere in the sky. > Hardly, as all ready explained, but how about this handle = open('foo.txt') for line in handle : ...do something... handle.close() Is that better? -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From cs at zip.com.au Wed Feb 17 21:22:49 2016 From: cs at zip.com.au (Cameron Simpson) Date: Thu, 18 Feb 2016 13:22:49 +1100 Subject: Will file be closed automatically in a "for ... in open..." statement? In-Reply-To: References: Message-ID: <20160218022249.GA56834@cskk.homeip.net> On 18Feb2016 02:05, Mark Lawrence wrote: >On 18/02/2016 01:29, jfong at ms4.hinet.net wrote: >>The "for ... open ..." is definitely not a good design pattern. It opens a file at "for" block but leaves it closed somewhere in the sky. >> > >Hardly, as all ready explained, but how about this > >handle = open('foo.txt') >for line in handle : > ...do something... >handle.close() > >Is that better? As already discussed in this thread, not in the face of an exception inside the loop or some other change of flow control (early "return" statement etc). I'm pretty sure jfong understands the issues now, certainly enough to make his/her own decisions on the benefits and risks of with versus "for ... open" and similar variants. Cheers, Cameron Simpson From python.list at tim.thechases.com Wed Feb 17 21:32:59 2016 From: python.list at tim.thechases.com (Tim Chase) Date: Wed, 17 Feb 2016 20:32:59 -0600 Subject: Will file be closed automatically in a "for ... in open..." statement? In-Reply-To: <56c40a67$0$11104$c3e8da3@news.astraweb.com> References: <230cfde5-e6f2-45fe-9739-e56c5c7739ab@googlegroups.com> <56c40a67$0$11104$c3e8da3@news.astraweb.com> Message-ID: <20160217203259.0f99b558@bigbox.christie.dr> On 2016-02-17 16:51, Steven D'Aprano wrote: > If you want the file to be closed immediately, you must: > > - use a with statement; > > - or explicitly call f.close() I have a lot of pre-"with" code (i.e., Py2.4) that looks like f = open(...) try: do_stuff() finally: f.close() To explicitly close() correctly, you still have to pay the one-level-of-indent cost regardless of whether you use a "with" or close() Now that we have the "with" statement, it's the same cost, with no lost functionality, but with fewer lines of messy code. So use the "with" unless you're sadly maintaining 2.4 code like me. :-) -tkc From no.email at nospam.invalid Wed Feb 17 23:38:14 2016 From: no.email at nospam.invalid (Paul Rubin) Date: Wed, 17 Feb 2016 20:38:14 -0800 Subject: asyncio - run coroutine in the background References: <8737sumpjl.fsf@elektro.pacujo.net> <87h9ha8lt0.fsf@jester.gateway.pace.com> <87d1rwpwo2.fsf@elektro.pacujo.net> <56c33d7e$0$1587$c3e8da3$5496439d@news.astraweb.com> <87vb5ok2ab.fsf@elektro.pacujo.net> Message-ID: <87vb5m7hvt.fsf@jester.gateway.pace.com> Marko Rauhamaa writes: > @asyncio.coroutine > def background_task(): ... > while time.time() - t < 10: > pass Wait, that's a cpu-busy loop, you can't do that in cooperative multitasking. Of course you need a wait there. From steve+comp.lang.python at pearwood.info Thu Feb 18 00:42:17 2016 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Thu, 18 Feb 2016 16:42:17 +1100 Subject: Python keyword args can be any string Message-ID: <56c559c2$0$2916$c3e8da3$76491128@news.astraweb.com> Today I learned that **kwargs style keyword arguments can be any string: py> def test(**kw): ... print(kw) ... py> kwargs = {'abc-def': 42, '': 23, '---': 999, '123': 17} py> test(**kwargs) {'': 23, '123': 17, '---': 999, 'abc-def': 42} Bug or feature? -- Steve From rosuav at gmail.com Thu Feb 18 00:57:23 2016 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 18 Feb 2016 16:57:23 +1100 Subject: Python keyword args can be any string In-Reply-To: <56c559c2$0$2916$c3e8da3$76491128@news.astraweb.com> References: <56c559c2$0$2916$c3e8da3$76491128@news.astraweb.com> Message-ID: On Thu, Feb 18, 2016 at 4:42 PM, Steven D'Aprano wrote: > Today I learned that **kwargs style keyword arguments can be any string: > > > py> def test(**kw): > ... print(kw) > ... > py> kwargs = {'abc-def': 42, '': 23, '---': 999, '123': 17} > py> test(**kwargs) > {'': 23, '123': 17, '---': 999, 'abc-def': 42} > > > Bug or feature? Probably neither. It's something that can't hurt, so there's no point specifically blocking it. You can do the same thing with other dictionaries: >>> globals()["abc-def"] = 42 >>> dir() ['__builtins__', '__doc__', '__loader__', '__name__', '__package__', '__spec__', 'abc-def'] >>> class Blob: pass ... >>> b = Blob() >>> b.__dict__[""] = 23 >>> dir(b) ['', '__class__', '__delattr__', '__dict__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__gt__', '__hash__', '__init__', '__le__', '__lt__', '__module__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '__weakref__'] I suppose it's possible for this to be a vulnerability, eg if you build up an XML node using keyword arguments for attributes, and end up accepting something with a space in it. But most of the time, the only consequence is that you use a dict to create a situation that can only be handled with another dict. Doesn't seem worth the hassle of preventing it, but I would also see this as a bizarre thing to deliberately exploit. ChrisA From ben+python at benfinney.id.au Thu Feb 18 00:59:47 2016 From: ben+python at benfinney.id.au (Ben Finney) Date: Thu, 18 Feb 2016 16:59:47 +1100 Subject: Python keyword args can be any string References: <56c559c2$0$2916$c3e8da3$76491128@news.astraweb.com> Message-ID: <857fi236ek.fsf@benfinney.id.au> Steven D'Aprano writes: > Today I learned that **kwargs style keyword arguments can be any string: > > > py> def test(**kw): > ... print(kw) > ... > py> kwargs = {'abc-def': 42, '': 23, '---': 999, '123': 17} > py> test(**kwargs) > {'': 23, '123': 17, '---': 999, 'abc-def': 42} > > > Bug or feature? Incidental feature, I think. If the caller is deliberately unpacking a dict, it's on them to ensure the keys are valid identifiers or wear the consequences. If the function cares so little about the keys in its kwargs that (as in your example) any string will do for each key, then it's a consenting actor IMO. The times when it's a problem ? i.e. that the function is trying to use items from that dictionary as name bindings ? the errors will be raised to the caller that unpacked that mapping. That's where the errors belong. -- \ ?There's a fine line between fishing and standing on the shore | `\ looking like an idiot.? ?Steven Wright | _o__) | Ben Finney From marko at pacujo.net Thu Feb 18 01:10:38 2016 From: marko at pacujo.net (Marko Rauhamaa) Date: Thu, 18 Feb 2016 08:10:38 +0200 Subject: asyncio - run coroutine in the background References: <8737sumpjl.fsf@elektro.pacujo.net> <87h9ha8lt0.fsf@jester.gateway.pace.com> <87d1rwpwo2.fsf@elektro.pacujo.net> <56c33d7e$0$1587$c3e8da3$5496439d@news.astraweb.com> <87vb5ok2ab.fsf@elektro.pacujo.net> <87vb5m7hvt.fsf@jester.gateway.pace.com> Message-ID: <87lh6i8s69.fsf@elektro.pacujo.net> Paul Rubin : > Marko Rauhamaa writes: >> @asyncio.coroutine >> def background_task(): ... >> while time.time() - t < 10: >> pass > > Wait, that's a cpu-busy loop, you can't do that in cooperative > multitasking. Of course you need a wait there. That was the very point: to demonstrate that coroutines monopolize the CPU. Marko From tjreedy at udel.edu Thu Feb 18 01:31:12 2016 From: tjreedy at udel.edu (Terry Reedy) Date: Thu, 18 Feb 2016 01:31:12 -0500 Subject: Python keyword args can be any string In-Reply-To: <857fi236ek.fsf@benfinney.id.au> References: <56c559c2$0$2916$c3e8da3$76491128@news.astraweb.com> <857fi236ek.fsf@benfinney.id.au> Message-ID: On 2/18/2016 12:59 AM, Ben Finney wrote: > Steven D'Aprano writes: > >> Today I learned that **kwargs style keyword arguments can be any string: >> >> >> py> def test(**kw): >> ... print(kw) >> ... >> py> kwargs = {'abc-def': 42, '': 23, '---': 999, '123': 17} >> py> test(**kwargs) >> {'': 23, '123': 17, '---': 999, 'abc-def': 42} >> >> >> Bug or feature? > > Incidental feature, I think. The was discussed on pydev some years ago. Guido says leave as is. This and similar cases of allowing non-identifier is string-dicts does not seem to cause a problem and apparently has a few rare legitimate usages. > If the caller is deliberately unpacking a dict, it's on them to ensure > the keys are valid identifiers or wear the consequences. > > If the function cares so little about the keys in its kwargs that (as in > your example) any string will do for each key, then it's a consenting > actor IMO. > > The times when it's a problem ? i.e. that the function is trying to use > items from that dictionary as name bindings ? the errors will be raised > to the caller that unpacked that mapping. That's where the errors belong. > -- Terry Jan Reedy From steve+comp.lang.python at pearwood.info Thu Feb 18 01:47:07 2016 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Thu, 18 Feb 2016 17:47:07 +1100 Subject: Guido on python3 for beginners References: <48762040-a7e0-434c-92e8-8a0969210e5b@googlegroups.com> <5d31dd14-1a60-4f80-8889-d0616f404c70@googlegroups.com> <4e902c58-3370-4adf-905d-74630a6eea90@googlegroups.com> <60036746-78f7-4f05-a6d8-04e88b09c00f@googlegroups.com> <032f0f66-34df-4325-98e0-1127025d2a94@googlegroups.com> <31a618b2-a407-4723-9a4f-9b756fc93b0b@googlegroups.com> <0a920e92-5d31-4866-815b-cae7e201e4d8@googlegroups.com> Message-ID: <56c568f2$0$2832$c3e8da3$76491128@news.astraweb.com> On Wednesday 17 February 2016 19:51, Rustom Mody wrote: > I hope someone can help me find this link: There is some record that Guido > has said that python3 is probably a bit harder on noobs than python2. > > Does anyone know/have that link? I can't say that I've seen it. I know that Raymond Hettinger is not too fond of adding new syntactic features that add little in the way of power but make the language harder to learn, but I don't recall anyone saying that it is harder for newbies to get started with Python 3 than Python 2. There are more features in Python 3, so in that trivial sense of "more to learn", I suppose that it is objectively correct that it is harder to learn than Python 2. But I don't think the learning curve is any steeper. If anything, the learning curve is ever-so-slightly less steep. -- Steve From ben+python at benfinney.id.au Thu Feb 18 01:57:43 2016 From: ben+python at benfinney.id.au (Ben Finney) Date: Thu, 18 Feb 2016 17:57:43 +1100 Subject: Guido on python3 for beginners References: <48762040-a7e0-434c-92e8-8a0969210e5b@googlegroups.com> <5d31dd14-1a60-4f80-8889-d0616f404c70@googlegroups.com> <4e902c58-3370-4adf-905d-74630a6eea90@googlegroups.com> <60036746-78f7-4f05-a6d8-04e88b09c00f@googlegroups.com> <032f0f66-34df-4325-98e0-1127025d2a94@googlegroups.com> <31a618b2-a407-4723-9a4f-9b756fc93b0b@googlegroups.com> <0a920e92-5d31-4866-815b-cae7e201e4d8@googlegroups.com> <56c568f2$0$2832$c3e8da3$76491128@news.astraweb.com> Message-ID: <8537sq33q0.fsf@benfinney.id.au> Steven D'Aprano writes: > There are more features in Python 3, so in that trivial sense of "more to > learn", I suppose that it is objectively correct that it is harder to learn > than Python 2. But I don't think the learning curve is any steeper. Set against the ?more features in Python 3? is the ?more inconsistencies and cruft in Python 2?. How to weigh those objectively is beyond me, but my experience is that the inconsistencies in Python 2 make it more difficult for newcomers than Python 3. > If anything, the learning curve is ever-so-slightly less steep. Yes. The language Python 3 is more consistent with fewer historical edge cases the beginner needs to know; and the new features don't have to be learned all at once. Both of which mean that the newcomer has an easier time in Python 3. -- \ ?Not to perambulate the corridors in the hours of repose in the | `\ boots of ascension.? ?ski hotel, Austria | _o__) | Ben Finney From no.email at nospam.invalid Thu Feb 18 01:59:14 2016 From: no.email at nospam.invalid (Paul Rubin) Date: Wed, 17 Feb 2016 22:59:14 -0800 Subject: Guido on python3 for beginners References: <48762040-a7e0-434c-92e8-8a0969210e5b@googlegroups.com> <5d31dd14-1a60-4f80-8889-d0616f404c70@googlegroups.com> <4e902c58-3370-4adf-905d-74630a6eea90@googlegroups.com> <60036746-78f7-4f05-a6d8-04e88b09c00f@googlegroups.com> <032f0f66-34df-4325-98e0-1127025d2a94@googlegroups.com> <31a618b2-a407-4723-9a4f-9b756fc93b0b@googlegroups.com> <0a920e92-5d31-4866-815b-cae7e201e4d8@googlegroups.com> <56c568f2$0$2832$c3e8da3$76491128@news.astraweb.com> Message-ID: <87r3ga7bct.fsf@jester.gateway.pace.com> Steven D'Aprano writes: > I suppose that it is objectively correct that it is harder to learn > than Python 2. But I don't think the learning curve is any steeper. If > anything, the learning curve is ever-so-slightly less steep. I think py3 has more learning curve because it uses iterators in places where py2 uses lists. That's a significant new concept and it can be bug-prone even for programmers who are experienced with it. From random832 at fastmail.com Thu Feb 18 02:00:55 2016 From: random832 at fastmail.com (Random832) Date: Thu, 18 Feb 2016 02:00:55 -0500 Subject: Guido on python3 for beginners In-Reply-To: <56c568f2$0$2832$c3e8da3$76491128@news.astraweb.com> References: <48762040-a7e0-434c-92e8-8a0969210e5b@googlegroups.com> <5d31dd14-1a60-4f80-8889-d0616f404c70@googlegroups.com> <4e902c58-3370-4adf-905d-74630a6eea90@googlegroups.com> <60036746-78f7-4f05-a6d8-04e88b09c00f@googlegroups.com> <032f0f66-34df-4325-98e0-1127025d2a94@googlegroups.com> <31a618b2-a407-4723-9a4f-9b756fc93b0b@googlegroups.com> <0a920e92-5d31-4866-815b-cae7e201e4d8@googlegroups.com> <56c568f2$0$2832$c3e8da3$76491128@news.astraweb.com> Message-ID: <1455778855.2341509.524603842.2D391124@webmail.messagingengine.com> On Thu, Feb 18, 2016, at 01:47, Steven D'Aprano wrote: > There are more features in Python 3, so in that trivial sense of "more to > learn", I suppose that it is objectively correct that it is harder to > learn > than Python 2. But I don't think the learning curve is any steeper. If > anything, the learning curve is ever-so-slightly less steep. I wonder if (with one of the major differences being the unicode thing) there is a differences between the learning curve for people whose primary prior experience is with languages that use byte strings for text (such as perl, [as typically used] C, shell/awk/etc, PHP, python 2) vs languages that use some form of unicode string (UTF-8 byte strings on a platform whose default encoding is also UTF-8 don't count) for text (such as java, C#, javascript). I feel like the unicode string stuff (issues like encodings etc) is something that you only have to learn _once_, and then if you really understand it then for a new language you can just look up how to do it. From rosuav at gmail.com Thu Feb 18 02:27:59 2016 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 18 Feb 2016 18:27:59 +1100 Subject: Guido on python3 for beginners In-Reply-To: <56c568f2$0$2832$c3e8da3$76491128@news.astraweb.com> References: <48762040-a7e0-434c-92e8-8a0969210e5b@googlegroups.com> <5d31dd14-1a60-4f80-8889-d0616f404c70@googlegroups.com> <4e902c58-3370-4adf-905d-74630a6eea90@googlegroups.com> <60036746-78f7-4f05-a6d8-04e88b09c00f@googlegroups.com> <032f0f66-34df-4325-98e0-1127025d2a94@googlegroups.com> <31a618b2-a407-4723-9a4f-9b756fc93b0b@googlegroups.com> <0a920e92-5d31-4866-815b-cae7e201e4d8@googlegroups.com> <56c568f2$0$2832$c3e8da3$76491128@news.astraweb.com> Message-ID: On Thu, Feb 18, 2016 at 5:47 PM, Steven D'Aprano wrote: > There are more features in Python 3, so in that trivial sense of "more to > learn", I suppose that it is objectively correct that it is harder to learn > than Python 2. But I don't think the learning curve is any steeper. If > anything, the learning curve is ever-so-slightly less steep. Let's see... changes in Py3. 1) Division of two integers now yields a float instead of flooring. For someone fresh to programming, that's a Py3 advantage, although it can cause surprises elsewhere. But since 1.0==1, it's not going to be a problem for a new programmer. Advantage: Py3. 2) Strings are Unicode text, and files etc may need to have their encodings declared. Definitely causes some issues in ASCII-only situations, where a lot of other languages (notably including PHP, for the people building web sites) let you be sloppy. Advantage: Py3 if you speak any language other than English; otherwise Py2 in the very short term, neither in the medium term, and most definitely Py3 in the long term (no more "funny characters break my program" errors long after deployment). 3) Laziness. When you explain to someone what the range() function does, Py2 makes a list, but Py3 makes... a range. It doesn't really answer the question at all. When you ask Py2 for a dictionary's keys/values, you get a list; Py3 gives you a thing that mostly acts like a list, only it isn't. If you map a function over a list, you get back a lazy thing that will eventually call that function. Py2 often has less levels of indirection, ergo less things to try to explain. Advantage: Py2; the benefits (lower memory usage, etc) aren't significant to new users. 4) Exception chaining. You get more information when errors cascade. Advantage: Py3, easily and without any question. 5) print statement/function. Py3 forces you to put parentheses on it, which is no different from C's printf() or Pike's write() or any number of other languages where console I/O needs no language support. Maybe a tiny TINY advantage to Py2 in the short term, but as soon as you introduce the less basic features, keyword arguments are way better than the magic syntax the statement needs. (Also, trying to explain the interaction between the print statement's "soft space" and other console I/O is not easy.) By the time you've really learned the language, the advantage belongs to Py3. 6) The fact that the name "python" may not invoke the interpreter you want. Advantage: Py2, if any; there'll be times when they're on par, but Py3 never comes out ahead. 7) Whether or not the interpreter comes pre-installed on your system. As of a few years ago, that was a clear advantage to Py2 (most systems would ship with both, or neither, or Py2 only), but that's shifting. It's only a small difference, though; on Windows, you generally get nothing, and on any system with a decent package manager, you should be able to request either version with ease. It's actually a pretty tough call. Most of the Py3 advantages aren't for the absolute beginner; it's not easier to write "Hello, world" in Py3, and aside from the change to integer division, most of the changes won't benefit small-to-medium scripts either. The biggest advantage (Unicode by default) really only shows itself by sparing you hassles later on - it's not going to make your life easier in the short term, ergo it's not going to make the language easier to learn. Py3 isn't so much easier as _better_. There are specific situations where it's massively better, but for the most part, they're about on par. ChrisA From breamoreboy at yahoo.co.uk Thu Feb 18 02:55:59 2016 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Thu, 18 Feb 2016 07:55:59 +0000 Subject: Python keyword args can be any string In-Reply-To: <56c559c2$0$2916$c3e8da3$76491128@news.astraweb.com> References: <56c559c2$0$2916$c3e8da3$76491128@news.astraweb.com> Message-ID: On 18/02/2016 05:42, Steven D'Aprano wrote: > Today I learned that **kwargs style keyword arguments can be any string: > > py> def test(**kw): > ... print(kw) > ... > py> kwargs = {'abc-def': 42, '': 23, '---': 999, '123': 17} > py> test(**kwargs) > {'': 23, '123': 17, '---': 999, 'abc-def': 42} > > Bug or feature? > I'm not sure, but what on earth got you to this in the first place? -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From songofacandy at gmail.com Thu Feb 18 03:00:32 2016 From: songofacandy at gmail.com (INADA Naoki) Date: Thu, 18 Feb 2016 17:00:32 +0900 Subject: Guido on python3 for beginners In-Reply-To: References: <48762040-a7e0-434c-92e8-8a0969210e5b@googlegroups.com> <5d31dd14-1a60-4f80-8889-d0616f404c70@googlegroups.com> <4e902c58-3370-4adf-905d-74630a6eea90@googlegroups.com> <60036746-78f7-4f05-a6d8-04e88b09c00f@googlegroups.com> <032f0f66-34df-4325-98e0-1127025d2a94@googlegroups.com> <31a618b2-a407-4723-9a4f-9b756fc93b0b@googlegroups.com> <0a920e92-5d31-4866-815b-cae7e201e4d8@googlegroups.com> <56c568f2$0$2832$c3e8da3$76491128@news.astraweb.com> Message-ID: In Python 3, I don't required to teach followings to newbies. 1. Don't do `class Foo:`, do `class Foo(object):`. 2. Don't do `isinstance(x, int)`, do `isinstance(x, (int, long))`. 3. Don't return non-ASCII string from `__repr__`, otherwise UnicodeError happens in logging and you will lost your important log. 4. Use %r instead of %s in logging to avoid UnicodeError when __str__ returns non ASCII strings. I think there are many pitfalls fixed in Python 3 other than above. Python 3 is far easier to teach and review code than Python 2. On Thu, Feb 18, 2016 at 4:27 PM, Chris Angelico wrote: > On Thu, Feb 18, 2016 at 5:47 PM, Steven D'Aprano > wrote: > > There are more features in Python 3, so in that trivial sense of "more to > > learn", I suppose that it is objectively correct that it is harder to > learn > > than Python 2. But I don't think the learning curve is any steeper. If > > anything, the learning curve is ever-so-slightly less steep. > > Let's see... changes in Py3. > > 1) Division of two integers now yields a float instead of flooring. > For someone fresh to programming, that's a Py3 advantage, although it > can cause surprises elsewhere. But since 1.0==1, it's not going to be > a problem for a new programmer. Advantage: Py3. > > 2) Strings are Unicode text, and files etc may need to have their > encodings declared. Definitely causes some issues in ASCII-only > situations, where a lot of other languages (notably including PHP, for > the people building web sites) let you be sloppy. Advantage: Py3 if > you speak any language other than English; otherwise Py2 in the very > short term, neither in the medium term, and most definitely Py3 in the > long term (no more "funny characters break my program" errors long > after deployment). > > 3) Laziness. When you explain to someone what the range() function > does, Py2 makes a list, but Py3 makes... a range. It doesn't really > answer the question at all. When you ask Py2 for a dictionary's > keys/values, you get a list; Py3 gives you a thing that mostly acts > like a list, only it isn't. If you map a function over a list, you get > back a lazy thing that will eventually call that function. Py2 often > has less levels of indirection, ergo less things to try to explain. > Advantage: Py2; the benefits (lower memory usage, etc) aren't > significant to new users. > > 4) Exception chaining. You get more information when errors cascade. > Advantage: Py3, easily and without any question. > > 5) print statement/function. Py3 forces you to put parentheses on it, > which is no different from C's printf() or Pike's write() or any > number of other languages where console I/O needs no language support. > Maybe a tiny TINY advantage to Py2 in the short term, but as soon as > you introduce the less basic features, keyword arguments are way > better than the magic syntax the statement needs. (Also, trying to > explain the interaction between the print statement's "soft space" and > other console I/O is not easy.) By the time you've really learned the > language, the advantage belongs to Py3. > > 6) The fact that the name "python" may not invoke the interpreter you > want. Advantage: Py2, if any; there'll be times when they're on par, > but Py3 never comes out ahead. > > 7) Whether or not the interpreter comes pre-installed on your system. > As of a few years ago, that was a clear advantage to Py2 (most systems > would ship with both, or neither, or Py2 only), but that's shifting. > It's only a small difference, though; on Windows, you generally get > nothing, and on any system with a decent package manager, you should > be able to request either version with ease. > > It's actually a pretty tough call. Most of the Py3 advantages aren't > for the absolute beginner; it's not easier to write "Hello, world" in > Py3, and aside from the change to integer division, most of the > changes won't benefit small-to-medium scripts either. The biggest > advantage (Unicode by default) really only shows itself by sparing you > hassles later on - it's not going to make your life easier in the > short term, ergo it's not going to make the language easier to learn. > Py3 isn't so much easier as _better_. There are specific situations > where it's massively better, but for the most part, they're about on > par. > > ChrisA > -- > https://mail.python.org/mailman/listinfo/python-list > -- INADA Naoki From wrong.address.1 at gmail.com Thu Feb 18 03:17:42 2016 From: wrong.address.1 at gmail.com (wrong.address.1 at gmail.com) Date: Thu, 18 Feb 2016 00:17:42 -0800 (PST) Subject: Considering migrating to Python from Visual Basic 6 for engineering applications In-Reply-To: References: <90cc50d2-1ce5-4588-9bfd-a49d439f00dd@googlegroups.com> Message-ID: <14c75a68-0d2e-45cc-8d73-0d71b6a6aea6@googlegroups.com> Thanks to all of you who have responded. One point two of you ask me is why I want to shift from Visual Basic. Most of my software is in VB6 (and VB3), and converting to VB.net is not very pleasant. I have learnt enough VB.net to manage most of the things I will need, but it always gives me a feeling that it is not meant for the kind of things I am doing. And you never know when Microsoft will kick us out of VB.net also. For the sake of keeping software compatible with new versions of Windows, you need to keep changing things in the software, which takes too much time. You might ask if it is really necessary to move from VB6 to VB.net because Microsoft will still not be able to kill VB6 very soon, thanks to VBA used in Excel. If my reasoning is wrong, please correct me. I am just an engineer, not a good programmer. Is it easy to create vector graphics files of plots in Python? In VB6, I wrote my own code to generate EPS files. Then people who demand jpg and gif files could be given those by converting the EPS with the desired resolution. More points in the text below. William Ray Wing writes: > > On Feb 17, 2016, at 2:49 PM, wrong.address.1 at gmail.com wrote: > > > > It seems Python is used more for web based applications. Is it equally fine for creating stand-alone *.exe's? Can the same code be compiled to run on Linux or Android or web-based? > > > > I'm not sure where you got that idea. Python has been and is being used for an extremely broad range of applications. At one extreme, the popular YouTube site is based on Python (and yes, that is a web application), at the other extreme much (most?) of the data analysis of the recent LIGO data leading up to the discovery of gravitational waves was done in Python, and numpy (a python numerical library); with the results displayed in matplotlib (a python library for scientific-engineering graphics). > This is good news, because I need several kinds of plots. > The basic python language and its libraries are completely cross-platform (Linux, OS-X, Windows, iOS, and Android). The same source code will run on all of them. However, because it is an Interpreted language, the step to a bundled, self-contained image is platform dependent. There python compliers (as opposed to interpreters) for some, but not all platforms, and there is a very active JiT compiler project that would fall somewhere in between. > OK. This is clear to me now. It is like needing different Fortran compilers for different operating systems, but Fortran code does not need much modifications (except file names/paths, etc.). > > Is it possible to create GUI elements with a good IDE? Can they be defined like in Visual Basic with given sizes, fonts, visible/invisible, etc.? > > > > There are several GUI libraries for Python. You might check out TK (which comes built-in), but there are also libraries like wxPython, Kivi (which is completely cross platform), and PyQt. Note that these are the GUI libraries themselves, not IDE's which tend to be independent. There are too many IDE's to list here - check with Google. > > > Is it easy to do matrix operations in Python? Or do I need to write subroutines like in Visual Basic? > > Matrix (and similar operations) can be done in Python directly, but if you need maximum performance you should use the optimized vector routines available Speed is usually not an issue. > in numpy, these are typically pre-compiled with C or FORTRAN and squeeze out the maximum speed your hardware is capable of. (There is work being done to migrate them to GPUs, but is VERY hardware dependent at this point.) If you need to do scientific/engineering analysis, you should also check out scipy, which is built on top of it and is also vector optimized. > > > > > Could someone kindly tell me advantages and disadvantages of Python? > > If you are completely satisfied with VB, there is no reason to change (other than the enjoyable exercise of learning a new and more powerful language). > Will VB6 run on Windows 11? Or can I compile VB6 on Windows 11. We can't know. But there will be easy ways to run Python on any Windows version or Linux or whatever. > On the other hand, Python has often been referred to as "executable pseudo code" - in other words, the english-like steps you might sketch out to outline the solution to a programming problem in another language is to all intents and purposes Python itself. This makes for very fast and productive code generation with typically fewer errors than are made in other languages. In addition, and as has been implied above, it is cross platform. The same python source code (with the appropriate libraries) will run on all the platforms mentioned above. One of the biggest productive features of Python is the fact that it is interpreted (in its most common instantiation). Thus, you can have two windows open side by side, with the source code in one and running code in the other. Changes in the source code can instantly be reflected in the output. > > > Or any better options? I have like 40-50 VB Forms and may be around 20000 lines of code. It will be a task to learn a new language and translate/re-write that code. > > > > Not necessarily better, but at least worth looking into would be the Swift language, which shares much of Python's modern structure, is now also open source, and has been ported to most flavors of Linux. A Window's port is promised, but I don't know how far along it is. It *is* a compiled language, so you would then be back in the edit, compile, link, and run loop as you debug your code. > > I'm sure you will get lots of other answers - Google each of the things I've mentioned and you will get a truck-load of info. You might start with Wikipedia and read there about Python and Swift. > > > Thanks for your responses. Thanks again to all of you who have responded. > > -- > > https://mail.python.org/mailman/listinfo/python-list From tjreedy at udel.edu Thu Feb 18 03:40:07 2016 From: tjreedy at udel.edu (Terry Reedy) Date: Thu, 18 Feb 2016 03:40:07 -0500 Subject: Guido on python3 for beginners In-Reply-To: References: <48762040-a7e0-434c-92e8-8a0969210e5b@googlegroups.com> <5d31dd14-1a60-4f80-8889-d0616f404c70@googlegroups.com> <4e902c58-3370-4adf-905d-74630a6eea90@googlegroups.com> <60036746-78f7-4f05-a6d8-04e88b09c00f@googlegroups.com> <032f0f66-34df-4325-98e0-1127025d2a94@googlegroups.com> <31a618b2-a407-4723-9a4f-9b756fc93b0b@googlegroups.com> <0a920e92-5d31-4866-815b-cae7e201e4d8@googlegroups.com> <56c568f2$0$2832$c3e8da3$76491128@news.astraweb.com> Message-ID: On 2/18/2016 2:27 AM, Chris Angelico wrote: > On Thu, Feb 18, 2016 at 5:47 PM, Steven D'Aprano > wrote: >> There are more features in Python 3, so in that trivial sense of "more to >> learn", I suppose that it is objectively correct that it is harder to learn >> than Python 2. But I don't think the learning curve is any steeper. If >> anything, the learning curve is ever-so-slightly less steep. > > Let's see... changes in Py3. > > 1) Division of two integers now yields a float instead of flooring. > For someone fresh to programming, that's a Py3 advantage, although it > can cause surprises elsewhere. But since 1.0==1, it's not going to be > a problem for a new programmer. Advantage: Py3. > > 2) Strings are Unicode text, and files etc may need to have their > encodings declared. Definitely causes some issues in ASCII-only > situations, where a lot of other languages (notably including PHP, for > the people building web sites) let you be sloppy. Advantage: Py3 if > you speak any language other than English; otherwise Py2 in the very > short term, neither in the medium term, and most definitely Py3 in the > long term (no more "funny characters break my program" errors long > after deployment). > > 3) Laziness. When you explain to someone what the range() function > does, Py2 makes a list, but Py3 makes... a range. It doesn't really > answer the question at all. When you ask Py2 for a dictionary's > keys/values, you get a list; Py3 gives you a thing that mostly acts > like a list, only it isn't. If you map a function over a list, you get > back a lazy thing that will eventually call that function. Py2 often > has less levels of indirection, ergo less things to try to explain. > Advantage: Py2; the benefits (lower memory usage, etc) aren't > significant to new users. > > 4) Exception chaining. You get more information when errors cascade. > Advantage: Py3, easily and without any question. > > 5) print statement/function. Py3 forces you to put parentheses on it, > which is no different from C's printf() or Pike's write() or any > number of other languages where console I/O needs no language support. > Maybe a tiny TINY advantage to Py2 in the short term, but as soon as > you introduce the less basic features, keyword arguments are way > better than the magic syntax the statement needs. (Also, trying to > explain the interaction between the print statement's "soft space" and > other console I/O is not easy.) By the time you've really learned the > language, the advantage belongs to Py3. > > 6) The fact that the name "python" may not invoke the interpreter you > want. Advantage: Py2, if any; there'll be times when they're on par, > but Py3 never comes out ahead. > > 7) Whether or not the interpreter comes pre-installed on your system. > As of a few years ago, that was a clear advantage to Py2 (most systems > would ship with both, or neither, or Py2 only), but that's shifting. > It's only a small difference, though; on Windows, you generally get > nothing, and on any system with a decent package manager, you should > be able to request either version with ease. > > It's actually a pretty tough call. Most of the Py3 advantages aren't > for the absolute beginner; it's not easier to write "Hello, world" in > Py3, and aside from the change to integer division, most of the > changes won't benefit small-to-medium scripts either. The biggest > advantage (Unicode by default) really only shows itself by sparing you > hassles later on - it's not going to make your life easier in the > short term, ergo it's not going to make the language easier to learn. > Py3 isn't so much easier as _better_. There are specific situations > where it's massively better, but for the most part, they're about on > par. 8. 2.x has two subtlely different types of classes. The 2.x docs do not document the type of builtin and stdlib classes. I discovered that tkinter classes are still old-style in 2.7 when I backported a patch from 3.x to 2.7 and it mysteriously did not work. Py 3 wins here. To me, this alone makes 2.x a bad choice for most beginners. 9. Two integer classes and the nuisance of 'L' suffix. Py 3 wins for beginners, at least. (Mentioned by Inaki). 10. Two except-clause syntaxes in 2.7, just one in 3.x. -1 syntax is +1 vote for 3.x 11. To test is something is text, isinstance s, c), where 'c' is one of str, bytes, unicode, basestring, (bytes, unicode), (str, unicode). +1 for 3.x. 12. 2.7 has two different open' functions, open and io.open. In 3.x these are the same opjects. I believe there are other 3.x backports like this. 13. 2.7 has two ways to apply arguments to functions: apply and *args. 3.x only has the latter. To my mind, the numerous duplications and overlaps in 2.7 that are gone in 3.x make 2.7 the worse version ever for beginners. -- Terry Jan Reedy From a24061 at ducksburg.com Thu Feb 18 04:26:58 2016 From: a24061 at ducksburg.com (Adam Funk) Date: Thu, 18 Feb 2016 09:26:58 +0000 Subject: Testing whether the VPN is running? Message-ID: <2cfgpcxdg2.ln2@news.ducksburg.com> I'd like to test (inside a python 3 program) whether the VPN is running or not. The only thing I can think of so far is to use subprocess to run the 'ifconfig' command, then check its output for 'tun0'. Is there a better way? Thanks. -- Nam Sibbyllam quidem Cumis ego ipse oculis meis vidi in ampulla pendere, et cum illi pueri dicerent: beable beable beable; respondebat illa: doidy doidy doidy. --- plorkwort From airween at gmail.com Thu Feb 18 04:45:31 2016 From: airween at gmail.com (Ervin =?utf-8?Q?Heged=C3=BCs?=) Date: Thu, 18 Feb 2016 10:45:31 +0100 Subject: Testing whether the VPN is running? In-Reply-To: <2cfgpcxdg2.ln2@news.ducksburg.com> References: <2cfgpcxdg2.ln2@news.ducksburg.com> Message-ID: <20160218094531.GA5510@arxnet.hu> Hi Adam, On Thu, Feb 18, 2016 at 09:26:58AM +0000, Adam Funk wrote: > I'd like to test (inside a python 3 program) whether the VPN is > running or not. The only thing I can think of so far is to use > subprocess to run the 'ifconfig' command, then check its output for > 'tun0'. Is there a better way? you didn't wrote, which system (OS) you want to use - based on the "ifconfig" and "tun0" keywords, possible that's a Linux. I think that the psutil modul could be better for you for this task: https://pypi.python.org/pypi/psutil/ and see the "Network" section. Regards, a. -- I ? UTF-8 From wxjmfauth at gmail.com Thu Feb 18 04:49:07 2016 From: wxjmfauth at gmail.com (wxjmfauth at gmail.com) Date: Thu, 18 Feb 2016 01:49:07 -0800 (PST) Subject: Considering migrating to Python from Visual Basic 6 for engineering applications In-Reply-To: <90cc50d2-1ce5-4588-9bfd-a49d439f00dd@googlegroups.com> References: <90cc50d2-1ce5-4588-9bfd-a49d439f00dd@googlegroups.com> Message-ID: Le mercredi 17 f?vrier 2016 20:49:44 UTC+1, wrong.a... at gmail.com a ?crit?: > I am mostly getting positive feedback for Python. > > It seems Python is used more for web based applications. Is it equally fine for creating stand-alone *.exe's? Can the same code be compiled to run on Linux or Android or web-based? > > Is it possible to create GUI elements with a good IDE? Can they be defined like in Visual Basic with given sizes, fonts, visible/invisible, etc.? > > Is it easy to do matrix operations in Python? Or do I need to write subroutines like in Visual Basic? > > Could someone kindly tell me advantages and disadvantages of Python? Or any better options? I have like 40-50 VB Forms and may be around 20000 lines of code. It will be a task to learn a new language and translate/re-write that code. > > Thanks for your responses. The single solid and reliable GUI toolkits are Jython or IronPython. A serious - exotic - solution, like "PyUNO" in LibreOffice, is all but satisfying. jmf From rosuav at gmail.com Thu Feb 18 04:57:17 2016 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 18 Feb 2016 20:57:17 +1100 Subject: Guido on python3 for beginners In-Reply-To: References: <48762040-a7e0-434c-92e8-8a0969210e5b@googlegroups.com> <5d31dd14-1a60-4f80-8889-d0616f404c70@googlegroups.com> <4e902c58-3370-4adf-905d-74630a6eea90@googlegroups.com> <60036746-78f7-4f05-a6d8-04e88b09c00f@googlegroups.com> <032f0f66-34df-4325-98e0-1127025d2a94@googlegroups.com> <31a618b2-a407-4723-9a4f-9b756fc93b0b@googlegroups.com> <0a920e92-5d31-4866-815b-cae7e201e4d8@googlegroups.com> <56c568f2$0$2832$c3e8da3$76491128@news.astraweb.com> Message-ID: On Thu, Feb 18, 2016 at 7:40 PM, Terry Reedy wrote: > To my mind, the numerous duplications and overlaps in 2.7 that are gone in > 3.x make 2.7 the worse version ever for beginners. Hmm. I was teaching on 2.7 up until last year, and for the most part, we taught a "compatible with Py3" subset of the language, without any significant cost. If you'd shown code saying "except ValueError, e:" to one of my Py2 students then, s/he would have been just as unfamiliar as one of my Py3 students would be today. That said, though, it's still that Py3 is no worse than Py2, and potentially better. The removal of L suffixes (and, similarly, the removal of u"..." prefixes on text strings) is a bigger deal to newbies than it is to experienced programmers, so that one definitely counts. "This is great, but how can I remove that u from the strings?" was a common question (eg when they're printing out a list of strings obtained from a database, or decoded from JSON). The removal of old-style classes is a definite improvement in Py3, as is the no-arg form of super(), which I'd consider a related change. So there's a bunch of tiny little "quality of life" improvements here. ChrisA From a24061 at ducksburg.com Thu Feb 18 05:03:20 2016 From: a24061 at ducksburg.com (Adam Funk) Date: Thu, 18 Feb 2016 10:03:20 +0000 Subject: Testing whether the VPN is running? References: <2cfgpcxdg2.ln2@news.ducksburg.com> Message-ID: <8ghgpcxrd3.ln2@news.ducksburg.com> On 2016-02-18, Ervin Heged?s wrote: > Hi Adam, > > On Thu, Feb 18, 2016 at 09:26:58AM +0000, Adam Funk wrote: >> I'd like to test (inside a python 3 program) whether the VPN is >> running or not. The only thing I can think of so far is to use >> subprocess to run the 'ifconfig' command, then check its output for >> 'tun0'. Is there a better way? > > you didn't wrote, which system (OS) you want to use - based on > the "ifconfig" and "tun0" keywords, possible that's a Linux. Oops, sorry! But your educated guess is right. :-) > I think that the psutil modul could be better for you for this > task: > > https://pypi.python.org/pypi/psutil/ > > and see the "Network" section. if 'tun0' in psutil.net_if_addrs(): # vpn is running Brilliant! I've used psutil for something else, but I didn't know it did that too. My excuse is that the version on my system was 2.2.1, which does not do that, so I installed the newer version with pip3. Thanks for pointing me to that. -- XML is like violence: if it doesn't solve the problem, use more. From oscar.j.benjamin at gmail.com Thu Feb 18 05:09:55 2016 From: oscar.j.benjamin at gmail.com (Oscar Benjamin) Date: Thu, 18 Feb 2016 10:09:55 +0000 Subject: Considering migrating to Python from Visual Basic 6 for engineering applications In-Reply-To: <14c75a68-0d2e-45cc-8d73-0d71b6a6aea6@googlegroups.com> References: <90cc50d2-1ce5-4588-9bfd-a49d439f00dd@googlegroups.com> <14c75a68-0d2e-45cc-8d73-0d71b6a6aea6@googlegroups.com> Message-ID: On 18 February 2016 at 08:17, wrote: > Is it easy to create vector graphics files of plots in Python? Yes > In VB6, I wrote my own code to generate EPS files. Then people who demand jpg and gif files could be given those by converting the EPS with the desired resolution. That sounds a lot harder than doing it in Python using matplotlib. To run the following you need Python, numpy and matplotlib: ----- from matplotlib import pyplot as plt xdata = [0, 1, 2, 3, 4] ydata = [0, 1, 0, 1, 0] fig = plt.figure(figsize=(4, 3)) ax = fig.add_axes([0.15, 0.15, 0.70, 0.70]) ax.plot(xdata, ydata, color='black', linewidth=2) ax.set_xlabel('$V_x$') # Latex math-mode in text ax.set_ylabel('$V_y$') plt.show() # Make the plot window appear on screen ----- The last line can be replaced with a call to fig.savefig if you want the plot to go straight to a file. The format is inferred from the filename so: fig.savefig('plot.eps') # Do you really want eps? fig.savefig('plot.pdf') # I'd use pdf fig.savefig('plot.jpg') # etc. -- Oscar From rosuav at gmail.com Thu Feb 18 05:16:15 2016 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 18 Feb 2016 21:16:15 +1100 Subject: Considering migrating to Python from Visual Basic 6 for engineering applications In-Reply-To: <14c75a68-0d2e-45cc-8d73-0d71b6a6aea6@googlegroups.com> References: <90cc50d2-1ce5-4588-9bfd-a49d439f00dd@googlegroups.com> <14c75a68-0d2e-45cc-8d73-0d71b6a6aea6@googlegroups.com> Message-ID: On Thu, Feb 18, 2016 at 7:17 PM, wrote: > Thanks to all of you who have responded. One point two of you ask me is why I want to shift from Visual Basic. > > Most of my software is in VB6 (and VB3), and converting to VB.net is not very pleasant. I have learnt enough VB.net to manage most of the things I will need, but it always gives me a feeling that it is not meant for the kind of things I am doing. And you never know when Microsoft will kick us out of VB.net also. For the sake of keeping software compatible with new versions of Windows, you need to keep changing things in the software, which takes too much time. You might ask if it is really necessary to move from VB6 to VB.net because Microsoft will still not be able to kill VB6 very soon, thanks to VBA used in Excel. If my reasoning is wrong, please correct me. I am just an engineer, not a good programmer. > Very fair point. And I fully support your move, here. Just make sure you start on the latest Python (currently that's 3.5), and you should have no difficulty moving forward in versions; as someone who started writing Python 3 code back when 3.3 was the newest and latest thing, I can attest that it's been easy to migrate 3.2->3.3->3.4->3.5->3.6; there are very few things that get broken by the upgrade, and they're mostly things that would have been bad code in the older version anyway. >> The basic python language and its libraries are completely cross-platform (Linux, OS-X, Windows, iOS, and Android). The same source code will run on all of them. However, because it is an Interpreted language, the step to a bundled, self-contained image is platform dependent. There python compliers (as opposed to interpreters) for some, but not all platforms, and there is a very active JiT compiler project that would fall somewhere in between. >> > > OK. This is clear to me now. It is like needing different Fortran compilers for different operating systems, but Fortran code does not need much modifications (except file names/paths, etc.). > Something like that. There are a lot of little traps you can fall into (as you say, file names can be different, especially system ones) that stop your code from running correctly on all platforms, but porting a Python script from Linux to Windows or vice versa is a matter of testing it on the new platform and seeing what breaks, not completely rewriting it in a new language. >> > Could someone kindly tell me advantages and disadvantages of Python? >> >> If you are completely satisfied with VB, there is no reason to change (other than the enjoyable exercise of learning a new and more powerful language). >> > > Will VB6 run on Windows 11? Or can I compile VB6 on Windows 11. We can't know. But there will be easy ways to run Python on any Windows version or Linux or whatever. > There will indeed. That's the advantage of a fully open system - even if a nuclear meltdown kills all the Python core devs in one stroke (we really shouldn't have had PyCon Chernobyl), anyone in the world can pick up the source code and keep going with it. And Microsoft, despite having a strongly anti-freedom history, has come far more on board of recent years; as I understand it, Steve Dower has been spending company time improving Python (notably in the area of Windows installers and such). The future of open source software is pretty secure, and the future of openly-developed languages even more so, because CPython isn't the only way to run your Python code. One Python advantage that you may or may not be aware of is its extensibility. To an extent that I haven't seen since working with REXX on OS/2, there is a thriving community building Python packages, many of which (about 75K!) are listed on PyPI [1]. Got this proprietary database that you need to get info from? Check PyPI - chances are someone's written a connector for it, so you can perform queries on it as if it had native support in the language. Once you install something using "pip" [2], you can import it into your script the exact same way that you would import something from the standard library - making PyPI kinda like the "extended library" for the language. The barrier to entry is low, though, so you do sometimes have to wade through a bunch of stagnant or expired projects to find the gems you want. But there's so much good stuff there that it's well worth the effort. ChrisA [1] https://pypi.python.org/pypi [2] No, not the chipmunk from Enchanted. From bthk55 at gmail.com Thu Feb 18 05:32:59 2016 From: bthk55 at gmail.com (bthk55 at gmail.com) Date: Thu, 18 Feb 2016 02:32:59 -0800 (PST) Subject: Trying to build Python from Source on HPUX 11.23 IA fails Message-ID: <6b54afa8-eeb7-4350-b06d-6109f9aef4a2@googlegroups.com> I am trying to build python from source on HPUX 11.23 IA I am using the latest python version 3.5.1. I am getting the following error: Error 172: "Python/pytime.c", line 627 # Undeclared variable 'CLOCK_MONOTONIC'. const clockid_t clk_id = CLOCK_MONOTONIC; ^^^^^^^^^^^^^^^ *** Error exit code 2 I used the following steps to build : export CC="cc +DD64" export CXX=aCC ./configure --prefix=/opt/soe/local/Python-3.5.1 Does anyone know what is the issue here. From cfkaran2 at gmail.com Thu Feb 18 05:57:33 2016 From: cfkaran2 at gmail.com (Cem Karan) Date: Thu, 18 Feb 2016 05:57:33 -0500 Subject: Guido on python3 for beginners In-Reply-To: References: <48762040-a7e0-434c-92e8-8a0969210e5b@googlegroups.com> <5d31dd14-1a60-4f80-8889-d0616f404c70@googlegroups.com> <4e902c58-3370-4adf-905d-74630a6eea90@googlegroups.com> <60036746-78f7-4f05-a6d8-04e88b09c00f@googlegroups.com> <032f0f66-34df-4325-98e0-1127025d2a94@googlegroups.com> <31a618b2-a407-4723-9a4f-9b756fc93b0b@googlegroups.com> <0a920e92-5d31-4866-815b-cae7e201e4d8@googlegroups.com> <56c568f2$0$2832$c3e8da3$76491128@news.astraweb.com> Message-ID: On Feb 18, 2016, at 4:57 AM, Chris Angelico wrote: > On Thu, Feb 18, 2016 at 7:40 PM, Terry Reedy wrote: >> To my mind, the numerous duplications and overlaps in 2.7 that are gone in >> 3.x make 2.7 the worse version ever for beginners. > > Hmm. I was teaching on 2.7 up until last year, and for the most part, > we taught a "compatible with Py3" subset of the language, without any > significant cost. If you'd shown code saying "except ValueError, e:" > to one of my Py2 students then, s/he would have been just as > unfamiliar as one of my Py3 students would be today. That said, > though, it's still that Py3 is no worse than Py2, and potentially > better. > > The removal of L suffixes (and, similarly, the removal of u"..." > prefixes on text strings) is a bigger deal to newbies than it is to > experienced programmers, so that one definitely counts. "This is > great, but how can I remove that u from the strings?" was a common > question (eg when they're printing out a list of strings obtained from > a database, or decoded from JSON). > > The removal of old-style classes is a definite improvement in Py3, as > is the no-arg form of super(), which I'd consider a related change. So > there's a bunch of tiny little "quality of life" improvements here. > > ChrisA I agree with Chris on all his points. My personal feeling is that Py3 is the way to go for teaching in the future; its just that little bit more consistent across the board. And the things that are confusing are not things that beginners will need to know about. About the only thing I've read where Py2 has a slight advantage is for scripts where you're suddenly surprised by Py2 starting up when you've been using a Py3 interactive interpreter. For me, I'd probably give my students a block of code that they are asked to copy at the start of their files to test for Py2 or Py3, and to raise an exception on Py2. After that, I just wouldn't worry about it. Thanks, Cem Karan From rosuav at gmail.com Thu Feb 18 06:07:05 2016 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 18 Feb 2016 22:07:05 +1100 Subject: Guido on python3 for beginners In-Reply-To: References: <48762040-a7e0-434c-92e8-8a0969210e5b@googlegroups.com> <5d31dd14-1a60-4f80-8889-d0616f404c70@googlegroups.com> <4e902c58-3370-4adf-905d-74630a6eea90@googlegroups.com> <60036746-78f7-4f05-a6d8-04e88b09c00f@googlegroups.com> <032f0f66-34df-4325-98e0-1127025d2a94@googlegroups.com> <31a618b2-a407-4723-9a4f-9b756fc93b0b@googlegroups.com> <0a920e92-5d31-4866-815b-cae7e201e4d8@googlegroups.com> <56c568f2$0$2832$c3e8da3$76491128@news.astraweb.com> Message-ID: On Thu, Feb 18, 2016 at 9:57 PM, Cem Karan wrote: > I agree with Chris on all his points. My personal feeling is that Py3 is the way to go for teaching in the future; its just that little bit more consistent across the board. And the things that are confusing are not things that beginners will need to know about. > > About the only thing I've read where Py2 has a slight advantage is for scripts where you're suddenly surprised by Py2 starting up when you've been using a Py3 interactive interpreter. For me, I'd probably give my students a block of code that they are asked to copy at the start of their files to test for Py2 or Py3, and to raise an exception on Py2. After that, I just wouldn't worry about it. > Another solution is to have a little bit of boilerplate at the shell first: python3 -m venv env source env/bin/activate Then both "python" and "python3" will run the binary from your virtual environment, and as a side bonus, you get to use pip without root privileges. By the way... For bash users, adding this to .bashrc may make venvs a bit easier to keep straight: checkdir() { [ -n "$VIRTUAL_ENV" ] && ! [[ `pwd` =~ `dirname $VIRTUAL_ENV`* ]] && echo Deactivating venv $VIRTUAL_ENV... && deactivate [ -z "$VIRTUAL_ENV" -a -d env ] && echo Activating venv `pwd`/env... && source env/bin/activate } PROMPT_COMMAND=checkdir (I'm more fluent in Python than in bash, so this is a bit ugly.) As soon as you change out of the directory that contains your venv, it'll be deactivated, so you can't accidentally run stuff in the "wrong" environment. And as soon as you change into a directory that contains an 'env' subdir, it'll activate it. (Also applies on first creation; as soon as "python3 -m venv env" returns, this will activate the env.) ChrisA From wrong.address.1 at gmail.com Thu Feb 18 06:11:41 2016 From: wrong.address.1 at gmail.com (wrong.address.1 at gmail.com) Date: Thu, 18 Feb 2016 03:11:41 -0800 (PST) Subject: Considering migrating to Python from Visual Basic 6 for engineering applications In-Reply-To: References: <90cc50d2-1ce5-4588-9bfd-a49d439f00dd@googlegroups.com> <14c75a68-0d2e-45cc-8d73-0d71b6a6aea6@googlegroups.com> Message-ID: Almost everything points positively for Python. Thanks to all of you who have responded. But please also tell me the disadvantages of Python. If I start using Python, I should be aware of the price I am paying. Speed is not a big problem for me, so an interpreted language is fine. Is packaging/installing very messy? Do I create dozens of files for a simple program calculating the sum of two numbers and product of two numbers in text boxes with one command to be clicked? Can I learn this much in the first couple of hours? torstai 18. helmikuuta 2016 12.16.30 UTC+2 Chris Angelico kirjoitti: > On Thu, Feb 18, 2016 at 7:17 PM, wrote: > > Thanks to all of you who have responded. One point two of you ask me is why I want to shift from Visual Basic. > > > > Most of my software is in VB6 (and VB3), and converting to VB.net is not very pleasant. I have learnt enough VB.net to manage most of the things I will need, but it always gives me a feeling that it is not meant for the kind of things I am doing. And you never know when Microsoft will kick us out of VB.net also. For the sake of keeping software compatible with new versions of Windows, you need to keep changing things in the software, which takes too much time. You might ask if it is really necessary to move from VB6 to VB.net because Microsoft will still not be able to kill VB6 very soon, thanks to VBA used in Excel. If my reasoning is wrong, please correct me. I am just an engineer, not a good programmer. > > > > Very fair point. And I fully support your move, here. Just make sure > you start on the latest Python (currently that's 3.5), and you should > have no difficulty moving forward in versions; as someone who started > writing Python 3 code back when 3.3 was the newest and latest thing, I > can attest that it's been easy to migrate 3.2->3.3->3.4->3.5->3.6; > there are very few things that get broken by the upgrade, and they're > mostly things that would have been bad code in the older version > anyway. > > >> The basic python language and its libraries are completely cross-platform (Linux, OS-X, Windows, iOS, and Android). The same source code will run on all of them. However, because it is an Interpreted language, the step to a bundled, self-contained image is platform dependent. There python compliers (as opposed to interpreters) for some, but not all platforms, and there is a very active JiT compiler project that would fall somewhere in between. > >> > > > > OK. This is clear to me now. It is like needing different Fortran compilers for different operating systems, but Fortran code does not need much modifications (except file names/paths, etc.). > > > > Something like that. There are a lot of little traps you can fall into > (as you say, file names can be different, especially system ones) that > stop your code from running correctly on all platforms, but porting a > Python script from Linux to Windows or vice versa is a matter of > testing it on the new platform and seeing what breaks, not completely > rewriting it in a new language. > > >> > Could someone kindly tell me advantages and disadvantages of Python? > >> > >> If you are completely satisfied with VB, there is no reason to change (other than the enjoyable exercise of learning a new and more powerful language). > >> > > > > Will VB6 run on Windows 11? Or can I compile VB6 on Windows 11. We can't know. But there will be easy ways to run Python on any Windows version or Linux or whatever. > > > > There will indeed. That's the advantage of a fully open system - even > if a nuclear meltdown kills all the Python core devs in one stroke (we > really shouldn't have had PyCon Chernobyl), anyone in the world can > pick up the source code and keep going with it. And Microsoft, despite > having a strongly anti-freedom history, has come far more on board of > recent years; as I understand it, Steve Dower has been spending > company time improving Python (notably in the area of Windows > installers and such). The future of open source software is pretty > secure, and the future of openly-developed languages even more so, > because CPython isn't the only way to run your Python code. > > One Python advantage that you may or may not be aware of is its > extensibility. To an extent that I haven't seen since working with > REXX on OS/2, there is a thriving community building Python packages, > many of which (about 75K!) are listed on PyPI [1]. Got this > proprietary database that you need to get info from? Check PyPI - > chances are someone's written a connector for it, so you can perform > queries on it as if it had native support in the language. Once you > install something using "pip" [2], you can import it into your script > the exact same way that you would import something from the standard > library - making PyPI kinda like the "extended library" for the > language. The barrier to entry is low, though, so you do sometimes > have to wade through a bunch of stagnant or expired projects to find > the gems you want. But there's so much good stuff there that it's well > worth the effort. > > ChrisA > > [1] https://pypi.python.org/pypi > [2] No, not the chipmunk from Enchanted. From rosuav at gmail.com Thu Feb 18 06:32:33 2016 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 18 Feb 2016 22:32:33 +1100 Subject: Considering migrating to Python from Visual Basic 6 for engineering applications In-Reply-To: References: <90cc50d2-1ce5-4588-9bfd-a49d439f00dd@googlegroups.com> <14c75a68-0d2e-45cc-8d73-0d71b6a6aea6@googlegroups.com> Message-ID: On Thu, Feb 18, 2016 at 10:11 PM, wrote: > Almost everything points positively for Python. Thanks to all of you who have responded. But please also tell me the disadvantages of Python. If I start using Python, I should be aware of the price I am paying. Speed is not a big problem for me, so an interpreted language is fine. Is packaging/installing very messy? Do I create dozens of files for a simple program calculating the sum of two numbers and product of two numbers in text boxes with one command to be clicked? Can I learn this much in the first couple of hours? > There are a few warts, particularly on Windows, as regards packaging and third-party modules. Anything that's written in pure Python is fairly easy; stuff that's written in C is sometimes a bit hairy. But that's a limitation on the "extended library" of PyPI, not the stuff that comes with Python itself. Your simple example could easily be a single file. Python tends to eschew boilerplate; its motto comes from Baloo - look for the bare necessities of life. For a first couple of hours, I would recommend working at the console or in Idle, rather than creating a GUI; a window takes a bit of extra effort (plus you have choices in how to do that), so just learn the language first. But you could easily advance to building a GUI within your first day of learning Python, if that's the direction you want to go. The biggest disadvantage of Python is that, in a number of ways, it surprises people. Significant whitespace bugs a lot of experienced programmers (and some tools, too), and there are a few other ways that you may find a team of people revile against it. But if you can deal with the social aspects, it's a solid language. ChrisA From tdsperth at gmail.com Thu Feb 18 07:08:53 2016 From: tdsperth at gmail.com (tdsperth at gmail.com) Date: Thu, 18 Feb 2016 04:08:53 -0800 (PST) Subject: pymysql question Message-ID: <0b27ecb5-4a3e-46dc-a83c-5ce806f46051@googlegroups.com> Hi All I have many mysql tables in a conversion I am doing - the table field names are a mixture of lower case and upper case - do I have to covert all to one case to over come Python not finding the field. rows ocur.fetchall() for row in rows: print(row['newname']) #which fails because it is upper case in the table Cheers Colin From rustompmody at gmail.com Thu Feb 18 07:25:17 2016 From: rustompmody at gmail.com (Rustom Mody) Date: Thu, 18 Feb 2016 04:25:17 -0800 (PST) Subject: Guido on python3 for beginners In-Reply-To: <56c568f2$0$2832$c3e8da3$76491128@news.astraweb.com> References: <48762040-a7e0-434c-92e8-8a0969210e5b@googlegroups.com> <5d31dd14-1a60-4f80-8889-d0616f404c70@googlegroups.com> <4e902c58-3370-4adf-905d-74630a6eea90@googlegroups.com> <60036746-78f7-4f05-a6d8-04e88b09c00f@googlegroups.com> <032f0f66-34df-4325-98e0-1127025d2a94@googlegroups.com> <31a618b2-a407-4723-9a4f-9b756fc93b0b@googlegroups.com> <0a920e92-5d31-4866-815b-cae7e201e4d8@googlegroups.com> <56c568f2$0$2832$c3e8da3$76491128@news.astraweb.com> Message-ID: On Thursday, February 18, 2016 at 12:17:26 PM UTC+5:30, Steven D'Aprano wrote: > On Wednesday 17 February 2016 19:51, Rustom Mody wrote: > > > I hope someone can help me find this link: There is some record that Guido > > has said that python3 is probably a bit harder on noobs than python2. > > > > Does anyone know/have that link? > > > I can't say that I've seen it. I know that Raymond Hettinger is not too fond > of adding new syntactic features that add little in the way of power but > make the language harder to learn, but I don't recall anyone saying that it > is harder for newbies to get started with Python 3 than Python 2. > > There are more features in Python 3, so in that trivial sense of "more to > learn", I suppose that it is objectively correct that it is harder to learn > than Python 2. But I don't think the learning curve is any steeper. If > anything, the learning curve is ever-so-slightly less steep. > > Thanks Steven! So its Raymond Hettinger... Good enough So now I find your earlier post: https://mail.python.org/pipermail/python-list/2015-March/688387.html Obviously google was not obliging when I searched for Guido :-) BTW I have no tongs in this 2 vs 3 fire -- I'll leave that to Rick jmf and other noble gentry. My beef is somewhat different: viz that post 70s (Pascal) and 80s (scheme) programming pedagogy has deteriorated with general purpose languages replacing 'teaching-purpose language' for teaching. Which is about as intelligent as calling Martha[1] and Rose[2] both pianists [1] https://www.youtube.com/watch?v=YLZLp6AcAi4 [2] https://www.youtube.com/watch?v=_bjKDJD-CLc From python at mrabarnett.plus.com Thu Feb 18 07:52:32 2016 From: python at mrabarnett.plus.com (MRAB) Date: Thu, 18 Feb 2016 12:52:32 +0000 Subject: pymysql question In-Reply-To: <0b27ecb5-4a3e-46dc-a83c-5ce806f46051@googlegroups.com> References: <0b27ecb5-4a3e-46dc-a83c-5ce806f46051@googlegroups.com> Message-ID: <56C5BE90.4050500@mrabarnett.plus.com> On 2016-02-18 12:08, tdsperth at gmail.com wrote: > Hi All > > I have many mysql tables in a conversion I am doing - the table field names are a mixture of lower case and upper case - do I have to covert all to one case to over come Python not finding the field. > > rows ocur.fetchall() > for row in rows: > print(row['newname']) > #which fails because it is upper case in the table > According to this: MySQL query to get column names? http://stackoverflow.com/questions/4165195/mysql-query-to-get-column-names it's possible to get the column names. You could then build a dict to map a lowercase column name to an actual column name. From eryksun at gmail.com Thu Feb 18 07:54:16 2016 From: eryksun at gmail.com (eryk sun) Date: Thu, 18 Feb 2016 06:54:16 -0600 Subject: extending PATH on Windows? In-Reply-To: <1d5acb1l234hka82flgiqiq13cbp5ra03c@4ax.com> References: <1d5acb1l234hka82flgiqiq13cbp5ra03c@4ax.com> Message-ID: On Wed, Feb 17, 2016 at 6:53 PM, Dennis Lee Bieber wrote: > On Wed, 17 Feb 2016 17:49:11 +0000 (UTC), Ulli Horlacher > declaimed the following: > >>Thorsten Kampe wrote: >> >>> By the way: there is a script called `win_add2path.py` in your Python >>> distribution >> >>I have >>"Python 2.7.11 (v2.7.11:6d1b6a68f775, Dec 5 2015, 20:32:19) [MSC >>v.1500 32 bit (Intel)] on win32" >>and there is no "win_add2path.py" >> > C:\Python_x64\Python27\Tools\scripts\win_add2path.py > > PythonWin 2.7.5 (default, Sep 16 2013, 23:11:01) [MSC v.1500 64 bit > (AMD64)] on win32. > Portions Copyright 1994-2008 Mark Hammond - see 'Help/About PythonWin' for > further copyright information. > > But I can't confirm that this is not something added in the ActiveState > release. It's here: https://hg.python.org/cpython/file/v2.7.11/Tools/scripts/win_add2path.py But there are a few issues with this script. * A default value of u"%PATH%" is wrong. That causes the system PATH to be concatenated with itself. * For the user scripts directory, replacing appdata with "%APPDATA%" causes os.path.isdir to always fail. * For QueryValueEx, the only error it should handle is ERROR_FILE_NOT_FOUND. Other OS errors are unlikely, but they shouldn't be masked. * In Python 2, the value returned by QueryValueEx is a unicode string, which this script assumes is ASCII only. So it could die on a UnicodeDecodeError. * REG_EXPAND_SZ should only be used when '%' occurs 2 or more times. Otherwise use REG_SZ. This is important for two-pass environment variable expansion. Thus an existing REG_SZ type should be preserved, if that's reasonable, because the user may be reusing PATH in a REG_EXPAND_SZ variable. * It doesn't broadcast a WM_SETTINGCHANGE message, so it needlessly forces the user to log off and back on in order to use the modified PATH. Here's a new version for Python 2. I generalized the shell-variable replacement to a list of well-known folders. import os import sys import site import ctypes import _winreg import warnings user32 = ctypes.WinDLL('user32', use_last_error=True) HWND_BROADCAST = 0xFFFF WM_SETTINGCHANGE = 0x001A SMTO_ABORTIFHUNG = 0x0002 ERROR_FILE_NOT_FOUND = 0x0002 ERROR_TIMEOUT = 0x05B4 HKCU = _winreg.HKEY_CURRENT_USER ENVIRONMENT = u"Environment" PATH = u"PATH" SCRIPTS = u"Scripts" dir_vars = [u"%APPDATA%", u"%LOCALAPPDATA%", u"%USERPROFILE%", u"%HOMEDRIVE%%HOMEPATH%", u"%ProgramFiles%", u"%ProgramFiles(x86)%"] dir_vals = [_winreg.ExpandEnvironmentStrings(v) for v in dir_vars] def broadcast_change(lparam): if not user32.SendMessageTimeoutW( HWND_BROADCAST, WM_SETTINGCHANGE, 0, ctypes.c_wchar_p(lparam), SMTO_ABORTIFHUNG, 1000, None): err = ctypes.get_last_error() if err != ERROR_TIMEOUT: raise ctypes.WinError(err) def modify(): executable = sys.executable.decode("mbcs") pythonpath = os.path.dirname(os.path.normpath(executable)) scripts = os.path.join(pythonpath, SCRIPTS) if hasattr(site, "USER_SITE"): userpath = site.USER_SITE.decode("mbcs") userscripts = os.path.join(userpath, SCRIPTS) else: userscripts = None with _winreg.CreateKey(HKCU, ENVIRONMENT) as key: try: envpath, dtype = _winreg.QueryValueEx(key, PATH) except WindowsError as e: if e.winerror != ERROR_FILE_NOT_FOUND: raise envpath, dtype = u"", _winreg.REG_EXPAND_SZ if dtype not in (_winreg.REG_SZ, _winreg.REG_EXPAND_SZ): raise TypeError(r"Existing PATH value is not a string.") if dtype == _winreg.REG_SZ and envpath.count(u"%") > 1: warnings.warn('Changing type to REG_EXPAND_SZ.') dtype = _winreg.REG_EXPAND_SZ paths = [envpath] for path in (pythonpath, scripts, userscripts): if path and path not in envpath and os.path.isdir(path): if dtype == _winreg.REG_EXPAND_SZ: for var, val in zip(dir_vars, dir_vals): if val in path: path = path.replace(val, var) break if path in envpath: continue paths.append(path) envpath = os.pathsep.join(paths) _winreg.SetValueEx(key, PATH, 0, dtype, envpath) broadcast_change(ENVIRONMENT) return paths, envpath def main(): paths, envpath = modify() if len(paths) > 1: print "Path(s) added:" print '\n'.join(paths[1:]) else: print "No path was added" print "\nCurrent user PATH is now:\n%s\n" % envpath print "Expanded:" print _winreg.ExpandEnvironmentStrings(envpath) if __name__ == '__main__': main() From mal at europython.eu Thu Feb 18 08:40:57 2016 From: mal at europython.eu (M.-A. Lemburg) Date: Thu, 18 Feb 2016 14:40:57 +0100 Subject: EuroPython 2016: Call for Proposals Message-ID: <56C5C9E9.8040206@europython.eu> We?re looking for proposals on every aspect of Python: programming from novice to advanced levels, applications and frameworks, or how you have been involved in introducing Python into your organization. EuroPython is a community conference and we are eager to hear about your experience. Please also forward this Call for Proposals to anyone that you feel may be interested. *** https://ep2016.europython.eu/en/call-for-proposals/ *** Submissions will be open until Sunday, March 6, 23:59:59 CET For full details, please see the above CFP page. We have many exciting things waiting for you: * a PyData satellite event * new formats (panels, interactive sessions) * tracks to focus on more specific domains, including a revival of the EuroPython Business Track * speaker discounts for more than just talks and trainings Please note that we are using a shorter CFP period of just 2.5 weeks this year, since we want to have the list of talks published sooner than last year. With gravitational regards, -- EuroPython 2016 Team http://ep2016.europython.eu/ http://www.europython-society.org/ From grsmith at atlanticbb.net Thu Feb 18 09:00:10 2016 From: grsmith at atlanticbb.net (grsmith at atlanticbb.net) Date: Thu, 18 Feb 2016 09:00:10 -0500 Subject: Question on keyword arguments Message-ID: <62136715E19142B6B318EA3A3DF04D40@OPTIPLEX990> Would this be the correct way to return a list as a default result. Also, would the list be the preferable result (to a python programmer) ? def test(command, return_type='LIST'): """ Go to database and return data""" if return_type == 'LIST': result = ['ONE', 'TWO', 'THREE'] else: result = r'0xfeONE\0exfeTWO\0xfeTHREE' return result if __name__ == '__main__': print(test('cmd')) print(test('cmd', 'LIST')) print(test('cmd', None)) print(test('cmd', 'string')) thanks, george From wrw at mac.com Thu Feb 18 09:07:36 2016 From: wrw at mac.com (William Ray Wing) Date: Thu, 18 Feb 2016 09:07:36 -0500 Subject: Considering migrating to Python from Visual Basic 6 for engineering applications In-Reply-To: <90cc50d2-1ce5-4588-9bfd-a49d439f00dd@googlegroups.com> References: <90cc50d2-1ce5-4588-9bfd-a49d439f00dd@googlegroups.com> Message-ID: > On Feb 17, 2016, at 2:49 PM, wrong.address.1 at gmail.com wrote: > > I am mostly getting positive feedback for Python. > > It seems Python is used more for web based applications. Is it equally fine for creating stand-alone *.exe's? Can the same code be compiled to run on Linux or Android or web-based? > > Is it possible to create GUI elements with a good IDE? Can they be defined like in Visual Basic with given sizes, fonts, visible/invisible, etc.? > > Is it easy to do matrix operations in Python? Or do I need to write subroutines like in Visual Basic? > > Could someone kindly tell me advantages and disadvantages of Python? Or any better options? I have like 40-50 VB Forms and may be around 20000 lines of code. It will be a task to learn a new language and translate/re-write that code. > At this point you?ve probably heard more than you wanted to *about* Python. The next step really ought to be simply to invest a few minutes in looking at the language itself. Start here by downloading a copy of the Python interpreter: https://www.python.org/downloads/windows/ Then, there is an extensive list of on-line tutorial material here: https://wiki.python.org/moin/BeginnersGuide/Programmers I?d recommend simply picking one and diving in. If you have questions (and you surely will), come back here or, at least initially, send them to the Python Tutor list (tutor at python.org). Good luck and have fun, Bill > Thanks for your responses. > -- > https://mail.python.org/mailman/listinfo/python-list From __peter__ at web.de Thu Feb 18 09:37:16 2016 From: __peter__ at web.de (Peter Otten) Date: Thu, 18 Feb 2016 15:37:16 +0100 Subject: Question on keyword arguments References: <62136715E19142B6B318EA3A3DF04D40@OPTIPLEX990> Message-ID: grsmith at atlanticbb.net wrote: > Would this be the correct way to return > a list as a default result. If it does what you want it to do it is "correct" as in "complies with specification". > Also, would the list be the preferable result (to a python programmer) ? A list as a return value is OK as is any other object, however, > def test(command, return_type='LIST'): passing the desired result type as an argument is usually not a good idea. > """ Go to database and return data""" > if return_type == 'LIST': > result = ['ONE', 'TWO', 'THREE'] > else: > result = r'0xfeONE\0exfeTWO\0xfeTHREE' > return result > > if __name__ == '__main__': > print(test('cmd')) > print(test('cmd', 'LIST')) > print(test('cmd', None)) > print(test('cmd', 'string')) Make it separate functions instead: def test(command): return ["ONE", "TWO", "THREE"] def test_as_string(command, sep=","): return sep.join(test(command)) if __name__ == '__main__': print(test("cmd")) print(test_as_string("cmd")) From nikos.gr33k at gmail.com Thu Feb 18 09:45:00 2016 From: nikos.gr33k at gmail.com (=?ISO-8859-7?B?1v7t9OHyIMvh5O/w8eHq/PDv9evv8g==?=) Date: Thu, 18 Feb 2016 06:45:00 -0800 (PST) Subject: Weird and sparese cgi:error In-Reply-To: References: <813cd989-bc60-4d24-96ef-f20244d91362@googlegroups.com> Message-ID: <00242dad-3be0-41cb-b15f-ef31d32f23b8@googlegroups.com> #!/usr/bin/python3 # coding=utf-8 import cgitb; cgitb.enable() import cgi, re, os, sys, socket, time, datetime, locale, codecs, random, smtplib, subprocess, pymysql, geoip2.database from datetime import datetime, timedelta from http import cookies =================== [root at nexus cgi-bin]# /usr/bin/python3 Python 3.3.2 (default, Aug 14 2014, 14:25:52) [GCC 4.8.2 20140120 (Red Hat 4.8.2-16)] on linux Type "help", "copyright", "credits" or "license" for more information. >>> ============================= [root at nexus cgi-bin]# /usr/bin/python3 ./metrites.py Error in sys.excepthook: ValueError: underlying buffer has been detached Original exception was: Traceback (most recent call last): File "/opt/rh/python33/root/usr/lib64/python3.3/os.py", line 673, in __getitem__ value = self._data[self.encodekey(key)] KeyError: b'HTTP_USER_AGENT' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "./metrites.py", line 22, in userinfo = os.environ['HTTP_USER_AGENT'] File "/opt/rh/python33/root/usr/lib64/python3.3/os.py", line 676, in __getitem__ raise KeyError(key) KeyError: 'HTTP_USER_AGENT' [root at nexus cgi-bin]# ======================================= everything is they should but still iam gettign that weird cgi:err error From nikos.gr33k at gmail.com Thu Feb 18 09:47:01 2016 From: nikos.gr33k at gmail.com (=?ISO-8859-7?B?1v7t9OHyIMvh5O/w8eHq/PDv9evv8g==?=) Date: Thu, 18 Feb 2016 06:47:01 -0800 (PST) Subject: Weird and sparese cgi:error In-Reply-To: References: <813cd989-bc60-4d24-96ef-f20244d91362@googlegroups.com> Message-ID: <5a3b0097-0be2-4f30-974d-1ee02a1d81d2@googlegroups.com> [root at nexus cgi-bin]# head -10 ./metrites.py #!/usr/bin/python3 # coding=utf-8 import cgitb; cgitb.enable() import cgi, re, os, sys, socket, time, datetime, locale, codecs, random, smtplib, subprocess, pymysql, geoip2.database from datetime import datetime, timedelta from http import cookies #needed line, script does *not* work without it [root at nexus cgi-bin]# ^C [root at nexus cgi-bin]# /usr/bin/python3 Python 3.3.2 (default, Aug 14 2014, 14:25:52) [GCC 4.8.2 20140120 (Red Hat 4.8.2-16)] on linux Type "help", "copyright", "credits" or "license" for more information. >>> KeyboardInterrupt >>> KeyboardInterrupt >>> quit() [root at nexus cgi-bin]# [root at nexus cgi-bin]# /usr/bin/python3 ./metrites.py Error in sys.excepthook: ValueError: underlying buffer has been detached Original exception was: Traceback (most recent call last): File "/opt/rh/python33/root/usr/lib64/python3.3/os.py", line 673, in __getitem__ value = self._data[self.encodekey(key)] KeyError: b'HTTP_USER_AGENT' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "./metrites.py", line 22, in userinfo = os.environ['HTTP_USER_AGENT'] File "/opt/rh/python33/root/usr/lib64/python3.3/os.py", line 676, in __getitem__ raise KeyError(key) KeyError: 'HTTP_USER_AGENT' [root at nexus cgi-bin]# ^C From python.list at tim.thechases.com Thu Feb 18 09:58:03 2016 From: python.list at tim.thechases.com (Tim Chase) Date: Thu, 18 Feb 2016 08:58:03 -0600 Subject: Question on keyword arguments In-Reply-To: <62136715E19142B6B318EA3A3DF04D40@OPTIPLEX990> References: <62136715E19142B6B318EA3A3DF04D40@OPTIPLEX990> Message-ID: <20160218085803.115b1417@bigbox.christie.dr> On 2016-02-18 09:00, grsmith at atlanticbb.net wrote: > Would this be the correct way to return > a list as a default result. > > Also, would the list be the preferable result (to a python > programmer) ? > > def test(command, return_type='LIST'): > """ Go to database and return data""" > if return_type == 'LIST': > result = ['ONE', 'TWO', 'THREE'] > else: > result = r'0xfeONE\0exfeTWO\0xfeTHREE' [I presume the "\0exfe" should just be "\0xfe" like the other two] > return result > > if __name__ == '__main__': > print(test('cmd')) > print(test('cmd', 'LIST')) > print(test('cmd', None)) > print(test('cmd', 'string')) The function should return "the most obvious way to do it". In this case, it appears that you should just always return a list. If you want to format it in your crazy-other-result format, create a utility function to do that: def test(command): return ['ONE', 'TWO', 'THREE'] def grsmithify(lst): return ''.join("\0xfe%s" % s for s in lst) print(test('cmd')) print(grsmithify(test('cmd'))) -tkc From oscar.j.benjamin at gmail.com Thu Feb 18 10:20:54 2016 From: oscar.j.benjamin at gmail.com (Oscar Benjamin) Date: Thu, 18 Feb 2016 15:20:54 +0000 Subject: Considering migrating to Python from Visual Basic 6 for engineering applications In-Reply-To: References: <90cc50d2-1ce5-4588-9bfd-a49d439f00dd@googlegroups.com> <14c75a68-0d2e-45cc-8d73-0d71b6a6aea6@googlegroups.com> Message-ID: On 18 February 2016 at 11:32, Chris Angelico wrote: > On Thu, Feb 18, 2016 at 10:11 PM, wrote: >> Almost everything points positively for Python. Thanks to all of you who have responded. But please also tell me the disadvantages of Python. If I start using Python, I should be aware of the price I am paying. Speed is not a big problem for me, so an interpreted language is fine. Is packaging/installing very messy? Do I create dozens of files for a simple program calculating the sum of two numbers and product of two numbers in text boxes with one command to be clicked? Can I learn this much in the first couple of hours? >> > > There are a few warts, particularly on Windows, as regards packaging > and third-party modules. Anything that's written in pure Python is > fairly easy; stuff that's written in C is sometimes a bit hairy. But > that's a limitation on the "extended library" of PyPI, not the stuff > that comes with Python itself. For packaging/installing it really depends on what you're trying to do. You have to understand that Python is used in many very different ways in different environments and ecosystems so there just isn't a single way of doing it. It sounds to me as if all of your needs can be solved in pure Python code possibly using some of the popular extension modules from PyPI. In this case it's actually very easy to package/install. You can package your code simply by zipping it up with a __main__.py file. Someone who wants to install it will simply have a two step process: first install Python (and possibly a few dependencies) and then obtain the zip file with your code in it. -- Oscar From wrong.address.1 at gmail.com Thu Feb 18 10:33:19 2016 From: wrong.address.1 at gmail.com (wrong.address.1 at gmail.com) Date: Thu, 18 Feb 2016 07:33:19 -0800 (PST) Subject: Considering migrating to Python from Visual Basic 6 for engineering applications In-Reply-To: References: <90cc50d2-1ce5-4588-9bfd-a49d439f00dd@googlegroups.com> <14c75a68-0d2e-45cc-8d73-0d71b6a6aea6@googlegroups.com> Message-ID: torstai 18. helmikuuta 2016 17.21.32 UTC+2 Oscar Benjamin kirjoitti: > On 18 February 2016 at 11:32, Chris Angelico wrote: > > On Thu, Feb 18, 2016 at 10:11 PM, wrote: > >> Almost everything points positively for Python. Thanks to all of you who have responded. But please also tell me the disadvantages of Python. If I start using Python, I should be aware of the price I am paying. Speed is not a big problem for me, so an interpreted language is fine. Is packaging/installing very messy? Do I create dozens of files for a simple program calculating the sum of two numbers and product of two numbers in text boxes with one command to be clicked? Can I learn this much in the first couple of hours? > >> > > > > There are a few warts, particularly on Windows, as regards packaging > > and third-party modules. Anything that's written in pure Python is > > fairly easy; stuff that's written in C is sometimes a bit hairy. But > > that's a limitation on the "extended library" of PyPI, not the stuff > > that comes with Python itself. > > For packaging/installing it really depends on what you're trying to > do. You have to understand that Python is used in many very different > ways in different environments and ecosystems so there just isn't a > single way of doing it. > > It sounds to me as if all of your needs can be solved in pure Python > code possibly using some of the popular extension modules from PyPI. > In this case it's actually very easy to package/install. You can > package your code simply by zipping it up with a __main__.py file. > Someone who wants to install it will simply have a two step process: > first install Python (and possibly a few dependencies) and then obtain > the zip file with your code in it. > > -- > Oscar This form of packing is not desirable. I can't ask other people to install Python on their machines, and I also would not want show most of the code doing the calculations. Another question I have is regarding reading numerical data from text files. Is it necessary to read one character at a time, or can one read like in Fortran and Basic (something like Input #5, X1, X2, X3)? From wrong.address.1 at gmail.com Thu Feb 18 10:35:53 2016 From: wrong.address.1 at gmail.com (wrong.address.1 at gmail.com) Date: Thu, 18 Feb 2016 07:35:53 -0800 (PST) Subject: Considering migrating to Python from Visual Basic 6 for engineering applications In-Reply-To: References: <90cc50d2-1ce5-4588-9bfd-a49d439f00dd@googlegroups.com> Message-ID: torstai 18. helmikuuta 2016 16.08.05 UTC+2 William Ray Wing kirjoitti: > > On Feb 17, 2016, at 2:49 PM, wrong.address.1 at gmail.com wrote: > > > > I am mostly getting positive feedback for Python. > > > > It seems Python is used more for web based applications. Is it equally fine for creating stand-alone *.exe's? Can the same code be compiled to run on Linux or Android or web-based? > > > > Is it possible to create GUI elements with a good IDE? Can they be defined like in Visual Basic with given sizes, fonts, visible/invisible, etc.? > > > > Is it easy to do matrix operations in Python? Or do I need to write subroutines like in Visual Basic? > > > > Could someone kindly tell me advantages and disadvantages of Python? Or any better options? I have like 40-50 VB Forms and may be around 20000 lines of code. It will be a task to learn a new language and translate/re-write that code. > > > > At this point you've probably heard more than you wanted to *about* Python. The next step really ought to be simply to invest a few minutes in looking at the language itself. > Start here by downloading a copy of the Python interpreter: > > https://www.python.org/downloads/windows/ > > Then, there is an extensive list of on-line tutorial material here: > > https://wiki.python.org/moin/BeginnersGuide/Programmers > > I'd recommend simply picking one and diving in. If you have questions (and you surely will), come back here or, at least initially, send them to the Python Tutor list (tutor at python.org). > > Good luck and have fun, > Bill > > > > Thanks for your responses. > > -- > > https://mail.python.org/mailman/listinfo/python-list I am almost eager to do this but want to be sure that I know the pitfalls in using Python for my purposes. Thanks for your encouraging response. From D.Strohl at F5.com Thu Feb 18 10:39:38 2016 From: D.Strohl at F5.com (Dan Strohl) Date: Thu, 18 Feb 2016 15:39:38 +0000 Subject: Question on keyword arguments In-Reply-To: References: <62136715E19142B6B318EA3A3DF04D40@OPTIPLEX990> Message-ID: <7278ec3f250548b192ce2d95c6fa07c5@seaexchmbx03.olympus.F5Net.com> I have approached this in a few different ways, depending on the use case. I have done the "return_type=list" before, though I don't really like it... it's too easy to forget or mistype the exact name used for the switch. If you do this, I also recommend setting some class or module variables to the text strings and pass those back and forth. I have also returned a different type depending on what is being requested... so, if the function user requests data that has multiple objects, return a list, if they request something that has just one object, return that object. This also has potential problems if the number of returned objects is not consistently predictable, you would have to always test your returned object to see how to handle it. I like the other suggestions thrown out as well, (having multiple methods and having one method and a converting function).. They are good, easy to read and use methods as long as you know at design time what type of object you want (which is probably the norm). Another approach that I like (where it makes sense) is to return another (custom) object that can act like either... that way, you have the content in hand and can use it either way. This approach works well when there is a high overhead in creating the data, that you don't want to have to do again, and you don't mind having a few more objects hanging around until cleaned up. So, define a return object like: from collections import UserList class TestResponse(UserList): def __str__(self): return '\0xfe%s' % '\0xfe'.join(self.data) ...and return that object. that way, you can iterate over it, treat it as a list, or whatever, or if you do print(str(TestResponse)), you get your r'0xfeONE\0exfeTWO\0xfeTHREE' In the end, your decision should be made based on: 1. What would the developer using this EXPECT to get back? (if the user normally expects one or the other, do that, if this is for a "public module", I would expect the list is more likely the common response. This list is probably not the right place for that question though, unless this is a very general utility, you want to ask the people coding in the domain that would mostly be using your function.) 2. What is the most likely response type? (if it is common to get it back different ways, or use it multiple times, this is probably mostly the same as #1, but can be different) 3. What will be the easiest to read and document? Returning a list is easy to read and expectable, but not if you have to then jump through some loops to convert it to something else immediately. 4. Is there a large enough benefit in processing that you want to be able to use it multiple ways without having to regenerate the list. (note that this is the last criteria, unless performance is really critical, readability and predictability is normally much more important than what is probably minor performance tuning.) Dan Strohl > -----Original Message----- > From: Python-list [mailto:python-list-bounces+d.strohl=f5.com at python.org] On > Behalf Of Peter Otten > Sent: Thursday, February 18, 2016 6:37 AM > To: python-list at python.org > Subject: Re: Question on keyword arguments > > grsmith at atlanticbb.net wrote: > > > Would this be the correct way to return a list as a default result. > > If it does what you want it to do it is "correct" as in "complies with > specification". > > > Also, would the list be the preferable result (to a python programmer) ? > > A list as a return value is OK as is any other object, however, > > > def test(command, return_type='LIST'): > > passing the desired result type as an argument is usually not a good idea. > > > """ Go to database and return data""" > > if return_type == 'LIST': > > result = ['ONE', 'TWO', 'THREE'] > > else: > > result = r'0xfeONE\0exfeTWO\0xfeTHREE' > > return result > > > > if __name__ == '__main__': > > print(test('cmd')) > > print(test('cmd', 'LIST')) > > print(test('cmd', None)) > > print(test('cmd', 'string')) > > Make it separate functions instead: > > def test(command): > return ["ONE", "TWO", "THREE"] > > def test_as_string(command, sep=","): > return sep.join(test(command)) > > if __name__ == '__main__': > print(test("cmd")) > print(test_as_string("cmd")) > > > -- > https://mail.python.org/mailman/listinfo/python-list From ganesh1pal at gmail.com Thu Feb 18 10:42:26 2016 From: ganesh1pal at gmail.com (Ganesh Pal) Date: Thu, 18 Feb 2016 21:12:26 +0530 Subject: TypeError: not all arguments converted during string formatting In-Reply-To: References: Message-ID: On Wed, Feb 17, 2016 at 7:32 PM, Chris Angelico wrote: > The print statement/function happily accepts multiple arguments, and > will join them according to a set of predefined rules. The logging > functions don't have those rules, so they take one message and some > optional parameters. Try this, instead: > > logging.info("%r %r %r", out, err, ret) > Thanks this solved my issue :) From rosuav at gmail.com Thu Feb 18 10:45:36 2016 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 19 Feb 2016 02:45:36 +1100 Subject: Question on keyword arguments In-Reply-To: <7278ec3f250548b192ce2d95c6fa07c5@seaexchmbx03.olympus.F5Net.com> References: <62136715E19142B6B318EA3A3DF04D40@OPTIPLEX990> <7278ec3f250548b192ce2d95c6fa07c5@seaexchmbx03.olympus.F5Net.com> Message-ID: On Fri, Feb 19, 2016 at 2:39 AM, Dan Strohl wrote: > So, define a return object like: > > from collections import UserList > class TestResponse(UserList): > def __str__(self): > return '\0xfe%s' % '\0xfe'.join(self.data) > > ...and return that object. Out of interest, why UserList rather than simply subclassing list? ChrisA From oscar.j.benjamin at gmail.com Thu Feb 18 10:47:46 2016 From: oscar.j.benjamin at gmail.com (Oscar Benjamin) Date: Thu, 18 Feb 2016 15:47:46 +0000 Subject: Considering migrating to Python from Visual Basic 6 for engineering applications In-Reply-To: References: <90cc50d2-1ce5-4588-9bfd-a49d439f00dd@googlegroups.com> <14c75a68-0d2e-45cc-8d73-0d71b6a6aea6@googlegroups.com> Message-ID: On 18 February 2016 at 15:33, wrote: >> It sounds to me as if all of your needs can be solved in pure Python >> code possibly using some of the popular extension modules from PyPI. >> In this case it's actually very easy to package/install. You can >> package your code simply by zipping it up with a __main__.py file. >> Someone who wants to install it will simply have a two step process: >> first install Python (and possibly a few dependencies) and then obtain >> the zip file with your code in it. > > This form of packing is not desirable. I can't ask other people to install Python on their machines, and I also would not want show most of the code doing the calculations. This is what sometimes makes packaging Python apps difficult. Distributing code as a zip file is easy and works out of the box. However a lot of people are very insistent that asking the end user to first install Python is unacceptable. That's why I said it depends what you're trying to do. I guess from your description that you want other people to be able to install and use your proprietary applications on whatever OS they're using. You would probably want to use something like pyinstaller then as this bundles a Python interpreter with your code. For Windows there's also the embedded Python distribution which you can ship as part of an installer for your program. For OSX/Linux Python is most likely already installed so this may be unnecessary. In terms of hiding the code doing the calculations: it doesn't work like that in Python. The code is not compiled so if the end user has your app then they have your code. You could probably use something like Cython to obfuscate it but then this means that your compiled code is architecture/platform dependent so you need to compile it separately for each platform and there are other problems. > Another question I have is regarding reading numerical data from text files. Is it necessary to read one character at a time, or can one read like in Fortran and Basic (something like Input #5, X1, X2, X3)? There are loads of ways to read your data in from text files. I don't know how it works in Fortran and Basic but I'm sure there will be something that does what you want. -- Oscar From wrong.address.1 at gmail.com Thu Feb 18 10:49:02 2016 From: wrong.address.1 at gmail.com (wrong.address.1 at gmail.com) Date: Thu, 18 Feb 2016 07:49:02 -0800 (PST) Subject: Considering migrating to Python from Visual Basic 6 for engineering applications In-Reply-To: References: <90cc50d2-1ce5-4588-9bfd-a49d439f00dd@googlegroups.com> <14c75a68-0d2e-45cc-8d73-0d71b6a6aea6@googlegroups.com> Message-ID: <0a56557a-3c41-4900-b03c-288f1af82c53@googlegroups.com> On Thursday, 18 February 2016 13:32:58 UTC+2, Chris Angelico wrote: > On Thu, Feb 18, 2016 at 10:11 PM, wrote: > > Almost everything points positively for Python. Thanks to all of you who have responded. But please also tell me the disadvantages of Python. If I start using Python, I should be aware of the price I am paying. Speed is not a big problem for me, so an interpreted language is fine. Is packaging/installing very messy? Do I create dozens of files for a simple program calculating the sum of two numbers and product of two numbers in text boxes with one command to be clicked? Can I learn this much in the first couple of hours? > > > > There are a few warts, particularly on Windows, as regards packaging > and third-party modules. Anything that's written in pure Python is > fairly easy; stuff that's written in C is sometimes a bit hairy. But > that's a limitation on the "extended library" of PyPI, not the stuff > that comes with Python itself. > > Your simple example could easily be a single file. Python tends to > eschew boilerplate; its motto comes from Baloo - look for the bare > necessities of life. > > For a first couple of hours, I would recommend working at the console > or in Idle, rather than creating a GUI; a window takes a bit of extra > effort (plus you have choices in how to do that), so just learn the > language first. But you could easily advance to building a GUI within > your first day of learning Python, if that's the direction you want to > go. > > The biggest disadvantage of Python is that, in a number of ways, it > surprises people. Significant whitespace bugs a lot of experienced > programmers (and some tools, too), and there are a few other ways that > you may find a team of people revile against it. But if you can deal > with the social aspects, it's a solid language. > > ChrisA Thanks. You have guided me quite well, and I am almost ready to declare that I will use Python for the next few decades. Imagine I want to read in two (or a few) numbers from a text file, display them in two (or more) text boxes, calculate their sums and products, may be take logarithms of them, and display them in other two text boxes or labels, and make some bar charts, scatter plots, and later draw curves and surfaces on a computer screen. Do I really need PyPi or other external stuff for this? Is Python well equipped for this? I will have to learn GUI creating quickly after I know the basics of reading and writing text files, and doing simple mathematical calculations. Later I can imagine using things like sending an SMS from a phone running this on Android, or placing a graph in a WhatsApp message, make a call to someone and tell him the temperature is now too high, etc. These things might need external libraries, but I can learn this later on. From random832 at fastmail.com Thu Feb 18 10:51:54 2016 From: random832 at fastmail.com (Random832) Date: Thu, 18 Feb 2016 10:51:54 -0500 Subject: Guido on python3 for beginners In-Reply-To: References: <48762040-a7e0-434c-92e8-8a0969210e5b@googlegroups.com> <5d31dd14-1a60-4f80-8889-d0616f404c70@googlegroups.com> <4e902c58-3370-4adf-905d-74630a6eea90@googlegroups.com> <60036746-78f7-4f05-a6d8-04e88b09c00f@googlegroups.com> <032f0f66-34df-4325-98e0-1127025d2a94@googlegroups.com> <31a618b2-a407-4723-9a4f-9b756fc93b0b@googlegroups.com> <0a920e92-5d31-4866-815b-cae7e201e4d8@googlegroups.com> <56c568f2$0$2832$c3e8da3$76491128@news.astraweb.com> Message-ID: <1455810714.2345775.525031114.34DE5B66@webmail.messagingengine.com> On Thu, Feb 18, 2016, at 07:25, Rustom Mody wrote: > My beef is somewhat different: viz that post 70s (Pascal) and 80s > (scheme) > programming pedagogy has deteriorated with general purpose languages > replacing > 'teaching-purpose language' for teaching. The flaw in this idea is right there in your post. Both languages you named are strongly tied to a single paradigm (procedural for Pascal, and functional for Scheme) which don't match the paradigm that real-world work is done in. Is there a new "teaching-purpose language"? From grsmith at atlanticbb.net Thu Feb 18 10:57:31 2016 From: grsmith at atlanticbb.net (grsmith at atlanticbb.net) Date: Thu, 18 Feb 2016 10:57:31 -0500 Subject: Question on keyword arguments In-Reply-To: References: <62136715E19142B6B318EA3A3DF04D40@OPTIPLEX990> <7278ec3f250548b192ce2d95c6fa07c5@seaexchmbx03.olympus.F5Net.com> Message-ID: <52FA2B3109D743E2BB8534CAC2B5E8A5@OPTIPLEX990> Thanks to all who responded, it is a big help. Tim, the 'crazy-other-result format' is the result returned by the database, nothing I can do about that :) Thanks again George -----Original Message----- From: Chris Angelico Sent: Thursday, February 18, 2016 10:45 AM Cc: python-list at python.org Subject: Re: Question on keyword arguments On Fri, Feb 19, 2016 at 2:39 AM, Dan Strohl wrote: > So, define a return object like: > > from collections import UserList > class TestResponse(UserList): > def __str__(self): > return '\0xfe%s' % '\0xfe'.join(self.data) > > ...and return that object. Out of interest, why UserList rather than simply subclassing list? ChrisA -- https://mail.python.org/mailman/listinfo/python-list From wrw at mac.com Thu Feb 18 10:59:29 2016 From: wrw at mac.com (William Ray Wing) Date: Thu, 18 Feb 2016 10:59:29 -0500 Subject: Considering migrating to Python from Visual Basic 6 for engineering applications In-Reply-To: References: <90cc50d2-1ce5-4588-9bfd-a49d439f00dd@googlegroups.com> <14c75a68-0d2e-45cc-8d73-0d71b6a6aea6@googlegroups.com> Message-ID: <38CBB103-F3DF-4E76-ABE8-43A91182AF04@mac.com> > On Feb 18, 2016, at 10:33 AM, wrong.address.1 at gmail.com wrote: > > torstai 18. helmikuuta 2016 17.21.32 UTC+2 Oscar Benjamin kirjoitti: >> On 18 February 2016 at 11:32, Chris Angelico wrote: >> [byte] >> It sounds to me as if all of your needs can be solved in pure Python >> code possibly using some of the popular extension modules from PyPI. >> In this case it's actually very easy to package/install. You can >> package your code simply by zipping it up with a __main__.py file. >> Someone who wants to install it will simply have a two step process: >> first install Python (and possibly a few dependencies) and then obtain >> the zip file with your code in it. >> >> -- >> Oscar > > This form of packing is not desirable. I can't ask other people to install Python on their machines, and I also would not want show most of the code doing the calculations. > Now things get tricky. I can understand you not wanting to force people to pre-install Python in order for your code to run, but just how deeply do you want to hide or obfuscate it? Is this potentially a commercial application to be sold, or are you simply trying to keep things clean and tidy within various divisions of your company. I?d hope not the former, because even VB can get you into tricky licensing issues (and in any case - even fully compiled code in any language these days can be de-compiled into logically correct source code, although with less than obvious variable names). At the other extreme, there are packaging programs (py2exe comes to mind, although I have no experience with it). These wrap the whole python interpreter, your code, and any needed libraries into an executable (clickable) package. Their only downside is that the output packages they produce tend to be large. However, any sophisticated user who digs into them WILL be able to find your source code, possibly only slightly obfuscated by being zipped. > Another question I have is regarding reading numerical data from text files. Is it necessary to read one character at a time, or can one read like in Fortran and Basic (something like Input #5, X1, X2, X3)? Python can read lines of text or whole blocks of text from source files. If those files are in more or less well-understood form (csv for example) it has libraries specifically designed for reading and extracting data from them (including VERY nice facilities for doing arithmetic and otherwise manipulating time and date data). Bill > -- > https://mail.python.org/mailman/listinfo/python-list From D.Strohl at F5.com Thu Feb 18 11:00:57 2016 From: D.Strohl at F5.com (Dan Strohl) Date: Thu, 18 Feb 2016 16:00:57 +0000 Subject: Considering migrating to Python from Visual Basic 6 for engineering applications In-Reply-To: References: <90cc50d2-1ce5-4588-9bfd-a49d439f00dd@googlegroups.com> <14c75a68-0d2e-45cc-8d73-0d71b6a6aea6@googlegroups.com> Message-ID: Disadvantages of python... (compared to VB) That's a hard one, but here are my thoughts: (keep in mind that these kinds of discussions are subjective and much is based on the background and experience of the coder, these are also assuming that 100% of your audience is on windows, as soon as you start talking about cross platform support, VB gets much harder to use (yes, you could use it on Linus, via a web based app, or an emulator if you HAD to). - I find that VB is much easier at making quick GUI apps (for windows). - VB is easier to integrate with office apps and other windows specific things (there are some python modules that will access office files etc, but if you want to have an engineering app that directly integrates / interacts with excel, VB is probably better. - I find VB is easier to package and distribute. (there are some good utilities that will take Python and package it into an exe, along with all of the needed pieces, interpreter, etc, but those all require some level of work to setup and make work... not a lot sometimes, but certainly more than "click compile" and copy the .exe file. - VB is often, for simple apps, often simpler to learn for non-programmers. (I know I will get slammed for that one). For complex apps, I find VB harder to do things than Python, but for example, if I wanted to make a quick windows calculator, I would probably go to VB first.) My approach is generally: I use Python for: - server apps, web based apps, plugins, modules, library development, or apps that I want to be able to expand later with plugins, console apps and utilities (things that I am only going to run from the CLI anyway), performance focused apps (unless I need to go all the way to C for performance), anything that might ever need to be cross platform, apps that interact with other (non-Microsoft) apps. I use VB for: Quick user focused, non-web apps, apps that are used in or directly with Microsoft Office apps, apps that I am developing for someone else that is a VB programmer (or non-programmer but might poke at them).. apps that I need to distribute to lots of less controlled workstations that don't have python on them already. These days, I find that I am using VB much less than Python, most of the reasons that I would pick VB can be overcome by developing a cloud app instead of a local app, but there are still times that VB is the right tool (for me at least). > -----Original Message----- > From: Python-list [mailto:python-list-bounces+d.strohl=f5.com at python.org] On > Behalf Of wrong.address.1 at gmail.com > Sent: Thursday, February 18, 2016 7:33 AM > To: python-list at python.org > Subject: Re: Considering migrating to Python from Visual Basic 6 for engineering > applications > > torstai 18. helmikuuta 2016 17.21.32 UTC+2 Oscar Benjamin kirjoitti: > > On 18 February 2016 at 11:32, Chris Angelico wrote: > > > On Thu, Feb 18, 2016 at 10:11 PM, wrote: > > >> Almost everything points positively for Python. Thanks to all of you who > have responded. But please also tell me the disadvantages of Python. If I start > using Python, I should be aware of the price I am paying. Speed is not a big > problem for me, so an interpreted language is fine. Is packaging/installing very > messy? Do I create dozens of files for a simple program calculating the sum of > two numbers and product of two numbers in text boxes with one command to > be clicked? Can I learn this much in the first couple of hours? > > >> > > > > > > There are a few warts, particularly on Windows, as regards packaging > > > and third-party modules. Anything that's written in pure Python is > > > fairly easy; stuff that's written in C is sometimes a bit hairy. But > > > that's a limitation on the "extended library" of PyPI, not the stuff > > > that comes with Python itself. > > > > For packaging/installing it really depends on what you're trying to > > do. You have to understand that Python is used in many very different > > ways in different environments and ecosystems so there just isn't a > > single way of doing it. > > > > It sounds to me as if all of your needs can be solved in pure Python > > code possibly using some of the popular extension modules from PyPI. > > In this case it's actually very easy to package/install. You can > > package your code simply by zipping it up with a __main__.py file. > > Someone who wants to install it will simply have a two step process: > > first install Python (and possibly a few dependencies) and then obtain > > the zip file with your code in it. > > > > -- > > Oscar > > This form of packing is not desirable. I can't ask other people to install Python on > their machines, and I also would not want show most of the code doing the > calculations. > > Another question I have is regarding reading numerical data from text files. Is it > necessary to read one character at a time, or can one read like in Fortran and > Basic (something like Input #5, X1, X2, X3)? > -- > https://mail.python.org/mailman/listinfo/python-list From rosuav at gmail.com Thu Feb 18 11:06:12 2016 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 19 Feb 2016 03:06:12 +1100 Subject: Considering migrating to Python from Visual Basic 6 for engineering applications In-Reply-To: <0a56557a-3c41-4900-b03c-288f1af82c53@googlegroups.com> References: <90cc50d2-1ce5-4588-9bfd-a49d439f00dd@googlegroups.com> <14c75a68-0d2e-45cc-8d73-0d71b6a6aea6@googlegroups.com> <0a56557a-3c41-4900-b03c-288f1af82c53@googlegroups.com> Message-ID: On Fri, Feb 19, 2016 at 2:49 AM, wrote: > Thanks. You have guided me quite well, and I am almost ready to declare that I will use Python for the next few decades. > Don't declare like that - just start using it, and see what you think :) But I would be very much surprised if Python doesn't get added to your toolbelt. A good programmer should always keep a selection of tools handy - seldom is one tool the right one for every job. > Imagine I want to read in two (or a few) numbers from a text file, display them in two (or more) text boxes, calculate their sums and products, may be take logarithms of them, and display them in other two text boxes or labels, and make some bar charts, scatter plots, and later draw curves and surfaces on a computer screen. Do I really need PyPi or other external stuff for this? Is Python well equipped for this? > You could do all of that with just the standard library, but then your only choice of GUI library is Tkinter, which is derived from Tcl/Tk. If you don't like how that looks (either in your code, or in the resulting GUI), you'll need to turn to PyPI for an alternative (eg something derived from GTK or wxWindows or Qt). I'm not sure how the graphing capabilities of Tkinter are, so you might want to grab matplotlib too - again, it'll give you a lot more flexibility than you would have if you restrict yourself to *just* the standard library. But I think you could do all of that - certainly most of it - with just the language and standard library. > I will have to learn GUI creating quickly after I know the basics of reading and writing text files, and doing simple mathematical calculations. > Reading and writing text files is easy, as is basic mathematics. You'll have that down in an hour or two, most likely, and then you'll have time to play GUIs. > Later I can imagine using things like sending an SMS from a phone running this on Android, or placing a graph in a WhatsApp message, make a call to someone and tell him the temperature is now too high, etc. These things might need external libraries, but I can learn this later on. > Those things will most likely require external libraries. But most of them are probably possible. ChrisA From framstag at rus.uni-stuttgart.de Thu Feb 18 11:24:00 2016 From: framstag at rus.uni-stuttgart.de (Ulli Horlacher) Date: Thu, 18 Feb 2016 16:24:00 +0000 (UTC) Subject: extending PATH on Windows? References: Message-ID: Dennis Lee Bieber wrote: > >I have > >"Python 2.7.11 (v2.7.11:6d1b6a68f775, Dec 5 2015, 20:32:19) [MSC > >v.1500 32 bit (Intel)] on win32" > >and there is no "win_add2path.py" > > > C:\Python_x64\Python27\Tools\scripts\win_add2path.py Ok, It is here in C:\Python27\Tools\scripts\win_add2path.py but windows "Search programs and files" was not able to find it. -- Ullrich Horlacher Server und Virtualisierung Rechenzentrum IZUS/TIK E-Mail: horlacher at tik.uni-stuttgart.de Universitaet Stuttgart Tel: ++49-711-68565868 Allmandring 30a Fax: ++49-711-682357 70550 Stuttgart (Germany) WWW: http://www.tik.uni-stuttgart.de/ From D.Strohl at F5.com Thu Feb 18 11:24:33 2016 From: D.Strohl at F5.com (Dan Strohl) Date: Thu, 18 Feb 2016 16:24:33 +0000 Subject: Considering migrating to Python from Visual Basic 6 for engineering applications In-Reply-To: References: <90cc50d2-1ce5-4588-9bfd-a49d439f00dd@googlegroups.com> <14c75a68-0d2e-45cc-8d73-0d71b6a6aea6@googlegroups.com> <0a56557a-3c41-4900-b03c-288f1af82c53@googlegroups.com> Message-ID: <550bd0626d9a48f1bcc7feaeba646b6a@seaexchmbx03.olympus.F5Net.com> I totally agree with Chris here, try it out, do some simpler things to get your feet wet before committing to redoing hundreds of thousands of lines of code in it. Find a few of the "deal breakers" and try to solve them in Python (create a hello world app in python, then package it in an exe and distribute it to some friends).. VB is a fine language, and is likely to suit your needs as well... even if we all believe that Python is better, the effort to convert from one to the other may not make sense. I would also suggest taking a few steps back and seeing if there is a different overall architectural approach that makes sense. If you are using a heavy weight local app, what about a web app? What about doing a combination approach? Have a back end running on a server with Python, that feeds a VB app on a client for displaying and manipulating data? (then a Java app for mobile clients or Linux ones?) Also, looking at existing frameworks and expandable apps out there that you might be able to do less work but still meet your goals with because 80% of the work was done for you... things like django, splunk, tableau, etc... Dan > -----Original Message----- > From: Python-list [mailto:python-list-bounces+d.strohl=f5.com at python.org] On > Behalf Of Chris Angelico > Sent: Thursday, February 18, 2016 8:06 AM > Cc: python-list at python.org > Subject: Re: Considering migrating to Python from Visual Basic 6 for engineering > applications > > On Fri, Feb 19, 2016 at 2:49 AM, wrote: > > Thanks. You have guided me quite well, and I am almost ready to declare that I > will use Python for the next few decades. > > > > Don't declare like that - just start using it, and see what you think > :) But I would be very much surprised if Python doesn't get added to your > toolbelt. A good programmer should always keep a selection of tools handy - > seldom is one tool the right one for every job. > > > Imagine I want to read in two (or a few) numbers from a text file, display them > in two (or more) text boxes, calculate their sums and products, may be take > logarithms of them, and display them in other two text boxes or labels, and > make some bar charts, scatter plots, and later draw curves and surfaces on a > computer screen. Do I really need PyPi or other external stuff for this? Is Python > well equipped for this? > > > > You could do all of that with just the standard library, but then your only choice > of GUI library is Tkinter, which is derived from Tcl/Tk. > If you don't like how that looks (either in your code, or in the resulting GUI), > you'll need to turn to PyPI for an alternative (eg something derived from GTK or > wxWindows or Qt). I'm not sure how the graphing capabilities of Tkinter are, so > you might want to grab matplotlib too - again, it'll give you a lot more flexibility > than you would have if you restrict yourself to *just* the standard library. > But I think you could do all of that - certainly most of it - with just the language > and standard library. > > > I will have to learn GUI creating quickly after I know the basics of reading and > writing text files, and doing simple mathematical calculations. > > > > Reading and writing text files is easy, as is basic mathematics. > You'll have that down in an hour or two, most likely, and then you'll have time to > play GUIs. > > > Later I can imagine using things like sending an SMS from a phone running this > on Android, or placing a graph in a WhatsApp message, make a call to someone > and tell him the temperature is now too high, etc. These things might need > external libraries, but I can learn this later on. > > > > Those things will most likely require external libraries. But most of them are > probably possible. > > ChrisA > -- > https://mail.python.org/mailman/listinfo/python-list From m at funkyhat.org Thu Feb 18 11:44:36 2016 From: m at funkyhat.org (Matt Wheeler) Date: Thu, 18 Feb 2016 16:44:36 +0000 Subject: Guido on python3 for beginners In-Reply-To: References: <48762040-a7e0-434c-92e8-8a0969210e5b@googlegroups.com> <5d31dd14-1a60-4f80-8889-d0616f404c70@googlegroups.com> <4e902c58-3370-4adf-905d-74630a6eea90@googlegroups.com> <60036746-78f7-4f05-a6d8-04e88b09c00f@googlegroups.com> <032f0f66-34df-4325-98e0-1127025d2a94@googlegroups.com> <31a618b2-a407-4723-9a4f-9b756fc93b0b@googlegroups.com> <0a920e92-5d31-4866-815b-cae7e201e4d8@googlegroups.com> <56c568f2$0$2832$c3e8da3$76491128@news.astraweb.com> Message-ID: On Thu, 18 Feb 2016 11:07 Chris Angelico wrote: > By the way... For bash users, adding this to .bashrc may make venvs a > bit easier to keep straight: > > checkdir() { > [ -n "$VIRTUAL_ENV" ] && ! [[ `pwd` =~ `dirname $VIRTUAL_ENV`* ]] > && echo Deactivating venv $VIRTUAL_ENV... && deactivate > [ -z "$VIRTUAL_ENV" -a -d env ] && echo Activating venv > `pwd`/env... && source env/bin/activate > } > PROMPT_COMMAND=checkdir > > (I'm more fluent in Python than in bash, so this is a bit ugly.) > > As soon as you change out of the directory that contains your venv, > it'll be deactivated, so you can't accidentally run stuff in the > "wrong" environment. And as soon as you change into a directory that > contains an 'env' subdir, it'll activate it. (Also applies on first > creation; as soon as "python3 -m venv env" returns, this will activate > the env.) > This is a bad idea for the same reason enabling all magic comments in your editor is a bad idea. (rvm does this but confirms that the rvmrc it's about to activate is one that you've previously approved, I wouldn't want to try to write a shell function that did that though :)). > From framstag at rus.uni-stuttgart.de Thu Feb 18 11:50:48 2016 From: framstag at rus.uni-stuttgart.de (Ulli Horlacher) Date: Thu, 18 Feb 2016 16:50:48 +0000 (UTC) Subject: extending PATH on Windows? References: <1d5acb1l234hka82flgiqiq13cbp5ra03c@4ax.com> Message-ID: eryk sun wrote: > https://hg.python.org/cpython/file/v2.7.11/Tools/scripts/win_add2path.py > > But there are a few issues with this script. (... lot of flaws ...) > Here's a new version for Python 2. I generalized the shell-variable > replacement to a list of well-known folders. Great script, I will save it for later usage! But in the meantime I came to another solution: Instead of modifying PATH systemwide, I modify it only for cmd.exe. To achieve this, I set the registry key "HKEY_CURRENT_USER\Software\Microsoft\Command Processor\AutoRun" to "%USERPROFILE%\autorun.cmd" and write in this file: set PATH=%PATH%;%USERPROFILE%\Desktop If there was already an old value in HKEY_CURRENT_USER\Software\Microsoft\Command Processor\AutoRun it will be saved to %USERPROFILE%\autorun.cmd, too This autorun.cmd is like .bashrc on UNIX. -- Ullrich Horlacher Server und Virtualisierung Rechenzentrum IZUS/TIK E-Mail: horlacher at tik.uni-stuttgart.de Universitaet Stuttgart Tel: ++49-711-68565868 Allmandring 30a Fax: ++49-711-682357 70550 Stuttgart (Germany) WWW: http://www.tik.uni-stuttgart.de/ From framstag at rus.uni-stuttgart.de Thu Feb 18 11:55:29 2016 From: framstag at rus.uni-stuttgart.de (Ulli Horlacher) Date: Thu, 18 Feb 2016 16:55:29 +0000 (UTC) Subject: extending PATH on Windows? References: Message-ID: Ulli Horlacher wrote: > > but simpler still and more reliable to just call QueryValueEx. > > I find it more complicated. I have now (after long studying docs and examples):: def get_winreg(key,subkey): try: rkey = winreg.OpenKey(winreg.HKEY_CURRENT_USER,key,0,winreg.KEY_READ) rvalue,rtype = winreg.QueryValueEx(rkey,subkey) winreg.CloseKey(rkey) except WindowsError: rvalue,rtype = None,None return rvalue def set_winreg(key,subkey,value): winreg.CreateKey(winreg.HKEY_CURRENT_USER,key) rkey = winreg.OpenKey(winreg.HKEY_CURRENT_USER,key,0,winreg.KEY_WRITE) winreg.SetValueEx(rkey,subkey,0,winreg.REG_SZ,value) winreg.CloseKey(rkey) -- Ullrich Horlacher Server und Virtualisierung Rechenzentrum IZUS/TIK E-Mail: horlacher at tik.uni-stuttgart.de Universitaet Stuttgart Tel: ++49-711-68565868 Allmandring 30a Fax: ++49-711-682357 70550 Stuttgart (Germany) WWW: http://www.tik.uni-stuttgart.de/ From wrong.address.1 at gmail.com Thu Feb 18 12:38:03 2016 From: wrong.address.1 at gmail.com (wrong.address.1 at gmail.com) Date: Thu, 18 Feb 2016 09:38:03 -0800 (PST) Subject: Considering migrating to Python from Visual Basic 6 for engineering applications In-Reply-To: References: <90cc50d2-1ce5-4588-9bfd-a49d439f00dd@googlegroups.com> <14c75a68-0d2e-45cc-8d73-0d71b6a6aea6@googlegroups.com> Message-ID: On Thursday, 18 February 2016 17:59:59 UTC+2, William Ray Wing wrote: > > On Feb 18, 2016, at 10:33 AM, wrong.address.1 at gmail.com wrote: > > > > torstai 18. helmikuuta 2016 17.21.32 UTC+2 Oscar Benjamin kirjoitti: > >> On 18 February 2016 at 11:32, Chris Angelico wrote: > >> > > [byte] > > >> It sounds to me as if all of your needs can be solved in pure Python > >> code possibly using some of the popular extension modules from PyPI. > >> In this case it's actually very easy to package/install. You can > >> package your code simply by zipping it up with a __main__.py file. > >> Someone who wants to install it will simply have a two step process: > >> first install Python (and possibly a few dependencies) and then obtain > >> the zip file with your code in it. > >> > >> -- > >> Oscar > > > > This form of packing is not desirable. I can't ask other people to install Python on their machines, and I also would not want show most of the code doing the calculations. > > > > Now things get tricky. I can understand you not wanting to force people to pre-install Python in order for your code to run, but just how deeply do you want to hide or obfuscate it? I don't have to hide it very carefully. If it creates machine code which can be decompiled, I am not very bothered, but I do not want to show many of the equations I will embed in the code, very openly. These will be different for different people, so nobody will have much motivation to break open the code. One *.exe is likely to be used by 2 or 3 people, and then for someone else, I would create another very similar *.exe with somewhat different equations. This is just one of the things I intend to do. Then there are several calculation and plotting routines for my work, which won't be given to anyone, where the issue of secrecy does not arise. > Is this potentially a commercial application to be sold, or are you simply trying to keep things clean and tidy within various divisions of your company. I'd hope not the former, because even VB can get you into tricky licensing issues (and in any case - even fully compiled code in any language these days can be de-compiled into logically correct source code, although with less than obvious variable names). Decompiling machine code I am not at all worried about. The decompiled code will be too messy for people to study. (I won't be writing simple algebraic equations in one or two lines.) > At the other extreme, there are packaging programs (py2exe comes to mind, although I have no experience with it). These wrap the whole python interpreter, your code, and any needed libraries into an executable (clickable) package. Their only downside is that the output packages they produce tend to be large. However, any sophisticated user who digs into them WILL be able to find your source code, possibly only slightly obfuscated by being zipped. > This is not good if the source code with actual variable names becomes visible. Then I would have to use misleading variable names, and call temperature a flow rate, and pressure a magnetic field; stuff a lot of matrix equations which do nothing but get the decoders to waste a huge amount of time. > > Another question I have is regarding reading numerical data from text files. Is it necessary to read one character at a time, or can one read like in Fortran and Basic (something like Input #5, X1, X2, X3)? > > Python can read lines of text or whole blocks of text from source files. If those files are in more or less well-understood form (csv for example) it has libraries specifically designed for reading and extracting data from them (including VERY nice facilities for doing arithmetic and otherwise manipulating time and date data). > That is good. VB.net also has, but in the beginning I got the impression that you could read only one character as a time in a stream. > Bill > > > -- > > https://mail.python.org/mailman/listinfo/python-list From sorsorday at gmail.com Thu Feb 18 12:41:33 2016 From: sorsorday at gmail.com (Herman) Date: Thu, 18 Feb 2016 09:41:33 -0800 Subject: How to properly override the default factory of defaultdict? Message-ID: d = dictutil.DefaultDictWithEnhancedFactory(lambda k: k) self.assertEqual("apple", d['apple']) From: Ben Finney > > you are using the inheritance hierarchy but thwarting it by not using > ?super?. Instead:: > > super().__init__(self, default_factory, *a, **kw) > > and:: > > super().__getitem__(self, key) > -- > \ "Those who will not reason, are bigots, those who cannot, are | > `\ fools, and those who dare not, are slaves." ??Lord? George | > _o__) Gordon Noel Byron | > Ben Finney super does not work for defaultdict. I am using python 2.7. If I use super(defaultdict, self).__init__(default_factory, *a, **kw), I get the error: super(defaultdict, self).__init__(default_factory, *a, **kw) TypeError: 'function' object is not iterable My use case is: d = dictutil.DefaultDictWithEnhancedFactory(lambda k: k) self.assertEqual("apple", d['apple']) > From: Chris Angelico > > Save yourself a lot of trouble, and just override __missing__: > > class DefaultDictWithEnhancedFactory(collections.defaultdict): > def __missing__(self, key): > return self.default_factory(key) > > ChrisA > This works! Thanks. From: "Steven D'Aprano" > > I want to pass in the key to the default_factory of defaultdict > > Just use a regular dict and define __missing__: > > class MyDefaultDict(dict): > def __missing__(self, key): > return "We gotcha key %r right here!" % key > > > If you want a per-instance __missing__, do something like this: > > > class MyDefaultDict(dict): > def __init__(self, factory): > self._factory = factory > def __missing__(self, key): > return self._factory(self, key) > > > d = MyDefaultDict(lambda self, key: ...) > > > -- > Steve > Look like inheriting from defaultdict is easier. I don't even have to override the constructor as suggested by Chris Angelico above. Thanks. From wrong.address.1 at gmail.com Thu Feb 18 12:44:03 2016 From: wrong.address.1 at gmail.com (wrong.address.1 at gmail.com) Date: Thu, 18 Feb 2016 09:44:03 -0800 (PST) Subject: Considering migrating to Python from Visual Basic 6 for engineering applications In-Reply-To: References: <90cc50d2-1ce5-4588-9bfd-a49d439f00dd@googlegroups.com> <14c75a68-0d2e-45cc-8d73-0d71b6a6aea6@googlegroups.com> Message-ID: <14835961-1ee1-44c6-a0a2-dd88d3f9298f@googlegroups.com> On Thursday, 18 February 2016 18:01:17 UTC+2, Dan Strohl wrote: > Disadvantages of python... (compared to VB) > > That's a hard one, but here are my thoughts: (keep in mind that these kinds of discussions are subjective and much is based on the background and experience of the coder, these are also assuming that 100% of your audience is on windows, as soon as you start talking about cross platform support, VB gets much harder to use (yes, you could use it on Linus, via a web based app, or an emulator if you HAD to). > > - I find that VB is much easier at making quick GUI apps (for windows). Yes, and I have used it for about 20 years. VB2 was wonderful for my work. VB3 was fine. VB6 is also OK, but VB.net is too bloated. But can I count on using VB6 for the next five years? Or more? > > - VB is easier to integrate with office apps and other windows specific things (there are some python modules that will access office files etc, but if you want to have an engineering app that directly integrates / interacts with excel, VB is probably better. > Python will read *.csv, and that will be enough. I guess writing *.csv will also be easy. > - I find VB is easier to package and distribute. (there are some good utilities that will take Python and package it into an exe, along with all of the needed pieces, interpreter, etc, but those all require some level of work to setup and make work... not a lot sometimes, but certainly more than "click compile" and copy the .exe file. > Is Python packaging a lot more complicated than VB.net? > - VB is often, for simple apps, often simpler to learn for non-programmers. (I know I will get slammed for that one). For complex apps, I find VB harder to do things than Python, but for example, if I wanted to make a quick windows calculator, I would probably go to VB first.) > > My approach is generally: > > I use Python for: > - server apps, web based apps, plugins, modules, library development, or apps that I want to be able to expand later with plugins, console apps and utilities (things that I am only going to run from the CLI anyway), performance focused apps (unless I need to go all the way to C for performance), anything that might ever need to be cross platform, apps that interact with other (non-Microsoft) apps. > For me, none of these things are very interesting. > I use VB for: > Quick user focused, non-web apps, apps that are used in or directly with Microsoft Office apps, apps that I am developing for someone else that is a VB programmer (or non-programmer but might poke at them).. apps that I need to distribute to lots of less controlled workstations that don't have python on them already. > > These days, I find that I am using VB much less than Python, most of the reasons that I would pick VB can be overcome by developing a cloud app instead of a local app, but there are still times that VB is the right tool (for me at least). > But how long can I continue to use VB6? > > -- > > https://mail.python.org/mailman/listinfo/python-list From wrong.address.1 at gmail.com Thu Feb 18 12:49:54 2016 From: wrong.address.1 at gmail.com (wrong.address.1 at gmail.com) Date: Thu, 18 Feb 2016 09:49:54 -0800 (PST) Subject: Considering migrating to Python from Visual Basic 6 for engineering applications In-Reply-To: References: <90cc50d2-1ce5-4588-9bfd-a49d439f00dd@googlegroups.com> <14c75a68-0d2e-45cc-8d73-0d71b6a6aea6@googlegroups.com> <0a56557a-3c41-4900-b03c-288f1af82c53@googlegroups.com> Message-ID: <9750b589-2045-4634-9364-01e8d5f3dc99@googlegroups.com> On Thursday, 18 February 2016 18:06:29 UTC+2, Chris Angelico wrote: > On Fri, Feb 19, 2016 at 2:49 AM, wrote: > > Thanks. You have guided me quite well, and I am almost ready to declare that I will use Python for the next few decades. > > > > Don't declare like that - just start using it, and see what you think > :) But I would be very much surprised if Python doesn't get added to > your toolbelt. A good programmer should always keep a selection of > tools handy - seldom is one tool the right one for every job. > > > Imagine I want to read in two (or a few) numbers from a text file, display them in two (or more) text boxes, calculate their sums and products, may be take logarithms of them, and display them in other two text boxes or labels, and make some bar charts, scatter plots, and later draw curves and surfaces on a computer screen. Do I really need PyPi or other external stuff for this? Is Python well equipped for this? > > > > You could do all of that with just the standard library, but then your > only choice of GUI library is Tkinter, which is derived from Tcl/Tk. > If you don't like how that looks (either in your code, or in the > resulting GUI), you'll need to turn to PyPI for an alternative (eg > something derived from GTK or wxWindows or Qt). I'm not sure how the > graphing capabilities of Tkinter are, so you might want to grab > matplotlib too - again, it'll give you a lot more flexibility than you > would have if you restrict yourself to *just* the standard library. > But I think you could do all of that - certainly most of it - with > just the language and standard library. > What do I lose by using an external library? Bigger size of the *.exe is not a problem, neither is speed an issue. It might even be easier to learn with these external libraries, and produce more readable code. > > I will have to learn GUI creating quickly after I know the basics of reading and writing text files, and doing simple mathematical calculations. > > > > Reading and writing text files is easy, as is basic mathematics. > You'll have that down in an hour or two, most likely, and then you'll > have time to play GUIs. That is good. > > > Later I can imagine using things like sending an SMS from a phone running this on Android, or placing a graph in a WhatsApp message, make a call to someone and tell him the temperature is now too high, etc. These things might need external libraries, but I can learn this later on. > > > > Those things will most likely require external libraries. But most of > them are probably possible. > > ChrisA Thanks again for your input. From wrong.address.1 at gmail.com Thu Feb 18 12:58:05 2016 From: wrong.address.1 at gmail.com (wrong.address.1 at gmail.com) Date: Thu, 18 Feb 2016 09:58:05 -0800 (PST) Subject: Considering migrating to Python from Visual Basic 6 for engineering applications In-Reply-To: References: <90cc50d2-1ce5-4588-9bfd-a49d439f00dd@googlegroups.com> <14c75a68-0d2e-45cc-8d73-0d71b6a6aea6@googlegroups.com> <0a56557a-3c41-4900-b03c-288f1af82c53@googlegroups.com> Message-ID: <9d101727-7e9d-4031-9814-95b81b7c5520@googlegroups.com> On Thursday, 18 February 2016 18:24:55 UTC+2, Dan Strohl wrote: > I totally agree with Chris here, try it out, do some simpler things to get your feet wet before committing to redoing hundreds of thousands of lines of code in it. Find a few of the "deal breakers" and try to solve them in Python (create a hello world app in python, then package it in an exe and distribute it to some friends).. VB is a fine language, and is likely to suit your needs as well... even if we all believe that Python is better, the effort to convert from one to the other may not make sense. > How long can I depend on VB? I don't really mind learning one more language if there is sufficient utility of it. Besides, I hear that Python is relatively easy to learn compared to C++ or Java. I have some smaller VB programs (single form, little math) for my own use. I could convert those first and get a feel of how it is going to be with larger programs with thousands of lines. > I would also suggest taking a few steps back and seeing if there is a different overall architectural approach that makes sense. If you are using a heavy weight local app, what about a web app? Some things I do can be done in a web app, but the important stuff has to be put into *.exe files for myself and others to use. > What about doing a combination approach? Have a back end running on a server with Python, that feeds a VB app on a client for displaying and manipulating data? (then a Java app for mobile clients or Linux ones?) > I do not know Java, and don't feel inclined to learn that. If I could use VB for years, I need not consider moving over to Python. The trouble is that I don't think we can trust VB (VB6 particularly, but even VB.net) for too long. > Also, looking at existing frameworks and expandable apps out there that you might be able to do less work but still meet your goals with because 80% of the work was done for you... things like django, splunk, tableau, etc... > > Dan > Thanks. > > > -----Original Message----- > > From: Python-list [mailto:python-list-bounces+d.strohl=f5.com at python.org] On > > Behalf Of Chris Angelico > > Sent: Thursday, February 18, 2016 8:06 AM > > Cc: python-list at python.org > > Subject: Re: Considering migrating to Python from Visual Basic 6 for engineering > > applications > > > > On Fri, Feb 19, 2016 at 2:49 AM, wrote: > > > Thanks. You have guided me quite well, and I am almost ready to declare that I > > will use Python for the next few decades. > > > > > > > Don't declare like that - just start using it, and see what you think > > :) But I would be very much surprised if Python doesn't get added to your > > toolbelt. A good programmer should always keep a selection of tools handy - > > seldom is one tool the right one for every job. > > > > > Imagine I want to read in two (or a few) numbers from a text file, display them > > in two (or more) text boxes, calculate their sums and products, may be take > > logarithms of them, and display them in other two text boxes or labels, and > > make some bar charts, scatter plots, and later draw curves and surfaces on a > > computer screen. Do I really need PyPi or other external stuff for this? Is Python > > well equipped for this? > > > > > > > You could do all of that with just the standard library, but then your only choice > > of GUI library is Tkinter, which is derived from Tcl/Tk. > > If you don't like how that looks (either in your code, or in the resulting GUI), > > you'll need to turn to PyPI for an alternative (eg something derived from GTK or > > wxWindows or Qt). I'm not sure how the graphing capabilities of Tkinter are, so > > you might want to grab matplotlib too - again, it'll give you a lot more flexibility > > than you would have if you restrict yourself to *just* the standard library. > > But I think you could do all of that - certainly most of it - with just the language > > and standard library. > > > > > I will have to learn GUI creating quickly after I know the basics of reading and > > writing text files, and doing simple mathematical calculations. > > > > > > > Reading and writing text files is easy, as is basic mathematics. > > You'll have that down in an hour or two, most likely, and then you'll have time to > > play GUIs. > > > > > Later I can imagine using things like sending an SMS from a phone running this > > on Android, or placing a graph in a WhatsApp message, make a call to someone > > and tell him the temperature is now too high, etc. These things might need > > external libraries, but I can learn this later on. > > > > > > > Those things will most likely require external libraries. But most of them are > > probably possible. > > > > ChrisA > > -- > > https://mail.python.org/mailman/listinfo/python-list From jrmy.lnrd at gmail.com Thu Feb 18 13:02:56 2016 From: jrmy.lnrd at gmail.com (Jeremy Leonard) Date: Thu, 18 Feb 2016 10:02:56 -0800 (PST) Subject: Will file be closed automatically in a "for ... in open..." statement? In-Reply-To: References: Message-ID: On Tuesday, February 16, 2016 at 3:39:34 AM UTC-5, jf... at ms4.hinet.net wrote: > I know > > with open('foo.txt') as f: > ...do something... > > will close the file automatically when the "with" block ends. > > I also saw codes in a book: > > for line in open('foo.txt'): > ...do something... > > but it didn't mention if the file will be closed automatically or not when the "for" block ends. Is there any document talking about this? and how to know if a file is in "open" or not? > > --Jach Fong One way of having a file automatically closed is to have something like: with open("file_name") as foo: for x in foo: # process line x Once the with block has completed, the file will be automatically closed. Sorry if this has already been posted, but I wasn't able to see it. From jcasale at activenetwerx.com Thu Feb 18 14:06:29 2016 From: jcasale at activenetwerx.com (Joseph L. Casale) Date: Thu, 18 Feb 2016 19:06:29 +0000 Subject: Passing data across callbacks in ThreadPoolExecutor In-Reply-To: References: , Message-ID: On Thur, Feb 17, 2016 at 9:24 AM, Ian Kelly wrote: >> What is the pattern for chaining execution of tasks with ThreadPoolExecutor? >> Callbacks is not an adequate facility as each task I have will generate new output. > > Can you specify in more detail what your use case is? > > If you don't mind having threads sitting around waiting, you can just > submit each chained task at the start and have each task wait on the > futures of its dependencies. The advantage of this is that it's easy > to conceptualize the dependency graph of the tasks. The disadvantage > is that it eats up extra threads. You'll probably want to increase the > size of the thread pool to handle the waiting tasks (or use a separate > ThreadPoolExecutor for each chained task). The thing with callbacks is each one gets the original tasks result. So the callback can't pass a new task result up and/or cancel the task "set". > Otherwise, is there some reason you can't use multiple callbacks, one > to handle the task's output and one to submit the chained task? > > E.g.: > > def chain_task2(input, f2): > f2 = executor.submit(task2, input, f2.result()) > f2.add_done_callback(handle_task2_done) > > f1 = executor.submit(task1, input) > f1.add_done_callback(handle_task1_done) > f1.add_done_callback(functools.partial(chain_task2, input)) Closing over the executer instance seems potentially race'y. What happens when the last task finishes and the executer checks (locking the queue) and the task also wants to add more work. I'll check the source, but I had hoped for a more succinct way. Thanks, jlc From python.list at tim.thechases.com Thu Feb 18 14:18:00 2016 From: python.list at tim.thechases.com (Tim Chase) Date: Thu, 18 Feb 2016 13:18:00 -0600 Subject: Considering migrating to Python from Visual Basic 6 for engineering applications In-Reply-To: <9d101727-7e9d-4031-9814-95b81b7c5520@googlegroups.com> References: <90cc50d2-1ce5-4588-9bfd-a49d439f00dd@googlegroups.com> <14c75a68-0d2e-45cc-8d73-0d71b6a6aea6@googlegroups.com> <0a56557a-3c41-4900-b03c-288f1af82c53@googlegroups.com> <9d101727-7e9d-4031-9814-95b81b7c5520@googlegroups.com> Message-ID: <20160218131800.40706b66@bigbox.christie.dr> On 2016-02-18 09:58, wrong.address.1 at gmail.com wrote: > How long can I depend on VB? Are you talking the VB6-and-before, or VB.Net? Given that MS dropped support for the VB6 line a decade ago (2005-2008 depending on whether you had extended support) with little to no help in transitioning to VB.Net which (pejoratively called "Visual Fred" because it's so different), I wouldn't want to depend on it at all. Especially since VB.Net is so radically different from VB6. As it seems you've found. > I don't really mind learning one more language if there is > sufficient utility of it. Besides, I hear that Python is relatively > easy to learn compared to C++ or Java. I find that Python is much more readable than the C-flavored languages. Also, while there are language gotchas in all of them, Python's make a lot more sense to me compared to those in the C-flavored languages. > I have some smaller VB programs (single form, little math) for my > own use. I could convert those first and get a feel of how it is > going to be with larger programs with thousands of lines. Excellent plan. -tkc From python.list at tim.thechases.com Thu Feb 18 14:20:55 2016 From: python.list at tim.thechases.com (Tim Chase) Date: Thu, 18 Feb 2016 13:20:55 -0600 Subject: Question on keyword arguments In-Reply-To: <52FA2B3109D743E2BB8534CAC2B5E8A5@OPTIPLEX990> References: <62136715E19142B6B318EA3A3DF04D40@OPTIPLEX990> <7278ec3f250548b192ce2d95c6fa07c5@seaexchmbx03.olympus.F5Net.com> <52FA2B3109D743E2BB8534CAC2B5E8A5@OPTIPLEX990> Message-ID: <20160218132055.71fae02c@bigbox.christie.dr> On 2016-02-18 10:57, grsmith at atlanticbb.net wrote: > Tim, the 'crazy-other-result format' is the > result returned by the database, nothing > I can do about that :) then, much like converting byte-strings to unicode strings as early as possible and converting them back to byte-strings as late as possible, I'd do the same with your DB results: change them into real Python lists as soon as they come out of the DB, operate on them throughout your code as real lists, and only convert them back to crazy-other-format at the time you write them back to the DB. My condolences on having to deal with ugly data. Been there, done that, still stuck with it. :-) -tkc From python.list at tim.thechases.com Thu Feb 18 14:28:07 2016 From: python.list at tim.thechases.com (Tim Chase) Date: Thu, 18 Feb 2016 13:28:07 -0600 Subject: Considering migrating to Python from Visual Basic 6 for engineering applications In-Reply-To: References: <90cc50d2-1ce5-4588-9bfd-a49d439f00dd@googlegroups.com> <14c75a68-0d2e-45cc-8d73-0d71b6a6aea6@googlegroups.com> Message-ID: <20160218132807.7ba2b43d@bigbox.christie.dr> On 2016-02-18 07:33, wrong.address.1 at gmail.com wrote: > Another question I have is regarding reading numerical data from > text files. Is it necessary to read one character at a time, or can > one read like in Fortran and Basic (something like Input #5, X1, > X2, X3)? A lot of my work is extracting data from text files. If it's CSV-format data, Python's "csv" module in the standard library offers some nice tools for doing this. If the numbers are in plain-text, but at fixed column-offsets, I usually create a mapping of slices to make it easier to work with. If this is the case and you want some example code, I can dig some up. Finally, if the data is actually binary, Python's "struct" module (also in the standard library) makes it a lot easier to work with such data. -tkc From john_ladasky at sbcglobal.net Thu Feb 18 15:22:58 2016 From: john_ladasky at sbcglobal.net (John Ladasky) Date: Thu, 18 Feb 2016 12:22:58 -0800 (PST) Subject: Guido on python3 for beginners In-Reply-To: References: <48762040-a7e0-434c-92e8-8a0969210e5b@googlegroups.com> <5d31dd14-1a60-4f80-8889-d0616f404c70@googlegroups.com> <4e902c58-3370-4adf-905d-74630a6eea90@googlegroups.com> <60036746-78f7-4f05-a6d8-04e88b09c00f@googlegroups.com> <032f0f66-34df-4325-98e0-1127025d2a94@googlegroups.com> <31a618b2-a407-4723-9a4f-9b756fc93b0b@googlegroups.com> <0a920e92-5d31-4866-815b-cae7e201e4d8@googlegroups.com> <56c568f2$0$2832$c3e8da3$76491128@news.astraweb.com> Message-ID: <592315d3-7560-41aa-a69b-3036895103d5@googlegroups.com> On Wednesday, February 17, 2016 at 11:28:17 PM UTC-8, Chris Angelico wrote: > 5) print statement/function. Py3 forces you to put parentheses on it, > which is no different from C's printf() or Pike's write() or any > number of other languages where console I/O needs no language support. > Maybe a tiny TINY advantage to Py2 in the short term, but as soon as > you introduce the less basic features, keyword arguments are way > better than the magic syntax the statement needs. (Also, trying to > explain the interaction between the print statement's "soft space" and > other console I/O is not easy.) By the time you've really learned the > language, the advantage belongs to Py3. Another advantage to the Py3 print function over the Py2 print statement is that it makes redirecting output SO much easier. If I write a function that might sometimes print output to the console, and might at some other time print to a GUI window, or to a log file, I can just pass in the relevant output function name as an argument to my own function. This is very helpful, and I use it often. I teach my new Python students to write their application's core logic, without regards to any GUI. Later, when I teach them to use a GUI, building it is easier because the process is modular. From maillist at schwertberger.de Thu Feb 18 15:37:19 2016 From: maillist at schwertberger.de (Dietmar Schwertberger) Date: Thu, 18 Feb 2016 21:37:19 +0100 Subject: Considering migrating to Python from Visual Basic 6 for engineering applications In-Reply-To: <9750b589-2045-4634-9364-01e8d5f3dc99@googlegroups.com> References: <90cc50d2-1ce5-4588-9bfd-a49d439f00dd@googlegroups.com> <14c75a68-0d2e-45cc-8d73-0d71b6a6aea6@googlegroups.com> <0a56557a-3c41-4900-b03c-288f1af82c53@googlegroups.com> <9750b589-2045-4634-9364-01e8d5f3dc99@googlegroups.com> Message-ID: On 18.02.2016 18:49, wrong.address.1 at gmail.com wrote: > What do I lose by using an external library? With using matplotlib for your plots, you can easily create many kinds of plots. On the other hand, if integration with MS Office is a concern, other options may be more suitable for you as matplotlib can not export EMF/WMF files any more. (Using svg or pdf as intermediate format is possible, but no fun.) With the drawing function of e.g. wxPython or PyQt you can either draw on the screen or into a Metafile context using the same code. So you can easily export high quality vector graphics via clipboard or file. Regards, Dietmar From auriocus at gmx.de Thu Feb 18 16:14:22 2016 From: auriocus at gmx.de (Christian Gollwitzer) Date: Thu, 18 Feb 2016 22:14:22 +0100 Subject: [STORY-TIME] THE BDFL AND HIS PYTHON PETTING ZOO In-Reply-To: <032f0f66-34df-4325-98e0-1127025d2a94@googlegroups.com> References: <48762040-a7e0-434c-92e8-8a0969210e5b@googlegroups.com> <5d31dd14-1a60-4f80-8889-d0616f404c70@googlegroups.com> <4e902c58-3370-4adf-905d-74630a6eea90@googlegroups.com> <60036746-78f7-4f05-a6d8-04e88b09c00f@googlegroups.com> <032f0f66-34df-4325-98e0-1127025d2a94@googlegroups.com> Message-ID: Am 16.02.16 um 03:02 schrieb Rick Johnson: > On Friday, February 12, 2016 at 1:51:35 AM UTC-6, John Ladasky wrote: >> I like lazy evaluation. > > Well, it is a "Pythonic feature" no doubt. ?? I'm confused. Does Python have lazy evaluation? I thought that Python does eager evaluation. At least this snippet seems to confirm: def arg(): print("Evaluating arg") return None def func(x): print("Evluating func") print(x) func(arg()) If I run it, the output is: Evaluating arg Evluating func and I think that with side effects, only eager evaluation is really predictable. Christian From cjcollier at linuxfoundation.org Thu Feb 18 16:14:27 2016 From: cjcollier at linuxfoundation.org (cjcollier at linuxfoundation.org) Date: Thu, 18 Feb 2016 13:14:27 -0800 (PST) Subject: PythonTidy In-Reply-To: References: <20061130024509.GA8961@loki> Message-ID: Hello Sir! First, thank you for creating this tool. next, I would recommend perltidy as a program to emulate. I'll paste the output of perltidy --help below. http://perltidy.sourceforge.net/ I could probably write a wrapper script to do this if you'd like. Cheers, C.J. $ perltidy --help This is perltidy version 20140328, a perl script indenter. Usage: perltidy [ options ] file1 file2 file3 ... (output goes to file1.tdy, file2.tdy, file3.tdy, ...) perltidy [ options ] file1 -o outfile perltidy [ options ] file1 -st >outfile perltidy [ options ] outfile Options have short and long forms. Short forms are shown; see man pages for long forms. Note: '=s' indicates a required string, and '=n' indicates a required integer. I/O control -h show this help -o=file name of the output file (only if single input file) -oext=s change output extension from 'tdy' to s -opath=path change path to be 'path' for output files -b backup original to .bak and modify file in-place -bext=s change default backup extension from 'bak' to s -q deactivate error messages (for running under editor) -w include non-critical warning messages in the .ERR error output -syn run perl -c to check syntax (default under unix systems) -log save .LOG file, which has useful diagnostics -f force perltidy to read a binary file -g like -log but writes more detailed .LOG file, for debugging scripts -opt write the set of options actually used to a .LOG file -npro ignore .perltidyrc configuration command file -pro=file read configuration commands from file instead of .perltidyrc -st send output to standard output, STDOUT -se send all error output to standard error output, STDERR -v display version number to standard output and quit Basic Options: -i=n use n columns per indentation level (default n=4) -t tabs: use one tab character per indentation level, not recommeded -nt no tabs: use n spaces per indentation level (default) -et=n entab leading whitespace n spaces per tab; not recommended -io "indent only": just do indentation, no other formatting. -sil=n set starting indentation level to n; use if auto detection fails -ole=s specify output line ending (s=dos or win, mac, unix) -ple keep output line endings same as input (input must be filename) Whitespace Control -fws freeze whitespace; this disables all whitespace changes and disables the following switches: -bt=n sets brace tightness, n= (0 = loose, 1=default, 2 = tight) -bbt same as -bt but for code block braces; same as -bt if not given -bbvt block braces vertically tight; use with -bl or -bli -bbvtl=s make -bbvt to apply to selected list of block types -pt=n paren tightness (n=0, 1 or 2) -sbt=n square bracket tightness (n=0, 1, or 2) -bvt=n brace vertical tightness, n=(0=open, 1=close unless multiple steps on a line, 2=always close) -pvt=n paren vertical tightness (see -bvt for n) -sbvt=n square bracket vertical tightness (see -bvt for n) -bvtc=n closing brace vertical tightness: n=(0=open, 1=sometimes close, 2=always close) -pvtc=n closing paren vertical tightness, see -bvtc for n. -sbvtc=n closing square bracket vertical tightness, see -bvtc for n. -ci=n sets continuation indentation=n, default is n=2 spaces -lp line up parentheses, brackets, and non-BLOCK braces -sfs add space before semicolon in for( ; ; ) -aws allow perltidy to add whitespace (default) -dws delete all old non-essential whitespace -icb indent closing brace of a code block -cti=n closing indentation of paren, square bracket, or non-block brace: n=0 none, =1 align with opening, =2 one full indentation level -icp equivalent to -cti=2 -wls=s want space left of tokens in string; i.e. -nwls='+ - * /' -wrs=s want space right of tokens in string; -sts put space before terminal semicolon of a statement -sak=s put space between keywords given in s and '('; -nsak=s no space between keywords in s and '('; i.e. -nsak='my our local' Line Break Control -fnl freeze newlines; this disables all line break changes and disables the following switches: -anl add newlines; ok to introduce new line breaks -bbs add blank line before subs and packages -bbc add blank line before block comments -bbb add blank line between major blocks -kbl=n keep old blank lines? 0=no, 1=some, 2=all -mbl=n maximum consecutive blank lines to output (default=1) -ce cuddled else; use this style: '} else {' -dnl delete old newlines (default) -l=n maximum line length; default n=80 -bl opening brace on new line -sbl opening sub brace on new line. value of -bl is used if not given. -bli opening brace on new line and indented -bar opening brace always on right, even for long clauses -vt=n vertical tightness (requires -lp); n controls break after opening token: 0=never 1=no break if next line balanced 2=no break -vtc=n vertical tightness of closing container; n controls if closing token starts new line: 0=always 1=not unless list 1=never -wba=s want break after tokens in string; i.e. wba=': .' -wbb=s want break before tokens in string Following Old Breakpoints -kis keep interior semicolons. Allows multiple statements per line. -boc break at old comma breaks: turns off all automatic list formatting -bol break at old logical breakpoints: or, and, ||, && (default) -bok break at old list keyword breakpoints such as map, sort (default) -bot break at old conditional (ternary ?:) operator breakpoints (default) -boa break at old attribute breakpoints -cab=n break at commas after a comma-arrow (=>): n=0 break at all commas after => n=1 stable: break unless this breaks an existing one-line container n=2 break only if a one-line container cannot be formed n=3 do not treat commas after => specially at all Comment controls -ibc indent block comments (default) -isbc indent spaced block comments; may indent unless no leading space -msc=n minimum desired spaces to side comment, default 4 -fpsc=n fix position for side comments; default 0; -csc add or update closing side comments after closing BLOCK brace -dcsc delete closing side comments created by a -csc command -cscp=s change closing side comment prefix to be other than '## end' -cscl=s change closing side comment to apply to selected list of blocks -csci=n minimum number of lines needed to apply a -csc tag, default n=6 -csct=n maximum number of columns of appended text, default n=20 -cscw causes warning if old side comment is overwritten with -csc -sbc use 'static block comments' identified by leading '##' (default) -sbcp=s change static block comment identifier to be other than '##' -osbc outdent static block comments -ssc use 'static side comments' identified by leading '##' (default) -sscp=s change static side comment identifier to be other than '##' Delete selected text -dac delete all comments AND pod -dbc delete block comments -dsc delete side comments -dp delete pod Send selected text to a '.TEE' file -tac tee all comments AND pod -tbc tee block comments -tsc tee side comments -tp tee pod Outdenting -olq outdent long quoted strings (default) -olc outdent a long block comment line -ola outdent statement labels -okw outdent control keywords (redo, next, last, goto, return) -okwl=s specify alternative keywords for -okw command Other controls -mft=n maximum fields per table; default n=40 -x do not format lines before hash-bang line (i.e., for VMS) -asc allows perltidy to add a ';' when missing (default) -dsm allows perltidy to delete an unnecessary ';' (default) Combinations of other parameters -gnu attempt to follow GNU Coding Standards as applied to perl -mangle remove as many newlines as possible (but keep comments and pods) -extrude insert as many newlines as possible Dump and die, debugging -dop dump options used in this run to standard output and quit -ddf dump default options to standard output and quit -dsn dump all option short names to standard output and quit -dln dump option long names to standard output and quit -dpro dump whatever configuration file is in effect to standard output -dtt dump all token types to standard output and quit HTML -html write an html file (see 'man perl2web' for many options) Note: when -html is used, no indentation or formatting are done. Hint: try perltidy -html -css=mystyle.css filename.pl and edit mystyle.css to change the appearance of filename.html. -nnn gives line numbers -pre only writes out 
..
code section -toc places a table of contents to subs at the top (default) -pod passes pod text through pod2html (default) -frm write html as a frame (3 files) -text=s extra extension for table of contents if -frm, default='toc' -sext=s extra extension for file content if -frm, default='src' A prefix of "n" negates short form toggle switches, and a prefix of "no" negates the long forms. For example, -nasc means don't add missing semicolons. If you are unable to see this entire text, try "perltidy -h | more" For more detailed information, and additional options, try "man perltidy", or go to the perltidy home page at http://perltidy.sourceforge.net On Thursday, November 30, 2006 at 6:52:40 PM UTC-8, Chuck Rhode wrote: > Thomas Heller wrote this on Thu, Nov 30, 2006 at 09:50:25PM +0100. My > reply is below. > > > The two things that bother me at the moment are how the comments are > > formatted (dunno if that can be customized or changed easily), and > > it would be good if the script took command line args instead of > > working as a filter only. > > Thank you for trying PythonTidy. > > o Command-line args: Please give an example of a standard command that > I might emulate w.r.t. standard argument use. > > o Comments: Input is parsed twice: I use *tokenize.generate_tokens* to > extract the comments and *compiler.parse* to generate the Abstract > Syntax Tree (AST). Other applications usually use the AST to generate > bytecode, so it contains no information about comments. The tokens > list identifies keywords (and comments and some whitespace) but > doesn't group them into statements. I need both: comments *and* > functional grouping. Fortunately both the AST and the tokens list > carry line numbers to reference the source. Unfortunately the AST > line numbers are relative to functional groups and do not necessarily > apply to the particular keywords that introduce each group. This > makes fixing the position of comments relative to reconstructed code a > bit of a challenge. For example, when a comment has a line number in > the beginning/ending range of what would normally be considered one > command, I have to assume it is an inline comment regardless of how it > may have appeared in the original code. > > Out-of-line comments should appear pretty much as they did in the > original code, however. Can you provide an example where they do not? > Would you prefer that they be left justified or wrapped? >-~ > > Doc strings (for modules, class declarations, and functions) are > another matter. PythonTidy should not mess with them (unless > LEFTJUST_DOC_STRINGS is True). They should appear exactly as > originally written. > > I was taught that code ought to be self documenting and that comments > more often than not diminish readability by padding the code beyond > what can be comprehended on one page (screen). I use them only > minimally and am not greatly inconvenienced when they are moved around > a little. > > -- > .. Chuck Rhode, Sheboygan, WI, USA > .. Weather: http://LacusVeris.com/WX > .. 21? -- Wind N 8 mph From srkunze at mail.de Thu Feb 18 16:37:23 2016 From: srkunze at mail.de (Sven R. Kunze) Date: Thu, 18 Feb 2016 22:37:23 +0100 Subject: Guido on python3 for beginners In-Reply-To: <87r3ga7bct.fsf@jester.gateway.pace.com> References: <48762040-a7e0-434c-92e8-8a0969210e5b@googlegroups.com> <5d31dd14-1a60-4f80-8889-d0616f404c70@googlegroups.com> <4e902c58-3370-4adf-905d-74630a6eea90@googlegroups.com> <60036746-78f7-4f05-a6d8-04e88b09c00f@googlegroups.com> <032f0f66-34df-4325-98e0-1127025d2a94@googlegroups.com> <31a618b2-a407-4723-9a4f-9b756fc93b0b@googlegroups.com> <0a920e92-5d31-4866-815b-cae7e201e4d8@googlegroups.com> <56c568f2$0$2832$c3e8da3$76491128@news.astraweb.com> <87r3ga7bct.fsf@jester.gateway.pace.com> Message-ID: <56C63993.8050005@mail.de> On 18.02.2016 07:59, Paul Rubin wrote: > Steven D'Aprano writes: >> I suppose that it is objectively correct that it is harder to learn >> than Python 2. But I don't think the learning curve is any steeper. If >> anything, the learning curve is ever-so-slightly less steep. > I think py3 has more learning curve because it uses iterators in places > where py2 uses lists. That's a significant new concept and it can be > bug-prone even for programmers who are experienced with it. That is indeed very true. Best, Sven From rosuav at gmail.com Thu Feb 18 16:54:50 2016 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 19 Feb 2016 08:54:50 +1100 Subject: [STORY-TIME] THE BDFL AND HIS PYTHON PETTING ZOO In-Reply-To: References: <48762040-a7e0-434c-92e8-8a0969210e5b@googlegroups.com> <5d31dd14-1a60-4f80-8889-d0616f404c70@googlegroups.com> <4e902c58-3370-4adf-905d-74630a6eea90@googlegroups.com> <60036746-78f7-4f05-a6d8-04e88b09c00f@googlegroups.com> <032f0f66-34df-4325-98e0-1127025d2a94@googlegroups.com> Message-ID: On Fri, Feb 19, 2016 at 8:14 AM, Christian Gollwitzer wrote: > Am 16.02.16 um 03:02 schrieb Rick Johnson: >> >> On Friday, February 12, 2016 at 1:51:35 AM UTC-6, John Ladasky wrote: >>> >>> I like lazy evaluation. >> >> >> Well, it is a "Pythonic feature" no doubt. > > > > ?? I'm confused. Does Python have lazy evaluation? I thought that Python > does eager evaluation. At least this snippet seems to confirm: > > def arg(): > print("Evaluating arg") > return None > > def func(x): > print("Evluating func") > print(x) > > func(arg()) > > If I run it, the output is: > > Evaluating arg > Evluating func > > and I think that with side effects, only eager evaluation is really > predictable. Python's form of lazy evaluation comes in the form of functions that return iterables, rather than concrete lists: >>> def square(x): ... print("Squaring %d..." % x) ... return x*x ... >>> squares = map(square, range(10)) >>> next(squares) Squaring 0... 0 >>> next(squares) Squaring 1... 1 >>> next(squares) Squaring 2... 4 ChrisA From chris at simplistix.co.uk Thu Feb 18 17:27:44 2016 From: chris at simplistix.co.uk (Chris Withers) Date: Thu, 18 Feb 2016 22:27:44 +0000 Subject: testfixtures 4.8.0 Released! Message-ID: <56C64560.6000409@simplistix.co.uk> Hi All, I'm pleased to announce the release of testfixtures 4.9.0 featuring the following: - Suffixes and well as prefixes for compare() assertion errors. - Appropriate metadata to indicate official support for Python 3.5. Thanks for Felix Yan for the metadata patch. Thanks to Wim Glenn for the suffix patch. The package is on PyPI and a full list of all the links to docs, issue trackers and the like can be found here: https://github.com/Simplistix/testfixtures Any questions, please do ask on the Testing in Python list or on the Simplistix open source mailing list... cheers, Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk From breamoreboy at yahoo.co.uk Thu Feb 18 17:35:16 2016 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Thu, 18 Feb 2016 22:35:16 +0000 Subject: testfixtures 4.8.0 Released! In-Reply-To: <56C64560.6000409@simplistix.co.uk> References: <56C64560.6000409@simplistix.co.uk> Message-ID: On 18/02/2016 22:27, Chris Withers wrote: > Hi All, > > I'm pleased to announce the release of testfixtures 4.9.0 featuring the > following: > 4.8.0 or 4.9.0, that is the question, or should it be just how good is your version control? :) -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From steve at pearwood.info Thu Feb 18 19:03:47 2016 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 19 Feb 2016 11:03:47 +1100 Subject: Python keyword args can be any string References: <56c559c2$0$2916$c3e8da3$76491128@news.astraweb.com> Message-ID: <56c65be5$0$1590$c3e8da3$5496439d@news.astraweb.com> On Thu, 18 Feb 2016 06:55 pm, Mark Lawrence wrote: > On 18/02/2016 05:42, Steven D'Aprano wrote: >> Today I learned that **kwargs style keyword arguments can be any string: >> >> py> def test(**kw): >> ... print(kw) >> ... >> py> kwargs = {'abc-def': 42, '': 23, '---': 999, '123': 17} >> py> test(**kwargs) >> {'': 23, '123': 17, '---': 999, 'abc-def': 42} >> >> Bug or feature? >> > > I'm not sure, but what on earth got you to this in the first place? A work colleague wanted to pass an argument starting with "-" to a function. Apparently he didn't have a specific argument in mind. He just wanted to test the function to breaking point by passing invalid argument names. -- Steven From steve at pearwood.info Thu Feb 18 19:30:27 2016 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 19 Feb 2016 11:30:27 +1100 Subject: Considering migrating to Python from Visual Basic 6 for engineering applications References: <90cc50d2-1ce5-4588-9bfd-a49d439f00dd@googlegroups.com> <14c75a68-0d2e-45cc-8d73-0d71b6a6aea6@googlegroups.com> Message-ID: <56c66225$0$1607$c3e8da3$5496439d@news.astraweb.com> On Thu, 18 Feb 2016 10:32 pm, Chris Angelico wrote: > The biggest disadvantage of Python is that, in a number of ways, it > surprises people. Significant whitespace bugs a lot of experienced > programmers This is why we cannot have nice things. "But what if we pass the source code through a system that strips out leading whitespace?" "Then don't do that. What if you pass the source code through a system that strips out braces?" -- Steven From rosuav at gmail.com Thu Feb 18 19:57:28 2016 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 19 Feb 2016 11:57:28 +1100 Subject: Considering migrating to Python from Visual Basic 6 for engineering applications In-Reply-To: <56c66225$0$1607$c3e8da3$5496439d@news.astraweb.com> References: <90cc50d2-1ce5-4588-9bfd-a49d439f00dd@googlegroups.com> <14c75a68-0d2e-45cc-8d73-0d71b6a6aea6@googlegroups.com> <56c66225$0$1607$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Fri, Feb 19, 2016 at 11:30 AM, Steven D'Aprano wrote: > On Thu, 18 Feb 2016 10:32 pm, Chris Angelico wrote: > >> The biggest disadvantage of Python is that, in a number of ways, it >> surprises people. Significant whitespace bugs a lot of experienced >> programmers > > This is why we cannot have nice things. > > "But what if we pass the source code through a system that strips out > leading whitespace?" > > "Then don't do that. What if you pass the source code through a system that > strips out braces?" You may note that I didn't say that this was a flaw in Python. And it's not a tooling issue either (that was mentioned as a parenthesis, but wasn't my main point). I'm talking about how people, those bags of flesh and thoughts, are bugged out by the notion that *whitespace* should matter (other than the mere presence/absence of it). It's the price you pay for being different - people will have trouble comprehending you. ChrisA From ben+python at benfinney.id.au Thu Feb 18 19:58:41 2016 From: ben+python at benfinney.id.au (Ben Finney) Date: Fri, 19 Feb 2016 11:58:41 +1100 Subject: Python keyword args can be any string References: <56c559c2$0$2916$c3e8da3$76491128@news.astraweb.com> <56c65be5$0$1590$c3e8da3$5496439d@news.astraweb.com> Message-ID: <85wpq11poe.fsf@benfinney.id.au> Steven D'Aprano writes: > A work colleague wanted to pass an argument starting with "-" to a > function. > > Apparently he didn't have a specific argument in mind. He just wanted > to test the function to breaking point by passing invalid argument > names. That seems a reasonable test. >>> kwargs = { ... 'spam': 13, ... 'eggs': 17, ... '-beans': 11, ... } >>> foo(**kwargs) # Should this fail? It exposes a design smell; by capturing ?**kwargs?, a function has deiberately not specified which keyword arguments it will accept. >>> def foo_captures_kwargs(*args, **kwargs): ... for (name, value) in kwargs.items(): ... print("Got argument {name}={value!r}".format( ... name=name, value=value)) ... >>> foo_captures_kwargs(**kwargs) Got argument eggs=17 Got argument spam=13 Got argument -beans=11 Still not a bug in Python IMO. It may be a bug in the program; the design of the function doesn't provide any way to know. Perhaps the design can be improved by not using ?**kwargs? at all, and instead using a specific set of keyword-only arguments. >>> def foo_names_every_argument(*, spam="Lorem", eggs="ipsum"): ... print("Got argument {name}={value!r}".format( ... name='spam', value=spam)) ... print("Got argument {name}={value!r}".format( ... name='eggs', value=eggs)) ... >>> foo_names_every_argument(**kwargs) Traceback (most recent call last): File "", line 1, in TypeError: foo_names_every_argument() got an unexpected keyword argument '-beans' This is IMO another good reason to migrate ASAP to Python 3; better design is easier that before. -- \ ?Judge: A law student who marks his own papers.? ?Henry L. | `\ Mencken | _o__) | Ben Finney From ben+python at benfinney.id.au Thu Feb 18 20:03:24 2016 From: ben+python at benfinney.id.au (Ben Finney) Date: Fri, 19 Feb 2016 12:03:24 +1100 Subject: Ohnoes significant whitespace (was: Considering migrating to Python from Visual Basic 6 for engineering applications) References: <90cc50d2-1ce5-4588-9bfd-a49d439f00dd@googlegroups.com> <14c75a68-0d2e-45cc-8d73-0d71b6a6aea6@googlegroups.com> <56c66225$0$1607$c3e8da3$5496439d@news.astraweb.com> Message-ID: <85si0p1pgj.fsf_-_@benfinney.id.au> Chris Angelico writes: > I'm talking about how people, those bags of flesh and thoughts, are > bugged out by the notion that *whitespace* should matter (other than > the mere presence/absence of it). It's the price you pay for being > different - people will have trouble comprehending you. To be fair, there is good reason for the programming (and broader IT) community to have a heuristic of ?significant whitespace is probably bad?. The few languages that did this badly (Makefile syntax, some Unix shell syntax) leave a legacy of countless headaches and you can't fix the language retroactively without breaking backward compatibility. So I am sympathetic to Python newcomers recoiling in horror from significant whitespace, *before* they try it. And because of that, we are burdened with forever needing to deal with that reaction and soothing it. Those people who claim to have tried Python and *still* complain about ?significant whitespace?, I have no sympathy for. Python clearly does it right, and it's a huge boon to readability and reducing simple errors. -- \ ?He was the mildest-mannered man / That ever scuttled ship or | `\ cut a throat.? ??Lord? George Gordon Noel Byron, _Don Juan_ | _o__) | Ben Finney From steve at pearwood.info Thu Feb 18 20:06:47 2016 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 19 Feb 2016 12:06:47 +1100 Subject: Considering migrating to Python from Visual Basic 6 for engineering applications References: <90cc50d2-1ce5-4588-9bfd-a49d439f00dd@googlegroups.com> Message-ID: <56c66aa9$0$1598$c3e8da3$5496439d@news.astraweb.com> On Fri, 19 Feb 2016 02:35 am, wrong.address.1 at gmail.com wrote: > I am almost eager to do this but want to be sure that I know the pitfalls > in using Python for my purposes. Thanks for your encouraging response. Honestly "wrong.address.1", the ONLY pitfall you need to be aware of is to beware of trying to write VB code using Python's interpreter. Python has it's own way of doing things, and if you insist on doing them "just like VB" you will end up with horrible, slow, inefficient, unusable code. That's not to say that VB is worse, or Python is worse, just that they are different. You don't use a hammer the same was a screwdriver. Python is a 20+ year old language used by millions of professionals all over the world for everything from quick scripting, scientific computing, web applications, long-running server applications, and everything in between. The answer to every one of your questions "Can Python do X?" will be one of: (1) Yes it can. (2) No, but instead it will do Y which gives you the same result. I'll be frank, to come here and ask a bunch of questions like: "Is it necessary to read one character at a time...?" is a little bit rude. I don't want to discourage you from asking questions, but think about *how* you ask them. What you're doing is a little bit like going to a car dealer, looking at the cars, then asking a bunch of questions: "So... does this car have a reverse gear or can it only go forward? Do the doors open for entry, or do I have to squeeze through the windows? Can I use the radio while driving, or is there only enough power for one at a time?" In most programming communities, if you start asking questions like that, you can expect to be ignored or abused. Good thing we're a friendly bunch :-) Instead, a good question is: "How do I read a bunch of numbers from a text file?" I'll show you, not only how to read a bunch of numbers, but how to write them first. Of course there are a million different ways you can do this, and for serious use you will probably want to use something like a CSV (Comma Separated Values) file like Excel produces, but for a quick idea of how Python works, let's write some numbers to a file, one per line. Comments start with # and have no effect. Remember that indentation is meaningful: you must use the same number of spaces as shown in my code. Copy and paste this code into the Python interpreter: # ===== cut ===== # Define some numbers. numbers = [1, 45, 38, 99, 1002, 83025, 234, 55, 273, 2] # Open a file for writing. with open("myfile.txt", "w") as f: # Write each number to the file, one per line. for number in numbers: print("Writing %d to the file." % number) f.write(str(number) + "\n") # ===== cut ===== And that's done. Five lines of code, ignoring comments. Now let's read them back and see if they're the same: # ===== cut ===== # Prepare a list to hold the numbers. data = [] # Open a file for reading. with open("myfile.txt", "r") as f: # Read each line. for line in f: value = line.strip() # Get rid of trailing newline. print("Read %s from the file." % value) data.append(int(value)) # Confirm the numbers read in are the same as those written out. if data != numbers: print("mismatch in values") # Print the sorted values. print(sorted(data)) # ===== cut ===== Does this help? -- Steven From breamoreboy at yahoo.co.uk Thu Feb 18 20:07:53 2016 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Fri, 19 Feb 2016 01:07:53 +0000 Subject: Considering migrating to Python from Visual Basic 6 for engineering applications In-Reply-To: References: <90cc50d2-1ce5-4588-9bfd-a49d439f00dd@googlegroups.com> <14c75a68-0d2e-45cc-8d73-0d71b6a6aea6@googlegroups.com> Message-ID: On 18/02/2016 11:32, Chris Angelico wrote: > On Thu, Feb 18, 2016 at 10:11 PM, wrote: >> Almost everything points positively for Python. Thanks to all of you who have responded. But please also tell me the disadvantages of Python. If I start using Python, I should be aware of the price I am paying. Speed is not a big problem for me, so an interpreted language is fine. Is packaging/installing very messy? Do I create dozens of files for a simple program calculating the sum of two numbers and product of two numbers in text boxes with one command to be clicked? Can I learn this much in the first couple of hours? >> > > There are a few warts, particularly on Windows, as regards packaging > and third-party modules. Anything that's written in pure Python is > fairly easy; stuff that's written in C is sometimes a bit hairy. But > that's a limitation on the "extended library" of PyPI, not the stuff > that comes with Python itself. > The vast majority of C code that you're ever likely to need can be obtained here http://www.lfd.uci.edu/~gohlke/pythonlibs/ if there isn't a whl file on pypi. The site might be headed "Unofficial Windows Binaries for Python Extension Packages" but it's as safe as houses, I've been using it for years without any problems at all. -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From steve at pearwood.info Thu Feb 18 20:17:59 2016 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 19 Feb 2016 12:17:59 +1100 Subject: Guido on python3 for beginners References: <48762040-a7e0-434c-92e8-8a0969210e5b@googlegroups.com> <5d31dd14-1a60-4f80-8889-d0616f404c70@googlegroups.com> <4e902c58-3370-4adf-905d-74630a6eea90@googlegroups.com> <60036746-78f7-4f05-a6d8-04e88b09c00f@googlegroups.com> <032f0f66-34df-4325-98e0-1127025d2a94@googlegroups.com> <31a618b2-a407-4723-9a4f-9b756fc93b0b@googlegroups.com> <0a920e92-5d31-4866-815b-cae7e201e4d8@googlegroups.com> <56c568f2$0$2832$c3e8da3$76491128@news.astraweb.com> Message-ID: <56c66d49$0$1620$c3e8da3$5496439d@news.astraweb.com> On Fri, 19 Feb 2016 02:51 am, Random832 wrote: > On Thu, Feb 18, 2016, at 07:25, Rustom Mody wrote: >> My beef is somewhat different: viz that post 70s (Pascal) and 80s >> (scheme) >> programming pedagogy has deteriorated with general purpose languages >> replacing >> 'teaching-purpose language' for teaching. > > The flaw in this idea is right there in your post. Both languages you > named are strongly tied to a single paradigm (procedural for Pascal, and > functional for Scheme) which don't match the paradigm that real-world > work is done in. Is there a new "teaching-purpose language"? Scratch seems very popular for teaching young children. If I were 10, I would definitely check it out. The emphasis is on writing re-usable code snippets, collaboration, and graphics and animation. https://scratch.mit.edu/ But apart from that, I think that "teaching" versus "doing" language is a false dichotomy. Teaching languages should have a shallow learning curve (easy to get started and learn the language, easy discoverability). Production languages should have deep functionality and power. Those two are not *necessarily* opposed[1]. Good languages should have both: a shallow learning curve leading to deep functionality. Pascal was easy to learn and powerful, but it made the mistake of not standardising on a few critical functions that production languages need, like strings. Nevertheless, for the first 10 or 15 years, Apple used a mix of Pascal and assembly to write not just the operating system but a whole lot of applications for the Macintosh. Anyone who says that Pascal is a toy language is just ignorant. [1] Except in the trivial sense that the more you have to learn, the longer it will take. -- Steven From drsalists at gmail.com Thu Feb 18 20:40:22 2016 From: drsalists at gmail.com (Dan Stromberg) Date: Thu, 18 Feb 2016 17:40:22 -0800 Subject: Weird python 2.7 import thing Message-ID: OK, I'd rather be on 3.x, but that's not going to happen today. So 2.7. I have added directories to my sys.path hundreds of times before importing from them, and know what to expect from that. I confess, I don't have a lot of package or egg experience. However, I have a somewhat special (apparently) directory that isn't working. The traceback, when I try to import it, looks like: Traceback (most recent call last): File "t", line 17, in from InfinityUnixHost.activities.ICWrap import ICWrap File "/home/dstromberg/PycharmProjects/infinity_PY/src/InfinityUnixHost/activities/ICWrap.py", line 8, in from InfinityUnixHost.IPGlobal import DB_POOL File "/home/dstromberg/PycharmProjects/infinity_PY/src/InfinityUnixHost/IPGlobal.py", line 12, in INTERNAL_VERSION = pkg_resources.require("InfinityUnixHost")[0].version File "/home/dstromberg/virtualenvs/dev/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 941, in require needed = self.resolve(parse_requirements(requirements)) File "/home/dstromberg/virtualenvs/dev/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 828, in resolve raise DistributionNotFound(req, requirers) pkg_resources.DistributionNotFound: The 'InfinityUnixHost' distribution was not found and is required by the application And a tiny test program that produces this error is: #!/usr/bin/env python import sys import os.path sys.path.insert(0, os.path.abspath('src')) sys.path.insert(0, os.path.abspath('eggs/InfinityCore-6-py2.7.egg')) #for directory in sys.path: # # print(directory) # if \ # os.path.exists(os.path.join(directory, 'InfinityUnixHost')) and \ # os.path.exists(os.path.join(directory, 'InfinityUnixHost/activities')) and \ # os.path.exists(os.path.join(directory, 'InfinityUnixHost/activities/ICWrap.py')): # print('found in %s' % directory) # from InfinityUnixHost.activities.ICWrap import ICWrap from InfinityUnixHost.activities.ICWrap import ICWrap dummy = ICWrap BTW, the commented-out for loop can see the file I am trying to import (when uncommented), but still, the import machinery doesn't load the module. I can't share much detail about the InfinityUnixHost internals unfortunately, but we can probably discuss it as needed. Does anyone know why import is failing in this odd way? It's not even an ImportError! BTW, this is not a package I pip installed from pypi - it's an internal-only project. Thanks! From bc at freeuk.com Thu Feb 18 21:14:51 2016 From: bc at freeuk.com (BartC) Date: Fri, 19 Feb 2016 02:14:51 +0000 Subject: Considering migrating to Python from Visual Basic 6 for engineering applications In-Reply-To: <56c66225$0$1607$c3e8da3$5496439d@news.astraweb.com> References: <90cc50d2-1ce5-4588-9bfd-a49d439f00dd@googlegroups.com> <14c75a68-0d2e-45cc-8d73-0d71b6a6aea6@googlegroups.com> <56c66225$0$1607$c3e8da3$5496439d@news.astraweb.com> Message-ID: On 19/02/2016 00:30, Steven D'Aprano wrote: > On Thu, 18 Feb 2016 10:32 pm, Chris Angelico wrote: > >> The biggest disadvantage of Python is that, in a number of ways, it >> surprises people. Significant whitespace bugs a lot of experienced >> programmers > > This is why we cannot have nice things. > > "But what if we pass the source code through a system that strips out > leading whitespace?" > > "Then don't do that. What if you pass the source code through a system that > strips out braces?" Python doesn't use braces, so that's not a problem! -- Bartc From ethan at stoneleaf.us Thu Feb 18 21:22:01 2016 From: ethan at stoneleaf.us (Ethan Furman) Date: Thu, 18 Feb 2016 18:22:01 -0800 Subject: [STORY-TIME] THE BDFL AND HIS PYTHON PETTING ZOO In-Reply-To: <48762040-a7e0-434c-92e8-8a0969210e5b@googlegroups.com> References: <48762040-a7e0-434c-92e8-8a0969210e5b@googlegroups.com> Message-ID: <56C67C49.9010001@stoneleaf.us> On 02/02/2016 07:26 PM, Rick Johnson wrote: a bunch of garbage. *sigh* Time to update my procmail filters... Anyone know one for deleting all responses to a troll post? -- ~Ethan~ From rustompmody at gmail.com Thu Feb 18 22:32:44 2016 From: rustompmody at gmail.com (Rustom Mody) Date: Thu, 18 Feb 2016 19:32:44 -0800 (PST) Subject: Guido on python3 for beginners In-Reply-To: References: <48762040-a7e0-434c-92e8-8a0969210e5b@googlegroups.com> <5d31dd14-1a60-4f80-8889-d0616f404c70@googlegroups.com> <4e902c58-3370-4adf-905d-74630a6eea90@googlegroups.com> <60036746-78f7-4f05-a6d8-04e88b09c00f@googlegroups.com> <032f0f66-34df-4325-98e0-1127025d2a94@googlegroups.com> <31a618b2-a407-4723-9a4f-9b756fc93b0b@googlegroups.com> <0a920e92-5d31-4866-815b-cae7e201e4d8@googlegroups.com> <56c568f2$0$2832$c3e8da3$76491128@news.astraweb.com> Message-ID: <026a9610-833b-4c8c-a1e9-793e80638f95@googlegroups.com> On Thursday, February 18, 2016 at 9:22:10 PM UTC+5:30, Random832 wrote: > On Thu, Feb 18, 2016, at 07:25, Rustom Mody wrote: > > My beef is somewhat different: viz that post 70s (Pascal) and 80s > > (scheme) > > programming pedagogy has deteriorated with general purpose languages > > replacing > > 'teaching-purpose language' for teaching. > > The flaw in this idea is right there in your post. Both languages you > named are strongly tied to a single paradigm (procedural for Pascal, and > functional for Scheme) which don't match the paradigm that real-world > work is done in. Factually incorrect. Scheme is as multi-paradigm as you can get -- SICP spends considerable time building OO systems, machine architectures etc Culturally is another matter -- scheme tends to draw FP aficionados as python seems to draw OO ones. The languages do not mandate this > Is there a new "teaching-purpose language"? No And that's the deterioration in the programming edu-world From ian.g.kelly at gmail.com Thu Feb 18 22:35:19 2016 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Thu, 18 Feb 2016 20:35:19 -0700 Subject: Passing data across callbacks in ThreadPoolExecutor In-Reply-To: References: Message-ID: On Thu, Feb 18, 2016 at 12:06 PM, Joseph L. Casale wrote: > On Thur, Feb 17, 2016 at 9:24 AM, Ian Kelly wrote: >>> What is the pattern for chaining execution of tasks with ThreadPoolExecutor? >>> Callbacks is not an adequate facility as each task I have will generate new output. >> >> Can you specify in more detail what your use case is? It's still not clear to me specifically what you're trying to do. It would really help if you would describe the problem in more detail. Here's what I think you're trying to do: 1) Submit a task to a ThreadPoolExecutor and get back a future. 2) When the task is complete, submit another task that needs the result of the first task to do its work. 3) When the chained task is complete, return the final result of the chained task back to whoever submitted the original task via the original task's future. The problem arises in that the original task's future already completed when the original task did. The chained task sets its result in a different future that the submitter didn't know about. >> If you don't mind having threads sitting around waiting, you can just >> submit each chained task at the start and have each task wait on the >> futures of its dependencies. The advantage of this is that it's easy >> to conceptualize the dependency graph of the tasks. The disadvantage >> is that it eats up extra threads. You'll probably want to increase the >> size of the thread pool to handle the waiting tasks (or use a separate >> ThreadPoolExecutor for each chained task). > > The thing with callbacks is each one gets the original tasks result. So the > callback can't pass a new task result up and/or cancel the task "set". If you start each individual task at time of submission, then you'll get separate futures for them, and you can add callbacks to each of the futures individually. Each callback will get the result of the task it was added to, not the original task. If you only care about the final result, then you would only add a callback to the future for the final task. >> Otherwise, is there some reason you can't use multiple callbacks, one >> to handle the task's output and one to submit the chained task? >> >> E.g.: >> >> def chain_task2(input, f2): >> f2 = executor.submit(task2, input, f2.result()) >> f2.add_done_callback(handle_task2_done) >> >> f1 = executor.submit(task1, input) >> f1.add_done_callback(handle_task1_done) >> f1.add_done_callback(functools.partial(chain_task2, input)) > > Closing over the executer instance seems potentially race'y. What happens when > the last task finishes and the executer checks (locking the queue) and the task also > wants to add more work. I'll check the source, but I had hoped for a more succinct way. Again, it's not clear to me what scenario you're talking about here. The executor never "checks" the queue; it's only used to submit tasks. The only things pulling from the queue are the worker threads themselves. Besides, if something is waiting on a queue, it's not going to hold the lock on the queue while it's waiting; that would make it impossible to put anything on the queue and it would just end up waiting forever. From phamp at mindspring.com Thu Feb 18 22:39:01 2016 From: phamp at mindspring.com (pyotr filipivich) Date: Thu, 18 Feb 2016 19:39:01 -0800 Subject: extending PATH on Windows? References: Message-ID: Dennis Lee Bieber on Thu, 18 Feb 2016 21:57:13 -0500 typed in comp.lang.python the following: >On Thu, 18 Feb 2016 16:24:00 +0000 (UTC), Ulli Horlacher > declaimed the following: > >>Dennis Lee Bieber wrote: >> >>> >I have >>> >"Python 2.7.11 (v2.7.11:6d1b6a68f775, Dec 5 2015, 20:32:19) [MSC >>> >v.1500 32 bit (Intel)] on win32" >>> >and there is no "win_add2path.py" >>> > >>> C:\Python_x64\Python27\Tools\scripts\win_add2path.py >> >>Ok, It is here in C:\Python27\Tools\scripts\win_add2path.py >>but windows "Search programs and files" was not able to find it. > > Windows (especially 7) search function is highly crippled. There is >some command sequence that will open it up to looking at other file types >and locations. > >http://answers.microsoft.com/en-us/windows/forum/windows_7-files/windows-7-search-does-not-find-files-that-it/61b88d5e-7df7-4427-8a2e-82b801a4a746?auth=1 Thanks. I've found it "simpler" to just open a command prompt, and use DOS. -- pyotr filipivich Next month's Panel: Graft - Boon or blessing? From rustompmody at gmail.com Thu Feb 18 22:39:34 2016 From: rustompmody at gmail.com (Rustom Mody) Date: Thu, 18 Feb 2016 19:39:34 -0800 (PST) Subject: Guido on python3 for beginners In-Reply-To: <56c66d49$0$1620$c3e8da3$5496439d@news.astraweb.com> References: <48762040-a7e0-434c-92e8-8a0969210e5b@googlegroups.com> <5d31dd14-1a60-4f80-8889-d0616f404c70@googlegroups.com> <4e902c58-3370-4adf-905d-74630a6eea90@googlegroups.com> <60036746-78f7-4f05-a6d8-04e88b09c00f@googlegroups.com> <032f0f66-34df-4325-98e0-1127025d2a94@googlegroups.com> <31a618b2-a407-4723-9a4f-9b756fc93b0b@googlegroups.com> <0a920e92-5d31-4866-815b-cae7e201e4d8@googlegroups.com> <56c568f2$0$2832$c3e8da3$76491128@news.astraweb.com> <56c66d49$0$1620$c3e8da3$5496439d@news.astraweb.com> Message-ID: <33b46714-8d9b-4159-ad26-4699d8b11c4d@googlegroups.com> On Friday, February 19, 2016 at 6:48:12 AM UTC+5:30, Steven D'Aprano wrote: > > > But apart from that, I think that "teaching" versus "doing" language is a > false dichotomy. Teaching languages should have a shallow learning curve > (easy to get started and learn the language, easy discoverability). > Production languages should have deep functionality and power. Those two > are not *necessarily* opposed[1]. Good languages should have both: a > shallow learning curve leading to deep functionality. > > [1] Except in the trivial sense that the more you have to learn, the longer > it will take. Consider as hypothesis L1: Needs 4 weeks to master L2: Needs 4 years to manage (if you want to make it less hypothetical think C++) Which would be more satisfying to a student? [Not a hypothetical question: Some people like to go mountaineering/bungee-jumping etc and breaks their necks. Likewise Ive seen students who were so C++-happy they failed other courses! I'd hazard you have forgotten what it feels like to be a student by some decades :-) From ian.g.kelly at gmail.com Thu Feb 18 23:07:23 2016 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Thu, 18 Feb 2016 21:07:23 -0700 Subject: [STORY-TIME] THE BDFL AND HIS PYTHON PETTING ZOO In-Reply-To: <032f0f66-34df-4325-98e0-1127025d2a94@googlegroups.com> References: <48762040-a7e0-434c-92e8-8a0969210e5b@googlegroups.com> <5d31dd14-1a60-4f80-8889-d0616f404c70@googlegroups.com> <4e902c58-3370-4adf-905d-74630a6eea90@googlegroups.com> <60036746-78f7-4f05-a6d8-04e88b09c00f@googlegroups.com> <032f0f66-34df-4325-98e0-1127025d2a94@googlegroups.com> Message-ID: On Mon, Feb 15, 2016 at 7:02 PM, Rick Johnson wrote: > WxPython is not ported either, much to my chagrin. If wxPython "Classic" had just been ported to Python 3, I'm sure it would be all done by now. But it was decided to rebuild wxPython from the ground up instead. Last I've heard, people generally seem to consider Phoenix beta-ready. I understand that it's slow-going though because there's basically only one maintainer and he has a day job. From marko at pacujo.net Fri Feb 19 01:36:21 2016 From: marko at pacujo.net (Marko Rauhamaa) Date: Fri, 19 Feb 2016 08:36:21 +0200 Subject: Ohnoes significant whitespace References: <90cc50d2-1ce5-4588-9bfd-a49d439f00dd@googlegroups.com> <14c75a68-0d2e-45cc-8d73-0d71b6a6aea6@googlegroups.com> <56c66225$0$1607$c3e8da3$5496439d@news.astraweb.com> Message-ID: <87bn7dqk9m.fsf@elektro.pacujo.net> Ben Finney : > So I am sympathetic to Python newcomers recoiling in horror from > significant whitespace, *before* they try it. And because of that, we > are burdened with forever needing to deal with that reaction and > soothing it. I remember being *very* doubtful how the whitespace would work in practice. I was won over by a colleague who had sloppy indentation habbits in C++ but produced tidy, readable code in Python. > Those people who claim to have tried Python and *still* complain about > ?significant whitespace?, I am bitten by it occasionally. The editor doesn't know from the context how a line or block should be (re)indented. In Emacs, TAB, BS and C-M-\, which keep source code properly indented in other programming languages, have been known to lead to accidental bugs in Python code. (The proper way in Emacs is to use C-> and C-< to manipulate blocks of Python code.) > I have no sympathy for. Python clearly does it right, and it's a huge > boon to readability and reducing simple errors. I don't mind Python's syntax. It's the implicit semicolons of JavaScript and Go that I dislike. Marko From srkunze at mail.de Fri Feb 19 04:52:23 2016 From: srkunze at mail.de (Sven R. Kunze) Date: Fri, 19 Feb 2016 10:52:23 +0100 Subject: benchmarking in general and using xheap Message-ID: <56C6E5D7.5030508@mail.de> Hi everybody, I've finally had the time to do the benchmarks and here you go: http://srkunze.blogspot.com/2016/02/the-xheap-benchmark.html The benchmark compares heapq, Heap, OrderHeap, RemovalHeap and XHeap regarding their operation heapify, push and pop. As expected wrapping results in some overhead. Most of the overhead consists of wrapper, super and descriptor calls. As with the current optimizations efforts, I expect this to be reduced even further. But even using current CPython 2.7 or 3.5, the overhead for simple heaps, heaps with a custom orders or heaps with removal can be considered affordable given the maintenance benefits. @srinivas The current removal implementation uses a index-tracking approach with quite some overhead for other operations. I am not sure if that is remediable with a mark-and-sweep approach but given the time I will definitely look into it for another benchmark post now that I have build the infrastructure for it. @all benchmark friends Not sure how you do your benchmarks, but I am somewhat dissatisfied with the current approach. I started out using unittests as they integrated nicely with my current toolchain and I could write actual code. Then I threw anything away and used timeit as suggested here https://mail.python.org/pipermail/python-list/2016-January/702571.html and grew my own set of tools around it to produce readable results; writing code as strings. :-/ And from what I know, there is no official infrastructure (tools, classes, ... such as there is for unittests) around timeit to encapsulate benchmarks, choosing a baseline, calculate ratios etc (and write code instead of strings). Does somebody have an idea here? Best, Sven From jon+usenet at unequivocal.co.uk Fri Feb 19 05:05:18 2016 From: jon+usenet at unequivocal.co.uk (Jon Ribbens) Date: Fri, 19 Feb 2016 10:05:18 -0000 (UTC) Subject: [STORY-TIME] THE BDFL AND HIS PYTHON PETTING ZOO References: <48762040-a7e0-434c-92e8-8a0969210e5b@googlegroups.com> Message-ID: On 2016-02-19, Ethan Furman wrote: > On 02/02/2016 07:26 PM, Rick Johnson wrote: > > a bunch of garbage. > > *sigh* > > Time to update my procmail filters... > > Anyone know one for deleting all responses to a troll post? Well for this particular imbecile, just blocking posts without any lower-case letters in the Subject line would apparently help... From chris at simplistix.co.uk Fri Feb 19 05:06:18 2016 From: chris at simplistix.co.uk (Chris Withers) Date: Fri, 19 Feb 2016 10:06:18 +0000 Subject: [TIP] testfixtures 4.8.0 Released! In-Reply-To: References: <56C64560.6000409@simplistix.co.uk> Message-ID: <56C6E91A.9020908@simplistix.co.uk> On 18/02/2016 22:35, Mark Lawrence wrote: > On 18/02/2016 22:27, Chris Withers wrote: >> Hi All, >> >> I'm pleased to announce the release of testfixtures 4.9.0 featuring the >> following: >> > > 4.8.0 or 4.9.0, that is the question, or should it be just how good is > your version control? :) > Gah, I really need to automate the release emails to stop stupidity like that :-S It's 4.9.0... Chris From steve at pearwood.info Fri Feb 19 05:18:27 2016 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 19 Feb 2016 21:18:27 +1100 Subject: Considering migrating to Python from Visual Basic 6 for engineering applications References: <90cc50d2-1ce5-4588-9bfd-a49d439f00dd@googlegroups.com> <14c75a68-0d2e-45cc-8d73-0d71b6a6aea6@googlegroups.com> <56c66225$0$1607$c3e8da3$5496439d@news.astraweb.com> Message-ID: <56c6ebf4$0$1583$c3e8da3$5496439d@news.astraweb.com> On Fri, 19 Feb 2016 01:14 pm, BartC wrote: > On 19/02/2016 00:30, Steven D'Aprano wrote: [...] >> "Then don't do that. What if you pass the source code through a system >> that strips out braces?" > > Python doesn't use braces, so that's not a problem! mydict = {23: 'twenty-three', 17: 'seventeen'} myset = {1, 2, 4, 8} :-) -- Steven From wxjmfauth at gmail.com Fri Feb 19 05:18:29 2016 From: wxjmfauth at gmail.com (wxjmfauth at gmail.com) Date: Fri, 19 Feb 2016 02:18:29 -0800 (PST) Subject: [STORY-TIME] THE BDFL AND HIS PYTHON PETTING ZOO In-Reply-To: References: <48762040-a7e0-434c-92e8-8a0969210e5b@googlegroups.com> <5d31dd14-1a60-4f80-8889-d0616f404c70@googlegroups.com> <4e902c58-3370-4adf-905d-74630a6eea90@googlegroups.com> <60036746-78f7-4f05-a6d8-04e88b09c00f@googlegroups.com> <032f0f66-34df-4325-98e0-1127025d2a94@googlegroups.com> Message-ID: <4a32d366-4031-4e67-91c9-5121c101d4e7@googlegroups.com> Le vendredi 19 f?vrier 2016 05:08:24 UTC+1, Ian a ?crit?: > On Mon, Feb 15, 2016 at 7:02 PM, Rick Johnson > wrote: > > WxPython is not ported either, much to my chagrin. > > If wxPython "Classic" had just been ported to Python 3, I'm sure it > would be all done by now. But it was decided to rebuild wxPython from > the ground up instead. > > Last I've heard, people generally seem to consider Phoenix beta-ready. > I understand that it's slow-going though because there's basically > only one maintainer and he has a day job. I contributed to wxPython from version 2.1 up to version 2.8. Then come the wxPython 2.9, which had the good idea to try the unification of the ANSI build and the unicode build. It stopped working correctly. wxPhoenix suffered from the same design issue (visible in the first few releases I tested. I doubt, it has changed.) I tried to explain... jmf PS Does it not remind you something ? From framstag at rus.uni-stuttgart.de Fri Feb 19 05:42:52 2016 From: framstag at rus.uni-stuttgart.de (Ulli Horlacher) Date: Fri, 19 Feb 2016 10:42:52 +0000 (UTC) Subject: extending PATH on Windows? References: Message-ID: pyotr filipivich wrote: > > Windows (especially 7) search function is highly crippled. There is > >some command sequence that will open it up to looking at other file types > >and locations. > > > >http://answers.microsoft.com/en-us/windows/forum/windows_7-files/windows-7-search-does-not-find-files-that-it/61b88d5e-7df7-4427-8a2e-82b801a4a746?auth=1 > > > Thanks. I've found it "simpler" to just open a command prompt, and > use DOS. How can one search for files with DOS? -- Ullrich Horlacher Server und Virtualisierung Rechenzentrum IZUS/TIK E-Mail: horlacher at tik.uni-stuttgart.de Universitaet Stuttgart Tel: ++49-711-68565868 Allmandring 30a Fax: ++49-711-682357 70550 Stuttgart (Germany) WWW: http://www.tik.uni-stuttgart.de/ From wrong.address.1 at gmail.com Fri Feb 19 05:47:12 2016 From: wrong.address.1 at gmail.com (wrong.address.1 at gmail.com) Date: Fri, 19 Feb 2016 02:47:12 -0800 (PST) Subject: Considering migrating to Python from Visual Basic 6 for engineering applications In-Reply-To: References: <90cc50d2-1ce5-4588-9bfd-a49d439f00dd@googlegroups.com> <14c75a68-0d2e-45cc-8d73-0d71b6a6aea6@googlegroups.com> Message-ID: <9e57761f-26e1-41c5-8e71-23800de1fdd3@googlegroups.com> On Thursday, 18 February 2016 21:31:20 UTC+2, Tim Chase wrote: > On 2016-02-18 07:33, wrong.address.1 at gmail.com wrote: > > Another question I have is regarding reading numerical data from > > text files. Is it necessary to read one character at a time, or can > > one read like in Fortran and Basic (something like Input #5, X1, > > X2, X3)? > > A lot of my work is extracting data from text files. If it's > CSV-format data, Python's "csv" module in the standard library offers > some nice tools for doing this. > > If the numbers are in plain-text, but at fixed column-offsets, I > usually create a mapping of slices to make it easier to work with. > If this is the case and you want some example code, I can dig some up. > > Finally, if the data is actually binary, Python's "struct" module > (also in the standard library) makes it a lot easier to work with such > data. > > -tkc Thanks. The data I will often have to read from text files could read like 2 12.657823 0.1823467E-04 114 0 3 4 5 9 11 "Lower" 278.15 Is it straightforward to read this, or does one have to read one character at a time and then figure out what the numbers are? From rosuav at gmail.com Fri Feb 19 05:48:45 2016 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 19 Feb 2016 21:48:45 +1100 Subject: extending PATH on Windows? In-Reply-To: References: Message-ID: On Fri, Feb 19, 2016 at 9:42 PM, Ulli Horlacher wrote: > pyotr filipivich wrote: > >> > Windows (especially 7) search function is highly crippled. There is >> >some command sequence that will open it up to looking at other file types >> >and locations. >> > >> >http://answers.microsoft.com/en-us/windows/forum/windows_7-files/windows-7-search-does-not-find-files-that-it/61b88d5e-7df7-4427-8a2e-82b801a4a746?auth=1 >> >> >> Thanks. I've found it "simpler" to just open a command prompt, and >> use DOS. > > How can one search for files with DOS? dir /s /b \*add2path.* ChrisA From wrong.address.1 at gmail.com Fri Feb 19 06:11:04 2016 From: wrong.address.1 at gmail.com (wrong.address.1 at gmail.com) Date: Fri, 19 Feb 2016 03:11:04 -0800 (PST) Subject: Considering migrating to Python from Visual Basic 6 for engineering applications In-Reply-To: <56c66aa9$0$1598$c3e8da3$5496439d@news.astraweb.com> References: <90cc50d2-1ce5-4588-9bfd-a49d439f00dd@googlegroups.com> <56c66aa9$0$1598$c3e8da3$5496439d@news.astraweb.com> Message-ID: <8c67cb1c-4318-4d3b-b425-da894e5138e4@googlegroups.com> On Friday, 19 February 2016 03:06:58 UTC+2, Steven D'Aprano wrote: > On Fri, 19 Feb 2016 02:35 am, wrong.address.1 at gmail.com wrote: > > > I am almost eager to do this but want to be sure that I know the pitfalls > > in using Python for my purposes. Thanks for your encouraging response. > > Honestly "wrong.address.1", the ONLY pitfall you need to be aware of is to > beware of trying to write VB code using Python's interpreter. Python has > it's own way of doing things, and if you insist on doing them "just like > VB" you will end up with horrible, slow, inefficient, unusable code. This I am aware of. And it is not a problem. > That's > not to say that VB is worse, or Python is worse, just that they are > different. You don't use a hammer the same was a screwdriver. > > Python is a 20+ year old language used by millions of professionals all over > the world for everything from quick scripting, scientific computing, web > applications, long-running server applications, and everything in between. > The answer to every one of your questions "Can Python do X?" will be one > of: > > (1) Yes it can. > > (2) No, but instead it will do Y which gives you the same result. > > > I'll be frank, to come here and ask a bunch of questions like: > > "Is it necessary to read one character at a time...?" > > is a little bit rude. I don't want to discourage you from asking questions, > but think about *how* you ask them. What you're doing is a little bit like > going to a car dealer, looking at the cars, then asking a bunch of > questions: No, I first saw this trouble in VB.net. Later I found there was an input command which allowed me to read numbers in varying formats more or less like in Fortran or Basic. I really hope Python has decent ways of reading numeric data which may not follow regular formats. > > "So... does this car have a reverse gear or can it only go forward? Do the > doors open for entry, or do I have to squeeze through the windows? Can I > use the radio while driving, or is there only enough power for one at a > time?" > > In most programming communities, if you start asking questions like that, > you can expect to be ignored or abused. Good thing we're a friendly > bunch :-) I hope it was not rude. At least I did not mean to be. > > > Instead, a good question is: > > "How do I read a bunch of numbers from a text file?" > > > I'll show you, not only how to read a bunch of numbers, but how to write > them first. > > Of course there are a million different ways you can do this, and for > serious use you will probably want to use something like a CSV (Comma > Separated Values) file like Excel produces, but for a quick idea of how > Python works, let's write some numbers to a file, one per line. Comments > start with # and have no effect. Remember that indentation is meaningful: > you must use the same number of spaces as shown in my code. > > Copy and paste this code into the Python interpreter: > > > > # ===== cut ===== > > # Define some numbers. > numbers = [1, 45, 38, 99, 1002, 83025, 234, 55, 273, 2] > # Open a file for writing. > with open("myfile.txt", "w") as f: > # Write each number to the file, one per line. > for number in numbers: > print("Writing %d to the file." % number) > f.write(str(number) + "\n") > > # ===== cut ===== > > > > And that's done. Five lines of code, ignoring comments. Now let's read them > back and see if they're the same: > > > # ===== cut ===== > > # Prepare a list to hold the numbers. > data = [] > # Open a file for reading. > with open("myfile.txt", "r") as f: > # Read each line. > for line in f: > value = line.strip() # Get rid of trailing newline. > print("Read %s from the file." % value) > data.append(int(value)) > I have not yet learnt any Python so I do not understand almost anything. Besides, I will have to read data given to me by people, which may not come in nice formats like CSV, or pre-written by Python. > # Confirm the numbers read in are the same as those written out. > if data != numbers: > print("mismatch in values") > > # Print the sorted values. > print(sorted(data)) > > # ===== cut ===== > > > > Does this help? > Yes. Thanks for your response. I am clear about that I will have to think in a different way and not try to convert VB to Python line by line. > > > -- > Steven From breamoreboy at yahoo.co.uk Fri Feb 19 06:23:59 2016 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Fri, 19 Feb 2016 11:23:59 +0000 Subject: Considering migrating to Python from Visual Basic 6 for engineering applications In-Reply-To: <9e57761f-26e1-41c5-8e71-23800de1fdd3@googlegroups.com> References: <90cc50d2-1ce5-4588-9bfd-a49d439f00dd@googlegroups.com> <14c75a68-0d2e-45cc-8d73-0d71b6a6aea6@googlegroups.com> <9e57761f-26e1-41c5-8e71-23800de1fdd3@googlegroups.com> Message-ID: On 19/02/2016 10:47, wrong.address.1 at gmail.com wrote: > > 2 12.657823 0.1823467E-04 114 0 > 3 4 5 9 11 > "Lower" > 278.15 > > Is it straightforward to read this, or does one have to read one character at a time and then figure out what the numbers are? > One character at a time in a high level language like Python, please. See http://nedbatchelder.com/text/python-parsers.html for a list of parsers that can do all sorts for you. Or the stdlib re module https://docs.python.org/3/library/re.html. Or a likely replacement for the re module https://pypi.python.org/pypi/regex. -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From info at egenix.com Fri Feb 19 06:25:49 2016 From: info at egenix.com (eGenix Team: M.-A. Lemburg) Date: Fri, 19 Feb 2016 12:25:49 +0100 Subject: ANN: eGenix PyRun - One file Python Runtime 2.1.2 Message-ID: <56C6FBBD.1040801@egenix.com> ________________________________________________________________________ ANNOUNCING eGenix PyRun - One file Python Runtime Version 2.1.2 An easy-to-use single file relocatable Python run-time - available for Linux, Mac OS X and Unix platforms, with support for Python 2.6, 2.7 and **also for Python 3.4**. This announcement is also available on our web-site for online reading: http://www.egenix.com/company/news/eGenix-PyRun-2.1.2-GA.html ________________________________________________________________________ INTRODUCTION eGenix PyRun is our open source, one file, no installation version of Python, making the distribution of a Python interpreter to run based scripts and applications to Unix based systems as simple as copying a single file. eGenix PyRun's executable only needs 11MB for Python 2 and 13MB for Python 3, but still supports most Python application and scripts - and it can be compressed to just 3-4MB using upx, if needed. Compared to a regular Python installation of typically 100MB on disk, eGenix PyRun is ideal for applications and scripts that need to be distributed to several target machines, client installations or customers. It makes "installing" Python on a Unix based system as simple as copying a single file. eGenix has been using eGenix PyRun internally in the mxODBC Connect Server product since 2008 with great success and decided to make it available as a stand-alone open-source product. We provide both the source archive to build your own eGenix PyRun, as well as pre-compiled binaries for Linux, FreeBSD and Mac OS X, as 32- and 64-bit versions. The binaries can be downloaded manually, or you can let our automatic install script install-pyrun take care of the installation: ./install-pyrun dir and you're done. Please see the product page for more details: http://www.egenix.com/products/python/PyRun/ ________________________________________________________________________ NEWS This patch level release of eGenix PyRun 2.1 comes with the following enhancements: Enhancements / Changes ---------------------- * Upgraded eGenix PyRun to work with and use Python 2.7.11 per default. Python 2.7.11 now uses computed gotos in the interpreter loop, adding some more performance. In addition, the vulnerable SSLv3 protocol was switched off in the defaults of the ssl module. * Upgraded eGenix PyRun to work with and use Python 3.4.4 for the Python 3 version. * Added an explicit --libpath configure setting to work a problem with some 64-bit platforms defaulting to lib64/. The new setting forces the libs to always go into a lib/ directory. install-pyrun Quick Install Enhancements --------------------------------------------- eGenix PyRun includes a shell script called install-pyrun, which greatly simplifies installation of PyRun. It works much like the virtualenv shell script used for creating new virtual environments (except that there's nothing virtual about PyRun environments). https://downloads.egenix.com/python/install-pyrun With the script, an eGenix PyRun installation is as simple as running: ./install-pyrun targetdir This will automatically detect the platform, download and install the right pyrun version into targetdir. We have updated this script since the last release: * Updated install-pyrun to default to eGenix PyRun 2.1.2 and its feature set. For a complete list of changes, please see the eGenix PyRun Changelog: http://www.egenix.com/products/python/PyRun/changelog.html ________________________________________________________________________ LICENSE eGenix PyRun is distributed under the eGenix.com Public License 1.1.0 which is an Open Source license similar to the Python license. You can use eGenix PyRun in both commercial and non-commercial settings without fee or charge. Please see our license page for more details: http://www.egenix.com/products/python/PyRun/license.html The package comes with full source code. ________________________________________________________________________ DOWNLOADS The download archives and instructions for installing eGenix PyRun can be found at: http://www.egenix.com/products/python/PyRun/ As always, we are providing pre-built binaries for all common platforms: Windows 32/64-bit, Linux 32/64-bit, FreeBSD 32/64-bit, Mac OS X 32/64-bit. Source code archives are available for installation on other platforms, such as Solaris, AIX, HP-UX, etc. _______________________________________________________________________ SUPPORT Commercial support for this product is available from eGenix.com. Please see http://www.egenix.com/services/support/ for details about our support offerings. ________________________________________________________________________ MORE INFORMATION For more information about eGenix PyRun, licensing and download instructions, please visit our web-site: http://www.egenix.com/products/python/PyRun/ About eGenix (http://www.egenix.com/): eGenix is a Python software project, consulting and product company delivering expert services and professional quality products for companies, Python users and developers. We specialize in database driven applications, large scale software designs and integration. Enjoy, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Experts (#1, Feb 19 2016) >>> Python Projects, Coaching and Consulting ... http://www.egenix.com/ >>> Python Database Interfaces ... http://products.egenix.com/ >>> Plone/Zope Database Interfaces ... http://zope.egenix.com/ ________________________________________________________________________ ::: We implement business ideas - efficiently in both time and costs ::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ http://www.malemburg.com/ From tony at vanderhoff.org Fri Feb 19 06:34:40 2016 From: tony at vanderhoff.org (Tony van der Hoff) Date: Fri, 19 Feb 2016 11:34:40 +0000 Subject: Considering migrating to Python from Visual Basic 6 for engineering applications In-Reply-To: References: <90cc50d2-1ce5-4588-9bfd-a49d439f00dd@googlegroups.com> <14c75a68-0d2e-45cc-8d73-0d71b6a6aea6@googlegroups.com> <9e57761f-26e1-41c5-8e71-23800de1fdd3@googlegroups.com> Message-ID: <56C6FDD0.7090109@vanderhoff.org> On 19/02/16 11:23, Mark Lawrence wrote: > On 19/02/2016 10:47, wrong.address.1 at gmail.com wrote: >> >> 2 12.657823 0.1823467E-04 114 0 >> 3 4 5 9 11 >> "Lower" >> 278.15 >> >> Is it straightforward to read this, or does one have to read one >> character at a time and then figure out what the numbers are? >> > > One character at a time in a high level language like Python, please. > See http://nedbatchelder.com/text/python-parsers.html for a list of > parsers that can do all sorts for you. Or the stdlib re module > https://docs.python.org/3/library/re.html. Or a likely replacement for > the re module https://pypi.python.org/pypi/regex. > Am I alone in thinking that this wrongaddress character is trolling? How much effort can it be to just install python, and try out these simple things *before* asking trivia here? -- Tony van der Hoff | mailto:tony at vanderhoff.org Buckinghamshire, England | From rustompmody at gmail.com Fri Feb 19 06:41:55 2016 From: rustompmody at gmail.com (Rustom Mody) Date: Fri, 19 Feb 2016 03:41:55 -0800 (PST) Subject: Python keyword args can be any string In-Reply-To: <56c559c2$0$2916$c3e8da3$76491128@news.astraweb.com> References: <56c559c2$0$2916$c3e8da3$76491128@news.astraweb.com> Message-ID: <95706d9d-1e34-4686-b928-146a66341711@googlegroups.com> On Thursday, February 18, 2016 at 11:12:45 AM UTC+5:30, Steven D'Aprano wrote: > Today I learned that **kwargs style keyword arguments can be any string: > > > py> def test(**kw): > ... print(kw) > ... > py> kwargs = {'abc-def': 42, '': 23, '---': 999, '123': 17} > py> test(**kwargs) > {'': 23, '123': 17, '---': 999, 'abc-def': 42} > Hoo boy! Namespaces are a honking great idea though sometimes the honk is too loud > > Bug or feature? Dunno... I dont like it From bthk55 at gmail.com Fri Feb 19 06:45:04 2016 From: bthk55 at gmail.com (bthk55 at gmail.com) Date: Fri, 19 Feb 2016 03:45:04 -0800 (PST) Subject: Compilation of Python 3.5.1 fails on Solaris 8 sparc Message-ID: <18c753d2-85b5-436b-9037-6edd40a12654@googlegroups.com> When I try to compile the python from source on Solaris 8. I getting the following error """ Modules/_localemodule.o: In function PyIntl_gettext': Modules/_localemodule.o(.text+0xb60): undefined reference to libintl_gettext' Modules/_localemodule.o: In function PyIntl_dgettext': Modules/_localemodule.o(.text+0xba8): undefined reference to libintl_dgettext' Modules/_localemodule.o: In function PyIntl_dcgettext': Modules/_localemodule.o(.text+0xbf8): undefined reference to libintl_dcgettext' Modules/_localemodule.o: In function PyIntl_textdomain': Modules/_localemodule.o(.text+0xc38): undefined reference to libintl_textdomain' Modules/_localemodule.o: In function PyIntl_bindtextdomain': Modules/_localemodule.o(.text+0xcec): undefined reference to libintl_bindtextdomain' Modules/_localemodule.o: In function PyIntl_bind_textdomain_codeset': Modules/_localemodule.o(.text+0xde0): undefined reference to libintl_bind_textdomain_codeset' collect2: ld returned 1 exit status make: *** [Programs/_freeze_importlib] Error 1 """ CPPFLAGS="-I/usr/local/include" export CPPFLAGS LD_OPTIONS='-L/usr/local/lib -R/usr/local/lib' export LD_OPTIONS LDFLAGS='-L/usr/local/lib -R/usr/local/lib' export LDFLAGS CC=/usr/local/bin/gcc OPT="-O2" ./configure --prefix=/opt/soe/local/Python-3.5.1 From bthk55 at gmail.com Fri Feb 19 06:49:01 2016 From: bthk55 at gmail.com (bthk55 at gmail.com) Date: Fri, 19 Feb 2016 03:49:01 -0800 (PST) Subject: Compilation of Python 3.5.1 fails on Solaris 8 sparc Message-ID: <3bf2f2cf-e3e6-424f-987a-3d0dc05f82dc@googlegroups.com> When I try to compile the python from source on Solaris 8. I getting the following error """ Modules/_localemodule.o: In function PyIntl_gettext': Modules/_localemodule.o(.text+0xb60): undefined reference to libintl_gettext' Modules/_localemodule.o: In function PyIntl_dgettext': Modules/_localemodule.o(.text+0xba8): undefined reference to libintl_dgettext' Modules/_localemodule.o: In function PyIntl_dcgettext': Modules/_localemodule.o(.text+0xbf8): undefined reference to libintl_dcgettext' Modules/_localemodule.o: In function PyIntl_textdomain': Modules/_localemodule.o(.text+0xc38): undefined reference to libintl_textdomain' Modules/_localemodule.o: In function PyIntl_bindtextdomain': Modules/_localemodule.o(.text+0xcec): undefined reference to libintl_bindtextdomain' Modules/_localemodule.o: In function PyIntl_bind_textdomain_codeset': Modules/_localemodule.o(.text+0xde0): undefined reference to libintl_bind_textdomain_codeset' collect2: ld returned 1 exit status make: *** [Programs/_freeze_importlib] Error 1 """ The steps that I followed : CPPFLAGS="-I/usr/local/include" export CPPFLAGS LD_OPTIONS='-L/usr/local/lib -R/usr/local/lib' export LD_OPTIONS LDFLAGS='-L/usr/local/lib -R/usr/local/lib' export LDFLAGS CC=/usr/local/bin/gcc OPT="-O2" ./configure --prefix=/opt/soe/local/Python-3.5.1 From wrong.address.1 at gmail.com Fri Feb 19 06:53:03 2016 From: wrong.address.1 at gmail.com (wrong.address.1 at gmail.com) Date: Fri, 19 Feb 2016 03:53:03 -0800 (PST) Subject: Considering migrating to Python from Visual Basic 6 for engineering applications In-Reply-To: References: <90cc50d2-1ce5-4588-9bfd-a49d439f00dd@googlegroups.com> <14c75a68-0d2e-45cc-8d73-0d71b6a6aea6@googlegroups.com> <9e57761f-26e1-41c5-8e71-23800de1fdd3@googlegroups.com> Message-ID: <729e6513-4d1d-4b9b-be8b-4b7664abac2e@googlegroups.com> On Friday, 19 February 2016 13:24:46 UTC+2, Mark Lawrence wrote: > On 19/02/2016 10:47, wrong.address.1 at gmail.com wrote: > > > > 2 12.657823 0.1823467E-04 114 0 > > 3 4 5 9 11 > > "Lower" > > 278.15 > > > > Is it straightforward to read this, or does one have to read one character at a time and then figure out what the numbers are? > > > > One character at a time in a high level language like Python, please. That is what VB.net seems designed for. I know nothing about Python, and have to decide soon, so I am asking these *very stupid* questions. > See http://nedbatchelder.com/text/python-parsers.html for a list of > parsers that can do all sorts for you. Or the stdlib re module I am an engineer, and do not understand most of the terminology used there. And do I need something fancy to read a line of numbers? Should it not be built into the language? The new languages are not designed by engineers or for engineers, so it is likely that what we take for granted may not be easy in Python. It was different when there was no such thing called "computer science" in the 1960s. > https://docs.python.org/3/library/re.html. Or a likely replacement for > the re module https://pypi.python.org/pypi/regex. > Thanks. > -- > My fellow Pythonistas, ask not what our language can do for you, ask > what you can do for our language. > > Mark Lawrence From python.list at tim.thechases.com Fri Feb 19 08:40:57 2016 From: python.list at tim.thechases.com (Tim Chase) Date: Fri, 19 Feb 2016 07:40:57 -0600 Subject: Considering migrating to Python from Visual Basic 6 for engineering applications In-Reply-To: <9e57761f-26e1-41c5-8e71-23800de1fdd3@googlegroups.com> References: <90cc50d2-1ce5-4588-9bfd-a49d439f00dd@googlegroups.com> <14c75a68-0d2e-45cc-8d73-0d71b6a6aea6@googlegroups.com> <9e57761f-26e1-41c5-8e71-23800de1fdd3@googlegroups.com> Message-ID: <20160219074057.16b9eddb@bigbox.christie.dr> On 2016-02-19 02:47, wrong.address.1 at gmail.com wrote: > 2 12.657823 0.1823467E-04 114 0 > 3 4 5 9 11 > "Lower" > 278.15 > > Is it straightforward to read this, or does one have to read one > character at a time and then figure out what the numbers are? -- It's easy to read. What you do with that mess of data is the complex part. They come in as byte-strings, but you'd have to convert them to the corresponding formats: from shlex import shlex USE_LEX = True # False with open('data.txt') as f: for i, line in enumerate(f, 1): if USE_LEX: bits = shlex(line) else: bits = line.split() for j, bit in enumerate(bits, 1): if bit.isdigit(): result = int(bit) t = "an int" elif '"' in bit: result = bit t = "a string" else: result = float(bit) t = "a float" print("On line %i I think that item %i %r is %s: %r" % ( i, j, bit, t, result, )) The USE_LEX controls whether the example code uses string-splitting on white-space, or uses the built-in "shlex" module to parse for quoted strings that might contain a space. The naive way of string-splitting will be faster, but choke on string-data containing spaces. You'd have to make up your own heuristics for determining what type each data "bit" is, parsing it out (with int(), float() or whatever), but the above gives you some rough ideas with at least one known bug/edge-case. But we can't do *all* the work for you ;-) -tkc From gvanem at yahoo.no Fri Feb 19 09:18:08 2016 From: gvanem at yahoo.no (Gisle Vanem) Date: Fri, 19 Feb 2016 15:18:08 +0100 Subject: extending PATH on Windows? In-Reply-To: References: Message-ID: <56C72420.8020607@yahoo.no> Dennis Lee Bieber wrote: >>> How can one search for files with DOS? >> >> dir /s /b \*add2path.* >> >> ChrisA > > Or move to PowerShell... > > Windows PowerShell > Copyright (C) 2009 Microsoft Corporation. All rights reserved. > > PS C:\Users\Wulfraed\Documents> Get-ChildItem -Path c:\ -Filter *add2path* > -Name -Recurse Clumsy IMHO. Try: c:\> envtool --path *ipython* Matches in %PATH: 12 Feb 2015 - 17:16:12: f:\ProgramFiler\Python27\Scripts\ipython-script.py 12 Feb 2015 - 17:16:12: f:\ProgramFiler\Python27\Scripts\ipython.exe 12 Feb 2015 - 17:16:12: f:\ProgramFiler\Python27\Scripts\ipython.exe.manifest 12 Feb 2015 - 17:16:12: f:\ProgramFiler\Python27\Scripts\ipython2-script.py 12 Feb 2015 - 17:16:12: f:\ProgramFiler\Python27\Scripts\ipython2.exe 12 Feb 2015 - 17:16:12: f:\ProgramFiler\Python27\Scripts\ipython2.exe.manifest 08 Feb 2015 - 12:48:56: f:\ProgramFiler\Python27\Scripts\ipython_win_post_install.py 7 matches found for "*ipython*". ------------------- Or find a spec on *all partitions* using Everything Search engine: c:\> envtool --evry *ipython* Matches from EveryThing: ... 133 matches found for "*ipython*". ------------------- Or for Python-path searches: c:\> envtool --python *ipython* Matches in "f:\ProgramFiler\Python27\python.exe" sys.path[]: 23 Jan 2016 - 13:50:30: f:\ProgramFiler\Python27\lib\site-packages\ipython-2.1.0-py2.7.egg\IPython\ 23 Jan 2016 - 13:50:26: f:\ProgramFiler\Python27\lib\site-packages\IPython\ 23 Jan 2016 - 13:50:30: f:\ProgramFiler\Python27\lib\site-packages\ipython-2.1.0-py2.7.egg\ 3 matches found for "*ipython*". ----- My envtool is at https://github.com/gvanem/EnvTool EveryThing is at http://voidtools.com/ From invalid at invalid.invalid Fri Feb 19 09:57:48 2016 From: invalid at invalid.invalid (Grant Edwards) Date: Fri, 19 Feb 2016 14:57:48 +0000 (UTC) Subject: Ohnoes significant whitespace (was: Considering migrating to Python from Visual Basic 6 for engineering applications) References: <90cc50d2-1ce5-4588-9bfd-a49d439f00dd@googlegroups.com> <14c75a68-0d2e-45cc-8d73-0d71b6a6aea6@googlegroups.com> <56c66225$0$1607$c3e8da3$5496439d@news.astraweb.com> Message-ID: On 2016-02-19, Ben Finney wrote: > So I am sympathetic to Python newcomers recoiling in horror from > significant whitespace, *before* they try it. And because of that, we > are burdened with forever needing to deal with that reaction and > soothing it. The first time I wrote Python (it was the only language I could find that was free and for which I found understandable examples on how to suck e-mail messages out of Outlook using DCOM -- which was the problem to be solved), I had an initial aversion to the "significant whitespace" concept. That immediately vanished once I started working on my first Python code. That was Python 1.5.2 back in 1999. Shortly after that, I went to the trouble to add raw socket support to the Python standard library "socket" module so that I could use Python for some other tasks. :) > Those people who claim to have tried Python and *still* complain > about ?significant whitespace?, I have no sympathy for. I, on the other hand, do feel sorry for them because their brains are evidently broken in some basic manner that can't help but cause them suffering. > Python clearly does it right, and it's a huge boon to readability > and reducing simple errors. Indeed. -- Grant Edwards grant.b.edwards Yow! Used staples are good at with SOY SAUCE! gmail.com From __peter__ at web.de Fri Feb 19 09:58:13 2016 From: __peter__ at web.de (Peter Otten) Date: Fri, 19 Feb 2016 15:58:13 +0100 Subject: Considering migrating to Python from Visual Basic 6 for engineering applications References: <90cc50d2-1ce5-4588-9bfd-a49d439f00dd@googlegroups.com> <14c75a68-0d2e-45cc-8d73-0d71b6a6aea6@googlegroups.com> <9e57761f-26e1-41c5-8e71-23800de1fdd3@googlegroups.com> <20160219074057.16b9eddb@bigbox.christie.dr> Message-ID: Tim Chase wrote: > On 2016-02-19 02:47, wrong.address.1 at gmail.com wrote: >> 2 12.657823 0.1823467E-04 114 0 >> 3 4 5 9 11 >> "Lower" >> 278.15 >> >> Is it straightforward to read this, or does one have to read one >> character at a time and then figure out what the numbers are? -- > > It's easy to read. What you do with that mess of data is the complex > part. They come in as byte-strings, but you'd have to convert them > to the corresponding formats: > > from shlex import shlex > USE_LEX = True # False > with open('data.txt') as f: > for i, line in enumerate(f, 1): > if USE_LEX: > bits = shlex(line) > else: > bits = line.split() > for j, bit in enumerate(bits, 1): > if bit.isdigit(): > result = int(bit) > t = "an int" > elif '"' in bit: > result = bit > t = "a string" > else: > result = float(bit) > t = "a float" > print("On line %i I think that item %i %r is %s: %r" % ( > i, > j, > bit, > t, > result, > )) > > The USE_LEX controls whether the example code uses string-splitting > on white-space, or uses the built-in "shlex" module to parse for > quoted strings that might contain a space. The naive way of > string-splitting will be faster, but choke on string-data containing > spaces. > > You'd have to make up your own heuristics for determining what type > each data "bit" is, parsing it out (with int(), float() or whatever), > but the above gives you some rough ideas with at least one known > bug/edge-case. Or just tell the parser what to expect: $ cat read_data_shlex2.py import shlex CONVERTERS = { "i": int, "f": float, "s": str } def parse_line(types, line=None, file=None): if line is None: line = file.readline() values = shlex.split(line) if len(values) != len(types): raise ValueError("Too few/many values %r <-- %r" % (types, values)) return tuple(CONVERTERS[t](v) for t, v in zip(types, values)) with open("data.txt") as f: print(parse_line("iffii", file=f)) print(parse_line("iiiii", file=f)) print(parse_line("s", file=f)) print(parse_line("fsi", file=f)) print(parse_line("ff", file=f)) $ cat data.txt 2 12.657823 0.1823467E-04 114 0 3 4 5 9 11 "Lower" 1.2 "foo \" bar \\ baz" 42 278.15 $ python3 read_data_shlex2.py (2, 12.657823, 1.823467e-05, 114, 0) (3, 4, 5, 9, 11) ('Lower',) (1.2, 'foo " bar \\ baz', 42) Traceback (most recent call last): File "read_data_shlex2.py", line 24, in print(parse_line("ff", file=f)) File "read_data_shlex2.py", line 15, in parse_line raise ValueError("Too few/many values %r <-- %r" % (types, values)) ValueError: Too few/many values 'ff' <-- ['278.15'] $ But we can't do *all* the work for you ;-) If this thread goes long enough eventually we will ;) From invalid at invalid.invalid Fri Feb 19 10:06:15 2016 From: invalid at invalid.invalid (Grant Edwards) Date: Fri, 19 Feb 2016 15:06:15 +0000 (UTC) Subject: Guido on python3 for beginners References: <48762040-a7e0-434c-92e8-8a0969210e5b@googlegroups.com> <5d31dd14-1a60-4f80-8889-d0616f404c70@googlegroups.com> <4e902c58-3370-4adf-905d-74630a6eea90@googlegroups.com> <60036746-78f7-4f05-a6d8-04e88b09c00f@googlegroups.com> <032f0f66-34df-4325-98e0-1127025d2a94@googlegroups.com> <31a618b2-a407-4723-9a4f-9b756fc93b0b@googlegroups.com> <0a920e92-5d31-4866-815b-cae7e201e4d8@googlegroups.com> <56c568f2$0$2832$c3e8da3$76491128@news.astraweb.com> <56c66d49$0$1620$c3e8da3$5496439d@news.astraweb.com> Message-ID: On 2016-02-19, Steven D'Aprano wrote: > Pascal was easy to learn and powerful, but it made the mistake of not > standardising on a few critical functions that production languages need, > like strings. Nevertheless, for the first 10 or 15 years, Apple used a mix > of Pascal and assembly to write not just the operating system but a whole > lot of applications for the Macintosh. Anyone who says that Pascal is a toy > language is just ignorant. At one point, Pascal was quite popular for emedded systems (e.g. microprocessor) development also. One of my first projects out of school back in the early 80's was working on cell-site radio firmware (Zilog Z8000 uP). There were a few small extensions to the standard langage to deal with the fact it was low-level code running on bare metal, but it was quite a nice language for writing stuff for small microprocessor-based systems. After you debugged everything, you could turn off array-bounds-checking, and the extra overhead compared to something like C vanished. -- Grant Edwards grant.b.edwards Yow! HAIR TONICS, please!! at gmail.com From bc at freeuk.com Fri Feb 19 10:34:30 2016 From: bc at freeuk.com (BartC) Date: Fri, 19 Feb 2016 15:34:30 +0000 Subject: Considering migrating to Python from Visual Basic 6 for engineering applications In-Reply-To: <90cc50d2-1ce5-4588-9bfd-a49d439f00dd@googlegroups.com> References: <90cc50d2-1ce5-4588-9bfd-a49d439f00dd@googlegroups.com> Message-ID: On 17/02/2016 19:49, wrong.address.1 at gmail.com wrote: > Could someone kindly tell me advantages and disadvantages of Python? Something I don't think anyone has mentioned is that Python is case-sensitive. So keywords, identifiers, library functions and so on have to be written just right. (For example the identifier "abcdef" can be written 64 different ways, such as "aBCdEf", and they will all be different symbols.) This is a complete pain if you're used to a language that ignores case (except in data). But pretty much every language now is the same, you can't do much about it, you just need to be aware of it. -- Bartc From rosuav at gmail.com Fri Feb 19 10:43:27 2016 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 20 Feb 2016 02:43:27 +1100 Subject: Considering migrating to Python from Visual Basic 6 for engineering applications In-Reply-To: References: <90cc50d2-1ce5-4588-9bfd-a49d439f00dd@googlegroups.com> Message-ID: On Sat, Feb 20, 2016 at 2:34 AM, BartC wrote: > On 17/02/2016 19:49, wrong.address.1 at gmail.com wrote: > >> Could someone kindly tell me advantages and disadvantages of Python? > > > Something I don't think anyone has mentioned is that Python is > case-sensitive. > > So keywords, identifiers, library functions and so on have to be written > just right. (For example the identifier "abcdef" can be written 64 different > ways, such as "aBCdEf", and they will all be different symbols.) > > This is a complete pain if you're used to a language that ignores case > (except in data). But pretty much every language now is the same, you can't > do much about it, you just need to be aware of it. Indeed. Most languages are case sensitive, because it's too risky to do anything else. Python 3 allows you to use other scripts than the basic Latin letters in the ASCII set: >>> ?????? = "hello" and of course you wouldn't expect that to collide with something written in a completely different script. As far as the interpreter's concerned, "P" and "p" are just as different as are "A", "?", "?", and so on. Like with Fortran's peculiar freedom in whitespace, case insensitivity in identifiers is an unwanted relic of a past age. ChrisA From invalid at invalid.invalid Fri Feb 19 10:59:30 2016 From: invalid at invalid.invalid (Grant Edwards) Date: Fri, 19 Feb 2016 15:59:30 +0000 (UTC) Subject: Considering migrating to Python from Visual Basic 6 for engineering applications References: <90cc50d2-1ce5-4588-9bfd-a49d439f00dd@googlegroups.com> Message-ID: On 2016-02-19, BartC wrote: > On 17/02/2016 19:49, wrong.address.1 at gmail.com wrote: > >> Could someone kindly tell me advantages and disadvantages of Python? > > Something I don't think anyone has mentioned is that Python is > case-sensitive. That's an _advantage_ not a disadvantage. Anybody who writes code that depends on the assumption that FOOBAR foobar and FooBar are identical deserves to have his keyboard explode under his fingertips. > So keywords, identifiers, library functions and so on have to be > written just right. IOW, you're expected to do things correctly rather that have the compiler try to guess the intent of your inept typing. [OK, I may be a bit touchy on this subject from dealing with code written by people used to working on Windows where they assume that file names are case insensitive, so therefore seem to feel the need to spice up life by using a variety of spellings for the same damned file.] -- Grant Edwards grant.b.edwards Yow! I would like to at urinate in an OVULAR, gmail.com porcelain pool -- From invalid at invalid.invalid Fri Feb 19 11:04:24 2016 From: invalid at invalid.invalid (Grant Edwards) Date: Fri, 19 Feb 2016 16:04:24 +0000 (UTC) Subject: Considering migrating to Python from Visual Basic 6 for engineering applications References: <90cc50d2-1ce5-4588-9bfd-a49d439f00dd@googlegroups.com> Message-ID: On 2016-02-19, Chris Angelico wrote: > and of course you wouldn't expect that to collide with something > written in a completely different script. As far as the interpreter's > concerned, "P" and "p" are just as different as are "A", "?", "?", and > [...] ...and a, ?, ?, ?, ?, ?, ?, ?, ?, etc. [Just wanted to see if I remembered how to do that...] -- Grant Edwards grant.b.edwards Yow! I like your SNOOPY at POSTER!! gmail.com From bc at freeuk.com Fri Feb 19 11:32:45 2016 From: bc at freeuk.com (BartC) Date: Fri, 19 Feb 2016 16:32:45 +0000 Subject: Considering migrating to Python from Visual Basic 6 for engineering applications In-Reply-To: References: <90cc50d2-1ce5-4588-9bfd-a49d439f00dd@googlegroups.com> Message-ID: On 19/02/2016 15:59, Grant Edwards wrote: > On 2016-02-19, BartC wrote: >> On 17/02/2016 19:49, wrong.address.1 at gmail.com wrote: >> >>> Could someone kindly tell me advantages and disadvantages of Python? >> >> Something I don't think anyone has mentioned is that Python is >> case-sensitive. > > That's an _advantage_ not a disadvantage. > > Anybody who writes code that depends on the assumption that FOOBAR > foobar and FooBar are identical deserves to have his keyboard explode > under his fingertips. > >> So keywords, identifiers, library functions and so on have to be >> written just right. > > IOW, you're expected to do things correctly You mean pedantically. In real life, names generally are not case sensitive. I can call myself bart or Bart or BART or any of the remaining 13 combinations, without anyone getting confused (but they might be puzzled as to why I'd choose to spell it bArT). And in speech there is no distinction between case (so anyone using voice-to-text is going to have trouble with writing code). Even in computing, many kinds of names are case-insensitive, emails and website names for example. I think even MS would struggle to register all the 32768 upper and lower case combinations of www dot microsoft dot com. It becomes nonsensical. It's just a popular fad in programming languages, probably originating in C and/or Unix, and doing a good job of infesting everything else. > rather that have the > compiler try to guess the intent of your inept typing. That's not being very helpful then. Do I type help, Help or HELP? Or would it be a better idea to recognise all of them, instead of stubbornly refusing to work until I get it exactly right? > [OK, I may be a bit touchy on this subject from dealing with code > written by people used to working on Windows where they assume that > file names are case insensitive, so therefore seem to feel the need to > spice up life by using a variety of spellings for the same damned > file.] But they're all the same file? If I had to give you the name of a file over the phone, wouldn't it make life much easier if I didn't have to describe, letter by letter, which was upper case and which was lower? [Most of the code I write uses lower-case anyway, but I also like to write temporary or debugging code in upper-case. That won't work in a language like Python. But it's also a nuisance in remembering if it's tkinter or Tkinter. Or are there actually two different packages differing only in the case of one letter of their names?] -- Bartc From rosuav at gmail.com Fri Feb 19 11:49:44 2016 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 20 Feb 2016 03:49:44 +1100 Subject: Considering migrating to Python from Visual Basic 6 for engineering applications In-Reply-To: References: <90cc50d2-1ce5-4588-9bfd-a49d439f00dd@googlegroups.com> Message-ID: On Sat, Feb 20, 2016 at 3:32 AM, BartC wrote: > In real life, names generally are not case sensitive. I can call myself bart > or Bart or BART or any of the remaining 13 combinations, without anyone > getting confused (but they might be puzzled as to why I'd choose to spell it > bArT). There are other conflations, too. You might transliterate your name into another script, or represent it using only sounds found in a particular language, to introduce yourself in a completely different culture. Would you expect a computer to recognize all those as identical, too? Should the computer treat "Dad" and "John" as the same variable, since my father's name happens to be John? (Or should it treat them as distinct, because the _computer's_ father wasn't named John?) > And in speech there is no distinction between case (so anyone using > voice-to-text is going to have trouble with writing code). That's true of all sorts of other spoken parallels too, though. If you're using speech-to-text to enter code, you will have to cope with those, and it's no extra complexity to cope with mixed case. > Even in computing, many kinds of names are case-insensitive, emails and > website names for example. I think even MS would struggle to register all > the 32768 upper and lower case combinations of www dot microsoft dot com. It > becomes nonsensical. DNS is a bit of a special case. It has a bizarre mess of ASCII-only and full-Unicode naming, such that it really shouldn't be held up as a model for programming language identifiers. > It's just a popular fad in programming languages, probably originating in C > and/or Unix, and doing a good job of infesting everything else. Go ask a German how ? should be capitalized. Then ask if it's identical to "ss", which also uppercases to "SS". Now, how do you feel about that whole case sensitivity thing? And is "?" equivalent to "O" or to "OE" or neither? (Answer: It depends on context.) Everyone I've heard arguing in favour of programming language case insensitivity has assumed ASCII-only (and English-only). It's 2016. Time to stop making that assumption. ChrisA From phamp at mindspring.com Fri Feb 19 12:04:08 2016 From: phamp at mindspring.com (pyotr filipivich) Date: Fri, 19 Feb 2016 09:04:08 -0800 Subject: extending PATH on Windows? References: Message-ID: Ulli Horlacher on Fri, 19 Feb 2016 10:42:52 +0000 (UTC) typed in comp.lang.python the following: >pyotr filipivich wrote: > >> > Windows (especially 7) search function is highly crippled. There is >> >some command sequence that will open it up to looking at other file types >> >and locations. >> > >> >http://answers.microsoft.com/en-us/windows/forum/windows_7-files/windows-7-search-does-not-find-files-that-it/61b88d5e-7df7-4427-8a2e-82b801a4a746?auth=1 >> >> >> Thanks. I've found it "simpler" to just open a command prompt, and >> use DOS. > >How can one search for files with DOS? cd %searchPathStart% dir /s *.php or variants. -- pyotr filipivich Next month's Panel: Graft - Boon or blessing? From m at funkyhat.org Fri Feb 19 12:05:48 2016 From: m at funkyhat.org (Matt Wheeler) Date: Fri, 19 Feb 2016 17:05:48 +0000 Subject: Considering migrating to Python from Visual Basic 6 for engineering applications In-Reply-To: References: <90cc50d2-1ce5-4588-9bfd-a49d439f00dd@googlegroups.com> <14c75a68-0d2e-45cc-8d73-0d71b6a6aea6@googlegroups.com> <9e57761f-26e1-41c5-8e71-23800de1fdd3@googlegroups.com> <20160219074057.16b9eddb@bigbox.christie.dr> Message-ID: On 19 February 2016 at 14:58, Peter Otten <__peter__ at web.de> wrote: > Tim Chase wrote: >> [a long thing] > > Or just tell the parser what to expect: > > [another long thing] If you're sure all of the words on a line are going to be numbers then >>> [float(x) for x in '2 12.657823 0.1823467E-04 114 0'.split()] [2.0, 12.657823, 1.823467e-05, 114.0, 0.0] This will of course break if anything on the line *doesn't* look like a float, but rather than jumping straight into a giant engineering exercise I'd probably start there. I'm sure there was a recent thread about returning the best fit type (i.e. int, if not then float, if not then str)? -- Matt Wheeler http://funkyh.at From jcasale at activenetwerx.com Fri Feb 19 12:18:55 2016 From: jcasale at activenetwerx.com (Joseph L. Casale) Date: Fri, 19 Feb 2016 17:18:55 +0000 Subject: Passing data across callbacks in ThreadPoolExecutor In-Reply-To: References: , Message-ID: <63d71b83fd9c43868f6a97ecaa6af31c@activenetwerx.com> > It's still not clear to me specifically what you're trying to do. It > would really help if you would describe the problem in more detail. > Here's what I think you're trying to do: > > 1) Submit a task to a ThreadPoolExecutor and get back a future. > > 2) When the task is complete, submit another task that needs the > result of the first task to do its work. > > 3) When the chained task is complete, return the final result of the > chained task back to whoever submitted the original task via the > original task's future. > > The problem arises in that the original task's future already > completed when the original task did. The chained task sets its result > in a different future that the submitter didn't know about. Yes, I may have 2 or more tasks that depend on the previous. They are each distinct tasks, or continuations, so they each want the result of the previous task however each one can cancel the set. The callback model doesn't apply, its based on the result of _one_ task. What I need I are continuations, much like the .NET TPL. def task_a(): return "a" def task_b(): return "b" def task_c(): return "c" So I submit task_a, if it completes successfully, task_b runs on its result. If task_b completes successfully, task_c runs on its result. They "look" like callbacks, but they are continuations. Task_b must be able to cancel the continuation of task_c if it see's task_a has failed. Thanks for the continued help! jlc From invalid at invalid.invalid Fri Feb 19 13:03:34 2016 From: invalid at invalid.invalid (Grant Edwards) Date: Fri, 19 Feb 2016 18:03:34 +0000 (UTC) Subject: Considering migrating to Python from Visual Basic 6 for engineering applications References: <90cc50d2-1ce5-4588-9bfd-a49d439f00dd@googlegroups.com> Message-ID: On 2016-02-19, BartC wrote: > >> IOW, you're expected to do things correctly > > You mean pedantically. :) > In real life, names generally are not case sensitive. I can call > myself bart or Bart or BART or any of the remaining 13 combinations, > without anyone getting confused (but they might be puzzled as to why > I'd choose to spell it bArT). You probably answer to half-a-dozen others things as well. Such natural-language concepts just don't work in code. > And in speech there is no distinction between case (so anyone using > voice-to-text is going to have trouble with writing code). That's a good point. > Even in computing, many kinds of names are case-insensitive, emails and > website names for example. I think even MS would struggle to register > all the 32768 upper and lower case combinations of www dot microsoft dot > com. It becomes nonsensical. True. >> [OK, I may be a bit touchy on this subject from dealing with code >> written by people used to working on Windows where they assume that >> file names are case insensitive, so therefore seem to feel the need to >> spice up life by using a variety of spellings for the same damned >> file.] > > But they're all the same file? Yes. Sometimes three or four different spellings scattered over multiple domains (Makefile, C source (e.g. #include directives), and the filesystem). -- Grant Edwards grant.b.edwards Yow! I'd like some JUNK at FOOD ... and then I want to gmail.com be ALONE -- From eryksun at gmail.com Fri Feb 19 13:04:29 2016 From: eryksun at gmail.com (eryk sun) Date: Fri, 19 Feb 2016 12:04:29 -0600 Subject: extending PATH on Windows? In-Reply-To: References: Message-ID: On Fri, Feb 19, 2016 at 4:48 AM, Chris Angelico wrote: > On Fri, Feb 19, 2016 at 9:42 PM, Ulli Horlacher > wrote: >> pyotr filipivich wrote: >> >>> > Windows (especially 7) search function is highly crippled. There is >>> >some command sequence that will open it up to looking at other file types >>> >and locations. >>> > >>> >http://answers.microsoft.com/en-us/windows/forum/windows_7-files/windows-7-search-does-not-find-files-that-it/61b88d5e-7df7-4427-8a2e-82b801a4a746?auth=1 >>> >>> >>> Thanks. I've found it "simpler" to just open a command prompt, and >>> use DOS. >> >> How can one search for files with DOS? > > dir /s /b \*add2path.* This search is carried out by Windows NT instead of MS-DOS in almost all cases nowadays: Z:\Python\repo\2.7>dir /s /b Tools\Scripts\win_add?*.* Breakpoint 0 hit ntdll!NtQueryDirectoryFile: 00007ffb`d9d138c0 4c8bd1 mov r10,rcx 0:000> dS poi(@rsp + a*8) 000000df`bab1ff9a "win_add><"*" NT has normal (i.e. sane) * and ? wildcard matching, so WinAPI FindFirstFile has to translate DOS wildcards to special values before calling NtQueryDirectoryFile. See the file-system runtime library function FsRtlIsNameInExpression [1]. Dot, question mark, and asterisk may be mapped (depending on context in the string) to double quote and the angle brackets, as shown in the above debugger output in which 'win_add?*.*' is passed to the system call as 'win_add><"*'. DOS_DOT (") Matches either a period or zero characters beyond the name string. DOS_QM (>) Matches any single character or, upon encountering a period or end of name string, advances the expression to the end of the set of contiguous DOS_QMs. DOS_STAR (<) Matches zero or more characters until encountering and matching the final . in the name. [1]: https://msdn.microsoft.com/en-us/library/ff546850 From eryksun at gmail.com Fri Feb 19 13:07:24 2016 From: eryksun at gmail.com (eryk sun) Date: Fri, 19 Feb 2016 12:07:24 -0600 Subject: extending PATH on Windows? In-Reply-To: References: Message-ID: On Thu, Feb 18, 2016 at 10:55 AM, Ulli Horlacher wrote: > Ulli Horlacher wrote: > >> > but simpler still and more reliable to just call QueryValueEx. >> >> I find it more complicated. > > I have now (after long studying docs and examples):: > > def get_winreg(key,subkey): > try: > rkey = winreg.OpenKey(winreg.HKEY_CURRENT_USER,key,0,winreg.KEY_READ) > rvalue,rtype = winreg.QueryValueEx(rkey,subkey) > winreg.CloseKey(rkey) > except WindowsError: > rvalue,rtype = None,None > return rvalue The 2nd parameter should be named "value_name", not "subkey". A registry key is similar to an NTFS file-system node, and values are similar to NTFS $DATA streams [1], but with different types of data [2] other than just REG_BINARY, to make it easier to store structured configuration data. [1]: https://msdn.microsoft.com/en-us/library/aa364404 [2]: https://msdn.microsoft.com/en-us/library/ms724884 From shaunak.bangale at gmail.com Fri Feb 19 13:07:41 2016 From: shaunak.bangale at gmail.com (shaunak.bangale at gmail.com) Date: Fri, 19 Feb 2016 10:07:41 -0800 (PST) Subject: Exception handling for socket.error in Python 3.5/RStudio In-Reply-To: <0e2fc62a-a4b8-4962-8737-b9cfdbafefc4@googlegroups.com> References: <1067e50c-bcbd-464e-8013-754bd66fda91@googlegroups.com> <0e2fc62a-a4b8-4962-8737-b9cfdbafefc4@googlegroups.com> Message-ID: On Monday, February 8, 2016 at 1:05:08 AM UTC-7, shaunak... at gmail.com wrote: > On Sunday, February 7, 2016 at 1:23:32 AM UTC-7, dieter wrote: > > Shaunak Bangale writes: > > > ... > > > while (count > 0): > > > try : > > > # read line from file: > > > print(file.readline()) > > > # parse > > > parse_json(file.readline()) > > > count = count - 1 > > > except socket.error as e: > > > print('Connection fail', e) > > > print(traceback.format_exc()) > > > ... > > > Error: > > > except socket.error as e: > > > ^ > > > SyntaxError: invalid syntax > > > > Are you sure, that there is no invisible character at the end > > of that line? > > > > When I try code like the above (in PYthon 2.7), there is no > > "SyntaxError": > > > > >>> while(False): > > ... try : > > ... # read line from file: > > ... print(file.readline()) > > ... # parse > > ... parse_json(file.readline()) > > ... count = count - 1 > > ... except socket.error as e: > > ... print('Connection fail', e) > > ... print(traceback.format_exc()) > > ... > > >>> > > > > > > Another reason for your problem could be an older Python version. > > The "as" construct as part of the "except" clause is a more recent > > addition to Python. > > Hi Dieter, > > I typed that line again but didn't help. I am using Python 3.5. ------------- Hi people, it finally worked. One problem was with the way modules were set up, so installed anacoda and using accessing it through PyCharm. It's working well now. Another problem was with the initiation command, had to renew the password and that's working too. I sincerely thank everyone here who tried to help me. :) Have a great weekend guys! -Shaun From jrmy.lnrd at gmail.com Fri Feb 19 13:13:58 2016 From: jrmy.lnrd at gmail.com (Jeremy Leonard) Date: Fri, 19 Feb 2016 10:13:58 -0800 (PST) Subject: Processing multiple forms in a cgi script Message-ID: <16807edd-4376-4a36-82a7-3bcff8e86a0a@googlegroups.com> I have a quick question regarding processing multiple forms. Most of my experience has been where there is just one form so the logic was pretty straight forward for me. My question revolves around how to handle multiple forms. I've seen that you can have an id for each form, but I'm unsure how to process that. Any info or a point/kick in the right direction would be appreciated. From wrong.address.1 at gmail.com Fri Feb 19 13:14:29 2016 From: wrong.address.1 at gmail.com (wrong.address.1 at gmail.com) Date: Fri, 19 Feb 2016 10:14:29 -0800 (PST) Subject: Considering migrating to Python from Visual Basic 6 for engineering applications In-Reply-To: References: <90cc50d2-1ce5-4588-9bfd-a49d439f00dd@googlegroups.com> <14c75a68-0d2e-45cc-8d73-0d71b6a6aea6@googlegroups.com> <9e57761f-26e1-41c5-8e71-23800de1fdd3@googlegroups.com> Message-ID: <7f9c473e-b0c2-4d77-91d1-d0733c93b12d@googlegroups.com> On Friday, 19 February 2016 16:08:46 UTC+2, Tim Chase wrote: > On 2016-02-19 02:47, wrong.address.1 at gmail.com wrote: > > 2 12.657823 0.1823467E-04 114 0 > > 3 4 5 9 11 > > "Lower" > > 278.15 > > > > Is it straightforward to read this, or does one have to read one > > character at a time and then figure out what the numbers are? -- > > It's easy to read. What you do with that mess of data is the complex > part. They come in as byte-strings, but you'd have to convert them > to the corresponding formats: > > from shlex import shlex > USE_LEX = True # False > with open('data.txt') as f: > for i, line in enumerate(f, 1): > if USE_LEX: > bits = shlex(line) > else: > bits = line.split() > for j, bit in enumerate(bits, 1): > if bit.isdigit(): > result = int(bit) > t = "an int" > elif '"' in bit: > result = bit > t = "a string" > else: > result = float(bit) > t = "a float" > print("On line %i I think that item %i %r is %s: %r" % ( > i, > j, > bit, > t, > result, > )) > All this I could do with one Read or Input statement in Fortran and Basic. > The USE_LEX controls whether the example code uses string-splitting > on white-space, or uses the built-in "shlex" module to parse for > quoted strings that might contain a space. The naive way of > string-splitting will be faster, but choke on string-data containing > spaces. > > You'd have to make up your own heuristics for determining what type > each data "bit" is, parsing it out (with int(), float() or whatever), > but the above gives you some rough ideas with at least one known > bug/edge-case. But we can't do *all* the work for you ;-) > > -tkc This is precisely reading one character at a time. If not exactly reading one character, it is effectively looking at each character to assemble the number. Not a good sign. I guess there might be libraries which will help read numbers better, but I would expect a good language to be able to handle this basic thing for engineers - to read numbers like Fortran and Basic do. Still, if I have misunderstood something, I will be glad to be corrected. I would generally know that the first three numbers will be floating point, the next will be something and then the next will be something else, etc. Does that help or does one still have to look at each character and determine how to place it in a number? Thanks for your patience. I don't want surprises later on, which is why I am asking very stupid questions. From jon+usenet at unequivocal.co.uk Fri Feb 19 13:15:33 2016 From: jon+usenet at unequivocal.co.uk (Jon Ribbens) Date: Fri, 19 Feb 2016 18:15:33 -0000 (UTC) Subject: Processing multiple forms in a cgi script References: <16807edd-4376-4a36-82a7-3bcff8e86a0a@googlegroups.com> Message-ID: On 2016-02-19, Jeremy Leonard wrote: > I have a quick question regarding processing multiple forms. Most of > my experience has been where there is just one form so the logic was > pretty straight forward for me. My question revolves around how to > handle multiple forms. I've seen that you can have an id for each > form, but I'm unsure how to process that. Any info or a point/kick > in the right direction would be appreciated. Only one form can be submitted at once. To identify in your code which one it was, either make the "action" attribute for each form point to different URLs, or put a unique "name" attribute on each submit button, or put an in each form with a unique name/value. From invalid at invalid.invalid Fri Feb 19 13:29:47 2016 From: invalid at invalid.invalid (Grant Edwards) Date: Fri, 19 Feb 2016 18:29:47 +0000 (UTC) Subject: Processing multiple forms in a cgi script References: <16807edd-4376-4a36-82a7-3bcff8e86a0a@googlegroups.com> Message-ID: On 2016-02-19, Jon Ribbens wrote: > On 2016-02-19, Jeremy Leonard wrote: > >> I have a quick question regarding processing multiple forms. Most >> of my experience has been where there is just one form so the logic >> was pretty straight forward for me. My question revolves around how >> to handle multiple forms. I've seen that you can have an id for >> each form, but I'm unsure how to process that. Any info or a >> point/kick in the right direction would be appreciated. > > Only one form can be submitted at once. Unless you do something rather obtuse like using Javascript to serialize multiple HTML forms into separate JSON objects and then combine those into a single JSON object and then submit that via an AJAX call. [You could use XML instead of JSON, but that'd be even worse.] I've no clue _why_ somebody would go to the trouble to do something like that -- if you know how to stuff like that, then you probably know better than to actually do it. ;) If the data on some set of multiple forms is always going to be submitted together, then just use a single form instead. If you want to handle multiple different forms _one_at_a_time_ using a single form handler, then just put a hidden input field on each form that identifies which one it is. -- Grant Edwards grant.b.edwards Yow! Are you selling NYLON at OIL WELLS?? If so, we can gmail.com use TWO DOZEN!! From ian.g.kelly at gmail.com Fri Feb 19 13:32:27 2016 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Fri, 19 Feb 2016 11:32:27 -0700 Subject: Passing data across callbacks in ThreadPoolExecutor In-Reply-To: <63d71b83fd9c43868f6a97ecaa6af31c@activenetwerx.com> References: <63d71b83fd9c43868f6a97ecaa6af31c@activenetwerx.com> Message-ID: On Fri, Feb 19, 2016 at 10:18 AM, Joseph L. Casale wrote: >> It's still not clear to me specifically what you're trying to do. It >> would really help if you would describe the problem in more detail. >> Here's what I think you're trying to do: >> >> 1) Submit a task to a ThreadPoolExecutor and get back a future. >> >> 2) When the task is complete, submit another task that needs the >> result of the first task to do its work. >> >> 3) When the chained task is complete, return the final result of the >> chained task back to whoever submitted the original task via the >> original task's future. >> >> The problem arises in that the original task's future already >> completed when the original task did. The chained task sets its result >> in a different future that the submitter didn't know about. > > Yes, I may have 2 or more tasks that depend on the previous. > They are each distinct tasks, or continuations, so they each want the result > of the previous task however each one can cancel the set. > > The callback model doesn't apply, its based on the result of _one_ task. > > What I need I are continuations, much like the .NET TPL. > > def task_a(): > return "a" > > def task_b(): > return "b" > > def task_c(): > return "c" > > So I submit task_a, if it completes successfully, task_b runs on its result. > If task_b completes successfully, task_c runs on its result. They "look" > like callbacks, but they are continuations. > > Task_b must be able to cancel the continuation of task_c if it see's task_a > has failed. Thanks for the clarification. The first suggestion that I gave in my initial reply will handle cancellations implicitly. If task_b() is waiting on future_a.result() and task_a() raises an uncaught exception, then future_a.result() will reraise the exception. If it then propagates uncaught out of task_b() then future_b.result() will again reraise the exception, and task_c() that is waiting on it will reraise it as well. See this in action below. You might also be interested in the promise decorator described at https://bloerg.net/2013/04/05/chaining-python-futures.html. It does essentially the same as what I've described, but it abstracts it into a decorator and allows the caller to determine which arguments to pass as futures and which to pass as concrete values, instead of hard-coding the resolution of futures of dependencies into the task function itself. In the course of digging that up, I also found this: https://github.com/dvdotsenko/python-future-then which is brand-new and looks promising, but I can't see how one would use it with a ThreadPoolExecutor. py> import concurrent.futures py> def task_a(): ... raise ValueError(42) ... py> def task_b(future_a): ... return future_a.result() ... py> def task_c(future_b): ... return future_b.result() ... py> executor = concurrent.futures.ThreadPoolExecutor() py> future_a = executor.submit(task_a) py> future_b = executor.submit(task_b, future_a) py> future_c = executor.submit(task_c, future_b) py> future_c.result() Traceback (most recent call last): File "", line 1, in File "/usr/local/lib/python3.5/concurrent/futures/_base.py", line 398, in result return self.__get_result() File "/usr/local/lib/python3.5/concurrent/futures/_base.py", line 357, in __get_result raise self._exception File "/usr/local/lib/python3.5/concurrent/futures/thread.py", line 55, in run result = self.fn(*self.args, **self.kwargs) File "", line 2, in task_c File "/usr/local/lib/python3.5/concurrent/futures/_base.py", line 398, in result return self.__get_result() File "/usr/local/lib/python3.5/concurrent/futures/_base.py", line 357, in __get_result raise self._exception File "/usr/local/lib/python3.5/concurrent/futures/thread.py", line 55, in run result = self.fn(*self.args, **self.kwargs) File "", line 2, in task_b File "/usr/local/lib/python3.5/concurrent/futures/_base.py", line 398, in result return self.__get_result() File "/usr/local/lib/python3.5/concurrent/futures/_base.py", line 357, in __get_result raise self._exception File "/usr/local/lib/python3.5/concurrent/futures/thread.py", line 55, in run result = self.fn(*self.args, **self.kwargs) File "", line 2, in task_a ValueError: 42 From drsalists at gmail.com Fri Feb 19 13:33:40 2016 From: drsalists at gmail.com (Dan Stromberg) Date: Fri, 19 Feb 2016 10:33:40 -0800 Subject: Weird python 2.7 import thing In-Reply-To: References: Message-ID: On Thu, Feb 18, 2016 at 5:40 PM, Dan Stromberg wrote: > > BTW, this is not a package I pip installed from pypi - it's an > internal-only project. I need to correct this: It's not installed from pypi.python.org, but it probably is installed from our internal pypi server. Thanks! From jrmy.lnrd at gmail.com Fri Feb 19 13:38:53 2016 From: jrmy.lnrd at gmail.com (Jeremy Leonard) Date: Fri, 19 Feb 2016 10:38:53 -0800 (PST) Subject: Processing multiple forms in a cgi script In-Reply-To: References: <16807edd-4376-4a36-82a7-3bcff8e86a0a@googlegroups.com> Message-ID: <54ef2615-15a9-406e-8257-d63faac177fb@googlegroups.com> On Friday, February 19, 2016 at 1:18:41 PM UTC-5, Jon Ribbens wrote: > On 2016-02-19, Jeremy Leonard wrote: > > I have a quick question regarding processing multiple forms. Most of > > my experience has been where there is just one form so the logic was > > pretty straight forward for me. My question revolves around how to > > handle multiple forms. I've seen that you can have an id for each > > form, but I'm unsure how to process that. Any info or a point/kick > > in the right direction would be appreciated. > > Only one form can be submitted at once. To identify in your code > which one it was, either make the "action" attribute for each form > point to different URLs, or put a unique "name" attribute on each > submit button, or put an in each form with > a unique name/value. Thank you. That helps. The hidden input will be perfect. From eryksun at gmail.com Fri Feb 19 13:42:41 2016 From: eryksun at gmail.com (eryk sun) Date: Fri, 19 Feb 2016 12:42:41 -0600 Subject: extending PATH on Windows? In-Reply-To: <56C72420.8020607@yahoo.no> References: <56C72420.8020607@yahoo.no> Message-ID: On Fri, Feb 19, 2016 at 8:18 AM, Gisle Vanem wrote: > Dennis Lee Bieber wrote: > >>>> How can one search for files with DOS? >>> >>> dir /s /b \*add2path.* >>> >>> ChrisA >> >> Or move to PowerShell... >> >> Windows PowerShell >> Copyright (C) 2009 Microsoft Corporation. All rights reserved. >> >> PS C:\Users\Wulfraed\Documents> Get-ChildItem -Path c:\ -Filter *add2path* >> -Name -Recurse > > Clumsy IMHO. Try: > c:\> envtool --path *ipython* > > Matches in %PATH: Your tool has special integration with Python's search paths, which is cool. But for someone that wants some of these features without installing another program and without learning PowerShell, where.exe may suffice: https://technet.microsoft.com/en-us/library/cc753148 Search PATH and the current directory: C:\>where kd C:\Program Files (x86)\Windows Kits\10\Debuggers\x64\kd.exe Search an environment variable that may contain one or more directories: C:\>set MYPATH=^ More? %SystemRoot%\System32;^ More? %SystemRoot%\System32\drivers C:\>where $MYPATH:conhost*.* C:\Windows\System32\conhost.exe C:\Windows\System32\ConhostV1.dll C:\Windows\System32\ConhostV2.dll Search a list of directories: C:\>where "%MYPATH%":con*.sys C:\Windows\System32\drivers\condrv.sys Recursively search a directory: C:\>where /r "%LocalAppData%\Programs" python.exe C:\Users\Administrator\AppData\Local\Programs\Python\ Python35-32\python.exe From jenn.duerr at gmail.com Fri Feb 19 13:46:49 2016 From: jenn.duerr at gmail.com (noydb) Date: Fri, 19 Feb 2016 10:46:49 -0800 (PST) Subject: downloading a CSV Message-ID: <1efa02db-a26a-44bd-a0ea-db81e612ccd6@googlegroups.com> Greetings All, Python v 3.4, windows I want to be able to download this CSV file and save to disk >> http://earthquake.usgs.gov/earthquakes/feed/v1.0/summary/all_month.csv This (interacting with the web using python) is very new to me, so can anyone provide direction on how to go about doing this? Is it involved? Is there a good example out there that can illustrate the needs/mechanics/how-to? Many thanks in advance! From python at mrabarnett.plus.com Fri Feb 19 14:24:33 2016 From: python at mrabarnett.plus.com (MRAB) Date: Fri, 19 Feb 2016 19:24:33 +0000 Subject: downloading a CSV In-Reply-To: <1efa02db-a26a-44bd-a0ea-db81e612ccd6@googlegroups.com> References: <1efa02db-a26a-44bd-a0ea-db81e612ccd6@googlegroups.com> Message-ID: <56C76BF1.9090609@mrabarnett.plus.com> On 2016-02-19 18:46, noydb wrote: > Greetings All, > > Python v 3.4, windows > > I want to be able to download this CSV file and save to disk >>> http://earthquake.usgs.gov/earthquakes/feed/v1.0/summary/all_month.csv > > This (interacting with the web using python) is very new to me, so can anyone provide direction on how to go about doing this? Is it involved? Is there a good example out there that can illustrate the needs/mechanics/how-to? > > Many thanks in advance! > Lookup "urlretrieve" in the help file. From invalid at invalid.invalid Fri Feb 19 14:43:01 2016 From: invalid at invalid.invalid (Grant Edwards) Date: Fri, 19 Feb 2016 19:43:01 +0000 (UTC) Subject: downloading a CSV References: <1efa02db-a26a-44bd-a0ea-db81e612ccd6@googlegroups.com> Message-ID: On 2016-02-19, noydb wrote: > Greetings All, > > Python v 3.4, windows > > I want to be able to download this CSV file and save to disk >>> http://earthquake.usgs.gov/earthquakes/feed/v1.0/summary/all_month.csv Unless you really want to write a Python program, "wget" is a good solution: http://gnuwin32.sourceforge.net/packages/wget.htm Just do this on a command line: wget http://earthquake.usgs.gov/earthquakes/feed/v1.0/summary/all_month.csv Curl is another good option: https://curl.haxx.se/ Just do this at the command line: curl http://earthquake.usgs.gov/earthquakes/feed/v1.0/summary/all_month.csv > all_month.csv -- Grant Edwards grant.b.edwards Yow! I just had my entire at INTESTINAL TRACT coated gmail.com with TEFLON! From drsalists at gmail.com Fri Feb 19 14:54:47 2016 From: drsalists at gmail.com (Dan Stromberg) Date: Fri, 19 Feb 2016 11:54:47 -0800 Subject: Weird python 2.7 import thing In-Reply-To: References: Message-ID: On Fri, Feb 19, 2016 at 10:33 AM, Dan Stromberg wrote: > On Thu, Feb 18, 2016 at 5:40 PM, Dan Stromberg wrote: > >> >> BTW, this is not a package I pip installed from pypi - it's an >> internal-only project. > > I need to correct this: It's not installed from pypi.python.org, but > it probably is installed from our internal pypi server. > > Thanks! I've taken this to Stackoverflow: http://stackoverflow.com/questions/35512989/weird-cpython-2-7-import-traceback-not-an-importerror Thanks again. From jenn.duerr at gmail.com Fri Feb 19 15:05:13 2016 From: jenn.duerr at gmail.com (noydb) Date: Fri, 19 Feb 2016 12:05:13 -0800 (PST) Subject: downloading a CSV In-Reply-To: <1efa02db-a26a-44bd-a0ea-db81e612ccd6@googlegroups.com> References: <1efa02db-a26a-44bd-a0ea-db81e612ccd6@googlegroups.com> Message-ID: <7abe9082-2720-439e-b592-432cf842874b@googlegroups.com> Thanks! That was pretty easy. import urllib.request url = 'http://earthquake.usgs.gov/earthquakes/feed/v1.0/summary/all_month.csv' urllib.request.urlretrieve(url, csv_file) csv_file = r"C:\Temp\earthquakeAll_last30days.csv" urllib.request.urlretrieve(url, csv_file) I do want to use python -- there's much more I need to do with the file after downloading. From joel.goldstick at gmail.com Fri Feb 19 16:03:23 2016 From: joel.goldstick at gmail.com (Joel Goldstick) Date: Fri, 19 Feb 2016 16:03:23 -0500 Subject: downloading a CSV In-Reply-To: <7abe9082-2720-439e-b592-432cf842874b@googlegroups.com> References: <1efa02db-a26a-44bd-a0ea-db81e612ccd6@googlegroups.com> <7abe9082-2720-439e-b592-432cf842874b@googlegroups.com> Message-ID: On Fri, Feb 19, 2016 at 3:05 PM, noydb wrote: > Thanks! That was pretty easy. > > import urllib.request > url = ' > http://earthquake.usgs.gov/earthquakes/feed/v1.0/summary/all_month.csv' > urllib.request.urlretrieve(url, csv_file) > csv_file = r"C:\Temp\earthquakeAll_last30days.csv" > urllib.request.urlretrieve(url, csv_file) > > I do want to use python -- there's much more I need to do with the file > after downloading. > -- > https://mail.python.org/mailman/listinfo/python-list > You might look at the Requests library. Its third party, but easier to understand and use than urllib -- Joel Goldstick http://joelgoldstick.com/ http://cc-baseballstats.info/ From bc at freeuk.com Fri Feb 19 18:06:07 2016 From: bc at freeuk.com (BartC) Date: Fri, 19 Feb 2016 23:06:07 +0000 Subject: Considering migrating to Python from Visual Basic 6 for engineering applications In-Reply-To: <7f9c473e-b0c2-4d77-91d1-d0733c93b12d@googlegroups.com> References: <90cc50d2-1ce5-4588-9bfd-a49d439f00dd@googlegroups.com> <14c75a68-0d2e-45cc-8d73-0d71b6a6aea6@googlegroups.com> <9e57761f-26e1-41c5-8e71-23800de1fdd3@googlegroups.com> <7f9c473e-b0c2-4d77-91d1-d0733c93b12d@googlegroups.com> Message-ID: On 19/02/2016 18:14, wrong.address.1 at gmail.com wrote: > On Friday, 19 February 2016 16:08:46 UTC+2, Tim Chase wrote: > All this I could do with one Read or Input statement in Fortran and Basic. I agree with you completely. This stuff is far more complicated than it need be. And the fact that there a hundred possible ways of doing it doesn't help! It seems modern languages don't like having i/o as part of the language but prefer to have function libraries deal with it. And it still seems to be largely DIY... Although the approach used by Peter Otten doesn't seem too bad. Then you would end up writing something like this: a,b,c,d,e = readline(f, "iffii") with f being a file handle. You'd need to deal with tokens (in the implementation of readline), but not individual characters of each number. (Remember that Fortran and VB6 are be statically typed, so the language knows what types to read into each of those variables. And the Fortran might have a 'format' to help out, unless they've got rid of those since 1979...) -- Bartc From ian.g.kelly at gmail.com Fri Feb 19 19:24:40 2016 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Fri, 19 Feb 2016 17:24:40 -0700 Subject: How to properly override the default factory of defaultdict? In-Reply-To: References: Message-ID: On Thu, Feb 18, 2016 at 10:41 AM, Herman wrote: > From: Ben Finney >> >> you are using the inheritance hierarchy but thwarting it by not using >> ?super?. Instead:: >> >> super().__init__(self, default_factory, *a, **kw) >> >> and:: >> >> super().__getitem__(self, key) >> -- >> \ "Those who will not reason, are bigots, those who cannot, are | >> `\ fools, and those who dare not, are slaves." ??Lord? George | >> _o__) Gordon Noel Byron | >> Ben Finney > > super does not work for defaultdict. I am using python 2.7. If I use > super(defaultdict, self).__init__(default_factory, *a, **kw), I get the > error: > > super(defaultdict, self).__init__(default_factory, *a, **kw) > TypeError: 'function' object is not iterable You're using it incorrectly. If your class is named DefaultDictWithEnhancedFactory, then the super call would be: super(DefaultDictWithEnhancedFactory, self).__init__(default_factory, *a, **kw) You pass in the current class so that super can look up the next class. If you pass defaultdict instead, super will think that it's being called *by* defaultdict and call the __init__ method on its own superclass, dict, which has a different signature. defaultdict.__init__ is effectively skipped. > Look like inheriting from defaultdict is easier. I don't even have to > override the constructor as suggested by Chris Angelico above. Thanks. True, although there's a faint code smell as this technically violates the Liskov Substitution Principle; the default_factory attribute on defaultdict instances is expected to be a function of zero arguments, not one. From python.list at tim.thechases.com Fri Feb 19 19:47:59 2016 From: python.list at tim.thechases.com (Tim Chase) Date: Fri, 19 Feb 2016 18:47:59 -0600 Subject: downloading a CSV In-Reply-To: <1efa02db-a26a-44bd-a0ea-db81e612ccd6@googlegroups.com> References: <1efa02db-a26a-44bd-a0ea-db81e612ccd6@googlegroups.com> Message-ID: <20160219184759.1367298d@bigbox.christie.dr> On 2016-02-19 10:46, noydb wrote: > I want to be able to download this CSV file and save to disk > >> http://earthquake.usgs.gov/earthquakes/feed/v1.0/summary/all_month.csv from urllib.request import urlopen data = urlopen("http://earthquake.usgs.gov/earthquakes/feed/v1.0/summary/all_month.csv") > This (interacting with the web using python) is very new to me, so > can anyone provide direction on how to go about doing this? Is it > involved? Is there a good example out there that can illustrate > the needs/mechanics/how-to? You can then either iterate over the incoming data and write it to a file: with open('output.csv', 'wb') as output: output.writelines(data) or you can process it as it comes in: import csv r = csv.DictReader(line.decode('utf8') for line in data) for i, row in enumerate(r): do_something(row) if i == 0: print(repr(r)) {'rms': '', 'gap': '158.52', 'latitude': '38.3625', 'magType': 'ml', 'net': 'nn', 'horizontalError': '', 'status': 'automatic', 'depth': '3', 'nst': '5', 'magError': '', 'magSource': 'nn', 'depthError': '', 'longitude': '-118.4693', 'updated': '2016-02-20T00:17:41.069Z', 'mag': '1.1', 'type': 'earthquake', 'time': '2016-02-20T00:15:24.120Z', 'id': 'nn00532282', 'locationSource': 'nn', 'dmin': '0.128', 'magNst': '', 'place': '22km SE of Hawthorne, Nevada'} -tkc From steve at pearwood.info Fri Feb 19 20:13:12 2016 From: steve at pearwood.info (Steven D'Aprano) Date: Sat, 20 Feb 2016 12:13:12 +1100 Subject: Considering migrating to Python from Visual Basic 6 for engineering applications References: <90cc50d2-1ce5-4588-9bfd-a49d439f00dd@googlegroups.com> <56c66aa9$0$1598$c3e8da3$5496439d@news.astraweb.com> <8c67cb1c-4318-4d3b-b425-da894e5138e4@googlegroups.com> Message-ID: <56c7bda9$0$1615$c3e8da3$5496439d@news.astraweb.com> On Sat, 20 Feb 2016 12:27 am, Dennis Lee Bieber wrote: > Then the best suggestion I have would be to take a weekend and just > read the language reference manual (it used to be about an 80-page PDF > file, but has no doubt grown into some less handy dynamically linked > HTML/"help" file). Then add the section of the library reference manual > that discusses data types. And while in that manual, scan the table of > contents -- you may find there are modules in the standard library that > already do what you need. Surely you should start with the tutorial, not the reference manual. -- Steven From rosuav at gmail.com Fri Feb 19 21:19:14 2016 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 20 Feb 2016 13:19:14 +1100 Subject: How to properly override the default factory of defaultdict? In-Reply-To: References: Message-ID: On Sat, Feb 20, 2016 at 11:24 AM, Ian Kelly wrote: >> Look like inheriting from defaultdict is easier. I don't even have to >> override the constructor as suggested by Chris Angelico above. Thanks. > > True, although there's a faint code smell as this technically violates > the Liskov Substitution Principle; the default_factory attribute on > defaultdict instances is expected to be a function of zero arguments, > not one. My suggestion was a minimal change against his own code, but really, the right way to do it is to not subclass defaultdict at all - just subclass dict and define __missing__. Then there's no violation of LSP, and probably no super() call either. And the correct way to use super() is with no arguments and Python 3. Much less fiddling around, no repetition of the class name, and you're not depending on a global name lookup that might fail: Python 2.7: >>> class Foo(object): ... def method(self): print("Base method") ... >>> class Bar(Foo): ... def method(self): ... super(Bar,self).method() ... print("Derived method") ... >>> b = Bar() >>> del Bar >>> b.method() Traceback (most recent call last): File "", line 1, in File "", line 3, in method NameError: global name 'Bar' is not defined Python 3.6: >>> class Foo(object): ... def method(self): print("Base method") ... >>> class Bar(Foo): ... def method(self): ... super().method() ... print("Derived method") ... >>> b = Bar() >>> del Bar >>> b.method() Base method Derived method Yes, that's a little contrived. But imagine the mess if you use a class decorator that returns a function, or accidentally reuse a name at the interactive prompt, or something. Needing to look something up just to find your own parent seems unnecessary. (That said, though, we depend on a name lookup to perform a recursive function call, and that's a lot more likely to be rebound than a class. But the no-arg super is still better for the other reasons.) ChrisA From steve at pearwood.info Fri Feb 19 21:36:23 2016 From: steve at pearwood.info (Steven D'Aprano) Date: Sat, 20 Feb 2016 13:36:23 +1100 Subject: Guido on python3 for beginners References: <48762040-a7e0-434c-92e8-8a0969210e5b@googlegroups.com> <5d31dd14-1a60-4f80-8889-d0616f404c70@googlegroups.com> <4e902c58-3370-4adf-905d-74630a6eea90@googlegroups.com> <60036746-78f7-4f05-a6d8-04e88b09c00f@googlegroups.com> <032f0f66-34df-4325-98e0-1127025d2a94@googlegroups.com> <31a618b2-a407-4723-9a4f-9b756fc93b0b@googlegroups.com> <0a920e92-5d31-4866-815b-cae7e201e4d8@googlegroups.com> <56c568f2$0$2832$c3e8da3$76491128@news.astraweb.com> <56c66d49$0$1620$c3e8da3$5496439d@news.astraweb.com> <33b46714-8d9b-4159-ad26-4699d8b11c4d@googlegroups.com> Message-ID: <56c7d128$0$1597$c3e8da3$5496439d@news.astraweb.com> On Fri, 19 Feb 2016 02:39 pm, Rustom Mody wrote: > Consider as hypothesis > L1: Needs 4 weeks to master A language that only takes 4 weeks to master, starting from zero programming experience, clearly doesn't include much in the way of features. > L2: Needs 4 years to manage (if you want to make it less hypothetical > think C++) Do you mean 4 years to *master*? Generally speaking, it takes 10 years of experience to become a master of a complex skill. You might learn all the syntax of C++ in 4 years, but that doesn't mean you have mastered the language :-) > Which would be more satisfying to a student? To be satisfying, a language needs to have a shallow enough learning curve that the student can make good progress right from the beginning, and enough features that they can actually do interesting things. Here's a language that a student can learn in about one minute, because it only has two features: Assignment: LET x = 99 Printing: print x But you can't do anything interesting with this language, so it is not satisfying. On the other hand, here's "Hello World" in another language, one which is Turing complete so it can do anything Python or C can do: (=<`#9]~6ZY32Vx/4Rs+0No-&Jk)"Fh}|Bcy?`=*z]Kw%oG4UUS0/@-ejc(:'8dc but the learning curve is steep enough that it will be frustrating rather than interesting. https://en.wikipedia.org/wiki/Malbolge -- Steven From steve at pearwood.info Fri Feb 19 21:36:53 2016 From: steve at pearwood.info (Steven D'Aprano) Date: Sat, 20 Feb 2016 13:36:53 +1100 Subject: How the heck does async/await work in Python 3.5 References: Message-ID: <56c7d145$0$1597$c3e8da3$5496439d@news.astraweb.com> On Thu, 18 Feb 2016 09:08 am, Mark Lawrence wrote: > Seeing there is a lot of interest in asyncio recently I figured people > might be interested in this > http://www.snarky.ca/how-the-heck-does-async-await-work-in-python-3-5 Thanks for the link, but I'm now no wiser than I was before :-( Can somebody explain Brett's explanation? -- Steven From wrw at mac.com Fri Feb 19 22:28:38 2016 From: wrw at mac.com (William Ray Wing) Date: Fri, 19 Feb 2016 22:28:38 -0500 Subject: Considering migrating to Python from Visual Basic 6 for engineering applications In-Reply-To: <56c7bda9$0$1615$c3e8da3$5496439d@news.astraweb.com> References: <90cc50d2-1ce5-4588-9bfd-a49d439f00dd@googlegroups.com> <56c66aa9$0$1598$c3e8da3$5496439d@news.astraweb.com> <8c67cb1c-4318-4d3b-b425-da894e5138e4@googlegroups.com> <56c7bda9$0$1615$c3e8da3$5496439d@news.astraweb.com> Message-ID: > On Feb 19, 2016, at 8:13 PM, Steven D'Aprano wrote: > > On Sat, 20 Feb 2016 12:27 am, Dennis Lee Bieber wrote: > > >> Then the best suggestion I have would be to take a weekend and just >> read the language reference manual (it used to be about an 80-page PDF >> file, but has no doubt grown into some less handy dynamically linked >> HTML/"help" file). Then add the section of the library reference manual >> that discusses data types. And while in that manual, scan the table of >> contents -- you may find there are modules in the standard library that >> already do what you need. > > Surely you should start with the tutorial, not the reference manual. > > > > -- > Steven > > -- > https://mail.python.org/mailman/listinfo/python-list Plus +1. The ref manual would be like learning English from a dictionary Bill From tjreedy at udel.edu Fri Feb 19 23:15:44 2016 From: tjreedy at udel.edu (Terry Reedy) Date: Fri, 19 Feb 2016 23:15:44 -0500 Subject: [STORY-TIME] THE BDFL AND HIS PYTHON PETTING ZOO In-Reply-To: <56C67C49.9010001@stoneleaf.us> References: <48762040-a7e0-434c-92e8-8a0969210e5b@googlegroups.com> <56C67C49.9010001@stoneleaf.us> Message-ID: On 2/18/2016 9:22 PM, Ethan Furman wrote: > Anyone know one for deleting all responses to a troll post? Thunderbird has 'ignore thread' on its context menu. -- Terry Jan Reedy From denis.akhiyarov at gmail.com Fri Feb 19 23:58:20 2016 From: denis.akhiyarov at gmail.com (Denis Akhiyarov) Date: Fri, 19 Feb 2016 20:58:20 -0800 (PST) Subject: Considering migrating to Python from Visual Basic 6 for engineering applications In-Reply-To: <90cc50d2-1ce5-4588-9bfd-a49d439f00dd@googlegroups.com> References: <90cc50d2-1ce5-4588-9bfd-a49d439f00dd@googlegroups.com> Message-ID: <2dd9429e-16db-451d-beb8-1248a7029f80@googlegroups.com> On Wednesday, February 17, 2016 at 1:49:44 PM UTC-6, wrong.a... at gmail.com wrote: > I am mostly getting positive feedback for Python. > > It seems Python is used more for web based applications. Is it equally fine for creating stand-alone *.exe's? Can the same code be compiled to run on Linux or Android or web-based? > > Is it possible to create GUI elements with a good IDE? Can they be defined like in Visual Basic with given sizes, fonts, visible/invisible, etc.? > > Is it easy to do matrix operations in Python? Or do I need to write subroutines like in Visual Basic? > > Could someone kindly tell me advantages and disadvantages of Python? Or any better options? I have like 40-50 VB Forms and may be around 20000 lines of code. It will be a task to learn a new language and translate/re-write that code. > > Thanks for your responses. I'm surprised that no one mentioned this tool called vb2py. It looks outdated, but I actually used it successfully to convert vba code to python, once all dependencies were installed correctly :) http://vb2py.sourceforge.net/ You can also understand how vb objects map to python objects. vb2py has also minimal support for GUI conversion. Someone has even forked it on github recently: https://github.com/reingart/vb2py From rustompmody at gmail.com Sat Feb 20 00:24:48 2016 From: rustompmody at gmail.com (Rustom Mody) Date: Fri, 19 Feb 2016 21:24:48 -0800 (PST) Subject: How the heck does async/await work in Python 3.5 In-Reply-To: <56c7d145$0$1597$c3e8da3$5496439d@news.astraweb.com> References: <56c7d145$0$1597$c3e8da3$5496439d@news.astraweb.com> Message-ID: <9d968a2e-f23c-4c93-979d-43dfa610c343@googlegroups.com> On Saturday, February 20, 2016 at 8:07:03 AM UTC+5:30, Steven D'Aprano wrote: > On Thu, 18 Feb 2016 09:08 am, Mark Lawrence wrote: > > > Seeing there is a lot of interest in asyncio recently I figured people > > might be interested in this > > http://www.snarky.ca/how-the-heck-does-async-await-work-in-python-3-5 > > > Thanks for the link, but I'm now no wiser than I was before :-( > > Can somebody explain Brett's explanation? Does "C++" light a bulb? Less snarkily looks like a series of bolt-ons after bolt-ons IMHO Guido's (otherwise) uncannily sound intuitions have been wrong right from 2001 when he overloaded def for generators. And after that its been slippery-slope down: reusing generator-yield (statement) for coroutine-yield (expression) Most recently choosing these async-await keywords instead of the more symmetric suggestions of Greg Ewing PS. Will be off email/net for about a week From rustompmody at gmail.com Sat Feb 20 00:34:17 2016 From: rustompmody at gmail.com (Rustom Mody) Date: Fri, 19 Feb 2016 21:34:17 -0800 (PST) Subject: How the heck does async/await work in Python 3.5 In-Reply-To: <9d968a2e-f23c-4c93-979d-43dfa610c343@googlegroups.com> References: <56c7d145$0$1597$c3e8da3$5496439d@news.astraweb.com> <9d968a2e-f23c-4c93-979d-43dfa610c343@googlegroups.com> Message-ID: On Saturday, February 20, 2016 at 10:55:02 AM UTC+5:30, Rustom Mody wrote: > On Saturday, February 20, 2016 at 8:07:03 AM UTC+5:30, Steven D'Aprano wrote: > > On Thu, 18 Feb 2016 09:08 am, Mark Lawrence wrote: > > > > > Seeing there is a lot of interest in asyncio recently I figured people > > > might be interested in this > > > http://www.snarky.ca/how-the-heck-does-async-await-work-in-python-3-5 > > > > > > Thanks for the link, but I'm now no wiser than I was before :-( > > > > Can somebody explain Brett's explanation? > > Does "C++" light a bulb? > > Less snarkily looks like a series of bolt-ons after bolt-ons > > IMHO Guido's (otherwise) uncannily sound intuitions have been wrong right from > 2001 when he overloaded def for generators. > And after that its been slippery-slope down: reusing generator-yield (statement) > for coroutine-yield (expression) > Most recently choosing these async-await keywords instead of the more symmetric > suggestions of Greg Ewing Forgot the probably most important: Not merging stackless into CPython From no.email at nospam.invalid Sat Feb 20 01:44:04 2016 From: no.email at nospam.invalid (Paul Rubin) Date: Fri, 19 Feb 2016 22:44:04 -0800 Subject: How the heck does async/await work in Python 3.5 References: <56c7d145$0$1597$c3e8da3$5496439d@news.astraweb.com> <9d968a2e-f23c-4c93-979d-43dfa610c343@googlegroups.com> Message-ID: <8760xj7uff.fsf@jester.gateway.pace.com> Rustom Mody writes: > Forgot the probably most important: Not merging stackless into CPython I thought there was some serious technical obstacle to that. Where can I find Greg Ewing's suggestions about Python coroutines? The async/await stuff seems ok on the surface. I liked the Lua paper about coroutines: http://www.inf.puc-rio.br/~roberto/docs/corosblp.pdf it has a fair amount of historical info, comparison of implementation techniques, etc. But frankly the stuff I'm seeing in this thread makes me sad for Python. It's an impossible dream but it would be awesome to have Erlang-like communicating microtasks in Python. From auriocus at gmx.de Sat Feb 20 01:53:37 2016 From: auriocus at gmx.de (Christian Gollwitzer) Date: Sat, 20 Feb 2016 07:53:37 +0100 Subject: How the heck does async/await work in Python 3.5 In-Reply-To: <56c7d145$0$1597$c3e8da3$5496439d@news.astraweb.com> References: <56c7d145$0$1597$c3e8da3$5496439d@news.astraweb.com> Message-ID: Am 20.02.16 um 03:36 schrieb Steven D'Aprano: > On Thu, 18 Feb 2016 09:08 am, Mark Lawrence wrote: > >> Seeing there is a lot of interest in asyncio recently I figured people >> might be interested in this >> http://www.snarky.ca/how-the-heck-does-async-await-work-in-python-3-5 > > > Thanks for the link, but I'm now no wiser than I was before :-( > > Can somebody explain Brett's explanation? If you have difficulties wit hthe overall concept, and if you are open to discussions in another language, take a look at this video: https://channel9.msdn.com/Shows/C9-GoingNative/GoingNative-39-await-co-routines MS has added coroutine support with very similar syntax to VC++ recently, and the developer tries to explain it to the "stackful" programmers. Christian From giriprasadhraghavan at gmail.com Sat Feb 20 02:34:47 2016 From: giriprasadhraghavan at gmail.com (Giriprasadh Raghavan) Date: Sat, 20 Feb 2016 13:04:47 +0530 Subject: I can't install python on my pc Message-ID: <56c81718.0207620a.f9e33.17c0@mx.google.com> I can?t install python on my pc because it encounters an error every time I try to install it. Please help as soon as possible. Sent from Mail for Windows 10 From no.email at nospam.invalid Sat Feb 20 02:40:01 2016 From: no.email at nospam.invalid (Paul Rubin) Date: Fri, 19 Feb 2016 23:40:01 -0800 Subject: asyncio - run coroutine in the background References: <8737sumpjl.fsf@elektro.pacujo.net> <87h9ha8lt0.fsf@jester.gateway.pace.com> <87d1rwpwo2.fsf@elektro.pacujo.net> <87r3gc608i.fsf@elektro.pacujo.net> Message-ID: <871t877ru6.fsf@jester.gateway.pace.com> Marko Rauhamaa writes: > "Frank Millman" : >> I would love to drive the database asynchronously, but of the three >> databases I use, only psycopg2 seems to have asyncio support. > Yes, asyncio is at its infancy. There needs to be a moratorium on > blocking I/O. Unfortunately there appears to be no way to open a file in Linux without at least potentially blocking (slow disk or whatever). You need separate threads or processes to do the right thing. From jenswaelkens at gmail.com Sat Feb 20 02:42:38 2016 From: jenswaelkens at gmail.com (jenswaelkens at gmail.com) Date: Fri, 19 Feb 2016 23:42:38 -0800 (PST) Subject: [newbie] Problem with matplotlib Message-ID: <1751492a-45ae-422d-8819-17de767a94c4@googlegroups.com> When I use either of the following commands I get an error for which I don't have a solution, could someone here help me further? These are the commands: import matplotlib.pyplot as plt or from matplotlib.pyplot import pyplot as plt This is the error I get: Traceback (most recent call last): File "/home/waelkens/matplotlibdemo.py", line 2, in from matplotlib.pyplot import pyplot as plt File "/usr/lib/python2.7/dist-packages/matplotlib/pyplot.py", line 27, in import matplotlib.colorbar File "/usr/lib/python2.7/dist-packages/matplotlib/colorbar.py", line 36, in import matplotlib.contour as contour File "/usr/lib/python2.7/dist-packages/matplotlib/contour.py", line 17, in import matplotlib.ticker as ticker File "/usr/lib/python2.7/dist-packages/matplotlib/ticker.py", line 152, in import decimal File "/usr/lib/python2.7/decimal.py", line 3744, in _numbers.Number.register(Decimal) AttributeError: 'module' object has no attribute 'Number' thanks in advance Jens From ian.g.kelly at gmail.com Sat Feb 20 02:48:31 2016 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Sat, 20 Feb 2016 00:48:31 -0700 Subject: How the heck does async/await work in Python 3.5 In-Reply-To: <9d968a2e-f23c-4c93-979d-43dfa610c343@googlegroups.com> References: <56c7d145$0$1597$c3e8da3$5496439d@news.astraweb.com> <9d968a2e-f23c-4c93-979d-43dfa610c343@googlegroups.com> Message-ID: On Fri, Feb 19, 2016 at 10:24 PM, Rustom Mody wrote: > Less snarkily looks like a series of bolt-ons after bolt-ons > > IMHO Guido's (otherwise) uncannily sound intuitions have been wrong right from > 2001 when he overloaded def for generators. > And after that its been slippery-slope down: reusing generator-yield (statement) > for coroutine-yield (expression) > Most recently choosing these async-await keywords instead of the more symmetric > suggestions of Greg Ewing Two out of three of those are fine. IMO generators should have required a separate keyword to distinguish them from ordinary functions, but it's a minor complaint and they're otherwise a great addition to the language. I do think it's pretty clear at this point though that PEP 342 was a mistake. We should never have introduced generator-based coroutines; it's an abstraction with too much leakage. It doesn't make sense that generator-based coroutines implement the iterator protocol, because there's no reason to ever try to iterate over them. It also doesn't make sense to ever iterate over a Future, and yet asyncio Futures, which aren't even coroutines, are nevertheless *forced* to be iterable just because that's how coroutines work. Otherwise you couldn't "yield from" a Future. As another point that happens to be fresh in my mind, awaiting a Future on which an exception gets set is supposed to propagate the exception. I recently found that this breaks if the exception in question happens to be StopIteration (granted not one that should generally be allowed to propagate anyway, but that's a separate discussion) for the simple reason that raising StopIteration in a generator is equivalent to returning None. The awaiting coroutine thus gets a non-exceptional result of None rather than the expected exception. The irritating thing to me is that this even plagues PEP 492 coroutines using "await" rather than "yield from", because the two are basically the same under the covers. The former gets an iterator by calling __await__ instead of __iter__, but it's still implemented using an iterator. If you look at the asyncio.Future implementation, __await__ is actually just a synonym of the __iter__ method. My hope is that someday we can get rid of PEP 342 coroutines entirely. Then maybe we can get a coroutine implementation that's actually sane. PEP 492 with async/await and non-iterable semantics is a step in the right direction, but ultimately I think we also need an underlying implementation that *isn't* fundamentally based on iteration. From orgnut at yahoo.com Sat Feb 20 02:49:06 2016 From: orgnut at yahoo.com (Larry Hudson) Date: Fri, 19 Feb 2016 23:49:06 -0800 Subject: Considering migrating to Python from Visual Basic 6 for engineering applications In-Reply-To: <7f9c473e-b0c2-4d77-91d1-d0733c93b12d@googlegroups.com> References: <90cc50d2-1ce5-4588-9bfd-a49d439f00dd@googlegroups.com> <14c75a68-0d2e-45cc-8d73-0d71b6a6aea6@googlegroups.com> <9e57761f-26e1-41c5-8e71-23800de1fdd3@googlegroups.com> <7f9c473e-b0c2-4d77-91d1-d0733c93b12d@googlegroups.com> Message-ID: On 02/19/2016 10:14 AM, wrong.address.1 at gmail.com wrote: [snip] > > This is precisely reading one character at a time. If not exactly reading one character, it is effectively looking at each character to assemble the number. Not a good sign. I guess there might be libraries which will help read numbers better, but I would expect a good language to be able to handle this basic thing for engineers - to read numbers like Fortran and Basic do. > > Still, if I have misunderstood something, I will be glad to be corrected. I would generally know that the first three numbers will be floating point, the next will be something and then the next will be something else, etc. Does that help or does one still have to look at each character and determine how to place it in a number? > > Thanks for your patience. I don't want surprises later on, which is why I am asking very stupid questions. > It absolutely does NOT require reading a character at a time! You are reading the data from a text file, which means everything is a string. These strings (or sub-strings) can represent integers, floats, dates, phone numbers or whatever. Your example data implies a free-form style, where it is then necessary to determine the data type for each of these (sub)strings individually. Of course, if your data has a defined fixed format, this is simplified -- but they are still initially strings that have to be converted. String processing in Python is very powerful and versatile. BTW, it does no good to continue to think strictly in BASIC techniques. Python is a different language with a different approach to attacking problems. If you try to write BASIC (or C or Java or ...) programs in Python syntax you'll just get bad programs. Forget trying to find features in Python that are identical to the features of BASIC. Python requires a different mind-set to use it properly -- just like any other language. Here is a rather naive somewhat brute-force way to read your example data. I'm using a Python list here to simulate the file reading. (Actually, using read() instead of readline() will also give this list.) Python lists are essentially the same as arrays in other languages, but much more powerful and versatile. Two examples of the differences between arrays and lists are: can use mixed data types, you are not restricted to all the same data type, and the size of lists are dynamic -- they grow or shrink as necessary. Comments start with a # Strings in Python can be delimited by single-quotes ('), double-quotes (") or triple-quotes (""" or '''). Triple-quoted strings can be multi-line text. The triple-quote here is used as what Python calls a docstring, which it saves internally for documenting your program. # Define a function to determine the data type a string represents def chktyp(s): """Check string s for int, float or string. Returns the data and a type code""" try: return int(s), 'int' # Try to convert to int, return it if successful except ValueError: # No, it's not an int pass # Drop into float test try: return float(s), 'float' # Try to convert to float, return it if successful except ValueError: # No, it's not a float return s, 'str' # It must be a string # Here is your (simulated) data as a list of strings data = [ '2 12.657823 0.1823467E-04 114 0', '3 4 5 9 11', '"Lower"', # Could be left simply as "Lower" '278.15'] # Process the data for line in data: # For each line of the data dat = line.split() # Make a list of the sub-strings in the line for stng in dat: # For each substring dt, tp = chktyp(stng) # Get the data and the type print('{} is a {}'.format(dt, tp)) # Print this data Running this example gives this result: 2 is a int 12.657823 is a float 1.823467e-05 is a float 114 is a int 0 is a int 3 is a int 4 is a int 5 is a int 9 is a int 11 is a int "Lower" is a str 278.15 is a float I hope this gives you a slight hint about the Python way of doing things. Yes, of course it's very different from BASIC, but if you can pick up on the Pythonic way of doing things I think you might at least find it useful. -=- Larry -=- From steve at pearwood.info Sat Feb 20 02:54:05 2016 From: steve at pearwood.info (Steven D'Aprano) Date: Sat, 20 Feb 2016 18:54:05 +1100 Subject: Considering migrating to Python from Visual Basic 6 for engineering applications References: <90cc50d2-1ce5-4588-9bfd-a49d439f00dd@googlegroups.com> <14c75a68-0d2e-45cc-8d73-0d71b6a6aea6@googlegroups.com> <9e57761f-26e1-41c5-8e71-23800de1fdd3@googlegroups.com> <729e6513-4d1d-4b9b-be8b-4b7664abac2e@googlegroups.com> Message-ID: <56c81b9f$0$1585$c3e8da3$5496439d@news.astraweb.com> On Fri, 19 Feb 2016 10:53 pm, wrong.address.1 at gmail.com wrote: >> See http://nedbatchelder.com/text/python-parsers.html for a list of >> parsers that can do all sorts for you. Or the stdlib re module > > I am an engineer, and do not understand most of the terminology used > there. Google, or the search engine of your choice, is your friend. https://duckduckgo.com/html/ https://startpage.com/eng/? Wikipedia even more so: Wikipedia has lots of good, useful articles on computing concepts. https://en.wikipedia.org/wiki/Category:Computing And use the search box visible at the top of every page. Or ask here. > And do I need something fancy to read a line of numbers? Should it > not be built into the language? In your own words: "I will have to read data given to me by people, which may not come in nice formats like CSV" I trust that you don't expect the language to come with pre-written functions to read numbers in every imaginable format. If you do, you will be disappointed -- no language could possible do this. To answer your question "Do I need something fancy...?", no, of course not, reading a line of numbers from a text file is easy. with open("numbers.txt", "r") as f: for line in f: numbers = [int(s) for s in split(line)] will read and convert integer-valued numbers separated by whitespace like: 123 654 9374 1 -45 3625 one line at a time. You can then collate them for later use, or process them as you go. If they're floating point numbers, change the call to int() to a call to float(). -- Steven From rosuav at gmail.com Sat Feb 20 02:57:23 2016 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 20 Feb 2016 18:57:23 +1100 Subject: How the heck does async/await work in Python 3.5 In-Reply-To: References: <56c7d145$0$1597$c3e8da3$5496439d@news.astraweb.com> <9d968a2e-f23c-4c93-979d-43dfa610c343@googlegroups.com> Message-ID: On Sat, Feb 20, 2016 at 6:48 PM, Ian Kelly wrote: > As another point that happens to be fresh in my mind, awaiting a > Future on which an exception gets set is supposed to propagate the > exception. I recently found that this breaks if the exception in > question happens to be StopIteration (granted not one that should > generally be allowed to propagate anyway, but that's a separate > discussion) for the simple reason that raising StopIteration in a > generator is equivalent to returning None. Solved by PEP 479. Use "from __future__ import generator_stop" to save yourself the pain. ChrisA From steve at pearwood.info Sat Feb 20 02:59:02 2016 From: steve at pearwood.info (Steven D'Aprano) Date: Sat, 20 Feb 2016 18:59:02 +1100 Subject: Guido on python3 for beginners References: <48762040-a7e0-434c-92e8-8a0969210e5b@googlegroups.com> <5d31dd14-1a60-4f80-8889-d0616f404c70@googlegroups.com> <4e902c58-3370-4adf-905d-74630a6eea90@googlegroups.com> <60036746-78f7-4f05-a6d8-04e88b09c00f@googlegroups.com> <032f0f66-34df-4325-98e0-1127025d2a94@googlegroups.com> <31a618b2-a407-4723-9a4f-9b756fc93b0b@googlegroups.com> <0a920e92-5d31-4866-815b-cae7e201e4d8@googlegroups.com> <56c568f2$0$2832$c3e8da3$76491128@news.astraweb.com> Message-ID: <56c81cc8$0$1585$c3e8da3$5496439d@news.astraweb.com> On Thu, 18 Feb 2016 07:40 pm, Terry Reedy wrote: > 8. 2.x has two subtlely different types of classes. The 2.x docs do not > document the type of builtin and stdlib classes. I discovered that > tkinter classes are still old-style in 2.7 when I backported a patch > from 3.x to 2.7 and it mysteriously did not work. Py 3 wins here. To > me, this alone makes 2.x a bad choice for most beginners. An excellent point! > 11. To test is something is text, isinstance s, c), where 'c' is one of > str, bytes, unicode, basestring, (bytes, unicode), (str, unicode). +1 > for 3.x. In Python 3, it should be much rarer to want to test whether something is unicode or bytes. You can't combine them easily, so you rarely want to handle them in identical ways. > 12. 2.7 has two different open' functions, open and io.open. In 3.x > these are the same opjects. I believe there are other 3.x backports > like this. There's also codecs.open(). -- Steven From marko at pacujo.net Sat Feb 20 03:13:11 2016 From: marko at pacujo.net (Marko Rauhamaa) Date: Sat, 20 Feb 2016 10:13:11 +0200 Subject: asyncio - run coroutine in the background References: <8737sumpjl.fsf@elektro.pacujo.net> <87h9ha8lt0.fsf@jester.gateway.pace.com> <87d1rwpwo2.fsf@elektro.pacujo.net> <87r3gc608i.fsf@elektro.pacujo.net> <871t877ru6.fsf@jester.gateway.pace.com> Message-ID: <87oabbpzoo.fsf@elektro.pacujo.net> Paul Rubin : > Marko Rauhamaa writes: >> "Frank Millman" : >>> I would love to drive the database asynchronously, but of the three >>> databases I use, only psycopg2 seems to have asyncio support. >> Yes, asyncio is at its infancy. There needs to be a moratorium on >> blocking I/O. > > Unfortunately there appears to be no way to open a file in Linux > without at least potentially blocking (slow disk or whatever). You > need separate threads or processes to do the right thing. I have been wondering about the same thing. It would appear that disk I/O is considered nonblocking at a very deep level: * O_NONBLOCK doesn't have an effect * a process waiting for the disk to respond cannot receive a signal * a process waiting for the disk to respond stays in the "ready" state Note that * most disk I/O operates on a RAM cache that is flushed irregularly * memory mapping and swapping make disk I/O and RAM access two sides of the same coin * page faults can turn any assembly language instruction into a blocking disk I/O operation * ordinary disks don't provide for much parallelism; processes are usually serialized for disk I/O If the file system happens to be NFS, a networking issue may paralyze the whole system. ... On the networking side, there is also a dangerous blocking operation: socket.getaddrinfo() (and friends). As a consequence, socket.bind(), socket.connect() may block indefinitely. In fact, even asyncio's BaseEventLoop.create_server() and BaseEventLoop.create_sonnection() may block indefinitely without yielding. SEE ALSO getaddrinfo_a(3) Marko From ian.g.kelly at gmail.com Sat Feb 20 03:14:48 2016 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Sat, 20 Feb 2016 01:14:48 -0700 Subject: How the heck does async/await work in Python 3.5 In-Reply-To: References: <56c7d145$0$1597$c3e8da3$5496439d@news.astraweb.com> <9d968a2e-f23c-4c93-979d-43dfa610c343@googlegroups.com> Message-ID: On Sat, Feb 20, 2016 at 12:57 AM, Chris Angelico wrote: > On Sat, Feb 20, 2016 at 6:48 PM, Ian Kelly wrote: >> As another point that happens to be fresh in my mind, awaiting a >> Future on which an exception gets set is supposed to propagate the >> exception. I recently found that this breaks if the exception in >> question happens to be StopIteration (granted not one that should >> generally be allowed to propagate anyway, but that's a separate >> discussion) for the simple reason that raising StopIteration in a >> generator is equivalent to returning None. > > Solved by PEP 479. Use "from __future__ import generator_stop" to save > yourself the pain. Nope. py> from __future__ import generator_stop py> import asyncio py> async def test_coro(): ... fut = asyncio.Future() ... fut.set_exception(StopIteration()) ... print('received %r' % await fut) ... py> list(test_coro().__await__()) received None [] I think because __future__ imports are per-file, and asyncio.Future.__iter__ is defined in a file outside my control that doesn't have the __future__ import. I suppose that when the generator_stop behavior becomes standard then it will work, but still that will just cause a RuntimeError to propagate instead of the desired StopIteration. It's not really that big a deal since there is a code smell to it, but it's surprising since intuitively StopIteration should have no special meaning to a PEP 492 coroutine (it's not an iterator, wink wink, nudge nudge), and the thing being awaited is a Future, which also doesn't intuitively look like an iterator. Note that if you just call Future.result(), then the exception propagates as expected; it's just awaiting it that doesn't work. From df at see.replyto.invalid Sat Feb 20 03:30:46 2016 From: df at see.replyto.invalid (Dave Farrance) Date: Sat, 20 Feb 2016 08:30:46 +0000 Subject: [newbie] Problem with matplotlib References: <1751492a-45ae-422d-8819-17de767a94c4@googlegroups.com> Message-ID: jenswaelkens at gmail.com wrote: > File "/usr/lib/python2.7/decimal.py", line 3744, in > _numbers.Number.register(Decimal) >AttributeError: 'module' object has no attribute 'Number' Your decimal module seems broken. Confirm that in the Python shell: import numbers print numbers.Number I'm guessing you'll get: AttributeError: 'module' object has no attribute 'Number' It appears to be the usual decimal module of the current linux system python 2.7, so a corrupt hard disk? From no.email at nospam.invalid Sat Feb 20 03:37:08 2016 From: no.email at nospam.invalid (Paul Rubin) Date: Sat, 20 Feb 2016 00:37:08 -0800 Subject: asyncio - run coroutine in the background References: <8737sumpjl.fsf@elektro.pacujo.net> <87h9ha8lt0.fsf@jester.gateway.pace.com> <87d1rwpwo2.fsf@elektro.pacujo.net> <87r3gc608i.fsf@elektro.pacujo.net> <871t877ru6.fsf@jester.gateway.pace.com> <87oabbpzoo.fsf@elektro.pacujo.net> Message-ID: <87twl36amj.fsf@jester.gateway.pace.com> Marko Rauhamaa writes: > It would appear that disk I/O is considered nonblocking at a very deep > level: > * O_NONBLOCK doesn't have an effect > * a process waiting for the disk to respond cannot receive a signal > * a process waiting for the disk to respond stays in the "ready" state You can handle those issues with AIO. It's open(2) that seems to have no asynchronous analog as far as I can tell. Actually, looking at the AIO man pages, it appears that the Linux kernel currently doesn't support it and it's instead simulated by a userspace library using threads. I didn't realize that before. But AIO is at least specified by POSIX, and there was some kernel work (io_setup(2) etc.) that may or may not still be in progress. It also doesn't have an open(2) analog, sigh. > On the networking side, there is also a dangerous blocking operation: > socket.getaddrinfo() (and friends). As a consequence, socket.bind(), > socket.connect() may block indefinitely. In fact, even asyncio's > BaseEventLoop.create_server() and BaseEventLoop.create_sonnection() may > block indefinitely without yielding. getaddrinfo is a notorious pain but I think it's just a library issue; an async version should be possible in principle. How does Twisted handle it? Does it have a version? I've just felt depressed whenever I've looked at any Python async stuff. I've written many Python programs with threads and not gotten into the trouble that people keep warning about. But I haven't really understood the warnings, so maybe they know something I don't. I just write in a multiprocessing style, with every mutable object owned by exactly one thread and accessed only by RPC through queues, sort of a poor man's Erlang. There's a performance hit but there's a much bigger one from using Python in the first place, so I just live with it. From orgnut at yahoo.com Sat Feb 20 03:37:51 2016 From: orgnut at yahoo.com (Larry Hudson) Date: Sat, 20 Feb 2016 00:37:51 -0800 Subject: Guido on python3 for beginners In-Reply-To: <56c7d128$0$1597$c3e8da3$5496439d@news.astraweb.com> References: <48762040-a7e0-434c-92e8-8a0969210e5b@googlegroups.com> <5d31dd14-1a60-4f80-8889-d0616f404c70@googlegroups.com> <4e902c58-3370-4adf-905d-74630a6eea90@googlegroups.com> <60036746-78f7-4f05-a6d8-04e88b09c00f@googlegroups.com> <032f0f66-34df-4325-98e0-1127025d2a94@googlegroups.com> <31a618b2-a407-4723-9a4f-9b756fc93b0b@googlegroups.com> <0a920e92-5d31-4866-815b-cae7e201e4d8@googlegroups.com> <56c568f2$0$2832$c3e8da3$76491128@news.astraweb.com> <56c66d49$0$1620$c3e8da3$5496439d@news.astraweb.com> <33b46714-8d9b-4159-ad26-4699d8b11c4d@googlegroups.com> <56c7d128$0$1597$c3e8da3$5496439d@news.astraweb.com> Message-ID: <5qudnTGfcOJ9uFXLnZ2dnUU7-R-dnZ2d@giganews.com> On 02/19/2016 06:36 PM, Steven D'Aprano wrote: > On Fri, 19 Feb 2016 02:39 pm, Rustom Mody wrote: > [snip] > But you can't do anything interesting with this language, so it is not > satisfying. On the other hand, here's "Hello World" in another language, > one which is Turing complete so it can do anything Python or C can do: > > (=<`#9]~6ZY32Vx/4Rs+0No-&Jk)"Fh}|Bcy?`=*z]Kw%oG4UUS0/@-ejc(:'8dc > > but the learning curve is steep enough that it will be frustrating rather > than interesting. > > https://en.wikipedia.org/wiki/Malbolge > Somewhat OT, but speaking of "Hello World", check out: http://www2.latech.edu/~acm/HelloWorld.shtml It's a collection of "Hello World" programs in umpteen languages. (Well, not really umpteen, but there are a lot of them.) ;-) -=- Larry -=- From breamoreboy at yahoo.co.uk Sat Feb 20 03:42:41 2016 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Sat, 20 Feb 2016 08:42:41 +0000 Subject: [newbie] Problem with matplotlib In-Reply-To: <1751492a-45ae-422d-8819-17de767a94c4@googlegroups.com> References: <1751492a-45ae-422d-8819-17de767a94c4@googlegroups.com> Message-ID: On 20/02/2016 07:42, jenswaelkens at gmail.com wrote: > When I use either of the following commands I get an error for which I don't have a solution, could someone here help me further? > These are the commands: > import matplotlib.pyplot as plt Are you certain that this is what you typed? C:\Users\Mark\Desktop>py -2.7 Python 2.7.10 (default, May 23 2015, 09:44:00) [MSC v.1500 64 bit (AMD64)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> import matplotlib.pyplot as plt >>> Please have another go. > > or > > from matplotlib.pyplot import pyplot as plt The above is incorrect, you are trying to import 'pyplot' from 'matplotlib.pyplot'. > > This is the error I get: > Traceback (most recent call last): > File "/home/waelkens/matplotlibdemo.py", line 2, in > from matplotlib.pyplot import pyplot as plt > File "/usr/lib/python2.7/dist-packages/matplotlib/pyplot.py", line 27, in > import matplotlib.colorbar > File "/usr/lib/python2.7/dist-packages/matplotlib/colorbar.py", line 36, in > import matplotlib.contour as contour > File "/usr/lib/python2.7/dist-packages/matplotlib/contour.py", line 17, in > import matplotlib.ticker as ticker > File "/usr/lib/python2.7/dist-packages/matplotlib/ticker.py", line 152, in > import decimal > File "/usr/lib/python2.7/decimal.py", line 3744, in > _numbers.Number.register(Decimal) > AttributeError: 'module' object has no attribute 'Number' > > thanks in advance > Jens > With this type of problem I'm inclined to throw the last line of the traceback, here the 'AttributeError' one, into a search engine and see what you come up with. The hits you get are often enough to help you diagnose the problem. -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From rosuav at gmail.com Sat Feb 20 03:49:04 2016 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 20 Feb 2016 19:49:04 +1100 Subject: How the heck does async/await work in Python 3.5 In-Reply-To: References: <56c7d145$0$1597$c3e8da3$5496439d@news.astraweb.com> <9d968a2e-f23c-4c93-979d-43dfa610c343@googlegroups.com> Message-ID: On Sat, Feb 20, 2016 at 7:14 PM, Ian Kelly wrote: > On Sat, Feb 20, 2016 at 12:57 AM, Chris Angelico wrote: >> On Sat, Feb 20, 2016 at 6:48 PM, Ian Kelly wrote: >>> As another point that happens to be fresh in my mind, awaiting a >>> Future on which an exception gets set is supposed to propagate the >>> exception. I recently found that this breaks if the exception in >>> question happens to be StopIteration (granted not one that should >>> generally be allowed to propagate anyway, but that's a separate >>> discussion) for the simple reason that raising StopIteration in a >>> generator is equivalent to returning None. >> >> Solved by PEP 479. Use "from __future__ import generator_stop" to save >> yourself the pain. > > Nope. > > py> from __future__ import generator_stop > py> import asyncio > py> async def test_coro(): > ... fut = asyncio.Future() > ... fut.set_exception(StopIteration()) > ... print('received %r' % await fut) > ... > py> list(test_coro().__await__()) > received None > [] > > I think because __future__ imports are per-file, and > asyncio.Future.__iter__ is defined in a file outside my control that > doesn't have the __future__ import. You need the future directive in the file that defines the function that raises, so I guess you'd need to apply that to an asyncio call. The tricky bit here is that it's a backward compatibility change, but since asyncio is flagged provisional, I suspect the future directive could be added (anyone who's depending on set_exception(StopIteration()) to terminate without an exception will have to change code). Actually, that mightn't be a bad thing. Maybe raise that as a tracker issue? I just tested, and slapping "from __future__ import generator_stop" at the top of Lib/asyncio/futures.py causes your example to raise an exception instead of returning None, and doesn't seem to break the test suite. > I suppose that when the generator_stop behavior becomes standard then > it will work, but still that will just cause a RuntimeError to > propagate instead of the desired StopIteration. That then becomes a pretty minor wart, on par with hash() never returning -1 (which I came across recently, but only by snooping the source) - it'll hack it to -2 instead, because -1 is used as an error signal. In the same way, StopIteration is special-cased as a return signal (because there needs to be _some_ mechanism for distinguishing between yield and raise and return; normally, the difference between "has a value to return" and "has no value to return" is indicated by raising in the latter case, but now we need even more distinguishments), it can't actually be raised per se. Since the exception chains, you can't get confused. > It's not really that big a deal since there is a code smell to it, but > it's surprising since intuitively StopIteration should have no special > meaning to a PEP 492 coroutine (it's not an iterator, wink wink, nudge > nudge), and the thing being awaited is a Future, which also doesn't > intuitively look like an iterator. Note that if you just call > Future.result(), then the exception propagates as expected; it's just > awaiting it that doesn't work. Definitely seems like it should be fixed, then; the current behaviour is that Future.result() raises RuntimeError if you raise StopIteration, so having await do the same would make sense. ChrisA From df at see.replyto.invalid Sat Feb 20 03:49:53 2016 From: df at see.replyto.invalid (Dave Farrance) Date: Sat, 20 Feb 2016 08:49:53 +0000 Subject: [newbie] Problem with matplotlib References: <1751492a-45ae-422d-8819-17de767a94c4@googlegroups.com> Message-ID: It occurs to me now that the trackback might misidentify the module in use, if say, you'd named a file "numbers.py" then got rid of it later leaving a "numbers.pyc" somewhere. If so, see where it is: import numbers print numbers.__file__ From rosuav at gmail.com Sat Feb 20 03:52:15 2016 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 20 Feb 2016 19:52:15 +1100 Subject: asyncio - run coroutine in the background In-Reply-To: <87twl36amj.fsf@jester.gateway.pace.com> References: <8737sumpjl.fsf@elektro.pacujo.net> <87h9ha8lt0.fsf@jester.gateway.pace.com> <87d1rwpwo2.fsf@elektro.pacujo.net> <87r3gc608i.fsf@elektro.pacujo.net> <871t877ru6.fsf@jester.gateway.pace.com> <87oabbpzoo.fsf@elektro.pacujo.net> <87twl36amj.fsf@jester.gateway.pace.com> Message-ID: On Sat, Feb 20, 2016 at 7:37 PM, Paul Rubin wrote: > getaddrinfo is a notorious pain but I think it's just a library issue; > an async version should be possible in principle. How does Twisted > handle it? Does it have a version? In a (non-Python) program of mine, I got annoyed by synchronous name lookups, so I hacked around it: instead of using the regular library functions, I just do a DNS lookup directly (which can then be event-based - send a UDP packet, get notified when a UDP packet arrives). Downside: Ignores /etc/nsswitch.conf and /etc/hosts, and goes straight to the name server. Upside: Is able to do its own caching, since the DNS library gives me the TTLs, but gethostbyname/getaddrinfo won't. ChrisA From marko at pacujo.net Sat Feb 20 03:59:15 2016 From: marko at pacujo.net (Marko Rauhamaa) Date: Sat, 20 Feb 2016 10:59:15 +0200 Subject: asyncio - run coroutine in the background References: <8737sumpjl.fsf@elektro.pacujo.net> <87h9ha8lt0.fsf@jester.gateway.pace.com> <87d1rwpwo2.fsf@elektro.pacujo.net> <87r3gc608i.fsf@elektro.pacujo.net> <871t877ru6.fsf@jester.gateway.pace.com> <87oabbpzoo.fsf@elektro.pacujo.net> <87twl36amj.fsf@jester.gateway.pace.com> Message-ID: <87io1jpxjw.fsf@elektro.pacujo.net> Chris Angelico : > In a (non-Python) program of mine, I got annoyed by synchronous name > lookups, so I hacked around it: instead of using the regular library > functions, I just do a DNS lookup directly (which can then be > event-based - send a UDP packet, get notified when a UDP packet > arrives). Downside: Ignores /etc/nsswitch.conf and /etc/hosts, and > goes straight to the name server. Upside: Is able to do its own > caching, since the DNS library gives me the TTLs, but > gethostbyname/getaddrinfo won't. Ditto in a Python program of mine, although I don't bother with caching: the DNS server is perfectly capable of caching the entries for me. Marko From rosuav at gmail.com Sat Feb 20 04:02:18 2016 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 20 Feb 2016 20:02:18 +1100 Subject: asyncio - run coroutine in the background In-Reply-To: <87io1jpxjw.fsf@elektro.pacujo.net> References: <8737sumpjl.fsf@elektro.pacujo.net> <87h9ha8lt0.fsf@jester.gateway.pace.com> <87d1rwpwo2.fsf@elektro.pacujo.net> <87r3gc608i.fsf@elektro.pacujo.net> <871t877ru6.fsf@jester.gateway.pace.com> <87oabbpzoo.fsf@elektro.pacujo.net> <87twl36amj.fsf@jester.gateway.pace.com> <87io1jpxjw.fsf@elektro.pacujo.net> Message-ID: On Sat, Feb 20, 2016 at 7:59 PM, Marko Rauhamaa wrote: > Chris Angelico : > >> In a (non-Python) program of mine, I got annoyed by synchronous name >> lookups, so I hacked around it: instead of using the regular library >> functions, I just do a DNS lookup directly (which can then be >> event-based - send a UDP packet, get notified when a UDP packet >> arrives). Downside: Ignores /etc/nsswitch.conf and /etc/hosts, and >> goes straight to the name server. Upside: Is able to do its own >> caching, since the DNS library gives me the TTLs, but >> gethostbyname/getaddrinfo won't. > > Ditto in a Python program of mine, although I don't bother with caching: > the DNS server is perfectly capable of caching the entries for me. If you know you have a local DNS server, sure. Mine is written for a generic situation where that can't be depended on, so it caches itself. But it's no big deal. ChrisA From ian.g.kelly at gmail.com Sat Feb 20 04:11:33 2016 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Sat, 20 Feb 2016 02:11:33 -0700 Subject: How the heck does async/await work in Python 3.5 In-Reply-To: References: <56c7d145$0$1597$c3e8da3$5496439d@news.astraweb.com> <9d968a2e-f23c-4c93-979d-43dfa610c343@googlegroups.com> Message-ID: On Sat, Feb 20, 2016 at 1:49 AM, Chris Angelico wrote: > Definitely seems like it should be fixed, then; the current behaviour > is that Future.result() raises RuntimeError if you raise > StopIteration, so having await do the same would make sense. Future.result() itself simply raises the StopIteration. If you call it inside a coroutine (which seems odd -- why not await it?) then it's the coroutine that reraises the StopIteration as RuntimeError. The __future__ import doesn't even seem to be needed in this case. From df at see.replyto.invalid Sat Feb 20 04:19:27 2016 From: df at see.replyto.invalid (Dave Farrance) Date: Sat, 20 Feb 2016 09:19:27 +0000 Subject: [newbie] Problem with matplotlib References: <1751492a-45ae-422d-8819-17de767a94c4@googlegroups.com> Message-ID: Dave Farrance wrote: >It occurs to me now that the trackback might misidentify the module in >use, if say, you'd named a file "numbers.py" then got rid of it later >leaving a "numbers.pyc" somewhere. If so, see where it is: > >import numbers >print numbers.__file__ I seem to have "numbers" on the brain. Replace with "decimal", of course. From ian.g.kelly at gmail.com Sat Feb 20 04:21:37 2016 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Sat, 20 Feb 2016 02:21:37 -0700 Subject: How the heck does async/await work in Python 3.5 In-Reply-To: References: <56c7d145$0$1597$c3e8da3$5496439d@news.astraweb.com> <9d968a2e-f23c-4c93-979d-43dfa610c343@googlegroups.com> Message-ID: On Sat, Feb 20, 2016 at 1:49 AM, Chris Angelico wrote: > Actually, that mightn't be a bad thing. Maybe raise that as a tracker > issue? I just tested, and slapping "from __future__ import > generator_stop" at the top of Lib/asyncio/futures.py causes your > example to raise an exception instead of returning None, and doesn't > seem to break the test suite. I added your suggestion to the existing http://bugs.python.org/issue26221. From marko at pacujo.net Sat Feb 20 04:28:42 2016 From: marko at pacujo.net (Marko Rauhamaa) Date: Sat, 20 Feb 2016 11:28:42 +0200 Subject: asyncio - run coroutine in the background References: <8737sumpjl.fsf@elektro.pacujo.net> <87h9ha8lt0.fsf@jester.gateway.pace.com> <87d1rwpwo2.fsf@elektro.pacujo.net> <87r3gc608i.fsf@elektro.pacujo.net> <871t877ru6.fsf@jester.gateway.pace.com> <87oabbpzoo.fsf@elektro.pacujo.net> <87twl36amj.fsf@jester.gateway.pace.com> Message-ID: <87egc7pw6t.fsf@elektro.pacujo.net> Paul Rubin : > I've just felt depressed whenever I've looked at any Python async > stuff. I've written many Python programs with threads and not gotten > into the trouble that people keep warning about. Programming-model-wise, asyncio is virtually identical with threads. In each, I dislike the implicit state concept. I want the state to stand out with big block letters. > I've just felt depressed whenever I've looked at any Python async > stuff. I've written many Python programs with threads and not gotten > into the trouble that people keep warning about. But I haven't really > understood the warnings, so maybe they know something I don't. I just > write in a multiprocessing style, with every mutable object owned by > exactly one thread and accessed only by RPC through queues, sort of a > poor man's Erlang. There's a performance hit but there's a much bigger > one from using Python in the first place, so I just live with it. Good for you if you have been able to choose your own programming model. Most people have to deal with a legacy mess. Also, maintainers who inherit your tidy code might not be careful to ship only nonmutable objects in the queues. Your way of using threads works, of course, with the caveat that it is not possible to get rid of a blocking thread from the outside. With asyncio, you can at least cancel tasks. Marko From eryksun at gmail.com Sat Feb 20 08:31:50 2016 From: eryksun at gmail.com (eryk sun) Date: Sat, 20 Feb 2016 07:31:50 -0600 Subject: extending PATH on Windows? In-Reply-To: References: <56C72420.8020607@yahoo.no> Message-ID: On Fri, Feb 19, 2016 at 6:57 PM, Dennis Lee Bieber wrote: > > Problem -- if the OP's PATH had contained the location of the add2path > script, the OP might not have needed to search for it... (presuming their > configuration also was set up to treat .py files as executable, entering > win_add2path would be all that was needed to run it on a command line) Ulli didn't need to run the file, but to read it as example code. > "where" only searches the locations defined in PATH I made it clear in several examples that where.exe is not limited to searching PATH. From kevinjacobconway at gmail.com Sat Feb 20 08:52:02 2016 From: kevinjacobconway at gmail.com (Kevin Conway) Date: Sat, 20 Feb 2016 13:52:02 +0000 Subject: asyncio - run coroutine in the background In-Reply-To: <87egc7pw6t.fsf@elektro.pacujo.net> References: <8737sumpjl.fsf@elektro.pacujo.net> <87h9ha8lt0.fsf@jester.gateway.pace.com> <87d1rwpwo2.fsf@elektro.pacujo.net> <87r3gc608i.fsf@elektro.pacujo.net> <871t877ru6.fsf@jester.gateway.pace.com> <87oabbpzoo.fsf@elektro.pacujo.net> <87twl36amj.fsf@jester.gateway.pace.com> <87egc7pw6t.fsf@elektro.pacujo.net> Message-ID: > getaddrinfo is a notorious pain but I think it's just a library issue; an async version should be possible in principle. How does Twisted handle it? Does it have a version? I think we're a little outside the scope of OP's question at this point, but for the sake of answering this: There are a few cases that I know of where Twisted uses the standard lib socket DNS methods. One is when resolving names to IPv6 addresses [1] when creating a client connection to a remote source. The other is in the default DNS resolver that is installed in the reactor [2]. Creating client connections allows the call to 'getaddrinfo' to block without mitigation. The default DNS resolver, unfortunately, dispatches calls of 'gethostbyname' to a thread pool. Without seeing the commit history, I'd assume the use of 'socket' and threads by default is an artifact that predates the implementation of the DNS protocol in Twisted. Twisted has, in 'twisted.names' [3], a DNS protocol that uses UDP and leverages the reactor appropriately. Thankfully, Twisted has a reactor method called 'installResolver' [4] that allows you to hook in any DNS resolver implementation you want so you aren't stuck using the default, threaded implementation. As far as asyncio, it also defaults to an implementation that delegates to an executor (default: threadpool). Unlike Twisted, though, it appears to require a subclass of the event loop to override the 'getaddrinfo' method [5]. [1] https://github.com/twisted/twisted/blob/trunk/twisted/internet/tcp.py#L622 [2] https://github.com/twisted/twisted/blob/trunk/twisted/internet/base.py#L257 [3] https://github.com/twisted/twisted/tree/trunk/twisted/names [4] https://github.com/twisted/twisted/blob/trunk/twisted/internet/base.py#L509 [5] https://github.com/python/cpython/blob/master/Lib/asyncio/base_events.py#L572 On Sat, Feb 20, 2016, 03:31 Marko Rauhamaa wrote: > Paul Rubin : > > > I've just felt depressed whenever I've looked at any Python async > > stuff. I've written many Python programs with threads and not gotten > > into the trouble that people keep warning about. > > Programming-model-wise, asyncio is virtually identical with threads. In > each, I dislike the implicit state concept. I want the state to stand > out with big block letters. > > > I've just felt depressed whenever I've looked at any Python async > > stuff. I've written many Python programs with threads and not gotten > > into the trouble that people keep warning about. But I haven't really > > understood the warnings, so maybe they know something I don't. I just > > write in a multiprocessing style, with every mutable object owned by > > exactly one thread and accessed only by RPC through queues, sort of a > > poor man's Erlang. There's a performance hit but there's a much bigger > > one from using Python in the first place, so I just live with it. > > Good for you if you have been able to choose your own programming model. > Most people have to deal with a legacy mess. Also, maintainers who > inherit your tidy code might not be careful to ship only nonmutable > objects in the queues. > > Your way of using threads works, of course, with the caveat that it is > not possible to get rid of a blocking thread from the outside. With > asyncio, you can at least cancel tasks. > > > Marko > -- > https://mail.python.org/mailman/listinfo/python-list > From breamoreboy at yahoo.co.uk Sat Feb 20 09:18:09 2016 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Sat, 20 Feb 2016 14:18:09 +0000 Subject: I can't install python on my pc In-Reply-To: <56c81718.0207620a.f9e33.17c0@mx.google.com> References: <56c81718.0207620a.f9e33.17c0@mx.google.com> Message-ID: On 20/02/2016 07:34, Giriprasadh Raghavan wrote: > > I can?t install python on my pc because it encounters an error every time I try to install it. Please help as soon as possible. > Sent from Mail for Windows 10 > Asked and answered repeatedly over the last few months, so please search the archives for the answer to your specific situation, whatever that may be. -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From nholtz at cee.carleton.ca Sat Feb 20 11:22:45 2016 From: nholtz at cee.carleton.ca (nholtz) Date: Sat, 20 Feb 2016 08:22:45 -0800 (PST) Subject: Considering migrating to Python from Visual Basic 6 for engineering applications In-Reply-To: <90cc50d2-1ce5-4588-9bfd-a49d439f00dd@googlegroups.com> References: <90cc50d2-1ce5-4588-9bfd-a49d439f00dd@googlegroups.com> Message-ID: <37913023-69f6-41d5-b351-9aff124167e4@googlegroups.com> On Wednesday, February 17, 2016 at 2:49:44 PM UTC-5, wrong.a... at gmail.com wrote: > I am mostly getting positive feedback for Python. > ... I'm surprised no one has mentioned jupyter yet, so here goes ... A browser-based notebook, see http://www.jupyter.org I think this is an unparalleled way to learn Python, by experimentation at the start and for development later. As an example, I wanted to do some table processing, so a couple of lines to experiment with Pandas reading CSV files, then a couple of more lines to start processing the data ... What a wonderful way to experimentally develop software ... Can even develop simple GUIs *really* easily (but packaging for distribution is probably not easy, unless distribution is only within an organization and that organization can run a private notebook server). If you want to try it, this page http://jupyter.readthedocs.org/en/latest/install.html recommends installing Anaconda Python which installs a *lot* of stuff (numpy, scipy, sympy, pandas, etc. etc.) I highly recommend both (I use them for Civil Engineering (structures) software and teach a related university course) From jenswaelkens at gmail.com Sat Feb 20 12:02:38 2016 From: jenswaelkens at gmail.com (jenswaelkens at gmail.com) Date: Sat, 20 Feb 2016 09:02:38 -0800 (PST) Subject: [newbie] Problem with matplotlib In-Reply-To: References: <1751492a-45ae-422d-8819-17de767a94c4@googlegroups.com> Message-ID: <0062d4be-6dfa-48e4-be08-050f33ea6d75@googlegroups.com> Op zaterdag 20 februari 2016 09:50:05 UTC+1 schreef Dave Farrance: > It occurs to me now that the trackback might misidentify the module in > use, if say, you'd named a file "numbers.py" then got rid of it later > leaving a "numbers.pyc" somewhere. If so, see where it is: > > import numbers > print numbers.__file__ Dear Farrance, You have solved my problem, there was indeed a program which I had called numbers.py and a numbers.pyc which caused this behaviour. Thanks a lot for helping me. kind regards, Jens From jenswaelkens at gmail.com Sat Feb 20 12:05:36 2016 From: jenswaelkens at gmail.com (jenswaelkens at gmail.com) Date: Sat, 20 Feb 2016 09:05:36 -0800 (PST) Subject: [newbie] Problem with matplotlib In-Reply-To: References: <1751492a-45ae-422d-8819-17de767a94c4@googlegroups.com> Message-ID: <5fefa9ca-31a2-41a2-8fd4-97367e0a6c3f@googlegroups.com> Op zaterdag 20 februari 2016 09:43:35 UTC+1 schreef Mark Lawrence: > On 20/02/2016 07:42, jenswaelkens at gmail.com wrote: > > When I use either of the following commands I get an error for which I don't have a solution, could someone here help me further? > > These are the commands: > > import matplotlib.pyplot as plt > > Are you certain that this is what you typed? > > C:\Users\Mark\Desktop>py -2.7 > Python 2.7.10 (default, May 23 2015, 09:44:00) [MSC v.1500 64 bit > (AMD64)] on win32 > Type "help", "copyright", "credits" or "license" for more information. > >>> import matplotlib.pyplot as plt > >>> > > Please have another go. > > > > > or > > > > from matplotlib.pyplot import pyplot as plt > > The above is incorrect, you are trying to import 'pyplot' from > 'matplotlib.pyplot'. You are right, this doesn't work, I got it from some source on the Internet tryin to solve the initial problem with the other statement...Anyway, as you see in my other reply, the problem has been solved. kind regards, Jens > > > > > This is the error I get: > > Traceback (most recent call last): > > File "/home/waelkens/matplotlibdemo.py", line 2, in > > from matplotlib.pyplot import pyplot as plt > > File "/usr/lib/python2.7/dist-packages/matplotlib/pyplot.py", line 27, in > > import matplotlib.colorbar > > File "/usr/lib/python2.7/dist-packages/matplotlib/colorbar.py", line 36, in > > import matplotlib.contour as contour > > File "/usr/lib/python2.7/dist-packages/matplotlib/contour.py", line 17, in > > import matplotlib.ticker as ticker > > File "/usr/lib/python2.7/dist-packages/matplotlib/ticker.py", line 152, in > > import decimal > > File "/usr/lib/python2.7/decimal.py", line 3744, in > > _numbers.Number.register(Decimal) > > AttributeError: 'module' object has no attribute 'Number' > > > > thanks in advance > > Jens > > > > With this type of problem I'm inclined to throw the last line of the > traceback, here the 'AttributeError' one, into a search engine and see > what you come up with. The hits you get are often enough to help you > diagnose the problem. > > -- > My fellow Pythonistas, ask not what our language can do for you, ask > what you can do for our language. > > Mark Lawrence From martin at linux-ip.net Sat Feb 20 12:45:02 2016 From: martin at linux-ip.net (Martin A. Brown) Date: Sat, 20 Feb 2016 09:45:02 -0800 Subject: asyncio - run coroutine in the background In-Reply-To: References: <8737sumpjl.fsf@elektro.pacujo.net> <87h9ha8lt0.fsf@jester.gateway.pace.com> <87d1rwpwo2.fsf@elektro.pacujo.net> <87r3gc608i.fsf@elektro.pacujo.net> <871t877ru6.fsf@jester.gateway.pace.com> <87oabbpzoo.fsf@elektro.pacujo.net> <87twl36amj.fsf@jester.gateway.pace.com> Message-ID: Hello there, I realize that this discussion of supporting asynchronous name lookup requests in DNS is merely a detour in this thread on asyncio, but I couldn't resist mentioning an existing tool. >> getaddrinfo is a notorious pain but I think it's just a library >> issue; an async version should be possible in principle. How >> does Twisted handle it? Does it have a version? > >In a (non-Python) program of mine, I got annoyed by synchronous >name lookups, so I hacked around it: instead of using the regular >library functions, I just do a DNS lookup directly (which can then >be event-based - send a UDP packet, get notified when a UDP packet >arrives). Downside: Ignores /etc/nsswitch.conf and /etc/hosts, and >goes straight to the name server. Upside: Is able to do its own >caching, since the DNS library gives me the TTLs, but >gethostbyname/getaddrinfo won't. Another (non-Python) DNS name lookup library that does practically the same thing (along with the shortcomingsn you mentioned, Chris: no NSS nor /etc/hosts) is the adns library. Well, it is DNS, after all. http://www.gnu.org/software/adns/ https://pypi.python.org/pypi/adns-python/1.2.1 And, there are Python bindings. I have been quite happy using the adns tools (and tools built on the Python bindings) for mass lookups (millions of DNS names). It works very nicely. Just sharing knowledge of an existing tool, -Martin -- Martin A. Brown http://linux-ip.net/ From wrong.address.1 at gmail.com Sat Feb 20 13:38:40 2016 From: wrong.address.1 at gmail.com (wrong.address.1 at gmail.com) Date: Sat, 20 Feb 2016 10:38:40 -0800 (PST) Subject: Considering migrating to Python from Visual Basic 6 for engineering applications In-Reply-To: References: <90cc50d2-1ce5-4588-9bfd-a49d439f00dd@googlegroups.com> <14c75a68-0d2e-45cc-8d73-0d71b6a6aea6@googlegroups.com> <9e57761f-26e1-41c5-8e71-23800de1fdd3@googlegroups.com> <7f9c473e-b0c2-4d77-91d1-d0733c93b12d@googlegroups.com> Message-ID: <23d8156f-1808-4395-9c04-27d2984fe67c@googlegroups.com> On Saturday, 20 February 2016 09:49:17 UTC+2, Larry Hudson wrote: > On 02/19/2016 10:14 AM, wrong.address.1 at gmail.com wrote: > [snip] > > > > This is precisely reading one character at a time. If not exactly reading one character, it is effectively looking at each character to assemble the number. Not a good sign. I guess there might be libraries which will help read numbers better, but I would expect a good language to be able to handle this basic thing for engineers - to read numbers like Fortran and Basic do. > > > > Still, if I have misunderstood something, I will be glad to be corrected. I would generally know that the first three numbers will be floating point, the next will be something and then the next will be something else, etc. Does that help or does one still have to look at each character and determine how to place it in a number? > > > > Thanks for your patience. I don't want surprises later on, which is why I am asking very stupid questions. > > > It absolutely does NOT require reading a character at a time! You are reading the data from a > text file, which means everything is a string. These strings (or sub-strings) can represent > integers, floats, dates, phone numbers or whatever. Your example data implies a free-form > style, where it is then necessary to determine the data type for each of these (sub)strings > individually. Of course, if your data has a defined fixed format, this is simplified -- but > they are still initially strings that have to be converted. String processing in Python is very > powerful and versatile. > > BTW, it does no good to continue to think strictly in BASIC techniques. Python is a different > language with a different approach to attacking problems. If you try to write BASIC (or C or > Java or ...) programs in Python syntax you'll just get bad programs. Forget trying to find > features in Python that are identical to the features of BASIC. Python requires a different > mind-set to use it properly -- just like any other language. > > Here is a rather naive somewhat brute-force way to read your example data. I'm using a Python > list here to simulate the file reading. (Actually, using read() instead of readline() will also > give this list.) Python lists are essentially the same as arrays in other languages, but much > more powerful and versatile. Two examples of the differences between arrays and lists are: can > use mixed data types, you are not restricted to all the same data type, and the size of lists > are dynamic -- they grow or shrink as necessary. > > Comments start with a # > Strings in Python can be delimited by single-quotes ('), double-quotes (") or triple-quotes (""" > or '''). Triple-quoted strings can be multi-line text. The triple-quote here is used as what > Python calls a docstring, which it saves internally for documenting your program. > > > # Define a function to determine the data type a string represents > def chktyp(s): > """Check string s for int, float or string. Returns the data and a type code""" > try: > return int(s), 'int' # Try to convert to int, return it if successful > except ValueError: # No, it's not an int > pass # Drop into float test > try: > return float(s), 'float' # Try to convert to float, return it if successful > except ValueError: # No, it's not a float > return s, 'str' # It must be a string > > # Here is your (simulated) data as a list of strings > data = [ > '2 12.657823 0.1823467E-04 114 0', > '3 4 5 9 11', > '"Lower"', # Could be left simply as "Lower" > '278.15'] > > # Process the data > for line in data: # For each line of the data > dat = line.split() # Make a list of the sub-strings in the line > for stng in dat: # For each substring > dt, tp = chktyp(stng) # Get the data and the type > print('{} is a {}'.format(dt, tp)) # Print this data > > > Running this example gives this result: > > 2 is a int > 12.657823 is a float > 1.823467e-05 is a float > 114 is a int > 0 is a int > 3 is a int > 4 is a int > 5 is a int > 9 is a int > 11 is a int > "Lower" is a str > 278.15 is a float > > I hope this gives you a slight hint about the Python way of doing things. Yes, of course it's > very different from BASIC, but if you can pick up on the Pythonic way of doing things I think > you might at least find it useful. > > -=- Larry -=- I realise that this file i/o will have to be thought of in a different way if I start to use Python. This may still be worthwhile inspite of the complexity and loss in readability of the code. To give an example of the kind of things I have to read (and I have to read numbers from several files usually), here is some VB6 code: Open "N020S.TXT" For Input As #8 Input #8, ND, NIN, NT ' all integers Text9.Text = ND Text1 = CStr(NIN) Text4 = NT Text12 = "" For i = 1 To NIN Input #8, DINCOL(i) ' single precision floating point vector Text12 = Text12.Text & DINCOL(i) & " " If Dvarname(1) <> "" Then varname(i) = Dvarname(NLCOL(i)) ' strings Next i etc.. The data file (written by another VB6 code) contains in the first few lines: 10 6 1 8.65 0.2192347 3.33E-4 44 0.0051 6 9 1 2 1 3 How complicated could this get in Python? Reading the numbers is one thing, and then placing the values in text boxes of the GUI. Or can I write my own reading subroutines which can then be called like ReadVBstyle 8, ND, NIN, NT to make the code more readable? From wrong.address.1 at gmail.com Sat Feb 20 13:45:17 2016 From: wrong.address.1 at gmail.com (wrong.address.1 at gmail.com) Date: Sat, 20 Feb 2016 10:45:17 -0800 (PST) Subject: Considering migrating to Python from Visual Basic 6 for engineering applications In-Reply-To: <56c81b9f$0$1585$c3e8da3$5496439d@news.astraweb.com> References: <90cc50d2-1ce5-4588-9bfd-a49d439f00dd@googlegroups.com> <14c75a68-0d2e-45cc-8d73-0d71b6a6aea6@googlegroups.com> <9e57761f-26e1-41c5-8e71-23800de1fdd3@googlegroups.com> <729e6513-4d1d-4b9b-be8b-4b7664abac2e@googlegroups.com> <56c81b9f$0$1585$c3e8da3$5496439d@news.astraweb.com> Message-ID: <22994701-0b2b-42ba-9ae5-36b8c428150b@googlegroups.com> On Saturday, 20 February 2016 09:54:15 UTC+2, Steven D'Aprano wrote: > On Fri, 19 Feb 2016 10:53 pm, wrong.address.1 at gmail.com wrote: > > >> See http://nedbatchelder.com/text/python-parsers.html for a list of > >> parsers that can do all sorts for you. Or the stdlib re module > > > > I am an engineer, and do not understand most of the terminology used > > there. > > Google, or the search engine of your choice, is your friend. > > https://duckduckgo.com/html/ > > https://startpage.com/eng/? > > > Wikipedia even more so: Wikipedia has lots of good, useful articles on > computing concepts. > > https://en.wikipedia.org/wiki/Category:Computing > > And use the search box visible at the top of every page. > > Or ask here. > > > > > And do I need something fancy to read a line of numbers? Should it > > not be built into the language? > > In your own words: > > "I will have to read data given to me by people, which may not come in nice > formats like CSV" > > I trust that you don't expect the language to come with pre-written > functions to read numbers in every imaginable format. If you do, you will > be disappointed -- no language could possible do this. > > To answer your question "Do I need something fancy...?", no, of course not, > reading a line of numbers from a text file is easy. > > with open("numbers.txt", "r") as f: > for line in f: > numbers = [int(s) for s in split(line)] > This looks good enough. This does not seem complicated (although I still understand almost nothing of what is written). I was simply not expressing myself in a way you people would understand. > > will read and convert integer-valued numbers separated by whitespace like: > > 123 654 9374 1 -45 3625 > > > one line at a time. You can then collate them for later use, or process them > as you go. If they're floating point numbers, change the call to int() to a > call to float(). > And I guess if the first three numbers are integers and the fourth is a float, then also there must be some equally straightforward way. Thanks for this explanation, which changes my view about reading numbers in Python. > > > > > > -- > Steven From wrong.address.1 at gmail.com Sat Feb 20 13:47:28 2016 From: wrong.address.1 at gmail.com (wrong.address.1 at gmail.com) Date: Sat, 20 Feb 2016 10:47:28 -0800 (PST) Subject: Considering migrating to Python from Visual Basic 6 for engineering applications In-Reply-To: <37913023-69f6-41d5-b351-9aff124167e4@googlegroups.com> References: <90cc50d2-1ce5-4588-9bfd-a49d439f00dd@googlegroups.com> <37913023-69f6-41d5-b351-9aff124167e4@googlegroups.com> Message-ID: On Saturday, 20 February 2016 18:23:26 UTC+2, nholtz wrote: > On Wednesday, February 17, 2016 at 2:49:44 PM UTC-5, wrong.a... at gmail.com wrote: > > I am mostly getting positive feedback for Python. > > ... > > I'm surprised no one has mentioned jupyter yet, so here goes ... > > A browser-based notebook, see http://www.jupyter.org > > I think this is an unparalleled way to learn Python, by experimentation at the start and for development later. As an example, I wanted to do some table > processing, so a couple of lines to experiment with Pandas reading CSV files, then a couple of more lines to start processing the data ... What a wonderful way to experimentally develop software ... > > Can even develop simple GUIs *really* easily (but packaging for distribution is probably not easy, unless distribution is only within an organization and that organization can run a private notebook server). > > If you want to try it, this page > > http://jupyter.readthedocs.org/en/latest/install.html > > recommends installing Anaconda Python which installs a *lot* of stuff (numpy, scipy, sympy, pandas, etc. etc.) > > I highly recommend both (I use them for Civil Engineering (structures) software and teach a related university course) Thanks. This gives me some more confidence that it will be a good choice for my work as well. From wrong.address.1 at gmail.com Sat Feb 20 13:50:43 2016 From: wrong.address.1 at gmail.com (wrong.address.1 at gmail.com) Date: Sat, 20 Feb 2016 10:50:43 -0800 (PST) Subject: Considering migrating to Python from Visual Basic 6 for engineering applications In-Reply-To: <2dd9429e-16db-451d-beb8-1248a7029f80@googlegroups.com> References: <90cc50d2-1ce5-4588-9bfd-a49d439f00dd@googlegroups.com> <2dd9429e-16db-451d-beb8-1248a7029f80@googlegroups.com> Message-ID: On Saturday, 20 February 2016 06:58:39 UTC+2, Denis Akhiyarov wrote: > On Wednesday, February 17, 2016 at 1:49:44 PM UTC-6, wrong.a... at gmail.com wrote: > > I am mostly getting positive feedback for Python. > > > > It seems Python is used more for web based applications. Is it equally fine for creating stand-alone *.exe's? Can the same code be compiled to run on Linux or Android or web-based? > > > > Is it possible to create GUI elements with a good IDE? Can they be defined like in Visual Basic with given sizes, fonts, visible/invisible, etc.? > > > > Is it easy to do matrix operations in Python? Or do I need to write subroutines like in Visual Basic? > > > > Could someone kindly tell me advantages and disadvantages of Python? Or any better options? I have like 40-50 VB Forms and may be around 20000 lines of code. It will be a task to learn a new language and translate/re-write that code. > > > > Thanks for your responses. > > I'm surprised that no one mentioned this tool called vb2py. It looks outdated, but I actually used it successfully to convert vba code to python, once all dependencies were installed correctly :) > > http://vb2py.sourceforge.net/ > > You can also understand how vb objects map to python objects. > > vb2py has also minimal support for GUI conversion. > > Someone has even forked it on github recently: > > https://github.com/reingart/vb2py This will definitely be useful to me in the early phase. From auriocus at gmx.de Sat Feb 20 14:21:13 2016 From: auriocus at gmx.de (Christian Gollwitzer) Date: Sat, 20 Feb 2016 20:21:13 +0100 Subject: Considering migrating to Python from Visual Basic 6 for engineering applications In-Reply-To: <22994701-0b2b-42ba-9ae5-36b8c428150b@googlegroups.com> References: <90cc50d2-1ce5-4588-9bfd-a49d439f00dd@googlegroups.com> <14c75a68-0d2e-45cc-8d73-0d71b6a6aea6@googlegroups.com> <9e57761f-26e1-41c5-8e71-23800de1fdd3@googlegroups.com> <729e6513-4d1d-4b9b-be8b-4b7664abac2e@googlegroups.com> <56c81b9f$0$1585$c3e8da3$5496439d@news.astraweb.com> <22994701-0b2b-42ba-9ae5-36b8c428150b@googlegroups.com> Message-ID: Am 20.02.16 um 19:45 schrieb wrong.address.1 at gmail.com: > On Saturday, 20 February 2016 09:54:15 UTC+2, Steven D'Aprano wrote: >> To answer your question "Do I need something fancy...?", no, of course not, >> reading a line of numbers from a text file is easy. >> >> with open("numbers.txt", "r") as f: >> for line in f: >> numbers = [int(s) for s in split(line)] >> > > This looks good enough. This does not seem complicated (although I still understand almost nothing of what is written). I was simply not expressing myself in a way you people would understand. > >> >> will read and convert integer-valued numbers separated by whitespace like: >> >> 123 654 9374 1 -45 3625 >> >> >> one line at a time. You can then collate them for later use, or process them >> as you go. If they're floating point numbers, change the call to int() to a >> call to float(). >> > > And I guess if the first three numbers are integers and the fourth is a float, then also there must be some equally straightforward way. If you know in advance, what you expect, then it is easy. Most people who gave you answers assumed that you have a messy file and don't know if an int or float follows, and you want a program which decides by itself whether to read an int, float or string. Whereas, if the format is fixed, for 3 ints and 1 float, you could do: str='1 2 3 3.14159' # some example fmt=(int,int,int,float) # define format a,b,c,d=[type(x) for (type,x) in zip(fmt, str.split())] It's one line, but it might look involved and actually it uses a list comprehension, tuple unpacking and first class functions, so it's certainly not comprehensible from the first Python lesson. Another alternative would be sscanf. This is a 3rd party module which simulates C sscanf, optimized more or less for this kind of number parsing: https://hkn.eecs.berkeley.edu/~dyoo/python/scanf/ After installing that, you can do: from scanf import sscanf str='1 2 3 3.14159' a,b,c,d=sscanf(str, '%d %d %d %f') whereby '%d' means integer and '%f' is float. sscanf also handles fixed-width columns which you often get from Fortran programs. Christian From oscar.j.benjamin at gmail.com Sat Feb 20 15:09:04 2016 From: oscar.j.benjamin at gmail.com (Oscar Benjamin) Date: Sat, 20 Feb 2016 20:09:04 +0000 Subject: I can't install python on my pc In-Reply-To: <56c81718.0207620a.f9e33.17c0@mx.google.com> References: <56c81718.0207620a.f9e33.17c0@mx.google.com> Message-ID: On 20 Feb 2016 14:05, "Giriprasadh Raghavan" wrote: > > > I can?t install python on my pc because it encounters an error every time I try to install it. Please help as soon as possible. You need to give more information than that to get clear help. What error do you see? If there is an error message what does it say? What exactly do you do that results in the message? From rosuav at gmail.com Sat Feb 20 16:47:28 2016 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 21 Feb 2016 08:47:28 +1100 Subject: asyncio - run coroutine in the background In-Reply-To: References: <8737sumpjl.fsf@elektro.pacujo.net> <87h9ha8lt0.fsf@jester.gateway.pace.com> <87d1rwpwo2.fsf@elektro.pacujo.net> <87r3gc608i.fsf@elektro.pacujo.net> <871t877ru6.fsf@jester.gateway.pace.com> <87oabbpzoo.fsf@elektro.pacujo.net> <87twl36amj.fsf@jester.gateway.pace.com> Message-ID: On Sun, Feb 21, 2016 at 4:45 AM, Martin A. Brown wrote: > Another (non-Python) DNS name lookup library that does practically > the same thing (along with the shortcomingsn you mentioned, Chris: > no NSS nor /etc/hosts) is the adns library. Well, it is DNS, after > all. > > http://www.gnu.org/software/adns/ > https://pypi.python.org/pypi/adns-python/1.2.1 > > And, there are Python bindings. I have been quite happy using the > adns tools (and tools built on the Python bindings) for mass lookups > (millions of DNS names). It works very nicely. > > Just sharing knowledge of an existing tool, > Ultimately, anything that replaces a gethostbyname/getaddrinfo call with an explicit DNS lookup is going to have the exact same benefit and downside: lookups won't freeze the program, but you can't use /etc/hosts any more. (Slightly sloppy language but that's how an end user will see it.) ChrisA From roel at roelschroeven.net Sat Feb 20 17:28:40 2016 From: roel at roelschroeven.net (Roel Schroeven) Date: Sat, 20 Feb 2016 23:28:40 +0100 Subject: Considering migrating to Python from Visual Basic 6 for engineering applications In-Reply-To: <9e57761f-26e1-41c5-8e71-23800de1fdd3@googlegroups.com> References: <90cc50d2-1ce5-4588-9bfd-a49d439f00dd@googlegroups.com> <14c75a68-0d2e-45cc-8d73-0d71b6a6aea6@googlegroups.com> <9e57761f-26e1-41c5-8e71-23800de1fdd3@googlegroups.com> Message-ID: wrong.address.1 at gmail.com schreef op 2016-02-19 11:47: > Thanks. The data I will often have to read from text files could read like > > 2 12.657823 0.1823467E-04 114 0 > 3 4 5 9 11 > "Lower" > 278.15 > > Is it straightforward to read this, or does one have to read one character at a time and then figure out what the numbers are? The question is: what is known about the format of the data? Is it fixed, or does the code need to be smart enough to be able to deal with varying formats? If we can assume that there are 4 lines, of which the first line contains floating point numbers, the second contains integers, the third contains one string and the fourth contains one floating point number, it's pretty easy. For example: def readinput(filename): with open(filename, 'rt') as f: lines = f.readlines() line0 = [float(part) for part in lines[0].split()] line1 = [int(part) for part in lines[1].split()] line2 = lines[2].strip()[1:-1] line3 = float(lines[3]) return line0, line1, line2, line3 line0, line1, line2, line3 = readinput('input.txt') print(line0) print(line1) print(line2) print(line3) Output: [2.0, 12.657823, 1.823467e-05, 114.0, 0.0] [3, 4, 5, 9, 11] Lower 278.15 This basically splits the two first line by whitespace, then converts each part (or the whole line in case of the last two lines) into the desired data type. -- The saddest aspect of life right now is that science gathers knowledge faster than society gathers wisdom. -- Isaac Asimov Roel Schroeven From mscir at yahoo.com Sat Feb 20 22:52:34 2016 From: mscir at yahoo.com (Mike S) Date: Sat, 20 Feb 2016 19:52:34 -0800 Subject: Considering migrating to Python from Visual Basic 6 for engineering applications In-Reply-To: <2dd9429e-16db-451d-beb8-1248a7029f80@googlegroups.com> References: <90cc50d2-1ce5-4588-9bfd-a49d439f00dd@googlegroups.com> <2dd9429e-16db-451d-beb8-1248a7029f80@googlegroups.com> Message-ID: On 2/19/2016 8:58 PM, Denis Akhiyarov wrote: > On Wednesday, February 17, 2016 at 1:49:44 PM UTC-6, wrong.a... at gmail.com wrote: >> I am mostly getting positive feedback for Python. >> >> It seems Python is used more for web based applications. Is it equally fine for creating stand-alone *.exe's? Can the same code be compiled to run on Linux or Android or web-based? >> >> Is it possible to create GUI elements with a good IDE? Can they be defined like in Visual Basic with given sizes, fonts, visible/invisible, etc.? >> >> Is it easy to do matrix operations in Python? Or do I need to write subroutines like in Visual Basic? >> >> Could someone kindly tell me advantages and disadvantages of Python? Or any better options? I have like 40-50 VB Forms and may be around 20000 lines of code. It will be a task to learn a new language and translate/re-write that code. >> >> Thanks for your responses. > > I'm surprised that no one mentioned this tool called vb2py. > It looks outdated, but I actually used it successfully to convert vba code to python,> once all dependencies were installed correctly :) > http://vb2py.sourceforge.net/ > You can also understand how vb objects map to python objects. > vb2py has also minimal support for GUI conversion. > Someone has even forked it on github recently: > https://github.com/reingart/vb2py Thanks! I have a lot of VB6 and VB.NET code and I'm learning Python thinking about doing machine learning, this might come in handy for other programs! From steve at pearwood.info Sun Feb 21 02:10:05 2016 From: steve at pearwood.info (Steven D'Aprano) Date: Sun, 21 Feb 2016 18:10:05 +1100 Subject: I can't install python on my pc References: Message-ID: <56c962cf$0$1592$c3e8da3$5496439d@news.astraweb.com> On Sat, 20 Feb 2016 06:34 pm, Giriprasadh Raghavan wrote: > > I can?t install python on my pc because it encounters an error every time > I try to install it. Please help as soon as possible. Sent from Mail for > Windows 10 Shall we guess what the error says? My guess is that you are out of disk space. You need to delete some files, or put in a bigger hard drive. -- Steven From steve at pearwood.info Sun Feb 21 02:17:09 2016 From: steve at pearwood.info (Steven D'Aprano) Date: Sun, 21 Feb 2016 18:17:09 +1100 Subject: How the heck does async/await work in Python 3.5 References: <56c7d145$0$1597$c3e8da3$5496439d@news.astraweb.com> <9d968a2e-f23c-4c93-979d-43dfa610c343@googlegroups.com> <8760xj7uff.fsf@jester.gateway.pace.com> Message-ID: <56c96477$0$1606$c3e8da3$5496439d@news.astraweb.com> On Sat, 20 Feb 2016 05:44 pm, Paul Rubin wrote: > But frankly the stuff I'm seeing in this thread makes me sad for Python. > It's an impossible dream but it would be awesome to have Erlang-like > communicating microtasks in Python. "But frankly the stuff I'm seeing in this thread makes me sad for *literally every programming language in existence except for Erlang and maybe one or two others*, which altogether about six people use in total..." :-) -- Steven From orgnut at yahoo.com Sun Feb 21 02:28:05 2016 From: orgnut at yahoo.com (Larry Hudson) Date: Sat, 20 Feb 2016 23:28:05 -0800 Subject: Considering migrating to Python from Visual Basic 6 for engineering applications In-Reply-To: <23d8156f-1808-4395-9c04-27d2984fe67c@googlegroups.com> References: <90cc50d2-1ce5-4588-9bfd-a49d439f00dd@googlegroups.com> <14c75a68-0d2e-45cc-8d73-0d71b6a6aea6@googlegroups.com> <9e57761f-26e1-41c5-8e71-23800de1fdd3@googlegroups.com> <7f9c473e-b0c2-4d77-91d1-d0733c93b12d@googlegroups.com> <23d8156f-1808-4395-9c04-27d2984fe67c@googlegroups.com> Message-ID: On 02/20/2016 10:38 AM, wrong.address.1 at gmail.com wrote: [snip] > How complicated could this get in Python? Reading the numbers is one thing, and then placing the values in text boxes of the GUI. If that is the only object of using these values, there is no conversions necessary. The data is read as text (strings), and you could simply write them directly into the text boxes. Of course, this is unlikely -- you surely want the actual numeric values for other purposes. As already pointed out, if you know in advance what the data types are, the necessary conversions are trivial. Just slightly more complex if you don't know the types in advance. OTOH, the conversion from int or float back to strings is also trivial. It's as simple as writing str(n), where n is an int OR a float -- it works automatically with either. (Data typing in Python is dynamic, you generally don't need to specify the types.) But if you want the strings to be in a specific format, this is also possible with a different syntax that lets you specify the output format. As an example, assume the variable val is 1.97834, and the formatting string is "${:.2f}".format(val) -- this will give you the string '$1.98'. This expression breaks down to: $ -> a literal dollar sign {} -> a placeholder, it is where the formatted data will be put : -> what follows is formatting code .2 -> round to, and print, two decimal places f -> the data is a float .format(val) -> the data to format BTW, this leaves the variable val unchanged -- it is NOT rounded, it still holds its original precision. It only affects how it is displayed. You CAN round it if you want, but that's an entirely different function. Naturally, learning all the formatting codes takes some effort, but it allows defining the appearance of the resulting strings in a very detailed and complete manner. [Aside: this is the "new" formatting method. Python also supports an "old" method, which is very much like the way strings are formatted in the C language.] > Or can I write my own reading subroutines which can then be called like > ReadVBstyle 8, ND, NIN, NT > to make the code more readable? ABSOLUTELY!! Most Python programming consists of defining the functions and classes needed, which definitely makes Python more readable. (Classes imply Object Oriented Programming. Python _allows_ OOP but does not _require_ it -- and is irrelevant here. Ignore anything I say about classes and OOP!) For your example here, it wouldn't _match_ the BASIC syntax, but would be used in an equivalent way. In fact, if you find yourself writing functions (or classes) that could be general-purpose routines, it is a trivial matter to put them into a personal library which you can then use in future programs. You don't need to rewrite them, just use them. Now, in practice, it takes a little care to write them as library functions. That is, the original version may rely on some details of the original program, so the library version will need to be tweaked a bit to remove these 'local' dependencies. But this is generally easily handled through the parameters to the functions. Also they are probably written with a bit more care to handle error conditions (which Python calls exceptions, Python has very extensive exception handling). This capability (a personal library) is enormously convenient. While I am saying 'personal', I really mean library(s) available to everyone involved in a programming project. No need for anyone to re-invent the wheel! ;-) Python calls them modules rather than libraries, but it's the same thing. -=- Larry -=- From no.email at nospam.invalid Sun Feb 21 02:34:49 2016 From: no.email at nospam.invalid (Paul Rubin) Date: Sat, 20 Feb 2016 23:34:49 -0800 Subject: How the heck does async/await work in Python 3.5 References: <56c7d145$0$1597$c3e8da3$5496439d@news.astraweb.com> <9d968a2e-f23c-4c93-979d-43dfa610c343@googlegroups.com> <8760xj7uff.fsf@jester.gateway.pace.com> <56c96477$0$1606$c3e8da3$5496439d@news.astraweb.com> Message-ID: <878u2e5xeu.fsf@jester.gateway.pace.com> Steven D'Aprano writes: > "But frankly the stuff I'm seeing in this thread makes me sad for > *literally every programming language in existence except for Erlang > and maybe one or two others*, which altogether about six people use in > total..." Erlang microtasks are more a matter of the runtime environment than the language. It might be possible to have something like it in PyPy. Other languages typically do concurrency with something like: 1. threads - various hazards that can be avoided if you're careful 2. callback chaining like in node.js - gets messy if the program is complicated, but conceptually simple 3. explicit state machines with something like libevent in C -- same as #2, simple but tedious 4. lightweight threads/tasks like in GHC and Erlang -- very nice though requires a sophisticated runtime system 5. cooperative multasking (small RTOS's, Forth, etc): simple, but apparently bug-prone when the program gets complicated Python's async stuff seems to combine various of the above approaches and (while I'm not saying it's objectively bad) the experiences I've had with it so far have been confusing and unpleasant. I do want to put some effort into understanding asyncio, but so far threads have worked ok for me. OK Web Server (uses a separate C++ process for each page on the site) looked straightforward and fast, though makes some architectural impositions. http://seastar-project.org/ seems interesting but I don't understand it at the moment. From cpoline95 at gmail.com Sun Feb 21 05:41:58 2016 From: cpoline95 at gmail.com (cpoline95 at gmail.com) Date: Sun, 21 Feb 2016 02:41:58 -0800 (PST) Subject: PyPDF2 merge / out of memory Message-ID: Hello, There is an issue with PyPDF2 and merging file https://github.com/mstamy2/PyPDF2/issues/189 Does anybody know an alternate library to merge PDF and produce optimized pdf file ? Thanks a lot Clement From breamoreboy at yahoo.co.uk Sun Feb 21 06:11:51 2016 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Sun, 21 Feb 2016 11:11:51 +0000 Subject: PyPDF2 merge / out of memory In-Reply-To: References: Message-ID: On 21/02/2016 10:41, cpoline95 at gmail.com wrote: > Hello, > > There is an issue with PyPDF2 and merging file > https://github.com/mstamy2/PyPDF2/issues/189 > > Does anybody know an alternate library to merge PDF and produce optimized pdf file ? > > Thanks a lot > > Clement > I did a bit of searching and found http://www.pdftron.com/pdfnet/features.html, which on pypi is described as:- A top notch PDF library for PDF rendering, conversion, content extraction, etc PDFNet SDK is an amazingly comprehensive, high-quality PDF developer toolkit for working with PDF files at all levels. Using the PDFNet PDF library, developers can create powerful PDF solutions and applications that can generate, manipulate, view, render and print PDF documents on Windows, Mac, and Linux. There's also https://bitbucket.org/vasudevram/xtopdf which states it's a This package (xtopdf) contains tools to convert various file formats (collectively called x) to PDF format. Vasudev Ram I know does a lot of work with PDF so a further search throws up articles on using his code, including some recipes on Activestate. Note I've used none of these myself, I just HTH. -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From bc at freeuk.com Sun Feb 21 08:16:07 2016 From: bc at freeuk.com (BartC) Date: Sun, 21 Feb 2016 13:16:07 +0000 Subject: Considering migrating to Python from Visual Basic 6 for engineering applications In-Reply-To: References: <90cc50d2-1ce5-4588-9bfd-a49d439f00dd@googlegroups.com> <14c75a68-0d2e-45cc-8d73-0d71b6a6aea6@googlegroups.com> <9e57761f-26e1-41c5-8e71-23800de1fdd3@googlegroups.com> <7f9c473e-b0c2-4d77-91d1-d0733c93b12d@googlegroups.com> <23d8156f-1808-4395-9c04-27d2984fe67c@googlegroups.com> Message-ID: On 21/02/2016 07:28, Larry Hudson wrote: > On 02/20/2016 10:38 AM, wrong.address.1 at gmail.com wrote: >> Or can I write my own reading subroutines which can then be called like >> ReadVBstyle 8, ND, NIN, NT >> to make the code more readable? > ABSOLUTELY!! Most Python programming consists of defining the functions > and classes needed, which definitely makes Python more readable. > No need for anyone to re-invent the > wheel! ;-) I keep seeing this in the thread. Python has all this capability, yet it still requires a lot of fiddly code to be added to get anywhere near as simple as this: read f, a, b, c And this is code that is not going to be obvious to anyone starting out. Even accepting that syntax limitations might require this to be written as: readline(f, a, b, c) I can't see a straightforward way of making this possible while still keeping a, b and c simple integer, float or string types (because Python's reference parameters don't work quite the right way). (There is also the question of 'readline' knowing what types of values to read. This information would not be needed in Fortran or Basic but somehow needs to be supplied here, if a particular set of types is to imposed on the input.) In other words, it seems this particular wheel does require re-inventing! -- Bartc From davidbenny2000 at gmail.com Sun Feb 21 08:34:50 2016 From: davidbenny2000 at gmail.com (davidbenny2000 at gmail.com) Date: Sun, 21 Feb 2016 05:34:50 -0800 (PST) Subject: how to get the list form dictionary's values Message-ID: <554c09f2-6a6b-4576-9c9d-50d6234c8583@googlegroups.com> File "mainpy.py", line 81 for functionlistelement in functionlist0 ^ SyntaxError: invalid syntax import asyncio def f000(): try: print "000" except: print "000 exception" def f001(): try: print "001" except: print "001 exception" def f002(): try: print "002" except: print "002 exception" def f003(): try: print "003" except: print "003 exception" def f004(): try: print "004" except: print "004 exception" machine = {} mappedfunc = {} functionlist000 = [] functionlist001 = [] functionlist002 = [] functionlist003 = [] functionlist004 = [] functionlist000.append('002') functionlist001.append('000') functionlist002.append('003') functionlist002.append('004') functionlist003.append('001') functionlist004.append('001') machine000 = {'000': functionlist000} machine001 = {'001': functionlist001} machine002 = {'002': functionlist002} machine003 = {'003': functionlist003} machine004 = {'004': functionlist004} machine.update(machine000) machine.update(machine001) machine.update(machine002) machine.update(machine003) machine.update(machine004) functionkey000 = {'000': f000 } functionkey001 = {'001': f001 } functionkey002 = {'002': f002 } functionkey002 = {'003': f003 } functionkey002 = {'004': f004 } mappedfunc.update(functionkey000) mappedfunc.update(functionkey001) mappedfunc.update(functionkey002) mappedfunc.update(functionkey003) mappedfunc.update(functionkey004) def workingthreadpool(currentnumber1, machine1): try: functionlist0 = machine1.get(currentnumber1) loop = asyncio.get_event_loop() tesks = [] j = 1 for functionlistelement in functionlist0 tesks.append(asyncio.ensure_future(mappedfunc.get(functionlistelement)())) if j > 1: workingthreadpool(functionlistelement) j = j + 1 loop.run_until_complete(asyncio.wait(tesks)) loop.close() except: print "workingthreadpool exception" currentnumber = "000" workingthreadpool(currentnumber, machine) From rosuav at gmail.com Sun Feb 21 08:54:06 2016 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 22 Feb 2016 00:54:06 +1100 Subject: Considering migrating to Python from Visual Basic 6 for engineering applications In-Reply-To: References: <90cc50d2-1ce5-4588-9bfd-a49d439f00dd@googlegroups.com> <14c75a68-0d2e-45cc-8d73-0d71b6a6aea6@googlegroups.com> <9e57761f-26e1-41c5-8e71-23800de1fdd3@googlegroups.com> <7f9c473e-b0c2-4d77-91d1-d0733c93b12d@googlegroups.com> <23d8156f-1808-4395-9c04-27d2984fe67c@googlegroups.com> Message-ID: On Mon, Feb 22, 2016 at 12:16 AM, BartC wrote: > On 21/02/2016 07:28, Larry Hudson wrote: >> >> On 02/20/2016 10:38 AM, wrong.address.1 at gmail.com wrote: > > >>> Or can I write my own reading subroutines which can then be called like >>> ReadVBstyle 8, ND, NIN, NT >>> to make the code more readable? > > >> ABSOLUTELY!! Most Python programming consists of defining the functions >> and classes needed, which definitely makes Python more readable. > > >> No need for anyone to re-invent the >> wheel! ;-) > > > I keep seeing this in the thread. Python has all this capability, yet it > still requires a lot of fiddly code to be added to get anywhere near as > simple as this: > > read f, a, b, c > > And this is code that is not going to be obvious to anyone starting out. > Even accepting that syntax limitations might require this to be written as: > > readline(f, a, b, c) > > I can't see a straightforward way of making this possible while still > keeping a, b and c simple integer, float or string types (because Python's > reference parameters don't work quite the right way). How about: a, b, c = readline(f) > (There is also the question of 'readline' knowing what types of values to > read. This information would not be needed in Fortran or Basic but somehow > needs to be supplied here, if a particular set of types is to imposed on the > input.) You'd still need to solve that, one way or another. But the Pythonic way is to distinguish between "input" and "output" parameters by having the input parameters as parameters, and the output parameters as the return value. And personally, I find it *way* more readable that way; while it's perfectly possible to pass a list as a parameter and have the function append to it, it's much clearer if your return values are on the left of the equals sign. ChrisA From rosuav at gmail.com Sun Feb 21 09:04:06 2016 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 22 Feb 2016 01:04:06 +1100 Subject: how to get the list form dictionary's values In-Reply-To: <554c09f2-6a6b-4576-9c9d-50d6234c8583@googlegroups.com> References: <554c09f2-6a6b-4576-9c9d-50d6234c8583@googlegroups.com> Message-ID: On Mon, Feb 22, 2016 at 12:34 AM, wrote: > File "mainpy.py", line 81 > for functionlistelement in functionlist0 > ^ > SyntaxError: invalid syntax Your actual syntax problem here is simply a missing colon. That's easily fixed. But please, PLEASE, don't do this: > def workingthreadpool(currentnumber1, machine1): > try: > functionlist0 = machine1.get(currentnumber1) > > loop = asyncio.get_event_loop() > tesks = [] > > j = 1 > for functionlistelement in functionlist0 > tesks.append(asyncio.ensure_future(mappedfunc.get(functionlistelement)())) > if j > 1: > workingthreadpool(functionlistelement) > j = j + 1 > > loop.run_until_complete(asyncio.wait(tesks)) > loop.close() > except: > print "workingthreadpool exception" 1) Indenting by a single space makes it hard to skim and see what's at what indentation level. Indent by at least four spaces (I've seen two, and it's still very narrow), or one tab. 2) Carrying a counter through a 'for' loop is much better done with enumerate(). But the counter's sole purpose is to ignore the first element, so you might want to find a completely different approach. 3) Why does this recurse anyway? Your function appends one future to 'tesks' (should that be 'tasks'?), then appends another... and then calls itself recursively. What's that doing, exactly? 4) Bare except clauses are almost always a bad idea. In fact, if a future version of Python raises a SyntaxWarning or even SyntaxError on the bare except, I would see it as no loss. As of Python 2.7, it's possible to raise an old-style class, which then won't be caught by "except Exception:" or even "except BaseException:", but I'm pretty sure there's nothing you can raise in Python 3 that isn't caught by "except BaseException:". Proper exception handling generally means either (a) catching a specific exception or list of exceptions, and handling them; or (b) catching everything, logging them in some way, and returning to some sort of main loop. Your code is simply reporting the presence of an exception, nothing more, and then bailing out. Let those exceptions bubble up to top level and be printed out! 5) Asynchronous code is way better supported in Python 3.5 than in 2.7. I strongly recommend upgrading your Python. These considerations are less simple than the syntax issue you were asking about, but they're also non-trivial problems. You might get your code to _run_, but can you get it to do the right thing, even after you tinker with it later on? Good code saves you time in the long run. ChrisA From __peter__ at web.de Sun Feb 21 09:06:55 2016 From: __peter__ at web.de (Peter Otten) Date: Sun, 21 Feb 2016 15:06:55 +0100 Subject: how to get the list form dictionary's values References: <554c09f2-6a6b-4576-9c9d-50d6234c8583@googlegroups.com> Message-ID: davidbenny2000 at gmail.com wrote: > File "mainpy.py", line 81 > for functionlistelement in functionlist0 > ^ > SyntaxError: invalid syntax > > > import asyncio [snip] > mappedfunc = {} > functionlist000 = [] > functionlist001 = [] > functionlist002 = [] > functionlist003 = [] > functionlist004 = [] [snip many names with numeric suffix and repeated method calls] I have no idea what you are trying to do; however, your code looks awfully redundant. This is errorprone: functionkey000 = {'000': f000 } functionkey001 = {'001': f001 } functionkey002 = {'002': f002 } functionkey002 = {'003': f003 } functionkey002 = {'004': f004 } > for functionlistelement in functionlist0 To answer what seems to be your actual question: a colon is missing at the end of this line. From davidbenny2000 at gmail.com Sun Feb 21 09:15:02 2016 From: davidbenny2000 at gmail.com (Ho Yeung Lee) Date: Sun, 21 Feb 2016 06:15:02 -0800 (PST) Subject: how to get the list form dictionary's values In-Reply-To: References: <554c09f2-6a6b-4576-9c9d-50d6234c8583@googlegroups.com> Message-ID: <813ad31d-b208-496f-91b0-5f39aef28965@googlegroups.com> Hi Chris, 0 ---> 2 --> 3--> 1 ---> 0 ---> 4 / i am practicing task flow in this graph situation when current state is 2 , there are 3 and 4 to run parallel and wait list of tasks finish before running to 1 , however i feel that my code has been wrong because 3 and 4 can not combine to run task 1 , which is the correct way to combine to run task 1? or is this graph situation an example impossible to do in computer science? Regards, Martin Lee Chris Angelico? 2016?2?21???? UTC+8??10?04?22???? > On Mon, Feb 22, 2016 at 12:34 AM, wrote: > > File "mainpy.py", line 81 > > for functionlistelement in functionlist0 > > ^ > > SyntaxError: invalid syntax > > Your actual syntax problem here is simply a missing colon. That's > easily fixed. But please, PLEASE, don't do this: > > > def workingthreadpool(currentnumber1, machine1): > > try: > > functionlist0 = machine1.get(currentnumber1) > > > > loop = asyncio.get_event_loop() > > tesks = [] > > > > j = 1 > > for functionlistelement in functionlist0 > > tesks.append(asyncio.ensure_future(mappedfunc.get(functionlistelement)())) > > if j > 1: > > workingthreadpool(functionlistelement) > > j = j + 1 > > > > loop.run_until_complete(asyncio.wait(tesks)) > > loop.close() > > except: > > print "workingthreadpool exception" > > 1) Indenting by a single space makes it hard to skim and see what's at > what indentation level. Indent by at least four spaces (I've seen two, > and it's still very narrow), or one tab. > > 2) Carrying a counter through a 'for' loop is much better done with > enumerate(). But the counter's sole purpose is to ignore the first > element, so you might want to find a completely different approach. > > 3) Why does this recurse anyway? Your function appends one future to > 'tesks' (should that be 'tasks'?), then appends another... and then > calls itself recursively. What's that doing, exactly? > > 4) Bare except clauses are almost always a bad idea. In fact, if a > future version of Python raises a SyntaxWarning or even SyntaxError on > the bare except, I would see it as no loss. As of Python 2.7, it's > possible to raise an old-style class, which then won't be caught by > "except Exception:" or even "except BaseException:", but I'm pretty > sure there's nothing you can raise in Python 3 that isn't caught by > "except BaseException:". Proper exception handling generally means > either (a) catching a specific exception or list of exceptions, and > handling them; or (b) catching everything, logging them in some way, > and returning to some sort of main loop. Your code is simply reporting > the presence of an exception, nothing more, and then bailing out. Let > those exceptions bubble up to top level and be printed out! > > 5) Asynchronous code is way better supported in Python 3.5 than in > 2.7. I strongly recommend upgrading your Python. > > These considerations are less simple than the syntax issue you were > asking about, but they're also non-trivial problems. You might get > your code to _run_, but can you get it to do the right thing, even > after you tinker with it later on? Good code saves you time in the > long run. > > ChrisA From davidbenny2000 at gmail.com Sun Feb 21 09:21:34 2016 From: davidbenny2000 at gmail.com (Ho Yeung Lee) Date: Sun, 21 Feb 2016 06:21:34 -0800 (PST) Subject: how to get the list form dictionary's values In-Reply-To: References: <554c09f2-6a6b-4576-9c9d-50d6234c8583@googlegroups.com> Message-ID: Peter Otten? 2016?2?21???? UTC+8??10?07?18???? > davidbenny2000 at gmail.com wrote: > > > File "mainpy.py", line 81 > > for functionlistelement in functionlist0 > > ^ > > SyntaxError: invalid syntax > > > > > > import asyncio > > [snip] > > > mappedfunc = {} > > functionlist000 = [] > > functionlist001 = [] > > functionlist002 = [] > > functionlist003 = [] > > functionlist004 = [] > > [snip many names with numeric suffix and repeated method calls] > > I have no idea what you are trying to do; however, your code looks awfully > redundant. This is errorprone: > > functionkey000 = {'000': f000 } > functionkey001 = {'001': f001 } > functionkey002 = {'002': f002 } > functionkey002 = {'003': f003 } > functionkey002 = {'004': f004 } > > > for functionlistelement in functionlist0 > > To answer what seems to be your actual question: a colon is missing at the > end of this line. hi Peter, i am running this task flow in this graph, these code are not redundant, because this writing can clearly show the work flow https://drive.google.com/file/d/0B7fHc_dTzkY_OGMtTGI2UnR6ZEE/view?usp=sharing Regards, Martin Lee From rosuav at gmail.com Sun Feb 21 10:01:37 2016 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 22 Feb 2016 02:01:37 +1100 Subject: how to get the list form dictionary's values In-Reply-To: <813ad31d-b208-496f-91b0-5f39aef28965@googlegroups.com> References: <554c09f2-6a6b-4576-9c9d-50d6234c8583@googlegroups.com> <813ad31d-b208-496f-91b0-5f39aef28965@googlegroups.com> Message-ID: On Mon, Feb 22, 2016 at 1:15 AM, Ho Yeung Lee wrote: > Hi Chris, > > 0 ---> 2 --> 3--> 1 ---> 0 > ---> 4 / > > i am practicing task flow in this graph situation > > when current state is 2 , there are 3 and 4 to run parallel and wait list of tasks finish before running to 1 , > > however i feel that my code has been wrong because 3 and 4 can not combine to run task 1 , > > which is the correct way to combine to run task 1? So after task 2 completes, tasks 3 and 4 both become available (and can run in parallel), but then task 1 must wait until both have finished before it runs? I'm not an asyncio expert, but this sounds like a fairly straight-forward dependency requirement. If you were to implement these as four separate threads, you would have threads 3 and 4 raise semaphores which thread 1 blocks on; I'm sure there'll be an equivalent in asyncio. One way to do it would be something like this: import asyncio loop = asyncio.get_event_loop() async def starter(): print("Starting!") global t1, t2 t1 = asyncio.Task(dep1()) t2 = asyncio.Task(dep2()) print("Done!") await dependent() async def dep1(): print("Dependency 1") await asyncio.sleep(3) print("Dep 1 done.") async def dep2(): print("Dependency 2") await asyncio.sleep(2) print("Dep 2 done.") async def dependent(): await t1 await t2 print("Deps are done - dependent can run.") loop.run_until_complete(starter()) The dependent task simply waits for each of its dependencies in turn. It doesn't matter which one finishes first; it won't do its main work until both are done. And if your starter and dependent are logically connected, you can just have the 'await' lines in the middle of one tidy function. (Can an asyncio expert tell me how to tidy this code up, please? I'm sure this isn't the cleanest.) ChrisA From jussi.piitulainen at helsinki.fi Sun Feb 21 10:08:49 2016 From: jussi.piitulainen at helsinki.fi (Jussi Piitulainen) Date: Sun, 21 Feb 2016 17:08:49 +0200 Subject: Considering migrating to Python from Visual Basic 6 for engineering applications References: <90cc50d2-1ce5-4588-9bfd-a49d439f00dd@googlegroups.com> <14c75a68-0d2e-45cc-8d73-0d71b6a6aea6@googlegroups.com> <9e57761f-26e1-41c5-8e71-23800de1fdd3@googlegroups.com> <7f9c473e-b0c2-4d77-91d1-d0733c93b12d@googlegroups.com> <23d8156f-1808-4395-9c04-27d2984fe67c@googlegroups.com> Message-ID: BartC writes: > On 21/02/2016 07:28, Larry Hudson wrote: >> On 02/20/2016 10:38 AM, wrong.address.1 at gmail.com wrote: > >>> Or can I write my own reading subroutines which can then be called like >>> ReadVBstyle 8, ND, NIN, NT >>> to make the code more readable? > >> ABSOLUTELY!! Most Python programming consists of defining the functions >> and classes needed, which definitely makes Python more readable. > >> No need for anyone to re-invent the >> wheel! ;-) > > I keep seeing this in the thread. Python has all this capability, yet > it still requires a lot of fiddly code to be added to get anywhere > near as simple as this: > > read f, a, b, c > > And this is code that is not going to be obvious to anyone starting > out. Even accepting that syntax limitations might require this to be > written as: > > readline(f, a, b, c) > > I can't see a straightforward way of making this possible while still > keeping a, b and c simple integer, float or string types (because > Python's reference parameters don't work quite the right way). > > (There is also the question of 'readline' knowing what types of values > to read. This information would not be needed in Fortran or Basic but > somehow needs to be supplied here, if a particular set of types is to > imposed on the input.) > > In other words, it seems this particular wheel does require > re-inventing! It's hardly Python's problem if an engineer is worried about some VB not being there in its current form in the future. The sample code upthread seemed gibberish to me - anybody capable of learning *that* should be *able* to also learn Python, and then the input parsing business is not only simple but also flexible, as already demonstrated by others. The following assumes that the author of the code knows the desired types, and does not assume a working DWIM feature. I wouldn't name things quite like that in real code. from io import StringIO def funcall(f, x): return f(x) def Input(source, *dims): return tuple(map(funcall, dims, next(source).split())) def Vector(dim, times): return tuple([dim] * times) N020SdotTXT = ''' 10 6 1 8.65 0.2192347 3.33E-4 44 0.0051 6 9 1 2 1 3 ''' Number8 = StringIO(N020SdotTXT) ND, NIN, NT = Input(Number8, int, int, int) DINCOL = Input(Number8, *Vector(float, NIN)) [NINE] = Input(Number8, int) [ONE] = Input(Number8, float) TWO = Input(Number8, float, int) [THREE] = Input(Number8, str) Hash = dict print('ND = {}, NIN = {}, NT = {}'.format(ND, NIN, NT), DINCOL, Hash(NINE = NINE, ONE = ONE, TWO = TWO, THREE = THREE), sep = '\n') # Output: # ND = 10, NIN = 6, NT = 1 # (8.65, 0.2192347, 0.000333, 44.0, 0.0051, 6.0) # {'NINE': 9, 'TWO': (2.0, 1), 'ONE': 1.0, 'THREE': '3'} /s From marko at pacujo.net Sun Feb 21 10:16:05 2016 From: marko at pacujo.net (Marko Rauhamaa) Date: Sun, 21 Feb 2016 17:16:05 +0200 Subject: how to get the list form dictionary's values References: <554c09f2-6a6b-4576-9c9d-50d6234c8583@googlegroups.com> <813ad31d-b208-496f-91b0-5f39aef28965@googlegroups.com> Message-ID: <87vb5inlfu.fsf@elektro.pacujo.net> Chris Angelico : > On Mon, Feb 22, 2016 at 1:15 AM, Ho Yeung Lee wrote: >> 0 ---> 2 --> 3--> 1 ---> 0 >> ---> 4 / > > So after task 2 completes, tasks 3 and 4 both become available (and > can run in parallel), but then task 1 must wait until both have > finished before it runs? I'm not an asyncio expert, but this sounds > like a fairly straight-forward dependency requirement. If you were to > implement these as four separate threads, you would have threads 3 and > 4 raise semaphores which thread 1 blocks on; I'm sure there'll be an > equivalent in asyncio. > > [...] > > (Can an asyncio expert tell me how to tidy this code up, please? I'm > sure this isn't the cleanest.) Not an expert, but this does the trick: ======================================================================== #!/usr/bin/env python3 import asyncio def main(): loop = asyncio.get_event_loop() loop.run_until_complete(t0()) async def t0(): print("start t0") await t2() await asyncio.wait([ t3(), t4() ]) await t1() print("finish t0") async def t1(): print("start t1") await asyncio.sleep(1) print("finish t1") async def t2(): print("start t2") await asyncio.sleep(1) print("finish t2") async def t3(): print("start t3") await asyncio.sleep(1) print("finish t3") async def t4(): print("start t4") await asyncio.sleep(1) print("finish t4") if __name__ == '__main__': main() ======================================================================== Marko From auriocus at gmx.de Sun Feb 21 10:21:29 2016 From: auriocus at gmx.de (Christian Gollwitzer) Date: Sun, 21 Feb 2016 16:21:29 +0100 Subject: Considering migrating to Python from Visual Basic 6 for engineering applications In-Reply-To: References: <90cc50d2-1ce5-4588-9bfd-a49d439f00dd@googlegroups.com> <14c75a68-0d2e-45cc-8d73-0d71b6a6aea6@googlegroups.com> <9e57761f-26e1-41c5-8e71-23800de1fdd3@googlegroups.com> <7f9c473e-b0c2-4d77-91d1-d0733c93b12d@googlegroups.com> <23d8156f-1808-4395-9c04-27d2984fe67c@googlegroups.com> Message-ID: Am 21.02.16 um 14:16 schrieb BartC: > Even accepting that syntax limitations might require this to be written as: > > readline(f, a, b, c) > > I can't see a straightforward way of making this possible while still > keeping a, b and c simple integer, float or string types (because > Python's reference parameters don't work quite the right way). > > (There is also the question of 'readline' knowing what types of values > to read. This information would not be needed in Fortran or Basic but > somehow needs to be supplied here, if a particular set of types is to > imposed on the input.) Are you sure that in Basic or Fortran the expected type is not supplied? I'm not too familiar with either, but I think that in Fortran the compiler deduces it from the (compile-time) static type of the variable, while in BASIC there used to be sigils (A$, A# etc.) to denote the type. A pythonic input function would look like this IMHO: a,b,c = readline(f, int, float, str) > In other words, it seems this particular wheel does require re-inventing! Yes, but the above seems quite trivial: Apfelkiste:Tests chris$ cat parseline.py def readline(f, *args): line=f.readline().split() return [type(x) for type,x in zip(args,line)] with open("mydata.dat", "r") as f: ND, NINT, NT=readline(f, int, int, int) # next line holds NINT floats dincol=readline(f, *NINT*[float]) # next line holds a string text=f.readline() print("Read: ", ND, NINT, NT) print(dincol) print(text) Apfelkiste:Tests chris$ cat mydata.dat 10 6 1 8.65 0.2192347 3.33E-4 44 0.0051 6 String Apfelkiste:Tests chris$ python parseline.py ('Read: ', 10, 6, 1) [8.65, 0.2192347, 0.000333, 44.0, 0.0051, 6.0] String Apfelkiste:Tests chris$ Christian From xapwing at gmail.com Sun Feb 21 10:43:30 2016 From: xapwing at gmail.com (Arie van Wingerden) Date: Sun, 21 Feb 2016 07:43:30 -0800 (PST) Subject: PyQt5 not found from Python Message-ID: <0df21c16-e39b-4711-b675-934ad98da61e@googlegroups.com> Python 3.5.1 installed in default Windows folder here: C:\Users\Arie\AppData\Local\Programs\Python\Python35-32 PyQt5 installed - it creates a folder c:\Python34 In the Python REPL i try: from PyQt5 import QtCore Traceback (most recent call last): File "", line 1, in ImportError: No module named 'PyQt5' So i guess Python does not see the QT stuff. What can i do about it? TIA From skip.montanaro at gmail.com Sun Feb 21 10:51:26 2016 From: skip.montanaro at gmail.com (Skip Montanaro) Date: Sun, 21 Feb 2016 09:51:26 -0600 Subject: Setting up/authenticating Google API? Message-ID: This isn't strictly a Python question, however... Once I get myself authenticated, I intend to use the Python Google API to pump archived mail messages from a few defunct mailing lists into Google Groups. I thought it would be pretty straightforward, but my attempts a few months ago were completely unsuccessful. I asked for help on the relevant Google product forum but got zero assistance. I'd really appreciate a little help with the setup. If anyone has had successful experience with this, please contact me off-list. Once I get something going, I'll write a blog post so others don't have the same problems I have. Thanks, Skip Montanaro From ganesh1pal at gmail.com Sun Feb 21 10:57:31 2016 From: ganesh1pal at gmail.com (Ganesh Pal) Date: Sun, 21 Feb 2016 21:27:31 +0530 Subject: Python unittest2.SkipTest and general suggestion Message-ID: Hello team, Please provide your guidance on how to proceed with the below test , Iam on python 2.6 and Linux. I have a linitation to use python 2.6 and unittest2 try: import unittest2 as unittest except ImportError: import unittest class isiCorruptTest(unittest.TestCase): corrupt_baddr = {} report= "" @classmethod def setUpClass(cls): cls.corruptSetup() @classmethod def corruptSetup(cls): """ Initial setup before unittest run """ logging.info("SETUP.....Started !!!") try: logging.info("Capturing data as part of prechecks for test_xx") capture_data() except Exception as e: logging.error(e) sys.exit("Corruption injection failed...Exiting !!!") try: corrupt.run_scan() except Exception as e: logging.error(e) raise unittest.SkipTest("Failure running Integrity Scan ") try: cls.report = corrupt.run_report() except Exception as e: logging.error(e) raise unittest.SkipTest("Failure running Reporting Tool ") #sys.exit(1) logging.info("SETUP.....Done !!!") def test_04_inode(self): """ test04: """ logging.info("### Executing test04: ###") self.assertTrue(corrupt.run_query(self.__class__.report, self.corrupt_baddr['test04'])) def main(): """ ---MAIN--- """ # both function are not shown in this code functions = [create_logdir, create_dataset ] for func in functions: try: func() except Exception as e: logging.error(e) return False unittest.main() if __name__ == '__main__': main() I have the below problems while running this code: 1. unittest.SkipTest does not the display the exception message that is caught. Example : The function corrupt.run_scan() returns none and False for failures 3. how do I the ensure that test_04 is run only it setUpClass succeeds ? Example: Iam getting this error message for each tests on the console. ====================================================================== ERROR: test_04_inode (__main__.isiCorruptTest) test04: inode offset, size corruption ---------------------------------------------------------------------- Traceback (most recent call last): File "c_t1.py", line 251, in test_04_inode_offset_size_corruption self.corrupt_baddr['test04'])) KeyError: 'test04' 3. Any other suggestion welcome Regards, Ganesh From ganesh1pal at gmail.com Sun Feb 21 11:12:18 2016 From: ganesh1pal at gmail.com (Ganesh Pal) Date: Sun, 21 Feb 2016 21:42:18 +0530 Subject: can try expect have if else. Message-ID: Hi Team, Iam on python 2.6 , need input on the below piece of code. EXIT_STATUS_ERROR=1 def create_dataset(): """ """ logging.info("Dataset create.....Started !!!") try: if os.path.ismount("/nfs_mount"): touch_file("inode_fixcrc.txt") logging.info("Dataset create.....Done !!!") else: raise Exception("/nfs_mount is not mounted. Dataset create failed !!!") return False except Exception as e: logging.error(e) sys.stderr.write("Dataset create failed...Exiting !!!") sys.exit(EXIT_STATUS_ERROR) return True 1. Can we have if else with in a try except block 2. How can the above code be improved Regards, Ganesh From bc at freeuk.com Sun Feb 21 11:16:28 2016 From: bc at freeuk.com (BartC) Date: Sun, 21 Feb 2016 16:16:28 +0000 Subject: Considering migrating to Python from Visual Basic 6 for engineering applications In-Reply-To: References: <90cc50d2-1ce5-4588-9bfd-a49d439f00dd@googlegroups.com> <14c75a68-0d2e-45cc-8d73-0d71b6a6aea6@googlegroups.com> <9e57761f-26e1-41c5-8e71-23800de1fdd3@googlegroups.com> <7f9c473e-b0c2-4d77-91d1-d0733c93b12d@googlegroups.com> <23d8156f-1808-4395-9c04-27d2984fe67c@googlegroups.com> Message-ID: On 21/02/2016 15:08, Jussi Piitulainen wrote: > BartC writes: >> In other words, it seems this particular wheel does require >> re-inventing! > > It's hardly Python's problem if an engineer is worried about some VB not > being there in its current form in the future. The sample code upthread > seemed gibberish to me I don't know VB6 but: Open "N020S.TXT" For Input As #8 Input #8, ND, NIN, NT ' all integers seems clear enough: read 3 integers from the text file just opened on channel 8, and store them in variables ND, NIN and NT. But this is not so much to do with VB6, but with a very fundamental capability that seems missing in modern languages. IIRC, the first programming exercise I ever did (in 1976 using Algol 60) involved reading 3 numbers from the teletype and working out if those could form sides of a triangle. It might have started off like this (I can't remember Algol 60): print "Type 3 numbers:" read a, b, c In Basic, that last line might be: input a, b, c instead (reading here as floats probably). Now, nearly 40 years later, I don't actually know how to do that in Python! Sure, I can cobble something together, with all sorts of functions and string operations (and I would need to go and look them up). But it's not just /there/ already. If you gave this a Python exercise to a class of students, you'd end up with 30 different solutions just for the first, easy part of the exercise! In fact it would be far more challenging than the triangle problem. That can't be right. -- Bartc From phil at riverbankcomputing.com Sun Feb 21 11:25:31 2016 From: phil at riverbankcomputing.com (Phil Thompson) Date: Sun, 21 Feb 2016 16:25:31 +0000 Subject: PyQt5 not found from Python In-Reply-To: <0df21c16-e39b-4711-b675-934ad98da61e@googlegroups.com> References: <0df21c16-e39b-4711-b675-934ad98da61e@googlegroups.com> Message-ID: On 21/02/2016 3:43 pm, Arie van Wingerden wrote: > Python 3.5.1 installed in default Windows folder here: > C:\Users\Arie\AppData\Local\Programs\Python\Python35-32 > > PyQt5 installed - it creates a folder c:\Python34 > > In the Python REPL i try: > from PyQt5 import QtCore > Traceback (most recent call last): > File "", line 1, in > ImportError: No module named 'PyQt5' > > So i guess Python does not see the QT stuff. > > What can i do about it? > > TIA There are no PyQt installers for Python 3.5 yet. There will be when PyQt v5.6 is released. Phil From breamoreboy at yahoo.co.uk Sun Feb 21 11:27:22 2016 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Sun, 21 Feb 2016 16:27:22 +0000 Subject: PyQt5 not found from Python In-Reply-To: <0df21c16-e39b-4711-b675-934ad98da61e@googlegroups.com> References: <0df21c16-e39b-4711-b675-934ad98da61e@googlegroups.com> Message-ID: On 21/02/2016 15:43, Arie van Wingerden wrote: > Python 3.5.1 installed in default Windows folder here: > C:\Users\Arie\AppData\Local\Programs\Python\Python35-32 > > PyQt5 installed - it creates a folder c:\Python34 > > In the Python REPL i try: > from PyQt5 import QtCore > Traceback (most recent call last): > File "", line 1, in > ImportError: No module named 'PyQt5' > > So i guess Python does not see the QT stuff. > > What can i do about it? > > TIA > There isn't a PyQt5 for Python 3.5 as I discovered about six hours ago when I did exactly what you've just done :) Assuming that you get a version of PyQt that matches with a Python version, I'd assume that you'd drop it into the appropriate lib\site-packages, but I'll happily stand corrected on that one. -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From xapwing at gmail.com Sun Feb 21 11:40:07 2016 From: xapwing at gmail.com (Arie van Wingerden) Date: Sun, 21 Feb 2016 08:40:07 -0800 (PST) Subject: PyQt5 not found from Python In-Reply-To: <0df21c16-e39b-4711-b675-934ad98da61e@googlegroups.com> References: <0df21c16-e39b-4711-b675-934ad98da61e@googlegroups.com> Message-ID: Thx. Didn't realize that! From breamoreboy at yahoo.co.uk Sun Feb 21 11:48:45 2016 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Sun, 21 Feb 2016 16:48:45 +0000 Subject: PyQt5 not found from Python In-Reply-To: References: <0df21c16-e39b-4711-b675-934ad98da61e@googlegroups.com> Message-ID: On 21/02/2016 16:40, Arie van Wingerden wrote: > Thx. Didn't realize that! > Please provide some context when you reply, thanks. -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From ben+python at benfinney.id.au Sun Feb 21 12:03:06 2016 From: ben+python at benfinney.id.au (Ben Finney) Date: Mon, 22 Feb 2016 04:03:06 +1100 Subject: Python unittest2.SkipTest and general suggestion References: Message-ID: <85d1rq0zed.fsf@benfinney.id.au> Ganesh Pal writes: > 1. unittest.SkipTest does not the display the exception message that > is caught. You are already supplying a custom message to ?self.skipTest?:: except Exception as exc: logging.error(exc) raise unittest.SkipTest("Failure running Integrity Scan ") So you can change that message by including the text representation of the exception object:: except Exception as exc: logging.error(exc) raise unittest.SkipTest( "Failure running Integrity Scan: {exc}".format(exc=exc)) > 3. how do I the ensure that test_04 is run only it setUpClass succeeds > ? If setupClass fails, why would you want to continue? Handle the error inside the code for ?setUpClass?. If that's not obvious, maybe you mean something different by ?only if setUpClass succeeds?. > 3. Any other suggestion welcome Please try hard to migrate to a currently-supported Python version. The Python Software Foundation no longer supports Python 2.6, and that translates into a justifiable desire in the Python community to also stop supporting it. -- \ ?We live in capitalism. Its power seems inescapable. So did the | `\ divine right of kings.? ?Ursula K. LeGuin, National Book Awards | _o__) acceptance speech, 2014-11-19 | Ben Finney From ben+python at benfinney.id.au Sun Feb 21 12:07:05 2016 From: ben+python at benfinney.id.au (Ben Finney) Date: Mon, 22 Feb 2016 04:07:05 +1100 Subject: can try expect have if else. References: Message-ID: <858u2e0z7q.fsf@benfinney.id.au> Ganesh Pal writes: > 1. Can we have if else with in a try except block What result do you get when running that code? What empirical reason do you have to think it would work or not work? > 2. How can the above code be improved The following sequence of statements:: raise Exception("/nfs_mount is not mounted. Dataset create failed !!!") return False do not make sense. The ?return? statement will never be reached. So your intent must be something other than what is expressed by that code. -- \ ?Do unto others twenty-five percent better than you expect them | `\ to do unto you. (The twenty-five percent is [to correct] for | _o__) error.)? ?Linus Pauling's Golden Rule | Ben Finney From k.d.jantzen at mailbox.org Sun Feb 21 12:11:18 2016 From: k.d.jantzen at mailbox.org (Klaus Jantzen) Date: Sun, 21 Feb 2016 18:11:18 +0100 Subject: SQLite Message-ID: <56C9EFB6.1090309@mailbox.org> Hello, I have downloaded Python3.5.1 as .targz, compiled it(configure, make,...) and it works (under Debian Wheezy AMD64) up to the moment I wanted to use SQLite. I get the following message: === jantzen at PC4:~$ python Python 3.5.0 (default, Dec? 2 2015, 14:16:16) [GCC 4.7.2] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import sqlite3 Traceback (most recent call last): ? File "", line 1, in ? File "/usr/local/lib/python3.5/sqlite3/__init__.py", line 23, in ??? from sqlite3.dbapi2 import * ? File "/usr/local/lib/python3.5/sqlite3/dbapi2.py", line 27, in ??? from _sqlite3 import * ImportError: No module named '_sqlite3' === Obviously something is missing. How do I solve the problem? Where do I find this module? Thanks for a hint. -- K.D.J. From mustang at NOMAIL.it Sun Feb 21 12:36:25 2016 From: mustang at NOMAIL.it (mustang) Date: Sun, 21 Feb 2016 18:36:25 +0100 Subject: show instant data on webpage References: Message-ID: > You need free and private? The price is low -- if this is a business why > not pay? If not a business, why private? it's not business it's like a school project, so no money :D > There is another charting library called pygal which is great. It can > produce output to .svg file format or .png. You could distribute the files > which will display in a browser. Or you could build them into web pages if > you are so inclined to build a website. It seems that isn't what you want > to do. You could also upload images to some free image site. Photobucket > supports .png images. pygal can create .png images > > > ok thanks. From ganesh1pal at gmail.com Sun Feb 21 12:39:30 2016 From: ganesh1pal at gmail.com (Ganesh Pal) Date: Sun, 21 Feb 2016 23:09:30 +0530 Subject: can try expect have if else. In-Reply-To: <858u2e0z7q.fsf@benfinney.id.au> References: <858u2e0z7q.fsf@benfinney.id.au> Message-ID: On Sun, Feb 21, 2016 at 10:37 PM, Ben Finney wrote: > What result do you get when running that code? What empirical reason do > you have to think it would work or not work? I wanted to know was is it good to have if else with in a try expect block , I was checking more from the programming perspective and if I need to avoid it and if its recommended >> 2. How can the above code be improved > > The following sequence of statements:: > > raise Exception("/nfs_mount is not mounted. Dataset create failed !!!") > return False > > do not make sense. The ?return? statement will never be reached. So your > intent must be something other than what is expressed by that code. > I was using it with create_data function in the below code . If create_dataset failed the exception would be caught , I think you are right I need not worry about returning False functions = [create_logdir, create_dataset] for func in functions: try: func() except Exception as e: logging.error(e) return False Regards, Ganesh From ganesh1pal at gmail.com Sun Feb 21 13:03:06 2016 From: ganesh1pal at gmail.com (Ganesh Pal) Date: Sun, 21 Feb 2016 23:33:06 +0530 Subject: Python unittest2.SkipTest and general suggestion In-Reply-To: <85d1rq0zed.fsf@benfinney.id.au> References: <85d1rq0zed.fsf@benfinney.id.au> Message-ID: On Sun, Feb 21, 2016 at 10:33 PM, Ben Finney wrote: > You are already supplying a custom message to ?self.skipTest?:: > > except Exception as exc: > logging.error(exc) > raise unittest.SkipTest("Failure running Integrity Scan ") > > So you can change that message by including the text representation of > the exception object:: > > except Exception as exc: > logging.error(exc) > raise unittest.SkipTest( > "Failure running Integrity Scan: {exc}".format(exc=exc)) > Thank you for the pointers , but I modified my code to use raise instead of exception , just wondering how will I display the message with the below code , Sample code : import os try: import unittest2 as unittest except ImportError: import unittest class MyTests(unittest.TestCase): @classmethod def setUpClass(cls): #x = False x = None if not x: raise unittest.SkipTest( "class setup failed") def test_one(self): print "test_one" def test_two(self): print "test_two" if __name__ == "__main__": unittest.main() output: gpal-3c6dc81-1# python c_4.py s ---------------------------------------------------------------------- Ran 0 tests in 0.000s OK (skipped=1) From python.list at tim.thechases.com Sun Feb 21 13:19:06 2016 From: python.list at tim.thechases.com (Tim Chase) Date: Sun, 21 Feb 2016 12:19:06 -0600 Subject: Considering migrating to Python from Visual Basic 6 for engineering applications In-Reply-To: References: <90cc50d2-1ce5-4588-9bfd-a49d439f00dd@googlegroups.com> <14c75a68-0d2e-45cc-8d73-0d71b6a6aea6@googlegroups.com> <9e57761f-26e1-41c5-8e71-23800de1fdd3@googlegroups.com> <7f9c473e-b0c2-4d77-91d1-d0733c93b12d@googlegroups.com> <23d8156f-1808-4395-9c04-27d2984fe67c@googlegroups.com> Message-ID: <20160221121906.095a57d0@bigbox.christie.dr> On 2016-02-21 13:16, BartC wrote: > > No need for anyone to re-invent the > > wheel! ;-) > > I keep seeing this in the thread. Python has all this capability, > yet it still requires a lot of fiddly code to be added to get > anywhere near as simple as this: > > read f, a, b, c > > And this is code that is not going to be obvious to anyone starting > out. Even accepting that syntax limitations might require this to > be written as: > > readline(f, a, b, c) Well, if you know what the line is going to contain, that can be written as a, b, c = f.readline().split() > I can't see a straightforward way of making this possible while > still keeping a, b and c simple integer, float or string types > (because Python's reference parameters don't work quite the right > way). However, that does give you byte-strings since that's what comes out of files. If you know they're ints, you can force that: a, b, c = map(int, f.readline().split()) > (There is also the question of 'readline' knowing what types of > values to read. This information would not be needed in Fortran or > Basic but somehow needs to be supplied here, if a particular set of > types is to imposed on the input.) > > In other words, it seems this particular wheel does require > re-inventing! In both Fortran & BASIC, you specify that information somewhere as well. However, it sounds like you define those at the variable-definition level (it's been over a decade since I done any BASIC and far longer since I've even touched any Fortran, so forgive me if I'm a tad rusty) instead of where its read from the file. -tkc From ian.g.kelly at gmail.com Sun Feb 21 14:22:02 2016 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Sun, 21 Feb 2016 12:22:02 -0700 Subject: can try expect have if else. In-Reply-To: References: Message-ID: On Feb 21, 2016 9:13 AM, "Ganesh Pal" wrote: > > Hi Team, > > Iam on python 2.6 , need input on the below piece of code. Python 2.6 isn't supported and hasn't been since October 2013. Is there something preventing you from upgrading to 2.7? > EXIT_STATUS_ERROR=1 > > def create_dataset(): > """ > """ Empty docstring is pointless. > logging.info("Dataset create.....Started !!!") This is just a pet peeve of mine, but do info-level log messages really need triple exclamation marks? > try: > if os.path.ismount("/nfs_mount"): > touch_file("inode_fixcrc.txt") Is this file really meant to be created in the current working directory, or is it supposed to be inside the /nfs_mount? > logging.info("Dataset create.....Done !!!") > else: > raise Exception("/nfs_mount is not mounted. Dataset create > failed !!!") > return False > except Exception as e: > logging.error(e) > sys.stderr.write("Dataset create failed...Exiting !!!") > sys.exit(EXIT_STATUS_ERROR) > return True > > 1. Can we have if else with in a try except block Yes, of course. From cs at zip.com.au Sun Feb 21 15:35:36 2016 From: cs at zip.com.au (Cameron Simpson) Date: Mon, 22 Feb 2016 07:35:36 +1100 Subject: Make a unique filesystem path, without creating the file In-Reply-To: References: Message-ID: <20160221203536.GA7520@cskk.homeip.net> On 16Feb2016 19:24, Alan Bawden wrote: >Ben Finney writes: >> Cameron Simpson writes: >>> I've been watching this for a few days, and am struggling to >>> understand your use case. >> >> Yes, you're not alone. This surprises me, which is why I'm persisting. >> >>> Can you elaborate with a concrete example and its purpose which would >>> work with a mktemp-ish official function? >> >> An example:: > >Let me present another example that might strike some as more >straightforward. > >If I want to create a temporary file, I can call mkstemp(). >If I want to create a temporary directory, I can call mkdtemp(). > >Suppose that instead of a file or a directory, I want a FIFO or a >socket. > >A FIFO is created by passing a pathname to os.mkfifo(). A socket is >created by passing a pathname to an AF_UNIX socket's bind() method. In >both cases, the pathname must not name anything yet (not even a symbolic >link), otherwise the call will fail. > >So in the FIFO case, I might write something like the following: > > def make_temp_fifo(mode=0o600): > while True: > path = tempfile.mktemp() > try: > os.mkfifo(path, mode=mode) > except FileExistsError: > pass > else: > return path > >mktemp() is convenient here, because I don't have to worry about whether >I should be using "/tmp" or "/var/tmp" or "c:\temp", or whether the >TMPDIR environment variable is set, or whether I have permission to >create entries in those directories. It just gives me a pathname >without making me think about the rest of that stuff. Yes, that is highly desirable. >Yes, I have to >defend against the possibility that somebody else creates something with >the same name first, but as you can see, I did that, and it wasn't >rocket science. > >So is there something wrong with the above code? Other than the fact >that the documentation says something scary about mktemp()? Well, it has a few shortcomings. It relies on mkfifo reliably failing if the name exists. It shounds like mkfifo is reliable this way, but I can imagine analogous use cases without such a convenient core action, and your code only avoids mktemp's security issue _because_ mkfifo has that fortuitous aspect. >It looks to me like mktemp() provides some real utility, packaged up in >a way that is orthogonal to the type of file system entry I want to >create, the permissions I want to give to that entry, and the mode I >want use to open it. It looks like a useful, albeit low-level, >primitive that it is perfectly reasonable for the tempfile module to >supply. Secondly, why is your example better than:: os.mkfifo(os.path.join(mkdtemp(), 'myfifo')) On that basis, this example doesn't present a use case what can't be addressed by mkstemp or mkdtemp. By contrast, Ben's example does look like it needs something like mktemp. >And yet the documentation condemns it as "deprecated", and tells me I >should use mkstemp() instead. You _do_ understand the security issue, yes? I sure looked like you did, until here. >(As if that would be of any use in the >situation above!) It looks like anxiety that some people might use >mktemp() in a stupid way has caused an over-reaction. No, it is anxiety that mktemp's _normal_ use is inherently unsafe. >Let the >documentation warn about the problem and point to prepackaged solutions >in the common cases of making files and directories, but I see no good >reason to deprecate this useful utility. I think it is like C's gets() function (albeit not as dangerous). It really shouldn't be used. One of the things about mktemp() is its raciness, which is the core of the security issue. People look at the term "security issue" and think "Ah, it can be attacked." But the flipside is that it is simply unreliable. Its normal use was to make an ordinary temp file. Consider the case where two instances of the same task are running at the same time, doing that. They can easily, by accident, end us using the same scratch file! This is by no means unlikely; any shell script running tasks in parallel can arrange it, any procmail script filing a message with a "copy" rule (which causes procmail simply to fork and proceed), etc. This is neither weird nor even unlikely which is why kmtemp is strongly discouraged - naive (and standard) use is not safe. That you have contrived a use case where you can _carefully_ use mktemp in safety in no way makes mktemp recommendable. In fact your use case isn't safe, because _another_ task using mktemp in conflict as a plain old temporary file may grab your fifo. Cheers, Cameron Simpson From sjeik_appie at hotmail.com Sun Feb 21 16:37:43 2016 From: sjeik_appie at hotmail.com (Albert-Jan Roskam) Date: Sun, 21 Feb 2016 21:37:43 +0000 Subject: SQLite In-Reply-To: <56C9EFB6.1090309@mailbox.org> References: <56C9EFB6.1090309@mailbox.org> Message-ID: (Sorry for top posting) IIRC, you have to do sudo apt-get install build-essential python-dev ... then re-compile python > To: python-list at python.org > From: k.d.jantzen at mailbox.org > Subject: SQLite > Date: Sun, 21 Feb 2016 18:11:18 +0100 > > Hello, > > I have downloaded Python3.5.1 as .targz, compiled it(configure, make,...) > and it works > (under Debian Wheezy AMD64) up to the moment I wanted to use SQLite. > > I get the following message: > === > jantzen at PC4:~$ python > Python 3.5.0 (default, Dec 2 2015, 14:16:16) > [GCC 4.7.2] on linux > Type "help", "copyright", "credits" or "license" for more information. > >>> import sqlite3 > Traceback (most recent call last): > File "", line 1, in > File "/usr/local/lib/python3.5/sqlite3/__init__.py", line 23, in > > from sqlite3.dbapi2 import * > File "/usr/local/lib/python3.5/sqlite3/dbapi2.py", line 27, in > from _sqlite3 import * > ImportError: No module named '_sqlite3' > === > > Obviously something is missing. > How do I solve the problem? Where do I find this module? > > Thanks for a hint. > -- > > K.D.J. > -- > https://mail.python.org/mailman/listinfo/python-list From jussi.piitulainen at helsinki.fi Sun Feb 21 16:52:11 2016 From: jussi.piitulainen at helsinki.fi (Jussi Piitulainen) Date: Sun, 21 Feb 2016 23:52:11 +0200 Subject: Considering migrating to Python from Visual Basic 6 for engineering applications References: <90cc50d2-1ce5-4588-9bfd-a49d439f00dd@googlegroups.com> <14c75a68-0d2e-45cc-8d73-0d71b6a6aea6@googlegroups.com> <9e57761f-26e1-41c5-8e71-23800de1fdd3@googlegroups.com> <7f9c473e-b0c2-4d77-91d1-d0733c93b12d@googlegroups.com> <23d8156f-1808-4395-9c04-27d2984fe67c@googlegroups.com> Message-ID: BartC writes: > But this is not so much to do with VB6, but with a very fundamental > capability that seems missing in modern languages. All that's missing is a minor convenience that turns out to be mainly awkward. Not at all fundamental. The capability of parsing simple input formats is easily composed of more generally useful components that are available. As demonstrated in this thread. I get a bit carried away about input() below, but really, it's the abstraction and composition of programs that is fundamental and exciting, Python has *that*, and that makes it possible to define the desired input parsers in a couple of lines. Simply. > IIRC, the first programming exercise I ever did (in 1976 using Algol > 60) involved reading 3 numbers from the teletype and working out if > those could form sides of a triangle. That was a lousy user interface even then - an inflexible user interaction without even a possibility of handling errors interactively? Command line arguments would have been better (if available, that is). In Python, use the interactive loop instead (or command line arguments, of course, or input(), and I think it's slightly simpler to use input() three times than instruct the user about how to separate the three numbers). > It might have started off like this (I can't remember Algol 60): > > print "Type 3 numbers:" > read a, b, c > > In Basic, that last line might be: > > input a, b, c > > instead (reading here as floats probably). > > Now, nearly 40 years later, I don't actually know how to do that in > Python! Sure, I can cobble something together, with all sorts of > functions and string operations (and I would need to go and look them > up). But it's not just /there/ already. It seems you'd need to look them up in Algol 60 and Basic, too. Nothing wrong with that. You can ask for a line of input with input('>>> ') in Python 3. The methods of parsing simple input formats, such as those in this thread, with or without validation, are generally useful, composable, flexible, and not at all difficult. You'll get far by line.split(), splitting at any amount of whitespace by default (note that the .strip() in the line.strip().split() that someone posted is redundant). It's *good* to be able to cobble together a simple one-liner that does what you want with fundamental, reusable components. Or a three-liner if you *want* something more complex. Or a page of code to fake something that is really AI-complete, if *that's* what you want. But mainly I mean that the present problem is just trivial, and what do you do with the Basic command if you have a slightly different input format - does it scale at all? If there is extra text on the line before the numbers? If the numbers are separated by commas, or semicolons? > If you gave this a Python exercise to a class of students, you'd end > up with 30 different solutions just for the first, easy part of the > exercise! In fact it would be far more challenging than the triangle > problem. That can't be right. You mean those students who have never even started a program from a command line? Whose first command-line interface ever is the Python interactive loop in IDLE or some other such integrated environment where they still don't learn to start a program from a command line? Those students? Well, why would they write code that asks for interactive input at all? They can simply call their triangle-testing function in the Python interactive loop and specify the integers as arguments: >>> def istriangle(*threeintegers): a,b,c = sorted(threeintegers) return a*a + b*b == c*c >>> istriangle(3,4,5) True >>> istriangle(3,1,4) False The development environment probably lets them send their code to the interaction window from an editor window. Some might be able to learn the advanced techniques required to import a program file into an interactive session launched in a terminal window. You insist on them asking for input anyway? How about telling them that they can get a line of input (if they have some sort of console available, I suppose) by calling, er, input, and the simplest way to parse a string, call it s, as an integer is int(s)? (Tell them that if they just guess that the input method might be called input they can get their guess confirmed by trying help(input). Another useful skill.) >>> def awkward(): a = int(input('int> ')) b = int(input('int> ')) c = int(input('int> ')) istriangle(a, b, c) >>> awkward() int> 3 int> 1 int> 4 >>> # oops >>> def awkward(): a = int(input('int> ')) b = int(input('int> ')) c = int(input('int> ')) return istriangle(a, b, c) >>> awkward() int> 5 int> 4 int> 3 True >>> Point out that it's a good idea to have a separate function to check for triangularity of the triple of numbers: it'll be composable with other, better sources of such triples (web form? command line arguments? random numbers? loop over a million random triples and calculate the proportion of triangles? database query?) and other consumers of such judgments (like that proportion calculator), and its easy to write tests for it (it's not easy to write automatic tests for functions that require interaction). Point out that the input-prompting interface is lousy (oops, wrong key, too late) and doing better is (1) far from trivial and either (2) already available to them in the Python interactive loop or (3) a number of separate topics. And then there are all the common data sources and formats (CSV, JSON, XML, ...) that change the game altogether. All the functions and expression types used to parse the simple input lines in this thread are still useful, but what use is that very special Basic input command now? None whatsoever. From bc at freeuk.com Sun Feb 21 18:05:56 2016 From: bc at freeuk.com (BartC) Date: Sun, 21 Feb 2016 23:05:56 +0000 Subject: Considering migrating to Python from Visual Basic 6 for engineering applications In-Reply-To: References: <90cc50d2-1ce5-4588-9bfd-a49d439f00dd@googlegroups.com> <14c75a68-0d2e-45cc-8d73-0d71b6a6aea6@googlegroups.com> <9e57761f-26e1-41c5-8e71-23800de1fdd3@googlegroups.com> <7f9c473e-b0c2-4d77-91d1-d0733c93b12d@googlegroups.com> <23d8156f-1808-4395-9c04-27d2984fe67c@googlegroups.com> Message-ID: On 21/02/2016 21:52, Jussi Piitulainen wrote: > BartC writes: > >> But this is not so much to do with VB6, but with a very fundamental >> capability that seems missing in modern languages. > > All that's missing is a minor convenience that turns out to be mainly > awkward. Not at all fundamental. It /is/ fundamental because it mimics how humans interact: "Give me 3 numbers:" "OK: ten, twenty and twenty-nine." "Yeah, they'd make a triangle." > The capability of parsing simple input formats is easily composed of > more generally useful components that are available. As demonstrated in > this thread. Yes, by having a box of parts you first have to figure out how to put together! >> IIRC, the first programming exercise I ever did (in 1976 using Algol >> 60) involved reading 3 numbers from the teletype and working out if >> those could form sides of a triangle. > That was a lousy user interface even then - an inflexible user > interaction without even a possibility of handling errors interactively? > Command line arguments would have been better (if available, that is). The big deal about that kind of interface was it that it /was/ interactive (compared with batch processing or job control or whatever. I remember using punched cards; those certainly weren't). Command line arguments are not interactive. If the name of the program is Triangle, and someone types: >Triangle the probably nothing happens. Or it generates an error message. Or it appears to hang. > In Python, use the interactive loop instead (or command line arguments, > of course, or input(), and I think it's slightly simpler to use input() > three times than instruct the user about how to separate the three > numbers). Basics also used to use interactive loops. Separating the numbers I don't think has ever been a big problem. Obviously you can't type ten, twenty and thirty as 102030, but will need to use white-space or punctuation. It doesn't take long to figure out! (And how do you separate command-line arguments? Have you even thought about it?) >> Now, nearly 40 years later, I don't actually know how to do that in >> Python! Sure, I can cobble something together, with all sorts of >> functions and string operations (and I would need to go and look them >> up). But it's not just /there/ already. > > It seems you'd need to look them up in Algol 60 and Basic, too. Nothing > wrong with that. Yes, and it would say use 'readln' or 'input' or whatever. In Python, I'd get two-inch-thick manual full of library functions and modules I can use to /implement/ 'readln' or 'input'! > It's *good* to be able to cobble together a simple one-liner that does > what you want with fundamental, reusable components. Or a three-liner if > you *want* something more complex. But this is advanced stuff for a beginner. (I'm not a beginner and it's advanced for /me/! Well, more of a nuisance when I want to quickly do X but first need to figure out Y, some input routine.) Or a page of code to fake something > that is really AI-complete, if *that's* what you want. But mainly I mean > that the present problem is just trivial, and what do you do with the > Basic command if you have a slightly different input format - does it > scale at all? If there is extra text on the line before the numbers? If > the numbers are separated by commas, or semicolons? I don't know the details of how Basic does it. But it seems to allow any reasonable separator, while asking for more input than is provided on the line sets those extra variables to 0 or "". >> If you gave this a Python exercise to a class of students, you'd end >> up with 30 different solutions just for the first, easy part of the >> exercise! > Well, why would they write code that asks for interactive input at all? > They can simply call their triangle-testing function in the Python > interactive loop and specify the integers as arguments: > >>>> def istriangle(*threeintegers): > a,b,c = sorted(threeintegers) > return a*a + b*b == c*c (That detects right-angled triangles. I think 'return a+b>c' is the correct test for any triangle. It becomes trivial when you have sort available, but that's part of the point of the exercise.) >>>> istriangle(3,4,5) > True >>>> istriangle(3,1,4) > False Yes, this is a good next step, to separate the logic from the job of acquiring the data (and to learn about with subroutines or functions). But this way, you're putting off the job of requesting interactive data from a user. If A is the chap writing the program and using the tools, he might want B to be the one running the program and entering the data. B won't be running the interactive loop; he won't even know what language it's in. (And the OP wants B to run an EXE.) > And then there are all the common data sources and formats (CSV, JSON, > XML, ...) that change the game altogether. All the functions and > expression types used to parse the simple input lines in this thread are > still useful, but what use is that very special Basic input command now? > None whatsoever. Not at all. Once you've got the hang of reading a line of interactive input, then the same program can be given redirected input from a file. And the OP's example was reading from a file anyway. (Some formats will need more sophisticating parsing; that's advanced stuff though. Advanced enough in fact that you wouldn't attempt parsing them; just use a library.) Reading stuff from an interactive console or terminal, is such an important part of the history of computing, still being used now, that you'd think a language would provide simple, ready-to-use methods ways to do it. Would it have hurt to have done so? Then it would be easier also to port existing programs. -- Bartc From rosuav at gmail.com Sun Feb 21 20:44:30 2016 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 22 Feb 2016 12:44:30 +1100 Subject: SQLite In-Reply-To: References: <56C9EFB6.1090309@mailbox.org> Message-ID: On Mon, Feb 22, 2016 at 8:37 AM, Albert-Jan Roskam wrote: > IIRC, you have to do > sudo apt-get install build-essential python-dev > ... then re-compile python That may well work, but it's probably easier to work this way: sudo apt-get build-dep python3 That should grab all the compilation dependencies of the python3 package, which on Wheezy is a 3.2 IIRC. The deps haven't changed since then AFAIK. When you build, you should get a summary at the end that tells you about any modules that couldn't be built. The most common reason for that is missing deps. If you still have any after running the above, post here and we may be able to more directly advise. ChrisA From denis.akhiyarov at gmail.com Mon Feb 22 03:02:10 2016 From: denis.akhiyarov at gmail.com (Denis Akhiyarov) Date: Mon, 22 Feb 2016 00:02:10 -0800 (PST) Subject: Considering migrating to Python from Visual Basic 6 for engineering applications In-Reply-To: References: <90cc50d2-1ce5-4588-9bfd-a49d439f00dd@googlegroups.com> <2dd9429e-16db-451d-beb8-1248a7029f80@googlegroups.com> Message-ID: Note that you can continue using your existing vb6 code from python through COM using pywin32 or pythonnet, until you decide to rewrite it. From k.d.jantzen at mailbox.org Mon Feb 22 03:32:43 2016 From: k.d.jantzen at mailbox.org (Klaus Jantzen) Date: Mon, 22 Feb 2016 09:32:43 +0100 Subject: SQLite In-Reply-To: References: <56C9EFB6.1090309@mailbox.org> Message-ID: <56CAC7AB.2000902@mailbox.org> On 02/21/2016 10:37 PM, Albert-Jan Roskam wrote: (Sorry for top posting) IIRC, you have to do sudo apt-get install build-essential python-dev ... then re-compile python > To: [1]python-list at python.org > From: [2]k.d.jantzen at mailbox.org > Subject: SQLite > Date: Sun, 21 Feb 2016 18:11:18 +0100 > > Hello, > > I have downloaded Python3.5.1 as .targz, compiled it(configure, make,...) > and it works > (under Debian Wheezy AMD64) up to the moment I wanted to use SQLite. > > I get the following message: > === > jantzen at PC4:~$ python > Python 3.5.0 (default, Dec? 2 2015, 14:16:16) > [GCC 4.7.2] on linux > Type "help", "copyright", "credits" or "license" for more information. > >>> import sqlite3 > Traceback (most recent call last): > ? File "", line 1, in > ? File "/usr/local/lib/python3.5/sqlite3/__init__.py", line 23, in > > ??? from sqlite3.dbapi2 import * > ? File "/usr/local/lib/python3.5/sqlite3/dbapi2.py", line 27, in > ??? from _sqlite3 import * > ImportError: No module named '_sqlite3' > === > > Obviously something is missing. > How do I solve the problem? Where do I find this module? > > Thanks for a hint. > -- > > K.D.J. > -- > [3]https://mail.python.org/mailman/listinfo/python-list Hello, thanks for your hint. That did it!!! -- K.D.J. References Visible links 1. mailto:python-list at python.org 2. mailto:k.d.jantzen at mailbox.org 3. https://mail.python.org/mailman/listinfo/python-list From k.d.jantzen at mailbox.org Mon Feb 22 03:40:59 2016 From: k.d.jantzen at mailbox.org (Klaus Jantzen) Date: Mon, 22 Feb 2016 09:40:59 +0100 Subject: SQLite In-Reply-To: References: <56C9EFB6.1090309@mailbox.org> Message-ID: <56CAC99B.8020104@mailbox.org> On 02/22/2016 02:44 AM, Chris Angelico wrote: On Mon, Feb 22, 2016 at 8:37 AM, Albert-Jan Roskam [1] wrote: IIRC, you have to do sudo apt-get install build-essential python-dev ... then re-compile python That may well work, but it's probably easier to work this way: sudo apt-get build-dep python3 That should grab all the compilation dependencies of the python3 package, which on Wheezy is a 3.2 IIRC. The deps haven't changed since then AFAIK. When you build, you should get a summary at the end that tells you about any modules that couldn't be built. The most common reason for that is missing deps. If you still have any after running the above, post here and we may be able to more directly advise. ChrisA That did not work because I did not install Python 3.5. with apt-get but downloaded the source and compiled myself. Thus apt-get does not have any information about the Python 3.5 installation. I got it to work following the hint by Albert-Jan Roskam. -- K.D.J. References Visible links 1. mailto:sjeik_appie at hotmail.com From jussi.piitulainen at helsinki.fi Mon Feb 22 03:50:07 2016 From: jussi.piitulainen at helsinki.fi (Jussi Piitulainen) Date: Mon, 22 Feb 2016 10:50:07 +0200 Subject: Considering migrating to Python from Visual Basic 6 for engineering applications References: <90cc50d2-1ce5-4588-9bfd-a49d439f00dd@googlegroups.com> <14c75a68-0d2e-45cc-8d73-0d71b6a6aea6@googlegroups.com> <9e57761f-26e1-41c5-8e71-23800de1fdd3@googlegroups.com> <7f9c473e-b0c2-4d77-91d1-d0733c93b12d@googlegroups.com> <23d8156f-1808-4395-9c04-27d2984fe67c@googlegroups.com> Message-ID: BartC writes: > On 21/02/2016 21:52, Jussi Piitulainen wrote: [...] >>>>> def istriangle(*threeintegers): >> a,b,c = sorted(threeintegers) >> return a*a + b*b == c*c > > (That detects right-angled triangles. I think 'return a+b>c' is the > correct test for any triangle. It becomes trivial when you have sort > available, but that's part of the point of the exercise.) Sorry about that. (Non-positive sides should also be ruled out.) [...] > Reading stuff from an interactive console or terminal, is such an > important part of the history of computing, still being used now, that > you'd think a language would provide simple, ready-to-use methods ways > to do it. I think it does. Do you at least agree that it provides a reasonably simple way to ask for a single line of text? stuff = input() Is that already too obscure or advanced for you? Because surely not? It can be used piecemeal: print("a b c = ") stuff = input() parts = stuff.split() a = int(parts[0]) b = int(parts[1]) c = int(parts[2]) The same can be done in one statement: a, b, c = map(int, input("a b c = ").split()) Here's a simpler user interface: a = int(input("a = ")) b = int(input("b = ")) c = int(input("c = ")) These building blocks can be used to define more complex dialogues in obvious ways, probably using exception handling. def please(*types, *, prompt = None): """Repeatedly prompts the console user for space-separated values in one line until an input line matches the given types. Returns the corresponding values. At end of input, raises UpsetException.""" ... a, b, c = please(int, int, int, prompt = "a b c = ") def readWithDWIM(source, *types): """Reads and returns whitespace-separated values from text source. Parses each as the corresponding type by calling the type on it. For invalid or missing values, substitutes the value of calling the type without arguments, or None if even that fails. May or may not discard the remaining part of the line that contained the last value. Use at your own risk!""" ... from itertools import repeat with open("f.txt" as f: a, b, c = readWithDWIM(f, int, int, int)) xs = readWithDWIM(f, *repeat(float, b)) If there really is demand for these, they should be all over the web already. I don't see why there should be demand. The simple case is simple enough, more complex specifications are either still simple enough to write from scratch, or too specific, or insane. [...] From rosuav at gmail.com Mon Feb 22 04:00:30 2016 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 22 Feb 2016 20:00:30 +1100 Subject: SQLite In-Reply-To: <56CAC99B.8020104@mailbox.org> References: <56C9EFB6.1090309@mailbox.org> <56CAC99B.8020104@mailbox.org> Message-ID: On Mon, Feb 22, 2016 at 7:40 PM, Klaus Jantzen wrote: > That did not work because I did not install Python 3.5. with apt-get > but downloaded the source and compiled myself. > Thus apt-get does not have any information about the Python 3.5 > installation. It generally does work; the build dependencies for CPython 3.2 are the same as for CPython 3.5, with maybe one or two new ones added for new modules. So you can ask apt-get to grab you the deps for 3.2 and then build 3.5 from source. That's what I usually do when I build from source control - whether it's CPython or Pike or git or FFMpeg or anything else, I can generally depend on the build deps for the old version being good enough to build the new version. Even when they're not exactly right, they're close enough that it saves a lot of time. ChrisA From nelle.varoquaux at gmail.com Mon Feb 22 04:15:51 2016 From: nelle.varoquaux at gmail.com (Nelle Varoquaux) Date: Mon, 22 Feb 2016 10:15:51 +0100 Subject: Scipy2016: call for proposals Message-ID: Dear all, SciPy 2016, the Fifteenth Annual Conference on Python in Science, takes place in Austin, TX on July, 11th to 17th. The conference features two days of tutorials by followed by three days of presentations, and concludes with two days of developer sprints on projects of interest to attendees. . The topics presented at SciPy are very diverse, with a focus on advanced software engineering and original uses of Python and its scientific libraries, either in theoretical or experimental research, from both academia and the industry. This year we are happy to announce two specialized tracks that run in parallel to the general conference (Data Science , High Performance Computing) and 8 mini-symposia (Earth and Space Science, Biology and Medicine, Engineering, Social Sciences, Special Purpose Databases, Case Studies in Industry, Education, Reproducibility) Submissions for talks and posters are welcome on our website ( http://scipy2016.scipy.org). In your abstract, please provide details on what Python tools are being employed, and how. The talk and poster submission deadline is March 25th, 2016, while the tutorial submission deadline is March, 21st, 2016. Important dates: Mar 21: Tutorial Proposals Due Mar 25: Talk and Poster Proposals Due May 11: Plotting Contest Submissions Due Apr 22: Tutorials Announced Apr 22: Financial Aid Submissions Due May 4: Talk and Posters Announced May 11: Financial Aid Recipients Notified May 22: Early Bird Registration Deadline Jul 11-12: SciPy 2016 Tutorials Jul 13-15: SciPy 2016 General Conference Jul 16-17: SciPy 2016 Sprints We look forward to an exciting conference and hope to see you in Austin in July! The Scipy 2016 http://scipy2016.scipy.org/ Conference Chairs: Aric Hagberg, Prabhu Ramachandran Tutorial Chairs: Justin Vincent, Ben Root Program Chair: Serge Rey, Nelle Varoquaux Proceeding Chairs: Sebastian Benthall From steve at pearwood.info Mon Feb 22 05:36:06 2016 From: steve at pearwood.info (Steven D'Aprano) Date: Mon, 22 Feb 2016 21:36:06 +1100 Subject: can try expect have if else. References: Message-ID: <56cae498$0$1615$c3e8da3$5496439d@news.astraweb.com> On Mon, 22 Feb 2016 03:12 am, Ganesh Pal wrote: > Hi Team, > > Iam on python 2.6 , need input on the below piece of code. > > > EXIT_STATUS_ERROR=1 > > def create_dataset(): > """ > """ > logging.info("Dataset create.....Started !!!") > try: > if os.path.ismount("/nfs_mount"): > touch_file("inode_fixcrc.txt") > logging.info("Dataset create.....Done !!!") > else: > raise Exception("/nfs_mount is not mounted. Dataset create > failed !!!") > return False You should not raised Exception directly. The "return False" line is dead code, it will never be executed. > except Exception as e: > logging.error(e) > sys.stderr.write("Dataset create failed...Exiting !!!") > sys.exit(EXIT_STATUS_ERROR) > return True > > 1. Can we have if else with in a try except block Did you try it to see what would happen? Yes, you can have an if...else block inside a try block. > 2. How can the above code be improved # Use a custom exception type. class CreateDatasetError(OSError): pass def touch_file(path, filename): """Raises OSError or IOError if path is not a mounted mount point, and path/filename cannot be created. """ if not os.path.ismount(path): raise CreateDatasetError('%s is not mounted, create failed.' % path) else: fullname = os.join(path, filename) f = open(fullname, 'w') f.close() EXIT_STATUS_ERROR=1 def create_dataset(): """ """ logging.info("Dataset create.....Started !!!") try: touch_file("/nfs_mount", "inode_fixcrc.txt") except (IOError, OSError) as e: logging.error(e) sys.stderr.write("Dataset create failed...Exiting !!!") sys.exit(EXIT_STATUS_ERROR) logging.info("Dataset create.....Done !!!") -- Steven From steve at pearwood.info Mon Feb 22 05:46:40 2016 From: steve at pearwood.info (Steven D'Aprano) Date: Mon, 22 Feb 2016 21:46:40 +1100 Subject: Considering migrating to Python from Visual Basic 6 for engineering applications References: <90cc50d2-1ce5-4588-9bfd-a49d439f00dd@googlegroups.com> <14c75a68-0d2e-45cc-8d73-0d71b6a6aea6@googlegroups.com> <9e57761f-26e1-41c5-8e71-23800de1fdd3@googlegroups.com> <7f9c473e-b0c2-4d77-91d1-d0733c93b12d@googlegroups.com> <23d8156f-1808-4395-9c04-27d2984fe67c@googlegroups.com> Message-ID: <56cae711$0$1586$c3e8da3$5496439d@news.astraweb.com> On Mon, 22 Feb 2016 08:52 am, Jussi Piitulainen wrote: > BartC writes: >> IIRC, the first programming exercise I ever did (in 1976 using Algol >> 60) involved reading 3 numbers from the teletype and working out if >> those could form sides of a triangle. > > That was a lousy user interface even then - an inflexible user > interaction without even a possibility of handling errors interactively? > Command line arguments would have been better (if available, that is). Jussi, I think you have an inflated expectation of what was available in 1976. Command line? What's that? Programs ran in batch mode, and 'interactive' meant that you could easily slip out one punched card, replace it with a different one, and run the program again. -- Steven From arsh840 at gmail.com Mon Feb 22 06:15:07 2016 From: arsh840 at gmail.com (Arshpreet Singh) Date: Mon, 22 Feb 2016 03:15:07 -0800 (PST) Subject: avoid for loop calling Generator function Message-ID: Hi, I am converting PDF into text file, I am using following code. from pypdf2 import PdfFileReader def read_pdf(pdfFileName): pdf = PdfFileReader(pdfFileName) yield from (pg.extractText() for pg in pdf.pages) for i in read_pdf('book.pdf'): print(i) I want to avoid for loop , I also tried to create another function and call read_pdf() inside that new function using yield from but I think I am missing real picture here From steve at pearwood.info Mon Feb 22 06:16:38 2016 From: steve at pearwood.info (Steven D'Aprano) Date: Mon, 22 Feb 2016 22:16:38 +1100 Subject: Considering migrating to Python from Visual Basic 6 for engineering applications References: <90cc50d2-1ce5-4588-9bfd-a49d439f00dd@googlegroups.com> <14c75a68-0d2e-45cc-8d73-0d71b6a6aea6@googlegroups.com> <9e57761f-26e1-41c5-8e71-23800de1fdd3@googlegroups.com> <7f9c473e-b0c2-4d77-91d1-d0733c93b12d@googlegroups.com> <23d8156f-1808-4395-9c04-27d2984fe67c@googlegroups.com> Message-ID: <56caee18$0$1612$c3e8da3$5496439d@news.astraweb.com> On Mon, 22 Feb 2016 12:16 am, BartC wrote: [...] >> No need for anyone to re-invent the >> wheel! ;-) > > I keep seeing this in the thread. Python has all this capability, yet it > still requires a lot of fiddly code to be added to get anywhere near as > simple as this: > > read f, a, b, c I can't say I really know what that means. My guess is that reads three things (a, b and c) from a file f, but what those things are (strings, binary blobs, floats, ints, something else?) is a mystery. One of the major problems with output parameters is that it isn't obvious what is an output parameter and what isn't. When it comes to read, perhaps you can guess, but when it comes to arbitrary functions, who can tell what is output and what is input? fromaginate m, x, y, z, a, b Whereas a functional design makes it obvious: output is on the left of the assignment symbol, input is on the right. x, y, z = fromaginate m, a, b > And this is code that is not going to be obvious to anyone starting out. > Even accepting that syntax limitations might require this to be written > as: > > readline(f, a, b, c) > > I can't see a straightforward way of making this possible while still > keeping a, b and c simple integer, float or string types (because > Python's reference parameters don't work quite the right way). Python doesn't have reference parameters. Please feel free to discuss further if you disagree, or want additional explanation. > (There is also the question of 'readline' knowing what types of values > to read. This information would not be needed in Fortran or Basic but > somehow needs to be supplied here, if a particular set of types is to > imposed on the input.) > > In other words, it seems this particular wheel does require re-inventing! Hmmm, well, yes, I think perhaps it is fair to say that there is middle ground where Python misses out. Python's raw IO is quite powerful, and it can handle unstructured binary of text files with no difficulty. It also comes with libraries for certain common file formats, like CSV, XML, JSON and others. But there's a middle ground, of *minimally structured* text files, where Python leaves it up to you. Admittedly it's not difficult, any minimally competent Python programmer should be able to read a bunch of ints or floats from a text file without working up a sweat, but beginners may find this tricky. So I think you are right: there is a narrow, but useful, niche of semi-structured textual data that BASIC and VB support that Python doesn't support out of the box. -- Steven From jussi.piitulainen at helsinki.fi Mon Feb 22 06:39:53 2016 From: jussi.piitulainen at helsinki.fi (Jussi Piitulainen) Date: Mon, 22 Feb 2016 13:39:53 +0200 Subject: Considering migrating to Python from Visual Basic 6 for engineering applications References: <90cc50d2-1ce5-4588-9bfd-a49d439f00dd@googlegroups.com> <14c75a68-0d2e-45cc-8d73-0d71b6a6aea6@googlegroups.com> <9e57761f-26e1-41c5-8e71-23800de1fdd3@googlegroups.com> <7f9c473e-b0c2-4d77-91d1-d0733c93b12d@googlegroups.com> <23d8156f-1808-4395-9c04-27d2984fe67c@googlegroups.com> <56cae711$0$1586$c3e8da3$5496439d@news.astraweb.com> Message-ID: Steven D'Aprano writes: > On Mon, 22 Feb 2016 08:52 am, Jussi Piitulainen wrote: > >> BartC writes: > >>> IIRC, the first programming exercise I ever did (in 1976 using Algol >>> 60) involved reading 3 numbers from the teletype and working out if >>> those could form sides of a triangle. >> >> That was a lousy user interface even then - an inflexible user >> interaction without even a possibility of handling errors >> interactively? Command line arguments would have been better (if >> available, that is). > > Jussi, I think you have an inflated expectation of what was available > in 1976. Command line? What's that? Programs ran in batch mode, and > 'interactive' meant that you could easily slip out one punched card, > replace it with a different one, and run the program again. You are probably right. I was thinking ten years off. Sorry. But if that's how one worked a teletype, then it sounds similar to command line arguments to me. Or to a configuration file that the program then when run. Not the interaction-prompting interface that I was thinking of when I called it lousy. From arsh840 at gmail.com Mon Feb 22 07:24:01 2016 From: arsh840 at gmail.com (Arshpreet Singh) Date: Mon, 22 Feb 2016 04:24:01 -0800 (PST) Subject: Setting up/authenticating Google API? In-Reply-To: References: Message-ID: On Sunday, 21 February 2016 21:21:54 UTC+5:30, Skip Montanaro wrote: > This isn't strictly a Python question, however... Once I get myself > authenticated, I intend to use the Python Google API to pump archived > mail messages from a few defunct mailing lists into Google Groups. I > thought it would be pretty straightforward, but my attempts a few > months ago were completely unsuccessful. I asked for help on the > relevant Google product forum but got zero assistance. > > I'd really appreciate a little help with the setup. If anyone has had > successful experience with this, please contact me off-list. Once I > get something going, I'll write a blog post so others don't have the > same problems I have. > > Thanks, > > Skip Montanaro Your question seems quite cinfusing to me but I think following may is what you are asking for.(Read the source) https://google-mail-oauth2-tools.googlecode.com/svn/trunk/python/oauth2.py After getting the authentication you can use imaplib to get all the gmail Data. From bc at freeuk.com Mon Feb 22 07:24:05 2016 From: bc at freeuk.com (BartC) Date: Mon, 22 Feb 2016 12:24:05 +0000 Subject: Considering migrating to Python from Visual Basic 6 for engineering applications In-Reply-To: References: <90cc50d2-1ce5-4588-9bfd-a49d439f00dd@googlegroups.com> <14c75a68-0d2e-45cc-8d73-0d71b6a6aea6@googlegroups.com> <9e57761f-26e1-41c5-8e71-23800de1fdd3@googlegroups.com> <7f9c473e-b0c2-4d77-91d1-d0733c93b12d@googlegroups.com> <23d8156f-1808-4395-9c04-27d2984fe67c@googlegroups.com> Message-ID: On 22/02/2016 08:50, Jussi Piitulainen wrote: > BartC writes: >> Reading stuff from an interactive console or terminal, is such an >> important part of the history of computing, still being used now, that >> you'd think a language would provide simple, ready-to-use methods ways >> to do it. > > I think it does. Do you at least agree that it provides a reasonably > simple way to ask for a single line of text? > > stuff = input() > > Is that already too obscure or advanced for you? Because surely not? That's a good starting point. It reads an entire line of text, as a string. So that's part of the job done. The rest of it needs some string processing. But even 'input' has some issues. While it seemed to work, the docs I looked at suggested it interpreted the input as an expression, and I needed to use raw_input(). Was this a Python 2/3 problem? The docs I used (https://en.wikibooks.org/wiki/Python_Programming/Input_and_Output) didn't appear to mention that, not in the title or around the edges anyway! Eventually I saw that as a italicised note within the text. So, it's input() on 3.x and raw_input() on 2.x. But another thing is that it doesn't scale up to read from a file, as far as I can see. File reading is rather different, and it behaves differently regarding line endings. (For comparison, here's how it works on another language I use: readln a,b,c This waits for a line of input (it becomes the read buffer), then reads three integers from the buffer. read d,e This reads two more integers from that buffer (if there is no more data, it reads zeros). readln @f,a,b,c This does the same, from file handle f. The equivalent of Python's input() might be: s := sreadln() # from console s := sreadln(f) # from a file s := sreadln("10,20") # from a string but the entire line is also placed into the read buffer so that subsequent read commands (or sread() function calls) can read individual values as before. Reading anything other than integers is bit more fiddly (you can't have everything!) So that means using something like this: read a:"r", b:"h", c:"s", d:"n" So reading as a float, an int using hexadecimal, a string (delimited by white space or punctuation, or quoted), a name (file names etc), and so on. There are limitations, but this stuff is there when I don't need anything more sophisticated.) -- Bartc From mal at europython.eu Mon Feb 22 07:38:46 2016 From: mal at europython.eu (M.-A. Lemburg) Date: Mon, 22 Feb 2016 13:38:46 +0100 Subject: EuroPython 2016: Early bird ticket sales Message-ID: <56CB0156.3090508@europython.eu> After the Call for Proposals for EuroPython 2016 last week, we are now calling out to everyone interested in signing up as EuroPython attendee early. You will be able to benefit from reduced ticket prices for a short period of time. Our early bird ticket sales are limited to 300 tickets. Regular sales start shortly after we?ve ended the early bird sales. *** https://ep2016.europython.eu/en/registration/ *** More Information and Ticket Shop We will be opening the flood gates on Tuesday, 09:00 CET, so set your alarms to make sure you can secure your ticket. Experience shows that early bird tickets sell out quickly. PS: Remember to submit your proposals for the conference. There are less than two weeks left: https://ep2016.europython.eu/en/call-for-proposals/ With gravitational regards, -- EuroPython 2016 Team http://ep2016.europython.eu/ http://www.europython-society.org/ From bc at freeuk.com Mon Feb 22 07:51:01 2016 From: bc at freeuk.com (BartC) Date: Mon, 22 Feb 2016 12:51:01 +0000 Subject: Considering migrating to Python from Visual Basic 6 for engineering applications In-Reply-To: <56caee18$0$1612$c3e8da3$5496439d@news.astraweb.com> References: <90cc50d2-1ce5-4588-9bfd-a49d439f00dd@googlegroups.com> <14c75a68-0d2e-45cc-8d73-0d71b6a6aea6@googlegroups.com> <9e57761f-26e1-41c5-8e71-23800de1fdd3@googlegroups.com> <7f9c473e-b0c2-4d77-91d1-d0733c93b12d@googlegroups.com> <23d8156f-1808-4395-9c04-27d2984fe67c@googlegroups.com> <56caee18$0$1612$c3e8da3$5496439d@news.astraweb.com> Message-ID: On 22/02/2016 11:16, Steven D'Aprano wrote: > On Mon, 22 Feb 2016 12:16 am, BartC wrote: > > [...] >>> No need for anyone to re-invent the >>> wheel! ;-) >> >> I keep seeing this in the thread. Python has all this capability, yet it >> still requires a lot of fiddly code to be added to get anywhere near as >> simple as this: >> >> read f, a, b, c > > I can't say I really know what that means. My guess is that reads three > things (a, b and c) from a file f, but what those things are (strings, > binary blobs, floats, ints, something else?) is a mystery. Yes, I mentioned that point. In the OP's language, the variables can be declared as a particular type; in Python they could perhaps be 'declared' by first assigning with 0, 0.0 or "" for example. But that would need reference parameters to make use of tidily. But even when declared, sometimes additional formatting info is needed (numbers might be in hex or binary for example). > One of the major problems with output parameters is that it isn't obvious > what is an output parameter and what isn't. When it comes to read, perhaps > you can guess, but when it comes to arbitrary functions, who can tell what > is output and what is input? > > fromaginate m, x, y, z, a, b > > > Whereas a functional design makes it obvious: output is on the left of the > assignment symbol, input is on the right. > > x, y, z = fromaginate m, a, b Old-style /statements/ such as 'read' aren't functional. Then it is easy to specify that the parameters need to be l-values. >> readline(f, a, b, c) >> >> I can't see a straightforward way of making this possible while still >> keeping a, b and c simple integer, float or string types (because >> Python's reference parameters don't work quite the right way). > > Python doesn't have reference parameters. > > Please feel free to discuss further if you disagree, or want additional > explanation. Well, I've just finished reimplementing a language so that it uses CPython-like references for objects (not, however, for types such as small integers which stay as value types). But that language retains the use of pointers which can be used /as well as/ references to provide pass-by-reference, even for integers. Then it is possible to write a function which can be called like: readline(f,a,b,c) and which would update the caller's a,b,c variables. Python can't do this, even with 100% reference objects. This is because most types are immutable and their values can be shared across unrelated variables. Try and change one of those, and all would be changed! In any case, doing an assignment to a parameter just replaces its local value. Caller's data can only be changed via an in-place modification of the parameter, which I believe only works for lists. (Maybe, something like this can be done with classes, but I did say you can't do it with simple types.) -- Bartc From rosuav at gmail.com Mon Feb 22 08:09:08 2016 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 23 Feb 2016 00:09:08 +1100 Subject: Considering migrating to Python from Visual Basic 6 for engineering applications In-Reply-To: References: <90cc50d2-1ce5-4588-9bfd-a49d439f00dd@googlegroups.com> <14c75a68-0d2e-45cc-8d73-0d71b6a6aea6@googlegroups.com> <9e57761f-26e1-41c5-8e71-23800de1fdd3@googlegroups.com> <7f9c473e-b0c2-4d77-91d1-d0733c93b12d@googlegroups.com> <23d8156f-1808-4395-9c04-27d2984fe67c@googlegroups.com> <56caee18$0$1612$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Mon, Feb 22, 2016 at 11:51 PM, BartC wrote: > Yes, I mentioned that point. In the OP's language, the variables can be > declared as a particular type; in Python they could perhaps be 'declared' by > first assigning with 0, 0.0 or "" for example. But that would need reference > parameters to make use of tidily. What you could do is something like this: def read_values(file, values): """Read one line from file and parse it into the given list. Prepopulate values with a series of types or instances of types. The line of text read will be parsed accordingly. Returns the number of elements successfully read. """ line = file.readline() for idx, val in enumerate(values): if not isinstance(val, type): val = type(val) # Now add the logic to read an int, a float, # a str, a Fraction, etc etc etc, from the line, # breaking out of the loop if one can't be # read, and stashing the result into values[idx] # if it can. return idx with open("inputfile") as f: values = [int, int, float, Fraction, str] while read_values(f, values) == 5: [index, level, percent, ratio, message] = values This lets you "declare" the values' types, and use an input/output parameter. It's not exactly the most Pythonic of techniques, but it does kinda work. I suspect, though, that what you'd _really_ want is something more like a sscanf string, or a "typed regex", which would allow you to specify a bit more flexibly what you're looking for. You could easily make a template string that has magic markers for "take an integer", which would insert [0-9]+ (or something more elaborate) and then capture it and call int() on the string before returning it. ChrisA From __peter__ at web.de Mon Feb 22 08:34:08 2016 From: __peter__ at web.de (Peter Otten) Date: Mon, 22 Feb 2016 14:34:08 +0100 Subject: avoid for loop calling Generator function References: Message-ID: Arshpreet Singh wrote: > Hi, I am converting PDF into text file, I am using following code. > > from pypdf2 import PdfFileReader > > def read_pdf(pdfFileName): > > pdf = PdfFileReader(pdfFileName) > > yield from (pg.extractText() for pg in pdf.pages) > > for i in read_pdf('book.pdf'): > print(i) > > I want to avoid for loop , I also tried to create another function and > call read_pdf() inside that new function using yield from but I think I am > missing real picture here While it is possible to replace the loop with next(filter(print, read_pdf("book.pdf")), None) or the slightly less convoluted sys.stdout.writelines(map("{}\n".format, read_pdf("book.pdf"))) the for loop is the obvious and therefore recommended solution. Personally, I would also replace > yield from (pg.extractText() for pg in pdf.pages) with the good old for pg in pdf.pages: yield pg.extractText() and reserve the generator expression for occasions where it has a demonstrable advantage in readability. From random832 at fastmail.com Mon Feb 22 09:49:38 2016 From: random832 at fastmail.com (Random832) Date: Mon, 22 Feb 2016 09:49:38 -0500 Subject: Considering migrating to Python from Visual Basic 6 for engineering applications In-Reply-To: <56cae711$0$1586$c3e8da3$5496439d@news.astraweb.com> References: <90cc50d2-1ce5-4588-9bfd-a49d439f00dd@googlegroups.com> <14c75a68-0d2e-45cc-8d73-0d71b6a6aea6@googlegroups.com> <9e57761f-26e1-41c5-8e71-23800de1fdd3@googlegroups.com> <7f9c473e-b0c2-4d77-91d1-d0733c93b12d@googlegroups.com> <23d8156f-1808-4395-9c04-27d2984fe67c@googlegroups.com> <56cae711$0$1586$c3e8da3$5496439d@news.astraweb.com> Message-ID: <1456152578.205902.528273162.19835767@webmail.messagingengine.com> On Mon, Feb 22, 2016, at 05:46, Steven D'Aprano wrote: > Jussi, I think you have an inflated expectation of what was available in > 1976. Command line? What's that? Programs ran in batch mode, > and 'interactive' meant that you could easily slip out one punched card, > replace it with a different one, and run the program again. User experience was hardly uniform across different systems in that era. The book "Hackers", for example, describes an interactive computer that was used at MIT in *1959*. More relevantly to the lineage of the systems we use today, PDP-7 Unix was first developed in 1969 - and PDP-11 6th Edition Unix, very close to a recognizable modern system, was released in 1975. From arsh840 at gmail.com Mon Feb 22 10:38:09 2016 From: arsh840 at gmail.com (Arshpreet Singh) Date: Mon, 22 Feb 2016 07:38:09 -0800 (PST) Subject: avoid for loop calling Generator function In-Reply-To: References: Message-ID: <0c289f93-ce64-4eff-93f3-c70d7ff50817@googlegroups.com> On Monday, 22 February 2016 19:05:24 UTC+5:30, Peter Otten wrote: > Arshpreet Singh wrote: > > > Hi, I am converting PDF into text file, I am using following code. > > > > from pypdf2 import PdfFileReader > > > > def read_pdf(pdfFileName): > > > > pdf = PdfFileReader(pdfFileName) > > > > yield from (pg.extractText() for pg in pdf.pages) > > > > for i in read_pdf('book.pdf'): > > print(i) > > > > I want to avoid for loop , I also tried to create another function and > > call read_pdf() inside that new function using yield from but I think I am > > missing real picture here > > While it is possible to replace the loop with > > next(filter(print, read_pdf("book.pdf")), None) Why we are w=using filter here? > or the slightly less convoluted > > sys.stdout.writelines(map("{}\n".format, read_pdf("book.pdf"))) Actually I am using this function in Android App which is being built using Kivy, Where I am returning whole text into a file, So what you think will be more efficient way? > the for loop is the obvious and therefore recommended solution. Personally, > I would also replace > > > yield from (pg.extractText() for pg in pdf.pages) > > with the good old > > for pg in pdf.pages: > yield pg.extractText() > > and reserve the generator expression for occasions where it has a > demonstrable advantage in readability. But when I am calling pdf_read() from nother function to avoid for loop why it is not working? say: def hello() yield from read_pdf('book.pdf') print(hello()) # still returns memory location instead of text. If I am not wrong yield from can be used to avoid for loop? From rosuav at gmail.com Mon Feb 22 10:46:33 2016 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 23 Feb 2016 02:46:33 +1100 Subject: avoid for loop calling Generator function In-Reply-To: <0c289f93-ce64-4eff-93f3-c70d7ff50817@googlegroups.com> References: <0c289f93-ce64-4eff-93f3-c70d7ff50817@googlegroups.com> Message-ID: On Tue, Feb 23, 2016 at 2:38 AM, Arshpreet Singh wrote: >> next(filter(print, read_pdf("book.pdf")), None) > > Why we are w=using filter here? It's a beautiful hack. It'll filter according to the "print" predicate, which always returns None, and will thus filter everything out. One single call to next() will thus process and print the entire PDF, and then - since it has a second parameter - return None instead of raising StopIteration. Or maybe it's a gross and ugly hack, with the exact same description. ChrisA From ian.g.kelly at gmail.com Mon Feb 22 11:11:11 2016 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Mon, 22 Feb 2016 09:11:11 -0700 Subject: avoid for loop calling Generator function In-Reply-To: <0c289f93-ce64-4eff-93f3-c70d7ff50817@googlegroups.com> References: <0c289f93-ce64-4eff-93f3-c70d7ff50817@googlegroups.com> Message-ID: On Mon, Feb 22, 2016 at 8:38 AM, Arshpreet Singh wrote: > On Monday, 22 February 2016 19:05:24 UTC+5:30, Peter Otten wrote: >> or the slightly less convoluted >> >> sys.stdout.writelines(map("{}\n".format, read_pdf("book.pdf"))) > > Actually I am using this function in Android App which is being built using Kivy, Where I am returning whole text into a file, So what you think will be more efficient way? Profile them and find out, but I don't think you'll find the difference is great enough to be overly concerned with. Pick the way that is more readable and doesn't introduce any gross inefficiencies (such as concatenating strings in a loop). > But when I am calling pdf_read() from nother function to avoid for loop why it is not working? > say: > > def hello() > yield from read_pdf('book.pdf') This uses yield from, which makes it a generator function. > > print(hello()) # still returns memory location instead of text. If I am not wrong yield from can be used to avoid for loop? hello is a generator function, so calling it just creates a generator object. Printing it then prints out the repr of that generator object, which is just something like . Notably, you haven't actually *executed* the generator object, which would require iterating over it, e.g.: for i in hello(): print(i) So you haven't actually avoided creating a for loop; you've just added a redundant layer between the for loop and the thing it's actually iterating over. From bc at freeuk.com Mon Feb 22 11:21:19 2016 From: bc at freeuk.com (BartC) Date: Mon, 22 Feb 2016 16:21:19 +0000 Subject: Considering migrating to Python from Visual Basic 6 for engineering applications In-Reply-To: <56cae711$0$1586$c3e8da3$5496439d@news.astraweb.com> References: <90cc50d2-1ce5-4588-9bfd-a49d439f00dd@googlegroups.com> <14c75a68-0d2e-45cc-8d73-0d71b6a6aea6@googlegroups.com> <9e57761f-26e1-41c5-8e71-23800de1fdd3@googlegroups.com> <7f9c473e-b0c2-4d77-91d1-d0733c93b12d@googlegroups.com> <23d8156f-1808-4395-9c04-27d2984fe67c@googlegroups.com> <56cae711$0$1586$c3e8da3$5496439d@news.astraweb.com> Message-ID: On 22/02/2016 10:46, Steven D'Aprano wrote: > On Mon, 22 Feb 2016 08:52 am, Jussi Piitulainen wrote: > >> BartC writes: > >>> IIRC, the first programming exercise I ever did (in 1976 using Algol >>> 60) involved reading 3 numbers from the teletype and working out if >>> those could form sides of a triangle. >> >> That was a lousy user interface even then - an inflexible user >> interaction without even a possibility of handling errors interactively? >> Command line arguments would have been better (if available, that is). > > Jussi, I think you have an inflated expectation of what was available in > 1976. Command line? What's that? Programs ran in batch mode, > and 'interactive' meant that you could easily slip out one punched card, > replace it with a different one, and run the program again. Our system must have been more advanced then, or designed for training. We used a time-sharing 'dec-system 10' and it was usually accessed via interactive terminals, either teletypes or the odd VDU. It still supported punched cards but that was more because they were still being used in the real world. -- Bartc From alan at csail.mit.edu Mon Feb 22 12:34:15 2016 From: alan at csail.mit.edu (Alan Bawden) Date: Mon, 22 Feb 2016 12:34:15 -0500 Subject: Make a unique filesystem path, without creating the file References: Message-ID: Cameron Simpson writes: > On 16Feb2016 19:24, Alan Bawden wrote: >>So in the FIFO case, I might write something like the following: >> >> def make_temp_fifo(mode=0o600): >> while True: >> path = tempfile.mktemp() >> try: >> os.mkfifo(path, mode=mode) >> except FileExistsError: >> pass >> else: >> return path >> >>So is there something wrong with the above code? Other than the fact >>that the documentation says something scary about mktemp()? > > Well, it has a few shortcomings. > > It relies on mkfifo reliably failing if the name exists. It shounds like > mkfifo is reliable this way, but I can imagine analogous use cases without > such a convenient core action, and your code only avoids mktemp's security > issue _because_ mkfifo has that fortuitous aspect. I don't understand your use of the word "fortuitous" here. mkfifo is defined to act that way according to POSIX. I wrote the code that way precisely because of that property. I sometimes write code knowing that adding two even numbers together results in an even answer. I suppose you might describe that as "fortuitous", but it's just things behaving as they are defined to behave! > Secondly, why is your example better than:: > > os.mkfifo(os.path.join(mkdtemp(), 'myfifo')) My way is not much better, but I think it is a little better because your way I have to worry about deleting both the file and the directory when I am done, and I have to get the permissions right on two filesystem objects. (If I can use a TemporaryDirectory() context manager, the cleaning up part does get easier.) And it also seems wasteful to me, given that the way mkdtemp() is implemented is to generate a possible name, try creating it, and loop if the mkdir() call fails. (POSIX makes the same guarantee for mkdir() as it does for mkfifo().) Why not just let me do an equivalent loop myself? > On that basis, this example doesn't present a use case what can't be > addressed by mkstemp or mkdtemp. Yes, if mktemp() were taken away from me, I could work around it. I'm just saying that in order to justify taking something like this away, it has to be both below some threshold of utility and above some threshold of dangerousness. In the canonical case of gets() in C, not only is fgets() almost a perfectly exact replacement for gets(), gets() is insanely dangerous. But the case of mktemp() doesn't seem to me to come close to this combination of redundancy and danger. > You _do_ understand the security issue, yes? I sure looked like you did, > until here. Well, it's always dangerous to say that you understand all the security issues of anything. In part that is why I wrote the code quoted above. I am open to the possibility that there is a security problem here that I haven't thought of. But so far the only problem anybody has with it is that you think there is something "fortuitous" about the way that it works. >>(As if that would be of any use in the >>situation above!) It looks like anxiety that some people might use >>mktemp() in a stupid way has caused an over-reaction. > > No, it is anxiety that mktemp's _normal_ use is inherently unsafe. So are you saying that the way I used mktemp() above is _abnormal_? > [ Here I have removed some perfectly reasonable text describing the > race condition in question -- yes I really do understand that. ] > > This is neither weird nor even unlikely which is why kmtemp is strongly > discouraged - naive (and standard) use is not safe. > > That you have contrived a use case where you can _carefully_ use mktemp in > safety in no way makes mktemp recommendable. OK, so you _do_ seem to be saying that I have used mktemp() in a "contrived" and "non-standard" (and "non-naive"!) way. I'm genuinely surprised. I though I was just writing straightforward correct code and demonstrating that this was a useful utility that it was not hard to use safely. You seem to think what I did is something that ordinary programmers can not be expected to do. Your judgement is definitely different from mine! And ultimately this does all boil down to making judgements. It does make sense to remove things from libraries that are safety hazards (like gets() in C), I'm just trying to argue that mktemp() isn't nearly dangerous enough to deserve more than a warning in its documentation. You don't agree. Oh well... Up until this point, you haven't said anything that I actually think is flat out wrong, we just disagree about what tools it is reasonable to take away from _all_ programmers just because _some_ programmers might use them to make a mess. > In fact your use case isn't safe, because _another_ task using mktemp > in conflict as a plain old temporary file may grab your fifo. But here in very last sentence I really must disagree. If the code I wrote above is "unsafe" because some _other_ process might be using mktemp() badly and stumble over the same path, then the current implementation of tempfile.mkdtemp() is also "unsafe" for exactly the same reason: some other process using mktemp() badly to create its own directory might accidentally grab the same directory. Heck, that other process doesn't even need to be using mktemp(). It might just be using the hardwired pathname "/tmp/tmp_xyzzy" -- that has _exactly_ the same chance of collision. Surely you don't think that the standard tempfile.mkdtemp() is "unsafe" (in whatever sense of the word you intend) as long as other programmers are allowed to be stupid and use constant pathnames? -- Alan Bawden From ethan at stoneleaf.us Mon Feb 22 13:11:44 2016 From: ethan at stoneleaf.us (Ethan Furman) Date: Mon, 22 Feb 2016 10:11:44 -0800 Subject: Make a unique filesystem path, without creating the file In-Reply-To: <85mvr26dij.fsf@benfinney.id.au> References: <85r3gf55k4.fsf@benfinney.id.au> <85mvr26dij.fsf@benfinney.id.au> Message-ID: <56CB4F60.4010201@stoneleaf.us> On 02/14/2016 04:08 PM, Ben Finney wrote: > I am unconcerned with whether there is a real filesystem entry of that > name; the goal entails having no filesystem activity for this. I want a > valid unique filesystem path, without touching the filesystem. This is impossible. If you don't touch the file system you have no way to know if the path is unique. -- ~Ethan~ From jon+usenet at unequivocal.co.uk Mon Feb 22 13:17:01 2016 From: jon+usenet at unequivocal.co.uk (Jon Ribbens) Date: Mon, 22 Feb 2016 18:17:01 -0000 (UTC) Subject: Make a unique filesystem path, without creating the file References: <85r3gf55k4.fsf@benfinney.id.au> <85mvr26dij.fsf@benfinney.id.au> Message-ID: On 2016-02-22, Ethan Furman wrote: > On 02/14/2016 04:08 PM, Ben Finney wrote: >> I am unconcerned with whether there is a real filesystem entry of that >> name; the goal entails having no filesystem activity for this. I want a >> valid unique filesystem path, without touching the filesystem. > > This is impossible. If you don't touch the file system you have no way > to know if the path is unique. Weeeeeell, I have a lot of sympathy for that point, but on the other hand the whole concept of UUIDs ("import uuid") is predicated on the opposite assumption. From rosuav at gmail.com Mon Feb 22 13:25:08 2016 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 23 Feb 2016 05:25:08 +1100 Subject: Make a unique filesystem path, without creating the file In-Reply-To: References: <85r3gf55k4.fsf@benfinney.id.au> <85mvr26dij.fsf@benfinney.id.au> Message-ID: On Tue, Feb 23, 2016 at 5:17 AM, Jon Ribbens wrote: > On 2016-02-22, Ethan Furman wrote: >> On 02/14/2016 04:08 PM, Ben Finney wrote: >>> I am unconcerned with whether there is a real filesystem entry of that >>> name; the goal entails having no filesystem activity for this. I want a >>> valid unique filesystem path, without touching the filesystem. >> >> This is impossible. If you don't touch the file system you have no way >> to know if the path is unique. > > Weeeeeell, I have a lot of sympathy for that point, but on the other > hand the whole concept of UUIDs ("import uuid") is predicated on the > opposite assumption. Not quite opposite. Ethan is asserting that you cannot be *certain* without actually checking the FS; the point of UUIDs is that you can be fairly *confident* that there won't be a collision. There is a nonzero probability of accidental collisions, and if an attacker is deliberately trying to _force_ a collision, it's most definitely possible. So both views are correct. ChrisA From jon+usenet at unequivocal.co.uk Mon Feb 22 13:39:58 2016 From: jon+usenet at unequivocal.co.uk (Jon Ribbens) Date: Mon, 22 Feb 2016 18:39:58 -0000 (UTC) Subject: Make a unique filesystem path, without creating the file References: <85r3gf55k4.fsf@benfinney.id.au> <85mvr26dij.fsf@benfinney.id.au> Message-ID: On 2016-02-22, Chris Angelico wrote: > On Tue, Feb 23, 2016 at 5:17 AM, Jon Ribbens > wrote: >> Weeeeeell, I have a lot of sympathy for that point, but on the other >> hand the whole concept of UUIDs ("import uuid") is predicated on the >> opposite assumption. > > Not quite opposite. Ethan is asserting that you cannot be *certain* > without actually checking the FS; the point of UUIDs is that you can > be fairly *confident* that there won't be a collision. There is a > nonzero probability of accidental collisions, and if an attacker is > deliberately trying to _force_ a collision, it's most definitely > possible. So both views are correct. I was under the impression that the point of UUIDs is that you can be *so* confident that there won't be a collision that for all practical purposes it's indistinguishable from being certain. From marko at pacujo.net Mon Feb 22 13:48:39 2016 From: marko at pacujo.net (Marko Rauhamaa) Date: Mon, 22 Feb 2016 20:48:39 +0200 Subject: Make a unique filesystem path, without creating the file References: <85r3gf55k4.fsf@benfinney.id.au> <85mvr26dij.fsf@benfinney.id.au> Message-ID: <87ziusmvi0.fsf@elektro.pacujo.net> Jon Ribbens : > I was under the impression that the point of UUIDs is that you can be > *so* confident that there won't be a collision that for all practical > purposes it's indistinguishable from being certain. Yes, if you generate a random 128-bit number, it will be unique -- unless someone clones it. Cloning will be a practical issue when you clone virtual machines, for example. Marko From cpoline95 at gmail.com Mon Feb 22 13:52:46 2016 From: cpoline95 at gmail.com (cpoline95 at gmail.com) Date: Mon, 22 Feb 2016 10:52:46 -0800 (PST) Subject: PyPDF2 merge / out of memory In-Reply-To: References: Message-ID: <100aa09b-2348-4f07-8db8-3325540add25@googlegroups.com> Le dimanche 21 f?vrier 2016 11:42:33 UTC+1, cpol... at gmail.com a ?crit?: > Hello, > > There is an issue with PyPDF2 and merging file > https://github.com/mstamy2/PyPDF2/issues/189 > > Does anybody know an alternate library to merge PDF and produce optimized pdf file ? > > Thanks a lot > > Clement Thanks Mark, I'll dig these tools. Regards, Clement From rosuav at gmail.com Mon Feb 22 14:04:02 2016 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 23 Feb 2016 06:04:02 +1100 Subject: Make a unique filesystem path, without creating the file In-Reply-To: References: <85r3gf55k4.fsf@benfinney.id.au> <85mvr26dij.fsf@benfinney.id.au> Message-ID: On Tue, Feb 23, 2016 at 5:39 AM, Jon Ribbens wrote: > On 2016-02-22, Chris Angelico wrote: >> On Tue, Feb 23, 2016 at 5:17 AM, Jon Ribbens >> wrote: >>> Weeeeeell, I have a lot of sympathy for that point, but on the other >>> hand the whole concept of UUIDs ("import uuid") is predicated on the >>> opposite assumption. >> >> Not quite opposite. Ethan is asserting that you cannot be *certain* >> without actually checking the FS; the point of UUIDs is that you can >> be fairly *confident* that there won't be a collision. There is a >> nonzero probability of accidental collisions, and if an attacker is >> deliberately trying to _force_ a collision, it's most definitely >> possible. So both views are correct. > > I was under the impression that the point of UUIDs is that you can be > *so* confident that there won't be a collision that for all practical > purposes it's indistinguishable from being certain. Maybe, if everyone's cooperating. I'm not sure how they fare in the face of malice though. ChrisA From felipe.gomez.rojas at gmail.com Mon Feb 22 14:18:55 2016 From: felipe.gomez.rojas at gmail.com (felipe.gomez.rojas at gmail.com) Date: Mon, 22 Feb 2016 11:18:55 -0800 (PST) Subject: 0x80070570-The file or directory is corrupted and unreadable In-Reply-To: References: Message-ID: <985ccc90-d68e-4f18-884d-7521f7df4dfa@googlegroups.com> El martes, 22 de diciembre de 2015, 13:46:01 (UTC-3), eryk sun escribi?: > On Tue, Dec 22, 2015 at 8:02 AM, muizz hasan wrote: > > Hi there! I've been recently trying to install Python for Windows 10 > > and I've been encountering some issues. Every time i try to install > > the program it just says"0x80070570-The file or directory is corrupted > > and unreadable". I have attached my log file and i hope that you guys > > might enlighten me on how to solve my problem. Thank you! > > Try downloading a new copy of the installer. Clear your browser cache first. Thank you!! this was usefull!! From jon+usenet at unequivocal.co.uk Mon Feb 22 14:22:35 2016 From: jon+usenet at unequivocal.co.uk (Jon Ribbens) Date: Mon, 22 Feb 2016 19:22:35 -0000 (UTC) Subject: Make a unique filesystem path, without creating the file References: <85r3gf55k4.fsf@benfinney.id.au> <85mvr26dij.fsf@benfinney.id.au> Message-ID: On 2016-02-22, Chris Angelico wrote: > On Tue, Feb 23, 2016 at 5:39 AM, Jon Ribbens > wrote: >> On 2016-02-22, Chris Angelico wrote: >>> On Tue, Feb 23, 2016 at 5:17 AM, Jon Ribbens >>> wrote: >>>> Weeeeeell, I have a lot of sympathy for that point, but on the other >>>> hand the whole concept of UUIDs ("import uuid") is predicated on the >>>> opposite assumption. >>> >>> Not quite opposite. Ethan is asserting that you cannot be *certain* >>> without actually checking the FS; the point of UUIDs is that you can >>> be fairly *confident* that there won't be a collision. There is a >>> nonzero probability of accidental collisions, and if an attacker is >>> deliberately trying to _force_ a collision, it's most definitely >>> possible. So both views are correct. >> >> I was under the impression that the point of UUIDs is that you can be >> *so* confident that there won't be a collision that for all practical >> purposes it's indistinguishable from being certain. > > Maybe, if everyone's cooperating. I'm not sure how they fare in the > face of malice though. Suppose you had code like this: filename = binascii.hexlify(os.urandom(16)).decode("ascii") Do we really think that is insecure or that there are any practical attacks against it? It would be basically the same as saying that urandom() is broken, surely? From no.email at nospam.invalid Mon Feb 22 14:22:56 2016 From: no.email at nospam.invalid (Paul Rubin) Date: Mon, 22 Feb 2016 11:22:56 -0800 Subject: Make a unique filesystem path, without creating the file References: <85r3gf55k4.fsf@benfinney.id.au> <85mvr26dij.fsf@benfinney.id.au> Message-ID: <878u2c4kj3.fsf@jester.gateway.pace.com> Chris Angelico writes: >> I was under the impression that the point of UUIDs is that you can be >> *so* confident that there won't be a collision that for all practical >> purposes it's indistinguishable from being certain. > Maybe, if everyone's cooperating. I'm not sure how they fare in the > face of malice though. There are different UUID algorithms, some of which have useful syntax but are easy to spoof. Uuid4 is random and implemented properly, should be hard to spoof. From marko at pacujo.net Mon Feb 22 14:32:44 2016 From: marko at pacujo.net (Marko Rauhamaa) Date: Mon, 22 Feb 2016 21:32:44 +0200 Subject: Make a unique filesystem path, without creating the file References: <85r3gf55k4.fsf@benfinney.id.au> <85mvr26dij.fsf@benfinney.id.au> Message-ID: <87vb5gmtgj.fsf@elektro.pacujo.net> Jon Ribbens : > Suppose you had code like this: > > filename = binascii.hexlify(os.urandom(16)).decode("ascii") > > Do we really think that is insecure or that there are any practical > attacks against it? It would be basically the same as saying that > urandom() is broken, surely? urandom() is not quite random and so should not be considered cryptographically airtight. Under Linux, /dev/random is the way to go when strong security is needed. Note that /dev/random is a scarce resource on ordinary systems. Marko From rosuav at gmail.com Mon Feb 22 14:37:28 2016 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 23 Feb 2016 06:37:28 +1100 Subject: Make a unique filesystem path, without creating the file In-Reply-To: References: <85r3gf55k4.fsf@benfinney.id.au> <85mvr26dij.fsf@benfinney.id.au> Message-ID: On Tue, Feb 23, 2016 at 6:22 AM, Jon Ribbens wrote: >> Maybe, if everyone's cooperating. I'm not sure how they fare in the >> face of malice though. > > Suppose you had code like this: > > filename = binascii.hexlify(os.urandom(16)).decode("ascii") > > Do we really think that is insecure or that there are any practical > attacks against it? It would be basically the same as saying that > urandom() is broken, surely? Sure, that would be safe. But UUIDs aren't necessarily based on "give me sixteen bytes from urandom". They can involve potentially-predictable information such as MAC addresses, current time of day, and so on, which gives them significantly less randomness. In that kind of usage, they're not intended to be cryptographically secure. ChrisA From random832 at fastmail.com Mon Feb 22 14:41:01 2016 From: random832 at fastmail.com (Random832) Date: Mon, 22 Feb 2016 14:41:01 -0500 Subject: Make a unique filesystem path, without creating the file In-Reply-To: <87vb5gmtgj.fsf@elektro.pacujo.net> References: <85r3gf55k4.fsf@benfinney.id.au> <85mvr26dij.fsf@benfinney.id.au> <87vb5gmtgj.fsf@elektro.pacujo.net> Message-ID: <1456170061.274843.528595898.3AB485CD@webmail.messagingengine.com> On Mon, Feb 22, 2016, at 14:32, Marko Rauhamaa wrote: > urandom() is not quite random and so should not be considered > cryptographically airtight. > > Under Linux, /dev/random is the way to go when strong security is > needed. Note that /dev/random is a scarce resource on ordinary systems. http://www.2uo.de/myths-about-urandom/ From marko at pacujo.net Mon Feb 22 15:41:58 2016 From: marko at pacujo.net (Marko Rauhamaa) Date: Mon, 22 Feb 2016 22:41:58 +0200 Subject: Make a unique filesystem path, without creating the file References: <85r3gf55k4.fsf@benfinney.id.au> <85mvr26dij.fsf@benfinney.id.au> <87vb5gmtgj.fsf@elektro.pacujo.net> Message-ID: <87povomq95.fsf@elektro.pacujo.net> Random832 : > On Mon, Feb 22, 2016, at 14:32, Marko Rauhamaa wrote: >> urandom() is not quite random and so should not be considered >> cryptographically airtight. >> >> Under Linux, /dev/random is the way to go when strong security is >> needed. Note that /dev/random is a scarce resource on ordinary >> systems. > > http://www.2uo.de/myths-about-urandom/ Did you post the link because you agreed with the Web pamphlet? Marko From no.email at nospam.invalid Mon Feb 22 16:05:14 2016 From: no.email at nospam.invalid (Paul Rubin) Date: Mon, 22 Feb 2016 13:05:14 -0800 Subject: Make a unique filesystem path, without creating the file References: <85r3gf55k4.fsf@benfinney.id.au> <85mvr26dij.fsf@benfinney.id.au> <87vb5gmtgj.fsf@elektro.pacujo.net> <87povomq95.fsf@elektro.pacujo.net> Message-ID: <874md04fsl.fsf@jester.gateway.pace.com> Marko Rauhamaa writes: >> http://www.2uo.de/myths-about-urandom/ > Did you post the link because you agreed with the Web pamphlet? I don't know what web pamphlet you mean, but the right thing to use now is getrandom(2). The random/urandom interface was poorly designed and misleadingly documented. From marko at pacujo.net Mon Feb 22 16:22:24 2016 From: marko at pacujo.net (Marko Rauhamaa) Date: Mon, 22 Feb 2016 23:22:24 +0200 Subject: Make a unique filesystem path, without creating the file References: <85r3gf55k4.fsf@benfinney.id.au> <85mvr26dij.fsf@benfinney.id.au> <87vb5gmtgj.fsf@elektro.pacujo.net> <87povomq95.fsf@elektro.pacujo.net> <874md04fsl.fsf@jester.gateway.pace.com> Message-ID: <87lh6cmodr.fsf@elektro.pacujo.net> Paul Rubin : >>> http://www.2uo.de/myths-about-urandom/ >> Did you post the link because you agreed with the Web pamphlet? > > I don't know what web pamphlet you mean, The only one linked above. Cryptography is tricky business, indeed. I know enough about it not to improvise too much. Infinitesimal weaknesses can make a difference between feasible and unfeasible attacks. > but the right thing to use now is getrandom(2). getrandom(2) is a good interface that distinguishes between the flag values 0 => /dev/urandom GRND_RANDOM => /dev/random GRND_RANDOM | GRND_NONBLOCK => /dev/random (O_NONBLOCK) However, although os.urandom() delegates to getrandom(), the documentation suggests it uses the flag value 0 (/dev/urandom). > The random/urandom interface was poorly designed and misleadingly > documented. It could be better I suppose, but I never found it particularly bad. The nice thing about it is that it is readily usable in shell scripts. Marko From greg.ewing at canterbury.ac.nz Mon Feb 22 16:45:50 2016 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Tue, 23 Feb 2016 10:45:50 +1300 Subject: Considering migrating to Python from Visual Basic 6 for engineering applications In-Reply-To: References: <90cc50d2-1ce5-4588-9bfd-a49d439f00dd@googlegroups.com> <14c75a68-0d2e-45cc-8d73-0d71b6a6aea6@googlegroups.com> <9e57761f-26e1-41c5-8e71-23800de1fdd3@googlegroups.com> <7f9c473e-b0c2-4d77-91d1-d0733c93b12d@googlegroups.com> <23d8156f-1808-4395-9c04-27d2984fe67c@googlegroups.com> <56cae711$0$1586$c3e8da3$5496439d@news.astraweb.com> Message-ID: BartC wrote: > Our system must have been more advanced then, or designed for training. > We used a time-sharing 'dec-system 10' and it was usually accessed via > interactive terminals, either teletypes or the odd VDU. According to Wikipedia the first interactive version of Dartmouth BASIC appeared in 1964: https://en.wikipedia.org/wiki/Dartmouth_BASIC Also, the *very* earliest computer systems were all interactive -- you sat in front of a panel flipping switches and reading lights! -- Greg From srkunze at mail.de Mon Feb 22 17:16:53 2016 From: srkunze at mail.de (Sven R. Kunze) Date: Mon, 22 Feb 2016 23:16:53 +0100 Subject: How the heck does async/await work in Python 3.5 In-Reply-To: References: <56c7d145$0$1597$c3e8da3$5496439d@news.astraweb.com> Message-ID: <56CB88D5.3090202@mail.de> On 20.02.2016 07:53, Christian Gollwitzer wrote: > If you have difficulties wit hthe overall concept, and if you are open > to discussions in another language, take a look at this video: > > https://channel9.msdn.com/Shows/C9-GoingNative/GoingNative-39-await-co-routines > > > MS has added coroutine support with very similar syntax to VC++ > recently, and the developer tries to explain it to the "stackful" > programmers. Thanks, Christian. Very informative video. Is something like shown in 12:50 ( cout << tcp_reader(1000).get() ) possible with asyncio? (tcp_reader would be async def) Best, Sven From cs at zip.com.au Mon Feb 22 17:25:36 2016 From: cs at zip.com.au (Cameron Simpson) Date: Tue, 23 Feb 2016 09:25:36 +1100 Subject: Make a unique filesystem path, without creating the file In-Reply-To: <56CB4F60.4010201@stoneleaf.us> References: <56CB4F60.4010201@stoneleaf.us> Message-ID: <20160222222536.GA84710@cskk.homeip.net> On 22Feb2016 10:11, Ethan Furman wrote: >On 02/14/2016 04:08 PM, Ben Finney wrote: > >>I am unconcerned with whether there is a real filesystem entry of that >>name; the goal entails having no filesystem activity for this. I want a >>valid unique filesystem path, without touching the filesystem. > >This is impossible. If you don't touch the file system you have no way to >know if the path is unique. I think Ben wants to avoid filesystem modification (let us ignore atime here). So one can read the filesystem to see what is current, but he does not want to actually make any new filesystem entry. Cheers, Cameron Simpson From ben+python at benfinney.id.au Mon Feb 22 17:48:59 2016 From: ben+python at benfinney.id.au (Ben Finney) Date: Tue, 23 Feb 2016 09:48:59 +1100 Subject: Make a unique filesystem path, without creating the file References: <85r3gf55k4.fsf@benfinney.id.au> <85mvr26dij.fsf@benfinney.id.au> <56CB4F60.4010201@stoneleaf.us> Message-ID: <85povoz7hg.fsf@benfinney.id.au> Ethan Furman writes: > On 02/14/2016 04:08 PM, Ben Finney wrote: > > > I am unconcerned with whether there is a real filesystem entry of that > > name; the goal entails having no filesystem activity for this. I want a > > valid unique filesystem path, without touching the filesystem. > > This is impossible. If you don't touch the file system you have no > way to know if the path is unique. That was unclear. Later in the same thread, I clarified that by ?unique? I mean nothing about entries already on the filesystem. Instead it means ?unpredictably different each time the function is called?. -- \ ?It is difficult to get a man to understand something when his | `\ salary depends upon his not understanding it.? ?Upton Sinclair, | _o__) 1935 | Ben Finney From ethan at stoneleaf.us Mon Feb 22 17:51:34 2016 From: ethan at stoneleaf.us (Ethan Furman) Date: Mon, 22 Feb 2016 14:51:34 -0800 Subject: Make a unique filesystem path, without creating the file In-Reply-To: <20160222222536.GA84710@cskk.homeip.net> References: <56CB4F60.4010201@stoneleaf.us> <20160222222536.GA84710@cskk.homeip.net> Message-ID: <56CB90F6.1040106@stoneleaf.us> On 02/22/2016 02:25 PM, Cameron Simpson wrote: > On 22Feb2016 10:11, Ethan Furman wrote: >> On 02/14/2016 04:08 PM, Ben Finney wrote: >> >>> I am unconcerned with whether there is a real filesystem entry of that >>> name; the goal entails having no filesystem activity for this. I want a >>> valid unique filesystem path, without touching the filesystem. >> >> This is impossible. If you don't touch the file system you have no >> way to know if the path is unique. > > I think Ben wants to avoid filesystem modification (let us ignore atime > here). So one can read the filesystem to see what is current, but he > does not want to actually make any new filesystem entry. Hmm -- well, he says "the goal entails having no filesystem activity for this", and seeing what already exists definitely requires file system activity . . . -- ~Ethan~ From cs at zip.com.au Mon Feb 22 18:02:48 2016 From: cs at zip.com.au (Cameron Simpson) Date: Tue, 23 Feb 2016 10:02:48 +1100 Subject: Make a unique filesystem path, without creating the file In-Reply-To: References: Message-ID: <20160222230248.GA90747@cskk.homeip.net> On 22Feb2016 12:34, Alan Bawden wrote: >Cameron Simpson writes: > >> On 16Feb2016 19:24, Alan Bawden wrote: >>>So in the FIFO case, I might write something like the following: >>> >>> def make_temp_fifo(mode=0o600): >>> while True: >>> path = tempfile.mktemp() >>> try: >>> os.mkfifo(path, mode=mode) >>> except FileExistsError: >>> pass >>> else: >>> return path >>> >>>So is there something wrong with the above code? Other than the fact >>>that the documentation says something scary about mktemp()? >> >> Well, it has a few shortcomings. >> >> It relies on mkfifo reliably failing if the name exists. It shounds like >> mkfifo is reliable this way, but I can imagine analogous use cases without >> such a convenient core action, and your code only avoids mktemp's security >> issue _because_ mkfifo has that fortuitous aspect. > >I don't understand your use of the word "fortuitous" here. mkfifo is >defined to act that way according to POSIX. I wrote the code that way >precisely because of that property. I sometimes write code knowing that >adding two even numbers together results in an even answer. I suppose >you might describe that as "fortuitous", but it's just things behaving >as they are defined to behave! I mean here that your scheme isn't adaptable to a system call which will reuse an existing name. Of course, mkfifo, mkdir and open(.., O_EXCL) all have this nice feature. >> Secondly, why is your example better than:: >> os.mkfifo(os.path.join(mkdtemp(), 'myfifo')) > >My way is not much better, but I think it is a little better because >your way I have to worry about deleting both the file and the directory >when I am done, and I have to get the permissions right on two >filesystem objects. (If I can use a TemporaryDirectory() context >manager, the cleaning up part does get easier.) > >And it also seems wasteful to me, given that the way mkdtemp() is >implemented is to generate a possible name, try creating it, and loop if >the mkdir() call fails. (POSIX makes the same guarantee for mkdir() as >it does for mkfifo().) Why not just let me do an equivalent loop >myself? Go ahead. But I think Ben's specificly trying to avoid writing his own loop. >> On that basis, this example doesn't present a use case what can't be >> addressed by mkstemp or mkdtemp. > >Yes, if mktemp() were taken away from me, I could work around it. I'm >just saying that in order to justify taking something like this away, it >has to be both below some threshold of utility and above some threshold >of dangerousness. In the canonical case of gets() in C, not only is >fgets() almost a perfectly exact replacement for gets(), gets() is >insanely dangerous. But the case of mktemp() doesn't seem to me to come >close to this combination of redundancy and danger. > >> You _do_ understand the security issue, yes? I sure looked like you did, >> until here. > >Well, it's always dangerous to say that you understand all the security >issues of anything. In part that is why I wrote the code quoted above. >I am open to the possibility that there is a security problem here that >I haven't thought of. But so far the only problem anybody has with it >is that you think there is something "fortuitous" about the way that it >works. > >>>(As if that would be of any use in the >>>situation above!) It looks like anxiety that some people might use >>>mktemp() in a stupid way has caused an over-reaction. >> >> No, it is anxiety that mktemp's _normal_ use is inherently unsafe. > >So are you saying that the way I used mktemp() above is _abnormal_? In that you're not making a file. I mean "abnormal" in a statistical sense, and also in the "anticipated use case for mktemp's design". I'm not suggestioning you're wrong to use it like this. >> [ Here I have removed some perfectly reasonable text describing the >> race condition in question -- yes I really do understand that. ] >> >> This is neither weird nor even unlikely which is why kmtemp is strongly >> discouraged - naive (and standard) use is not safe. >> >> That you have contrived a use case where you can _carefully_ use mktemp in >> safety in no way makes mktemp recommendable. > >OK, so you _do_ seem to be saying that I have used mktemp() in a >"contrived" and "non-standard" (and "non-naive"!) way. I'm genuinely >surprised. I though I was just writing straightforward correct code and >demonstrating that this was a useful utility that it was not hard to use >safely. You seem to think what I did is something that ordinary >programmers can not be expected to do. Your judgement is definitely >different from mine! No, I meant only that (a) mktemp is normally used for regular files and (b) that mkdtemp()/mkfifo() present equivalent results without hand making a pick-a-name loop. Of course any programmer should be able to read the mktemp() spec and built from it. >And ultimately this does all boil down to making judgements. It does >make sense to remove things from libraries that are safety hazards (like >gets() in C), I'm just trying to argue that mktemp() isn't nearly >dangerous enough to deserve more than a warning in its documentation. >You don't agree. Oh well... I think mktemp() is like system() (or popen with a shell command string instead of a list of arguments suitable for exec()): it is easy and obvious and there is a much safe tool sitting right beside it that will frequently be overlooked, and that is why I'm on the "discourage its use" side of the discussion, even to the point of deprecating its use. It isn't that it cannot be carefully used ever, it is that it is too easy to use when there is a more reliable way to do it. >Up until this point, you haven't said anything that I actually think is >flat out wrong, we just disagree about what tools it is reasonable to >take away from _all_ programmers just because _some_ programmers might >use them to make a mess. > >> In fact your use case isn't safe, because _another_ task using mktemp >> in conflict as a plain old temporary file may grab your fifo. > >But here in very last sentence I really must disagree. If the code I >wrote above is "unsafe" because some _other_ process might be using >mktemp() badly and stumble over the same path, then the current >implementation of tempfile.mkdtemp() is also "unsafe" for exactly the >same reason: some other process using mktemp() badly to create its own >directory might accidentally grab the same directory. When the other taks goes mkdir with the generated name it will fail, so no. In your mkfifo case, another task using mktemp thus: path = mktemp() tmpfp = open(path, "w") can succeed in the circumstance where your fifo gets made first. >Heck, that other process doesn't even need to be using mktemp(). It >might just be using the hardwired pathname "/tmp/tmp_xyzzy" -- that has >_exactly_ the same chance of collision. Surely you don't think that the >standard tempfile.mkdtemp() is "unsafe" (in whatever sense of the word >you intend) as long as other programmers are allowed to be stupid and >use constant pathnames? Of course not. Cheers, Cameron Simpson From no.email at nospam.invalid Mon Feb 22 18:26:29 2016 From: no.email at nospam.invalid (Paul Rubin) Date: Mon, 22 Feb 2016 15:26:29 -0800 Subject: Make a unique filesystem path, without creating the file References: <85r3gf55k4.fsf@benfinney.id.au> <85mvr26dij.fsf@benfinney.id.au> <87vb5gmtgj.fsf@elektro.pacujo.net> <87povomq95.fsf@elektro.pacujo.net> <874md04fsl.fsf@jester.gateway.pace.com> <87lh6cmodr.fsf@elektro.pacujo.net> Message-ID: <87wppw2uoq.fsf@jester.gateway.pace.com> Marko Rauhamaa writes: >>>> http://www.2uo.de/myths-about-urandom/ >> I don't know what web pamphlet you mean, > The only one linked above. Oh, I wouldn't have called that a pamphlet. I could quibble with the writing style but the points in the article are basically correct. > getrandom(2) is a good interface that distinguishes between the flag > values > 0 => /dev/urandom > GRND_RANDOM => /dev/random > GRND_RANDOM | GRND_NONBLOCK => /dev/random (O_NONBLOCK) > However, although os.urandom() delegates to getrandom(), the > documentation suggests it uses the flag value 0 (/dev/urandom). Flag value 0 does the right thing and blocks if the entropy pool is not yet initialized, and doesn't block after that. That fixes the errors of both urandom (fails to block before there's enough entropy) and random (blocks even after there's enough entropy). The getrandom doc is also misleading about the workings of the entropy pools but that's ok. The actual algorithm is described here: http://www.pinkas.net/PAPERS/gpr06.pdf It's pretty clumsy but discussions about replacing it have gotten bogged down several times. OTOH maybe I'm out of date on this. >> The random/urandom interface was poorly designed and misleadingly >> documented. > It could be better I suppose, but I never found it particularly bad. The > nice thing about it is that it is readily usable in shell scripts. DJB describes the problems: https://groups.google.com/forum/#!msg/randomness-generation/4opmDHA6_3w/__TyKhbnNWsJ Regarding shell scripts, it should be a simple matter to put a wrapper around the system call. From steve at pearwood.info Mon Feb 22 18:37:23 2016 From: steve at pearwood.info (Steven D'Aprano) Date: Tue, 23 Feb 2016 10:37:23 +1100 Subject: Make a unique filesystem path, without creating the file References: <85r3gf55k4.fsf@benfinney.id.au> <85mvr26dij.fsf@benfinney.id.au> <87ziusmvi0.fsf@elektro.pacujo.net> Message-ID: <56cb9bb5$0$1595$c3e8da3$5496439d@news.astraweb.com> On Tue, 23 Feb 2016 05:48 am, Marko Rauhamaa wrote: > Jon Ribbens : > >> I was under the impression that the point of UUIDs is that you can be >> *so* confident that there won't be a collision that for all practical >> purposes it's indistinguishable from being certain. > > Yes, if you generate a random 128-bit number, it will be unique -- If you generate a second random 128 bit number, you have a chance of 1 in 2**128 of a collision. All you can say is that it will be *very probably* unique. (I might even allow "almost certainly" unique.) If you generate 2**128 + 1 such numbers, you are *guaranteed* to have at least one collision. If I can arrange matters so that I am using the same seed as you, then I can generate the same UUIDs as you. If I know you are using the Mersenne Twister PRNG, and I can get hold of (by memory) 128 consecutive UUIDs, I can reconstruct the seed you are using and generate all future (and past) UUIDs the same as yours. (Well, when I say "I can", I don't mean *me*, I mean some attacker who is smarter than me, but not that much smarter.) > unless someone clones it. > > Cloning will be a practical issue when you clone virtual machines, for > example. This is certainly a practical issue that people have to be aware of. -- Steven From steve at pearwood.info Mon Feb 22 18:45:37 2016 From: steve at pearwood.info (Steven D'Aprano) Date: Tue, 23 Feb 2016 10:45:37 +1100 Subject: Make a unique filesystem path, without creating the file References: <85r3gf55k4.fsf@benfinney.id.au> <85mvr26dij.fsf@benfinney.id.au> <878u2c4kj3.fsf@jester.gateway.pace.com> Message-ID: <56cb9da3$0$1583$c3e8da3$5496439d@news.astraweb.com> On Tue, 23 Feb 2016 06:22 am, Paul Rubin wrote: > Chris Angelico writes: >>> I was under the impression that the point of UUIDs is that you can be >>> *so* confident that there won't be a collision that for all practical >>> purposes it's indistinguishable from being certain. >> Maybe, if everyone's cooperating. I'm not sure how they fare in the >> face of malice though. > > There are different UUID algorithms, some of which have useful syntax > but are easy to spoof. Uuid4 is random and implemented properly, should > be hard to spoof. I'm not sure what you mean by "spoof" in this context. Do you mean generate collisions? Do you mean "pretend to generate a UUID, but without actually doing so"? That's how I interpret "spoof", but I don't quite understand why that would be difficult. Here's one I just made now: {00010203-0405-0607-0809-0a0b0c0d0e0f} And another: {836313e2-3b8a-53f2-9b90-0c9ade199e5d} They weren't hard to spoof :-) -- Steven From oscar.j.benjamin at gmail.com Mon Feb 22 18:59:25 2016 From: oscar.j.benjamin at gmail.com (Oscar Benjamin) Date: Mon, 22 Feb 2016 23:59:25 +0000 Subject: Make a unique filesystem path, without creating the file In-Reply-To: <85povoz7hg.fsf@benfinney.id.au> References: <85r3gf55k4.fsf@benfinney.id.au> <85mvr26dij.fsf@benfinney.id.au> <56CB4F60.4010201@stoneleaf.us> <85povoz7hg.fsf@benfinney.id.au> Message-ID: On 22 Feb 2016 22:50, "Ben Finney" wrote: > > Ethan Furman writes: > > > On 02/14/2016 04:08 PM, Ben Finney wrote: > > > > > I am unconcerned with whether there is a real filesystem entry of that > > > name; the goal entails having no filesystem activity for this. I want a > > > valid unique filesystem path, without touching the filesystem. > > > > This is impossible. If you don't touch the file system you have no > > way to know if the path is unique. > > That was unclear. Later in the same thread, I clarified that by ?unique? > I mean nothing about entries already on the filesystem. Instead it means > ?unpredictably different each time the function is called?. What does unpredictable mean in this context? Maybe I'm reading too much into that... What's wrong with the example I posted before? -- Oscar From steve at pearwood.info Mon Feb 22 19:03:06 2016 From: steve at pearwood.info (Steven D'Aprano) Date: Tue, 23 Feb 2016 11:03:06 +1100 Subject: Make a unique filesystem path, without creating the file References: <85r3gf55k4.fsf@benfinney.id.au> <85mvr26dij.fsf@benfinney.id.au> Message-ID: <56cba1bc$0$1604$c3e8da3$5496439d@news.astraweb.com> On Tue, 23 Feb 2016 06:22 am, Jon Ribbens wrote: > Suppose you had code like this: > > filename = binascii.hexlify(os.urandom(16)).decode("ascii") > > Do we really think that is insecure or that there are any practical > attacks against it? It would be basically the same as saying that > urandom() is broken, surely? Correct. Any attack against urandom would be an attack on this. You would just have to trust that the kernel devs have made urandom as secure as possible, and pay no attention to what the man page says, as its wrong. By the way, Python 3.6 will have (once Guido formally approves it) a new module, "secrets", for securely generating (pseudo)random tokens like this: import secrets filename = secrets.token_hex(16) https://www.python.org/dev/peps/pep-0506/ -- Steven From jon+usenet at unequivocal.co.uk Mon Feb 22 19:08:48 2016 From: jon+usenet at unequivocal.co.uk (Jon Ribbens) Date: Tue, 23 Feb 2016 00:08:48 -0000 (UTC) Subject: Make a unique filesystem path, without creating the file References: <85r3gf55k4.fsf@benfinney.id.au> <85mvr26dij.fsf@benfinney.id.au> <87ziusmvi0.fsf@elektro.pacujo.net> <56cb9bb5$0$1595$c3e8da3$5496439d@news.astraweb.com> Message-ID: On 2016-02-22, Steven D'Aprano wrote: > On Tue, 23 Feb 2016 05:48 am, Marko Rauhamaa wrote: >> Jon Ribbens : >>> I was under the impression that the point of UUIDs is that you can be >>> *so* confident that there won't be a collision that for all practical >>> purposes it's indistinguishable from being certain. >> >> Yes, if you generate a random 128-bit number, it will be unique -- > > If you generate a second random 128 bit number, you have a chance of 1 in > 2**128 of a collision. All you can say is that it will be *very probably* > unique. (I might even allow "almost certainly" unique.) If you are not prepared to say that something with a 340282366920938463463374607431768211455 / 340282366920938463463374607431768211456 chance of being true is not "certainly true" then I'm not sure how you would not be too scared to ever leave the house. Or not leave the house. I mean, you're probably going to be hit by 10^25 meteorites, which sounds painful. > If you generate 2**128 + 1 such numbers, you are *guaranteed* to ... have expired due to the heat death of the universe. From jon+usenet at unequivocal.co.uk Mon Feb 22 19:11:05 2016 From: jon+usenet at unequivocal.co.uk (Jon Ribbens) Date: Tue, 23 Feb 2016 00:11:05 -0000 (UTC) Subject: Make a unique filesystem path, without creating the file References: <85r3gf55k4.fsf@benfinney.id.au> <85mvr26dij.fsf@benfinney.id.au> <56cba1bc$0$1604$c3e8da3$5496439d@news.astraweb.com> Message-ID: On 2016-02-23, Steven D'Aprano wrote: > On Tue, 23 Feb 2016 06:22 am, Jon Ribbens wrote: >> Suppose you had code like this: >> >> filename = binascii.hexlify(os.urandom(16)).decode("ascii") >> >> Do we really think that is insecure or that there are any practical >> attacks against it? It would be basically the same as saying that >> urandom() is broken, surely? > > Correct. Any attack against urandom would be an attack on this. You would > just have to trust that the kernel devs have made urandom as secure as > possible, and pay no attention to what the man page says, as its wrong. > > By the way, Python 3.6 will have (once Guido formally approves it) a new > module, "secrets", for securely generating (pseudo)random tokens like this: > > import secrets > filename = secrets.token_hex(16) +1 From ben+python at benfinney.id.au Mon Feb 22 19:18:20 2016 From: ben+python at benfinney.id.au (Ben Finney) Date: Tue, 23 Feb 2016 11:18:20 +1100 Subject: Make a unique filesystem path, without creating the file References: <85r3gf55k4.fsf@benfinney.id.au> <85mvr26dij.fsf@benfinney.id.au> <56CB4F60.4010201@stoneleaf.us> <85povoz7hg.fsf@benfinney.id.au> Message-ID: <85lh6cz3cj.fsf@benfinney.id.au> Oscar Benjamin writes: > What does unpredictable mean in this context? Maybe I'm reading too > much into that... I think you may be, yes. The request in this thread requires making direct use of the ?generate a new valid temporary fielsystem path? functionality already implemented in ?tempfile?. Implementations of that functionality outside of ?tempfile? are a fun exercise, but miss the point of this thread. -- \ ?But Marge, what if we chose the wrong religion? Each week we | `\ just make God madder and madder.? ?Homer, _The Simpsons_ | _o__) | Ben Finney From steve at pearwood.info Mon Feb 22 19:18:34 2016 From: steve at pearwood.info (Steven D'Aprano) Date: Tue, 23 Feb 2016 11:18:34 +1100 Subject: Make a unique filesystem path, without creating the file References: <85r3gf55k4.fsf@benfinney.id.au> <85mvr26dij.fsf@benfinney.id.au> Message-ID: <56cba55c$0$1605$c3e8da3$5496439d@news.astraweb.com> On Tue, 23 Feb 2016 05:17 am, Jon Ribbens wrote: > On 2016-02-22, Ethan Furman wrote: >> On 02/14/2016 04:08 PM, Ben Finney wrote: >>> I am unconcerned with whether there is a real filesystem entry of that >>> name; the goal entails having no filesystem activity for this. I want a >>> valid unique filesystem path, without touching the filesystem. >> >> This is impossible. If you don't touch the file system you have no way >> to know if the path is unique. > > Weeeeeell, I have a lot of sympathy for that point, but on the other > hand the whole concept of UUIDs ("import uuid") is predicated on the > opposite assumption. You're referring to uuid4, presumably, as the other varieties of UUID use non-secret information, such as the time, or a namespace, either of which is potentially public knowledge. Only uuid4 is considered "globally unique", and that's not *certainly* globally unique, only that the chances of an *accidental* collision is below some threshold deemed "small enough that we don't care". Deliberate collisions of public UUIDs are *trivial*. Pick a UUID you know is already in use, and use it again. There's a lot of assumptions involved in the "globally unique" claim, and there are probably ways to contrive to generate the same UUIDs as someone else. But to what benefit? UUIDs are not intended as security tokens, and are not hardened against attack. Even uuid4 may not be suitable for security, since it may use a cryptographically weak PRNG such as Mersenne Twister. -- Steven From rosuav at gmail.com Mon Feb 22 19:18:51 2016 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 23 Feb 2016 11:18:51 +1100 Subject: Make a unique filesystem path, without creating the file In-Reply-To: References: <85r3gf55k4.fsf@benfinney.id.au> <85mvr26dij.fsf@benfinney.id.au> <87ziusmvi0.fsf@elektro.pacujo.net> <56cb9bb5$0$1595$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Tue, Feb 23, 2016 at 11:08 AM, Jon Ribbens wrote: > On 2016-02-22, Steven D'Aprano wrote: >> On Tue, 23 Feb 2016 05:48 am, Marko Rauhamaa wrote: >>> Jon Ribbens : >>>> I was under the impression that the point of UUIDs is that you can be >>>> *so* confident that there won't be a collision that for all practical >>>> purposes it's indistinguishable from being certain. >>> >>> Yes, if you generate a random 128-bit number, it will be unique -- >> >> If you generate a second random 128 bit number, you have a chance of 1 in >> 2**128 of a collision. All you can say is that it will be *very probably* >> unique. (I might even allow "almost certainly" unique.) > > If you are not prepared to say that something with a > 340282366920938463463374607431768211455 / > 340282366920938463463374607431768211456 chance of being true > is not "certainly true" then I'm not sure how you would not > be too scared to ever leave the house. Or not leave the house. > I mean, you're probably going to be hit by 10^25 meteorites, > which sounds painful. > >> If you generate 2**128 + 1 such numbers, you are *guaranteed* to > > ... have expired due to the heat death of the universe. Maybe... but by the time you get to 2**64 of them, you have a 50% chance of a collision. (That's either utterly intuitive or completely counter-intuitive, depending on who you are.) ChrisA From jon+usenet at unequivocal.co.uk Mon Feb 22 19:26:01 2016 From: jon+usenet at unequivocal.co.uk (Jon Ribbens) Date: Tue, 23 Feb 2016 00:26:01 -0000 (UTC) Subject: Make a unique filesystem path, without creating the file References: <85r3gf55k4.fsf@benfinney.id.au> <85mvr26dij.fsf@benfinney.id.au> <87ziusmvi0.fsf@elektro.pacujo.net> <56cb9bb5$0$1595$c3e8da3$5496439d@news.astraweb.com> Message-ID: On 2016-02-23, Chris Angelico wrote: > On Tue, Feb 23, 2016 at 11:08 AM, Jon Ribbens > wrote: >>> If you generate 2**128 + 1 such numbers, you are *guaranteed* to >> >> ... have expired due to the heat death of the universe. > > Maybe... but by the time you get to 2**64 of them, you have a 50% > chance of a collision. (That's either utterly intuitive or completely > counter-intuitive, depending on who you are.) Um, did you mean to say 2**127? Are you thinking of the birthday paradox or something, which doesn't apply here? From steve at pearwood.info Mon Feb 22 19:33:09 2016 From: steve at pearwood.info (Steven D'Aprano) Date: Tue, 23 Feb 2016 11:33:09 +1100 Subject: Make a unique filesystem path, without creating the file References: <85r3gf55k4.fsf@benfinney.id.au> <85mvr26dij.fsf@benfinney.id.au> <87vb5gmtgj.fsf@elektro.pacujo.net> Message-ID: <56cba8c7$0$1611$c3e8da3$5496439d@news.astraweb.com> On Tue, 23 Feb 2016 06:32 am, Marko Rauhamaa wrote: > Jon Ribbens : > >> Suppose you had code like this: >> >> filename = binascii.hexlify(os.urandom(16)).decode("ascii") >> >> Do we really think that is insecure or that there are any practical >> attacks against it? It would be basically the same as saying that >> urandom() is broken, surely? > > urandom() is not quite random and so should not be considered > cryptographically airtight. > > Under Linux, /dev/random is the way to go when strong security is > needed. Note that /dev/random is a scarce resource on ordinary systems. That's actually incorrect, but you're not the only one to have been mislead by the man pages. http://sockpuppet.org/blog/2014/02/25/safely-generate-random-numbers/ On non-Linux Unixes, the difference between urandom and random is mostly, or entirely, gone, in favour of urandom's non-blocking behaviour. And it's a myth that the output of random is "more random" or "more pure" than urandom's. In reality, on Linux both urandom and random use exactly the same CSPRNG. See also: http://www.2uo.de/myths-about-urandom/ for a good explanation of how random and urandom actually work on Linux. -- Steven From rosuav at gmail.com Mon Feb 22 19:33:55 2016 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 23 Feb 2016 11:33:55 +1100 Subject: Make a unique filesystem path, without creating the file In-Reply-To: References: <85r3gf55k4.fsf@benfinney.id.au> <85mvr26dij.fsf@benfinney.id.au> <87ziusmvi0.fsf@elektro.pacujo.net> <56cb9bb5$0$1595$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Tue, Feb 23, 2016 at 11:26 AM, Jon Ribbens wrote: > On 2016-02-23, Chris Angelico wrote: >> On Tue, Feb 23, 2016 at 11:08 AM, Jon Ribbens >> wrote: >>>> If you generate 2**128 + 1 such numbers, you are *guaranteed* to >>> >>> ... have expired due to the heat death of the universe. >> >> Maybe... but by the time you get to 2**64 of them, you have a 50% >> chance of a collision. (That's either utterly intuitive or completely >> counter-intuitive, depending on who you are.) > > Um, did you mean to say 2**127? Are you thinking of the > birthday paradox or something, which doesn't apply here? By the time you generate 2**64 of them, you have a 50% chance that some pair of them collides. Yes, the birthday paradox does apply here. ChrisA From jon+usenet at unequivocal.co.uk Mon Feb 22 19:44:39 2016 From: jon+usenet at unequivocal.co.uk (Jon Ribbens) Date: Tue, 23 Feb 2016 00:44:39 -0000 (UTC) Subject: Make a unique filesystem path, without creating the file References: <85r3gf55k4.fsf@benfinney.id.au> <85mvr26dij.fsf@benfinney.id.au> <87ziusmvi0.fsf@elektro.pacujo.net> <56cb9bb5$0$1595$c3e8da3$5496439d@news.astraweb.com> Message-ID: On 2016-02-23, Chris Angelico wrote: > On Tue, Feb 23, 2016 at 11:26 AM, Jon Ribbens > wrote: >> On 2016-02-23, Chris Angelico wrote: >>> On Tue, Feb 23, 2016 at 11:08 AM, Jon Ribbens >>> wrote: >>>>> If you generate 2**128 + 1 such numbers, you are *guaranteed* to >>>> >>>> ... have expired due to the heat death of the universe. >>> >>> Maybe... but by the time you get to 2**64 of them, you have a 50% >>> chance of a collision. (That's either utterly intuitive or completely >>> counter-intuitive, depending on who you are.) >> >> Um, did you mean to say 2**127? Are you thinking of the >> birthday paradox or something, which doesn't apply here? > > By the time you generate 2**64 of them, you have a 50% chance that > some pair of them collides. Yes, the birthday paradox does apply here. Oh, I see, you're thinking of it differently. I was thinking of it as Alice is choosing a filename and Mallet is trying to guess it, in which case the birthday paradox doesn't apply. You're thinking of it as Alice is generating many random filenames and, even though she could avoid collisions with 100% certainty by remembering what she's already had, isn't doing so, and must avoid colliding with herself. I don't think your version makes has much relevance as an attack model. From ian.g.kelly at gmail.com Mon Feb 22 19:48:59 2016 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Mon, 22 Feb 2016 17:48:59 -0700 Subject: How the heck does async/await work in Python 3.5 In-Reply-To: <56CB88D5.3090202@mail.de> References: <56c7d145$0$1597$c3e8da3$5496439d@news.astraweb.com> <56CB88D5.3090202@mail.de> Message-ID: On Mon, Feb 22, 2016 at 3:16 PM, Sven R. Kunze wrote: > Is something like shown in 12:50 ( cout << tcp_reader(1000).get() ) possible > with asyncio? (tcp_reader would be async def) loop = asyncio.get_event_loop() print(loop.run_until_complete(tcp_reader(1000))) From jon+usenet at unequivocal.co.uk Mon Feb 22 19:53:51 2016 From: jon+usenet at unequivocal.co.uk (Jon Ribbens) Date: Tue, 23 Feb 2016 00:53:51 -0000 (UTC) Subject: Make a unique filesystem path, without creating the file References: <85r3gf55k4.fsf@benfinney.id.au> <85mvr26dij.fsf@benfinney.id.au> <56CB4F60.4010201@stoneleaf.us> <85povoz7hg.fsf@benfinney.id.au> Message-ID: On 2016-02-23, Ben Finney wrote: > Oscar Benjamin writes: >> What does unpredictable mean in this context? Maybe I'm reading too >> much into that... > > I think you may be, yes. The request in this thread requires making > direct use of the ?generate a new valid temporary fielsystem path? > functionality already implemented in ?tempfile?. > > Implementations of that functionality outside of ?tempfile? are a fun > exercise, but miss the point of this thread. I think you have missed the point of your own thread. You can't do what you wanted using tempfile, the only possible answer is to choose a filename that is sufficiently random that your hope that it is unique won't be proven futile. tempfile has two main modes, mktemp which meets your requirements but should never be used as it is insecure, and mkstemp which doesn't meet your requirements because it fundamentally operates by actually creating the file in question and relying on the filesystem to guarantee uniqueness. From rosuav at gmail.com Mon Feb 22 19:56:24 2016 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 23 Feb 2016 11:56:24 +1100 Subject: Make a unique filesystem path, without creating the file In-Reply-To: References: <85r3gf55k4.fsf@benfinney.id.au> <85mvr26dij.fsf@benfinney.id.au> <87ziusmvi0.fsf@elektro.pacujo.net> <56cb9bb5$0$1595$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Tue, Feb 23, 2016 at 11:44 AM, Jon Ribbens wrote: > On 2016-02-23, Chris Angelico wrote: >> On Tue, Feb 23, 2016 at 11:26 AM, Jon Ribbens >> wrote: >>> On 2016-02-23, Chris Angelico wrote: >>>> On Tue, Feb 23, 2016 at 11:08 AM, Jon Ribbens >>>> wrote: >>>>>> If you generate 2**128 + 1 such numbers, you are *guaranteed* to >>>>> >>>>> ... have expired due to the heat death of the universe. >>>> >>>> Maybe... but by the time you get to 2**64 of them, you have a 50% >>>> chance of a collision. (That's either utterly intuitive or completely >>>> counter-intuitive, depending on who you are.) >>> >>> Um, did you mean to say 2**127? Are you thinking of the >>> birthday paradox or something, which doesn't apply here? >> >> By the time you generate 2**64 of them, you have a 50% chance that >> some pair of them collides. Yes, the birthday paradox does apply here. > > Oh, I see, you're thinking of it differently. I was thinking of it as > Alice is choosing a filename and Mallet is trying to guess it, in which > case the birthday paradox doesn't apply. You're thinking of it as Alice > is generating many random filenames and, even though she could avoid > collisions with 100% certainty by remembering what she's already had, > isn't doing so, and must avoid colliding with herself. I don't think > your version makes has much relevance as an attack model. Ah. Steven was talking about collisions; once you have 2**128+1 of them, you're guaranteed a collision (pigeonhole principle). What you're talking about gives certainty slightly sooner - specifically, once you've tried 2**128 of them, you're guaranteed to have hit it :) ChrisA From no.email at nospam.invalid Mon Feb 22 21:27:46 2016 From: no.email at nospam.invalid (Paul Rubin) Date: Mon, 22 Feb 2016 18:27:46 -0800 Subject: Make a unique filesystem path, without creating the file References: <85r3gf55k4.fsf@benfinney.id.au> <85mvr26dij.fsf@benfinney.id.au> <56cba1bc$0$1604$c3e8da3$5496439d@news.astraweb.com> Message-ID: <87si0k2mal.fsf@jester.gateway.pace.com> Steven D'Aprano writes: > https://www.python.org/dev/peps/pep-0506/ I didn't know about this! The discussion was all on mailing lists? A few things I suggest changing: 1) the default system RNG for Linux should be getrandom(2) on kernels that support it (3.17 and later). 2) Some effort should be directed at simulating getrandom's behaviour on kernels that don't have it, using the /dev/random entropy estimator and the /dev/urandom interface. I.e. it should block if the system hasn't seen enough entropy to get the CSPRNG started securely, and never block after that. 3) The default token length should be long enough to not have to "change in the future". If the user wants a shorter token, they ask for that, or can truncate a longer one that they receive from the default. There are a few other choices in the PEP whose benefit is unclear to me, but they aren't harmful, and I guess the decisions have already been made. From rosuav at gmail.com Mon Feb 22 21:53:36 2016 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 23 Feb 2016 13:53:36 +1100 Subject: Make a unique filesystem path, without creating the file In-Reply-To: <87si0k2mal.fsf@jester.gateway.pace.com> References: <85r3gf55k4.fsf@benfinney.id.au> <85mvr26dij.fsf@benfinney.id.au> <56cba1bc$0$1604$c3e8da3$5496439d@news.astraweb.com> <87si0k2mal.fsf@jester.gateway.pace.com> Message-ID: On Tue, Feb 23, 2016 at 1:27 PM, Paul Rubin wrote: > 3) The default token length should be long enough to not have to "change > in the future". If the user wants a shorter token, they ask for that, > or can truncate a longer one that they receive from the default. How much future are you expecting? ChrisA From no.email at nospam.invalid Mon Feb 22 22:26:01 2016 From: no.email at nospam.invalid (Paul Rubin) Date: Mon, 22 Feb 2016 19:26:01 -0800 Subject: Make a unique filesystem path, without creating the file References: <85r3gf55k4.fsf@benfinney.id.au> <85mvr26dij.fsf@benfinney.id.au> <56cba1bc$0$1604$c3e8da3$5496439d@news.astraweb.com> <87si0k2mal.fsf@jester.gateway.pace.com> Message-ID: <87povow1iu.fsf@jester.gateway.pace.com> Chris Angelico writes: > How much future are you expecting? This is old but its methodology still seems ok: http://saluc.engr.uconn.edu/refs/keymgr/blaze95minimalkeylength.pdf I also like this: http://cr.yp.to/talks/2015.10.05/slides-djb-20151005-a4.pdf Quote (slide 37): The crypto users' fantasy is boring crypto: crypto that simply works, solidly resists attacks, never needs any upgrades. HN discussion: https://news.ycombinator.com/item?id=10345965 From mscir at yahoo.com Tue Feb 23 01:48:27 2016 From: mscir at yahoo.com (Mike S) Date: Mon, 22 Feb 2016 22:48:27 -0800 Subject: good python tutorial Message-ID: This site was recommended by a friend, it looks really well put together, I thought it might be of interest to people considering online tutorials. http://www.python-course.eu/index.php From marko at pacujo.net Tue Feb 23 01:54:19 2016 From: marko at pacujo.net (Marko Rauhamaa) Date: Tue, 23 Feb 2016 08:54:19 +0200 Subject: Make a unique filesystem path, without creating the file References: <85r3gf55k4.fsf@benfinney.id.au> <85mvr26dij.fsf@benfinney.id.au> <87vb5gmtgj.fsf@elektro.pacujo.net> <56cba8c7$0$1611$c3e8da3$5496439d@news.astraweb.com> Message-ID: <877fhvnch0.fsf@elektro.pacujo.net> Steven D'Aprano : > On Tue, 23 Feb 2016 06:32 am, Marko Rauhamaa wrote: >> Under Linux, /dev/random is the way to go when strong security is >> needed. Note that /dev/random is a scarce resource on ordinary >> systems. > > That's actually incorrect, but you're not the only one to have been > mislead by the man pages. > > http://sockpuppet.org/blog/2014/02/25/safely-generate-random-numbers/ Still, mostly hypnotic repetitions. However, it admits: But /dev/random also tries to keep track of how much entropy remains in its kernel pool, and will occasionally go on strike if it decides not enough remains. That's the whole point. /dev/random will rather block the program than lower the quality of the random numbers below a threshold. /dev/urandom has no such qualms. If you use /dev/random instead of urandom, your program will unpredictably (or, if you?re an attacker, very predictably) hang when Linux gets confused about how its own RNG works. Yes, possibly indefinitely, too. Using /dev/random will make your programs less stable, but it won?t make them any more cryptographically safe. It is correct that you shouldn't use /dev/random as a routine source of bulk random numbers. It is also correct that /dev/urandom depletes the entropy pool as effectively as /dev/random. However, when you are generating signing or encryption keys, you should use /dev/random. As stated in : /dev/urandom should be used for essentially all random numbers required, but /dev/random is sometimes used for things like extremely sensitive, long-lived keys (e.g. GPG) or one-time pads. > See also: > > http://www.2uo.de/myths-about-urandom/ Already addressed. Marko From no.email at nospam.invalid Tue Feb 23 02:18:21 2016 From: no.email at nospam.invalid (Paul Rubin) Date: Mon, 22 Feb 2016 23:18:21 -0800 Subject: Make a unique filesystem path, without creating the file References: <85r3gf55k4.fsf@benfinney.id.au> <85mvr26dij.fsf@benfinney.id.au> <87vb5gmtgj.fsf@elektro.pacujo.net> <56cba8c7$0$1611$c3e8da3$5496439d@news.astraweb.com> <877fhvnch0.fsf@elektro.pacujo.net> Message-ID: <87lh6bx5c2.fsf@jester.gateway.pace.com> Marko Rauhamaa writes: > It is also correct that /dev/urandom depletes the entropy pool as > effectively as /dev/random. I think see what's confusing you: the above is a misconception that is probably held by lots of people. Entropy is not water and from a cryptographic standpoint there is essentially no such thing as "depleting" an entropy pool. There is either enough entropy (say 256 bits or more) in the PRNG or else there isn't. If there's not enough, urandom can misbehave by giving you bad output because it doesn't block until more is gathered. If there is enough, /dev/random misbehaves by blocking under this bogus concept of "depletion". If you have a seed with 256 bits of entropy and you generate a gigabyte of random numbers from it, you have not increased the predictability of the seed in any significant way. So once /dev/random unblocks, it should never again block, the behavior of getrandom. There used to be an article on David Wagner's web site (cs.berkeley.edu/~daw) about the concept of "depleting" entropy by iterated hashing, but I can't find it now. That's unfortunate since it might help cast light on the subject. >> http://www.2uo.de/myths-about-urandom/ > Already addressed. No really, all you've done is repeat bad advice. The people cited in that article are very knowledgeable and the stuff they say makes good mathematical sense. The stuff you say makes no sense and you haven't given any convincing reason for anyone to listen to you. From breamoreboy at yahoo.co.uk Tue Feb 23 03:09:35 2016 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Tue, 23 Feb 2016 08:09:35 +0000 Subject: Make a unique filesystem path, without creating the file In-Reply-To: <87si0k2mal.fsf@jester.gateway.pace.com> References: <85r3gf55k4.fsf@benfinney.id.au> <85mvr26dij.fsf@benfinney.id.au> <56cba1bc$0$1604$c3e8da3$5496439d@news.astraweb.com> <87si0k2mal.fsf@jester.gateway.pace.com> Message-ID: On 23/02/2016 02:27, Paul Rubin wrote: > Steven D'Aprano writes: >> https://www.python.org/dev/peps/pep-0506/ > > I didn't know about this! The discussion was all on mailing lists? https://mail.python.org/pipermail/python-ideas/2015-September/036333.html then http://www.gossamer-threads.com/lists/python/dev/1223780 > > A few things I suggest changing: > > 1) the default system RNG for Linux should be getrandom(2) on kernels > that support it (3.17 and later). > > 2) Some effort should be directed at simulating getrandom's behaviour > on kernels that don't have it, using the /dev/random entropy estimator > and the /dev/urandom interface. I.e. it should block if the system > hasn't seen enough entropy to get the CSPRNG started securely, and > never block after that. > > 3) The default token length should be long enough to not have to "change > in the future". If the user wants a shorter token, they ask for that, > or can truncate a longer one that they receive from the default. > > There are a few other choices in the PEP whose benefit is unclear to me, > but they aren't harmful, and I guess the decisions have already been > made. > The PEP status is draft so is subject to change. -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From no.email at nospam.invalid Tue Feb 23 03:22:47 2016 From: no.email at nospam.invalid (Paul Rubin) Date: Tue, 23 Feb 2016 00:22:47 -0800 Subject: Make a unique filesystem path, without creating the file References: <85r3gf55k4.fsf@benfinney.id.au> <85mvr26dij.fsf@benfinney.id.au> <56cba1bc$0$1604$c3e8da3$5496439d@news.astraweb.com> <87si0k2mal.fsf@jester.gateway.pace.com> Message-ID: <87h9gzx2co.fsf@jester.gateway.pace.com> Mark Lawrence writes: > https://mail.python.org/pipermail/python-ideas/2015-September/036333.html > then http://www.gossamer-threads.com/lists/python/dev/1223780 Thanks. It would be nice if those were gatewayed to usenet like this group is. I can't bring myself to subscribe to mailing lists. >> There are a few other choices in the PEP whose benefit is unclear to me, >> but they aren't harmful, and I guess the decisions have already been >> made. > The PEP status is draft so is subject to change. Well they might be changeable but it sounds like there's a level of consensus by now, that wouldn't be helped by more bikeshedding over relatively minor stuff. I might write up some further comments and post them here From cs at zip.com.au Tue Feb 23 03:24:45 2016 From: cs at zip.com.au (Cameron Simpson) Date: Tue, 23 Feb 2016 19:24:45 +1100 Subject: Testing whether the VPN is running? In-Reply-To: <8ghgpcxrd3.ln2@news.ducksburg.com> References: <8ghgpcxrd3.ln2@news.ducksburg.com> Message-ID: <20160223082445.GA73615@cskk.homeip.net> On 18Feb2016 10:03, Adam Funk wrote: >On 2016-02-18, Ervin Heged?s wrote: >> I think that the psutil modul could be better for you for this >> task: >> https://pypi.python.org/pypi/psutil/ >> >> and see the "Network" section. > >if 'tun0' in psutil.net_if_addrs(): > # vpn is running > >Brilliant! I've used psutil for something else, but I didn't know it >did that too. My excuse is that the version on my system was 2.2.1, >which does not do that, so I installed the newer version with pip3. >Thanks for pointing me to that. You might also want to check that the interface is up. My personal hack (not for a VPN, but for "being online", which turns my ssh tunnels on and off) is to look in the output of "netstat -rn" for a default route. This may imply that an alternative test for you is to test for a route to your VPN's address range? Just an idea. Cheers, Cameron Simpson From __peter__ at web.de Tue Feb 23 03:40:37 2016 From: __peter__ at web.de (Peter Otten) Date: Tue, 23 Feb 2016 09:40:37 +0100 Subject: Make a unique filesystem path, without creating the file References: <85r3gf55k4.fsf@benfinney.id.au> <85mvr26dij.fsf@benfinney.id.au> <56cba1bc$0$1604$c3e8da3$5496439d@news.astraweb.com> <87si0k2mal.fsf@jester.gateway.pace.com> <87h9gzx2co.fsf@jester.gateway.pace.com> Message-ID: Paul Rubin wrote: > Mark Lawrence writes: >> https://mail.python.org/pipermail/python-ideas/2015-September/036333.html >> then http://www.gossamer-threads.com/lists/python/dev/1223780 > > Thanks. It would be nice if those were gatewayed to usenet like this > group is. I can't bring myself to subscribe to mailing lists. They are available via news.gmane.org as gmane.comp.python.devel gmane.comp.python.ideas From breamoreboy at yahoo.co.uk Tue Feb 23 04:00:28 2016 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Tue, 23 Feb 2016 09:00:28 +0000 Subject: Make a unique filesystem path, without creating the file In-Reply-To: <87h9gzx2co.fsf@jester.gateway.pace.com> References: <85r3gf55k4.fsf@benfinney.id.au> <85mvr26dij.fsf@benfinney.id.au> <56cba1bc$0$1604$c3e8da3$5496439d@news.astraweb.com> <87si0k2mal.fsf@jester.gateway.pace.com> <87h9gzx2co.fsf@jester.gateway.pace.com> Message-ID: On 23/02/2016 08:22, Paul Rubin wrote: > Mark Lawrence writes: >> https://mail.python.org/pipermail/python-ideas/2015-September/036333.html >> then http://www.gossamer-threads.com/lists/python/dev/1223780 > > Thanks. It would be nice if those were gatewayed to usenet like this > group is. I can't bring myself to subscribe to mailing lists. Piece of cake using even a semi-decent email client (I use Thunderbird on Windows) via gmane. It provides access to hundreds of Python mailing lists, blogs and even updates to the Activestate recipes :) > >>> There are a few other choices in the PEP whose benefit is unclear to me, >>> but they aren't harmful, and I guess the decisions have already been >>> made. >> The PEP status is draft so is subject to change. > > Well they might be changeable but it sounds like there's a level of > consensus by now, that wouldn't be helped by more bikeshedding over > relatively minor stuff. I might write up some further comments and post > them here > You might as well, can't do any harm and somebody might pick up on something. -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From gvanem at yahoo.no Tue Feb 23 04:40:27 2016 From: gvanem at yahoo.no (Gisle Vanem) Date: Tue, 23 Feb 2016 10:40:27 +0100 Subject: Testing whether the VPN is running? In-Reply-To: <20160223082445.GA73615@cskk.homeip.net> References: <8ghgpcxrd3.ln2@news.ducksburg.com> <20160223082445.GA73615@cskk.homeip.net> Message-ID: <56CC290B.8000509@yahoo.no> Cameron Simpson: > You might also want to check that the interface is up. > > My personal hack (not for a VPN, but for "being online", which turns my ssh tunnels on and off) is to look in the output > of "netstat -rn" for a default route. This may imply that an alternative test for you is to test for a route to your > VPN's address range? Just an idea. And for VPN in Windows *and* using WinPcap, one can check if this device exist: \Device\NPF_GenericDialupAdapter (controllable using 'sq query rasdial'). This works if you use PPTP or L2TP but not OpenVPN. BTW, a VPN on Windows doesn't come back up automatically after a sleep/hibernation. The WOSB tool at http://www.dennisbabkin.com/wosb/ could be handy. What happens with your VPN on Linux (?) after coming back from sleep/hibernation? Automatic or you need to script that too? -- --gv From mal at europython.eu Tue Feb 23 07:15:14 2016 From: mal at europython.eu (M.-A. Lemburg) Date: Tue, 23 Feb 2016 13:15:14 +0100 Subject: EuroPython 2016: Early bird sales started Message-ID: <56CC4D52.5020405@europython.eu> We have opened up the early-bird sales today and tickets are selling fast. The first 100 early-bird tickets, we have available, will be gone in less than an hour. If you want to benefit from reduced ticket prices too, please register soon. Our early bird ticket sales are limited to 300 tickets. Regular ticket sales will start shortly after we?ve ended the early bird sales. *** https://ep2016.europython.eu/en/registration/ *** More Information and Ticket Shop PS: Remember to submit your proposals for the conference. There are less than two weeks left: https://ep2016.europython.eu/en/call-for-proposals/ With gravitational regards, -- EuroPython 2016 Team http://ep2016.europython.eu/ http://www.europython-society.org/ From davidbenny2000 at gmail.com Tue Feb 23 07:15:40 2016 From: davidbenny2000 at gmail.com (Ho Yeung Lee) Date: Tue, 23 Feb 2016 04:15:40 -0800 (PST) Subject: how to notify among the def and can network communication such as zeromq for python do this? Message-ID: <3f18c177-a7d7-42a7-9134-bb1abc3ba7b8@googlegroups.com> in the following code, node 3 and node 4 running parallel if there are 100 nodes running parallel, how can they notify each other i find this post stackoverflow, http://stackoverflow.com/questions/29324346/how-do-i-connect-asyncio-coroutines-that-continually-produce-and-consume-data if zeromq for python can not do notify among parallel nodes which means give handle to another def my goal is to make each node can notify all other nodes if each node need import asyncio import time from concurrent.futures import ProcessPoolExecutor def f000(): try: print "000" ex = ProcessPoolExecutor(2) yield from loop.run_in_executor(ex, time.sleep, 10) except: print "000 exception" def f001(): try: print "001" ex = ProcessPoolExecutor(2) yield from loop.run_in_executor(ex, time.sleep, 10) except: print "001 exception" def f002(): try: print "002" ex = ProcessPoolExecutor(2) yield from loop.run_in_executor(ex, time.sleep, 10) except: print "002 exception" def f003(): try: print "003" ex = ProcessPoolExecutor(2) yield from loop.run_in_executor(ex, time.sleep, 10) except: print "003 exception" def f004(): try: print "004" ex = ProcessPoolExecutor(2) yield from loop.run_in_executor(ex, time.sleep, 10) except: print "004 exception" machine = {} mappedfunc = {} functionlist000 = [] functionlist001 = [] functionlist002 = [] functionlist003 = [] functionlist004 = [] functionlist000.append('002') functionlist001.append('000') functionlist002.append('003') functionlist002.append('004') functionlist003.append('001') functionlist004.append('001') machine000 = {'000': functionlist000} machine001 = {'001': functionlist001} machine002 = {'002': functionlist002} machine003 = {'003': functionlist003} machine004 = {'004': functionlist004} machine.update(machine000) machine.update(machine001) machine.update(machine002) machine.update(machine003) machine.update(machine004) functionkey000 = {'000': f000 } functionkey001 = {'001': f001 } functionkey002 = {'002': f002 } functionkey002 = {'003': f003 } functionkey002 = {'004': f004 } mappedfunc.update(functionkey000) mappedfunc.update(functionkey001) mappedfunc.update(functionkey002) mappedfunc.update(functionkey003) mappedfunc.update(functionkey004) def workingthreadpool(currentnumber1): i = 1 #while i < 7: #i = i + 1 functionlist0 = list(machine.get(currentnumber1).values()) loop = asyncio.get_event_loop() tesks = [] j = 1 for functionlistelement in functionlist0 tesks.append(asyncio.ensure_future(mappedfunc.get(functionlistelement)())) if j > 1: workingthreadpool(functionlistelement) j = j + 1 loop.run_until_complete(asyncio.wait_for(tesks, 1)) loop.close() currentnumber = "000" workingthreadpool(currentnumber) From invalid at invalid.invalid Tue Feb 23 10:14:20 2016 From: invalid at invalid.invalid (Grant Edwards) Date: Tue, 23 Feb 2016 15:14:20 +0000 (UTC) Subject: Make a unique filesystem path, without creating the file References: <85r3gf55k4.fsf@benfinney.id.au> <85mvr26dij.fsf@benfinney.id.au> <56cba1bc$0$1604$c3e8da3$5496439d@news.astraweb.com> <87si0k2mal.fsf@jester.gateway.pace.com> <87h9gzx2co.fsf@jester.gateway.pace.com> Message-ID: On 2016-02-23, Mark Lawrence wrote: > On 23/02/2016 08:22, Paul Rubin wrote: >> Mark Lawrence writes: >>> https://mail.python.org/pipermail/python-ideas/2015-September/036333.html >>> then http://www.gossamer-threads.com/lists/python/dev/1223780 >> >> Thanks. It would be nice if those were gatewayed to usenet like >> this group is. I can't bring myself to subscribe to mailing lists. > > Piece of cake using even a semi-decent email client (I use Thunderbird > on Windows) via gmane. And gmane is even better using a decent news (NNTP) client. I prefer slrn, but that may be a bit old-school for many. Technically, gmane's news server is not "Usenet", but the UI is the same. Gmane's internal search facility is a bit lame, but searching gmane with Google works fairly well. > It provides access to hundreds of Python mailing lists, blogs and > even updates to the Activestate recipes :) -- Grant Edwards grant.b.edwards Yow! World War Three can at be averted by adherence gmail.com to a strictly enforced dress code! From garyfallidis at gmail.com Tue Feb 23 11:08:28 2016 From: garyfallidis at gmail.com (Eleftherios Garyfallidis) Date: Tue, 23 Feb 2016 11:08:28 -0500 Subject: DIPY 0.11.0 is now available for download Message-ID: Dear all, We are excited to announce a new public release of Diffusion Imaging in Python (DIPY). The 0.11 release follows closely on the heels of the 0.10 release, resolving issues that existed in that release on the Windows 64 bit platform. New features of the 0.11 and 0.10 release cycles include many additional fixes and new frameworks. Here are some of the highlights: DIPY 0.11.0 (Monday, 21 February 2016): - New framework for contextual enhancement of ODFs. - Compatibility with new version of numpy (1.11). - Compatibility with VTK 7.0 which supports Python 3.x. - Faster PIESNO for noise estimation. - Reorient gradient directions according to motion correction parameters. - Supporting Python 3.3+ but not 3.2. - Reduced memory usage in DTI prediction. - DSI now can use datasets with multiple b0s. - Fixed different issues with Windows 64bit and Python 3.5. DIPY 0.10.1 (Friday, 4 December 2015): - Compatibility with new versions of scipy (0.16) and numpy (1.10). - New cleaner visualization API, including compatibility with VTK 6, and functions to create your own interactive visualizations. - Diffusion Kurtosis Imaging (DKI): Google Summer of Code work by Rafael Henriques. - Mean Apparent Propagator (MAP) MRI for tissue microstructure estimation. - Anisotropic Power Maps from spherical harmonic coefficients. - A new framework for affine registration of images. Detailed release notes can be found here: http://dipy.org/release0.11.html http://dipy.org/release0.10.html To upgrade, run the following command in your terminal: pip install --upgrade dipy For the complete installation guide look here: http://dipy.org/installation.html For any questions go to http://dipy.org, or https://neurostars.org or send an e-mail to neuroimaging at python.org We also have a new instant messaging service and chat room available at https://gitter.im/nipy/dipy On behalf of the DIPY developers, Eleftherios Garyfallidis & Ariel Rokem http://dipy.org/developers.html From kevind0718 at gmail.com Tue Feb 23 11:30:06 2016 From: kevind0718 at gmail.com (kevind0718 at gmail.com) Date: Tue, 23 Feb 2016 08:30:06 -0800 (PST) Subject: setting the font of the root title Message-ID: Hello: Newbee here. I need to change the font of the title of the root. Actually I just need it to be larger, which means I may need the top boarder to be larger. I wrote the bit of code below, hoping for the desired effect. Also googled around regarding the toplevel widget No luck. Your kind assistance is requested. from Tkinter import * import tkFont root = Tk( ) root.option_add("*font, ", "Times 16") root.title("Database Logon Info") w = Label(root, text="Hello World") w.pack() root.mainloop() From random832 at fastmail.com Tue Feb 23 11:45:55 2016 From: random832 at fastmail.com (Random832) Date: Tue, 23 Feb 2016 11:45:55 -0500 Subject: setting the font of the root title In-Reply-To: References: Message-ID: <1456245955.573829.529562674.186C84A9@webmail.messagingengine.com> On Tue, Feb 23, 2016, at 11:30, kevind0718 at gmail.com wrote: > Hello: > > Newbee here. > > I need to change the font of the title of the root. > Actually I just need it to be larger, which means I may need the > top boarder to be larger. > > I wrote the bit of code below, hoping for the desired effect. > Also googled around regarding the toplevel widget > No luck. Generally the title bar and window border is provided by the window manager or operating system and therefore its appearance is not under the control of your program. From srkunze at mail.de Tue Feb 23 11:50:05 2016 From: srkunze at mail.de (Sven R. Kunze) Date: Tue, 23 Feb 2016 17:50:05 +0100 Subject: How the heck does async/await work in Python 3.5 In-Reply-To: References: <56c7d145$0$1597$c3e8da3$5496439d@news.astraweb.com> <56CB88D5.3090202@mail.de> Message-ID: <56CC8DBD.9080604@mail.de> On 23.02.2016 01:48, Ian Kelly wrote: > On Mon, Feb 22, 2016 at 3:16 PM, Sven R. Kunze wrote: >> Is something like shown in 12:50 ( cout << tcp_reader(1000).get() ) possible >> with asyncio? (tcp_reader would be async def) > loop = asyncio.get_event_loop() > print(loop.run_until_complete(tcp_reader(1000))) I see. Thanks. :) How come that Python (compared to C++) needs much more boilerplate to use async programming? Historically, it was the other way round. Best, Sven From aaron.christensen at gmail.com Tue Feb 23 12:34:40 2016 From: aaron.christensen at gmail.com (Aaron Christensen) Date: Tue, 23 Feb 2016 12:34:40 -0500 Subject: good python tutorial In-Reply-To: References: Message-ID: Thanks for sharing! On Tue, Feb 23, 2016 at 1:48 AM, Mike S via Python-list < python-list at python.org> wrote: > This site was recommended by a friend, it looks really well put together, > I thought it might be of interest to people considering online tutorials. > > http://www.python-course.eu/index.php > -- > https://mail.python.org/mailman/listinfo/python-list > From drsalists at gmail.com Tue Feb 23 12:36:03 2016 From: drsalists at gmail.com (Dan Stromberg) Date: Tue, 23 Feb 2016 09:36:03 -0800 Subject: nosepipe error Message-ID: Hi folks. I'm using (or trying to use) nosepipe to get nose test isolation; I'm working on an almost-big test suite, and the tests appear to be lacking test isolation. I'm encountering a bug in nosepipe: ====================================================================== ERROR: testCreateFileMux (testFilemux.TestFilemux) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/dstromberg/virtualenvs/dev/lib/python2.7/site-packages/nose/case.py", line 133, in run self.runTest(result) File "/home/dstromberg/virtualenvs/dev/lib/python2.7/site-packages/nose/case.py", line 151, in runTest test(result) File "/home/dstromberg/virtualenvs/dev/lib/python2.7/site-packages/nosepipe.py", line 179, in __call__ (header + data).decode("latin1")) Exception: short message body (want 1433299041, got 80) Something went wrong Message: Unhandled exception in thread started by sys.excepthook is missing lost sys.stderr That 1433299041 looks like ASCII: >>> hex(1433299041) '0x556e6861' >>> chr(0x55) + chr(0x6e) + chr(0x68) + chr(0x61) 'Unha' ...but I'm not sure where to go with that from there. The authors appear to be: John J. Lee, Dan McCombs, and Vadim Markovtsev. I've e-mailed John and Dan - though that was only last night. I don't know how to get in touch with Vadim. Are there any nosepipe users out there? Does anyone know how best to reach John, Dan and/or Vadim? Thanks. From ian.g.kelly at gmail.com Tue Feb 23 12:37:31 2016 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Tue, 23 Feb 2016 10:37:31 -0700 Subject: How the heck does async/await work in Python 3.5 In-Reply-To: <56CC8DBD.9080604@mail.de> References: <56c7d145$0$1597$c3e8da3$5496439d@news.astraweb.com> <56CB88D5.3090202@mail.de> <56CC8DBD.9080604@mail.de> Message-ID: On Tue, Feb 23, 2016 at 9:50 AM, Sven R. Kunze wrote: > On 23.02.2016 01:48, Ian Kelly wrote: >> >> On Mon, Feb 22, 2016 at 3:16 PM, Sven R. Kunze wrote: >>> >>> Is something like shown in 12:50 ( cout << tcp_reader(1000).get() ) >>> possible >>> with asyncio? (tcp_reader would be async def) >> >> loop = asyncio.get_event_loop() >> print(loop.run_until_complete(tcp_reader(1000))) > > > I see. Thanks. :) > > How come that Python (compared to C++) needs much more boilerplate to use > async programming? Historically, it was the other way round. It's not entirely clear to me what the C++ is actually doing. With Python we have an explicit event loop that has to be started to manage resuming the coroutines. Since it's explicit, you could easily drop in a different event loop, such as Tornado or curio, if desired. If your coroutine never awaits anything that isn't already done then technically you don't need an event loop, but at that point you might as well be using ordinary functions. The C++ on the other hand seems to be doing something implicit at the compiler level to make everything happen automatically inside the future.get() call, but I don't know what that is. You could wrap up the boilerplate in Python if you like: def get(coro, loop=None): if loop is None: loop = asyncio.get_event_loop() return loop.run_until_complete(coro) print(get(tcp_reader(1000))) From rosuav at gmail.com Tue Feb 23 12:43:18 2016 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 24 Feb 2016 04:43:18 +1100 Subject: nosepipe error In-Reply-To: References: Message-ID: On Wed, Feb 24, 2016 at 4:36 AM, Dan Stromberg wrote: > Message: Unhandled exception in thread started by > sys.excepthook is missing > lost sys.stderr > > That 1433299041 looks like ASCII: >>>> hex(1433299041) > '0x556e6861' >>>> chr(0x55) + chr(0x6e) + chr(0x68) + chr(0x61) > 'Unha' > > ...but I'm not sure where to go with that from there. My eye sees those characters as being the beginning of "Unhandled exception in thread". You could be dealing with multiple levels of cascading exceptions. ChrisA From drsalists at gmail.com Tue Feb 23 13:11:43 2016 From: drsalists at gmail.com (Dan Stromberg) Date: Tue, 23 Feb 2016 10:11:43 -0800 Subject: nosepipe error In-Reply-To: References: Message-ID: On Tue, Feb 23, 2016 at 9:43 AM, Chris Angelico wrote: > On Wed, Feb 24, 2016 at 4:36 AM, Dan Stromberg wrote: >> Message: Unhandled exception in thread started by >> sys.excepthook is missing >> lost sys.stderr >> >> That 1433299041 looks like ASCII: >>>>> hex(1433299041) >> '0x556e6861' >>>>> chr(0x55) + chr(0x6e) + chr(0x68) + chr(0x61) >> 'Unha' >> >> ...but I'm not sure where to go with that from there. > > My eye sees those characters as being the beginning of "Unhandled > exception in thread". You could be dealing with multiple levels of > cascading exceptions. Agreed - at least "Unhandled exception". I'm seeing, in the nosepipe code: # we use stdout for IPC, so block all other output self._stream = sys.__stdout__ I'm not sure using stdout/stdin for IPC is a great thing; a lot of tests write to stdout or stderr, whether intentionally or unintentionally. I'm attempting to replace the use of stdout/stdin with os.pipe(), but so far I'm not having much luck. I get three write()'s, and then a hang. So far, I'm ignoring the fact that os.read() and os.write() might split or aggregate chunks over a socket, since this is a socketpair communicating on the same machine. From rosuav at gmail.com Tue Feb 23 13:42:15 2016 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 24 Feb 2016 05:42:15 +1100 Subject: nosepipe error In-Reply-To: References: Message-ID: On Wed, Feb 24, 2016 at 5:11 AM, Dan Stromberg wrote: > I'm seeing, in the nosepipe code: > # we use stdout for IPC, so block all other output > self._stream = sys.__stdout__ > > I'm not sure using stdout/stdin for IPC is a great thing; a lot of > tests write to stdout or stderr, whether intentionally or > unintentionally. Using standard streams for IPC is well recognized. If it's known in advance, it shouldn't be a problem - writing to stdout/stderr would simply result in a test failure. > I'm attempting to replace the use of stdout/stdin with os.pipe(), but > so far I'm not having much luck. I get three write()'s, and then a > hang. So far, I'm ignoring the fact that os.read() and os.write() > might split or aggregate chunks over a socket, since this is a > socketpair communicating on the same machine. Sounds like the buffer's full. You'll need to have another thread reading before you start writing, to prevent that. ChrisA From marko at pacujo.net Tue Feb 23 14:04:15 2016 From: marko at pacujo.net (Marko Rauhamaa) Date: Tue, 23 Feb 2016 21:04:15 +0200 Subject: Make a unique filesystem path, without creating the file References: <85r3gf55k4.fsf@benfinney.id.au> <85mvr26dij.fsf@benfinney.id.au> <87vb5gmtgj.fsf@elektro.pacujo.net> <56cba8c7$0$1611$c3e8da3$5496439d@news.astraweb.com> <877fhvnch0.fsf@elektro.pacujo.net> <87lh6bx5c2.fsf@jester.gateway.pace.com> Message-ID: <87povnl040.fsf@elektro.pacujo.net> Paul Rubin : > Marko Rauhamaa writes: >> It is also correct that /dev/urandom depletes the entropy pool as >> effectively as /dev/random. > > I think see what's confusing you: the above is a misconception that is > probably held by lots of people. Entropy is not water and from a > cryptographic standpoint there is essentially no such thing as > "depleting" an entropy pool. There is either enough entropy (say 256 > bits or more) in the PRNG or else there isn't. If there's not enough, > urandom can misbehave by giving you bad output because it doesn't block > until more is gathered. If there is enough, /dev/random misbehaves by > blocking under this bogus concept of "depletion". You are making my point. /dev/random is correct to block until top-quality random numbers can be supplied. That's not misbehaving. > So once /dev/random unblocks, it should never again block, the behavior > of getrandom. What you are saying is that /dev/random has no reason to exist (and the GRND_RANDOM flag to getrandom() is redundant). I'm no cryptographer and can't judge that. However, as long as the distinction is maintained, I have to abide by the documented characteristics. > No really, all you've done is repeat bad advice. The people cited in > that article are very knowledgeable and the stuff they say makes good > mathematical sense. The stuff you say makes no sense and you haven't > given any convincing reason for anyone to listen to you. Thing is, neither you nor me nor the cited articles has provided any more info than insisting on a position, my position being relying on the documented API. So we have * /dev/urandom vs /dev/random * getrandom(0) vs getrandom(GRND_RANDOM) * GCRY_STRONG_RANDOM ("Use this level for session keys and similar purposes") vs GCRY_VERY_STRONG_RANDOM ("Use this level for long term key material") (in libgcrypt) You don't need to convince me that that distinction is silly. You need to convince the crypto facility providers. Marko From srkunze at mail.de Tue Feb 23 14:42:49 2016 From: srkunze at mail.de (Sven R. Kunze) Date: Tue, 23 Feb 2016 20:42:49 +0100 Subject: How the heck does async/await work in Python 3.5 In-Reply-To: References: <56c7d145$0$1597$c3e8da3$5496439d@news.astraweb.com> <56CB88D5.3090202@mail.de> <56CC8DBD.9080604@mail.de> Message-ID: <56CCB639.7050101@mail.de> On 23.02.2016 18:37, Ian Kelly wrote: > It's not entirely clear to me what the C++ is actually doing. With > Python we have an explicit event loop that has to be started to manage > resuming the coroutines. Since it's explicit, you could easily drop in > a different event loop, such as Tornado or curio, if desired. If your > coroutine never awaits anything that isn't already done then > technically you don't need an event loop, but at that point you might > as well be using ordinary functions. I don't think taking the shortcut to ordinary functions will work on the big scale. I certainly agree that asynchronous operations can/should? be the very core of everything (files, sockets, timers, etc.); just as Microsoft is pushing with their Windows API. Chaining async ops together just works now with async/await in Python as well. However, in the end of the chaining there'll always be synchronous code that e.g. initializes the event loop. Real world code works the same. Imagine, in some near/distant future, Python might have all its core components (like reading a file, etc. etc.) converted to async. It would be great for many larger applications if there one could introduce async gently. So, laying out an async foundation (the building blocks) but the wiring synchronous operations still work as they should. Once, the team decides they want to leverage the async potential in their code (as the building blocks COULD be executed concurrently), they will then be able to replace the synchronous wires with an event loop. So, I see quite some potential here. > The C++ on the other hand seems to be doing something implicit at the > compiler level to make everything happen automatically inside the > future.get() call, but I don't know what that is. > > You could wrap up the boilerplate in Python if you like: > > def get(coro, loop=None): > if loop is None: > loop = asyncio.get_event_loop() > return loop.run_until_complete(coro) > > print(get(tcp_reader(1000))) As usual. :) Best, Sven From srkunze at mail.de Tue Feb 23 16:05:16 2016 From: srkunze at mail.de (Sven R. Kunze) Date: Tue, 23 Feb 2016 22:05:16 +0100 Subject: How the heck does async/await work in Python 3.5 In-Reply-To: References: <56c7d145$0$1597$c3e8da3$5496439d@news.astraweb.com> Message-ID: <56CCC98C.5060504@mail.de> On 20.02.2016 07:53, Christian Gollwitzer wrote: > If you have difficulties wit hthe overall concept, and if you are open > to discussions in another language, take a look at this video: > > https://channel9.msdn.com/Shows/C9-GoingNative/GoingNative-39-await-co-routines > > > MS has added coroutine support with very similar syntax to VC++ > recently, and the developer tries to explain it to the "stackful" > programmers. Because of this thread, I finally finished an older post collecting valuable insights from last year discussions regarding concurrency modules available in Python: http://srkunze.blogspot.com/2016/02/concurrency-in-python.html It appears to me that it would fit here well. @python-ideas Back then, the old thread ("Concurrency Modules") was like basically meant to result in something useful. I hope the post covers the essence of the discussion. Some even suggested putting the table into the Python docs. I am unaware of the formal procedure here but I would be glad if somebody could point be at the right direction if that the survey table is wanted in the docs. Best, Sven From jsbueno at python.org.br Tue Feb 23 16:25:47 2016 From: jsbueno at python.org.br (Joao S. O. Bueno) Date: Tue, 23 Feb 2016 18:25:47 -0300 Subject: [Python-ideas] How the heck does async/await work in Python 3.5 In-Reply-To: <56CCC98C.5060504@mail.de> References: <56c7d145$0$1597$c3e8da3$5496439d@news.astraweb.com> <56CCC98C.5060504@mail.de> Message-ID: Today I also stumbled on this helpful "essay" from Brett Cannon about the same subject http://www.snarky.ca/how-the-heck-does-async-await-work-in-python-3-5 On 23 February 2016 at 18:05, Sven R. Kunze wrote: > On 20.02.2016 07:53, Christian Gollwitzer wrote: > > If you have difficulties wit hthe overall concept, and if you are open to > discussions in another language, take a look at this video: > > https://channel9.msdn.com/Shows/C9-GoingNative/GoingNative-39-await-co-routines > > MS has added coroutine support with very similar syntax to VC++ recently, > and the developer tries to explain it to the "stackful" programmers. > > > Because of this thread, I finally finished an older post collecting valuable > insights from last year discussions regarding concurrency modules available > in Python: http://srkunze.blogspot.com/2016/02/concurrency-in-python.html It > appears to me that it would fit here well. > > @python-ideas > Back then, the old thread ("Concurrency Modules") was like basically meant > to result in something useful. I hope the post covers the essence of the > discussion. > Some even suggested putting the table into the Python docs. I am unaware of > the formal procedure here but I would be glad if somebody could point be at > the right direction if that the survey table is wanted in the docs. > > Best, > Sven > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ From kevind0718 at gmail.com Tue Feb 23 16:39:24 2016 From: kevind0718 at gmail.com (kevind0718 at gmail.com) Date: Tue, 23 Feb 2016 13:39:24 -0800 (PST) Subject: exit from Tkinter mainloop Python 2.7 Message-ID: <56e451a3-01f7-4883-b4db-c3a6a10729a9@googlegroups.com> Hello: Newbie here. Spent the a good part of the day tinkering and reading tutorials, I was able to create a sample that is very close to my requirement. When I execute the code below the Dialog displayed as expected and I can enter data into the textboxes. All good. When I click on butGo I get the error below. So is this some sort of scope error? why does entryName not exist for me to grab it's value? Your kind assistance is requested. Traceback (most recent call last): File "C:\Users\kduffy12\workspace\testPythonA\testA\simpleDialog.py", line 25, in print entryName.get("1.0", "end-1c" ) AttributeError: 'NoneType' object has no attribute 'get' from Tkinter import * def butContinue(): root1.destroy() root1 = Tk() root1.geometry("500x250") lblTop = Label(root1, text= ' Enter Values Below', font="Helvetica 14").grid(row=0, column=0, columnspan=2 , pady=5) ##lblTop.pack(side = TOP) lblDB = Label(root1, text= ' Weight').grid(row=1, column=0 ) lblPWord = Label(root1, text= ' Height').grid(row=2,column=0) entryName = Entry(root1).grid(row=1, column=1, pady=5) entryPWord = Entry(root1).grid(row=2, column=1, pady = 5) butGo = Button(root1, text=" Continue " , command=butContinue ).grid(row=3, column=1, sticky=W, pady=10) root1.mainloop() print entryName.get("1.0", "end-1c" ) print entryPWord.get("1.0", "end-1c" ) From best_lay at yahoo.com Tue Feb 23 17:19:43 2016 From: best_lay at yahoo.com (Wildman) Date: Tue, 23 Feb 2016 16:19:43 -0600 Subject: [Newbie] Tkinter Question Message-ID: I am familiar with OO programming but I am new to Python and Tkinter. I am working on a gui program that creates a couple of temporary files. As part of the Exit button command they are deleted. If the program is shut down using the window close button [X], the exit button code is not executed and the temporary files are left behind. That is a serious no-no. Is there a way to capture close button click to prevent the program from closing and instead execute a command? I'm talking about something like the Form_QueryUnload event in Visual Basic. Alternately is there a way to just disable the button or prevent it from being displayed in the first place? Any help appreciated. -- GNU/Linux user #557453 "Be at war with your vices, at peace with your neighbors, and let every new year find you a better man." -Benjamin Franklin From auriocus at gmx.de Tue Feb 23 17:29:31 2016 From: auriocus at gmx.de (Christian Gollwitzer) Date: Tue, 23 Feb 2016 23:29:31 +0100 Subject: exit from Tkinter mainloop Python 2.7 In-Reply-To: <56e451a3-01f7-4883-b4db-c3a6a10729a9@googlegroups.com> References: <56e451a3-01f7-4883-b4db-c3a6a10729a9@googlegroups.com> Message-ID: Am 23.02.16 um 22:39 schrieb kevind0718 at gmail.com: > from Tkinter import * > > def butContinue(): > root1.destroy() > [...] > entryName = Entry(root1).grid(row=1, column=1, pady=5) > [...] > butGo = Button(root1, text=" Continue " , command=butContinue ).grid(row=3, column=1, sticky=W, pady=10) > > root1.mainloop() > > print entryName.get("1.0", "end-1c" ) > When I click on butGo I get the error below. > So is this some sort of scope error? > why does entryName not exist for me to grab it's value? You call destroy() on the root window of Tk. If you destroy a window, that will destroy all of it's children. Therefore by deleting the root window, also the entryName widget was deleted. You need to export the values before you close the window, i.e. in the butContinue() function. There is another pitfall with Tk: if you delete the main window, you can get problems if you try to recreate it. If you want to show several pages in sequence or the like, you should withdraw the main window root1.withdraw() and popup a fresh toplevel by t=Toplevel() and do everything in t. Once you are finished, destroying t will keep your application alive (because the real root window is just hidden) Another comment: > root1.geometry("500x250") Do not do this. The grid and pack geometry managers compute the needed space automatically. A fixed size in pixels can easily break, when you go to another computer with a different font size, screen resolution, or a different OS. If you are not satisfied with the whitespace, use the padding options of grid or pack. Christian From auriocus at gmx.de Tue Feb 23 17:33:17 2016 From: auriocus at gmx.de (Christian Gollwitzer) Date: Tue, 23 Feb 2016 23:33:17 +0100 Subject: exit from Tkinter mainloop Python 2.7 In-Reply-To: <56e451a3-01f7-4883-b4db-c3a6a10729a9@googlegroups.com> References: <56e451a3-01f7-4883-b4db-c3a6a10729a9@googlegroups.com> Message-ID: Am 23.02.16 um 22:39 schrieb kevind0718 at gmail.com: > lblTop = Label(root1, text= ' Enter Values Below', font="Helvetica 14").grid(row=0, column=0, columnspan=2 , pady=5) > ##lblTop.pack(side = TOP) > > lblDB = Label(root1, text= ' Weight').grid(row=1, column=0 ) > lblPWord = Label(root1, text= ' Height').grid(row=2,column=0) Also here, the labels look odd. Have you tried to do some alignment of the text with spaces? Remove the sapces and look at the "justify" option of the label widget and the "sticky" option for grid. Christian From auriocus at gmx.de Tue Feb 23 17:37:20 2016 From: auriocus at gmx.de (Christian Gollwitzer) Date: Tue, 23 Feb 2016 23:37:20 +0100 Subject: [Newbie] Tkinter Question In-Reply-To: References: Message-ID: Am 23.02.16 um 23:19 schrieb Wildman: > I am familiar with OO programming but I am new to Python > and Tkinter. I am working on a gui program that creates > a couple of temporary files. As part of the Exit button > command they are deleted. If the program is shut down > using the window close button [X], the exit button code > is not executed and the temporary files are left behind. > That is a serious no-no. > > Is there a way to capture close button click to prevent > the program from closing and instead execute a command? It's done using the WM_DELETE_WINDOW protocol: http://stackoverflow.com/questions/111155/how-do-i-handle-the-window-close-event-in-tkinter Your program could still be killed by another way, e.g. from the task manager, which would leave the files behind. Christian From ckaynor at zindagigames.com Tue Feb 23 17:41:07 2016 From: ckaynor at zindagigames.com (Chris Kaynor) Date: Tue, 23 Feb 2016 14:41:07 -0800 Subject: [Newbie] Tkinter Question In-Reply-To: References: Message-ID: On Tue, Feb 23, 2016 at 2:19 PM, Wildman via Python-list < python-list at python.org> wrote: > I am familiar with OO programming but I am new to Python > and Tkinter. I am working on a gui program that creates > a couple of temporary files. As part of the Exit button > command they are deleted. If the program is shut down > using the window close button [X], the exit button code > is not executed and the temporary files are left behind. > That is a serious no-no. > > Is there a way to capture close button click to prevent > the program from closing and instead execute a command? > I'm talking about something like the Form_QueryUnload > event in Visual Basic. > > Alternately is there a way to just disable the button > or prevent it from being displayed in the first place? > > Any help appreciated. > There are a few options involved, however one other case to consider is if the user force kills the app (or the whole system is shut down improperly, such as a power loss). In those cases, should the temp files be removed? If yes, you'll need to use OS features to ensure the files are properly deleted by letting the OS know they are temporary. If the temp files need to be removed no matter what, the primary option is something like https://docs.python.org/2/library/tempfile.html#tempfile.NamedTemporaryFile using the delete=True arguments. Alternatively, you can probably use os.open with O_TEMPORARY (I belive this is the same as FILE_FLAG_DELETE_ON_CLOSE in the Win32 API, and there is probably something similar for Linux). If you only care if the user closes the app properly (no ending the process on Windows), you can perform the clean-up when the mainloop returns, aka Tkinter.Tk().mainloop(). Based off other GUI systems, this will only occur once all windows have been closed. Another option would be to use Python's atexit library: https://docs.python.org/3/library/atexit.html. atexit has the advantage that you could register/unregister the handlers when needed, rather than maintaining your own list to clean-up. Tkinter may also have some hook you can use to get when it is exiting, but I do not know what that is. The same limitations regarding OS exits will apply, however. If you need somewhere in the middle, you can try signal handlers, which, for some types of external exits, will fire, but it is not certain. Chris From __peter__ at web.de Tue Feb 23 17:45:08 2016 From: __peter__ at web.de (Peter Otten) Date: Tue, 23 Feb 2016 23:45:08 +0100 Subject: exit from Tkinter mainloop Python 2.7 References: <56e451a3-01f7-4883-b4db-c3a6a10729a9@googlegroups.com> Message-ID: Christian Gollwitzer wrote: > Am 23.02.16 um 22:39 schrieb kevind0718 at gmail.com: >> from Tkinter import * >> >> def butContinue(): >> root1.destroy() >> [...] >> entryName = Entry(root1).grid(row=1, column=1, pady=5) >> [...] >> butGo = Button(root1, text=" Continue " , command=butContinue >> ).grid(row=3, column=1, sticky=W, pady=10) >> >> root1.mainloop() >> >> print entryName.get("1.0", "end-1c" ) > >> When I click on butGo I get the error below. >> So is this some sort of scope error? >> why does entryName not exist for me to grab it's value? > > You call destroy() on the root window of Tk. If you destroy a window, > that will destroy all of it's children. Therefore by deleting the root > window, also the entryName widget was deleted. You need to export the > values before you close the window, i.e. in the butContinue() function. Even when you follow this advice, entryName was set to None by the line >> entryName = Entry(root1).grid(row=1, column=1, pady=5) as grid() always returns None. You need two steps entryName = Entry(root1) entryName.grid(row=1, column=1, pady=5) Also, >> print entryName.get("1.0", "end-1c" ) I believe that the Entry widget's get() method doesn't take any arguments. From steve at pearwood.info Tue Feb 23 20:40:07 2016 From: steve at pearwood.info (Steven D'Aprano) Date: Wed, 24 Feb 2016 12:40:07 +1100 Subject: Make a unique filesystem path, without creating the file References: <85r3gf55k4.fsf@benfinney.id.au> <85mvr26dij.fsf@benfinney.id.au> <87vb5gmtgj.fsf@elektro.pacujo.net> <56cba8c7$0$1611$c3e8da3$5496439d@news.astraweb.com> <877fhvnch0.fsf@elektro.pacujo.net> Message-ID: <56cd09f9$0$1620$c3e8da3$5496439d@news.astraweb.com> On Tue, 23 Feb 2016 05:54 pm, Marko Rauhamaa wrote: > Steven D'Aprano : > >> On Tue, 23 Feb 2016 06:32 am, Marko Rauhamaa wrote: >>> Under Linux, /dev/random is the way to go when strong security is >>> needed. Note that /dev/random is a scarce resource on ordinary >>> systems. >> >> That's actually incorrect, but you're not the only one to have been >> mislead by the man pages. >> >> http://sockpuppet.org/blog/2014/02/25/safely-generate-random-numbers/ > > Still, mostly hypnotic repetitions. Repetition for the sake of emphasis, because there are so many misled and confused people on the internet who misunderstand the difference between urandom and random and consequently give bad advice. I believe that the Linux man page for urandom is to blame, although I don't know why it hasn't been fixed. Possibly because it is *technically* correct, in the sense of "if you are concerned by the risk of being hit by a meteorite, wearing a stainless steel cooking pot on your head will give you some protection from meteorite strikes to the head". Everything in it is technically correct, but misleading. > However, it admits: > > But /dev/random also tries to keep track of how much entropy remains > in its kernel pool, and will occasionally go on strike if it decides > not enough remains. > > That's the whole point. Exactly, but you've missed the point. That is precisely why the blocking random is HARMFUL and should not be used. There is one, and only one, scenario when your CSPRNG should block: before the system has enough entropy to securely seed the CSPRNG and it is at risk of returning predictable numbers. But after that point has passed, there is no test you can perform to distinguish the outputs of /dev/random and /dev/urandom (apart from the blocking behaviour itself). If I give you a million numbers, there is no way you can tell whether I used random or urandom. The important thing here is that there is no difference in "quality" (whatever that means!) between the random numbers generated by urandom and those generated by random. They are equally unpredictable. They pass the same randomness tests. Neither is "better" or "worse" than the other, because they are both generated by the same CSPRNG or HRNG. Here is a summary of the random/urandom distinction on various Unixes: Linux: random blocks, urandom never blocks, both use the same CSPRNG based on SHA-1 hashes, both will use a HRNG if available FreeBSD: urandom is a link to random, which never blocks; uses 256-bit Yarrow CSPRNG, will use a HRNG if available OpenBSD: both never block; both use a variant of the RC4 CSPRNG (misleadingly renamed ARC4 due to licencing issues), in newer versions use the ChaCha20 CSPRNG OS X: both never block and use 160-bit Yarrow NetBSD: random blocks, urandom never blocks, both use the same AES-128 CSPRNG The NetBSD man pages are quite scathing: "The entropy accounting described here is not grounded in any cryptography theory. It is done because it was always done, and because it gives people a warm fuzzy feeling about information theory. ... History is littered with examples of broken entropy sources and failed system engineering for random number generators. Nobody has ever reported distinguishing AES ciphertext from uniform random without side channels, nor reported computing SHA-1 preimages faster than brute force. The folklore information- theoretic defence against computationally unbounded attackers replaces system engineering that successfully defends against realistic threat models by imaginary theory that defends only against fantasy threat models." To be clear, the "folklore information-theoretic defence" they are referring to is /dev/random's blocking behaviour. http://netbsd.gw.com/cgi-bin/man-cgi?rnd+4+NetBSD-current The blocking behaviour of /dev/random (on Linux) doesn't solve any real problems, but it *creates* new problems. /dev/random can block for minutes or even hours, especially straight after booting a freshly installed OS. This can be considered a Denial Of Service attack, and even if it isn't, it encourages developers to "fix" the problem by using their own home-brewed random numbers, weakening the security of the system. There's even a minority viewpoint that constantly adding new entropy to the CSPRNG is useless. Apart from collecting sufficient entropy for the initial seed, you should never add new entropy to the CSPRNG. Your CSPRNG is either cryptographically strong, or it isn't. If it is, then it is already unpredictable and adding more entropy is a waste of time. If it isn't, then adding more entropy isn't going to help you. Adding entropy is just one more component that can contain bugs (see the NetSBD comment about "broken entropy sources") or even allow an attack on the CSPRNG: http://blog.cr.yp.to/20140205-entropy.html There's one good argument for adding new entropy: imagine an attacker who, somehow, manages to get access to the internal state of your CSPRNG, but nothing else. (If they have access to your whole system, they own you, and the game is over.) In this scenario, instead of being able to predict the random numbers you produce forever, they will only be able to predict them until you've added sufficient fresh entropy into the system. Assuming they can't read or modify the entropy going in. > /dev/random will rather block the program than > lower the quality of the random numbers below a threshold. /dev/urandom > has no such qualms. No, that's wrong. There is no "lower the quality below a threshold". Both /dev/random and /dev/urandom use the same CSPRNG on all the Unixes I know of. So long as the CSPRNG has been seeded with enough entropy at the start, they are both of equally good quality. If the OS supports a HRNG, both will use it. On at least one Unix, FreeBSD, the two block devices are literally identical, and one is a link to the other. > If you use /dev/random instead of urandom, your program will > unpredictably (or, if you?re an attacker, very predictably) hang when > Linux gets confused about how its own RNG works. > > Yes, possibly indefinitely, too. > > Using /dev/random will make your programs less stable, but it won?t > make them any more cryptographically safe. > > It is correct that you shouldn't use /dev/random as a routine source of > bulk random numbers. It is also correct that /dev/urandom depletes the > entropy pool as effectively as /dev/random. Correct so far. But then: > However, when you are > generating signing or encryption keys, you should use /dev/random. And that is where you repeat something which is rank superstition. > As stated in : > > /dev/urandom should be used for essentially all random numbers > required, but /dev/random is sometimes used for things like extremely > sensitive, long-lived keys (e.g. GPG) or one-time pads. That doesn't mean anything beyond a statement of fact that people sometimes use /dev/random. Yeah, okay. So what? People sometimes use urandom for the same purposes too. I'm sure that, somewhere in the world, some poor dofus has used a lousy 16-bit PRNG to generate his GPG key. Doesn't mean that it is good to do so, or necessary to do so. -- Steven From nikhil.amraotkar1 at gmail.com Tue Feb 23 21:54:36 2016 From: nikhil.amraotkar1 at gmail.com (nikhil amraotkar) Date: Tue, 23 Feb 2016 18:54:36 -0800 (PST) Subject: Network Simulator Message-ID: <3fb8c52b-4d7c-474d-bef9-26a0a24b1d77@googlegroups.com> Hi...I need help to design a network simulator consisting for 5 routers in python...Any help would be appretiated... Thanks.. From joel.goldstick at gmail.com Tue Feb 23 22:42:36 2016 From: joel.goldstick at gmail.com (Joel Goldstick) Date: Tue, 23 Feb 2016 22:42:36 -0500 Subject: Network Simulator In-Reply-To: <3fb8c52b-4d7c-474d-bef9-26a0a24b1d77@googlegroups.com> References: <3fb8c52b-4d7c-474d-bef9-26a0a24b1d77@googlegroups.com> Message-ID: On Tue, Feb 23, 2016 at 9:54 PM, nikhil amraotkar < nikhil.amraotkar1 at gmail.com> wrote: > Hi...I need help to design a network simulator consisting for 5 routers in > python...Any help would be appretiated... > Thanks.. > -- > https://mail.python.org/mailman/listinfo/python-list > -- Joel Goldstick http://joelgoldstick.com/ http://cc-baseballstats.info/ From subhabangalore at gmail.com Tue Feb 23 23:04:51 2016 From: subhabangalore at gmail.com (subhabangalore at gmail.com) Date: Tue, 23 Feb 2016 20:04:51 -0800 (PST) Subject: How may I change values in tuples of list of lists? Message-ID: Hi I am trying to use the following set of tuples in list of lists. I am using a Python based library named, NLTK. >>> import nltk >>> from nltk.corpus import brown as bn >>> bt=bn.tagged_sents() >>> bt_5=bt[:5] >>> print bt [[(u'The', u'AT'), (u'Fulton', u'NP-TL'), (u'County', u'NN-TL'), (u'Grand', u'JJ-TL'), (u'Jury', u'NN-TL'), (u'said', u'VBD'), (u'Friday', u'NR'), (u'an', u'AT'), (u'investigation', u'NN'), (u'of', u'IN'), (u"Atlanta's", u'NP$'), (u'recent', u'JJ'), (u'primary', u'NN'), (u'election', u'NN'), (u'produced', u'VBD'), (u'``', u'``'), (u'no', u'AT'), (u'evidence', u'NN'), (u"''", u"''"), (u'that', u'CS'), (u'any', u'DTI'), (u'irregularities', u'NNS'), (u'took', u'VBD'), (u'place', u'NN'), (u'.', u'.')], [(u'The', u'AT'), (u'jury', u'NN'), (u'further', u'RBR'), (u'said', u'VBD'), (u'in', u'IN'), (u'term-end', u'NN'), (u'presentments', u'NNS'), (u'that', u'CS'), (u'the', u'AT'), (u'City', u'NN-TL'), (u'Executive', u'JJ-TL'), (u'Committee', u'NN-TL'), (u',', u','), (u'which', u'WDT'), (u'had', u'HVD'), (u'over-all', u'JJ'), (u'charge', u'NN'), (u'of', u'IN'), (u'the', u'AT'), (u'election', u'NN'), (u',', u','), (u'``', u'``'), (u'deserves', u'VBZ'), (u'the', u'AT'), (u'praise', u'NN'), (u'and', u'CC'), (u'thanks', u'NNS'), (u'of', u'IN'), (u'the', u'AT'), (u'City', u'NN-TL'), (u'of', u'IN-TL'), (u'Atlanta', u'NP-TL'), (u"''", u"''"), (u'for', u'IN'), (u'the', u'AT'), (u'manner', u'NN'), (u'in', u'IN'), (u'which', u'WDT'), (u'the', u'AT'), (u'election', u'NN'), (u'was', u'BEDZ'), (u'conducted', u'VBN'), (u'.', u'.')], ...] >>> Now if I want to change the values of tags like 'AT', 'NP-TL', 'NN-TL', etc. to some arbitrary ones like XX,YY,ZZ and yet preserve total structure of tuples in list of lists, please suggest how may I do it. I donot think it is an NLTK issue, rather a Python issue. I am trying to access and change but using for i,j in enumerate(bt_5), etc. bit long stuff. If any one may kindly suggest a smart line of code. I am using Python2.7.11 on MS-Windows-10. My NLTK version is 3.1 Thanks in advance. Regards, RP. From ben+python at benfinney.id.au Wed Feb 24 00:34:35 2016 From: ben+python at benfinney.id.au (Ben Finney) Date: Wed, 24 Feb 2016 16:34:35 +1100 Subject: How may I change values in tuples of list of lists? References: Message-ID: <85fuwizn6c.fsf@benfinney.id.au> subhabangalore at gmail.com writes: > Now if I want to change the values of tags like 'AT', 'NP-TL', > 'NN-TL', etc. to some arbitrary ones like XX,YY,ZZ and yet preserve > total structure of tuples in list of lists, please suggest how may I > do it. Changing items in lists is done by assigning to an index in that list, using index syntax:: big_list[17] = new_value This works because a list is a homogeneous mutable sequence: no position in the sequence has any special meaning, and you can change any of the items in the sequence to refer to a different value. Choosing the list type to represent a structure is a choice to express ?this sequence doesn't mean anything except the particular ordering of these items, and changing any of the values doesn't change the meaning of this sequence?. I think your question boils down to ?how do I change one item in a tuple??. If that's the essence of the question, the other stuff about where the tuple is found are irrelevant to solving this. The answer to that question is: you don't. Instead, you create a new tuple. A tuple is immutable; the programmer, by choosing the tuple type, has chosen to express ?this sequence is a single complex value, where the total set of items in this order has its own meaning, and if any of its items were different this would have a different meaning?. So, to create a new tuple based on an existing tuple, you need to specify each item in the new tuple. If you want some of the items in the new tuple to be from the existing tuple, you need to access those. foo = ('Fulton', 'NP-TL') bar = tuple(foo[0], 'XX') Is this awkward? Yes, so you should be thinking hard about whether you're fighting against the intent of the existing data structure. The programmer either made a bad design choice; or the choice of tuple type was for a good purpose. You should lean toward the latter until you know more about the program. -- \ ?It seems intuitively obvious to me, which means that it might | `\ be wrong.? ?Chris Torek | _o__) | Ben Finney From marko at pacujo.net Wed Feb 24 02:20:44 2016 From: marko at pacujo.net (Marko Rauhamaa) Date: Wed, 24 Feb 2016 09:20:44 +0200 Subject: Make a unique filesystem path, without creating the file References: <85r3gf55k4.fsf@benfinney.id.au> <85mvr26dij.fsf@benfinney.id.au> <87vb5gmtgj.fsf@elektro.pacujo.net> <56cba8c7$0$1611$c3e8da3$5496439d@news.astraweb.com> <877fhvnch0.fsf@elektro.pacujo.net> <56cd09f9$0$1620$c3e8da3$5496439d@news.astraweb.com> Message-ID: <87a8mqlgkz.fsf@elektro.pacujo.net> Steven D'Aprano : > On Tue, 23 Feb 2016 05:54 pm, Marko Rauhamaa wrote: >> However, when you are generating signing or encryption keys, you >> should use /dev/random. > > And that is where you repeat something which is rank superstition. Can you find info to back that up. All I've seen so far is forceful claims that's superstition ("These are not the droids you're looking for"). Even the ssh-keygen man page has: The reseeding of the OpenSSL random generator is usually done from /dev/urandom. If the SSH_USE_STRONG_RNG environment vari? able is set to value other than 0 the OpenSSL random generator is reseeded from /dev/random. Marko From nomail at invalid.com Wed Feb 24 03:08:21 2016 From: nomail at invalid.com (ast) Date: Wed, 24 Feb 2016 09:08:21 +0100 Subject: How to define what a class is ? Message-ID: <56cd64fb$0$9220$426a74cc@news.free.fr> Hi Since a class is an object, I ask myself how to define rigorously what a class is. classes are instances from type, but not all, since a class may be an instance of a metaclass A class is always callable A class inherit from some others classes, so they have a bases attribute any thing else ? Suppose I provide to you an object and that I ask to you to tell me if it is a class or not. How would you proceed ? thx From ian.g.kelly at gmail.com Wed Feb 24 03:46:48 2016 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Wed, 24 Feb 2016 01:46:48 -0700 Subject: How to define what a class is ? In-Reply-To: <56cd64fb$0$9220$426a74cc@news.free.fr> References: <56cd64fb$0$9220$426a74cc@news.free.fr> Message-ID: On Wed, Feb 24, 2016 at 1:08 AM, ast wrote: > Hi > > Since a class is an object, I ask myself how to define rigorously what a > class is. > > classes are instances from type, but not all, since > a class may be an instance of a metaclass All metaclasses are subclasses of type, so all classes are instances of type. > A class is always callable It doesn't strictly have to be. You could override the __call__ method in the metaclass to raise a TypeError if you don't want the class to be callable. I don't know of a use case for this, though. > Suppose I provide to you an object and that I > ask to you to tell me if it is a class or not. How > would you proceed ? import inspect inspect.isclass(x) From ben+python at benfinney.id.au Wed Feb 24 04:11:36 2016 From: ben+python at benfinney.id.au (Ben Finney) Date: Wed, 24 Feb 2016 20:11:36 +1100 Subject: How to define what a class is ? References: <56cd64fb$0$9220$426a74cc@news.free.fr> Message-ID: <85bn76zd4n.fsf@benfinney.id.au> "ast" writes: > Since a class is an object, I ask myself how to define rigorously what > a class is. A class is a type. This bears stating, because for a large part of Python's history, the two were distinct. A lot of Python documentation that has its roots in that history still is careful to maintain the distinction, which in current Python is completely obsolete. Every type (and therefore every class) is a template for instances. The type defines what values are possible for those instances, and also defines what behaviour those instances will have in common. > classes are instances from type, but not all, since a class may be an > instance of a metaclass Yes. Every class (every type) is an instance of a metaclass, and there is a particular metaclass named ?type?. > A class is always callable By default, calling a class creates an instance of that class, and returns that instance to the caller. > A class inherit from some others classes, so they have a bases > attribute That's not part of the definition, really. You need to know that, but it's not necessary to say what a class is. > any thing else ? See the reference documentation on this topic . > Suppose I provide to you an object and that I ask to you to tell me if > it is a class or not. How would you proceed ? If the object is an instance of the ?type? metaclass, the object is a type (i.e. a class). Metaclasses are callable, and return a type (a class) to the caller. The ?type? metaclass, if called with an object as a parameter, will return the type of that object. If you have the name ?foo? bound to an object, you can call:: type(foo) and the metaclass ?type? will return the instance of that object's type. For example:: >>> type("spam") >>> type(None) >>> type(3) >>> type(int) >>> type(type) If you want a boolean test:: >>> isinstance(3, type) False >>> isinstance("spam", type) False >>> isinstance(int, type) True >>> isinstance(type, type) True -- \ ?The fact of your own existence is the most astonishing fact | `\ you'll ever have to confront. Don't dare ever see your life as | _o__) boring, monotonous, or joyless.? ?Richard Dawkins, 2010-03-10 | Ben Finney From df at see.replyto.invalid Wed Feb 24 04:34:43 2016 From: df at see.replyto.invalid (Dave Farrance) Date: Wed, 24 Feb 2016 09:34:43 +0000 Subject: exit from Tkinter mainloop Python 2.7 References: <56e451a3-01f7-4883-b4db-c3a6a10729a9@googlegroups.com> Message-ID: kevind0718 at gmail.com wrote: >from Tkinter import * > >def butContinue(): > root1.destroy() As Christian said, you're destroying the root window and its children, so instead use root1.quit() here. > ... > >root1.mainloop() > >print entryName.get("1.0", "end-1c" ) >print entryPWord.get("1.0", "end-1c" ) And your root1.destroy() goes here instead. (The root window would normally be destroyed on the script exit, but some IDE debuggers will leave it open.) From p.f.moore at gmail.com Wed Feb 24 04:59:24 2016 From: p.f.moore at gmail.com (Paul Moore) Date: Wed, 24 Feb 2016 09:59:24 +0000 Subject: [Python-ideas] How the heck does async/await work in Python 3.5 In-Reply-To: References: <56c7d145$0$1597$c3e8da3$5496439d@news.astraweb.com> <56CCC98C.5060504@mail.de> Message-ID: On 24 February 2016 at 02:37, Terry Reedy wrote: > > In this essay, Brett says that asyncio added an event loop to Python. It > did, but it was the second. The tk event loop was added about 20 years ago > with tkinter. One of the things I would love to see (but don't have the time to work on) is a GUI event loop based around async/await. It would be a very useful example to make it clear to people that async/await isn't just about network protocols. Paul From victor.stinner at gmail.com Wed Feb 24 05:01:26 2016 From: victor.stinner at gmail.com (Victor Stinner) Date: Wed, 24 Feb 2016 11:01:26 +0100 Subject: [Python-ideas] How the heck does async/await work in Python 3.5 In-Reply-To: References: <56c7d145$0$1597$c3e8da3$5496439d@news.astraweb.com> <56CCC98C.5060504@mail.de> Message-ID: See also Doug Hellmann article on asyncio, from its serie of "Python 3 Module of the Week" articles: https://pymotw.com/3/asyncio/index.html Victor 2016-02-23 22:25 GMT+01:00 Joao S. O. Bueno : > Today I also stumbled on this helpful "essay" from Brett Cannon about > the same subject > > http://www.snarky.ca/how-the-heck-does-async-await-work-in-python-3-5 > > On 23 February 2016 at 18:05, Sven R. Kunze wrote: >> On 20.02.2016 07:53, Christian Gollwitzer wrote: >> >> If you have difficulties wit hthe overall concept, and if you are open to >> discussions in another language, take a look at this video: >> >> https://channel9.msdn.com/Shows/C9-GoingNative/GoingNative-39-await-co-routines >> >> MS has added coroutine support with very similar syntax to VC++ recently, >> and the developer tries to explain it to the "stackful" programmers. >> >> >> Because of this thread, I finally finished an older post collecting valuable >> insights from last year discussions regarding concurrency modules available >> in Python: http://srkunze.blogspot.com/2016/02/concurrency-in-python.html It >> appears to me that it would fit here well. >> >> @python-ideas >> Back then, the old thread ("Concurrency Modules") was like basically meant >> to result in something useful. I hope the post covers the essence of the >> discussion. >> Some even suggested putting the table into the Python docs. I am unaware of >> the formal procedure here but I would be glad if somebody could point be at >> the right direction if that the survey table is wanted in the docs. >> >> Best, >> Sven >> >> _______________________________________________ >> Python-ideas mailing list >> Python-ideas at python.org >> https://mail.python.org/mailman/listinfo/python-ideas >> Code of Conduct: http://python.org/psf/codeofconduct/ > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ From wjun77 at gmail.com Wed Feb 24 05:40:08 2016 From: wjun77 at gmail.com (=?UTF-8?B?546L54+6?=) Date: Wed, 24 Feb 2016 18:40:08 +0800 Subject: [Python-ideas] How the heck does async/await work in Python 3.5 In-Reply-To: <56CCC98C.5060504@mail.de> References: <56c7d145$0$1597$c3e8da3$5496439d@news.astraweb.com> <56CCC98C.5060504@mail.de> Message-ID: It seems an event loop is required for all async programs in python, but sometimes I need only lazy i/o. Is it possible with asyncio? Suppose io_operation() takes 3 seconds, then how can I write something like future = io_operation() print('Start') time.sleep(1) print('Something') time.sleep(2) print(future.result()) that print 'Start' immediately and the result of io_operation() 3 seconds later. The actual io operations are done by the io device that the main (and only) thread is not interrupted at all, if my understanding is correct. 2016-02-24 5:05 GMT+08:00 Sven R. Kunze : > On 20.02.2016 07:53, Christian Gollwitzer wrote: > > If you have difficulties wit hthe overall concept, and if you are open to > discussions in another language, take a look at this video: > > > https://channel9.msdn.com/Shows/C9-GoingNative/GoingNative-39-await-co-routines > > MS has added coroutine support with very similar syntax to VC++ recently, > and the developer tries to explain it to the "stackful" programmers. > > > Because of this thread, I finally finished an older post collecting > valuable insights from last year discussions regarding concurrency modules > available in Python: > http://srkunze.blogspot.com/2016/02/concurrency-in-python.html It appears > to me that it would fit here well. > > @python-ideas > Back then, the old thread ("Concurrency Modules") was like basically meant > to result in something useful. I hope the post covers the essence of the > discussion. > Some even suggested putting the table into the Python docs. I am unaware > of the formal procedure here but I would be glad if somebody could point be > at the right direction if that the survey table is wanted in the docs. > > Best, > Sven > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > From jdege at jdege.us Wed Feb 24 09:02:07 2016 From: jdege at jdege.us (jdege at jdege.us) Date: Wed, 24 Feb 2016 06:02:07 -0800 (PST) Subject: Could not find suitable distribution for Requirement.parse('urllib2') Message-ID: <8a91aabc-cf23-4352-b440-881227e05b91@googlegroups.com> I've been running a simple python program on my Linux Mint 13 box for some years that depends upon https://github.com/ryazwinski/pythings. I'm currently moving to a new Linux Mint 17 box, and I'm trying to move the program. I'm having a problem with installing python.py. The package comes with a setup.py: from setuptools import setup setup( name='pythings', py_modules=['pythings'], version='0.1', description='Python Library for interfacing with Withings API', install_requires=['simplejson', 'urllib2', 'hashlib'], ) When I run the install (sudo python setup.py install) I get an error: Could not find suitable distribution for Requirement.parse('urllib2') Any ideas on how I should proceed? Is there some repository that contains urllib2 that needs to be added to the setuptools configuration? Is there some new library that has supplanted urllib2? Help would be appreciated. From reevesa1 at ohio.edu Wed Feb 24 09:08:04 2016 From: reevesa1 at ohio.edu (Reeves, Andrew) Date: Wed, 24 Feb 2016 14:08:04 +0000 Subject: HELP Message-ID: I don't know what's going on. I am running windows 10 64 bit and per the pictures I am providing. I am also running Replicator G 0040. Just built a new computer with a fresh windows install. All drivers for the USB interface are up to date. But what ever is trying to install on your end is failing. What can I do or you do or someone do to get this issue fixed for me??? Please see attached photo Andrew Reeves, OIT Service Desk 740-593-1222 From breamoreboy at yahoo.co.uk Wed Feb 24 09:26:28 2016 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Wed, 24 Feb 2016 14:26:28 +0000 Subject: Could not find suitable distribution for Requirement.parse('urllib2') In-Reply-To: <8a91aabc-cf23-4352-b440-881227e05b91@googlegroups.com> References: <8a91aabc-cf23-4352-b440-881227e05b91@googlegroups.com> Message-ID: On 24/02/2016 14:02, jdege at jdege.us wrote: > I've been running a simple python program on my Linux Mint 13 box for some years that depends upon https://github.com/ryazwinski/pythings. > > I'm currently moving to a new Linux Mint 17 box, and I'm trying to move the program. I'm having a problem with installing python.py. > > The package comes with a setup.py: > > from setuptools import setup > setup( > name='pythings', > py_modules=['pythings'], > version='0.1', > description='Python Library for interfacing with Withings API', > install_requires=['simplejson', 'urllib2', 'hashlib'], > ) > > > When I run the install (sudo python setup.py install) I get an error: > > Could not find suitable distribution for Requirement.parse('urllib2') > > Any ideas on how I should proceed? Is there some repository that contains urllib2 that needs to be added to the setuptools configuration? Is there some new library that has supplanted urllib2? > > Help would be appreciated. > I'll guess that your new OS is running Python 3 by default, in which case urllib2 has gone. If I'm correct you should to be able to run the code through 2to3. -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From rtempl31 at gmail.com Wed Feb 24 09:39:29 2016 From: rtempl31 at gmail.com (Tem Pl) Date: Wed, 24 Feb 2016 06:39:29 -0800 (PST) Subject: [Python-ideas] How the heck does async/await work in Python 3.5 In-Reply-To: References: <56c7d145$0$1597$c3e8da3$5496439d@news.astraweb.com> <56CCC98C.5060504@mail.de> Message-ID: Here are some concurrency benchmarks for python vs other languages. https://github.com/atemerev/skynet/pull/53 Is there something wrong with this implementation? "Hope I suck at coroutines, because the results are abysmal. I get around 63s on my i5 MacBook Air Early 2015. For reference, the go version got around 700ms with GOMAXPROCS=4 and 1.7s with GOMAXPROCS=1 on this same machine." On Wednesday, February 24, 2016 at 5:02:16 AM UTC-5, Victor Stinner wrote: > > See also Doug Hellmann article on asyncio, from its serie of "Python 3 > Module of the Week" articles: > https://pymotw.com/3/asyncio/index.html > > Victor > > 2016-02-23 22:25 GMT+01:00 Joao S. O. Bueno >: > > Today I also stumbled on this helpful "essay" from Brett Cannon about > > the same subject > > > > http://www.snarky.ca/how-the-heck-does-async-await-work-in-python-3-5 > > > > On 23 February 2016 at 18:05, Sven R. Kunze > wrote: > >> On 20.02.2016 07:53, Christian Gollwitzer wrote: > >> > >> If you have difficulties wit hthe overall concept, and if you are open > to > >> discussions in another language, take a look at this video: > >> > >> > https://channel9.msdn.com/Shows/C9-GoingNative/GoingNative-39-await-co-routines > >> > >> MS has added coroutine support with very similar syntax to VC++ > recently, > >> and the developer tries to explain it to the "stackful" programmers. > >> > >> > >> Because of this thread, I finally finished an older post collecting > valuable > >> insights from last year discussions regarding concurrency modules > available > >> in Python: > http://srkunze.blogspot.com/2016/02/concurrency-in-python.html It > >> appears to me that it would fit here well. > >> > >> @python-ideas > >> Back then, the old thread ("Concurrency Modules") was like basically > meant > >> to result in something useful. I hope the post covers the essence of > the > >> discussion. > >> Some even suggested putting the table into the Python docs. I am > unaware of > >> the formal procedure here but I would be glad if somebody could point > be at > >> the right direction if that the survey table is wanted in the docs. > >> > >> Best, > >> Sven > >> > >> _______________________________________________ > >> Python-ideas mailing list > >> Python... at python.org > >> https://mail.python.org/mailman/listinfo/python-ideas > >> Code of Conduct: http://python.org/psf/codeofconduct/ > > _______________________________________________ > > Python-ideas mailing list > > Python... at python.org > > https://mail.python.org/mailman/listinfo/python-ideas > > Code of Conduct: http://python.org/psf/codeofconduct/ > _______________________________________________ > Python-ideas mailing list > Python... at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > From __peter__ at web.de Wed Feb 24 09:45:40 2016 From: __peter__ at web.de (Peter Otten) Date: Wed, 24 Feb 2016 15:45:40 +0100 Subject: Could not find suitable distribution for Requirement.parse('urllib2') References: <8a91aabc-cf23-4352-b440-881227e05b91@googlegroups.com> Message-ID: jdege at jdege.us wrote: > I've been running a simple python program on my Linux Mint 13 box for some > years that depends upon https://github.com/ryazwinski/pythings. > > I'm currently moving to a new Linux Mint 17 box, and I'm trying to move > the program. I'm having a problem with installing python.py. > > The package comes with a setup.py: > > from setuptools import setup > setup( > name='pythings', > py_modules=['pythings'], > version='0.1', > description='Python Library for interfacing with Withings API', > install_requires=['simplejson', 'urllib2', 'hashlib'], > ) > > > When I run the install (sudo python setup.py install) I get an error: > > Could not find suitable distribution for Requirement.parse('urllib2') > > Any ideas on how I should proceed? Is there some repository that contains > urllib2 that needs to be added to the setuptools configuration? Is there > some new library that has supplanted urllib2? > > Help would be appreciated. If you are using Python 2 try replacing > install_requires=['simplejson', 'urllib2', 'hashlib'], with install_requires=['simplejson'], as the last two dependencies are probably already satisfied by the standard library. From breamoreboy at yahoo.co.uk Wed Feb 24 10:04:25 2016 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Wed, 24 Feb 2016 15:04:25 +0000 Subject: HELP In-Reply-To: References: Message-ID: On 24/02/2016 14:08, Reeves, Andrew wrote: > I don't know what's going on. I am running windows 10 64 bit and per the pictures I am providing. I am also running Replicator G 0040. Just built a new computer with a fresh windows install. All drivers for the USB interface are up to date. But what ever is trying to install on your end is failing. What can I do or you do or someone do to get this issue fixed for me??? > > Please see attached photo > > Andrew Reeves, > > OIT Service Desk > 740-593-1222 > Your Python problem is what exactly, or have you simply sent your message to the wrong place? -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From marko at pacujo.net Wed Feb 24 10:23:32 2016 From: marko at pacujo.net (Marko Rauhamaa) Date: Wed, 24 Feb 2016 17:23:32 +0200 Subject: [Python-ideas] How the heck does async/await work in Python 3.5 References: <56c7d145$0$1597$c3e8da3$5496439d@news.astraweb.com> <56CCC98C.5060504@mail.de> Message-ID: <87vb5ejfnv.fsf@elektro.pacujo.net> Tem Pl : > Here are some concurrency benchmarks for python vs other languages. > > https://github.com/atemerev/skynet/pull/53 > > Is there something wrong with this implementation? It's a "fork bomb". Marko From ian.g.kelly at gmail.com Wed Feb 24 10:41:25 2016 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Wed, 24 Feb 2016 08:41:25 -0700 Subject: [Python-ideas] How the heck does async/await work in Python 3.5 In-Reply-To: <87vb5ejfnv.fsf@elektro.pacujo.net> References: <56c7d145$0$1597$c3e8da3$5496439d@news.astraweb.com> <56CCC98C.5060504@mail.de> <87vb5ejfnv.fsf@elektro.pacujo.net> Message-ID: On Wed, Feb 24, 2016 at 8:23 AM, Marko Rauhamaa wrote: > Tem Pl : > >> Here are some concurrency benchmarks for python vs other languages. >> >> https://github.com/atemerev/skynet/pull/53 >> >> Is there something wrong with this implementation? > > It's a "fork bomb". Isn't that the point of the benchmark? But yeah, I tried playing with this and here are my results. It's clearly pushing all those coroutines down into the event loop that causes the problem. If you remove the asyncio.as_completed calls and just await the descendant coroutines in order, then it's a lot faster -- but this may be technically cheating because then the coroutines are effectively just running recursively without involving the event loop, so there's no exercise of actual concurrency. # Straight-up recursion: 400 ms def func(level=0, index=0): if level < LEVELS: sons = [func(level=level+1, index=index*SONS + x) for x in range(SONS)] sum_ = 0 for son in sons: sum_ += son return sum_ else: return index import timeit print(timeit.repeat('func()', 'from __main__ import func', number=10)) # [4.016848850995302, 4.1330014310078695, 4.149791953997919] # Concurrent coroutines with event loop: 30 s import asyncio async def coroutine(level=0, index=0): if level < LEVELS: sons = [coroutine(level=level+1, index=index*SONS + x) for x in range(SONS)] sum_ = 0 for f in asyncio.as_completed(sons): got = await f sum_ += got return sum_ else: return index print(timeit.repeat('loop.run_until_complete(coroutine())', 'from __main__ import coroutine; import asyncio; loop = asyncio.get_event_loop()', number=1)) # [29.884846250002738, 30.26590966898948, 30.716448744002264] # Recursion with coroutines: 600 ms async def coro2(level=0, index=0): if level < LEVELS: sons = [coro2(level=level+1, index=index*SONS + x) for x in range(SONS)] sum_ = 0 for son in sons: got = await son sum_ += got return sum_ else: return index print(timeit.repeat('loop.run_until_complete(coro2())', 'from __main__ import coro2; import asyncio; loop = asyncio.get_event_loop()', number=1)) # [0.6264017040084582, 0.590631059021689, 0.5807875629980117] # Recursive coroutines, no event loop: 600 ms print(timeit.repeat('list(coro2().__await__())', 'from __main__ import coro2', number=10)) # [6.028770218021236, 6.002665672975127, 5.987304503010819] From sruthi223 at gmail.com Wed Feb 24 10:42:40 2016 From: sruthi223 at gmail.com (pyfreek) Date: Wed, 24 Feb 2016 07:42:40 -0800 (PST) Subject: child.before taking almost 1 minute to execute Message-ID: The following snippet alone is taking 1 minute to execute. is there any best way to find 'No such file' other than using child.before if not scrutinFile.startswith('/') : scrutinFile = '/'+ scrutinFile scrutinFileFtp = directory + scrutinFile filePath, file = os.path.split(scrutinFileFtp) p.sendline('cd %s'%(filePath)) p.expect([pexpect.EOF,pexpect.TIMEOUT]) if 'No such file' in p.before: print "No such directory exists!!" sys.exit(1) pyfreek From ikorot01 at gmail.com Wed Feb 24 10:43:01 2016 From: ikorot01 at gmail.com (Igor Korot) Date: Wed, 24 Feb 2016 10:43:01 -0500 Subject: HELP In-Reply-To: References: Message-ID: Hi, On Wed, Feb 24, 2016 at 10:04 AM, Mark Lawrence wrote: > On 24/02/2016 14:08, Reeves, Andrew wrote: >> >> I don't know what's going on. I am running windows 10 64 bit and per the >> pictures I am providing. I am also running Replicator G 0040. Just built a >> new computer with a fresh windows install. All drivers for the USB interface >> are up to date. But what ever is trying to install on your end is failing. >> What can I do or you do or someone do to get this issue fixed for me??? >> >> Please see attached photo >> >> Andrew Reeves, >> >> OIT Service Desk >> 740-593-1222 >> > > Your Python problem is what exactly, or have you simply sent your message to > the wrong place? > > -- > My fellow Pythonistas, ask not what our language can do for you, ask > what you can do for our language. .... and forgot to attach a picture? Thank you. > > Mark Lawrence > > -- > https://mail.python.org/mailman/listinfo/python-list From breamoreboy at yahoo.co.uk Wed Feb 24 10:50:07 2016 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Wed, 24 Feb 2016 15:50:07 +0000 Subject: HELP In-Reply-To: References: Message-ID: On 24/02/2016 15:43, Igor Korot wrote: > Hi, > > On Wed, Feb 24, 2016 at 10:04 AM, Mark Lawrence wrote: >> On 24/02/2016 14:08, Reeves, Andrew wrote: >>> >>> I don't know what's going on. I am running windows 10 64 bit and per the >>> pictures I am providing. I am also running Replicator G 0040. Just built a >>> new computer with a fresh windows install. All drivers for the USB interface >>> are up to date. But what ever is trying to install on your end is failing. >>> What can I do or you do or someone do to get this issue fixed for me??? >>> >>> Please see attached photo >>> >>> Andrew Reeves, >>> >>> OIT Service Desk >>> 740-593-1222 >>> >> >> Your Python problem is what exactly, or have you simply sent your message to >> the wrong place? >> >> -- >> My fellow Pythonistas, ask not what our language can do for you, ask >> what you can do for our language. > > .... and forgot to attach a picture? Attachments quite often do not come through. -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From martin at linux-ip.net Wed Feb 24 10:55:19 2016 From: martin at linux-ip.net (Martin A. Brown) Date: Wed, 24 Feb 2016 07:55:19 -0800 Subject: Network Simulator In-Reply-To: <3fb8c52b-4d7c-474d-bef9-26a0a24b1d77@googlegroups.com> References: <3fb8c52b-4d7c-474d-bef9-26a0a24b1d77@googlegroups.com> Message-ID: >Hi...I need help to design a network simulator consisting for 5 >routers in python...Any help would be appretiated... Have you looked at existing network simulators? On two different ends of the spectrum are: Switchyard, a small network simulator intended for pedagogy https://github.com/jsommers/switchyard NS-3, the researcher's toolkit https://www.nsnam.org/ https://www.nsnam.org/wiki/Python_bindings Good luck, -Martin -- Martin A. Brown http://linux-ip.net/ From marko at pacujo.net Wed Feb 24 11:13:40 2016 From: marko at pacujo.net (Marko Rauhamaa) Date: Wed, 24 Feb 2016 18:13:40 +0200 Subject: [Python-ideas] How the heck does async/await work in Python 3.5 References: <56c7d145$0$1597$c3e8da3$5496439d@news.astraweb.com> <56CCC98C.5060504@mail.de> <87vb5ejfnv.fsf@elektro.pacujo.net> Message-ID: <87r3g2jdcb.fsf@elektro.pacujo.net> Ian Kelly : > On Wed, Feb 24, 2016 at 8:23 AM, Marko Rauhamaa wrote: >> Tem Pl : >>> Is there something wrong with this implementation? >> >> It's a "fork bomb". > > Isn't that the point of the benchmark? I don't quite see the point of the program as it doesn't resemble anything I'd ever have an urge to write. As for benchmarks, Python is not supposed to be fast nor small. Python is actually quite slow and the objects have a large footprint. However, Python's strength is in its powerful expressivity. Still, it may be that coroutines end up being especially slow; that's how they played out in my head when I first was imagining what all those nested generators must be doing to produce their magic. Also, I'm afraid the expressivity part suffers a blow with the whole coroutine paradigm -- without having the otherworldly appeal of Scheme's continuations. Marko From ian.g.kelly at gmail.com Wed Feb 24 11:47:41 2016 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Wed, 24 Feb 2016 09:47:41 -0700 Subject: [Python-ideas] How the heck does async/await work in Python 3.5 In-Reply-To: <87r3g2jdcb.fsf@elektro.pacujo.net> References: <56c7d145$0$1597$c3e8da3$5496439d@news.astraweb.com> <56CCC98C.5060504@mail.de> <87vb5ejfnv.fsf@elektro.pacujo.net> <87r3g2jdcb.fsf@elektro.pacujo.net> Message-ID: On Wed, Feb 24, 2016 at 9:13 AM, Marko Rauhamaa wrote: > Ian Kelly : > >> On Wed, Feb 24, 2016 at 8:23 AM, Marko Rauhamaa wrote: >>> Tem Pl : >>>> Is there something wrong with this implementation? >>> >>> It's a "fork bomb". >> >> Isn't that the point of the benchmark? > > I don't quite see the point of the program as it doesn't resemble > anything I'd ever have an urge to write. That's reasonable. asyncio is designed for I/O-bound work, so a fair response to the benchmark is that subjecting it to a CPU-bound stress test is perhaps missing the point. I was curious about how much the selector-based event loop might be influencing the timing, so I tried instrumenting the selector. class ProfiledSelector(selectors.DefaultSelector): def __init__(self): super().__init__() self._calls = collections.Counter() def select(self, timeout=None): self._calls['select'] += 1 return super().select(timeout) def register(self, fileobj, events, data=None): self._calls['register'] += 1 return super().register(fileobj, events, data) selector = ProfiledSelector() loop = asyncio.SelectorEventLoop(selector) asyncio.set_event_loop(loop) loop.run_until_complete(coroutine()) # 499999500000 selector._calls # Counter({'select': 20, 'register': 1}) Only 20 iterations of the event loop? That doesn't seem unreasonable. From skip.montanaro at gmail.com Wed Feb 24 13:57:56 2016 From: skip.montanaro at gmail.com (Skip Montanaro) Date: Wed, 24 Feb 2016 12:57:56 -0600 Subject: Setting up/authenticating Google API? In-Reply-To: References: Message-ID: > > Your question seems quite cinfusing to me but I think following may is what you are asking for.(Read the source) > > https://google-mail-oauth2-tools.googlecode.com/svn/trunk/python/oauth2.py > > After getting the authentication you can use imaplib to get all the gmail Data. Thanks, I will take a look. Skip From aaron.christensen at gmail.com Wed Feb 24 14:27:06 2016 From: aaron.christensen at gmail.com (Aaron Christensen) Date: Wed, 24 Feb 2016 14:27:06 -0500 Subject: Network Simulator In-Reply-To: <3fb8c52b-4d7c-474d-bef9-26a0a24b1d77@googlegroups.com> References: <3fb8c52b-4d7c-474d-bef9-26a0a24b1d77@googlegroups.com> Message-ID: On Feb 23, 2016 9:55 PM, "nikhil amraotkar" wrote: > > Hi...I need help to design a network simulator consisting for 5 routers in python...Any help would be appretiated... > Thanks.. > -- > https://mail.python.org/mailman/listinfo/python-list What is the purpose for designing it in Python? I'm a network engineer and use GNS3 and Packet Tracer. I could see the benefit of creating Python scripts to manage networking devices but not sure why you would want to use Python. I use Python to generate configuration scripts but haven't used it for any management yet. From greg.ewing at canterbury.ac.nz Wed Feb 24 15:44:33 2016 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Thu, 25 Feb 2016 09:44:33 +1300 Subject: How to define what a class is ? In-Reply-To: References: <56cd64fb$0$9220$426a74cc@news.free.fr> Message-ID: Ian Kelly wrote: > All metaclasses are subclasses of type, so all classes are instances of type. I think that's about the most general definition you can find. Almost everything else that you might think of as being part of the classness of a class can be overridden. Another definition might be that it's something that is potentially the result of type(x) for some x. I think that's equivalent, because at the C level type(x) returns the value of x->ob_type, which always points to an instance of type or a subclass thereof. -- Greg From emile at fenx.com Wed Feb 24 15:57:31 2016 From: emile at fenx.com (Emile van Sebille) Date: Wed, 24 Feb 2016 12:57:31 -0800 Subject: child.before taking almost 1 minute to execute In-Reply-To: References: Message-ID: On 2/24/2016 7:42 AM, pyfreek wrote: > The following snippet alone is taking 1 minute to execute. is there any best way to find 'No such file' other than using child.before > > if not scrutinFile.startswith('/') : > scrutinFile = '/'+ scrutinFile > scrutinFileFtp = directory + scrutinFile > filePath, file = os.path.split(scrutinFileFtp) > p.sendline('cd %s'%(filePath)) > p.expect([pexpect.EOF,pexpect.TIMEOUT]) > if 'No such file' in p.before: > print "No such directory exists!!" > sys.exit(1) I'd guess that you've got your p.expect line wrong -- it looks to me like you're allowing that line to complete only upon seeing an EOF or TIMEOUT, and that it's timing out as a result. Emile From grsmith at atlanticbb.net Wed Feb 24 15:59:40 2016 From: grsmith at atlanticbb.net (grsmith at atlanticbb.net) Date: Wed, 24 Feb 2016 15:59:40 -0500 Subject: Nested List question Message-ID: <4AD2810BE3B346399F777EDEAB079BDA@OPTIPLEX990> All, Can you have a phython list like: ['George', 'Soros', ['99 First Street', '33 Broadway Avenue', ['Apt 303'], '1 Park Avenue'], 'New York', 'NY'] In other words how do you correctly nest the ['Apt 303'] so it goes with 33 Broadway Avenue. Also, I tried several ways and could not figure out how to get the ['Apt 303'] out of the list. How can you do that. It is the ['Apt 303'] that is the problem, I know how to do the other elements. thanks George From greg.ewing at canterbury.ac.nz Wed Feb 24 16:00:29 2016 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Thu, 25 Feb 2016 10:00:29 +1300 Subject: [Python-ideas] How the heck does async/await work in Python 3.5 In-Reply-To: References: <56c7d145$0$1597$c3e8da3$5496439d@news.astraweb.com> <56CCC98C.5060504@mail.de> Message-ID: ?? wrote: > Suppose io_operation() takes 3 seconds, then how can I write something like > > future = io_operation() > print('Start') > time.sleep(1) > print('Something') > time.sleep(2) > print(future.result()) > > that print 'Start' immediately and the result of io_operation() 3 seconds > later. Yes, Python can do this, but you probably need to use real threads. The only exception would be if io_operation() were something you could fire off with a single system call that's guaranteed not to block, and then wait for the result later. Opportunities for things like that are rare in unix. (Windows might be different, I'm not sure.) -- Greg From greg.ewing at canterbury.ac.nz Wed Feb 24 16:09:52 2016 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Thu, 25 Feb 2016 10:09:52 +1300 Subject: child.before taking almost 1 minute to execute In-Reply-To: References: Message-ID: pyfreek wrote: > The following snippet alone is taking 1 minute to execute. is there any best way to find 'No such file' other than using child.before > > if not scrutinFile.startswith('/') : > scrutinFile = '/'+ scrutinFile > scrutinFileFtp = directory + scrutinFile > filePath, file = os.path.split(scrutinFileFtp) > p.sendline('cd %s'%(filePath)) > p.expect([pexpect.EOF,pexpect.TIMEOUT]) > if 'No such file' in p.before: > print "No such directory exists!!" > sys.exit(1) If you're talking to an ftp client here, you might like to consider using the ftplib module in the standard library. It ought to take care of the messy details of error detection for you. -- Greg From amfarrell at mit.edu Wed Feb 24 16:11:35 2016 From: amfarrell at mit.edu (Andrew Farrell) Date: Wed, 24 Feb 2016 15:11:35 -0600 Subject: Nested List question In-Reply-To: <4AD2810BE3B346399F777EDEAB079BDA@OPTIPLEX990> References: <4AD2810BE3B346399F777EDEAB079BDA@OPTIPLEX990> Message-ID: you can indeed have a nested list. One way you could do that looks like donor = [ 'George', 'Soros', [ #<- 2nd element of outermost list '99 First Street', [ #<- 1st element of middling list '33 Broadway Avenue', 'Apt 303', #<- 1st element of innermost list ], '1 Park Avenue' ], 'New York', 'NY', ] Then, to extract "Apt 303", you could do donor[2][1][1] A better explaination of nested lists can be found if you ctrl-f search for "nested list" on this chapter of the book How to Think Like a Computer Scientist with python. On Wed, Feb 24, 2016 at 2:59 PM, wrote: > All, > > Can you have a phython list like: > ['George', > 'Soros', > ['99 First Street', > '33 Broadway Avenue', ['Apt 303'], > '1 Park Avenue'], > 'New York', 'NY'] > > In other words how do you correctly nest the > ['Apt 303'] so it goes with 33 Broadway Avenue. > > Also, I tried several ways and could not figure out > how to get the ['Apt 303'] out of the list. How can > you do that. It is the ['Apt 303'] that is the problem, > I know how to do the other elements. > > thanks > George > -- > https://mail.python.org/mailman/listinfo/python-list > From invalid at invalid.invalid Wed Feb 24 16:14:46 2016 From: invalid at invalid.invalid (Grant Edwards) Date: Wed, 24 Feb 2016 21:14:46 +0000 (UTC) Subject: Nested List question References: Message-ID: On 2016-02-24, wrote: > All, > > Can you have a phython list like: > ['George', > 'Soros', > ['99 First Street', > '33 Broadway Avenue', ['Apt 303'], > '1 Park Avenue'], > 'New York', 'NY'] Sure: $ python3 Python 3.4.3 (default, Feb 12 2016, 15:58:12) [GCC 4.9.3] on linux Type "help", "copyright", "credits" or "license" for more information. >>> ['George', ... 'Soros', ... ['99 First Street', ... '33 Broadway Avenue', ['Apt 303'], ... '1 Park Avenue'], ... 'New York', 'NY'] ['George', 'Soros', ['99 First Street', '33 Broadway Avenue', ['Apt 303'], '1 Park Avenue'], 'New York', 'NY'] > In other words how do you correctly nest the > ['Apt 303'] so it goes with 33 Broadway Avenue. I'm afraid I don't know what you mean by "goes with". > Also, I tried several ways and could not figure out how to get the > ['Apt 303'] out of the list. How can you do that. Use the "del" operator: >>> foo = ['George', ... 'Soros', ... ['99 First Street', ... '33 Broadway Avenue', ['Apt 303'], ... '1 Park Avenue'], ... 'New York', 'NY'] >>> foo ['George', 'Soros', ['99 First Street', '33 Broadway Avenue', ['Apt 303'], '1 Park Avenue'], 'New York', 'NY'] >>> del foo[2][2] >>> foo ['George', 'Soros', ['99 First Street', '33 Broadway Avenue', '1 Park Avenue'], 'New York', 'NY'] > It is the ['Apt 303'] that is the problem, I know how to do the > other elements. I don't think I understand what your problem is. -- Grant Edwards grant.b.edwards Yow! I feel like I'm at in a Toilet Bowl with a gmail.com thumbtack in my forehead!! From python at lucidity.plus.com Wed Feb 24 16:28:56 2016 From: python at lucidity.plus.com (Erik) Date: Wed, 24 Feb 2016 21:28:56 +0000 Subject: Nested List question In-Reply-To: <4AD2810BE3B346399F777EDEAB079BDA@OPTIPLEX990> References: <4AD2810BE3B346399F777EDEAB079BDA@OPTIPLEX990> Message-ID: <56CE2098.2070507@lucidity.plus.com> On 24/02/16 20:59, grsmith at atlanticbb.net wrote: > Can you have a phython list like: > ['George', > 'Soros', > ['99 First Street', > '33 Broadway Avenue', ['Apt 303'], > '1 Park Avenue'], > 'New York', 'NY'] > > In other words how do you correctly nest the > ['Apt 303'] so it goes with 33 Broadway Avenue. The way to keep those particular items together is to have something like: data = \ ['George', 'Soros', [ '99 First Street', ['33 Broadway Avenue', 'Apt 303'], '1 Park Avenue' ], 'New York', 'NY' ] And then, guessing from the structure of your list, you'd do something like: forename, surname, addresses, city, state = data And then, something like: for address in addresses: if type(address) == StringType: apt = None else: address, apt = address That should work to pick apart with what you have presented above, but I think you'd be better off presenting the data with a bit more structure in the first place. > Also, I tried several ways and could not figure out > how to get the ['Apt 303'] out of the list. How can > you do that. It would be much easier to work out what you mean by this if you'd have actually said what you'd tried and what you had expected to happen and why what did happen wasn't right for you. E. From breamoreboy at yahoo.co.uk Wed Feb 24 16:38:09 2016 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Wed, 24 Feb 2016 21:38:09 +0000 Subject: Nested List question In-Reply-To: <4AD2810BE3B346399F777EDEAB079BDA@OPTIPLEX990> References: <4AD2810BE3B346399F777EDEAB079BDA@OPTIPLEX990> Message-ID: On 24/02/2016 20:59, grsmith at atlanticbb.net wrote: > All, > > Can you have a phython list like: > ['George', > 'Soros', > ['99 First Street', > '33 Broadway Avenue', ['Apt 303'], > '1 Park Avenue'], > 'New York', 'NY'] > > In other words how do you correctly nest the > ['Apt 303'] so it goes with 33 Broadway Avenue. > > Also, I tried several ways and could not figure out > how to get the ['Apt 303'] out of the list. How can > you do that. It is the ['Apt 303'] that is the problem, > I know how to do the other elements. > > thanks > George > As you've all ready had answers, I'd simply ask is a list the best data structure for your application? -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From grsmith at atlanticbb.net Wed Feb 24 17:33:20 2016 From: grsmith at atlanticbb.net (grsmith at atlanticbb.net) Date: Wed, 24 Feb 2016 17:33:20 -0500 Subject: Nested List question In-Reply-To: <56CE2098.2070507@lucidity.plus.com> References: <4AD2810BE3B346399F777EDEAB079BDA@OPTIPLEX990> <56CE2098.2070507@lucidity.plus.com> Message-ID: <9491F945BC274294822A6E07D37D2FB9@OPTIPLEX990> Erik, Works perfectly, thanks much !!! Mark, I am tied to data structure. for address in addresses: if isinstance(address, str): apt = None print(address, apt) else: address, apt = address print(address, apt) 99 First Street None 33 Broadway Avenue Apt 303 1 Park Avenue None -----Original Message----- From: Erik Sent: Wednesday, February 24, 2016 4:28 PM To: grsmith at atlanticbb.net ; python-list Subject: Re: Nested List question On 24/02/16 20:59, grsmith at atlanticbb.net wrote: > Can you have a phython list like: > ['George', > 'Soros', > ['99 First Street', > '33 Broadway Avenue', ['Apt 303'], > '1 Park Avenue'], > 'New York', 'NY'] > > In other words how do you correctly nest the > ['Apt 303'] so it goes with 33 Broadway Avenue. The way to keep those particular items together is to have something like: data = \ ['George', 'Soros', [ '99 First Street', ['33 Broadway Avenue', 'Apt 303'], '1 Park Avenue' ], 'New York', 'NY' ] And then, guessing from the structure of your list, you'd do something like: forename, surname, addresses, city, state = data And then, something like: for address in addresses: if type(address) == StringType: apt = None else: address, apt = address That should work to pick apart with what you have presented above, but I think you'd be better off presenting the data with a bit more structure in the first place. > Also, I tried several ways and could not figure out > how to get the ['Apt 303'] out of the list. How can > you do that. It would be much easier to work out what you mean by this if you'd have actually said what you'd tried and what you had expected to happen and why what did happen wasn't right for you. E. From mscir at yahoo.com Wed Feb 24 18:46:53 2016 From: mscir at yahoo.com (Mike S) Date: Wed, 24 Feb 2016 15:46:53 -0800 Subject: Python used in several places in LIGO effort Message-ID: https://www.reddit.com/r/IAmA/comments/45g8qu/we_are_the_ligo_scientific_collaboration_and_we/czxnlux?imm_mid=0e0d97&cmp=em-data-na-na-newsltr_20160224 From wjun77 at gmail.com Wed Feb 24 19:40:44 2016 From: wjun77 at gmail.com (=?UTF-8?B?546L54+6?=) Date: Thu, 25 Feb 2016 08:40:44 +0800 Subject: [Python-ideas] How the heck does async/await work in Python 3.5 In-Reply-To: References: <56c7d145$0$1597$c3e8da3$5496439d@news.astraweb.com> <56CCC98C.5060504@mail.de> Message-ID: I think this is possible if I understand what happens under the hood. I wonder how event loop and async io functions such as asyncio.open_connection cooperate to do async io in one thread. Maybe it exploits low-level details and is OS or even device specific. I think I should take a look at the source code when I have time. 2016-02-25 5:00 GMT+08:00 Gregory Ewing : > ?? wrote: > >> Suppose io_operation() takes 3 seconds, then how can I write something >> like >> >> future = io_operation() >> print('Start') >> time.sleep(1) >> print('Something') >> time.sleep(2) >> print(future.result()) >> >> that print 'Start' immediately and the result of io_operation() 3 seconds >> later. >> > > Yes, Python can do this, but you probably need to use real > threads. The only exception would be if io_operation() were > something you could fire off with a single system call that's > guaranteed not to block, and then wait for the result later. > Opportunities for things like that are rare in unix. > (Windows might be different, I'm not sure.) > > -- > Greg > -- > https://mail.python.org/mailman/listinfo/python-list > From steve at pearwood.info Wed Feb 24 19:41:55 2016 From: steve at pearwood.info (Steven D'Aprano) Date: Thu, 25 Feb 2016 11:41:55 +1100 Subject: Make a unique filesystem path, without creating the file References: <85r3gf55k4.fsf@benfinney.id.au> <85mvr26dij.fsf@benfinney.id.au> <56cba1bc$0$1604$c3e8da3$5496439d@news.astraweb.com> <87si0k2mal.fsf@jester.gateway.pace.com> <87h9gzx2co.fsf@jester.gateway.pace.com> Message-ID: <56ce4dd5$0$1614$c3e8da3$5496439d@news.astraweb.com> On Tue, 23 Feb 2016 07:22 pm, Paul Rubin wrote: > Mark Lawrence writes: >> https://mail.python.org/pipermail/python-ideas/2015-September/036333.html >> then http://www.gossamer-threads.com/lists/python/dev/1223780 > > Thanks. It would be nice if those were gatewayed to usenet like this > group is. I can't bring myself to subscribe to mailing lists. > >>> There are a few other choices in the PEP whose benefit is unclear to me, >>> but they aren't harmful, and I guess the decisions have already been >>> made. >> The PEP status is draft so is subject to change. > > Well they might be changeable but it sounds like there's a level of > consensus by now, that wouldn't be helped by more bikeshedding over > relatively minor stuff. I might write up some further comments and post > them here If you're going to do so, please do so in the next few days (or write to me off list to ask for an extension) because I intend to ask Guido for a ruling early next week. -- Steven From af154 at evansville.edu Wed Feb 24 20:01:09 2016 From: af154 at evansville.edu (Feagans, Mandy) Date: Thu, 25 Feb 2016 01:01:09 +0000 Subject: Computational Chemistry Analysis Message-ID: Dear Python, Hi! I am a student interested in conducting computational analysis of protein-ligand binding for drug development analysis. Recently, I read of an individual using a python program for their studies of protein-ligand binding. As I have been reading about Python programs, however, all I have been able to find are programs that use the python script (i.e. Autodock). I was hoping to see if there were any programs specifically run through the python programming that ran similar analysis to Autodock? Thank you for any assistance you may be able to provide! Thank you, Mandy Feagans From drsalists at gmail.com Wed Feb 24 20:07:33 2016 From: drsalists at gmail.com (Dan Stromberg) Date: Wed, 24 Feb 2016 17:07:33 -0800 Subject: "from module import data; print(data)" vs "import module; print(module.data)" Message-ID: Could people please compare and contrast the two ways of doing imports in the Subject line? I've long favored the latter, but I'm working in a code base that prefers the former. Is it fair to say that the former increases the surface area of your shared (sometimes mutable) state? It's clear that the former saves keystrokes. I find the latter a little more clear, because you don't have to go look for where a symbol came from. Anything else? Thanks! PS: Haskell seems better at the former than Python; Haskell tells you if you import two identical symbols from two different places, when you try to use one of them - not at import time. I believe in Python, whichever symbol you import last, wins. Haskell does not warn you at import time, which is fine. Not sure about OCaml or whatever else. From steve at pearwood.info Wed Feb 24 20:10:12 2016 From: steve at pearwood.info (Steven D'Aprano) Date: Thu, 25 Feb 2016 12:10:12 +1100 Subject: HELP References: Message-ID: <56ce5476$0$1615$c3e8da3$5496439d@news.astraweb.com> On Thu, 25 Feb 2016 01:08 am, Reeves, Andrew wrote: > I don't know what's going on. And neither do we. Andrew, can you please describe using words: * what you are trying to do * what you expect to happen * what actually happens * if it is not obvious, what it has to do with Python Don't use screen shots or pictures, at least, don't use them ALONE. Some of us are visually impaired. We have screen-readers that can deal with text, but pictures might as well not exist. Others are reading this from a newsgroup on Usenet, and attachments are not permitted. If you must provide a screen shot, post it on Imgur and provide the link. Most other photo-sharing sites are a pain to deal with for the recipient. If you're thinking "That's a lot of effort for me to go through, just for free technical advice!" you're absolutely right. Feel free to pay for commercial advice from a professional computer consultant instead, who will charge for the time they spend trying to understand your problem before they even begin trying to solve it. If you do decide to ask here, this is a bad way to ask for help: "HELP! I'm running a Computertron 3000 and nothing works! PLEASE HELP!!!" This is a good way to ask for help: "Hi, I'm running a Computertron 3000. You might not have heard of it, so here is a link to its home page: http:// ... It is running Windows 10 as the OS. (I assume you've heard of that :-) When I try to install Python, the installation seems to succeeds but running the Python interpreter causes a Yellow Screen Of Death. Here are the steps I took: - downloaded the Windows installer from here http:// ... - ran the installer by double-clicking - at the prompt, answered "Yes" to the question "Add icons?" There were no other messages, so I thought it was installed successfully. When I double-click the Python icon, I get a Yellow Screen Of Death. I expected the interpreter to run. In case it helps, I took a picture of the YSOD and posted it on imgur here: http:// ... Thanks in advance." -- Steven From kdiegorsantos at gmail.com Wed Feb 24 20:14:44 2016 From: kdiegorsantos at gmail.com (kdiegorsantos at gmail.com) Date: Wed, 24 Feb 2016 17:14:44 -0800 (PST) Subject: How to get all IP addresses in python? In-Reply-To: References: Message-ID: <65a86196-0711-4252-aa66-fb9ef4514dc4@googlegroups.com> On Friday, July 11, 2003 at 1:54:47 AM UTC-3, Yun Mao wrote: > Hi. I did a little homework on google for this question. The answer was: > >>> import socket > >>> hostname = socket.gethostname() > >>> ip = socket.gethostbyname(hostname) > >>> print ip > 192.168.55.101 > But what if I have several interfaces, say eth0, eth0:1, eth1, etc. How to > get all of them? It would be a little undesirable if people suggest to parse > the result of ifconfig because I in particular want the code to be cross > platform. Thanks a lot! > > Yun import socket def get_ipaddr(): proc = subprocess.Popen(["ip addr show | egrep inet | awk '{{print $2}}' | awk -F'/' '{{print $1}}' | egrep -v '^127' | xargs'"],stdout=subprocess.PIPE, shell=True) x = proc.communicate()[0] return x.strip() From rosuav at gmail.com Wed Feb 24 20:19:20 2016 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 25 Feb 2016 12:19:20 +1100 Subject: Import collisions (was Re: "from module import data; print(data)") Message-ID: On Thu, Feb 25, 2016 at 12:07 PM, Dan Stromberg wrote: > PS: Haskell seems better at the former than Python; Haskell tells you > if you import two identical symbols from two different places, when > you try to use one of them - not at import time. I believe in Python, > whichever symbol you import last, wins. Haskell does not warn you at > import time, which is fine. Not sure about OCaml or whatever else. In a sense, it's not the two imports that matter, but the situation that the second import statement is rebinding a name. Would it be possible to create a "mollyguard" import hook that snoops the globals of the importing module and raises an ImportWarning if it detects a rebind? Not that I've ever been in the situation of wanting one, but just as a point of intellectual curiosity. ChrisA From cs at zip.com.au Wed Feb 24 20:22:36 2016 From: cs at zip.com.au (Cameron Simpson) Date: Thu, 25 Feb 2016 12:22:36 +1100 Subject: "from module import data; print(data)" vs "import module; print(module.data)" In-Reply-To: References: Message-ID: <20160225012236.GA12050@cskk.homeip.net> On 24Feb2016 17:07, Dan Stromberg wrote: >Could people please compare and contrast the two ways of doing imports >in the Subject line? > >I've long favored the latter, but I'm working in a code base that >prefers the former. I largely use the former "from module import name, name..." over "import module". It makes your code more readable by making your calls to module-supplied names less verbose. That readbility does rely on the names being not too many or not to vague. As a glaring counter example to my preference, consider: from os.path import join "join" is a pretty generic name, so my code almost always goes either: import os.path and uses "os.path.join(a, b)" or (recently, in code doing a lot of path manipulation): from os.path import join as joinpath as with (real example): from os.path import abspath, normpath, join as joinpath and then using "joinpath(a, b)". >Is it fair to say that the former increases the surface area of your >shared (sometimes mutable) state? It can go the other way. Consider: import module module.data = 1 versus: from module import data data = 1 The former directly affects the name "data" in "module"; other users of "module" will see this change. The latter binds the _local_ name "data" to 1, leaving "module.data" as it was. Of course both of these are things that are almost never done (look up the term "monkey patch" for more info, but the difference is salient. >It's clear that the former saves keystrokes. >I find the latter a little more clear, because you don't have to go >look for where a symbol came from. I take the view that your should broadly know where a name came from, at least as far as what it does, and I will take shorter readably code over long.module.names.with.dots. And as mentioned above, is a name is vague (or conflicting), there is always the useful: from module import name as better_name >PS: Haskell seems better at the former than Python; Haskell tells you >if you import two identical symbols from two different places, when >you try to use one of them - not at import time. I believe in Python, >whichever symbol you import last, wins. It wins in exactly the same way that: x = 1 x = 2 wins for 2: they both run, and after they've run "x" will be bound to 2. Python is dynamic, and this is legal. It may be that linting tools like pylint will warn about conflicting names from imports. Cheers, Cameron Simpson From MrJean1 at gmail.com Wed Feb 24 21:21:50 2016 From: MrJean1 at gmail.com (MrJean1) Date: Wed, 24 Feb 2016 18:21:50 -0800 (PST) Subject: How to get all IP addresses in python? In-Reply-To: References: Message-ID: <6fef9aa3-cf83-4363-8aff-cd4ece2f281d@googlegroups.com> Try function getIPs from this module /Jean From steve+comp.lang.python at pearwood.info Wed Feb 24 22:39:12 2016 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Thu, 25 Feb 2016 14:39:12 +1100 Subject: "from module import data; print(data)" vs "import module; print(module.data)" References: Message-ID: <56ce7762$0$11100$c3e8da3@news.astraweb.com> On Thursday 25 February 2016 12:07, Dan Stromberg wrote: > Could people please compare and contrast the two ways of doing imports > in the Subject line? from module import data; print(data) import module; print(module.data) > I've long favored the latter, but I'm working in a code base that > prefers the former. > > Is it fair to say that the former increases the surface area of your > shared (sometimes mutable) state? No, I can't say that it is. If anything, the opposite is the case: it decreases the surface area, because `data` now is local to the importing (not imported) module. Rebinding data will not affect anything else. There are two scenarios (`from module import...` versus `import ...`), each with two cases: mutation, and rebinding: (1) from module import data (a) Mutation (only applies to mutable data like lists, dicts etc) In place mutation affects everything relying on module.data regardless of how or where it is accessed. (b) Rebinding (e.g. `data = []`) Because data is now a local name, you only affect the local scope. (2) import module (a) Mutation is no different from (1)(a) above. No change. (b) Rebinding `module.data = []` affects the imported module, and therefore everything that relies on it. So the `from module import data` scenario reduces the number of ways that you can affect other modules. That may be important to you, but usually the answer to that is "well don't rebind attributes of modules unless you really mean to". Sometimes doing `module.data = ...` is not a bad thing. On the other hand, seeing something like: module.data.append(999) should stand out to a reasonably experienced developer as possibly effecting other modules. But: data.append(999) may fool them into thinking that the change is only local to the current module. Dangerous things should look dangerous: this is a point in favour of using `import module` in preference to `from`. > It's clear that the former saves keystrokes. Code is read about 100 times more than it is written. Saving keystokes should be quite low on the list of priorities. It also saves line width, which may sometimes impact readability. But then readability is only weakly correlated to line length. As you say: > I find the latter a little more clear, because you don't have to go > look for where a symbol came from. which is also a very good point. Personally, I prefer the `import module` over `from module` except when I don't. Consequently I will sometimes even use both in the same module: import os from itertools import zip_longest depending on what's more convenient and useful for each. I think that having a hard rule that you MUST use one or the other is silly, but I guess that since Python makes "brace wars" irrelevant people need to find something else to argue about. http://encyclopedia2.thefreedictionary.com/1+true+brace+style > PS: Haskell seems better at the former than Python; Haskell tells you > if you import two identical symbols from two different places, when > you try to use one of them - not at import time. I believe in Python, > whichever symbol you import last, wins. Haskell does not warn you at > import time, which is fine. Not sure about OCaml or whatever else. Haskell assumes that importing the same style from two places is an error that needs warning about. I don't think it is, any more than: s = 'foo' s = 'bar' is necessarily an error (particular if the two rebindings are separated by code that does things). Both `import` and `from ... import` are rebinding operations, like = assignment. -- Steve From best_lay at yahoo.com Thu Feb 25 00:13:53 2016 From: best_lay at yahoo.com (Wildman) Date: Wed, 24 Feb 2016 23:13:53 -0600 Subject: [Newbie] Tkinter Question References: Message-ID: On Tue, 23 Feb 2016 16:19:43 -0600, Wildman wrote: > Thanks to Christian and Chris. You both gave me much to think about and to experiment with. That adds to my on-going learning experience. This is the first thing I tried: The Exit button has this: command=self.quit Then I have this: def clean_up(): # delete temp files root = Tk() app = Window(root) root.protocol("DELETE_WINDOW", clean_up) root.mainloop() And I also tried: root.wm_protocol("WM_DELETE_WINDOW", clean_up) Both worked when the window [X] button was clicked but neither worked when the Exit button was clicked. However, when I removed the protocol statement and replaced with this, it worked for both: import atexit atexit.register(clean_up) Nothing I tried would fire the clean_up routine when the program was force killed. In addition I should mention that I can't us the OS to delete the temp files because a shelled process creates them. It may be possible to still do it but my limited experience does not allow me to pursue it. Anyway, I am happy with the outcome even though I have not found a way to detect when the program is force killed. It is unlikely that would ever occur as long as the program does not lock up. It is up to me to make sure that doesn't happen. :-) Thanks again. -- GNU/Linux user #557453 "Philosophy is common sense with big words." -James Madison From steve+comp.lang.python at pearwood.info Thu Feb 25 00:38:19 2016 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Thu, 25 Feb 2016 16:38:19 +1100 Subject: Make a unique filesystem path, without creating the file References: <85r3gf55k4.fsf@benfinney.id.au> <85mvr26dij.fsf@benfinney.id.au> <87vb5gmtgj.fsf@elektro.pacujo.net> <56cba8c7$0$1611$c3e8da3$5496439d@news.astraweb.com> <877fhvnch0.fsf@elektro.pacujo.net> <56cd09f9$0$1620$c3e8da3$5496439d@news.astraweb.com> <87a8mqlgkz.fsf@elektro.pacujo.net> Message-ID: <56ce9350$0$1599$c3e8da3$5496439d@news.astraweb.com> On Wednesday 24 February 2016 18:20, Marko Rauhamaa wrote: > Steven D'Aprano : > >> On Tue, 23 Feb 2016 05:54 pm, Marko Rauhamaa wrote: >>> However, when you are generating signing or encryption keys, you >>> should use /dev/random. >> >> And that is where you repeat something which is rank superstition. > > Can you find info to back that up. The links already provided go through the evidence. For example, they explain that /dev/random and /dev/urandom both use the exact same CSPRNG. If you don't believe that, you can actually read the source to Linux, FreeBSD, OpenBSD and NetBSD. (But not OS X, sorry.) Put aside the known bug in Linux, where urandom will provide predictable values in the period following a fresh install before the OS has collected enough entropy. We all agree that's a problem, and that the right solution is to block. The question is, outside of that narrow set of circumstances, when it is appropriate to block? As I mentioned, most Unixes don't block. urandom and random behave exactly the same way in three of the most popular Unixes (FreeBSD, OpenBSD, OS X): https://en.wikipedia.org/wiki//dev/random so let's consider just those that do block (Linux, and NetBSD). They both use the same CSPRNG. Do you dispute that? Then read the source. For one to be "better" than the other, there would need to be a detectable difference between the two. Nobody has ever found one, and nor will they, because they're both coming from the same CSPRNG (AES in the case of NetBSD, I'm not sure what in the case of Linux). If you don't trust the CSPRNG, then you shouldn't trust it whether it comes from /dev/random or /dev/urandom. If you do trust it, then why would you want it to block? Blocking doesn't make it more random. That's not how it works. It just makes you vulnerable to a Denial Of Service attack. There really doesn't seem to be any valid reason for random blocking. It's like warnings and timers on fans in South Korea to prevent fan death: http://www.snopes.com/medical/freakish/fandeath.asp (My favourite explanation is that the blades of the fan chop the oxygen molecules in two.) I'm not surprised that there is so much misinformation about random/urandom. Here's a blog post by somebody wanting to port arc4 to Linux, so he clearly knows a few things about crypto. I can't judge whether arc4 is better or worse than what Linux already uses, but look at this quote: http://insanecoding.blogspot.com.au/2014/05/a-good-idea-with-bad-usage- devurandom.html Quote: Linux is well known for inventing and supplying two default files, /dev/random and /dev/urandom (unlimited random). The former is pretty much raw entropy, while the latter is the output of a CSPRNG function like OpenBSD's arc4random family. The former can be seen as more random, and the latter as less random, but the differences are extremely hard to measure, which is why CSPRNGs work in the first place. Since the former is only entropy, it is limited as to how much it can output, and one needing a lot of random data can be stuck waiting a while for it to fill up the random buffer. Since the latter is a CSPRNG, it can keep outputting data indefinitely, without any significant waiting periods." There's that myth about urandom being "less random" than random again, but even this guy admits that the difference is "extremely hard" (actually: impossible) to measure, and that CSPRNG's "work". Which is precisely why OpenBSD uses arc4random for their /dev/random and /dev/urandom, and presumably why he wants to bring it to Linux. This author is *completely wrong* to say that /dev/random is "pretty much raw entropy". If it were, it would be biased, and easily manipulated by an attacker. Entropy is collected from (among other things) network traffic, which would allow an attacker to control at least one source of entropy and hence (in theory) make it easier to predict the output of /dev/random. But fortunately it is not true. Linux's random system works like this: - entropy is collected from various sources and fed into a pool; - entropy from that pool is fed through a CSPRNG into two separate pools, one each for /dev/random and /dev/urandom; - when you read from /dev/random or urandom, they both collect entropy from their own pool, and again pass it through a CSPRNG; - /dev/random has a throttle (it blocks if you take out too much); - /dev/urandom doesn't have a throttle. https://events.linuxfoundation.org/images/stories/pdf/lceu2012_anvin.pdf Somebody criticized the author for spreading this misapprehension that /dev/random is "raw entropy" and here is his response: I tried giving an explanation which should be simple for a layman to follow of what goes on. I wouldn't take it as precise fact, especially when there's a washing machine involved in the explanation ;) Or, in other words, "When I said the moon was made of green cheese, I was simplifying it for the layman. I didn't mean it was precisely green cheese. It's just like green cheese." Sure. But it is *more like* rock and dust than green cheese. In fact, apart from both being made of atoms, it's hard to see any similarity between the moon and green cheese at all, just as there is no real similarity between what /dev/random actually does, and what this guy says it does. The bottom line is, nobody can distinguish the output of urandom and random (apart from the blocking behaviour). Nobody has demonstrated any way to distinguish the output of either random or urandom from "actual randomness". There are theoretical attacks on urandom that random might be immune to, but if so, I haven't heard what they are. Apart from pointing you at the source code, I don't know what else I can say to prove this. I can't prove that there's no known attack on Linux's CSPRNG, just as I can't prove that there's no tea cup in orbit around Jupiter, or that there wasn't a nuclear war between China and the USSR in 1985 and both countries covered it up. Maybe there was. But I wouldn't bet on it. > All I've seen so far is forceful > claims that's superstition ("These are not the droids you're looking > for"). > > Even the ssh-keygen man page has: > > The reseeding of the OpenSSL random generator is usually done from > /dev/urandom. If the SSH_USE_STRONG_RNG environment vari? able is > set to value other than 0 the OpenSSL random generator is reseeded > from /dev/random. If they had called the environment variable SSH_MAKE_MONEY_FAST would you believe it? What evidence do they give that /dev/urandom is weak? If it is weak, why are they using it as the default? -- Steve From steve+comp.lang.python at pearwood.info Thu Feb 25 01:50:51 2016 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Thu, 25 Feb 2016 17:50:51 +1100 Subject: How to read from a file to an arbitrary delimiter efficiently? Message-ID: <56cea44e$0$11128$c3e8da3@news.astraweb.com> I have a need to read to an arbitrary delimiter, which might be any of a (small) set of characters. For the sake of the exercise, lets say it is either ! or ? (for example). I want to read from files reasonably efficiently. I don't mind if there is a little overhead, but my first attempt is 100 times slower than the built-in "read to the end of the line" method. Here is the function I came up with: # Read a chunk of bytes/characters from an open file. def chunkiter(f, delim): buffer = [] b = f.read(1) while b: buffer.append(b) if b in delim: yield ''.join(buffer) buffer = [] b = f.read(1) if buffer: yield ''.join(buffer) And here is some test code showing how slow it is: # Create a test file. FILENAME = '/tmp/foo' s = """\ abcdefghijklmnopqrstuvwxyz! abcdefghijklmnopqrstuvwxyz? """ * 500 with open(FILENAME, 'w') as f: f.write(s) # Run some timing tests, comparing to reading lines from a file. def readlines(f): f.seek(0) for line in f: pass def readchunks(f): f.seek(0) for chunk in chunkiter(f, '!?'): pass from timeit import Timer SETUP = 'from __main__ import readlines, readchunks, FILENAME; ' SETUP += 'open(FILENAME)' t1 = Timer('readlines(f)', SETUP) t2 = Timer('readchunks(f)', SETUP) # Time them. x = t1.repeat(number=10) # Ignore the first run, in case of caching issues. x = min(t1.repeat(number=1000, repeat=9)) y = t2.repeat(number=10) y = min(t2.repeat(number=1000, repeat=9)) print('reading lines:', x, 'reading chunks:', y) On my laptop, the results I get are: reading lines: 0.22584209218621254 reading chunks: 21.716224210336804 Is there a better way to read chunks from a file up to one of a set of arbitrary delimiters? Bonus for it working equally well with text and bytes. (You can assume that the delimiters will be no more than one byte, or character, each. E.g. "!" or "?", but never "!?" or "?!".) -- Steve From marko at pacujo.net Thu Feb 25 01:54:55 2016 From: marko at pacujo.net (Marko Rauhamaa) Date: Thu, 25 Feb 2016 08:54:55 +0200 Subject: Make a unique filesystem path, without creating the file References: <85r3gf55k4.fsf@benfinney.id.au> <85mvr26dij.fsf@benfinney.id.au> <87vb5gmtgj.fsf@elektro.pacujo.net> <56cba8c7$0$1611$c3e8da3$5496439d@news.astraweb.com> <877fhvnch0.fsf@elektro.pacujo.net> <56cd09f9$0$1620$c3e8da3$5496439d@news.astraweb.com> <87a8mqlgkz.fsf@elektro.pacujo.net> <56ce9350$0$1599$c3e8da3$5496439d@news.astraweb.com> Message-ID: <87a8mpjn40.fsf@elektro.pacujo.net> Steven D'Aprano : > On Wednesday 24 February 2016 18:20, Marko Rauhamaa wrote: >> Steven D'Aprano : >>> And that is where you repeat something which is rank superstition. >> >> Can you find info to back that up. > > The links already provided go through the evidence. For example, they > explain that /dev/random and /dev/urandom both use the exact same > CSPRNG. A non-issue. The question is, after the initial entropy is collected and used to seed the CSPRNG, is any further entropy needed for any cryptographic purposes? Are there any nagging fears that weaknesses could be found in the deterministic sequence? /dev/random is supposed to be hardened against such concerns by stirring the pot constantly (if rather slowly). Here's what Linus Torvalds said on the matter years back: > No, it says /dev/random is primarily useful for generating large > (>>160 bit) keys. Which is exactly what something like sshd would want to use for generating keys for the machine, right? That is _the_ primary reason to use /dev/random. Yet apparently our /dev/random has been too conservative to be actually useful, because (as you point out somewhere else) even sshd uses /dev/urandom for the host key generation by default. That is really sad. That is the _one_ application that is common and that should really have a reason to maybe care about /dev/random vs urandom. And that application uses urandom. To me that says that /dev/random has turned out to be less than useful in real life. Is there anything that actually uses /dev/random at all (except for clueless programs that really don't need to)? > If you don't trust the CSPRNG, then you shouldn't trust it whether it comes > from /dev/random or /dev/urandom. If you do trust it, then why would you > want it to block? Blocking doesn't make it more random. It might not make it more secure cryptographically, but the point is that it should make it more genuinely random. > That's not how it works. It just makes you vulnerable to a Denial Of > Service attack. Understood. You should not use /dev/random for any reactive purposes (like nonces or session encryption keys). > There's that myth about urandom being "less random" than random again, > but even this guy admits that the difference is "extremely hard" > (actually: impossible) to measure, and that CSPRNG's "work". Which is > precisely why OpenBSD uses arc4random for their /dev/random and > /dev/urandom, and presumably why he wants to bring it to Linux. That's for the cryptographic experts to judge. CSPRNG's aren't always as CS as one would think: In December 2013, a Reuters news article alleged that in 2004, before NIST standardized Dual_EC_DRBG, NSA paid RSA Security $10 million in a secret deal to use Dual_EC_DRBG as the default in the RSA BSAFE cryptography library, which resulted in RSA Security becoming the most important distributor of the insecure algorithm. > The bottom line is, nobody can distinguish the output of urandom and > random (apart from the blocking behaviour). Nobody has demonstrated > any way to distinguish the output of either random or urandom from > "actual randomness". There are theoretical attacks on urandom that > random might be immune to, but if so, I haven't heard what they are. What I'm looking for is a cryptography mailing list (or equivalent) giving their stamp of approval. As can be seen above, NIST ain't it. It seems, though, that cryptography researchers are not ready to declare any scheme void of vulnerabilities. At best they can mention that there are no *known* vulnerabilities. > What evidence do they give that /dev/urandom is weak? If it is weak, > why are they using it as the default? It's a big mess, but not a mess I would disentangle. Once the crypto libraries, utilities, facilities and the OS come to a consensus, I can hope they've done their homework. As it stands, the STRONG vs VERY STRONG dichotomy seems to be alive all over the place. Marko From rosuav at gmail.com Thu Feb 25 02:30:25 2016 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 25 Feb 2016 18:30:25 +1100 Subject: How to read from a file to an arbitrary delimiter efficiently? In-Reply-To: <56cea44e$0$11128$c3e8da3@news.astraweb.com> References: <56cea44e$0$11128$c3e8da3@news.astraweb.com> Message-ID: On Thu, Feb 25, 2016 at 5:50 PM, Steven D'Aprano wrote: > > # Read a chunk of bytes/characters from an open file. > def chunkiter(f, delim): > buffer = [] > b = f.read(1) > while b: > buffer.append(b) > if b in delim: > yield ''.join(buffer) > buffer = [] > b = f.read(1) > if buffer: > yield ''.join(buffer) How bad is it if you over-read? If it's absolutely critical that you not read anything from the buffer that you shouldn't, then yeah, it's going to be slow. But if you're never going to read the file using anything other than this iterator, the best thing to do is to read more at a time. Simple and naive method: def chunkiter(f, delim): """Don't use [ or ] as the delimiter, kthx""" buffer = "" b = f.read(256) while b: buffer += b *parts, buffer = re.split("["+delim+"]", buffer) yield from parts if buffer: yield buffer How well does that perform? ChrisA From wolfgang.maier at biologie.uni-freiburg.de Thu Feb 25 02:37:14 2016 From: wolfgang.maier at biologie.uni-freiburg.de (Wolfgang Maier) Date: Thu, 25 Feb 2016 08:37:14 +0100 Subject: How to read from a file to an arbitrary delimiter efficiently? In-Reply-To: <56cea44e$0$11128$c3e8da3@news.astraweb.com> References: <56cea44e$0$11128$c3e8da3@news.astraweb.com> Message-ID: On 25.02.2016 07:50, Steven D'Aprano wrote: > I have a need to read to an arbitrary delimiter, which might be any of a > (small) set of characters. For the sake of the exercise, lets say it is > either ! or ? (for example). > You are not alone with your need. http://bugs.python.org/issue1152248 discusses the problem and has some code snippets that you may be interested in. While there is no trivial solution there are certainly faster ways than your first attempt. Wolfgang From no.email at nospam.invalid Thu Feb 25 02:48:59 2016 From: no.email at nospam.invalid (Paul Rubin) Date: Wed, 24 Feb 2016 23:48:59 -0800 Subject: How to read from a file to an arbitrary delimiter efficiently? References: <56cea44e$0$11128$c3e8da3@news.astraweb.com> Message-ID: <871t81w7pw.fsf@jester.gateway.pace.com> Steven D'Aprano writes: > while b: > buffer.append(b) This looks bad because of the overhead of list elements, and also the reading of 1 char at a time. If it's bytes that you're reading, try using bytearray instead of list: def chunkiter(f,delim): buf = bytearray() bufappend = buf.append # avoid an attribute lookup when calling fread = f.read # similar while True: c = fread(1) bufappend(c) if c in delim: yield str(buf) del buf[:] If that's still not fast enough, you could do a more hacky thing of reading large chunks of input at once (f.read(4096) or whatever), splitting on the delimiter set with re.split, and yielding the split output, refilling the buffer when you don't find more delimiters. That doesn't tell you what delimiters actually match: do you need that? Maybe there is nicer a way to get at it than adding up the lengths of the chunks to index into the buffer. How large do you expect the chunks to be? From gandalf at shopzeus.com Thu Feb 25 03:08:15 2016 From: gandalf at shopzeus.com (=?UTF-8?Q?Nagy_L=c3=a1szl=c3=b3_Zsolt?=) Date: Thu, 25 Feb 2016 09:08:15 +0100 Subject: asyncmongo + python 3: No module named 'errors' Message-ID: <56CEB66F.30308@shopzeus.com> ayncmongo requirements are "pymongo" and "tornado". I have a fresh installation of Python 3.5, pymongo 3.2 and tornado 4.3, but I cannot import asyncmongo. Tracelog below. P:\WinPython-64bit-3.5.1.2\python-3.5.1.amd64>python Python 3.5.1 (v3.5.1:37a07cee5969, Dec 6 2015, 01:54:25) [MSC v.1900 64 bit (AMD64)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> import pymongo >>> pymongo.version '3.2' >>> import tornado >>> tornado.version '4.3' >>> import asyncmongo Traceback (most recent call last): File "", line 1, in File "P:\WinPython-64bit-3.5.1.2\python-3.5.1.amd64\lib\site-packages\asyncmongo\__init__.py", line 37, in from errors import (Error, InterfaceError, AuthenticationError, DatabaseError, RSConnectionError, ImportError: No module named 'errors' >>> From gandalf at shopzeus.com Thu Feb 25 03:10:43 2016 From: gandalf at shopzeus.com (=?UTF-8?Q?Nagy_L=c3=a1szl=c3=b3_Zsolt?=) Date: Thu, 25 Feb 2016 09:10:43 +0100 Subject: asyncmongo + python 3: No module named 'errors' In-Reply-To: <56CEB66F.30308@shopzeus.com> References: <56CEB66F.30308@shopzeus.com> Message-ID: <56CEB703.9020104@shopzeus.com> 2016.02.25. 9:08 keltez?ssel, Nagy L?szl? Zsolt ?rta: > ayncmongo requirements are "pymongo" and "tornado". > > I have a fresh installation of Python 3.5, pymongo 3.2 and tornado 4.3, > but I cannot import asyncmongo. This seems to be the exact same bug report: https://github.com/bitly/asyncmongo/issues/51 Last comment in 2013. There was no response. Does it mean that asyncmongo does not work with Python 3? From songofacandy at gmail.com Thu Feb 25 03:16:01 2016 From: songofacandy at gmail.com (INADA Naoki) Date: Thu, 25 Feb 2016 17:16:01 +0900 Subject: asyncmongo + python 3: No module named 'errors' In-Reply-To: <56CEB703.9020104@shopzeus.com> References: <56CEB66F.30308@shopzeus.com> <56CEB703.9020104@shopzeus.com> Message-ID: > > > Does it mean that asyncmongo does not work with Python 3? > > Yes. I recommend you to use motor [1]. * [1] https://github.com/mongodb/motor -- INADA Naoki From steve+comp.lang.python at pearwood.info Thu Feb 25 03:21:15 2016 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Thu, 25 Feb 2016 19:21:15 +1100 Subject: Make a unique filesystem path, without creating the file References: <85r3gf55k4.fsf@benfinney.id.au> <85mvr26dij.fsf@benfinney.id.au> <87vb5gmtgj.fsf@elektro.pacujo.net> <56cba8c7$0$1611$c3e8da3$5496439d@news.astraweb.com> <877fhvnch0.fsf@elektro.pacujo.net> <56cd09f9$0$1620$c3e8da3$5496439d@news.astraweb.com> <87a8mqlgkz.fsf@elektro.pacujo.net> <56ce9350$0$1599$c3e8da3$5496439d@news.astraweb.com> <87a8mpjn40.fsf@elektro.pacujo.net> Message-ID: <56ceb97e$0$2924$c3e8da3$76491128@news.astraweb.com> On Thursday 25 February 2016 17:54, Marko Rauhamaa wrote: > Steven D'Aprano : > >> On Wednesday 24 February 2016 18:20, Marko Rauhamaa wrote: >>> Steven D'Aprano : >>>> And that is where you repeat something which is rank superstition. >>> >>> Can you find info to back that up. >> >> The links already provided go through the evidence. For example, they >> explain that /dev/random and /dev/urandom both use the exact same >> CSPRNG. > > A non-issue. The question is, after the initial entropy is collected and > used to seed the CSPRNG, is any further entropy needed for any > cryptographic purposes? The short answer: "yes". The long answer: "probably not, but it can't hurt". The longer answer: "probably not, and it usually won't hurt, but it could". If, somehow, an attacker manages to work out the state of your CSPRNG, including the entropy pool, then they can predict what values you get until they no longer know the state of the CSPRNG. The idea is that if, somehow, somebody knows the current state of the CSPRNG (including the entropy pool), but can't influence what future values go into the entropy pool, then they will only be able to predict the output values for a short time. But it's hard to think of any actual attack where somebody can see what's in the entropy pool but can't influence the values going into it. It seems to me that this is an unrealistic attack: "Assume that you're kidnapped by somebody with no arms or legs..." The conventional wisdom is that adding poor sources of entropy into the pool will never hurt, but that is actually wrong. If an attacker knows what is in the entropy pool, and can craft the values going in, they can force the CSPRNG to return more predictable values. So sometimes adding more entropy can hurt. And it usually won't help. It *might* help if your system is compromised, but if so, it's not really clear how the attacker has compromised your current entropy pool but not the future ones. > Are there any nagging fears that weaknesses > could be found in the deterministic sequence? Of course there are. Nobody really knows what capabilities the NSA have, but they almost surely aren't *that* advanced. CSPRNGs are subject to much the same sort of issues as other crypto, such as hash functions: http://valerieaurora.org/hash.html and encryption algorithms. (The main real difference between a hash function and encryption algorithm is that hashes don't have to be reversible.) Expect the current crop of CSPRNGs (Yarrow, AES, whatever Linux uses) to be replaced long before there is a proven attack on them. > /dev/random is supposed to be hardened against such concerns by stirring > the pot constantly (if rather slowly). As is /dev/urandom. > Here's what Linus Torvalds said on the matter years back: > > > No, it says /dev/random is primarily useful for generating large > > (>>160 bit) keys. > > Which is exactly what something like sshd would want to use for > generating keys for the machine, right? That is _the_ primary reason > to use /dev/random. > > Yet apparently our /dev/random has been too conservative to be > actually useful, because (as you point out somewhere else) even sshd > uses /dev/urandom for the host key generation by default. > > That is really sad. That is the _one_ application that is common and > that should really have a reason to maybe care about /dev/random vs > urandom. And that application uses urandom. To me that says that > /dev/random has turned out to be less than useful in real life. > > Is there anything that actually uses /dev/random at all (except for > clueless programs that really don't need to)? Most other Unixes have decided that /dev/random is unnecessary, and urandom is the right thing to do. SSH uses urandom by default, but allows the paranoid/clueless to use /dev/random if they insist. -- Steve From gandalf at shopzeus.com Thu Feb 25 03:46:00 2016 From: gandalf at shopzeus.com (=?UTF-8?Q?Nagy_L=c3=a1szl=c3=b3_Zsolt?=) Date: Thu, 25 Feb 2016 09:46:00 +0100 Subject: asyncmongo + python 3: No module named 'errors' In-Reply-To: References: <56CEB66F.30308@shopzeus.com> <56CEB703.9020104@shopzeus.com> Message-ID: <56CEBF48.2060508@shopzeus.com> > Yes. > I recommend you to use motor [1]. > > > * [1] https://github.com/mongodb/motor Thanks. I have checked motor, and here are some problems with it: * I cannot install it with "pip3 install motor", because it is trying to downgrade (!!!) pymongo to version 2.8, but it fails to do so. There might be a binary version available somewhere. Is there a reason why it works with pymongo 2.8 only? * The API works with callbacks. I would like to use an API that has await-able method. I'm at early stage of development, so I can migrate to a different nosql database. Is there a good nosql alternative? My requirements are: * scalable and fault tolerant - data can be distributed between multiple nodes * can be run as a portable application * tornado compatible Python API with awaitable methods instead of callbacks Facts:* *- pymongo knows all, except that it is not async - ayncmongo does not work with python 3 - motor requires downgraded pymongo, and it has no awaitable methods I have found codernitydb https://pypi.python.org/pypi/CodernityDB/ which seems to be able to do sharding, but I'm not sure if it can be setup as a fault tolerant server. Thanks, Laszlo From nomail at invalid.com Thu Feb 25 04:54:08 2016 From: nomail at invalid.com (ast) Date: Thu, 25 Feb 2016 10:54:08 +0100 Subject: How to define what a class is ? In-Reply-To: References: <56cd64fb$0$9220$426a74cc@news.free.fr> Message-ID: <56cecf47$0$3305$426a74cc@news.free.fr> "Ian Kelly" a ?crit dans le message de news:mailman.85.1456303651.20994.python-list at python.org... > On Wed, Feb 24, 2016 at 1:08 AM, ast wrote: > All metaclasses are subclasses of type, so all classes are instances of type. Ah ! I didn't know that if an object Obj is an instance of Myclass and MyClass inherit from MyClass2, then Ojb is an instance of MyClass2 too >>> from enum import Enum, EnumMeta >>> >>> isinstance(Enum, EnumMeta) True >>> isinstance(EnumMeta, type) True >>> isinstance(Enum, type) True that's correct >> Suppose I provide to you an object and that I >> ask to you to tell me if it is a class or not. How >> would you proceed ? > > import inspect > inspect.isclass(x) So we can conclude that inspect.isclass(x) is equivalent to isinstance(x, type) lets have a look at the source code of isclass: def isclass(object): """Return true if the object is a class. Class objects provide these attributes: __doc__ documentation string __module__ name of module in which this class was defined""" return isinstance(object, type) correct From jon+usenet at unequivocal.co.uk Thu Feb 25 05:05:25 2016 From: jon+usenet at unequivocal.co.uk (Jon Ribbens) Date: Thu, 25 Feb 2016 10:05:25 -0000 (UTC) Subject: Make a unique filesystem path, without creating the file References: <85r3gf55k4.fsf@benfinney.id.au> <85mvr26dij.fsf@benfinney.id.au> <87vb5gmtgj.fsf@elektro.pacujo.net> <56cba8c7$0$1611$c3e8da3$5496439d@news.astraweb.com> <877fhvnch0.fsf@elektro.pacujo.net> <56cd09f9$0$1620$c3e8da3$5496439d@news.astraweb.com> <87a8mqlgkz.fsf@elektro.pacujo.net> <56ce9350$0$1599$c3e8da3$5496439d@news.astraweb.com> Message-ID: On 2016-02-25, Steven D'Aprano wrote: > The links already provided go through the evidence. For example, they > explain that /dev/random and /dev/urandom both use the exact same CSPRNG. If > you don't believe that, you can actually read the source to Linux, FreeBSD, > OpenBSD and NetBSD. (But not OS X, sorry.) Actually yes OS X: http://www.opensource.apple.com/source/xnu/xnu-3248.20.55/bsd/dev/random/ http://www.opensource.apple.com/source/xnu/xnu-3248.20.55/osfmk/prng/ From eryksun at gmail.com Thu Feb 25 06:21:43 2016 From: eryksun at gmail.com (eryk sun) Date: Thu, 25 Feb 2016 05:21:43 -0600 Subject: How to define what a class is ? In-Reply-To: <56cecf47$0$3305$426a74cc@news.free.fr> References: <56cd64fb$0$9220$426a74cc@news.free.fr> <56cecf47$0$3305$426a74cc@news.free.fr> Message-ID: On Thu, Feb 25, 2016 at 3:54 AM, ast wrote: > So we can conclude that inspect.isclass(x) is equivalent > to isinstance(x, type) > > lets have a look at the source code of isclass: > > def isclass(object): > """Return true if the object is a class. > > Class objects provide these attributes: > __doc__ documentation string > __module__ name of module in which this class was defined""" > return isinstance(object, type) Except Python 2 old-style classes (i.e. 2.x classes that aren't a subclass of `object`) are not instances of `type`. Prior to new-style classes, only built-in types were instances of `type`. An old-style class is an instance of "classobj", and its instances have the "instance" type. >>> class A: pass ... >>> type(A) >>> type(A()) Note that "classobj" and "instance" are instances of `type`. The `isclass` check in Python 2 has to instead check isinstance(object, (type, types.ClassType)). >>> types.ClassType From breamoreboy at yahoo.co.uk Thu Feb 25 08:13:56 2016 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Thu, 25 Feb 2016 13:13:56 +0000 Subject: Computational Chemistry Analysis In-Reply-To: References: Message-ID: On 25/02/2016 01:01, Feagans, Mandy wrote: > Dear Python, > > Hi! I am a student interested in conducting computational analysis of protein-ligand binding for drug development analysis. Recently, I read of an individual using a python program for their studies of protein-ligand binding. As I have been reading about Python programs, however, all I have been able to find are programs that use the python script (i.e. Autodock). I was hoping to see if there were any programs specifically run through the python programming that ran similar analysis to Autodock? > > Thank you for any assistance you may be able to provide! > > Thank you, > > Mandy Feagans > I'll guess that you start here http://biopython.org/wiki/Main_Page -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From a24061 at ducksburg.com Thu Feb 25 08:25:27 2016 From: a24061 at ducksburg.com (Adam Funk) Date: Thu, 25 Feb 2016 13:25:27 +0000 Subject: Testing whether the VPN is running? References: <8ghgpcxrd3.ln2@news.ducksburg.com> Message-ID: <7vb3qcxocs.ln2@news.ducksburg.com> On 2016-02-23, Cameron Simpson wrote: > On 18Feb2016 10:03, Adam Funk wrote: >>On 2016-02-18, Ervin Heged?s wrote: >>> I think that the psutil modul could be better for you for this >>> task: >>> https://pypi.python.org/pypi/psutil/ >>> >>> and see the "Network" section. >> >>if 'tun0' in psutil.net_if_addrs(): >> # vpn is running >> >>Brilliant! I've used psutil for something else, but I didn't know it >>did that too. My excuse is that the version on my system was 2.2.1, >>which does not do that, so I installed the newer version with pip3. >>Thanks for pointing me to that. > > You might also want to check that the interface is up. > > My personal hack (not for a VPN, but for "being online", which turns my ssh > tunnels on and off) is to look in the output of "netstat -rn" for a default > route. This may imply that an alternative test for you is to test for a route > to your VPN's address range? Just an idea. Also interesting to know, thanks. -- Mrs CJ and I avoid clich?s like the plague. From sapna at aries.res.in Thu Feb 25 09:37:08 2016 From: sapna at aries.res.in (Sapna Mishra) Date: Thu, 25 Feb 2016 20:07:08 +0530 (IST) Subject: Pyraf problem In-Reply-To: <705629636.351550.1456410947390.JavaMail.zimbra@aries.res.in> References: <705629636.351550.1456410947390.JavaMail.zimbra@aries.res.in> Message-ID: <2042039661.351558.1456411028503.JavaMail.zimbra@aries.res.in> Dear Sir/Mam, I am using python for my astronomy purpose, for that I want to use PyRaf, but strange thing is occurring that pyraf is getting open as a root but out side my root user directory when I am typing pyraf I am getting like; No graphics/display possible for this session. Tkinter import failed. The numpy package is required by PyRAF and was not found. Please visit http://numpy.scipy.org I have added bath of pyraf in my bashrc file , I have even downloaded numpy but again it is showing same error. Any help will be highly appreciated. Thanks and regards, ***************************************************************************************** Sapna Mishra JRF Aryabhatta Research Institute of Observational Sciences (ARIES) Manora Peak Nainital-263002 Uttarakhand (India) email: sapna.intell at gmail.com sapna at aries.res.in From random832 at fastmail.com Thu Feb 25 10:03:14 2016 From: random832 at fastmail.com (Random832) Date: Thu, 25 Feb 2016 10:03:14 -0500 Subject: Make a unique filesystem path, without creating the file In-Reply-To: <87h9gzx2co.fsf@jester.gateway.pace.com> References: <85r3gf55k4.fsf@benfinney.id.au> <85mvr26dij.fsf@benfinney.id.au> <56cba1bc$0$1604$c3e8da3$5496439d@news.astraweb.com> <87si0k2mal.fsf@jester.gateway.pace.com> <87h9gzx2co.fsf@jester.gateway.pace.com> Message-ID: <1456412594.2565424.531726610.4B08CA66@webmail.messagingengine.com> On Tue, Feb 23, 2016, at 03:22, Paul Rubin wrote: > Thanks. It would be nice if those were gatewayed to usenet like this > group is. I can't bring myself to subscribe to mailing lists. Have you tried gmane? From sruthi223 at gmail.com Thu Feb 25 10:12:08 2016 From: sruthi223 at gmail.com (sruthi223 at gmail.com) Date: Thu, 25 Feb 2016 07:12:08 -0800 (PST) Subject: child.before taking almost 1 minute to execute In-Reply-To: References: Message-ID: <720905f6-dfd2-4580-8cd2-abb41a27343a@googlegroups.com> On Wednesday, February 24, 2016 at 3:59:01 PM UTC-5, Emile van Sebille wrote: > On 2/24/2016 7:42 AM, pyfreek wrote: > > The following snippet alone is taking 1 minute to execute. is there any best way to find 'No such file' other than using child.before > > > > if not scrutinFile.startswith('/') : > > scrutinFile = '/'+ scrutinFile > > scrutinFileFtp = directory + scrutinFile > > filePath, file = os.path.split(scrutinFileFtp) > > p.sendline('cd %s'%(filePath)) > > p.expect([pexpect.EOF,pexpect.TIMEOUT]) > > if 'No such file' in p.before: > > print "No such directory exists!!" > > sys.exit(1) > > I'd guess that you've got your p.expect line wrong -- it looks to me > like you're allowing that line to complete only upon seeing an EOF or > TIMEOUT, and that it's timing out as a result. > > Emile HI EMile, Thanks for the reply. If I exit based only on EOF, then does it work? From sruthi223 at gmail.com Thu Feb 25 10:12:55 2016 From: sruthi223 at gmail.com (sruthi223 at gmail.com) Date: Thu, 25 Feb 2016 07:12:55 -0800 (PST) Subject: child.before taking almost 1 minute to execute In-Reply-To: References: Message-ID: <6df04be8-136f-4558-8e7e-874f7717f4b6@googlegroups.com> On Wednesday, February 24, 2016 at 4:10:13 PM UTC-5, Gregory Ewing wrote: > pyfreek wrote: > > The following snippet alone is taking 1 minute to execute. is there any best way to find 'No such file' other than using child.before > > > > if not scrutinFile.startswith('/') : > > scrutinFile = '/'+ scrutinFile > > scrutinFileFtp = directory + scrutinFile > > filePath, file = os.path.split(scrutinFileFtp) > > p.sendline('cd %s'%(filePath)) > > p.expect([pexpect.EOF,pexpect.TIMEOUT]) > > if 'No such file' in p.before: > > print "No such directory exists!!" > > sys.exit(1) > > If you're talking to an ftp client here, you might like > to consider using the ftplib module in the standard > library. It ought to take care of the messy details > of error detection for you. > > -- > Greg Thanks greg, but my organization is using pexpect as a part of its libraries. so I am trying to use the same. From oscar.j.benjamin at gmail.com Thu Feb 25 10:31:54 2016 From: oscar.j.benjamin at gmail.com (Oscar Benjamin) Date: Thu, 25 Feb 2016 15:31:54 +0000 Subject: Computational Chemistry Analysis In-Reply-To: References: Message-ID: On 25 February 2016 at 01:01, Feagans, Mandy wrote: > Hi! I am a student interested in conducting computational analysis of protein-ligand binding for drug development analysis. Recently, I read of an individual using a python program for their studies of protein-ligand binding. As I have been reading about Python programs, however, all I have been able to find are programs that use the python script (i.e. Autodock). I was hoping to see if there were any programs specifically run through the python programming that ran similar analysis to Autodock? You may have better luck asking on the scipy user list: http://www.scipy.org/scipylib/mailing-lists.html Those lists are specifically for people doing scientific work with Python. -- Oscar From steve at pearwood.info Thu Feb 25 11:14:15 2016 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 26 Feb 2016 03:14:15 +1100 Subject: Pyraf problem References: <705629636.351550.1456410947390.JavaMail.zimbra@aries.res.in> Message-ID: <56cf2858$0$1607$c3e8da3$5496439d@news.astraweb.com> On Fri, 26 Feb 2016 01:37 am, Sapna Mishra wrote: > Dear Sir/Mam, > > I am using python for my astronomy purpose, for that I want to use PyRaf, > but strange thing is occurring that pyraf is getting open as a root but I am sorry, I don't understand what "getting open as a root" means. > out side my root user directory when I am typing pyraf I am getting like; > No graphics/display possible for this session. Tkinter import failed. The > numpy package is required by PyRAF and was not found. Please visit > http://numpy.scipy.org Where are you getting this? Is it a Python traceback? If so, please copy and paste the entire traceback, starting with the line: Traceback and ending with the last of the error message. Where are you typing pyraf? In the Python interpreter or the operating system shell? Is the prompt >>> or $ or something else? Are you using the default Python interactive interpreter, IDLE, or some third party IDE? Please explain how you launch Python, e.g.: "I double-click on the Python icon on the desktop" "I open a terminal, and enter python at the shell prompt" Or something else? What version of Python are you using? What operating system? > I have added bath of pyraf in my bashrc file , I have even downloaded > numpy but again it is showing same error. What have you added to your bashrc file? Did you add it to the PATH or the PYTHONPATH? What happens if you run this at the bash prompt? python -c "import sys; print(sys.path)" Do you see the path to pyraf in Python's path? (Python does not use the same path as bash.) -- Steven From drsalists at gmail.com Thu Feb 25 11:15:16 2016 From: drsalists at gmail.com (Dan Stromberg) Date: Thu, 25 Feb 2016 08:15:16 -0800 Subject: "from module import data; print(data)" vs "import module; print(module.data)" In-Reply-To: <56ce7762$0$11100$c3e8da3@news.astraweb.com> References: <56ce7762$0$11100$c3e8da3@news.astraweb.com> Message-ID: On Wed, Feb 24, 2016 at 7:39 PM, Steven D'Aprano wrote: > On Thursday 25 February 2016 12:07, Dan Stromberg wrote: > >> Could people please compare and contrast the two ways of doing imports >> in the Subject line? > > from module import data; print(data) > > import module; print(module.data) >> Is it fair to say that the former increases the surface area of your >> shared (sometimes mutable) state? > > No, I can't say that it is. If anything, the opposite is the case: it > decreases the surface area, because `data` now is local to the importing > (not imported) module. Rebinding data will not affect anything else. > There are two scenarios (`from module import...` versus `import ...`), each > with two cases: mutation, and rebinding: > > (1) from module import data > > (a) Mutation (only applies to mutable data like lists, dicts etc) > > In place mutation affects everything relying on module.data regardless of > how or where it is accessed. We have some scenarios where "data" is an instance of a class, and we need to monkey patch it for unit testing. My intuition is telling me that "module.data; data.mutate()" would be easier to monkey patch in a way that all modules will see. Is that fair to say? Thanks. From drsalists at gmail.com Thu Feb 25 11:20:59 2016 From: drsalists at gmail.com (Dan Stromberg) Date: Thu, 25 Feb 2016 08:20:59 -0800 Subject: "from module import data; print(data)" vs "import module; print(module.data)" In-Reply-To: References: <56ce7762$0$11100$c3e8da3@news.astraweb.com> Message-ID: On Thu, Feb 25, 2016 at 8:15 AM, Dan Stromberg wrote: > On Wed, Feb 24, 2016 at 7:39 PM, Steven D'Aprano > wrote: >> On Thursday 25 February 2016 12:07, Dan Stromberg wrote: >> >>> Could people please compare and contrast the two ways of doing imports >>> in the Subject line? >> >> from module import data; print(data) >> >> import module; print(module.data) > >>> Is it fair to say that the former increases the surface area of your >>> shared (sometimes mutable) state? >> >> No, I can't say that it is. If anything, the opposite is the case: it >> decreases the surface area, because `data` now is local to the importing >> (not imported) module. Rebinding data will not affect anything else. > >> There are two scenarios (`from module import...` versus `import ...`), each >> with two cases: mutation, and rebinding: >> >> (1) from module import data >> >> (a) Mutation (only applies to mutable data like lists, dicts etc) >> >> In place mutation affects everything relying on module.data regardless of >> how or where it is accessed. > > We have some scenarios where "data" is an instance of a class, and we > need to monkey patch it for unit testing. > > Please ignore: My intuition is telling me that "module.data; data.mutate()" would be > easier to monkey patch in a way that all modules will see. Is that > fair to say? > > Thanks. Correction! My intuition is telling me that "module.data; module.data.mutate()" would be easier to monkey patch in a way that all modules will see. Is that fair to say? From ethan at stoneleaf.us Thu Feb 25 11:35:52 2016 From: ethan at stoneleaf.us (Ethan Furman) Date: Thu, 25 Feb 2016 08:35:52 -0800 Subject: "from module import data; print(data)" vs "import module; print(module.data)" In-Reply-To: <56ce7762$0$11100$c3e8da3@news.astraweb.com> References: <56ce7762$0$11100$c3e8da3@news.astraweb.com> Message-ID: <56CF2D68.7020905@stoneleaf.us> On 02/24/2016 07:39 PM, Steven D'Aprano wrote: > (2) import module > > (a) Mutation is no different from (1)(a) above. No change. > > (b) Rebinding `module.data = []` affects the imported module, and therefore > everything that relies on it. More accurate: and therefore everything that has not already done a `from module import data` as those will be unaffected by the rebinding. -- ~Ethan~ From ethan at stoneleaf.us Thu Feb 25 11:51:39 2016 From: ethan at stoneleaf.us (Ethan Furman) Date: Thu, 25 Feb 2016 08:51:39 -0800 Subject: "from module import data; print(data)" vs "import module; print(module.data)" In-Reply-To: References: <56ce7762$0$11100$c3e8da3@news.astraweb.com> Message-ID: <56CF311B.0@stoneleaf.us> On 02/25/2016 08:20 AM, Dan Stromberg wrote: > My intuition is telling me that "module.data; module.data.mutate()" > would be easier to monkey patch in a way that all modules will see. > Is that fair to say? It is fair to say that if you need to monkey-patch a module, you should import the module. :) You should also import it before anything else does to make sure your change is visible to all other code. -- ~Ethan~ From alexamici at gmail.com Thu Feb 25 11:55:35 2016 From: alexamici at gmail.com (Alessandro Amici) Date: Thu, 25 Feb 2016 16:55:35 +0000 Subject: ANN: pytest-nodev v0.9.3 - Test-driven code search Message-ID: I am pleased to announce the release of the new beta release of pytest-nodev (was pytest-wish) a test-driven code search plugin for pytest: https://pytest-nodev.readthedocs.org/en/stable/quickstart.html Changes: - renamed the package to pytest-nodev from pytest-wish (sorry!) - refuse to run potentially dangerous `--wish-from-all` by default and... - ... document how to run inside a container to safely enable all the features - simpler command line usage - more documentation and references to the academic research on the test-driven code search subject Development effort is directed to fixing bugs and writing better documentation before the 1.0 release. Help is welcomed and appreciated: https://github.com/nodev-io/pytest-nodev User questions are best directed to http://stackoverflow.com/search?q=pytest-nodev and general questions can be sent to pytest-dev at python.org. Thanks, Alessandro From randy.day at sasktel.netx Thu Feb 25 12:13:53 2016 From: randy.day at sasktel.netx (Randy Day) Date: Thu, 25 Feb 2016 11:13:53 -0600 Subject: [Newbie] Tkinter Question References: Message-ID: In article , best_lay at yahoo.com says... [snip] > Anyway, I am happy with the outcome even though I have > not found a way to detect when the program is force > killed. It is unlikely that would ever occur as long When your peogram starts, have it create a small file. As part of your graceful shutdown, delete the file. Any subsequent processes can test for the existence of the file; if it exists, your program did not exit gracefully. --- news://freenews.netfront.net/ - complaints: news at netfront.net --- From rui.vapps at gmail.com Thu Feb 25 12:55:49 2016 From: rui.vapps at gmail.com (Ray) Date: Thu, 25 Feb 2016 09:55:49 -0800 (PST) Subject: tcp networking question (CLOSE_WAIT) Message-ID: <7ff15c67-102e-4b02-a2c8-87b24f1b8351@googlegroups.com> hi, I'm new to python networking. I am waiting TCP server/client app by using python built-in SocketServer. My problem is if client get killed, then the tcp port will never get released, in CLOSE_WAIT maybe I didn't do the handler right? or anyway I can catch the client get killed? I wrote following simple code to explain the issue I have, server listen on 127.0.0.1:1234 client need to send multi-messages (I am sending 2 in this test) if client get killed (I just do kill client_pid) then if I check lsof, I will see the new allocated tcp do not get released ########server.py######## #!/usr/bin/python import threading import SocketServer import time class TCPServerRequestHandler(SocketServer.BaseRequestHandler): def handle(self): while True: data=self.request.recv(1024) if not data: time.sleep(0.1) continue print data self.request.send(data) if data=='end': break def main(): server = TCPServer(('127.0.0.1', 1234), TCPServerRequestHandler) ip, port = server.server_address server_thread = threading.Thread(target=server.serve_forever) server_thread.daemon = True server_thread.start() time.sleep(600) server.shutdown() server.server_close() if __name__=='__main__': main() ########client.py######## import socket import time def client(): sock=socket.socket(socket.AF_INET, socket.SOCK_STREAM) sock.connect(('127.0.0.1', 1234)) sock.sendall('hello') response=sock.recv(1024) print "Received:", response time.sleep(60) sock.sendall('end') response=sock.recv(1024) print "Received:", response sock.close() if __name__=='__main__': client() when I run it, client has 60 seconds sleep, so I just kill the process from OS kill 93948 then I'm running lsof to find my TCP/UDP lsof -n -p $(ps -ef|grep serve[r]|awk '{print $2}')|egrep '(TCP|UDP)'|awk '{print $NF}' TCP 127.0.0.1:search-agent (LISTEN) TCP 127.0.0.1:search-agent->127.0.0.1:59411 (CLOSE_WAIT) I will see the CLOSE_WAIT, this is where the client get killed. any help would be great! Thanks a lot! From erikkrogen at gmail.com Thu Feb 25 13:04:48 2016 From: erikkrogen at gmail.com (erikkrogen at gmail.com) Date: Thu, 25 Feb 2016 10:04:48 -0800 (PST) Subject: New Module: Bond - A Spy-Based Testing and Mocking Library Message-ID: <57725418-5199-45d8-bb69-885834c0710e@googlegroups.com> Hi all - As part of my work as a Computer Science Masters student at UC Berkeley I've been working with Prof. George Necula to develop a new testing library, called Bond, which introduces a new style of test assertions that we refer to as spy-based testing. Essentially, rather than explicitly coding all assertions about your test data, you allow the library to take observations of your state/variables/objects at specified "spy" points, and you can simply approve these observations when your code runs correctly to have them saved as reference observations for future runs of the test. This saves time when initially writing tests, but really shines when updating tests to new behavior, as we have an interactive flow that allows you to quickly reconcile differences in new behavior and old behavior (often without changing any code at all). There is a blog post with much more detailed information and examples available here: http://dreaming-neptune.blogspot.com/2016/02/bond-spy-based-testing-and-mocking.html, and the package is available on PyPI here: https://pypi.python.org/pypi/bond/ (you can `pip install bond` to install). Work on the library is ongoing so any feedback here or as comments on the blog would be much appreciated! Thanks and enjoy! Erik From recruitrohitkoul at gmail.com Thu Feb 25 13:07:50 2016 From: recruitrohitkoul at gmail.com (Rohit Koul) Date: Thu, 25 Feb 2016 10:07:50 -0800 (PST) Subject: Job opportunity - Sr. Python Developer - Morrisville, NC Message-ID: Title: Senior Python Developer Location: Morrisville, NC, United States Contract : 6+ months Description: Qualifications: The Senior Python Developer will lead and serve as part of a team supporting established projects and creating products from the ground up. The ideal candidate is leader, a team player and is passionate about learning and applying their technical skills and experience to solve real-world problems. Responsibilities: - Contribute to the Architecture, Design and Development across the full stack of products. - Work with multiple teams to guide and influence the architecture and design of the various products - Investigating technologies that could benefit our development and success of our products Thanks and regards, Rohit Rohit Koul Technical Recruiter Email:-rohit at adeptincorp.com From martin at linux-ip.net Thu Feb 25 13:17:27 2016 From: martin at linux-ip.net (Martin A. Brown) Date: Thu, 25 Feb 2016 10:17:27 -0800 Subject: tcp networking question (CLOSE_WAIT) In-Reply-To: <7ff15c67-102e-4b02-a2c8-87b24f1b8351@googlegroups.com> References: <7ff15c67-102e-4b02-a2c8-87b24f1b8351@googlegroups.com> Message-ID: >I'm new to python networking. I am waiting TCP server/client app by >using python built-in SocketServer. My problem is if client get >killed, then the tcp port will never get released, in CLOSE_WAIT I did not thoroughly review your code (other than to see that you are not using SO_REUSEADDR). This is the most likely problem. Suggestion: man 7 socket Look for SO_REUSEADDR. Then, apply what you have learned to your code. -Martin -- Martin A. Brown http://linux-ip.net/ From rui.vapps at gmail.com Thu Feb 25 13:25:23 2016 From: rui.vapps at gmail.com (Ray) Date: Thu, 25 Feb 2016 10:25:23 -0800 (PST) Subject: tcp networking question (CLOSE_WAIT) In-Reply-To: References: <7ff15c67-102e-4b02-a2c8-87b24f1b8351@googlegroups.com> Message-ID: <86a8ad2a-3185-4ff3-abe0-7345411fa0c2@googlegroups.com> On Thursday, February 25, 2016 at 1:18:05 PM UTC-5, Martin A. Brown wrote: > >I'm new to python networking. I am waiting TCP server/client app by > >using python built-in SocketServer. My problem is if client get > >killed, then the tcp port will never get released, in CLOSE_WAIT > > I did not thoroughly review your code (other than to see that you > are not using SO_REUSEADDR). This is the most likely problem. > > Suggestion: > > man 7 socket > > Look for SO_REUSEADDR. Then, apply what you have learned to your > code. > > -Martin > > -- > Martin A. Brown > http://linux-ip.net/ it's not I can't bind the address, my problem is: server is long run. if client die without "disconnect" then server will leak one socket. by using the built-in thread socket server. the extra tcp port are opened by built-in class itself. if the handler() is finish correctly (the line with break) then this socket will get cleaned up. but if client dies, then I am never get out from that True loop. so the socket will keep in close_wait From rui.vapps at gmail.com Thu Feb 25 13:44:52 2016 From: rui.vapps at gmail.com (Ray) Date: Thu, 25 Feb 2016 10:44:52 -0800 (PST) Subject: tcp networking question (CLOSE_WAIT) In-Reply-To: <7ff15c67-102e-4b02-a2c8-87b24f1b8351@googlegroups.com> References: <7ff15c67-102e-4b02-a2c8-87b24f1b8351@googlegroups.com> Message-ID: On Thursday, February 25, 2016 at 12:56:10 PM UTC-5, Ray wrote: > hi, > > I'm new to python networking. I am waiting TCP server/client app by using python built-in SocketServer. My problem is if client get killed, then the tcp port will never get released, in CLOSE_WAIT > > maybe I didn't do the handler right? or anyway I can catch the client get killed? > > I wrote following simple code to explain the issue I have, server listen on 127.0.0.1:1234 > client need to send multi-messages (I am sending 2 in this test) if client get killed (I just do kill client_pid) then if I check lsof, I will see the new allocated tcp do not get released > > ########server.py######## > > #!/usr/bin/python > import threading > import SocketServer > import time > > class TCPServerRequestHandler(SocketServer.BaseRequestHandler): > > def handle(self): > while True: > data=self.request.recv(1024) > if not data: > time.sleep(0.1) > continue > print data > self.request.send(data) > if data=='end': > break > > def main(): > server = TCPServer(('127.0.0.1', 1234), TCPServerRequestHandler) > ip, port = server.server_address > server_thread = threading.Thread(target=server.serve_forever) > server_thread.daemon = True > server_thread.start() > time.sleep(600) > server.shutdown() > server.server_close() > if __name__=='__main__': > main() > > ########client.py######## > import socket > import time > > def client(): > sock=socket.socket(socket.AF_INET, socket.SOCK_STREAM) > sock.connect(('127.0.0.1', 1234)) > sock.sendall('hello') > response=sock.recv(1024) > print "Received:", response > time.sleep(60) > sock.sendall('end') > response=sock.recv(1024) > print "Received:", response > sock.close() > > if __name__=='__main__': > client() > > > when I run it, client has 60 seconds sleep, so I just kill the process from OS > kill 93948 > > then I'm running lsof to find my TCP/UDP > lsof -n -p $(ps -ef|grep serve[r]|awk '{print $2}')|egrep '(TCP|UDP)'|awk '{print $NF}' > > TCP 127.0.0.1:search-agent (LISTEN) > TCP 127.0.0.1:search-agent->127.0.0.1:59411 (CLOSE_WAIT) > > I will see the CLOSE_WAIT, this is where the client get killed. > > > any help would be great! > > Thanks a lot! I fond the issue. it's my own stupid issue. i did "continue" if no data received. just break from it then it will be fine From martin at linux-ip.net Thu Feb 25 13:56:00 2016 From: martin at linux-ip.net (Martin A. Brown) Date: Thu, 25 Feb 2016 10:56:00 -0800 Subject: tcp networking question (CLOSE_WAIT) In-Reply-To: <86a8ad2a-3185-4ff3-abe0-7345411fa0c2@googlegroups.com> References: <7ff15c67-102e-4b02-a2c8-87b24f1b8351@googlegroups.com> <86a8ad2a-3185-4ff3-abe0-7345411fa0c2@googlegroups.com> Message-ID: Hello again Ray, >> >I'm new to python networking. I am waiting TCP server/client app by >> >using python built-in SocketServer. My problem is if client get >> >killed, then the tcp port will never get released, in CLOSE_WAIT >> >> I did not thoroughly review your code (other than to see that you >> are not using SO_REUSEADDR). This is the most likely problem. >> >> Suggestion: >> >> man 7 socket >> >> Look for SO_REUSEADDR. Then, apply what you have learned to your >> code. > >it's not I can't bind the address, my problem is: server is long >run. if client die without "disconnect" then server will leak one >socket. Sorry for my trigger-happy, and incorrect reply. After so many years, I should know better than to reply without completely processing questions. Apologies. >by using the built-in thread socket server. the extra tcp port are >opened by built-in class itself. if the handler() is finish >correctly (the line with break) then this socket will get cleaned >up. but if client dies, then I am never get out from that True >loop. so the socket will keep in close_wait > >I fond the issue. it's my own stupid issue. >i did "continue" if no data received. >just break from it then it will be fine Well, I'm glad you found the issue. Best of luck, -Martin -- Martin A. Brown http://linux-ip.net/ From rui.vapps at gmail.com Thu Feb 25 13:59:03 2016 From: rui.vapps at gmail.com (Ray) Date: Thu, 25 Feb 2016 10:59:03 -0800 (PST) Subject: tcp networking question (CLOSE_WAIT) In-Reply-To: References: <7ff15c67-102e-4b02-a2c8-87b24f1b8351@googlegroups.com> <86a8ad2a-3185-4ff3-abe0-7345411fa0c2@googlegroups.com> Message-ID: On Thursday, February 25, 2016 at 1:56:21 PM UTC-5, Martin A. Brown wrote: > Hello again Ray, > > >> >I'm new to python networking. I am waiting TCP server/client app by > >> >using python built-in SocketServer. My problem is if client get > >> >killed, then the tcp port will never get released, in CLOSE_WAIT > >> > >> I did not thoroughly review your code (other than to see that you > >> are not using SO_REUSEADDR). This is the most likely problem. > >> > >> Suggestion: > >> > >> man 7 socket > >> > >> Look for SO_REUSEADDR. Then, apply what you have learned to your > >> code. > > > >it's not I can't bind the address, my problem is: server is long > >run. if client die without "disconnect" then server will leak one > >socket. > > Sorry for my trigger-happy, and incorrect reply. > > After so many years, I should know better than to reply without > completely processing questions. Apologies. > > >by using the built-in thread socket server. the extra tcp port are > >opened by built-in class itself. if the handler() is finish > >correctly (the line with break) then this socket will get cleaned > >up. but if client dies, then I am never get out from that True > >loop. so the socket will keep in close_wait > > > >I fond the issue. it's my own stupid issue. > >i did "continue" if no data received. > >just break from it then it will be fine > > Well, I'm glad you found the issue. > > Best of luck, > > -Martin > > -- > Martin A. Brown > http://linux-ip.net/ Thank you very much for reply and help. From ben+python at benfinney.id.au Thu Feb 25 14:17:42 2016 From: ben+python at benfinney.id.au (Ben Finney) Date: Fri, 26 Feb 2016 06:17:42 +1100 Subject: Use the Python Job Board for recruitment (was: Job opportunity - Sr. Python Developer - Morrisville, NC) In-Reply-To: (Rohit Koul's message of "Thu, 25 Feb 2016 10:07:50 -0800 (PST)") References: Message-ID: <851t80zjjd.fsf@benfinney.id.au> Rohit Koul writes: > Title: Senior Python Developer > Location: Morrisville, NC, United States Please do not use the Python discussion forum for recruitment. Instead, use the Python Job Board which is explicitly for this purpose . -- \ ?One of the most important things you learn from the internet | `\ is that there is no ?them? out there. It's just an awful lot of | _o__) ?us?.? ?Douglas Adams | Ben Finney From sohcahtoa82 at gmail.com Thu Feb 25 15:00:49 2016 From: sohcahtoa82 at gmail.com (sohcahtoa82 at gmail.com) Date: Thu, 25 Feb 2016 12:00:49 -0800 (PST) Subject: "from module import data; print(data)" vs "import module; print(module.data)" In-Reply-To: References: Message-ID: <535b56b5-f836-4cd6-ae95-33f18004c90b@googlegroups.com> On Wednesday, February 24, 2016 at 5:07:57 PM UTC-8, Dan Stromberg wrote: > Could people please compare and contrast the two ways of doing imports > in the Subject line? > > I've long favored the latter, but I'm working in a code base that > prefers the former. > > Is it fair to say that the former increases the surface area of your > shared (sometimes mutable) state? > > It's clear that the former saves keystrokes. > > I find the latter a little more clear, because you don't have to go > look for where a symbol came from. > > Anything else? > > Thanks! If I'm only importing a single class or function from a module AND that class or function has a very meaningful name, I will use "from MyModule import MyUsefulClass". Otherwise, I just use "import MyModule". Now, I've noticed people talking about importing os.path. Is there any reason to use "import os.path" rather than "import os"? Both of them will still put the "os" module into the global namespace. From greg.ewing at canterbury.ac.nz Thu Feb 25 17:20:14 2016 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Fri, 26 Feb 2016 11:20:14 +1300 Subject: "from module import data; print(data)" vs "import module; print(module.data)" In-Reply-To: <535b56b5-f836-4cd6-ae95-33f18004c90b@googlegroups.com> References: <535b56b5-f836-4cd6-ae95-33f18004c90b@googlegroups.com> Message-ID: sohcahtoa82 at gmail.com wrote: > Now, I've noticed people talking about importing os.path. Is there any > reason to use "import os.path" rather than "import os"? Both of them will > still put the "os" module into the global namespace. In the case of os.path it doesn't matter, because the os module imports the appropriate path module automatically. But in general, importing a package won't necessarily import submodules under it, so sometimes you need to import somepackage.somemodule explicitly. -- Greg From rgacote at appropriatesolutions.com Thu Feb 25 17:23:50 2016 From: rgacote at appropriatesolutions.com (Ray Cote) Date: Thu, 25 Feb 2016 17:23:50 -0500 Subject: Looking for examples of using aiopg with aiohttp and non-async startup. Message-ID: Hello: I have an aiohttp project that starts in the usual way: app = web.Application() app.router.add_route(?POST?, ?/?, handler) web.run_app(app) My question is, how do I work with an aiopg.pool with aiohttp? There only seems to be async interfaces into aiopg ? but I don?t want to create the pool in my handler since that requires a connection on each transaction. I found one example that was app[?db?] = await aiopg.create_pool(dsn) but that doesn?t seen correct since we?re not yet in a loop. Can someone please provide an example showing the proper way to integrate aiopg pool into an aiohttp web application? Regards ?Ray -- Raymond Cote, President voice: +1.603.924.6079 email: rgacote at AppropriateSolutions.com skype: ray.cote From drsalists at gmail.com Thu Feb 25 18:01:22 2016 From: drsalists at gmail.com (Dan Stromberg) Date: Thu, 25 Feb 2016 15:01:22 -0800 Subject: Testing whether the VPN is running? In-Reply-To: <20160218094531.GA5510@arxnet.hu> References: <2cfgpcxdg2.ln2@news.ducksburg.com> <20160218094531.GA5510@arxnet.hu> Message-ID: On Thu, Feb 18, 2016 at 1:45 AM, Ervin Heged?s wrote: > Hi Adam, > > On Thu, Feb 18, 2016 at 09:26:58AM +0000, Adam Funk wrote: >> I'd like to test (inside a python 3 program) whether the VPN is >> running or not. The only thing I can think of so far is to use >> subprocess to run the 'ifconfig' command, then check its output for >> 'tun0'. Is there a better way? > > you didn't wrote, which system (OS) you want to use - based on > the "ifconfig" and "tun0" keywords, possible that's a Linux. > > I think that the psutil modul could be better for you for this > task: > > https://pypi.python.org/pypi/psutil/ > > and see the "Network" section. psutil is probably clean on Linux. However, using the subprocess module to run ifconfig -a and check for tun0 / utun0 should get you compatibility with both Linux and Mac. Note that if I'm running a Linux in a VMware, and it's hosted on a Mac, and the Mac has one of my VPN's up, then the Linux doesn't have a tun0, but it can still see stuff only accessible via that VPN. For that, it might be better to try ping'ing some host that's behind the VPN. In a way, that's a better test anyway. From ben+python at benfinney.id.au Thu Feb 25 18:38:36 2016 From: ben+python at benfinney.id.au (Ben Finney) Date: Fri, 26 Feb 2016 10:38:36 +1100 Subject: "from module import data; print(data)" vs "import module; print(module.data)" References: <535b56b5-f836-4cd6-ae95-33f18004c90b@googlegroups.com> Message-ID: <85wppsxsw3.fsf@benfinney.id.au> Gregory Ewing writes: > sohcahtoa82 at gmail.com wrote: > > Now, I've noticed people talking about importing os.path. Is there any > > reason to use "import os.path" rather than "import os"? Both of them will > > still put the "os" module into the global namespace. > > In the case of os.path it doesn't matter, because the > os module imports the appropriate path module automatically. My position is that behaviour violates one of the principles of the Zen of Python: ?Special cases aren't special enough to break the rules.? > But in general, importing a package won't necessarily > import submodules under it, so sometimes you need to > import somepackage.somemodule explicitly. Because that's normally the case, I choose not to rely on that special behaviour of ?os?. If I need ?os.path?, I import it explicitly so no reader needs to guess:: import os import os.path -- \ ?Crime is contagious? if the government becomes a lawbreaker, | `\ it breeds contempt for the law.? ?Justice Louis Brandeis | _o__) | Ben Finney From steve at pearwood.info Thu Feb 25 19:40:31 2016 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 26 Feb 2016 11:40:31 +1100 Subject: "from module import data; print(data)" vs "import module; print(module.data)" References: <535b56b5-f836-4cd6-ae95-33f18004c90b@googlegroups.com> Message-ID: <56cf9f01$0$1620$c3e8da3$5496439d@news.astraweb.com> On Fri, 26 Feb 2016 10:38 am, Ben Finney wrote: > Gregory Ewing writes: > >> sohcahtoa82 at gmail.com wrote: >> > Now, I've noticed people talking about importing os.path. Is there any >> > reason to use "import os.path" rather than "import os"? Both of them >> > will still put the "os" module into the global namespace. >> >> In the case of os.path it doesn't matter, because the >> os module imports the appropriate path module automatically. > > My position is that behaviour violates one of the principles of the Zen > of Python: ?Special cases aren't special enough to break the rules.? But it's not special. It's the standard behaviour of any module which offers a public name: `import os` makes all the names in the os name space available. There's no difference between (say) `os.listdir` and `os.path` except that listdir happens to be a function and path happens to be a module. `path` has been a documented public attribute of the `os` module since at least Python 1.5: [steve at ando ~]$ python1.5 -c "import os; print os.path" and likely older. The documentation from help(os) says: DESCRIPTION This exports: - all functions from posix, nt, os2, mac, or ce, e.g. unlink, stat, etc. - os.path is one of the modules posixpath, ntpath, or macpath (The above is from 2.4, the oldest Python I have that supports docstrings; in 3.3 it is substantially the same except macpath is removed.) Since `os` is not a package, the fact that importing `os.path` works at all is surprising. The os module has to make a special effort to support importing os.path. From the source code: sys.modules['os.path'] = path If you take "Special cases are not special enough" seriously, you will not use `import os.path` since os is not a package: py> os.__package__ '' and os.path is not part of os, it's just a publicly exposed attribute which merely happens to be a module. Being a module doesn't make it special, it's just another name in the os namespace. I trust that you wouldn't insist on writing: import os.listdir (especially since that doesn't work). Neither should you insist on writing `import os.path`, since path is documented as a public part of the os module. It has done so since at least Python 1.5. >> But in general, importing a package won't necessarily >> import submodules under it, so sometimes you need to >> import somepackage.somemodule explicitly. > > Because that's normally the case, I choose not to rely on that special > behaviour of ?os?. If I need ?os.path?, I import it explicitly so no > reader needs to guess:: > > import os > import os.path As I have shown above, the fact that os.path is importable at all is the special case. -- Steven From ian.g.kelly at gmail.com Thu Feb 25 22:56:18 2016 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Thu, 25 Feb 2016 20:56:18 -0700 Subject: "from module import data; print(data)" vs "import module; print(module.data)" In-Reply-To: <56cf9f01$0$1620$c3e8da3$5496439d@news.astraweb.com> References: <535b56b5-f836-4cd6-ae95-33f18004c90b@googlegroups.com> <56cf9f01$0$1620$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Thu, Feb 25, 2016 at 5:40 PM, Steven D'Aprano wrote: > If you take "Special cases are not special enough" seriously, you will not > use `import os.path` since os is not a package: > > py> os.__package__ > '' > > and os.path is not part of os, it's just a publicly exposed attribute which > merely happens to be a module. Being a module doesn't make it special, it's > just another name in the os namespace. I trust that you wouldn't insist on > writing: > > import os.listdir > > (especially since that doesn't work). Neither should you insist on writing > `import os.path`, since path is documented as a public part of the os > module. It has done so since at least Python 1.5. I disagree. The fact that os is not a package is an implementation detail. I for one wasn't even aware of it prior to reading your post. The name of the concurrent.futures module is "concurrent.futures". If you want to use it, you import concurrent.futures, not concurrent. Likewise, the name of the os.path module is "os.path". If you want to use it, you import os.path, not os. The fact that concurrent and os are two different types of things is irrelevant. Besides, packages *are* modules. To take another example, collections is a package (I checked), and collections.abc is a module. But collections also contains things that aren't modules. If you want to use collections.abc, you have to import it, but at the same time you don't import collections.Counter. This demonstrates that the analog you suggest between os.path and os.listdir is flawed. From rosuav at gmail.com Thu Feb 25 23:11:51 2016 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 26 Feb 2016 15:11:51 +1100 Subject: "from module import data; print(data)" vs "import module; print(module.data)" In-Reply-To: References: <535b56b5-f836-4cd6-ae95-33f18004c90b@googlegroups.com> <56cf9f01$0$1620$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Fri, Feb 26, 2016 at 2:56 PM, Ian Kelly wrote: > On Thu, Feb 25, 2016 at 5:40 PM, Steven D'Aprano wrote: >> If you take "Special cases are not special enough" seriously, you will not >> use `import os.path` since os is not a package: >> >> py> os.__package__ >> '' >> >> and os.path is not part of os, it's just a publicly exposed attribute which >> merely happens to be a module. Being a module doesn't make it special, it's >> just another name in the os namespace. I trust that you wouldn't insist on >> writing: >> >> import os.listdir >> >> (especially since that doesn't work). Neither should you insist on writing >> `import os.path`, since path is documented as a public part of the os >> module. It has done so since at least Python 1.5. > > I disagree. The fact that os is not a package is an implementation > detail. I for one wasn't even aware of it prior to reading your post. > > The name of the concurrent.futures module is "concurrent.futures". If > you want to use it, you import concurrent.futures, not concurrent. > > Likewise, the name of the os.path module is "os.path". If you want to > use it, you import os.path, not os. > > The fact that concurrent and os are two different types of things is irrelevant. > > Besides, packages *are* modules. To take another example, collections > is a package (I checked), and collections.abc is a module. But > collections also contains things that aren't modules. If you want to > use collections.abc, you have to import it, but at the same time you > don't import collections.Counter. This demonstrates that the analog > you suggest between os.path and os.listdir is flawed. Steven is right, though. You can "from module import X" and it doesn't matter whether 'module' is a package or a non-package module; but if you say "import module.X", aside from special cases like os.path, you're asserting that 'module' is a package and 'X' is a module within that package. Yes, a package is a module, but the dotted lookup requires a package. ChrisA From ben+python at benfinney.id.au Thu Feb 25 23:37:14 2016 From: ben+python at benfinney.id.au (Ben Finney) Date: Fri, 26 Feb 2016 15:37:14 +1100 Subject: "from module import data; print(data)" vs "import module; print(module.data)" References: <535b56b5-f836-4cd6-ae95-33f18004c90b@googlegroups.com> <56cf9f01$0$1620$c3e8da3$5496439d@news.astraweb.com> Message-ID: <85si0gxf2d.fsf@benfinney.id.au> Steven D'Aprano writes: > On Fri, 26 Feb 2016 10:38 am, Ben Finney wrote: > > > Gregory Ewing writes: > > > >> sohcahtoa82 at gmail.com wrote: > >> > Now, I've noticed people talking about importing os.path. Is there any > >> > reason to use "import os.path" rather than "import os"? Both of them > >> > will still put the "os" module into the global namespace. > >> > >> In the case of os.path it doesn't matter, because the > >> os module imports the appropriate path module automatically. > > > > My position is that behaviour violates one of the principles of the Zen > > of Python: ?Special cases aren't special enough to break the rules.? > > But it's not special. It's the standard behaviour of any module which offers > a public name That is the special behaviour. ?os? is a package that has a sub-module ?path?. Normally, to get at a module inside a package, you import it with a qualified name:: import os.path The special case is that ?os? also wants the ?path? module itself, and so the name happens to be available as a module attribute. That's a special case that can not be depended on for other cases. > `import os` makes all the names in the os name space available. Since ?os.path? is a sub-module of the ?os? package, it should not also be exported from the ?os? module attributes. The special case is confusing. > There's no difference between (say) `os.listdir` and `os.path` except > that listdir happens to be a function and path happens to be a module. There's no difference between ?logging.info? and ?logging.config? except that ?info? happens to be a function and ?config? happens to be a module. The difference is salient:: >>> import logging >>> logging.info >>> logging.config Traceback (most recent call last): File "", line 1, in AttributeError: module 'logging' has no attribute 'config' That's the case normally, when one name is primarily an attribute and another name is primarily a sub-module. The ?os.path? case is not special enough to break that rule IMO. > `path` has been a documented public attribute of the `os` module since at > least Python 1.5: Then it's been violating that principle for that long :-) > If you take "Special cases are not special enough" seriously, you will > not use `import os.path` since os is not a package The implementation is special. That doesn't exempt it from the principle that a special case isn't special enough to break expectations. > and os.path is not part of os, it's just a publicly exposed attribute > which merely happens to be a module. You seem to be making my case for me: ?os? is indeed special. It goes to some legth to hide that specialness; I think it doesn't go far enough. -- \ ?If you don't want your beliefs to be ridiculed, don't have | `\ such ridiculous beliefs.? ?Greta Christina, 2011-10-22 | _o__) | Ben Finney From mscir at yahoo.com Fri Feb 26 00:01:02 2016 From: mscir at yahoo.com (Mike S) Date: Thu, 25 Feb 2016 21:01:02 -0800 Subject: Computational Chemistry Analysis In-Reply-To: References: Message-ID: On 2/25/2016 7:31 AM, Oscar Benjamin wrote: > On 25 February 2016 at 01:01, Feagans, Mandy wrote: >> Hi! I am a student interested in conducting computational analysis of protein-ligand binding for drug development analysis. Recently, I read of an individual using a python program for their studies of protein-ligand binding. As I have been reading about Python programs, however, all I have been able to find are programs that use the python script (i.e. Autodock). I was hoping to see if there were any programs specifically run through the python programming that ran similar analysis to Autodock? > You may have better luck asking on the scipy user list: > http://www.scipy.org/scipylib/mailing-lists.html > Those lists are specifically for people doing scientific work with Python. > Oscar HtH http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3607193/ From mscir at yahoo.com Fri Feb 26 00:39:50 2016 From: mscir at yahoo.com (Mike S) Date: Thu, 25 Feb 2016 21:39:50 -0800 Subject: Speech recognition and synthesi in Python Message-ID: Pretty nice example code... https://ggulati.wordpress.com/2016/02/24/coding-jarvis-in-python-3-in-2016/ From plucena24 at gmail.com Fri Feb 26 02:44:05 2016 From: plucena24 at gmail.com (Pablo Lucena) Date: Thu, 25 Feb 2016 23:44:05 -0800 Subject: Cycling through iterables diagonally Message-ID: Hello, I am trying to accomplish the following: Say I have a group of 4 lists as follows: l1 = ['a1', 'a2', 'a3', 'a4'] l2 = ['b1', 'b2', 'b3', 'b4'] l3 = ['c1', 'c2', 'c3', 'c4'] l4 = ['d1', 'd2', 'd3', 'd4'] I would like to cycle through these lists "diagonally" in groups of len(list) (in this example, each list has 4 items). cycle1: a1, b2, b3, b4 cycle2: a2, b3, c4, d1 cycle3: a3, b4, c1, d2 cycle4: a4, b1, c2, d3 The way I thought about doing this is as follows: from collections import deque from itertools import cycle l1 = deque(['a1', 'a2', 'a3', 'a4']) l2 = deque(['b1', 'b2', 'b3', 'b4']) l3 = deque(['c1', 'c2', 'c3', 'c4']) l4 = deque(['d1', 'd2', 'd3', 'd4']) l1.rotate(-0) l2.rotate(-1) l3.rotate(-2) l4.rotate(-3) groups = cycle([l1, l2, l3, l4]) In [115]: for group in groups: .....: if not group: .....: break .....: print(group.popleft()) .....: a1 b2 c3 d4 a2 b3 c4 d1 a3 b4 c1 d2 a4 b1 c2 d3 Prior to this I was mucking around with index counting while looping, and popping lists out of a deque, popping an item out of the list, and appending the list back into the deque during each iteration. Is there a better/cleaner way to do this? I was hoping for some cool itertools logic =) Thanks! -- *Pabl?o?* From jussi.piitulainen at helsinki.fi Fri Feb 26 04:21:34 2016 From: jussi.piitulainen at helsinki.fi (Jussi Piitulainen) Date: Fri, 26 Feb 2016 11:21:34 +0200 Subject: Cycling through iterables diagonally References: Message-ID: Pablo Lucena writes: > Hello, > > I am trying to accomplish the following: > > Say I have a group of 4 lists as follows: > > l1 = ['a1', 'a2', 'a3', 'a4'] > l2 = ['b1', 'b2', 'b3', 'b4'] > l3 = ['c1', 'c2', 'c3', 'c4'] > l4 = ['d1', 'd2', 'd3', 'd4'] > > I would like to cycle through these lists "diagonally" in groups of > len(list) (in this example, each list has 4 items). > > cycle1: a1, b2, b3, b4 > cycle2: a2, b3, c4, d1 > cycle3: a3, b4, c1, d2 > cycle4: a4, b1, c2, d3 First line should be a1, b2, c3, d4, right? > The way I thought about doing this is as follows: > > from collections import deque > from itertools import cycle [...] > Prior to this I was mucking around with index counting while looping, > and popping lists out of a deque, popping an item out of the list, and > appending the list back into the deque during each iteration. > > Is there a better/cleaner way to do this? I was hoping for some cool > itertools logic =) Here's a combination of itertools (chaining of slices for the rotations) and builtin machinery (argument spreading, a generator expression, zip, enumerate) that seems to have the desired effect. from itertools import chain, islice l1 = ['a1', 'a2', 'a3', 'a4'] l2 = ['b1', 'b2', 'b3', 'b4'] l3 = ['c1', 'c2', 'c3', 'c4'] l4 = ['d1', 'd2', 'd3', 'd4'] n = 4 chainfrom = chain.from_iterable print(*chainfrom(zip(*(chain(islice(m, k, n), islice(m, 0, k)) for k, m in enumerate([l1, l2, l3, l4]))))) # prints: a1 b2 c3 d4 a2 b3 c4 d1 a3 b4 c1 d2 a4 b1 c2 d3 From __peter__ at web.de Fri Feb 26 04:59:40 2016 From: __peter__ at web.de (Peter Otten) Date: Fri, 26 Feb 2016 10:59:40 +0100 Subject: Cycling through iterables diagonally References: Message-ID: Pablo Lucena wrote: > Say I have a group of 4 lists as follows: > > l1 = ['a1', 'a2', 'a3', 'a4'] > l2 = ['b1', 'b2', 'b3', 'b4'] > l3 = ['c1', 'c2', 'c3', 'c4'] > l4 = ['d1', 'd2', 'd3', 'd4'] > > I would like to cycle through these lists "diagonally" in groups of > len(list) (in this example, each list has 4 items). > Prior to this I was mucking around with index counting while looping, and > popping lists out of a deque, popping an item out of the list, and > appending the list back into the deque during each iteration. > > Is there a better/cleaner way to do this? I was hoping for some cool > itertools logic =) I have a weak spot for the itertools myself, but I think in terms of clarity it is hard to beat the conventional def diagonals(a): N = len(a) for i in range(N): for k in range(N): yield a[k][(k+i)%N] print(list(diagonals([l1, l2, l3, l4]))) Of course that's as uncool as it can get ;) From breamoreboy at yahoo.co.uk Fri Feb 26 06:00:31 2016 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Fri, 26 Feb 2016 11:00:31 +0000 Subject: Cycling through iterables diagonally In-Reply-To: References: Message-ID: On 26/02/2016 09:59, Peter Otten wrote: > Pablo Lucena wrote: > >> Say I have a group of 4 lists as follows: >> >> l1 = ['a1', 'a2', 'a3', 'a4'] >> l2 = ['b1', 'b2', 'b3', 'b4'] >> l3 = ['c1', 'c2', 'c3', 'c4'] >> l4 = ['d1', 'd2', 'd3', 'd4'] >> >> I would like to cycle through these lists "diagonally" in groups of >> len(list) (in this example, each list has 4 items). > >> Prior to this I was mucking around with index counting while looping, and >> popping lists out of a deque, popping an item out of the list, and >> appending the list back into the deque during each iteration. >> >> Is there a better/cleaner way to do this? I was hoping for some cool >> itertools logic =) > > I have a weak spot for the itertools myself, but I think in terms of clarity > it is hard to beat the conventional > > def diagonals(a): > N = len(a) > for i in range(N): > for k in range(N): > yield a[k][(k+i)%N] > > print(list(diagonals([l1, l2, l3, l4]))) > > Of course that's as uncool as it can get ;) > It might be uncool, but at least I can read it, unlike the earlier Jussi Piitulainen answer that made my head spin :) -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From ganesh1pal at gmail.com Fri Feb 26 06:07:11 2016 From: ganesh1pal at gmail.com (Ganesh Pal) Date: Fri, 26 Feb 2016 16:37:11 +0530 Subject: How to remove the line numbers from the file in python Message-ID: what would be the easiest way to remove the lines in the leading numbers 1.e 1 ,2,.... 19 from this file using python ? 1 import os 2 Suite = "Test Mail" 3 4 def sendMail(x): 5 text = x 6 sendmail_location = "/home/prasad/onefs/share/sendmail" # sendmail location 7 p = os.popen("%s -t" % sendmail_location, "w") 8 p.write("From: %s\n" % "ganesh.pal at gmail.com") 9 p.write("To: %s\n" % "ganesh.pal at gmail.com") 10 #p.write("To: %s\n" % "umamaheshwar.b at gmail.com") 11 p.write("Subject: Suite : %s \n" % (Suite)) 12 p.write("\n") # blank line separating headers from body 13 p.write("%s" %text) 14 status = p.close() 15 16 if status != 0: 17 print "Sendmail exit status", status 18 19 sendMail("Test Mail") Regards, Ganesh From __peter__ at web.de Fri Feb 26 06:31:05 2016 From: __peter__ at web.de (Peter Otten) Date: Fri, 26 Feb 2016 12:31:05 +0100 Subject: How to remove the line numbers from the file in python References: Message-ID: Ganesh Pal wrote: > what would be the easiest way to remove the lines in the leading > numbers 1.e 1 ,2,.... 19 from this file using python ? > > > 1 import os > 2 Suite = "Test Mail" > 3 > 4 def sendMail(x): > 5 text = x > 6 sendmail_location = "/home/prasad/onefs/share/sendmail" # > sendmail location > 7 p = os.popen("%s -t" % sendmail_location, "w") > 8 p.write("From: %s\n" % "ganesh.pal at gmail.com") > 9 p.write("To: %s\n" % "ganesh.pal at gmail.com") > 10 #p.write("To: %s\n" % "umamaheshwar.b at gmail.com") > 11 p.write("Subject: Suite : %s \n" % (Suite)) > 12 p.write("\n") # blank line separating headers from body > 13 p.write("%s" %text) > 14 status = p.close() > 15 > 16 if status != 0: > 17 print "Sendmail exit status", status > 18 > 19 sendMail("Test Mail") sys.stdout.writelines( line.lstrip().lstrip("0123456789")[1:] or "\n" for line in sys.stdin) Replace stdin/out with files as needed. From peter.heitzer at rz.uni-regensburg.de Fri Feb 26 06:31:32 2016 From: peter.heitzer at rz.uni-regensburg.de (Peter Heitzer) Date: 26 Feb 2016 11:31:32 GMT Subject: How to remove the line numbers from the file in python References: Message-ID: Ganesh Pal wrote: >what would be the easiest way to remove the lines in the leading >numbers 1.e 1 ,2,.... 19 from this file using python ? import sys,re for line in sys.stdin: print re.sub('^\d+','',line).rstrip() From shweta.d.ece at gmail.com Fri Feb 26 08:20:16 2016 From: shweta.d.ece at gmail.com (Shweta Dinnimani) Date: Fri, 26 Feb 2016 18:50:16 +0530 Subject: subprocess startup error Message-ID: hi hello, I'm begineer to python programming.. I had installed python 3.5.1 version on my windows 7 system. I was fine earlier and now when i was trying the programs on string i'm facing the subprocess startup error. IDLE is not connecting. And python shell is also not opening. I tried uninstalling and installing the python shell but Im facing the problem.Please do help me -- Thank you with best regards Shweta Dinnimani +91-8762650817 From shweta.d.ece at gmail.com Fri Feb 26 08:22:08 2016 From: shweta.d.ece at gmail.com (Shweta Dinnimani) Date: Fri, 26 Feb 2016 18:52:08 +0530 Subject: subprocess startup error In-Reply-To: References: Message-ID: forgot to mention.. i saved my file as string.py since than i'm facing this error On Fri, Feb 26, 2016 at 6:50 PM, Shweta Dinnimani wrote: > hi > > hello, I'm begineer to python programming.. I had installed python 3.5.1 > version on my windows 7 system. I was fine earlier and now when i was > trying the programs on string i'm facing the subprocess startup error. IDLE > is not connecting. And python shell is also not opening. I tried > uninstalling and installing the python shell but Im facing the > problem.Please do help me > > -- > Thank you > with best regards > > Shweta Dinnimani > +91-8762650817 > -- Thank you with best regards Shweta Dinnimani +91-8762650817 From __peter__ at web.de Fri Feb 26 08:30:13 2016 From: __peter__ at web.de (Peter Otten) Date: Fri, 26 Feb 2016 14:30:13 +0100 Subject: subprocess startup error References: Message-ID: Shweta Dinnimani wrote: > forgot to mention.. > > i saved my file as string.py since than i'm facing this error Rename that file to something that does not clash with the module names in the standard library, mystring.py for example, and IDLE should work again. > On Fri, Feb 26, 2016 at 6:50 PM, Shweta Dinnimani > wrote: > >> hi >> >> hello, I'm begineer to python programming.. I had installed python 3.5.1 >> version on my windows 7 system. I was fine earlier and now when i was >> trying the programs on string i'm facing the subprocess startup error. >> IDLE is not connecting. And python shell is also not opening. I tried >> uninstalling and installing the python shell but Im facing the >> problem.Please do help me From marco.nawijn at colosso.nl Fri Feb 26 09:27:50 2016 From: marco.nawijn at colosso.nl (marco.nawijn at colosso.nl) Date: Fri, 26 Feb 2016 06:27:50 -0800 (PST) Subject: Cycling through iterables diagonally In-Reply-To: References: Message-ID: On Friday, February 26, 2016 at 8:44:38 AM UTC+1, Pablo Lucena wrote: > Hello, > > I am trying to accomplish the following: > > Say I have a group of 4 lists as follows: > > l1 = ['a1', 'a2', 'a3', 'a4'] > l2 = ['b1', 'b2', 'b3', 'b4'] > l3 = ['c1', 'c2', 'c3', 'c4'] > l4 = ['d1', 'd2', 'd3', 'd4'] > > I would like to cycle through these lists "diagonally" in groups of > len(list) (in this example, each list has 4 items). > > cycle1: a1, b2, b3, b4 > cycle2: a2, b3, c4, d1 > cycle3: a3, b4, c1, d2 > cycle4: a4, b1, c2, d3 > > The way I thought about doing this is as follows: > > from collections import deque > from itertools import cycle > > l1 = deque(['a1', 'a2', 'a3', 'a4']) > l2 = deque(['b1', 'b2', 'b3', 'b4']) > l3 = deque(['c1', 'c2', 'c3', 'c4']) > l4 = deque(['d1', 'd2', 'd3', 'd4']) > > l1.rotate(-0) > l2.rotate(-1) > l3.rotate(-2) > l4.rotate(-3) > > groups = cycle([l1, l2, l3, l4]) > > In [115]: for group in groups: > .....: if not group: > .....: break > .....: print(group.popleft()) > .....: > a1 > b2 > c3 > d4 > a2 > b3 > c4 > d1 > a3 > b4 > c1 > d2 > a4 > b1 > c2 > d3 > > Prior to this I was mucking around with index counting while looping, and > popping lists out of a deque, popping an item out of the list, and > appending the list back into the deque during each iteration. > > Is there a better/cleaner way to do this? I was hoping for some cool > itertools logic =) > > Thanks! > > > -- > *Pablo* Hello Pablo, If you don't mind using third-party packages you could this quite straighforward with numpy: (Python 2.7) import numpy as np a = np.array([ ('a1', 'a2', 'a3', 'a4'), ('b1', 'b2', 'b3', 'b4'), ('c1', 'c2', 'c3', 'c4'), ('d1', 'd2', 'd3', 'd4')], dtype=str) nrows, ncols = a.shape for i in range(ncols): print np.diag(np.roll(a, -i, axis=1)) This prints: ['a1' 'b2' 'c3' 'd4'] ['a2' 'b3' 'c4' 'd1'] ['a3' 'b4' 'c1' 'd2'] ['a4' 'b1' 'c2' 'd3'] From oscar.j.benjamin at gmail.com Fri Feb 26 09:39:43 2016 From: oscar.j.benjamin at gmail.com (Oscar Benjamin) Date: Fri, 26 Feb 2016 14:39:43 +0000 Subject: Sorry, here are the details. In-Reply-To: <56cf3629.46f0420a.503bd.ffffe2f8@mx.google.com> References: <56c81718.0207620a.f9e33.17c0@mx.google.com> <56cf3629.46f0420a.503bd.ffffe2f8@mx.google.com> Message-ID: Please reply to the list rather than directly to me (and then you will get a quicker response from someone else). On 25 February 2016 at 17:13, Giriprasadh Raghavan wrote: > I open the setup file of python 3.5.1 and click then install button. Then I > am displayed an error that says: > > One or more issues caused the setup to fail. Please fix the issues an then > retry the setup. For more info see the log file. > > 0x80070570 ? The file or directory is corrupted and unreadable. > > (I actually downloaded it for the 3rd time as it said the file or directory > is corrupted and unreadable.) This problem (0x80070570) has been reported a few times: https://mail.python.org/pipermail/python-list/2015-September/697447.html https://mail.python.org/pipermail/python-list/2016-January/702590.html https://mail.python.org/pipermail/python-list/2015-December/700998.html However I haven't seen a fix for it. It is apparently due to problems with corrupted files or with broken hard disks or something. I don't remember hearing of this problem before Python 3.5 and AFAIK it only occurs with 3.5. Could you try installing 3.4 and let us know if that works? If so it may be a bug (I don't see any tracker issues for it yet). -- Oscar From oscar.j.benjamin at gmail.com Fri Feb 26 09:49:16 2016 From: oscar.j.benjamin at gmail.com (Oscar Benjamin) Date: Fri, 26 Feb 2016 14:49:16 +0000 Subject: subprocess startup error In-Reply-To: References: Message-ID: On 26 February 2016 at 13:30, Peter Otten <__peter__ at web.de> wrote: > Shweta Dinnimani wrote: >>> >>> hello, I'm begineer to python programming.. I had installed python 3.5.1 >>> version on my windows 7 system. I was fine earlier and now when i was >>> trying the programs on string i'm facing the subprocess startup error. >>> IDLE is not connecting. And python shell is also not opening. I tried >>> uninstalling and installing the python shell but Im facing the >>> problem.Please do help me > >> forgot to mention.. >> >> i saved my file as string.py since than i'm facing this error > > Rename that file to something that does not clash with the module names in > the standard library, mystring.py for example, and IDLE should work again. It's ridiculous that it's this easy to accidentally crash IDLE and the Python shell. -- Oscar From info at wingware.com Fri Feb 26 09:49:25 2016 From: info at wingware.com (Wingware) Date: Fri, 26 Feb 2016 09:49:25 -0500 Subject: ANN: Wing IDE 5.1.10 released Message-ID: <56D065F5.6050602@wingware.com> Hi, Wingware has released version 5.1.10 of Wing IDE, our cross-platform integrated development environment for the Python programming language. Wing IDE features a professional code editor with vi, emacs, visual studio, and other key bindings, auto-completion, call tips, context-sensitive auto-editing, goto-definition, find uses, refactoring, a powerful debugger, version control, unit testing, search, project management, and many other features. This release includes the following minor improvements: Fix issues with Django app creation French localization updates (thanks to Jean Sanchez) Improve analysis cache performance Don't specify Qt dependencies in Debian packages Fix occasional hangs when debugging multi-threaded code Fix display of decimal instances in stack data Don't hide the auto-completer when a debug process exits or on other events that cause update of menus and toolbars Don't show duplicate entries for the same file in Compare to Repository Fix crash triggered by some mako files Fix problem with inspection of bound methods in Python 3 12 other bug fixes For details see http://wingware.com/news/2016-02-25 and http://wingware.com/pub/wingide/5.1.10/CHANGELOG.txt What's New in Wing 5.1: Wing IDE 5.1 adds multi-process and child process debugging, syntax highlighting in the shells, support for pytest, Find Symbol in Project, persistent time-stamped unit test results, auto-conversion of indents on paste, an XCode keyboard personality, support for Flask, Django 1.7, 1.8, and 1.9, Python 3.5 and recent Google App Engine versions, improved auto-completion for PyQt, recursive snippet invocation, and many other minor features and improvements. Free trial: http://wingware.com/wingide/trial Downloads: http://wingware.com/downloads Feature list: http://wingware.com/wingide/features Sales: http://wingware.com/store/purchase Upgrades: https://wingware.com/store/upgrade Questions? Don't hesitate to email us at support at wingware.com. Thanks, -- Stephan Deibel Wingware | Python IDE The Intelligent Development Environment for Python Programmers wingware.com From rgacote at appropriatesolutions.com Fri Feb 26 09:52:12 2016 From: rgacote at appropriatesolutions.com (Ray Cote) Date: Fri, 26 Feb 2016 09:52:12 -0500 Subject: Looking for examples of using aiopg with aiohttp and non-async startup. In-Reply-To: References: Message-ID: Answer (obvious after a refreshing sleep): Just run a separate async pool connection prior to kicking off the aiohttp web application. The startup now looks like: async def connect(): return await aiopg.create_pool(?) if __name__ == ?__main__?: loop = asyncio.get_event_loop() pool = loop.run_until_complete(connect()) app = web.Application() app["pool"] = pool app.router.add_route(?POST', '/pv/v1/', handle_v1) web.run_app(app) Then, in the handle_v1 code: pool = request.app["pool"] connection = await pool.acquire() cursor = await connection.cursor() ?r On Thu, Feb 25, 2016 at 5:23 PM, Ray Cote wrote: > Hello: > > I have an aiohttp project that starts in the usual way: > > app = web.Application() > app.router.add_route(?POST?, ?/?, handler) > web.run_app(app) > > My question is, how do I work with an aiopg.pool with aiohttp? > There only seems to be async interfaces into aiopg ? but I don?t want to > create the pool in my handler since that requires a connection on each > transaction. > > I found one example that was > app[?db?] = await aiopg.create_pool(dsn) > but that doesn?t seen correct since we?re not yet in a loop. > > Can someone please provide an example showing the proper way to integrate > aiopg pool into an aiohttp web application? > > Regards > ?Ray > > -- > Raymond Cote, President > voice: +1.603.924.6079 email: rgacote at AppropriateSolutions.com skype: > ray.cote > > > -- Raymond Cote, President voice: +1.603.924.6079 email: rgacote at AppropriateSolutions.com skype: ray.cote From emile at fenx.com Fri Feb 26 09:57:09 2016 From: emile at fenx.com (Emile van Sebille) Date: Fri, 26 Feb 2016 06:57:09 -0800 Subject: subprocess startup error In-Reply-To: References: Message-ID: On 2/26/2016 6:49 AM, Oscar Benjamin wrote: > On 26 February 2016 at 13:30, Peter Otten <__peter__ at web.de> wrote: >> Shweta Dinnimani wrote: >>> i saved my file as string.py since than i'm facing this error >> >> Rename that file to something that does not clash with the module names in >> the standard library, mystring.py for example, and IDLE should work again. > > It's ridiculous that it's this easy to accidentally crash IDLE and the > Python shell. I don't think there's a way to protect against it -- how could you work on string.py otherwise? Emile From arsh840 at gmail.com Fri Feb 26 10:04:44 2016 From: arsh840 at gmail.com (Arshpreet Singh) Date: Fri, 26 Feb 2016 07:04:44 -0800 (PST) Subject: Pyraf problem In-Reply-To: References: <705629636.351550.1456410947390.JavaMail.zimbra@aries.res.in> Message-ID: On Thursday, 25 February 2016 20:07:52 UTC+5:30, Sapna Mishra wrote: > Dear Sir/Mam, > > I am using python for my astronomy purpose, for that I want to use PyRaf, but strange thing is occurring that pyraf is getting open as a root but out side my root user directory when I am typing pyraf I am getting like; > No graphics/display possible for this session. > Tkinter import failed. > The numpy package is required by PyRAF and was not found. Please visit http://numpy.scipy.org Hi Sapna, You can't run directly X-window system from root account, SO you are getting Tkinter error. You can try to install as follows: sudo pip install pyraf But befoore that you also need to install numpy as it is required for pyraf or scipy too. SO you need to run sudo pip install numpy scipy Above instructions will work only for Python2.X From wolfgang.maier at biologie.uni-freiburg.de Fri Feb 26 10:08:17 2016 From: wolfgang.maier at biologie.uni-freiburg.de (Wolfgang Maier) Date: Fri, 26 Feb 2016 16:08:17 +0100 Subject: subprocess startup error In-Reply-To: References: Message-ID: <56D06A61.3030008@biologie.uni-freiburg.de> On 26.02.2016 15:57, Emile van Sebille wrote: > On 2/26/2016 6:49 AM, Oscar Benjamin wrote: >> On 26 February 2016 at 13:30, Peter Otten <__peter__ at web.de> wrote: >>> Shweta Dinnimani wrote: >>>> i saved my file as string.py since than i'm facing this error >>> >>> Rename that file to something that does not clash with the module >>> names in >>> the standard library, mystring.py for example, and IDLE should work >>> again. >> >> It's ridiculous that it's this easy to accidentally crash IDLE and the >> Python shell. > > I don't think there's a way to protect against it -- how could you work > on string.py otherwise? > Emitting a warning message in such situations is being discussed in http://bugs.python.org/issue23809 though it is not a very lively discussion. From wolfgang.maier at biologie.uni-freiburg.de Fri Feb 26 10:08:17 2016 From: wolfgang.maier at biologie.uni-freiburg.de (Wolfgang Maier) Date: Fri, 26 Feb 2016 16:08:17 +0100 Subject: subprocess startup error In-Reply-To: References: Message-ID: <56D06A61.3030008@biologie.uni-freiburg.de> On 26.02.2016 15:57, Emile van Sebille wrote: > On 2/26/2016 6:49 AM, Oscar Benjamin wrote: >> On 26 February 2016 at 13:30, Peter Otten <__peter__ at web.de> wrote: >>> Shweta Dinnimani wrote: >>>> i saved my file as string.py since than i'm facing this error >>> >>> Rename that file to something that does not clash with the module >>> names in >>> the standard library, mystring.py for example, and IDLE should work >>> again. >> >> It's ridiculous that it's this easy to accidentally crash IDLE and the >> Python shell. > > I don't think there's a way to protect against it -- how could you work > on string.py otherwise? > Emitting a warning message in such situations is being discussed in http://bugs.python.org/issue23809 though it is not a very lively discussion. From breamoreboy at yahoo.co.uk Fri Feb 26 10:47:16 2016 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Fri, 26 Feb 2016 15:47:16 +0000 Subject: subprocess startup error In-Reply-To: <56D06A61.3030008@biologie.uni-freiburg.de> References: <56D06A61.3030008@biologie.uni-freiburg.de> Message-ID: On 26/02/2016 15:08, Wolfgang Maier wrote: > On 26.02.2016 15:57, Emile van Sebille wrote: >> On 2/26/2016 6:49 AM, Oscar Benjamin wrote: >>> On 26 February 2016 at 13:30, Peter Otten <__peter__ at web.de> wrote: >>>> Shweta Dinnimani wrote: >>>>> i saved my file as string.py since than i'm facing this error >>>> >>>> Rename that file to something that does not clash with the module >>>> names in >>>> the standard library, mystring.py for example, and IDLE should work >>>> again. >>> >>> It's ridiculous that it's this easy to accidentally crash IDLE and the >>> Python shell. >> >> I don't think there's a way to protect against it -- how could you work >> on string.py otherwise? >> > > Emitting a warning message in such situations is being discussed in > http://bugs.python.org/issue23809 though it is not a very lively > discussion. > Thanks for the link. I knew that I remembered a discussion about this but frustratingly couldn't find it for myself :( -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From lbolla at gmail.com Fri Feb 26 11:43:28 2016 From: lbolla at gmail.com (Lorenzo Bolla) Date: Fri, 26 Feb 2016 08:43:28 -0800 (PST) Subject: Reduce memory fragmentation with MALLOC_MMAP_THRESHOLD_ and MALLOC_MMAP_MAX_ Message-ID: <39f8429e-9a0a-4045-aa2e-7dfae2c82496@googlegroups.com> Hi all, I've been experimenting with MALLOC_MMAP_THRESHOLD_ and MALLOC_MMAP_MAX_ env variables to affect memory management in a long-running Python 2 process. See http://man7.org/linux/man-pages/man3/mallopt.3.html I got the idea from this bug report: http://bugs.python.org/issue11849 The results I have are encouraging: memory fragmentation is reduced and the typical high-water mark visible in memory used by long-running processes is lower. My only concern is if there are other side effects that may bite back, when using such low level tweaks. Does anyone have any experience in using them? Here is an example script that shows how those variables affect RSS memory in a script that generate a large dictionary: https://gist.github.com/lbolla/8e2640133032b0a6bb9c Just run "alloc.sh" and compare the output. Thanks, L. From anitagoyal571 at gmail.com Fri Feb 26 14:35:58 2016 From: anitagoyal571 at gmail.com (Anita Goyal) Date: Fri, 26 Feb 2016 11:35:58 -0800 (PST) Subject: The Real-Time Use of Python in Data Science World! Message-ID: <709dbf52-4b56-4bed-946d-0e6ecba59874@googlegroups.com> This course will help you to expertise the usage of Python in Data Science world. Carter your Python Knowledge so that it can be utilized to get the Insights of Data using Methodologies and Techniques of Data Science... Objective: Understand the concepts of Data science and Python You will be able to use Python in Discovering Data. You will have an idea of Statistical and Analytical methods to deal with huge data sets. You will gain an expertise on Regular Expressions, looping functions and concepts of Object Oriented Programming. You will be able to create business algorithms and data models using Python and it's techniques. Work on Real-life Projects will help you to get a practical experience of real scenarios of IT Industry. Start learning Python for Data Science from basics to advance levels here... https://goo.gl/070wXw From joel.goldstick at gmail.com Fri Feb 26 14:47:59 2016 From: joel.goldstick at gmail.com (Joel Goldstick) Date: Fri, 26 Feb 2016 14:47:59 -0500 Subject: The Real-Time Use of Python in Data Science World! In-Reply-To: <709dbf52-4b56-4bed-946d-0e6ecba59874@googlegroups.com> References: <709dbf52-4b56-4bed-946d-0e6ecba59874@googlegroups.com> Message-ID: On Fri, Feb 26, 2016 at 2:35 PM, Anita Goyal wrote: > This course will help you to expertise the usage of Python in Data Science > world. > > Carter your Python Knowledge so that it can be utilized to get the > Insights of Data using Methodologies and Techniques of Data Science... > > Objective: > Understand the concepts of Data science and Python > You will be able to use Python in Discovering Data. > You will have an idea of Statistical and Analytical methods to deal with > huge data sets. > You will gain an expertise on Regular Expressions, looping functions and > concepts of Object Oriented Programming. > You will be able to create business algorithms and data models using > Python and it's techniques. > Work on Real-life Projects will help you to get a practical experience of > real scenarios of IT Industry. > > Start learning Python for Data Science from basics to advance levels > here... > https://goo.gl/070wXw > -- > https://mail.python.org/mailman/listinfo/python-list > Second post by this person. Both advertising classes for a fee. Is this proper posting? I tend to think it is spam -- Joel Goldstick http://joelgoldstick.com/ http://cc-baseballstats.info/ From best_lay at yahoo.com Fri Feb 26 16:17:26 2016 From: best_lay at yahoo.com (Wildman) Date: Fri, 26 Feb 2016 15:17:26 -0600 Subject: Request Help With Displaying XBM Image Message-ID: As part of a program I am working on I want to display a 48x48 XBM image on the main window. I have done a lot of searching for code snippets and I found what appears to be the correct way to do it using a Label. So far I have not been able to get it to work. I have tried different variations of the syntax but nothing will display the image. The Label will expand to size of the image but nothing is there. I have the background of the Label set to white so it can be seen. For experimentation I create a small program for the sole purpose of displaying an XBM image. The complete script is pasted below. I am thinking the problem might have something with the way the image object is being created. I'm probable making a dumb newbie mistake. BTW, if I run the script from a terminal window, I do not get any errors. Any guidance is appreciated. #!/usr/bin/python from Tkinter import * import tkFileDialog, os, Image, ImageTk class cv(): default_dir = os.environ['HOME'] class Window(Frame): def __init__(self, master = None): Frame.__init__(self,master) self.master = master self.init_window() def init_window(self): self.master.title("XBM Image Test") self.pack(fill=BOTH, expand=1) self.openButton = Button(self, text="Open Image", width = 10, command=self.open_image) self.openButton.place(x=50, y=100) self.quitButton = Button(self, text="Quit", width=10, command=self.quit) self.quitButton.place(x=50, y=140) self.xbmImage = Label(self, bg="white") self.xbmImage.place(x=80, y=25) def open_image(self): file_filter = [ ('X BitMap', '*.xbm *.XBM'), ('all files', '*.*') ] fileName = tkFileDialog.askopenfilename(parent=root, initialdir=cv.default_dir, filetypes=file_filter, title="Open XBM Image") if fileName: cv.default_dir = os.path.dirname(fileName) openImage = Image.open(fileName) imageFile = ImageTk.BitmapImage(openImage) self.xbmImage.config(image=imageFile) else: return None root = Tk() root.minsize(width=200, height=180) root.maxsize(width=200, height=180) app = Window(root) root.mainloop() -- GNU/Linux user #557453 May the Source be with you. From best_lay at yahoo.com Fri Feb 26 16:38:37 2016 From: best_lay at yahoo.com (Wildman) Date: Fri, 26 Feb 2016 15:38:37 -0600 Subject: The Real-Time Use of Python in Data Science World! References: <709dbf52-4b56-4bed-946d-0e6ecba59874@googlegroups.com> Message-ID: On Fri, 26 Feb 2016 11:35:58 -0800, Anita Goyal wrote: > This course (...) What!? No peas? I like peas with spam. -- GNU/Linux user #557453 The cow died so I don't need your bull! From __peter__ at web.de Fri Feb 26 16:49:58 2016 From: __peter__ at web.de (Peter Otten) Date: Fri, 26 Feb 2016 22:49:58 +0100 Subject: Request Help With Displaying XBM Image References: Message-ID: Wildman via Python-list wrote: > As part of a program I am working on I want to display a > 48x48 XBM image on the main window. I have done a lot > of searching for code snippets and I found what appears > to be the correct way to do it using a Label. So far I > have not been able to get it to work. I have tried > different variations of the syntax but nothing will > display the image. The Label will expand to size of > the image but nothing is there. I have the background > of the Label set to white so it can be seen. > > For experimentation I create a small program for the > sole purpose of displaying an XBM image. The complete > script is pasted below. I am thinking the problem > might have something with the way the image object > is being created. I'm probable making a dumb newbie > mistake. BTW, if I run the script from a terminal > window, I do not get any errors. Any guidance is > appreciated. It's not you, the program as you wrote it should and would show the image, were it not for an odd quirk in how images are handled in tkinter: You have to keep an explicit reference of the Image to prevent it from being garbage-collected. Changing open_image() as follows > def open_image(self): > file_filter = [ > ('X BitMap', '*.xbm *.XBM'), > ('all files', '*.*') > ] > fileName = tkFileDialog.askopenfilename(parent=root, > initialdir=cv.default_dir, > filetypes=file_filter, > title="Open XBM Image") > if fileName: > cv.default_dir = os.path.dirname(fileName) > openImage = Image.open(fileName) self.imageFile = ImageTk.BitmapImage(openImage) self.xbmImage.config(image=self.imageFile) > else: > return None should achieve that. From srkunze at mail.de Fri Feb 26 17:08:41 2016 From: srkunze at mail.de (Sven R. Kunze) Date: Fri, 26 Feb 2016 23:08:41 +0100 Subject: Bug in Python? Message-ID: <56D0CCE9.2000301@mail.de> Hi everybody, I recognized the following oddity (background story: http://srkunze.blogspot.com/2016/02/lets-go-down-rabbit-hole.html). Python sometimes seems not to hop back and forth between C and Python code. Can somebody explain this? class MyList(list): count = 0 def __setitem__(self, key, value): self.count += 1 super(MyList, self).__setitem__(key, value) # using heapq directly from heapq import heappop ml = MyList(range(10)) heappop(ml) # that's C print(ml.count) # print 0 # using exact copy from heapq from heapq import _siftup def my_heappop(heap): lastelt = heap.pop() if heap: returnitem = heap[0] heap[0] = lastelt _siftup(heap, 0) # that's C return returnitem return lastelt ml = MyList(range(10)) my_heappop(ml) print(ml.count) # print 6 Best, Sven From ian.g.kelly at gmail.com Fri Feb 26 17:37:15 2016 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Fri, 26 Feb 2016 15:37:15 -0700 Subject: Bug in Python? In-Reply-To: <56D0CCE9.2000301@mail.de> References: <56D0CCE9.2000301@mail.de> Message-ID: On Fri, Feb 26, 2016 at 3:08 PM, Sven R. Kunze wrote: > Python sometimes seems not to hop back and forth between C and Python code. C code as a rule tends to ignore dunder methods. Those are used to implement Python operations, not C operations. > _siftup(heap, 0) # that's C Your comment here appears to be incorrect. >>> from heapq import _siftup >>> type(_siftup) >>> import inspect >>> print(inspect.getsource(_siftup)) def _siftup(heap, pos): endpos = len(heap) startpos = pos newitem = heap[pos] # Bubble up the smaller child until hitting a leaf. childpos = 2*pos + 1 # leftmost child position while childpos < endpos: # Set childpos to index of smaller child. rightpos = childpos + 1 if rightpos < endpos and not heap[childpos] < heap[rightpos]: childpos = rightpos # Move the smaller child up. heap[pos] = heap[childpos] pos = childpos childpos = 2*pos + 1 # The leaf at pos is empty now. Put newitem there, and bubble it up # to its final resting place (by sifting its parents down). heap[pos] = newitem _siftdown(heap, startpos, pos) So I would guess that the difference here is because one implementation is entirely C, and the other implementation is entirely Python. From best_lay at yahoo.com Fri Feb 26 17:37:18 2016 From: best_lay at yahoo.com (Wildman) Date: Fri, 26 Feb 2016 16:37:18 -0600 Subject: Request Help With Displaying XBM Image References: Message-ID: On Fri, 26 Feb 2016 22:49:58 +0100, Peter Otten wrote: > Wildman via Python-list wrote: > It's not you, the program as you wrote it should and would show the image, > were it not for an odd quirk in how images are handled in tkinter: > > You have to keep an explicit reference of the Image to prevent it from being > garbage-collected. Changing open_image() as follows > >> def open_image(self): >> file_filter = [ >> ('X BitMap', '*.xbm *.XBM'), >> ('all files', '*.*') >> ] >> fileName = tkFileDialog.askopenfilename(parent=root, >> initialdir=cv.default_dir, >> filetypes=file_filter, >> title="Open XBM Image") >> if fileName: >> cv.default_dir = os.path.dirname(fileName) >> openImage = Image.open(fileName) > self.imageFile = ImageTk.BitmapImage(openImage) > self.xbmImage.config(image=self.imageFile) >> else: >> return None > > should achieve that. You, Sir, are a scholar and a gentleman. Your change worked perfectly. A thousand thank-you's. -- GNU/Linux user #557453 "Be at war with your vices, at peace with your neighbors, and let every new year find you a better man." -Benjamin Franklin From eryksun at gmail.com Fri Feb 26 18:07:20 2016 From: eryksun at gmail.com (eryk sun) Date: Fri, 26 Feb 2016 17:07:20 -0600 Subject: Bug in Python? In-Reply-To: <56D0CCE9.2000301@mail.de> References: <56D0CCE9.2000301@mail.de> Message-ID: On Fri, Feb 26, 2016 at 4:08 PM, Sven R. Kunze wrote: > Python sometimes seems not to hop back and forth between C and Python code. > Can somebody explain this? Normally a C extension would call PySequence_SetItem, which would call the type's sq_ass_item, which for MyList is slot_sq_ass_item. The latter function bridges the CPython and Python sides by binding and calling the overridden __setitem__ method. However, the _heapq extension module uses `PyList_SET_ITEM(heap, 0, lastelt)`. This macro expands to `((PyListObject *)(heap))->ob_item[0] = lastelt`. This directly modifies the internal ob_item array of the list, so the overridden __setitem__ method is never called. I presume it was implemented like this with performance in mind, but I don't know whether or not that justifies the loss of generality. From eryksun at gmail.com Fri Feb 26 18:29:54 2016 From: eryksun at gmail.com (eryk sun) Date: Fri, 26 Feb 2016 17:29:54 -0600 Subject: Bug in Python? In-Reply-To: References: <56D0CCE9.2000301@mail.de> Message-ID: On Fri, Feb 26, 2016 at 4:37 PM, Ian Kelly wrote: > So I would guess that the difference here is because one > implementation is entirely C, and the other implementation is entirely > Python. Exactly, the C implementation of siftup is only called internally. So there's no need to export it as a function. Calling the C heappop calls this internal siftup implementation, which uses the _PyList_ITEMS macro to reference the list's internal ob_item array. Thus all of the assignment operations party directly on the array instead of using the abstract API. From mentificium at gmail.com Fri Feb 26 21:21:26 2016 From: mentificium at gmail.com (mentificium at gmail.com) Date: Fri, 26 Feb 2016 18:21:26 -0800 (PST) Subject: Bug in Python? In-Reply-To: References: Message-ID: <18dc5c03-6ad0-4a87-8f26-26474cd18504@googlegroups.com> On Friday, February 26, 2016 at 2:09:07 PM UTC-8, Sven R. Kunze wrote: > Hi everybody, > > I recognized the following oddity (background story: > http://srkunze.blogspot.com/2016/02/lets-go-down-rabbit-hole.html). > > Python sometimes seems not to hop back and forth between C and Python code. > [...] http://srkunze.blogspot.com/2016/02/lets-go-down-rabbit-hole.html ? If there are too many bugs in Python, you could switch to Perl (Perl6 just came out :-) especially for artificial intelligence (Strong AI) and for webservers running a "Ghost" AI Mind. See http://www.sourcecodeonline.com/details/ghost_perl_webserver_strong_ai.html or you could use "AI Steps" to code Strong AI in Python. Arthur -- http://ai.neocities.org/AiSteps.html http://www.amazon.com/dp/B00FKJY1WY http://mind.sourceforge.net/python.html http://www.sourcecodeonline.com/details/ghost_perl_webserver_strong_ai.html From tjreedy at udel.edu Sat Feb 27 03:33:14 2016 From: tjreedy at udel.edu (Terry Reedy) Date: Sat, 27 Feb 2016 03:33:14 -0500 Subject: subprocess startup error In-Reply-To: References: Message-ID: On 2/26/2016 9:49 AM, Oscar Benjamin wrote: > On 26 February 2016 at 13:30, Peter Otten <__peter__ at web.de> wrote: >> Shweta Dinnimani wrote: >>>> >>>> hello, I'm begineer to python programming.. I had installed python 3.5.1 >>>> version on my windows 7 system. I was fine earlier and now when i was >>>> trying the programs on string i'm facing the subprocess startup error. >>>> IDLE is not connecting. And python shell is also not opening. I tried >>>> uninstalling and installing the python shell but Im facing the >>>> problem.Please do help me What do you mean by 'python shell' and how do you uninstall and install it? >>> forgot to mention.. >>> >>> i saved my file as string.py since than i'm facing this error >> >> Rename that file to something that does not clash with the module names in >> the standard library, mystring.py for example, and IDLE should work again. > > It's ridiculous that it's this easy to accidentally crash IDLE and the > Python shell. I agree and hope to fix this for IDLE before the next round of releases. Any Python app that does *not* want to import non-stdlib files on startup must remove '' from sys.path. Console python should not be affected by files in the current working directory. -- Terry Jan Reedy From tjreedy at udel.edu Sat Feb 27 03:38:27 2016 From: tjreedy at udel.edu (Terry Reedy) Date: Sat, 27 Feb 2016 03:38:27 -0500 Subject: subprocess startup error In-Reply-To: <56D06A61.3030008@biologie.uni-freiburg.de> References: <56D06A61.3030008@biologie.uni-freiburg.de> Message-ID: On 2/26/2016 10:08 AM, Wolfgang Maier wrote: > On 26.02.2016 15:57, Emile van Sebille wrote: >> On 2/26/2016 6:49 AM, Oscar Benjamin wrote: >>> On 26 February 2016 at 13:30, Peter Otten <__peter__ at web.de> wrote: >>>> Shweta Dinnimani wrote: >>>>> i saved my file as string.py since than i'm facing this error >>>> >>>> Rename that file to something that does not clash with the module >>>> names in >>>> the standard library, mystring.py for example, and IDLE should work >>>> again. >>> >>> It's ridiculous that it's this easy to accidentally crash IDLE and the >>> Python shell. >> >> I don't think there's a way to protect against it -- how could you work >> on string.py otherwise? >> > > Emitting a warning message in such situations is being discussed in > http://bugs.python.org/issue23809 though it is not a very lively > discussion. This is a separate issue of user code importing a user's own file instead of an stdlib file. -- Terry Jan Reedy From tjreedy at udel.edu Sat Feb 27 03:55:32 2016 From: tjreedy at udel.edu (Terry Reedy) Date: Sat, 27 Feb 2016 03:55:32 -0500 Subject: Bug in Python? In-Reply-To: References: <56D0CCE9.2000301@mail.de> Message-ID: On 2/26/2016 6:07 PM, eryk sun wrote: > On Fri, Feb 26, 2016 at 4:08 PM, Sven R. Kunze wrote: >> Python sometimes seems not to hop back and forth between C and Python code. >> Can somebody explain this? > > Normally a C extension would call PySequence_SetItem, which would call > the type's sq_ass_item, which for MyList is slot_sq_ass_item. The > latter function bridges the CPython and Python sides by binding and > calling the overridden __setitem__ method. However, the _heapq > extension module uses `PyList_SET_ITEM(heap, 0, lastelt)`. This macro > expands to `((PyListObject *)(heap))->ob_item[0] = lastelt`. This > directly modifies the internal ob_item array of the list, so the > overridden __setitem__ method is never called. I presume it was > implemented like this with performance in mind, but I don't know > whether or not that justifies the loss of generality. In other words, when that doc says *list*, it means a *list*. "To create a heap, use a list initialized to [], or you can transform a populated list into a heap via function heapify()." Note: when the doc says 'dict' rather than 'dictionary' or 'mapping', it is pretty obvious it means builtin dict. It may sometimes say 'dict or dict subclass'. Ditto for 'str' versus 'string' or 'text'. However, 'list' (Python builtin) and generic 'list are easy to confuse. In this case, 'initialized to []' is a hint. However, the doc could be made clearer. How about "A heap must be an instance of *list* (and not a subclass thereof). To create a heap, start with [] or transform an existing list into a heap via function heapify()." *list* means to display it blue, linked to the list class. -- Terry Jan Reedy From steve at pearwood.info Sat Feb 27 04:13:12 2016 From: steve at pearwood.info (Steven D'Aprano) Date: Sat, 27 Feb 2016 20:13:12 +1100 Subject: The Real-Time Use of Python in Data Science World! References: <709dbf52-4b56-4bed-946d-0e6ecba59874@googlegroups.com> Message-ID: <56d168aa$0$1612$c3e8da3$5496439d@news.astraweb.com> On Sat, 27 Feb 2016 06:47 am, Joel Goldstick wrote: > On Fri, Feb 26, 2016 at 2:35 PM, Anita Goyal > wrote: > >> This course will help you to expertise the usage of Python in Data >> Science world. [...] > Second post by this person. Both advertising classes for a fee. Is this > proper posting? I tend to think it is spam That's a tricky one. It's on-topic and reasonable honest (no obfuscated text, and although they don't explicitly state there is a fee it should be obvious that most courses that aren't labelled as free do cost money). It is advertising, but advertising isn't necessarily evil. And it's being sent to a public list, rather than spammed out to millions of individual email addresses. I think it's in a grey area, but so long as the sender doesn't abuse the privilege by flooding the group with advertising, it falls just slightly on the "acceptable" side of the line. But I wouldn't be surprised if some people disagree. I think it would be more acceptable to me if the sender labelled the subject line as "Advertising". -- Steven From tjreedy at udel.edu Sat Feb 27 04:14:31 2016 From: tjreedy at udel.edu (Terry Reedy) Date: Sat, 27 Feb 2016 04:14:31 -0500 Subject: {off topic] Re: Bug in Python? In-Reply-To: <18dc5c03-6ad0-4a87-8f26-26474cd18504@googlegroups.com> References: <18dc5c03-6ad0-4a87-8f26-26474cd18504@googlegroups.com> Message-ID: On 2/26/2016 9:21 PM, mentificium at gmail.com wrote: > On Friday, February 26, 2016 at 2:09:07 PM UTC-8, Sven R. Kunze wrote: >> Hi everybody, >> >> I recognized the following oddity (background story: >> http://srkunze.blogspot.com/2016/02/lets-go-down-rabbit-hole.html). >> >> Python sometimes seems not to hop back and forth between C and Python code. >> [...] > > http://srkunze.blogspot.com/2016/02/lets-go-down-rabbit-hole.html ? > > If there are too many bugs in Python, As it turns out, there was no bug, just a misunderstanding of heapq and its doc. > you could switch to Perl (Perl6 just came out :-) > especially for artificial intelligence (Strong AI) > and for webservers running a "Ghost" AI Mind. See > > http://www.sourcecodeonline.com/details/ghost_perl_webserver_strong_ai.html > > or you could use "AI Steps" to code Strong AI in Python. > > Arthur For those who don't know, Arthur has be working on and promoting on his 'AI Mind' project for at least a decade, probably more. AFAIK, it has been a few years since his last off-topic response like this. Arthur, please promote you once-Forth, now Perl project elsewhere. Everyone else, please don't respond further on python-list. -- Terry Jan Reedy From steve at pearwood.info Sat Feb 27 04:16:43 2016 From: steve at pearwood.info (Steven D'Aprano) Date: Sat, 27 Feb 2016 20:16:43 +1100 Subject: Redmonk popularity rankings place Python at #4 Message-ID: <56d1697c$0$1612$c3e8da3$5496439d@news.astraweb.com> The latest popularity rankings from Redmonk are available, based on the number of projects on Github and posts on Stackoverflow, and Python comes in at #4, just behind Javascript, Java and PHP, and just ahead of C#, C++ and Ruby. http://redmonk.com/sogrady/2016/02/19/language-rankings-1-16/ -- Steven From subhabangalore at gmail.com Sat Feb 27 04:17:45 2016 From: subhabangalore at gmail.com (subhabangalore at gmail.com) Date: Sat, 27 Feb 2016 01:17:45 -0800 (PST) Subject: Error in Tree Structure Message-ID: <33c93316-0ae3-4b8f-b2f6-29f76c8b3f32@googlegroups.com> I was trying to implement the code, import nltk import nltk.tag, nltk.chunk, itertools def ieertree2conlltags(tree, tag=nltk.tag.pos_tag): words, ents = zip(*tree.pos()) iobs = [] prev = None for ent in ents: if ent == tree.node: iobs.append('O') prev = None elif prev == ent: iobs.append('I-%s' % ent) else: iobs.append('B-%s' % ent) prev = ent words, tags = zip(*tag(words)) return itertools.izip(words, tags, iobs) def ieer_chunked_sents(tag=nltk.tag.pos_tag): for doc in ieer.parsed_docs(): tagged = ieertree2conlltags(doc.text, tag) yield nltk.chunk.conlltags2tree(tagged) from chunkers import ieer_chunked_sents, ClassifierChunker from nltk.corpus import treebank_chunk ieer_chunks = list(ieer_chunked_sents()) chunker = ClassifierChunker(ieer_chunks[:80]) print chunker.parse(treebank_chunk.tagged_sents()[0]) score = chunker.evaluate(ieer_chunks[80:]) print score.accuracy() It is running fine. But as I am trying to rewrite the code as, chunker = ClassifierChunker(list1), where list1 is same value as, ieer_chunks[:80] only I am pasting the value as [Tree('S', [Tree('LOCATION', [(u'NAIROBI', 'NNP')]), (u',', ','), Tree('LOCATION', [(u'Kenya', 'NNP')]), (u'(', '('), Tree('ORGANIZATION', [(u'AP', 'NNP')]), (u')', ')'), (u'_', 'NNP'), Tree('CARDINAL', [(u'Thousands', 'NNP')]), (u'of', 'IN'), (u'laborers,', 'JJ'), (u'students', 'NNS'), (u'and', 'CC'), (u'opposition', 'NN'), (u'politicians', 'NNS'), (u'on', 'IN'), Tree('DATE', [(u'Saturday', 'NNP')]), (u'protested', 'VBD'), (u'tax', 'NN'), (u'hikes', 'NNS'), (u'imposed', 'VBN'), (u'by', 'IN'), (u'their', 'PRP$'), (u'cash-strapped', 'JJ'), (u'government,', 'NN'), (u'which', 'WDT'), (u'they', 'PRP'), (u'accused', 'VBD'), (u'of', 'IN'), (u'failing', 'VBG'), (u'to', 'TO'), (u'provide', 'VB'), (u'basic', 'JJ'), (u'services.', 'NN'),....(u'(cm-kjd)', 'NN')])] the value of whole list directly I am getting syntax error. I tried to paste it in Python IDE outside code there also it is giving syntax error. If I do not paste the value and and rename ieer_chunks[:80] as list1 there is no error. I may be doing some problem while copying the value and pasting it. But I did not change anything there. Is it any error in Python part or in NLTK part? Thanks in advance. If any one may guide me what is the error I am doing and how may I solve it. From steve at pearwood.info Sat Feb 27 04:37:45 2016 From: steve at pearwood.info (Steven D'Aprano) Date: Sat, 27 Feb 2016 20:37:45 +1100 Subject: [Off-topic] Requests author discusses MentalHealthError exception Message-ID: <56d16e6a$0$1593$c3e8da3$5496439d@news.astraweb.com> The author of Requests, Kenneth Reitz, discusses his recent recovery from a MentalHealthError exception. http://www.kennethreitz.org/essays/mentalhealtherror-an-exception-occurred Although the connection to Python is only quite slim, I found it fascinating to read. -- Steven From vek.m1234 at gmail.com Sat Feb 27 04:39:56 2016 From: vek.m1234 at gmail.com (Veek. M) Date: Sat, 27 Feb 2016 15:09:56 +0530 Subject: Lookahead while doing: for line in fh.readlines(): Message-ID: I want to do something like: #!/usr/bin/env python3 fh = open('/etc/motd') for line in fh.readlines(): print(fh.tell()) why doesn't this work as expected.. fh.readlines() should return a generator object and fh.tell() ought to start at 0 first. Instead i get the final count repeated for the number of lines. What i'm trying to do is lookahead: #!whatever fh = open(whatever) for line in fh.readlines(): x = fh.tell() temp = fh.readline() fh.seek(x) From steve at pearwood.info Sat Feb 27 04:44:30 2016 From: steve at pearwood.info (Steven D'Aprano) Date: Sat, 27 Feb 2016 20:44:30 +1100 Subject: Bug in Python? References: <56D0CCE9.2000301@mail.de> Message-ID: <56d16fff$0$1605$c3e8da3$5496439d@news.astraweb.com> On Sat, 27 Feb 2016 07:55 pm, Terry Reedy wrote: > In other words, when that doc says *list*, it means a *list*. > > "To create a heap, use a list initialized to [], or you can transform a > populated list into a heap via function heapify()." [...] > "A heap must be an instance of *list* (and not a subclass thereof). To > create a heap, start with [] or transform an existing list into a heap > via function heapify()." I think that's a sad decision. heapq ought to be able to handle any list subclass, not just actual lists. Preferably it ought to handle duck-typed lists too, anything with a list-like interface. It is okay if the optimized C version only works with actual lists, and falls back to a slower Python version for anything else. -- Steven From steve at pearwood.info Sat Feb 27 04:49:43 2016 From: steve at pearwood.info (Steven D'Aprano) Date: Sat, 27 Feb 2016 20:49:43 +1100 Subject: How to read from a file to an arbitrary delimiter efficiently? References: <56cea44e$0$11128$c3e8da3@news.astraweb.com> Message-ID: <56d17138$0$1605$c3e8da3$5496439d@news.astraweb.com> On Thu, 25 Feb 2016 06:30 pm, Chris Angelico wrote: > On Thu, Feb 25, 2016 at 5:50 PM, Steven D'Aprano > wrote: >> >> # Read a chunk of bytes/characters from an open file. >> def chunkiter(f, delim): >> buffer = [] >> b = f.read(1) >> while b: >> buffer.append(b) >> if b in delim: >> yield ''.join(buffer) >> buffer = [] >> b = f.read(1) >> if buffer: >> yield ''.join(buffer) > > How bad is it if you over-read? Pretty bad :-) Ideally, I'd rather not over-read at all. I'd like the user to be able to swap from "read N bytes" to "read to the next delimiter" (and possibly even "read the next line") without losing anything. If there's absolutely no other way to speed this up by at least a factor of ten, I'll consider reading into a buffer and losing the ability to mix different kinds of reads. -- Steven From breamoreboy at yahoo.co.uk Sat Feb 27 04:50:38 2016 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Sat, 27 Feb 2016 09:50:38 +0000 Subject: [Off-topic] Requests author discusses MentalHealthError exception In-Reply-To: <56d16e6a$0$1593$c3e8da3$5496439d@news.astraweb.com> References: <56d16e6a$0$1593$c3e8da3$5496439d@news.astraweb.com> Message-ID: On 27/02/2016 09:37, Steven D'Aprano wrote: > The author of Requests, Kenneth Reitz, discusses his recent recovery from a > MentalHealthError exception. > > http://www.kennethreitz.org/essays/mentalhealtherror-an-exception-occurred > > Although the connection to Python is only quite slim, I found it fascinating > to read. > Thanks for the link even if it did touch an extremely raw nerve. -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From steve at pearwood.info Sat Feb 27 05:40:17 2016 From: steve at pearwood.info (Steven D'Aprano) Date: Sat, 27 Feb 2016 21:40:17 +1100 Subject: How to read from a file to an arbitrary delimiter efficiently? References: <56cea44e$0$11128$c3e8da3@news.astraweb.com> Message-ID: <56d17d13$0$1596$c3e8da3$5496439d@news.astraweb.com> On Thu, 25 Feb 2016 06:37 pm, Wolfgang Maier wrote: > On 25.02.2016 07:50, Steven D'Aprano wrote: >> I have a need to read to an arbitrary delimiter, which might be any of a >> (small) set of characters. For the sake of the exercise, lets say it is >> either ! or ? (for example). >> > > You are not alone with your need. > > http://bugs.python.org/issue1152248 discusses the problem and has some > code snippets that you may be interested in. While there is no trivial > solution there are certainly faster ways than your first attempt. Wow. Ten years and still no solution :-( Thanks for finding the issue, but the solutions given don't suit my use case. I don't want an iterator that operates on pre-read blocks, I want something that will read a record from a file, and leave the file pointer one entry past the end of the record. Oh, and records are likely fairly short, but there may be a lot of them. -- Steven From wrong.address.1 at gmail.com Sat Feb 27 06:18:57 2016 From: wrong.address.1 at gmail.com (wrong.address.1 at gmail.com) Date: Sat, 27 Feb 2016 03:18:57 -0800 (PST) Subject: Everything good about Python except GUI IDE? Message-ID: <64a6599c-fae1-469d-bcee-875165b3cc7d@googlegroups.com> I have some VB forms with more than a hundred objects. If I cannot drag and drop text boxes, list boxes, labels, etc., it will be too much work to create that with several lines of code for each object. Isn't there any good GUI IDE like Visual Basic? I hope there are some less well known GUI IDEs which I did not come across. Thanks. From marco.buttu at gmail.com Sat Feb 27 06:25:32 2016 From: marco.buttu at gmail.com (Marco Buttu) Date: Sat, 27 Feb 2016 12:25:32 +0100 Subject: The Real-Time Use of Python in Data Science World! In-Reply-To: <56d168aa$0$1612$c3e8da3$5496439d@news.astraweb.com> References: <709dbf52-4b56-4bed-946d-0e6ecba59874@googlegroups.com> <56d168aa$0$1612$c3e8da3$5496439d@news.astraweb.com> Message-ID: <56D187AC.2040803@oa-cagliari.inaf.it> On 27/02/2016 10:13, Steven D'Aprano wrote: > I think it would be more acceptable to me if the sender > labelled the subject line as "Advertising". +1 -- Marco Buttu INAF-Osservatorio Astronomico di Cagliari Via della Scienza n. 5, 09047 Selargius (CA) Phone: 070 711 80 217 Email: mbuttu at oa-cagliari.inaf.it From steve at pearwood.info Sat Feb 27 06:29:17 2016 From: steve at pearwood.info (Steven D'Aprano) Date: Sat, 27 Feb 2016 22:29:17 +1100 Subject: Error in Tree Structure References: <33c93316-0ae3-4b8f-b2f6-29f76c8b3f32@googlegroups.com> Message-ID: <56d1888f$0$1586$c3e8da3$5496439d@news.astraweb.com> On Sat, 27 Feb 2016 08:17 pm, subhabangalore at gmail.com wrote: > Is it any error in Python part or in NLTK part? Neither. Any time you think there is an error in Python, it is 99.9% sure that the error is in your code, not Python. If the error is a SyntaxError, that is 99.99999%. > If any one may guide me what is the error I am doing and how may I solve > it. Look at the SyntaxError traceback and read what it says. Does it tell you what the error is? Does it use a ^ as an arrow to point to the error, or immediately after the error? Chances are, the error is that you have added or deleted a bracket or parenthesis somewhere. Dealing with nested lists like this is usually awful, because they are unreadable. Can you avoid copying and pasting the nested list? If not, try re-formatting it so you can at least read it: # Unreadable, bad: thelist = [Tree('S', [Tree('LOCATION', [(u'NAIROBI', 'NNP')]), (u',', ','), Tree('LOCATION', [(u'Kenya', 'NNP')]), (u'(', '('), Tree('ORGANIZATION', [(u'AP', 'NNP')]), (u')', ')'), (u'_', 'NNP'), Tree('CARDINAL', [(u'Thousands', 'NNP')]), (u'of', 'IN'), (u'laborers,', 'JJ'), (u'students', 'NNS'), (u'and', 'CC'), (u'opposition', 'NN'), (u'politicians', 'NNS'), (u'on', 'IN'), Tree('DATE', [(u'Saturday', 'NNP')]), (u'protested', 'VBD'), (u'tax', 'NN'), (u'hikes', 'NNS'), (u'imposed', 'VBN'), (u'by', 'IN'), (u'their', 'PRP$'), (u'cash-strapped', 'JJ'), (u'government,', 'NN'), (u'which', 'WDT'), (u'they', 'PRP'), (u'accused', 'VBD'), (u'of', 'IN'), (u'failing', 'VBG'), (u'to', 'TO'), (u'provide', 'VB'), (u'basic', 'JJ'), (u'services.', 'NN'), (u'(cm-kjd)', 'NN')])] # Slightly more readable, good: thelist = [ Tree('S', [ Tree( 'LOCATION', [ (u'NAIROBI', 'NNP') ] ), (u',', ','), Tree( 'LOCATION', [ (u'Kenya', 'NNP') ] ), (u'(', '('), Tree( 'ORGANIZATION', [ (u'AP', 'NNP') ] ), (u')', ')'), (u'_', 'NNP'), Tree( 'CARDINAL', [ (u'Thousands', 'NNP') ] ), (u'of', 'IN'), (u'laborers,', 'JJ'), (u'students', 'NNS'), (u'and', 'CC'), (u'opposition', 'NN'), (u'politicians', 'NNS'), (u'on', 'IN'), Tree( 'DATE', [ (u'Saturday', 'NNP') ] ), (u'protested', 'VBD'), (u'tax', 'NN'), (u'hikes', 'NNS'), (u'imposed', 'VBN'), (u'by', 'IN'), (u'their', 'PRP$'), (u'cash-strapped', 'JJ'), (u'government,', 'NN'), (u'which', 'WDT'), (u'they', 'PRP'), (u'accused', 'VBD'), (u'of', 'IN'), (u'failing', 'VBG'), (u'to', 'TO'), (u'provide', 'VB'), (u'basic', 'JJ'), (u'services.', 'NN'), (u'(cm-kjd)', 'NN') ] ) ] If you format your giant list so you can see the structure, then you will likely find where there is a problem. Perhaps you have: - missing or too many [ ( ) or ] - a missing or extra comma - a missing or extra quotation marks - unexpected symbols like .... inside the list -- Steven From steve at pearwood.info Sat Feb 27 06:36:51 2016 From: steve at pearwood.info (Steven D'Aprano) Date: Sat, 27 Feb 2016 22:36:51 +1100 Subject: Everything good about Python except GUI IDE? References: <64a6599c-fae1-469d-bcee-875165b3cc7d@googlegroups.com> Message-ID: <56d18a56$0$1586$c3e8da3$5496439d@news.astraweb.com> On Sat, 27 Feb 2016 10:18 pm, wrong.address.1 at gmail.com wrote: > I have some VB forms with more than a hundred objects. https://www.appnovation.com/sites/default/files/attachments/yourproduct.jpg Your users probably hate you... *wink* -- Steven From tjreedy at udel.edu Sat Feb 27 06:46:24 2016 From: tjreedy at udel.edu (Terry Reedy) Date: Sat, 27 Feb 2016 06:46:24 -0500 Subject: Lookahead while doing: for line in fh.readlines(): In-Reply-To: References: Message-ID: On 2/27/2016 4:39 AM, Veek. M wrote: > I want to do something like: > > #!/usr/bin/env python3 > > fh = open('/etc/motd') > for line in fh.readlines(): > print(fh.tell()) > > why doesn't this work as expected.. fh.readlines() should return a > generator object and fh.tell() ought to start at 0 first. Not after you have already read some data. Readlines() reads the entire file and splits it into lines. readline reads at least a single block. Reading a single byte or character at a time looking for /n would be too slow, so even after readline, the file pointer will be somewhere past the end of the last line returned. > Instead i get the final count repeated for the number of lines. > > What i'm trying to do is lookahead: > #!whatever > > fh = open(whatever) > for line in fh.readlines(): > x = fh.tell() > temp = fh.readline() > fh.seek(x) > -- Terry Jan Reedy From tjreedy at udel.edu Sat Feb 27 06:48:51 2016 From: tjreedy at udel.edu (Terry Reedy) Date: Sat, 27 Feb 2016 06:48:51 -0500 Subject: Bug in Python? In-Reply-To: <56d16fff$0$1605$c3e8da3$5496439d@news.astraweb.com> References: <56D0CCE9.2000301@mail.de> <56d16fff$0$1605$c3e8da3$5496439d@news.astraweb.com> Message-ID: On 2/27/2016 4:44 AM, Steven D'Aprano wrote: > On Sat, 27 Feb 2016 07:55 pm, Terry Reedy wrote: > >> In other words, when that doc says *list*, it means a *list*. >> >> "To create a heap, use a list initialized to [], or you can transform a >> populated list into a heap via function heapify()." > [...] >> "A heap must be an instance of *list* (and not a subclass thereof). To >> create a heap, start with [] or transform an existing list into a heap >> via function heapify()." > > I think that's a sad decision. heapq ought to be able to handle any list > subclass, not just actual lists. Preferably it ought to handle duck-typed > lists too, anything with a list-like interface. It is okay if the optimized > C version only works with actual lists, and falls back to a slower Python > version for anything else. Propose that on the tracker, after checking previous issues. -- Terry Jan Reedy From rustompmody at gmail.com Sat Feb 27 07:02:44 2016 From: rustompmody at gmail.com (Rustom Mody) Date: Sat, 27 Feb 2016 04:02:44 -0800 (PST) Subject: Everything good about Python except GUI IDE? In-Reply-To: <64a6599c-fae1-469d-bcee-875165b3cc7d@googlegroups.com> References: <64a6599c-fae1-469d-bcee-875165b3cc7d@googlegroups.com> Message-ID: <21a891de-b3b7-46ec-a263-a08487d86c1d@googlegroups.com> On Saturday, February 27, 2016 at 4:49:21 PM UTC+5:30, wrong.a... at gmail.com wrote: > I have some VB forms with more than a hundred objects. If I cannot drag and drop text boxes, list boxes, labels, etc., it will be too much work to create that with several lines of code for each object. > > Isn't there any good GUI IDE like Visual Basic? I hope there are some less well known GUI IDEs which I did not come across. Thanks. Good... I do not know [Do not know much of the area] However there is glade, qtdesigner, wxglade for gtk, qt and wx. How full-featured/bug-free etc I cannot say. If you find that they just dont match up to what windows has on offer, I wont be surprised :-( From rosuav at gmail.com Sat Feb 27 07:07:40 2016 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 27 Feb 2016 23:07:40 +1100 Subject: Everything good about Python except GUI IDE? In-Reply-To: <64a6599c-fae1-469d-bcee-875165b3cc7d@googlegroups.com> References: <64a6599c-fae1-469d-bcee-875165b3cc7d@googlegroups.com> Message-ID: On Sat, Feb 27, 2016 at 10:18 PM, wrote: > I have some VB forms with more than a hundred objects. If I cannot drag and drop text boxes, list boxes, labels, etc., it will be too much work to create that with several lines of code for each object. > > Isn't there any good GUI IDE like Visual Basic? I hope there are some less well known GUI IDEs which I did not come across. Thanks. Sounds like the advantage lies with Python here... Don't make a UI by dragging and dropping that many widgets. If it takes "several lines of code" for each object, it might be worth creating a utility function. For example, a database form might include a good number of fields with associated labels; you could create a function that creates (in a single line) a label and its input field. Or create a list and iterate over it, creating all the widgets as you step through the loop. That's the advantage of code - you can't easily create reusable 'clumps' with a drag-and-drop builder, but in code, it's the exact same thing as any other form of code reuse. ChrisA From rosuav at gmail.com Sat Feb 27 07:17:36 2016 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 27 Feb 2016 23:17:36 +1100 Subject: How to read from a file to an arbitrary delimiter efficiently? In-Reply-To: <56d17138$0$1605$c3e8da3$5496439d@news.astraweb.com> References: <56cea44e$0$11128$c3e8da3@news.astraweb.com> <56d17138$0$1605$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Sat, Feb 27, 2016 at 8:49 PM, Steven D'Aprano wrote: > On Thu, 25 Feb 2016 06:30 pm, Chris Angelico wrote: > >> On Thu, Feb 25, 2016 at 5:50 PM, Steven D'Aprano >> wrote: >>> >>> # Read a chunk of bytes/characters from an open file. >>> def chunkiter(f, delim): >>> buffer = [] >>> b = f.read(1) >>> while b: >>> buffer.append(b) >>> if b in delim: >>> yield ''.join(buffer) >>> buffer = [] >>> b = f.read(1) >>> if buffer: >>> yield ''.join(buffer) >> >> How bad is it if you over-read? > > Pretty bad :-) > > Ideally, I'd rather not over-read at all. I'd like the user to be able to > swap from "read N bytes" to "read to the next delimiter" (and possibly > even "read the next line") without losing anything. If those are the *only* two operations, you should be able to maintain your own buffer. Something like this: class ChunkIter: def __init__(self, f, delim): self.f = f self.delim = re.compile("["+delim+"]") self.buffer = "" def read_to_delim(self): """Return characters up to the next delim, or remaining chars, or "" if at EOF""" while "delimiter not found": *parts, self.buffer = self.delim.split(self.buffer, 1) if parts: return parts[0] b = self.f.read(256) if not b: return self.buffer self.buffer += b def read(self, nbytes): need = nbytes - len(self.buffer) if need > 0: self.buffer += self.f.read(need) ret, self.buffer = self.buffer[:need], self.buffer[need:] return ret It still might over-read from the underlying file, but those extra chars will be available to the read(N) function. ChrisA From rosuav at gmail.com Sat Feb 27 07:18:49 2016 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 27 Feb 2016 23:18:49 +1100 Subject: How to read from a file to an arbitrary delimiter efficiently? In-Reply-To: References: <56cea44e$0$11128$c3e8da3@news.astraweb.com> <56d17138$0$1605$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Sat, Feb 27, 2016 at 11:17 PM, Chris Angelico wrote: >> Ideally, I'd rather not over-read at all. I'd like the user to be able to >> swap from "read N bytes" to "read to the next delimiter" (and possibly >> even "read the next line") without losing anything. > > If those are the *only* two operations, you should be able to maintain > your own buffer. And, I started out by thinking "to next delimiter" and "next line" were the same thing with different delimiters, but then went and coded the delimiter so that wouldn't work. Whatevs. If those are the only *three* operations, the same class with one more method could do it. ChrisA From bc at freeuk.com Sat Feb 27 08:35:23 2016 From: bc at freeuk.com (BartC) Date: Sat, 27 Feb 2016 13:35:23 +0000 Subject: Everything good about Python except GUI IDE? In-Reply-To: <64a6599c-fae1-469d-bcee-875165b3cc7d@googlegroups.com> References: <64a6599c-fae1-469d-bcee-875165b3cc7d@googlegroups.com> Message-ID: On 27/02/2016 11:18, wrong.address.1 at gmail.com wrote: > I have some VB forms with more than a hundred objects. If I cannot drag and drop text boxes, list boxes, labels, etc., it will be too much work to create that with several lines of code for each object. > > Isn't there any good GUI IDE like Visual Basic? I hope there are some less well known GUI IDEs which I did not come across. Thanks. All the dialog boxes I've ever created (if that's what you mean by forms), have been done by writing code. Never with drag and drop. Provided you know what to write in those few lines (which would be the stumbling block for me), then a few hundred lines of script code isn't a lot. And Python is a scripting language. And once one form or part of one is done, you might able to just copy and adapt the code instead of having to write everything from scratch. -- Bartc From maillist at schwertberger.de Sat Feb 27 09:20:48 2016 From: maillist at schwertberger.de (Dietmar Schwertberger) Date: Sat, 27 Feb 2016 15:20:48 +0100 Subject: Everything good about Python except GUI IDE? In-Reply-To: <64a6599c-fae1-469d-bcee-875165b3cc7d@googlegroups.com> References: <64a6599c-fae1-469d-bcee-875165b3cc7d@googlegroups.com> Message-ID: On 27.02.2016 12:18, wrong.address.1 at gmail.com wrote: > Isn't there any good GUI IDE like Visual Basic? I hope there are some less well known GUI IDEs which I did not come across. Thanks. As of today, there's no Python GUI builder comparable to VB 6. There are some like QtDesigner or wxGlade, but they either don't generate Python code directly or they can only be used if you know the underlying toolkit good enough to create the GUI yourself. You may try out some, but I can almost guarantee you that you will come to the same result. If you want a GUI, create it yourself using either wxPython or PyQt. For engineering applications that's probably the weakest point that Python has. It's holding back a lot of people... Well, for most measurement or control software a GUI is not really needed, but still people want it. Regards, Dietmar From miragewebstudio12 at gmail.com Sat Feb 27 09:35:39 2016 From: miragewebstudio12 at gmail.com (MWS) Date: Sat, 27 Feb 2016 20:05:39 +0530 Subject: Everything good about Python except GUI IDE? In-Reply-To: <64a6599c-fae1-469d-bcee-875165b3cc7d@googlegroups.com> References: <64a6599c-fae1-469d-bcee-875165b3cc7d@googlegroups.com> Message-ID: <56D1B43B.8020304@gmail.com> On Saturday 27 February 2016 04:48 PM, wrong.address.1 at gmail.com wrote: > I have some VB forms with more than a hundred objects. If I cannot drag and drop text boxes, list boxes, labels, etc., it will be too much work to create that with several lines of code for each object. > > Isn't there any good GUI IDE like Visual Basic? I hope there are some less well known GUI IDEs which I did not come across. Thanks. Hello, I too faced such a problem, being familiar with basic i too found it difficult to transition from designer to text only, but i found the solution, i.e. qt, try pyqt, for desigining try qt creator, i don't know if qt designer is still installable but qt creator is. you can easily design your forms and generate python code and then integrate them with your program. a little commandline stuff, but you can always batch it up. George From dan at tombstonezero.net Sat Feb 27 09:40:33 2016 From: dan at tombstonezero.net (Dan Sommers) Date: Sat, 27 Feb 2016 14:40:33 -0000 (UTC) Subject: How to read from a file to an arbitrary delimiter efficiently? References: <56cea44e$0$11128$c3e8da3@news.astraweb.com> <56d17d13$0$1596$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Sat, 27 Feb 2016 21:40:17 +1100, Steven D'Aprano wrote: > Thanks for finding the issue, but the solutions given don't suit my > use case. I don't want an iterator that operates on pre-read blocks, I > want something that will read a record from a file, and leave the file > pointer one entry past the end of the record. A file is a stream of bytes, but you want to view it as a stream of records. It sounds like you want an abstraction layer, and it sounds like you also want to let the file leak through that layer when it's convenient. (Yes, I spun that horribly on purpose, and I understand the use case of imposing some structure on part of a file, and possibly a different structure on a different part of a file. MIME messages and literate programming files spring to mind.) Perhaps (as I think ChrisA suggested), you could provide your own buffering/chunking layer between your application and the file itself, and never let the application see the file directly. From storchaka at gmail.com Sat Feb 27 10:23:40 2016 From: storchaka at gmail.com (Serhiy Storchaka) Date: Sat, 27 Feb 2016 17:23:40 +0200 Subject: How to read from a file to an arbitrary delimiter efficiently? In-Reply-To: <56d17138$0$1605$c3e8da3$5496439d@news.astraweb.com> References: <56cea44e$0$11128$c3e8da3@news.astraweb.com> <56d17138$0$1605$c3e8da3$5496439d@news.astraweb.com> Message-ID: On 27.02.16 11:49, Steven D'Aprano wrote: > On Thu, 25 Feb 2016 06:30 pm, Chris Angelico wrote: >> How bad is it if you over-read? > > Pretty bad :-) > > Ideally, I'd rather not over-read at all. I'd like the user to be able to > swap from "read N bytes" to "read to the next delimiter" (and possibly > even "read the next line") without losing anything. > > > If there's absolutely no other way to speed this up by at least a factor of > ten, I'll consider reading into a buffer and losing the ability to mix > different kinds of reads. If the file is buffered, you can use Chris's receipt, but with peek(). Otherwise you should fall back to slow one-byte read. From dkprojektaismtp at gmail.com Sat Feb 27 10:32:17 2016 From: dkprojektaismtp at gmail.com (II DK projektai) Date: Sat, 27 Feb 2016 15:32:17 +0000 Subject: How to optimize operations Message-ID: <01020153235be181-e1c4ef1d-2f4e-48cc-9630-40f6c1f013c8-000000@eu-west-1.amazonses.com> Hello, Do you ever question how effectively you can use your video surveillance system in search of "uninvited guests", "unpaid client", "person damaged the vehicle" or simply to get information about the guest arriving in real time? Or maybe your company needs the number plate recognition systems? Or maybe you have some thoughts that no one has not realized due to high prices and expensive solutions? If you are applicable to these issues, we advance that you familiarize yourself with the automatic license plate recognition?system capabilities by visiting our?website or ask a question by Email. DKprojektai From rustompmody at gmail.com Sat Feb 27 11:13:45 2016 From: rustompmody at gmail.com (Rustom Mody) Date: Sat, 27 Feb 2016 08:13:45 -0800 (PST) Subject: Error in Tree Structure In-Reply-To: <33c93316-0ae3-4b8f-b2f6-29f76c8b3f32@googlegroups.com> References: <33c93316-0ae3-4b8f-b2f6-29f76c8b3f32@googlegroups.com> Message-ID: <573e58fa-36af-4e32-9460-5335d889df26@googlegroups.com> On Saturday, February 27, 2016 at 2:47:53 PM UTC+5:30, subhaba... at gmail.com wrote: > I was trying to implement the code, > > import nltk > import nltk.tag, nltk.chunk, itertools > def ieertree2conlltags(tree, tag=nltk.tag.pos_tag): > words, ents = zip(*tree.pos()) > iobs = [] > prev = None > for ent in ents: > if ent == tree.node: > iobs.append('O') > prev = None > elif prev == ent: > iobs.append('I-%s' % ent) > else: > iobs.append('B-%s' % ent) > prev = ent > words, tags = zip(*tag(words)) > return itertools.izip(words, tags, iobs) > > def ieer_chunked_sents(tag=nltk.tag.pos_tag): > for doc in ieer.parsed_docs(): > tagged = ieertree2conlltags(doc.text, tag) > yield nltk.chunk.conlltags2tree(tagged) > > > from chunkers import ieer_chunked_sents, ClassifierChunker > from nltk.corpus import treebank_chunk > ieer_chunks = list(ieer_chunked_sents()) > chunker = ClassifierChunker(ieer_chunks[:80]) > print chunker.parse(treebank_chunk.tagged_sents()[0]) > score = chunker.evaluate(ieer_chunks[80:]) > print score.accuracy() > > It is running fine. > But as I am trying to rewrite the code as, > chunker = ClassifierChunker(list1), > where list1 is same value as, > ieer_chunks[:80] > only I am pasting the value as > [Tree('S', [Tree('LOCATION', [(u'NAIROBI', 'NNP')]), (u',', ','), Tree('LOCATION', [(u'Kenya', 'NNP')]), (u'(', '('), Tree('ORGANIZATION', [(u'AP', 'NNP')]), (u')', ')'), (u'_', 'NNP'), Tree('CARDINAL', [(u'Thousands', 'NNP')]), (u'of', 'IN'), (u'laborers,', 'JJ'), (u'students', 'NNS'), (u'and', 'CC'), (u'opposition', 'NN'), (u'politicians', 'NNS'), (u'on', 'IN'), Tree('DATE', [(u'Saturday', 'NNP')]), (u'protested', 'VBD'), (u'tax', 'NN'), (u'hikes', 'NNS'), (u'imposed', 'VBN'), (u'by', 'IN'), (u'their', 'PRP$'), (u'cash-strapped', 'JJ'), (u'government,', 'NN'), (u'which', 'WDT'), (u'they', 'PRP'), (u'accused', 'VBD'), (u'of', 'IN'), (u'failing', 'VBG'), (u'to', 'TO'), (u'provide', 'VB'), (u'basic', 'JJ'), (u'services.', 'NN'),....(u'(cm-kjd)', 'NN')])] > the value of whole list directly I am getting syntax error. Dunno how literally you intend this but there is a "...." near the end of the list. Intended? From bc at freeuk.com Sat Feb 27 11:35:15 2016 From: bc at freeuk.com (BartC) Date: Sat, 27 Feb 2016 16:35:15 +0000 Subject: How to read from a file to an arbitrary delimiter efficiently? In-Reply-To: <56cea44e$0$11128$c3e8da3@news.astraweb.com> References: <56cea44e$0$11128$c3e8da3@news.astraweb.com> Message-ID: On 25/02/2016 06:50, Steven D'Aprano wrote: > I have a need to read to an arbitrary delimiter, which might be any of a > (small) set of characters. For the sake of the exercise, lets say it is > either ! or ? (for example). > > # Read a chunk of bytes/characters from an open file. > def chunkiter(f, delim): > buffer = [] > b = f.read(1) > while b: > buffer.append(b) > if b in delim: > yield ''.join(buffer) > buffer = [] > b = f.read(1) > if buffer: > yield ''.join(buffer) At first sight, it's not surprising it's slow when you throw in generators and whatnot in there. However those aren't the main reasons for the poor speed. The limiting factor here is reading one byte at a time. Just a loop like this: while f.read(1): pass without doing anything else, seems to take most of the time. (3.6 seconds, compared with 5.6 seconds of your readchunks() on a 6MB version of your test file, on Python 2.7. readlines() took about 0.2 seconds.) Any faster solutions would need to read more than one byte at a time. (This bottleneck occurs in C too if you try and do read a file using only fgetc(), compared with any buffered solutions.) -- bartc From ganesh1pal at gmail.com Sat Feb 27 11:38:40 2016 From: ganesh1pal at gmail.com (Ganesh Pal) Date: Sat, 27 Feb 2016 22:08:40 +0530 Subject: How to remove the line numbers from the file in python In-Reply-To: References: Message-ID: Thanks it works fine :) On Fri, Feb 26, 2016 at 5:01 PM, Peter Heitzer wrote: > Ganesh Pal wrote: >>what would be the easiest way to remove the lines in the leading >>numbers 1.e 1 ,2,.... 19 from this file using python ? > import sys,re > for line in sys.stdin: > print re.sub('^\d+','',line).rstrip() > > -- > https://mail.python.org/mailman/listinfo/python-list From ganesh1pal at gmail.com Sat Feb 27 11:50:10 2016 From: ganesh1pal at gmail.com (Ganesh Pal) Date: Sat, 27 Feb 2016 22:20:10 +0530 Subject: list index out of range Error , need to fix it or ignore it Message-ID: Iam on python 2.6 and Linux , I need input on the below program , here is the spinet of my program filename='/tmp2/2.txt' def check_file(): """ Run the command parallel on all the machines , if there is a file named /tmp/file2.txt extract file2.txt """ global filename baddr = '' cmd = ("run_al_paral 'ls -al %s'" % (filename)) print(cmd) stdout, stderr, exitcode = run(cmd) print(stdout) lines = stdout.strip().split('\n') print(lines) for line in lines: if 'exited' in lines: continue file = lines[0].split()[9][6:] break print file return file def main(): functions = [check_file] for func in functions: try: func() except Exception as e: return False if __name__ == '__main__': main() 1.If the file is present in any one of the machine the program works fine , example if the file is in machine4 it works fine, machine-4# touch /tmp2/2.txt machine-4# python c_4.py run_al_parall 'ls -al /tmp2/2.txt' machine-4: -rw-r--r-- 1 root wheel 0 Feb 27 08:15 /tmp2/2.txt gpal-machine-2 exited with status 1 gpal-machine-5 exited with status 1 gpal-machine-3 exited with status 1 gpal-machine-1 exited with status 1 ['machine-4: -rw-r--r-- 1 root wheel 0 Feb 27 08:15 /tmp2/2.txt', 'gpal-machine-2 exited with status 1', 'gpal-machine-5 exited with status 1', 'gpal-machine-3 exited with status 1', 'gpal-machine-1 exited with status 1'] 2.txt 2. But if the file is not present we get index out of range error , do we need to fix this or its expected ? or its ok. machine-4# python c_4.py isi_for_array 'ls -al /tmp2/2.txt' machine-2 exited with status 1 machine-1 exited with status 1 machine-4 exited with status 1 machine-5 exited with status 1 machine-3 exited with status 1 ['machine-2 exited with status 1', 'machine-1 exited with status 1', 'machine-4 exited with status 1', 'machine-5 exited with status 1', 'machine-3 exited with status 1'] ERROR:root:list index out of range 3. Any other tips to improve the program Regards, Ganesh From ganesh1pal at gmail.com Sat Feb 27 12:01:57 2016 From: ganesh1pal at gmail.com (Ganesh Pal) Date: Sat, 27 Feb 2016 22:31:57 +0530 Subject: list index out of range Error , need to fix it or ignore it In-Reply-To: References: Message-ID: changed baddr="" to file ="" in the example program , sorry for the typo > filename='/tmp2/2.txt' > > def check_file(): > """ > Run the command parallel on all the machines , if there is a > file named /tmp/file2.txt extract file2.txt > > """ > global filename > file = '' > cmd = ("run_al_paral 'ls -al %s'" % (filename)) > print(cmd) > stdout, stderr, exitcode = run(cmd) > print(stdout) > lines = stdout.strip().split('\n') > print(lines) > for line in lines: > if 'exited' in lines: > continue > > file = lines[0].split()[9][6:] > break > print file > return file > > def main(): > functions = [check_file] > for func in functions: > try: > func() > except Exception as e: > return False > if __name__ == '__main__': > main() > From joel.goldstick at gmail.com Sat Feb 27 12:21:10 2016 From: joel.goldstick at gmail.com (Joel Goldstick) Date: Sat, 27 Feb 2016 12:21:10 -0500 Subject: list index out of range Error , need to fix it or ignore it In-Reply-To: References: Message-ID: On Sat, Feb 27, 2016 at 12:01 PM, Ganesh Pal wrote: > changed baddr="" to file ="" in the example program , sorry for the typo > > > filename='/tmp2/2.txt' > > > > def check_file(): > don't use global filename. just pass filename into check_file def check_file(filename): > > """ > > Run the command parallel on all the machines , if there is a > > file named /tmp/file2.txt extract file2.txt > > > > """ > > global filename > > file = '' > > cmd = ("run_al_paral 'ls -al %s'" % (filename)) > > print(cmd) > > stdout, stderr, exitcode = run(cmd) > what is run(...) > > print(stdout) > > lines = stdout.strip().split('\n') > > print(lines) > > for line in lines: > > if 'exited' in lines: > > continue > > > > file = lines[0].split()[9][6:] > > break > > print file > > return file > > > > def main(): > > functions = [check_file] > > for func in functions: > > try: > > func() > > except Exception as e: > not a good idea to have catchall exception > > return False > > if __name__ == '__main__': > > main() > > > -- > copy and paste your traceback > https://mail.python.org/mailman/listinfo/python-list > -- Joel Goldstick http://joelgoldstick.com/ http://cc-baseballstats.info/ From marko at pacujo.net Sat Feb 27 12:47:34 2016 From: marko at pacujo.net (Marko Rauhamaa) Date: Sat, 27 Feb 2016 19:47:34 +0200 Subject: How to read from a file to an arbitrary delimiter efficiently? References: <56cea44e$0$11128$c3e8da3@news.astraweb.com> <56d17d13$0$1596$c3e8da3$5496439d@news.astraweb.com> Message-ID: <87bn72f3k9.fsf@elektro.pacujo.net> Dennis Lee Bieber : > On Sat, 27 Feb 2016 21:40:17 +1100, Steven D'Aprano > declaimed the following: >>Thanks for finding the issue, but the solutions given don't suit my >>use case. I don't want an iterator that operates on pre-read blocks, I >>want something that will read a record from a file, and leave the file >>pointer one entry past the end of the record. >> >>Oh, and records are likely fairly short, but there may be a lot of them. > > Considering that most of the world has settled on the view that > files are just linear streams (curse you, UNIX) anything working with > "records" has to build the concept on top of the stream. Either by > making records "fixed width" (allowing for fast random access: > recNum*recLen => seek position), though likely giving up the stream > access... Or by wrapping the stream with something that does > parsing/buffering. It may be instructive to see how the Linux/UNIX utility head(1) operates. It actually reads its input greedily but once it has seen enough, it uses lseek(2) to move the seek position back. Not all file-like objects can seek so head(1) may fail to operate as advertised: ======================================================================== $ seq 10000 >/tmp/data.txt $ { > head -n 5 >/dev/null > head -n 5 > } head -n 5 >/dev/null > head -n 5 > } 1861 1862 1863 1864 $ ======================================================================== Marko From marco.kaulea at gmail.com Sat Feb 27 12:57:43 2016 From: marco.kaulea at gmail.com (Marco Kaulea) Date: Sat, 27 Feb 2016 18:57:43 +0100 Subject: Everything good about Python except GUI IDE? In-Reply-To: <64a6599c-fae1-469d-bcee-875165b3cc7d@googlegroups.com> References: <64a6599c-fae1-469d-bcee-875165b3cc7d@googlegroups.com> Message-ID: Hi, Haven't tried it myself, but pyforms[1] might suit your needs. - Marco [1] http://pyforms.readthedocs.org/en/latest/ On Sat, Feb 27, 2016 at 12:18 PM, wrote: > I have some VB forms with more than a hundred objects. If I cannot drag > and drop text boxes, list boxes, labels, etc., it will be too much work to > create that with several lines of code for each object. > Isn't there any good GUI IDE like Visual Basic? I hope there are some less > well known GUI IDEs which I did not come across. Thanks. > From wrong.address.1 at gmail.com Sat Feb 27 13:13:21 2016 From: wrong.address.1 at gmail.com (wrong.address.1 at gmail.com) Date: Sat, 27 Feb 2016 10:13:21 -0800 (PST) Subject: Everything good about Python except GUI IDE? In-Reply-To: References: <64a6599c-fae1-469d-bcee-875165b3cc7d@googlegroups.com> Message-ID: On Saturday, 27 February 2016 18:08:36 UTC+2, Dietmar Schwertberger wrote: > On 27.02.2016 12:18, wrong.address.1 at gmail.com wrote: > > Isn't there any good GUI IDE like Visual Basic? I hope there are some less well known GUI IDEs which I did not come across. Thanks. > > As of today, there's no Python GUI builder comparable to VB 6. > Thanks for stating this clearly. Everyone here has been trying to show me various ways to do the kind of things I will want to, but nobody clearly admits the limitations I will have to accept if I start with Python. I am starting to wonder if VB.net would be a better solution for the time being. I have learnt enough VB.net to manage my work but it is bloated and Microsoft dependent. > There are some like QtDesigner or wxGlade, but they either don't > generate Python code directly or they can only be used if you know the > underlying toolkit good enough to create the GUI yourself. You may try > out some, but I can almost guarantee you that you will come to the same > result. > If you want a GUI, create it yourself using either wxPython or PyQt. I will check it. I got the impression that you can create a GUI but that has to be converted to Python, and then you need a wrapper to put these forms in, and then they can be compiled or converted to *.exe with py2exe. Not a good way for development/debugging. > > For engineering applications that's probably the weakest point that > Python has. > It's holding back a lot of people... > > Well, for most measurement or control software a GUI is not really > needed, but still people want it. > In the 1980s everyone was happy with inputs from the command line on a line editor, but today people expect GUIs with graphics and often even animations. It is surprising that a language which seems very popular does not have GUI development infrastructure in place these many years after it got into common use. > > Regards, > > Dietmar From rosuav at gmail.com Sat Feb 27 13:29:00 2016 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 28 Feb 2016 05:29:00 +1100 Subject: Everything good about Python except GUI IDE? In-Reply-To: References: <64a6599c-fae1-469d-bcee-875165b3cc7d@googlegroups.com> Message-ID: On Sun, Feb 28, 2016 at 5:13 AM, wrote: > In the 1980s everyone was happy with inputs from the command line on a line editor, but today people expect GUIs with graphics and often even animations. > > It is surprising that a language which seems very popular does not have GUI development infrastructure in place these many years after it got into common use. There IS development infrastructure for building GUIs. It's just that the best way to build a cross-platform GUI is code, not drag-and-drop. There are a number of ways to build a GUI in Python (wxPython, PyGTK/PyGObject, PyQt, Tkinter), and some of those have drag-and-drop builders, but not all. Even back in the 90s, when "cross-platform" wasn't a big thing, it was possible to build a window layout using code instead of the builder, and there were a number of situations when that was better (I gave the example of a database form; one of my projects was a generic table editing tool, and what it did was ask the database for a list of columns, and create label+entry field for each one - purely under script control). It's worth noting, too, that the language and the GUI toolkits are independent (only one of the ones I mentioned is even packaged with Python, and it's an optional part); and, thanks to code, the GUI toolkits and GUI builders are also independent. You can pick up a third-party window builder, and as long as it emits valid Python code, it'll work. I've written huge numbers of Python scripts and applications, and I don't think I've ever built a GUI in Python for anything other than testing. ChrisA From marko at pacujo.net Sat Feb 27 13:35:17 2016 From: marko at pacujo.net (Marko Rauhamaa) Date: Sat, 27 Feb 2016 20:35:17 +0200 Subject: Everything good about Python except GUI IDE? References: <64a6599c-fae1-469d-bcee-875165b3cc7d@googlegroups.com> Message-ID: <877fhqf1cq.fsf@elektro.pacujo.net> wrong.address.1 at gmail.com: > In the 1980s everyone was happy with inputs from the command line on a > line editor, but today people expect GUIs with graphics and often even > animations. I don't know who these "people" are, but GUIs certainly have their uses. If you are doing your work professionally, I suppose product management tells you what features are needed in the product. > It is surprising that a language which seems very popular does not > have GUI development infrastructure in place these many years after it > got into common use. It's suprprising that there don't seem to be good GUI applications in existence. Web apps are notoriously bad. (My relatively small experiences with GUI development suggest GUI builders are to blame for the bad GUIs much of the time.) Marko From maillist at schwertberger.de Sat Feb 27 13:51:45 2016 From: maillist at schwertberger.de (Dietmar Schwertberger) Date: Sat, 27 Feb 2016 19:51:45 +0100 Subject: Everything good about Python except GUI IDE? In-Reply-To: References: <64a6599c-fae1-469d-bcee-875165b3cc7d@googlegroups.com> Message-ID: <4714c622-863d-ffbd-357b-d955c22b950e@schwertberger.de> On 27.02.2016 19:13, wrong.address.1 at gmail.com wrote: > I am starting to wonder if VB.net would be a better solution for the time being. I have learnt enough VB.net to manage my work but it is bloated and Microsoft dependent. I would recommend the Python option... > I will check it. I got the impression that you can create a GUI but that has to be converted to Python, and then you need a wrapper to put these forms in, and then they can be compiled or converted to *.exe with py2exe. Not a good way for development/debugging. For PyQt with Qt Designer the development flow looks like e.g. https://nikolak.com/pyqt-qt-designer-getting-started/ For non-trivial GUIs you will probably find it easier to create the GUI in Python code yourself. But maybe the created Python code will show you how to use PyQt in the beginning. You will find many "Getting started with..." for both PyQt and wxPython. For either GUI toolkit, you should get a book - either "wxPython in Action" or "Rapid GUI Programming with Python and Qt" (or both). For both toolkits the electronic documentation is not for Python but for C++. For wxPython you get a .chm help file, which contains all documentation in one file and a wxPython demo which contains a browser and example code for almost all GUI components. For PyQt the documentation is online. While it is quite complete, I always have difficulties to find the information I'm looking for... wxPython is currently not really an option for Python 3, but this will change very soon. (Well, it can be used with Python 3 already, but for a beginner I would recommend to start with the so called "classic" version for Python 2, especially as the included demo will be the main source of information.) For debugging you just run the interpreter with the .py file. It's reasonably simple to debug with e.g. WingIDE. When you create a GUI in Python code, it's often the easiest way to explore properties and methods of the GUI classes from the debugger prompt. Regards, Dietmar From joel.goldstick at gmail.com Sat Feb 27 14:11:08 2016 From: joel.goldstick at gmail.com (Joel Goldstick) Date: Sat, 27 Feb 2016 14:11:08 -0500 Subject: How to optimize operations In-Reply-To: <01020153235be181-e1c4ef1d-2f4e-48cc-9630-40f6c1f013c8-000000@eu-west-1.amazonses.com> References: <01020153235be181-e1c4ef1d-2f4e-48cc-9630-40f6c1f013c8-000000@eu-west-1.amazonses.com> Message-ID: On Sat, Feb 27, 2016 at 10:32 AM, II DK projektai wrote: > Hello, > > Do you ever question how effectively you can use your video surveillance > system in search of "uninvited guests", "unpaid client", "person damaged > the vehicle" or simply to get information about the guest arriving in real > time? Or maybe your company needs the number plate recognition systems? Or > maybe you have some thoughts that no one has not realized due to high > prices and expensive solutions? > > If you are applicable to these issues, we advance that you familiarize > yourself with the automatic license plate recognition system capabilities > by visiting our website or ask a question by Email. > > DKprojektai > -- > https://mail.python.org/mailman/listinfo/python-list > Normally this stuff annoys me, but I got a laugh from this one. I really want to visit the website ;) I wonder where it is? I wonder if i am applicable -- Joel Goldstick http://joelgoldstick.com/ http://cc-baseballstats.info/ From anthony at cajuntechie.org Sat Feb 27 14:45:53 2016 From: anthony at cajuntechie.org (Anthony Papillion) Date: Sat, 27 Feb 2016 13:45:53 -0600 Subject: Everything good about Python except GUI IDE? In-Reply-To: <64a6599c-fae1-469d-bcee-875165b3cc7d@googlegroups.com> References: <64a6599c-fae1-469d-bcee-875165b3cc7d@googlegroups.com> Message-ID: <53F4B293-39C5-40F5-9FB9-CED36B4E8E10@cajuntechie.org> I would absolutely recommend you take a look at the Qt stuff. Very modern, easy to use, and free for non-commercial products. Anthony On February 27, 2016 5:18:57 AM CST, wrong.address.1 at gmail.com wrote: >I have some VB forms with more than a hundred objects. If I cannot drag >and drop text boxes, list boxes, labels, etc., it will be too much work >to create that with several lines of code for each object. > >Isn't there any good GUI IDE like Visual Basic? I hope there are some >less well known GUI IDEs which I did not come across. Thanks. >-- >https://mail.python.org/mailman/listinfo/python-list -- Sent from my Android device with K-9 Mail. Please excuse my brevity. From bc at freeuk.com Sat Feb 27 15:03:42 2016 From: bc at freeuk.com (BartC) Date: Sat, 27 Feb 2016 20:03:42 +0000 Subject: How to read from a file to an arbitrary delimiter efficiently? In-Reply-To: References: <56cea44e$0$11128$c3e8da3@news.astraweb.com> Message-ID: On 27/02/2016 16:35, BartC wrote: > On 25/02/2016 06:50, Steven D'Aprano wrote: >> I have a need to read to an arbitrary delimiter, which might be any of a >> (small) set of characters. For the sake of the exercise, lets say it is >> either ! or ? (for example). > However those aren't the main reasons for the poor speed. The limiting > factor here is reading one byte at a time. Just a loop like this: > > while f.read(1): > pass > > without doing anything else, seems to take most of the time. (3.6 > seconds, compared with 5.6 seconds of your readchunks() on a 6MB version > of your test file, on Python 2.7. readlines() took about 0.2 seconds.) > > Any faster solutions would need to read more than one byte at a time. I've done some more test using Python 3.4, with the same 200,000 line 6MB test file: 0.25 seconds Scan the file with 'for line in f' 2.25 seconds Scan the file with your readlines() routine 4.0 seconds Scan the file with your readchunks() routine 0.65 seconds Scan the file with using a buffer This latter test uses a 64-byte buffer, reading not more than an extra 63 bytes, but resetting the file position to just past the end of of each identified chunk so that any subsequent read works as expected. This test (the code is too untidy to post) only checks for two specific delimiters (not an arbitrary string fill of them). (It also counts EOF as a valid delimiter so counts one more chunk.) Increasing the buffer size doesn't help, and beyond 256 bytes slowed things down (for this input) as it spends too long rereading data. -- Bartc From bc at freeuk.com Sat Feb 27 15:28:40 2016 From: bc at freeuk.com (BartC) Date: Sat, 27 Feb 2016 20:28:40 +0000 Subject: How to read from a file to an arbitrary delimiter efficiently? In-Reply-To: References: <56cea44e$0$11128$c3e8da3@news.astraweb.com> Message-ID: On 27/02/2016 20:03, BartC wrote: > On 27/02/2016 16:35, BartC wrote: >> Any faster solutions would need to read more than one byte at a time. > > I've done some more test using Python 3.4, with the same 200,000 line > 6MB test file: > > 0.25 seconds Scan the file with 'for line in f' > 2.25 seconds Scan the file with your readlines() routine That's not right. 0.25 seconds was for readlines(). 2.25 for a f.read(1) loop. From breamoreboy at yahoo.co.uk Sat Feb 27 15:52:16 2016 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Sat, 27 Feb 2016 20:52:16 +0000 Subject: Everything good about Python except GUI IDE? In-Reply-To: References: <64a6599c-fae1-469d-bcee-875165b3cc7d@googlegroups.com> Message-ID: On 27/02/2016 17:57, Marco Kaulea wrote: > Hi, > > Haven't tried it myself, but pyforms[1] might suit your needs. > > - Marco > > [1] http://pyforms.readthedocs.org/en/latest/ > PyForms 0.1.3 last updated on pypi 2016-01-17 and it's 2.7 only, just awesome, not. -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From python at mrabarnett.plus.com Sat Feb 27 16:35:00 2016 From: python at mrabarnett.plus.com (MRAB) Date: Sat, 27 Feb 2016 21:35:00 +0000 Subject: Everything good about Python except GUI IDE? In-Reply-To: References: <64a6599c-fae1-469d-bcee-875165b3cc7d@googlegroups.com> Message-ID: <56D21684.6070000@mrabarnett.plus.com> On 2016-02-27 20:52, Mark Lawrence wrote: > On 27/02/2016 17:57, Marco Kaulea wrote: >> Hi, >> >> Haven't tried it myself, but pyforms[1] might suit your needs. >> >> - Marco >> >> [1] http://pyforms.readthedocs.org/en/latest/ >> > > PyForms 0.1.3 last updated on pypi 2016-01-17 and it's 2.7 only, just > awesome, not. > It's a little ambiguous. It says "Pyforms is a Python 2.7.x and 3.x cross-enviroment framework", but the Requirements list Python 2.7 only. From ethan at stoneleaf.us Sat Feb 27 17:06:56 2016 From: ethan at stoneleaf.us (Ethan Furman) Date: Sat, 27 Feb 2016 14:06:56 -0800 Subject: [Off-topic] Requests author discusses MentalHealthError exception In-Reply-To: <56d16e6a$0$1593$c3e8da3$5496439d@news.astraweb.com> References: <56d16e6a$0$1593$c3e8da3$5496439d@news.astraweb.com> Message-ID: <56D21E00.9040608@stoneleaf.us> On 02/27/2016 01:37 AM, Steven D'Aprano wrote: > Although the connection to Python is only quite slim, I found it fascinating > to read. Thanks to you and Kenneth for sharing that. -- ~Ethan~ From maillist at schwertberger.de Sat Feb 27 18:20:34 2016 From: maillist at schwertberger.de (Dietmar Schwertberger) Date: Sun, 28 Feb 2016 00:20:34 +0100 Subject: Everything good about Python except GUI IDE? In-Reply-To: <4714c622-863d-ffbd-357b-d955c22b950e@schwertberger.de> References: <64a6599c-fae1-469d-bcee-875165b3cc7d@googlegroups.com> <4714c622-863d-ffbd-357b-d955c22b950e@schwertberger.de> Message-ID: <356345ec-df12-e0bc-d331-c0be78dc8e0a@schwertberger.de> On 27.02.2016 19:51, Dietmar Schwertberger wrote: > You will find many "Getting started with..." for both PyQt and wxPython. P.S.: I forgot to mention that both toolkits have very helpful mailing lists. The toolkits have different licenses. With wxPython you don't need to worry. For PyQt you should check which versions and licenses of PyQt and Qt are applicable and whether they are OK for your purpose. Regards, Dietmar From jsf80238 at gmail.com Sat Feb 27 19:00:13 2016 From: jsf80238 at gmail.com (Jason Friedman) Date: Sat, 27 Feb 2016 17:00:13 -0700 Subject: [Off-topic] Requests author discusses MentalHealthError exception In-Reply-To: <56d16e6a$0$1593$c3e8da3$5496439d@news.astraweb.com> References: <56d16e6a$0$1593$c3e8da3$5496439d@news.astraweb.com> Message-ID: Yes, thank you for sharing. Stories from people we know, or know of, leads to normalization: mental illness is a routine illness like Type I diabetes or appendicitis. On Sat, Feb 27, 2016 at 2:37 AM, Steven D'Aprano wrote: > The author of Requests, Kenneth Reitz, discusses his recent recovery from a > MentalHealthError exception. > > http://www.kennethreitz.org/essays/mentalhealtherror-an-exception-occurred > > Although the connection to Python is only quite slim, I found it fascinating > to read. > > > > -- > Steven > > -- > https://mail.python.org/mailman/listinfo/python-list From gordon at address.invalid Sun Feb 28 00:49:59 2016 From: gordon at address.invalid (Gordon Levi) Date: Sun, 28 Feb 2016 16:49:59 +1100 Subject: Everything good about Python except GUI IDE? References: <64a6599c-fae1-469d-bcee-875165b3cc7d@googlegroups.com> Message-ID: wrong.address.1 at gmail.com wrote: >On Saturday, 27 February 2016 18:08:36 UTC+2, Dietmar Schwertberger wrote: >> On 27.02.2016 12:18, wrong.address.1 at gmail.com wrote: >> > Isn't there any good GUI IDE like Visual Basic? I hope there are some less well known GUI IDEs which I did not come across. Thanks. >> >> As of today, there's no Python GUI builder comparable to VB 6. >> > >Thanks for stating this clearly. Everyone here has been trying to show me various ways to do the kind of things I will want to, but nobody clearly admits the limitations I will have to accept if I start with Python. > >I am starting to wonder if VB.net would be a better solution for the time being. I have learnt enough VB.net to manage my work but it is bloated and Microsoft dependent. When I moved from VB "Classic" to new a language I chose Java. The Netbeans IDE includes a drag and drop WYSIWYG form designer similar to, but slightly better than, VB6 . Alas, thick desktop applications are rapidly becoming out of date. When I chose Java it was the most popular language for commercial programming apart from .NET. I was subsequently tempted to move to Python because I have been programming for the Raspberry Pi and Python is the language of choice for the Pi. In addition I find Python more readable than Java. I have been deterred because, like VB, there are two distinct flavours and, unlike VB, there is no IDE with a form designer. From steve at pearwood.info Sun Feb 28 01:34:30 2016 From: steve at pearwood.info (Steven D'Aprano) Date: Sun, 28 Feb 2016 17:34:30 +1100 Subject: Everything good about Python except GUI IDE? References: <64a6599c-fae1-469d-bcee-875165b3cc7d@googlegroups.com> Message-ID: <56d294f8$0$1604$c3e8da3$5496439d@news.astraweb.com> On Sat, 27 Feb 2016 11:07 pm, Chris Angelico wrote: >> Isn't there any good GUI IDE like Visual Basic? I hope there are some >> less well known GUI IDEs which I did not come across. Thanks. > > Sounds like the advantage lies with Python here... > > Don't make a UI by dragging and dropping that many widgets. And later, in another post: > the best way to build a cross-platform GUI is code, not drag-and-drop. I think that's out-and-out wrong, and harmful to the developer community. I think that we're stuck in the equivalent of the pre-WYSIWYG days of word processing: you can format documents as nicely as you like, but you have to use a separate mode to see it. Drag-and-drop GUI builders have the same advantages over code as Python has over languages with distinct compile/execute steps: rapid development, prototyping, exploration and discovery. Of course, any decent modern builder won't limit you to literally drag-and-drop, but will offer functionality like duplicating elements, aligning them, magnetic guides, etc. GUI elements are by definition graphical in nature, and like other graphical elements, manipulation by hand is superior to command-based manipulation. Graphical interfaces for manipulating graphics have won the UI war so effectively that some people have forgotten there ever was a war. Can you imagine using Photoshop without drag and drop? And yet programming those graphical interfaces is an exception. There, with very few exceptions, we still *require* a command interface. Not just a command interface, but an *off-line* command interface, where you batch up all your commands then run them at once, as if we were Neanderthals living in a cave. An effective and modern GUI builder UI should be programmable without requiring programming. About thirty years ago Apple came up with the ideal mix of graphical and programmatic development for its Hypercard product. You built applications by dragging and dropping widgets on the screen, or by copying and pasting them from a library of pre-made widgets. (By 2016 standards the UI of Hypercard would seem a bit old fashioned and primitive -- black and white, bitmapped graphics, no magnetic guides or "replicate" command, etc -- but it was the mid 80s. A modern GUI builder wouldn't have those limitations.) Hypercard let you create a place widgets by hand using the mouse, or by running a one-line command in the "Message Box" (a simple command interpreter), and for complex tasks, or by writing a script and executing it. You didn't have to script the *entire* application, just as much or as little as needed scripting. Python would be especially well-suited to this, being an interpreter. Why should I have to batch up all the GUI elements and run them all at once to see what my application looks like? Why can't I add elements interactively? I've made a number of attempts to get into GUI development in Python, and every time I flounder at the lousy state of the art in GUI builder tools. Yes, the widget set is fantastic. But putting the GUIs together is primitive beyond words. Hypercard, bless Bill Atkinson, is long gone but not forgotten. But today it lives on in the guise of LiveCode: http://livecode.com/ If only I could get the Linux installer to actually, you know, *work*. -- Steven From rustompmody at gmail.com Sun Feb 28 02:39:23 2016 From: rustompmody at gmail.com (Rustom Mody) Date: Sat, 27 Feb 2016 23:39:23 -0800 (PST) Subject: Everything good about Python except GUI IDE? In-Reply-To: <56d294f8$0$1604$c3e8da3$5496439d@news.astraweb.com> References: <64a6599c-fae1-469d-bcee-875165b3cc7d@googlegroups.com> <56d294f8$0$1604$c3e8da3$5496439d@news.astraweb.com> Message-ID: <225c7518-5af5-42ab-a0fc-653ee14deb0f@googlegroups.com> On Sunday, February 28, 2016 at 12:04:52 PM UTC+5:30, Steven D'Aprano wrote: > On Sat, 27 Feb 2016 11:07 pm, Chris Angelico wrote: > > >> Isn't there any good GUI IDE like Visual Basic? I hope there are some > >> less well known GUI IDEs which I did not come across. Thanks. > > > > Sounds like the advantage lies with Python here... > > > > Don't make a UI by dragging and dropping that many widgets. > > > And later, in another post: > > > the best way to build a cross-platform GUI is code, not drag-and-drop. > > > I think that's out-and-out wrong, and harmful to the developer community. I > think that we're stuck in the equivalent of the pre-WYSIWYG days of word > processing: you can format documents as nicely as you like, but you have to > use a separate mode to see it. > > Drag-and-drop GUI builders have the same advantages over code as Python has > over languages with distinct compile/execute steps: rapid development, > prototyping, exploration and discovery. Of course, any decent modern > builder won't limit you to literally drag-and-drop, but will offer > functionality like duplicating elements, aligning them, magnetic guides, > etc. > > GUI elements are by definition graphical in nature, and like other graphical > elements, manipulation by hand is superior to command-based manipulation. > Graphical interfaces for manipulating graphics have won the UI war so > effectively that some people have forgotten there ever was a war. Can you > imagine using Photoshop without drag and drop? > > And yet programming those graphical interfaces is an exception. There, with > very few exceptions, we still *require* a command interface. Not just a > command interface, but an *off-line* command interface, where you batch up > all your commands then run them at once, as if we were Neanderthals living > in a cave. > > An effective and modern GUI builder UI should be programmable without > requiring programming. About thirty years ago Apple came up with the ideal > mix of graphical and programmatic development for its Hypercard product. > You built applications by dragging and dropping widgets on the screen, or > by copying and pasting them from a library of pre-made widgets. > > (By 2016 standards the UI of Hypercard would seem a bit old fashioned and > primitive -- black and white, bitmapped graphics, no magnetic guides > or "replicate" command, etc -- but it was the mid 80s. A modern GUI builder > wouldn't have those limitations.) > > Hypercard let you create a place widgets by hand using the mouse, or by > running a one-line command in the "Message Box" (a simple command > interpreter), and for complex tasks, or by writing a script and executing > it. You didn't have to script the *entire* application, just as much or as > little as needed scripting. > > Python would be especially well-suited to this, being an interpreter. Why > should I have to batch up all the GUI elements and run them all at once to > see what my application looks like? Why can't I add elements interactively? > > I've made a number of attempts to get into GUI development in Python, and > every time I flounder at the lousy state of the art in GUI builder tools. > Yes, the widget set is fantastic. But putting the GUIs together is > primitive beyond words. > > Hypercard, bless Bill Atkinson, is long gone but not forgotten. But today it > lives on in the guise of LiveCode: > > http://livecode.com/ > > > If only I could get the Linux installer to actually, you know, *work*. > > > -- > Steven A sensible view And more helpful than pretending that neanderthal == civilized Chris: Is it easier to work out the best-lookkng colors with a color picker or with hacking through a million #rrggbb combos? From ganesh1pal at gmail.com Sun Feb 28 03:30:14 2016 From: ganesh1pal at gmail.com (Ganesh Pal) Date: Sun, 28 Feb 2016 14:00:14 +0530 Subject: list index out of range Error , need to fix it or ignore it In-Reply-To: References: Message-ID: >> > what is run(...) > The run (_ is a wrapper it uses suprocess.Popen and returns stdout ,error and extitcod e > not a good idea to have catchall exception how to fix this ? > >> > return False >> > if __name__ == '__main__': >> > main() >> > >> -- >> > copy and paste your traceback > I get " ERROR:root:list index out of range" error if the file is not found in any of the machine , I dont have a trackback back def check_file(): """ 333333333333333333333333333333333333333333333333333333333333333333333333333333333333 Run the command parallel on all the machines , if there is a file named /tmp/file2.txt extract file2.txt """ global filename file = '' cmd = ("run_al_paral 'ls -al %s'" % (filename)) print(cmd) stdout, stderr, exitcode = run(cmd) print(stdout) lines = stdout.strip().split('\n') print(lines) for line in lines: if 'exited' in lines: continue file = lines[0].split()[9][6:] break print file return file def main(): functions = [check_file] for func in functions: try: func() except Exception as e: return False if __name__ == '__main__': main() 1. But if the file is not present we get index out of range error , do we need to fix this or its expected ? or its ok. machine-4# python c_4.py isi_for_array 'ls -al /tmp2/2.txt' machine-2 exited with status 1 machine-1 exited with status 1 machine-4 exited with status 1 machine-5 exited with status 1 machine-3 exited with status 1 ['machine-2 exited with status 1', 'machine-1 exited with status 1', 'machine-4 exited with status 1', 'machine-5 exited with status 1', 'machine-3 exited with status 1'] ERROR:root:list index out of range 3. Any other tips to improve the program From rosuav at gmail.com Sun Feb 28 03:44:55 2016 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 28 Feb 2016 19:44:55 +1100 Subject: Everything good about Python except GUI IDE? In-Reply-To: <56d294f8$0$1604$c3e8da3$5496439d@news.astraweb.com> References: <64a6599c-fae1-469d-bcee-875165b3cc7d@googlegroups.com> <56d294f8$0$1604$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Sun, Feb 28, 2016 at 5:34 PM, Steven D'Aprano wrote: > On Sat, 27 Feb 2016 11:07 pm, Chris Angelico wrote: > >>> Isn't there any good GUI IDE like Visual Basic? I hope there are some >>> less well known GUI IDEs which I did not come across. Thanks. >> >> Sounds like the advantage lies with Python here... >> >> Don't make a UI by dragging and dropping that many widgets. > > > And later, in another post: > >> the best way to build a cross-platform GUI is code, not drag-and-drop. > > > I think that's out-and-out wrong, and harmful to the developer community. I > think that we're stuck in the equivalent of the pre-WYSIWYG days of word > processing: you can format documents as nicely as you like, but you have to > use a separate mode to see it. > > Drag-and-drop GUI builders have the same advantages over code as Python has > over languages with distinct compile/execute steps: rapid development, > prototyping, exploration and discovery. Of course, any decent modern > builder won't limit you to literally drag-and-drop, but will offer > functionality like duplicating elements, aligning them, magnetic guides, > etc. Alright, but how do you go about doing, with a drag-and-drop builder, all those things we're used to with code - composing new named actions out of primitives, observing the changes to a program through source control, unit testing (maybe), and code review? The only way I know of to build a "function" in a DnD builder is to create a composite widget (eg "horizontal box containing label and entry field"), which is extremely useful, but limited - it's like saying that the only way to reuse code is single-inheritance. How would you create a higher-order operation in a DnD builder? How would you write something that does some sort of sweep over a set of widgets and does the same thing to them? All these sorts of things are possible... but we're getting right back to code again. People have tried to create graphical code builders for decades; they've never been sufficient. Code - textual commands to manipulate a system - is still the most powerful and flexible way to do things. By the way, you'll notice that I said "dragging and dropping **that many** widgets" (emphasis added). GUI builders can be great for simple jobs, and a really awesome one can work well for more complex jobs too, but the asymptotic complexity of using drag and drop is worse than that of code. > GUI elements are by definition graphical in nature, and like other graphical > elements, manipulation by hand is superior to command-based manipulation. > Graphical interfaces for manipulating graphics have won the UI war so > effectively that some people have forgotten there ever was a war. Can you > imagine using Photoshop without drag and drop? No, I can't. But I also can't imagine git-managing Photoshop files, and IMO, that is a serious loss. Would you accept a programming language that forced all files to be edited by a single person? Collaboration is pretty important these days. How would you review edits to approve/deny them if you can't see what was edited? > And yet programming those graphical interfaces is an exception. There, with > very few exceptions, we still *require* a command interface. Not just a > command interface, but an *off-line* command interface, where you batch up > all your commands then run them at once, as if we were Neanderthals living > in a cave. I think there's room to improve the textual interface without throwing it away. Certainly several of the Python toolkits are quite primitive in their style; everything has to be executed imperatively, with no room to build a layout declaratively. But thanks to the flexibility of code, this is a very real possibility. Nobody can usefully *extend* the VB UI builder except its owner; anybody can write a function that simplifies one small part of code generation. Python's existing APIs definitely need some work, but I've used some other systems that were closer. Imagine something like this: win = gtk.Window( title="Demo Window", transient_for=mainwindow, child=gtk.labelled(["User name", "Email", "Password", "Confirm", gtk.okbtn()]), ) For more complicated work, build up new primitives and keep right on going. Do it right, and you should be able to have live editing with a preview window, and *maybe* even some kind of manipulation tool, although I'm not sure how best to do that. This doesn't have to be a dichotomy. ChrisA From rosuav at gmail.com Sun Feb 28 03:49:52 2016 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 28 Feb 2016 19:49:52 +1100 Subject: Everything good about Python except GUI IDE? In-Reply-To: <225c7518-5af5-42ab-a0fc-653ee14deb0f@googlegroups.com> References: <64a6599c-fae1-469d-bcee-875165b3cc7d@googlegroups.com> <56d294f8$0$1604$c3e8da3$5496439d@news.astraweb.com> <225c7518-5af5-42ab-a0fc-653ee14deb0f@googlegroups.com> Message-ID: On Sun, Feb 28, 2016 at 6:39 PM, Rustom Mody wrote: > A sensible view > And more helpful than pretending that neanderthal == civilized > > Chris: Is it easier to work out the best-lookkng colors with a color picker or > with hacking through a million #rrggbb combos? Given that "best-looking" is a vague thing that requires a human eye, the easiest way is to try it, in real-time. Whether you see the #rrggbb or drag something with the mouse doesn't matter. But this is an example (single color selection) that is dead simple; whether you use a picker or explicit RGB, you're still picking up 24 bits of information that can be comfortably represented in six characters (seven if you count the hash as part of it). If you mess something up, you hit Ctrl-Z and go back to the previous color. When it's more complicated (say, when you're overlaying multiple brushes in multiple colors), the cost of having a non-textual representation becomes more serious. ChrisA From srkunze at mail.de Sun Feb 28 04:52:45 2016 From: srkunze at mail.de (Sven R. Kunze) Date: Sun, 28 Feb 2016 10:52:45 +0100 Subject: Bug in Python? In-Reply-To: References: <56D0CCE9.2000301@mail.de> Message-ID: <56D2C36D.6090207@mail.de> On 26.02.2016 23:37, Ian Kelly wrote: > On Fri, Feb 26, 2016 at 3:08 PM, Sven R. Kunze wrote: >> Python sometimes seems not to hop back and forth between C and Python code. > C code as a rule tends to ignore dunder methods. Those are used to > implement Python operations, not C operations. Ah, good to know. >> _siftup(heap, 0) # that's C > Your comment here appears to be incorrect. > > [snip] > > So I would guess that the difference here is because one > implementation is entirely C, and the other implementation is entirely > Python. You are damn right. While implementing xheap and looking at the C implementation, I just assumed that all Python functions would be replaced by when the C module is imported. Furthermore, the C module exports some internal functions (leading _ ) such as _heappop_max. Thus, I assumed this is true for _siftup without verifying it by looking too closely at the PyMethodDef and PyModuleDef structs. Sorry for that. Best, Sven From srkunze at mail.de Sun Feb 28 05:08:55 2016 From: srkunze at mail.de (Sven R. Kunze) Date: Sun, 28 Feb 2016 11:08:55 +0100 Subject: Bug in Python? In-Reply-To: References: <56D0CCE9.2000301@mail.de> Message-ID: <56D2C737.9000209@mail.de> On 27.02.2016 00:07, eryk sun wrote: > On Fri, Feb 26, 2016 at 4:08 PM, Sven R. Kunze wrote: >> Python sometimes seems not to hop back and forth between C and Python code. >> Can somebody explain this? > Normally a C extension would call PySequence_SetItem, which would call > the type's sq_ass_item, which for MyList is slot_sq_ass_item. The > latter function bridges the CPython and Python sides by binding and > calling the overridden __setitem__ method. However, the _heapq > extension module uses `PyList_SET_ITEM(heap, 0, lastelt)`. This macro > expands to `((PyListObject *)(heap))->ob_item[0] = lastelt`. This > directly modifies the internal ob_item array of the list, so the > overridden __setitem__ method is never called. I presume it was > implemented like this with performance in mind, but I don't know > whether or not that justifies the loss of generality. I think this is true and it explains the huge performance penalty of the current RemovalHeap and XHeap implementation as it basically uses Python only (results here: http://bit.ly/1KU7CyW). Shoot! I could have seen this earlier. I thought the performance penalty was due to calling __setitem__ and dict operations. But having all heap operations carried out in Python slows things down considerably of course. Let's see if I can manage to create a more efficient mark-and-sweep approach which uses the C module. Best, Sven From srkunze at mail.de Sun Feb 28 05:24:00 2016 From: srkunze at mail.de (Sven R. Kunze) Date: Sun, 28 Feb 2016 11:24:00 +0100 Subject: Bug in Python? In-Reply-To: References: <56D0CCE9.2000301@mail.de> <56d16fff$0$1605$c3e8da3$5496439d@news.astraweb.com> Message-ID: <56D2CAC0.3020700@mail.de> On 27.02.2016 12:48, Terry Reedy wrote: > On 2/27/2016 4:44 AM, Steven D'Aprano wrote: >> On Sat, 27 Feb 2016 07:55 pm, Terry Reedy wrote: >> >>> In other words, when that doc says *list*, it means a *list*. >>> >>> "To create a heap, use a list initialized to [], or you can transform a >>> populated list into a heap via function heapify()." >> [...] >>> "A heap must be an instance of *list* (and not a subclass thereof). To >>> create a heap, start with [] or transform an existing list into a heap >>> via function heapify()." >> >> I think that's a sad decision. heapq ought to be able to handle any list >> subclass, not just actual lists. Preferably it ought to handle >> duck-typed >> lists too, anything with a list-like interface. It is okay if the >> optimized >> C version only works with actual lists, and falls back to a slower >> Python >> version for anything else. > I agree that it would increase comprehensibility. I took me a fair amount of time to see why things are not working as intended (and as you see I wasn't even close to the real explanation). However, heapq might work well the way it is as long as you have the chance to get your hands on one of the other heap implementations out there. > Propose that on the tracker, after checking previous issues. :) Best, Sven From rustompmody at gmail.com Sun Feb 28 05:25:56 2016 From: rustompmody at gmail.com (Rustom Mody) Date: Sun, 28 Feb 2016 02:25:56 -0800 (PST) Subject: Everything good about Python except GUI IDE? In-Reply-To: References: <64a6599c-fae1-469d-bcee-875165b3cc7d@googlegroups.com> <56d294f8$0$1604$c3e8da3$5496439d@news.astraweb.com> Message-ID: <62084c14-abd1-4214-af08-70ce8449c83e@googlegroups.com> On Sunday, February 28, 2016 at 2:15:10 PM UTC+5:30, Chris Angelico wrote: > On Sun, Feb 28, 2016 at 5:34 PM, Steven D'Aprano wrote: > > On Sat, 27 Feb 2016 11:07 pm, Chris Angelico wrote: > > > >>> Isn't there any good GUI IDE like Visual Basic? I hope there are some > >>> less well known GUI IDEs which I did not come across. Thanks. > >> > >> Sounds like the advantage lies with Python here... > >> > >> Don't make a UI by dragging and dropping that many widgets. > > > > > > And later, in another post: > > > >> the best way to build a cross-platform GUI is code, not drag-and-drop. > > > > > > I think that's out-and-out wrong, and harmful to the developer community. I > > think that we're stuck in the equivalent of the pre-WYSIWYG days of word > > processing: you can format documents as nicely as you like, but you have to > > use a separate mode to see it. > > > > Drag-and-drop GUI builders have the same advantages over code as Python has > > over languages with distinct compile/execute steps: rapid development, > > prototyping, exploration and discovery. Of course, any decent modern > > builder won't limit you to literally drag-and-drop, but will offer > > functionality like duplicating elements, aligning them, magnetic guides, > > etc. > > Alright, but how do you go about doing, with a drag-and-drop builder, > all those things we're used to with code - composing new named actions > out of primitives, observing the changes to a program through source > control, unit testing (maybe), and code review? The only way I know of > to build a "function" in a DnD builder is to create a composite widget > (eg "horizontal box containing label and entry field"), which is > extremely useful, but limited - it's like saying that the only way to > reuse code is single-inheritance. How would you create a higher-order > operation in a DnD builder? How would you write something that does > some sort of sweep over a set of widgets and does the same thing to > them? Alright... I can handcraft better assembly than gcc generates. Shall I ditch gcc for assembly? And I can work out more sophisticated memory mgmt and other algos with C than with python (vide Steven's recent thread on delimiter-reading), Shall we prefer C over python? Moore's law dictates that "penny-wise, pound-foolish" gets multiplied by an exponential curve and the 'really-wise' puts up with a low(er) level mess for a higher level good. Sure the well handcrafted gui code will be smaller, neater, faster, what-have-u than builder=generated code. So what? The only reasonable answer is the corollary to Moore's law: The (differential) cost of programmer-time increases exponentially > > All these sorts of things are possible... but we're getting right back > to code again. People have tried to create graphical code builders for > decades; they've never been sufficient. Code - textual commands to > manipulate a system - is still the most powerful and flexible way to > do things. > > By the way, you'll notice that I said "dragging and dropping **that > many** widgets" (emphasis added). GUI builders can be great for simple > jobs, and a really awesome one can work well for more complex jobs > too, but the asymptotic complexity of using drag and drop is worse > than that of code. Code is always the last resort for arbitrary complexity Lets keep it the last resort. If the bottom-line is that python's GUI-builders are so deep into suxland that they are best avoided in place of hand-written code, thats fine (by me). Lets please not make a virtue of a lack From rosuav at gmail.com Sun Feb 28 05:34:39 2016 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 28 Feb 2016 21:34:39 +1100 Subject: Everything good about Python except GUI IDE? In-Reply-To: <62084c14-abd1-4214-af08-70ce8449c83e@googlegroups.com> References: <64a6599c-fae1-469d-bcee-875165b3cc7d@googlegroups.com> <56d294f8$0$1604$c3e8da3$5496439d@news.astraweb.com> <62084c14-abd1-4214-af08-70ce8449c83e@googlegroups.com> Message-ID: On Sun, Feb 28, 2016 at 9:25 PM, Rustom Mody wrote: > Code is always the last resort for arbitrary complexity > Lets keep it the last resort. > > If the bottom-line is that python's GUI-builders are so deep into suxland > that they are best avoided in place of hand-written code, thats fine (by me). > > Lets please not make a virtue of a lack Once someone figures out a way to usefully merge independent changes (you know, the way source control tools do every single day for code), maybe I'll consider that. Until then, the last resort is also my first response. ChrisA From nulla.epistola at web.de Sun Feb 28 05:46:44 2016 From: nulla.epistola at web.de (Sibylle Koczian) Date: Sun, 28 Feb 2016 11:46:44 +0100 Subject: Everything good about Python except GUI IDE? In-Reply-To: References: <64a6599c-fae1-469d-bcee-875165b3cc7d@googlegroups.com> Message-ID: <56D2D014.5090804@web.de> Am 27.02.2016 um 19:13 schrieb wrong.address.1 at gmail.com: > On Saturday, 27 February 2016 18:08:36 UTC+2, Dietmar Schwertberger wrote: >> On 27.02.2016 12:18, wrong.address.1 at gmail.com wrote: >>> Isn't there any good GUI IDE like Visual Basic? I hope there are some less well known GUI IDEs which I did not come across. Thanks. >> >> As of today, there's no Python GUI builder comparable to VB 6. >> > > Thanks for stating this clearly. Everyone here has been trying to show me various ways to do the kind of things I will want to, but nobody clearly admits the limitations I will have to accept if I start with Python. > > I am starting to wonder if VB.net would be a better solution for the time being. I have learnt enough VB.net to manage my work but it is bloated and Microsoft dependent. > Might be quite a good idea. As much as I love Python myself, the WPF GUI builder is very nice. Layout done declaratively, xaml really well documented, and I can work directly with the xaml text and see what it does at the same time. Code completion works, which is quite important with those overlong .NET names. The xaml file only contains the properties of controls I've set myself, not all the default values. If I really want to, I can drag and drop. And the WPF has a reasonable geometry management, like PyQt, tkinter and PyGtk. But then it comes to writing the code and things start getting tedious. So, is there a GUI library for Python that lets me _write_ declarative GUI descriptions, separate from the Python code, but easily connected with it? Sibylle From nulla.epistola at web.de Sun Feb 28 05:46:44 2016 From: nulla.epistola at web.de (Sibylle Koczian) Date: Sun, 28 Feb 2016 11:46:44 +0100 Subject: Everything good about Python except GUI IDE? In-Reply-To: References: <64a6599c-fae1-469d-bcee-875165b3cc7d@googlegroups.com> Message-ID: <56D2D014.5090804@web.de> Am 27.02.2016 um 19:13 schrieb wrong.address.1 at gmail.com: > On Saturday, 27 February 2016 18:08:36 UTC+2, Dietmar Schwertberger wrote: >> On 27.02.2016 12:18, wrong.address.1 at gmail.com wrote: >>> Isn't there any good GUI IDE like Visual Basic? I hope there are some less well known GUI IDEs which I did not come across. Thanks. >> >> As of today, there's no Python GUI builder comparable to VB 6. >> > > Thanks for stating this clearly. Everyone here has been trying to show me various ways to do the kind of things I will want to, but nobody clearly admits the limitations I will have to accept if I start with Python. > > I am starting to wonder if VB.net would be a better solution for the time being. I have learnt enough VB.net to manage my work but it is bloated and Microsoft dependent. > Might be quite a good idea. As much as I love Python myself, the WPF GUI builder is very nice. Layout done declaratively, xaml really well documented, and I can work directly with the xaml text and see what it does at the same time. Code completion works, which is quite important with those overlong .NET names. The xaml file only contains the properties of controls I've set myself, not all the default values. If I really want to, I can drag and drop. And the WPF has a reasonable geometry management, like PyQt, tkinter and PyGtk. But then it comes to writing the code and things start getting tedious. So, is there a GUI library for Python that lets me _write_ declarative GUI descriptions, separate from the Python code, but easily connected with it? Sibylle From vs at it.uu.se Sun Feb 28 06:26:24 2016 From: vs at it.uu.se (Virgil Stokes) Date: Sun, 28 Feb 2016 12:26:24 +0100 Subject: Everything good about Python except GUI IDE? In-Reply-To: References: <64a6599c-fae1-469d-bcee-875165b3cc7d@googlegroups.com> Message-ID: <56D2D960.6020109@it.uu.se> On 2016-Feb-27 19:13, wrong.address.1 at gmail.com wrote: > On Saturday, 27 February 2016 18:08:36 UTC+2, Dietmar Schwertberger wrote: >> On 27.02.2016 12:18, wrong.address.1 at gmail.com wrote: >>> Isn't there any good GUI IDE like Visual Basic? I hope there are some less well known GUI IDEs which I did not come across. Thanks. >> As of today, there's no Python GUI builder comparable to VB 6. >> > Thanks for stating this clearly. Everyone here has been trying to show me various ways to do the kind of things I will want to, but nobody clearly admits the limitations I will have to accept if I start with Python. > > I am starting to wonder if VB.net would be a better solution for the time being. I have learnt enough VB.net to manage my work but it is bloated and Microsoft dependent. > >> There are some like QtDesigner or wxGlade, but they either don't >> generate Python code directly or they can only be used if you know the >> underlying toolkit good enough to create the GUI yourself. You may try >> out some, but I can almost guarantee you that you will come to the same >> result. >> If you want a GUI, create it yourself using either wxPython or PyQt. > I will check it. I got the impression that you can create a GUI but that has to be converted to Python, and then you need a wrapper to put these forms in, and then they can be compiled or converted to *.exe with py2exe. Not a good way for development/debugging. > >> For engineering applications that's probably the weakest point that >> Python has. >> It's holding back a lot of people... >> >> Well, for most measurement or control software a GUI is not really >> needed, but still people want it. >> > In the 1980s everyone was happy with inputs from the command line on a line editor, but today people expect GUIs with graphics and often even animations. > > It is surprising that a language which seems very popular does not have GUI development infrastructure in place these many years after it got into common use. > >> Regards, >> >> Dietmar I agree (at least largely) with the author of this email, in response to Dietmar. I have been working with Python for several years and often a GUI is needed, not by me; but, for users of my software where my target is usually numerical and image processing with a "don't make me think too much" GUI. I have mainly used wxPython (which is rather good, with good support); but, I find it rather awkward in practice and making an *.exe for users that includes wxPython is often a tedious process (at least from my own experiences). Perhaps my skills with wxPython and its API are lacking :-( . After re-reading some of the postings that are connected to GUI problems in the python-list, wxPython-users, and PyQT, it seems to me that an "improved IDLE" for Python might have helped to solve some of their problems. I am quite sure such a development would be well received by Python beginners and those migrating to Python from other languages (e.g. VB). While working on my first wxPython GUI project, I actually switched to VB to test my GUI design and to create an *.exe for the project --- this went rather fast, considering that I had no previous experience with VB. Only afterwards, did I return to wxPython, for implementation in Python 2.7, which took much longer with extensive refactoring. And Dietmar, please don't take this the wrong way, I also agree with some of the points that you have made. And I do like wxPython :-) From srkunze at mail.de Sun Feb 28 07:23:07 2016 From: srkunze at mail.de (Sven R. Kunze) Date: Sun, 28 Feb 2016 13:23:07 +0100 Subject: Everything good about Python except GUI IDE? In-Reply-To: <56d294f8$0$1604$c3e8da3$5496439d@news.astraweb.com> References: <64a6599c-fae1-469d-bcee-875165b3cc7d@googlegroups.com> <56d294f8$0$1604$c3e8da3$5496439d@news.astraweb.com> Message-ID: <56D2E6AB.9090604@mail.de> On 28.02.2016 07:34, Steven D'Aprano wrote: > I think that's out-and-out wrong, and harmful to the developer community. I > think that we're stuck in the equivalent of the pre-WYSIWYG days of word > processing: you can format documents as nicely as you like, but you have to > use a separate mode to see it. Good point. > Drag-and-drop GUI builders have the same advantages over code as Python has > over languages with distinct compile/execute steps: rapid development, > prototyping, exploration and discovery. Of course, any decent modern > builder won't limit you to literally drag-and-drop, but will offer > functionality like duplicating elements, aligning them, magnetic guides, > etc. Another good point. I will get to this later. > GUI elements are by definition graphical in nature, and like other graphical > elements, manipulation by hand is superior to command-based manipulation. > Graphical interfaces for manipulating graphics have won the UI war so > effectively that some people have forgotten there ever was a war. Can you > imagine using Photoshop without drag and drop? > (you can measure this by counting the numbers of replies to a thread) That's whole different topic. What is Photoshop manipulating? Layers of pixels. That's an extremely simplified model. There is no dynamic behavior as there is with GUIs. > And yet programming those graphical interfaces is an exception. There, with > very few exceptions, we still *require* a command interface. Not just a > command interface, but an *off-line* command interface, where you batch up > all your commands then run them at once, as if we were Neanderthals living > in a cave. Not sure if I agree with you here. Let's ask ourselves, what is so different about, say, a complex mathematical function and a complex GUI? In other words: why do you can live with a text representation of that function whereas you cannot live with a text representation of a GUI? One difference is the number of interactions you can do with a function and a GUI. A function takes some numbers whereas a GUI takes some complex text/mouse/finger/voice interactions. So, I've never heard of any complains when it comes to mathematical functions represented in some source code. But, I've heard a lot of complains regarding GUI design and interaction tests (even when they are done graphically) -- also in WPF. Both text representations are abstract descriptions of the real thing (function and GUI). You need some imagination to get them right, to debug them, to maintain them, to change them. We could blame Python here but it's due to the problem realm and to the people working there: Functions -> mathematicians/computer scientists, work regularly with highly abstract objects GUI -> designers, never really got the same education for programming/abstraction as the former group has So, (and I know that from where I am involved with) GUI research (development, evaluation etc.) is not a topic considered closed. No serious computer scientist really knows the "right" way. But, hey, people are working on it at least. Usually, you start out simple. As the time flies, you put in more and more features and things become more and more complex (we all know that all non-toy projects will). And so does a GUI. At a certain point, there is no other way than going into the code and do something nasty by utilizing the Turing-completeness of the underlying language. Generated code always looks creepy, bloaty with a lot of boilerplate. If you really really need to dig deeper, you will have a hard time finding out what of the boilerplate is really needed and what was added by the code-generator. In the end, you might even break the "drag-n-drop"ability. :-( That is the reason, why traditional CASE tools never really got started, why we still need programmers, why we still have text. From my point of view (highly subjective), start by using general building blocks (text, functions, classes, ...) is better long-term; not by starting with a cage (the GUI) and subsequently adding more and more holes not fitting the original concept. History so far as agreed with this; professional software development always uses text tools for which LATER somebody built a GUI. I cannot remember it being the other way round. Furthermore, I agree with Chris about the version control problem. Last but not least, GUIs are a place for bike shedding because almost everybody is able to see them and can start having an opinion about them: Who loves the new Windows modern UI? Either you like it or you hate it. What about the Riemann zeta function? Anybody? Best, Sven PS: another thought. I recently introduced LaTeX to my girlfriend. LaTeX is quite ugly and it has this "distinct compile/execute step", so initially I hesitated to show it to her. But her MS Word experience got worse and worse the more complex (and especially larger) her workload became. Word became less responsive and results became even less reproducible (footnote numbering, styling, literature, etc.). She needed to invest some time to learn LaTeX and to tweak the initial template to fit her needs. Though, in the end, she's much happier and get reproducible results. She still uses a GUI for writing LaTeX. It helps her avoiding mistakes. So, I don't think it's GUI vs text but rather how can they complement each other. From bc at freeuk.com Sun Feb 28 07:38:17 2016 From: bc at freeuk.com (BartC) Date: Sun, 28 Feb 2016 12:38:17 +0000 Subject: Everything good about Python except GUI IDE? In-Reply-To: <56d294f8$0$1604$c3e8da3$5496439d@news.astraweb.com> References: <64a6599c-fae1-469d-bcee-875165b3cc7d@googlegroups.com> <56d294f8$0$1604$c3e8da3$5496439d@news.astraweb.com> Message-ID: On 28/02/2016 06:34, Steven D'Aprano wrote: > GUI elements are by definition graphical in nature, and like other graphical > elements, manipulation by hand is superior to command-based manipulation. > Graphical interfaces for manipulating graphics have won the UI war so > effectively that some people have forgotten there ever was a war. Can you > imagine using Photoshop without drag and drop? > > And yet programming those graphical interfaces is an exception. There, with > very few exceptions, we still *require* a command interface. Not just a > command interface, but an *off-line* command interface, where you batch up > all your commands then run them at once, as if we were Neanderthals living > in a cave. You've got that back to front. It's the GUI users who are the Neanderthals, having to effectively point at things with sticks. Or have to physically move that rock themselves (ie. drag a file to a wastebasket). More advanced uses have the power of language, with all its sophistications (ie. commands lines and scripting). And they don't need to move that rock, they can tell someone else to do it! But with far more control: all rocks of a certain size and colour, and at sunrise tomorrow. Some things are just more easily described with a script or formula or algorithm which is then 'rendered' to produce the result. Not quite right? Change one parameter to re-render to instantly produce a new version, that would have taken minutes or hours to do manually. > An effective and modern GUI builder UI should be programmable without > requiring programming. About thirty years ago Apple came up with the ideal > mix of graphical and programmatic development for its Hypercard product. > You built applications by dragging and dropping widgets on the screen, or > by copying and pasting them from a library of pre-made widgets. You have to give someone some shopping to do. What's quicker, jotting down a list of milk, bread, eggs and so on, or invoking some GUI program where you have to first look for each category, then have to choose the exact subcategory, size, quantity... -- Bartc From maillist at schwertberger.de Sun Feb 28 07:38:33 2016 From: maillist at schwertberger.de (Dietmar Schwertberger) Date: Sun, 28 Feb 2016 13:38:33 +0100 Subject: Everything good about Python except GUI IDE? In-Reply-To: <56D2E6AB.9090604@mail.de> References: <64a6599c-fae1-469d-bcee-875165b3cc7d@googlegroups.com> <56d294f8$0$1604$c3e8da3$5496439d@news.astraweb.com> <56D2E6AB.9090604@mail.de> Message-ID: <9ed69848-bfb2-a6ef-555e-01e9424875cc@schwertberger.de> On 28.02.2016 13:23, Sven R. Kunze wrote: > I recently introduced LaTeX to my girlfriend. LaTeX is quite ugly and > it has this "distinct compile/execute step", so initially I hesitated > to show it to her. But her MS Word experience got worse and worse the > more complex (and especially larger) her workload became. Word became > less responsive and results became even less reproducible (footnote > numbering, styling, literature, etc.). Well, MS Word is a bad example, as it's lacking even some very basic features like a good user interface for styles. (This is the reason why most people are using Word just like a typwriter instead of creating structured documents.) There were and probably still are better word processors, especially for scientific use. The discussion here is not about "any GUI builder", but about a "good and easy to use GUI builder". Regards, Dietmar From steve at pearwood.info Sun Feb 28 07:50:57 2016 From: steve at pearwood.info (Steven D'Aprano) Date: Sun, 28 Feb 2016 23:50:57 +1100 Subject: Everything good about Python except GUI IDE? References: <64a6599c-fae1-469d-bcee-875165b3cc7d@googlegroups.com> <56d294f8$0$1604$c3e8da3$5496439d@news.astraweb.com> Message-ID: <56d2ed33$0$1588$c3e8da3$5496439d@news.astraweb.com> On Sun, 28 Feb 2016 07:44 pm, Chris Angelico wrote: > On Sun, Feb 28, 2016 at 5:34 PM, Steven D'Aprano > wrote: [...] >> Drag-and-drop GUI builders have the same advantages over code as Python >> has over languages with distinct compile/execute steps: rapid >> development, prototyping, exploration and discovery. Of course, any >> decent modern builder won't limit you to literally drag-and-drop, but >> will offer functionality like duplicating elements, aligning them, >> magnetic guides, etc. > > Alright, but how do you go about doing, with a drag-and-drop builder, > all those things we're used to with code - composing new named actions > out of primitives, observing the changes to a program through source > control, unit testing (maybe), and code review? These are all good questions. Let's see if I can give good answers: (1) "composing new named actions" -- I'm not entirely sure what you mean. Do you mean new named *widgets*? A good builder app should give you the ability to Group widgets into a single element, this is functionality which has existed in vector-drawing programs since at least MacDraw in 1984 so it shouldn't be hard. This is composition, a fundamental, powerful and rich design pattern for making new widgets (classes) out of simpler parts. If objects have a name, now you can refer to CompositeMenuDateColourPicker by name. You can copy it, paste it, replicate it 30 times, whatever you like. Possibly the GUI builder will even add it to the "Insert Widget" menu, or put it on the toolbar. Surely the builder app will use a plug-in architecture to control what widgets are available. How easy is it to create new plug-ins from within the builder? This is a "quality of implementation" issue. Presumably a modern GUI builder will have the ability to export to source code, so you can export your CompositeMenuDateColourPicker to a file, then re-use it over and over again in project after project. (2) "source control" -- the world is full of document types that aren't plain text source code, and people have found ways to manage collaborative editing and change management for them. Why don't we ask game developers how they manage changes to the non-code elements in their applications? Textures, icons, player avatars, sound effects, maps, etc. Surely you don't suggest that game developers edit their background images in a hex editor? (3) "unit testing" -- I'm not sure that unit testing is what is needed for the GUI elements of your application. It's more like integration testing, to ensure that the entire application works together as a seamless whole. I'm not sure what the state of the art in GUI application integration testing is like. I suspect crap, judging by the state of the art in GUI applications. But whatever it is, it will surely operate the same regardless of whether you have built the application using code or a graphical builder. (The GUI framework itself may have some analogue to unit testing for the individual widgets, but that's not your responsibility as the user of the framework. It's not up to you to test that menus drop down when clicked, but it is your responsibility to check that the right menus exist and that they do what they are supposed to.) (4) "code review" -- the usual way to review the graphical elements of a GUI app is to call somebody over, sit them down at the running application, and say "What do you think of this?". They will usually answer "that icon needs to be a bit more to the left, and can you make that button blue instead of green?". > The only way I know of > to build a "function" in a DnD builder is to create a composite widget > (eg "horizontal box containing label and entry field"), which is > extremely useful, but limited - it's like saying that the only way to > reuse code is single-inheritance. A poor analogy. Composition is equivalent to multiple inheritance, except without any of the weaknesses of MI. > How would you create a higher-order > operation in a DnD builder? How would you write something that does > some sort of sweep over a set of widgets and does the same thing to > them? In Hypercard, if it was a once-off processing task, I would create a button, edit the button's script: on mouseUp: -- my memory of HC syntax and functions is a bit rusty -- this may not be correct for btnNum in 1 to the number of buttons: if btnNum is the number of me: continue end if set the textsize of button btnNum to 9 set the textstyle of button btnNum to bold,italic if the name of button btnNum starts with "Spam": set the icon of button btnNum to SpamIcon end if end for end mouseUp then I would click on that button and run the script. Then, once I have satisfied myself that it has done what was needed, I'd delete the button. If this was something that needed to run each time the application ran, I would put the script in some other layer of the application, say, in the card layer, in a named handler: on setBtnStyle: for btnNum in 1 to the number of buttons: set the textsize of button btnNum to 9 set the textstyle of button btnNum to bold,italic if the name of button btnNum starts with "Spam": set the icon of button btnNum to SpamIcon end if end for end setBtnStyle then call that handler on some event: on openCard: setBtnStyle end openCard (In Hypercard, the UI model is of a Rolodex or set of cards, once card displayed at a time in a window. Each card can contain widgets and graphics. Displaying a card sends the openCard message to that card, which runs whatever code is in the openCard handler. (Don't be put off by the language of sending messages, that's identical to a method call.) If this sounds kinda Javascript-y, it's because HC is one of the admitted influences of early Javascript. If the business of sending messages sounds kinda like Smalltalk, that's because HC was influenced by it. Apple history buffs will see the influence of Alan Kay and Xerox PARC all over this... Hypercard was not limited to manipulating the state of existing widgets. It could programmatically create new ones. (Apple's Hypercard has a horrible API for that where you literally gave a command to select a tool, then drag from one point to another to create the widget. But later competing products added more sensible commands for this.) If you had a complex layout, you could code that part of the layout, but the point is that you weren't forced to code everything. [...] > This doesn't have to be a dichotomy. I didn't say it did :-) -- Steven From cl at isbd.net Sun Feb 28 07:52:33 2016 From: cl at isbd.net (cl at isbd.net) Date: Sun, 28 Feb 2016 12:52:33 +0000 Subject: Everything good about Python except GUI IDE? References: <64a6599c-fae1-469d-bcee-875165b3cc7d@googlegroups.com> <56d294f8$0$1604$c3e8da3$5496439d@news.astraweb.com> <56D2E6AB.9090604@mail.de> Message-ID: Dietmar Schwertberger wrote: > On 28.02.2016 13:23, Sven R. Kunze wrote: > > I recently introduced LaTeX to my girlfriend. LaTeX is quite ugly and > > it has this "distinct compile/execute step", so initially I hesitated > > to show it to her. But her MS Word experience got worse and worse the > > more complex (and especially larger) her workload became. Word became > > less responsive and results became even less reproducible (footnote > > numbering, styling, literature, etc.). > Well, MS Word is a bad example, as it's lacking even some very basic > features like a good user interface for styles. (This is the reason why > most people are using Word just like a typwriter instead of creating > structured documents.) > There were and probably still are better word processors, especially for > scientific use. > E.g. Latex or reStructuredText, not GUI. -- Chris Green ? From rustompmody at gmail.com Sun Feb 28 07:54:07 2016 From: rustompmody at gmail.com (Rustom Mody) Date: Sun, 28 Feb 2016 04:54:07 -0800 (PST) Subject: Everything good about Python except GUI IDE? In-Reply-To: References: <64a6599c-fae1-469d-bcee-875165b3cc7d@googlegroups.com> <56d294f8$0$1604$c3e8da3$5496439d@news.astraweb.com> Message-ID: <234a398e-1b0f-467b-a8cb-d7ca748f8062@googlegroups.com> On Sunday, February 28, 2016 at 6:08:44 PM UTC+5:30, BartC wrote: > On 28/02/2016 06:34, Steven D'Aprano wrote: > > > GUI elements are by definition graphical in nature, and like other graphical > > elements, manipulation by hand is superior to command-based manipulation. > > Graphical interfaces for manipulating graphics have won the UI war so > > effectively that some people have forgotten there ever was a war. Can you > > imagine using Photoshop without drag and drop? > > > > And yet programming those graphical interfaces is an exception. There, with > > very few exceptions, we still *require* a command interface. Not just a > > command interface, but an *off-line* command interface, where you batch up > > all your commands then run them at once, as if we were Neanderthals living > > in a cave. > > You've got that back to front. > > It's the GUI users who are the Neanderthals, having to effectively point > at things with sticks. Or have to physically move that rock themselves > (ie. drag a file to a wastebasket). Creature A: Plays with a toy -- usually called 'child' Creature B: Makes toys, possibly designs new ones... Can be child Are these same? Steven is talking of GUI *programmers* You are talking of GUI *users* > > More advanced uses have the power of language, with all its > sophistications (ie. commands lines and scripting). And they don't need > to move that rock, they can tell someone else to do it! But with far > more control: all rocks of a certain size and colour, and at sunrise > tomorrow. You seem to have a rather limited view of language. Math is a language Music is a language -- and sophisticated music analysis can slot music according to genre etc So also GUIs > > Some things are just more easily described with a script or formula or > algorithm which is then 'rendered' to produce the result. Not quite > right? Change one parameter to re-render to instantly produce a new > version, that would have taken minutes or hours to do manually. > > > An effective and modern GUI builder UI should be programmable without > > requiring programming. About thirty years ago Apple came up with the ideal > > mix of graphical and programmatic development for its Hypercard product. > > You built applications by dragging and dropping widgets on the screen, or > > by copying and pasting them from a library of pre-made widgets. > > You have to give someone some shopping to do. What's quicker, jotting > down a list of milk, bread, eggs and so on, or invoking some GUI program > where you have to first look for each category, then have to choose the > exact subcategory, size, quantity... Dunno what that has to do with GUI It seems to be to do with 'coding-up' The string "milk" codes up milk more efficiently than category navigation and manipulation That programmers are 50 years behind laypersons in terms of computer USAGE: http://blog.languager.org/2013/09/poorest-computer-users-are-programmers.html From steve at pearwood.info Sun Feb 28 08:00:44 2016 From: steve at pearwood.info (Steven D'Aprano) Date: Mon, 29 Feb 2016 00:00:44 +1100 Subject: list index out of range Error , need to fix it or ignore it References: Message-ID: <56d2ef7e$0$1584$c3e8da3$5496439d@news.astraweb.com> On Sun, 28 Feb 2016 03:50 am, Ganesh Pal wrote: > Iam on python 2.6 and Linux , I need input on the below program , > here is the spinet of my program > > > filename='/tmp2/2.txt' > > def check_file(): > """ > Run the command parallel on all the machines , if there is a > file named /tmp/file2.txt extract file2.txt > > """ > global filename Since filename is never assigned to inside the function, you don't need to declare it global. But better is to make the filename an argument of the function. That will allow you to check for different files, not always the same one. > baddr = '' Not used. Get rid of it. > cmd = ("run_al_paral 'ls -al %s'" % (filename)) > print(cmd) > stdout, stderr, exitcode = run(cmd) run is not defined. > print(stdout) > lines = stdout.strip().split('\n') > print(lines) > for line in lines: > if 'exited' in lines: > continue > file = lines[0].split()[9][6:] > break > print file > return file > > def main(): > functions = [check_file] > for func in functions: > try: > func() > except Exception as e: > return False Why do you bother to return False? Nothing checks the return value of main(). As far as the try...except block, please read this: https://realpython.com/blog/python/the-most-diabolical-python-antipattern/ > if __name__ == '__main__': > main() > > > > 1.If the file is present in any one of the machine the program works > fine , example if the file is in machine4 it works fine, > > > machine-4# touch /tmp2/2.txt > machine-4# python c_4.py > run_al_parall 'ls -al /tmp2/2.txt' > machine-4: -rw-r--r-- 1 root wheel 0 Feb 27 08:15 /tmp2/2.txt > gpal-machine-2 exited with status 1 > gpal-machine-5 exited with status 1 > gpal-machine-3 exited with status 1 > gpal-machine-1 exited with status 1 You really should get out of the bad habit of running code as root. Once day you will accidentally have a bug in your code that will do something it shouldn't do (like delete the root file system) and leave your machine unusable. You should create a user with the minimum privileges needed to get the work done. At the very least, you should TEST your code while running as an unprivileged user. It is very dangerous to run buggy code as root. Who knows what it will do? > 2. But if the file is not present we get index out of range error , do > we need to fix this or its expected ? or its ok. Of course you need to fix it. There's no way of telling where that error occurs or why it is being printed. -- Steven From bc at freeuk.com Sun Feb 28 08:07:21 2016 From: bc at freeuk.com (BartC) Date: Sun, 28 Feb 2016 13:07:21 +0000 Subject: Everything good about Python except GUI IDE? In-Reply-To: <234a398e-1b0f-467b-a8cb-d7ca748f8062@googlegroups.com> References: <64a6599c-fae1-469d-bcee-875165b3cc7d@googlegroups.com> <56d294f8$0$1604$c3e8da3$5496439d@news.astraweb.com> <234a398e-1b0f-467b-a8cb-d7ca748f8062@googlegroups.com> Message-ID: On 28/02/2016 12:54, Rustom Mody wrote: > On Sunday, February 28, 2016 at 6:08:44 PM UTC+5:30, BartC wrote: >> You have to give someone some shopping to do. What's quicker, jotting >> down a list of milk, bread, eggs and so on, or invoking some GUI program >> where you have to first look for each category, then have to choose the >> exact subcategory, size, quantity... > > Dunno what that has to do with GUI > It seems to be to do with 'coding-up' It's comparing a drag-and-drop approach with just writing a list or script using text. And in a situation where there can be thousands of possibilities. To extend it further, imagine having to write a document using a mouse rather than a keyboard. And doing so by having to bring up the right word each time and drag it into place. It would take forever. Going back to GUI for creating dialogs, it just doesn't work for me (admittedly I've never tried it except for some tinkering decades ago). The first dialog I create will be bound to have a conditional layout which depends on parameters now known until runtime. Or when one element has a dependency on another. -- Bartc From gordon at address.invalid Sun Feb 28 08:08:26 2016 From: gordon at address.invalid (Gordon Levi) Date: Mon, 29 Feb 2016 00:08:26 +1100 Subject: Everything good about Python except GUI IDE? References: <64a6599c-fae1-469d-bcee-875165b3cc7d@googlegroups.com> <56d294f8$0$1604$c3e8da3$5496439d@news.astraweb.com> <62084c14-abd1-4214-af08-70ce8449c83e@googlegroups.com> Message-ID: <6dq5db5j0hg2evl7t334ftdm5sk8n5itge@4ax.com> Chris Angelico wrote: >On Sun, Feb 28, 2016 at 9:25 PM, Rustom Mody wrote: >> Code is always the last resort for arbitrary complexity >> Lets keep it the last resort. >> >> If the bottom-line is that python's GUI-builders are so deep into suxland >> that they are best avoided in place of hand-written code, thats fine (by me). >> >> Lets please not make a virtue of a lack > >Once someone figures out a way to usefully merge independent changes >(you know, the way source control tools do every single day for code), >maybe I'll consider that. Until then, the last resort is also my first >response. Why can't whatever is generated by a GUI form designer be stored in source control along with all the other project files? The only restriction would be that everyone who wants to change the UI would have to use the same form designer. From rustompmody at gmail.com Sun Feb 28 08:13:57 2016 From: rustompmody at gmail.com (Rustom Mody) Date: Sun, 28 Feb 2016 05:13:57 -0800 (PST) Subject: Everything good about Python except GUI IDE? In-Reply-To: <6dq5db5j0hg2evl7t334ftdm5sk8n5itge@4ax.com> References: <64a6599c-fae1-469d-bcee-875165b3cc7d@googlegroups.com> <56d294f8$0$1604$c3e8da3$5496439d@news.astraweb.com> <62084c14-abd1-4214-af08-70ce8449c83e@googlegroups.com> <6dq5db5j0hg2evl7t334ftdm5sk8n5itge@4ax.com> Message-ID: On Sunday, February 28, 2016 at 6:38:40 PM UTC+5:30, Gordon Levi wrote: > Chris Angelico wrote: > > >On Sun, Feb 28, 2016 at 9:25 PM, Rustom Mody wrote: > >> Code is always the last resort for arbitrary complexity > >> Lets keep it the last resort. > >> > >> If the bottom-line is that python's GUI-builders are so deep into suxland > >> that they are best avoided in place of hand-written code, thats fine (by me). > >> > >> Lets please not make a virtue of a lack > > > >Once someone figures out a way to usefully merge independent changes > >(you know, the way source control tools do every single day for code), > >maybe I'll consider that. Until then, the last resort is also my first > >response. > > Why can't whatever is generated by a GUI form designer be stored in > source control along with all the other project files? The only > restriction would be that everyone who wants to change the UI would > have to use the same form designer. Glade generates XML (last I saw) XML is text... kinda... but not quite eg XML is sometimes/somewhere space sensitive, sometimes not This can generate messy diffs Best I can see, these are not exactly trivial nor quite impossible to solve problems From maillist at schwertberger.de Sun Feb 28 08:19:19 2016 From: maillist at schwertberger.de (Dietmar Schwertberger) Date: Sun, 28 Feb 2016 14:19:19 +0100 Subject: Everything good about Python except GUI IDE? In-Reply-To: References: <64a6599c-fae1-469d-bcee-875165b3cc7d@googlegroups.com> <56d294f8$0$1604$c3e8da3$5496439d@news.astraweb.com> <56D2E6AB.9090604@mail.de> Message-ID: <2d771167-3f32-5dde-e083-3d24911aa03e@schwertberger.de> On 28.02.2016 13:52, cl at isbd.net wrote: > Dietmar Schwertberger wrote: >> There were and probably still are better word processors, especially for >> scientific use. > E.g. Latex or reStructuredText, not GUI. I was more thinking of TechWriter on RISC OS. For Windows there are probably similar offerings. Try selling LaTex as standard word processor to a corporate environment. I guarantee you that you will fail. Regards, Dietmar From rustompmody at gmail.com Sun Feb 28 08:20:19 2016 From: rustompmody at gmail.com (Rustom Mody) Date: Sun, 28 Feb 2016 05:20:19 -0800 (PST) Subject: Everything good about Python except GUI IDE? In-Reply-To: References: <64a6599c-fae1-469d-bcee-875165b3cc7d@googlegroups.com> <56d294f8$0$1604$c3e8da3$5496439d@news.astraweb.com> <234a398e-1b0f-467b-a8cb-d7ca748f8062@googlegroups.com> Message-ID: <84922f24-3e00-4a23-b26d-5e6c0d8e7e04@googlegroups.com> On Sunday, February 28, 2016 at 6:37:49 PM UTC+5:30, BartC wrote: > On 28/02/2016 12:54, Rustom Mody wrote: > > On Sunday, February 28, 2016 at 6:08:44 PM UTC+5:30, BartC wrote: > > >> You have to give someone some shopping to do. What's quicker, jotting > >> down a list of milk, bread, eggs and so on, or invoking some GUI program > >> where you have to first look for each category, then have to choose the > >> exact subcategory, size, quantity... > > > > Dunno what that has to do with GUI > > It seems to be to do with 'coding-up' > > > It's comparing a drag-and-drop approach with just writing a list or > script using text. And in a situation where there can be thousands of > possibilities. > > To extend it further, imagine having to write a document using a mouse > rather than a keyboard. And doing so by having to bring up the right > word each time and drag it into place. It would take forever. > > Going back to GUI for creating dialogs, it just doesn't work for me > (admittedly I've never tried it except for some tinkering decades ago). > The first dialog I create will be bound to have a conditional layout > which depends on parameters now known until runtime. Or when one element > has a dependency on another. Your argument assumes GUI = not Text Interface whereas in fact every significant GUI embeds text (possibly recursively) eg TI inside GUI -- think of text inside gimp GUI inside TI -- think of Word embedding other doc types including pictures Which can be recursive -- WOrd embeds a picture embeds text From gordon at address.invalid Sun Feb 28 08:24:24 2016 From: gordon at address.invalid (Gordon Levi) Date: Mon, 29 Feb 2016 00:24:24 +1100 Subject: Everything good about Python except GUI IDE? References: <64a6599c-fae1-469d-bcee-875165b3cc7d@googlegroups.com> <56d294f8$0$1604$c3e8da3$5496439d@news.astraweb.com> <62084c14-abd1-4214-af08-70ce8449c83e@googlegroups.com> <6dq5db5j0hg2evl7t334ftdm5sk8n5itge@4ax.com> Message-ID: <93t5db9sib9ldgktrt7523fnis4tgq2uev@4ax.com> Rustom Mody wrote: >On Sunday, February 28, 2016 at 6:38:40 PM UTC+5:30, Gordon Levi wrote: >> Chris Angelico wrote: >> >> >On Sun, Feb 28, 2016 at 9:25 PM, Rustom Mody wrote: >> >> Code is always the last resort for arbitrary complexity >> >> Lets keep it the last resort. >> >> >> >> If the bottom-line is that python's GUI-builders are so deep into suxland >> >> that they are best avoided in place of hand-written code, thats fine (by me). >> >> >> >> Lets please not make a virtue of a lack >> > >> >Once someone figures out a way to usefully merge independent changes >> >(you know, the way source control tools do every single day for code), >> >maybe I'll consider that. Until then, the last resort is also my first >> >response. >> >> Why can't whatever is generated by a GUI form designer be stored in >> source control along with all the other project files? The only >> restriction would be that everyone who wants to change the UI would >> have to use the same form designer. > >Glade generates XML (last I saw) >XML is text... kinda... but not quite >eg XML is sometimes/somewhere space sensitive, sometimes not >This can generate messy diffs That is also true of Python code but does not preclude effective source control. > >Best I can see, these are not exactly trivial nor quite impossible to solve problems From rustompmody at gmail.com Sun Feb 28 08:49:38 2016 From: rustompmody at gmail.com (Rustom Mody) Date: Sun, 28 Feb 2016 05:49:38 -0800 (PST) Subject: Everything good about Python except GUI IDE? In-Reply-To: <93t5db9sib9ldgktrt7523fnis4tgq2uev@4ax.com> References: <64a6599c-fae1-469d-bcee-875165b3cc7d@googlegroups.com> <56d294f8$0$1604$c3e8da3$5496439d@news.astraweb.com> <62084c14-abd1-4214-af08-70ce8449c83e@googlegroups.com> <6dq5db5j0hg2evl7t334ftdm5sk8n5itge@4ax.com> <93t5db9sib9ldgktrt7523fnis4tgq2uev@4ax.com> Message-ID: <9064f06c-cbd5-4bf4-98d7-24bed0e78c10@googlegroups.com> On Sunday, February 28, 2016 at 6:54:40 PM UTC+5:30, Gordon Levi wrote: > Rustom Mody wrote: > >Glade generates XML (last I saw) > >XML is text... kinda... but not quite > >eg XML is sometimes/somewhere space sensitive, sometimes not > >This can generate messy diffs > > That is also true of Python code but does not preclude effective > source control. Yes as I said its not satisfactory but not impossible to manage Heck Current state of art VCSes cannot even manage mismatching EOL conventions cleanly. And as usual they make a virtue out of the lack: "git stores binary data not text" which means that opening a file created on windows on linux and saving it in WITHOUT a SINGLE CHANGE can give you a 10,000 line diff!! From marko at pacujo.net Sun Feb 28 08:51:57 2016 From: marko at pacujo.net (Marko Rauhamaa) Date: Sun, 28 Feb 2016 15:51:57 +0200 Subject: Everything good about Python except GUI IDE? References: <64a6599c-fae1-469d-bcee-875165b3cc7d@googlegroups.com> <56d294f8$0$1604$c3e8da3$5496439d@news.astraweb.com> <234a398e-1b0f-467b-a8cb-d7ca748f8062@googlegroups.com> <84922f24-3e00-4a23-b26d-5e6c0d8e7e04@googlegroups.com> Message-ID: <87y4a5c58i.fsf@elektro.pacujo.net> Rustom Mody : > whereas in fact every significant GUI embeds text (possibly recursively) > > eg TI inside GUI -- think of text inside gimp > GUI inside TI -- think of Word embedding other doc types including pictures > Which can be recursive -- WOrd embeds a picture embeds text Sigh, still nobody has mentioned an exemplary GUI application. An anecdote: Some weeks back my son and I were struggling to get the right kind of graph out of Excel. After four hours of Google, Youtube, trial and error, we gave up, took out a pad of millimeter paper and some colored pencils. The whole job took my son an hour and the end result looked great. He snapped a picture and sent it to the teacher by email. A 2nd anecdote. I occasionally need to make technical presentations to an audience. Do I use PowerPoint or Impress? No, I use emacs, M-x picture-mode and raw HTML (without styles). I get to concentrate on producing effective communication, and nobody has complained about the lack of imagery or funny animation. Marko From kwpolska at gmail.com Sun Feb 28 09:00:37 2016 From: kwpolska at gmail.com (Chris Warrick) Date: Sun, 28 Feb 2016 15:00:37 +0100 Subject: Everything good about Python except GUI IDE? In-Reply-To: <9064f06c-cbd5-4bf4-98d7-24bed0e78c10@googlegroups.com> References: <64a6599c-fae1-469d-bcee-875165b3cc7d@googlegroups.com> <56d294f8$0$1604$c3e8da3$5496439d@news.astraweb.com> <62084c14-abd1-4214-af08-70ce8449c83e@googlegroups.com> <6dq5db5j0hg2evl7t334ftdm5sk8n5itge@4ax.com> <93t5db9sib9ldgktrt7523fnis4tgq2uev@4ax.com> <9064f06c-cbd5-4bf4-98d7-24bed0e78c10@googlegroups.com> Message-ID: On 28 February 2016 at 14:49, Rustom Mody wrote: > On Sunday, February 28, 2016 at 6:54:40 PM UTC+5:30, Gordon Levi wrote: >> Rustom Mody wrote: >> >Glade generates XML (last I saw) >> >XML is text... kinda... but not quite >> >eg XML is sometimes/somewhere space sensitive, sometimes not >> >This can generate messy diffs >> >> That is also true of Python code but does not preclude effective >> source control. > > Yes as I said its not satisfactory but not impossible to manage > > Heck Current state of art VCSes cannot even manage mismatching EOL conventions > cleanly. > And as usual they make a virtue out of the lack: > "git stores binary data not text" > > which means that opening a file created on windows on linux and saving it in > WITHOUT a SINGLE CHANGE > can give you a 10,000 line diff!! You clearly haven?t ever done that. 1. git can manage EOL changing if you want to enforce a newline style that way. 2. A good editor can read and write any newline style. It should also not convert without asking the user. -- Chris Warrick PGP: 5EAAEA16 From rustompmody at gmail.com Sun Feb 28 09:03:09 2016 From: rustompmody at gmail.com (Rustom Mody) Date: Sun, 28 Feb 2016 06:03:09 -0800 (PST) Subject: Everything good about Python except GUI IDE? In-Reply-To: <87y4a5c58i.fsf@elektro.pacujo.net> References: <64a6599c-fae1-469d-bcee-875165b3cc7d@googlegroups.com> <56d294f8$0$1604$c3e8da3$5496439d@news.astraweb.com> <234a398e-1b0f-467b-a8cb-d7ca748f8062@googlegroups.com> <84922f24-3e00-4a23-b26d-5e6c0d8e7e04@googlegroups.com> <87y4a5c58i.fsf@elektro.pacujo.net> Message-ID: On Sunday, February 28, 2016 at 7:22:08 PM UTC+5:30, Marko Rauhamaa wrote: > Rustom Mody : > > > whereas in fact every significant GUI embeds text (possibly recursively) > > > > eg TI inside GUI -- think of text inside gimp > > GUI inside TI -- think of Word embedding other doc types including pictures > > Which can be recursive -- WOrd embeds a picture embeds text > > Sigh, still nobody has mentioned an exemplary GUI application. > > An anecdote: Some weeks back my son and I were struggling to get the > right kind of graph out of Excel. After four hours of Google, Youtube, > trial and error, we gave up, took out a pad of millimeter paper and some > colored pencils. The whole job took my son an hour and the end result > looked great. He snapped a picture and sent it to the teacher by email. Point being?? If I tried to speak Finnish I'd look a fool. Makes me a fool? Best I can see you dont know excel [Nor do I] Lets guess that you've spent 10 hours struggling with excel Now compare with how much with programming and 'classical' CS Would a couple of thousand hours be an overestimate? Is the inherent difficulty of excel to programming consistent with that ratio? > > A 2nd anecdote. I occasionally need to make technical presentations to > an audience. Do I use PowerPoint or Impress? No, I use emacs, M-x > picture-mode and raw HTML (without styles). I get to concentrate on > producing effective communication, and nobody has complained about the > lack of imagery or funny animation. Yeah so do I emacs ? org mode ? export html Works but not ideal From rustompmody at gmail.com Sun Feb 28 09:11:54 2016 From: rustompmody at gmail.com (Rustom Mody) Date: Sun, 28 Feb 2016 06:11:54 -0800 (PST) Subject: Everything good about Python except GUI IDE? In-Reply-To: References: <64a6599c-fae1-469d-bcee-875165b3cc7d@googlegroups.com> <56d294f8$0$1604$c3e8da3$5496439d@news.astraweb.com> <62084c14-abd1-4214-af08-70ce8449c83e@googlegroups.com> <6dq5db5j0hg2evl7t334ftdm5sk8n5itge@4ax.com> <93t5db9sib9ldgktrt7523fnis4tgq2uev@4ax.com> <9064f06c-cbd5-4bf4-98d7-24bed0e78c10@googlegroups.com> Message-ID: <90f65ae1-c3d8-4a36-bc9f-860403a0633c@googlegroups.com> On Sunday, February 28, 2016 at 7:30:57 PM UTC+5:30, Chris Warrick wrote: > On 28 February 2016 at 14:49, Rustom Mody wrote: > > On Sunday, February 28, 2016 at 6:54:40 PM UTC+5:30, Gordon Levi wrote: > >> Rustom Mody wrote: > >> >Glade generates XML (last I saw) > >> >XML is text... kinda... but not quite > >> >eg XML is sometimes/somewhere space sensitive, sometimes not > >> >This can generate messy diffs > >> > >> That is also true of Python code but does not preclude effective > >> source control. > > > > Yes as I said its not satisfactory but not impossible to manage > > > > Heck Current state of art VCSes cannot even manage mismatching EOL conventions > > cleanly. > > And as usual they make a virtue out of the lack: > > "git stores binary data not text" > > > > which means that opening a file created on windows on linux and saving it in > > WITHOUT a SINGLE CHANGE > > can give you a 10,000 line diff!! > > > 2. A good editor can read and write any newline style. It should also > not convert without asking the user. git is a *collaborative* tool and should work when the other party is using notepad. > 1. git can manage EOL changing if you want to enforce a newline style that way. Only out-of-band You store autocrlf etc in your config not in the repo [And pray that the other (semi-literate) collaborator does likewise] > You clearly haven't ever done that. You specialize in crystal balls? Here's my report about CRLF issues in CPython https://mail.python.org/pipermail/python-dev/2015-June/140563.html Bug report: http://bugs.python.org/issue24507 From kwpolska at gmail.com Sun Feb 28 09:26:52 2016 From: kwpolska at gmail.com (Chris Warrick) Date: Sun, 28 Feb 2016 15:26:52 +0100 Subject: Everything good about Python except GUI IDE? In-Reply-To: <90f65ae1-c3d8-4a36-bc9f-860403a0633c@googlegroups.com> References: <64a6599c-fae1-469d-bcee-875165b3cc7d@googlegroups.com> <56d294f8$0$1604$c3e8da3$5496439d@news.astraweb.com> <62084c14-abd1-4214-af08-70ce8449c83e@googlegroups.com> <6dq5db5j0hg2evl7t334ftdm5sk8n5itge@4ax.com> <93t5db9sib9ldgktrt7523fnis4tgq2uev@4ax.com> <9064f06c-cbd5-4bf4-98d7-24bed0e78c10@googlegroups.com> <90f65ae1-c3d8-4a36-bc9f-860403a0633c@googlegroups.com> Message-ID: On 28 February 2016 at 15:11, Rustom Mody wrote: > On Sunday, February 28, 2016 at 7:30:57 PM UTC+5:30, Chris Warrick wrote: >> On 28 February 2016 at 14:49, Rustom Mody wrote: >> > On Sunday, February 28, 2016 at 6:54:40 PM UTC+5:30, Gordon Levi wrote: >> >> Rustom Mody wrote: >> >> >Glade generates XML (last I saw) >> >> >XML is text... kinda... but not quite >> >> >eg XML is sometimes/somewhere space sensitive, sometimes not >> >> >This can generate messy diffs >> >> >> >> That is also true of Python code but does not preclude effective >> >> source control. >> > >> > Yes as I said its not satisfactory but not impossible to manage >> > >> > Heck Current state of art VCSes cannot even manage mismatching EOL conventions >> > cleanly. >> > And as usual they make a virtue out of the lack: >> > "git stores binary data not text" >> > >> > which means that opening a file created on windows on linux and saving it in >> > WITHOUT a SINGLE CHANGE >> > can give you a 10,000 line diff!! >> > >> >> 2. A good editor can read and write any newline style. It should also >> not convert without asking the user. > > git is a *collaborative* tool and should work when the other party is using > notepad. What should git do if someone saves, say, Ruby code as a .py file? Should it rename it? Or should it figure out an equivalent snippet of Python? You probably have some rules in your project such as ?Code must be written in Python? or ?Use 4-space soft tabs?. Your rulebook should also include ?Use an editor that understands LF line endings?. Notepad is a joke that you should not tolerate. Problem solved. (Notepad does not understand LF line endings and replaces them with boxes. I also don?t think a Notepad user is likely to provide good contributions, but that?s another thing) -- Chris Warrick PGP: 5EAAEA16 From bc at freeuk.com Sun Feb 28 09:29:18 2016 From: bc at freeuk.com (BartC) Date: Sun, 28 Feb 2016 14:29:18 +0000 Subject: Everything good about Python except GUI IDE? In-Reply-To: References: <64a6599c-fae1-469d-bcee-875165b3cc7d@googlegroups.com> <56d294f8$0$1604$c3e8da3$5496439d@news.astraweb.com> <234a398e-1b0f-467b-a8cb-d7ca748f8062@googlegroups.com> <84922f24-3e00-4a23-b26d-5e6c0d8e7e04@googlegroups.com> <87y4a5c58i.fsf@elektro.pacujo.net> Message-ID: On 28/02/2016 14:03, Rustom Mody wrote: > On Sunday, February 28, 2016 at 7:22:08 PM UTC+5:30, Marko Rauhamaa wrote: >> Rustom Mody : >> >>> whereas in fact every significant GUI embeds text (possibly recursively) >>> >>> eg TI inside GUI -- think of text inside gimp >>> GUI inside TI -- think of Word embedding other doc types including pictures >>> Which can be recursive -- WOrd embeds a picture embeds text >> >> Sigh, still nobody has mentioned an exemplary GUI application. >> >> An anecdote: Some weeks back my son and I were struggling to get the >> right kind of graph out of Excel. After four hours of Google, Youtube, >> trial and error, we gave up, took out a pad of millimeter paper and some >> colored pencils. The whole job took my son an hour and the end result >> looked great. He snapped a picture and sent it to the teacher by email. > > Point being?? > If I tried to speak Finnish I'd look a fool. > Makes me a fool? > > Best I can see you dont know excel [Nor do I] > Lets guess that you've spent 10 hours struggling with excel > Now compare with how much with programming and 'classical' CS > Would a couple of thousand hours be an overestimate? > Is the inherent difficulty of excel to programming consistent with that ratio? Some programs are just difficult to use. Yes maybe you can achieve something with enough knowledge and training, but then you're getting the result despite the obstacles put in your way rather than because the software is so helpful. 20 years ago, when these things were simpler, MS Word had a mind of its own even then. I had to produce a manual of few hundred pages, with diagrams and images, and it just wasn't going to happen. Not without spending a year on it. And employing someone to do it cost thousands. In the end I spent a week or two throwing together some simple mark-up language, written in my own editor, which was then processed by my own script language and ending up (via my own graphics software along the way) as Postscript. The results were perfect. (Have you ever had a situation where you have to edit a bit of text where a word is in italic or has some particular style. You delete the word, and try and add some more text, but it persists in using the style of the deleted text rather than the current style. You move further away; still the same. You end up deleting everything including every trace of that word and its style. Yet start typing on a now blank document, and it's still the wrong style! That's what I mean by these applications having minds of their own. And with a 300-page document you can't just start all over, you need something reliable, and not so smart.) -- Bartc From marko at pacujo.net Sun Feb 28 10:08:31 2016 From: marko at pacujo.net (Marko Rauhamaa) Date: Sun, 28 Feb 2016 17:08:31 +0200 Subject: Everything good about Python except GUI IDE? References: <64a6599c-fae1-469d-bcee-875165b3cc7d@googlegroups.com> <56d294f8$0$1604$c3e8da3$5496439d@news.astraweb.com> <234a398e-1b0f-467b-a8cb-d7ca748f8062@googlegroups.com> <84922f24-3e00-4a23-b26d-5e6c0d8e7e04@googlegroups.com> <87y4a5c58i.fsf@elektro.pacujo.net> Message-ID: <87twksdg9c.fsf@elektro.pacujo.net> Rustom Mody : > On Sunday, February 28, 2016 at 7:22:08 PM UTC+5:30, Marko Rauhamaa wrote: >> Rustom Mody : >> >> > whereas in fact every significant GUI embeds text (possibly >> > recursively) >> > >> > eg TI inside GUI -- think of text inside gimp GUI inside TI -- >> > think of Word embedding other doc types including pictures Which >> > can be recursive -- WOrd embeds a picture embeds text >> >> Sigh, still nobody has mentioned an exemplary GUI application. >> >> An anecdote: Some weeks back my son and I were struggling to get the >> right kind of graph out of Excel. After four hours of Google, >> Youtube, trial and error, we gave up, took out a pad of millimeter >> paper and some colored pencils. The whole job took my son an hour and >> the end result looked great. He snapped a picture and sent it to the >> teacher by email. > > Point being?? I'm awaiting for an opposite anecdote, a GUI app you absolutely love and recommend as a model for GUI developers. We are talking about GUI tools to produce what? Crap? Marko From rustompmody at gmail.com Sun Feb 28 11:41:21 2016 From: rustompmody at gmail.com (Rustom Mody) Date: Sun, 28 Feb 2016 08:41:21 -0800 (PST) Subject: Everything good about Python except GUI IDE? In-Reply-To: <87twksdg9c.fsf@elektro.pacujo.net> References: <64a6599c-fae1-469d-bcee-875165b3cc7d@googlegroups.com> <56d294f8$0$1604$c3e8da3$5496439d@news.astraweb.com> <234a398e-1b0f-467b-a8cb-d7ca748f8062@googlegroups.com> <84922f24-3e00-4a23-b26d-5e6c0d8e7e04@googlegroups.com> <87y4a5c58i.fsf@elektro.pacujo.net> <87twksdg9c.fsf@elektro.pacujo.net> Message-ID: <8801b430-f87c-4b42-8833-775b9f56d317@googlegroups.com> On Sunday, February 28, 2016 at 8:38:49 PM UTC+5:30, Marko Rauhamaa wrote: > Rustom Mody : > > > On Sunday, February 28, 2016 at 7:22:08 PM UTC+5:30, Marko Rauhamaa wrote: > >> Rustom Mody : > >> > >> > whereas in fact every significant GUI embeds text (possibly > >> > recursively) > >> > > >> > eg TI inside GUI -- think of text inside gimp GUI inside TI -- > >> > think of Word embedding other doc types including pictures Which > >> > can be recursive -- WOrd embeds a picture embeds text > >> > >> Sigh, still nobody has mentioned an exemplary GUI application. > >> > >> An anecdote: Some weeks back my son and I were struggling to get the > >> right kind of graph out of Excel. After four hours of Google, > >> Youtube, trial and error, we gave up, took out a pad of millimeter > >> paper and some colored pencils. The whole job took my son an hour and > >> the end result looked great. He snapped a picture and sent it to the > >> teacher by email. > > > > Point being?? > > I'm awaiting for an opposite anecdote, a GUI app you absolutely love and > recommend as a model for GUI developers. GUI app I absolutely love... um hard But what does it prove? Best I can see it proves I am an old (Unix) geezer brought up on text files :-) Still as an approximation... I edit audio with audacity And type music with musescore Both could be better (And Ive heard people swear by ardour... No experience myself) The alternative -- stay with lilypond workflow -- is too primitive In particular if musescore could input and output to lilypond or some such it would have been optimal ...for me; ie clickety clicking 100s of notes is painful compared to typing them in say emacs For a much more professional pianist friend: He connects a small 2 octave keyboard to his mac Plays out what he needs to enter And then cleans it up... in Sibelius > > We are talking about GUI tools to produce what? Crap? You (seem to) be saying that excel is crap. Do you know it well enough to make that judgement? Do you have equivalent functionality in some other tool? From rustompmody at gmail.com Sun Feb 28 11:50:11 2016 From: rustompmody at gmail.com (Rustom Mody) Date: Sun, 28 Feb 2016 08:50:11 -0800 (PST) Subject: Everything good about Python except GUI IDE? In-Reply-To: References: <64a6599c-fae1-469d-bcee-875165b3cc7d@googlegroups.com> <56d294f8$0$1604$c3e8da3$5496439d@news.astraweb.com> <62084c14-abd1-4214-af08-70ce8449c83e@googlegroups.com> <6dq5db5j0hg2evl7t334ftdm5sk8n5itge@4ax.com> <93t5db9sib9ldgktrt7523fnis4tgq2uev@4ax.com> <9064f06c-cbd5-4bf4-98d7-24bed0e78c10@googlegroups.com> <90f65ae1-c3d8-4a36-bc9f-860403a0633c@googlegroups.com> Message-ID: <8e62f384-c4c5-4429-a1bc-b7134b7cbbf4@googlegroups.com> On Sunday, February 28, 2016 at 7:57:17 PM UTC+5:30, Chris Warrick wrote: > (Notepad does not understand LF line endings and replaces them with > boxes. I also don't think a Notepad user is likely to provide good > contributions, but that's another thing) You seem to have not worked in a web dev team wherein the frontend guys only know 3 tools - dreamweaver - photoshop - notepad From breamoreboy at yahoo.co.uk Sun Feb 28 12:09:31 2016 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Sun, 28 Feb 2016 17:09:31 +0000 Subject: Everything good about Python except GUI IDE? In-Reply-To: References: <64a6599c-fae1-469d-bcee-875165b3cc7d@googlegroups.com> <56d294f8$0$1604$c3e8da3$5496439d@news.astraweb.com> <62084c14-abd1-4214-af08-70ce8449c83e@googlegroups.com> <6dq5db5j0hg2evl7t334ftdm5sk8n5itge@4ax.com> <93t5db9sib9ldgktrt7523fnis4tgq2uev@4ax.com> <9064f06c-cbd5-4bf4-98d7-24bed0e78c10@googlegroups.com> Message-ID: On 28/02/2016 14:00, Chris Warrick wrote: > On 28 February 2016 at 14:49, Rustom Mody wrote: >> On Sunday, February 28, 2016 at 6:54:40 PM UTC+5:30, Gordon Levi wrote: >>> Rustom Mody wrote: >>>> Glade generates XML (last I saw) >>>> XML is text... kinda... but not quite >>>> eg XML is sometimes/somewhere space sensitive, sometimes not >>>> This can generate messy diffs >>> >>> That is also true of Python code but does not preclude effective >>> source control. >> >> Yes as I said its not satisfactory but not impossible to manage >> >> Heck Current state of art VCSes cannot even manage mismatching EOL conventions >> cleanly. >> And as usual they make a virtue out of the lack: >> "git stores binary data not text" >> >> which means that opening a file created on windows on linux and saving it in >> WITHOUT a SINGLE CHANGE >> can give you a 10,000 line diff!! > > You clearly haven?t ever done that. > > 1. git can manage EOL changing if you want to enforce a newline style that way. > 2. A good editor can read and write any newline style. It should also > not convert without asking the user. > Those who can, do, those who can't, teach? -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From maillist at schwertberger.de Sun Feb 28 12:41:27 2016 From: maillist at schwertberger.de (Dietmar Schwertberger) Date: Sun, 28 Feb 2016 18:41:27 +0100 Subject: Everything good about Python except GUI IDE? In-Reply-To: References: <64a6599c-fae1-469d-bcee-875165b3cc7d@googlegroups.com> <56d294f8$0$1604$c3e8da3$5496439d@news.astraweb.com> <56D2E6AB.9090604@mail.de> <9ed69848-bfb2-a6ef-555e-01e9424875cc@schwertberger.de> Message-ID: <2e5e420d-79c0-fe71-4dc4-ebed58b6b24d@schwertberger.de> On 28.02.2016 18:03, Dennis Lee Bieber wrote: > Office 2002 (or was it 2003... Whatever I had on my WinXP box) was, I > thought, rather easy to work styles in... Automatically shifting from > header to body text style on ; Having first body paragraph after a > header use block format while subsequent paragraphs used indented format... The style editor was always crappy. Most people won't notice due to the lack of comparison. Word processors like Fireworkz and TechWriter got it right to handle layered styles and effects. Easy to edit and easy to apply. And also easy to get rid of them again with a region editor which made it easy to select all modifications that were applied to a text and to delete or modify them. In comparison, MS Word is a mechanical typewriter. Regards, Dietmar From rosuav at gmail.com Sun Feb 28 12:53:35 2016 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 29 Feb 2016 04:53:35 +1100 Subject: Everything good about Python except GUI IDE? In-Reply-To: <56d2ed33$0$1588$c3e8da3$5496439d@news.astraweb.com> References: <64a6599c-fae1-469d-bcee-875165b3cc7d@googlegroups.com> <56d294f8$0$1604$c3e8da3$5496439d@news.astraweb.com> <56d2ed33$0$1588$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Sun, Feb 28, 2016 at 11:50 PM, Steven D'Aprano wrote: > On Sun, 28 Feb 2016 07:44 pm, Chris Angelico wrote: > >> On Sun, Feb 28, 2016 at 5:34 PM, Steven D'Aprano >> wrote: > [...] >>> Drag-and-drop GUI builders have the same advantages over code as Python >>> has over languages with distinct compile/execute steps: rapid >>> development, prototyping, exploration and discovery. Of course, any >>> decent modern builder won't limit you to literally drag-and-drop, but >>> will offer functionality like duplicating elements, aligning them, >>> magnetic guides, etc. >> >> Alright, but how do you go about doing, with a drag-and-drop builder, >> all those things we're used to with code - composing new named actions >> out of primitives, observing the changes to a program through source >> control, unit testing (maybe), and code review? > > These are all good questions. Let's see if I can give good answers: > > (1) "composing new named actions" -- I'm not entirely sure what you mean. Do > you mean new named *widgets*? A good builder app should give you the > ability to Group widgets into a single element, this is functionality which > has existed in vector-drawing programs since at least MacDraw in 1984 so it > shouldn't be hard. This is composition, a fundamental, powerful and rich > design pattern for making new widgets (classes) out of simpler parts. If > objects have a name, now you can refer to CompositeMenuDateColourPicker by > name. You can copy it, paste it, replicate it 30 times, whatever you like. A good number of GUI builders do offer this functionality - composition. (Even some of what we would call primitives are actually composed of multiple widgets; a drop-down combo box is an entry field, a button, and a possibly-hidden list box.) But there are other actions than "put this widget here". For example, you could go and adjust some widget's size or style, or reposition it according to some new rules (hopefully most of your positioning rules can be codified, eg "use a grid layout, put this in this cell", but a lot of the people who ask for drag-and-drop GUI building are thinking in terms of "place this right here", rather than using layout managers; and sometimes there are rules that don't really fit the layout manager per se, or are layered on top of the layout manager's rules). These kinds of actions can be represented as functions and then applied everywhere, such that you can change the precise appearance by editing that one function. For example, I have a Dungeons & Dragons character sheet display, in which there are large numbers of entry fields (editable), labels (non-editable display, usually calculated from other fields), and less commonly, drop-down lists and multi-line text fields. Call that four primitives. Now, some of the fields need to be highlighted for the human's attention ("this is the actual value you want to be reading, most of the time"). Currently, I do this with a blue border around it and its label. Okay, so I can no doubt create a "readme display" widget that has the border (a GTK2 Frame) and two labels; and then a "readme editable" with the border, one label, and one entry field. That's already split it out into two options, where a parameterized function could simply generate a border, a label, and *whatever object it was given*. And what if I want to change the look of those readme objects? What if, instead of surrounding them with a Frame, I want to put a little icon to the right of them? With code, all I have to do is change the definition of the function; it still takes a widget and returns a widget, but now it returns (say) a horizontal box containing a label, the provided widget, and the icon. With a GUI builder, how do you redefine a function that isn't just simple composition? This additional meta-level is one that is absent from a *lot* of modern graphical environments. Look at spreadsheets - the classic Lotus 1-2-3 model has stayed with us all through, with MS Excel, OO/LO Calc, etc, etc generally following it. And that's fine; spreadsheets do a lot of very good work for a lot of people. Now, suppose in cell C2 you have this formula: "=A2+B2*.05". You can copy that down the rest of the column, and C9 will say "=A9+B9*.05", and so on; that's what spreadsheets do well. But once you've done that copy operation, those cells lose all track of the fact that they got their formula from C2. If you edit C2, you have to re-copy-down. That's not too hard for a lot of people to remember, but what happens when that's forgotten? In a spreadsheet with huge numbers of calculated cells, how are you going to debug the "oops I forgot to re-copy that" problem? There is nothing in the document itself that helps you; if you see that C2 has "=A2+B2*.051" and that this pattern continues down as far as C10 but no further, do you assume that someone was tinkering and didn't edit all down the column (which is tedious), or do you assume that it's intentional? (Of course, the tendency for spreadsheets to lack comments is a separate issue.) There is one spreadsheet (that I know of) that gives this extra meta-level: Mesa for OS/2. If you copy down like that, the cells will say "SAME(C2)" instead of actually copying anything. Then, if you change C2, it'll use that formula for all the others. Why is this a rare feature in spreadsheets? It's pretty obvious that you can do this in code - you simply call that function. In anything drag-and-drop, it has to be an explicitly provided feature. > (2) "source control" -- the world is full of document types that aren't > plain text source code, and people have found ways to manage collaborative > editing and change management for them. Why don't we ask game developers > how they manage changes to the non-code elements in their applications? > Textures, icons, player avatars, sound effects, maps, etc. Surely you don't > suggest that game developers edit their background images in a hex editor? Yes, and while you're at it, ask the critical question: What happens when two people make edits to the same object? Non-text source code is fine as long as only one person at a time is ever editing. Source control will happily track them. But merging is virtually impossible; taking one base case and two modified files and producing a unified result is hard for either a human or a machine, when the files are non-text blobs. My suspicion is that such game devs would have strict rules about simultaneous editing of files. While that quite probably works fairly well, it's a limitation that most of us would not accept in any code project. Imagine if the Python bug tracker required you to send, not a patch file, but the entire source file that has the edit you want. How would you manage those sorts of edits? > (3) "unit testing" -- I'm not sure that unit testing is what is needed for > the GUI elements of your application. It's more like integration testing, > to ensure that the entire application works together as a seamless whole. > I'm not sure what the state of the art in GUI application integration > testing is like. I suspect crap, judging by the state of the art in GUI > applications. > > But whatever it is, it will surely operate the same regardless of whether > you have built the application using code or a graphical builder. > > (The GUI framework itself may have some analogue to unit testing for the > individual widgets, but that's not your responsibility as the user of the > framework. It's not up to you to test that menus drop down when clicked, > but it is your responsibility to check that the right menus exist and that > they do what they are supposed to.) Right. Testing the widgets isn't your responsibility, although testing a composite widget might (imagine building a drop-down combo box out of the three parts - you should be able to test that in isolation). But I was stretching a bit here. Feel free to drop this one. > (4) "code review" -- the usual way to review the graphical elements of a GUI > app is to call somebody over, sit them down at the running application, and > say "What do you think of this?". They will usually answer "that icon needs > to be a bit more to the left, and can you make that button blue instead of > green?". That's if you review the entire app. I'm talking more about the equivalent of sharing patches; "here, I changed this window, do you accept my change or not?" is a hard question to ask if you can't see exactly what changed. Closely related to the source control issue - if you can make patches/diffs for one, you can do it for the other. >> The only way I know of >> to build a "function" in a DnD builder is to create a composite widget >> (eg "horizontal box containing label and entry field"), which is >> extremely useful, but limited - it's like saying that the only way to >> reuse code is single-inheritance. > > A poor analogy. Composition is equivalent to multiple inheritance, except > without any of the weaknesses of MI. The analogy is of having a grossly-restricted form of code reuse as your only form. Composition is one specific option, but that's not the only thing you can do with code. >> How would you create a higher-order >> operation in a DnD builder? How would you write something that does >> some sort of sweep over a set of widgets and does the same thing to >> them? > > In Hypercard, if it was a once-off processing task, I would create a button, > edit the button's script: > > on mouseUp: > -- my memory of HC syntax and functions is a bit rusty > -- this may not be correct > for btnNum in 1 to the number of buttons: > if btnNum is the number of me: > continue > end if > set the textsize of button btnNum to 9 > set the textstyle of button btnNum to bold,italic > if the name of button btnNum starts with "Spam": > set the icon of button btnNum to SpamIcon > end if > end for > end mouseUp > > then I would click on that button and run the script. Then, once I have > satisfied myself that it has done what was needed, I'd delete the button. This is like running a sed script over your code. That's still not the same thing as higher-order code. > If this was something that needed to run each time the application ran, I > would put the script in some other layer of the application, say, in the > card layer, in a named handler: > > on setBtnStyle: > for btnNum in 1 to the number of buttons: > set the textsize of button btnNum to 9 > set the textstyle of button btnNum to bold,italic > if the name of button btnNum starts with "Spam": > set the icon of button btnNum to SpamIcon > end if > end for > end setBtnStyle > > > then call that handler on some event: > > on openCard: > setBtnStyle > end openCard I suppose that's a reasonable simulation of higher-order code. If it's stuff you're happy to do at run time (ie it won't hurt that you can't see it in the editor), this will work. It's still not as good IMO though. >> This doesn't have to be a dichotomy. > > I didn't say it did :-) Good :) ChrisA From none at invalid.com Sun Feb 28 13:47:34 2016 From: none at invalid.com (mm0fmf) Date: Sun, 28 Feb 2016 18:47:34 +0000 Subject: Everything good about Python except GUI IDE? In-Reply-To: References: <64a6599c-fae1-469d-bcee-875165b3cc7d@googlegroups.com> Message-ID: On 27/02/2016 18:13, wrong.address.1 at gmail.com wrote: > On Saturday, 27 February 2016 18:08:36 UTC+2, Dietmar Schwertberger wrote: >> On 27.02.2016 12:18, wrong.address.1 at gmail.com wrote: >>> Isn't there any good GUI IDE like Visual Basic? I hope there are some less well known GUI IDEs which I did not come across. Thanks. >> >> As of today, there's no Python GUI builder comparable to VB 6. >> > > Thanks for stating this clearly. Everyone here has been trying to show me various ways to do the kind of things I will want to, but nobody clearly admits the limitations I will have to accept if I start with Python. > > I am starting to wonder if VB.net would be a better solution for the time being. I have learnt enough VB.net to manage my work but it is bloated and Microsoft dependent. > >> There are some like QtDesigner or wxGlade, but they either don't >> generate Python code directly or they can only be used if you know the >> underlying toolkit good enough to create the GUI yourself. You may try >> out some, but I can almost guarantee you that you will come to the same >> result. >> If you want a GUI, create it yourself using either wxPython or PyQt. > > I will check it. I got the impression that you can create a GUI but that has to be converted to Python, and then you need a wrapper to put these forms in, and then they can be compiled or converted to *.exe with py2exe. Not a good way for development/debugging. > >> >> For engineering applications that's probably the weakest point that >> Python has. >> It's holding back a lot of people... >> >> Well, for most measurement or control software a GUI is not really >> needed, but still people want it. >> > > In the 1980s everyone was happy with inputs from the command line on a line editor, but today people expect GUIs with graphics and often even animations. > > It is surprising that a language which seems very popular does not have GUI development infrastructure in place these many years after it got into common use. > >> >> Regards, >> >> Dietmar > I'm no C# expert but I inherited the support of some C# projects. One uses a form to hold the UI objects. When the program is loaded in VS, you see the form and you can drag and drop objects to the form and edit the object properties (text, font, colours etc.). The result of your visual work is rendered in the C# source with some code folding options. If you don't click the folds in the editor you don't get to see that the form editor generates the C# code you need to call to generate the objects. There are suitable comments through the generated code warning you not to edit it as it is regenerated etc. The result is you use a visual tool to generate the boiler plate code. Knowing MS tools I'd be very suprised if the same idea is not used in VB. Somewhere there will be a text file with the VB boilerplate code to generate the form. You need to locate that and use it to drive your VB to Python conversion process. I'd write myself a script that takes the autogenerated source and converts it to Python. You may need to polish the output but if you have hundreds of objects this sounds to be simpler and quicker than starting from scratch. This is from my experience using C# with ASP.NET and .Net 4.0+ and VS2010 Professional. I think I've played with VS2013 Pro and it's the same. YMMV. From maillist at schwertberger.de Sun Feb 28 14:09:51 2016 From: maillist at schwertberger.de (Dietmar Schwertberger) Date: Sun, 28 Feb 2016 20:09:51 +0100 Subject: Everything good about Python except GUI IDE? In-Reply-To: References: <64a6599c-fae1-469d-bcee-875165b3cc7d@googlegroups.com> Message-ID: On 28.02.2016 19:47, mm0fmf wrote: > I'm no C# expert but I inherited the support of some C# projects. One > uses a form to hold the UI objects. When the program is loaded in VS, > you see the form and you can drag and drop objects to the form and > edit the object properties (text, font, colours etc.). The result of > your visual work is rendered in the C# source with some code folding > options. If you don't click the folds in the editor you don't get to > see that the form editor generates the C# code you need to call to > generate the objects. There are suitable comments through the > generated code warning you not to edit it as it is regenerated etc. > > The result is you use a visual tool to generate the boiler plate code. > Knowing MS tools I'd be very suprised if the same idea is not used in > VB. Somewhere there will be a text file with the VB boilerplate code > to generate the form. The VB 6 setup was different. GUI editor, IDE and runtime were tightly integrated. In VB 6, you don't see such boiler plate code. You only see the form, the textual representation of the form and the code that you enter into the event handlers etc. The form is then rendered by the VB runtime. But what you outlined for C# is probably the way that the ideal Python GUI editor would go. When you think about how a RAD tool could look like and how to integrate with IDEs, you automatically come to such a setup with comments as markers/separators for the automatically generated code. Regards, Dietmar From oscar.j.benjamin at gmail.com Sun Feb 28 14:28:04 2016 From: oscar.j.benjamin at gmail.com (Oscar Benjamin) Date: Sun, 28 Feb 2016 19:28:04 +0000 Subject: list index out of range Error , need to fix it or ignore it In-Reply-To: References: Message-ID: On 27 February 2016 at 16:50, Ganesh Pal wrote: > Iam on python 2.6 and Linux , I need input on the below program , > here is the spinet of my program It would be much better if you presented a complete program here. Otherwise the missing parts will confuse people. See: http://sscce.org/ > filename='/tmp2/2.txt' > > def check_file(): > """ > Run the command parallel on all the machines , if there is a > file named /tmp/file2.txt extract file2.txt > > """ > global filename > baddr = '' > cmd = ("run_al_paral 'ls -al %s'" % (filename)) > print(cmd) > stdout, stderr, exitcode = run(cmd) > print(stdout) > lines = stdout.strip().split('\n') .splitlines() is more portable than .split('\n') as it handles '\r' as well. > print(lines) There are a few problems with the code below: > for line in lines: > if 'exited' in lines: > continue > > file = lines[0].split()[9][6:] Okay so you're skipping lines that contain the text "exited" in order to find the line you want. That's fine except that you then try to access the line you've found as lines[0] rather than line (singular). This means that although your loop continues when the line contains "exited" you're still accessing the first element of lines (which may be one of the ones that contains "exited") in any case. I think that you meant line rather than lines[0] here. You're assuming that lines has at least one element. Otherwise lines[0] would give IndexError. This won't raise IndexError since the line of code "file = ..." is only executed inside the body of the for loop which means that if lines is empty then execution doesn't reach this point. I don't think that that was your intention though as really it should be line rather than lines[0]. You're then assuming that lines[0].split() (which should really be line.split())) has at least 10 elements since you ask for lines[0].split()[9]. This will raise IndexError if .split() returnd fewer than 10 substrings. This is most likely the IndexError that you get. Your incorrect use of line rather than lines[0] means you're reading one of the lines that contains "exited" and so when you call .split() on that it gives fewer than 10 elements and so the last index in lines[0].split()[9] raises IndexError. > break > print file Here you're assuming that lines contains at least one string that does not contain the substring "exited". Otherwise this line will leas to a NameError since the name file is not bound to anything unless the "file=" line is executed. > return file > > def main(): > functions = [check_file] > for func in functions: > try: > func() > except Exception as e: > return False Here you've crippled Python's ability to give quite useful error messages. Replace the above with: def main(): check_file() and then Python will print a helpful error message. More than that if you learn to use a debugger you can hook into the exact moment where an *uncaught* error is raised. Since you catch the error that won't be possible. The traceback that Python prints *if you don't catch the error* contains useful information about the problem. Even if you don't understand that information other people do so if you post it here then you can get much better help. It shows the line at which the error occurred and has more description than simply IndexError. If you remove the try/except you can then use pdb. Just rerun your code as $ python -m pdb myscript.py Type "c" (and hit enter) to "continue". This will run your script right up to the point of the uncaught exception and then hook you into a debugger at the moment the exception was raised. You can then use "l" to show the surrounding code. Use "p file" to show the value of for example the file variable if that seems to be the problem. -- Oscar From tomwilliamson115 at gmail.com Sun Feb 28 14:58:45 2016 From: tomwilliamson115 at gmail.com (tomwilliamson115 at gmail.com) Date: Sun, 28 Feb 2016 11:58:45 -0800 (PST) Subject: Help Message-ID: <6cdcd063-1c22-4f08-900c-f3260a543018@googlegroups.com> I need to write a program that allows the user to enter a sentence then asks them which word they wish to find- and then tell them the position that word was within the sentence. E.g. Please help with this code Then they enter help it would return that it appears in the 2nd position in the sentence. >From what I gather it appears to be a list function but I am struggling to come up with a solution. Thanks. From oscar.j.benjamin at gmail.com Sun Feb 28 15:28:29 2016 From: oscar.j.benjamin at gmail.com (Oscar Benjamin) Date: Sun, 28 Feb 2016 20:28:29 +0000 Subject: How to read from a file to an arbitrary delimiter efficiently? In-Reply-To: <56cea44e$0$11128$c3e8da3@news.astraweb.com> References: <56cea44e$0$11128$c3e8da3@news.astraweb.com> Message-ID: On 25 February 2016 at 06:50, Steven D'Aprano wrote: > > I have a need to read to an arbitrary delimiter, which might be any of a > (small) set of characters. For the sake of the exercise, lets say it is > either ! or ? (for example). > > I want to read from files reasonably efficiently. I don't mind if there is a > little overhead, but my first attempt is 100 times slower than the built-in > "read to the end of the line" method. You can get something much faster using mmap and searching for a single delimiter: def readuntil(m, delim): start = m.tell() index = m.find(delim, start) if index == -1: return m.read() else: return m.read(index - start) def readmmap(f): m = mmap.mmap(f.fileno(), 0, access=mmap.ACCESS_READ) f.seek(0) while True: chunk = readuntil(m, b'!') # Note byte-string if not chunk: return # Do stuff with chunk pass My timing makes that ~7x slower than iterating over the lines of the file but still around 100x faster than reading individual characters. I'm not sure how to generalise it to looking for multiple delimiters without dropping back to reading individual characters though. -- Oscar From jacob at blindza.co.za Sun Feb 28 15:50:47 2016 From: jacob at blindza.co.za (jacob Kruger) Date: Sun, 28 Feb 2016 22:50:47 +0200 Subject: Help In-Reply-To: <6cdcd063-1c22-4f08-900c-f3260a543018@googlegroups.com> References: <6cdcd063-1c22-4f08-900c-f3260a543018@googlegroups.com> Message-ID: <56D35DA7.1000509@blindza.co.za> On 2016-02-28 9:58 PM, tomwilliamson115 at gmail.com wrote: > I need to write a program that allows the user to enter a sentence then asks them which word they wish to find- and then tell them the position that word was within the sentence. > > E.g. Please help with this code > Then they enter help it would return that it appears in the 2nd position in the sentence. > >>From what I gather it appears to be a list function but I am struggling to come up with a solution. > > Thanks. > Something along lines of loading the sentence into a string, using the str.split() function to split it into a list object, then cycling through to strip all the spaces and/or punctuation out of the elements/items, and then the list.index() function can return the index of a word in the list, which is the zero-based position of the item in that list. #something along lines of s = input("enter sentence") s.replace(",", "") #do this for all common punctuation characters l = s.split(" ") #space is actually default s2 = input("enter word") i = l.index(s2) print("Your word is at position " + str(i+1) + " in the sentence") #end code That's just typed here in message - HTH HTH Jacob Kruger Blind Biker Skype: BlindZA "Roger Wilco wants to welcome you...to the space janitor's closet..." From timothy.c.delaney at gmail.com Sun Feb 28 16:00:22 2016 From: timothy.c.delaney at gmail.com (Tim Delaney) Date: Mon, 29 Feb 2016 08:00:22 +1100 Subject: How to read from a file to an arbitrary delimiter efficiently? In-Reply-To: References: <56cea44e$0$11128$c3e8da3@news.astraweb.com> Message-ID: On 29 February 2016 at 07:28, Oscar Benjamin wrote: > On 25 February 2016 at 06:50, Steven D'Aprano > wrote: > > > > I have a need to read to an arbitrary delimiter, which might be any of a > > (small) set of characters. For the sake of the exercise, lets say it is > > either ! or ? (for example). > > > > I want to read from files reasonably efficiently. I don't mind if there > is a > > little overhead, but my first attempt is 100 times slower than the > built-in > > "read to the end of the line" method. > > You can get something much faster using mmap and searching for a > single delimiter: > > My timing makes that ~7x slower than iterating over the lines of the > file but still around 100x faster than reading individual characters. > I'm not sure how to generalise it to looking for multiple delimiters > without dropping back to reading individual characters though. > You can use an mmapped file as the input for regular expressions. May or may not be particularly efficient. Otherwise, if reading from a file I think read a chunk, and seek() back to the delimiter is probably going to be most efficient whilst leaving the file position just after the delimiter. If reading from a stream, I think Chris' read a chunk and maintain an internal buffer, and don't give access to the underlying stream. Tim Delaney From marko at pacujo.net Sun Feb 28 16:38:22 2016 From: marko at pacujo.net (Marko Rauhamaa) Date: Sun, 28 Feb 2016 23:38:22 +0200 Subject: Everything good about Python except GUI IDE? References: <64a6599c-fae1-469d-bcee-875165b3cc7d@googlegroups.com> <56d294f8$0$1604$c3e8da3$5496439d@news.astraweb.com> <234a398e-1b0f-467b-a8cb-d7ca748f8062@googlegroups.com> <84922f24-3e00-4a23-b26d-5e6c0d8e7e04@googlegroups.com> <87y4a5c58i.fsf@elektro.pacujo.net> <87twksdg9c.fsf@elektro.pacujo.net> <8801b430-f87c-4b42-8833-775b9f56d317@googlegroups.com> Message-ID: <87k2locy7l.fsf@elektro.pacujo.net> Rustom Mody : > On Sunday, February 28, 2016 at 8:38:49 PM UTC+5:30, Marko Rauhamaa wrote: >> We are talking about GUI tools to produce what? Crap? > > You (seem to) be saying that excel is crap. > Do you know it well enough to make that judgement? > Do you have equivalent functionality in some other tool? Well, now you mention it, I have a third anecdote. For years I was the treasurer of a soccer team. My predecessors had used Excel for the purpose, with varying success. The accounting software I chose for the job was -- Python. I entered the transactions in Python and had the main program output all the needed reports: Excel, HTML, email. I believe it would have been a nightmare to do it any other way. As for Excel being crap, I have seen admirable, heroic, colossal efforts put in a lot of GUI apps (web or otherwise) but the results still coming short: Excel, Atlassian, Eclipse. Thing is, I wonder if the whole GUI paradigm is only suitable for very simple, naive things. An exception to that comment would be games. They really are often awe-inspiring achievements. Marko From steve at pearwood.info Sun Feb 28 19:14:54 2016 From: steve at pearwood.info (Steven D'Aprano) Date: Mon, 29 Feb 2016 11:14:54 +1100 Subject: Everything good about Python except GUI IDE? References: <64a6599c-fae1-469d-bcee-875165b3cc7d@googlegroups.com> <56d294f8$0$1604$c3e8da3$5496439d@news.astraweb.com> <234a398e-1b0f-467b-a8cb-d7ca748f8062@googlegroups.com> <84922f24-3e00-4a23-b26d-5e6c0d8e7e04@googlegroups.com> <87y4a5c58i.fsf@elektro.pacujo.net> Message-ID: <56d38d83$0$1594$c3e8da3$5496439d@news.astraweb.com> On Mon, 29 Feb 2016 12:51 am, Marko Rauhamaa wrote: > Rustom Mody : > >> whereas in fact every significant GUI embeds text (possibly recursively) >> >> eg TI inside GUI -- think of text inside gimp >> GUI inside TI -- think of Word embedding other doc types including >> pictures Which can be recursive -- WOrd embeds a picture embeds text > > Sigh, still nobody has mentioned an exemplary GUI application. Possibly because of the poor state of GUI application development, I don't think there are many. There are many which are *acceptable*, to some lesser or greater degree, but none that come to mind which make me go "wow, that's amazing!". Or perhaps I'm just jaded after 30+ years of using GUI apps and can only see the bugs and not the features any more. > An anecdote: Some weeks back my son and I were struggling to get the > right kind of graph out of Excel. After four hours of Google, Youtube, > trial and error, we gave up, took out a pad of millimeter paper and some > colored pencils. The whole job took my son an hour and the end result > looked great. He snapped a picture and sent it to the teacher by email. I feel that the state of the art of graphing tools is horrid. It's acceptable for business-style graphs (e.g. barcharts and pie charts) with just a handful of items but rapidly goes downhill from there. There is too much emphasis on making the graph look "fancy" at the expense of communicating information (3D pie charts, really?). But for graphing mathematical functions, it's just horrible. A good graphing application should: - by default show the zero point on the Y-axis; - by default scale the Y-axis so as to fit the entire graph; - label the axes in such a way that they are readable; - be as easy as saying "graph f(x) for x between a and b" and have the graphing application pick sensible defaults; - allow easy discovery of features (which usually means a graphical interface -- e.g. click on the axis and be presented with options to modify that axis). > A 2nd anecdote. I occasionally need to make technical presentations to > an audience. Do I use PowerPoint or Impress? No, I use emacs, M-x > picture-mode and raw HTML (without styles). I get to concentrate on > producing effective communication, and nobody has complained about the > lack of imagery or funny animation. You don't know if they haven't complained because they don't miss them, or haven't complained because they fell asleep and are too embarrassed to complain "your presentation put me to sleep due to the lack of dancing cows". :-) -- Steven From steve at pearwood.info Sun Feb 28 19:39:13 2016 From: steve at pearwood.info (Steven D'Aprano) Date: Mon, 29 Feb 2016 11:39:13 +1100 Subject: Everything good about Python except GUI IDE? References: <64a6599c-fae1-469d-bcee-875165b3cc7d@googlegroups.com> <56d294f8$0$1604$c3e8da3$5496439d@news.astraweb.com> <62084c14-abd1-4214-af08-70ce8449c83e@googlegroups.com> <6dq5db5j0hg2evl7t334ftdm5sk8n5itge@4ax.com> <93t5db9sib9ldgktrt7523fnis4tgq2uev@4ax.com> <9064f06c-cbd5-4bf4-98d7-24bed0e78c10@googlegroups.com> <90f65ae1-c3d8-4a36-bc9f-860403a0633c@googlegroups.com> Message-ID: <56d39335$0$1622$c3e8da3$5496439d@news.astraweb.com> On Mon, 29 Feb 2016 01:26 am, Chris Warrick wrote: >> git is a *collaborative* tool and should work when the other party is >> using notepad. > > What should git do if someone saves, say, Ruby code as a .py file? > Should it rename it? Or should it figure out an equivalent snippet of > Python? Don't be ridiculous. That's completely over the top. It isn't asking too much for version control systems to *not care* about line ending changes. Who cares if the file changes from \n to \r \r\n? It shouldn't matter, or at least, it shouldn't matter much. > You probably have some rules in your project such as ?Code must be > written in Python? In one of my Python projects, I have (as well as various .py files) an R script and a Gnumeric spreadsheet. In another Python project, I have a couple of bash scripts. In nearly every one of my repos, I have plain text files that don't contain Python code. Should the VCS refuse to track them because they don't contain Python code? Of course not. If I open a Python file in my editor, accidentally or deliberately change the content to Ruby code, save and commit, then the VCS obviously should track the changes because they are actual changes. But if I open a Python file in my editor, accidentally or deliberately change the line endings from \n to \r, save and commit, then it is a weakness and limitation of the current generation of VCSs that they will treat this as "every line has changed". Or worse, "10,000 lines have collapsed to a single line containing \r characters". Changing line endings is neither a structural nor a semantic change to the content of the file. It's effectively metadata, not data. The Python interpreter doesn't care which line ending you use. Neither will decent text editors. If some tools, like git, do, then that's a weakness of git, not a feature. Changing the permissions on a 10,000 line file doesn't give you a 10,000 line diff, and neither should changing the line ending. Your VCS absolutely should track line ending changes. In a perfect world, we should never care about the line ending, but so long as there are users and tools that cannot transparently deal with one line ending or another, there may be times were we do care about line endings, and therefore want to run "git blame" to find out which idiot changed the line ending of my file to \r. But that should be treated as a metadata change, not a change to the content. I know it isn't "really" metadata, it's "actually" content, but regardless, it should be *treated* as metadata. Who is the boss here? The user of the tool, or the tool? -- Steven From tjreedy at udel.edu Sun Feb 28 19:49:00 2016 From: tjreedy at udel.edu (Terry Reedy) Date: Sun, 28 Feb 2016 19:49:00 -0500 Subject: Everything good about Python except GUI IDE? In-Reply-To: References: <64a6599c-fae1-469d-bcee-875165b3cc7d@googlegroups.com> <56d294f8$0$1604$c3e8da3$5496439d@news.astraweb.com> <234a398e-1b0f-467b-a8cb-d7ca748f8062@googlegroups.com> <84922f24-3e00-4a23-b26d-5e6c0d8e7e04@googlegroups.com> <87y4a5c58i.fsf@elektro.pacujo.net> Message-ID: On 2/28/2016 9:29 AM, BartC wrote: > (Have you ever had a situation where you have to edit a bit of text > where a word is in italic or has some particular style. You delete the > word, and try and add some more text, but it persists in using the style > of the deleted text rather than the current style. You move further > away; still the same. You end up deleting everything including every > trace of that word and its style. Yet start typing on a now blank > document, and it's still the wrong style! Wordperfect had a Reveal Codes option that made it easy to exactly fix such mixups. I think it a tragedy that MS Word wiped it out. > That's what I mean by these applications having minds of their own. And > with a 300-page document you can't just start all over, you need > something reliable, and not so smart.) I did a 200 page book with WP. -- Terry Jan Reedy From steve at pearwood.info Sun Feb 28 19:49:10 2016 From: steve at pearwood.info (Steven D'Aprano) Date: Mon, 29 Feb 2016 11:49:10 +1100 Subject: Everything good about Python except GUI IDE? References: <64a6599c-fae1-469d-bcee-875165b3cc7d@googlegroups.com> <56d294f8$0$1604$c3e8da3$5496439d@news.astraweb.com> <234a398e-1b0f-467b-a8cb-d7ca748f8062@googlegroups.com> <84922f24-3e00-4a23-b26d-5e6c0d8e7e04@googlegroups.com> <87y4a5c58i.fsf@elektro.pacujo.net> Message-ID: <56d39589$0$1617$c3e8da3$5496439d@news.astraweb.com> On Mon, 29 Feb 2016 01:29 am, BartC wrote: > 20 years ago, when these things were simpler, MS Word had a mind of its > own even then. I had to produce a manual of few hundred pages, with > diagrams and images, and it just wasn't going to happen. Not without > spending a year on it. And employing someone to do it cost thousands. So you're saying that somebody else knew how to do it, but you didn't, so it would take you a year? I find it hard to believe that Word 20 years ago couldn't deal with something as small as a couple of hundred pages with diagrams. I find it much easier to believe that you're the sort of guy who would rather spend three days inventing your solution from scratch (involving your own custom programming language no less) than three hours reading the manual of the existing solution. > In the end I spent a week or two throwing together some simple mark-up > language, written in my own editor, which was then processed by my own > script language and ending up (via my own graphics software along the > way) as Postscript. The results were perfect. /s/three days/two weeks/ > (Have you ever had a situation where you have to edit a bit of text > where a word is in italic or has some particular style. You delete the > word, and try and add some more text, but it persists in using the style > of the deleted text rather than the current style. You move further > away; still the same. You end up deleting everything including every > trace of that word and its style. Yet start typing on a now blank > document, and it's still the wrong style! You know that Word lets you reset the style to plain (Roman) text? There's no need to delete the entire line, let alone the entire document. Text styles are toggled: just hit Ctrl-I (or is it Ctrl-Shift-I, it's been a while since I've used Word) to toggle italic on and off. Even if the current paragraph defaults to italic, you can still toggle it off that way. -- Steven From rosuav at gmail.com Sun Feb 28 19:54:36 2016 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 29 Feb 2016 11:54:36 +1100 Subject: Everything good about Python except GUI IDE? In-Reply-To: <56d39335$0$1622$c3e8da3$5496439d@news.astraweb.com> References: <64a6599c-fae1-469d-bcee-875165b3cc7d@googlegroups.com> <56d294f8$0$1604$c3e8da3$5496439d@news.astraweb.com> <62084c14-abd1-4214-af08-70ce8449c83e@googlegroups.com> <6dq5db5j0hg2evl7t334ftdm5sk8n5itge@4ax.com> <93t5db9sib9ldgktrt7523fnis4tgq2uev@4ax.com> <9064f06c-cbd5-4bf4-98d7-24bed0e78c10@googlegroups.com> <90f65ae1-c3d8-4a36-bc9f-860403a0633c@googlegroups.com> <56d39335$0$1622$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Mon, Feb 29, 2016 at 11:39 AM, Steven D'Aprano wrote: > If I open a Python file in my editor, accidentally or deliberately change > the content to Ruby code, save and commit, then the VCS obviously should > track the changes because they are actual changes. "Oops, I accidentally pressed Esc-Meta-Alt-R-Q and translated my code into Ruby. Now what?" > Changing line endings is neither a structural nor a semantic change to the > content of the file. It's effectively metadata, not data. The Python > interpreter doesn't care which line ending you use. Neither will decent > text editors. If some tools, like git, do, then that's a weakness of git, > not a feature. Changing the permissions on a 10,000 line file doesn't give > you a 10,000 line diff, and neither should changing the line ending. > > Your VCS absolutely should track line ending changes. In a perfect world, we > should never care about the line ending, but so long as there are users and > tools that cannot transparently deal with one line ending or another, there > may be times were we do care about line endings, and therefore want to > run "git blame" to find out which idiot changed the line ending of my file > to \r. But that should be treated as a metadata change, not a change to the > content. I know it isn't "really" metadata, it's "actually" content, but > regardless, it should be *treated* as metadata. > > Who is the boss here? The user of the tool, or the tool? This is a tricky question, and there's a slight difference between perms and line endings in that it's perfectly possible to have mixed line endings. The normal way to resolve this is to tell git to convert line endings *for text files* such that source control always uses \n, and then individual users get to choose whether to check out with \n or convert to \r\n for their working copies. This destroys any information that had been represented in mixed line endings - but that's information that no project I've ever worked with uses. However, since this is not something that git dares do on its own, it's not the default action - though the Git Windows installer does prompt you to pick those options on installation. Once you tell git to treat line endings the same way ("git config --global core.autocrlf true", if memory serves me), it's not even metadata, it's just non-information. ChrisA From ben+python at benfinney.id.au Sun Feb 28 20:05:32 2016 From: ben+python at benfinney.id.au (Ben Finney) Date: Mon, 29 Feb 2016 12:05:32 +1100 Subject: Everything good about Python except GUI IDE? References: <64a6599c-fae1-469d-bcee-875165b3cc7d@googlegroups.com> <56d294f8$0$1604$c3e8da3$5496439d@news.astraweb.com> <62084c14-abd1-4214-af08-70ce8449c83e@googlegroups.com> <6dq5db5j0hg2evl7t334ftdm5sk8n5itge@4ax.com> <93t5db9sib9ldgktrt7523fnis4tgq2uev@4ax.com> <9064f06c-cbd5-4bf4-98d7-24bed0e78c10@googlegroups.com> <90f65ae1-c3d8-4a36-bc9f-860403a0633c@googlegroups.com> <56d39335$0$1622$c3e8da3$5496439d@news.astraweb.com> Message-ID: <85k2loxr4z.fsf@benfinney.id.au> Steven D'Aprano writes: > Changing line endings is neither a structural nor a semantic change to > the content of the file. It's effectively metadata, not data. Hmm. Unlike other examples you give (like filesystem permissions on the file) the line endings *are* content in the file. You may say they're metadata, and maybe there's a case for that; I think that doesn't stop line endings in the file from being content ? and changes to those are changes to content. I'd say ?what line ending style is in use? is closer to the information about text encoding. Yes, it is metadata; it is *also* content, because a change to the text encoding must be reflected as a change to the file's content. The same is not true for (e.g.) filesystem permissions. I would say that a change to the text encoding in the file should be reflected as a change to the file content, and presented as such by the VCS. Do you agree? Whatever your answer to that, I would be interested to know whether you think the answer should be different for line ending changes. > Who is the boss here? The user of the tool, or the tool? We are unfortunately a slave to decisions made long in the past, to record some metadata ? line endings, text encoding ? as in-band content rather than out-of-band pure metadata. The VCS must, IMO, be at least aware that the content has changed when those in-band data change. -- \ ?Visitors are expected to complain at the office between the | `\ hours of 9 and 11 a.m. daily.? ?hotel, Athens | _o__) | Ben Finney From rosuav at gmail.com Sun Feb 28 20:13:24 2016 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 29 Feb 2016 12:13:24 +1100 Subject: Everything good about Python except GUI IDE? In-Reply-To: <85k2loxr4z.fsf@benfinney.id.au> References: <64a6599c-fae1-469d-bcee-875165b3cc7d@googlegroups.com> <56d294f8$0$1604$c3e8da3$5496439d@news.astraweb.com> <62084c14-abd1-4214-af08-70ce8449c83e@googlegroups.com> <6dq5db5j0hg2evl7t334ftdm5sk8n5itge@4ax.com> <93t5db9sib9ldgktrt7523fnis4tgq2uev@4ax.com> <9064f06c-cbd5-4bf4-98d7-24bed0e78c10@googlegroups.com> <90f65ae1-c3d8-4a36-bc9f-860403a0633c@googlegroups.com> <56d39335$0$1622$c3e8da3$5496439d@news.astraweb.com> <85k2loxr4z.fsf@benfinney.id.au> Message-ID: On Mon, Feb 29, 2016 at 12:05 PM, Ben Finney wrote: >> Who is the boss here? The user of the tool, or the tool? > > We are unfortunately a slave to decisions made long in the past, to > record some metadata ? line endings, text encoding ? as in-band content > rather than out-of-band pure metadata. The VCS must, IMO, be at least > aware that the content has changed when those in-band data change. Not sure what you mean here. It's metadata that lets you properly interpret the content; by definition, that has to be out of band information with in-band consequences. It's like using a shebang to interpret an executable file - you can't just casually change the first line from "#!/usr/bin/python2" to "#!/usr/bin/python3" and deem that your code has now been ported to Py3. ChrisA From torriem at gmail.com Sun Feb 28 20:24:29 2016 From: torriem at gmail.com (Michael Torrie) Date: Sun, 28 Feb 2016 18:24:29 -0700 Subject: Everything good about Python except GUI IDE? In-Reply-To: References: <64a6599c-fae1-469d-bcee-875165b3cc7d@googlegroups.com> Message-ID: <56D39DCD.4070107@gmail.com> On 02/27/2016 11:13 AM, wrong.address.1 at gmail.com wrote: > On Saturday, 27 February 2016 18:08:36 UTC+2, Dietmar Schwertberger >> As of today, there's no Python GUI builder comparable to VB 6. > Thanks for stating this clearly. Everyone here has been trying to > show me various ways to do the kind of things I will want to, but > nobody clearly admits the limitations I will have to accept if I > start with Python. Every language and environment has limitations and tradeoffs, including the venerable VB6 model. In my opinion, no one including Microsoft has ever duplicated the functionality of VB6. > I am starting to wonder if VB.net would be a better solution for the > time being. I have learnt enough VB.net to manage my work but it is > bloated and Microsoft dependent. Maybe you should post a small VB6 project on github (something that isn't very secret and could be licensed in an open way perhaps) and then see if we can come together here on the forum to convert it to Python using one of the various GUI toolkits. Could be a fun challenge and would be instructive to you. >> There are some like QtDesigner or wxGlade, but they either don't >> generate Python code directly or they can only be used if you know >> the underlying toolkit good enough to create the GUI yourself. You >> may try out some, but I can almost guarantee you that you will come >> to the same result. If you want a GUI, create it yourself using >> either wxPython or PyQt. > > I will check it. I got the impression that you can create a GUI but > that has to be converted to Python, and then you need a wrapper to > put these forms in, and then they can be compiled or converted to > *.exe with py2exe. Not a good way for development/debugging. You are laboring under some misconceptions in general I think. All the GUI designers talked about here work with XML definition files (each has its own flavor) and then either generate Python code from it, or you can load the XML file at runtime and the GUI is created under the hood from that (similar to .nib files on OS X). Compiling to .exe is an entirely different matter. If you think Python will help you easily create .exes you might be barking up the wrong tree. Python is an interpreted language (soon to be JIT'ed with PyPy). It's not a compiler. You can edit and run the python files directly for debugging. You can use py2exe to create a final executable if you want, but this is not compiling and it won't keep your code secret. > In the 1980s everyone was happy with inputs from the command line on > a line editor, but today people expect GUIs with graphics and often > even animations. Qt does this very well with less and less programming all the time. QtQuick and QML can define pretty nice transitions and animations that can then be combined with Python programming. But there's more than one way to do things. I use a program called Glade3 for doing GTK+ apps in Python (I only use Linux). In PyQt I use the amazing QtDesigner. > It is surprising that a language which seems very popular does not > have GUI development infrastructure in place these many years after > it got into common use. Well it's just that the infrastructure is very different than you're used to. There are lots of different choices. I think it turns out that a VB6-style IDE just has never been necessarily for the vast majority of Python programmers. And there are very fancy, complicated GUI apps written in Python. For example, the calibre ebook management program is a nice GUI with animations like coverflow, and it's entirely in Python and PyQt. I'm sure the author uses Qt Designer to do GUI layout. From rustompmody at gmail.com Sun Feb 28 20:39:25 2016 From: rustompmody at gmail.com (Rustom Mody) Date: Sun, 28 Feb 2016 17:39:25 -0800 (PST) Subject: Lineendings (was Everything good about Python except GUI IDE?) In-Reply-To: <56d39335$0$1622$c3e8da3$5496439d@news.astraweb.com> References: <64a6599c-fae1-469d-bcee-875165b3cc7d@googlegroups.com> <56d294f8$0$1604$c3e8da3$5496439d@news.astraweb.com> <62084c14-abd1-4214-af08-70ce8449c83e@googlegroups.com> <6dq5db5j0hg2evl7t334ftdm5sk8n5itge@4ax.com> <93t5db9sib9ldgktrt7523fnis4tgq2uev@4ax.com> <9064f06c-cbd5-4bf4-98d7-24bed0e78c10@googlegroups.com> <90f65ae1-c3d8-4a36-bc9f-860403a0633c@googlegroups.com> <56d39335$0$1622$c3e8da3$5496439d@news.astraweb.com> Message-ID: <92074551-a917-43d2-b72f-f184798ab7e8@googlegroups.com> On Monday, February 29, 2016 at 6:09:33 AM UTC+5:30, Steven D'Aprano wrote: > On Mon, 29 Feb 2016 01:26 am, Chris Warrick wrote: > > >> git is a *collaborative* tool and should work when the other party is > >> using notepad. > > > > What should git do if someone saves, say, Ruby code as a .py file? > > Should it rename it? Or should it figure out an equivalent snippet of > > Python? > > Don't be ridiculous. That's completely over the top. > > It isn't asking too much for version control systems to *not care* about > line ending changes. Who cares if the file changes from \n to \r \r\n? It > shouldn't matter, or at least, it shouldn't matter much. Unfortunately that's the outlook all major VCSes (not just git) have started with and its wrong. Speaking somewhat simplistically: On windows one should see CRLF On *nix LF And this SHOULD NOT be a diff! [Assuming the VCS is serious about collaboration] Analogy: I stick my flash drive into linux and get /media/rusi/Transcend On windows (I guess) its H:\Transcend The SAME files and filesystem should be thus different right? .gitattributes does make these (declarations) possible ... in a half-assed afterthought sort of way with safecrlf and autocrlf as earlier poor bug-ridden bugfixes From rosuav at gmail.com Sun Feb 28 20:49:57 2016 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 29 Feb 2016 12:49:57 +1100 Subject: Lineendings (was Everything good about Python except GUI IDE?) In-Reply-To: <92074551-a917-43d2-b72f-f184798ab7e8@googlegroups.com> References: <64a6599c-fae1-469d-bcee-875165b3cc7d@googlegroups.com> <56d294f8$0$1604$c3e8da3$5496439d@news.astraweb.com> <62084c14-abd1-4214-af08-70ce8449c83e@googlegroups.com> <6dq5db5j0hg2evl7t334ftdm5sk8n5itge@4ax.com> <93t5db9sib9ldgktrt7523fnis4tgq2uev@4ax.com> <9064f06c-cbd5-4bf4-98d7-24bed0e78c10@googlegroups.com> <90f65ae1-c3d8-4a36-bc9f-860403a0633c@googlegroups.com> <56d39335$0$1622$c3e8da3$5496439d@news.astraweb.com> <92074551-a917-43d2-b72f-f184798ab7e8@googlegroups.com> Message-ID: On Mon, Feb 29, 2016 at 12:39 PM, Rustom Mody wrote: > Unfortunately that's the outlook all major VCSes (not just git) have started > with and its wrong. > Speaking somewhat simplistically: > On windows one should see CRLF > On *nix LF > And this SHOULD NOT be a diff! > [Assuming the VCS is serious about collaboration] > If you want this, simply run this on your Windows computer: git config --global core.autocrlf true Job done. The repo will record LF, but you'll get CRLF on the Windows box. Alternatively, if you're happy with LF on Windows, but want protection against accidentally checking in a CRLF: git config --global core.autocrlf input Either way, this means that line endings get folded to LF for consistency. ChrisA From rustompmody at gmail.com Sun Feb 28 20:55:08 2016 From: rustompmody at gmail.com (Rustom Mody) Date: Sun, 28 Feb 2016 17:55:08 -0800 (PST) Subject: Lineendings (was Everything good about Python except GUI IDE?) In-Reply-To: References: <64a6599c-fae1-469d-bcee-875165b3cc7d@googlegroups.com> <56d294f8$0$1604$c3e8da3$5496439d@news.astraweb.com> <62084c14-abd1-4214-af08-70ce8449c83e@googlegroups.com> <6dq5db5j0hg2evl7t334ftdm5sk8n5itge@4ax.com> <93t5db9sib9ldgktrt7523fnis4tgq2uev@4ax.com> <9064f06c-cbd5-4bf4-98d7-24bed0e78c10@googlegroups.com> <90f65ae1-c3d8-4a36-bc9f-860403a0633c@googlegroups.com> <56d39335$0$1622$c3e8da3$5496439d@news.astraweb.com> <92074551-a917-43d2-b72f-f184798ab7e8@googlegroups.com> Message-ID: <47e3847b-1fe5-4319-9de9-5d64c996195a@googlegroups.com> On Monday, February 29, 2016 at 7:20:19 AM UTC+5:30, Chris Angelico wrote: > On Mon, Feb 29, 2016 at 12:39 PM, Rustom Mody wrote: > > Unfortunately that's the outlook all major VCSes (not just git) have started > > with and its wrong. > > Speaking somewhat simplistically: > > On windows one should see CRLF > > On *nix LF > > And this SHOULD NOT be a diff! > > [Assuming the VCS is serious about collaboration] > > > > If you want this, simply run this on your Windows computer: > > git config --global core.autocrlf true > > Job done. The repo will record LF, but you'll get CRLF on the Windows > box. Alternatively, if you're happy with LF on Windows, but want > protection against accidentally checking in a CRLF: > > git config --global core.autocrlf input > > Either way, this means that line endings get folded to LF for consistency. > > ChrisA Just check stackoverflow for how often this FAILS to work From rosuav at gmail.com Sun Feb 28 21:02:46 2016 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 29 Feb 2016 13:02:46 +1100 Subject: Lineendings (was Everything good about Python except GUI IDE?) In-Reply-To: <47e3847b-1fe5-4319-9de9-5d64c996195a@googlegroups.com> References: <64a6599c-fae1-469d-bcee-875165b3cc7d@googlegroups.com> <56d294f8$0$1604$c3e8da3$5496439d@news.astraweb.com> <62084c14-abd1-4214-af08-70ce8449c83e@googlegroups.com> <6dq5db5j0hg2evl7t334ftdm5sk8n5itge@4ax.com> <93t5db9sib9ldgktrt7523fnis4tgq2uev@4ax.com> <9064f06c-cbd5-4bf4-98d7-24bed0e78c10@googlegroups.com> <90f65ae1-c3d8-4a36-bc9f-860403a0633c@googlegroups.com> <56d39335$0$1622$c3e8da3$5496439d@news.astraweb.com> <92074551-a917-43d2-b72f-f184798ab7e8@googlegroups.com> <47e3847b-1fe5-4319-9de9-5d64c996195a@googlegroups.com> Message-ID: On Mon, Feb 29, 2016 at 12:55 PM, Rustom Mody wrote: > On Monday, February 29, 2016 at 7:20:19 AM UTC+5:30, Chris Angelico wrote: >> On Mon, Feb 29, 2016 at 12:39 PM, Rustom Mody wrote: >> > Unfortunately that's the outlook all major VCSes (not just git) have started >> > with and its wrong. >> > Speaking somewhat simplistically: >> > On windows one should see CRLF >> > On *nix LF >> > And this SHOULD NOT be a diff! >> > [Assuming the VCS is serious about collaboration] >> > >> >> If you want this, simply run this on your Windows computer: >> >> git config --global core.autocrlf true >> >> Job done. The repo will record LF, but you'll get CRLF on the Windows >> box. Alternatively, if you're happy with LF on Windows, but want >> protection against accidentally checking in a CRLF: >> >> git config --global core.autocrlf input >> >> Either way, this means that line endings get folded to LF for consistency. >> >> ChrisA > > Just check stackoverflow for how often this FAILS to work Never has for any of my projects. Examples please? Actual real problems? I've been using git for years, on mixed platforms for a lot of that, and not had a single problem. ChrisA From rustompmody at gmail.com Sun Feb 28 21:08:30 2016 From: rustompmody at gmail.com (Rustom Mody) Date: Sun, 28 Feb 2016 18:08:30 -0800 (PST) Subject: Lineendings (was Everything good about Python except GUI IDE?) In-Reply-To: References: <64a6599c-fae1-469d-bcee-875165b3cc7d@googlegroups.com> <56d294f8$0$1604$c3e8da3$5496439d@news.astraweb.com> <62084c14-abd1-4214-af08-70ce8449c83e@googlegroups.com> <6dq5db5j0hg2evl7t334ftdm5sk8n5itge@4ax.com> <93t5db9sib9ldgktrt7523fnis4tgq2uev@4ax.com> <9064f06c-cbd5-4bf4-98d7-24bed0e78c10@googlegroups.com> <90f65ae1-c3d8-4a36-bc9f-860403a0633c@googlegroups.com> <56d39335$0$1622$c3e8da3$5496439d@news.astraweb.com> <92074551-a917-43d2-b72f-f184798ab7e8@googlegroups.com> <47e3847b-1fe5-4319-9de9-5d64c996195a@googlegroups.com> Message-ID: On Monday, February 29, 2016 at 7:33:18 AM UTC+5:30, Chris Angelico wrote: > On Mon, Feb 29, 2016 at 12:55 PM, Rustom Mody wrote: > > On Monday, February 29, 2016 at 7:20:19 AM UTC+5:30, Chris Angelico wrote: > >> On Mon, Feb 29, 2016 at 12:39 PM, Rustom Mody wrote: > >> > Unfortunately that's the outlook all major VCSes (not just git) have started > >> > with and its wrong. > >> > Speaking somewhat simplistically: > >> > On windows one should see CRLF > >> > On *nix LF > >> > And this SHOULD NOT be a diff! > >> > [Assuming the VCS is serious about collaboration] > >> > > >> > >> If you want this, simply run this on your Windows computer: > >> > >> git config --global core.autocrlf true > >> > >> Job done. The repo will record LF, but you'll get CRLF on the Windows > >> box. Alternatively, if you're happy with LF on Windows, but want > >> protection against accidentally checking in a CRLF: > >> > >> git config --global core.autocrlf input > >> > >> Either way, this means that line endings get folded to LF for consistency. > >> > >> ChrisA > > > > Just check stackoverflow for how often this FAILS to work > > Never has for any of my projects. Examples please? Actual real > problems? I've been using git for years, on mixed platforms for a lot > of that, and not had a single problem. Pragmatically: As I said just search stackoverflow for git:crlf Theoretically: Its a borked (leaky) abstraction From steve at pearwood.info Sun Feb 28 21:22:28 2016 From: steve at pearwood.info (Steven D'Aprano) Date: Mon, 29 Feb 2016 13:22:28 +1100 Subject: Everything good about Python except GUI IDE? References: <64a6599c-fae1-469d-bcee-875165b3cc7d@googlegroups.com> <56d294f8$0$1604$c3e8da3$5496439d@news.astraweb.com> <56d2ed33$0$1588$c3e8da3$5496439d@news.astraweb.com> Message-ID: <56d3ab66$0$1601$c3e8da3$5496439d@news.astraweb.com> On Mon, 29 Feb 2016 04:53 am, Chris Angelico wrote: > For example, I have a Dungeons & Dragons character sheet display, in > which there are large numbers of entry fields (editable), labels > (non-editable display, usually calculated from other fields), and less > commonly, drop-down lists and multi-line text fields. Call that four > primitives. Now, some of the fields need to be highlighted for the > human's attention [...] > With a GUI builder, how do > you redefine a function that isn't just simple composition? I'm not entirely sure what your objection here is. You're still coding. Your widgets will have callbacks, or have handlers (to use Hypercard terminology). Just because you're placing the widget by point-and-click or drag-and-drop doesn't mean that you don't have to write any code at all. If the highlighting is done at runtime, there's probably a handler or callback that handles it. So you edit that code, wherever it happens to be. > This additional meta-level is one that is absent from a *lot* of > modern graphical environments. That's exactly why I miss Hypercard so much. The builder and the runtime are the same thing. (In practice, you could use a simple permissions-based model to disable access to the builder at runtime, if desired.) I really cannot stress enough how similar it is to Python's REPL. In Python, you can choose at an extremely fine-grained level: - open a file in an editor, write a function or class, save, import that module into the REPL; - or write the function directly in the REPL; - what's that, I need to operate on that function as data? I can write a decorator-type function in the REPL and apply it to the function as needed, on the fly. I can *build code* and *run code* in the same environment. I don't *have* to use the REPL. I can write my code in a text editor, of course, and for many purposes that is idea. But having the interpreter available at the same time is a HUGE benefit. Languages without a REPL are comparatively crippled by comparison, as far as rapid development, exploration, discovery and testing are concerned. Think of the GUI builder as the REPL for building GUIs. You shouldn't have to do all your work in the REPL, but neither should you be limited without one. If the GUI builder doesn't let you write and run code on the fly, it's a crap builder. Hypercard had one powerful feature that the default Python REPL lacks. When you quit Python's interactive interpreter, all the functions, classes and variables you have created disappear. When you quit Hypercard, all the widgets and data and code you have created is saved in the current document (the "stack") so that they are available next time you open that document. I think that iPython workbooks (notebooks?) are the closest equivalent in the Python world. Certainly they are a powerful and popular feature in the sciences (iPython copied them from Mathematica). > Look at spreadsheets - the classic > Lotus 1-2-3 model has stayed with us all through, with MS Excel, OO/LO > Calc, etc, etc generally following it. And that's fine; spreadsheets > do a lot of very good work for a lot of people. Now, suppose in cell > C2 you have this formula: "=A2+B2*.05". You can copy that down the > rest of the column, and C9 will say "=A9+B9*.05", and so on; that's > what spreadsheets do well. But once you've done that copy operation, > those cells lose all track of the fact that they got their formula > from C2. If you edit C2, you have to re-copy-down. That's not too hard > for a lot of people to remember, but what happens when that's > forgotten? What you are describing is effectively a form of refactoring. What happens when you decide to use a set instead of a tuple for some data structure? You have to go through your entire code base looking for all references to those tuples (it might not be just one!) and change them to a set, and change the code that operates on them. Don't say "you should have hidden the fact that this is a tuple behind some interface layer", because this is the implementation layer. (There's always an implementation layer.) Some editors have refactoring tools for this sort of thing, but most don't, and refactoring tools are never perfect nor fully automated. >> (2) "source control" -- the world is full of document types that aren't >> plain text source code, and people have found ways to manage >> collaborative editing and change management for them. Why don't we ask >> game developers how they manage changes to the non-code elements in their >> applications? Textures, icons, player avatars, sound effects, maps, etc. >> Surely you don't suggest that game developers edit their background >> images in a hex editor? > > Yes, and while you're at it, ask the critical question: What happens > when two people make edits to the same object? You mean, something like I change the File menu to say "New; Save; Quit" and you change it to say "Create; Save; Exit"? Then there's a conflict, same as in text code. How would you merge those two changes as text? [...] > My suspicion is that such game devs would have strict rules about > simultaneous editing of files. While that quite probably works fairly > well, it's a limitation that most of us would not accept in any code > project. I think you mean, "in any code project apart from games". > Imagine if the Python bug tracker required you to send, not a > patch file, but the entire source file that has the edit you want. How > would you manage those sorts of edits? Patch files are for the convenience of the code reviewer, not the person making the change. It's *less* work for me to email or upload the entire .py file than to create a patch and email that, but it's harder for the guy who wants to see what changes I've made before applying them. -- Steven From gansteed at gmail.com Sun Feb 28 21:26:56 2016 From: gansteed at gmail.com (gansteed) Date: Mon, 29 Feb 2016 10:26:56 +0800 Subject: why not define a EVENT_ERROR in selectors module? Message-ID: as the title: I'm reading the source code of selectors, I got this: # generic events, that must be mapped to implementation-specific ones EVENT_READ = (1 << 0) EVENT_WRITE = (1 << 1) maybe we can add definitions for EVENT_ERROR like this: # Choose the best implementation, roughly: # epoll|kqueue|devpoll > poll > select. # select() also can't accept a FD > FD_SETSIZE (usually around 1024) if 'KqueueSelector' in globals(): DefaultSelector = KqueueSelector EVENT_ERROR = EVENT_WRITE << 1 elif 'EpollSelector' in globals(): ... elif ...: ... From ben+python at benfinney.id.au Sun Feb 28 21:35:06 2016 From: ben+python at benfinney.id.au (Ben Finney) Date: Mon, 29 Feb 2016 13:35:06 +1100 Subject: Lineendings (was Everything good about Python except GUI IDE?) References: <64a6599c-fae1-469d-bcee-875165b3cc7d@googlegroups.com> <56d294f8$0$1604$c3e8da3$5496439d@news.astraweb.com> <62084c14-abd1-4214-af08-70ce8449c83e@googlegroups.com> <6dq5db5j0hg2evl7t334ftdm5sk8n5itge@4ax.com> <93t5db9sib9ldgktrt7523fnis4tgq2uev@4ax.com> <9064f06c-cbd5-4bf4-98d7-24bed0e78c10@googlegroups.com> <90f65ae1-c3d8-4a36-bc9f-860403a0633c@googlegroups.com> <56d39335$0$1622$c3e8da3$5496439d@news.astraweb.com> <92074551-a917-43d2-b72f-f184798ab7e8@googlegroups.com> <47e3847b-1fe5-4319-9de9-5d64c996195a@googlegroups.com> Message-ID: <85d1rgxmzp.fsf@benfinney.id.au> Rustom Mody writes: > On Monday, February 29, 2016 at 7:33:18 AM UTC+5:30, Chris Angelico wrote: > > Never has for any of my projects. Examples please? Actual real > > problems? I've been using git for years, on mixed platforms for a lot > > of that, and not had a single problem. > > Pragmatically: As I said just search stackoverflow for git:crlf Don't ask Chris to *guess* which search results are representative of what you're asserting. Please provide concrete examples that demonstrate specifically what you're describing. -- \ ?I'd take the awe of understanding over the awe of ignorance | `\ any day.? ?Douglas Adams | _o__) | Ben Finney From jason.swails at gmail.com Sun Feb 28 22:58:44 2016 From: jason.swails at gmail.com (Jason Swails) Date: Sun, 28 Feb 2016 22:58:44 -0500 Subject: Computational Chemistry Analysis In-Reply-To: References: Message-ID: On Wed, Feb 24, 2016 at 8:01 PM, Feagans, Mandy wrote: > Dear Python, > > > Hi! I am a student interested in conducting computational analysis of > protein-ligand binding for drug development analysis. Recently, I read of > an individual using a python program for their studies of protein-ligand > binding. As I have been reading about Python programs, however, all I have > been able to find are programs that use the python script (i.e. Autodock). > I was hoping to see if there were any programs specifically run through the > python programming that ran similar analysis to Autodock? > ?Being trained as a computational chemist myself (and having learned Python by writing a program for carrying out protein-ligand free energy calculations: http://pubs.acs.org/doi/abs/10.1021/ct300418h), I had to answer... There are a number of programs out there that are aimed at computational modeling of biomolecules, and many of them are written in Python or have Python interfaces (pymol, chimera, MMPBSA.py, OpenMM to name a very small few). However, this is definitely not the right forum to ask such questions as a very small number of people who frequent this list are computational chemists (I check it only occasionally). Looking for a program to do something because it's written in Python is the wrong way to go about this. You need to design your experiment (i.e., what you want to test and what you hope to learn), then try to design a set of calculations and analyses that will allow you to probe your underlying hypothesis. Then you should pick the software to perform these analyses based on the best choice. That choice is very frequently whatever others in your lab are using. A research group builds up experience (based originally on the experience of the PI, typically) in a set of programs they use for their computational experiments, and deviating from that set of programs essentially discards potentially decades worth of experience. So my suggestion -- ask other group members or the professor what softwares they use, and consult Google if you wish to branch out a little. You can also find more applicable mailing lists to ask questions about computational chemistry, like the CCL or molecular dynamics news (the CCL being the defacto "Computational Chemistry List"). That should help give perhaps more helpful places to start. FWIW, I did all my work with the AMBER and OpenMM software suites, (and wrote a substantial amount of code for both projects). But those are far from the only options out there. HTH, Jason -- Jason M. Swails BioMaPS, Rutgers University Postdoctoral Researcher From greg.ewing at canterbury.ac.nz Sun Feb 28 23:40:33 2016 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Mon, 29 Feb 2016 17:40:33 +1300 Subject: Everything good about Python except GUI IDE? In-Reply-To: <56d3ab66$0$1601$c3e8da3$5496439d@news.astraweb.com> References: <64a6599c-fae1-469d-bcee-875165b3cc7d@googlegroups.com> <56d294f8$0$1604$c3e8da3$5496439d@news.astraweb.com> <56d2ed33$0$1588$c3e8da3$5496439d@news.astraweb.com> <56d3ab66$0$1601$c3e8da3$5496439d@news.astraweb.com> Message-ID: Steven D'Aprano wrote: > That's exactly why I miss Hypercard so much. The builder and the runtime are > the same thing. Maybe someone would like to resurrect this project: http://pythoncard.sourceforge.net/ -- Greg From gordon at address.invalid Sun Feb 28 23:47:21 2016 From: gordon at address.invalid (Gordon Levi) Date: Mon, 29 Feb 2016 15:47:21 +1100 Subject: Everything good about Python except GUI IDE? References: <64a6599c-fae1-469d-bcee-875165b3cc7d@googlegroups.com> <56d294f8$0$1604$c3e8da3$5496439d@news.astraweb.com> <234a398e-1b0f-467b-a8cb-d7ca748f8062@googlegroups.com> <84922f24-3e00-4a23-b26d-5e6c0d8e7e04@googlegroups.com> <87y4a5c58i.fsf@elektro.pacujo.net> <87twksdg9c.fsf@elektro.pacujo.net> Message-ID: Marko Rauhamaa wrote: >Rustom Mody : > >> On Sunday, February 28, 2016 at 7:22:08 PM UTC+5:30, Marko Rauhamaa wrote: >>> Rustom Mody : >>> >>> > whereas in fact every significant GUI embeds text (possibly >>> > recursively) >>> > >>> > eg TI inside GUI -- think of text inside gimp GUI inside TI -- >>> > think of Word embedding other doc types including pictures Which >>> > can be recursive -- WOrd embeds a picture embeds text >>> >>> Sigh, still nobody has mentioned an exemplary GUI application. >>> >>> An anecdote: Some weeks back my son and I were struggling to get the >>> right kind of graph out of Excel. After four hours of Google, >>> Youtube, trial and error, we gave up, took out a pad of millimeter >>> paper and some colored pencils. The whole job took my son an hour and >>> the end result looked great. He snapped a picture and sent it to the >>> teacher by email. >> >> Point being?? > >I'm awaiting for an opposite anecdote, a GUI app you absolutely love and >recommend as a model for GUI developers. > >We are talking about GUI tools to produce what? Crap? Nobody likes filling in forms but how do you suggest converting a form based app into something loveable. What interface would make you love adding a new contact to your address book? From rustompmody at gmail.com Sun Feb 28 23:48:55 2016 From: rustompmody at gmail.com (Rustom Mody) Date: Sun, 28 Feb 2016 20:48:55 -0800 (PST) Subject: Lineendings (was Everything good about Python except GUI IDE?) In-Reply-To: References: <64a6599c-fae1-469d-bcee-875165b3cc7d@googlegroups.com> <56d294f8$0$1604$c3e8da3$5496439d@news.astraweb.com> <62084c14-abd1-4214-af08-70ce8449c83e@googlegroups.com> <6dq5db5j0hg2evl7t334ftdm5sk8n5itge@4ax.com> <93t5db9sib9ldgktrt7523fnis4tgq2uev@4ax.com> <9064f06c-cbd5-4bf4-98d7-24bed0e78c10@googlegroups.com> <90f65ae1-c3d8-4a36-bc9f-860403a0633c@googlegroups.com> <56d39335$0$1622$c3e8da3$5496439d@news.astraweb.com> <92074551-a917-43d2-b72f-f184798ab7e8@googlegroups.com> <47e3847b-1fe5-4319-9de9-5d64c996195a@googlegroups.com> Message-ID: <6260cd54-2c50-4a55-925f-cb6fa2c0bbab@googlegroups.com> On Monday, February 29, 2016 at 8:05:33 AM UTC+5:30, Ben Finney wrote: > Rustom Mody writes: > > > On Monday, February 29, 2016 at 7:33:18 AM UTC+5:30, Chris Angelico wrote: > > > Never has for any of my projects. Examples please? Actual real > > > problems? I've been using git for years, on mixed platforms for a lot > > > of that, and not had a single problem. > > > > Pragmatically: As I said just search stackoverflow for git:crlf > > Don't ask Chris to *guess* which search results are representative of > what you're asserting. Please provide concrete examples that demonstrate > specifically what you're describing. I was answering Chris' specific "Never has for any of my projects" by saying there are dozens of questions about this (FAQ) on stackoverflow 1st one it gives: http://stackoverflow.com/questions/170961/whats-the-best-crlf-carriage-return-line-feed-handling-strategy-with-git Also some discussions by git devs: https://groups.google.com/forum/#!topic/msysgit/Gb5tlbfEyPk There are more acrimonious dev-exchanges that can be hunted down Nevertheless my basic point is somehow a bit different: A text file (content) is a list of lines A line is a list of char A char is... wont open that can of worms! If git says it stores EOL as LF and can convert to LF or CRLF on demand there itself is the leaked abstraction. Knowing that git uses LF internally should be as relevant as say machine endianness is to how python stores the tuple (1,2) From alan at csail.mit.edu Mon Feb 29 00:47:54 2016 From: alan at csail.mit.edu (Alan Bawden) Date: Mon, 29 Feb 2016 00:47:54 -0500 Subject: Make a unique filesystem path, without creating the file References: Message-ID: Cameron Simpson writes: > On 22Feb2016 12:34, Alan Bawden wrote: I have deleted the part of discussion where it seems that we must simply agree to disagree. You think mktemp() is _way_ more dangerous that I do. >>> In fact your use case isn't safe, because _another_ task using mktemp >>> in conflict as a plain old temporary file may grab your fifo. >> >>But here in very last sentence I really must disagree. If the code I >>wrote above is "unsafe" because some _other_ process might be using >>mktemp() badly and stumble over the same path, then the current >>implementation of tempfile.mkdtemp() is also "unsafe" for exactly the >>same reason: some other process using mktemp() badly to create its own >>directory might accidentally grab the same directory. > > When the other taks goes mkdir with the generated name it will fail, so no. Quite right. I sabotaged my own argument by picking mkdtemp() instead of mkstemp(). I was trying to shorten my text by taking advantage of the fact that I had _already_ mentioned that mkdtemp() performs exactly the same generate-and-open loop than the code I had written. I apologize for the confusion. In fact, mkstemp() also performs that same generate-and-open loop, and of course it is careful to use os.O_EXCL along with os.O_CREAT when it opens the file. So let me re-state my argument using mkstemp() instead: If the code I wrote in my original message is "unsafe" because some _other_ process might be using mktemp() badly and stumble over the same path, then the current implementation of tempfile.mkstemp() is also "unsafe" for exactly the same reason: some other process badly using mktemp() to create its own file might accidentally grab the same file. In other words, if that other process does: path = mktemp() tmpfp = open(path, "w") Then yes indeed, it might accidentally grab my fifo when I used my original code for making a temporary fifo. But it might _also_ succeed in grabbing any temporary files I make using tempfile.mkstemp()! So if you think what I wrote is "unsafe", it seems that you must conclude that the standard tempfile.mkstemp() is exactly as "unsafe". So is that what you think? -- Alan Bawden From marko at pacujo.net Mon Feb 29 01:18:11 2016 From: marko at pacujo.net (Marko Rauhamaa) Date: Mon, 29 Feb 2016 08:18:11 +0200 Subject: Everything good about Python except GUI IDE? References: <64a6599c-fae1-469d-bcee-875165b3cc7d@googlegroups.com> <56d294f8$0$1604$c3e8da3$5496439d@news.astraweb.com> <234a398e-1b0f-467b-a8cb-d7ca748f8062@googlegroups.com> <84922f24-3e00-4a23-b26d-5e6c0d8e7e04@googlegroups.com> <87y4a5c58i.fsf@elektro.pacujo.net> <87twksdg9c.fsf@elektro.pacujo.net> Message-ID: <87d1rgca58.fsf@elektro.pacujo.net> Gordon Levi : > Nobody likes filling in forms but how do you suggest converting a form > based app into something loveable. Straight HTML does forms just fine without CSS or JavaScript, yet few can resist. > What interface would make you love adding a new contact to your > address book? In my case, the address book is a ~/.mailrc file, which I edit using emacs. Marko From cs at zip.com.au Mon Feb 29 01:38:35 2016 From: cs at zip.com.au (Cameron Simpson) Date: Mon, 29 Feb 2016 17:38:35 +1100 Subject: Make a unique filesystem path, without creating the file In-Reply-To: References: Message-ID: <20160229063835.GA51600@cskk.homeip.net> On 29Feb2016 00:47, Alan Bawden wrote: >Cameron Simpson writes: >> On 22Feb2016 12:34, Alan Bawden wrote: > >I have deleted the part of discussion where it seems that we must simply >agree to disagree. You think mktemp() is _way_ more dangerous that I >do. I certainly think the habit of using it is. And thus we're off into the realm of risk assessment I suppose, where one's value sets greatly affect the outcome. But there are concrete arguments to be made about risks. To your other question... [...] >In fact, mkstemp() also performs that same generate-and-open loop, and of >course it is careful to use os.O_EXCL along with os.O_CREAT when it >opens the file. So let me re-state my argument using mkstemp() instead: > >If the code I wrote in my original message is "unsafe" because some >_other_ process might be using mktemp() badly and stumble over the same >path, then the current implementation of tempfile.mkstemp() is also >"unsafe" for exactly the same reason: some other process badly using >mktemp() to create its own file might accidentally grab the same file. > >In other words, if that other process does: > > path = mktemp() > tmpfp = open(path, "w") > >Then yes indeed, it might accidentally grab my fifo when I used my >original code for making a temporary fifo. But it might _also_ succeed >in grabbing any temporary files I make using tempfile.mkstemp()! So if >you think what I wrote is "unsafe", it seems that you must conclude that >the standard tempfile.mkstemp() is exactly as "unsafe". > >So is that what you think? Yes and no? You're quite right that a task using mkstemp is not safe against a task misusing mktemp. _However_: In a space where everyone uses mktemp, everyone is unsafe from collision. In a space where everyone uses mkstemp, everyone is safe from collision. So provided everyone "upgrades", safety is reliable without any added burden in program complexity. Of course, that sidesteps the scenario where someone is using mktemp to obtain a pathname for a non-file, but I am of the opinion that in almost all such cases the programmer is better off using mkdtemp and making their non-file inside the temporary directory. Again, provided everyone "upgrades" to such a practice, safety is arranged. Because of this, I think that _any_ use of mktemp invites risk of collision, and needs to be justified with a robust argument establishing that the problem cannot be solved with mkstemp or mkdtemp. Your example was not such a case. Ben's is, in that (a) he needs a "valid" name and (b) he isn't going to make an actual filesystem object using the name obtained. As it happens it looks like the uuid generation functions from the stdlib may meet his needs, addressing his desire to do it simply with the stdlib instead of making his own wheel. So I remain against mktemp without an outsandingly special use case. Cheers, Cameron Simpson From alien2utoo at gmail.com Mon Feb 29 01:40:04 2016 From: alien2utoo at gmail.com (alien2utoo at gmail.com) Date: Sun, 28 Feb 2016 22:40:04 -0800 (PST) Subject: Reason for not allowing import twice but allowing reload() Message-ID: <645cdd46-d4a4-49b3-a0d8-848608d70d73@googlegroups.com> Hello list, We can not import a module twice in a session of Python (subsequent attempts to import same module don't result in any error though, but it is not-effective). However after making change to module, we can reload() it (if not reload(), we could possibly have reimport statement) to get the updates in module. I am a newbie in Python (learning via IDLE) and trying to understand - what is extra (special) that import does, that reload() doesn't? - if both do the same internally and externally, why subsequent imports of same module are ineffective? - if subsequent imports of same module in a session are not effective, why not simply flag those attempts as an error, rather than letting them go effect-less. Kindly help me understand the reasons behind. Regards Sandeep From rosuav at gmail.com Mon Feb 29 02:01:30 2016 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 29 Feb 2016 18:01:30 +1100 Subject: Reason for not allowing import twice but allowing reload() In-Reply-To: <645cdd46-d4a4-49b3-a0d8-848608d70d73@googlegroups.com> References: <645cdd46-d4a4-49b3-a0d8-848608d70d73@googlegroups.com> Message-ID: On Mon, Feb 29, 2016 at 5:40 PM, wrote: > We can not import a module twice in a session of Python (subsequent attempts to import same module don't result in any error though, but it is not-effective). > > However after making change to module, we can reload() it (if not reload(), we could possibly have reimport statement) to get the updates in module. > > I am a newbie in Python (learning via IDLE) and trying to understand > > - what is extra (special) that import does, that reload() doesn't? > - if both do the same internally and externally, why subsequent imports of same module are ineffective? > - if subsequent imports of same module in a session are not effective, why not simply flag those attempts as an error, rather than letting them go effect-less. > > Kindly help me understand the reasons behind. Normally, you import a module because you want its features. For instance, you can "import re" to get access to the Regular Expression parser, or "import pandas" to load up the rather hefty third-party numerical computation/analysis library of that name. Sometimes, this can take a lot of work - importing pandas loads several hundred modules in total, and can take a second or two, if it's not already in the OS's disk cache. (Even if it is in cache, it takes a visible pause - maybe 100-300 ms.) If two parts of your project need the same module, it's way more efficient to make use of the already-loaded module. Additionally, sometimes it makes a lot of difference, and you _want_ to use the same module. When you "import sys", you absolutely must access the same sys module that everyone else does; otherwise, changing sys.stdout or sys.path wouldn't have any effect. And depending on how it's implemented, sys.intern could create very subtle bugs if you had two versions of it around. So here's what happens (massively oversimplified) when you "import modulename": 1) Is modulename in sys.modules? If so, return that. 2) Step through sys.path, looking for the module. Can't find it? ImportError. 3) Run that program and create a module object. Put it into sys.modules, and return it. You can test that first step fairly easily: >>> sys.modules["asdf"]="Haha" >>> import asdf >>> asdf 'Haha' As a general rule, this is what you want (apart from sticking strings into sys.modules, which is a VERY strange thing to do!). In fact, I would recommend never using reload() unless you're absolutely sure of what it does and how it's important. I've never used reload() in any production code, ever. Ever. The reason that repeating the import isn't flagged as an error is that it's simply an assignment. If you write this code, Python won't warn you: >>> os = sys.modules["os"] >>> os = sys.modules["os"] >>> os = sys.modules["os"] >>> os = sys.modules["os"] Yet this is effectively what "import os" does. It's assignment, just like any other assignment operation. I believe it is theoretically possible to ask Python to give you a warning (using an import hook), but generally, you should think about imports simply as assignment. ChrisA From ian.g.kelly at gmail.com Mon Feb 29 02:02:13 2016 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Mon, 29 Feb 2016 00:02:13 -0700 Subject: Reason for not allowing import twice but allowing reload() In-Reply-To: <645cdd46-d4a4-49b3-a0d8-848608d70d73@googlegroups.com> References: <645cdd46-d4a4-49b3-a0d8-848608d70d73@googlegroups.com> Message-ID: On Sun, Feb 28, 2016 at 11:40 PM, wrote: > Hello list, > > We can not import a module twice in a session of Python (subsequent attempts to import same module don't result in any error though, but it is not-effective). > > However after making change to module, we can reload() it (if not reload(), we could possibly have reimport statement) to get the updates in module. > > I am a newbie in Python (learning via IDLE) and trying to understand > > - what is extra (special) that import does, that reload() doesn't? When you import a module, it's cached in the sys.modules dict. Subsequent imports of the same module find the cached module rather than reimport it. However, reload() ignores the cache and always reloads the module. > - if subsequent imports of same module in a session are not effective, why not simply flag those attempts as an error, rather than letting them go effect-less. Because there are legitimate reasons for importing the same module multiple times. For example, I have a script that imports the sys module. It also imports the os module, which imports the sys module. Both the main module of my script and the os module have need of the sys module, so they both import it. If subsequent imports of the module raised an exception, this wouldn't be possible. However, they shouldn't get two separate copies of the sys module, because that would be wasteful and potentially error-prone if the module is stateful In the case of the sys module, there is state -- the sys.modules cache that I mentioned earlier. Imagine the confusion if two separate imports of the sys module resulted in two separate instances with two separate caches of imported modules that were in disagreement. From rosuav at gmail.com Mon Feb 29 02:11:33 2016 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 29 Feb 2016 18:11:33 +1100 Subject: Reason for not allowing import twice but allowing reload() In-Reply-To: References: <645cdd46-d4a4-49b3-a0d8-848608d70d73@googlegroups.com> Message-ID: On Mon, Feb 29, 2016 at 6:02 PM, Ian Kelly wrote: >> - if subsequent imports of same module in a session are not effective, why not simply flag those attempts as an error, rather than letting them go effect-less. > > Because there are legitimate reasons for importing the same module > multiple times. For example, I have a script that imports the sys > module. It also imports the os module, which imports the sys module. > Both the main module of my script and the os module have need of the > sys module, so they both import it. If subsequent imports of the > module raised an exception, this wouldn't be possible. I think the OP's talking more about the situation of having an active session (IDLE was mentioned), importing a local module (a .py file from the current directory), then editing the file and re-importing, which has no effect. While I am sympathetic to the problem, I don't believe the language should be changed here; what might be useful is something that notices that an already-loaded module is now out of date, but really, the best solution is a change of workflow that no longer has long-running modules loading live-updated code. I would, instead, run a short-lived module and simply restart it after any edit (possibly dropping to interactive mode at the end). Save, restart. Save, restart. Python isn't optimized for the "keep running but change code" execution model (for that, use Pike, and pay the complexity overhead). ChrisA From rustompmody at gmail.com Mon Feb 29 02:20:13 2016 From: rustompmody at gmail.com (Rustom Mody) Date: Sun, 28 Feb 2016 23:20:13 -0800 (PST) Subject: Everything good about Python except GUI IDE? In-Reply-To: <87d1rgca58.fsf@elektro.pacujo.net> References: <64a6599c-fae1-469d-bcee-875165b3cc7d@googlegroups.com> <56d294f8$0$1604$c3e8da3$5496439d@news.astraweb.com> <234a398e-1b0f-467b-a8cb-d7ca748f8062@googlegroups.com> <84922f24-3e00-4a23-b26d-5e6c0d8e7e04@googlegroups.com> <87y4a5c58i.fsf@elektro.pacujo.net> <87twksdg9c.fsf@elektro.pacujo.net> <87d1rgca58.fsf@elektro.pacujo.net> Message-ID: On Monday, February 29, 2016 at 11:48:25 AM UTC+5:30, Marko Rauhamaa wrote: > Gordon Levi : > > > Nobody likes filling in forms but how do you suggest converting a form > > based app into something loveable. > > Straight HTML does forms just fine without CSS or JavaScript, yet few > can resist. Abjure JS/CSS is a virtue? Why? In any case with or without its still a form not plain (printf/scanf) text. So what exactly are we talking of? > > > What interface would make you love adding a new contact to your > > address book? > > In my case, the address book is a ~/.mailrc file, which I edit using > emacs. Even in the emacs world people find this painful enough that they use tools like bbdb and more modern ones like org-contacts From anthony at cajuntechie.org Mon Feb 29 03:10:00 2016 From: anthony at cajuntechie.org (Anthony Papillion) Date: Mon, 29 Feb 2016 02:10:00 -0600 Subject: Sending an email with a binary attachment Message-ID: <56D3FCD8.8070509@cajuntechie.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 Hello Everyone, I've been trying to write a /very/ simple script that will email a binary file to an email address when run. Pretty simple. I've done quite a bit of research and have finally settled on the following code: http://pastebin.com/sryj98wW For some reason though, sending mail is failing every time. I've made sure that the password is correct (which seems to be the most usual error). Still, I just can't get it to work. Can someone take a look at this code and give me some advice? Thanks! Anthony -----BEGIN PGP SIGNATURE----- iQIcBAEBCgAGBQJW0/zYAAoJEAKK33RTsEsVH5YP/jisLkuK4SAD6E06dO/czn1V bO1QlzqTje8giXUWGyVeGS9MHAA1S2TlHc/rp01/DHTt1iYT50qGIaPxzUxBTMEX mBCFFUSgg6LjkY+9b0c+o4OmviZlX7/KA6wT9IpFEG0l86Uj+z39rcBHm+GauiX+ IgMS5lWXfPmIEwiKzByMduxEgfj6kQW6PhkR5hTmOvd3sbILR91U9NiigrUD1ukM +eN4LnKVu5Qpyb8X34Ze0NF7tiMr7tTQyBIq6C6ORGR/IPri407Ro8ACHBd55xZV QxHjFVemms9s7ala0aoBcpRKr4PG35Ebm+7CcZaDj0fdk11mRvpqCZtGfodw/+sL Crrxoz11hC//DHslZrEuNlJHgRqnsw+abKG8jIaJCrIW1f9BiyiWse+RRwbpN8IY 1C0gD4E1N3dUqGFrhyD/Xuy0tkXWpfiYhcLJQY+j7exrf00s+ASUTrd92lDCzxW4 pwuPZYN6nXY6dh9q72fgoEzqU0eDIzSgX99fODcxqp1L2ieczWMMajlWQJrJEcUe UgxB8IvlvuD3NpzZXisEG8ZzqhZT6GOurHGbBusAW2M9zaqImb4qGEid7HS4BL/c Kj7PspXz+8gD3SKZ/dOBdFlKK1KM1utekB0UnmBboZ9nXVhz5qVPaiVgjPunB5TG 1fMuuGiy2yl5pfTNvH8a =Ff+J -----END PGP SIGNATURE----- From ganesh1pal at gmail.com Mon Feb 29 03:13:29 2016 From: ganesh1pal at gmail.com (Ganesh Pal) Date: Mon, 29 Feb 2016 13:43:29 +0530 Subject: usage of try except for review. Message-ID: Iam on python 2.6 and Linux , need your suggestion on the usage of try and except in this program and Modified code: #!/usr/bin/env python """ """ import os import shlex import subprocess import sys import time import logging import run import pdb def run_cmd_and_verify(cmd, timeout=1000): try: out, err, ret = run(cmd, timeout=timeout) assert ret ==0,"ERROR (ret %d): " \ " \nout: %s\nerr: %s\n" % (ret, out, err) except Exception as e: logging.error("Failed to run %s got %s" % (cmd, e)) return False return True def run_test(): """ Mount """ pdb.set_trace() for cmd in ["mount /nfs_mount1", "mount /cifs_mount1"]: try: if not run_cmd_and_verify(cmd, timeout=3600): return False except: logging.error("Failure while running command %") logging.info("Setup and Creation ....Done !!!") # cmd = "run_scan" out, err, ret = run(cmd) for cmd in ["create_data.py -nfs ", "validate.py -30 "]: try: if not run_cmd_and_verify(cmd, timeout=3600): return False except: logging.error("") return False logging.info("Mount IS START.....Done !!!") def main(): if not run_test(): sys.exit("Exiting Main") if __name__ == '__main__': main() Question 1: 1. Have I used try and expect block correctly ? , In my case I have the except block that's is not needed it just gives an message I have still included for the sake of try block try: if not run_cmd_and_verify(cmd, timeout=3600): return False except: logging.error("inside except") return False 2. If a failure?s are encountered the error by assert condition the errors are now displayed on the screen , how do I redirect it to log file using logging error def run_cmd_and_verify(cmd, timeout=1000): try: out, err, ret = run(cmd, timeout=timeout) assert ret ==0,"ERROR (ret %d): " \ " \nout: %s\nerr: %s\n" % (ret, out, err) except Exception as e: logging.error("Failed to run %s got %s" % (cmd, e)) return False return True #script_10.py Failed to run mount /nfs got ERROR (ret 1): out: host-44-3 exited with status 1 err: host-44-3: mount_efs: on /nfs: efs is already mounted 3. my function def has 1000 but Iam using 3600 in the calling fnx etc , Time out value are overwritten ? 4. Any further improvement particularly on try -except ? Regards, Ganesh From rosuav at gmail.com Mon Feb 29 03:20:44 2016 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 29 Feb 2016 19:20:44 +1100 Subject: Everything good about Python except GUI IDE? In-Reply-To: References: <64a6599c-fae1-469d-bcee-875165b3cc7d@googlegroups.com> <56d294f8$0$1604$c3e8da3$5496439d@news.astraweb.com> <234a398e-1b0f-467b-a8cb-d7ca748f8062@googlegroups.com> <84922f24-3e00-4a23-b26d-5e6c0d8e7e04@googlegroups.com> <87y4a5c58i.fsf@elektro.pacujo.net> <87twksdg9c.fsf@elektro.pacujo.net> <87d1rgca58.fsf@elektro.pacujo.net> Message-ID: On Mon, Feb 29, 2016 at 6:20 PM, Rustom Mody wrote: > On Monday, February 29, 2016 at 11:48:25 AM UTC+5:30, Marko Rauhamaa wrote: >> Gordon Levi : >> >> > Nobody likes filling in forms but how do you suggest converting a form >> > based app into something loveable. >> >> Straight HTML does forms just fine without CSS or JavaScript, yet few >> can resist. > > Abjure JS/CSS is a virtue? Why? > In any case with or without its still a form not plain (printf/scanf) text. > So what exactly are we talking of? Abjuring JS may be a virtue (or at least, making it a non-critical part of your web site), but CSS is important to document structure and layout. The combination of HTML and CSS provides a logical structure with separate styling, which IMO is an excellent thing. It's possible to accomplish the same goal in other ways (for instance, Markdown source code for the structure and HTML for the layout), but CSS provides enough flexibility to do this just fine on its own. Incidentally, HTML+CSS is another excellent example of code being used to create a visual effect. While there *are* WYSIWYG HTML editors, I'm not familiar with any WYISWYG HTML+CSS editors, and I much more often see a fast-turnaround code editing system such as codepen.io - you change the HTML in one box, or the CSS in another, and the result down below changes in real-time. It wouldn't be too hard to create something like this for a GUI, and it'd remove some of that feeling of non-interactivity while still retaining all the benefits of code above drag-and-drop. ChrisA From marko at pacujo.net Mon Feb 29 03:25:56 2016 From: marko at pacujo.net (Marko Rauhamaa) Date: Mon, 29 Feb 2016 10:25:56 +0200 Subject: Everything good about Python except GUI IDE? References: <64a6599c-fae1-469d-bcee-875165b3cc7d@googlegroups.com> <56d294f8$0$1604$c3e8da3$5496439d@news.astraweb.com> <234a398e-1b0f-467b-a8cb-d7ca748f8062@googlegroups.com> <84922f24-3e00-4a23-b26d-5e6c0d8e7e04@googlegroups.com> <87y4a5c58i.fsf@elektro.pacujo.net> <87twksdg9c.fsf@elektro.pacujo.net> <87d1rgca58.fsf@elektro.pacujo.net> Message-ID: <87h9grorcb.fsf@elektro.pacujo.net> Rustom Mody : > On Monday, February 29, 2016 at 11:48:25 AM UTC+5:30, Marko Rauhamaa wrote: >> Gordon Levi : >> >> > Nobody likes filling in forms but how do you suggest converting a form >> > based app into something loveable. >> >> Straight HTML does forms just fine without CSS or JavaScript, yet few >> can resist. > > Abjure JS/CSS is a virtue? Why? I have used some minute amounts of CSS when I have wanted to pay attention to the aesthetics (). However, then the point is conveying/exchanging info, straight HTML is all that is needed: . As for why you should avoid JS/CSS, Web pages open very slowly, jump around wildly during rendering and have unexpected artifacts (not to mention the numerous data collection abuses) when they are encumbered with truckloads of state-of-the-art web dev gimmicks. Marko From rosuav at gmail.com Mon Feb 29 03:33:03 2016 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 29 Feb 2016 19:33:03 +1100 Subject: Everything good about Python except GUI IDE? In-Reply-To: <87h9grorcb.fsf@elektro.pacujo.net> References: <64a6599c-fae1-469d-bcee-875165b3cc7d@googlegroups.com> <56d294f8$0$1604$c3e8da3$5496439d@news.astraweb.com> <234a398e-1b0f-467b-a8cb-d7ca748f8062@googlegroups.com> <84922f24-3e00-4a23-b26d-5e6c0d8e7e04@googlegroups.com> <87y4a5c58i.fsf@elektro.pacujo.net> <87twksdg9c.fsf@elektro.pacujo.net> <87d1rgca58.fsf@elektro.pacujo.net> <87h9grorcb.fsf@elektro.pacujo.net> Message-ID: On Mon, Feb 29, 2016 at 7:25 PM, Marko Rauhamaa wrote: > As for why you should avoid JS/CSS, Web pages open very slowly, jump > around wildly during rendering and have unexpected artifacts (not to > mention the numerous data collection abuses) when they are encumbered > with truckloads of state-of-the-art web dev gimmicks. And when I pick up a paintbrush, canvas, and oil paints, the result is appallingly hard on the eyes. Clearly oil paints should not be used, and we should just place the brush tastefully on the canvas, because that is guaranteed to look good. Don't blame the tool for its poor users. ChrisA From marko at pacujo.net Mon Feb 29 03:37:51 2016 From: marko at pacujo.net (Marko Rauhamaa) Date: Mon, 29 Feb 2016 10:37:51 +0200 Subject: Everything good about Python except GUI IDE? References: <64a6599c-fae1-469d-bcee-875165b3cc7d@googlegroups.com> <56d294f8$0$1604$c3e8da3$5496439d@news.astraweb.com> <234a398e-1b0f-467b-a8cb-d7ca748f8062@googlegroups.com> <84922f24-3e00-4a23-b26d-5e6c0d8e7e04@googlegroups.com> <87y4a5c58i.fsf@elektro.pacujo.net> <87twksdg9c.fsf@elektro.pacujo.net> <87d1rgca58.fsf@elektro.pacujo.net> Message-ID: <87d1rfoqsg.fsf@elektro.pacujo.net> Chris Angelico : > CSS is important to document structure and layout. Most Web pages had better not concern themselves with layout but with info content only. Unfortunately, HTML is a dirt-poor language for markup (no extensibility or programmability) so I can understand the temptation to side-step it with CSS. CSS has tons of esoterics but seems to lack some very basic facilities so I can understand the temptation to side-step it with JavaScript. > Incidentally, HTML+CSS is another excellent example of code being used > to create a visual effect. While there *are* WYSIWYG HTML editors, I'm > not familiar with any WYISWYG HTML+CSS editors, and I much more often > see a fast-turnaround code editing system such as codepen.io - you > change the HTML in one box, or the CSS in another, and the result down > below changes in real-time. It wouldn't be too hard to create > something like this for a GUI, and it'd remove some of that feeling of > non-interactivity while still retaining all the benefits of code above > drag-and-drop. Yes, WYSIWYG is a doomed approach. There's a difference between input and output. Marko From marko at pacujo.net Mon Feb 29 03:46:10 2016 From: marko at pacujo.net (Marko Rauhamaa) Date: Mon, 29 Feb 2016 10:46:10 +0200 Subject: Everything good about Python except GUI IDE? References: <64a6599c-fae1-469d-bcee-875165b3cc7d@googlegroups.com> <56d294f8$0$1604$c3e8da3$5496439d@news.astraweb.com> <234a398e-1b0f-467b-a8cb-d7ca748f8062@googlegroups.com> <84922f24-3e00-4a23-b26d-5e6c0d8e7e04@googlegroups.com> <87y4a5c58i.fsf@elektro.pacujo.net> <87twksdg9c.fsf@elektro.pacujo.net> <87d1rgca58.fsf@elektro.pacujo.net> <87h9grorcb.fsf@elektro.pacujo.net> Message-ID: <878u23oqel.fsf@elektro.pacujo.net> Chris Angelico : > On Mon, Feb 29, 2016 at 7:25 PM, Marko Rauhamaa wrote: >> As for why you should avoid JS/CSS, Web pages open very slowly, jump >> around wildly during rendering and have unexpected artifacts (not to >> mention the numerous data collection abuses) when they are encumbered >> with truckloads of state-of-the-art web dev gimmicks. > > And when I pick up a paintbrush, canvas, and oil paints, the result is > appallingly hard on the eyes. Clearly oil paints should not be used, > and we should just place the brush tastefully on the canvas, because > that is guaranteed to look good. > > Don't blame the tool for its poor users. In that vein... The men turned away and went toward Sodom, but Abraham remained standing before the Lord. Then Abraham approached him and said: ?Will you sweep away the righteous with the wicked? What if there are fifty righteous people in the city? Will you really sweep it away and not spare the place for the sake of the fifty righteous people in it? Far be it from you to do such a thing?to kill the righteous with the wicked, treating the righteous and the wicked alike. Far be it from you! Will not the Judge of all the earth do right?? The Lord said, ?If I find fifty righteous people in the city of Sodom, I will spare the whole place for their sake.? Then Abraham spoke up again: ?Now that I have been so bold as to speak to the Lord, though I am nothing but dust and ashes, what if the number of the righteous is five less than fifty? Will you destroy the whole city for lack of five people?? ?If I find forty-five there,? he said, ?I will not destroy it.? Once again he spoke to him, ?What if only forty are found there?? He said, ?For the sake of forty, I will not do it.? Then he said, ?May the Lord not be angry, but let me speak. What if only thirty can be found there?? He answered, ?I will not do it if I find thirty there.? Abraham said, ?Now that I have been so bold as to speak to the Lord, what if only twenty can be found there?? He said, ?For the sake of twenty, I will not destroy it.? Then he said, ?May the Lord not be angry, but let me speak just once more. What if only ten can be found there?? He answered, ?For the sake of ten, I will not destroy it.? When the Lord had finished speaking with Abraham, he left, and Abraham returned home. Early the next morning Abraham got up and returned to the place where he had stood before the Lord. He looked down toward Sodom and Gomorrah, toward all the land of the plain, and he saw dense smoke rising from the land, like smoke from a furnace. Marko From ganesh1pal at gmail.com Mon Feb 29 05:03:18 2016 From: ganesh1pal at gmail.com (Ganesh Pal) Date: Mon, 29 Feb 2016 15:33:18 +0530 Subject: usage of try except for review. In-Reply-To: References: Message-ID: Iam really sorry , I will have to resend my question again , because the earlier post had mistakes and formatting was bad , so apologies for top posting will try to avoid such mistakes in future. Iam on python 2.6 and Linux , need your suggestion on the usage of try and except in this program #!/usr/bin/env python """ """ import os import shlex import subprocess import sys import time import logging import run import pdb def run_cmd_and_verify(cmd, timeout=1000): try: out, err, ret = run(cmd, timeout=timeout) assert ret ==0,"ERROR (ret %d): " \ " \nout: %s\nerr: %s\n" % (ret, out, err) except Exception as e: logging.error("Failed to run %s got %s" % (cmd, e)) return False return True def run_test(): """ Mount """ pdb.set_trace() for cmd in ["mount /nfs_mount1", "mount /cifs_mount1"]: try: if not run_cmd_and_verify(cmd, timeout=3600): return False except: logging.error("Some meaningful message") logging.info("Setup and Creation ....Done !!!") # cmd = "run_scan" out, err, ret = run(cmd) for cmd in ["create_data.py -nfs ", "validate.py -30 "]: try: if not run_cmd_and_verify(cmd, timeout=3600): return False except: logging.error("some meaningful message") return False logging.info("Mount IS START.....Done !!!") def main(): if not run_test(): sys.exit("Exiting Main") if __name__ == '__main__': main() Question 1: (a) Have I used try and expect block correctly ? in run_test() I have expect block which displays some error message instead of pass , is it fine ? (b) Have I returned True and False correctly , looks fine for me try: if not run_cmd_and_verify(cmd, timeout=3600): return False except: logging.error("inside except") return False Question 2. (a) If a failure?s are encountered the error by assert condition the errors are now displayed on the screen , how do I redirect it to log file using logging error because the moment assert ret==0 becomes true the program exits def run_cmd_and_verify(cmd, timeout=1000): try: out, err, ret = run(cmd, timeout=timeout) assert ret ==0,"ERROR (ret %d): " \ " \nout: %s\nerr: %s\n" % (ret, out, err) except Exception as e: logging.error("Failed to run %s got %s" % (cmd, e)) return False return True #script_10.py Failed to run mount /nfs got ERROR (ret 1): out: host-44-3 exited with status 1 err: host-44-3: mount_efs: on /nfs: efs is already mounted 3. my function def has 1000 but Iam using 3600 in the calling fnx etc , Time out value are overwritten ? 4. Any further improvement particularly on try -except ? Regards, Ganesh From bc at freeuk.com Mon Feb 29 06:56:05 2016 From: bc at freeuk.com (BartC) Date: Mon, 29 Feb 2016 11:56:05 +0000 Subject: Everything good about Python except GUI IDE? In-Reply-To: <56d39589$0$1617$c3e8da3$5496439d@news.astraweb.com> References: <64a6599c-fae1-469d-bcee-875165b3cc7d@googlegroups.com> <56d294f8$0$1604$c3e8da3$5496439d@news.astraweb.com> <234a398e-1b0f-467b-a8cb-d7ca748f8062@googlegroups.com> <84922f24-3e00-4a23-b26d-5e6c0d8e7e04@googlegroups.com> <87y4a5c58i.fsf@elektro.pacujo.net> <56d39589$0$1617$c3e8da3$5496439d@news.astraweb.com> Message-ID: On 29/02/2016 00:49, Steven D'Aprano wrote: > On Mon, 29 Feb 2016 01:29 am, BartC wrote: > >> 20 years ago, when these things were simpler, MS Word had a mind of its >> own even then. I had to produce a manual of few hundred pages, with >> diagrams and images, and it just wasn't going to happen. Not without >> spending a year on it. And employing someone to do it cost thousands. > > So you're saying that somebody else knew how to do it, but you didn't, so it > would take you a year? An outsider would be using their own methods, perhaps some more apt DTP solution; I wouldn't know. > I find it hard to believe that Word 20 years ago couldn't deal with > something as small as a couple of hundred pages with diagrams. Possibly, but just because it could, maybe it does so in a complicated manner. Or maybe it's wasn't flexible. Or maybe some things I needed just weren't possible. (How did it do contents, appendices, indices, references to labelled items in the rest of the text? One manual I did was for the very script language I was using: how could Word render code, and do so with automatic highlighting (eg boldening) of keywords for a language it knew nothing about?) And my experience is that a lot of these things were incredibly fiddly anyway. I'm really the sort of guy who needs to have separate plain-text and rendered verions of a document. > I find it > much easier to believe that you're the sort of guy who would rather spend > three days inventing your solution from scratch (involving your own custom > programming language no less) than three hours reading the manual of the > existing solution. The language wasn't created just for this purpose. It was part of the application the manual was about. And in fact it had its own manual later on. The graphics application was also the one used to preview the rendered pages before sending them off to a Postscript printer. (And actually some of the diagrams were in a 3D vector format belonging to the application, a projection of which could sent as vector graphics to PS; I would have needed to rasterise them for Word. So in some ways, my solution was more sophisticated than using Word.) >> In the end I spent a week or two throwing together some simple mark-up >> language, written in my own editor, which was then processed by my own >> script language and ending up (via my own graphics software along the >> way) as Postscript. The results were perfect. > > /s/three days/two weeks/ Actually I can't remember how long it took. I know I spent some time building font-data tables for the fonts which only existed inside the Postscript printer. >> (Have you ever had a situation where you have to edit a bit of text >> where a word is in italic or has some particular style. You delete the >> word, and try and add some more text, but it persists in using the style >> of the deleted text rather than the current style. > You know that Word lets you reset the style to plain (Roman) text? There's > no need to delete the entire line, let alone the entire document. Text > styles are toggled: just hit Ctrl-I (or is it Ctrl-Shift-I, it's been a > while since I've used Word) to toggle italic on and off. Even if the > current paragraph defaults to italic, you can still toggle it off that way. Yes but you have to keep doing it. It's not just Word; even Thunderbird (which I'm using right now) sometimes gets in a twist about quoted text (shown in blue) and new text shown in black. It wouldn't matter but quoted text doesn't wrap at the ends of lines. It also keeps screwing up any copy and paste that happens to have indents. Here's one example: One Two Three And if I do a simple copy and paste: One Two Three And it's not always as simple as just stripping leading spaces; sometimes they are just munged. And I haven't even posted yet which often gives yet more unexpected results. Now someone is going to tell me what I'm doing wrong. I need to set X in Y to Z; obviously! The point is that it should Just Work. Multiply these little things by dozens of examples and you will see how using Other People's Software can often be a complete pain. -- Bartc From rustompmody at gmail.com Mon Feb 29 07:42:48 2016 From: rustompmody at gmail.com (Rustom Mody) Date: Mon, 29 Feb 2016 04:42:48 -0800 (PST) Subject: Correct IDLE usage (was Reason for not allowing import twice but allowing reload()) In-Reply-To: <645cdd46-d4a4-49b3-a0d8-848608d70d73@googlegroups.com> References: <645cdd46-d4a4-49b3-a0d8-848608d70d73@googlegroups.com> Message-ID: <075122af-1a9c-44b6-97bb-521ebcb3e4ae@googlegroups.com> On Monday, February 29, 2016 at 12:10:28 PM UTC+5:30, alien2utoo wrote: > Hello list, > > We can not import a module twice in a session of Python (subsequent attempts to import same module don't result in any error though, but it is not-effective). > > However after making change to module, we can reload() it (if not reload(), we could possibly have reimport statement) to get the updates in module. > > I am a newbie in Python (learning via IDLE) and trying to understand > Hi and welcome! > - what is extra (special) that import does, that reload() doesn't? > - if both do the same internally and externally, why subsequent imports of same module are ineffective? > - if subsequent imports of same module in a session are not effective, why not simply flag those attempts as an error, rather than letting them go effect-less. > > Kindly help me understand the reasons behind. I guess Ian and Chris have answered well enough in a general way. However I wonder at a more pragmatic level: Is import needed at all when trying out in Idle? [Maybe Idle experts can comment...] In more detail: 1. Start idle 2. Open new with Ctrl-n 3. Put the following into foo.py ------------------- x = 3 def foo(y) : return x+y ------------------- 4. Load it into python with F5 (or Run-> Run_Module) And have this interaction >>> ================================ RESTART ================================ >>> >>> x 3 >>> foo(2) 5 Note the restart 5. Switch back to foo.py and change the '+' to '*'; F5 >>> ================================ RESTART ================================ >>> >>> foo(2) 6 SO THE CHANGE IS EFFECTED 6. Yes there is a difference between importing and 'F5-ing'; To see that I add to bottom of foo.py if __name__ == '__main__': print "if" else: print "else" 7. Now with F5: >>> ================================ RESTART ================================ >>> if Whereas with import: >>> import foo else >>> So it does appear that 1. import not necessary with(in) idle 2. However import and f5 (ie is run as main) are different May some idle experts elaborate on this? Whats the idle idiom of import-ing? From tomwilliamson115 at gmail.com Mon Feb 29 07:53:59 2016 From: tomwilliamson115 at gmail.com (tomwilliamson115 at gmail.com) Date: Mon, 29 Feb 2016 04:53:59 -0800 (PST) Subject: Help In-Reply-To: References: <6cdcd063-1c22-4f08-900c-f3260a543018@googlegroups.com> Message-ID: Thanks. If a word appears more than once how would I bring back both locations? From alien2utoo at gmail.com Mon Feb 29 08:00:05 2016 From: alien2utoo at gmail.com (alien2utoo at gmail.com) Date: Mon, 29 Feb 2016 05:00:05 -0800 (PST) Subject: Reason for not allowing import twice but allowing reload() In-Reply-To: <645cdd46-d4a4-49b3-a0d8-848608d70d73@googlegroups.com> References: <645cdd46-d4a4-49b3-a0d8-848608d70d73@googlegroups.com> Message-ID: <9d7e1d73-e424-4d6b-8234-a625fd3a306b@googlegroups.com> Thanks Chris and Ian, Your suggested experiments and explanations have really provided some good insights, something Tutorial didn't indicate. Ian's explanation reminds me of #ifndef _HEADER_H_ #define _HEADER_H_ ... #endif // _HEADER_H_ in C/C++, and I can draw parallels to why subsequent attempts to import same module shouldn't result in flagging an error, and need for multiple imports to refer to same information. From joel.goldstick at gmail.com Mon Feb 29 08:05:22 2016 From: joel.goldstick at gmail.com (Joel Goldstick) Date: Mon, 29 Feb 2016 08:05:22 -0500 Subject: Help In-Reply-To: References: <6cdcd063-1c22-4f08-900c-f3260a543018@googlegroups.com> Message-ID: On Mon, Feb 29, 2016 at 7:53 AM, wrote: > Thanks. If a word appears more than once how would I bring back both > locations? > This is not generally a free coding service. Fortunately for you Tom, Jacob was kind enough to do your homework for you. The problem with doing someone's homework for them is that they don't learn anything. Why don't you go line by line through his code and understand what he has written. I think if you do that you will see how to solve the problem of finding then next match (if there is one) > -- > https://mail.python.org/mailman/listinfo/python-list > -- Joel Goldstick http://joelgoldstick.com/ http://cc-baseballstats.info/ From alien2utoo at gmail.com Mon Feb 29 08:22:44 2016 From: alien2utoo at gmail.com (alien2utoo at gmail.com) Date: Mon, 29 Feb 2016 05:22:44 -0800 (PST) Subject: Reason for not allowing import twice but allowing reload() In-Reply-To: <645cdd46-d4a4-49b3-a0d8-848608d70d73@googlegroups.com> References: <645cdd46-d4a4-49b3-a0d8-848608d70d73@googlegroups.com> Message-ID: <5c8e3283-2013-4f68-87b8-6311ccee64a3@googlegroups.com> Hello Rustom, F5 in Idle restarts the Python interpreter (that's what my impression is). Whatever you have done earlier at Idle prompt (in Idle session) before F5 is gone after F5. Try simple experiment at prompt. >>> myvar="hello" >>> myvar 'hello' myvar is gone after F5. As for need of import in Idle session, I use it to - import sys - sys.append.path('D:\\Where\\Ever\\My\\Modules\\Lie') - import mymodule From alien2utoo at gmail.com Mon Feb 29 08:25:59 2016 From: alien2utoo at gmail.com (alien2utoo at gmail.com) Date: Mon, 29 Feb 2016 05:25:59 -0800 (PST) Subject: Reason for not allowing import twice but allowing reload() In-Reply-To: <5c8e3283-2013-4f68-87b8-6311ccee64a3@googlegroups.com> References: <645cdd46-d4a4-49b3-a0d8-848608d70d73@googlegroups.com> <5c8e3283-2013-4f68-87b8-6311ccee64a3@googlegroups.com> Message-ID: <3ec34f5c-7d18-49c8-8d38-dcd4e7c58e9b@googlegroups.com> > As for need of import in Idle session, I use it to > - import sys > - sys.append.path('D:\\Where\\Ever\\My\\Modules\\Lie') Kindly read above as sys.path.append(....) > - import mymodule From breamoreboy at yahoo.co.uk Mon Feb 29 08:41:53 2016 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Mon, 29 Feb 2016 13:41:53 +0000 Subject: Help In-Reply-To: References: <6cdcd063-1c22-4f08-900c-f3260a543018@googlegroups.com> Message-ID: On 29/02/2016 12:53, tomwilliamson115 at gmail.com wrote: > Thanks. If a word appears more than once how would I bring back both locations? > I've no idea without any context, would you please care to elucidate, thanks. -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From jonas.thornvall at gmail.com Mon Feb 29 08:45:18 2016 From: jonas.thornvall at gmail.com (jonas.thornvall at gmail.com) Date: Mon, 29 Feb 2016 05:45:18 -0800 (PST) Subject: General computer language, syntax question. Message-ID: <5b619ffa-aae6-4fa8-9f62-58532668ec93@googlegroups.com> I have a problem programming uniform networks, "x nodes with y links" that turn out to be really hairy to solve for me and i feel i really lack the programming features "Actually i program in javascript" but the problem seem general for all programming languages including Pyhton. For a beautiful solution it would require "If in list/array return boolean" "If not in list/array return boolean". But there is no such feature Python nor Javascript, so instead i set boolean value "inlist" to false and loop thru, to see if it is already in list. If not it is added to list. So if the current node i generate links for is x and i try to generate a link to node z, and z's links exhausted i will add it to exhausted list. And if there is node-x exhausted entries in list, well then script should break because it will lock into a cycle. The cyclic lockup is due to only a subset of the networks on the form (links*deep)+1=nodes is possible. So what i need is to know howto write "if list/array ***empty*** do {something}" I sure know howto check if an array have 1 element 2,3,4 but how do you check for empty. I think it is crazy that you can not do general comparissons of the type If in list/array If not in list/array But it is even more crazy that you can not check if list/array is empty, that is just madness. From rustompmody at gmail.com Mon Feb 29 08:51:52 2016 From: rustompmody at gmail.com (Rustom Mody) Date: Mon, 29 Feb 2016 05:51:52 -0800 (PST) Subject: Reason for not allowing import twice but allowing reload() In-Reply-To: <5c8e3283-2013-4f68-87b8-6311ccee64a3@googlegroups.com> References: <645cdd46-d4a4-49b3-a0d8-848608d70d73@googlegroups.com> <5c8e3283-2013-4f68-87b8-6311ccee64a3@googlegroups.com> Message-ID: On Monday, February 29, 2016 at 6:53:09 PM UTC+5:30, alien... at gmail.com wrote: > Hello Rustom, > > F5 in Idle restarts the Python interpreter (that's what my impression is). Whatever you have done earlier at Idle prompt (in Idle session) before F5 is gone after F5. > > Try simple experiment at prompt. > > >>> myvar="hello" > >>> myvar > 'hello' > > myvar is gone after F5. > > As for need of import in Idle session, I use it to > - import sys > - sys.append.path('D:\\Where\\Ever\\My\\Modules\\Lie') > - import mymodule Why does one use (something like) idle? To experiment. So what's your experiment-focus? If it is mymodule then mymodule should be open in idle file window and idle will take care of paths If its someothermodule.py that has an import of mymodule.py then someothermodule should be handling the paths issue. In neither case I see a reason to do it at idle prompt [At least that's my understanding, hope an idle expert will weigh in on this] In linux this question does not typically arise as one starts idle from the shell in the same directory as the python files one wants to play with In windows... not sure of idiom... Maybe right-click the idle icon and change its 'start-in' to the path where your python files are situated? My general impression (best-practices not semantics) is that changing sys.path is allowed but hackish. 1. Modify sys.path 2. Using PYTHONPATH env variable 3. Use relative imports 4. Use packages ?. .pth files is roughly decreasing in hackishness though unfortunately increasing in sophistication http://stackoverflow.com/questions/19917492/how-to-use-pythonpath http://stackoverflow.com/questions/18521503/pydev-how-to-avoid-adding-sub-directories-to-python-path-in-order-to-fix-unres From joel.goldstick at gmail.com Mon Feb 29 08:55:19 2016 From: joel.goldstick at gmail.com (Joel Goldstick) Date: Mon, 29 Feb 2016 08:55:19 -0500 Subject: General computer language, syntax question. In-Reply-To: <5b619ffa-aae6-4fa8-9f62-58532668ec93@googlegroups.com> References: <5b619ffa-aae6-4fa8-9f62-58532668ec93@googlegroups.com> Message-ID: On Mon, Feb 29, 2016 at 8:45 AM, wrote: > I have a problem programming uniform networks, "x nodes with y links" that > turn out to be really hairy to solve for me and i feel i really lack the > programming features > > "Actually i program in javascript" but the problem seem general for all > programming languages including Pyhton. > > For a beautiful solution it would require "If in list/array return > boolean" "If not in list/array return boolean". > > But there is no such feature Python nor Javascript, so instead i set > boolean value "inlist" to false and loop thru, to see if it is already in > list. If not it is added to list. > > So if the current node i generate links for is x and i try to generate a > link to node z, and z's links exhausted i will add it to exhausted list. > > And if there is node-x exhausted entries in list, well then script should > break because it will lock into a cycle. The cyclic lockup is due to only a > subset of the networks on the form (links*deep)+1=nodes is possible. > > So what i need is to know howto write "if list/array ***empty*** do > {something}" > > I sure know howto check if an array have 1 element 2,3,4 but how do you > check for empty. > > I think it is crazy that you can not do general comparissons of the type > If in list/array > If not in list/array > > But it is even more crazy that you can not check if list/array is empty, > that is just madness. > -- > https://mail.python.org/mailman/listinfo/python-list > >>> l = [1,2,3,4] >>> if 1 in l: ... print 1 ... 1 >>> if 6 in l: ... print 1 ... >>> >>> l = [] >>> if len(l) == 0: ... print 'empty' ... empty -- Joel Goldstick http://joelgoldstick.com/ http://cc-baseballstats.info/ From jonas.thornvall at gmail.com Mon Feb 29 08:56:52 2016 From: jonas.thornvall at gmail.com (jonas.thornvall at gmail.com) Date: Mon, 29 Feb 2016 05:56:52 -0800 (PST) Subject: General computer language, syntax question. In-Reply-To: <5b619ffa-aae6-4fa8-9f62-58532668ec93@googlegroups.com> References: <5b619ffa-aae6-4fa8-9f62-58532668ec93@googlegroups.com> Message-ID: Den m?ndag 29 februari 2016 kl. 14:45:37 UTC+1 skrev jonas.t... at gmail.com: > I have a problem programming uniform networks, "x nodes with y links" that turn out to be really hairy to solve for me and i feel i really lack the programming features > > "Actually i program in javascript" but the problem seem general for all programming languages including Pyhton. > > For a beautiful solution it would require "If in list/array return boolean" "If not in list/array return boolean". > > But there is no such feature Python nor Javascript, so instead i set boolean value "inlist" to false and loop thru, to see if it is already in list. If not it is added to list. > > So if the current node i generate links for is x and i try to generate a link to node z, and z's links exhausted i will add it to exhausted list. > > And if there is node-x exhausted entries in list, well then script should break because it will lock into a cycle. The cyclic lockup is due to only a subset of the networks on the form (links*deep)+1=nodes is possible. > > So what i need is to know howto write "if list/array ***empty*** do {something}" > > I sure know howto check if an array have 1 element 2,3,4 but how do you check for empty. > > I think it is crazy that you can not do general comparissons of the type > If in list/array > If not in list/array > > But it is even more crazy that you can not check if list/array is empty, that is just madness. I mean for for example Javascript if (typeof array[index] !== 'undefined' && array[index] !== null) { or this one if (array[index] != null) { I mean how do they come up with such convoluted syntax, do they pull it out of ass? Well one can always say it is needed because undefined not equal to null. But would not if (array[index]==empty) suffice and be alot clearer? From jonas.thornvall at gmail.com Mon Feb 29 09:05:11 2016 From: jonas.thornvall at gmail.com (jonas.thornvall at gmail.com) Date: Mon, 29 Feb 2016 06:05:11 -0800 (PST) Subject: General computer language, syntax question. In-Reply-To: References: <5b619ffa-aae6-4fa8-9f62-58532668ec93@googlegroups.com> Message-ID: Den m?ndag 29 februari 2016 kl. 14:57:04 UTC+1 skrev jonas.t... at gmail.com: > Den m?ndag 29 februari 2016 kl. 14:45:37 UTC+1 skrev jonas.t... at gmail.com: > > I have a problem programming uniform networks, "x nodes with y links" that turn out to be really hairy to solve for me and i feel i really lack the programming features > > > > "Actually i program in javascript" but the problem seem general for all programming languages including Pyhton. > > > > For a beautiful solution it would require "If in list/array return boolean" "If not in list/array return boolean". > > > > But there is no such feature Python nor Javascript, so instead i set boolean value "inlist" to false and loop thru, to see if it is already in list. If not it is added to list. > > > > So if the current node i generate links for is x and i try to generate a link to node z, and z's links exhausted i will add it to exhausted list. > > > > And if there is node-x exhausted entries in list, well then script should break because it will lock into a cycle. The cyclic lockup is due to only a subset of the networks on the form (links*deep)+1=nodes is possible. > > > > So what i need is to know howto write "if list/array ***empty*** do {something}" > > > > I sure know howto check if an array have 1 element 2,3,4 but how do you check for empty. > > > > I think it is crazy that you can not do general comparissons of the type > > If in list/array > > If not in list/array > > > > But it is even more crazy that you can not check if list/array is empty, that is just madness. > > I mean for for example Javascript > if (typeof array[index] !== 'undefined' && array[index] !== null) { > or this one > if (array[index] != null) { > > I mean how do they come up with such convoluted syntax, do they pull it out of ass? Well one can always say it is needed because undefined not equal to null. > > But would not if (array[index]==empty) suffice and be alot clearer? Sorry but would not if (array==empty) suffice and be alot clearer? From python at mrabarnett.plus.com Mon Feb 29 09:07:02 2016 From: python at mrabarnett.plus.com (MRAB) Date: Mon, 29 Feb 2016 14:07:02 +0000 Subject: General computer language, syntax question. In-Reply-To: <5b619ffa-aae6-4fa8-9f62-58532668ec93@googlegroups.com> References: <5b619ffa-aae6-4fa8-9f62-58532668ec93@googlegroups.com> Message-ID: <56D45086.5090100@mrabarnett.plus.com> On 2016-02-29 13:45, jonas.thornvall at gmail.com wrote: > I have a problem programming uniform networks, "x nodes with y links" that turn out to be really hairy to solve for me and i feel i really lack the programming features > > "Actually i program in javascript" but the problem seem general for all programming languages including Pyhton. > > For a beautiful solution it would require "If in list/array return boolean" "If not in list/array return boolean". > > But there is no such feature Python nor Javascript, so instead i set boolean value "inlist" to false and loop thru, to see if it is already in list. If not it is added to list. > > So if the current node i generate links for is x and i try to generate a link to node z, and z's links exhausted i will add it to exhausted list. > > And if there is node-x exhausted entries in list, well then script should break because it will lock into a cycle. The cyclic lockup is due to only a subset of the networks on the form (links*deep)+1=nodes is possible. > > So what i need is to know howto write "if list/array ***empty*** do {something}" > > I sure know howto check if an array have 1 element 2,3,4 but how do you check for empty. > > I think it is crazy that you can not do general comparissons of the type > If in list/array > If not in list/array > > But it is even more crazy that you can not check if list/array is empty, that is just madness. > Does this help? >>> items = ['a', 'b', 'c', 'd', 'e'] >>> 'c' in items True >>> 'f' in items False >>> len(items) 5 >>> len([]) 0 From joel.goldstick at gmail.com Mon Feb 29 09:38:50 2016 From: joel.goldstick at gmail.com (Joel Goldstick) Date: Mon, 29 Feb 2016 09:38:50 -0500 Subject: General computer language, syntax question. In-Reply-To: References: <5b619ffa-aae6-4fa8-9f62-58532668ec93@googlegroups.com> Message-ID: On Mon, Feb 29, 2016 at 9:05 AM, wrote: > Den m?ndag 29 februari 2016 kl. 14:57:04 UTC+1 skrev jonas.t... at gmail.com: > > Den m?ndag 29 februari 2016 kl. 14:45:37 UTC+1 skrev > jonas.t... at gmail.com: > > > I have a problem programming uniform networks, "x nodes with y links" > that turn out to be really hairy to solve for me and i feel i really lack > the programming features > > > > > > "Actually i program in javascript" but the problem seem general for > all programming languages including Pyhton. > > > > > > For a beautiful solution it would require "If in list/array return > boolean" "If not in list/array return boolean". > > > > > > But there is no such feature Python nor Javascript, so instead i set > boolean value "inlist" to false and loop thru, to see if it is already in > list. If not it is added to list. > > > > > > So if the current node i generate links for is x and i try to generate > a link to node z, and z's links exhausted i will add it to exhausted list. > > > > > > And if there is node-x exhausted entries in list, well then script > should break because it will lock into a cycle. The cyclic lockup is due to > only a subset of the networks on the form (links*deep)+1=nodes is possible. > > > > > > So what i need is to know howto write "if list/array ***empty*** do > {something}" > > > > > > I sure know howto check if an array have 1 element 2,3,4 but how do > you check for empty. > > > > > > I think it is crazy that you can not do general comparissons of the > type > > > If in list/array > > > If not in list/array > > > > > > But it is even more crazy that you can not check if list/array is > empty, that is just madness. > > > > I mean for for example Javascript > > if (typeof array[index] !== 'undefined' && array[index] !== null) { > > or this one > > if (array[index] != null) { > > > > I mean how do they come up with such convoluted syntax, do they pull it > out of ass? Well one can always say it is needed because undefined not > equal to null. > > > > But would not if (array[index]==empty) suffice and be alot clearer? > > Sorry but would not if (array==empty) suffice and be alot clearer? > -- > https://mail.python.org/mailman/listinfo/python-list > You might have better luck on a javascript mailing list. Not sure crazy, madness, ass are descriptive to your problems -- Joel Goldstick http://joelgoldstick.com/ http://cc-baseballstats.info/ From breamoreboy at yahoo.co.uk Mon Feb 29 09:39:49 2016 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Mon, 29 Feb 2016 14:39:49 +0000 Subject: General computer language, syntax question. In-Reply-To: <5b619ffa-aae6-4fa8-9f62-58532668ec93@googlegroups.com> References: <5b619ffa-aae6-4fa8-9f62-58532668ec93@googlegroups.com> Message-ID: On 29/02/2016 13:45, jonas.thornvall at gmail.com wrote: > I have a problem programming uniform networks, "x nodes with y links" that turn out to be really hairy to solve for me and i feel i really lack the programming features > > "Actually i program in javascript" but the problem seem general for all programming languages including Pyhton. > > For a beautiful solution it would require "If in list/array return boolean" "If not in list/array return boolean". > > But there is no such feature Python nor Javascript, so instead i set boolean value "inlist" to false and loop thru, to see if it is already in list. If not it is added to list. > > So if the current node i generate links for is x and i try to generate a link to node z, and z's links exhausted i will add it to exhausted list. > > And if there is node-x exhausted entries in list, well then script should break because it will lock into a cycle. The cyclic lockup is due to only a subset of the networks on the form (links*deep)+1=nodes is possible. > > So what i need is to know howto write "if list/array ***empty*** do {something}" > > I sure know howto check if an array have 1 element 2,3,4 but how do you check for empty. > > I think it is crazy that you can not do general comparissons of the type > If in list/array > If not in list/array > > But it is even more crazy that you can not check if list/array is empty, that is just madness. > Please see https://docs.python.org/3/library/stdtypes.html#truth-value-testing -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From joel.goldstick at gmail.com Mon Feb 29 09:48:48 2016 From: joel.goldstick at gmail.com (Joel Goldstick) Date: Mon, 29 Feb 2016 09:48:48 -0500 Subject: General computer language, syntax question. In-Reply-To: References: <5b619ffa-aae6-4fa8-9f62-58532668ec93@googlegroups.com> Message-ID: On Mon, Feb 29, 2016 at 9:39 AM, Mark Lawrence wrote: > On 29/02/2016 13:45, jonas.thornvall at gmail.com wrote: > >> I have a problem programming uniform networks, "x nodes with y links" >> that turn out to be really hairy to solve for me and i feel i really lack >> the programming features >> >> "Actually i program in javascript" but the problem seem general for all >> programming languages including Pyhton. >> >> For a beautiful solution it would require "If in list/array return >> boolean" "If not in list/array return boolean". >> >> But there is no such feature Python nor Javascript, so instead i set >> boolean value "inlist" to false and loop thru, to see if it is already in >> list. If not it is added to list. >> >> So if the current node i generate links for is x and i try to generate a >> link to node z, and z's links exhausted i will add it to exhausted list. >> >> And if there is node-x exhausted entries in list, well then script should >> break because it will lock into a cycle. The cyclic lockup is due to only a >> subset of the networks on the form (links*deep)+1=nodes is possible. >> >> So what i need is to know howto write "if list/array ***empty*** do >> {something}" >> >> I sure know howto check if an array have 1 element 2,3,4 but how do you >> check for empty. >> >> I think it is crazy that you can not do general comparissons of the type >> If in list/array >> If not in list/array >> >> But it is even more crazy that you can not check if list/array is empty, >> that is just madness. >> >> > Please see > https://docs.python.org/3/library/stdtypes.html#truth-value-testing > > -- > My fellow Pythonistas, ask not what our language can do for you, ask > what you can do for our language. > > Mark Lawrence > > -- > https://mail.python.org/mailman/listinfo/python-list > >>> l = [] >>> l [] >>> if l: ... print 'not empty' ... >>> -- Joel Goldstick http://joelgoldstick.com/ http://cc-baseballstats.info/ From ian.g.kelly at gmail.com Mon Feb 29 09:54:05 2016 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Mon, 29 Feb 2016 07:54:05 -0700 Subject: General computer language, syntax question. In-Reply-To: References: <5b619ffa-aae6-4fa8-9f62-58532668ec93@googlegroups.com> Message-ID: On Feb 29, 2016 7:11 AM, wrote: > > Sorry but would not if (array==empty) suffice and be alot clearer? In Python, you can just do "if len(array) == 0" or "if not array". In JavaScript you have "if (array.length === 0)". Is there some problem with that? I would prefer this over your suggestion since it doesn't require some variable named "empty" to have a sensible value set. On the container test, as others pointed out, Python has the "in" operator. JavaScript has a draft Array.prototype.includes method, but it doesn't have broad cross-browser support yet. From alien2utoo at gmail.com Mon Feb 29 10:13:56 2016 From: alien2utoo at gmail.com (alien2utoo at gmail.com) Date: Mon, 29 Feb 2016 07:13:56 -0800 (PST) Subject: Reason for not allowing import twice but allowing reload() In-Reply-To: References: <645cdd46-d4a4-49b3-a0d8-848608d70d73@googlegroups.com> <5c8e3283-2013-4f68-87b8-6311ccee64a3@googlegroups.com> Message-ID: > Why does one use (something like) idle? > To experiment. > > So what's your experiment-focus? True. Experiment only. What happens (and in environment) when you use - import module - from module import name - from module import name as othername Idle is start point, but we aren't always going to be Idle-ing, so need to work towards that as well while practicing basic concepts. [P.S.- I think we are digressing a lot from original thread question.] From ganesh1pal at gmail.com Mon Feb 29 10:18:57 2016 From: ganesh1pal at gmail.com (Ganesh Pal) Date: Mon, 29 Feb 2016 20:48:57 +0530 Subject: common mistakes in this simple program Message-ID: Iam on python 2.6 , need inputs on the common mistakes in this program , may be you suggest what need to be improved from 1. usage of try- expect 2. Return of True/ False 3. Other improvement #!/usr/bin/env python """ """ import os import shlex import subprocess import sys import time import logging import run import pdb def run_cmd_and_verify(cmd, timeout=1000): try: pdb.set_trace() out, err, ret = run(cmd, timeout=timeout) assert ret ==0,"ERROR (ret %d): " \ " \nout: %s\nerr: %s\n" % (ret, out, err) except Exception as e: print("Failed to run %s got %s" % (cmd, e)) return False return True def prep_host(): """ Prepare clustering """ for cmd in ["ls -al", "touch /tmp/file1", "mkdir /tmp/dir1"]: try: if not run_cmd_and_verify(cmd, timeout=3600): return False except: print("Error: While preparing cluster !!!") return False print("Preparing Cluster.....Done !!!") return True def main(): functions = [prep_host] for func in functions: try: func() except Exception as e: print(e) return False if __name__ == '__main__': main() Regards, Gpal From bc at freeuk.com Mon Feb 29 10:33:25 2016 From: bc at freeuk.com (BartC) Date: Mon, 29 Feb 2016 15:33:25 +0000 Subject: Reason for not allowing import twice but allowing reload() In-Reply-To: References: <645cdd46-d4a4-49b3-a0d8-848608d70d73@googlegroups.com> Message-ID: On 29/02/2016 07:11, Chris Angelico wrote: > On Mon, Feb 29, 2016 at 6:02 PM, Ian Kelly wrote: >>> - if subsequent imports of same module in a session are not effective, why not simply flag those attempts as an error, rather than letting them go effect-less. >> >> Because there are legitimate reasons for importing the same module >> multiple times. For example, I have a script that imports the sys >> module. It also imports the os module, which imports the sys module. >> Both the main module of my script and the os module have need of the >> sys module, so they both import it. If subsequent imports of the >> module raised an exception, this wouldn't be possible. > > I think the OP's talking more about the situation of having an active > session (IDLE was mentioned), importing a local module (a .py file > from the current directory), then editing the file and re-importing, > which has no effect. While I am sympathetic to the problem, I don't > believe the language should be changed here; what might be useful is > something that notices that an already-loaded module is now out of > date, but really, the best solution is a change of workflow that no > longer has long-running modules loading live-updated code. Not in Python (this was pre-Python actually) but I once used such a technique all the time. The main set of modules would be running a GUI application which also had a command line. Some commands included editing, compiling [a discrete step] and running modules of the application that were under development. Then you had a very fast edit-run cycle for those modules without having to restart the entire application, and possibly reload the graphical data that was under test (which could be large), or having to do any other set-ups required for the test. (Another reason in those days was that not all modules would fit into memory at once and a module could be unloaded once it had finished (in Python terms, running off the end of the module) to make room for another.) What I'm saying is, you shouldn't dismiss something just because you can't think of enough uses for it. -- Bartc From nomail at invalid.com Mon Feb 29 10:36:47 2016 From: nomail at invalid.com (ast) Date: Mon, 29 Feb 2016 16:36:47 +0100 Subject: Dynamic object attribute creation Message-ID: <56d46597$0$22756$426a74cc@news.free.fr> Hello Object's attributes can be created dynamically, ie class MyClass: pass obj = MyClass() obj.test = 'foo' but why doesn't it work with built-in classes int, float, list.... ? L = [1, 8, 0] L.test = 'its a list !' (however lists are mutable, int, float ... are not) Traceback (most recent call last): File "", line 1, in L.test = 'its a list !' AttributeError: 'list' object has no attribute 'test' but it works on functions: def funct(a,b): ....print(a+b) funct.test = 'this is a function' funct.test 'this is a function' From invalid at invalid.invalid Mon Feb 29 10:43:49 2016 From: invalid at invalid.invalid (Grant Edwards) Date: Mon, 29 Feb 2016 15:43:49 +0000 (UTC) Subject: Everything good about Python except GUI IDE? References: <64a6599c-fae1-469d-bcee-875165b3cc7d@googlegroups.com> <56d294f8$0$1604$c3e8da3$5496439d@news.astraweb.com> <234a398e-1b0f-467b-a8cb-d7ca748f8062@googlegroups.com> <84922f24-3e00-4a23-b26d-5e6c0d8e7e04@googlegroups.com> <87y4a5c58i.fsf@elektro.pacujo.net> <87twksdg9c.fsf@elektro.pacujo.net> <87d1rgca58.fsf@elektro.pacujo.net> Message-ID: On 2016-02-29, Chris Angelico wrote: > Abjuring JS may be a virtue (or at least, making it a non-critical > part of your web site), Except the marketing people who decide on the requirements will never, ever settle for what you can do with plain HTML/CSS. In my experience, HTML/CSS makes a pretty awful GUI for a non-trivial application. With some Javascript and sweat, you can almost make it to mediocre. > but CSS is important to document structure and layout. The > combination of HTML and CSS provides a logical structure with > separate styling, which IMO is an excellent thing. Indeed. Separating the visual appearnce stuff from the "structure" makes it a lot easier to deal with both, and the result is a lot more robust as well. -- Grant Edwards grant.b.edwards Yow! The Korean War must at have been fun. gmail.com From invalid at invalid.invalid Mon Feb 29 10:49:02 2016 From: invalid at invalid.invalid (Grant Edwards) Date: Mon, 29 Feb 2016 15:49:02 +0000 (UTC) Subject: Help References: <6cdcd063-1c22-4f08-900c-f3260a543018@googlegroups.com> Message-ID: On 2016-02-29, Joel Goldstick wrote: > On Mon, Feb 29, 2016 at 7:53 AM, wrote: > >> Thanks. If a word appears more than once how would I bring back both >> locations? > > This is not generally a free coding service. Fortunately for you Tom, > Jacob was kind enough to do your homework for you. The problem with doing > someone's homework for them is that they don't learn anything. And thus don't compete for the good jobs with us who did learn things. At least in theory. In real life they often still graduate and get hired anyway, get put in the cubicle next to you, and you spend half your time cleaning up after them. -- Grant Edwards grant.b.edwards Yow! NEWARK has been at REZONED!! DES MOINES has gmail.com been REZONED!! From mal at europython.eu Mon Feb 29 10:49:29 2016 From: mal at europython.eu (M.-A. Lemburg) Date: Mon, 29 Feb 2016 16:49:29 +0100 Subject: EuroPython 2016: Regular ticket prices Message-ID: <56D46889.3060906@europython.eu> We will be switching to regular ticket prices very soon now: * Student: EUR 120.00 (only available for students) * Personal: EUR 360.00 (for people enjoying Python from home) * Business: EUR 580.00 (for people using Python to make a living) but still have a few early-bird tickets left. If you buy until tomorrow at midnight (Tuesday, March 1st, 23:59 CET), you can save up to EUR 200 on early-bird prices, compared to the above regular prices: *** https://ep2016.europython.eu/en/registration/ *** More Information and Ticket Shop PS: Please remember to submit your proposals for the conference. There are only a few days left for submission. The deadline is Sunday, March 6: https://ep2016.europython.eu/en/call-for-proposals/ With gravitational regards, -- EuroPython 2016 Team http://ep2016.europython.eu/ http://www.europython-society.org/ From rosuav at gmail.com Mon Feb 29 11:05:24 2016 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 1 Mar 2016 03:05:24 +1100 Subject: Reason for not allowing import twice but allowing reload() In-Reply-To: References: <645cdd46-d4a4-49b3-a0d8-848608d70d73@googlegroups.com> Message-ID: On Tue, Mar 1, 2016 at 2:33 AM, BartC wrote: >> I think the OP's talking more about the situation of having an active >> session (IDLE was mentioned), importing a local module (a .py file >> from the current directory), then editing the file and re-importing, >> which has no effect. While I am sympathetic to the problem, I don't >> believe the language should be changed here; what might be useful is >> something that notices that an already-loaded module is now out of >> date, but really, the best solution is a change of workflow that no >> longer has long-running modules loading live-updated code. > > > Not in Python (this was pre-Python actually) but I once used such a > technique all the time. > > The main set of modules would be running a GUI application which also had a > command line. I use this technique all the time, in current code. But not in Python. The main reason is to have a server that permits clients to connect and remain connected, and to update code in ways that affect those connected clients. So by definition, I can't restart the server (it would drop the connections), and the critical main loop doesn't reload, but all the rest of the code does. It's pretty much the same style you had. Doing the same thing in Python would probably best be done using "exec" rather than "import". It'd work, but you'd have to differentiate in code between "load this module which can be reloaded" and "import this module from the standard library" (where the latter would be the regular "import" statement). ChrisA From random832 at fastmail.com Mon Feb 29 11:06:04 2016 From: random832 at fastmail.com (Random832) Date: Mon, 29 Feb 2016 11:06:04 -0500 Subject: Dynamic object attribute creation In-Reply-To: <56d46597$0$22756$426a74cc@news.free.fr> References: <56d46597$0$22756$426a74cc@news.free.fr> Message-ID: <1456761964.822300.535132330.33A18241@webmail.messagingengine.com> On Mon, Feb 29, 2016, at 10:36, ast wrote: > but why doesn't it work with built-in classes int, float, list.... ? > > L = [1, 8, 0] > L.test = 'its a list !' > > (however lists are mutable, int, float ... are not) Because those classes do not have attribute dictionaries, in order to save space. You can make a class without an attribute dictionary, by using __slots__. From wij at totalbb.net.tw Mon Feb 29 11:16:53 2016 From: wij at totalbb.net.tw (wij at totalbb.net.tw) Date: Mon, 29 Feb 2016 08:16:53 -0800 (PST) Subject: Xn project v002 is released Message-ID: <000f1f08-69d3-4df2-b4e4-4cfa025fb6ee@googlegroups.com> Xn project tries to develop a more uniform language that can be used in almost anywhere. It can be used for general files (e.g. executables), general communication language, or even for an alive program that can grow. Xn project is licensed as Public Domain (Historical Fact Principle). Anyone can start anything from Xn, as long as the historical fact is maintained. Suggestions are also welcome. https://sourceforge.net/projects/systemnode/ From rosuav at gmail.com Mon Feb 29 11:17:58 2016 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 1 Mar 2016 03:17:58 +1100 Subject: Everything good about Python except GUI IDE? In-Reply-To: References: <64a6599c-fae1-469d-bcee-875165b3cc7d@googlegroups.com> <56d294f8$0$1604$c3e8da3$5496439d@news.astraweb.com> <234a398e-1b0f-467b-a8cb-d7ca748f8062@googlegroups.com> <84922f24-3e00-4a23-b26d-5e6c0d8e7e04@googlegroups.com> <87y4a5c58i.fsf@elektro.pacujo.net> <87twksdg9c.fsf@elektro.pacujo.net> <87d1rgca58.fsf@elektro.pacujo.net> Message-ID: On Tue, Mar 1, 2016 at 2:43 AM, Grant Edwards wrote: > On 2016-02-29, Chris Angelico wrote: > >> Abjuring JS may be a virtue (or at least, making it a non-critical >> part of your web site), > > Except the marketing people who decide on the requirements will never, > ever settle for what you can do with plain HTML/CSS. > > In my experience, HTML/CSS makes a pretty awful GUI for a non-trivial > application. With some Javascript and sweat, you can almost make it > to mediocre. That's why I said "may be". A pure CGI web site is pretty annoying unless it's really brilliantly done. However, I prefer to see JS restricted to actual interaction, instead of making it critical to the basic layout. A lot of web sites these days load nothing but a script that goes and loads everything else, while you gaze at a splash screen. IMO that's unideal. However, even that is probably a losing battle. :( ChrisA From larry.martell at gmail.com Mon Feb 29 11:21:59 2016 From: larry.martell at gmail.com (Larry Martell) Date: Mon, 29 Feb 2016 11:21:59 -0500 Subject: [Off-topic] Requests author discusses MentalHealthError exception In-Reply-To: <56d16e6a$0$1593$c3e8da3$5496439d@news.astraweb.com> References: <56d16e6a$0$1593$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Sat, Feb 27, 2016 at 4:37 AM, Steven D'Aprano wrote: > The author of Requests, Kenneth Reitz, discusses his recent recovery from a > MentalHealthError exception. > > http://www.kennethreitz.org/essays/mentalhealtherror-an-exception-occurred > > Although the connection to Python is only quite slim, I found it fascinating > to read. I found this to be a very sad story. Sure, he had some issues, but I don't think they needed to drug him, and take all that he had learned away from him and turn him back into what he was before. From ian.g.kelly at gmail.com Mon Feb 29 11:29:21 2016 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Mon, 29 Feb 2016 09:29:21 -0700 Subject: common mistakes in this simple program In-Reply-To: References: Message-ID: On Mon, Feb 29, 2016 at 8:18 AM, Ganesh Pal wrote: > Iam on python 2.6 Python 2.6 has been unsupported since October 2013. Among other things, that means it is no longer receiving security updates like more recent versions. Unless you have an extremely strong reason for wanting to stay to Python 2.6, you should update your Python version. > 1. usage of try- expect try-except in every single function is a code smell. You should only be using it where you're actually going to handle the exception. If you catch an exception just to log it, you generally should also reraise it so that something further up the call chain has the opportunity to handle it. > 2. Return of True/ False If you're returning True to mean "This completed without exception" and False to mean "This raised an exception", then the more Pythonic thing to do would just be to let the exception propagate, giving the caller the opportunity to catch, inspect, and handle the exception. > """ > """ An empty docstring is pointless. > import os > import shlex > import subprocess > import sys > import time > import logging > import run > import pdb Most of these are unused. Only import modules that your code is actually using. > def run_cmd_and_verify(cmd, timeout=1000): > try: > pdb.set_trace() > out, err, ret = run(cmd, timeout=timeout) What is "run"? It's imported like a module above, but here you're using it like a function. > assert ret ==0,"ERROR (ret %d): " \ > " \nout: %s\nerr: %s\n" % (ret, out, err) > except Exception as e: > print("Failed to run %s got %s" % (cmd, e)) > return False > return True > > def prep_host(): > """ > Prepare clustering > """ > for cmd in ["ls -al", > "touch /tmp/file1", > "mkdir /tmp/dir1"]: > try: > if not run_cmd_and_verify(cmd, timeout=3600): > return False > except: What exceptions are you expecting this to catch? run_cmd_and_verify already catches any expected exceptions that it raises. Also, you should almost never use a bare except like this. Use "except Exception" instead. A bare except will catch things like KeyboardInterrupt and SystemExit that should not be caught. > print("Error: While preparing cluster !!!") > return False > print("Preparing Cluster.....Done !!!") > return True > > > def main(): > functions = [prep_host] > for func in functions: > try: > func() > except Exception as e: As above, what exceptions are you expecting here? > print(e) > return False > if __name__ == '__main__': > main() From ian.g.kelly at gmail.com Mon Feb 29 11:33:48 2016 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Mon, 29 Feb 2016 09:33:48 -0700 Subject: Dynamic object attribute creation In-Reply-To: <1456761964.822300.535132330.33A18241@webmail.messagingengine.com> References: <56d46597$0$22756$426a74cc@news.free.fr> <1456761964.822300.535132330.33A18241@webmail.messagingengine.com> Message-ID: On Mon, Feb 29, 2016 at 9:06 AM, Random832 wrote: > On Mon, Feb 29, 2016, at 10:36, ast wrote: >> but why doesn't it work with built-in classes int, float, list.... ? >> >> L = [1, 8, 0] >> L.test = 'its a list !' >> >> (however lists are mutable, int, float ... are not) > > Because those classes do not have attribute dictionaries, in order to > save space. > > You can make a class without an attribute dictionary, by using > __slots__. You can also make a list that does have an attribute dictionary by subclassing it. >>> class MyList(list): pass ... >>> obj = MyList() >>> obj.test = 'foo' From ian.g.kelly at gmail.com Mon Feb 29 11:47:49 2016 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Mon, 29 Feb 2016 09:47:49 -0700 Subject: [Off-topic] Requests author discusses MentalHealthError exception In-Reply-To: References: <56d16e6a$0$1593$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Mon, Feb 29, 2016 at 9:21 AM, Larry Martell wrote: > On Sat, Feb 27, 2016 at 4:37 AM, Steven D'Aprano wrote: >> The author of Requests, Kenneth Reitz, discusses his recent recovery from a >> MentalHealthError exception. >> >> http://www.kennethreitz.org/essays/mentalhealtherror-an-exception-occurred >> >> Although the connection to Python is only quite slim, I found it fascinating >> to read. > > I found this to be a very sad story. Sure, he had some issues, but I > don't think they needed to drug him, and take all that he had learned > away from him and turn him back into what he was before. "All that he had learned" meaning his delusions and psychoses? From ben+python at benfinney.id.au Mon Feb 29 11:56:08 2016 From: ben+python at benfinney.id.au (Ben Finney) Date: Tue, 01 Mar 2016 03:56:08 +1100 Subject: [Off-topic] Requests author discusses MentalHealthError exception References: <56d16e6a$0$1593$c3e8da3$5496439d@news.astraweb.com> Message-ID: <854mcrxxp3.fsf@benfinney.id.au> Ian Kelly writes: > On Mon, Feb 29, 2016 at 9:21 AM, Larry Martell wrote: > > I found this to be a very sad story. Sure, he had some issues, but I > > don't think they needed to drug him, and take all that he had > > learned away from him and turn him back into what he was before. > > "All that he had learned" meaning his delusions and psychoses? Indeed. If a revelation is unconnected to reality, it's misleading to say that one has ?learned? it. Also, I don't see how anyone ?took away? those experiences. He wrote an article describing them and clearly still retains those experiences as memories to recall as he wishes. When someone describes the ill effects their mental illness produced, I find it rather condescending for an observer to express regret that the person no longer experiences those ill effects. -- \ ?We have clumsy, sputtering, inefficient brains?. It is a | `\ *struggle* to be rational and objective, and failures are not | _o__) evidence for an alternative reality.? ?Paul Z. Myers, 2010-10-14 | Ben Finney From rustompmody at gmail.com Mon Feb 29 12:08:05 2016 From: rustompmody at gmail.com (Rustom Mody) Date: Mon, 29 Feb 2016 09:08:05 -0800 (PST) Subject: [Off-topic] Requests author discusses MentalHealthError exception In-Reply-To: References: <56d16e6a$0$1593$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Monday, February 29, 2016 at 10:26:32 PM UTC+5:30, Ben Finney wrote: > Ian Kelly writes: > > > On Mon, Feb 29, 2016 at 9:21 AM, Larry Martell wrote: > > > I found this to be a very sad story. Sure, he had some issues, but I > > > don't think they needed to drug him, and take all that he had > > > learned away from him and turn him back into what he was before. > > > > "All that he had learned" meaning his delusions and psychoses? > > Indeed. If a revelation is unconnected to reality, it's misleading to > say that one has "learned" it. And who is the last arbiter on that 'reality'? Niels Bohr random quote: If anybody says he can think about quantum physics without getting giddy, that only shows he has not understood the first thing about them. Some more such: http://www.brainyquote.com/quotes/authors/n/niels_bohr.html From pkpearson at nowhere.invalid Mon Feb 29 12:08:32 2016 From: pkpearson at nowhere.invalid (Peter Pearson) Date: 29 Feb 2016 17:08:32 GMT Subject: Sending an email with a binary attachment References: Message-ID: On Mon, 29 Feb 2016 02:10:00 -0600, Anthony Papillion wrote: [snip] > > http://pastebin.com/sryj98wW To improve odds of your getting a response, let's get the code into this thread. Here it is, minus the #! preamble: from email.mime.text import MIMEText from email.mime.application import MIMEApplication from email.mime.multipart import MIMEMultipart from smtplib import SMTP import sys FILE_TO_SEND = "/home/anthony/Desktop/passwords.kdb" message = MIMEMultipart() message['Subject'] = 'Password database update' message['From'] = 'Database Update Script For some reason though, sending mail is failing every time. I've made > sure that the password is correct (which seems to be the most usual > error). > > Still, I just can't get it to work. Can someone take a look at this > code and give me some advice? What exactly do you mean by "failing"? If there's an error message, show it. If an exception is being generated, at least print some information about the exception. -- To email me, substitute nowhere->runbox, invalid->com. From rosuav at gmail.com Mon Feb 29 12:13:59 2016 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 1 Mar 2016 04:13:59 +1100 Subject: Sending an email with a binary attachment In-Reply-To: References: Message-ID: On Tue, Mar 1, 2016 at 4:08 AM, Peter Pearson wrote: > try: > smtp.sendmail(message['From'], > message['To'], > message.as_string()) > except: > print "Message sending has failed" > sys.exit(1) > print "Message sending was successful" > sys.exit(0) > This is the problem, right here. Replace this code with: smtp.sendmail(message['From'], message['To'], message.as_string()) ChrisA From subhabangalore at gmail.com Mon Feb 29 12:23:42 2016 From: subhabangalore at gmail.com (subhabangalore at gmail.com) Date: Mon, 29 Feb 2016 09:23:42 -0800 (PST) Subject: Error in Tree Structure In-Reply-To: <573e58fa-36af-4e32-9460-5335d889df26@googlegroups.com> References: <33c93316-0ae3-4b8f-b2f6-29f76c8b3f32@googlegroups.com> <573e58fa-36af-4e32-9460-5335d889df26@googlegroups.com> Message-ID: <07d3149e-9867-478a-bc1c-65263ad50659@googlegroups.com> On Saturday, February 27, 2016 at 9:43:56 PM UTC+5:30, Rustom Mody wrote: > On Saturday, February 27, 2016 at 2:47:53 PM UTC+5:30, subhaba... at gmail.com wrote: > > I was trying to implement the code, > > > > import nltk > > import nltk.tag, nltk.chunk, itertools > > def ieertree2conlltags(tree, tag=nltk.tag.pos_tag): > > words, ents = zip(*tree.pos()) > > iobs = [] > > prev = None > > for ent in ents: > > if ent == tree.node: > > iobs.append('O') > > prev = None > > elif prev == ent: > > iobs.append('I-%s' % ent) > > else: > > iobs.append('B-%s' % ent) > > prev = ent > > words, tags = zip(*tag(words)) > > return itertools.izip(words, tags, iobs) > > > > def ieer_chunked_sents(tag=nltk.tag.pos_tag): > > for doc in ieer.parsed_docs(): > > tagged = ieertree2conlltags(doc.text, tag) > > yield nltk.chunk.conlltags2tree(tagged) > > > > > > from chunkers import ieer_chunked_sents, ClassifierChunker > > from nltk.corpus import treebank_chunk > > ieer_chunks = list(ieer_chunked_sents()) > > chunker = ClassifierChunker(ieer_chunks[:80]) > > print chunker.parse(treebank_chunk.tagged_sents()[0]) > > score = chunker.evaluate(ieer_chunks[80:]) > > print score.accuracy() > > > > It is running fine. > > But as I am trying to rewrite the code as, > > chunker = ClassifierChunker(list1), > > where list1 is same value as, > > ieer_chunks[:80] > > only I am pasting the value as > > [Tree('S', [Tree('LOCATION', [(u'NAIROBI', 'NNP')]), (u',', ','), Tree('LOCATION', [(u'Kenya', 'NNP')]), (u'(', '('), Tree('ORGANIZATION', [(u'AP', 'NNP')]), (u')', ')'), (u'_', 'NNP'), Tree('CARDINAL', [(u'Thousands', 'NNP')]), (u'of', 'IN'), (u'laborers,', 'JJ'), (u'students', 'NNS'), (u'and', 'CC'), (u'opposition', 'NN'), (u'politicians', 'NNS'), (u'on', 'IN'), Tree('DATE', [(u'Saturday', 'NNP')]), (u'protested', 'VBD'), (u'tax', 'NN'), (u'hikes', 'NNS'), (u'imposed', 'VBN'), (u'by', 'IN'), (u'their', 'PRP$'), (u'cash-strapped', 'JJ'), (u'government,', 'NN'), (u'which', 'WDT'), (u'they', 'PRP'), (u'accused', 'VBD'), (u'of', 'IN'), (u'failing', 'VBG'), (u'to', 'TO'), (u'provide', 'VB'), (u'basic', 'JJ'), (u'services.', 'NN'),....(u'(cm-kjd)', 'NN')])] > > the value of whole list directly I am getting syntax error. > > Dunno how literally you intend this but there is a "...." near the end > of the list. Intended? It is intended. As actual list was large. And most likely I could solve the problem, with from nltk.tree import Tree I missed in my code. Thank you for your kind time and discussion. Regards, RP From ganesh1pal at gmail.com Mon Feb 29 12:26:43 2016 From: ganesh1pal at gmail.com (Ganesh Pal) Date: Mon, 29 Feb 2016 22:56:43 +0530 Subject: common mistakes in this simple program In-Reply-To: References: Message-ID: On Mon, Feb 29, 2016 at 9:59 PM, Ian Kelly wrote: > On Mon, Feb 29, 2016 at 8:18 AM, Ganesh Pal wrote: >> Iam on python 2.6 >> 1. usage of try- expect > > try-except in every single function is a code smell. You should only > be using it where you're actually going to handle the exception. If > you catch an exception just to log it, you generally should also > reraise it so that something further up the call chain has the > opportunity to handle it. How do we reraise the exception in python , I have used raise not sure how to reraise the exception > >> def run_cmd_and_verify(cmd, timeout=1000): >> try: >> pdb.set_trace() >> out, err, ret = run(cmd, timeout=timeout) > > What is "run"? It's imported like a module above, but here you're > using it like a function. Sorry run is a function which was imported from a library , the function had to be # from utility import run , >> assert ret ==0,"ERROR (ret %d): " \ >> " \nout: %s\nerr: %s\n" % (ret, out, err) >> except Exception as e: >> print("Failed to run %s got %s" % (cmd, e)) >> return False >> return True >> >> def prep_host(): >> """ >> Prepare clustering >> """ >> for cmd in ["ls -al", >> "touch /tmp/file1", >> "mkdir /tmp/dir1"]: >> try: >> if not run_cmd_and_verify(cmd, timeout=3600): >> return False >> except: > > What exceptions are you expecting this to catch? run_cmd_and_verify > already catches any expected exceptions that it raises. This is a wrapper for Popen it runs the command and returns stdout ,stderror and returncode .The only exception it can return is a time out exception Here is the modified buggy code , Can I have Try and except with Pass , how do I modify the try and expect in the pre-host ? #!/usr/bin/env python """ bugging code """ import logging from utility import run def run_cmd_and_verify(cmd, timeout=1000): try: out, err, ret = run(cmd, timeout=timeout) assert ret ==0,"ERROR (ret %d): " \ " \nout: %s\nerr: %s\n" % (ret, out, err) except Exception as e: logging.error("Failed to run %s got %s" % (cmd, e)) return False return True def prep_host(): """ Prepare clustering """ for cmd in ["ls -al", "touch /tmp/file1", "mkdir /tmp/dir1"]: try: if not run_cmd_and_verify(cmd, timeout=3600): logging.info("Preparing cluster failed ...") return False except: pass logging.info("Preparing Cluster.....Done !!!") return True def main(): functions = [prep_host] for func in functions: try: func() except Exception as e: logging.info(e) return False if __name__ == '__main__': main() Regards, Gpal From ben+python at benfinney.id.au Mon Feb 29 12:33:50 2016 From: ben+python at benfinney.id.au (Ben Finney) Date: Tue, 01 Mar 2016 04:33:50 +1100 Subject: [Off-topic] Requests author discusses MentalHealthError exception References: <56d16e6a$0$1593$c3e8da3$5496439d@news.astraweb.com> Message-ID: <85ziujwhdt.fsf@benfinney.id.au> Rustom Mody writes: > On Monday, February 29, 2016 at 10:26:32 PM UTC+5:30, Ben Finney wrote: > > Ian Kelly writes: > > > > > On Mon, Feb 29, 2016 at 9:21 AM, Larry Martell wrote: > > > > I found this to be a very sad story. Sure, he had some issues, but I > > > > don't think they needed to drug him, and take all that he had > > > > learned away from him and turn him back into what he was before. > > > > > > "All that he had learned" meaning his delusions and psychoses? > > > > Indeed. If a revelation is unconnected to reality, it's misleading to > > say that one has "learned" it. > > And who is the last arbiter on that 'reality'? A nice academic question. Not appropriate when we're discussing a real person's psychotic episodes. Neither you nor I are experts in whether an experience is psychosis, and the person already has an expert diagnosis. So please don't de-rail into the weeds of second-guessing an expert in mental health. Unless you know the person's mental health better than their doctor (and you do not), don't second-guess the diagnosis they received. -- \ ?When [science] permits us to see the far side of some new | `\ horizon, we remember those who prepared the way ? seeing for | _o__) them also.? ?Carl Sagan, _Cosmos_, 1980 | Ben Finney From ganesh1pal at gmail.com Mon Feb 29 12:34:44 2016 From: ganesh1pal at gmail.com (Ganesh Pal) Date: Mon, 29 Feb 2016 23:04:44 +0530 Subject: usage of try except for review. In-Reply-To: References: Message-ID: On Mon, Feb 29, 2016 at 10:10 PM, Dennis Lee Bieber wrote: > Ask yourself: Will my program still work if I remove all the assert > statements. If the answer is "No", then you should not be using an assert. You meant if the answer is "NO" then I should be using asset ? > Can your "run()" raise an exception? Since you never show it to us we > can't tell. And if it can, which ones? However, checking a return code is > not an exceptional condition -- that's expected logic. > The run api is like a wrapper to sub-process module it does a Popen run's the command and returns stdout ,err and ret. The only exception it can raise is a timeout error. >> try: >> if not run_cmd_and_verify(cmd, timeout=3600): > > Since your version of rcav() always trapped exceptions internally, this > call will never raise an exception, so using a try: block is meaningless >> return False > > And your conditional basically comes down to: > > if False: > return False: > > >> except: >> logging.error("Some meaningful message") >> logging.info("Setup and Creation ....Done !!!") > > But you fall off and return Null on success... Iam using the try expect block to loop over the list do I have alternatives but would prefer the for loop because in the actual program there are 100 of command and putting them in a list is quite easy and running over with the same operation saves many lines of code , Cam I modify it something like a try except with pass in the except ? or any suggestions for cmd in ["mount /nfs_mount1", "mount /cifs_mount1"]: try: if not run_cmd_and_verify(cmd, timeout=3600): return False except: pass logging.info("Setup and Creation ....Done !!!") Regards, Ganesh From subhabangalore at gmail.com Mon Feb 29 12:34:45 2016 From: subhabangalore at gmail.com (subhabangalore at gmail.com) Date: Mon, 29 Feb 2016 09:34:45 -0800 (PST) Subject: Problem in creating list of lists Message-ID: I have few sentences, like, the film was nice. leonardo is great. it was academy award. Now I want them to be tagged with some standards which may look like, the DT film NN was AV nice ADJ leonardo NN is AV great ADJ it PRP was AV academy NN award NN I could do it but my goal is to see it as, [[('the','DT'),('film', 'NN'),('was','AV'),('nice','ADJ')],[('leonardo','NN'),('is','AV'),('great','ADJ')],[('it','PRP'), ('was','AV'),('academy','NN'),('award','NN')]] that is a list of lists where in each list there is a set of tuples. I could solve each one like I am getting one list with tuples, but not all within one. As it is PoS Tagging so I am not being able to use zip. If any one may please suggest. If any one may kindly suggest a solution. Thanks in advance. From rosuav at gmail.com Mon Feb 29 12:38:20 2016 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 1 Mar 2016 04:38:20 +1100 Subject: usage of try except for review. In-Reply-To: References: Message-ID: On Tue, Mar 1, 2016 at 4:34 AM, Ganesh Pal wrote: > On Mon, Feb 29, 2016 at 10:10 PM, Dennis Lee Bieber > wrote: > >> Ask yourself: Will my program still work if I remove all the assert >> statements. If the answer is "No", then you should not be using an assert. > > You meant if the answer is "NO" then I should be using asset ? No, Dennis was correct. You should assume that "assert" can potentially be replaced with "pass" and your program will continue to work. ChrisA From ian.g.kelly at gmail.com Mon Feb 29 12:45:41 2016 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Mon, 29 Feb 2016 10:45:41 -0700 Subject: common mistakes in this simple program In-Reply-To: References: Message-ID: On Mon, Feb 29, 2016 at 10:26 AM, Ganesh Pal wrote: > On Mon, Feb 29, 2016 at 9:59 PM, Ian Kelly wrote: >> On Mon, Feb 29, 2016 at 8:18 AM, Ganesh Pal wrote: >>> Iam on python 2.6 > >>> 1. usage of try- expect >> >> try-except in every single function is a code smell. You should only >> be using it where you're actually going to handle the exception. If >> you catch an exception just to log it, you generally should also >> reraise it so that something further up the call chain has the >> opportunity to handle it. > > How do we reraise the exception in python , I have used raise not > sure how to reraise the exception raise with no arguments will reraise the exception currently being handled. except Exception: logging.error("something went wrong") raise >>> assert ret ==0,"ERROR (ret %d): " \ >>> " \nout: %s\nerr: %s\n" % (ret, out, err) >>> except Exception as e: >>> print("Failed to run %s got %s" % (cmd, e)) >>> return False >>> return True >>> >>> def prep_host(): >>> """ >>> Prepare clustering >>> """ >>> for cmd in ["ls -al", >>> "touch /tmp/file1", >>> "mkdir /tmp/dir1"]: >>> try: >>> if not run_cmd_and_verify(cmd, timeout=3600): >>> return False >>> except: >> >> What exceptions are you expecting this to catch? run_cmd_and_verify >> already catches any expected exceptions that it raises. > > This is a wrapper for Popen it runs the command and returns stdout > ,stderror and returncode .The only exception it can return is a time > out exception But that exception is already caught by the run_cmd_and_verify function, so what exception are you expecting to be caught *here*? > Here is the modified buggy code , Can I have Try and except with Pass > , how do I modify the try and expect in the pre-host ? You should virtually never just pass in an exception handler. Either handle the exception, or log it and reraise it. If you're going to do neither of those things, then don't use a try-except at all. From rustompmody at gmail.com Mon Feb 29 12:54:31 2016 From: rustompmody at gmail.com (Rustom Mody) Date: Mon, 29 Feb 2016 09:54:31 -0800 (PST) Subject: [Off-topic] Requests author discusses MentalHealthError exception In-Reply-To: References: <56d16e6a$0$1593$c3e8da3$5496439d@news.astraweb.com> Message-ID: <3a45b886-a9fb-43cb-aff3-a507811486ce@googlegroups.com> On Monday, February 29, 2016 at 11:04:20 PM UTC+5:30, Ben Finney wrote: > Rustom Mody writes: > > > On Monday, February 29, 2016 at 10:26:32 PM UTC+5:30, Ben Finney wrote: > > > Ian Kelly writes: > > > > > > > On Mon, Feb 29, 2016 at 9:21 AM, Larry Martell wrote: > > > > > I found this to be a very sad story. Sure, he had some issues, but I > > > > > don't think they needed to drug him, and take all that he had > > > > > learned away from him and turn him back into what he was before. > > > > > > > > "All that he had learned" meaning his delusions and psychoses? > > > > > > Indeed. If a revelation is unconnected to reality, it's misleading to > > > say that one has "learned" it. > > > > And who is the last arbiter on that 'reality'? > > A nice academic question. Not appropriate when we're discussing a real > person's psychotic episodes. Neither you nor I are experts in whether an > experience is psychosis, and the person already has an expert diagnosis. > > So please don't de-rail into the weeds of second-guessing an expert in > mental health. Unless you know the person's mental health better than > their doctor (and you do not), don't second-guess the diagnosis they > received. No need to second guess Article's last line: Avoid falling in love with hyper-intelligent pan-dimensional beings. IOW Larry makes an important point The fact that he claims to be better now than in that period is more significant than the medical diagnoses It neglects the possibility that these are not the only two possibilities From ganesh1pal at gmail.com Mon Feb 29 12:57:18 2016 From: ganesh1pal at gmail.com (Ganesh Pal) Date: Mon, 29 Feb 2016 23:27:18 +0530 Subject: usage of try except for review. In-Reply-To: References: Message-ID: > No, Dennis was correct. You should assume that "assert" can > potentially be replaced with "pass" and your program will continue to > work. Thanks Chris for clarifying Dennis point of view , >> try: >> if not run_cmd_and_verify(cmd, timeout=3600): > > Since your version of rcav() always trapped exceptions internally, this > call will never raise an exception, so using a try: block is meaningless >> return False > > And your conditional basically comes down to: > > if False: > return False: What would be the alternative for try expect in the FOR loop section of the program , will this work fine i.e expect with pass , or any better alternative for this ? I have tried down the code to #!/usr/bin/env python """ """ import os import shlex import subprocess import sys import time import logging from utils import run def run_cmd_and_verify(cmd, timeout=1000): try: out, err, ret = run(cmd, timeout=timeout) assert ret ==0,"ERROR (ret %d): " \ " \nout: %s\nerr: %s\n" % (ret, out, err) except Exception as e: logging.error("Failed to run %s got %s" % (cmd, e)) return False return True def run_test(): """ Mount """ # For Loop section of the program for cmd in ["mount /nfs_mount1", "mount /cifs_mount1"]: try: if not run_cmd_and_verify(cmd, timeout=3600): logging.error("mount Failed") return False except: pass logging.info("Setup and Creation ....Done !!!") def main(): if not run_test(): sys.exit("Exiting Main") if __name__ == '__main__': main() Regards, Ganesh From ganesh1pal at gmail.com Mon Feb 29 12:59:04 2016 From: ganesh1pal at gmail.com (Ganesh Pal) Date: Mon, 29 Feb 2016 23:29:04 +0530 Subject: usage of try except for review. In-Reply-To: References: Message-ID: > I have tried down the code to Read "I have tried down the code to " as I have trimmed down the code as below Ganesh From invalid at invalid.invalid Mon Feb 29 13:17:20 2016 From: invalid at invalid.invalid (Grant Edwards) Date: Mon, 29 Feb 2016 18:17:20 +0000 (UTC) Subject: Everything good about Python except GUI IDE? References: <64a6599c-fae1-469d-bcee-875165b3cc7d@googlegroups.com> <56d294f8$0$1604$c3e8da3$5496439d@news.astraweb.com> <234a398e-1b0f-467b-a8cb-d7ca748f8062@googlegroups.com> <84922f24-3e00-4a23-b26d-5e6c0d8e7e04@googlegroups.com> <87y4a5c58i.fsf@elektro.pacujo.net> <87twksdg9c.fsf@elektro.pacujo.net> <87d1rgca58.fsf@elektro.pacujo.net> Message-ID: On 2016-02-29, Chris Angelico wrote: > On Tue, Mar 1, 2016 at 2:43 AM, Grant Edwards wrote: >> On 2016-02-29, Chris Angelico wrote: >> >>> Abjuring JS may be a virtue (or at least, making it a non-critical >>> part of your web site), >> >> Except the marketing people who decide on the requirements will never, >> ever settle for what you can do with plain HTML/CSS. >> >> In my experience, HTML/CSS makes a pretty awful GUI for a non-trivial >> application. With some Javascript and sweat, you can almost make it >> to mediocre. > > That's why I said "may be". A pure CGI web site is pretty annoying > unless it's really brilliantly done. However, I prefer to see JS > restricted to actual interaction, instead of making it critical to the > basic layout. Agreed. That's an excellent rule to follow. The introduction of the "flex" display type in CSS has (for me) completely eliminated the need for JS to concern itself with display and layout (other than the basic hide/show enable/disable of optional elements as part of an application's interaction logic). I don't know why it took so long for CSS to grok the basic idea that you almost always want to specify that some blocks should expand/contract to fill available space and some should remain at their "natural" size. GUI toolkits and markup languages like LaTeX seem to have understood that this was a very basic need for many decades, but there was never a clean, reliable way to do it CSS until recently. > A lot of web sites these days load nothing but a script > that goes and loads everything else, while you gaze at a splash > screen. IMO that's unideal. :) "Unideal" is too kind -- I would described it as something more like "the evil spawn of brain-dead incompetents". I often wonder what sequence of decisions/accidents get people to "solutions" like that. > However, even that is probably a losing battle. :( At least the era of using giant tables full of fragements of a single large gif image to do page-layout seems to have died a long-overdue death... -- Grant Edwards grant.b.edwards Yow! does your DRESSING at ROOM have enough ASPARAGUS? gmail.com From ganesh1pal at gmail.com Mon Feb 29 13:21:37 2016 From: ganesh1pal at gmail.com (Ganesh Pal) Date: Mon, 29 Feb 2016 23:51:37 +0530 Subject: common mistakes in this simple program In-Reply-To: References: Message-ID: >> How do we reraise the exception in python , I have used raise not >> sure how to reraise the exception > > raise with no arguments will reraise the exception currently being handled. > > except Exception: > logging.error("something went wrong") > raise Thanks Ian for taking time and looking into the code , o k raise keyword for raising exception is fine . >>>> assert ret ==0,"ERROR (ret %d): " \ >>>> " \nout: %s\nerr: %s\n" % (ret, out, err) >>>> except Exception as e: >>>> print("Failed to run %s got %s" % (cmd, e)) >>>> return False >>>> return True >>>> >>>> def prep_host(): >>>> """ >>>> Prepare clustering >>>> """ >>>> for cmd in ["ls -al", >>>> "touch /tmp/file1", >>>> "mkdir /tmp/dir1"]: >>>> try: >>>> if not run_cmd_and_verify(cmd, timeout=3600): >>>> return False >>>> except: >>> >>> What exceptions are you expecting this to catch? run_cmd_and_verify >>> already catches any expected exceptions that it raises. In my case the exception is nothing but the error example if we plan to run the command say #ifconfig -a and the command fails because of a type ( say u ran #igconfig -a). we will the output as # Failed to run igconfig -a got Error (ret=127) out : error: command not found: igconfig So the execption is the error i.e Error (ret=127) out : error: command not found: igconfig, Iam fine with this behaviour. > > But that exception is already caught by the run_cmd_and_verify > function, so what exception are you expecting to be caught *here*? I wanted to run the command in a loop and have a fxn for the pattern that repeats in this case the function is run_cmd_and_verify , the only known way to me was using try with expect I thought I will use try and have pass in except which you don't recommend for cmd in ["ls -al", "touch /tmp/file1", "mkdir /tmp/dir1"]: try: if not run_cmd_and_verify(cmd, timeout=3600): print "running command failed " return False except: pass > You should virtually never just pass in an exception handler. Either > handle the exception, or log it and reraise it. If you're going to do > neither of those things, then don't use a try-except at all. What alternative do I have other than try-expect ? can try - else be used for my case? Regards, GPal From invalid at invalid.invalid Mon Feb 29 13:22:29 2016 From: invalid at invalid.invalid (Grant Edwards) Date: Mon, 29 Feb 2016 18:22:29 +0000 (UTC) Subject: Sending an email with a binary attachment References: Message-ID: On 2016-02-29, Peter Pearson wrote: > On Mon, 29 Feb 2016 02:10:00 -0600, Anthony Papillion wrote: > >> For some reason though, sending mail is failing every time. I've made >> sure that the password is correct (which seems to be the most usual >> error). >> >> Still, I just can't get it to work. Can someone take a look at this >> code and give me some advice? > > What exactly do you mean by "failing"? If there's an error message, > show it. If an exception is being generated, at least print some > information about the exception. If you aren't getting any messages, then tell the smtp object you want some: >From TFM (https://docs.python.org/3/library/smtplib.html#smtp-objects): An SMTP instance has the following methods: SMTP.set_debuglevel(level) Set the debug output level. A value of 1 or True for level results in debug messages for connection and for all messages sent to and received from the server. A value of 2 for level results in these messages being timestamped. If all else fails, fire up wireshark and watch the conversation between the SMTP object and the server. -- Grant Edwards grant.b.edwards Yow! Can you MAIL a BEAN at CAKE? gmail.com From rosuav at gmail.com Mon Feb 29 13:31:22 2016 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 1 Mar 2016 05:31:22 +1100 Subject: Everything good about Python except GUI IDE? In-Reply-To: References: <64a6599c-fae1-469d-bcee-875165b3cc7d@googlegroups.com> <56d294f8$0$1604$c3e8da3$5496439d@news.astraweb.com> <234a398e-1b0f-467b-a8cb-d7ca748f8062@googlegroups.com> <84922f24-3e00-4a23-b26d-5e6c0d8e7e04@googlegroups.com> <87y4a5c58i.fsf@elektro.pacujo.net> <87twksdg9c.fsf@elektro.pacujo.net> <87d1rgca58.fsf@elektro.pacujo.net> Message-ID: On Tue, Mar 1, 2016 at 5:17 AM, Grant Edwards wrote: >> A lot of web sites these days load nothing but a script >> that goes and loads everything else, while you gaze at a splash >> screen. IMO that's unideal. > > :) > > "Unideal" is too kind -- I would described it as something more like > "the evil spawn of brain-dead incompetents". I often wonder what > sequence of decisions/accidents get people to "solutions" like that. I do understand where the thinking comes from, though. 1) Pages should be more responsive than the 1990s-style CGI model, where every action is a link or button click that completely replaces the current page. 2) Many actions therefore must be implemented using JavaScript, an AJAX call (if necessary), and DOM manipulation to change just the parts of the page that need to be changed. 3) This means that the entire content of the page must be under JS control, and the entire "user-accessible" part of the back end should be returning JSON, since that's what #2 requires. 4) If everything has to be under JS control, why not just send out a stub to start with, and begin everything with an AJAX call? That way, we don't need to maintain two of everything! The logic is reasonable. Deduplication is valuable. But the result is a web site that has a stubby splash screen until the back end responds, and that's why I call it merely "unideal" rather than "evil spawn &c". >> However, even that is probably a losing battle. :( > > At least the era of using giant tables full of fragements of a single > large gif image to do page-layout seems to have died a long-overdue > death... Oh yes, I do not miss those days! And it was compounded by browsers that, when they started receiving a tag, drew nothing whatsoever until they received the
and could lay the whole thing out. Add to that slow internet connections, so it would likely be some seconds or even a minute or more before the arrives, and you have a perfect recipe for frustration. But no web site these days would ever make you wait for its contents. 'Course not. ChrisA From rosuav at gmail.com Mon Feb 29 13:34:41 2016 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 1 Mar 2016 05:34:41 +1100 Subject: common mistakes in this simple program In-Reply-To: References: Message-ID: On Tue, Mar 1, 2016 at 5:21 AM, Ganesh Pal wrote: > > In my case the exception is nothing but the error example if we plan > to run the command say #ifconfig -a and the command fails because of > a type ( say u ran #igconfig -a). > > we will the output as > > # Failed to run igconfig -a got Error (ret=127) > out : > error: command not found: igconfig > > So the execption is the error i.e Error (ret=127) out : error: command > not found: igconfig, Iam fine with this behaviour. > You're falling into the trap of assuming that the only exception you can ever get is the one that you're planning for, and then handling ALL exceptions as though they were that one. Instead catch ONLY the exception that you're expecting to see, and ignore everything else. Do not use a bare "except:" clause, nor even "except Exception:", for this. You will appreciate it later on. ChrisA From eshaman at list.ru Mon Feb 29 14:19:50 2016 From: eshaman at list.ru (Shamanov) Date: Mon, 29 Feb 2016 22:19:50 +0300 Subject: No subject Message-ID: I can't work with Python From sohcahtoa82 at gmail.com Mon Feb 29 14:23:22 2016 From: sohcahtoa82 at gmail.com (sohcahtoa82 at gmail.com) Date: Mon, 29 Feb 2016 11:23:22 -0800 (PST) Subject: common mistakes in this simple program In-Reply-To: References: Message-ID: <45787234-afd1-4e45-a34d-3bf4bdc241f1@googlegroups.com> On Monday, February 29, 2016 at 10:21:57 AM UTC-8, Ganesh Pal wrote: > >> How do we reraise the exception in python , I have used raise not > >> sure how to reraise the exception > > > > raise with no arguments will reraise the exception currently being handled. > > > > except Exception: > > logging.error("something went wrong") > > raise > > Thanks Ian for taking time and looking into the code , o k raise > keyword for raising exception is fine . > > >>>> assert ret ==0,"ERROR (ret %d): " \ > >>>> " \nout: %s\nerr: %s\n" % (ret, out, err) > >>>> except Exception as e: > >>>> print("Failed to run %s got %s" % (cmd, e)) > >>>> return False > >>>> return True > >>>> > >>>> def prep_host(): > >>>> """ > >>>> Prepare clustering > >>>> """ > >>>> for cmd in ["ls -al", > >>>> "touch /tmp/file1", > >>>> "mkdir /tmp/dir1"]: > >>>> try: > >>>> if not run_cmd_and_verify(cmd, timeout=3600): > >>>> return False > >>>> except: > >>> > >>> What exceptions are you expecting this to catch? run_cmd_and_verify > >>> already catches any expected exceptions that it raises. > > In my case the exception is nothing but the error example if we plan > to run the command say #ifconfig -a and the command fails because of > a type ( say u ran #igconfig -a). > > we will the output as > > # Failed to run igconfig -a got Error (ret=127) > out : > error: command not found: igconfig > > So the execption is the error i.e Error (ret=127) out : error: command > not found: igconfig, Iam fine with this behaviour. > > > > > > But that exception is already caught by the run_cmd_and_verify > > function, so what exception are you expecting to be caught *here*? > > I wanted to run the command in a loop and have a fxn for the pattern > that repeats in this case the function is run_cmd_and_verify , the > only known way to me was using try with expect > > I thought I will use try and have pass in except which you don't recommend > > for cmd in ["ls -al", > "touch /tmp/file1", > "mkdir /tmp/dir1"]: > try: > if not run_cmd_and_verify(cmd, timeout=3600): > print "running command failed " > return False > except: > pass > > > You should virtually never just pass in an exception handler. Either > > handle the exception, or log it and reraise it. If you're going to do > > neither of those things, then don't use a try-except at all. > > What alternative do I have other than try-expect ? can try - else be > used for my case? > > Regards, > GPal Every time you say "try-expect", my head wants to explode. It is called a "try-except" block, because you're using the key words "try" and "except" when you make one. From ganesh1pal at gmail.com Mon Feb 29 14:49:30 2016 From: ganesh1pal at gmail.com (Ganesh Pal) Date: Tue, 1 Mar 2016 01:19:30 +0530 Subject: common mistakes in this simple program In-Reply-To: References: Message-ID: On Mar 1, 2016 12:06 AM, "Chris Angelico" wrote > > You're falling into the trap of assuming that the only exception you > can ever get is the one that you're planning for, and then handling. Ok sure ! > ALL exceptions as though they were that one. Instead catch ONLY the > exception that you're expecting to see, and ignore everything else. Do > not use a bare "except:" clause, nor even "except Exception:", for > this. You will appreciate it later on. What option do I have now in so ensure that I loop over the For loop. Try except is ruled out ? From jonas.thornvall at gmail.com Mon Feb 29 15:07:25 2016 From: jonas.thornvall at gmail.com (jonas.thornvall at gmail.com) Date: Mon, 29 Feb 2016 12:07:25 -0800 (PST) Subject: Loop awareness Message-ID: <205018d7-f74d-4fbb-88e1-90c58200486a@googlegroups.com> This program creates a uniform linktree of x nodes, and it knows when it get stuck in a loop generating random values. Because the networks random generated, only a subset of the permutations will generate a uniform network most get stuck in loops generating random values. But the program keep track the exhausted links and knows when no uniform network possible. Is this related to the halting problem? http://jt.node365.se/mydebug1.html From joel.goldstick at gmail.com Mon Feb 29 15:26:16 2016 From: joel.goldstick at gmail.com (Joel Goldstick) Date: Mon, 29 Feb 2016 15:26:16 -0500 Subject: common mistakes in this simple program In-Reply-To: References: Message-ID: On Mon, Feb 29, 2016 at 2:49 PM, Ganesh Pal wrote: > On Mar 1, 2016 12:06 AM, "Chris Angelico" wrote > > > > You're falling into the trap of assuming that the only exception you > > can ever get is the one that you're planning for, and then handling. > > Ok sure ! > > > ALL exceptions as though they were that one. Instead catch ONLY the > > exception that you're expecting to see, and ignore everything else. Do > > not use a bare "except:" clause, nor even "except Exception:", for > > this. You will appreciate it later on. > > What option do I have now in so ensure that I loop over the For loop. Try > except is ruled out ? > Its not ruled out. It is pointless. Look at the documents. Here is a start: https://wiki.python.org/moin/HandlingExceptions > -- > https://mail.python.org/mailman/listinfo/python-list > -- Joel Goldstick http://joelgoldstick.com/ http://cc-baseballstats.info/ From marko at pacujo.net Mon Feb 29 15:29:54 2016 From: marko at pacujo.net (Marko Rauhamaa) Date: Mon, 29 Feb 2016 22:29:54 +0200 Subject: common mistakes in this simple program References: <45787234-afd1-4e45-a34d-3bf4bdc241f1@googlegroups.com> Message-ID: <878u23cla5.fsf@elektro.pacujo.net> sohcahtoa82 at gmail.com: > Every time you say "try-expect", my head wants to explode. > > It is called a "try-except" block, because you're using the key words > "try" and "except" when you make one. Ah, I remember a Python-based test system idea where the "except" keyword meant "expect": try: event() except user.CloseRequest(): ... except user.SendRequest(): ... except peer.DataIndication(): ... except Timeout(): ... etc. In the end, though, we came up with something better. Marko From ian.g.kelly at gmail.com Mon Feb 29 15:31:56 2016 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Mon, 29 Feb 2016 13:31:56 -0700 Subject: Loop awareness In-Reply-To: <205018d7-f74d-4fbb-88e1-90c58200486a@googlegroups.com> References: <205018d7-f74d-4fbb-88e1-90c58200486a@googlegroups.com> Message-ID: On Mon, Feb 29, 2016 at 1:07 PM, wrote: > This program creates a uniform linktree of x nodes, and it knows when it get stuck in a loop generating random values. > > Because the networks random generated, only a subset of the permutations will generate a uniform network most get stuck in loops generating random values. But the program keep track the exhausted links and knows when no uniform network possible. > > Is this related to the halting problem? > > http://jt.node365.se/mydebug1.html No, the halting problem is that you can't algorithmically determine whether an *arbitrary* program given *arbitrary* input will ever halt. When you narrow down the scope of the problem considerably, as you've done here, then it's no longer necessarily undecidable. From martin at linux-ip.net Mon Feb 29 16:11:16 2016 From: martin at linux-ip.net (Martin A. Brown) Date: Mon, 29 Feb 2016 13:11:16 -0800 Subject: common mistakes in this simple program In-Reply-To: References: Message-ID: Greetings Ganesh, >> You're falling into the trap of assuming that the only exception you >> can ever get is the one that you're planning for, and then handling. > >Ok sure ! This point is very important, so I'll reiterate it. I hope the poor horse lives. >> ALL exceptions as though they were that one. Instead catch ONLY the >> exception that you're expecting to see, and ignore everything else. Do >> not use a bare "except:" clause, nor even "except Exception:", for >> this. You will appreciate it later on. > >What option do I have now in so ensure that I loop over the For >loop. >Try except is ruled out ? No, no not at all! #1: Yes, you (probably) should be using the try-except construct. #2: No, you should not (ever) use a bare try-except construct. Please read below. I will take a stab at explaining the gaps of understanding you seem to have (others have tried already, but I'll try, as well). I am going to give you four different functions which demonstrate how to use exceptions. You may find it instructive to paste these functions into an interactive Python shell and try them out, as well. I will explain a few different cases, which I will show below, but I think you'll need to figure out how to apply this to your situation. Step 1: catch a specific Exception ---------------------------------- def catchValueError(val, default=0): '''somebody passed in a non-convertible type, return a default''' try: return int(val) except ValueError: return default Now, let's try passing a few values into that function: >>> catchValueError(0) 0 >>> catchValueError('-3') -3 >>> catchValueError('Catullus') 0 The last example above is the most interesting. We fed a string to the builtin function int() and it raised a ValueError. Our little try-except block, however, caught the ValueError and performed our desired action (in this case, returning a default value). Step 2: catch a specific Exception ---------------------------------- But, what about this: >>> catchValueError(dict()) Traceback (most recent call last): File "", line 1, in File "", line 4, in catchValueError TypeError: int() argument must be a string or a number, not 'dict' Now, you can see that there's a different problem. The exception is different if we feed different data to the function. So, let's try again, but maybe we catch the TypeError instead. def catchTypeError(val, default=0): '''somebody passed in a non-convertible type, return a default''' try: return int(val) except TypeError: return default This seems to work--but, now, if we try our other values (which worked before), they don't work now: >>> catchTypeError(dict()) 0 >>> catchTypeError(0) 0 >>> catchTypeError('-3') -3 >>> catchTypeError('Catullus') Traceback (most recent call last): File "", line 1, in File "", line 4, in catchTypeError ValueError: invalid literal for int() with base 10: 'Catullus' Step 3: catch several different classes of Exception ---------------------------------------------------- You can a single try-except block to catch multiple classes of specific Exception (and then perform exception handling). def catchTVError(val, default=0): '''somebody passed in a non-convertible type, return a default''' try: return int(val) except (ValueError, TypeError): return default In this little toy demonstration, you can see that this is probably the desired functionality. >>> catchTVError(dict()) 0 >>> catchTVError(0) 0 >>> catchTVError(-3) -3 >>> catchTVError('Catullus') 0 Step 4: catch many different classes of Exception ------------------------------------------------- Now, suppose you want to deal with each possible exception in a different manner....you can have different exception-handling behaviour for each class of exception. def altCatchTVError(val, default=42): '''somebody passed in a non-convertible type, return a default''' try: return int(val) except ValueError: return abs(default) except TypeError: return 0 - abs(default) As you can see, this offers a good deal more flexibility for handling specific exceptions that you might encounter. >>> altCatchTVError(0) 0 >>> altCatchTVError(22) 22 >>> altCatchTVError('17') 17 >>> altCatchTVError('-3') -3 >>> altCatchTVError('str') 42 >>> altCatchTVError(dict()) -42 Interlude and recommendation ---------------------------- As you can see, there are many possible Exceptions that can be raised when you are calling a simple builtin function, int(). Consider now what may happen when you call out to a different program; you indicated that your run() function calls out to subprocess.Popen(). There are many more possible errors that can occur, just a few that can come to my mind: * locating the program on disk * setting up the file descriptors for the child process * fork()ing and exec()ing the program * memory issues * filesystem disappears (network goes away or block device failure) Each one of these possible errors may translate to a different exception. You have been tempted to do: try: run() except: pass This means that, no matter what happens, you are going to try to keep continuing, even in the face of massive failure. To (#1) improve the safety of your program and the environments in which it operates, to (#2) improve your defensive programming posture and to (#3) avoid frustrating your own debugging at some point in the future, you would be well-advised to identify which specific exceptions you want to ignore. As you first try to improve the resilience of your program, you may not be certain which exceptions you want to catch and which represent a roadblock for your progam. This is something that usually comes with experience. To get that experience you can define your own exception (it'll never get raised unless you raise it, so do not worry). Then, create your try-except block to catch only that one. As you encounter other exception that you are certain you wish to handle, you can do something with them: class UnknownException(Exception): pass def prep_host(): """ Prepare clustering """ for cmd in ["ls -al", "touch /tmp/file1", "mkdir /tmp/dir1"]: try: if not run_cmd_and_verify(cmd, timeout=3600): logging.info("Preparing cluster failed ...") return False except (UnknownException,): pass logging.info("Preparing Cluster.....Done !!!") return True Now, as you develop your program and encounter new exceptions, you can add new except clauses to the above block with appropriate handling, or (re-)raising the caught exception. Comments on shelling out to other programs and using exceptions --------------------------------------------------------------- Exceptions are great for catching logic errors, type errors, filesystem errors and all manner of other errors within Python programs and runtime environments. You introduce a significant complexity the moment you fork a child (calling subprocess.Popen). It is good, though, that you are testing the return code of the cmd that you pass to the run() function. Final advice: ------------- Do not use a bare try-except. You will frustrate your own debugging and your software may end up trying to excecute code paths (or external programs, as you are doing right now) for which you were sanity checking. -Martin -- Martin A. Brown http://linux-ip.net/ From best_lay at yahoo.com Mon Feb 29 16:51:08 2016 From: best_lay at yahoo.com (Wildman) Date: Mon, 29 Feb 2016 15:51:08 -0600 Subject: Request More Help With XBM Image Message-ID: <8rKdnb6SKYLRIEnLnZ2dnUU7-f3NnZ2d@giganews.com> I want to take an image file, convert it to XBM format and display it. Thanks to Mr. Otten I can open and display the XBM image without any problems. The script first calls an external program for the image conversion then I can open and display it. Of course, I am left with the XBM file that needs to be deleted. It seemed to me to be a better approach to use stdout and pipe thereby eliminating the XBM file altogether. Here is code I have so far but I'm not sure what to do next... convert = "convert " + fileName + " -resize 48x48! -threshold 55% xbm:-" p = subprocess.Popen([convert], stdout=subprocess.PIPE, shell=True) xbmFile, err = p.communicate() The variable fileName contains the image file path and name. The variable convert contains the complete command. The last argument in the command tells the convert utility to covert to an XBM and to direct the output to stdout. After the above code runs xbmFile contains the actual image, which is plain text. (X BitMap (XBM) is a plain text binary image format.) My question is how do I take the xbmFile variable and convert it to an image object that can be displayed? The technique for displaying an image from a file does not work or at least I have not been able to get it to work. -- GNU/Linux user #557453 "Be at war with your vices, at peace with your neighbors, and let every new year find you a better man." -Benjamin Franklin From jonas.thornvall at gmail.com Mon Feb 29 16:53:46 2016 From: jonas.thornvall at gmail.com (jonas.thornvall at gmail.com) Date: Mon, 29 Feb 2016 13:53:46 -0800 (PST) Subject: Loop awareness In-Reply-To: References: <205018d7-f74d-4fbb-88e1-90c58200486a@googlegroups.com> Message-ID: Den m?ndag 29 februari 2016 kl. 21:32:51 UTC+1 skrev Ian: > On Mon, Feb 29, 2016 at 1:07 PM, wrote: > > This program creates a uniform linktree of x nodes, and it knows when it get stuck in a loop generating random values. > > > > Because the networks random generated, only a subset of the permutations will generate a uniform network most get stuck in loops generating random values. But the program keep track the exhausted links and knows when no uniform network possible. > > > > Is this related to the halting problem? > > > > http://jt.node365.se/mydebug1.html > > No, the halting problem is that you can't algorithmically determine > whether an *arbitrary* program given *arbitrary* input will ever halt. > When you narrow down the scope of the problem considerably, as you've > done here, then it's no longer necessarily undecidable. Here i told it to only report the unfiorm subset of working networks. http://jt.node365.se/mydebug2.html From larry.martell at gmail.com Mon Feb 29 17:36:18 2016 From: larry.martell at gmail.com (Larry Martell) Date: Mon, 29 Feb 2016 17:36:18 -0500 Subject: [Off-topic] Requests author discusses MentalHealthError exception In-Reply-To: References: <56d16e6a$0$1593$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Mon, Feb 29, 2016 at 11:47 AM, Ian Kelly wrote: > On Mon, Feb 29, 2016 at 9:21 AM, Larry Martell wrote: >> On Sat, Feb 27, 2016 at 4:37 AM, Steven D'Aprano wrote: >>> The author of Requests, Kenneth Reitz, discusses his recent recovery from a >>> MentalHealthError exception. >>> >>> http://www.kennethreitz.org/essays/mentalhealtherror-an-exception-occurred >>> >>> Although the connection to Python is only quite slim, I found it fascinating >>> to read. >> >> I found this to be a very sad story. Sure, he had some issues, but I >> don't think they needed to drug him, and take all that he had learned >> away from him and turn him back into what he was before. > > "All that he had learned" meaning his delusions and psychoses? How do you know they were delusions? I think his only problem was not sleeping, which can kill you. From larry.martell at gmail.com Mon Feb 29 17:38:15 2016 From: larry.martell at gmail.com (Larry Martell) Date: Mon, 29 Feb 2016 17:38:15 -0500 Subject: [Off-topic] Requests author discusses MentalHealthError exception In-Reply-To: <854mcrxxp3.fsf@benfinney.id.au> References: <56d16e6a$0$1593$c3e8da3$5496439d@news.astraweb.com> <854mcrxxp3.fsf@benfinney.id.au> Message-ID: On Mon, Feb 29, 2016 at 11:56 AM, Ben Finney wrote: > Ian Kelly writes: > >> On Mon, Feb 29, 2016 at 9:21 AM, Larry Martell wrote: >> > I found this to be a very sad story. Sure, he had some issues, but I >> > don't think they needed to drug him, and take all that he had >> > learned away from him and turn him back into what he was before. >> >> "All that he had learned" meaning his delusions and psychoses? > > Indeed. If a revelation is unconnected to reality, it's misleading to > say that one has ?learned? it. But what is reality? > Also, I don't see how anyone ?took away? those experiences. He wrote an > article describing them and clearly still retains those experiences as > memories to recall as he wishes. They made him believe they were not true. > When someone describes the ill effects their mental illness produced, I don't consider him to have been mentally ill. From larry.martell at gmail.com Mon Feb 29 17:40:20 2016 From: larry.martell at gmail.com (Larry Martell) Date: Mon, 29 Feb 2016 17:40:20 -0500 Subject: [Off-topic] Requests author discusses MentalHealthError exception In-Reply-To: <85ziujwhdt.fsf@benfinney.id.au> References: <56d16e6a$0$1593$c3e8da3$5496439d@news.astraweb.com> <85ziujwhdt.fsf@benfinney.id.au> Message-ID: On Mon, Feb 29, 2016 at 12:33 PM, Ben Finney wrote: > Rustom Mody writes: > >> On Monday, February 29, 2016 at 10:26:32 PM UTC+5:30, Ben Finney wrote: >> > Ian Kelly writes: >> > >> > > On Mon, Feb 29, 2016 at 9:21 AM, Larry Martell wrote: >> > > > I found this to be a very sad story. Sure, he had some issues, but I >> > > > don't think they needed to drug him, and take all that he had >> > > > learned away from him and turn him back into what he was before. >> > > >> > > "All that he had learned" meaning his delusions and psychoses? >> > >> > Indeed. If a revelation is unconnected to reality, it's misleading to >> > say that one has "learned" it. >> >> And who is the last arbiter on that 'reality'? > > A nice academic question. Not appropriate when we're discussing a real > person's psychotic episodes. Neither you nor I are experts in whether an > experience is psychosis, and the person already has an expert diagnosis. > > So please don't de-rail into the weeds of second-guessing an expert in > mental health. Unless you know the person's mental health better than > their doctor (and you do not), don't second-guess the diagnosis they > received. I think for the most part, the mental health industry is most interested in pushing drugs and forcing people into some status quo. From cs at zip.com.au Mon Feb 29 18:14:21 2016 From: cs at zip.com.au (Cameron Simpson) Date: Tue, 1 Mar 2016 10:14:21 +1100 Subject: common mistakes in this simple program In-Reply-To: References: Message-ID: <20160229231421.GA71350@cskk.homeip.net> On 29Feb2016 10:45, Ian Kelly wrote: >On Mon, Feb 29, 2016 at 10:26 AM, Ganesh Pal wrote: >> On Mon, Feb 29, 2016 at 9:59 PM, Ian Kelly wrote: >>> On Mon, Feb 29, 2016 at 8:18 AM, Ganesh Pal wrote: >>>> 1. usage of try- expect >>> >>> try-except in every single function is a code smell. You should only >>> be using it where you're actually going to handle the exception. If >>> you catch an exception just to log it, you generally should also >>> reraise it so that something further up the call chain has the >>> opportunity to handle it. >> >> How do we reraise the exception in python , I have used raise not >> sure how to reraise the exception > >raise with no arguments will reraise the exception currently being handled. > >except Exception: > logging.error("something went wrong") > raise Another remark here: if you're going to log, log the exception as well: logging.error("something went wrong: %s", e) Ian's example code is nice and simple to illustrate "log and then reraise" but few things are as annoying as log files reciting "something went wrong" or the equivalent without any accompanying context information. Cheers, Cameron Simpson From rgaddi at highlandtechnology.invalid Mon Feb 29 18:20:15 2016 From: rgaddi at highlandtechnology.invalid (Rob Gaddi) Date: Mon, 29 Feb 2016 23:20:15 -0000 (UTC) Subject: common mistakes in this simple program References: Message-ID: Cameron Simpson wrote: > On 29Feb2016 10:45, Ian Kelly wrote: >>On Mon, Feb 29, 2016 at 10:26 AM, Ganesh Pal wrote: >>> On Mon, Feb 29, 2016 at 9:59 PM, Ian Kelly wrote: >>>> On Mon, Feb 29, 2016 at 8:18 AM, Ganesh Pal wrote: >>>>> 1. usage of try- expect >>>> >>>> try-except in every single function is a code smell. You should only >>>> be using it where you're actually going to handle the exception. If >>>> you catch an exception just to log it, you generally should also >>>> reraise it so that something further up the call chain has the >>>> opportunity to handle it. >>> >>> How do we reraise the exception in python , I have used raise not >>> sure how to reraise the exception >> >>raise with no arguments will reraise the exception currently being handled. >> >>except Exception: >> logging.error("something went wrong") >> raise > > Another remark here: if you're going to log, log the exception as well: > > logging.error("something went wrong: %s", e) > > Ian's example code is nice and simple to illustrate "log and then reraise" but > few things are as annoying as log files reciting "something went wrong" or the > equivalent without any accompanying context information. > > Cheers, > Cameron Simpson Or, for that matter: logging.exception('something went wrong') Which gives you the whole traceback as well and doesn't require you to explictly grab the exception. -- Rob Gaddi, Highland Technology -- www.highlandtechnology.com Email address domain is currently out of order. See above to fix. From breamoreboy at yahoo.co.uk Mon Feb 29 18:29:43 2016 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Mon, 29 Feb 2016 23:29:43 +0000 Subject: [Off-topic] Requests author discusses MentalHealthError exception In-Reply-To: References: <56d16e6a$0$1593$c3e8da3$5496439d@news.astraweb.com> <85ziujwhdt.fsf@benfinney.id.au> Message-ID: On 29/02/2016 22:40, Larry Martell wrote: > On Mon, Feb 29, 2016 at 12:33 PM, Ben Finney wrote: >> Rustom Mody writes: >> >>> On Monday, February 29, 2016 at 10:26:32 PM UTC+5:30, Ben Finney wrote: >>>> Ian Kelly writes: >>>> >>>>> On Mon, Feb 29, 2016 at 9:21 AM, Larry Martell wrote: >>>>>> I found this to be a very sad story. Sure, he had some issues, but I >>>>>> don't think they needed to drug him, and take all that he had >>>>>> learned away from him and turn him back into what he was before. >>>>> >>>>> "All that he had learned" meaning his delusions and psychoses? >>>> >>>> Indeed. If a revelation is unconnected to reality, it's misleading to >>>> say that one has "learned" it. >>> >>> And who is the last arbiter on that 'reality'? >> >> A nice academic question. Not appropriate when we're discussing a real >> person's psychotic episodes. Neither you nor I are experts in whether an >> experience is psychosis, and the person already has an expert diagnosis. >> >> So please don't de-rail into the weeds of second-guessing an expert in >> mental health. Unless you know the person's mental health better than >> their doctor (and you do not), don't second-guess the diagnosis they >> received. > > I think for the most part, the mental health industry is most > interested in pushing drugs and forcing people into some status quo. > I am disgusted by your comments. I'll keep my original reply in reserve. -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From joel.goldstick at gmail.com Mon Feb 29 18:33:03 2016 From: joel.goldstick at gmail.com (Joel Goldstick) Date: Mon, 29 Feb 2016 18:33:03 -0500 Subject: [Off-topic] Requests author discusses MentalHealthError exception In-Reply-To: References: <56d16e6a$0$1593$c3e8da3$5496439d@news.astraweb.com> <85ziujwhdt.fsf@benfinney.id.au> Message-ID: On Mon, Feb 29, 2016 at 6:29 PM, Mark Lawrence wrote: > On 29/02/2016 22:40, Larry Martell wrote: > >> On Mon, Feb 29, 2016 at 12:33 PM, Ben Finney >> wrote: >> >>> Rustom Mody writes: >>> >>> On Monday, February 29, 2016 at 10:26:32 PM UTC+5:30, Ben Finney wrote: >>>> >>>>> Ian Kelly writes: >>>>> >>>>> On Mon, Feb 29, 2016 at 9:21 AM, Larry Martell wrote: >>>>>> >>>>>>> I found this to be a very sad story. Sure, he had some issues, but I >>>>>>> don't think they needed to drug him, and take all that he had >>>>>>> learned away from him and turn him back into what he was before. >>>>>>> >>>>>> >>>>>> "All that he had learned" meaning his delusions and psychoses? >>>>>> >>>>> >>>>> Indeed. If a revelation is unconnected to reality, it's misleading to >>>>> say that one has "learned" it. >>>>> >>>> >>>> And who is the last arbiter on that 'reality'? >>>> >>> >>> A nice academic question. Not appropriate when we're discussing a real >>> person's psychotic episodes. Neither you nor I are experts in whether an >>> experience is psychosis, and the person already has an expert diagnosis. >>> >>> So please don't de-rail into the weeds of second-guessing an expert in >>> mental health. Unless you know the person's mental health better than >>> their doctor (and you do not), don't second-guess the diagnosis they >>> received. >>> >> >> I think for the most part, the mental health industry is most >> interested in pushing drugs and forcing people into some status quo. >> >> > I am disgusted by your comments. I'll keep my original reply in reserve. > > > -- > My fellow Pythonistas, ask not what our language can do for you, ask > what you can do for our language. > > Mark Lawrence > > -- > https://mail.python.org/mailman/listinfo/python-list > I really like requests. I recommend it whenever people have trouble with urllib. As to Kenneth's personal struggle, good for him. An interesting and honest appraisal of his personal stuff. Viva la internet -- Joel Goldstick http://joelgoldstick.com/ http://cc-baseballstats.info/ From ian.g.kelly at gmail.com Mon Feb 29 18:39:37 2016 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Mon, 29 Feb 2016 16:39:37 -0700 Subject: common mistakes in this simple program In-Reply-To: <20160229231421.GA71350@cskk.homeip.net> References: <20160229231421.GA71350@cskk.homeip.net> Message-ID: On Mon, Feb 29, 2016 at 4:14 PM, Cameron Simpson wrote: > Another remark here: if you're going to log, log the exception as well: > > logging.error("something went wrong: %s", e) > > Ian's example code is nice and simple to illustrate "log and then reraise" > but few things are as annoying as log files reciting "something went wrong" > or the equivalent without any accompanying context information. The proper way to pass in the exception is as a keyword argument: logging.error("something went wrong", exc_info=e) But I actually meant to use logging.exception, which handles this automatically. From changernli at yahoo.com Mon Feb 29 18:42:06 2016 From: changernli at yahoo.com (quoc tuong) Date: Mon, 29 Feb 2016 23:42:06 +0000 (UTC) Subject: The debug process never connected back to Wing IDE References: <1167897796.1336230.1456789327016.JavaMail.yahoo.ref@mail.yahoo.com> Message-ID: <1167897796.1336230.1456789327016.JavaMail.yahoo@mail.yahoo.com> Hi python stopped working on this error: The debug process never connected back to Wing IDE: Aborting debug session. See Trouble-shooting Failure to Debug in the product manual. can you please show me how to fix this issue, thanks. From no.email at nospam.invalid Mon Feb 29 19:35:16 2016 From: no.email at nospam.invalid (Paul Rubin) Date: Mon, 29 Feb 2016 16:35:16 -0800 Subject: [Off-topic] Requests author discusses MentalHealthError exception References: <56d16e6a$0$1593$c3e8da3$5496439d@news.astraweb.com> Message-ID: <87io17t4qj.fsf@jester.gateway.pace.com> Ben Finney writes: >> "All that he had learned" meaning his delusions and psychoses? > Indeed. If a revelation is unconnected to reality, it's misleading to > say that one has ?learned? it.... > When someone describes the ill effects their mental illness produced, I > find it rather condescending for an observer to express regret that > the person no longer experiences those ill effects. The story reminded me of "A Beautiful Mind" by Sylvia Nasar, about mathematician John Nash who suffered from mental illness in the 1970s-80's(?) but later recovered and won the Nobel Prize in economics. The book is excellent and I recommend it if you find that sort of thing interesting. They made a movie from it that was popular but I gather was Hollywood-fluffified as one could expect (I haven't seen it). From ethan at stoneleaf.us Mon Feb 29 19:54:32 2016 From: ethan at stoneleaf.us (Ethan Furman) Date: Mon, 29 Feb 2016 16:54:32 -0800 Subject: [Off-topic] Requests author discusses MentalHealthError exception In-Reply-To: <87io17t4qj.fsf@jester.gateway.pace.com> References: <56d16e6a$0$1593$c3e8da3$5496439d@news.astraweb.com> <87io17t4qj.fsf@jester.gateway.pace.com> Message-ID: <56D4E848.60404@stoneleaf.us> On 02/29/2016 04:35 PM, Paul Rubin wrote: > The story reminded me of "A Beautiful Mind" by Sylvia Nasar, about > mathematician John Nash who suffered from mental illness in the > 1970s-80's(?) but later recovered and won the Nobel Prize in economics. > The book is excellent and I recommend it if you find that sort of thing > interesting. They made a movie from it that was popular but I gather > was Hollywood-fluffified as one could expect (I haven't seen it). I'm sure it was fluffified, but it is a good movie. -- ~Ethan~ From reachparagm at gmail.com Mon Feb 29 21:11:16 2016 From: reachparagm at gmail.com (reachparagm at gmail.com) Date: Mon, 29 Feb 2016 18:11:16 -0800 (PST) Subject: Paramiko Help. Execute command to Interactive Shell which is opened by SSHClient() In-Reply-To: References: Message-ID: <18e71a0e-7525-47e4-a7b7-0c6fed153fef@googlegroups.com> Sagar, Have you worked on RobotFramework-sshlibrary. It seem to have very simple interface for both command processing/execution as well as interactive session to grep responses and decide the flow. Let me know if you face any issues. Can help you solve. -paragm On Thursday, August 8, 2013 at 12:20:25 AM UTC-7, sagar varule wrote: > Hi All, > > Im using Paramiko for my SSH automation. Im using method that is shown in demo_simple.py example which comes with Paramiko. Below is code from demo_simple.py. > > As you can make out, below code opens SSH connection and opens Interactie Shell, and then wait for the command from user. > I want to submit the command to this Interactive Shell using code. > > try: > client = paramiko.SSHClient() > client.load_system_host_keys() > client.set_missing_host_key_policy(paramiko.WarningPolicy()) > print '*** Connecting...' > client.connect(hostname, port, username, password) > chan = client.invoke_shell() > print repr(client.get_transport()) > print '*** Here we go!' > print > interactive.interactive_shell(chan) > chan.close() > client.close() > > Well Another approach I tried is instead of opening interactive_shell, directly issue command using; > > stdin, stdout, stderr = client.exec_command(bv_cmd) > for line in stderr.readlines(): > print line > for line in stdout.readlines(): > print line > But problem here is client.exec_command(bv_cmd) waits for command to execute completely and then it returns to stdout,stderr. And I want to see the ouput from the command during its execution. Because my command takes long time for execution. > > Big Picture in My Mind: Big Picture I that want to achieve is, Opening different SSH connection to different host, and it will issue commands to all host, wait for execution. All execution should happen parallel.(just wanted to share my thought, and wanted to hear opinions from you all. Is this possible??). I am not big programmer, just 2 years experience with asp.net C# 2.0 so i would appreciate if discussion happens in simple english. From tjreedy at udel.edu Mon Feb 29 21:33:48 2016 From: tjreedy at udel.edu (Terry Reedy) Date: Mon, 29 Feb 2016 21:33:48 -0500 Subject: Everything good about Python except GUI IDE? In-Reply-To: References: <64a6599c-fae1-469d-bcee-875165b3cc7d@googlegroups.com> <56d294f8$0$1604$c3e8da3$5496439d@news.astraweb.com> <234a398e-1b0f-467b-a8cb-d7ca748f8062@googlegroups.com> <84922f24-3e00-4a23-b26d-5e6c0d8e7e04@googlegroups.com> <87y4a5c58i.fsf@elektro.pacujo.net> <87twksdg9c.fsf@elektro.pacujo.net> <87d1rgca58.fsf@elektro.pacujo.net> Message-ID: On 2/29/2016 3:20 AM, Chris Angelico wrote: > Incidentally, HTML+CSS is another excellent example of code being used > to create a visual effect. While there *are* WYSIWYG HTML editors, I'm > not familiar with any WYISWYG HTML+CSS editors, and I much more often > see a fast-turnaround code editing system such as codepen.io - you > change the HTML in one box, or the CSS in another, and the result down > below changes in real-time. Does it change with each key stroke (if the keystroke leaves the text in a coherent, non-error state) or only on some special input? > It wouldn't be too hard to create > something like this for a GUI, and it'd remove some of that feeling of > non-interactivity while still retaining all the benefits of code above > drag-and-drop. Keystroke auto-updates hardly make sense. On-demand updates for tkinter are already possible. By default, tk root windows, Python interpreter, Idle Shell, and IDLE Editors windows appear in the upper left of a screen. Move either the tk or the python window(s) to the right. 1. REPL (python or IDLE): Possible updates are signaled by \n. Tkinter visibly responds, when appropriate, on a statement by statement basis. >>> import tkinter as tk >>> root = tk.Tk() # root window appears >>> b = tk.Button(root) # text='', bg='white' are defaults >>> b.grid() # button appears in root window >>> b['text'] = 'Hello World!' # text appears in button >>> b['bg'] = 'red' # background color changes (Non-default options can also be set when a widget is created.) One can even, I just discovered, setup timer events interactively. >>> root.after(500, lambda: b.config(bg='blue', text='done')) # after 1/2 second (500 milliseconds), bg and text change The disadvantage of this method is that the REPL record must be saved and processed to reuse it. 2. Editor (IDLE): Run with F5 to see the result of any changes. The root window appears in a fraction of a second and population by widgets hardly takes much longer. I think that this is close enough to interactive for most purposes. 3. Editor + REPL (IDLE): Since F5 simulates running the file with 'python -i', this is automatic if the file in the editor does not run 'root.mainloop()'. If it does, the following in the file will allow one to end the blocking mainloop() call without also calling root.destroy and destroying the gui. tk.Button(root, text='quit mainloop', command=root.quit).grid() 4. Integrated Editor and Display: your idea. I think "It wouldn't be too hard" is wildly over-optimistic, as I will now explain> The turtle demo (python -m turtledemo) has read-only text on the left, a turtle screen on the right, and a 'Start' button. The text is read-only to avoid overwriting the supplied demo files. People who want to edit them are supposed to copy and paste into an editor, such as IDLE's.) The demo files must be and are (after failures) written in a special style to run properly within the demo framework, ... and to not disturb the buildbots when imported as part of test/test_all. Top level code should only have definitions, including a 'def main' entry point, non-gui statements, and a '__main__'-guarded conditional statement for initiating gui actions. They must also not bypass the turtle wrapping of tkinter to make direct tkinter calls that would disable the demo runner itself. So even if turtledemo were turned into turtle-designer, by replacing the text pane with an editor, it could not be used for all legal turtle programs. A GUI designer would replace the turtle screen also, with a gui frame. The special style would be something like the following template. import tkinter as tk # or "from tkinter import x, y, z, ..." # define classes and functions def main(parent): # call classes, passing parent # set up timers with parent.after # anything else requiring that root exist if __name__ == '__main__': root = tk.Tk() main(root) root.mainloop() The important point is that it must be possible to import the script without it creating a new root window or creating events. The app would run the user code with user_gui = importlib.import_module(file_path) user_gui.main(gui_frame) Importing the code after saving, rather than using exec on the editor contents, isolates it somewhat from the app, but not enough. Passing app's gui_frame into main and making that the parent of everything in the user gui ties the user gui into the app. However, it also allows the app to inadvertently affect the app. Non-interference between IDLE and user code, especially tkinter code, was the reason that IDLE was re-written, around a decade ago, to run user code in a separate process. Another issue is handling user code errors. Syntax errors can be marked in the editor, but tracebacks need a separate pane. And it is also useful to be able to interact with the gui without altering the code in the editor. Summary: if one starts with the idea of an interactive, unrestricted, code based, gui designer, especially one for tkinter using tkinter, and thinks carefully through the possible problems, one can easily end up with something similar to what we already have. On the other hand, I think an interactive, restricted, gui-based, tkinter Frame or Canvas designer could be written and would be quite useful. The generated code could easily be code that a human might write. There may be something already available on pypi. -- Terry Jan Reedy From nimbiotics at gmail.com Mon Feb 29 23:11:11 2016 From: nimbiotics at gmail.com (Mario R. Osorio) Date: Mon, 29 Feb 2016 20:11:11 -0800 (PST) Subject: [Off-topic] Requests author discusses MentalHealthError exception In-Reply-To: <56d16e6a$0$1593$c3e8da3$5496439d@news.astraweb.com> References: <56d16e6a$0$1593$c3e8da3$5496439d@news.astraweb.com> Message-ID: <6a970593-c99a-4b11-b8dd-95fa1bab3177@googlegroups.com> On Saturday, February 27, 2016 at 4:39:12 AM UTC-5, Steven D'Aprano wrote: > The author of Requests, Kenneth Reitz, discusses his recent recovery from a > MentalHealthError exception. > > http://www.kennethreitz.org/essays/mentalhealtherror-an-exception-occurred > > Although the connection to Python is only quite slim, I found it fascinating > to read. > > > > -- > Steven Live and let live, but DO learn from other's experiences in the mean time otherwise, your life has been wasted. If you are one of those "hyper-intelligent pan-dimensional beings", good for you! but I think he made it very clear that is NOT what he should have been into. Larry, you seem to know nothing about the mental health system(s) and might actually be in desperate need for help. I myself have been helped A LOT with the right combination of psychologists, psychiatric doctors and medication. You just have to be careful when selecting those professionals. Larry ... just get a life ... for yourself. (OH, and avoid new age and the such!!) From rosuav at gmail.com Mon Feb 29 23:31:38 2016 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 1 Mar 2016 15:31:38 +1100 Subject: Everything good about Python except GUI IDE? In-Reply-To: References: <64a6599c-fae1-469d-bcee-875165b3cc7d@googlegroups.com> <56d294f8$0$1604$c3e8da3$5496439d@news.astraweb.com> <234a398e-1b0f-467b-a8cb-d7ca748f8062@googlegroups.com> <84922f24-3e00-4a23-b26d-5e6c0d8e7e04@googlegroups.com> <87y4a5c58i.fsf@elektro.pacujo.net> <87twksdg9c.fsf@elektro.pacujo.net> <87d1rgca58.fsf@elektro.pacujo.net> Message-ID: On Tue, Mar 1, 2016 at 1:33 PM, Terry Reedy wrote: > On 2/29/2016 3:20 AM, Chris Angelico wrote: > >> Incidentally, HTML+CSS is another excellent example of code being used >> to create a visual effect. While there *are* WYSIWYG HTML editors, I'm >> not familiar with any WYISWYG HTML+CSS editors, and I much more often >> see a fast-turnaround code editing system such as codepen.io - you >> change the HTML in one box, or the CSS in another, and the result down >> below changes in real-time. > > > Does it change with each key stroke (if the keystroke leaves the text in a > coherent, non-error state) or only on some special input? In the case of codepen, it's not quite either; when you finish typing and leave it for half a second, it updates. Not quite instant, but automiatic. ChrisA