Mailman 3 January 2011 - Python-Dev

Re: [Python-Dev] MSI: Remove dependency from win32com.client module (issue4080047)
by techtonik＠gmail.com Feb. 2, 2011

Feb. 2, 2011

There is no b.p.o issue as it's not a bug, but a tiny copy/paste patch to clean up the code a bit while I am trying to understand how to add Python to the PATH. I see no reason for b.p.o bureaucracy. Mercurial-style workflow [1] is more beneficial to development as it doesn't require switching from console to browser for submitting changes. This way tiny changes can be integrated/updated more rapidly. 1. http://mercurial.selenic.com/wiki/ContributingChanges#The_basics:_patches_b… http://codereview.appspot.com/4080047/

12 20

Mercurial style patch submission (Was: MSI: Remove dependency from win32com.client module (issue4080047))
by anatoly techtonik Feb. 1, 2011

Feb. 1, 2011

On Mon, Jan 31, 2011 at 10:54 PM, Antoine Pitrou <solipsis(a)pitrou.net> wrote: > On Mon, 31 Jan 2011 20:45:45 +0000 > techtonik(a)gmail.com wrote: >> I see no reason for b.p.o bureaucracy. Mercurial-style workflow [1] is >> more beneficial to development as it doesn't require switching from >> console to browser for submitting changes. > > Ok, why don't you contribute to Mercurial instead? If you don't want to receive a stupid answer, why don't you read the … [View More]

7 6

Re: [Python-Dev] MSI: Remove dependency from win32com.client module (issue4080047)
by georg.brandl＠gmail.com Jan. 31, 2011

Jan. 31, 2011

Is there a bugs.python.org issue for this? http://codereview.appspot.com/4080047/

1 0

MSI: Remove dependency from win32com.client module (issue4080047)
by techtonik＠gmail.com Jan. 31, 2011

Jan. 31, 2011

Reviewers: , Please review this at http://codereview.appspot.com/4080047/ Affected files: M Tools/msi/msi.py M Tools/msi/msilib.py Index: Tools/msi/msi.py =================================================================== --- Tools/msi/msi.py (revision 88279) +++ Tools/msi/msi.py (working copy) @@ -4,7 +4,6 @@ import msilib, schema, sequence, os, glob, time, re, shutil, zipfile from msilib import Feature, CAB, Directory, Dialog, Binary, add_data import uisample -from … [View More]win32com.client import constants from distutils.spawn import find_executable from uuids import product_codes import tempfile @@ -1360,7 +1359,7 @@ # Step 2: Add CAB files i = msilib.MakeInstaller() - db = i.OpenDatabase(msi, constants.msiOpenDatabaseModeTransact) + db = i.OpenDatabase(msi, msilib.msiOpenDatabaseModeTransact) v = db.OpenView("SELECT LastSequence FROM Media") v.Execute(None) Index: Tools/msi/msilib.py =================================================================== --- Tools/msi/msilib.py (revision 88279) +++ Tools/msi/msilib.py (working copy) @@ -4,7 +4,6 @@ import win32com.client.gencache import win32com.client import pythoncom, pywintypes -from win32com.client import constants import re, string, os, sets, glob, subprocess, sys, _winreg, struct try: @@ -29,6 +28,18 @@ knownbits = datasizemask | type_valid | type_localizable | \ typemask | type_nullable | type_key +# Constants from Windows Installer SDK +msiOpenDatabaseModeReadOnly = 0 +msiOpenDatabaseModeTransact = 1 +msiOpenDatabaseModeDirect = 2 +msiOpenDatabaseModeCreate = 3 +msiColumnInfoNames = 0 +msiColumnInfoTypes = 1 +msiReadStreamInteger = 0 +msiReadStreamBytes = 1 +msiViewModifyInsert = 1 +msidbFileAttributesVital = 512 + # Summary Info Property IDs PID_CODEPAGE=1 PID_TITLE=2 @@ -141,8 +152,7 @@ def gen_schema(destpath, schemapath): d = MakeInstaller() - schema = d.OpenDatabase(schemapath, - win32com.client.constants.msiOpenDatabaseModeReadOnly) + schema = d.OpenDatabase(schemapath, msiOpenDatabaseModeReadOnly) # XXX ORBER BY v=schema.OpenView("SELECT * FROM _Columns") @@ -196,8 +206,7 @@ def gen_sequence(destpath, msipath): dir = os.path.dirname(destpath) d = MakeInstaller() - seqmsi = d.OpenDatabase(msipath, - win32com.client.constants.msiOpenDatabaseModeReadOnly) + seqmsi = d.OpenDatabase(msipath, msiOpenDatabaseModeReadOnly) v = seqmsi.OpenView("SELECT * FROM _Tables"); v.Execute(None) @@ -212,7 +221,7 @@ f.write("%s = [\n" % table) v1 = seqmsi.OpenView("SELECT * FROM `%s`" % table) v1.Execute(None) - info = v1.ColumnInfo(constants.msiColumnInfoTypes) + info = v1.ColumnInfo(msiColumnInfoTypes) while 1: r = v1.Fetch() if not r:break @@ -226,7 +235,7 @@ rec.append(r.StringData(i)) elif info.StringData(i)[0]=="v": size = r.DataSize(i) - bytes = r.ReadStream(i, size, constants.msiReadStreamBytes) + bytes = r.ReadStream(i, size, msiReadStreamBytes) bytes = bytes.encode("latin-1") # binary data represented "as-is" if table == "Binary": fname = rec[0]+".bin" @@ -275,7 +284,7 @@ r.SetStream(i+1, field.name) else: raise TypeError, "Unsupported type %s" % field.__class__.__name__ - v.Modify(win32com.client.constants.msiViewModifyInsert, r) + v.Modify(msiViewModifyInsert, r) r.ClearData() v.Close() @@ -298,8 +307,7 @@ ProductCode = ProductCode.upper() d = MakeInstaller() # Create the database - db = d.OpenDatabase(name, - win32com.client.constants.msiOpenDatabaseModeCreate) + db = d.OpenDatabase(name, msiOpenDatabaseModeCreate) # Create the tables for t in schema.tables: t.create(db) @@ -538,7 +546,7 @@ short = self.make_short(file) full = "%s|%s" % (short, file) filesize = os.stat(absolute).st_size - # constants.msidbFileAttributesVital + # msidbFileAttributesVital # Compressed omitted, since it is the database default # could add r/o, system, hidden attributes = 512 [View Less]

2 1

short fetch for NEXTARG macro (was: one byte byte code arguments)
by Jurjen N.E. Bos Jan. 31, 2011

Jan. 31, 2011

I just did it: my first python source code hack. I replaced the NEXTARG and PEEKARG macros in ceval.c using a cast to short pointer, and lo and behold, a crude measurement indicates one to two percent speed increase. That isn't much, but it is virtually for free! Here are the macro's I used: #define NEXTARG() (next_instr +=2, *(short*)&next_instr[-2]) #define PEEKARG() (*(short*)&next_instr[1]) - Jurjen

4 3

Byte code arguments from two to one byte: did anyone try this?
by Jurjen N.E. Bos Jan. 31, 2011

Jan. 31, 2011

I tried to find any research on this subject, but I couldn't find any, so I'll be daring and vulnerable and just try it out to see what your thoughts are. I single stepped a simple loop in Python to see where the efficiency bottlenecks are. I was impressed by the optimizations already in there, but I still dare to suggest an optimization that from my estimates might shave off a few cycles, speeding up Python about 5%. The idea is simple: change the byte code argument values from two … [View More]bytes to one. Implications are: - code changes are relatively simple, see below - fewer memory reads, which are becoming more and more expensive - saves three instructions for every opcode with args (i.e. most of them) Code changes are, as far as I could find: compile.c: assemble_emit must produce extended opcodes for all cases of more than 8 bits instead of 16 ceval.c: NEXTARG and PEEKARG need adjustment EXTENDED_ARG needs adjustment (this will be a four byte instruction, which is ugly, I agree) peephole.c: GETARG, SETARG, need adjustment also GETJUMPTGT, CODESIZE routine tuple_of_constants, fold_binops_on_constants, PyCode_Optimize are dependent on instruction length, which will be 2 instead of 3 (search for the digit 3 will find all cases, as far as I checked) you probably will have to write a macro for codestr[i+3] there is a check for code length >32700, but I think this one might stay, maybe if a few extra checks are added. dis: minor adjustments Estimation of speed impact: about 80% of the instructions seem to have an argument, and I never saw an opcode >255 while looking at bytecode, so they are probably not frequent. The NEXTARG macro expands on my Macbook to: mov -408(%ebp),%edx (next_instr) movzbl 2(%edx),%eax (*second byte) shl $0x8,%eax (*shift) movzbl 1(%edx),%edx (first byte) add %edx,%eax (*combine) and the starred instructions will vanish. The main loop is approximately 40 instructions, so a saving of three instructions is significant. I don't dare to claim 3/40 = 7.5% savings, but I think 5% may be realistic. Did anyone try this already? If not, I might take up the gauntlet and try it myself, but I never did this before... - Jurjen PS I also saw that some scratch variables, mainly v and x, are carefull stored back in memory by the compiler and the end of the big interpreter loop, while their value isn't used anymore, of course. A few carefully placed braces might tell the compiler how useless this is and save another few percent. [View Less]

5 4

Followup: Byte code arguments from two to one byte: did anyone try this?
by Jurjen N.E. Bos Jan. 31, 2011

Jan. 31, 2011

> What version of CPython did you try that with? The latest py3k branch? I had a quick look at 3.2, 2.5 and 2.7 and got the impression that the savings is more if the interpreter loop is faster: the fewer instructions there are, the bigger a 3 instruction difference would make. The NEXTARG macro is the same in all three versions: #define NEXTARG() (next_instr += 2, (next_instr[-1]<<8) + next_instr[-2]) and the compiler compiles this to two separate fetches. I found … [View More]

1 0

[RELEASED] Python 3.2 rc 2
by Georg Brandl Jan. 31, 2011

Jan. 31, 2011

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On behalf of the Python development team, I'm quite happy to announce the second release candidate of Python 3.2. Python 3.2 is a continuation of the efforts to improve and stabilize the Python 3.x line. Since the final release of Python 2.7, the 2.x line will only receive bugfixes, and new features are developed for 3.x only. Since PEP 3003, the Moratorium on Language Changes, is in effect, there are no changes in Python's syntax and built-in … [View More]

1 0

Re: [Python-Dev] [Python-checkins] r88273 - python/branches/py3k/Doc/whatsnew/3.2.rst
by Raymond Hettinger Jan. 31, 2011

Jan. 31, 2011

On Jan 30, 2011, at 8:21 PM, eli.bendersky wrote: Please use the open tracker item and do not edit the document directly. Raymond

1 0

Stable buildbots
by Antoine Pitrou Jan. 30, 2011

Jan. 30, 2011

Hi, Just to let you know that we now have 8 stable buildbots, including Barry's own PPC Ubuntu machine (even though the Windows buildbots give a rather unconventional meaning to the word "stability"). Right now they are mostly green: http://www.python.org/dev/buildbot/all/waterfall?category=3.x.stable cheers Antoine.

11 23