PyConfig.h On Solaris

Hi, I apologise if this is an inappropriate post for this list; my query is concerned with the internals of Python, honest :) I am trying to compile a C module on Solaris, to return extended process information (args greater than 80 characters). However, the _FILE_OFFSET_BITS in pyconfig.h is causing me a bit of bother. It is preventing me, from correctly parsing /proc - according to the man pages, any handling of /proc entries must be done with 32bit structures, variables, etc. I've tried undefining it at compile time, without success. I've had to define _LP64, include <procfs.h>, then undefine _LP64. Doing this, allows me to use structured proc, instead of the deprecatd ioctl method. However, I don't think this is a working solution, as the code compiles, runs, but returns dubious results (I have a pure C version of the code, which works fine). I've included the code below. Is there a way, to tell Python to compile a module in 32bit mode? Or, is there a workaround for this Solaris oddity? Regards John #include "Python.h" #include <stdio.h> #define _LP64 #include <procfs.h> #undef _LP64 #include <fcntl.h> #include <sys/types.h> #include <sys/stat.h> static PyObject * psinfo_fullpidinfo(PyObject *self, PyObject *args) { PyObject *queryPid; char procBuf[50]; char argStr[512]; char **argvp; int fdPSInfoFile, fdASFile, counter; struct psinfo psBuf; if (!PyArg_ParseTuple(args, "i:pidinfo", &queryPid) ) { return NULL; } (void)sprintf(procBuf,"/proc/%i/psinfo", (int)queryPid); (void)sprintf(asBuf,"/proc/%i/as", (int)queryPid); if ((fdPSInfoFile = open(procBuf, O_RDONLY)) != -1 ) { if (read(fdPSInfoFile, &psBuf, sizeof(struct psinfo)) == sizeof(struct psinfo) ) { close(fdPSInfoFile); if ((fdASFile = open(asBuf, O_RDONLY)) == -1 ) { printf(" Open Of AS Failed"); } else { printf( "%d", psBuf.pr_argc); argvp=(char **) malloc(sizeof(char *) * (psBuf.pr_argc + 1)); if (lseek(fdASFile, psBuf.pr_argv, SEEK_SET) == -1) { printf ("Initial LSeek Failed" ); return NULL; } if (read(fdASFile, argvp, sizeof(char *) * (psBuf.pr_argc + 1)) == -1 ) { printf( "Read Failed" ); return NULL; } for ( counter=0; counter < psBuf.pr_argc; counter++ ) { if (lseek(fdASFile, argvp[ counter ], SEEK_SET) == -1) { printf( "2nd LSeek Failed"); return NULL; } if (read(fdASFile, argStr, 512) == -1) { printf( "Read Of AS File Failed"); return NULL; } printf( "Got %s.\n", argStr ); } close(fdASFile); } } } return Py_BuildValue( "s", argStr ); } static struct PyMethodDef PSInfoMethods[]={ { "fullpidinfo", psinfo_fullpidinfo, METH_VARARGS, "Display Full Arg Details For A Given PID" }, { NULL, NULL, 0, NULL } }; void initpsinfo(void) { (void)Py_InitModule("psinfo", PSInfoMethods); }

John Abel <johnfabel@btinternet.com> writes:
Is there a way, to tell Python to compile a module in 32bit mode?
You could edit pyconfig.h after configuring but before compiling Python. I would advise against doing so.
Or, is there a workaround for this Solaris oddity?
You could split your code into two files, with LFS-independent data structures in-between them. IOW, define JA_psinfo_t, which has just the fields of psinfo_t that you are interested in, using LFS-independent data types. In the source file opening /proc, copy all interesting fields in the data structure pointed-to by the caller, and compile this without LFS. Then, in the Python module proper, call these helper functions, passing JA_psinfo_t. AFAICT, it is only priovec_t, anyway, which is affected by the incompatibility, so if you don't use that, you could just pass psinfo_t through. I might be wrong here, though. Regards, Martin

Hi! Just a quick update. As a final attempt, before I implemented Martin's suggestion, I chaged the order of the includes. Having Python.h last in the includes, solved everything (apart from a warning, about _FILE_OFFSET_BITS being redefined. If it's already defined by the OS, does pyconfig.h have to define it?). Is there somewhere to log things such as this? Does it warrant a bug report? Regards John Martin v. Löwis wrote:
John Abel <johnfabel@btinternet.com> writes:
Is there a way, to tell Python to compile a module in 32bit mode?
You could edit pyconfig.h after configuring but before compiling Python. I would advise against doing so.
Or, is there a workaround for this Solaris oddity?
You could split your code into two files, with LFS-independent data structures in-between them. IOW, define JA_psinfo_t, which has just the fields of psinfo_t that you are interested in, using LFS-independent data types. In the source file opening /proc, copy all interesting fields in the data structure pointed-to by the caller, and compile this without LFS. Then, in the Python module proper, call these helper functions, passing JA_psinfo_t.
AFAICT, it is only priovec_t, anyway, which is affected by the incompatibility, so if you don't use that, you could just pass psinfo_t through. I might be wrong here, though.
Regards, Martin
_______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev

John Abel <johnfabel@btinternet.com> writes:
Just a quick update. As a final attempt, before I implemented Martin's suggestion, I chaged the order of the includes.
Then you are operating Python out of its specification. See http://www.python.org/doc/current/api/includes.html Good luck, and be prepared for random crashes.
Having Python.h last in the includes, solved everything (apart from a warning, about _FILE_OFFSET_BITS being redefined. If it's already defined by the OS, does pyconfig.h have to define it?). Is there somewhere to log things such as this? Does it warrant a bug report?
There is no bug AFAICT. Regards, Martin

John Abel <johnfabel@btinternet.com> writes:
Just a quick update. As a final attempt, before I implemented Martin's suggestion, I chaged the order of the includes.
Then you are operating Python out of its specification. See
http://www.python.org/doc/current/api/includes.html
Good luck, and be prepared for random crashes.
Having Python.h last in the includes, solved everything (apart from a warning, about _FILE_OFFSET_BITS being redefined. If it's already defined by the OS, does pyconfig.h have to define it?). Is there somewhere to log things such as this? Does it warrant a bug report?
There is no bug AFAICT.
OK, I can see why Python.h has to be first, but surely, abiding by the spec shouldn't prevent code from executing properly? Should I raise a bug report for this? I'm guessing that only Solaris is affected (though maybe other BSD-types are, too? AFAIK, only Solaris has the /proc oddity). I'd like to be able to produce compliant modules. I'd like to help, if I can. Regards John

"John Abel" <john.abel@pa.press.net> writes:
OK, I can see why Python.h has to be first, but surely, abiding by the spec shouldn't prevent code from executing properly? Should I raise a bug report for this?
For what? That you can't compile a certain Solaris header, if the application adds some system-specifed #defines? That is a Solaris bug, not a Python bug. All system headers ought to work under _LARGEFILE_SOURCE and _FILE_OFFSET_BITS=64.
I'm guessing that only Solaris is affected (though maybe other BSD-types are, too? AFAIK, only Solaris has the /proc oddity). I'd like to be able to produce compliant modules. I'd like to help, if I can.
I don't see that Python can or should change here. We have to define _FILE_OFFSET_BITS, and we have to define it consistently across all object files, or else we can't free exchange data structures which contain off_t fields across functions. Regards, Martin

OK, I can see why Python.h has to be first, but surely, abiding by the spec shouldn't prevent code from executing properly? Should I raise a bug report for this?
For what? That you can't compile a certain Solaris header, if the application adds some system-specifed #defines? That is a Solaris bug, not a Python bug. All system headers ought to work under _LARGEFILE_SOURCE and _FILE_OFFSET_BITS=64.
I'm guessing that only Solaris is affected (though maybe other BSD-types are, too? AFAIK, only Solaris has the /proc oddity). I'd like to be able to produce compliant modules. I'd like to help, if I can.
I don't see that Python can or should change here. We have to define _FILE_OFFSET_BITS, and we have to define it consistently across all object files, or else we can't free exchange data structures which contain off_t fields across functions.
Martin, Thanks for your help. It's a shame that Sun don't hurry up, and deprecate the ioctl method of reading /proc, as it seems that where most of the bother comes from. Maybe that's why no-one else seems to have written anything for parsing of /proc (in Python, or Perl). The module I've written, only uses open, lseek, and read. When it's finally finished, I'll distribute it with a warning. Regards John PS, for those that are interested, I've included the working (non-compliant) code. I'd appreciate any pointers. /* psinfomodule.c - a Python module to return PS information */ #include <procfs.h> #include <stdio.h> #include <unistd.h> #include <fcntl.h> #include <sys/types.h> #include <sys/stat.h> #include "Python.h" static PyObject * psinfo_pidinfo(PyObject *self, PyObject *args) { PyObject *queryPid; char procBuf[32]; char cmdLine[255]; char cmdArgs[255]; psinfo_t psBuf; int fdPSFile; if (!PyArg_ParseTuple(args, "l:pidinfo", &queryPid) ) { return NULL; } (void)sprintf(procBuf,"/proc/%ld/psinfo", (long)queryPid); if ((fdPSFile = open(procBuf, O_RDONLY)) != -1 ) { if ( read(fdPSFile, &psBuf, sizeof(psBuf)) == sizeof(psBuf) ) { (void)sprintf(cmdLine, "%s", psBuf.pr_fname ); (void)sprintf(cmdArgs, "%s", psBuf.pr_psargs ); } else{ return NULL; } close(fdPSFile); } return Py_BuildValue( "ss", cmdLine, cmdArgs ); } static PyObject * psinfo_fullpidinfo(PyObject *self, PyObject *args) { PyObject *queryPid; char procBuf[50]; char asBuf[50]; char argStr[512], cmdArgs[1024]; char **argvp; int fdFile, counter; struct psinfo psBuf; if (!PyArg_ParseTuple(args, "i:pidinfo", &queryPid) ) { return NULL; } (void)sprintf(procBuf,"/proc/%i/psinfo", (int)queryPid); (void)sprintf(asBuf,"/proc/%i/as", (int)queryPid); if ((fdFile = open(procBuf, O_RDONLY)) != -1 ) { if (read(fdFile, &psBuf, sizeof(struct psinfo)) == -1 ) { return NULL; } close(fdFile); if ((fdFile = open(asBuf, O_RDONLY)) == -1 ) { return NULL; } else { argvp=(char **) malloc(sizeof(char *) * (psBuf.pr_argc + 1)); if (lseek(fdFile, psBuf.pr_argv, SEEK_SET) == -1) { return NULL; } if (read(fdFile, argvp, sizeof(char *) * (psBuf.pr_argc + 1)) == -1 ) { printf( "Read Failed" ); return NULL; } for ( counter=0; counter < psBuf.pr_argc; counter++ ) { if (lseek(fdFile, argvp[ counter ], SEEK_SET) == -1) { printf( "2nd LSeek Failed"); return NULL; } if (read(fdFile, argStr, 512) == -1) { printf( "Read Of AS File Failed"); return NULL; } strcat( cmdArgs, argStr ); strcat( cmdArgs, " " ); } close(fdFile); } } else { return NULL; } return Py_BuildValue( "s", cmdArgs ); } static struct PyMethodDef PSInfoMethods[]={ { "pidinfo", psinfo_pidinfo, METH_VARARGS, "Display Details For A Given PID" }, { "fullpidinfo", psinfo_fullpidinfo, METH_VARARGS, "Display Full Arg Details For A Given PID" }, { NULL, NULL, 0, NULL } }; void initpsinfo(void) { (void)Py_InitModule("psinfo", PSInfoMethods); }
participants (3)
-
John Abel
-
John Abel
-
martin@v.loewis.de