Mailman 3 failure building numpy using icc - NumPy-Discussion

failure building numpy using icc

Glen W. Mabey

28 Feb 2008 28 Feb '08

1:21 p.m.

Hello, I'm using svn numpy and get the following error upon executing /usr/local/bin/python2.5 setup.py config --noisy --cc=/opt/intel/cce/10.0.025/bin/icc --compiler=intel --fcompiler=intel build_clib build_ext I see: conv_template:> build/src.linux-x86_64-2.5/numpy/core/src/scalartypes.inc Traceback (most recent call last): File "setup.py", line 96, in <module> setup_package() File "setup.py", line 89, in setup_package configuration=configuration ) File "/home/gmabey/src/DiamondBack/Diamondback/src/numpy-20080228_svn/numpy/distutils/core.py", line 184, in setup return old_setup(**new_attr) File "/usr/local/lib/python2.5/distutils/core.py", line 151, in setup dist.run_commands() File "/usr/local/lib/python2.5/distutils/dist.py", line 974, in run_commands self.run_command(cmd) File "/usr/local/lib/python2.5/distutils/dist.py", line 994, in run_command cmd_obj.run() File "/home/gmabey/src/DiamondBack/Diamondback/src/numpy-20080228_svn/numpy/distutils/command/build_ext.py", line 56, in run self.run_command('build_src') File "/usr/local/lib/python2.5/distutils/cmd.py", line 333, in run_command self.distribution.run_command(command) File "/usr/local/lib/python2.5/distutils/dist.py", line 994, in run_command cmd_obj.run() File "/home/gmabey/src/DiamondBack/Diamondback/src/numpy-20080228_svn/numpy/distutils/command/build_src.py", line 130, in run self.build_sources() File "/home/gmabey/src/DiamondBack/Diamondback/src/numpy-20080228_svn/numpy/distutils/command/build_src.py", line 147, in build_sources self.build_extension_sources(ext) File "/home/gmabey/src/DiamondBack/Diamondback/src/numpy-20080228_svn/numpy/distutils/command/build_src.py", line 252, in build_extension_sources sources = self.template_sources(sources, ext) File "/home/gmabey/src/DiamondBack/Diamondback/src/numpy-20080228_svn/numpy/distutils/command/build_src.py", line 359, in template_sources outstr = process_c_file(source) File "/home/gmabey/src/DiamondBack/Diamondback/src/numpy-20080228_svn/numpy/distutils/conv_template.py", line 185, in process_file % (sourcefile, process_str(''.join(lines)))) File "/home/gmabey/src/DiamondBack/Diamondback/src/numpy-20080228_svn/numpy/distutils/conv_template.py", line 150, in process_str newstr[sub[0]:sub[1]], sub[4]) File "/home/gmabey/src/DiamondBack/Diamondback/src/numpy-20080228_svn/numpy/distutils/conv_template.py", line 117, in expand_sub % (line, template_re.sub(namerepl, substr))) File "/home/gmabey/src/DiamondBack/Diamondback/src/numpy-20080228_svn/numpy/distutils/conv_template.py", line 113, in namerepl return names[name][thissub[0]] KeyError: 'PREFIX' And I do not see any errors when building the same svn version with gcc (on a different machine). I've unsuccessfully tried to follow that backtrace of functions to figure out exactly what is going on. Any hints/suggestions? Thanks, Glen Mabey

Show replies by date

Robert Kern

1 Mar 1 Mar

2:17 a.m.

On Thu, Feb 28, 2008 at 1:21 PM, Glen W. Mabey wrote:

...

Hello,

I'm using svn numpy and get the following error upon executing

/usr/local/bin/python2.5 setup.py config --noisy --cc=/opt/intel/cce/10.0.025/bin/icc --compiler=intel --fcompiler=intel build_clib build_ext

I see:

conv_template:> build/src.linux-x86_64-2.5/numpy/core/src/scalartypes.inc Traceback (most recent call last): File "setup.py", line 96, in <module> setup_package() File "setup.py", line 89, in setup_package configuration=configuration ) File "/home/gmabey/src/DiamondBack/Diamondback/src/numpy-20080228_svn/numpy/distutils/core.py", line 184, in setup return old_setup(**new_attr) File "/usr/local/lib/python2.5/distutils/core.py", line 151, in setup dist.run_commands() File "/usr/local/lib/python2.5/distutils/dist.py", line 974, in run_commands self.run_command(cmd) File "/usr/local/lib/python2.5/distutils/dist.py", line 994, in run_command cmd_obj.run() File "/home/gmabey/src/DiamondBack/Diamondback/src/numpy-20080228_svn/numpy/distutils/command/build_ext.py", line 56, in run self.run_command('build_src') File "/usr/local/lib/python2.5/distutils/cmd.py", line 333, in run_command self.distribution.run_command(command) File "/usr/local/lib/python2.5/distutils/dist.py", line 994, in run_command cmd_obj.run() File "/home/gmabey/src/DiamondBack/Diamondback/src/numpy-20080228_svn/numpy/distutils/command/build_src.py", line 130, in run self.build_sources() File "/home/gmabey/src/DiamondBack/Diamondback/src/numpy-20080228_svn/numpy/distutils/command/build_src.py", line 147, in build_sources self.build_extension_sources(ext) File "/home/gmabey/src/DiamondBack/Diamondback/src/numpy-20080228_svn/numpy/distutils/command/build_src.py", line 252, in build_extension_sources sources = self.template_sources(sources, ext) File "/home/gmabey/src/DiamondBack/Diamondback/src/numpy-20080228_svn/numpy/distutils/command/build_src.py", line 359, in template_sources outstr = process_c_file(source) File "/home/gmabey/src/DiamondBack/Diamondback/src/numpy-20080228_svn/numpy/distutils/conv_template.py", line 185, in process_file % (sourcefile, process_str(''.join(lines)))) File "/home/gmabey/src/DiamondBack/Diamondback/src/numpy-20080228_svn/numpy/distutils/conv_template.py", line 150, in process_str newstr[sub[0]:sub[1]], sub[4]) File "/home/gmabey/src/DiamondBack/Diamondback/src/numpy-20080228_svn/numpy/distutils/conv_template.py", line 117, in expand_sub % (line, template_re.sub(namerepl, substr))) File "/home/gmabey/src/DiamondBack/Diamondback/src/numpy-20080228_svn/numpy/distutils/conv_template.py", line 113, in namerepl return names[name][thissub[0]] KeyError: 'PREFIX'

And I do not see any errors when building the same svn version with gcc (on a different machine).

I've unsuccessfully tried to follow that backtrace of functions to figure out exactly what is going on.

Any hints/suggestions?

Off-hand, no, sorry. I'm not sure why the compiler would matter in this part of the code, though. Can you try using gcc on the same machine? -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco

Glen W. Mabey

29 Apr 29 Apr

10:21 a.m.

Robert, I do appreciate your response to this question (oh way back when) and am just now getting back to this problem. On Sat, Mar 01, 2008 at 02:17:46AM -0600, Robert Kern wrote:

...

On Thu, Feb 28, 2008 at 1:21 PM, Glen W. Mabey wrote:

...
Hello,

I'm using svn numpy and get the following error upon executing

/usr/local/bin/python2.5 setup.py config --noisy --cc=/opt/intel/cce/10.0.025/bin/icc --compiler=intel --fcompiler=intel build_clib build_ext

I see:

conv_template:> build/src.linux-x86_64-2.5/numpy/core/src/scalartypes.inc Traceback (most recent call last): File "setup.py", line 96, in <module> setup_package() File "setup.py", line 89, in setup_package configuration=configuration ) File "/home/gmabey/src/DiamondBack/Diamondback/src/numpy-20080228_svn/numpy/distutils/core.py", line 184, in setup return old_setup(**new_attr) File "/usr/local/lib/python2.5/distutils/core.py", line 151, in setup dist.run_commands() File "/usr/local/lib/python2.5/distutils/dist.py", line 974, in run_commands self.run_command(cmd) File "/usr/local/lib/python2.5/distutils/dist.py", line 994, in run_command cmd_obj.run() File "/home/gmabey/src/DiamondBack/Diamondback/src/numpy-20080228_svn/numpy/distutils/command/build_ext.py", line 56, in run self.run_command('build_src') File "/usr/local/lib/python2.5/distutils/cmd.py", line 333, in run_command self.distribution.run_command(command) File "/usr/local/lib/python2.5/distutils/dist.py", line 994, in run_command cmd_obj.run() File "/home/gmabey/src/DiamondBack/Diamondback/src/numpy-20080228_svn/numpy/distutils/command/build_src.py", line 130, in run self.build_sources() File "/home/gmabey/src/DiamondBack/Diamondback/src/numpy-20080228_svn/numpy/distutils/command/build_src.py", line 147, in build_sources self.build_extension_sources(ext) File "/home/gmabey/src/DiamondBack/Diamondback/src/numpy-20080228_svn/numpy/distutils/command/build_src.py", line 252, in build_extension_sources sources = self.template_sources(sources, ext) File "/home/gmabey/src/DiamondBack/Diamondback/src/numpy-20080228_svn/numpy/distutils/command/build_src.py", line 359, in template_sources outstr = process_c_file(source) File "/home/gmabey/src/DiamondBack/Diamondback/src/numpy-20080228_svn/numpy/distutils/conv_template.py", line 185, in process_file % (sourcefile, process_str(''.join(lines)))) File "/home/gmabey/src/DiamondBack/Diamondback/src/numpy-20080228_svn/numpy/distutils/conv_template.py", line 150, in process_str newstr[sub[0]:sub[1]], sub[4]) File "/home/gmabey/src/DiamondBack/Diamondback/src/numpy-20080228_svn/numpy/distutils/conv_template.py", line 117, in expand_sub % (line, template_re.sub(namerepl, substr))) File "/home/gmabey/src/DiamondBack/Diamondback/src/numpy-20080228_svn/numpy/distutils/conv_template.py", line 113, in namerepl return names[name][thissub[0]] KeyError: 'PREFIX'

And I do not see any errors when building the same svn version with gcc (on a different machine).

I've unsuccessfully tried to follow that backtrace of functions to figure out exactly what is going on.

Any hints/suggestions?

Off-hand, no, sorry. I'm not sure why the compiler would matter in this part of the code, though. Can you try using gcc on the same machine?

I did successfully use gcc on the same machine without any problems. However, I still encounter a similar error with today's svn. Now, I don't know anything about how code is produced within numpy, but within pdb I was able to find this string that some sort of pattern substitution is acting on. It suspiciously looks like the first function belongs after the last one ... static PyObject * gentype_@name@(PyObject *m1) { PyObject *arr, *ret; arr = PyArray_FromScalar(m1, NULL); if (arr == NULL) return NULL; ret = arr->ob_type->tp_as_number->nb_@name@(arr); Py_DECREF(arr); return ret; } /**end repeat**/ static int gentype_nonzero_number(PyObject *m1) { PyObject *arr; int ret; arr = PyArray_FromScalar(m1, NULL); if (arr == NULL) return -1; ret = arr->ob_type->tp_as_number->nb_nonzero(arr); Py_DECREF(arr); return ret; } static PyObject * gentype_str(PyObject *self) { PyArrayObject *arr; PyObject *ret; arr = (PyArrayObject *)PyArray_FromScalar(self, NULL); if (arr==NULL) return NULL; ret = PyObject_Str((PyObject *)arr); Py_DECREF(arr); return ret; } static PyObject * gentype_repr(PyObject *self) { PyArrayObject *arr; PyObject *ret; arr = (PyArrayObject *)PyArray_FromScalar(self, NULL); if (arr==NULL) return NULL; ret = PyObject_Str((PyObject *)arr); Py_DECREF(arr); return ret; } /**begin repeat #name=float, double, longdouble# #NAME=FLOAT, DOUBLE, LONGDOUBLE# #PREFIX=NPY_,NPY_,NPY_# */ static void format_@name@(char *buf, size_t buflen, @name@ val, unsigned int precision) { char *cp; PyOS_snprintf(buf, buflen, "%.*" @PREFIX@@NAME@_FMT, precision, val); cp = buf; if (*cp == '-') cp++; for (; *cp != '\0'; cp++) { if (!isdigit(Py_CHARMASK(*cp))) break; } if (*cp == '\0') { *cp++ = '.'; *cp++ = '0'; *cp++ = '\0'; } } And the command I ran to get to this point was: python setup.py config --noisy --cc=/opt/intel/cce/10.0.025/bin/icc --compiler=intel --fcompiler=intel build_clib build_ext Thank you, Glen Mabey

Charles R Harris

10:41 a.m.

On Tue, Apr 29, 2008 at 9:21 AM, Glen W. Mabey wrote:

...

Robert,

<snip> Could you post the new output? The conv_template routine has changed a bit, you should make sure you have the latest version. Also, it is pure python and the use of icc shouldn't make a difference in substituting the keys in the template to generate the source files. Also try deleting the build directory before the new build. Chuck

Glen W. Mabey

10:50 a.m.

On Tue, Apr 29, 2008 at 10:41:25AM -0500, Charles R Harris wrote:

...

Could you post the new output? The conv_template routine has changed a bit, you should make sure you have the latest version. Also, it is pure python and the use of icc shouldn't make a difference in substituting the keys in the template to generate the source files. Also try deleting the build directory before the new build.

Okay, I deleted the build directory, and here is the full output: executing numpy/core/code_generators/generate_array_api.py adding 'build/src.linux-x86_64-2.5/numpy/core/__multiarray_api.h' to sources. creating build/src.linux-x86_64-2.5/numpy/core/src conv_template:> build/src.linux-x86_64-2.5/numpy/core/src/scalartypes.inc Traceback (most recent call last): File "setup.py", line 96, in <module> setup_package() File "setup.py", line 89, in setup_package configuration=configuration ) File "/home/gmabey/src/DiamondBack/Diamondback/src/numpy-20080429_svn/numpy/distutils/core.py", line 184, in setup return old_setup(**new_attr) File "/usr/local/lib/python2.5/distutils/core.py", line 151, in setup dist.run_commands() File "/usr/local/lib/python2.5/distutils/dist.py", line 974, in run_commands self.run_command(cmd) File "/usr/local/lib/python2.5/distutils/dist.py", line 994, in run_command cmd_obj.run() File "/home/gmabey/src/DiamondBack/Diamondback/src/numpy-20080429_svn/numpy/distutils/command/build_ext.py", line 56, in run self.run_command('build_src') File "/usr/local/lib/python2.5/distutils/cmd.py", line 333, in run_command self.distribution.run_command(command) File "/usr/local/lib/python2.5/distutils/dist.py", line 994, in run_command cmd_obj.run() File "/home/gmabey/src/DiamondBack/Diamondback/src/numpy-20080429_svn/numpy/distutils/command/build_src.py", line 130, in run self.build_sources() File "/home/gmabey/src/DiamondBack/Diamondback/src/numpy-20080429_svn/numpy/distutils/command/build_src.py", line 147, in build_sources self.build_extension_sources(ext) File "/home/gmabey/src/DiamondBack/Diamondback/src/numpy-20080429_svn/numpy/distutils/command/build_src.py", line 252, in build_extension_sources sources = self.template_sources(sources, ext) File "/home/gmabey/src/DiamondBack/Diamondback/src/numpy-20080429_svn/numpy/distutils/command/build_src.py", line 359, in template_sources outstr = process_c_file(source) File "/home/gmabey/src/DiamondBack/Diamondback/src/numpy-20080429_svn/numpy/distutils/conv_template.py", line 252, in process_file code = process_str(''.join(lines)) File "/home/gmabey/src/DiamondBack/Diamondback/src/numpy-20080429_svn/numpy/distutils/conv_template.py", line 222, in process_str code.extend(parse_string(astr, global_names, 0, 1)) File "/home/gmabey/src/DiamondBack/Diamondback/src/numpy-20080429_svn/numpy/distutils/conv_template.py", line 210, in parse_string newcode = parse_string(text, newenv, newlevel, newline) File "/home/gmabey/src/DiamondBack/Diamondback/src/numpy-20080429_svn/numpy/distutils/conv_template.py", line 216, in parse_string code.append(replace_re.sub(replace, astr)) File "/home/gmabey/src/DiamondBack/Diamondback/src/numpy-20080429_svn/numpy/distutils/conv_template.py", line 187, in replace raise KeyError, msg KeyError: "#line 497\n: 'PREFIX'" Thank you, Glen Mabey

Charles R Harris

11:24 a.m.

On Tue, Apr 29, 2008 at 9:50 AM, Glen W. Mabey wrote:

...

...
Could you post the new output? The conv_template routine has changed a bit, you should make sure you have the latest version. Also, it is pure

On Tue, Apr 29, 2008 at 10:41:25AM -0500, Charles R Harris wrote: python and the use of icc shouldn't make a difference in substituting the keys in the template to generate the source files. Also try deleting the build directory before the new build.

Okay, I deleted the build directory, and here is the full output:

executing numpy/core/code_generators/generate_array_api.py adding 'build/src.linux-x86_64-2.5/numpy/core/__multiarray_api.h' to sources. creating build/src.linux-x86_64-2.5/numpy/core/src conv_template:> build/src.linux-x86_64-2.5/numpy/core/src/scalartypes.inc Traceback (most recent call last): File "setup.py", line 96, in <module> setup_package() File "setup.py", line 89, in setup_package configuration=configuration ) File "/home/gmabey/src/DiamondBack/Diamondback/src/numpy-20080429_svn/numpy/distutils/core.py", line 184, in setup return old_setup(**new_attr) File "/usr/local/lib/python2.5/distutils/core.py", line 151, in setup dist.run_commands() File "/usr/local/lib/python2.5/distutils/dist.py", line 974, in run_commands self.run_command(cmd) File "/usr/local/lib/python2.5/distutils/dist.py", line 994, in run_command cmd_obj.run() File "/home/gmabey/src/DiamondBack/Diamondback/src/numpy-20080429_svn/numpy/distutils/command/build_ext.py", line 56, in run self.run_command('build_src') File "/usr/local/lib/python2.5/distutils/cmd.py", line 333, in run_command self.distribution.run_command(command) File "/usr/local/lib/python2.5/distutils/dist.py", line 994, in run_command cmd_obj.run() File "/home/gmabey/src/DiamondBack/Diamondback/src/numpy-20080429_svn/numpy/distutils/command/build_src.py", line 130, in run self.build_sources() File "/home/gmabey/src/DiamondBack/Diamondback/src/numpy-20080429_svn/numpy/distutils/command/build_src.py", line 147, in build_sources self.build_extension_sources(ext) File "/home/gmabey/src/DiamondBack/Diamondback/src/numpy-20080429_svn/numpy/distutils/command/build_src.py", line 252, in build_extension_sources sources = self.template_sources(sources, ext) File "/home/gmabey/src/DiamondBack/Diamondback/src/numpy-20080429_svn/numpy/distutils/command/build_src.py", line 359, in template_sources outstr = process_c_file(source) File "/home/gmabey/src/DiamondBack/Diamondback/src/numpy-20080429_svn/numpy/distutils/conv_template.py", line 252, in process_file code = process_str(''.join(lines)) File "/home/gmabey/src/DiamondBack/Diamondback/src/numpy-20080429_svn/numpy/distutils/conv_template.py", line 222, in process_str code.extend(parse_string(astr, global_names, 0, 1)) File "/home/gmabey/src/DiamondBack/Diamondback/src/numpy-20080429_svn/numpy/distutils/conv_template.py", line 210, in parse_string newcode = parse_string(text, newenv, newlevel, newline) File "/home/gmabey/src/DiamondBack/Diamondback/src/numpy-20080429_svn/numpy/distutils/conv_template.py", line 216, in parse_string code.append(replace_re.sub(replace, astr)) File "/home/gmabey/src/DiamondBack/Diamondback/src/numpy-20080429_svn/numpy/distutils/conv_template.py", line 187, in replace raise KeyError, msg KeyError: "#line 497\n: 'PREFIX'"

Hmm, something must be mucking with the sources. You can run conv_template on the pure file from the src directory with python ../../distutils/conv_template.py scalartypes.inc.src However, when called from within python the routine also processes the include files and I suspect that may be the problem. Try going into distutils/conv_template and find the line include_src_re = re.compile(r"(\n|\A)#include\s*['\"]" r"(?P<name>[\w\d./\\]+[.]src)['\"]", re.I) and replace it with include_src_re = re.compile(r"(\n|\A)\s*#include\s*['\"]" r"(?P<name>[\w\d./\\]+[.]src)['\"]", re.I) Which should knock off any whitespace before #include. This is not a fix, but it might help us find the problem. Chuck

Glen W. Mabey

12:19 p.m.

On Tue, Apr 29, 2008 at 11:24:56AM -0500, Charles R Harris wrote:

...

Hmm, something must be mucking with the sources. You can run conv_template on the pure file from the src directory with

python ../../distutils/conv_template.py scalartypes.inc.src

However, when called from within python the routine also processes the include files and I suspect that may be the problem. Try going into distutils/conv_template and find the line

include_src_re = re.compile(r"(\n|\A)#include\s*['\"]" r"(?P<name>[\w\d./\\]+[.]src)['\"]", re.I)

and replace it with

include_src_re = re.compile(r"(\n|\A)\s*#include\s*['\"]" r"(?P<name>[\w\d./\\]+[.]src)['\"]", re.I)

Which should knock off any whitespace before #include. This is not a fix, but it might help us find the problem.

Humm. I couldn't see any difference in the behavior. Is there something that I should be looking for? Glen

Charles R Harris

1:01 p.m.

On Tue, Apr 29, 2008 at 11:19 AM, Glen W. Mabey wrote:

...

On Tue, Apr 29, 2008 at 11:24:56AM -0500, Charles R Harris wrote:

...
Hmm, something must be mucking with the sources. You can run conv_template on the pure file from the src directory with

python ../../distutils/conv_template.py scalartypes.inc.src

However, when called from within python the routine also processes the include files and I suspect that may be the problem. Try going into distutils/conv_template and find the line

include_src_re = re.compile(r"(\n|\A)#include\s*['\"]" r"(?P<name>[\w\d./\\]+[.]src)['\"]", re.I)

and replace it with

include_src_re = re.compile(r"(\n|\A)\s*#include\s*['\"]" r"(?P<name>[\w\d./\\]+[.]src)['\"]", re.I)

Which should knock off any whitespace before #include. This is not a fix, but it might help us find the problem.

Humm. I couldn't see any difference in the behavior. Is there something that I should be looking for?

Not really, it was just a shot in the dark. The thing is, if conv_python is being run on the same source each time, it should give the same result. So I suspect that something is changing the sources and that is what I'm trying to track down. One of the build guys should know... Hmmm..., I notice that there is a line break in line 18 of scalartypes.inc.src that doesn't belong there. Could you fix that? And the line number of the loop is 497, which is the loop before the one with PREFIX defined. For some reason the parser is missing the /**end repeat**/ on line 509. Hmmm, try putting a blank line before it. Just fishing here... Why that should happen with icc and not with gnu is a mystery. Chuck

Glen W. Mabey

4:43 p.m.

On Tue, Apr 29, 2008 at 01:01:55PM -0500, Charles R Harris wrote:

...

Not really, it was just a shot in the dark. The thing is, if conv_python is being run on the same source each time, it should give the same result. So I suspect that something is changing the sources and that is what I'm trying to track down. One of the build guys should know...

Hmmm..., I notice that there is a line break in line 18 of scalartypes.inc.src that doesn't belong there. Could you fix that?

Okay, got that one.

...

And the line number of the loop is 497, which is the loop before the one with PREFIX defined. For some reason the parser is missing the /**end repeat**/ on line 509. Hmmm, try putting a blank line before it.

Putting a blank line before it didn't do the trick. I have been looking at the python code that should have found the /**end repeat**/ mark and I can't understand why it didn't find it. In fact, watch this great trick in pdb: ipdb> text[265:281] '/**end repeat**/' ipdb> text.find( text[265:281] ) -1 Isn't that cool? I can only assume that it is a compiler bug and I will have to upgrade to a newer version of icc (I'm using 10.0.025, actually it's cce). After I do that, I'll post again if I have trouble. Thanks for your help. Glen

Glen W. Mabey

15 May 15 May

11:28 a.m.

On Tue, Apr 29, 2008 at 04:43:09PM -0500, Glen W. Mabey wrote:

...

Isn't that cool? I can only assume that it is a compiler bug and I will have to upgrade to a newer version of icc (I'm using 10.0.025, actually it's cce).

After I do that, I'll post again if I have trouble.

Just to follow up, compiling python with 10.1.015 resolved the problem. Glen

5824

Age (days ago)

5901

Last active (days ago)

List overview

Download

9 comments

3 participants

participants (3)

Charles R Harris
Glen W. Mabey
Robert Kern

failure building numpy using icc

Glen W. Mabey

Robert Kern

Glen W. Mabey

Charles R Harris

Glen W. Mabey

Charles R Harris

Glen W. Mabey

Charles R Harris

Glen W. Mabey

Glen W. Mabey

tags

participants (3)