numpy and the Google App Engine
Greetings, Google provides a product called App Engine. The description from their site follows, "Google App Engine enables you to build and host web apps on the same systems that power Google applications. App Engine offers fast development and deployment; simple administration, with no need to worry about hardware, patches or backups; and effortless scalability. " You can deploy applications written in either Python or JAVA. There are free and paid versions of the service. The Google App Engine would appear to be a powerful source of CPU cycles for scientific computing. Unfortunately this is currently not the case because numpy is not one of the supported libraries. The Python App Engine allows only the installation of user supplied pure Python code. I have recently returned from attending the Google I/O conference in San Francisco. While there I inquired into the possibility of getting numpy added. The basic response was that there doesn't appear to be much interest from the community given the amount of work it would take to vet and add numpy. I would like to ask your help in changing this perception. The quickest and easiest thing you can do would be to add your "me too" to this feature request (item #190) on the support site: http://code.google.com/p/googleappengine/issues/detail?id=190 If this issue is important to you could also consider raising this issue in the related Google Group: http://groups.google.com/group/google-appengine Letting Google know how you will use numpy would be helpful. If you or your institution would be willing to pay for service if you could deploy cloud applications that required numpy would be helpful to let them know as well. Finally, if you run into any App Engine developers (Guido included) let them know that you would like to see numpy added. Thank you for your time and consideration. Chris -- Christopher Hanley Senior Systems Software Engineer Space Telescope Science Institute 3700 San Martin Drive Baltimore MD, 21218 (410) 338-4338
Christopher Hanley wrote:
Greetings,
Google provides a product called App Engine. The description from their site follows,
"Google App Engine enables you to build and host web apps on the same systems that power Google applications. App Engine offers fast development and deployment; simple administration, with no need to worry about hardware, patches or backups; and effortless scalability. "
You can deploy applications written in either Python or JAVA. There are free and paid versions of the service.
The Google App Engine would appear to be a powerful source of CPU cycles for scientific computing. Unfortunately this is currently not the case because numpy is not one of the supported libraries. The Python App Engine allows only the installation of user supplied pure Python code.
I have recently returned from attending the Google I/O conference in San Francisco. While there I inquired into the possibility of getting numpy added. The basic response was that there doesn't appear to be much interest from the community given the amount of work it would take to vet and add numpy.
Something to keep in mind: It's rather trivial to write code to intentionally crash the Python interpreter using pure Python code and NumPy (or overwrite data in it, run custom assembly code...in short, NumPy is a big gaping security hole in this context). This obviously can't go on in the AppEngine. So this probably involves a considerable amount of work in the NumPy source code base as well, it's not simply about verifying. -- Dag Sverre
On Wed, May 26, 2010 at 12:49 PM, Dag Sverre Seljebotn <dagss@student.matnat.uio.no> wrote:
Christopher Hanley wrote:
Greetings,
Google provides a product called App Engine. The description from their site follows,
"Google App Engine enables you to build and host web apps on the same systems that power Google applications. App Engine offers fast development and deployment; simple administration, with no need to worry about hardware, patches or backups; and effortless scalability. "
You can deploy applications written in either Python or JAVA. There are free and paid versions of the service.
The Google App Engine would appear to be a powerful source of CPU cycles for scientific computing. Unfortunately this is currently not the case because numpy is not one of the supported libraries. The Python App Engine allows only the installation of user supplied pure Python code.
I have recently returned from attending the Google I/O conference in San Francisco. While there I inquired into the possibility of getting numpy added. The basic response was that there doesn't appear to be much interest from the community given the amount of work it would take to vet and add numpy.
Something to keep in mind: It's rather trivial to write code to intentionally crash the Python interpreter using pure Python code and NumPy (or overwrite data in it, run custom assembly code...in short, NumPy is a big gaping security hole in this context). This obviously can't go on in the AppEngine. So this probably involves a considerable amount of work in the NumPy source code base as well, it's not simply about verifying.
Agreed. Perhaps the recently discussed rework of the C internals will better allow a security audit of numpy. At that point perhaps the numpy community could more easily work with Google to fix security problems.
-- Dag Sverre _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
-- Christopher Hanley Senior Systems Software Engineer Space Telescope Science Institute 3700 San Martin Drive Baltimore MD, 21218 (410) 338-4338
On Wed, May 26, 2010 at 10:37 AM, Christopher Hanley <chanley@stsci.edu>wrote:
On Wed, May 26, 2010 at 12:49 PM, Dag Sverre Seljebotn <dagss@student.matnat.uio.no> wrote:
Christopher Hanley wrote:
Greetings,
Google provides a product called App Engine. The description from their site follows,
"Google App Engine enables you to build and host web apps on the same systems that power Google applications. App Engine offers fast development and deployment; simple administration, with no need to worry about hardware, patches or backups; and effortless scalability. "
You can deploy applications written in either Python or JAVA. There are free and paid versions of the service.
The Google App Engine would appear to be a powerful source of CPU cycles for scientific computing. Unfortunately this is currently not the case because numpy is not one of the supported libraries. The Python App Engine allows only the installation of user supplied pure Python code.
I have recently returned from attending the Google I/O conference in San Francisco. While there I inquired into the possibility of getting numpy added. The basic response was that there doesn't appear to be much interest from the community given the amount of work it would take to vet and add numpy.
Something to keep in mind: It's rather trivial to write code to intentionally crash the Python interpreter using pure Python code and NumPy (or overwrite data in it, run custom assembly code...in short, NumPy is a big gaping security hole in this context). This obviously can't go on in the AppEngine. So this probably involves a considerable amount of work in the NumPy source code base as well, it's not simply about verifying.
Agreed. Perhaps the recently discussed rework of the C internals will better allow a security audit of numpy.
My guess is that when "the fur begins to fly," submitted tickets will receive more attention, i.e., if you really want to see this done...file a ticket. (IMO, it's *never* wasted effort to do this: the worst that can happen is that some - recorded - person will close it as "will not do," and if for some unforeseeable reason they're unwilling to include an explanation as to why, well, you'll "know where they live," so to speak.) DG
At that point perhaps the numpy community could more easily work with Google to fix security problems.
-- Dag Sverre _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
-- Christopher Hanley Senior Systems Software Engineer Space Telescope Science Institute 3700 San Martin Drive Baltimore MD, 21218 (410) 338-4338 _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
-- Mathematician: noun, someone who disavows certainty when their uncertainty set is non-empty, even if that set has measure zero. Hope: noun, that delusive spirit which escaped Pandora's jar and, with her lies, prevents mankind from committing a general suicide. (As interpreted by Robert Graves)
On Wed, May 26, 2010 at 12:19, Christopher Hanley <chanley@stsci.edu> wrote:
Greetings,
Google provides a product called App Engine. The description from their site follows,
"Google App Engine enables you to build and host web apps on the same systems that power Google applications. App Engine offers fast development and deployment; simple administration, with no need to worry about hardware, patches or backups; and effortless scalability. "
You can deploy applications written in either Python or JAVA. There are free and paid versions of the service.
The Google App Engine would appear to be a powerful source of CPU cycles for scientific computing.
Not really. It is not intended for such purposes. It is intended for the easy deployment and horizontal scaling of web applications. Each individual request is very short; it is limited to 10 seconds of CPU time. While numpy would be useful for scientific web applications (not least because it would help you keep to that 10 second limit when doing things like simple image processing or summary statistics or whatever), it is not a source of CPU cycles. Services like Amazon EC2 or Rackspace Cloud are much closer to what you want. PiCloud provides an even nicer interface for you: http://www.picloud.com/ Disclosure: Enthought partners with PiCloud to provide most EPD libraries. I can't say I'm disinterested in promoting it, but it *is* a really powerful product that *does* provide CPU cycles for scientific computing with an interface much more suited to it than GAE.
Unfortunately this is currently not the case because numpy is not one of the supported libraries. The Python App Engine allows only the installation of user supplied pure Python code.
I have recently returned from attending the Google I/O conference in San Francisco. While there I inquired into the possibility of getting numpy added. The basic response was that there doesn't appear to be much interest from the community given the amount of work it would take to vet and add numpy.
I would like to ask your help in changing this perception.
The quickest and easiest thing you can do would be to add your "me too" to this feature request (item #190) on the support site:
http://code.google.com/p/googleappengine/issues/detail?id=190
My understanding is that they hate "me too" comments. They ask that you star the issue instead. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco
On Wed, May 26, 2010 at 12:54 PM, Robert Kern <robert.kern@gmail.com> wrote:
On Wed, May 26, 2010 at 12:19, Christopher Hanley <chanley@stsci.edu> wrote:
Greetings,
Google provides a product called App Engine. The description from their site follows,
"Google App Engine enables you to build and host web apps on the same systems that power Google applications. App Engine offers fast development and deployment; simple administration, with no need to worry about hardware, patches or backups; and effortless scalability. "
You can deploy applications written in either Python or JAVA. There are free and paid versions of the service.
The Google App Engine would appear to be a powerful source of CPU cycles for scientific computing.
Not really. It is not intended for such purposes. It is intended for the easy deployment and horizontal scaling of web applications. Each individual request is very short; it is limited to 10 seconds of CPU time. While numpy would be useful for scientific web applications (not least because it would help you keep to that 10 second limit when doing things like simple image processing or summary statistics or whatever), it is not a source of CPU cycles. Services like Amazon EC2 or Rackspace Cloud are much closer to what you want. PiCloud provides an even nicer interface for you:
In my conversations with the developers they indicated that it could be used for both. However, either use case would be useful for scientific computing.
Disclosure: Enthought partners with PiCloud to provide most EPD libraries. I can't say I'm disinterested in promoting it, but it *is* a really powerful product that *does* provide CPU cycles for scientific computing with an interface much more suited to it than GAE.
Unfortunately this is currently not the case because numpy is not one of the supported libraries. The Python App Engine allows only the installation of user supplied pure Python code.
I have recently returned from attending the Google I/O conference in San Francisco. While there I inquired into the possibility of getting numpy added. The basic response was that there doesn't appear to be much interest from the community given the amount of work it would take to vet and add numpy.
I would like to ask your help in changing this perception.
The quickest and easiest thing you can do would be to add your "me too" to this feature request (item #190) on the support site:
http://code.google.com/p/googleappengine/issues/detail?id=190
My understanding is that they hate "me too" comments. They ask that you star the issue instead.
I would be happy to see any support either starring or "me too" comments. Their comments to me was that they saw no interest. In my opinion any indication of interest would be a positive.
-- Robert Kern
"I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
participants (4)
-
Christopher Hanley -
Dag Sverre Seljebotn -
David Goldsmith -
Robert Kern