I made a thing:

https://github.com/lordmauve/edubundle/releases/tag/0.1-beta

I think it should just unzip to a USB stick and you're away... you've obviously got things like Pygame Zero, Python Repl, Mu, a command prompt in which you can use pip etc. IPython Notebook is in there for more advanced stuff like graph plotting.

This was mainly a learning experience for me, and to see how it can be done. The real work will be in automating the building of it and polishing up the rough edges. 

This was made much easier because various difficult dependencies (Pygame, PyQt) are shipping or working on Windows wheels. Also for the most part Python itself is quite happy without an absolute path (it has to be relative so that it can be simply unzipped or mounted as different drive letters). However pip creates binary executables for command line programs that do not, by default, contain relative paths - I think that could be changed.

Windows .lnk files present additional problems. A good idea would be to use the setuptools stuff to create little stub binaries that launch Python with the right path config, plus contain an appropriate icon resource.

Another avenue for improvement, now that PyQt is in the bundle, would be to write some additional GUI tools to make it a bit easier to do cool things - perhaps a "piptitude" GUI for pip?