[Edu-sig] PySqueak: More on Croquet and 3D (was: Just ran...)

[Paul D. Fernhout]

kirby urner wrote:
> All good R&D Paul.  In my book, these are some of the deep issues, and
> finding the right combination of libraries is not an easy job.
> Regardless of the ultimate fate of your particular project, you're
> adding to your repertoire plus carving out the basis of some kind of
> graphical framework others might use, and that's a positive thing.

Thanks for the kind words.

Also, I am forced to wrestle with the issue of what are my specific 
subgoals here (thinking of a PySqueak which is easy for beginners to use 
and experts want to use as well)?
They sort of are, roughly speaking:
* to have Self prototypes in Python as an option (check, though 
inefficient :-),
* to have a Self GUI in Python (brooding over),
* to port our 3D educational software to Python and this system (half 
done, the translating to Python part, but need to settle on widgets),
* and to make Python development with running systems as interactive and 
fun to use as that of a modern Smalltalk (just scratching at the network 
debugging approach outlined earlier, but also already have the reloader 
window I use with Jython),
* 3D simulated worlds in Python (less importance now to me, but of 
potential long term interest in terms of simulation).

I've been trying out the latest version of Croquet some more as I ponder 
this 3D and widget issue. I think I am getting past the Wow factor -- 
though it still remains pretty neat. Of course, as always, I was impressed 
more by watching the Alan Kay presentation videos like of drawing a fish 
in 2D and having software make it 3D and swim away than by using it myself.
   http://video.google.com/videoplay?docid=-9055536763288165825
Ignoring some walkbacks and exceptions when starting some of the demos, 
par for the course for my experience Squeaking, what I am left with is the 
same feeling I get when I watch pretty good 3D modeling of people -- which 
is sometimes a bit disconcerting (in a bad way). :-)

To explain, when you see cartoons of people, or mechanically rendered 
sprites of people with no serious attempts at skin texture or hair luster, 
you think, wow this is cool, and it is fun to look at. In my mind, that is 
something like using 2D apps or the command line. It does cool things its 
own way, and you can respect that. You get the immediacy people here talk 
about, and having no (or few) expectations about what it should do base 
don reality, nothing feels wrong (although it may still be confusing). 
But, when you look at images of people rendered in 3D that is pretty good 
(say 99%), they often look like corpses, because they are good enough to 
set in motion a whole bunch of reality based expectations we have for how 
other people should look, while not meeting them. Slightly drab hair? 
Maybe they are sick? Skin that looks like rubber? A sign of death. For 
more on this, see:
   "Realistic Human Graphics Look Creepy"
   http://games.slashdot.org/article.pl?sid=04/06/10/1327236
   http://www.slate.com/id/2102086 [article discussed]
So, by analogy, when you make fairly realistic 3D environments, but not 
100% realistic, you may be worse off for trying, because you raise 
expectations you can not fulfill, and the differences make people 
unconsciously feel uneasy, creepy, or frustrated. To be clear, I'm trying 
to make an analogy here; Croquet as a 1.0 beta demo does not attempt 
realistic human avatars, just a silly bunny rabbit avatar which looks 
cute. My point is about the 3D world aspect versus the real world.

So, in this case, with Croquet and 3D, perhaps more realism is actually a 
worse experience. And perhaps that unconsciously drives some of Alan Kay's 
interest in better 3D rendering? Yes, in theory you can get really good, 
but until you get there (when? 2020? 2040?), the results are sometimes 
almost more disappointing the closer you get to realism. I think Croquet 
may be facing that in the 3D realm. It activates expectations, like moving 
in 3D, but then you are confronted with how awkward it is to use the mouse 
to virtually "walk". You see a cube of blocks that invites you to take 
one, but when you put it back, there is no gravity to make it drop back 
into place, so it just oddly floats there. A slip of the mouse and the 
rabbit falls off the green carpet, to where? And it takes a bit of 
slogging to get back to anywhere interesting. I can see portals, but I 
can't shortcut click to them; I need to proceed slowly to them, and if I 
could click, and I suppose there is some trick perhaps, then I break the 
experience.

Obviously, specific popular 3D games solve many of these problems within 
the game in a fun "suspension of disbelief" way, and can be entertaining 
for twenty hours or so, and even leave you wanting more, e.g. a recent 
favorite of ours:
    "Beyond Good and Evil"
    http://www.gamespot.com/ps2/adventure/beyondgoodevil/
but, they do not seek to be the comprehensive life-time-of-use experiences 
that Croquet is implicitly claiming to be for right now (or at least, soon).

So, I wonder if better 3D graphics will really solve the problem in the 
near future. Short of Sutherland's "The Ultimate Display" (same as the 
Star Trek Holodeck), 3D worlds can be pretty dissatisfying or unfulfilling 
by themselves (ignoring their use as settings for games) if for no other 
reason (even if 100% visually realistic) than lack of force feedback or 
smell or air movement. Now, I'll say they can be useful, just like 2D 
white boards are, or conference calls are, and they can be fun (like many 
people find 3D games fun), but, ultimately, aside from any addicting game 
like aspects ("Evercrack") the 3D aspect by itself isn't a long term win 
(and it seems like the implicit notion of the Croquet work is that Croquet 
will be where we all spend most of our computer time). And the closer 
Croquet or similar systems get to reality, in some sense the more 
disappointing they are (up until we have the Holodeck or direct neural 
stimulation or something like that). Better to be in reality, I think. And 
stick with 2D or 3D graphics geared more towards efficient creation and 
collaboration than semi-realistic virtual creation and collaboration.

There's another thing that bothers me with the 3D world approach, and that 
is, it tends to run counter to the notion of moving fluidly between levels 
of abstraction, where you can see multiple representations of the same 
data. Our Garden Simulator (circa 1997) http://www.gardenwithinsight.com/ 
[See the second picture from the top on the right] tried to lead kids (or 
adults) from working in a relatively non-abstract garden setting onwards 
to using a level of abstraction using a numerical and graphical browser to 
visualize current state and change it, and then from that on to using 
graphs to visualize state over time, and so moving to higher levels of 
abstraction at each level. You could do that in a 3D Croquet world, say, 
by havinga 3D garden and a sheet of virtual graph paper, but it seems to 
me the 3D world is getting in the way a bit somehow, when you could just 
as well have a flat 2D window on your screen with a graph next to a 3D 
widnow of the garden. Granted, having a separate window for a 2D thing 
violates the paradigm Croquet is working towards, where multiple people 
can see what each other are looking at, so perhaps I need to understand 
more if that is really compelling and useful. But, on the other hand, even 
knowing where an avatar is pointing can't tell you what someone is 
thinking about. One big value of computers is that they can provide 
alternative ways to do things and look at things than is possible in the 
"real world". [One reason command line diehards like the command line so 
much is it makes some things easier to do with a mini-language than with a 
2D graphical interface, like change a bunch of files at once.] So, while 
the 3D Croquet interface does not demand giving up various levels of 
abstraction, it would seem to me it sort of encourages it. But that is not 
an opinion from using it much in practice, so perhaps in practice may be 
different.

Now this isn't to dismiss Croquet as a valuable experiment or as 
innovation. It's a good thing to have tried. And to keep trying with. And 
Python as a platform may well benefit from having some of this technology. 
It's just to consider it in the light of lessons learned and where to go 
next (for me or Python)?

But then you are left with the question -- how should people of varying 
abilities best interact with the computer in general and for various 
tasks. And, while I can see the value in a Croquet or ActiveWorlds 3D 
approach in some situations (I can see the potential value of being able 
to see where others are looking in the virtual world when collaborating, 
for example), I think there remains a lot of value in other interface 
paradigms, including both plain 2D widgets and also using a 3D window 
surrounded by 2D controls. (Obviously, whether OpenGL might render all 
that 2D stuff as well as the 3D stuff is just an implementation point in 
some sense.) So, in looking at the strengths and weaknesses of the 
Squeak/Croquet hybrid, derived from Self's Morphic and 2D collaborative 
model (and other things), I am again back to thinking that supporting 
collaboration in a 2D space (and ideally using prototypes) is potentially 
still a very useful thing, and a valuable first step. And so are any other 
approaches towards providing the option of doing development with Python 
in a way more like development under Smalltalk. And when six years ago I 
first brought up Squeak here, it was before Croquet, and in the context of 
these basic thing, plus some 2D learning applications built on top of them 
(more eToy-ish and HyperCard-ish and Virtual-Robot-programming-ish)

Anyway, having used Smalltalk for years (and ZetaLisp+Flavors before 
that), I am still confronted on a daily basis with the knowledge that it 
is productive and fun and empowering to develop with the level of 
interactivity Smalltalk (or the Symbolics) provides as an environment, and 
no Python IDE I have tried can deliver that (parts yes, the whole thing, 
no). Yet, I am also left with the reality that Python has gained 
acceptance because it is closer to C and BASIC in appearance, and because 
of its ability to play nice with other systems (especially ones written in 
C, or now, Java), as well as other factors, including that the license 
works out well for today's economic structures. So, on the one hand, I 
know what is possible, but on the other, I find myself often not able to 
make use of those systems in practice, which is what drives my interest on 
making such additions to Python more than anything.

> Incidentally, I just bought Civilization IV for my daughter [1], so we
> could give my wife's new computer a workout in the newly rented
> workspace [2], and she (my daughter) noticed right away that it
> contains Python bindings of some kind.  I haven't researched this yet,
> but I do find it exciting that a high powered game engine would make
> Python a part of its commercial front end, even mentioning Python on
> the box.

Your point is exactly what gives me pause about the Squeak / Croquet 
approach, and why Python has had more widespread commercial success. 
Scripting such things is just left out as an encouraged possibility with 
Squeak / Croquet (anything is possible with Squeak of course; it's a 
matter of focus and clutter). Squeak and Croquet are, as has been said 
here, sort of anti-collaborative. Which is ironic, since the whole point 
of Croquet is to be collaborative. [That was brought up years ago on this 
list.] Or perhaps, collaborative only on Croquet's terms? I liked your(?) 
proposal a way back on how what we need are standards for communicating 
between virtual world interfaces, so they could be implemented in various 
ways. Still, as Alan Kay has said, "any standard of more than three lines 
is ambiguous", and so you really need at least reference implementations, 
which Croquet is.

Yet, having said that, I am also thinking that given Python's success as 
gluing things together, and how all GUI toolkits are somewhat closed 
systems, see for example:
   "Why Gtk/Qt/WxWidgets... are bad"
   http://www.pygame.org/wiki/gui
I don't think it is unreasonable to argue for making another widget set 
(perhaps heavily copied from an existing one, like the pure Python on SDL 
OcempGUI)
   http://www.pygame.org/projects/9/125/
if there is a compelling and overwhelming reason to in terms of creating a 
good learning or developing environment in Python supporting a different 
paradigm of development ("modeless" versus "edit/run", and 
"prototype-oriented" instead of  "class-oriented"). That system can still 
be used to call existing (non GUI) libraries through Python. I'm not 
specifically making that argument right now, just saying, I think one 
could make it. For example, my very first Prototype proof of concept used 
TK widgets which were inherited from by prototypes. But I found when I 
copied one, that the widget was not copied, since internally two 
prototypes both point to the same proxied TK widget (as it was not 
copied). That's the sort of problem a pure Python solution might work 
around better (where a copied prototype would be a different GUI object). 
Not to say one might not be able to clone the TK widget, or a wx widget, 
just that it is more machinery, and perhaps some other unknown 
difficulties may exist in retrieving state from live widgets implemented 
in C/C++. But it is still hard to look at all the nice widgets in, say, 
the latest wx demo and think there is any point to writing yet another 
widget set... With all that entails...

Incidentally, I spent some time with the latest Debian versions of 
PythonCard and found a few issues from the standpoint of my using it. One 
is the paradigm -- mostly edit and run. Yes, you can change widget 
information while it is running, but that does not seem how it is set up. 
(HyperCard had a run/edit model too). Also, under GNU/Linux at least, you 
could not drag a button or list after you placed it -- they absorbed the 
clicks. Other components could be dragged. So, an inconsistency. Not sure 
if this is easily fixable with live wx components. In any case, that 
paradigm mismatch issue is potentially a large one.

--Paul Fernhout