I decided to turn this into a more regular journal thing. I need to invent a way to make it easier - maybe a database of entries or something like that. In 1994 or so when I first discovered HTML I had a lot of fun writing web pages, and when people ask I still tell them that HTML is very easy to learn, that they don't need special tools like FrontPage and that it is better to learn how to write it by hand. But I must consider it a drudgery because I sure put off writing anything that I feel like writing, like these journal entries. I've taken to documenting good ideas much more than I used to however - I have a directory with a text file for each idea.

My latest idea which I've been putting some time into is what I call "metawidgets". It is inspired by the fact that there is no "one true" GUI. There are dependencies on things like input and output devices, the physical capabilities of the user, etc. For example if one is using a mouse, then the MacOS GUI makes a lot of sense. The menubar being at the top of the screen (see question 5 at this link) means that you can just throw your mouse all the way to the top to reliably access any application's menu, and this takes less thought and dexterity than it does to precisely position the mouse over a menubar attached to an application window the way we see it done in practically all other GUIs. But then we are taking advantage of a mouse ideosyncracy, aren't we. OTOH Windows popularized the idea of using a right-click with the mouse to access a contextual menu related to whatever the mouse was hovering over when clicked. It's a really compelling idea, to the point where it's hard to imagine a modern GUI without contextual menus. Since Mac mice only have one button, this idea was reproduced clumsily by holding down the Option key while clicking. So something conceptually simple requires two input devices and two hands to accomplish.

Anyway mice suck. It's high time they were obsolete. They are the major cause of carpal tunnel syndrome. They are too indirect; moving the mouse in a horizontal plane causes the cursor to move in the (usually) vertical plane of the screen.

Let's consider the touchscreen. For one thing, you can't right-click with your finger so contextual menus would have to be accessed using some other action. We can put that aside for now (but I'm thinking touch-and-hold, say for a half-second or so, just longer than you'd take to normally just punch a button). Another issue is that menus themselves are clumsy. You reach up somewhere near the top of the screen, hold your finger to the screen, and a menu drops down underneath your hand where you can't see it, and then you have to wipe your finger downwards and release over a fairly small target. Yuck. I think touchscreens are better served with buttons for everything, large ones that you can punch on the first try. This must have been what the designers of the fictional LCARS system on Star Trek had in mind. However isn't it odd that they tend to put a column of buttons along the left side of the screen? It looks logical in that we tend to read left-to-right. Web sites tend to have navigation buttons along the left. Microsoft Outlook has icons along the left to switch between the contexts. But with a touchscreen if you are right handed again you cover up the screen with your arm in order to hit those buttons. If you are left handed then you would want the buttons to be along the left, if you are right handed you would maybe want them on the right; it's ergonomically better even if it is linguistically confusing. That is an interesting tradeoff.

We don't know yet what general-purpose 3D VR GUI's are going to look like. Some guesses have been made but people have mostly found them suboptimal to date. You can be sure VR will become more practical over time however. Wearable computers are now at the stage that PDAs were at in, say, 1994 or 95. They are starting to exist but aren't quite practical yet. I would predict that by 2005 wearable computers with "reality augmentation" see-through displays will be somewhat common. This will possibly stimulate further development of the 3D GUI, depending on whether monocular or stereo displays are more popular.

Anyway it is a shame that every time there is a UI paradigm shift applications must be re-written. In many cases they need it anyway - it's just a good excuse to make an old application better. But good software has been lost too just because their textual UIs of yore were not converted to modern GUIs, or because they ran on an OS that lost its popularity. So I think it would be a good idea to separate the GUI from the application more fully than has been done to date. So far the best abstraction has been the X window system. As the oldest client-server GUI system it obviously has withstood the test of time. It is independent from some of the fads; you can use a lot of different widget toolkits to build applications, but they all emit the same X protocol to the network socket, therefore there is no problem using 10-year-old Xterms with modern apps as long as the apps don't use some new-fangled X features like shaped windows and whatnot. You can also use any window manager. So the lesson to be learned is that X has endured because its protocol was low-level; the looks of the widgets are completely separate from the mechanism used to render them. OTOH X protocol is very verbose, and it has become quite a problem to use X applications over slow WAN or modem connections. There are multiple solutions, none of them perfect. VNC is one. Its protocol is very simple - every time some part of the display changes, just send an updated image of that area, using one of a handful of compression algorithms. You'd think sending pictures rather than vector graphic commands would really suck bandwidth but the compression is quite good and the result is that VNC is much more responsive over slow connections than plain X protocol. Sun's answer to that is the Sun Ray. They haven't documented their protocol as far as I have seen, but based on the recommendations for network bandwidth it would appear that the compression is not very good. They instead aim for full multimedia capabilities; overbuild now so that the terminals will not be obsolete anytime soon, and will be able to keep up with fast-moving GUIs. Alternatively there is a compressed version of X. I think it both cuts some corners in the amount of traffic to be sent back and forth, and also does some kind of sliding-window compression but I haven't studied it in detail. And of course there is the Citrix solution which Microsoft commandeered and renamed to WinFrame. From what I have heard it makes fairly good sense - not as verbose as X and easier to understand.

But now for something completely different... all those remote GUI protocols are just various ways to represent the graphics themselves. The programmer of the application and/or toolkit still has control over what the GUI is going to look like. For a while now I've been turning over in the back of my mind the idea that interaction with the user is at some level a simpler thing; you show him some kind of changing display, which he might optionally be able to interact directly with, and you give him some widgets which can be used to execute simple parameterless actions, and you give him places to enter text or other kinds of values like dates and numbers, and sometimes there are actions which do take parameters and you have to pop up a dialog or something like that to collect the parameters. And with the prevalance of hypertext, things tend to be linked a lot. So I would like to find a way to specify GUIs in terms of the kind of input and output that it is supposed to accomplish, rather than the details of how it ought to look. At first I thought of doing it with a class library but then I thought it would be even better to make it a remote-able protocol like X. And use XML of course since that is the hot new thing and seems like a good fit. It would also solve my immediate problem - I want to use touchscreens around the house for a variety of different things such as controlling lights, to replace the phonebook by the phone in the living room, controlling the security system, viewing video from security cameras (maybe - requires better hardware than I currently have), looking at recipes in the kitchen, etc. Well really it's just that touchscreens are geeky and cool and I'm looking for situations in which they'd be useful. But I have two kinds of machines with touchscreens now - a small pile of old 386 laptops with 2 megs RAM and 5 megs FLASH, and two color CRT touchscreens for the Mac. Both are too underpowered to run Linux and an X server (the obvious choice for a remote GUI). VNC has been somewhat disappointing also. It wouldn't run on the 386's (needs some work to become compatible with more kinds of video chipsets) and on the Mac, I got it working but there are a couple of drawbacks. It won't go fullscreen (the menu and titlebar won't go away) and it's a little slow - not obscenely slow but enough to notice.

An example from my "idea file" for this metawidget thing:

Another example: let's nest some objects.  Our "desktop" (the old standby
metaphor... but nothing in the design of metawidgets should enforce any metaphors
at all!  someday we will be using VR and "desktops" will be passe.  It's just that
the example runs on a 2D display so it's still an appropriate metaphor; so says
our hypothetical user.) consists of a dayplanner, a phonebook, a couple of VNC
desktops from other machines for access to legacy apps, a hierarchical object
organization tool (otherwise known as a file manager, but we don't have files,
only objects) containing work related to various projects, and a desktop manager
UI which allows us to decide which objects will be shown at the top level.  The
dayplanner consists of a month-view calendar, a week-view calendar, a todo list, a
"today's outlook" page (the default), and a journal page.  The pages can be
defined elsewhere; let's just look at the top level:

<object-ui name=com.ecloud.desktop>
	<altset name=desktopChoices default=dayplanner>
		<object-ui name=com.ecloud.dayplanner>
			<group name=dayplanner>
				<altset name=plannerChoices default=outlook>
					<object-ui name=com.ecloud.dayplanner.outlook>
					</object-ui>
					<object-ui name=com.ecloud.dayplanner.monthView>
					</object-ui>
					<object-ui name=com.ecloud.dayplanner.weekView>
					</object-ui>
					<object-ui name=com.ecloud.dayplanner.todo>
					</object-ui>
					<object-ui name=com.ecloud.dayplanner.journal>
					</object-ui>
				</altset>
			</group name=dayplanner>
		</object-ui>
		<object-ui name=com.ecloud.nettebook label=phonebook>
		</object-ui>
		<object-ui name=com.ecloud.vncviewer label=electron>
		</object-ui>
		<object-ui name=com.ecloud.vncviewer label=nucleus>
		</object-ui>
		<object-ui name=com.ecloud.projectOrganizer label=projects>
		</object-ui>
		<object-ui name=com.ecloud.desktopManager label=desktop>
		</object-ui>
	</altset>
</object-ui>

The fact that the default view in the planner is the outlook page is only used the
first time the user logs in and the client machine sees the above XML.  The
planner object decided to make that the default based on some preference probably,
or just that the outlook was what the user was last seen looking at.  If the user
leaves the planner and then comes back, it should redisplay the sub-ui within the
planner that he was looking at before, not use the default again.

The above tree is conceptual - it's not all found in one big XML file necessarily. 
I think we should use a "pull" model.  When the client logs in to the server it
gets only this:

<object-ui name=com.ecloud.desktop/>

and sends back this, to request the content of the desktop:

<request-object-ui name=com.ecloud.desktop>

at which point it gets the top-level altset and its object-ui's.  Then it sees
that the default is the dayplanner so it requests that, and gets back more or less
what we see above for the dayplanner.  Then it requests the outlook page.  When
there are no more container-type tags with nothing in them, it stops.  If the user
clicks the phonebook selector control (whatever form it might take), it has to
request that.  Once fetched, object UIs may never change (unless perhaps the
server sends a rare, special sort of "refresh" tag to indicate that the object
behind the UI just changed in an incompatible way and the UI needs to be
refetched).  They will be saved in memory in parsed object form (not the original
XML) for the lifetime of the login session, and maybe beyond, to save time for the
next access of each previously-accessed UI.

As you can see I can envision a merging of VNC into the metawidget system to handle specialized UIs which cannot be represented using "common" metawidgets; and I think one of the first applications which I will develop will probably be an alternative front-end for NetteBook.

So for the last week in my spare time I have been hacking away at a widget set for this hypothetical XML metawidget system. I considered which platform I should start with, for a quick and dirty first implementation. X is so complex; I think it will take me a long time to learn how to program it (and I want to do it at a low level, like X intrinsics, not with Motif which is what most X programming books are about). If I built an x client it would be handy to be able to do development at my main Linux workstation, but for use on the touchscreens I'd still be reliant upon VNC. I researched what tools are available on my old Macs. One Mac IIci that was given to me has a copy of ThinkC on it but I'm not sure if I legally own it at this point. I discovered that Mac programs tend to use resources for all the widgets. So it would take more digging and going against the grain to figure out how to programmatically create widgets in response to XML tags. And I'd be going against the grain big-time to try doing a full-screen application with no menubar. I found no info at all on how to do that, either in ThinkC docs, examples, header files, or online.

But for DOS, Turbo C 2.01 is free now at the Borland web page so I can use it guilt-free with the idea of releasing the results as freeware if I actually finish the project. I put one of the spare 486's back together, upgraded its video card, installed Win95 (I wanted to be able to share files over the network with the Linux box, and don't know how to do that with plain DOS) and Turbo C and set it up in the bedroom. I was using a temporary CD-ROM to install Windows, sitting on top of a trash can next to the machine so I wouldn't have to actually install it inside, and the disc was vibrating a lot (it's a 40x drive, I've noticed the faster the drive, the more they tend to do that sometimes). I was going to eject it and put it back in to try and get it centered on the spindle better (assuming that's why it was off balance). When I grabbed the drive it went "scrape" and made some scrape marks on the disc, but fortunately it still worked well enough to finish the installation. What a crappy drive.

I found a freeware C XML parser also. So far I have implemented LCARS-style buttons and windows using only plain C code but trying to embody OO concepts using structures where I would normally have classes. It seems to be working. The application is up to 82k or so now. Old-fashioned C sure is nice on memory usage. There was no mouse support in that version of Turbo C's graphics library so I had to manually call interrupt 33h and get the values out of the registers afterwards. But it is working and I can click the buttons now. I built an on-screen QWERTY keyboard for entering data in text fields (but there is no text field widget yet). Next step is to get the XML parser integrated, implement a simple windows-and-buttons-only DTD and try to get it putting up some windows and buttons given an XML file. I also need to install the app on one of the laptops and make sure the DOS touchscreen driver will work with my mouse handling methods. Unfortunately I will have to do most of this work over again to use it on the Macs. I might do a more Mac-style GUI instead of LCARS-style. The whole point is that the XML description of the metawidgets should be independent of the widgets being used, so it will be a good test to use the same app on two very different GUI systems.

Anyway all this lower-level graphics hacking has been refreshing. I haven't been so addicted to a programming project in a long time. It bothers me that I have been distracted from NetteBook, and HamNet, and another web idea that might actually make some money, and my research into Xanadu and other more advanced hypertext systems but all these things will tie together. My idea of the "ultimate computer system" now goes like this - it uses a capability-based OS to manage a global virtual memory space. All objects are persistent unless marked transient, and must never be explicity saved or loaded. The language of choice is Python but others are supported. GUIs tend to be implemented with metawidgets and are as accessible to old 286 DOS machines as they are to PalmPilots or modern workstations. Objects are equally accessible regardless where in the world they are located. They get cached and replicated as necessary for speed, without losing their identity in the global address space. Legacy text is stored xanalogically. All hyperlinks are 2-way. Much of the text on the web is just a "view" of an underlying OO knowledge base. Translations to all known languages are done accurately because the computer understands the meaning of what is being said - it in fact stores the facts and can expound upon them on demand and with sensitivity to the context using appropriate idioms in any language.

BTW like most of my ideas, it turns out that metawidgets have been thought of before, in some form. Here are some links (and I have more reading to do at the library for some of the journal articles referenced in these links):

• Info about a metawidgets project at UC Davis, with significant bibliography
• CV of Meera Blattner, apparent inventor of metawidgets
• "Modality" evidently means the ability to access a UI using a choice of I/O devices
• Tangentially related stuff about sounds that give feedback about meaning of UIs

Well it was a busy weekend. Haven't got any coding done since Friday night. Saturday morning I had to get up early for a hamfest out in Mesa. I got a lot of good deals though. The Kenwood TR7625 2M transceiver with the remote head (see picture) for $60 was an especially good deal, and it actually works too. I got a 5-element Larsen beam for the 440 band, and another 2M Ringo, and half a dozen or so Mouse Systems type optical mice (none with the pads included, unfortunately), and two 900VA APS UPSs with the passive serial-port status indicator thingies, and some coax, and a bag of misc. connectors and stuff. There was a guy selling PC power cords for 25 cents. I got some memory that might turn out to be 8-meg SIMMs with parity if my guess was good; I'm hoping I can use them in the DEC Multia system (it requires matched pairs of parity SIMMs). I was hoping to find some big solar panels but there weren't any this time except for an overpriced vehicle battery trickle-charger type. Keith Justice was there and gave me a Mocom radio which is on the 439.350 frequency which is turning into another 9600 baud packet backbone. I will try it out one of these days, and pay the guy he's selling it for if it works out; that's what I got the 440 beam for. Ideally I'd get a tower put up first but that's probably going to take a lot of time.

Saturday afternoon I put up Christmas lights. I put 4 25-light strings of the large kind on the big privet bush out front, which I had gotten trimmed a couple weeks ago so it looks pretty nice. Amazing how much growing it did this year. I got a couple more X10 controllers so each string is on a separate one, and I couldn't find my script from last year so rewrote a better one to go around and set each string to a random brightness level. So the strings slowly wink on and off. X10 isn't very fast unfortunately. I also put up the usual strings of lights along the front awning (1 1/2 strings - last year I cut one to length so it ends right at the point where the awning meets the house). The Velcro makes it fairly easy... a couple years ago I put little strips of the fuzzy side of the Velcro along the underside of the facia board where each light goes, with staples and construction adhesive; and used construction adhesive to fasten strips of the "hook" side to each light. The glue is coming loose so this time I used self-stick Velcro for the ones that had come off completely. We'll see if that works better. With all those lights I pretty well used up my supply of spare bulbs and will have to stock up again the day after Christmas like I did last year. Today (Sunday) I put about 12 100-light 2-way flashing strings of mini-lights on the hedge. That looks really nice; blotches of light flash on and off at random, and I like the blotches a lot better than the straight lines of lights on the privet bush, that obviously belong to a string. I also put 3 sets on the juniper bush next to the privet, hooked up to the same controllers as the lights on the privet, so that they wink on and off too.

Saturday night there was a Homebid Christmas party at the home of the company president. I'm guessing it's at least a half-million dollar home; quite a nice place. He was showing off the wine cellar, which is accessible via a hidden stairwell behind a bookcase that slides away to one side. It's air-conditioned to a constant 58 degrees. The guest house is almost as big as most people's houses. There is a pool, and a basketball/volleyball court, and a hottub, and a trampoline built in at ground-level so you don't have to climb up onto it. The yard is so deeply sculpted and irregular with perfect golf-course grass it's obvious they are having it professionally maintained. But the whole neighborhood's full of fancy houses like that. Anyway it was the nicest party I've ever been to I think. The food was catered, and very gourmet. There must have been half a dozen caterer people - two for the bar, two running around with trays of horsdeouvres, a guy cutting ham, and probably a cook or two. They parked a moving-van type mobile kitchen truck next to the garage and did all the preparation in there.

Tonight I finished making turkey soup with the leftover turkey from Thanksgiving. I made homemade noodles. It's not bad.

Last weekend I worked on the sliding steel door project some more. I made what I hope will be an air cylinder to power it; simply a larger diameter steel pipe (1 1/4" I think) in which a piston will go back and forth, and a small pipe running along it to the end so that I can pump air into that end. The other end will be exposed, sticking out of the edge of the door, so I can pump air into that end also. It's a double-action cylinder to either open or close the door. I still need to make the piston, and find a really polished stainless steel push rod, and make a cover for the exposed end of the pipe that will allow the push rod to stick out and will seal it well. I think a custom rubber boot would be just the thing but I don't know where I could get one. I also will need to either get a lathe (a decent idea anyway) or find someone to make the piston. Could also make a boot that way out of nylon, or could use steel but then I'd have to put o-rings in to seal it where it rubs against the push rod. Well I was proud of myself for figuring out how to attach the smaller pipe to the large one, so I can blow air in that end, and sealing it up without sealing off the orifice that lets in the air (which took several tries - I kept welding, grinding it down, testing for leaks with water, finding some pinhole ones and doing it again); but then I did something really stupid - I installed it in the door and then afterwards realized I had to cut out some holes in the top of the door for access to the nuts that will hold it to the bolts where it hangs from the "trucks" or "dogs" or whatever you call those little 4-wheeled barn door hanger things. I ended up cutting into the cylinder, and had to weld it back up. I'm sure the inside of the cylinder is not so perfectly round in that area now, and will leak air past the piston - "blowby" as it were. Question is whether it will leak enough to really matter; at 90PSI there will probably be an excess of power anyway, just to push the door back and forth. Well, lessee, if it is 1 1/4" pipe, pi r ^ 2 yields 1.22 square inches, so I could expect 110 pounds of force if there were no leaks. We'll see. I lightly welded the cylinder in place so I could easily remove it and make another one if necessary.

I also built a locking mechanism consisting of a screw drive that pushes a 1/2" steel rod in and out. It's going to be installed in the forward edge of the bottom of the door, and to lock it will push the rod into a hole in the floor. I bought a power screwdriver and used the motor and gearhead with a screw and nut to push the rod in and out. It's not pretty but it works. It takes a longer time than I expected to fully eject or retract the shaft so I won't want to use it frequently; probably just have the security system lock it automatically when I'm gone and unlock when I get home so I don't have to wait for it when I'm in a hurry to get the door open. I found some chintzy little microswitches at the hamfest this weekend which I could use to detect end-of-travel on this locking mechanism but I have doubts they would last well enough. There will have to be defeat mechanisms of course, can't get myself into a situation where the door won't unlock. The lock will be physically removable from the inside of the room, and there will be hidden electrical override switches to power the lock manually, ignoring the microswitches and whatever electronic controls I end up using.

Last weekend while going to get the steel pipe and stuff at Davis Salvage, I got the truck heated up enough to confirm that it really does have a radiator leak, it's not just the overflow. Another project.

These newsletter things take too long. No wonder I don't do it very often.