I wanted to put together a low-power home automation system, with a fixed set of tasks, with read-only rootfs such that if anything goes wrong it's OK to just hit reset. (Or if the power goes out it will reboot drama-free by itself.) For low-power hardware I wanted to make use of an old 233MHz Cyrix MediaGX system which has not had much use until now. With only the motherboard, an AT power supply modified with a Noctua fan, PCI USB and S3 Virge boards, and a 2 GB CF card for the "hard drive", it takes in the range of 20-25 watts... not bad compared to around 100 watts for a more modern PC. A Via EPIA or some such would work just as well if that's what you happen to have lying around. A SheevaPlug might even be enough, actually. My EPIA board died after only a year or two of use, due to bad caps, so I'd just as soon re-use old junk that still works rather than "investing" in a replacement.
So... given that it's not quite a 686-compatible, we need a Linux distro which doesn't make such assumptions about the hardware (increasingly rare). Debian 5.0 still runs on a 486. (I'm normally a gentoo user but not too keen on doing so much building on such a slow machine. And have tried building a custom distro with openembedded too, but bleh... it's too much work, and not every package that I want is available.) So I installed with default options onto the CF card, then went about modifying it to work even if I mount the root filesystem read-only.
grub is already configured "ro" because Debian still mounts the root filesystem ro first, then does fsck then mounts rw again, even with modern journaling filesystems (go figure). It could be there's a good reason for that, but I think to the extent that there are no bugs in ext3 (heh) it should be able to deal with being mounted read-write from the start. At least, reiserfs always could... but ext3 will give you a warning after a while that fsck is recommended because it's been too many days (oh ye of little faith). I think I will just deal with that manually since it's going to be mounted read-only most of the time. This is the kind of system that's going to (in theory) go for years at a time without significant modifications.
So I added a "maintenance mode" entry, and turned off swap on the default entry (/boot/grub/menu.lst):
title Debian GNU/Linux, kernel 2.6.26-2-486 root (hd0,0) kernel /boot/vmlinuz-2.6.26-2-486 root=/dev/hda1 ro noswap initrd /boot/initrd.img-2.6.26-2-486 title Debian GNU/Linux, kernel 2.6.26-2-486 read/write root (hd0,0) kernel /boot/vmlinuz-2.6.26-2-486 root=/dev/hda1 rw initrd /boot/initrd.img-2.6.26-2-486
Then we have to stop it from fscking and remounting read-write, which is simply a matter of removing any symlinks that point to checkroot.sh from /etc/rc*.
In /etc/fstab there are some tmpfs filesystems:
proc /proc proc defaults 0 0 /dev/hda1 / ext3 errors=remount-ro 0 1 /dev/hda5 none swap sw 0 0 tmpfs /tmp tmpfs rw 0 0 tmpfs /etc/network/run tmpfs rw 0 0
/var needs to be tmpfs too, but that gets tricky, because /var normally holds more stuff than I could fit in 64 megs of RAM (most of the big stuff isn't really "variable"). So I made /var-ro and /var-tmp, and moved stuff from /var which I wanted to be read-only (in normal mode) into /var-ro, and at boot it mounts /var as tmpfs then copies everything from /var-tmp into there... so we want only the bare directory structures in /var-tmp, as well as symlinks to the stuff that is actually in /var-ro.
[earll][08:58:25 PM] du -sh /var-ro/* 8.0K /var-ro/backups 162M /var-ro/cache 52M /var-ro/lib 236K /var-ro/run 8.0K /var-ro/www [earll][08:58:27 PM] ll /var-tmp/ total 40 drwxr-xr-x 10 root root 4096 Jul 24 20:11 . drwxr-xr-x 23 root root 4096 Jul 24 19:41 .. lrwxrwxrwx 1 root root 15 Jul 24 19:53 backups -> /var-ro/backups lrwxrwxrwx 1 root root 13 Jul 24 19:40 cache -> /var-ro/cache lrwxrwxrwx 1 root root 11 Jul 24 19:41 lib -> /var-ro/lib drwxrwsr-x 2 root staff 4096 Jun 18 15:17 local drwxrwxrwt 3 root root 4096 Jul 24 19:13 lock drwxr-xr-x 9 root root 4096 Jul 24 20:11 log drwxrwsr-x 2 root mail 4096 Nov 6 2005 mail drwxr-xr-x 2 root root 4096 Nov 6 2005 opt drwxr-xr-x 6 root root 4096 Jul 24 19:55 run drwxr-xr-x 6 root root 4096 Nov 6 2005 spool drwxrwxrwt 2 root root 4096 Jun 18 15:17 tmp lrwxrwxrwx 1 root root 11 Jul 24 19:40 www -> /var-ro/www [earll][08:58:53 PM] find /var-tmp /var-tmp /var-tmp/mail /var-tmp/www /var-tmp/local /var-tmp/cache /var-tmp/opt /var-tmp/backups /var-tmp/lib /var-tmp/run /var-tmp/run/apache2 /var-tmp/run/portmap_mapping /var-tmp/run/exim4 /var-tmp/run/dbus /var-tmp/run/utmp /var-tmp/run/network /var-tmp/run/motd /var-tmp/spool /var-tmp/spool/mail /var-tmp/spool/exim4 /var-tmp/spool/exim4/input /var-tmp/spool/exim4/db /var-tmp/spool/exim4/gnutls-params /var-tmp/spool/exim4/msglog /var-tmp/spool/samba /var-tmp/spool/cups /var-tmp/spool/cups/tmp /var-tmp/spool/cron /var-tmp/spool/cron/crontabs /var-tmp/spool/cron/atjobs /var-tmp/spool/cron/atjobs/.SEQ /var-tmp/spool/cron/atspool /var-tmp/tmp /var-tmp/log /var-tmp/log/installer /var-tmp/log/installer/cdebconf /var-tmp/log/apache2 /var-tmp/log/exim4 /var-tmp/log/samba /var-tmp/log/samba/cores /var-tmp/log/samba/cores/smbd /var-tmp/log/samba/cores/winbindd /var-tmp/log/samba/cores/nmbd /var-tmp/log/news /var-tmp/log/fsck /var-tmp/log/apt /var-tmp/lock /var-tmp/lock/apache2
And in /etc/init.d/mountkernfs.sh, we do the copying into the tmpfs, right after the part where it optionally mounts /var/run and /var/lock as tmpfs (we don't enable that option since all of /var is basically tmpfs):
# Mount /var as tmpfs and copy some initial stuff into there domount tmpfs "" /var var -omode=0777 cp -a /var-tmp/* /var
And we need to limit the logging since there's only 64M total... no point in filling it up until there's no RAM left for running processes:
/etc/mtab can be replaced with a link to /proc/mounts. /etc/adjtime can be replaced with a link to /tmp/adjtime.
That's about it... it boots with only a couple of complaints about "read-only file system", e.g. "cannot remove /var/lib/exim4/config.autogenerated.tmp" (which is not actually there anyhow), etc.
If I want to change files which are normally read-only, I can either reboot into maintenance mode or just
mount / -o remount,rw
then reboot again afterwards.