Continuing from Compiling a kernel I wanted to boot the existing X-DSL installation with my custom kernel. As noted in the concluding sections of that article, the kernel did boot but wouldn't find the hard disk. Examining the kernel messages on the screen I noticed devfs being mounted slightly before the stall. Comparing the output with the stock kernel I found that this was devfs was not used before.
In the beginning (before hot-plugging, when all computer peripherals were screwed together and everything was static) /dev devices all used static numbers and were present no matter if the hardware device they identified existed or not. devfs was an early attempt at hardware autodetection which created and removed device files while the system was running. It was later replaced by udev which is still used in modern Linux distributions.
This hardware autodetection includes probing hard disks for partitions. However, the Xbox doesn't have a partition table! All partitions are located at static offsets stored in the kernel. The existence of a partition or not is indicated by a magic identifier string (BRFR) at the static offsets. My guess is that devfs removes all the static block devices (which are included in the X-DSL initrd corresponding to the hard disk partitions when it is mounted on top of /dev. It is hard to say for sure though, because when the partition identification failed I was dropped into a very limited Almquist shell which of course didn't have ls (since this is before init is started so there can only be a single process executing on the system) and didn't even seem to do shell globbing.
Researching the issue I ended up at a TLDP article that explained devfs and how to enable/disable it. It is controlled by the kernel config parameters CONFIG_DEVFS_FS (whether to build it) and CONFIG_DEVFS_MOUNT (whether to mount on boot). Quickest way to disable it on boot is through the kernel cmdline parameter devfs=nomount.
With this change the system booted fully, and I finally had a kernel build environment where I could start to make changes to the Xbox-Linux kernel. So in my first patch against the Xbox-Linux kernel I will disable devfs.
Eventually I might collect all my Xbox-Linux patches somewhere, but for now this minor kernel config patch is attached to this article: disable-devfs.patch