pages tagged xbox-linuxAnton Eliassonhttp://www.antoneliasson.se//tags/xbox-linux/Anton Eliassonikiwiki2019-03-20T21:48:15ZXbox-Linux: Booting with nfsroot (rootfs on NFS)http://www.antoneliasson.se//journal/xbox-linux-nfs-root/2019-03-20T21:48:15Z2016-08-18T18:15:00Z
<p>To expedite the development of <a href="http://www.antoneliasson.se//journal/reviving-xbox-linux-15-years-later/">my contemplated Xbox rsync file server Linux distribution</a>, I would like the ability to mount the root filesystem over the network using NFS. The kernel requirements for is documented in <code>Documentation/filesystems/nfs/nfsroot.txt</code> in the kernel source tree. nfsroot was slightly less featureful in 2.4 than in the current kernel, so make sure to get the old version if you are coding along. Also, if you <em>are</em> coding along then please do get in touch so I will know that I'm not the only person in the world to program for the classic Xbox in 2016.</p>
<hr />
<p>Continuing from <a href="http://www.antoneliasson.se//journal/xbox-linux-booting-xdsl-custom-kernel/">Booting X-DSL with a custom kernel</a> where we disabled devfsd via kernel config patch, a suitable <code>linuxboot.cfg</code> for nfsroot looks something like this:</p>
<pre><code>title XDSL unstable 2.4.32 NFS root
kernel KNOPPIX/bzImage.32
append root=/dev/nfs nfsroot=/srv/xbox-linux ip=192.168.1.188:192.168.1.21:::::off init=/etc/init rw video=xbox:640x480 kbd-reset
</code></pre>
<p>Note that no initrd should be necessary. 192.168.1.188 is the client (Xbox) IP address and 192.168.1.21 is the NFS server IP address. I could have used more DHCP (the kernel built in this article supports it) but then I would also have to reconfigure my DHCP server, and that seemed like <em>a lot of work</em> for this toy project.</p>
<p>Start by opening <code>fs/nfs/nfsroot.c</code> and replacing</p>
<pre><code>#undef NFSROOT_DEBUG
</code></pre>
<p>with</p>
<pre><code>#define NFSROOT_DEBUG
</code></pre>
<p>to get some feedback from the NFS mounting process. Otherwise you will just get the uninformative message of</p>
<pre><code>VFS: Cannot open root device "nfs" or 00:ff
Please append a correct "root=" boot option
Kernel panic: VFS: Unable to mount root fs on 00:ff
</code></pre>
<p>which tells you that /dev/nfs, which was specified in the <code>root</code> kernel command line parameter, was not set up correctly.</p>
<h2>KCONFIG time</h2>
<p>I will present several patches in this section. To keep track of them I recommend using the program <em>quilt</em> as I wrote about <a href="http://www.antoneliasson.se//journal/xbox-linux-patch-management-quilt/">in a separate article</a>.</p>
<p>Referring to <code>Documentation/filesystems/nfs/nfsroot.txt</code>, the first thing to enable is NFS support and IP autoconfiguration. Apply <a href="http://www.antoneliasson.se//journal/xbox-linux-nfs-root/enable-nfsroot.patch">enable-nfsroot.patch</a>, <code>make bzImage</code>, transfer the kernel image and boot.</p>
<p>This should fail because there are no network adapters available. The reason for this is that the Ethernet driver is built as a module instead of being included inside the kernel image. For a special purpose Linux distribution like X-DSL or Xebian that will basically never be run on anything other than an actual Xbox, there is no gain in having essential drivers in modules.</p>
<p>Apply <a href="http://www.antoneliasson.se//journal/xbox-linux-nfs-root/enable-builtin-nforce-driver.patch">enable-builtin-nforce-driver.patch</a> and try again. Now NFS will actually fire up and try to mount the root filesystem. There is confusion between the client and server however, resulting in errors being printed in the kernel console. Examining the situation closer using Wireshark revealed that the ancient Linux 2.4 kernel tries to use NFSv2 instead of version 3 or 4 which are more common today. The NFS server on my desktop computer (Linux 4.4) claims to still support NFSv2, but it doesn't seem to work very well. If the client requests a non-existing file, the server appears to respond with an "unsupported operation" which seems strange to me.</p>
<p>Fortunately I won't have to downgrade my desktop to Linux 2.4, since the kernel does support NFSv3, it's just not the default (yet). To enable support for it, apply <a href="http://www.antoneliasson.se//journal/xbox-linux-nfs-root/enable-nfsv3.patch">enable-nfsv3.patch</a> and rebuild. Also add <code>v3</code> as an NFS mounting option in <code>linuxboot.cfg</code> like so:</p>
<pre><code>append root=/dev/nfs nfsroot=/srv/xbox-linux,v3 ip=...
</code></pre>
<p>If you are using quilt, your <code>series</code> file should at this point look like this:</p>
<pre><code>disable-devfs.patch
enable-nfsroot.patch
enable-builtin-nforce-driver.patch
enable-nfsv3.patch
</code></pre>
<h2>(Partial) Success</h2>
<p>Finally, the root filesystem is mounted and the system boots! Almost... It fails during fsck because the device backing the root filesystem (normally /dev/loop0) is not found. Disabling fsck in /etc/fstab fixes this and allows the boot procedure to continue and eventually land in a proper root shell.</p>
<p>However, the system will not shut down cleanly. At this part it becomes clear that DSL is maybe not a very solid foundation to build a specialized operating system on. The major part of X-DSL's shutdown process is implemented by a gigantic pile of shellscript in /etc/init.d/knoppix-halt. As part of this procedure, the network interfaces are disabled before the filesystems are unmounted. It should in theory unmount network filesystems first but, being a gigantic pile of shellscript, it understandably isn't completely reliable. I guess this is what they call a "minimalistic init system".</p>
<p>I could try to reorder lines in this file, but that would create a maintenance burden that I'm not very interested in. I might come back to this later, but first I should investigate Xebian, a Debian port for the classic Xbox. The reason I tried X-DSL before Xebian is that it seemed to have been maintained for longer. Xebian basically had one major release with some minor updates before its developers moved on. Still, being based on Debian I expect it to be fairly versatile and solid.</p>
Xbox-Linux: Creating a PC-BIOS/Xbox hybrid bootable ISO imagehttp://www.antoneliasson.se//journal/xbox-linux-pc-bios-xbox-hybrid-bootable-iso/2019-03-20T21:48:15Z2016-08-18T17:00:00Z
<p>I will skip right to the point: The magic command for creating an ISO9660 image for a disc that is bootable both on a PC BIOS and on a modded Xbox goes something like this:</p>
<pre><code>$ mkisofs -udf -r -joliet -eltorito-catalog isolinux/boot.cat -boot-info-table -eltorito-boot isolinux/isolinux.bin -no-emul-boot -boot-load-size 4 -o basic.iso basic/
</code></pre>
<h2>Explanations follow</h2>
<hr />
<ul>
<li><code>-udf</code> creates a UDF filesystem in addition to the standard ISO9660. Apparently this is required for the Xbox to recognize the disc as bootable.</li>
<li><code>-r</code> enables the creation of Rock Ridge file metadata. This allows for longer filenames, permissions and other improvements in POSIX systems. Probably used once the Linux kernel is booted from the disc. Unlike the <code>-rock</code> parameter, <code>-r</code> overrides some of the metadata with default values that make more sense for removable media.</li>
<li><code>-joliet</code> enables the creation of Joliet file metadata. Same purpose as Rock Ridge but for Windows based systems. Not sure if strictly required here (but it seems related to UDF according to genisoimage's man page).</li>
<li><code>-eltorito-catalog</code> specifies the location of the "boot catalog" that is supposedly ignored by everything but is still required according to the spec. Note that isolinux/boot.cat specifies a file to be created inside the image file. It does not (need to) exist in the source directory basic/.</li>
<li><code>-eltorito-boot</code> specifies a boot image to be embedded in the image. This is the path to an existing file inside the CD root directory specified last on the command line. The boot image is an executable file to be run without an operating system, typically a boot loader. As I understand it this is comparable to the code inside the MBR of a hard drive.</li>
<li><code>-boot-info-table</code> specifies that the "partition table" inside the boot image should indeed be updated with the actual CD layout before embedding it in the image file. Note that this makes permanent changes to the source file.</li>
<li><code>-no-emul-boot</code> specifies that the boot image should be run "as is". Default is to emulate a floppy drive, in which case the boot image must be an actual floppy image.</li>
<li><code>-boot-load-size</code> specifies the size of the boot image in fake "sectors" (512 byte). This should typically be the size of a CD sector, which is 2048 bytes in "mode 1". If this is smaller than the boot image (as in this case) it is truncated, which ISOLINUX is designed to handle. If unspecified the entire image is embedded.</li>
<li>Finally <code>-o</code> specifies the filename to write and basic/ is the directory which will become the CD root directory in the image.</li>
</ul>
<h2>Further details</h2>
<p>The ISO9660 format, due to being very old, is fairly limited in its vanilla shape and has therefore been extended several times during the years. That's why there are now four ways of encoding the filename and other metadata on a CD (ISO9660, Rock Ridge Interchange Protocol, Joliet and UDF).</p>
<p>Vanilla ISO9660 supports filenames of up to 31 characters, and the allowed list of characters is short. But for compatibility with DOS, typically only 8+3 characters are allowed in implementations (like genisoimage/mkisofs). <code>-l</code> unlocks the full length of 31 chars, but I'm not sure if this is supported by the Xbox BIOSes.</p>
<p>As eluded to earlier, <code>-r</code> overrides some of the file metadata with sensible values. Examples are squashing uid/gid to 0 (since the user who inserted the disc is usually treated as the "owner" of its contents), removing write permissions for everyone (read-only media), and allowing read access to both user/group/others (CD's are not really multi-user systems).</p>
<p>El Torito is the name of the ISO9660 extension for bootable optical media. This is used by all PC's in BIOS mode. (U)EFI was just a baby when the Xbox was new, works differently and is outside the scope of this article.</p>
<p>The Xbox BIOS does not consider El Torito however. It simply looks for a <code>default.xbe</code> file and executes it if found (and the disc is valid, unless those checks have been defeated like in a modded Xbox). The gotcha is that the Xbox original BIOS (and hacked versions thereof) only supports UDF, none of the other file metadata formats. It makes some sense since games are normally only stored on DVD's, and DVD's typically (but not always) use UDF.</p>
<p>The free BIOS Cromwell however, does support one or more of the other formats (I haven't checked which). And as you may know it does not work with XBE's at all, instead Cromwell is typically used to directly load a Linux kernel as specified in the <code>linuxboot.cfg</code> text file.</p>
<h2>Test case</h2>
<p>This was tested using Ed's Xebian 1.1.4 basic edition, available from the <a href="https://sourceforge.net/projects/xbox-linux">Xbox-Linux Sourceforge project page</a>.</p>
<h2>References</h2>
<p>See messages <a href="http://www.antoneliasson.se//journal/xbox-linux-pc-bios-xbox-hybrid-bootable-iso/20040714-Re_dual_boot_iso-4805.eml">20040714-Re dual boot iso-4805.eml</a> and <a href="http://www.antoneliasson.se//journal/xbox-linux-pc-bios-xbox-hybrid-bootable-iso/20040714-Re_dual_boot_iso-4807.eml">20040714-Re dual boot iso-4807.eml</a> on the xbox-linux-user mailing list.</p>
Xbox-Linux interlude: Patch management with quilthttp://www.antoneliasson.se//journal/xbox-linux-patch-management-quilt/2019-03-20T21:48:15Z2016-08-13T15:00:00Z
<p>Before continuing the series on Xbox-Linux, I want to write down how I currently manage my local kernel patches. While it is possible to use git and its rebase functionality for this purpose, it is not optimal. The way I currently work is to start with a vanilla Linux kernel tarball, extract it and apply the <em>big</em> Xbox-Linux patch. I then make changes to this tree, one change per commit/patch as a disciplined VCS user. Eventually I expect to want to update the Linux kernel by fetching a new tarball and applying all patches again. There is a tool which was created for exactly this workflow, and it's called <a href="https://web.archive.org/web/20100331173400/http://www.suse.de/%7Eagruen/quilt.pdf">quilt</a>. It is commonly used in Debian to keep track of downstream changes to deb packages. Unlike git, it's been around forever so I can even use it on my ancient Debian 3.1 building VM.</p>
<hr />
<p>quilt requires a slightly different workflow than a modern version control system. Instead of making changes first and committing them later, you must first declare a patch name and the files to patch before making changes. This is because by design there is no local repository. Patches are stored in the subdirectory <em>patches/</em> along with a <em>series</em> file that declare the order the patches should be applied. quilt also stores some bookkeeping information in the <em>.pc/</em> subdirectory.</p>
<p>To start a new patch while in the source tree top dir, do</p>
<pre><code>$ quilt new change_name.patch
</code></pre>
<p>To declare a file to be included in the patch, do</p>
<pre><code>$ quilt add path/to/file
</code></pre>
<p>Then make changes. To update the current patch with the change, do</p>
<pre><code>$ quilt refresh
</code></pre>
<p>That's it! <code>quilt refresh</code> can be run multiple times to update the current patch. To make another change, just execute <code>quilt new another_change.patch</code> and start over.</p>
<p>To apply the patch set to a different tree, simply copy the patches directory to the tree. To apply the patches in order, one at a time, do</p>
<pre><code>$ quilt push
</code></pre>
<p>repeatedly. Or <code>quilt push -a</code> to apply all patches listed in the series file. Likewise, patches can be unapplied by <code>pop</code>ing them.</p>
<p>Patches can have text headers which describe their functional changes. To edit the header of the latest patch, execute</p>
<pre><code>$ quilt header -e
</code></pre>
<p>The old version of quilt that is packaged for Debian 3.1 doesn't have this option though, but it's easy enough to add a patch header by hand. There are more options to quilt, but these are enough for its basic functionality.</p>
Xbox-Linux: Booting X-DSL with a custom kernelhttp://www.antoneliasson.se//journal/xbox-linux-booting-xdsl-custom-kernel/2019-03-20T21:48:15Z2016-08-13T14:35:00Z
<p>Continuing from <a href="http://www.antoneliasson.se//journal/xbox-linux-compiling-kernel/">Compiling a kernel</a> I wanted to boot the existing X-DSL installation with my custom kernel. As noted in the concluding sections of that article, the kernel did boot but wouldn't find the hard disk. Examining the kernel messages on the screen I noticed <em>devfs</em> being mounted slightly before the stall. Comparing the output with the stock kernel I found that this was devfs was not used before.</p>
<p>In the beginning (before hot-plugging, when all computer peripherals were screwed together and everything was static) /dev devices all used static numbers and were present no matter if the hardware device they identified existed or not. devfs was an early attempt at hardware autodetection which created and removed device files while the system was running. It was later replaced by udev which is still used in modern Linux distributions.</p>
<p>This hardware autodetection includes probing hard disks for partitions. However, the Xbox doesn't have a partition table! All partitions are located at static offsets stored in the kernel. The existence of a partition or not is indicated by a <a href="https://web.archive.org/web/20080406052808/http://www.xbox-linux.org/docs/partidea.html">magic identifier string (BRFR) at the static offsets</a>. My guess is that devfs removes all the static block devices (which are <a href="https://web.archive.org/web/20090709115900/http://www.x-dsl.org/wiki/Differences_between_XDSL_and_DSL">included in the X-DSL initrd</a> corresponding to the hard disk partitions when it is mounted on top of /dev. It is hard to say for sure though, because when the partition identification failed I was dropped into a <em>very</em> limited <a href="https://en.wikipedia.org/wiki/Almquist_shell">Almquist shell</a> which of course didn't have <em>ls</em> (since this is before <em>init</em> is started so there can only be a single process executing on the system) and didn't even seem to do shell globbing.</p>
<p>Researching the issue I ended up at <a href="http://www.tldp.org/HOWTO/SCSI-2.4-HOWTO/devfs.html">a TLDP article</a> that explained devfs and how to enable/disable it. It is controlled by the kernel config parameters CONFIG_DEVFS_FS (whether to build it) and CONFIG_DEVFS_MOUNT (whether to mount on boot). Quickest way to disable it on boot is through the kernel cmdline parameter <em>devfs=nomount</em>.</p>
<p>With this change the system booted fully, and I finally had a kernel build environment where I could start to make changes to the Xbox-Linux kernel. So in my first patch against the Xbox-Linux kernel I will disable devfs.</p>
<p>Eventually I might collect all my Xbox-Linux patches somewhere, but for now this minor kernel config patch is attached to this article:</p>
<hr />
<div class="highlight-patch"><pre class="hl">
Index: linux-2.4.32/.config
===================================================================
<span class="hl kwb">--- linux-2.4.32.orig/.config 2016-08-11 09:59:13.000000000 +0200</span>
<span class="hl kwa">+++ linux-2.4.32/.config 2016-08-11 10:00:18.000000000 +0200</span>
<span class="hl kwd">@@ -692,8 +692,8 @@</span>
# CONFIG_NTFS_RW is not set
# CONFIG_HPFS_FS is not set
CONFIG_PROC_FS=y
<span class="hl kwb">-CONFIG_DEVFS_FS=y</span>
<span class="hl kwb">-CONFIG_DEVFS_MOUNT=y</span>
<span class="hl kwa">+# CONFIG_DEVFS_FS is not set</span>
<span class="hl kwa">+# CONFIG_DEVFS_MOUNT is not set</span>
# CONFIG_DEVFS_DEBUG is not set
CONFIG_DEVPTS_FS=y
# CONFIG_QNX4FS_FS is not set
</pre></div>
<p><a href="http://www.antoneliasson.se//journal/xbox-linux-booting-xdsl-custom-kernel/disable-devfs.patch">View raw file</a></p>
<hr />
Xbox-Linux rsync file server: Compiling a kernelhttp://www.antoneliasson.se//journal/xbox-linux-compiling-kernel/2019-03-20T21:48:15Z2016-08-12T10:15:00Z
<p>This is a continuation of <a href="http://www.antoneliasson.se//journal/reviving-xbox-linux-15-years-later/">Reviving Xbox-Linux, 15 years later</a>. Refer to that article for an introduction.</p>
<p>As a first step I'm going to try to compile and hopefully boot the latest available Xbox-Linux kernel (2.4.32). The Xbox-Linux <a href="https://web.archive.org/web/20100617014359/http://www.xbox-linux.org/wiki/Making_a_Linux_distribution_Xbox_compatible_HOWTO">page about porting existing Linux distributions to the Xbox</a> suggests starting by making the kernel mount the root filesystem over NFS, as this will make it easier to make changes to it. While I'm not sure these instructions were ever tested, it does sound like a good idea.</p>
<hr />
<p>The guide also suggests using Etherboot in the Cromwell BIOS to load the kernel over the network. Somebody on the xbox-linux-user mailing list posted instructions for building an Etherboot compatible image from a kernel and an initrd. Gmane has shut down operations so instead of linking I'm <a href="http://www.antoneliasson.se//journal/xbox-linux-compiling-kernel/20041208-Re_etherboot_in_cromwell-6354.eml">re-hosting that e-mail in its entirety here</a>. I tried these instructions initially but never got it to work. Some web searching revealed that <a href="http://forum.linuxmce.org/index.php?topic=1791">csudcy at the LinuxMCE forums tried the same thing in 2007</a>. I basically tried the same steps and got the same results that he did so I won't bother documenting it further. <strong>Suffice to say that Etherboot is broken in the latest and last Cromwell (2.41-dev).</strong> Maybe it never worked. For now I'll use the built-in FTP server in the UnleashX dashboard that my Xbox runs on startup to transfer my homemade kernels to the hard drive.</p>
<h2>Building ancient Linux kernels</h2>
<p>There is a decent <a href="http://xbox-linux.cvs.sourceforge.net/viewvc/xbox-linux/kernel/Documentation/README.xbox?view=markup">README in Xbox-Linux's CVS repository on Sourceforge</a> that walks one through the build process of an Xbox compatible Linux kernel.</p>
<p>The first problem is that modern desktops run 64-bit CPU's and operating systems, which didn't exist in 2001 (at least not the amd64 architecture). An <a href="https://askubuntu.com/questions/27148/compile-32bit-kernel-on-64bit-machine">Ask Ubuntu</a> user suggests using sbuild to create a 32-bit chroot and build the kernel in that. I created an Ubuntu 16.04 32-bit chroot and attempted a kernel build. It failed with <code>error: array type has incomplete element type 'struct foo_bar'</code> on a file that it didn't look like the Xbox-Linux patch touched. I figure this is because the current version of gcc (5.3.1) enforces constraints in the C programming language that weren't around when the kernel code was written. The easiest solution is likely to get an older compiler.</p>
<p>In 2001-2004 gcc-3.x was the latest and greatest version. So let's try a Linux distribution that bundles gcc-3.3 or maybe 3.4. This would have to be Ubuntu 5.04 or Debian 3.1 or earlier.</p>
<p><code>mk-sbuild</code> can only install Ubuntu releases that are currently officially supported (>=12.04). It can install ancient Debian releases, but I found that <code>debootstrap</code> tries to use options that the archaic APT doesn't yet understand (<code>--no-install-recommends</code>).</p>
<p>I was hoping to not have to resort to a virtual machine for this adventure, but right now it seems to be the easiest option. http://cdimage.debian.org/mirror/cdimage/archive/ still serves historical Debian installation ISO images and http://archive.debian.org/debian/dists/ carries binary packages for posterity. Hurray for Debian! So I got the netinstall image for Debian 3.1 i386 and fired it up in VirtualBox.</p>
<h3>Installing a prehistoric Debian</h3>
<p>First it wouldn't find the HDD because VirtualBox defaults to creating a SATA controller for the hard drives. SATA did actually exist when Debian Sarge was new, but Linux support for it came first with kernel 2.6.19 and Sarge bundles 2.4.27. <a href="https://en.wikipedia.org/wiki/Serial_ATA">ref</a></p>
<p>I also tried enabling multiple CPU cores, and while Debian did boot, only one CPU was detected. Guess when Symmetric Multiprocessing support was added to Linux? <a href="https://www.ibm.com/developerworks/library/l-linux-smp/">Kernel 2.6</a>. Doesn't matter, my 2012 CPU is blazing fast with 2005 standards at single core applications as well.</p>
<p>Eventually the virtual machine booted properly and I was able to install Debian. Since most Debian mirrors drop the packages for unsupported releases I had some trouble configuring APT. I resorted to setting it up with only the installation CD as a package repository so that the installer would continue, and manually added archive.debian.org to /etc/apt/sources.list after the installation.</p>
<h3>This just in: Linux-2.4.32</h3>
<p>Finally I was able to follow the build instructions in <a href="http://xbox-linux.cvs.sourceforge.net/viewvc/xbox-linux/kernel/Documentation/README.xbox?view=markup">README.xbox</a>. I downloaded linux-2.4.32 from kernel.org and the corresponding patch from the Xbox-Linux project page on Sourceforge. The README mentions that after the bundled kernel.config has been copied to .config, one must run <code>make oldconfig</code> to "cause some actions to be made by the makefile". <strong>I later found that this has to be done every time the kernel configuration has been changed though <code>make menuconfig</code>, otherwise there would be linker errors during <code>make bzImage</code>.</strong></p>
<p>Since this is a throwaway build machine, I opted to <code>make modules_install</code> as root after the compilation to collect all modules into /lib/modules/2.4.32-xbox that I could then tar up and transfer to the Xbox hard drive.</p>
<h3>Booting the kernel</h3>
<p>Well, it didn't work. To be precise, the kernel did boot, but the init script in the initrd wouldn't find the hard disk. I still consider this part of the project a success, since I was eventually able to compile a bootable kernel image. <a href="http://www.antoneliasson.se//journal/xbox-linux-booting-xdsl-custom-kernel/">In the next segment I'll investigate why it wouldn't find the hard drive</a>.</p>
<h2>Misc resources</h2>
<p>While researching Xbox-Linux and related projects I found some articles that were interesting if not immediately relevant to my projects.</p>
<ul>
<li><a href="https://web.archive.org/web/20100708015200/http://www.xbox-linux.org/wiki/Porting_an_Operating_System_to_the_Xbox_HOWTO">Porting_an_Operating_System_to_the_Xbox_HOWTO</a> documents peculiarities of the Xbox hardware that an operating system must take into consideration.</li>
<li><a href="https://web.archive.org/web/20111021083253/http://tobias.schroepf.de/doku/doku.php?id=xbox:porting_openbsd_to_the_xbox">An enthusiastic BSD user ported parts of OpenBSD to the Xbox</a></li>
<li><a href="https://web.archive.org/web/20100617001210/http://www.xbox-linux.org/wiki/Contents">Xbox-Linux wiki Contents</a> has a lot of useful links.</li>
</ul>
Reviving Xbox-Linux, 15 years laterhttp://www.antoneliasson.se//journal/reviving-xbox-linux-15-years-later/2019-03-20T21:48:15Z2016-08-12T09:30:00Z
<p>I've recently acquired a fondness for the original Xbox from 2001, henceforth known as the Xbox Classic. It is very similar to an Intel Pentium III based PC with an Nvidia GPU. It is also a game console, which means that its hardware is (or was) completely static (with a few well-documented exceptions between the different versions) so you can be sure that even low-level programs written for one Xbox will run on another. The Xbox's security features were thoroughly broken soon after its release, so there are many ways of getting unsigned code to run on it. Further, even though I insist on calling it a classic console, it is usually not yet regarded as a retro console so there are lots of tiny things to improve on it for the keen hacker. All of these reasons tempt me to try to develop some toy programs for the Xbox Classic.</p>
<hr />
<h2>File transfer protocol(s)</h2>
<p>A practical example that I have in mind is a file server. Not for actually storing generic files, but for easily transferring Xbox games to and from it. The Xbox scene settled on FTP for this, but FTP is a terrible protocol. See <a href="http://mywiki.wooledge.org/FtpMustDie">FTP Must Die</a> for a whole list of its shortcomings.</p>
<p>Fast local area networks that were around even in the early 2000's hide some of FTP's deficiencies. For example, setting up a new TCP connection for every file isn't that costly when the network latency is less than a millisecond. And when everything runs at the same speed on an unsaturated network there is next to no packet loss. But fast forward 10 years and gigabit speed network equipment is getting into homes. So now we have mixed-speed networks and FTP's problems are getting noticeable again because it now needs TCP to do flow control.</p>
<p>It seems to me that the TCP stack in the Microsoft XDK (pirated versions of which have been used to compile the popular dashboards and utility programs for the Xbox) maybe isn't as good as it could have been, as it doesn't seem to handle lost packets well, but nevertheless there is the issue of negotiating a new TCP connection for every file transfer.</p>
<p>Looking for alternatives to FTP I wanted something that was fairly simple to understand the workings of. It should run on a single or at least constant number of TCP connections. It should preferably not attempt to be encrypted or secure in any way other than maybe having a simple authentication scheme. Let's be honest here: this is a toy project that I will probably abandon when version 1.0 is out (if not sooner) and I will certainly not be immediately issuing security updates when new bugs are found in SSH or TLS. It's going to be designed for secure (home) networks. The protocol should also be light on multi-user features like locking, as this is basically a single user machine.</p>
<p>Rsync is such a protocol. While often run over SSH, rsync can also run in daemon mode, serving files over its custom protocol. Since it is bundled with most Linux distributions, most Linux users will not have to download anything in order to transfer files to and from their Xbox. There are also Windows clients available.</p>
<p>As a proof of concept I have built the latest version of rsync on X-DSL (Xbox-Damn Small Linux) and run it in server mode (rsyncd). It works quite well and will almost saturate the 100 Mbps Ethernet connection (around 9 MB/s average for a typical Xbox game directory tree).</p>
<p>So this will be my new project: A tiny Linux distribution that basically only runs rsyncd and serves the Xbox local file systems over the network. <a href="http://www.antoneliasson.se//journal/xbox-linux-compiling-kernel/">Next up I'll compile an Xbox compatible Linux 2.4 kernel</a>.</p>