Mixing Fortran and C++

Introduction

WORK IN PROGRESS

This guide aims to give some advice on installing Debian GNU/Linux onto an external USB hard drive. In my case, I used an external Maxtor One-Touch 160Gb unit with a USB2 interface (there are also firewire variants). My main motivation in doing this is to give myself some insurance against the future failure of the hard-drive in my laptop. I have already had one failure after only 4 months of operation, and I fully expect to experience another one sooner or later. Having a fully functional GNU/Linux installation on an external drive, ready to boot and with data backed up, means that a future hard-drive failure will (hopefully) be only inconvenient and will not leave me without computing access.

Please note that it is crucial that your computer BIOS be capable of booting from an external USB HDD. I performed the following operations using a DELL Inspiron 1150 (Aug 2004). Activating USB boot on a one-off basis is as simple as pressing F12 during the initial POST (Power-On-Self-Test), and then choosing USB-device from the resulting menu (NB: the drive must be turned on, powered, and connected to the USB port in order to appear in the menu). Consult your BIOS/PC documentatation for more information.

Finally, note that I typed these notes up shortly AFTER having performed this installation. I hope shortly to run quickly through the process again to make sure that I haven't left anything important out or misremembered any details. If you're using these notes to do an installation of your own, then keep your eyes open for mistakes!

First Steps

To begin, you need to partition the external drive. In the following, be VERY VERY CAREFUL to always use the correct drive letters!!!. Failure to do so may result in the destruction of your current file-systems!

First, you must have access to the USB drive from GNU/Linux. If you've been using it for storage already then this is taken care of. If not, then you'll need the following kernel modules (or alternatively you'll need to have these drivers compiled directly into the kernel):

usbcore
uhci_hcd
ehci_hcd
scsi_mod
sd_mod
usb_storage
Typically, with distro-supplied kernels, you'll have the requisite drivers to access external usb devices. Potentially, you may have to issue the command "modprobe usb_storage" to load the driver, though it is also likely that the hotplug subsystem will auto-load the drivers for you. If the driver loads correctly, then a peek in /var/log/syslog should show something similar to the following lines:
Apr  2 15:36:38 localhost kernel: Initializing USB Mass Storage driver...
Apr  2 15:36:38 localhost kernel: scsi1 : SCSI emulation for USB Mass Storage devices
Apr  2 15:36:38 localhost kernel: usbcore: registered new driver usb-storage
Apr  2 15:36:38 localhost kernel: USB Mass Storage support registered.
Apr  2 15:36:38 localhost kernel: usb-storage: device found at 2
Apr  2 15:36:38 localhost kernel: usb-storage: waiting for device to settle before scanning
Apr  2 15:36:43 localhost kernel:   Vendor: Maxtor    Model: OneTouch          Rev: 0201
Apr  2 15:36:43 localhost kernel:   Type:   Direct-Access                      ANSI SCSI revision: 00
Apr  2 15:36:43 localhost kernel: SCSI device sda: 320171008 512-byte hdwr sectors (163928 MB)
Apr  2 15:36:43 localhost kernel: sda: assuming drive cache: write through
Apr  2 15:36:43 localhost kernel: SCSI device sda: 320171008 512-byte hdwr sectors (163928 MB)
Apr  2 15:36:43 localhost kernel: sda: assuming drive cache: write through
Apr  2 15:36:43 localhost kernel:  /dev/scsi/host1/bus0/target0/lun0: p1 p2 p3 p4 < p5 p6 >
Apr  2 15:36:43 localhost kernel: Attached scsi disk sda at scsi1, channel 0, id 0, lun 0
Apr  2 15:36:43 localhost kernel: usb-storage: device scan complete
Apr  2 15:36:43 localhost udev[7860]: configured rule in '/etc/udev/rules.d/z_hal-plugdev.rules[2]' applied, 'sda' becomes '%k'
Apr  2 15:36:43 localhost udev[7860]: creating device node '/dev/sda'
Apr  2 15:36:43 localhost scsi.agent[7885]: disk at /devices/pci0000:00/0000:00:1d.7/usb1/1-1/1-1:1.0/host1/target1:0:0/1:0:0:0
Apr  2 15:36:43 localhost udev[7873]: configured rule in '/etc/udev/rules.d/z_hal-plugdev.rules[2]' applied, 'sda1' becomes '%k'
Apr  2 15:36:43 localhost udev[7873]: creating device node '/dev/sda1'
Apr  2 15:36:44 localhost udev[7874]: configured rule in '/etc/udev/rules.d/z_hal-plugdev.rules[2]' applied, 'sda2' becomes '%k'
Apr  2 15:36:44 localhost udev[7874]: creating device node '/dev/sda2'
Apr  2 15:36:44 localhost udev[7877]: configured rule in '/etc/udev/rules.d/z_hal-plugdev.rules[2]' applied, 'sda5' becomes '%k'
Apr  2 15:36:44 localhost udev[7877]: creating device node '/dev/sda5'
Apr  2 15:36:44 localhost udev[7878]: configured rule in '/etc/udev/rules.d/z_hal-plugdev.rules[2]' applied, 'sda6' becomes '%k'
Apr  2 15:36:44 localhost udev[7878]: creating device node '/dev/sda6'
Apr  2 15:36:44 localhost udev[7876]: configured rule in '/etc/udev/rules.d/z_hal-plugdev.rules[2]' applied, 'sda4' becomes '%k'
Apr  2 15:36:44 localhost udev[7876]: creating device node '/dev/sda4'
Apr  2 15:36:44 localhost udev[7875]: configured rule in '/etc/udev/rules.d/z_hal-plugdev.rules[2]' applied, 'sda3' becomes '%k'
Apr  2 15:36:44 localhost udev[7875]: creating device node '/dev/sda3'
Your output may be different for a couple of reasons. First, my drive is already partitioned and formatted with a bunch of filesystems, hence so many device nodes. Second, I'm using udev to manage the device nodes in /dev, which you may not be. The important thing to note is that the drive appears as /dev/sda, and its partitions appear as /dev/sda1, /dev/sda2, etc. If you had two USB drives, you might have a different drive letter (e.g. /dev/sdb) and it is crucial that you use the correct letter.

Now, your brand-new external hard-drive will typically come with a nice big fat32 filesystem on it. This is handy as most anything can read such a filesystem, but it's not ideal for booting GNU/Linux off. We want instead to repartition the drive. NOTE, doing this will delete any and all data on the drive!. On a brand-new drive, no problem, but on one you've been diligently backing-up on, make sure you know what you're up to.

My preferred tool for partitioning is cfdisk, though a very nice alternative with a graphical user interface is qtparted. Using cfdisk, select the single large partition on the drive, and delete it. Then choose the huge-resulting block of "free space" and choose the option "create" and create a partition of (for example) 10Gb with type "Linux" (this will be default). You can do what you like with regard to partitioning the rest of the disk (my own preference was to create a couple of 10Gb partitions for trial distro-installs, then a 40Gb fat32 partition to share data between windows/linux machines, and finally a large linux partition for backups and similar occupying the remainder of the disk.

With the disk partitioned, you'll need to create a file-system on the install-target. I chose to go with the reliable and unexotic ext3 filesystem, which you can create using a command like (MAKE SURE DEVICE NAME IS CORRECT):

mkfs.ext3 /dev/sda1
Once this is done, you can continue to create filesystems on the other partitions on your drive.

Creating a Base System

At this point, you have at least one GNU/Linux ext3 partition on your USB HDD. To create a basic Debian GNU/Linux installation on it, you'll need to use the debootstrap command. If you don't have this tool installed yet, use the magic of apt and as root type
apt-get install debootstrap
Then, before running the command, you'll need to mount the target partition. Again as root, type commands like (changing device names as appropriate if necessary):
mkdir /mnt/usb1
mount /dev/sda1 /mnt/usb1
Now, ready to run debootstrap, you have a choice as to what flavour of Debian you want to install. I chose to go with sarge and to use the Irish debian mirror, and so issued the command:
debootstrap sarge /mnt/usb1 http://ftp.ie.debian.org/debian/
This will run for a while, and download a bunch of packages, and install them within the /mnt/usb1 directory.

Once this process has finished, you can go on to install further packages. To accomplish this, you must chroot into the new system root. The effect of this is that your shell will start to behave as if its root directory was /mnt/usb1, and any commands you run will be found from /mnt/usb1/... rather than from /... To get access to networking, you'll also have to mount proc from within the chrooted environment. Also, to have networking work properly, you'll need to ensure the new system's /etc/resolv.conf file contains appropriate name-server information

cat /etc/resolv.conf >> /mnt/usb1/etc/resolv.conf
chroot /mnt/usb1
# Then, from within the new chrooted shell:
mount -t proc proc /proc
Now, you can run dselect and install some new packages. Ones you'll need to install include
    grub
    devfsd

You'll also need to do some work on the /etc/fstab file, which by default will be empty. The basic /etc/fstab I created goes as follows:

proc            /proc   proc    defaults                        0 0
/dev/sda1       /       ext3    defaults,errors=remount-ro      0 0
You may need to change the device node for the root "/" file-system, or to add more entries if (for example) you put /home on another partition.

Making It Bootable

Following on from the previous section, we now have a basic Debian installation in a partition on our external hard-drive. The next task is to make it bootable. This requires two basic steps:
  • Install a suitable kernel-image.
  • Install a bootloader onto the mbr (master boot record) of the external drive.

Regarding the kernel image, I went not with a standard Debian kernel, but rather with one I compiled myself (but using the Debian kernel-packaging system). The configuration of a kernel is always important, and especially so when you're booting the system in an unusual way such as from a USB drive. The bootloader will be able to get your kernel from the drive and get it going, but you must make provision so that the kernel is able to access the root file-system early in the boot-process. This clearly means that you cannot rely on the kernel loading modules for usb_storage and co. from the hard-drive! There are two ways around this problem. One is to compile all the relevant drivers into the kernel. The alternative is to compile the drivers as modules, but to use an initial ram disk, or initrd, to give the kernel access to the modules before the root file-system on the USB HDD is mounted.

The creation of the mkinitrd file-system is controlled by some config files contained in the directory /etc/mkinitrd/. You should edit the file "modules", and add the names of modules required to access the root-filesystem. The ones I included were:

    usbcore
    uhci_hcd
    ehci_hcd
    scsi_mod
    sd_mod
    usb_storage
    ide_generic
    ide_cd
Then, editing mkinitrd.conf, you should set the options ROOT and DELAY as follows:
DELAY=10
ROOT=/dev/sda1
The second of these is clear enough. The DELAY option is necessary so that the boot process pauses for a few seconds to allow the usb_storage driver to load correctly, before going on to access the root file-system. The Debian kernel compilation process (described in subsequent paragraphs) will create and install an initrd file. You can create such a file by hand using a command like:
    mkinitrd -k -o initrd.img -r /dev/sda1
Here, -k means we want to preserve the working directory after the image is created. -o simply chooses the output file. -r can be used to over-ride the ROOT setting in /etc/mkinitrd/mkinitrd.conf.

Compiling a Debian kernel is a very straight-forward process. First unpack the kernel-sources. You can download these by going to ftp://ftp.kernel.org/ or your local mirror. I used the 2.6.10 sources for the current example. After unpacking the sources in the directory /usr/local/src/linux:

tar -xzvf linux-2.6.10.tar.gz   # If you downloaded a .gz archive
tar -xjvf linux-2.6.10.tar.bz2  # If you downloaded a .bz2 archive
change directory into the unpacked kernel source and clean them.
cd linux-2.6.10 && make mrproper
Before beginning the compilation, you'll need to configure the sources. If you've already compiled earlier kernels (e.g. 2.6.9) you can use the config file from that version as a starting point as follows (for example):
    cp ../linux-2.6.9/.config ./
    make oldconfig
    # Now you'll be prompted for input just on new/changed options
Alternatively, you can start from scratch. This is easiest to do via the menuconfig interface:
    make menuconfig
If you've never done this before, then it's very likely you'll make a mistake and leave out something that's important for your system. To become familiar with the process I reccommend you should configure and later compile a kernel for your current system that keeps all your hardware working (in particular make sure it still allows you to mount external USB storage devices!). You'll also need to include support for cramfs (CONFIG_CRAMFS=y), and for initial ram disk (CONFIG_RAMFS=y, CONFIG_BLK_DEV_RAM=y, CONFIG_BLK_DEV_INITRD=y). You can take a look at my .config file to get an idea of the setup to use, though of course you will more than likely want to change many of the options (e.g. what network card drivers to use, sound-drivers, etc.,).

The actual compilation step follows the following pattern:

     fakeroot make-kpkg clean && fakeroot make-kpkg --revision=yourid.1.0 --append-to-version=c --initrd kernel_image
Note that this requires that you have access to two Debian packages, which you can install using
    apt-get install fakeroot kernel-package
In the compilation command, you can use whatever "yourid.1.0" you want for the version number. It is recommended that it begin with a letter though. The "append-to-version" part allows you to have multiple versions of a kernel-version installed. When the configuration process finishes, you'll have a file in the parent directory (e.g. /usr/local/src/linux) called kernel-image-2.6.10c_yourid.1.0_i386.deb

I found that later configuration (e.g. of bootloader and initrds) went more smoothly if I was running the same kernel version in my parent system as the version I was attempting to install into the USB HDD. Installing the kernel-image into your parent system is then simply a matter of

dpkg -i kernel-image-2.6.10c_yourid.1.0_i386.deb
This should (via the magic of Debian) also create the appropriate entries in your bootloader menus to boot the kernel. To actually use the kernel, you must reboot the computer (and choose the appropriate kernel from the GRUB or LILO boot-menu).

To install the kernel into the USB-based system, copy the deb file above into the mounted filesystem, then chroot in and run the commands as follows (and, as always, making appropriate changes if required):

cp kernel-image-2.6.10c_yourid.1.0_i386.deb /mnt/usb1
chroot /mnt/usb1
mount -t proc proc /proc
dpkg -i kernel-image-2.6.10c_yourid.1.0_i386.deb

Next, you need to configure the boot-loader. I worked with GRUB, though LILO should also be suitable. From your parent system, you need to run the command:

grub-install --root-directory=/mnt/usb1 /dev/sda

The GRUB configuration is stored in the /mnt/usb1/boot/grub directory. Now, there's a small gotcha regarding the GRUB configuration. As things stand, while running your parent system, the hard-drive you booted off is more than likely known to GRUB as (hd0), and the USB drive is then (hd1). However, when your computer BIOS starts to boot your system off the USB drive, then that drive will become the primary drive and grub will identify it as (hd0). To reflect this, you should edit the file /mnt/usb1/boot/grub/device.map and change it to read:

(hd0)   /dev/sda
The only other file you may need to edit is /mnt/usb1/boot/grub/menu.lst. The stanza that finally worked for me runs as follows:
    # Next line is how the kernel will be identified in the menu
    title           Debian GNU/Linux, kernel 2.6.10c
    # The location of root fs, note (hd0,0) is the USB drive when
    # we're actually booting off the MBR on the drive
    root            (hd0,0)
    # The kernel to boot, and where it'll find its root-fs (in linux device notation)
    kernel          /boot/vmlinuz-2.6.10c root=/dev/sda1 rw
    # The relevant initrd file (note I made this one by hand)
    initrd          /boot/mc_initrd.img
    # The next (commented out) line would use the default initrd
    # installed by the Debian kernel package created earlier:
    #initrd         /boot/initrd.img-2.6.10c
    savedefault
    boot
Then, with GRUB configured, we can install it into the boot-sector of the USB HDD using the command
    grub-install --root-directory=/mnt/usb1 /dev/sda

Problems

Although the process described here is really relatively straight-forward, I found there was huge scope for mishaps and problems.
  • Mounted-permissions. Typically shows up when you try to chroot into a directory and are told that permission is denied. Often this is because the partition is mounted "noexec", so you cannot execute anything on the partition (e.g. bash). Solution is to (as root) unmount the partition, and then to mount it along the lines of
    mount /dev/sda1 /mnt/usb1
    
    This will avoid any options set in /etc/fstab
  • Kernel Panics when booting due to no root-fs. This may be because you don't in fact have the right modules in the initrd image. This can occur if the initrd is for a different kernel version than the one you're booting. If you mount the image using a command like "mount -o loop initrd.img /mnt/test" you can have a look around and see if it is as it should be. If that looks OK, then try accessing the shell during the boot process when invited to (you must have non-zero DELAY in mkinitrd.conf).
  • A more subtle problem I came across relates to the timing of operations. The loading of the usb_storage module went according to plan, but immediately I got a kernel-panic afterwards, then following that message, some more messages from usb_storage appeared. This was a consequence of the slow loading of the usb_storage module. The solution I found was to introduce a DELAY of 10 seconds. This gave enough time for the module to fully load, and allowed the kernel to subsequently access the root filesystem without problems.
Copyright © 2004 Michael Conry Disclaimer Wed Sep 14 19:36:36 2005