Mister Muffin Blog

For a linux install fest I was recently setting up a server that would be able to serve a number of different linux live CD's over nfs after selecting one by booting over PXE. I was very surprised how simple such a setup is and that the only trouble really only is to specify the correct kernel commandline options or initrd append options to boot a live system over nfs.

To enable pxe one needs a dhcp server that advertises it and a tftp server that serves the pxelinux.0 images, the config files and the kernel and initrd. Gladly, dnsmasq can act as both and is incredibly light weight as well (even more when comparing to the ISC dhcp3 server).

My dnsmasq config:

root@kirkwood:~# cat /etc/dnsmasq.d/pxeboot 
dhcp-range=192.168.0.50,192.168.0.150,12h
dhcp-boot=pxelinux.0
enable-tftp
tftp-root=/var/tftpd
dhcp-option=3
dhcp-option=6

This will make dnsmasq serve IP addresses in the range from 192.168.0.50 to 192.168.0.150, inform clients about the existance of pxelinux.0, enable the tftp functionality, set the tftp root directory to /var/tftpd and disable sending default gateway and DNS server addresses. The numbers 3 and 6 are taken from this specification

Configuring nfs is even more simple. After installing the nfs-kernel-server package, just set the exports as the following:

root@kirkwood:~# cat /etc/exports
/var/tftpd/ 192.168.0.1/24(ro,async,no_root_squash,no_subtree_check)

My network configuration looks like this, btw:

root@kirkwood:~# cat /etc/network/interfaces 
auto lo
iface lo inet loopback

auto eth0
iface eth0 inet static
        address 192.168.0.1
        netmask 255.255.255.0
        gateway 192.168.0.200

Then get pxelinux.0 vesamenu.c32, which is part of syslinux and begin populating /var/tftpd

apt-get install syslinux-common
mkdir -p /var/tftpd/pxelinux.cfg
cp /usr/lib/syslinux/pxelinux.0 /var/tftpd/
cp /usr/lib/syslinux/vesamenu32.c32 /var/tftpd/pxelinux.cfg/

Then populate a directory structure in /var/tftpd/ with "extracted" live CDs.

To "extract" the iso9660 images, mount them and copy their content but remember not to cp -r /mnt/* with the star wildcard as this will miss potential directories starting with a dot. This is especially important for the ubuntu live CDs as they contain a .disk directory containing metadata information about the live CD.

mount -o loop -t iso9660 linux-live.iso /mnt
cp -a /mnt/. /var/tftpd/some/destination
umount /mnt

My directory structure looked like this:

/var/tftpd
/var/tftpd/arch
/var/tftpd/arch/2011.08.19/i686
/var/tftpd/arch/2011.08.19/amd64
/var/tftpd/debian
/var/tftpd/debian/6.0.2/gnome
/var/tftpd/debian/6.0.2/gnome/i386
/var/tftpd/debian/6.0.2/gnome/amd64
/var/tftpd/debian/6.0.2/kde
/var/tftpd/debian/6.0.2/kde/i386
/var/tftpd/debian/6.0.2/kde/amd64
/var/tftpd/debian/6.0.2/xfce
/var/tftpd/debian/6.0.2/xfce/i386
/var/tftpd/debian/6.0.2/xfce/amd64
/var/tftpd/debian/6.0.2/lxde
/var/tftpd/debian/6.0.2/lxde/i386
/var/tftpd/debian/6.0.2/lxde/amd64

and so on...

Now, when a client boots over the network, after retrieving pxelinux.0 over tftp, it will then try to acquire pxelinux.cfg/default which also has to be filled accordingly.

Let me just paste you the files I compiled and spare you with the details:

• pxelinux.cfg/default
• pxelinux.cfg/debian.menu
• pxelinux.cfg/ubuntu.menu

Basically those files specify the menu layout and structure and point to the kernel images and initrds that are served via tftp, giving them the correct arguments to boot via nfs.

Now clients can attach to the machine running the dnsmasq powered dhcp and tftp server, boot pxelinux.0, select a distribution and boot them over nfs.

The following links are some helpful resources on the topic as well:

• LiveCDNetboot
• PXEInstallMultiDistro

The CS-club at Jacobs University recently organized a Linux install fest where we installed mostly Ubuntu 10.10 on people's machines. It turned out that in some cases (reasons yet unknown) grub failed to install correctly and the computer would still boot into Windows.

Eventually we figured out how to solve this problem and this is how.

There are two ways of doing it and they depend on whether one can chroot into the target system or not from the live CD. This again depends on the architectures being compatible with each other as using qemu user mode emulation is not an option on a live CD.

The reason one has to have the target linux installation as the root is update-grub which takes no argument to specify a differently mounted /. There is possible an environment variable somewhere that fixes this but we werent able to find one.

Method 1 (no chroot)

So in case chroot is not possible, the steps are:

1. install grub to the MBR from the live CD
2. manually boot into the already installed linux via the grub cli
3. run grub-update in the installed system

After booting into the live system, do

mount /dev/sdZX /mnt

where /dev/sdZX is the /boot partition

grub-install --recheck --root-directory=/mnt /dev/sdZ

where /dev/sdZ is the primary harddisk.

While you are still in the live system, it's also easy to take some notes which partition / is on.

reboot

then in grub do the following:

set root=(hd0,X)
linux /vmlinuz... root=/dev/sdZY
initrd /initrd...
boot

The first root= specifies your /boot partition. To get an overview of the available partitions, run ls from the grub prompt. To have a look whether you selected the right partition as the grub root, check its contents with ls / . When specifying the linux kernel image and the initrd, instead of the ... press [TAB] to automatically complete the filenames. Carefully adjust the root= kernel commandline option as /dev/sdZY has to point to the partition containing /.

If everything was entered correctly, your linux will boot and after opening a terminal you can let grub autoconfigure itself and install it to the MBR once again (where /dev/sdZ is again your primary harddrive).

update-grub
grub-install --recheck /dev/sdZ

Method 2 (with chroot)

In case it is possible to chroot into the installed system from the live system the process is a bit more complex but less time consuming due to no reboots or mess ups at the grub cli wrt. partition numbers.

The steps are:

1. mount the root file system from the harddrive somewhere
2. bindmount all important pseudo filesystems into it
3. chroot into the system
4. update-grub and grub-install
5. umount everything and reboot

After booting into the live system, do

mount /dev/sdZX /mnt

where /dev/sdZX is your / partition and possibly also do:

mount /dev/sdZY /mnt/boot

where /dev/sdZY is your /boot partition in case you have an extra /boot partition.

mount --bind /dev /mnt/dev
mount --bind /dev/pts /mnt/dev/pts
mount --bind /proc /mnt/proc
mount --bind /sys /mnt/sys

grub seems to need at least /dev to be bindmounted correctly.

chroot /mnt

then, in the chroot, do:

update-grub
grub-install --recheck /dev/sdZ

where /dev/sdZ is your primary harddrive.

Exit the chroot and reboot.

Problems with update-grub not detecting windows

this problem happened a few times as well but afaik catalin is still investigating how to fix this issue. Will probably update this section later on.

So I was copying files on a vfat-formatted usb stick when this error popped up:

Invalid or incomplete multibyte or wide character

Same issue with cp, rsync and tar. So I just thought this would be an issue with stupid vfat so I mkfs.ntfs the drive and tried again - same issue.

So while I thought that at least NTFS would allow any character except NULL I was proven wrong. Looking it up in wikipedia indeed yielded that ntfs file names are limited to UTF-16 code points - in contrast to ext2/3/4, reiserfs, Btrfs, XFS and surely even more which allow any byte except NULL.

So what my files contained was a number of iso 8859-1 or latin1 characters like 0xe8 (è), 0xe0 (à), 0xf9 (ù) or 0xec (ì). It might also have been Windows-1252 but apparently the bytes between 0x80 and 0x9F didnt appear.

This is how to finally batch-convert character sets in filenames:

convmv -f latin1 -t utf8 --notest *.flac

I had some operas lying around amongst others Wagner's "Ring Des Nibelungen", Puccini's "La Boheme" and Verdi's "La Traviata". Sadly, some of them were not stored as flac but as monkey's audio (extension .ape) which is a non-free lossless codec but fortunately ffmpeg has a GPL'd decoder included. Additionally some CD's were stored as a single file with a cue sheet next to them.

So my task was: convert ape to flac, split audio by information from cue files and tag everything correctly.

These were the lines I used to split the audio:

ffmpeg -i CD1.ape CD1.wav
shnsplit -o flac -f CD1.cue -t "%a %n %t" CD1.wav
rm *pregap.flac CD1.wav CD1.ape
cuetag CD1.cue *.flac

First ape is converted to wav so that shnsplit can read it. Shnsplit will then take the timing information from the cue sheet and split the wav into chunks which it conveniently also converts to flac on the fly. Sadly it doesnt do tagging of those files so this is done by cuetag afterward with information from the cue sheet. You need the shntool and cuetools packages.

Other data was already existing as separate tracks and only had to be converted from ape to flac. Sadly flac offers no means for batch conversion but using a for loop and basename the same effect is created. Conveniently ffmpeg will copy the tags from the ape files to the new flac files as well.

for f in *.ape; do b=`basename $f .ape`; ffmpeg -i "$b".ape "$b".flac; done

The resulting flac files were about 3-5% larger than the ape files which is a totally acceptable tradeoff for being able to ditch an unfree format.

thomasg pointed out to me, that using the zsh there is an even neater way to do this loop without basename(1) and without some shell-loop keywords:

for f in *.ape; ffmpeg -i $f $f:r.flac

I was like so surprised how easy it is to install debian from an usb stick :D

curl http://http.us.debian.org/debian/dists/stable/main/installer-i386/current/images/hd-media/boot.img.gz | zcat > /dev/sdc
mount /dev/sdc /mnt
( cd mount; wget http://cdimage.debian.org/debian-cd/6.0.2.1/i386/iso-cd/debian-6.0.2.1-i386-netinst.iso; )
umount /mnt

bam!