I found this over at the Ubuntu Forums, but since it took me forever to find, I’m dropping notes here. In RPM-based distros, you can do ‘rpm -qa > somefile.txt’. In Debian/Ubuntu, do this:

dpkg --get-selections > machineA.txt

In true apt fashion, if you then want to have machine B have all the software machine A has, do this:

dpkg --set-selections < machineA.txt && dselect

Enjoy!

NetworkManager is fast becoming the de facto network provider in desktop Linux distributions. The reason it’s so popular is that it “does the right thing” 99% of the time. However, there’s not many examples out there that extend that functionality. NetworkManager provides hooks in which you can have scripts launch when network settings change. In today’s post, I will show you how to launch the Synergy client whenever you plug into your corporate network.

I’m lazy. I hate having to fire up my Synergy client on my laptop to connect to my desktop at work when I get to work everyday. Now, I could just fire up the Synergy client at boot, but when I’m at home if I have the VPN connected, Synergy will hook up and I don’t want it to.

I needed a way to fire a script that says “if I’m on this network, then fire up synergyc, otherwise do nothing”. Writing the script was pretty easy, but I was stumped on how to get to run not only at bootup, but whenever I change networks – I very often suspend my laptop at night at home and resume it at work the next morning.

It turns out that NetworkManager has a piece called NetworkManagerDispatcher that does all of this for us. Any script in /etc/NetworkManager/dispatcher.d will be called with two arguments, the name of the interface, and the status of the interface (up/down).

If a picture is worth a thousand words, scripts are worth a million, so let’s get to it.

First, a little background is needed. I know that I am on my corporate network if my eth0 interface has obtained an IP in the 10.0.0.0/8 subnet. Without further ado, I present to you /etc/NetworkManager/dispatcher.d/99smartsynergy.sh:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30

#!/bin/sh
IF=$1
STATUS=$2
USER=justintime

wait_for_process() {
  PNAME=$1
  PID=`/usr/bin/pgrep $PNAME`
  while [ -z "$PID" ]; do
        sleep 3;
        PID=`/usr/bin/pgrep $PNAME`
  done
}

start_synergy() {
     wait_for_process nm-applet
     /bin/su $USER -c "/usr/bin/synergyc $1"
}

if [ "$IF" = "eth0" ] && [ "$STATUS" = "up" ]; then

        #LAN Subnet at work
        NETMASK="10.0.0.0/8"
        if [ -n "`/sbin/ip addr show $IF to $NETMASK`" ]; then
                ARGS="jentoo.bucklehq.com"
                start_synergy $ARGS
                exit $?
        fi

fi

The IF and STATUS variables are those fed in from NetworkManager. The USER variable is the user that I run synergyc as. You could add some intelligence here, but it was overkill for my situation.

The if at the bottom states that we are only concerned if the interface eth0 has changed it’s status to “up”. I then use the /sbin/ip command to determine if eth0 is within the 10.0.0.0/8 subnet. If so, then I call start_synergy, passing it my desktop’s hostname.

Within the start_synergy() function we call the wait_for_process function, passing it nm-applet. We need this function because if we try to run synergyc before I’ve logged in via GDM, it will exit (this happens on bootup). By calling wait_for_process, we create a way to make synergyc wait until after the nm-applet (NetworkManager Applet) has started. Finally, once nm-applet has been detected as running, the script executes synergyc, and exits.

TODO’s

• I should probably create a function that kills the remaining synergyc processes when eth0 goes down.
• If there’s already a synergyc running, we should just exit as the client will continue to try to reconnect.

Conclusion

The purpose of this article wasn’t to show you how to launch synergyc (although I think it’s really handy), it was to get the creative juices flowing. Have you already utilized NetworkManagerDispatcher for something? What would you like to have it do? Comment away!

RHEL/CentOS has had support for LVM in setup for quite some time now, but for whatever reason, Ubuntu has been slow at adopting support for LVM at installation. Usually, I just grumble and move on with ext3 – not today. Convinced that I couldn’t be the only person wanting LVM support, I set out to do just that. Luckily, it wasn’t hard at all! As of Intrepid (8.10), the installer has basic support for LVM in the disk partitioner part of the installation. It’s not pretty, but it works. The key is that the LVM binaries aren’t installed into the image, so the installer won’t detect the presence of the LVM PV’s, VG’s, and LV’s. Here’s how to a) create a new LVM setup during installation, and b) how to use an existing LVM setup during installation.

Pre-installation Common Steps

You need to do this section no matter what path you are taking.

• Boot into the Live CD mode - not the direct installation.
• Once you’re looking at your desktop, we need to install LVM. Hit Alt+F2, and type in gnome-terminal, hit the OK button.
• At the terminal, type sudo apt-get install lvm2. Note that this installs the binaries into the ramdisk, not your final installation. We’ll need to install LVM to the hard drive later.
• Once installed, we need to load the device mapper module into the kernel: sudo modprobe dm-mod If you’re setting up LVM from scratch, continue with New LVM Setup below. Otherwise, skip to Use an Existing LVM Setup further down.

New LVM Setup

Creating LVM setups from scratch is not a “cookie cutter” operation, everyone’s setup will be different. If you’re trying to do LVM on installation, you likely already know what you’re doing here. However, your general steps should be something like:

• Use sudo fdisk /dev/sda to setup your Linux boot partition and LVM (type 8e) partition(s)
• Update the kernel with the new partitions by running sudo partprobe.
• I recommend using ext2 on your boot partition. If boot will reside on partition 1, then do a sudo mkfs.ext2 /dev/sda1 to format the partition with Ext2.
• Create the physical volume(s) (PV’s) on the partitions of type 8e by using sudo pvcreate /dev/sdaN.
• Create a volume group (VG): sudo vgcreate MyLVMVG /dev/sdaN
• Create logical volumes (LV’s) within the group: sudo lvcreate -n root -L 5G MyLVMVG && sudo lvcreate -n swap -L 1G MyLVMVG
• Last, we need to put filesystems on the LV’s. sudo mkfs.ext3 /dev/MyLVMVG/root && mkfs.swap /dev/MyLVMVG/swap Skip the next section and continue on to Post-configuration Common Steps.

Use an Existing LVM Setup

In my personal case, I was replacing CentOS with Ubuntu. I like CentOS for it’s stability, but my laptop needs a more bleeding edge in order to run the way I want it. I had setup an efficient LVM configuration already, I just wanted to reformat it during installation and use what I already had. Here’s what you need to do:

• If you don’t already know it, we need the volume group (VG) names from our current setup. You can obtain that by running sudo vgscan | grep Found. In cases where CentOS/RHEL did the setup, you’ll likely have a VG name of VolGroup00.
• We need to tell the running kernel to scan for and activate the current logical volumes (LV’s). Using the VG from above, run sudo lvchange -ay VolGroup00
• At this point, your LV’s should be found and available to the kernel. You can verify this by running sudo lvdisplay | grep “LV NAME|LV Status” At this point, your LV’s will be visible to the installer. Continue on to Post-configuration Common Steps below.

Post-configuration Common Steps

Now, you need to launch the installer. You can double-click the icon, or you can type ubiquity in your already open terminal. Continue through the installer as normal, but make sure you select the Manual radio button on the Prepare Disk Space panel of the installer. When you click the Forward button, you’ll be presented with a list of devices that includes your /dev/mapper/VG/LV entries. Go ahead and click on each one, click the Edit Partition button, and set the mountpoint. Write down or memorize what LV’s are mounted where, and which partition is your boot partition – you’ll need it later. Format them if you used an existing setup. Continue through the installer just as you normally would, but make sure to click “Continue using the liveCD” at the end of installation! Failure to do so will give you a system that won’t boot. Here’s the only tricky part – we need to mount our partitions that we installed to, and install LVM into it. Since you wrote down everything above, it’s not hard. First, we need to make a mountpoint, and mount our root LV to it:

sudo su - mkdir /mnt/chroot/
mount /dev/VolGroup00/root /mnt/chroot

Now, everyone will need to do this for the /boot partition, but you may need to mount /var, /home/, etc., if you created separate mount points for them. Here’s what I had to do in my particular setup:

mount /dev/VolGroup00/home /mnt/chroot/home
mount /dev/sda3 /mnt/chroot/boot

Now, we need to chroot into our environment, and install LVM.

chroot /mnt/chroot apt-get update apt-get install lvm2
exit

At this point, you can restart the system by clicking System->Shut Down->Restart. If you’re paranoid you can unmount the partitions, but the live cd will do that for you.

Conclusion

While it’s not that hard to use LVM during installation of Ubuntu, it’s still harder than it should be. It looks like something the Ubuntu devs are working on, as the ubiquity installer already has rudimentary support for it. Maybe 9.04 will have it – if not, check back here for a post on how to do it there too!

Like many of you other sysadmins, I run a lot of ad-hoc, long running jobs. Also like many of you, I have a full plate and can’t stand to sit around watching things run. Often times, I will start such a job and forget to come back to it until the end of the day. I needed a way to find out quickly about how long these tasks would take to run so that I could make a mental note or set a reminder to check the task later. Over time, I noticed that the vast majority of the jobs I ran could be quantified and estimated by coming up with custom shell pipeline. For instance, I need to iterate over something, and write output to a file. I know that I’ll be close to being done when I hit 100,000 lines in that file. If the file was named output.log, then I could monitor progress by running ‘wc -l output.log’.

These conditions gave birth to progress.sh - a shell script that runs a command that you specify once every 30 seconds. The command needs to give back an integer. You can optionally specify an end integer, and it will (fairly accurately) give you back an estimated completion time. progress.sh takes two arguments, one required, the other optional. The first argument is the command that generates an integer. The second argument is optional, it’s the end integer that will allow the script to estimate time remaining. Perhaps the script is best explained by giving a full example. Open up a terminal, and issue the following at the command line:

echo "" > test.out; for i in `seq 1 600`; do echo "SUCCESS" >> test.out; echo "$i" >> test.out; sleep 3; done

This is a one-liner script that echoes two log lines to a file once every 3 seconds for 30 minutes. Before hitting the enter key, open up another terminal, and run this in it:

./progress.sh 'grep -c SUCCESS test.out' 600

It takes an iteration or two to get enough data to make a good estimate, but you should start seeing output like this:

Current=.13/sec TotalAvg=.13/sec Total=4/600 0% 1.27 hrs left
Current=.33/sec TotalAvg=.23/sec Total=14/600 2.00% 42.46 mins left
Current=.33/sec TotalAvg=.26/sec Total=24/600 4.00% 36.92 mins left
Current=.33/sec TotalAvg=.28/sec Total=34/600 5.00% 33.69 mins left
Current=.33/sec TotalAvg=.29/sec Total=44/600 7.00% 31.95 mins left
Current=.33/sec TotalAvg=.29/sec Total=54/600 9.00% 31.37 mins left
Current=.33/sec TotalAvg=.30/sec Total=64/600 10.00% 29.77 mins left

The first column is the statistics related to average units per second. Let’s examine the last line. Now, progress.sh executes the command you give it once every thirty seconds. The .33/sec figure means that in the last iteration, the integer incremented .33 units per second. Since we echo the word “SUCCESS” out to the file once every 3 seconds, the math here is sound. The tTotalAvg of .30/sec is the same metric, except instead of being for the last execution, it is the average since the startup of progress.sh. The second column shows the completion status. Again, looking at the last line, we are told that we have 64 units out of our total 600, meaning we’re roughly 10% done. The final column tells us what the script estimates as the time remaining before we are at 100% done.

Caveats

• I’m not a math major. This is simple stuff. If you know ‘bc’ and have suggestions, I’d be happy to hear them.
• Watch your quoting on the first argument. If you’re doing complex pipelines, you will likely need to escape some characters. Check out the quoting section of the bash-scripting howto for tips.
• Let me know if it works great for you. Let me know if it doesn’t. I’ll try to make it better if I can. I use it all the time and it works well for me.

Download

Get progress.sh here: https://github.com/justintime/progress.sh.

So, you store all your CD’s as FLAC, and encode FLAC to MP3 on the fly. Now, you’ve gone and lost that CD, or in my case, your 3 year old daughter loses it for you. How do you regenerate a CD from your FLAC’s? mp3cd does just that. I was all set to code something up myself, but mp3cd is currently maintained, and even available in the Ubuntu repositories! It has a man page, and it even works – why reinvent the wheel? Note that if you do use Ubuntu, and you’re getting this error:

sox soxio: Failed reading `01.wav': unknown file type `auto'

Then you just need to install some packages:

sudo apt-get install libsox-fmt-all

I refuse to do DRM. If there’s an album I want, I buy the CD. The first thing I do after opening a new disc is to rip the disc to FLAC. The second thing I do is to copy the disc so that the loader in my car doesn’t do permanent damage to the master copy. The third thing I do is to put the album on my MP3 player. Now, my player runs Rockbox, so I can play FLAC files, but they eat up too much space. However, I hate to keep both an MP3 and a FLAC laying around when I only need access to the MP3 once. Enter MP3FS - a fuse filesystem that converts FLAC to MP3 on the fly. It’s a beautiful thing. I keep all my FLAC files on my NAS, which is exported via NFS to all my systems. On my laptop, I have my FLAC export mounted at /mnt/FLAC. I have MP3FS configured to convert /mnt/FLAC, and it’s mounted at ~/MP3. I plug in my MP3 player, and browse all my MP3’s on ~/MP3 (they don’t really exist). When I copy the MP3 from ~/MP3 to my MP3 player, MP3FS transcodes the FLAC files to MP3 on the fly. It even adds ID3 tags to the MP3! Here’s how you can setup the same thing.

Download and install MP3FS

Most distro’s don’t include MP3FS as a package, but pretty much any modern distro supplies the prerequisite packages. On Ubuntu, run the following to satisfy the prerequisites:

sudo apt-get install build-
essential fuse fuse-utils liblame-dev libflac-dev libfuse-dev libid3tag0-dev

Next download MP3FS from http://mp3fs.sourceforge.net/, extract the file, change into the new directory, and do the normal GNU thing:

tar -xzvf mp3fs-0.12.tar.gz cd mp3fs-0.12 ./configure
make sudo make install

Test

Before we mount this thing via fstab, we’ll get it working first. First, create the mountpoint - in my example, it’s ~/MP3.

mkdir
~/MP3

Now, if your FLAC files are at /mnt/FLAC, and you want 192K MP3’s, run this command:

mp3fs /mnt/FLAC/,192
~/MP3/ -o ro

Now, browse the new MP3 directory using Nautilus, ls, or whatever. Cool ‘eh? Note the file isn’t actually transcoded until you try to access the contents of the file. Just doing directory listings doesn’t transcode. Go ahead, pick an MP3 to play in your favorite player. You’ll likely find that transcoding happens pretty quickly.

Set it and forget it

Now, we can setup this mount in /etc/fstab and configure it to mount on bootup, so it’s there waiting for you all the time. Unmount the directory, add the entry to fstab, and mount it.

sudo umount
/home/justintime/MP3/ sudo sh -c "umount /home/justintime/MP3; echo
'mp3fs#/mnt/FLAC,192 /home/justintime/MP3 fuse ro 0 0' >> /etc/fstab" mount
~/MP3 ls ~/MP3

At this point, you are ready to go. If you don’t like to brag about your uptime, go ahead and reboot and make sure the mountpoint is there. Otherwise, trust me ;-) Hope you enjoy MP3FS as much as I do!

Just a quick note for myself. We have a Sony HandyCam that burns video to DVD’s. I recently needed to upload a video of my daughter to YouTube to share with relatives. After a few iterations, here’s what I settled on:

mencoder -ovc xvid -oac mp3lame -af resample=44100:0:0 -xvidencopts \
bitrate=2200 -o MyVideo.avi dvd://${TITLE} -chapter ${CHAPTER}

If anyone else has any settings that work better, please share!

While reading over my RSS feed, I came across Chris Siebenmann’s post titled “A thesis: Sun should fork Solaris”. It’s one of those posts that brings up an idea that’s so simple, it’s brilliant. I’ve been a Linux user since Red Hat 5.2 was released somewhere around 10 years ago. I’ve been working with Solaris for about 2 years now, but where I was excited to learn Linux, I must admit that I dread learning Solaris. Why? It feels old and monolithic to me. I love ZFS, and I’m sure if I took the time to learn DTrace I would come to love that too. But why in the hell can I not do a ‘tar -czvf my.tar.gz .’ without installing an extra package? Where is the ‘apt-get upgrade’ or the ‘yum update’? Of course I know the answer - it’s to keep binary compatability with previous releases. Joerg Moellenkamp’s article over at c0t0d0s0 does a great job at describing why. Some people would say, why not use OpenSolaris? The simple answer, stability. If I wanted something that was under active development with bleeding edge packages, I would just use Linux. Bottom line, I want my cake, and I want to eat it too. I want the convenience and ease of maintenance of Linux, but with the advanced features, stability, and scalability that comes with Solaris 10. Heck, I’ve even learned to love Sparc. Just give me that damn -z option to tar!!! In all seriousness, what Chris refers to would make me quite happy, and would like keep the legacy customers happy as well. Heck, I’ll even let them use the name Solaris-NG free of charge saving millions of money on marketing salaries. What are your thoughts?

I've done a lot with LVM in the past, but up until now had never really played around with LVM snapshots. I recently used LVM snapshots to implement a "hot backup" of my LAMP stack running this blog. I quote "hot backup" because, while mysql is indeed running, I do have to place a read lock on all tables for a second or two. You don't need to do this if you're using Innodb, but you do if you use the MyISAM engine (which Drupal does by default). The key to doing LVM snapshots is that you save some unassigned space in your volume group when you setup LVM in the first place. If you don't you can shrink existing logical volumes to make space, but that's beyond the scope of this article. For this article, the following assumptions are made:

• The LVM logical volume we want to back up is /dev/VolGroup00/apps
• The LVM snapshot is named appsnap (clever, huh)
• My personal use of snapshots is to replicate my current production LAMP stack to my development machine. You can use snapshots for just about anything - backup with Amanda or Bacula, or hand-rolled rsync/tar backups like mine. Implementation is the same, it's just the rsync command that you might choose to tweak

How much space does my snapshot logical volume need?

That’s a good question, and unfortunately it’s one that is answered with another question: “How long will your backup take, and how much writing to your source logical volume will occur during that time?”. I would have to assume that 500MB would be good for most setups, but YMMV. In the scripts I present later, our snapshot partition will take whatever is available to the volume group.

Step One: Create the snapshot

Assuming that your MySQL data files reside on the LVM partition that you’re snapshotting, and that you’re using the MyISAM table engine, we need to temporarily lock the tables. In my particular setup, creating the snapshot takes less than 2 seconds, so it’s no big deal to lock my tables for that long. Thanks to commenter Jay on how to fix my bug! Here’s the steps:

TMP_FILE="/tmp/msqlbackup-$$.sql"
/bin/cat > $TMP_FILE >>EOD
FLUSH TABLES WITH READ LOCK;
\! /usr/sbin/lvcreate -l100%FREE --snapshot --name appsnap /dev/VolGroup00/apps
UNLOCK TABLES;
EOD
/usr/bin/mysql -u root -pmypassword < $TMP_FILE
/bin/rm $TMP_FILE

At this point, we have a logical volume at /dev/VolGroup00/appsnap that contains our data exactly as it existed at the time of the lvcreate command above.

Step Two: Mount the snapshot

Pretty straightforward here:

/bin/mkdir /backups
/bin/mount /dev/VolGroup00/appsnap /backups

Now, we have our snapshot mounted at /backups. Next, let's back it up.

Step Three: Backup the mountpoint

This step is really up to you. You can tar up the contents, rsync it off somewhere, whatever you feel like doing. If you want to see my rsync command, keep reading.

Step Four: Remove the snapshot

Since the space required by the snapshot gets larger as more writes are made to our apps logical volume, you don't want this thing sitting around long. Let's unmount it, and remove it altogether since we have our data:

/bin/umount /backups
/usr/sbin/lvremove -f /dev/VolGroup00/appsnap
/bin/rmdir /backups

Tada, all done! Pretty painless, really. Now, we're sysadmins, and man, all that typing is for the birds. We need a script!

Tying it all together: Moving prod to dev

First, to make this work well, setup key-based authentication between your dev and prod servers (Google is your friend). On your production server, we need to create two scripts -- one to create the snapshot, one to remove it. I called mine makelvmsnapshot.sh and removelvmsnapshot.sh. Security Note: I placed my root mysql password in makelvmsnapshot.sh! I'm okay with this, make sure that you are before you do this. We'll set permissions on this file so that only root can see it. Here's the contents of makelvmsnapshot.sh:

#!/bin/bash
/bin/mkdir /backups
echo "FLUSH TABLES WITH READ LOCK;" | /usr/bin/mysql -u root -pmypassword 
/usr/sbin/lvcreate -l100%FREE --snapshot --name appsnap /dev/VolGroup00/apps
echo "UNLOCK TABLES;" | /usr/bin/mysql -u root -pmypassword 
/bin/mount /dev/VolGroup00/appsnap /backups

Since our password is in that file, do a 'chown root makelvmsnapshot.sh && chmod 700 makelvmsnapshot.sh' for some very basic security. Here is my removelvmsnapshot.sh:

#!/bin/bash

/bin/umount /backups
/usr/sbin/lvremove -f /dev/VolGroup00/appsnap
/bin/rmdir /backups

Now, on your dev server, create a script named prod2dev.sh, and place it in /usr/local/bin/. Here's what I have in it:

#!/bin/bash

#Stop Services
for s in httpd mysqld; do
    /etc/init.d/$s stop
done

#Create snapshot lv on prod
/usr/bin/ssh root@myprodserver.com /apps/scripts/makelvmsnap.sh

#Rsync w/delete
/usr/bin/rsync -aHvz --delete --exclude=httpd/log/* root@myprodserver.com:/backups/* /apps/

#Remove snapshot on prod
/usr/bin/ssh root@myprodserver.com /apps/scripts/removelvmsnap.sh

#Start services
for s in httpd mysqld; do
    /etc/init.d/$s start
done

Let's go over that a bit.

• First, I stop mysqld and httpd on my dev server.
• Next, I ssh to the prod server, and call the script that creates the snapshot.
• With the snapshot mounted at /backups in prod, I rsync the contents of prod back to dev (skipping apache access logs).
• Once the rsync is done, I remove the snapshot on production, and start mysqld and httpd on dev.

Conclusion

LVM snapshots are easy, fast, and effective ways to take a slice in time. The hardest part of using LVM snapshots is planning ahead and leaving some free space in your volume group for the snapshot logical volume. By utilizing LVM snapshots you can efficiently replicate setups between machines, or get backups of your LAMP stack. Even more important, you can test the restoration process on a daily basis. Your backups are only as good as your restoration procedure!

Linode is, in my opinion, the best webhost out there. I recently switched this site from Wordpress to Drupal. In preparation for this switch, I wanted to be able to start with a clean slate - wipe the O/S and all, and install Drupal on top of that. Since I'm a sysadmin, I hate downtime! Read on to learn how I made my Linode portable. Linode is a provider of Xen-based vm's. The intelligent way that they have provisioned their machines gives you a way to clone your VM. Using this idea, I was able to take an image of a Linode-ized CentOS installation, and run it as a VM under Xen on a spare workstation at home. I configured this Xen VM at home, but with every change I made to the image I updated a shell script to do the same. When I had my VM at home ready to go, I sufferred only 5 minutes of downtime while my Linode was being reconfigured! Here's the specifics on how to do this yourself.

Setup Finnix on your Linode

Follow the instructions for "Setting It Up" from the Linode Wiki article.

Setup the base OS

In your Linode dashboard, click "Deploy a Linux Distribution", and select settings like the ones in this screenshot: Now, after creating the partition, rename it to something like "Virgin CentOS Image", and resize it as small as it can go - I used 600M for mine. After resizing the partition, create a new raw partition of 10GB or so - I use this as an LVM partition and it's where I install all of my custom applications. Now, we need to make these new partitions available to your existing VM (or Finnix if this is your first). From the dashboard, click on the Finnix profile (or your custom profile) to edit it. Under the drive setup area, set /dev/xvdg to be the "Virgin CentOS Image" from above, and set /dev/xvdh to be the raw partition from above. If you changed settings to a profile that was already running, you need to reboot for the settings to take effect.

Setup your Xen server at home

For me, I just popped in a Centos 5.2 CD, and installed the base OS plus the virtualization package. Do not partition all the drive space, leave enough unpartitioned space to equal that of what you're using for your raw partion on your Linode. Once you've booted, we need to setup our raw partition space to use as swap and applications to the domU. Depending on your hard drive size and partition layout, there's no copy and paste solution for this. If you're geek enough to want to do this, you know how to use fdisk, don't you? For the sake of this article, /dev/hda3 is a 512M swap partition(type 82), and /dev/hda5 is a LVM partition of 10GB (type 8e).

Make the base image

Now, we need to make a file-backed image of the CentOS installation on our Linode. At the same time, we will create a gzipped backup so we have something local to restore from if we hose something up or want to test our provisioning script. On your Xen server at home, run the following as root: mkdir -p /var/vms/linode-clone/ cd /var/vms/linode-clone/ ssh root@mylinode.com "dd if=/dev/xvdg bs=102400" | gzip -dc | dd of=root.img bs=102400 gzip -c root.img > virgin.img.gz dd if=/dev/zero bs=1048576 count=2000 >> root.img e2fsck -f root.img resize2fs -p root.img Phew! What did that do? The ssh + dd command made a bit for bit copy of the virgin partition on your Linode, and created a file on the local host with it's contents. The gzip command made a compressed copy of the image to restore from later. The dd using /dev/zero padded the end of that file by 2GB, giving us 2GB at the end of the partition to work with. The e2fsck and resize2fs updated the filesystem contained within that file to the new size. Now, on to setting up our kernel.

Kernel setup

We need to grab a copy of the kernel source from Linode, and build it for our Xen domU. You can browse the Linode kernels at http://www.linode.com/src. I recommend 2.6.18.8-linode10.tar.bz2. From your Xen server, run the following: yum install gcc gcc-c++ kernel-devel mkdir -p /usr/src/ cd /usr/src/ wget http://www.linode.com/src/2.6.18.8-linode10.tar.bz2 tar -xjvf 2.6.18.8-linode10.tar.bz2 cd 2.6.18.8-linode10 ssh root@mylinode.com "cat /proc/config.gz" | gzip -dc > .config make oldconfig make cp arch/i386/boot/vmlinuz /var/vms/linode-clone/kernel-2.6.18.8-linode10.gz This creates a kernel identical to the one that your Linode host is running. Next up, setting up our Xen config.

Xen configuration

Change to /etc/xen, and create a file named linode-clone. Paste the following into it:

name = "linode-clone"
uuid = "a6864d4d-1c6f-ea12-3e08-b6b89015bb77"
maxmem = 540
memory = 540
vcpus = 1
kernel = "/var/vms/linode-clone/kernel-2.6.18.8-linode10.gz"
root = "/dev/xvda ro"
on_poweroff = "destroy"
on_reboot = "restart"
on_crash = "restart"
vfb = [  ]
disk = [ "file:/var/vms/linode-clone/root.img,xvda,w", "phy:/dev/hda3,xvdb,w", "phy:/dev/hda5,xvdc,w" ]
vif = [ "mac=00:16:3e:57:53:3f,bridge=xenbr0" ]
extra = "xencons=tty"

At this point, you should have a Xen domU ready-to-run. To boot your Xen VM and attach a console, run this:

xm create -c linode-clone

It should boot for you, and drop you into a login prompt. Feel free to login as root (using the password specified during the linode setup phase). Make some changes to the filesystem. CTRL+] will disconnect you from the domU console, 'xm console linode-clone' will re-attach. Now, power off your domU by doing a 'shutdown -h now' from within the domU. You can restore back to a virgin install by doing a "gzip -dc /var/vms/linode-clone/linode-virgin-centos.img.gz > /var/vms/linode-clone/root.img" -- make sure the domU is shut down first! When you boot back in by doing a 'xm create -c linode-clone', you'll see that your filesystem changes are gone. Stay tuned for part two, where I'll show you the basics of creating a provisioning script for your domU. Part three will then cover how to push your local images back up to your Linode.

Shameless plugging

Scattered throughout this post are links to Linode. If you're not a customer and would like to be, please consider clicking on one of those links to sign up. It will cost you nothing, but will give me a $20 credit on my account if you sign up and stay a customer for 90 days.