Inchoate Curmudgeon

I’m setting up a development environment on a Solaris x86 machine (well, actually a VMware virtual machine) and have a couple of goals:

• Use zones to set up, and clone, AMP (Apache, MySQL, PHP) and Rails development environments. The idea is once I get things the way I want, I should be able to quickly set up another zone with the same development bits.
• Use zfs to snapshot zones, so that I always have a “trail of breadcrumbs” to restore that zone.
• Use zfs clones or copes to speed zone deployment.

This blog entry covers basic setup. Future entries will cover AMP setup and cloning.

You might ask why I would use Solaris virtualization and zfs if I have VMware ESX? The main reason, over and above needing to fully understand Solaris virtualization and zfs for my job, is that I’m hoping with a combination of zfs and zones I can get finer grained control of the replicated environments and faster setup of new environments. Zones are a lot more efficent than VMware virtual machines. I eventually intend to create a VM on my Mac with Parallels or Fusion, and I should be able to get a bunch of usable zones out of a single Solaris instance.

As I go along, I’m going to try to minimize the amount of duplication of bits in the non-global zones with a combination of zfs mounts and loopback mounts into the global zone. My goal is to be able to boot a “clean” development zone in a few minutes. BTW, Blastwave is great for package management on Solaris. It’s “zone aware” and is able to manage updates pretty well.

The first step for me was to pre-populate my DNS server with some machine names and IP addresses for my zones. I use static IP addresses in my lab environment. Right now, my DNS is on a separate Ubuntu machine, but eventually I’ll move IT to a zone and maybe set up the Ubuntu DNS as a backup.

The next step was to set up a zfs pool for building my zones from. I want each zone to be in its own zfs file system. Since I’m dealing with a Solaris x86 VM on VMware, I created a couple of virtual disks and mounted them to the machine. If I didn’t have a disk to mount, I’d consider creating zfs filesystem from a file anyway, even though its “not recommended”, so I could use the zfs snapshot facilities I’m playing with.

So I ssh into my Solaris machine.

Notice I’m on Nevada build 48. All of this should work on any of the later Solaris 10 builds, but you should use Solaris Express and get one of the Nevada (Solaris 11) builds. With that, you get Firefox and Thunderbird as part of the default install. (Besides, if its good enough for Joyent, its good enough for me! I think they are on Open Solaris Nevada build 62.)

Next, I carefully run format, so I can “see” what disks are mounted to my machine. (Any experienced Solaris admin is probably groaning, since there’s probably a better and safer way to do this.) Anyway, format shows a list of disks, something like this:

Ignore everything after “0.” and “1.” on “AVAILABLE DISK SELECTIONS”. Those are iSCSI targets being picked up by the iSCSI initiator autodiscovery. The innocuous “Bad read of fdisk partition” is also related to this: format is picking up an iSCSI disk that my VMware instance has mounted, or I have a goobered up partition on one of the iSCSI luns. We don’t care.

Based on what format is telling me I’m pretty sure that c1t1d0 is the disk I’m going to use for my zpool, but just to make sure, I do a “df -k” and see what has a mounted filesystem on it. So I control-c out of format and do that.

I have some NFS mounts (Goliath, Gravid and Zaftig), but, it looks like c1t1d0 is not being used. We’ll check one more thing, since df doesn’t show the disks that comprise zpools. We run “zpool status”, which shows the status of any zpools, and what they are made of. Since we haven’t created any pools, it had better return “no pools available”. If it doesn’t, it will show any devices we have already assigned to zpools.

With my paranoia sufficiently assuaged, I can create a pool and a zfs file system with c1t1d0. In my scheme, I want to create a separate zfs filesystem for each zone I will create. I’m creating a very simple zpool, with no compression or mirroring. If you have redundant spindles you can create a zpool with a mirror or zraid to protect your storage.

OK, so “zpool create zpool01 c1t1d0″ does its thing, and returns nothing. That wasn’t very satisfying. (I guess the zfs commands aren’t very chatty.) We then run a “zpool list” and a “zpool status” to check everything out. This all looks good. Now we need to create a zfs filesystem hierarchy to put our zones in.

Our taciturn zfs commands create a zfs filesystem and mount it in the /export directory. Next we need to create a zone in this filesystem. For this, download “zonemgr” and install it somewhere in your path. We are using version 1.8. Zonemgr is the 80% solution for zone management that is much easier to use than the raw Solaris commands. For the actual zone creation, create a little bash script. We are using quite a few of the options, and you’ll want to reuse the script for creating other zones.

So, here is our script in vi. Lovely. The second line is ths first zonemgr command, and its so log it wraps. But, here is what the script does.

1. First it creates a zfs filesystem for our zone within the zpool/amp zfs filesystem. This will be mounted at /export/amp/leto because of where /export/amp is mounted.
2. Next is the big harry create zone command.

• -a add is the “add zone” command
• -n leto says “leto” is the zone name
• -Z “/export/amp/leto” tells zonemgr where to put the zone
• -t s says create a “sparse” zone
• -I “192.168.11.30|e1000g0|16|leto” says the IP address, the ethernet adapter I’m binding to (e1000g0), that this is a slash 16 subnet (netmask 255.255.0.0), and the hostname is “leto”, the same as the zone name
• -D says my domain is “home.louspringer.com”, so my fully-qualified host name is leto.home.louspringer.com. My nameserver (DNS) needs to resolve this, or the zone won’t boot very happily, thus one of the reasons I run a DNS server here in my office.
• -d “192.168.1.165″ says my DNS server is 19.2.168.1.165
• -P “secret” says what the root passwrod for the zone will be

3. The next line sets the leto zone to autoboot.
4. The last two lines deal with tarring the /opt/csw files from the global zone and untarring then into the leto zone. This is a kludge until I can figure out why zonemgr isn’t doing this. (If I do a pkginfo in the zone before I copy the csw directory, pkginfo says all this balstwave stuff is installed. It’s probably time to install a later version of Solaris 11.)

These commands create the zone and copies a bunch of packages into it. In my case, it copies all the blastwave packages I have installed in the global zone, and I have a bunch. Probably too many. This takes a while, and one of the things we may do is fix this so that we use loopback mounts where we are just copying packages. But, the upside of copying all these packages is that we can change or patch the zone installation differently than the other zones if need be.

I’ll do one last thing from the global zone to set up my vncserver and check out my brand spanking new zone.

cp -R /.vnc /export/amp/leto/root zlogin leto "env USER=root /opt/csw/bin/vncserver -geometry 1024x768 -depth 24"

This should start my vncserver in the zone with a copy of my vnc configuration from the global zone. So I fire up my vnc client on my Mac (Chicken of the VNC, gotta love that!)

Woo hoo! A new sandbox to crap up!

Future blog entries will install the optimized “AMP cool stack” and work with zfs snapshots.

Trackback URI | Comments RSS