Cat in the Red Hat

Blog has moved!

2008-07-25T06:24:00.001-07:00

I've moved my blog to my own server at http://tylerlesmann.com.

PXE Boot the VMware ESX installer

2008-07-15T13:04:00.000-07:00

If you are like me, then you want to use network booting for everything, especially installs. PXE booting the VMware installer with pxelinux is much easier than I thought it would be. First, copy the entire contents of the installer CD to your tftp directory. I copied mine to /tftpboot/esx/3.5.

Now, open your pxelinux.cfg/default. Add this entry, changing the path to each of the components to coincide with where you placed the files within your tftp directory.

label esx
kernel esx/3.5/mboot.c32
append esx/3.5/vmkernel.gz --- esx/3.5/binmod.tgz --- esx/3.5/ienviron.tgz --- esx/3.5/cim.tgz --- esx/3.5/oem.tgz --- esx/3.5/license.tgz --- esx/3.5/install.tgz

That's all you have to do.

Solaris: Improvements that could be made

2008-07-13T18:30:00.000-07:00

I've been studying Solaris 10 and 11 recently because I'm aiming to acquire a SCSA. It is not surprising that the most popular flavor of commercial Unix is losing ground.

Getting Solaris

One should not have to register/login to download Solaris or packages for Solaris. Get with the times, Sun. Solaris is supposed to be open source. That's means everyone should get easy access, no questions/logins. I'm talking about anonymous ftp, rsync, and bittorrent. What is more important? Solaris's statistics or relevance.

Package Management

It's not completely horrible. It does figure out dependencies. It's essentially just the equivalent of rpm or dpkg though. There is nothing like yum or apt-get on Solaris 10 or 11. This is a problem. Installing and updating software is a chore, especially installing Solaris Companion software, like vim on Solaris 10. The solution to this is easy. Sun needs to integrate the IPS system from OpenSolaris into their commercial versions.

The Shell

Why are Solaris administrators still using plain Bourne shell by default? Sun, be nice to your sysadmins and give them tab completion and command history by default. Bash is plenty mature to be the default shell. I cannot see any serious admin using Bourne, if they don't absolutely have to. Sun should see this too.

Caching Kerberos credentials for offline logins

2008-07-13T18:26:00.000-07:00

Having kerberos is a nice way to centralize your user passwords, but what if you have laptops that you would like to log onto away from your network? You have to setup credential caching. This is achieved through PAM and the pam_ccreds module. On a Fedora, or a Red Hat derivative, system, you only need to edit one file, /etc/pam.d/system-auth-ac. This file exists on any PAM aware system. Here is mine in its entirety.

auth required pam_env.so
auth sufficient pam_unix.so nullok try_first_pass
auth requisite pam_succeed_if.so uid >= 500 quiet
auth [default=ignore success=1 service_err=reset] pam_krb5.so use_first_pass
auth [default=die success=done] pam_ccreds.so action=validate use_first_pass
auth sufficient pam_ccreds.so action=store use_first_pass
auth required pam_deny.so

account required pam_unix.so broken_shadow
account sufficient pam_localuser.so
account sufficient pam_succeed_if.so uid < 500 quiet
account [default=bad success=ok user_unknown=ignore] pam_krb5.so
account required pam_permit.so

password requisite pam_cracklib.so try_first_pass retry=3
password sufficient pam_unix.so sha512 shadow nullok try_first_pass use_authtok
password sufficient pam_krb5.so use_authtok
password required pam_deny.so

session optional pam_keyinit.so revoke
session required pam_limits.so
session [success=1 default=ignore] pam_succeed_if.so service in crond quiet use_uid
session required pam_unix.so
session optional pam_krb5.so

I've highlighted the lines that do the work. It took me a while to make this. I've read tons of tutorials, as you probably have, to get this to work. In the end, I ended up mastering PAM configuration since none of the tutorial worked properly. I'm going to explain each line so that you will understand what is happening.

auth [default=ignore success=1 service_err=reset] pam_krb5.so use_first_pass

This line is the most complicated. The default=ignore tells PAM, if this module fails, like cannot reach kerberos server or password does not match, go on to the next step. The success=1 tells PAM if the module succeeds, a correct password verified by the kerberos server, skip one step. The service_err=reset, tells PAM to try again if the server reachable but down for some reason. The pam_krb5.so is the name of the module. Finally, the use_first_pass tells PAM to use the password that was first given to PAM during this run through and do not ask for a new one.

auth [default=die success=done] pam_ccreds.so action=validate use_first_pass

If kerberos was unavailable, PAM will run this line. The default=die means, if this module fails, everything fails and don't give the user access. The success=done means the auth portion of PAM is complete if this succeeds. pam_ccreds.so is the module name. The module, pam_ccreds, takes arguments and the action=validate has pam_ccreds check to see if the given password matches what is cached in /var/cache/.security.db.

auth sufficient pam_ccreds.so action=store use_first_pass

If the user entered a password that was verified by the kerberos server, PAM will apply this rule. The sufficient tells PAM if this module succeeds, the auth portion is complete. Otherwise, continue through the rest of the auth portion. The action=store, if you haven't guessed, stores the encrypted password in /var/cache/.security.db.

Now the system will cache kerberos credentials!

One gotcha on SELinux enabled systems, I have not been able to get this to work in enforcing mode. PAM is horrible restricted. I cannot find an appropriate file context to apply to /var/cache/.security.db that will allow pam_ccreds to write to it. I recommend using Permissive mode until I find a solution. This should not be too big a deal because you should only be using this on laptops, not critical systems.

One more gotcha. For some reason I have not discovered yet, xscreensaver cannot use ccreds properly. If you lock your machine, then you will not be able to log in without setting up a local password. I'm working on a solution for this.

Fun with Preseed

2008-07-06T13:58:00.000-07:00

One of the things I do is give away older computers with Ubuntu on them. I've given away quite a few and the installs can take a while, plus I like to install flash and dvd decryption for the new users. I learned about preseed from this post. It didn't have all the information I needed to create this, so that is why I'm sharing my experience.

Preseed is Debian's/Ubuntu's answer to unattended installation. It will do everything for you, from partitioning to setting up users. You just need to create a text file, make it available, and tell the installer where it is. First, here is my preseed for ubuntu hardy.

### Network configuration
d-i netcfg/choose_interface select eth0
d-i netcfg/get_hostname string ubuntu

### Mirror settings
d-i mirror/country string enter information manually
d-i mirror/http/hostname string ftp.deathcat.dci
d-i mirror/http/directory string /ubuntu
d-i mirror/http/proxy string

### Partitioning
d-i partman-auto/init_automatically_partition
d-i partman-auto/disk string /dev/sda
d-i partman-auto/method string regular
d-i partman-auto/purge_lvm_from_device boolean true
d-i partman-auto/expert_recipe string boot-root :: 100 50 100 ext3 \
$primary{ } \
$bootable{ } \
method{ format } \
format{ } \
use_filesystem{ } \
filesystem{ ext3 } \
mountpoint{ /boot } \
. \
512 512 200% linux-swap \
method{ swap } \
format{ } \
. \
512 10000 1024 reiserfs \
method{ format } \
format{ } \
use_filesystem{ } \
filesystem{ reiserfs } \
mountpoint{ / } \
. \
512 10000 1024 reiserfs \
method{ format } \
format{ } \
use_filesystem{ } \
filesystem{ reiserfs } \
mountpoint{ /tmp } \
. \
1000 10000 2000 reiserfs \
method{ format } \
format{ } \
use_filesystem{ } \
filesystem{ reiserfs } \
mountpoint{ /var } \
. \
3000 10000 4000 reiserfs \
method{ format } \
format{ } \
use_filesystem{ } \
filesystem{ reiserfs } \
mountpoint{ /usr } \
. \
100 10000 100000000 reiserfs \
method{ format } \
format{ } \
use_filesystem{ } \
filesystem{ reiserfs } \
mountpoint{ /home } \
.
d-i partman/confirm_write_new_label boolean true
d-i partman/choose_partition \
select Finish partitioning and write changes to disk
d-i partman/confirm boolean true

### Clock and time zone setup
d-i clock-setup/utc boolean true
d-i time/zone string America/Denver

### Installation setup
d-i prebaseconfig/reboot_in_progress note

d-i debconf/priority select critical
debconf debconf/priority select critical

base-config base-config/intro note
base-config base-config/login note

# The user's name and login.
passwd passwd/user-fullname string Ubuntu
passwd passwd/username string ubuntu
# And their password, but use caution!
passwd passwd/user-password ubuntu ubuntu
passwd passwd/user-password-again ubuntu ubuntu

# install desktop + standard packages
tasksel tasksel/first multiselect ubuntu-standard, ubuntu-desktop
d-i finish-install/reboot_in_progress note

# post install scripts
d-i preseed/late_command string \
cd /target; \
wget ftp://ftp.deathcat.dci/pub/ubuntu/scripts/post-install; \
chmod +x ./post-install; \
chroot ./ ./post-install; \
rm -f ./post-install

A lot of stuff is in there as you can see. Let's look at the part under Network Configuration. The first line, d-i netcfg/choose_interface select eth0, tells the installer to use eth0 for networking during the install. This does absolute nothing during a PXE boot install, which is irratating if you are installing on to laptops with a wireless card. The next line, d-i netcfg/get_hostname string ubuntu, tells the install to give the new install the hostname ubuntu.

Now, to the Mirror settings section. This section tells the installer I want to manually set the location of the installer packages to http://ftp.deathcat.dci/ubuntu and not to use a proxy.

The Partitioning portion is the most perplexing. It was for me anyway. I like to set up several partitions, for /boot, swap, /, /var/, /usr, and /home. I also like to use reiserfs for speed. The first two lines tell the installer to automatically wipe /dev/sda and prepare to partition it. The next line tells it to use regular partitions opposed to lvm and after that is installer is told to not warn when deleting an lvm. Now, for the meat and potatoes of the partition. This one line( Yes, that is one line and must be so to function ) defines how the installer should do the actual partitioning.

d-i partman-auto/expert_recipe string boot-root :: 100 50 100 ext3 \
$primary{ } \
$bootable{ } \
method{ format } \
format{ } \
use_filesystem{ } \
filesystem{ ext3 } \
mountpoint{ /boot } \
.

The first thing is leave the 100 50 100 ext3 on the first line. Don't attempt to place the backslash after the :: and continue on the next. It doesn't work. I wish it did. Anyway, the first 100 says to give this partition a minimal size of 100MB. The 50 is the priority. The lower it is, the more likely it will be filled to maximum size, which is the second 100. Ext3 is the partition type to mount this partition as. $primary{ } ensures this partition will be a primary partition and $bootable{ } will set the boot flag. method{ format } tells it to format this partition and not preserve the filesystem as does the next line. Preseed is redundant, is it? Use_filesystem{ } tells it to auto detect the filesystem for mounting this time. Filesystem{ ext3 } says use ext3 for formating. Finally, mountpoint{ /boot } tells the installer to mount this to /boot. The "." terminates the configuration of the one partition. Read the rest of that line for more examples.

The three lines after that one big one, tell the installer to write the partitions to disk, format, and not ask questions.

The Clock and time zone setup section sets up the hard clock to use UTC and the timezone to that of Denver, Colorado.

The Installation Setup tells the system to first reboot automatically after install. It also tells the system to crash if debconf doesn't like the rest of the file.

The user's name and login portion is self-explanatory. You'll note that the password ubuntu is written four times, twice on two lines. Why it is required to type it more than once, I don't know. Preseed is redundant!

The next section tells the installer to install the ubuntu-standard and ubuntu-desktop packages and their dependencies. It also tells the installer to reboot after install...again.

At the end of the preseed, I have the installer download a file and execute it in a chroot of the new install. This is that file with comments detailing what's going on.

#!/bin/sh

SCRIPTURL=ftp://ftp.deathcat.dci/pub/ubuntu/scripts

cd /tmp

# Setup auto login
UBUNTUUSER=$( getent passwd 1000 | cut -d : -f 1 )
wget "$SCRIPTURL/gdm.conf-custom"
sed "s/AutomaticLogin=ubuntu/AutomaticLogin=$UBUNTUUSER/g" \
gdm.conf-custom > gdm.conf-custom2
mv /etc/gdm/gdm.conf-custom /etc/gdm/gdm.conf-custom.orig
cp gdm.conf-custom2 /etc/gdm/gdm.conf-custom
rm -f gdm.conf-custom gdm.conf-custom2

# Setup ubuntu sources
wget "$SCRIPTURL/sources.list"
rm -f /etc/apt/sources.list
cp sources.list /etc/apt/sources.list
rm -f sources.list

# Get updates
apt-get update
apt-get -y upgrade

# Install flash, vlc, and dvd playing software
apt-get -y install \
gstreamer0.10-plugins-ugly-multiverse \
gstreamer0.10-plugins-bad-multiverse \
gstreamer0.10-plugins-bad \
gstreamer0.10-plugins-ugly \
gstreamer0.10-ffmpeg \
libxine1-ffmpeg \
libdvdread3 \
flashplugin-nonfree \
vlc

# Setup DVD decryption
/usr/share/doc/libdvdread3/install-css.sh

exit 0

And that's it. One more note before I finish. To make this entirely hand-free, you will have to past a few arguments at boot: The locale, keymap, network interface to use, and the location of the preseed file. I do this in my pxelinux.cfg/default.

label ubuntu
kernel ubuntu/hardy/i386/linux
append vga=normal initrd=ubuntu/hardy/i386/initrd.gz \
locale=en_US.UTF-8 debian-installer/keymap=us \
netcfg/wireless_wep= netcfg/choose_interface=eth0 \
netcfg/get_hostname=ubuntu \
preseed/url=ftp://ftp.deathcat.dci/pub/linux/ubuntu/preseed/easy.cfg --

Loading pxegrub from pxelinux!

2008-07-06T13:40:00.000-07:00

Pxelinux can do a lot of things, but loading the solaris installer is not one of them. For that, it is easier to use pxegrub. What can you do if you still want to use pxelinux as your primary network bootloader? You can easily make an entry in your pxelinux.cfg/default for pxegrub!

First of all, here is what my /tftpboot directory without the many linux, bsd, and solaris boot files:

/tftpboot/boot/grub/menu.lst
/tftpboot/pxelinux.cfg/default
/tftpboot/pxegrub.0
/tftpboot/pxelinux.0

If you are paying attention, you'll notice that pxegrub's filename is different than normal. You have to end the filename with a .0. If you don't, then pxelinux will not load it properly, which caused me a bit of confusion. Also, you will need to place pxegrub's configuration, menu.lst, in /boot/grub at the root of your tftp. Pxegrub looks there by default.

If you have everything set as so, you will be able to load pxegrub from pxelinux with the following clause:

label g
kernel pxegrub.0

It's really that easy.

Solaris: Setting up a NFS server

2008-06-29T18:34:00.000-07:00

This assumes you've already setup the OpenSSH server on the machine.

First, you will need these packages installed in this order:

SUNWgssk
SUNWrsgk
SUNWnfscr
SUNWnfsckr
SUNWnfscu
SUNWnfssr
SUNWnfsskr
SUNWnfssu

Now, you must define your exports before nfsd will start. You do this by altering /etc/dfs/dfstab. To share /export/install for instance, where I have the Solaris installer files, I would add this line:

share -F nfs -o ro /export/install

This line gives everyone read-only access to my install files.

Now, nfs can be started and made to start at boot with these commands:

svcadm enable -t nfs/server
svcadm enable nfs/server

You can check to see if everything is working with a showmount -e.

Solaris: Installing SSHD

2008-06-29T18:19:00.000-07:00

This assumes the Reduced Network Support software group was chosen during initial installation.

These packages will need to be installed in this order:

SUNWgssc
SUNWgss
SUNWsshcu
SUNWsshdr
SUNWsshdu

After these are installed, you will have to generate host keys for sshd. You do this with the following command:

/lib/svc/method/sshd -c

Now that the keys exist sshd can be started with this command:

svcadm enable -t ssh

And made to start at boot with this command:

svcadm enable ssh

Windows Entropy

2007-10-25T15:52:00.000-07:00

If you've used a Windows installation for an extended period, six months or more, then you may have noticed the system gets slower and slower as time passes. This is what I call Windows Entropy.

The main cause is the New Technology File System, or NTFS.

When you first install Windows, or format a new NTFS partition, a few things are created. The two that matter here are:

The Master File Table, or MFT, which is usually contiguous, reserved space for storing file metadata, like location, size, acls, and timestamps.
The unreserved space, where data is placed.

Initially, the data and metadata are close together on the disk. As the data grows to fill the available space, the distance between the metadata and data grows because Windows will try to keep the MFT contiguous and in one spot [source]. This distance increases seek time, which slows the whole system down because metadata is accessed frequently.

What makes it worse is the partition does not have to fill up to much cause this. If you delete a file and create a new same size or smaller file, Windows may not place the new file in the space of the old file. The data may be placed instead sparsely [source]. This further increases the distance between the data and the MFT. Other file systems do this as well because it avoids fragmentation by giving files room to grow, which is exactly why Microsoft implemented it in NTFS5. These other systems do not try to store their metadata in one contiguous space though. Ext2, for instance, stores its metadata in inodes, which reside in block groups, which are placed throughout the partition [source].

Creating and moving your Linux software raid with mdadm

2007-09-11T09:33:00.000-07:00

I thought I would document this for reference for myself and anyone else who might benefit. I recently upgraded with main workstation, replacing my 1.2TB LVM made up of five 250GB drives with a 2TB raid array of five 500GB drives. The additional challenges were that I would have to transfer the data off of the old LVM onto the new raid and I would have to do this over the network as neither motherboard has 10 SATA connections. To further complicate things, I want to use one of the current drives of the LVM as my new system drive. Here is the process.

After setting up the hardware for both systems, I booted the new raid system with a Fedora Core 7 DVD and entered rescue mode by entering linux rescue at the ISOLINUX prompt. Normally, I would setup the raid during the normal Fedora installation. However, I want to use one of the LVM members for the new installation, which I also need to pull data off of first.

Now that I'm in rescue mode, the first thing to do was partition each raid member with one big Linux Raid Autodetect or fd type partition. Once this was done, I could create a level 5 raid with this command:

mdadm --create /dev/md0 --level=5 --raid-devices=5 /dev/sd[abcde]1

When I ran this command the first time, mdadm told me that /dev/sda1 was too small to be part of the array. For some reason, the kernel detected the changes in the other drives, but not the first one. I rebooted the system, as partprobe is not available in rescue mode, and then command worked like a charm.

The next step was to put a file system on the new raid device. I used XFS because the files that would live on this file system were very large, about 100MB to several gigabytes. I formatted /dev/md0 with this command, which is about how you could format any drive with XFS:

mkfs.xfs /dev/md0

Next, I mounted the raid, setup an NFS export on the old system, and mirrored about a terabyte of data. This took about 16 hours over gigabit ethernet.

The next day, after copying completion, I installed Fedora Core 7 on one of the old members of the LVM. After installing, I would then configure the raid as my /home. I had to assemble the raid before I could use it. You could use this method to assemble a raid moved from another system as well as what I am doing.

Before I could assemble it, I needed the raid's UUID. This is easy to get. All you need to know is the device file of one of the raid members, which the first on mine is /dev/sdb1, and you can ask mdadm.

mdadm --examine /dev/sdb1

In the cascade of output, you will see UUID presented as a string of hexidecimal characters. Now, we run this command to assemble the raid device:

mdadm --assemble --name=/dev/md0 --uuid=PUT_UUID_HERE /dev/sd[bcdef]1

The raid device is all ready to mount. Now, I want this raid device to be prepared at every boot. I do this by creating /etc/mdadm.conf and insert this:

DEVICE /dev/sd[bcdef]1

ARRAY /dev/md0 UUID=PUT_UUID_HERE

MAILADDR root@localhost

That's it. Remember to update your /etc/fstab, if you want your raid to be mounted every boot.

X.org for Sharp Aquos and nVidia 8800 GTS

2007-09-07T06:52:00.000-07:00

Configuring X for functionality with an HDTV and any nVidia card is quite a frustrating adventure. This is the procedure I took to configure X for my nVidia 8800 GTS ( g80 ) connected to my Sharp Aquos LC-32GP1U through DVI-D. This Aquos is a 1080p monitor, so I am configuring it to run at 1920x1080.

When I first start X, I get a display. The resolution is only 1280x1024. So the first step is to see what X's problem is. The easiest way is to first enter runlevel 3 and start X from there with added switches for verbosity.

Use Ctrl+Alt+F1 to switch to the first virtual terminal.
Login and run telinit 3 to enter runlevel 3.
Start X with startx -- -verbose 6 --logverbose 6 2> /tmp/startx.log
As soon as X displays the screen, press Ctrl+Alt+Backspace to kill X.

Now, we look at /tmp/startx.log. We are looking for the mode validation checking for possible resolutions, specifically 1920x1080. In this output, X tells us that 1920x1080 is to large for DFP, which it isn't because this is a 1080p monitor. We need to tell X to not worry about DFP. Insert the following line in the Device section:

Option "ModeValidation" "NoDFPNativeResolutionCheck"

Now start X with startx -- -verbose 6 --logverbose 6 2> /tmp/startx.log. We'll see nothing has changed. Kill X and look at /tmp/startx.log. Look at the mode validation. X is now complaining about timings. We need to tell X to use exact timings, which we do by adding this line to our Device section:

Option "ExactModeTimingsDVI" "True"

Now start X with startx -- -verbose 6 --logverbose 6 2> /tmp/startx.log. We'll see that the screen is deformed. Kill X and look at /tmp/startx.log. Nothing will scream out the problem unfortunately. The resolution is actually correct, 1920x1080. However, the image is not correctly scaled. It took me many frustrating google searches to find a solution for this. We need to add this line to our Device section to have the video card use the monitor's native scaling.

Option "FlatPanelProperties" "Scaling=Native"

Start X and voila. Beautiful 1080p resolution, perfectly scaled. Here is the entire /etc/X11/xorg.conf from my machine:

Section "ServerLayout"
Identifier "single head configuration"
Screen 0 "Screen0" 0 0
InputDevice "Keyboard0" "CoreKeyboard"
EndSection

Section "Files"
ModulePath "/usr/lib64/xorg/modules/extensions/nvidia"
ModulePath "/usr/lib64/xorg/modules"
EndSection

Section "ServerFlags"
Option "AIGLX" "on"
EndSection

Section "InputDevice"
Identifier "Keyboard0"
Driver "kbd"
Option "XkbModel" "pc105"
Option "XkbLayout" "us"
EndSection

Section "Monitor"
Identifier "Monitor0"
EndSection

Section "Device"
Identifier "Videocard0"
Driver "nvidia"
Option "ExactModeTimingsDVI" "True"
Option "ModeValidation" "NoDFPNativeResolutionCheck"
Option "FlatPanelProperties" "Scaling=Native"
Option "AddARGBGLXVisuals" "True"
Option "DisableGLXRootClipping" "True"
EndSection

Section "Screen"
Identifier "Screen0"
Device "Videocard0"
Monitor "Monitor0"
DefaultDepth 24
SubSection "Display"
Viewport 0 0
Depth 24
Modes "1920x1080"
EndSubSection
EndSection

Section "Extensions"
Option "Composite" "Enable"
EndSection