Setting Up VRRP on Netgear Managed L3 Switches

Preliminary steps

1. Make sure your two switches are connected together, preferably via port-channels (lag)

2. make sure the VLANs you want to apply VRRP to are enabled on the port channels so that they span both switches



Configuration

In this example, we'll create a new routing VLAN (VLAN 15 for this example) on two switches, switch-a and switch-b. The primary ip will be 192.168.211.1, and the secondary will be 192.168.211.2. the VRID for the VRRP will be 15 (for convenience - but not strictly necessary. VRID 1 would work as well, I just find it easier to name it after the vlan.) The virtual IP will be 192.168.211.254.

Master switch

1. log into switch-a

2. enable

3.

vlan database
vlan 15
vlan routing 15
exit

4.

configure
ip routing
ip vrrp
interface vlan 15
ip address 192.168.211.1 255.255.255.0
ip vrrp 15
ip vrrp 15 ip 192.168.211.254
ip vrrp 15 preempt
ip vrrpt 15 authentication simple my_pass
ip vrrp 15 mode (enables vrrp on this interface)
exit
exit

5. verify that VRRP 15 is behaving correctly:

sh ip vrrp interface vlan 15 15
Primary IP Address............................. 192.168.211.254
VMAC Address................................... 00:00:5e:00:01:50
Authentication Type............................ None
Priority....................................... 100
Advertisement Interval (secs).................. 1
Pre-empt Mode.................................. Enable
Administrative Mode............................ Enable
State.......................................... Master

Backup Switch

1. log into switch-b

2. enable

3. enter the following commands:

vlan database
vlan 30
vlan routing 30
exit

4.

configure
ip routing
ip vrrp
interface vlan 15
ip address 192.168.211.2 255.255.255.0
ip vrrp 15
ip vrrp 15 ip 192.168.211.254
ip vrrp 15 priority 1 (makes this a secondary - it defaults to 100)
ip vrrp 15 preempt
ip vrrpt 15 authentication simple my_pass
ip vrrp 15 mode (enables vrrp on this interface)
exit
exit

5. verify that VRRP is working correctly

sh ip vrrp interface vlan 15 15
sh ip vrrp interface vlan 15 15
Primary IP Address............................. 192.168.211.254
VMAC Address................................... 00:00:5e:00:01:50
Authentication Type............................ None
Priority....................................... 1
Advertisement Interval (secs).................. 1
Pre-empt Mode.................................. Enable
Administrative Mode............................ Enable
State.......................................... Backup

Automount Home Directories Over NFS in Linux

On RedHat Enterprise/CentOS...
assuming your home server is called "fs1" and is sharing out /home over NFS. You'll also want to make sure that both your home directory server and the server you're running autofs on has the same passwd/user info - ids, gids.

1. Make sure autofs is installed and running:

sudo chkconfig autofs on
sudo service autofs start

2. add "/home /etc/auto.home" to /etc/auto.master

3. create the file /etc/auto.home and add these lines:

* -fstype=nfs,rw,nosuid,soft fs1:/home/&

4. if you already have a /home dir, move it out of the way:

sudo mv /home /home.old

5. reload autofs

sudo service autofs reload

6. ls /home

You should see the users' home directories.

Building Bacula Client RPMs for CentOS 5.x

This is a quick and dirty explanation of the process. I'm building as root - which you should probably avoid doing.

Download and install the srpm file from bacula's site (sudo rpm -ivh name-of-srpm.srpm)

You'll need to install the rpmbuild utility:



sudo yum -y install rpm-build


To build the client only:

Install a couple of prerequisite rpms:
sudo yum -y install ncurses-devel libtermcap-devel

rpmbuild --define build_client_only=1 --define build_centos5=1 -bb /usr/src/redhat/SPECS/bacula.spec

Cisco ASA VPN and RSA SecurID Appliance

I recently set up an RSA SecurID Appliance as a authentication source for a Cisco ASA 5510 running 8.0.x firmware. The basic setup of the box was pretty straightforward. It runs a stripped down Linux distribution with a 2.6.24.x kernel.

Anyway, after setting up an authentication source using a Windows 2003/2008 Active Directory domain controller and importing a batch of time based RSA key token, I set up the ASA to authenticate off the Radius server. Here's the necessary config on the ASA:

aaa-server rsaapp protocol sdi
aaa-server rsaapp (INSIDE) host 10.14.14.50 MY_PASSWORD_FOR_RADIUS_CLIENT

tunnel-group employees type remote-access
tunnel-group employees general-attributes
address-pool employees-pool
authentication-server-group rsaapp
default-group-policy operations
tunnel-group operations ipsec-attributes
pre-shared-key *

Here are several important things to do:

1. set up DNS entries for the RSA box and the ASA, both forward and reverse/PTR. The box seems to be looking for its FQDN. You can use the host file for setup.

2. make sure the ASA, RSA box, and domain controller all have accurate time (via NTP, etc.)

3. setup a radius client on the RSA box and use the same pass phrase you used in the ASA aaa-server config

4. assign token devices to users... start off with one user for testing.

5. Re-synchronize the token. I'm not 100% sure this is necessary, but I tried several tokens, and this seemed necessary.

6. Have the user log into the self-service console:

https://myrsaappliance.mydomain.local:7004/console-selfservice

He or she should log into the console with their active directory username and password. He or she should then set a PIN on the token.

7. Wait for a minute or a two, and then have the user log into the VPN appliance with the Cisco client. This seemed to be necessary, as the token didn't seem to work at first. After running through the configuration again, I tried waiting, and this worked.

Entourage 2004 Crashes After Updates on OS X 10.5.8

I recently set up a new Mac Pro. I installed Office 2004 and copied the user's files over. Entourage started fine, and the user's Exchange mailbox seemed to open correctly.

I patched the installation of office, and Entourage started crashing shortly after launching it. I then removed Office 2004, and reinstalled it. This time, Entourage said that it could not open this version of the database. I patched 10.5.2 to 10.5.8, to no avail.

I tried various fixes other people suggested on the web. In the end, I patched 2004 with the latest patch, killed the Microsoft Database process. Since the user did not keep a PST, I simply moved the user's Entourage data in documents (In a Microsoft folder) to another location - and then reconfigured the entourage client for exchange.

Entourage opened with no issues.

Exchange 2003 Not Starting - EventID 5197

We recently had a problem with an Exchange server running Windows 2003 server standard and enterprise exchange. We recabled the entire network, but did not perform any address changes. However, the exchange server would not boot correctly after restart. The Windows Server would hang on "applying settings" - unless you unplugged the NIC. If you unplugged the NIC, the box would use your cached local account and allow you to log in.

At any rate, I checked the two domain controllers for signs of problems - but could not find any. The error in the event log on the exchange server was event 9157. There was an MS article vaguely relating to it (http://support.microsoft.com/kb/297295) - but the exchange server's account was definitely present in AD users and computers. In spite of that, I did use ADSI Edit to make the suggested changes in that article, but it did not help.

As it turned out, someone had remove the Exchange security groups out of the default users OU a month before this happened. It just so happened that the exchange server had not been rebooted since then. I'm guessing that one of the group policies didn't apply - as it was probably hardcoded to an entry in ad users and computers

I moved both of those groups back to the users OU, and restarted the exchange server; it came back up after reboot. What I guess happened was that

1-Wire Temperature Sensors...

I recently bought a 1-Wire serial interface (Link45) and two T-sensors from iButtonlink as well as a power injector and power adapter. The power adapter was back-ordered, but arrived today. Using a USB serial adapter and Digitemp, I get results like:

user@testmachine$ sudo digitemp_DS9097U -s /dev/ttyUSB0 -a
DigiTemp v3.5.0 Copyright 1996-2007 by Brian C. Lane
GNU Public License v2.0 - http://www.digitemp.com
Jul 06 22:26:20 Sensor 0 C: 22.81 F: 73.06
Jul 06 22:26:21 Sensor 1 C: 22.56 F: 72.61

One of the nice things is that you can add more sensors and also things like humidity sensors.

It's pretty useful. I discovered that someone had written a little plugin/wrapper for Nagios here.

Windows Vista Displays the Wrong CPU Device

I recently upgraded a laptop's CPU to a higher clocked model. The bios recognized the new CPU immediately, of course, but the device manager in Vista (Ultimate 64bit) still showed the old dual cores (T6570 @2.1GHz.) Running scans for hardware changes did not help, nor did updating the performance tool.

I did some web searches (though none too thorough) and didn't find any real answers. I felt brave (don't try this at home!) and went into device manager and "uninstalled" one of the cores of the processor and rebooted.

When the box came back up, one of the cores was labeled T6570 @2.1Ghz, but the other core was now labeled correctly: T9300 @2.5Ghz. I repeated the procedure with the second core and rebooted. Voila! Both cores show the correct name and speed.

If you try to do this yourself, you do so at your own risk, of course. It'd probably be smart to do a backup, but I was too impatient.

DB9 - RJ45 Serial Pinouts

I frequently use serial connections for access to network devices. I find it a hassle to carry around the old serial cables... especially with limited space in my bag. On the other hand, I always have a few cat 5/5e cables handy. I decided years ago to just start using the DB9 to CAT5 adapters you can get at places like Radio Shack.

I labeled these pinouts A and B, but those names are arbitrary. You'll find that layout B is most commonly used in the blue cisco cables... particularly the older cables that had a separate connector. In both cases, the green wire is optional (and is soldered to the red wire, at least in the Cisco version) - at least with all the network gear I've worked with in the last few years. Also, pins 1 and 9 aren't used in either version.

The advantage of these two are that you need only two of these adapters (one of each) and a cat 5 or cat 5e cable.

If you need to connect to a piece of Cisco gear (they typically use the RJ45 serial port,) just use the type A connector and a cat5 cable direct to the Cisco box.

If you need to connect to a piece of gear that uses DB9, place one connector on each end of the cat 5 cable.

type	Pin 1	Pin 2	Pin 3	Pin 4	Pin 5	Pin 6	Pin 7	Pin 8	Pin 9
A	x	black	yellow	brown	red (+green)	orange	white	blue	x
B	x	yellow	black	orange	red (+green)	brown	blue	white	x

Nagios, FreeBSD, and Lilac

In the earlier post, I talked about setting up nconf on FreeBSD. I decided to give Fruity/Lilac a try. It's been around longer, anyway, and it supports imports of existing configs. It also can use nmap to generate a basic configuration.

Anyway, the rough steps are:

1. install mysql50-server

2. install apache22

3. install php5

4. install php5-extensions with the following modules enabled:

json, pcntl, posix, mysql, curl

5. install php5-pdo_mysql

6. install nmap

7. untar the lilac source to /usr/local/www/apache22/data/lilac

8. sudo chown www:www /usr/local/www/apache22/data/lilac/includes

9. restart apache.

Note: in order to get apache to work with php, I created a php config file in /usr/local/etc/apache22/Includes/php.conf:

DirectoryIndex index.php index.html index.htm
AddType application/x-httpd-php .php .htm .html
AddType application/x-httpd-php-source .phps

Netgear Switches (FSM7352S) and Disabling Stackports

These are decent switches. The FSM7352S is a 48 port layer 3 10/100 switch with 4 copper/fiber combo ports. The last two ports are set to be stacking ports by default, and cannot be used for normal purposes.

To disable stacking from the command line:

enable (if you aren't there already)
configure
stack
stack-port 1/0/51 ethernet
stack-port 1/0/52 ethernet

To revert them back to stack ports:

configure
stack
stack-port 1/0/51 stack
stack-port 1/0/52 stack

Nagios, FreeBSD, and nconf

I just noticed nconf on the Nagios.org site the other day. Another GUI for configuration. Anyway, I was building a new VM for use by our consultants to run nagios, RT, and cacti, and I figured I would give it a shot.

Nconf is yet another web frontend for the configuration of Nagios. It seems to work okay. It's PHP based. It's not in the ports tree, but then again, it hasn't been around very long. Here's what I did to install it:

1. install nagios, apache, php5, php5-extensions, mysql (It can't use PostgreSQL at this point) etc.

2. Create an appropriate php5.ini in /usr/local/etc and make sure it contains at least:

short_open_tag = On
register_globals = Off
magic_quotes_gpc = Off

3. untar nconf into /usr/local/www/apache22/data/nconf

4. cd to that dir

chown www config output temp

5. sudo mysql (-p if you have a root password set at this point, if not it's a good idea)

CREATE DATABASE nconf;
GRANT SELECT, INSERT, UPDATE, DELETE, CREATE, ALTER, DROP on nconf.* to 'nconf_user'@'localhost' identified by 'difficult_password';
flush privileges;


6. add www to nagios group

7. restart apache (I also had to add:

AddType application/x-httpd-php .php .htm .html
AddType application/x-httpd-php-source .phps

to /usr/local/etc/apache22/httpd.conf)

8. Browse to http://your-host-ip-or-name/nconf and go through the setup (set the path to nagios to be /usr/local/bin/nagios)

9. remove INSTALL.php and the INSTALL directory in nconf

10. Download the default imagepak-base.tar.tar from here.)

11. rename the image pack to imagepak-base.tar.gz

12. cd /usr/local/www/nagios/images/logos
sudo gunzip /path/to/the/imagepak.tar.gz
sudo tar -xf /path/to/the/imagepak.tar

The base install is done.

It will generate a tarball with the configs in /usr/local/www/apache22/data/nconf/output. You'll need to untar it into /usr/local/etc/nagios and configure nagios.conf to parse the directories for configs.

I hope the project lasts. There have certainly been a lot of webguis for Nagios that have died. Hopefully, they will add the ability to import existing configs at some point.

ZFS, the ZIL, iSCSI, NFS, and ESXi

too many acronyms for one title...

Anyway,

I've been working with Solaris 10 u6 x86 and ZFS. I started dabbling in it about two years ago, but I wasn't too interested. I guess I just needed another couple of years worth of pain wrestling with competing volume managers like Sun Volume Manager, LVM, etc.

We have several clients with real interest in a Solaris/ZFS solution for fileservers between branch offices and for DR planning. To prepare for this, I've been testing Solaris 10u6 with a couple of ASUS barebones, 4GB RAM, 4x 320GB 7,200RPM 16MB cache 2.5" SATA drives, and a dual core celeron clocked at 2GHZ. The boxes also have an Intel PCI-E desktop NIC. Not terrible, but nothing like the "Thumpers" that all the blogging Sun engineers are using (128GB RAM!)

The testing has been going fairly well. I know the system could use more RAM (I'm working on getting a used Dell PE and adding 16GB RAM) - but this will be good for demos. The snapshotting alone should be able to hook quite a few people. I recently purchased an OCZ Vertex 30GB SSD to test out dedicated ZILs. For those that don't know already, the ZIL is the log device for a given pool - by default, stored across the drives in a pool. The ZIL is used in client caching. If you do use a separate device for the ZIL, it would be safest to use ZIL devices in pairs as mirrors.

Anyway, I added it to my little pool as a dedicated ZIL device. I noticed right away that it had zero impact on iSCSI performance (I guess I should have realized that the ZIL wasn't for use by the iSCSI targetd.) I was getting near full line speeds on a gigabit Windows Vista client using the windows iSCSI initiator, a ZFS backed iSCSI lun, and the ATTO disk benchmark.

I decided to create a regular ZFS volume and share it out in NFS (zfs create -osharenfs=root=my.esxi.box's.ip-address mypool01/vols/nfs1) so I could mount it in ESXi as the datastore. I set up an ESXi install on a USB stick and booted another machine with a Q9400 Intel processor (4 cores at 2.5GHZ) with 4GB RAM and only the USB stick for a hard disk. I then added the NFS share as a datastore, and proceeded to install FreeBSD 7.1 i386 on a new vm.

It changed everything in regard to the ZIL. Whereas I was seeing zero activity on the ZIL disk, I was now seeing heavy activity on the ZIL, with up to 20 seconds of no activity on the four disks. I was seeing between 2 and 5 thousand kps on the ZIL, plus around 600 transactions per second. The transactions bit was pretty interesting, as I was used to seeing about 30-40 tps for each of the regular drives in iSCSI testing.

I later compiled cvsup on the VM and I noticed that there was much less activity. Bonnie++ produced a heavier throughput (~12,000kps) on the ZIL, but less transactions per second (around 350.) The drives were writing every 10 to 15 seconds, and were sustaining a speed of about 12,000kps.

I'll probably do a make buildworld on the vm later to get a better feel for the performance... As you can tell, this testing isn't even remotely scientific or thorough.

Extending the System Partition in Windows 2003 Server

Before I start, I'd like to mention that the instructions in the following post (and all posts with instructions) is to be followed at your own risk; no guarantee or warranty is implied.


Someone asked how one might resize a Windows 2003 system partition (typically C:.)

There are several utilities that can do this: Acronis and Easeus are two that come to mind. However, they aren't all that cheap, especially if you need to perform this operation on multiple servers.

Window's disk manager does not work - I even tried converting the disk to dynamic. Parted couldn't extend it, either. A friend suggested gparted.

I've had some issues with gparted before, but I decided to give it a try.

- I booted the virtual machine in VMWare workstation (you didn't think I'd try this on a real machine first, did you?) and noted the partition size of C:. It was 20GB. - I then shut it down, and used vmware-vdiskmanager to extend the volume to 30GB.
- I made a snapshot of the vm
- I then booted from the gparted live CD and extended the C: drive to 27GB from there.
- I rebooted and logged in. C: was now 27GB. I even checked the disk with the windows disk tools, and there were no errors.

Note: you'll need to have free space on your disk for this to work (obviously.) Back up your C: drive/system drive before starting (you can use ntbackup - make sure you get the system state, too.)

OpenBSD 4.x, OpenVPN, and Kerberos Authentication

OpenVPN works fairly well with OpenVPN. The one caveat being that OpenBSD does not have PAM support... making secondary authentication, using a user account, more complicated. It is possible to install /usr/ports/net/openvpn_bsdauth to use local user accounts, but what if you want a group certificate with authentication against a Windows Active Directory installation?

Since AD does have Kerberos support, it is possible.

1. Install OpenVPN from ports

2. Install the p5-Authen-Krb5-Simple perl module from ports (/usr/ports/security/p5-Authen-Krb5-Simple)

3. Add a script like so:
/etc/openvpn/krb5-auth.pl

#!/usr/bin/perl
use strict;
use Authen::Krb5::Simple;
# change the next variable to 1 to log errors to /tmp/autherror.txt
my $debug = 0;
my $user = $ENV{'username'};
my $pass = $ENV{'password'};
chomp ($user, $pass);
my $krb = Authen::Krb5::Simple->new([realm => 'YOURREALM.LOCAL']);
# Authenticate a user.
#
my $authen = $krb->authenticate($user, $pass);

unless($authen) {
my $errmsg = $krb->errstr();
if ($debug == 1) {
open ASD, ">/tmp/autherror.xt";
print ASD "User: $user authentication failed: $errmsg\n";
close ASD;
}
die "User: $user authentication failed: $errmsg\n";
}


Your script can be more complex than this, but this should work.

4. Add the following line to your client config:
auth-user-pass


5. Add the following lines to your server config:
auth-user-pass-verify /etc/openvpn/krb5-auth.pl via-env

6. create /etc/kerberosV/krb5.conf and add something along the lines of:

[libdefaults]
# Set the realm of this host here
default_realm = YOURREALM.LOCAL

# Maximum allowed time difference between KDC and this host
clockskew = 300

# Uncomment this if you run NAT on the client side of kauth.
# This may be considered a security issue though.
# no-addresses = yes

[realms]
YOURREALM.LOCAL = {
# Specify KDC here
kdc = mydomaincontroller.my.domain.local

# Administration server, used for creating users etc.
# admin_server = kerberos.my.domain
}


7. test kerberos:

kinit your_windows_username@YOUR_FQDN_WINDOWS_DOMAIN.IN_ALL_CAPS

If you get no error, run klist and you should see a ticket.

8. Make sure the time is accurate on your OpenBSD server.

IPSec tunnels on a dual homed Cisco ASA 5510

I recently had an issue where a client wished to route one IPSec tunnel over one ISP, and another tunnel over another ISP. One ISP was on the outside interface, and the other ISP was on an interface called backup-link.

I assumed, incorrectly, that it was going to be as simple as adding a static route for the IP of the destination to route through the second ISP's gateway. That did allow me to bring up the tunnel, but traffic would not pass.

The route I added was something along the lines of:

route backup-link my.external.address my.netmask my.2nd.isps.gateway 1


As it turns out, the ASA assumes that even IPSec tunneled traffic will be using the default gateway, so I had to add another route like so:

route backup-link my.internal.subnet.at.the.other.office my.netmask my.2nd.isps.gateway 1

And that seemed to work.