RedHat/CentOS error: connection activation failed: connection 'x' is not available on the device y

You'll see this error using nmcli if you try to bring up a connection in the nmcli that references a NIC that is disconnected/unplugged - either unplugged from a baremetal server or disconnected in VMware. Here are some examples of what it might look like:

connection activation failed: connection 'ethernet' is not available on the device ens32

connection activation failed: connection 'eth0' is not available on the device ens192

etc.

The fix is really as simple as connecting the cable or re-enabling the NIC in VMware and running the nmcli command again.

Simple Clustering on CentOS 7/RHEL 7 for an haproxy Load Balancer

Here are the steps to get a simple cluster going. We're not going to share storage, so the quorom isn't going to work. We'll disable stonith and quorom. 

pre-req: add hosts file entries for all nodes on all nodes, or at least make sure DNS is working correctly. You might receive errors like:

Error: unable to get crm_config, is pacemaker running?

yum install pcs fence-agents-all -y
firewall-cmd --permanent --add-service=high-availability
# set the password for the hacluster user - it should probably be the same on all nodes
passwd hacluster
# disable haproxy, as the cluster will start it

systemctl disable haproxy

# enable the services
systemctl enable pcsd
systemctl enable corosync
systemctl enable pacemaker
systemctl start pcsd.service

### we're not going to have a stonith nor a quorom
pcs property set stonith-enabled=false
pcs property set no-quorum-policy=ignore

# check to see if it's alive
systemctl is-active pcsd.service
pcs cluster auth `hostname`
pcs cluster setup --start --name myclustername `hostname`
pcs cluster status
# add an IP as a resource
pcs resource create vip1 IPaddr2 ip=172.29.23.80 cidr_netmask=22 --group haproxy

to add an additional node called "mynode2"
authorize it on the master:

 pcs cluster auth mynode2

(authenticate using "hacluster" user)

add it:

pcs cluster node add mynode2


You'll need to start the other node:

pcs cluster start mynode2

To see the status of the nodes:


pc status nodes

example:
pcs status nodes
Pacemaker Nodes:
 Online: mynode1
 Standby:
 Offline: mynode2

Now we add haproxy. Since the haproxy service wouldn't be too useful without the IP address, we'll set up a colocation rule as well.

pcs resource create HAproxy --group haproxy  systemd:haproxy op monitor interval=10s
pcs constraint colocation add HAproxy vip1
pcs constraint order vip1 then HAproxy

# optional - we want the cluster to "favor" mynode1.
# if mynode1 is restarted, for example, mynode2 will get the resources, 
# until mynode1 is back and running
pcs constraint location vip11 prefers mynode1 

We'll want to turn off haproxy in systemd as the cluster will start it:

systemctl disable haproxy 

RHEL/CentOS nmcli Tips

You may need to disable selinux temporarily to make changes to these files (setenforce 0)

(note that many nmcli commands will fail if the underlying device is not active (i.e., disconnected in VMware))

1. List connections

nmcli c show

2. rename an connection id called "outside" - change it to eth0

nmcli c modify outside connection.id eth0

3. change a nic (with a connection name of "ethernet" and a device name of ens32) to static and assign an address, gw, dns, etc. (172.19.22.1 is the default gateway. Separate additional addresses with commas, leaving a space before the default gateway.)

nmcli c modify ethernet connection.interface-name ens32 ipv4.method static ipv4.addresses "172.19.22.3/24 172.19.22.1" ipv4.dns 172.19.22.10,172.19.22.11 ipv4.dns-search mydomain.local

4. Bring up your new connection:

nmcli con up ethernet

5. delete a connection called "wired":

nmcli con delete wired

6. create a new connection (called "eth0") using an ethernet device called "ens32":

nmcli con add type ethernet con-name eth0 ifname ens32

7. change the hostname:

nmcli general hostname new_hostname

and restart hostnamed to pick up the change (your shell prompt won't change until you exec a new shell or reboot):

systemctl restart systemd-hostnamed

RedHat Enterprise/CentOS Kickstart with Bonding

In order to set up the bonded interfaces (in 802.3ad mode) when first booting, you'll need to supply the following arguments to kickstart:


ks=http://name_of_apache_server/myconfig.cfg ip=your.ip.address netmask=yournetmask gateway=yourgateway dns=nameserver1,nameserver2 hostname=yourfqdn bond=bond0:eth0,eth1:mode=802.3ad:miimon:100

Moving from Ntop to Ntopng

I used to start ntop this way:
screen -d -m ntopng -u ntop  -m my.subnets,myothersubnets -i eth2,eth3 -W 4443 -w 40000 -M &


But this failed as the redis cache was not running (but was installed as part of the dependencies):

18/Mar/2014 16:50:33 [Redis.cpp:43] ERROR: ntopng requires redis server to be up and running
18/Mar/2014 16:50:33 [Redis.cpp:44] ERROR: Please start it and try again or use -r
18/Mar/2014 16:50:33 [Redis.cpp:45] ERROR: to specify a redis server other than the default

I now need redis to be running. I modified  /etc/redis.conf to point to /opt/redisdb for its "dir" variable and changed the owner of the dir to redis as well as chmoding the directory to 700.
So, in redis.conf,

#dir /var/lib/redis/
dir /opt/redisdb/

You'll probably want to copy the selinux context info, if you're using selinux:

As you can see:

ls -laZ /var/run/redis/

drwxr-xr-x. redis root  system_u:object_r:var_run_t:s0   .

drwxr-xr-x. root  root  system_u:object_r:var_run_t:s0   ..

-rw-r--r--. redis redis unconfined_u:object_r:initrc_var_run_t:s0 redis.pid

chcon --reference /var/run/redis /opt/redisdb

 I started redis, which was listening to 6379 on localhost only (sudo service redis start)

Ntopng also likes to have a data directory, so I created /opt/ntopng:

sudo mkdir /opt/ntopng
sudo chown ntop /opt/ntopng
sudo chmod 700 /opt/ntopng

sudo screen -d -m ntopng -u ntop  -r localhost:6379 -m my.subnets,myothersubnets -i eth2,eth3 -W 4443 -w 40000 -M &

But now, it was listening on eth0 as it didn't like the ordering of arguments. I saw this error:

18/Mar/2014 16:45:58 [NetworkInterface.cpp:79] WARNING: No capture interface specified
18/Mar/2014 16:45:58 [NetworkInterface.cpp:1438] Available interfaces (-i ):

18/Mar/2014 16:45:58 [NetworkInterface.cpp:1459] 1. eth0 (eth0)

18/Mar/2014 16:45:58 [NetworkInterface.cpp:1459] 2. eth1 (eth1)
18/Mar/2014 16:45:58 [NetworkInterface.cpp:1459] 3. usbmon1 (USB bus number 1)
18/Mar/2014 16:45:58 [NetworkInterface.cpp:1459] 4. eth2 (eth2)
18/Mar/2014 16:45:58 [NetworkInterface.cpp:1459] 5. usbmon2 (USB bus number 2)
18/Mar/2014 16:45:58 [NetworkInterface.cpp:1459] 6. usbmon3 (USB bus number 3)
18/Mar/2014 16:45:58 [NetworkInterface.cpp:1459] 7. usbmon4 (USB bus number 4)
18/Mar/2014 16:45:58 [NetworkInterface.cpp:1459] 8. any (Pseudo-device that captures on all interfaces)
18/Mar/2014 16:45:58 [NetworkInterface.cpp:1459] 9. lo (lo)

18/Mar/2014 16:49:52 [PcapInterface.cpp:68] Reading packets from interface eth0...

18/Mar/2014 16:49:52 [Ntop.cpp:573] Registered interface eth0 [id: 0]

Not desirable, so that becomes (I removed -M (I'm not sure what replaces "don't merge interfaces") as well as changed -u to -U and added -n 1 to resolve only ip addresses listed in -m (local))

sudo screen -d -m ntopng -i eth1 -i eth2 -d /opt/ntopng -n 0 -W 4443 -w 40000 -m mysubnets -r localhost:6379 -U ntop &

One last thing, you now need to set the password for admin, either via a file, by the gui (after logging in as admin/admin) or by the redis-cli client. I chose the latter.

redis-cli SET user.admin.password `echo -n "mylousypassword" | md5sum | cut -f 1 -d " "`

You can see the users in the gui or here:

redis-cli KEYS user* 

You can a new user either through the gui like so:

 redis-cli SET user.mynewuser.password `echo -n "mylousypassword" | md5sum | cut -f 1 -d " "`

Blocking Outbound File Attachments In MailScanner

MailScanner is a popular anti-virus/anti-spam open source package that supports several operating systems, including Linux and FreeBSD. It can be helpful for minor compliance work.

For exmaple, Company X wants to block outbound Microsoft Word Docs (both .doc and .docx) but allow people from outside to send them in. To set this up, you'll need to create a couple of files and modify the main config, Mailsca

In order to block only, say, word docs in Mailscanner, you need to do a few things.

Steps

1. add a rule to split the filename processing. On CentOS, we'll call this /etc/MailScanner/rules/filenameconf.rules

The contents are:

From:   *@mydomain.com /etc/MailScanner/filename.mydomain.rules.conf
From:       default /etc/MailScanner/filename.rules.conf

2. copy /etc/MailScanner/filename.rules.conf to /etc/MailScanner/filename.mydomain.rules.conf

3. edit /etc/MailScanner/filename.mydomain.rules.conf and set the rules you want. To block doc/docs:

deny   \.docx?$        Windows Word Doc                                                               Word  documents may contain sensitive information or viruses

(note, the fields are tab delimited, not space delimited!)

4. modify the main config file, /etc/MailScanner/MailScanner.conf and comment out this line:

Filename Rules = %etc-dir%/filename.rules.conf

and replace it with this:

Filename Rules = %rules-dir%/filenameconf.rules

5. restart mailscanner (sudo service MailScanner restart)

OSSEC Agent Installation on RedHat Enterprise and/or CentOS

I consider a HIDS to be one of the very most important security systems in a network. It's very important to know if someone is attempting to brute force his way into one of your servers, but if the attacker breaks in, it's even more important to know what changed. While log monitors and tools like Splunk can detect the former, they're not designed to note changes in binaries. That's where a HIDS (Host-based Intrusion Detection System) comes into play. One of the most popular is OSSEC. It's open sourced and free to deploy. It notes troublesome issues in the logs (like login failures) and can even take proactive measures, such as blocking an attacking host by manipulating the firewall.

OSSEC works on Windows, various flavors of unix and linux, as well as network devices such as switches, routers, and firewalls. It can run stand alone or report to a master server.

Agent installation on RHEL/CentOS

1. Install the Atomic Yum repo:

sudo "wget -q -O - http://www.atomicorp.com/installers/atomic | sh"

2. Install the EPEL repo (We're using 64bit Linux here)

 If RHEL 5/CentOS 5, 

sudo rpm -ivh http://download.fedoraproject.org/pub/epel/5/i386/epel-release-5-4.noarch.rpm

If RHEL 6/CentOS 6,

sudo rpm -ivh http://download.fedoraproject.org/pub/epel/6/i386/epel-release-6-8.noarch.rpm

CentOS 6

 3. Install the RPMs from yum (should be about 4 RPMs or so):

sudo yum -y install ossec-hids-client

3a. (optional) Disable the Yum repos for Atomic and EPEL:

sed -i /etc/yum.repos.d/atomic.repo -e 's/enabled = 1/enabled = 0/g'
sed -i /etc/yum.repos.d/epel.repo -e 's/enabled = 1/enabled = 0/g'

4. You'll need to set your server (if you even have a central OSSEC server) in the config file like so:

5. (optional) make an exception for the central server (again, if you even have one):

sudo iptables -A INPUT -p udp --dport 1514 -j ACCEPT

 That's basically it. It's most definitely worth reading up on the documentation and you should definitely test OSSEC out before using the active rules.

 

Building Snort RPMs On CentOS 6.x x86_64

This was done on

You'll want epel, most likely - to pick up the libdnet dependencies for snort. (sudo rpm -ivh http://dl.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm)


1. install the prerequisites:


 sudo yum install -y pcre pcre-devel gcc flex byacc bison l libxml2-devel kernel-devel  libdnet libdnet-devel  autoconf automake libpcap-devel rpm-build

2. install the src rpms (do not do this as root)

3. This will create an rpmbuild dir in your home, go ahead and build daq:

rpmbuild -bb ~/rpmbuild/SPECS/daq.spec

4. Install the daq libraries you just built:

  sudo rpm -ivh RPMS/x86_64/daq-*.rpm

5. Now, build snort

rpmbuild -bb ~/rpmbuild/SPECS/snort.spec

6. Remove the daq libraries you installed in step 4:

 sudo rpm -e daq

You should have two libraries in RPMS/x86_64. You'll need to install both for Snort to function.

Autofs direct maps for NFS mounts

I had a situation where I needed to automount an NFS share on the root level, i.e.,

mynfsserver:/data/nfsshar1 on /myNFSshare on a local NFS client.

In order to do this, I had to set up a direct map. In /etc/auto.master, I added:


/-      /etc/auto.direct

and then I created the file /etc/auto.direct with the following entry:

/myNFSshare         mynfsserver:/data/nfsshar1

I reloaded autofs and was then able to do a directory listing on the share. This works for Rhel 5 & 6 and CentOS 5&6.

Racktables on CentOS 5.6

I find Racktables to be a very useful tool. Because it requires newer versions of PHP, it's a bit of a hassle to get it running on CentOS 5.x. Here are the steps:

1. Get the CentOS testing repo (for new versions of PHP.)

 cd /etc/yum.repos.d

 wget http://dev.centos.org/centos/5/CentOS-Testing.repo

 # now we enable the new repo - at least for long enough to satisfy the dependencies

 sed -i CentOS-Testing.repo -e 's/enabled=0/enabled=1/g'

2. update PHP to 5.3.x

 yum -y install php53 (you may need to remove php-common, if it's installed)

2b. optional: lock down MySQL:

 sudo /usr/bin/mysql_secure_installation

You'll want to assign a root password as well as delete anonymous users and disallow remote root logins.

3. Create the database and database user:

 mysql -u root -p

  create database racktables;

grant all on racktables.* to racktablesuser;

grant all on racktables.* to racktablesuser@localhost;

set password for racktablesuser@localhost=password('your_super_password');

flush privileges;

4. untar the racktables source and do a make && make install.

5. create entries for this host in DNS - A records and/or CNAMEs. For example, create an A record called racktables.yourdomain.local. 

5b. Create a configuration entry in apache. Edit /etc/httpd/conf.d/racktables.conf:

 ServerName racktables.your_domain.local

 DocumentRoot /usr/local/share/RackTables/wwwroot/

Obviously, this is not running over SSL. It's not a bad idea to run Racktables over HTTPS.

5c. Restart apache (sudo /sbin/service httpd restart)

6. Connect to your host. You'll probably get this error message:

 Database connection parameters are read from /usr/local/share/RackTables/wwwroot/inc/secret.php file,  which cannot be found. You probably need to complete the installation procedure by following this link.

7. Step through the links till you get to the dependency page (step 2 of 6.)

8. You'll see that you are missing numerous dependencies. Go ahead and fix the dependencies:

sudo yum -y install php-mbstring

sudo yum -y install php-gd

sudo yum -y install php-snmp

sudo yum -y install php-pdo

sudo yum -y install php-mysql

9. In order to get the unicode support in php pcre

sudo yum update pcre

10. We'll leave pcntl as is for now. In the next step, you'll see a warning about the secret.php file. To fix it:

sudo touch '/usr/local/share/RackTables/wwwroot/inc/secret.php'; sudo chmod 666 '/usr/local/share/RackTables/wwwroot/inc/secret.php'

(you'll need to either configure SELinux to allow access to that file or disable SELinux altogether. 

11. Enter the db name and db username and password from earlier. 

12. Set an administrator password in the next step.

You should be all set.

CentOS 5.5, Samba/Winbind, Windows 2008R2 Active Directory

In order to facilitate client backups, I set up samba on CentOS on a Windows 2008R2 based domain. I created an empty directory, /etc/skel2, as I was not planning on letting users log in via ssh or the console.

 Unfortunately, the samba 3.0.x line wouldn't work correctly. I could join the domain, but not connect to shares. Samba logged the following message every connection attempt:


  read_data: read failure for 4 bytes to client 192.168.70.23. Error = Connection reset by peer

I upgraded to the Samba3 package with yum (Samba 3.3.x) and rejoined. I added a pam.d entry in the samba config:

 session required pam_mkhomedir.so skel=/etc/skel2 umask=0077


( I don't want users to see each other's directories. I did not modify system-auth, as I did not want the users to log in with anything but samba.)

And added this to smb.conf:


  winbind separator = \
  # use uids from 10000 to 20000 for domain users
  idmap uid = 10000-20000
 # use gids from 10000 to 20000 for domain groups
 idmap gid = 10000-20000
 # allow enumeration of winbind users and groups
 winbind enum users = yes
 winbind enum groups = yes
 winbind use default domain = yes
 # give winbind users a real shell (only needed if they have telnet access)
 template homedir = /data/clientdata/backups/%D/%U
 template shell = /bin/bash
 obey pam restrictions = yes

An important note, the samba3 rpm separates the smbd and nmbd init scripts, so you'll need to do a separate "chkconfig nmb on."

Trouble sending out email from Bacula 5.0.2 and CentOS 5.5

I recently noticed that Bacula installations on CentOS 5.x (primarily 5.5) boxes were not sending out email after job completion. It was baffling as bsmtp worked fine on the command line... at least when I ran it as root. I tried running it as the bacula user (my Bacula director runs as the Bacula user, not root) and it failed:

$ su - bacula -s "/usr/sbin/bsmtp"
-bsmtp: error while loading shared libraries: libbac-5.0.1.so: cannot open shared object file: Permission denied


Ah... I checked the file permissions:

ls -la /usr/lib/libbac-5.0.1.so
-rwxr-x--- 1 root root 330240 Jun 22 12:46 /usr/lib/libbac-5.0.1.so

Simple fix:

chmod o+r /usr/lib/libbac-5.0.1.so

On 64 bit CentOS/Rhel, it will be /usr/lib64/libbac-50.0.1.so

I'm guessing that there was a problem in the spec file (I built it from an srpm.)

Automount Home Directories Over NFS in Linux

On RedHat Enterprise/CentOS...
assuming your home server is called "fs1" and is sharing out /home over NFS. You'll also want to make sure that both your home directory server and the server you're running autofs on has the same passwd/user info - ids, gids.

1. Make sure autofs is installed and running:

sudo chkconfig autofs on
sudo service autofs start

2. add "/home /etc/auto.home" to /etc/auto.master

3. create the file /etc/auto.home and add these lines:

* -fstype=nfs,rw,nosuid,soft fs1:/home/&

4. if you already have a /home dir, move it out of the way:

sudo mv /home /home.old

5. reload autofs

sudo service autofs reload

6. ls /home

You should see the users' home directories.

Building Bacula Client RPMs for CentOS 5.x

This is a quick and dirty explanation of the process. I'm building as root - which you should probably avoid doing.

Download and install the srpm file from bacula's site (sudo rpm -ivh name-of-srpm.srpm)

You'll need to install the rpmbuild utility:



sudo yum -y install rpm-build


To build the client only:

Install a couple of prerequisite rpms:
sudo yum -y install ncurses-devel libtermcap-devel

rpmbuild --define build_client_only=1 --define build_centos5=1 -bb /usr/src/redhat/SPECS/bacula.spec