While upgrading a pair of non-stacked 3850-48Ps, I encountered this error message:
[1]: % An internal error was encountered. Operation aborted.
I did have some space issues, so I deleted previous versions. Even that did not fix the issue. I tried installing it from tftp, to no avail.
Eventually, I ran the software install command with "new force"
software install file tftp://10.20.30.40/cat3k_caa-universalk9.SPA.03.06.09.E.152-2.E9.bin new force
Success. I rebooted both switches. One crashed on reboot, but rebooted cleanly afterward.
Tuesday, November 27, 2018
Sunday, November 25, 2018
Cisco 3850 WS-C3850-48P-E Switch with Wrong License Level Lanbase
I received a couple of refurbished 3850 PoE Enterprise switches. One switch was perfectly fine, with IP services enabled permanently.
The other:
I issued a request for an RMA - however, the switch was labeled correctly on the case: WS-C3850-48P-E.
On a whim, I ran this command:
I rebooted as instructed from the switch, and now I have:
Switch#show license right-to-use
Slot# License name Type Count Period left
----------------------------------------------------------
1 ipservices permanent N/A Lifetime
1 lanbase permanent N/A Lifetime
License Level on Reboot: ipservices
The other:
Slot# License name
Type Count Period left
----------------------------------------------------------
1
lanbase permanent
N/A Lifetime
License Level on Reboot: lanbase
I issued a request for an RMA - however, the switch was labeled correctly on the case: WS-C3850-48P-E.
On a whim, I ran this command:
license
right-to-use activate ipservices
I rebooted as instructed from the switch, and now I have:
Switch#show license right-to-use
Slot# License name Type Count Period left
----------------------------------------------------------
1 ipservices permanent N/A Lifetime
1 lanbase permanent N/A Lifetime
License Level on Reboot: ipservices
So, yay - no need for an RMA.
Friday, November 23, 2018
Upgrading pfsense 2.4.3_1 to 2.4.4 - Issue Replacing PHP 5.x with 7.x Due to FRR
I ran into a problem upgrading pfsense 2.4.3_1 to 2.4.4 due to a dependency on the old PHP 5.6.x from the older package of FRR .2_2 (Free Range Router.) The problem was that PHP56 was marked as vital due to the FRR dependency, and it was not possible to update FRR to .2_4, as FRR .2_4 needed pfsense 2.4.4 (I could not decide whether this was a modern O. Henry story or a missing epilogue from Catch 22.)
The fix:
1. connect to the pfsense server with SSH
2. Open a shell
3. Run this command:
pkg set -v 0 php56
Your upgrade should work - I had to do a couple of update attempts before it worked.
The fix:
1. connect to the pfsense server with SSH
2. Open a shell
3. Run this command:
pkg set -v 0 php56
Your upgrade should work - I had to do a couple of update attempts before it worked.
Wednesday, March 21, 2018
SSH No Matching Cipher Found With SSH to Older Cisco Gear
I've seen this error on recent SSH versions, including the OpenSSH used by OS X and others:
Unable to negotiate with 192.168.77.1 port 22: no matching cipher found. Their offer: aes128-cbc,3des-cbc,aes192-cbc,aes256-cbc
Several people suggest modifying your local ssh client config. However, I'd rather not use 3DES unless I have to. So, I created an alias and put it in .bash_profile:
alias ssho='ssh -c 3des-cbc'
after a quick . ~/.bash_profile
I can ssh with the alias:
ssho 192.168.77.1
password:
Of course, the best fix would be to update IOS on the Cisco gear.
Unable to negotiate with 192.168.77.1 port 22: no matching cipher found. Their offer: aes128-cbc,3des-cbc,aes192-cbc,aes256-cbc
Several people suggest modifying your local ssh client config. However, I'd rather not use 3DES unless I have to. So, I created an alias and put it in .bash_profile:
alias ssho='ssh -c 3des-cbc'
after a quick . ~/.bash_profile
I can ssh with the alias:
ssho 192.168.77.1
password:
Of course, the best fix would be to update IOS on the Cisco gear.
Thursday, March 1, 2018
Openstack Ansible Pike - Cannot Create Image in Horizon
There is a bug with OpenStack Pike deployments using Ansible. When you attempt to create an image in Horizon, you will see the following error:
Followed by this entry in the images list after closing this:
You should still be able to create images via command line, but Horizon won't work.
There is a bug filed for this:
Their fix is simple: connect to the horizon container from an infrastructure node:
lxc-attach -n `lxc-ls -1 | grep horizon | tail -1`
edit /etc/horizon/local_settings.py
and change:
HORIZON_IMAGES_UPLOAD_MODE = 'direct'
to
HORIZON_IMAGES_UPLOAD_MODE = 'remote'
After that, restart the horizon container
lxc-stop -n `lxc-ls -1 | grep horizon | tail -1`
lxc-start -n `lxc-ls -1 | grep horizon | tail -1`
And simply wait a few minutes. Image creation should work.
Thursday, July 6, 2017
Powercenter Integration Service Won't Start: Error SF_34004 - Missing Environment Variables
I recently had an issue where I could not get the Informatica PowerCenter Integration Service to start after an install. The service generated this error code: SF_34004. There is an Informatica KB article describing this problem and the appropriate fix:
https://kb.informatica.com/solution/18/Pages/120251.aspx
Basically, it's suggesting that the Integration Service was configured to connect to the PowerCenter repo using the schema credentials and not the domain administrator.
However, I found that this did not help. the repo service was running correctly and the integration service was using the correct credentials. I checked the user running PowerCenter, and realized the INFA_HOME, INFA_DOMAIN_FILE, and other variables were not set correctly. After setting these and restarting PowerCenter, I was able to enable the Integration Service.
https://kb.informatica.com/solution/18/Pages/120251.aspx
Basically, it's suggesting that the Integration Service was configured to connect to the PowerCenter repo using the schema credentials and not the domain administrator.
However, I found that this did not help. the repo service was running correctly and the integration service was using the correct credentials. I checked the user running PowerCenter, and realized the INFA_HOME, INFA_DOMAIN_FILE, and other variables were not set correctly. After setting these and restarting PowerCenter, I was able to enable the Integration Service.
Friday, May 5, 2017
Abnormally Long First Boot Times on Cisco 3750x's with 15.2.x code
I recently upgraded some Cisco 3750x (WS-C3750X-48T-S to be precise) from 12.2.x code to 15.2 code (15.2(4)E4.)
The reboot took a rather long time due to microcode updates. It was about 25-30 minutes. The following messages were displayed:
ssssssssspssssssssssssssssssspssssssssssssssssssspssssssssssssssssssspssssssssssssssssssspsssssssssp
Front-end Microcode IMG MGR: Preparing to program device[0]...25166 bytes.
Front-end Microcode IMG MGR: Programming device 0...rrrrrrwssssssssssssssssssspssssssssssssssssssspp
Front-end Microcode IMG MGR: Preparing to program device[0]...86370 bytes.
Front-end Microcode IMG MGR: Programming device 0...rrrrrrwssssssssssssssssssspssssssssssssssssssspw
% Front-end Microcode IMG MGR: HW image is upgraded. MCU reset causes the switch to reload
If you're making such an upgrade, be prepared for a long wait. It will likely be worse on a stack.
While the 3750x is not a speedy switch, management-wise, this was impressively slow. The regular boot time after the upgrade was about 5-6 minutes, so be patient. Switches in a stack will take even longer to boot.
Tuesday, February 28, 2017
Android Nougat and OTG Sound Blaster E3 Sound Issues
I recently upgraded to Nougat and experienced sound issues using an OTG cable and my Sound Blaster E3 DAC. The sound was basically panned hard to the left, slower than normal, and there was some distortion.
I realized that ES File Explorer was activating each time I plugged the E3 in. The simple fix was to remove ES File Explorer. That being said, I'll probably experiment with its settings as it's a useful utility.
I realized that ES File Explorer was activating each time I plugged the E3 in. The simple fix was to remove ES File Explorer. That being said, I'll probably experiment with its settings as it's a useful utility.
Sunday, October 30, 2016
Raspberry Pi and 1-Wire Temperature/Environmental Sensors
I've discussed 1-wire before. In this case, I needed a cheap temperature sensor to alert on environmental conditions in an server room.
I set up a Raspberry Pi with stock Raspian and install digitemp like so:
apt install digitemp
I used an iButtonlink usb master (here: http://www.ibuttonlink.com/products/linkusb and a t-sense sensor here:
http://www.ibuttonlink.com/products/t-sense )
The USB device shows up as /dev/ttyUSB0 as an FTDI serial port.
I can walk the 1-wire tree like so:
sudo digitemp_DS9097U -s /dev/ttyUSB0 -w
DigiTemp v3.6.0 Copyright 1996-2007 by Brian C. Lane
GNU General Public License v2.0 - http://www.digitemp.com
Turning off all DS2409 Couplers
..
Devices on the Main LAN
2149B214090000D1 : DS18B20 Temperature Sensor
01C3235419000000 : DS2401/DS1990A Serial Number iButton
That being said, you'll want to create a dot file to use this like so:
sudo digitemp_DS9097U -s /dev/ttyUSB0 -i
DigiTemp v3.6.0 Copyright 1996-2007 by Brian C. Lane
GNU General Public License v2.0 - http://www.digitemp.com
Turning off all DS2409 Couplers
..
Searching the 1-Wire LAN
2149B214090000D1 : DS18B20 Temperature Sensor
ROM #0 : 2149B214090000D1
Wrote .digitemprc
I have one temperature sensor at the moment. After creating the .digitemprc, I can query it:
sudo digitemp_DS9097U -s /dev/ttyUSB0 -t 0
DigiTemp v3.6.0 Copyright 1996-2007 by Brian C. Lane
GNU General Public License v2.0 - http://www.digitemp.com
Oct 30 20:16:45 Sensor 0 C: 21.00 F: 69.80
I set up a Raspberry Pi with stock Raspian and install digitemp like so:
apt install digitemp
I used an iButtonlink usb master (here: http://www.ibuttonlink.com/products/linkusb and a t-sense sensor here:
http://www.ibuttonlink.com/products/t-sense )
The USB device shows up as /dev/ttyUSB0 as an FTDI serial port.
I can walk the 1-wire tree like so:
sudo digitemp_DS9097U -s /dev/ttyUSB0 -w
DigiTemp v3.6.0 Copyright 1996-2007 by Brian C. Lane
GNU General Public License v2.0 - http://www.digitemp.com
Turning off all DS2409 Couplers
..
Devices on the Main LAN
2149B214090000D1 : DS18B20 Temperature Sensor
01C3235419000000 : DS2401/DS1990A Serial Number iButton
That being said, you'll want to create a dot file to use this like so:
sudo digitemp_DS9097U -s /dev/ttyUSB0 -i
DigiTemp v3.6.0 Copyright 1996-2007 by Brian C. Lane
GNU General Public License v2.0 - http://www.digitemp.com
Turning off all DS2409 Couplers
..
Searching the 1-Wire LAN
2149B214090000D1 : DS18B20 Temperature Sensor
ROM #0 : 2149B214090000D1
Wrote .digitemprc
I have one temperature sensor at the moment. After creating the .digitemprc, I can query it:
sudo digitemp_DS9097U -s /dev/ttyUSB0 -t 0
DigiTemp v3.6.0 Copyright 1996-2007 by Brian C. Lane
GNU General Public License v2.0 - http://www.digitemp.com
Oct 30 20:16:45 Sensor 0 C: 21.00 F: 69.80
Thursday, October 6, 2016
Setting Up RAID Volumes on Cisco UCS C series with Avago/LSI/MegaRAID Controllers
On a C220M4, I discovered that the controller configures the drives as JBOD. Fixing this is easy, but not quite as straightforward as one would expect... especially considering that the controller has been a mainstay of servers for a very long time.
1. Go into the controller firmware (I had to do a control-M before I saw the control-R option):
Now you have the option to create a volume.
1. Go into the controller firmware (I had to do a control-M before I saw the control-R option):
As you can see, create virtual drives is not an option. Select "Make Unconfigured Good"
Select all the drives and click OK.
Cisco UCS C220M4s and the Purple Screen of Death - Could not start pcpu 1: TSC sync timed out
Recently, I had to install ESXi 5.5 on a Cisco rackmount server. VMware crashed with the purple screen of death.
Here's the display:
These are relatively new servers, so there was nothing especially helpful on the web. Other people had this problem, but I couldn't find any real solutions, other than a reference to C states on the CPU.
I experimented a bit and figured that CPU power savings might be the problem (I'd seen this before on Opteron servers.)
The fix is to set CPU performance to HPC (it defaults to Enterprise) in BIOS:
Here's the display:
These are relatively new servers, so there was nothing especially helpful on the web. Other people had this problem, but I couldn't find any real solutions, other than a reference to C states on the CPU.
I experimented a bit and figured that CPU power savings might be the problem (I'd seen this before on Opteron servers.)
The fix is to set CPU performance to HPC (it defaults to Enterprise) in BIOS:
That's advanced -> CPU Performance -> HPC (from Enterprise)
This solved the problem.
Wednesday, October 5, 2016
Avocent ACS 6016/6008 Renaming Ports From CLI
These console servers are fairly easy to setup, but the CLI is segmented to facilitate differing levels of access.
If you wish to rename a serial port (i.e., with hostname) from the CLI, you need to do the following
1. Figure out which port is being renamed (i.e., port 11)
2. log in with appropriate credentials via SSH (or direct serial)
3.
cd ports
4.
cd serial_ports
4b. You can check out the parameters here:
If you wish to rename a serial port (i.e., with hostname) from the CLI, you need to do the following
1. Figure out which port is being renamed (i.e., port 11)
2. log in with appropriate credentials via SSH (or direct serial)
3.
cd ports
4.
cd serial_ports
4b. You can check out the parameters here:
--:#- [serial_ports/physical] cli-> show
port: 11
enable_cisco_rj45_pin-out = yes
status = enabled
speed = 9600
parity = none
data_bits = 8
stop_bits = 1
flow_control = none
5. set port name
set port_name=my_new_name
i.e.,
--:#- [serial_ports/cas] cli-> set port_name=unused
8. save the config
save
That's it, basically.
Tuesday, August 30, 2016
AnyConnect 4.x on ASA 8.4.x: AnyConnect is not enabled on the VPN server
I recently had to do a failover on a pair of ASA5520s. On failing over to the secondary, AnyConnect stopped working. I did not see a license error, but users who connected received this error:
AnyConnect is not enabled on the VPN server
There weren't any smoking guns in the logs. I did notice that the newest version of the config was missing the client image definitions in webvpn. They were in the flash drive, but not copied to the secondary. So, when the secondary took over, this portion of the config was dropped.
The solution was to re-add the images:
config t
webvpn
anyconnect image disk0:/anyconnect-macosx-i386-4.3.00748-k9.pkg 1
anyconnect image
disk0:/anyconnect-win-4.3.00748-k9.pkg 2
Tuesday, March 8, 2016
Cisco ASA tftp Copies and Issues with atftpd
while attempting to copy a new anyconnect pkg file from a CentOS based server running atftpd, I received the following error on a Cisco ASA 5505:
%Error reading tftp://mytftpserver/anyconnect-win-3.1.14018-k9.pkg (Undefined error code)
On the tftp server in /var/log/messages, I saw:
%Error reading tftp://mytftpserver/anyconnect-win-3.1.14018-k9.pkg (Undefined error code)
On the tftp server in /var/log/messages, I saw:
Mar 6 14:49:19 mytftpserver atftpd[15147]: Serving anyconnect-win-3.1.14018-k9.pkg to 192.168.47.5:32947
Mar 6 14:49:19 mytftpserver atftpd[15147]: Requested file to big, increase BLKSIZE
I had installed tftp a while ago... specifically atftp-server. After looking at the man page for in.tftpd, I concluded there was no good way to set BLKSIZE on this package. The fix was:
sudo yum erase atftp-server
sudo yum install tftp-server
I then edited the xinet.d/tftp file to enable it and change the target directory and restarted xinetd. After this, I was able to copy the pkg file over successfully.
Monday, December 28, 2015
Mellanox ConnectX-2 10GB Interface on FreeBSD 10.2
Although Mellanox's FreeBSD driver for the ConnectX-2 is included in the kernel source, the kernel modules are not included with the generic kernel. To use this card, build a custom kernel.
1. update the FreeBSD source using freebsd-update
sudo freebsd-update fetch
sudo freebsd-update install
2. copy the generic kernel config to a custom kernel
cd /usr/src/sys/amd64/conf
sudo cp GENERIC MYKERNEL01
4. edit the config file (sudo vi MYKERNEL01.) Add these lines to the bottom:
#### Mellanox ConnectX-2 support
options OFED
options IPOIB_CM
device ipoib
device mlx4ib
device mlxen
5. Compile the kernel and install it
cd /usr/src
sudo make buildkernel KERNCONF=MYKERNEL01
sudo make installkernel KERNCONF=MYKERNEL01
6. reboot
sudo shutdown -r now
Your new interface should show up when the machine comes back:
mlxen0: flags=8802 metric 0 mtu 1500
options=d07bb
ether 00:02:c9:52:ad:23
nd6 options=29
media: Ethernet autoselect (autoselect)
status: active
You can configure however you'd like (IP, MTU, etc.)
If you run freebsd-update on a regular basis, you may want to append a kernel rebuild (step #5) to a script so you can rebuild the kernel automatically if there is a kernel source update.
1. update the FreeBSD source using freebsd-update
sudo freebsd-update fetch
sudo freebsd-update install
2. copy the generic kernel config to a custom kernel
cd /usr/src/sys/amd64/conf
sudo cp GENERIC MYKERNEL01
4. edit the config file (sudo vi MYKERNEL01.) Add these lines to the bottom:
#### Mellanox ConnectX-2 support
options OFED
options IPOIB_CM
device ipoib
device mlx4ib
device mlxen
5. Compile the kernel and install it
cd /usr/src
sudo make buildkernel KERNCONF=MYKERNEL01
sudo make installkernel KERNCONF=MYKERNEL01
6. reboot
sudo shutdown -r now
Your new interface should show up when the machine comes back:
mlxen0: flags=8802
options=d07bb
ether 00:02:c9:52:ad:23
nd6 options=29
media: Ethernet autoselect (autoselect
status: active
You can configure however you'd like (IP, MTU, etc.)
If you run freebsd-update on a regular basis, you may want to append a kernel rebuild (step #5) to a script so you can rebuild the kernel automatically if there is a kernel source update.
Thursday, November 19, 2015
2010 Macbook Pro and problems with Yosemite and/or El Capitan
I have had terrible issues with a mid-2010 MBP 15" core i7 laptop. Doing a clean installation of either Yosemite or El Capitan, the installer fails toward the end or crashes on first boot. I have not tried a fresh Mavericks install. However, a restore from a Mavericks time machine backup works perfectly fine. Alternatively, a linux installation also worked well with no crashes.
Quite a few posts suggest it is a log board problem and that Apple would fix it. However, being impatient, I tried my own work around. I noticed that the problem didn't happen if I reduced the RAM from 8GB to 4GB (1 stick of DDR 3 instead of 2.)
A prevailing theory for this failure is that the Nvidia 330M card was crashing the system when it switched from the onboard to the 330M.
A couple of suggested work arounds:
1. disable automatic video switching in power preferences. This will likely consume battery at a faster rate
2. use gfxCardStatus and force it to either onboard or discrete
What I did was:
1. do a fresh install of El Capitan with only 1 stick of DDR 3 (4GB)
2. fully patched the OS
3. installed gfxcardstatus and forced it to onboard only.
I haven't had a crash since.
Quite a few posts suggest it is a log board problem and that Apple would fix it. However, being impatient, I tried my own work around. I noticed that the problem didn't happen if I reduced the RAM from 8GB to 4GB (1 stick of DDR 3 instead of 2.)
A prevailing theory for this failure is that the Nvidia 330M card was crashing the system when it switched from the onboard to the 330M.
A couple of suggested work arounds:
1. disable automatic video switching in power preferences. This will likely consume battery at a faster rate
2. use gfxCardStatus and force it to either onboard or discrete
What I did was:
1. do a fresh install of El Capitan with only 1 stick of DDR 3 (4GB)
2. fully patched the OS
3. installed gfxcardstatus and forced it to onboard only.
I haven't had a crash since.
Wednesday, May 27, 2015
Windows Netstat and findstr - Same as find | grep on Linux/Unix
Unix/Linux users are familiar with the following patterns of usage for finding listening ports:
looking for services listening on ports 443 or 444:
netstat -a | grep 44[34]
find a service listening on port 602, 612, 604, 614, 608, or 618:
netstat -an | grep 6[01][248]
To do the same on windows, use netstat and findstr:
find a window's service listening on port 602, 612, 604, 614, 608, or 618:
netstat -an | findstr /R 6[01][248]
looking for services listening on ports 443 or 444:
netstat -a | grep 44[34]
find a service listening on port 602, 612, 604, 614, 608, or 618:
netstat -an | grep 6[01][248]
To do the same on windows, use netstat and findstr:
find a window's service listening on port 602, 612, 604, 614, 608, or 618:
netstat -an | findstr /R 6[01][248]
Subscribe to:
Posts (Atom)


