IPMI Configuration and Monitoring
Contents |
IPMI Remote Management
Did you know our new Dells essentially have a built in text based KVM over IP?
Intelligent Platform Monitoring Interface allow you to manage a server remotely. The IPMI services are controlled by a separate on board controlled, usually referred to as a Baseboard Management Console (BMC). There is no requirement for there to be an OS on the server to access IPMI features. If an OS is present, you can still connect regardless of the stats of the OS. Even if the OS has kernel paniced, or is completely locked up or unresponsive. This is accomplished via a separate dedicated service process, independent of the CPU.
IPMI provides two primary methods of access. First is a simple shell interface. By running a local client (impitools) you can remotely access a shell interface on the server. There is a hierarchical menu system of available commands. You can power cycle the system, select the boot device and view error logs.
IPMI can also use Serial over Lan (SoL) to redirect console output over the network. This provides text based access over the network. The great thing here is you have complete system access to BIOS, RAID configuration and if the OS is configured it as well. You can console the server from miles away, reboot it, enter BIOS, etc.
Most new servers support IPMI, though it is often not uses. A main reason is that configuring it is not intuitive. However, the steps on our Dells fairly straight forward and you once you do it once you can easily set this up in a few minutes.
We can set this up for customers who need a KVM over IP type solution (on Linux boxes only) on an as needed basis. It would require having IPMI on the NIC 1. You configure IPMI with one IP, and configure the OS to have the other. Both IPs are using the same physical ethernet cable. The customer can connect to one IP with ssh and to the other with impitools.
IPMI Configuration
Setting up IPMI requires configuring the BMC fireware in BIOS and also modifying two OS files.
Setup IPMI
(Example using Dell 1950s)
1) Enter BMC config from POST, usually Ctrl-E and the BMC prompt.
2) "LAN Paramaters" --> Set the IP, subnet mask, gateway (or set for DHCP) and configure VLAN information.
3) "LAN User Configuration" --> set up a user name and password to connect with.
(You can also set the account name and privilege level.)
4) Connect the LAN cable to interface 0
(You can also set up separate IPMI user, enable encryption, etc. here)
Testing IPMI
First ping the BMC IP. You don't need to set any IP on the OS, or even have hard drives in it - you are pinging the internal IPMI interface. It has its own internal NIC (in addition to the two external NICs) NOTE: Sometimes you don't get ping responses on this - if not try to connect using the following command anyway.
If that works, try connecting using IMPItools (apt-get install impitools) to the shell interface:
ipmitool -I lan -H 192.168.42.x -U login shell
You can enter different modes in the shell to get information, execute remote reboots, etc. Use ? for a full menu. Each command when entered alone will give its options, etc.
Configure SoL
With IPMI version 2.0 you have something similar to a built in text KVM over IP. It is Serial over Lan (SoL).
IPMI will redirect console output to the LAN and forward that over the network. It is text only, but provides BIOS level access, RAID configuration, booting other kernels, booting to single user mode, etc. all done remotely.
1)Reboot and go into regular BIOS setup (F2). Enter the "Serial Communication" menu and set up:
- Set Serial Communication to On with Console Redirection via COM2 (mandatory)
- Set Remote Terminal Type to VT100/VT220
- Set Redirection After Boot to Enabled
2) Configure the OS to direct boot messages and console messages to be redirected via SoL:
A) Modify /etc/inittab and add:
T0:23:respawn:/sbin/getty -L ttyS1 57600 vt100
B) Append this boot option to the kernel line in /etc/grub/menu1.st: console=tty0 console=ttyS1,57600
For example, the full kernel line might be like:
kernel /vmlinuz-2.6.18-92.1.6.el5 ro root=/dev/VolGroup00/LogVol00 console=tty0 console=ttyS1,57600
(Note: the option listed last - console=ttyS1,115200 in this case - is the primary output device.)
IMPI Commands
ipmitool and openimpi are two toolset you can use to access an IPMI enabled device.
support@support-desktop:~$ ipmitool -U root -H 192.168.42.100 shell
Starts an interactive IMPI shell session
support@support-desktop:~$ ipmitool -vv -I lanplus -U root -H 192.168.42.101 sol activate
Starts a SoL session. If configured console output will be redirected over LAN to remote impi client. Console a server remotely. Add -vv for verbose packet analyze.
support@support-desktop:~$ ipmitool -I lan -U root -H 192.168.42.100 -a user list
View all sensor information:
ipmitool> sdr
Temp | 40 degrees C | ok
Temp | 41 degrees C | ok
Ambient Temp | 30 degrees C | ok
Planar Temp | 37 degrees C | ok
Riser Temp | 39 degrees C | ok
Temp | 40 degrees C | ok
CMOS Battery | 3.08 Volts | ok
And much more . . .
And what DO those yellow blinking lights on Dells mean?
ipmitool> sel list 1 | 04/19/2005 | 09:20:58 | Event Logging Disabled #0x51 | Log area reset/cleared | Asserted 2 | 05/18/2005 | 08:40:44 | Physical Security #0x52 | General Chassis intrusion | Asserted 3 | 05/19/2005 | 06:07:23 | Physical Security #0x52 | General Chassis intrusion | Deasserted 2f | 01/20/2006 | 19:08:02 | Temperature #0x03 | Upper Non-critical going high 30 | 01/20/2006 | 19:08:47 | Temperature #0x03 | Upper Non-critical going high 9d | 11/07/2007 | 16:43:39 | Slot/Connector #0x01 | Fault Status | Asserted
More IPMI commands are at:
Dell 1850s
The above steps for basic IPMI setup and using the IPMI shell work on the Dell 1850s The BIOS and BMC configuration is slightly different. Note that SoL does not appear to work, and the latest BMC firmware (ver. 1.73) only supports IPMI ver. 1.5. However, you can still use the shell, view system logs, sensors, etc.
BIOS: Console redirection --> Serial Port1 Integrated Devices --> BMC NIC
There are options listed in the shell for setting up SoL using ver. 1.5 but they do not seem to work.
Additional Information
I found this site to be particularly helpful in setting IPMI up.
The following is from http://bgoglin.livejournal.com/13317.html
IPMI is now configured correctly. You should be able to ping the IPMI IP addresses.
$ ping 10.0.99.x
Now, you may for instance reboot a node using the following line. Replace cycle with status to see the status,
off to shutdown, on to start.
ipmitool -I lan -H 10.0.99.x -U login -P passwd chassis power cycle
Now we need to configure console redirection. It makes it possible to send the BIOS, Grub, and ttyS1 output
through IPMI on the network on the first network interface. Note that COM2/ttyS1 is mandatory, it may not be
COM1/ttyS0 instead. After booting, press F2 to enter the BIOS. Go in the Serial Communication menu:
* Set Serial Communication to On with Console Redirection via COM2
* Set Remote Terminal Type to VT100/VT220
* Set Redirection After Boot to Enabled
With this configuration, you should see the BIOS and Grub output remotely using:
$ ipmitool -I lanplus -H 10.0.99.x -U login -P password sol activate
Then we want to see the kernel booting remotely. This is done by adding the following to the kernel command line.
With Grub, you might want to add it to # kopt=... and then run update-grub to update all automatic entries.
console=ttyS1,57600 console=tty0
And finally, you might want to get a console login remotely through IPMI. To do so, add the following line to /etc/inittab:
T0:23:respawn:/sbin/getty -L ttyS1 57600 vt100
With all this setup, the above ipmitool sol activate line will display the same thing than the physical console
on the machine, which makes it very nice to configure the BIOS, change the kernel, debug, ... Note that ~ is the
control character when using the console redirection. And ~. may be used to leave the console. Also ipmitool sol
deactivate may help if somebody did not leave the console correctly.
Dell 1950 Test Notes:
/boot/grub/menu1.st
kernel /vmlinuz-2.6.18-92.1.6.el5 ro root=/dev/VolGroup00/LogVol00 console=tty0 console=ttyS1,115200
/etc/inittab
T0:23:respawn:/sbin/agetty -L ttyS1 57600 vt100
s0:2345:respawn:/sbin/agetty ttyS1 1 57600 vt100-nav
Dell's RH 5.1 IMPI update tools package: http://ftp.us.dell.com/sysman/openipmi-39.0.1.RHEL5-1dkms-A00.tar.gz
Yum install OpenIPMI OpenIPMI-devel -tools -libs
Additional Testing Info
http://www.dell.com/downloads/global/power/ps4q04-20040204-Murphy.pdf
impitool -I open channel info
If you run ipmitool and get an error the device is not available:
modprobe ipmi_msghandler modprobe ipmi_devintf modprobe ipmi_kcs_drv modprobe ipmi_kcs_drv lsmod
You can get Dell versions of OpenIPMI at:
http://linux.dell.com/files/openipmi/
Or the sourceforge versions at:
http://openipmi.sourceforge.net/
References
http://bgoglin.livejournal.com/13317.html
Info on setting up SOL and IPMI on Dell servers
http://www.opengear.com/SP-IPMI_SOL.html
IPMI tools on Red Hat systems
Info on secure remote access proxy ideas.
http://sourceforge.net/projects/openipmi/ http://openipmi.sourceforge.net/
OpenIPMI project
http://linux.die.net/man/1/openipmigui
ftp://ftp.supermicro.com/CDR-0010_2.03_for_IPMI_Server_Managment/Manuals/IPMIView20.pdf
