Ipmi access from a local network of computers. Quantifier - system integrator
Frequently Asked Questions about IPMI motherboards Supermicro: why is it needed, features, visual demonstration, pitfalls.
1) The most popular myth about KVM-over-LAN is that it is "not needed". For some reason, many people confuse it with the usual remote access to the text / graphical OS console, i.e. "why do we need KVM-over-LAN when we have SSH, VNC, RDP and Radmin?"
The difference is fundamental: KVM-over-LAN is not just access to the OS console, it does not depend on the OS in any way, it is access to the console of the server itself (and not only to the console, details will be below), i.e. we can, for example: go into the motherboard BIOS or add. controller, install an OS, configure sensor monitoring (which, again, is very important - independent of the OS) via SNMP, etc.
2) How it works? It works quite simply, here is the block diagram (Supermicro has been using Winbond or Nuvoton WPCM450 lately:
All the work is done by the BMC (Baseboard management controller) processor - it has its own memory and a specialized OS (usually based on Linux). The BMC uses the USB bus to connect virtual keyboards, mice and CD/DVD/FDD drives. Other buses collect readings of fan rotation sensors, temperature, power management, access to the COM port (for remote access to the serial console). BMC is also engaged in capturing and redirecting the contents of the video buffer - modern BMCs already have an integrated video core, once, in the first implementations, BMC was engaged in digitizing an analog signal from a separate VGA controller. Data exchange with the "outside world" occurs via ethernet (dedicated port or one of the ports located on the ethernet controller motherboard).
Actually, WPCM450 in Supermicro products is a system based on ARM processor 926EJ-S operating under Linux on the 2.6 kernel.
For diagnostic purposes, you can log in via SSH, where busybox will be waiting for you.
3) How much does it cost?
Supermicro is currently integrating the IPMI BMC directly onto motherboards. Previously, additional modules were used (for example - AOC-SIMSO +). The presence of an IPMI BMC on a Supermicro board is denoted by the letter F, for example: X8DTi and X8DTi-F, and the price difference does not exceed $50, which is already an obvious answer to the question “should or not”, because external IP-KVM will cost you much more. Installing a regular KVM and every time you need to reinstall or reflash something, go to the server room is also not an option, it's a waste of time, because your server can be located tens and hundreds of kilometers away, and, in the end, it's just inconvenient.
4) What does it look like in practice?
IPMI can be accessed either through IPMIView (a specialized utility in Java, there are builds for Windows and Linux) or through a regular browser (an additional java application will be launched when connected to the graphical console).
Consider IPMIView. To begin with, you can configure the IPMI ethernet port in the BIOS: the default is DHCP, but you can manually set the desired IP / mask / gateway, VLAN tag (all stung management is best allocated to a separate subnet for security and convenience reasons).
We launch IPMIView and find the desired server (you can scan the address range for IPMI), the default login/password is ADMIN/ADMIN.
- graceful shutdown- a quick press of the Power button is simulated so that the OS can correctly turn off the power on its own.
- power cycle is a Graceful shutdown followed by a power on
- reset and power up speak for themselves
- power down- hard shutdown, with a long press of the Power button
Network settings can be changed if necessary on the BMC Setting tab (remember that you can lose remote access to the server by mistake). By the way, about remote access: the following ports are used (they will need to be forwarded through NAT or provide a way to access the management network, for example, through a VPN):
TCP:
- http:80
- HTTPS: 443
- 5901 - needed for graphic console
- 5900 - HID, virtual keyboard/mouse traffic
- 5120 - virtual CD/DVD drive traffic
- 5123 - virtual FDD traffic
- IPMI: 623
On the tab KVM Console you can see the most useful thing - the graphical server console. There are also buttons for taking a screenshot, managing virtual media (Virtual Media), switching to full screen mode and an additional soft keyboard. Why is it needed if there is a hardware one? For various key combinations that for some reason cannot be sent to the server from the hardware keyboard, for example, the classic Ctrl+Alt+Delete.
Here are a few features:
- If you have an LSI controller, then you'll be familiar with the mouse-oriented LSI Webbios interface. So: a mouse in Webbios via IPMI does not work in modern Supermicro boards, this is due to the fact that Webbios is designed for a PS / 2 mouse, and IPMI emulates USB mouse. In the FAQ section technical support Supermicro is considering this issue, but the recommendations do not help. You have to use only the keyboard, which is not very convenient, because. not all items can be navigated using Tab - you need combinations with Alt, which are worked out only through the Soft Keyboard.
Switching layout in WinPE 3.0 does not work. So build WinPE with the English layout, for this you need to add the following command to the build script after mounting the Wim image:
Dism /image:C:\winpe64\mount /Set-InputLocale:1033:00000409 "C:\winpe64\mount" - image mount point.
If you wish, you can change the locale and interface language at the same time - see the documentation for WinPE.- For non-Windows OS, change the cursor synchronization mode from absolute to relative.
You can either redirect the local drive or mount the image, which is much more convenient. A reboot may be required for the corresponding device to appear in the BIOS boot menu.
P.S. How to reset IPMI password? Only using the ipmicfg console utility. Available in DOS, Windows and Linux versions. Running ipmicfg -fd resets all settings and sets the login / password of a user with administrative rights to the standard ADMIN / ADMIN.
Update from 04/18/2010. At some point after next update Java, trying to mount iso image in IPMIView started causing it to crash (Windows 7 64bit with the latest updates). A new release of IPMIView has been released (build 110412 dated April 12). I have not yet checked whether this bug has been fixed there, since you can use the launch of the console through the web interface.
Go to tab remote control, press launch console and get in a separate window Redirect Viewer, similar in functionality to the console in IPMIView.
It is worth adding that the web interface does not make it easy to access the console from an external network - Redirect Viewer is not a Java applet but a standalone Java application and uses the same ports for video, HID and virtual drives: 5900, 5901, 5120, 5123.
P.S. from 01.12.2011. Additional article: FreeIPMI.
P.S. from 06.10.2013. Similar.
P.S. dated 11/10/2013.
P.S. dated 06/20/2014. Again .
All servers offered by us for rent are equipped with an IPMI (Intelligent Platform Management Interface) controller that allows you to turn on, off, remote connection(KVM) with the ability to mount ISO images, as well as providing access to information about the current state of the server.
With the help of IPMI, after issuing the server, you can install the operating system and initial configuration of the server. Since the IPMI controller is connected via a separate cable and has its own IP address, even if you lose access to the OS, you can always manage the server remotely without the need for direct physical access.
How to install OS?
To install the operating system, you will need to connect to the IPMI server using one of the methods convenient for you: via the Web interface or using the IPMIView program. The article will describe both options, but we recommend using the second method.
Web interface
To connect via the web interface, enter the IPMI address in the address bar of your browser and log in. You can find the address and details for authorization in the letter with access to the server or in your personal account. Next, you need to go to the Remote Control -> Console Redirection tab and click the Launch Console button.
In the Device 1 section, select ISO File in the Logical Drive Type drop-down list and specify the path to the image on your disk using the Open Image button, then connect the image with the Plug In button.
After connecting the image, restart your server in the Power Control section ->
Some servers allow you to connect up to three devices using the Device 2 or Device 3 tabs under Virtual Media -> Virtual Storage . This can be useful if you need to install additional drivers during installation.
If you plan to install Windows OS on the server, you can pass the Ctrl+Alt+Del key combination in the Macro - Macro section.
Instructions for working with the web interface in older versions of IPMI firmware
IPMIView
To use the Supermicro IPMIView program, you need to download it from the official website, specifying your data, or directly from the ftp server: https://www.supermicro.com/wftp/utility/IPMIView/
After installing and running the program, you need to add your server in the File - New - System section.
As System Name, specify the name of your server, and in the IP address column, enter the IPMI address, which you can find in the letter with access to the server or in your personal account. Make sure there are no spaces at the end of the line in the address column and click OK .
In the list on the left, double-click on the name of the added server, after which you will see the authorization window. Fill it out using the details from the letter or personal account and click Login . In case of a successful connection, you will see the inscription Connected, and additional tabs for managing the server will appear at the very bottom of the window.
To manage the server remotely, go to the KVM Console section and click the Launch KVM Console button.
Further steps are similar to working through
With the IPMIView program, you can also reboot, enable or disable your server in the IPMI Device tab.
Not so long ago, I managed to work with new servers for me Supermicro remote control which is carried out using the interface IPMI. In this article, I will try to cover the main points in the IPMI settings on Supermciro servers, show the main menu items of the IPMI interface, and also talk about additional utilities, commands and ways to monitor the server using ipmi.
IPMI(Intelligent Platform Management Interface) is an intelligent platform management interface designed for autonomous monitoring and management of functions built directly into the hardware and firmware of server platforms. (Information taken from an official source). IPMI is implemented as a separate server controller that does not depend on the operating system, BIOS, CPU of the server and allows you to remotely control physical equipment.
Initial configuration of the IPMI console and familiarity with the interface
Setting up IPMI begins with setting the IP address of the interface, which must be specified in the BIOS. If the server is installed in the office, this may be a gray IP address, but if your equipment is located in the Data Center, then you are probably using a white static IP address.
On Supermicro servers, the BIOS can be accessed by pressing the “ Del” when loading the server, I will not focus on this and will immediately go to the interface of the BIOS itself.
As you can see in the screenshot, I entered the IPMI menu item and activated the Lan configuration option for the interface (Update IPMI Lan Configuration=Yes, Configuration Address Source = Static), specified the IP address, subnet mask and default gateway.
You can apply the settings by clicking the button. F4, after which the server will reboot.
If you did everything correctly, then by entering the IPMI IP specified for the IPMI interface in the browser, authorization will open:
Now let's go through the main points.
When buying a new server, a user has already been created in IPMI ADMIN with password ADMIN, it is with this data that we authorize. I always recommend creating a new user, and deleting the standard user or changing his password, so leaving such a username and password is extremely unsafe. You can create a new user or change passwords / privileges for current ones in the menu Configuration -> Users.
Restarting the IPMI interface can be done from the menu Maintenance -> Unit Reset.
Mounting the iso image of the OS for installation on the server is performed in the menu Virtual Media -> CD-ROM Image.
My OS iso images are stored on the Samba server on the same subnet as the IPMI interfaces of the servers. I give the address samba servers, the path to the installation ISO image with the OS, after which I mount the ISO image and proceed to install the operating system.
Remote graphical Java console of the server ( KVM-over-IP) can be opened via Remote Control -> Console Redirection.
The console is generally convenient, but after installing the OS, I usually use ssh clients to work with the server. Sometimes there are times when this is not possible, for example, if the network on the server does not work or is not yet configured, or there are some access problems. That's when the Remote Console comes to the rescue.
You don’t need to make any separate settings in the console itself, I just want to add that it has a built-in keyboard that can be called from the menu Virtual Media -> Virtual Keyboard.
Advice. After working with SUpermicro servers for a long time, I found one annoying bug. After installing the operating Centos systems 7 and installing KVM on it, the ability to use the Remote Console disappears. While the server is in the process of loading, the console responds and you can go to Bios or see the loading of the OS kernel. But as soon as operating system loaded, the video in the console disappears. Through experimentation, I managed to defeat this bug. You need to add the nomodeset boot parameter to the kernel boot. To do this, after installing the OS, you need to run the command:
grubby --args "nomodeset" --update-kernel /boot/vmlinuz-`uname -r`
After that, Remote Console works properly.
Server management through the IPMICFG utility from SuperMicro
To manage servers via IPMI, SuperMicro is developing its own utility IPMICFG.
You can download the IPMICFG file using the command:
wget ftp://ftp.supermicro.com/utility/IPMICFG/IPMICFG_1.30.0_build.190710.zip
At the time of publication, the file was with this name, I recommend visiting the SuperMicro repository using the link ftp://ftp.supermicro.com/utility/IPMICFG/ and copy the url of the actual file.
Unpack the downloaded file into the directory where we are:
ln -s /root/IPMI*/Linux/64bit/IPMICFG-Linux.x86_64 /usr/local/sbin/ipmicfg
Now we can run the utility using the ipmicfg command (symbolic link). Consider the basic features of the ipmicfg utility.
If we run the ipmicfg -help command, we will get a list of all possible utility options.
Let's analyze the main commands:
- ipmicfg -help - full help on the utility;
- ipmicfg -m - view the current IP and mac address for IPMI;
- ipmicfg -k - view the subnet mask;
- ipmicfg -g - view the specified gateway;
- ipmicfg -fd - reset IPMI to factory settings;
- ipmicfg -user list - View created users and their privileges.
You can change the IP address, mask and gateway of the IPMI interface:
- ipmicfg -m 192.168.1.200
- ipmicfg -k 255.255.255.0
- ipmicfg -g 192.168.1.1
Create a new IPMI user and password:
ipmicfg -user add Using the utility, we do the following: ipmicfg -user add 6 test 123456 4 Thus, we have created a test user with a password of 123456 and administrator privileges. The list of IPMI users and their privileges in the system can be displayed with the commands: As you can see in the screenshot, the test user has been created. To change (reset) the password of an IPMI user, use the command: ipmicfg -user setpwd And I will give a couple more examples of the use of this utility: Through IPMI, monitoring the temperature and operation of the hardware of the SuperMicro server is quite simple. Complete information about the server hardware is contained in the System -> Hardware Information item. Information on the state of the processor, RAM and fans can be viewed by going to the Server Health -> Sensor Readings tab. For ease of viewing, you can change the display categories of sensors, for example, temperature: Or voltage sensors: Since at the moment there are no problems on our server, neither with temperature nor with voltage, all sensors are in the green zone. If the server is experiencing temperature or voltage issues, the green rectangles will turn red to signal that your server is being tested. Checking the status of power supplies: # ipmicfg -pminfo item | Value ---- | ----- Status | (00h) Input Voltage | 217.5V Input Current | 1.06 A Main Output Voltage | 12.28 V Main Output Current | 17.93 A Temperature 1 | 23C/73F Temperature 2 | 21C/70F Fan 1 | 2064 RPM Fan 2 | 2032 RPM Main Output Power | 220W input power | 228W PMBus Revision | 0x22 PWS Serial Number | P2K4FCH02LT0125 PWS Module Number | PWS-2K04F-1R PWS Revision | REV1.0 Current Sharing Control | PEC error Item | Value ---- | ----- Status | (00h) Input Voltage | 217.5V Input Current | 1.09 A Main Output Voltage | 12.30 V Main Output Current | 18.09 A Temperature 1 | 24C/75F Temperature 2 | 22C/72F Fan 1 | 2064 RPM Fan 2 | 2064 RPM Main Output Power | 223W input power | 234W PMBus Revision | 0x22 PWS Serial Number | P2K4FCH02LT0126 PWS Module Number | PWS-2K04F-1R PWS Revision | REV1.0 Current Sharing Control | PEC error You can view the processor temperature with the command: ipmicfg -nm oemgettemp You can also check what mode the fans are in and change the mode if necessary. Checking Fan Status and Configuration # ipmicfg -fan Current Fan Speed Mode is [ PUE2 Optimal Mode ] Supported Fan modes: 0:Standard 1:Full 3:PUE2 Optimal 4:Heavy IO Changing the cooler mode: ipmicfg -fan IPMI version and firmware information: # ipmicfg -nm deviceid Device ID = 50h Firmware Version = 4.1.4.54 IPMI Version = 2.000000 Manufacturer ID = 57 01 00 Product ID Minor Ver = Greenlow platform Implemented DCMI version = DCMI not implemented/enabled Firmware implemented version = NM Revision 4.0 Image Flag = operational image 1 raw = 50 01 04 14 02 21 57 01 00 09 0b 04 05 40 01 And you can see all the sensors with the command ipmicfg -sdr In the output, we see that there are additional columns that display information about the limits of the lower and upper limits. There are also quite a few utilities that can be used to monitor and automate this process, for example, with nagios. At the moment, we will not focus on this, since the purpose of the article is to tell the main points in working with IPMI. If you are interested in the topic of monitoring, you can leave your wishes and perhaps in the future we will cover this topic as well. I hope this article will be useful for you! Frequently asked questions about IPMI in Supermicro motherboards: why is it needed, features, visual demonstration, pitfalls. All the work is done by the BMC (Baseboard management controller) processor - it has its own memory and a specialized OS (usually based on Linux). The BMC uses a USB bus to connect virtual keyboards, mice, and CD/DVD/FDD drives. Other buses collect readings of fan rotation sensors, temperature, power management, access to the COM port (for remote access to the serial console). BMC is also engaged in capturing and redirecting the contents of the video buffer - modern BMCs already have an integrated video core, once, in the first implementations, BMC was engaged in digitizing an analog signal from a separate VGA controller. Data exchange with the "outside world" occurs via ethernet (dedicated port or one of the ports located on the ethernet controller motherboard). For diagnostic purposes, you can log in via SSH, where busybox will be waiting for you. 3) How much does it cost? Network settings can be changed if necessary on the BMC Setting tab (remember that you can lose remote access to the server by mistake). By the way, about remote access: the following ports are used (they will need to be forwarded through NAT or provide a way to access the management network, for example, via VPN): On the tab KVM Console you can see the most useful thing - the graphical server console. There are also buttons for taking a screenshot, managing virtual media (Virtual Media), switching to full screen mode and an additional soft keyboard. Why is it needed if there is a hardware one? For various key combinations that for some reason cannot be sent to the server from the hardware keyboard, for example, the classic Ctrl+Alt+Delete. You can either redirect the local drive or mount the image, which is much more convenient. A reboot may be required for the corresponding device to appear in the BIOS boot menu. Go to tab remote control, press launch console and get in a separate window Redirect Viewer, similar in functionality to the console in IPMIView.
It is worth adding that the web interface does not make it easy to access the console from an external network - Redirect Viewer- this is not a Java applet, but a separate Java application and uses the same ports for video, HID and virtual drives traffic: 5900, 5901, 5120, 5123. Internet servers are the "backbone" of any company's online presence. Making sure that they remain fully operational is crucial to maintaining website uptime and functionality. The current industry standard for monitoring and managing a server installation is known as Intelligent Platform Management Interface and commonly referred to as IPMI. Generally speaking, IPMI is a standardized set of specifications for a hardware system, which allows a web host or data center to centrally monitor and control all of the servers it is managing. It was originally developed by Intel with support from Hewlett Packard, Dell, and NEC, and is now supported by most of the industry. IPMI works in tandem with two other standard specification sets, IPMB (Intelligent Platform Management Bus) and ICMB (Intelligent Chassis Management Bus), which handle the management functions within a computer and between the machines being managed. Communication is usually handled through a direct out-of-band LAN, but it’s also possible to use a sideband LAN through a NIC card which is a less expensive approach. Most of the key factors in a server’s hardware operation can be monitored via IPMI, including the health of the power supply, chassis security, and fans. It also tracks power levels, temperature and other important environmental factors. Additionally, the interface can check each machine's hardware logs, can receive pre-defined alerts, and can send messages to a server to reboot or power down. It even allows remote adjustment of BIOS settings. The front-end of an IPMI system is extremely user-friendly. Keyboard, mouse and video access to individual servers functions in the same way they would for an engineer or technician working directly on the machine. In the event a server is inaccessible, the error is clearly displayed on the monitoring console and the user may login directly from IPMI to modify specific network configurations. A modified and simplified variant on IPMI, known as DCMI (Data Center Management Interface), is often used by data centers because it includes some functions important for their systems (such as capping power to a server) while eliminating others which aren't needed for their purposes. There are a number of reasons IPMI is superior to more traditional software used for server system diagnosis. Most of them are based on the facts that IPMI is able to manage machines in multiple physical locations, and that it is able to monitor machines “from without” rather than “from within”; that is, it is firmware running on a machine's motherboard and is not dependent on a machine's operating system. The major benefits: There are only a few major drawbacks to IPMI, and in almost all cases, they are far outweighed by the benefits. Despite these few issues, IPMI has been almost universally adopted by data centers and web hosts as the most efficient and economical way to monitor and manage their networks.
Monitoring hardware on a SuperMicro server via IPMI and the IPMICFG utility
SuperMicro Server Monitoring via IPMI
Monitoring with the ipmicfg utility
1) The most popular myth about KVM-over-LAN is that it is "not needed". For some reason, many people confuse it with the usual remote access to the text / graphical OS console, i.e. "why do we need KVM-over-LAN when we have SSH, VNC, RDP and Radmin?"
The difference is fundamental: KVM-over-LAN is not just access to the OS console, it does not depend on the OS in any way, it is access to the console of the server itself (and not only to the console, details will be below), i.e. we can, for example: go into the motherboard BIOS or add. controller, install an OS, configure sensor monitoring (which, again, is very important - independent of the OS) via SNMP, etc.
2) How it works? It works quite simply, here is the block diagram (Supermicro has been using Winbond or Nuvoton WPCM450 lately:
Actually, the WPCM450 in Supermicro products is a system based on the ARM 926EJ-S processor running Linux on the 2.6 kernel.
Supermicro is currently integrating the IPMI BMC directly onto motherboards. Previously, additional modules were used (for example - AOC-SIMSO +). The presence of an IPMI BMC on a Supermicro board is denoted by the letter F, for example: X8DTi and X8DTi-F, and the price difference does not exceed $50, which is already an obvious answer to the question “should or not”, because external IP-KVM will cost you much more. Installing a regular KVM and every time you need to reinstall or reflash something, go to the server room is also not an option, it's a waste of time, because your server can be located tens and hundreds of kilometers away, and, in the end, it's just inconvenient.
4) What does it look like in practice?
IPMI can be accessed either through IPMIView (a specialized utility in Java, there are builds for Windows and Linux) or through a regular browser (an additional java application will be launched when connected to the graphical console).
Consider IPMIView. To begin with, you can configure the IPMI ethernet port in the BIOS: the default is DHCP, but you can manually set the desired IP / mask / gateway, VLAN tag (all stung management is best allocated to a separate subnet for security and convenience reasons).
We launch IPMIView and find the desired server (you can scan the address range for IPMI), the default login/password is ADMIN/ADMIN.
Server health can be monitored on the tab Sensors(readings of temperature and fan sensors you will see only when the server is turned on):
TCP:
UDP:
Here are a few features:
And the final paragraph is the use of virtual media.
Switching layout in WinPE 3.0 does not work. So build WinPE with the English layout, for this you need to add the following command to the build script after mounting the Wim image:
Dism /image:C:\winpe64\mount /Set-InputLocale:1033:00000409 "C:\winpe64\mount" - image mount point.
If you wish, you can change the locale and interface language at the same time - see the documentation for WinPE.
P.S. How to reset IPMI password? Only using the ipmicfg console utility. Available in DOS, Windows and Linux versions. Running ipmicfg -fd resets all settings and sets the login / password of a user with administrative rights to the standard ADMIN / ADMIN.
Update from 04/18/2010. At some point, after the next Java update, an attempt to mount the iso image in IPMIView began to lead to its crash (Windows 7 64bit with the latest updates). A new release of IPMIView has been released (build 110412 dated April 12). I have not yet checked whether this bug has been fixed there, since you can use the launch of the console through the web interface.
P.S. from 01.12.2011. Additional article: .
P.S. from 06.10.2013. Similar.
P.S. dated 11/10/2013. IPMI and security.
P.S. dated 06/20/2014. AgainWhat is IPMI?
Benefits Of IPMI
Drawbacks Of IPMI
