Posts Tagged ‘blade’

Painless Dell FX2 Firmware Upgrade

April 10, 2016

Overview

Recently I’ve had a chance to play with Dell’s FX2 chassis for a bit. Dell FX2 falls into the category of blade chassis and can hold up to 8 blades with Atom or 4 blades with Xeon CPUs in a 2U chassis.

Dell_FX2

Besides the compute blades FX2 also supports storage blades, which you can dedicate to particular compute blades and use as additional storage.

On the networking side you can choose from either pass-through modules or three types of I/O aggregators – four 10G SFP+ ports, four 10GBASE-T ports, or two Fibre Channel plus two SFP+ external ports.

The chassis itself also comes in two flavors – FX2 or FX2s. The main difference between the two is that FX2s additionally has PCIe slots at the back, which can be mapped to the server blades to provide additional connectivity.

Dell_FX2_Rear

First step of every hardware solution deployment is a firmware upgrade. But when it comes to firmware on Dell blade equipment be it M1000e, VRTX or FX2 you can quickly get confused. Especially when you go to the blade section and see a dozen of hardware components. Download and update each of them individually would be daunting. Fortunately there is an better way.

blade_firmware

CMC Firmware

Upgrade starts from the chassis management controller, which has two components: Chassis Infrastructure Firmware (or Main Board) and the CMC itself. You can find them on the Chassis Overview > Update tab.

CMC firmware comes as an .exe package, which you can extract. You really need just the fx2_cmc.bin file. During upgrade you will lose access to CMC for 5-10 minutes, while CMC is rebooting.

For the infrastructure firmware you’ll need the fx2_mainboard.bin file. The gotcha with the infrastructure firmware upgrade is that you’ll need all blades to be powered off. So if you have just one chassis this might be tricky.

Blade Firmware

Blades firmware is where this gets interesting. You can certainly upgrade all blades from the CMC by downloading firmware from the Dell support web-site and choosing one component at a time in Chassis Overview > Server Overview > Update section. CMC is capable of upgrading say iDRAC across all blades simultaneously, but it’s still about a dozen components.

The easier approach would be to use Dell Repository Manager (DRM). DRM can download firmware for virtually any blade or rack server (including some of the storage and network hardware) and build a bootable ISO image for an easy upgrade.

To build a bootable ISO follow the following steps:

  • Download and install Dell Repository Manager from the Dell support web-site
  • Add a source by going to Source > View Dell Online Catalog
  • Create a repository by going to Repository > New > Create New Repository
  • In the wizard select your hardware (I selected PowerEdge FC630 from the Blade category) and choose Linux (32-bit and 64-bit) as a DUP format (I’ll explain that later).
  • Go to the newly created repository, select the bundle and click Export

export_bundle

DRM can export bundles in multiple forms, we are interested in a bootable ISO and this is why we selected the Linux DUP format when we created the repository. DRM creates a Linux bootable ISO, so there was no point selecting Windows bundles.

  • Select “Create Bootable ISO (Linux Only)” and continue with the default settings for the rest

As a result you will get an .iso file, which you can mount to the server via iDRAC Remote Console and boot from it for a firmware upgrade.

Network I/O Aggregators

FX2 I/O aggregators are Dell Force10 switches, which use Force10 OS (FTOS). FTOS firmware is NOT available from the Dell web-site. You’ll need to register an account at https://www.force10networks.com to download the firmware.

Make sure to download firmware release specifically built for FX2 I/O aggregators, which can be found in M-Series Software section.

aggregators_firmware

To upgrade the aggregators go to Chassis Overview > I/O Module Overview > Update. Aggregators reset after a reboot, so make sure to upgrade them one at a time. Or if you stacked them instead of using VLT or standalone mode, you’ll have to have a downtime, as stacked switches reboot together.

Conclusion

There is nothing fancy in upgrading firmware on a blade chassis, you want it to be quick and painless. Make sure to use Dell Repository Manager for blades upgrade. It may save you heaps of time and make your life easier.

Force10 MXL: Initial Configuration

March 14, 2015

Continuing a series of posts on how to deal with Force10 MXL switches. This one is about VLANs, port channels, tagging and all the basic stuff. It’s not much different from other vendors like Cisco or HP. At the end of the day it’s the same networking standards.

If you want to match the terminology with Cisco for instance, then what you used to as EtherChannels is Port Channels on Force10. And trunk/access ports from Cisco are called tagged/untagged ports on Force10.

Configure Port Channels

If you are after dynamic LACP port channels (as opposed to static), then they are configured in two steps. First step is to create a port channel itself:

# conf t
# interface port-channel 1
# switchport
# no shutdown

And then you enable LACP on the interfaces you want to add to the port channel. I have a four switch stack and use 0/.., 1/.. type of syntax:

# conf t
# int range te0/51-52 , te1/51-52 , te2/51-52 , te3/51-52
# port-channel-protocol lacp
# port-channel 1 mode active

To check if the port channel has come up use this command. Port channel obviously won’t init if it’s not set up on the other side of the port channel as well.

# show int po1 brief

port_channel

Configure VLANs

Then you create your VLANs and add ports. Typically if you have vSphere hosts connected to the switch, you tag traffic on ESXi host level. So both your host ports and port channel will need to be added to VLANs as tagged. If you have any standalone non-virtualized servers – you’ll use untagged.

# conf t
# interface vlan 120
# description Management
# tagged Te0/1-4
# tagged Te2/1-4
# tagged Po1
# no shutdown
# copy run start

I have four hosts. Each host has a dual-port NIC which connects to two fabrics – switches 0 and 2 in the stack (1 port per fabric). I allow VLAN 120 traffic from these ports through the port channel to the upstream core switch.

You’ll most likely have more than one VLAN. At least one for Management and one for Production if it’s vSphere. But process for the rest is exactly the same.

The other switch

Just to give you a whole picture I’ll include the configuration of the switch on the other side of the trunk. I had a modular HP switch with 10Gb modules. A config for it would look like the following:

# conf t
# trunk I1-I8 trk1 lacp
# vlan 120 tagged trk1
# write mem

I1 to I8 here are ports, where I – is the module and 1 to 8 are ports within that module.

HP BladeSystem c3000

October 29, 2011

We have High Performace Computing (HPC) cluster I’d like to show. It has 72 cores and 152GB of RAM in total. We use ROCKS as cluster middleware. Interconnect is DDR InfiniBand.

We have two groups of servers. First group is two BL2x220c  blades. Since they are double-sided it’s actually four servers. Each with two 4-core CPUs and 16GB of RAM. Second group consists of five BL280c. Each of them also has two 4-core CPUs but 24 GB of RAM. Eighth server is BL260c. This blade serves as master server.

Click pictures to enlarge.

BL280c blade server. This dude has 8 Xeon cores and 24GB of RAM.

Every component of HP BladeSystem c3000 is hot-swappable. Here I show how I disconnect Onboard Adminstrator on fully operational system.

Fans, power supplies and all interconnects are on the back.

Here is the 16-port DDR InfiniBand switch. Each port’s throughput is 80GB/s FDX.

Uplink ports for Onboard Administrator.

16 ports of Ethernet pass-through for blade servers.


Six power supplies in N+1 redundant configuration. Each is capable of 1200 Watts. 7200 Watts in total.

Inside blade server.

InfiniBand mezzanine. One such module is capable of 80Gb/s FDX.

If you are interested in benchmarking results find them here for pure IB and here for IBoIP.

HP BladeSystem c3000 power issue

September 28, 2011

Today I faced a problem when HP ProLiant BL260c G5  blade rejected to boot with “System Halted until Power Condition is Corrected” error. Blade sits in HP BladeSystem c3000 chassis. Incident which led to this error was lost of power by one of power supplies. Even though issue was resolved and power restored blade still rejected to boot.

Same problem is described here http://h30499.www3.hp.com/t5/HP-BladeSystem-Chassis-Power/Blade-Power-Error/td-p/1155736 and here http://h30499.www3.hp.com/t5/HP-BladeSystem-Chassis-Power/BL685c-G6-power-issue-Need-advice/td-p/1164218.

Solution to this error was as simple as power cycle the blade by physically disconnecting it from the chassis and plugging back in.