06272008008

04292008132

04292008131

04282008120

04112008086

Windows 2008 Server, EqualLogic, MPIO, Intel I/OAT & QLogic 4062C HBA *Part 1*

I had a very productive day of iSCSI configuration with Bill from VLSystems. Turns out it was his first install with any Windows 2008 (seems to be the case more often than not!) servers.

After his recommendation with the QLogic HBAs (see previous post), I went ahead and got a QLogic 4062C PCIe iSCSI HBA, the dual port derivative.

With both the Intel Pro 1000/PT Quad port and QLogic 4062C in hand, made it a point to have my own little bake off. The QLogic would reside in the Engineering server, while the Intel would get cozy in the Estimating server. Both of these are the PowerEdge 1950 III that I've mentioned in my previous couple posts.

Estimating server (Intel Quad Port)

There were a few things I wanted to accomplish with this server:

1. Enable MPIO
2. Configure iSCSI targets for each interface
3. Test without Intel I/OAT.
4. Enable the ever reclusive Intel I/OAT.
5. Test again with I/OAT

First of course, is to setup the physical connections to your iSCSI box, servers, switches, Aggregate Trunks, IP Addresses for each of your iSCSI interfaces. All that stuff has been written up tons of times already, so I'll skip it.

A recap of my setup for perspective, I have 2 Dell 5448 switches with a LAG Trunk of 4 ports (4Gbps) between the 2 switches. All connections were done according to EqualLogic's best practices. IP addresses were setup for the interfaces as well... Now to the juicy stuff.

In order to enable MPIO, there are a few items of interest with Windows 2008. Where everything was in the iSCSI initiator for 2003 server, there are 2 areas we need to look into to make sure MPIO is enabled. If we head to the Control Panel, and assuming you've added the File Server role to your server, you should see the items "iSCSI Initiator" and "MPIO"


Where you want to go first is MPIO.

Of particular interest is the second tab, "Discover Multi-Paths." There is a checkbox for "Add support for iSCSI devices" that needs to be checked. After this is selected and applied, your server needs to be rebooted for it to stick. Mine's already done thus the greyed out box. After you reboot and go back into the MPIO Control Panel, you should see something similar to this:

This tells us that MPIO has been started. Next we want to goto the iSCSI Initiator control panel item, and the Discovery tab. Here we can add a "Target Portal" which will be the Group IP Address for our EqualLogic SAN.


After this is done, we should be able to goto the "Targets" Tab to see the available iSCSI Volumes made available by the EqualLogic Boxes. In this case they're still inactive, but lets fix that. I'm going to do testing on our SAS volume.

To connect to the Inactive target, all we have to do is select the target we want to log onto and click on "Log on..." In this case, I set the security on the EqualLogic side to restrict connections by IP address. Other options are Radius Server, CHAP authentication, etc.

Make sure you select "Automatically restore..." and "Enable Multi-path." Click on the "advance" button to select the local adapter as "Microsoft iSCSI Initiator" (do not leave as default) and the Source IP to one of the NICs you'll be using to access the SAN. Click OK, the OK. After a brief pause, you should see connected under status of the target you selected.

Follow these steps again with the SAME CONNECTED TARGET starting from selected that same target and click on "Log on..." This time, in the advanced window, select the Source IP of the OTHER NIC you wish to multi-path with. At this point you should be MPIO'd. You can check by selecting the "Details" of the target. You should see 2 or more identifiers, depending on how many NICs you've used in the connection.

Now you need to setup the disk just like any other new disk in the diskmanagement MMC. After formating it and mounting it to the appropriate letter, its important to go back into the iSCSI initiator page and configure the "Volumes and Devices" tab. I just pressed the "Autoconfigure" button to let windows discover my iSCSI targets and what drive letters they will reside at.


Whew... thats a long post. I'm going to finish this up a bit later...

PE1950 1U Servers

I got off the phone with VL Systems, the integrator who will be helping us out with our EqualLogic SAN integration, and they suggest that I not use the Intel Pro 1000/PT quad port NICs for our iSCSI implementation. Doesn't have TOE or iSCSI optimization. doh.

Although it does not support TOE natively, when used with dual quad core xeons, it should be enough to keep up. We're going to run some benchmarks when they come do the install.

In the mean time... I need to find a suitable iSCSI card in case it does not live up to performance expectations. Something with a Broadcom chipset such as the HP NC380T but not HP. Dell servers do not like HP NICs.

Windows Server 2008...

Does not like you to disable its firewall. In fact, it seems to lock itself down when you disable it. We had a server with the firewall disabled and we could not reach it whether it be by pinging or DNS replication.

Very counter-intuitive.

Joikuspot on N95 8GB NAM

Joiku Light Beta v1.1 on Nokia N95 8gb AT&T w/ 3.5G showing 75% signal strength.

I'll have to test my Sprint EVDO rev.A broadband card for comparison.

MBA Side view

MBA Open

New MBA

Kaiser Ontario JV trailer

A shot of the in progress setup in Ontario.
On the left side you can see the Shoregear 90 phone switch along with a cheapo netgear wireless router hooked up to the Adtran T1/VOIP box. Theres also a Dlink 26 port POE switch there to power all the phones. Gone missing was a watchguard router and Dell 48 port POE switch. They just got up and walked away! damnit.

Shoretel

Shoretel IP115 phone. Why did I shoot this upside down? i have no idea. If you want a better look at it, do a headstand.

ADTRAN eth & DS1 ports

Dell PowerConnect 5448 iSCSI switch

2 for <$2000!

Supports jumbo frames, flow control, layer2, w/ VLAN support. Currently testing it out with our EqualLogic SANs.

Dell PE1950 III

What a steal for these machines. Even direct from Dell they're a great deal.

• Dual Quad Core 2.5GhZ Xeons w/ 12MB Cache (45nm chips)
• 4GB (win 2k8 32bit machines) or 32GB (for VMWare hosts)
• 256MB Cache BBU Hardware RAID 1 with 2x 146GB 15K RPM Drives
• DRAC 5 (remote access card)
• Quad Port Intel Pro 1000 PT PCIe Card
• Redundant Power Supplies

4GB version costs us less than the base model HP DL360 G5, and the 32GB version costs us about 2/3rds of what a comparable HP.

Infrastructure update

I've started this blog while in the middle of our big infrastructure update.



You name it, we're updating it:

1. New Cat6e homeruns


2. All new Dell PE1950 III Servers


3. Windows 2008 Server


4. VMware VI3


5. EqualLogic SAN (PS100E + PS5000XV)

Yes we're putting in an iSCSI SAN. I've even purchased a couple of the new Dell PowerConnect 5448 Switches ($900 each!) that are supposedly designed especially for iSCSI traffic. Although I've heard bad things about Dell switches, I could not pass them up at that price. I'll just test them out and if they don't meet performance expectations, I'll ship them back and pick up either a Cisco 2960G or HP 2810 Switch. I don't have the budget to do redundant switches if I Cisco or HP, though.

Back to the EqualLogic iSCSI boxes. We looked at 5 vendors total: LeftHand, IBM, EMC, NetApp, and EqualLogic. Here is my progression of logic in choosing EqualLogic over LeftHand:

• EqualLogic & Lefthand are not "tier one" providers of SANS (some may argue this now that Dell has bought EqualLogic.) In comparing these two solutions, I tried digging into independent reviews/comparisons of midrange SANs. Lefthand does not appear in many of these reviews as it seems like many in the storage industry do not consider LeftHand as serious a contender in this space as NetApp, EMC, or EqualLogic


• Lefthand SANs configured properly and within my budget would only give me 50% of RAW disk space as usable space. Now this changes when you add more boxes of disks, but at this point, we're looking at what we can immediately use.


• Lefthand had single controllers in its boxes.


• According to Microsoft ESRP (Exchange Solution Review Program), LeftHand would handle less users per disk.


• Lefthand was only recently certified with VMware

Performance was a major driver in this project as CAD Engineers are apparently not very patient. ;) With LeftHand off the table, I focused my attention on NetApp, EMC, and IBM.

• EMC, NetApp and IBM (DS3000) are all controller + "dumb bay of disks" type SANs. EqualLogic has 2GB cache + redundant 3x GigE Ports on each bay of disks. Everytime you install a new bay, you gain more cache and more potential bandwidth.


• EMC, NetApp & IBM were very expensive for yearly maintenance (10K+) on hardware and software was ala carte. EqualLogic was all inclusive (like LeftHand).


• NetApp does have dedupe included which was very attractive when considering we will be moving our servers to VI3.


• The IBM DS3000 is a rebranded LSI product which is what the Dell MD3000 is based on. Overall not a great performing product in comparison to the EqualLogic.


• EMC was only certified "five 9's" reliability rating less than a year ago in their midrange product: http://chucksblog.typepad.com/chucks_blog/2007/05/five_nines_and_.html


• EqualLogic has been "five 9's" for 3+ years.


• Performace: EqualLogic has all comers (even many fiber channel!) beat: http://technet.microsoft.com/en-us/exchange/bb412165.aspx

I believe I'll have the SAN in hand next week so I'll do some testing and report back!

Peace out!

Hello!

I'm starting this blog to keep track of the IT projects I have here at Murray.