The other day I mentioned that I was starting to build a home vSphere lab. While we have two very good demo and engineering labs at Varrow I wanted something I could break and rebuild on my own schedule. I’m also planning to do some good VMware 101 type posts and videos and I think this mini-lab will be a great environment for those. Many people have asked me to post my build when I’m finished so I thought I’d go through each section and detail why I made the choices that I did. In no particular order other than the fact I ordered this first, we’ll start with storage.
Shared storage is a key requirement for my lab. A number of other lab builds I saw (such as rootwyrm’s here) use local storage in their vSphere servers but I want shared storage so I can use the “enterprise” features in vSphere. One lesser-cost option would be to add local storage to my vSphere hosts and then run something like EMC’s Uber VSA, which is a virtual NAS appliance. The VSA works very, very well and let’s you even configure replication that is supported by Site Recovery Manager. But…I wanted dedicated storage and I also wanted it to perform well. This is a problem as I learned earlier when I bought my HP EX495 Windows Home Server NAS. To get good performance you need to spend some money. I thought it would be easy to find a home NAS system that could easily fill a Gb connection for a reasonable amount of money…and I was wrong. You can see that very easily in Small Net Builder’s (SNB) charts right here. My main storage requirements were:
- VMware vSphere Certified – I don’t want to be chasing odd issues in an NFS stack on an untested platform.
- Performance – Saturate, or come close to it, a Gb Ethernet link on sequential reads and writes.
- Energy Efficient – I’m a power/cooling geek at times and this is especially true in my home office. I don’t want a sauna and I don’t want to pay the price of the NAS every year again to power it.
- Quiet – Along with not wanting to be in a sauna at home I also don’t want to listen to whining fans and disk drives all day. My iMac is quiet. My HP EX495 is quiet. This needs to follow along.
- NFS & iSCSI Support – While I mainly plan to use NFS I do want iSCSI support. No Fibre Channel in the home lab….yet.
The requirements that I have cut down a lot of the options out there for NAS. When you get to it, the field was pretty much Iomega, Synology, and QNAP. My first thought was just to go with an Iomega IX4-200D seeing as I’m an EMC guy and I work for the best EMC partner in the land (Spoiler Alert: EMC owns Iomega). That’s a pretty standard device that a lot of my peers use in their labs. The software it has is very good and supports replication (though not supported by SRM). A friend who deals in storage issues at VMware commented that the IX4 is a very reliable device. The IX4 is also very reasonably priced at $550 to $600 for the 4TB (4x1TB disks) model. Most others I was looking at were that much or more without drives. You can’t choose the model drives you get in the Iomega but I see that as a minor inconvenience. So, easy choice, right? Not really.
The Iomega IX4′s big weakness is performance. Let’s refer back to SNB’s charts here. If you look at the data you’ll see that the IX4′s performance is not very good and in some tests it’s downright abysmal. Performance is a key requirement both for the lab environment as well as some other tasks that I plan to put on the NAS. This device will also house some media for streaming to my Mac Mini running Plex as well as probably handling the Time Machine backups of our household Macs. So while I’m a big EMC proponent I decided against the Iomega. I bet Iomega releases like better performing model in a month just to spite me!
The next two options, Synology and QNAP, are very similar. They have similar models, features and performance. They also have models that are VMware Ready certified, though I expect almost all of their models would work fine as they run the same operating system. In the end I chose a Synology DS1010+ for several reasons…mainly price/performance, community support, and the very well reviewed management interface and firmware on the Synology.
The DS1010+ is a five-bay system that uses a dual-core Atom D510 processor running at 1.66GHz. The charts above show the performance that is capable of achieving. It has two Gb network ports that can be used for two separate networks or aggregated together manually or by using LACP. The management interface is a very fast and robust AJAX-based system that has been very easy to work with during configuration. Unlike my HP EX495 that requires a few manual steps and a client install for Time Machine support the Synology systems directly support Time Machine, which is a big thing in this Mac-centric house.
The next decision was around which hard drives to put in the NAS. My HP EX495 has WD Green drives in it as I wanted it to be as quiet as possible and performance wasn’t a primary concern, though I often question that decision. Since performance is a concern and a large reason I got the more expensive DS1010+ I decided to go with faster drives. The hard part was finding drives with fast performance but were not also loud and annoying. Probably the best drive to meet those requirements right now are the 2TB Seagate Barracudas. They benchmark very well and are noted for being very quiet…they are also about $170/drive right now which is more than I wanted to spend since I was buying 5 of them. The 1TB WD Caviar Black drives also benchmark well in IOP and seek time tests. Unfortunately, they just released a relatively new revision of that drive and no one has really tested the acoustics. The previous generation drive was described as “noisy” by some but it occurred to me I actually had one of those drives already in my 27″ iMac that sits right in front of me all day and I almost never hear that drive. So I took a chance and ordered 5 of the current generation 1TB WD Caviar Black drives. This new model, WD1002FAEX, is a 7,200 RPM drive with 64MB of cache.
I’m happy to report that the new drives are very quiet. The Synology DS1010+ is not silent…you can hear its two fans running. While they aren’t loud or annoying I can hear them in my office. It sounds like when the AC in my house is running and I hear the air coming through the vent. Any other ambient noise overrides it. Initial power consumption tests are also very good. It looks like most of the time the DS1010+ populated with 5 of the WD Blacks uses 60w of power.
Finally, I have done some initial tests with the DS1010+ and my first vSphere host and setup was easy and works great. No problems at all. So, I don’t think you can go wrong with this storage setup for vSphere or for a good general purpose NAS that also supports other useful features such as Time Machine, Bit Torrent Client, Media streaming, iTunes streaming, and others. One other nice option is that ou can easily upgrade the unit from 1GB of RAM to 3GB. Right now my DS1010+ sits at about 25% memory utilization so it’s a very lean system but I do plan to upgrade it to 3GB as any unused RAM gets allocated to caching which will provide performance improvements across the board. The final cost of this configuration was $799 for the DS1010+ and $89/each for the drives for a total of $1,244. That’s about double what I expected to spend when I initially spec’d out an Iomega IX4-200D. Is it worth twice the price? That depends on your requirements. I expect to be using this NAS for a while due to its performance and capabilities.
Nice post Jason, and I look forward to reading the subsequent ones. My house is pretty much going the all-Mac approach as I’m sure I will be buying one for my wife who still sits in awe at the beauty of the 27″ iMac.
My job recently changed to one in which I will need to learn more about Citrix and if I decide to build a home lab to study both VMware and Citrix, this NAS is certified for both. I have a good friend that has a QNAP device and he swears buy it. How’s the build quality on this thing? The QNAP, TS-459 Pro, is a very nicely built device and runs pretty quiet.
-Geoff
The build quality is very good. The fans are maglev type fans with a very long life and low noise. I mentioned the noise in the post but it was just to say it’s not completely silent.
Between QNAP and Synology I don’t think you can go wrong…I just thought the online community support of Synology was bigger and I know a few Synology users. The management interface is great and one thing I didn’t mention is that Synology has some client-side apps for monitoring their device and they do have an OSX version. So the Mac support from Synology is excellent.
Love your posts Jason! I can’t wait to see the rest of the hardware you plan to build your home lab around as I am trying to convince my wife to let me do something similar.
Great post Jason.
I am curious if you looked at Drobo. I recall some of their devices are vSphere certified. I believe the price is comparable to what you paid for the Synology system.
Thanks also for doing the legwork on cost since my environment at home is not too dissimilar to yours (minus all the Macs) and I too wouldn’t mind getting better centralized storage and a sandbox for testing at the house.
You need a follow up post though:
“How to convince your spouse that you NEED a home vSphere lab”
Looking forward to hearing about the rest of your lab.
That’s a good question. I did look at Drobo previously but didn’t mention it. The Drobo is simple and easy, but the performance is very lacking. I have also seen numerous reports of data loss on Drobo systems so I didn’t feel comfortable using one. Also, the current Synology firmware supports a RAID system very similar to the Drobo so you can use different sized drives and add as you want. That’s an option during configuration. I’m doing straight RAID5 since I bought all 5 disks on day 1…but the option is there and I’m seeing really good reports from it.
I can weigh in on the Drobo as we have 6 of them deployed and in the past week we have had 3 of them replaced for various reasons. So I can attest to what you’re reading,
Disappointing to hear, but definitely glad to get the feedback about Drobo. I have no interest in putting production data there, but had considered Drobo for POC and testing. I’ll keep my options (and wallet) open probably.
There any good (free) options out there for some iSCSI practice. We primarily use FC and I have only assisted on a few iSCSI implementations. Definitely want a little hands on practice with iSCSI security and troubleshooting before working on one of the VCAP exams.
@Josh A Avoid any NAS that runs IET (iSCSI Enterprise Target) as that is not certified for VMware and has several issues (see VMW KBs)
@nashwj I went for similar RAID config except that I use QNAP TS-459 and TS-639 devices along with WDC Raid Edition disks. Indeed WDC doesn’t support any of its desktop drives such the green, blue or black series in RAID setup but Raid Edition (RE3/4) series only. Something to do with the TLER…
Another thing to note, there is a small overhead running iSCSI on such NAS’es. Remember that its just a (big) file sitting on top of a Linux EXT3/4 volume.
And another tip, if you plan to buy a SSD to speed up your storage, check that the OS on your NAS can TRIM it otherwie you’ll suffer from wear leveling and erase cycles quite fast. A tools like HDPARM can do that if you NAS can install third party tools. It’s the case for QNAP, not sure Synology can do that…
Here is a photo of my own lab… in a cellar
http://deinoscloud.files.wordpress.com/2010/11/homelabinacellar.png?w=500&h=888
Cheers,
Didier
Thanks Didier. I just commented on your 1810G post. That’s my next post (and my chosen switch) but having issues with trunking.
On the Synology (and I expect the QNAP) you don’t need to worry about TLER. TLER is a concern on hardware RAID controllers and the Synology is a software implementation. I confirmed with them that TLER wasn’t an issue. They hand’e bad sectors differently in software and do not rely on TLER.
The Synology let’s you do iSCSI two ways. One is that you can create a RAID set and just format it for iSCSI. So I could have taken all 5 drives and made a RAID 5 4+1 set and just presented that as one large iSCSI target. You can also do a file blob on the file system like most NAS…when I use iSCSI that’s what I’ll do because I didn’t want to dedicate physical disks to iSCSI.
I would assume you can use HDPARM on the Synology. It’s Linux using an EXT4 filesystem so HDPARM and wiper.sh should work fine.
Hi Jason,
As far as I know, my QNAP TS-459 runs an Intel ICH9R, the R stand for RAID. OK it’s ‘software’ RAID as opposed to pricy ‘hardware’ RAID, but it’s still a RAID controller, thus TLER is quite is an important issue in my case.
I don’t know what kind of hardware is it on your Synology…
Thx for your comment on my blog post regarding HP Procurve and port trunking… I hope you’ll fix your issue with the tunk port.
Cheers,
Didier
Good stuff….I know a couple people who really like Synology (that model even) for small departmental share usage/raw space (stuff they don’t want to put on their enterprise array just due to price points).
So far the Synology route has worked out quite well for them.
Thanks Jason! I need to buy a new hdd next month, and you just game me an ideea.
[...] 5, 2010 by nashwj We’ve covered the networking and storage components of my home vSphere lab build and now it’s time to talk servers. This is where [...]
[...] Part 1 – Storage [...]
Jason, Your posts about your lab build were very informative and helpful when selecting gear for my own lab recently. I’m wondering if you are happy with your choice of Synology? I purchased the new model DS1511+ and while it has worked great for NFS I have loads of trouble with iSCSI reliability. I’m wondering if you have had any similar troubles?
Thanks,
Jason
Thanks Jason! I love my DS1010+. My only regret is that it isn’t a DS1511.
I’m about to add a DX510 to it for media storage outside of my VMware lab and the option for a second add-on cabinet would be nice…oh well…
To be honest I’ve only done NFS with it. I know of a few others that have done iSCSI and I’ve heard some issues. Are you using the latest DSM 3.1? Maybe I need to throw an iSCSI LUN out there and test it.
Thanks for the reply. I am running the latest DSM 3.1. I too have no problems with NFS. With iSCSI I can create a VM and move data to and from the LUN but I cannot install an OS. I get a message after the disk format that setup failed to format the disk and the disk may be corrupt.
I don’t want to rope you into a tech support discussion on your blog but, if you ever find the time I would be very interested in your results with iSCSI. Our labs are nearly identical so it would be a good test. Thanks again for your response and for the record, apart from my little iSCSI issue, I too love my DS1511+!
Shoot me an email to jason@varrow.com and I’ll test this on my setup.
Are you aware of the HP Proliant Microserver? (I know this article is quite old now). It runs a dual-core 1.3Ghz AMD Athlon and comes with 1GB RAM and 250GB Hard Disk for £100 (£200 with £100 mail-in rebate).
I bought one, added 4x3TB hard disks (£400) and 8GB RAM (£80)- the only thing that leaves is the optional IPMI (out of band management add-in which would cost £60). I can’t hear the unit and it runs at roughly 40W. I’m currently running WHS2011 on it but haven’t yet decided whether i’m going to continue with it in the long term…
Thanks
I am familiar with them and looked at those. I think they are cheaper for you than they are here. I’ve actually heard of people buying them there and sending them back to the US. When I looked at the cost of getting one and adding a NIC (only had one NIC port) it wasn’t much different than building my own and getting what I thought was a better system. They’ve dropped in price a bit now though and I think they are a good buy.
[...] topic of my first post. Jason Nash of Varrow was kind enough to provide a pretty through hardware guide to building a home VMWare cluster about a year ago. Additionally one of my consulting clients [...]
[...] http://jasonnash.com/2010/11/17/vsphere-home-lab-part-1-storage/ - Jason Nash’s home lab [...]
[...] year I built my home vSphere lab and I documented it here on my blog in several parts. When I started to build the lab I thought I’d use it off and [...]