It’s been 2 months since I have written a blog post that wasn’t about the Tohoku earthquake and tsunami or the Fukushima 1 nuclear disaster, but today I am taking a break from those subjects. The reason is that I replaced my local Ubuntu server with newer hardware. The primary requirements were:
- GNU/Linux (Ubuntu)
- Reasonably low power usage
- Large and very reliable storage
I was considering boards ranging from the new AMD Zacate E-350 dual core to LGA-1155 (“Sandy Bridge”) boards with the Core i5 2500K. First Intel’s P67/H67 chip set problems and then the disaster in Japan prompted me to postpone the purchase.
Finally I picked the GigaByte GA-H67MA-UD2H-B3, a MicroATX board with 4 SIMM slots in conjunction with the Core i3 2100T, a 35W TDP part with dual cores and 4 threads. The boxed version of the Intel chip comes with a basic fan that didn’t sound too noisy to me. I installed two 4 GB DDR3 modules for a total of 8 GB of RAM, with two slots still available. When you install two memory modules on this board you should install them in memory slots of the same colour (either the blue or the white pair) to get the benefit of dual channel.
I chose a H67 board because of the lower power usage of the on-chip video and the 2100T has the lowest TDP of any Core 2000 chip. I don’t play games and my video needs are like for basic office PCs. Unlike P67 boards, H67 boards can not be overclocked. If you’re a gamer and care more about ultimate performance than power usage you would probably go for a P67 or Q67 board with an i5 2500K or i7 2600K with a discrete video card.
To minimize power use at the wall socket I picked an 80 Plus power supply (PSU), the Antec EarthWatts EA-380D Green. It meets the 80 Plus Bronze standard, which means it converts AC to DC with at least 82% efficiency at 20% load, at least 85% load at 50% load and at least 82% at full load. It’s the lowest capacity 80Plus PSU I could find here. 20% load for a 380W PSU is 76W. Since the standard does not specify the efficiency achieved below 20% of rated output and typically efficiency drops at the lower end, it doesn’t pay to pick an over-sized PSU.
Disk storage is provided by four Hitachi Deskstar 5K3000 drives of 2 TB each (HDS5C3020ALA632). These are SATA 6 Gbps drives, though that was not really a criterium (the 3 Gbps interface is still fast enough for any magnetic disks). I just happened to find them cheaper than the Samsung HD204UI that I was also considering and the drive had good reports from people who had used them for RAID5. The 2TB Deskstar is supposed to draw a little over 4W per drive at idle. I don’t use 7200 rpm drives in my office much because of heat, noise and power usage. Both types that I had considered have three platters of 667 GB each instead of 4 platters of 500 GB in older 2 TB drives: Fewer platters means less electricity and less heat. A three platter 2 TB drive should draw no more power than a 1.5 TB (3×500 TB) drive.
There are “enterprise class” drives designed specifically for RAID, but they cost two to three times more than desktop drives — so much for the “I” in RAID that is supposed to stand for “inexpensive”. These drives support a special error handling mode known as CCTL or TLER which some hardware RAID controllers and Windows require, but apparently the Linux software RAID driver copes fine with cheap desktop drives. The expensive drives also have better seek mechanisms to deal with vibration problems, but at least some of those vibration problems are worse with 7200 rpm drives than the 5400 rpm drives that I tend to buy.
The case I picked was the CoolerMaster Centurion 5 II, which as you can see above is pretty large for a MicroATX board like the GA-H67MA-UD2H-B3, but I wanted enough space for at least 4 hard disks without crowding them in. Most cases that take only MicroATX boards and not full size ATX tend to have less space for internal hard disks or squeeze them in too tightly for good airflow. This case comes with two 12 cm fans and space to install three more 12 or 14 cm fans, not that I would need them. One of these fans blows cool air across the hard disks, which should minimize thermal problems even if you work those disks hard.
One slight complication was that the hard disks in the internal 3 1/4″ slots needed to be installed the opposite way most people expect: You have to take off both covers of the case, then connect power and SATA cables from the rear end (view to bottom of the motherboard) after sliding the drives in from the front side (view to top of motherboard). Once you do that you don’t even need L-shaped SATA cables. I could use the 4 SATA 6 Gbps cables that came with the GigaByte board. Most people expect to be able to install the hard disks just opening the front cover of the case and then run into trouble. It’s not a big deal once you figure it out, but quite irritating until then.
I installed Ubuntu 11.4, which has just been released, using the AMD64 alternate CD using a USB DVD drive. I configured the space for the /boot file system as a RAID1 with 4 drives and the / file system as a RAID6 with 4 drives with most of the space. Initially I had problems installing Grub as a boot loader after the manual partitioning, but the reason was that I needed to create a “bios_grub” partition on every drive before creating my boot and data RAID partitions.
RAID6 is like RAID5 but with two sets of parity data. Where the smallest RAID5 consists of three drives, a minimal RAID6 has four, with both providing two drives’ worth of net storage space. A degraded RAID6 (i.e. with one dead drive) effectively becomes a RAID5. That avoids nasty surprises that can happen with RAID5 when one of the other disks goes bad during a rebuild of a failed drive. If you order a spare when you purchase a RAID5 set and plan to keep the drive in a drawer until one of the others fails, you might as well go for a RAID6 to start with and gain the extra safety margin from day 1.
I had problems getting the on-board network port to work, so I first used a USB 2.0 network adapter and later installed an Intel Gigabit CT Desktop Adapter (EXPI9301CT). With two network interfaces you can use any Linux machine as a broadband router, there are various pre-configured packets for that.
While the RAID6 array was still syncing (writing checksums computed from data on two drives to two other drives) and therefore keeping all disks and partly the CPU busy the machine was drawing about 58W at the wall socket, as measured by my WattChecker Plus. Later, when the RAID had finished rebuilding and the server was just handling my spam feed traffic, power usage dropped to 52W at the wall socket. That’s about 450 kWh per year.
The total cost for the server with Core i3 2100T, 8 GB DDR3 RAM (1333), H67 MicroATX board, PCIe Ethernet card, 4 x 2 TB SATA drives, case and 380W PSU was just under 80,000 yen including tax, under US$1,000.
So why Ubuntu AMD64 CD when you have an intel i3 processor? Just wondering. Not a big hardware guy but I was eyeballing this same motherboard with an i7 processor.
I might have considered the i7 if I was a gamer or other wise needed a lot of CPU power, but for me anything faster than a cheap 35W TDP chip would have been a waste.
In case you’re confused by the “AMD64” label of the CD image, it’s not just for AMD’s 64 bit CPUs but any chip compatible with either AMD64 or EM64T (which is basically Intel’s name for AMD’s 64bit extensions), which includes all Core 2 / i3 / i5 / i7 and even some Atom CPUs.
If you have any 64bit capable Intel CPU and plan to use 4 GB or more of RAM, you should consider the 64bit version. If you’re only ever going to use 3 GB or less the 32 bit version would be fine too. I use 8 GB, hence my choice of the 64bit version.
Nice article, thanks!
I assembled pretty much the same configuration as yours, with a Gigabyte H67A-USB3-B3, and a core i3 LGA1155, but I’m currently struggling with the Ubuntu Server 11.04 (amd64) installer: the boot menu appears after booting from the USB key, but nothing happens when selecting an item.
Before trying all kind of things to install Ubuntu, I was just wondering if you had to do any particular thing to make Ubuntu 11.04 boot and install on this hardware?
No, I don’t recall doing anything unsual to install 11.4 (amd64), but I was doing it off optical media, not a USB key. I don’t think that should make a big difference though. Do you have an optical drive that you could attach temporarily?
I’m extremely happy with the machine. It has been rock solid while running 24/7. I recently equipped it with battery backup using an APC Smart-UPS 750 with apcupsd to shut it down before battery runs out in an extended power outage.
I read your interesting article today. I also choose the GA-H67 board.
I first configured it within an aluminum chassis I had on hand, but I am unhappy with the support along the rear headder (interfers with cable inputs)…and rather noise…from al top panel.
So plan to migrate to a different case.
I am configuring it for a more home office and solid backup for files via raid.
Your article and pictures gave me some new ideas on configuration as I migrate to the new build. Thanks for the article.