TB Server

[JohnL]

I saw this little thing http://rack.modzone.dk/ and it inspired me.  I know that I would never have a use for 4.5TB of storage like he put into this homebrew rack, but... maybe I could use 1TB... 
I'm thinking I cool map drives on my pvr and have it record directly to a big old file server... say with around 1TB of storage.  I want to build the server with a Mini-ITX motherboard from VIA... low power, Small size.  I was thinking of putting together a custom case and making it be around .5'x1'x1' (lwh) and putting it together with 2x3's or metal rods of some sort.  Will be running windows server 2003 so that I can span drives together with Dynamic Discs without needing to have raid (too expensive....)

Wanted to know if anyone had any ideas or suggestions they wanted to throw at me

[RedR]

Heya,

Well one thing I read recently was a great implementation I thought. The main PVR handles the video input and output, while your main PC handles storage, or in an idea like this a dedicated file server handles the storage. Thus allowing for a much louder file server while the HTPC/PVR can be dead silent. After all we all know storage itself can be loud, let alone proper cooling add's to that noise. I would highly suggest going with gigabit connections though. Now'a days its not that much more money for the proper equipment to fully support gigabit.

Enjoy,
RedR

[JohnL]

Thanks for the reply...

I would have thought that a full 100mbit connection would be sufficient, just because theres only going to be 4 computers on the network at all (pvr, mine, girlfriends, fileserver)...  Maybe I am wrong, although I do think it might be kind of a pain to do    Also, with my Media MVP setup, streaming mpg2 from the pvr only sucks down a max of 8mbps on the network.  So I think that 100mbit would be ok (I am open to correction though)
I would need to get an add in nic card for the pvr (and I want as many slots open as possible for tuners) a new switch (which I already have one that Im not using) and make sure to get a gigabit card for the server.  I am starting to think that maybe using the epia motherboard from VIA might be a little small for the project at hand.

Another thing that I thought would be mighty awesome was to have my PVR pc automatically queue Mpeg4 transcodings on the fileserver.  Like if I put together a Sempron 2500+ system, or even a XP 2500+, and had it doing the transcoding seperate from the pvr.  I have also thought of getting a RAID card to do Raid0 to speed things up and take some of the disk read/writes off of the CPU.  (Any suggestions on good hardware raid cards?)
I wonder if GB-PVR could be setup (or a plugin written?) to do this kind of thing.  I think that would be awesome... save space, on the file server that doesnt need space saved 

[planetjay]

What I'm doing right now is building a linux box with a 1TB of usable space. Here's what I know so far: http://www.tldp.org/HOWTO/Software-RAID-HOWTO.html. 

History: I did a software RAID on my Mac. 4 - 250GB drives mirrored into 2 - 250GB volumes. It sucks. I installed them in a FireWire box that I got off eBay ( Superldg store Firewire hd enclosures ) and it seemed nice for a while. Then one day (I think a cable got knocked loose) only 1/2 of the drives came up and I got an error. The mirrors had split. Then I find out they only way to get them back is to copy off the data and build a new mirror set. No way to make it rebuild. Google it and all you'll find is people bitching about it. But I still have my data so it could have been worse.

So a few weeks ago I decided to reconcider. The end result is that I can do RAID5 in linux and get n-1 capacity, rebuild on the fly and even automagically^TM, and gain speed too. I had 2 ATA-100 PCI cards but needed 3 to pull this off the way I wanted. Also I needed 2 more drives. I picked this up on eBay http://cgi.ebay.com/ws/eBayISAPI.dll?ViewItem&rd=1&item=5217982882&ssPageName=STRK:MEWA:IT on Sunday but the guy only ships on Saturday (bastard) So maybe next weekend he'll bother to ship and then in another week I'll get them. But the price was right. Also yesterday I bought 2 more 250GB drives at Sam's Club. Total now is 6.

The plan is 5 drives which will give me (n-1) or (5 - 1 * 250Gb = 1TB). Also yes I know that it's not a true Terabyte, but a drive company's cheated 1000 Byte vs 1024 Byte, Terabyte. But who cares? I also plan the 6th drive to be a hot spare that will swap in and replace a failed drive. The reason for for all the controllers is that every ATA drive MUST BE A MASTER. NO SLAVES! Otherwise you'll only be able to access one drive at a time per controller. Also if you lost a master you'd loose it's slave too.

I also plan to pickup a Gigabit NIC for that server. Both my PowerMac G4s have builtin Gigabit and I have a Gigabit switch already.

I'm not using hardware raid, BTW, for money reasons. Also RedR mentioned cooling... I don't think that's an issue for me. http://gallery.planetjay.com/CaseMod/6_G  The trick is to add too many fans and then lower the voltage on the fas and slow them down to a quiet level. I'm also working on some other ideas for quieting as well. More on that later.

And for the curious... I'm using an old (2000) MB with a P3-666. I think I have about 256 MB of RAM. a 4 GB drive for boot and another 4 GB drive for swap. I used the whole drive for swap only to keep anything else from using it and slowing it down. Again both drives are masters. I'm using WhiteBox Linux http://www.whiteboxlinux.org/, made down here in South Louisiana. After everything is done I'll remove the monitor, keyboard, and mouse and have only power and ethernet connected. I may hide it in another room out of the way too.

[planetjay]

And since Rampy asked... the Firewire box is here http://cgi.ebay.com/ws/eBayISAPI.dll?ViewItem&item=5221799506&category=167&rd=1.

[JohnL]

Wont running a software Raid-5 Array kill your cpu?  I've read that running Raid-5 in software will bog down the entire system because the cpu has to create all the parity information as things are being written and read.  Thats why I wanted to do hardware raid (Which I might chunk out the window and just use drive spanning in windows 2k3 server)

I was thinking of doing drive spanning instead of raid because then I could use different sized hard drives (I'm building it a piece at a time, not all at once... I dont get paid enough ) and I could just tack them onto the end of the ntfs partition as I went on.  I know that Linux can do this using LVM, but the last time I tried to use LVM, things werent good.  I also wanted to use windows so that GB-PVR can communicate without a problem to some sort of client program to do transcoding at some given hour every night so that programs will take less space, and gb-pvr will be updated on the file names (dont know if this part is possible... worth a try though right?)

I like that little fan setup youve got there, very nice.  I've got a couple of 120mm server rack fans that a guy from work didnt need any more, and I think he has more, so I'll have a butt load of aluminum 120mm server rack fans blowing over the hard drives and such

Frankly I cant wait to get started...  I need to brain storm an idea for a case... maybe buy/find a cheap old case and modify it?  Put two together?...  Ideas!  Ideas!

BTW: Props to our new mention in the paper!!

[planetjay]

You'll notice that on the RAID HowTo the guy was using a Dual P Pro 150. But I have no CPU concerns as I'm only going to use the box as a file server and maybe for a few other near 0 CPU uses. Anyway it's LINUX. It's designed to run on 386s.

For a Windows 2003 Server... I'd just go ahead and delete the data and save the money and time. 

[JohnL]

Lol...  Well, I'd like to give win2k3 a shot, if that doesnt work well, Ive got Fedora3 to fall back on.  Well, the results are in, I went down to my neightborhood Fry's Electronics last night and picked up on a killer deal.  I got a Sempron 2800+ (which was on sale for $99) and some PC Chips motherboard for $74.99.  I figure even if the mobo is crap crap crap, I still got the cpu for $75.  So I'm going to put those to use shortly.  Also picked up a $35 case that I think I am going to chop up...  I'm thinking of buying another one of the $35 cases and putting them together (dont know how yet) and using the second one for hard drives...

I saw a "TeraStation" (http://www.buffalotech.com/products/product-detail.php?productid=97&categoryid=19) for $999.99...  and thought to myself "I'll probably spend less than half of that, and have one TB, and still be able to add to it"

Has anyone here ever played with LVM in linux?  If I could get that working, then I would probably have no reason to run a win2k3 box (although I still want to try and see it in action)

[turajb]

JohnL & planetjay,

Personally I would hate to spend all that money on M$ Windows, just for a drive spanning solution for the OS.  There are more effective, less system demanding, open source solutions available to accomplish that task.

In case this helps, I use LVM on all my newer setup Linux boxes.  All of my servers and more demanding PCs run CentOS 4, which is a RedHat Enterpise Linux clone.

In CentOS 4, LVM is setup to be used by default & will automatically span your multiple hard drives when you first attempt to define the partitions on a fresh installation.  This will make it easy for you to setup/span your multiple hard drives and have it appear to your OS as a singular drive.

There is also a great GUI tool in CentOS which allows for easy LVM management.  It's aptly named "Logical Volume Management" & would be found under the 'System Settings' menu when using Gnome as the desktop.  This may be worth a look, if you are looking at LVM & need an easy way to manage it.  This tool is also available in Fedora Core 3 & 4.

As how to be able to use a Linux file server with your Windows PVR software (which doesn't support remote file storage/retrieval), there is a very easy solution.  Setup Samba on your Linux file server (for example, named "fileserver"), and have it share a folder from your LVM drive (for example, called "pvrdata") as a network share (for example,  named "pvrshare").  Then within all of your Windows PCs, you can simply map an open drive letter (for example, drive letter "z") to that network share ("\\fileserver\pvrshare") within your Network Neighborhood.

To your applications (for example "GB-PVR"), the mapped drive letter ("z") would appear as simply another hard drive letter in Windows.  You can then use the settings/options within your PVR software to select the mapped drive ("z"), instead of a locally connected hard drive (for example, drive letter "c") on that PC.  As such, when the PVR software records the shows to the mapped drive ("z"), the data is sent through the network to the file server and stored in the shared folder ("pvrdata").  When the PVR software wants to play back a recorded show, it request the file from the mapped drive letter ("z") and the data is streamed from your file server to the PC requesting it.

I can personally state that running Samba, with a gigabit network connection between several Windows PCs & a Linux file server, can produce a pretty smooth running remote network storage solution.  What makes it simple is that all of the Windows PCs can remotely access the content by simply calling the files from their mapped drive.

Attempting this solution would save you from having to spend a lot of time setting up or writing a plug-in to interface your software (PVR or otherwise) to the file server which holds the content.  This type of setup has worked well for me in my office, as well as at home.  And above all else, it is very easy to replicate over and over.

Should you want to try this approach, I can produce a more detailed write-up on how to set this up, and include a sample samba config file which will take a lot of time out of getting Samba to share a Linux drive/folder across a Windows network.  Let me know if this this will be of help...

[planetjay]

For what you want to do, I agree with turajb. 

And in other news... I ripped 2 drives out of the Firewire box and installed them in the Server along with 2 new ones. I still lack the 3rd contoller to complete it but wanted to run some tests. Then I realized I don't have enough power connections and I'm out of Ys. The Windows box may loose the SilverStone Power Supply.

I took a 5 drive cage out of an old case and added it to this one. I starting to have concern. 4 drives weigh a freakin' ton! And I'll be adding another. I'm also going to have no place for a 6th drive. No hot spare. 

A new case may be around the corner. 

[planetjay]

Some new pics.

[JohnL]

Well, I work for and attend a state university right now.  I'm also working in the lan management department for the libraries and MS has a licensing agreement with the school.  That translates to $7 a CD, for just about anything I would want (windows, office, vb, etc....)
I'm going to look into centos, I like the idea of linux better because its linux...  Also, I know how to do the samba and mapping thing, the reason I wanted to use windows was to try and write up some sort of mpg4 transcoding thing (well, a scheduler only really) so that the pvr box can queue up a transcode and the server will do it without a performance impact on the pvr.  so if I am recording aqua teen, it can finish and transcode while robot chicken is being recorded.  See what I mean?

I'll have some pictures of the cheap ass thing up soon...  its amazing how crappy cases get in the $30 range (psu included...  will be buying a new one shortly)

[planetjay]

$7 a disc? That is almost worth it. Except for the viruses and spyware.

Here's some new tips for you.

1. Webmin is the best way to setup your raid in Linux. Does LVM too.

2. Never. Never. Never. Never. Never. NEVER run your RAID box with the cover open for hours. I almost burned my hand. REALLY. I may have to rethink the fan arrangement and add temp monitoring. At 1% built I shut it down.

3. Lather. Rinse. Repeat.

[JohnL]

I would love to use Raid, I just dont know if it will have the expandability I need (hardware raid anyways) because a 4 channel IDE raid card can only support 4 discs (I'd like expandability to .... would there be a way to raid two raids?  lol...  or something along those lines, I would love to have the speed of raid, but dont want to have to tear completely down and rebuild everytime I add a drive (right now I only have one 250gb disk in it)

[planetjay]

Actually you can RAID your RAIDs. That's how you get 1+0, 0+1, and all the other multi number variations.