Building a decent size media storage server

Aychamo BanBan

Aychamo BanBan

<Banned>
6,338
7,144
Hi all,

I'm hoping for some help building a media storage server. I'm not the biggest tech guy any more, and I've basically lost track of what speed processors are needed to accomplish what these days. And I have no idea how to "size" a power supply to know if things will be adequately powered or not.

Requirements:
- 8-10+ Tb, via 4-5 drives
- RAID5
- Gigabit ethernet (dual with LAG would be great)
- Easily setup shares to OS X (ie once they are setup, to just work..)
- Rock solid, no hassles

Would be great if possible:
- Rack mounted, or at least a tower that is less than 19" tall so I could lay it on its side
- "hot swap" would be great, but not necessary

I'm guessing this would run FreeNAS or something. I think that's what it is. Whatever it is that uses xfs, because doesn't that help prevent bit rot? I tried pricing something similar out to this once, but the CPU on the all-in-one board was supposedly a bit slow so it made file transfers painful. Does anyone have a "go to" setup for something like this?

Thank you very much.
 
Void

Void

Experiencer
<Gold Donor>
9,441
11,118
You're a doctor and should be rolling in money, so I'd just suggest this:http://www.amazon.com/Synology-Perfo...d_sim_sbs_pc_2
or the 8-bay model for $200 more (that's the one I have, was on sale for under $1k at one point so look around). Yeah you can definitely do it for less yourself, but it depends on how much you value your time, because this thing takes no time at all.
 
Aychamo BanBan

Aychamo BanBan

<Banned>
6,338
7,144
Vvoid_sl said:
You're a doctor and should be rolling in money, so I'd just suggest this:http://www.amazon.com/Synology-Perfo...d_sim_sbs_pc_2
or the 8-bay model for $200 more (that's the one I have, was on sale for under $1k at one point so look around). Yeah you can definitely do it for less yourself, but it depends on how much you value your time, because this thing takes no time at all.
Click to expand...
Well, I was kind of interested in building one to get the xfs. Otherwise I'd probably go with a Promise pegasus RAID box or something.
 
Void

Void

Experiencer
<Gold Donor>
9,441
11,118
Well, that brings up some questions then. Are you planning on having the same drives in it for long enough to worry about bit rot (honestly I'd never even heard of that until you brought it up)? And is the data that important that you absolutely can't lose a single bit of it? Or is it for streaming movies and such? Because you'll probably buy new drives in a few years anyway as you need more space and the gb per dollar is exponentially better. Not to mention that spending the money on a dedicated battery backup for the raid server is probably the biggest "security" move you could make.

If the data is that important to you that you can't lose a single piece of it, you really need to be paying Carbonite or someone similar a few extra bucks a month to ensure its safety offsite.

Also, obviously you might simply have a preference, or want to just mess around and learn some new stuff, which is fine. I can't answer your original question then because I don't have the required knowledge, so hopefully someone else can.
 
A

aegdaien

86
0
I'm doing this for my Movies/TV Shows and plan on purchasingthis Synology ds412+to do the job.

I've looked at zfs/unraid solutions and the reality is it's going to cost you the same amount or more to get what the Synology has out of the box plus the giant hassle of setting it all up. You get more space using SHR raid rather than RAID5 if you have any missized harddrives (I'll be putting in 2x3TB, 2x2TB which results in 7TB usable vs 6TB on RAID5). The Synology GUI is really nice and integrates well with home security cams and other crap. See a demohere.

That said - I'm sure what I said sounds like I'm a Synology shill or something, but I've been doing research on this topic with basically your identical requirements for the last month straight now. As far as data integrity goes - I don't think SHR or RAID5 are designed to be 100% reliable in case of a HDD failure, so if the data is critical use a backup solution. My Movies and TV Shows aren't going to be backed up more than the 1 disk parity SHR provides, but my photos and some other documents are backed up to Carbonite on a regular basis.
 
Alkorin

Chancellor Alkorin

Part-Time Sith
<Granularity Engineer>
6,029
5,915
aegdaien_sl said:
As far as data integrity goes - I don't think SHR or RAID5 are designed to be 100% reliable in case of a HDD failure, so if the data is critical use a backup solution.
Click to expand...
Yes. A thousand times yes. Never rely on a RAID for backups. It's a patchwork solution at best and doesn't protect against a lot of things, like file system corruption, multiple drive failures, accidental file deletion, etc.
 
Joeboo

Joeboo

Molten Core Raider
8,157
140
Has anyone used the Carbonite back up service? I see their home plans can be anywhere from $59 - $149 per year, and they all say "unlimited GB" backup, but I wonder how true that is. Are they going to balk if I try to backup my 5-6 TB of videos? I notice the business services have size limits like 250GB or 500GB, so it makes me wonder if they are truly "unlimited". I'm always worried about my storage HDs failing and losing years worth of downloaded movies, TV shows, games, etc.
 
W

Wolfen_sl

shitlord
746
12
I think the Bitfenix Prodigy is an awesome case. Small, yet has room for 5 drives.
rrr_img_8008.jpg
 
Eomer

Eomer

Trakanon Raider
5,472
272
I just run a QNAP 439 Pro (whatever the exact model number is), and backup the most important stuff from that on to a 2tb drive in my desktop PC. I don't bother backing up movies and porn. I really recommend going with a pre-built NAS, they're pretty awesome.
 
A

aegdaien

86
0
joeboo_sl said:
Has anyone used the Carbonite back up service? I see their home plans can be anywhere from $59 - $149 per year, and they all say "unlimited GB" backup, but I wonder how true that is. Are they going to balk if I try to backup my 5-6 TB of videos? I notice the business services have size limits like 250GB or 500GB, so it makes me wonder if they are truly "unlimited". I'm always worried about my storage HDs failing and losing years worth of downloaded movies, TV shows, games, etc.
Click to expand...
Theoretically you can backup anything since the space is unlimited. However, in practice I've found Carbonite limits your upload speed to their servers at around 2Mbps, so trying to upload TBs of data isn't really feasible. I only have ~100GB of photos and other important things backed up there and the initial backup took weeks.
 
K

Kedwyn

Silver Squire
3,915
80
Eomer_sl said:
I just run a QNAP 439 Pro (whatever the exact model number is), and backup the most important stuff from that on to a 2tb drive in my desktop PC. I don't bother backing up movies and porn. I really recommend going with a pre-built NAS, they're pretty awesome.
Click to expand...
Pretty much my setup. Qnap is amazing.
 
Aychamo BanBan

Aychamo BanBan

<Banned>
6,338
7,144
Thank you all for the information. I may have misspoke about bitrot - it was something I had heard about and perhaps it's something I don't really need to worry about. This will be a media storage server. It's not the worlds most important data, but it'd be absolutely horrendous to replace. My plan is to use RAID5 on the server because there is (in my opinion) about a75% chance it can recover from a failed drive. But whatever I purchase, I was going to purchase 2 of them so that I can have two identical copies of the data. Which would give me two identical RAID5 arrays, each with a good chance of recovering from a failed drive. It doesn't protect me from an accidentally deleted file, but a single accidentally deleted file is easy for me to recover from. Two separate units would protect me from hardware failure, but not a catastrophic event. I've looked at CrashPlan, but it would take me probably a year to upload my data (2TB now), and I have a stupid 300GB monthly cap thanks to my ISP. I do have a UPS currently on my setup, and it works great because it has the USB thing to my server to shut it down. That's one reason I was kind of hoping for a DAS, so that the server could spin down the drives and allow them to gracefully shutdown when it tries to shut down in event of power loss.



aegdaien_sl said:
I'm doing this for my Movies/TV Shows and plan on purchasingthis Synology ds412+to do the job.

I've looked at zfs/unraid solutions and the reality is it's going to cost you the same amount or more to get what the Synology has out of the box plus the giant hassle of setting it all up. You get more space using SHR raid rather than RAID5 if you have any missized harddrives (I'll be putting in 2x3TB, 2x2TB which results in 7TB usable vs 6TB on RAID5). The Synology GUI is really nice and integrates well with home security cams and other crap. See a demohere.

That said - I'm sure what I said sounds like I'm a Synology shill or something, but I've been doing research on this topic with basically your identical requirements for the last month straight now. As far as data integrity goes - I don't think SHR or RAID5 are designed to be 100% reliable in case of a HDD failure, so if the data is critical use a backup solution. My Movies and TV Shows aren't going to be backed up more than the 1 disk parity SHR provides, but my photos and some other documents are backed up to Carbonite on a regular basis.
Click to expand...
Does the Synology work well with shares? I wouldn't use any of their 'iTunes server" stuff, or really any of it's built in software. I'd just want it to sit there as a dumb box with "Share1" available with it's 9 tb or whatever of storage allowing me to do whatever I want.
 
Alkorin

Chancellor Alkorin

Part-Time Sith
<Granularity Engineer>
6,029
5,915
aychamo_aycono_sl said:
My plan is to use RAID5 on the server because there is (in my opinion) about a75% chance it can recover from a failed drive. But whatever I purchase, I was going to purchase 2 of them so that I can have two identical copies of the data. Which would give me two identical RAID5 arrays, each with a good chance of recovering from a failed drive. It doesn't protect me from an accidentally deleted file, but a single accidentally deleted file is easy for me to recover from. Two separate units would protect me from hardware failure, but not a catastrophic event. [...] That's one reason I was kind of hoping for a DAS, so that the server could spin down the drives and allow them to gracefully shutdown when it tries to shut down in event of power loss.
Click to expand...
This is exactly what I'm doing. I bought a simple JBOD case (http://www.newegg.ca/Product/Product...82E16816111177) and I use software RAID 5 in Windows with 4x 3TB drives in the server, then another 4x 3TB drives in the DAS, and I rsync the content nightly (because rsync does differential backups and I have a bunch of large virtual machine files with only small daily changes). I turned on compression on the backup volume so that I can back up the OS there as well, and it does well enough that I can fill up the 9TB on the main RAID5 and still have a little more room on the backup RAID5 to store the OS. I was also looking at CrashPlan (I bought one of their 1 year deals on sale for like $6) but it would take the better part of 3 years to back up my data there, so... yeah.

So far, the solution is working perfectly. It's a good way to back everything up on the cheap, and you have full access to do whatever with the space as it's fully available to the server (as if the drives were plugged in inside the case). The only caveat with this setup is that your eSATA port needs to support multiple drives at once, which the majority will.
 
S

Silence_sl

shitlord
2,459
4
aychamo_aycono_sl said:
Thank you all for the information. I may have misspoke about bitrot - it was something I had heard about and perhaps it's something I don't really need to worry about. This will be a media storage server. It's not the worlds most important data, but it'd be absolutely horrendous to replace. My plan is to use RAID5 on the server because there is (in my opinion) about a75% chance it can recover from a failed drive. But whatever I purchase, I was going to purchase 2 of them so that I can have two identical copies of the data. Which would give me two identical RAID5 arrays, each with a good chance of recovering from a failed drive. It doesn't protect me from an accidentally deleted file, but a single accidentally deleted file is easy for me to recover from. Two separate units would protect me from hardware failure, but not a catastrophic event. I've looked at CrashPlan, but it would take me probably a year to upload my data (2TB now), and I have a stupid 300GB monthly cap thanks to my ISP. I do have a UPS currently on my setup, and it works great because it has the USB thing to my server to shut it down. That's one reason I was kind of hoping for a DAS, so that the server could spin down the drives and allow them to gracefully shutdown when it tries to shut down in event of power loss.





Does the Synology work well with shares? I wouldn't use any of their 'iTunes server" stuff, or really any of it's built in software. I'd just want it to sit there as a dumb box with "Share1" available with it's 9 tb or whatever of storage allowing me to do whatever I want.
Click to expand...
Any RAID beyond controller agnostic RAID 1, and you are just asking for all sorts of trouble. Hardware RAID is just asking for a bloody asshole, imo. I've looked at the Synology and QNAP units, and while nice, don't offer controller agnostic RAID.

I'm going to order a TS 430 and put 2TB reds in there...gonna retire the peerage model of data safety...which has worked really well so far, but I want a single point of admin and this fits.
 
spronk

spronk

FPS noob
22,642
25,727
i may be wrong, but doesn't Plex transcode files? ie translate mkv source into mpeg2 streams for older devices. If so, that is pretty terrible and you are losing video/audio quality. Something you should definitely see, maybe play plex shares with videolan or something that lets you see the exact stream quality and settings, and compare that to the source MKV file.

anyways re: OP, you can do all kinds of fancy shit but imo stick to the simplest. Wire your house with cat6/gigabit ethernet, so all the important places (living room, office, bedrooms, bathroom) have giga ethernet ports. Now setup/build your own PC so its the baddest motherfucker around, either stick in a ton of 3TB drives or attach a raid array to it.

Finally setup a boxee in your living room and bedroom, there were a TON of new android-based TV devices introduced at CES this year so will be a shitload of competition, don't spend too much money on the end part now since there will be better stuff. But for now, a boxee will play just fine all 1080p content over a wired network, without any need for extra software or shit.

Also don't get into the mindset that you are going to create an ultimate video library with a billion shows and videos, the reality is that too much content is overwhelming. I have a 12TB media array and I only share a few folders out of it, and constantly prune shit. My wife wants 10-15 choices to watch, not fucking 2000 like netflix. I delete 99% of the movies I see because they are pretty shitty, in the past 10 years maybe 2 dozen movies are worth rewatching.

Likewise, use the internet as your storage device. I like Rome, have seen it twice, but deleted it because fuck it, I can just redownload the torrent (and I own the bluray) if I feel the need to watch it again. There is no point whatsoever in hoarding every single bluray and TV show and movie you want, just watch and purge.
 
Joeboo

Joeboo

Molten Core Raider
8,157
140
Probably true, but but I do worry about the possibility that 5 years from now, easily torrenting anything and everything you want might be a hell of a lot more difficult than it is now. Hope not, but it's possible, so I'm getting as much as I can while I can get it
 
Aychamo BanBan

Aychamo BanBan

<Banned>
6,338
7,144
spronk_sl said:
i may be wrong, but doesn't Plex transcode files? ie translate mkv source into mpeg2 streams for older devices. If so, that is pretty terrible and you are losing video/audio quality. Something you should definitely see, maybe play plex shares with videolan or something that lets you see the exact stream quality and settings, and compare that to the source MKV file.

anyways re: OP, you can do all kinds of fancy shit but imo stick to the simplest. Wire your house with cat6/gigabit ethernet, so all the important places (living room, office, bedrooms, bathroom) have giga ethernet ports. Now setup/build your own PC so its the baddest motherfucker around, either stick in a ton of 3TB drives or attach a raid array to it.

Finally setup a boxee in your living room and bedroom, there were a TON of new android-based TV devices introduced at CES this year so will be a shitload of competition, don't spend too much money on the end part now since there will be better stuff. But for now, a boxee will play just fine all 1080p content over a wired network, without any need for extra software or shit.

Also don't get into the mindset that you are going to create an ultimate video library with a billion shows and videos, the reality is that too much content is overwhelming. I have a 12TB media array and I only share a few folders out of it, and constantly prune shit. My wife wants 10-15 choices to watch, not fucking 2000 like netflix. I delete 99% of the movies I see because they are pretty shitty, in the past 10 years maybe 2 dozen movies are worth rewatching.

Likewise, use the internet as your storage device. I like Rome, have seen it twice, but deleted it because fuck it, I can just redownload the torrent (and I own the bluray) if I feel the need to watch it again. There is no point whatsoever in hoarding every single bluray and TV show and movie you want, just watch and purge.
Click to expand...
Good advice! In fact I've already done basically this. I had Cat6 ran to the places I needed it, and I have a big rack mount media thing setup. The four white network cables run to Apple Airport Expresses that are hooked into amps that stream audio to different parts of the house. For streaming video I use Apple TVs at the destination. The server is a pretty beefy Mac mini and the storage (can't be seen in pic) is all direct attached. I have 3 x 3TB drives, and basically they are all clones. (3 copies of data.) I'm liking the Synology idea. Seems pretty straightforward and painless. The new Thunderbolt drobo seems like a nice idea too, but it's about $300 more expensive per unit.
 
You must log in or register to reply here.