NAS

[ÈlTwo]

I am using my NAS as a DLNA server, really for music.
If I'm going to watch a movie, I don't mind getting up to get the disc, but for music I have too much to start swapping discs, and playlists are great.
It's easy to access my music on my main system, on any computer in the house, on the laptop, on the tablet, on my son's iPod, and on our phones.

I like playing with computers, so it's enjoyable for me to mess around with these things. I actually picked up the QNAP and the LG NAS boxes on ebay; the LG is an okay plastic box, the QNAP is built like a tank.

[copperpipe]

Apr 8, 2014 11:57:14 GMT -5 KeithL said:

Just to shock everyone, I'm going to make a case for simplicity here...... 

Think about how you actually use or play your music and what you need.

A NAS is cool; it lets you access your music from multiple locations, holds a huge amount of music, and, if properly configured, can even back itself up (and repair itself if something goes wrong). HOWEVER, setting up and administering a NAS can get complicated. Do you buy it already loaded with drives? You can probably save money by buying the NAS "box" and adding your own drives, but then you're going to have to do some research to find out which drives work well in your unit. Also, remember that a NAS unit itself is a computer - which means that it can have all the problems that happen to computers. So, if you happen to like computers, or have a 14 year old nephew who likes to show off, then you're fine. Otherwise, it may always be smooth sailing, or you may be in for rough seas and a few headaches. (Also don't forget that fancy RAID arrays WON'T protect you if someone steals your drive, or if the house burns down and takes the drive with it, of if you get a solid lightning strike that fries it. In fact, a RAID array won't even protect you if your player/computer starts corrupting files while its playing them; or if you hit "delete" by mistake - RAID ONLY protects you from drive failures. To protect against that other stuff, you'll need to make backups.)

So what's the alternative?

Well, a 3 tB desktop USB drive currently goes for about $120 at Walmart (or your favorite discount store). A 3 tB drive can hold about SIX OR EIGHT THOUSAND albums as CD-quality FLACs (or half that many uncompressed WAV files, or close to a thousand 24/192 versions). A simple drive doesn't have all the fancy RAID redundancy and automatic backup that you'd get with a NAS, but you also don't have all that stuff to worry about and set up. To make a backup, simply buy ANOTHER 3 tB drive and, every so often, copy your entire music drive to it. (It might take overnight, but so what?)

Now, if you really need to be able to play your music in multiple locations, or if you have an amazingly large album collection, or if we're including videos in the discussion, then a NAS may be for you. Or, if you really want an included DLNA server, then that's another thing most NAS's include. BUT, if you really just need a drive to store music on, and to have sitting next to a computer so you can play that music, then a simple USB drive may be all you need - and a lot simpler (and cheaper) than the alternatives. You also avoid "network issues" and all that fun stuff.

Incidentally, whether you do a NAS or not, do remember to BACK UP YOUR MUSIC. Even the fanciest RAID array only protects you from drive failures; it won't stop you from deleting a file by mistake, and won't tell you if one of your files gets corrupted by a program (like your music player). Your music library, AND THE EFFORT THAT WENT INTO ORGANIZING IT, is a HUGE investment. Backups are cheap; loss is painful; do the right thing!

Here's another major suggestion: invest in a program that does checksums or file verification.....

If you have a backup copy of your music library, the most obvious way to make sure that it remains 100% intact is to periodically compare the "main copy" to the backup. However, there are THREE problems with doing this:

1) Comparing that many files takes a long time and can be somewhat complicated (depending on how you did the backup).

2) If your backup is slightly outdated, it gets VERY complicated to tell what's damaged and what's just different.

3) Even if your backup is current, it can still be a problem to tell which copy is "the good one" if your main copy and backup disagree on a few files. (It's also possible that the file got corrupted a long time ago, and you just haven't played it since then, in which case the backup could also be bad... and, if you keep multiple backups, it's going to be tricky to tell which one is the good one.)

In contrast, you can run a checksum of all the files on the drive. The program calculates a number for each file - using a special type of algorithm - and stores the numbers for all the files in a list. (That special number, properly called a CRC, is calculated such that, even though it is only a single number, and so doesn't take up much storage space, it is VERY unlikely that you could change or damage the file without also changing its checksum.) Then, when you want to check your files, it does the same thing again - only, this time, it compares the new checksums to the stored list and confirms that they match. Doing this is a lot faster than comparing every file every time, and the program will give you a list of every file that has changed since the original checksum list was stored.

Here's the program I use (it costs $15 for a personal license)

www.kvipu.com/CDCheck/download.php

Keith, you would love ZFS It does protect against deletions (you can make drive snapshots), it protects against corruption (it maintains a SHA-256 hash) etc, it offers compression, it is far more advanced than RAID etc. For example, you can increase the size of your pool (total available disk space) by replacing one disk at a time with a new disk that is twice the size, then letting the pool rebuild. Do the same thing for the next drive until all drives are replaced at which point you now have a much bigger pool. All this while the pool is still online and available. I've done this once already and will probably do it at several more times until it's time to upgrade the OS itself.
en.wikipedia.org/wiki/ZFS

I'm running it on a straight FreeBSD 9 install; my NAS is not just my music server, it is also my video server, my email server (getmail + dovecot + roundcube), my database server (postgresql + mysql), and it runs VirtualBox to host my windows xp + windows 7 testing machines.

I do agree that ZFS still does not protect against lightning or floods etc; but I backup all my important stuff to a remote machine, my media files don't get backed up because it's just entertainment.

[hemster]

^I'd have to do some research. Is ZFS anything like UFS (Unix File System)?

[yves]

Apr 8, 2014 7:39:45 GMT -5 copperpipe said:

Apr 8, 2014 3:59:24 GMT -5 yves said:

I own 14 external harddrives and a pair of USB hubs. In total, they give me 28 TB of external harddrive storage space, and, if I had chosen to go for a typical NAS solution, it would have more than doubled the total cost, yet it wouldn't have added anything that can be useful to me. [ I known I don't mention this very often, but I have a formal degree in IT ].

I sure hope you are using some sort of redundancy (raid or better yet zfs) with those 14 drives (or alternatively you don't care much what's on them).

RAID or ZFS is not needed. I unplug my external "backup drives" completely, i.e. from both the computer and the wall, and, as a result from the fact I do not use RAID, during playback of my media files, only the drive that contains the file that is being played back has to be kept spinning, i.e. both the longevity of all of the drives and the overall power cosumption can be optimized.

en.wikipedia.org/wiki/Massive_array_of_idle_disks

[hemster]

Apr 8, 2014 17:01:35 GMT -5 yves said:

Apr 8, 2014 7:39:45 GMT -5 copperpipe said:

I sure hope you are using some sort of redundancy (raid or better yet zfs) with those 14 drives (or alternatively you don't care much what's on them).

RAID or ZFS is not needed. I unplug my external "backup drives" completely, i.e. from both the computer and the wall, and, as a result from the fact I do not use RAID, during playback of my media files, only the drive that contains the file that is being played back has to be kept spinning, i.e. both the longevity of all of the drives and the overall power cosumption can be optimized.

en.wikipedia.org/wiki/Massive_array_of_idle_disks

^That's exactly what I do. My backup drives are purely offline backups. emotivalounge.proboards.com/post/630577

[igor]

Hi,

I also needed a solution for this, but also for streaming hd video, and did some research a few months ago.

First, as others say, RAID is not a backup solution, so why get it...speed? What speed? Streaming flac can be done on a 5y old cellphone let alone any modern harddrive. Reduce downtime? Cmon, a hd breakdown every second year tops and a few minutes of downtime worst case is NOT worth investing time and money in (while swapping the hd).

So skip that bull, sounds nice but it's only good for uptime and/or speed which is not a problem in this case.

Then we have the network-attached part. In my case I also needed a decent router for fast wifi (if wifi is not needed for you, there probably are slightly cheaper solutions), and while many routers and network storage gadgets boast different speeds, to really reliably be able to saturate a 100mbit network through wifi I'd say anything that doesn't use the "ac" wifi standard won't do it (unless maybe in the same room), but the client machine would need an "ac" capable wifi as well (there are usb-sticks...). There are plenty of routers with usb to attach drives of different sorts, but not many have the processing horsepower to get good throughput and also usb3. But what I found was the then rather new ASUS 68u router that has this, and I can reliably get 30mb/s (not mbit, I get about 240 mbit/s) through wifi, a couple of rooms away, which is more than some "real" NAS:es provide. I have connected a cheap (~$50) five-hd-slot usb3-hardrive box to it's usb3 port (which supports raid but I don't use it, but it also sleeps the disks after 15 min of no use), but so far I've "only" populated it with 3x3TB WD Red series harddrives.

Works like a charm both for streaming HD music and supplying my mobile devices with great bandwidth and full speed access to my 100mbit internet connection.

As for backup: For really important stuff, buy a large but slow (to reduce price) HD and store offsite except when backing up (othwerwise a fire for example would wipe both the data and backups), or for more convenience, for example google has lowered the price for online storage substantially if not that much data ($9.99/TB per month). For the rest I'm lazy and simply map the network drives to my windows machine (like Z:\ so it looks like a local hd to my computer) and run a simple one-line "robocopy" script between them now and then (but it could easily be scheduled or even probably run on the router not needing a computer to run the backups, if logging to the router with ssg and using similar tools in the routers under-the-hood linux OS to set up the script and schedule it), using one of the drives as "master" and the other one as backup:

robocopy Y:\Music Z:\Audio\Music /MIR /FFT /LOG:MusicBackup.log /TEE

(The script simply copies anything changed in "Y:\Music" to "Z:\Audio\Music" and removes files removed from y, and creates a logfile)

Cheap (relative to a NAS of similar performance), great wifi speeds all over, safe, and a supberb router for other purposes if you need those (guest networks, lots of firewall/portforwarding and similar functionality etc), really works well for me.

Good luck!

[Jim]

Apr 8, 2014 17:09:07 GMT -5 hemster said:

Apr 8, 2014 17:01:35 GMT -5 yves said:

RAID or ZFS is not needed. I unplug my external "backup drives" completely, i.e. from both the computer and the wall, and, as a result from the fact I do not use RAID, during playback of my media files, only the drive that contains the file that is being played back has to be kept spinning, i.e. both the longevity of all of the drives and the overall power cosumption can be optimized.

en.wikipedia.org/wiki/Massive_array_of_idle_disks

^That's exactly what I do. My backup drives are purely offline backups. emotivalounge.proboards.com/post/630577

Online storage and raid and everything is all great and dandy until the file system gets hosed. That can kill everything - regardless of disk redundancy. I was in the "will never happen to me" until i accidentally screwed up a raid 5 partition.

And if you have two locations that mirror each other... You delete something, guess what? Gone from both places.

Offline backups are the way to go if you want real safety.

[hemster]

Apr 8, 2014 18:45:40 GMT -5 Jim said:

Apr 8, 2014 17:09:07 GMT -5 hemster said:

^That's exactly what I do. My backup drives are purely offline backups. emotivalounge.proboards.com/post/630577

Online storage and raid and everything is all great and dandy until the file system gets hosed. That can kill everything - regardless of disk redundancy. I was in the "will never happen to me" until i accidentally screwed up a raid 5 partition.

And if you have two locations that mirror each other... You delete something, guess what? Gone from both places.

Offline backups are the way to go if you want real safety.

AMEN!

[klinemj]

I also subscribe to the simple approach... My NAS is not raided, but I back it up to an independent device even as I burn a new CD.

I only have mine networked (instead of a simple USB drive)so I can access the data from my central system (via SONOS) or from my 2-channel system (via PC or SONOS) for different needs. That I can also access my WD NAS via the cloud is a bonus.

Mark

[Deleted]

Ever look at iTunes Match? The simplest solution of all - offline backup for security and streaming to devices that support Airplay. It's cheap at around $20/yr. Only drawback is that you're limited and confined to the Apple universe somewhat. I use a combination of iTunes Match and NAS for local backup.

[yves]

Apr 8, 2014 17:09:07 GMT -5 hemster said:

Apr 8, 2014 17:01:35 GMT -5 yves said:

RAID or ZFS is not needed. I unplug my external "backup drives" completely, i.e. from both the computer and the wall, and, as a result from the fact I do not use RAID, during playback of my media files, only the drive that contains the file that is being played back has to be kept spinning, i.e. both the longevity of all of the drives and the overall power cosumption can be optimized.

en.wikipedia.org/wiki/Massive_array_of_idle_disks

^That's exactly what I do. My backup drives are purely offline backups. emotivalounge.proboards.com/post/630577

Yes, it's the most cost effective strategy for home users AFAIK. However, you should always remember to unplug (B) *before* plugging in (C), and unplug (C) *before* plugging in (B) because else you're not really keeping it strictly offline. Further, keeping a drive offline for numerous extended periods of time can diminish the longevity of the drive (see the Wikipedia article I linked, ref. #12). For this reason, when you perform your next backup cycle, IMO you should perhaps consider switching the "primary role" from (A) to (B). When you perform yet another backup cycle after that, you can switch this role from (B) to (C). Yet another backup cycle, switch from (C) to (A) again, and so forth.

A fairly similar strategy can be achieved using just 2 external drives and an internal one, as long as you always remember to unplug one of both external drives before plugging in the other one. The files on the internal drive can optionally be deleted after they have been safely copied to both external ones, and, also optionally, you can use a free software tool such as TeraCopy in order to speed up the copying of files a little bit. [ For *really* fast searching of files and folders, I always use Everything ].

[yves]

Apr 8, 2014 18:45:40 GMT -5 Jim said:

Offline backups are the way to go if you want real safety.

The feds stole some of my online backups after they.. ehh, RAIDed Kim Dotcom.. Anyway, I also kept an offline copy so no panic.

[Jim]

Apr 8, 2014 19:30:56 GMT -5 yves said:

Apr 8, 2014 18:45:40 GMT -5 Jim said:

Offline backups are the way to go if you want real safety.

The feds stole some of my online backups after they.. ehh, RAIDed Kim Dotcom.. Anyway, I also kept an offline copy so no panic.

Friend of mine lost a boatload of pics that were stored on a cloud service. He went to look for them and found out that most was lost due to the provider reducing the allotted storage. Doh!

"Raided". That's punny.

[yves]

Apr 8, 2014 19:40:43 GMT -5 Jim said:

Apr 8, 2014 19:30:56 GMT -5 yves said:

The feds stole some of my online backups after they.. ehh, RAIDed Kim Dotcom.. Anyway, I also kept an offline copy so no panic.

Friend of mine lost a boatload of pics that were stored on a cloud service. He went to look for them and found out that most was lost due to the provider reducing the allotted storage. Doh!

"Raided". That's punny.

"why WD is expensive"

[lsdeep]

i would not go for a 'pre-made' nas solution. i used myself first many separate external usb drives, later qnap and buffalo nas boxes. of course it depends also on your storage need at the moment... AND projected for the near/ medium future.
for redundancy you would poss look at raid 1 or 5 (maybe 4 or 6). all this is fine and dandy but locks you in with pre-made  nas solutions for insanely high prices.

my recommendation would be, build your own! (gasp!!!). it is not rocket science! even if you doubt your own ability to build a simple pc (even qnap, buffalo etc. are nothing else basically), have somebody build it for you (like your it-guy).
from my own experience i can highly recommend lime-tech's unraid software. using it now for about 2 yrs. it saved my butt once already (well, it saved my files   and my sanity). it offers many advantages in flexibility over standard raid solutions, offers outstanding expand-ability and is easy to scale up for future needs. ( lime technology ).
you can basically use all your existing drives, i de-housed many of my external drives even to use in the unraid server. lately bought and added drives involved several external wd drives (for some insane reason they where cheaper than the barebone drives!!!). don't get 'sucked' into the more hardware is better train of thought. it needs very little in processing power! also there you can scale up later as and if needed. you might have parts laying around, which are perfectly usable for a project like that (i only had to buy a motherboard and sufficient power supply). at this point  i am running a 32tb (on 10 data drives + 1 parity + 1 cache) without any problems. it gives me peace of mind for about 65.000 albums stored on it.
the community at unraid is great (like the one here for emotiva, just for unraid), means help is always just a click away. no grumpy support from buffalo, wd, qnap or else to deal with. the software gets constantly developed and offers new features, up to date programming. there are other benefits as well (like plugins to expand the capabilties etc.)

my first setup (keep in mind, capable to handle 12hdd's) cost little more than $275 without drives, try to find a easy to expand, reliable and flexible nas system for that.

cheers, L

[lsdeep]

Apr 8, 2014 19:15:09 GMT -5 johnnyd70 said:

Ever look at iTunes Match? The simplest solution of all - offline backup for security and streaming to devices that support Airplay. It's cheap at around $20/yr. Only drawback is that you're limited and confined to the Apple universe somewhat. I use a combination of iTunes Match and NAS for local backup.

beside being cheap ($20 for how many tb???) - i just don't trust cloud storage for anything important to me. the company might pack in, raise prices, big brother is going to your stuff.....

[hemster]

Apr 8, 2014 19:19:35 GMT -5 yves said:

Apr 8, 2014 17:09:07 GMT -5 hemster said:

^That's exactly what I do. My backup drives are purely offline backups. emotivalounge.proboards.com/post/630577

Yes, it's the most cost effective strategy for home users AFAIK. However, you should always remember to unplug (B) *before* plugging in (C), and unplug (C) *before* plugging in (B) because else you're not really keeping it strictly offline. Further, keeping a drive offline for numerous extended periods of time can diminish the longevity of the drive (see the Wikipedia article I linked, ref. #12). For this reason, when you perform your next backup cycle, IMO you should perhaps consider switching the "primary role" from (A) to (B). When you perform yet another backup cycle after that, you can switch this role from (B) to (C). Yet another backup cycle, switch from (C) to (A) again, and so forth.

A fairly similar strategy can be achieved using just 2 external drives and an internal one, as long as you always remember to unplug one of both external drives before plugging in the other one. The files on the internal drive can optionally be deleted after they have been safely copied to both external ones, and, also optionally, you can use a free software tool such as TeraCopy in order to speed up the copying of files a little bit. [ For *really* fast searching of files and folders, I always use Everything ].

Yes, every quarter I rotate which drive is acting as the primary "A" drive.

Also, the only time the "B" and "C" drives are plugged in to the computer is during the sync process - otherwise they're completely off-line. I do not use an internal drive on my machine as I travel with my machine (a laptop) and it only has  500 gb internal drive which has 25% free space now. 

[ÈlTwo]

It doesn't matter how many ways you make it redundant: back it up, copy it, save it, scan it, store it all on hard drives, CDs, DVDs or tape, or use any variety of RAID. There is a possibility of all your data getting hosed no matter what you do.

The real question is how much redundancy do you really need? Is what you're doing getting you from an 65% of not getting totally hosed to 95%, or is it getting you from 99.2% to 99.5%? What is reasonable and what is not cost effective?

[GreenKiwi]

Actually, it's off site, not off line:

• At least three copies
• In two different formats
• with one of those copies off-site.

blog.trendmicro.com/trendlabs-security-intelligence/world-backup-day-the-3-2-1-rule/

I disagree with others comments about the most cost effective being offline backups. Assuming you are not on a metered broadband connection and have at least 10mbps upstream, $50/year gets you 1TB,2TB,3TB of backup? With crashplan the same backup app will backup to both a local drive and off site. So you get fast recovery and true disaster safety.

[GreenKiwi]

As for systems, I'm moving to a less is more mindset, with dedicated devices rather than complex devices like a NAS. (And mind you, I'm a computer programmer and network engineer. My synology can pretty much saturate a GigE port for both reads and writes. We're talking 90-100 MB/s.)

But in the end it comes down to actual using experience rather than everything else and I have a wife who just wants stuff to work.

I actually have pretty much given up on local video. It may come back as iTunes via AirPlay or direct, we'll see. But I get almost all my video through streaming. Netflix, AppleTV, AirPlay and all the various TV apps that let you just watch shows. If I were doing it, I'd use Plex on a PC or Mac and have various front end devices to play my videos.

For music, I'm moving to a pure dedicated Olive One. (If that fails, it will be an inexpensive touch screen laptop like the Lenovo Yoda running JRiver.) I'd have gotten sonos if they did HD, I really enjoyed their system... and I was tempted by using squeeze devices, but they were all pretty much going the DIY route and I really missed "auto generated playlists", i.e. Jriver Play Doctor and Genius Playlists.