Fog Creek Software
Discussion Board




How to back up 1 Tb?

We're a victim of low priced disks outstripping our backup facility, can some one suggest a cheap backup capacity for the following senario?

We've got four 600Gb raid-5 devices all attached to Linux servers. Two of the raids are on-line smb/nfs servers and are rsync'd every Friday to the 2 backup raid-5's in the basement (same building) across the nerwork. While this is fine for coping with broken raids (never happened in 3+ yrs) i.e. we can cart one of the backups from the basement into the office while the office one is being fixed since I live in an earthquake/typhoon zone its time to look at other methods of backup.

The budget is limited to about US$ 7,500, each set of backup media should be about to handle 500Gb - 1Tb  ish and be easy to cart offsite using a bag on the bus - no private cars on site here! Most of our data is already keep in a compressed format so a backup that says 300Gb native/600 compressed is not useful. The backup system must be Linux compatible. There is no requirement for encrypting the data.

Any ideas? I looked at exabyte stuff but the libraries are too expensive fro us.

Thanks
Jon

Jon Wright
Thursday, May 01, 2003

Maxtor 250GB drives are $250US each - $1000 gets you a TB. Figure out a way to put four drives in an enclosure with an interface and just take the whole enclosure offsite.

Philo

Philo
Thursday, May 01, 2003

Sorry, I should have been more specific, I'm not looking for a hard disk based backup solution. With two raids in the basement already I am looking for something else instead.

Jon Wright
Thursday, May 01, 2003

According to the IBM website, an IBM 3581 7-tape Ultrium autoloader (700GB native) will cost about US$8000, without media.  Media costs US$133 for each 100GB tape.  You may want to reconsider the portable hard drive option, it's a lot cheaper.

I've got a similar setup (not quite as much data), and I'm using Maxtor 250GB Firewire drives for backups.  Works perfectly under linux.

Tim Evans
Friday, May 02, 2003

UseNet.

alt.binaries.backups.super-sensitive.encrypted

And you can get at it from anywhere...

:)

Geoff Bennett
Friday, May 02, 2003

Would it be possible to delete some of the data and put the remainder on a floppy?

I bet a terabyte backup is a long endeavor.. is your data live or static during the backup?

Li-fan Chen
Friday, May 02, 2003

Strange as it seems, the price of backup tape media can be greater than the price of the equivelent space in hard drives.  With the size of your backup only very expensive tape systems - using expensive tapes - will handle it.  With your budget constraints, the idea of using IDE hard drives is going to be hard to match.  You can buy a LOT of hard drives, and you don't have to buy them all at once.  (With the steady drop in price/GB you probably want to buy them as needed.)

For a belt-and-suspenders approach, write to mirrored drives, and then the drives of each pair to different places.

RH
Friday, May 02, 2003

Short answer is that you can't have a terabyte of critical changing data and a $7500 budget.  A couple of overlapping portable disk arrays would be a good solution for disaster recovery.  If you also want a history of archives you're going to have to limit the content or radically adjust the price tag.  How can you have so much data? 

John Aitken
Friday, May 02, 2003

'Overlapping' was the wrong word.  Probably you know what I meant: one on site / one off site (or nearby), traded each day or backup interval.

John Aitken
Friday, May 02, 2003

Okay,

The concensus seems to be pretty much using hard disks and moving them elsewhere.

The majority of the data is from molecular dynamics simulations, each simulation can be 4-5 Gb and people might want to do 10-20 per project so it mounts quite quickly. We have a 64 cpu beowulf so generating rubbish is easy esp if students are involved.

The basements raids are mirror images of the live raids and are made via rsync each friday. This is to cover a raid breakdown.
What scares me is that is the place caught fire/flooded we could lose everything. I guess each single person does not need more than 120Gb thou so we can manage with that.

Thanks for you help
Jon

Jon Wright
Friday, May 02, 2003

"Strange as it seems, the price of backup tape media can be greater than the price of the equivelent space in hard drives."

If everybody had been as rigourous in backing up all those years the backup device market would be equal in size to the HD market and we might have comparable pricing. So it is all the sloppy sysadmins out there that are responsible for this imbalance.

Well, that and maybe partly the fact that most of the worlds hardisks are just filled to the brim with the same WaReZ, P0rnZ, MP3's and DVI's that aren't too business critical.

Just me (Sir to you)
Friday, May 02, 2003

As far as the budget, up yours.

Mike
Friday, May 02, 2003

How long do each of these 4-5Gb simulations take to run on your Beowulf? It is less than a day or so, you may want to think about not backing them up - just backup the input data, and maybe the executable that ran it, and most users can generate the data again.

David Clayworth
Monday, May 05, 2003

<troll>
Is this one of those microsoft questions?
</troll>

www.marktaw.com
Monday, May 05, 2003

*  Recent Topics

*  Fog Creek Home