[Lustre-discuss] MDT backup procedure

[John White]

Ah, if I had only read on one mail I would have seen this procedure is  
far more complex than described in the Operations Manual.  I suppose  
that's one of the benefits of an object based FS.
----------------
John White
High Performance Computing Services (HPCS)
(510) 486-7307
One Cyclotron Rd, MS: 50B-3209C
Lawrence Berkeley National Lab
Berkeley, CA 94720








On Jun 18, 2009, at 10:58 PM, John White wrote:

> On Jun 17, 2009, at 4:09 PM, Adam Knight wrote:
>
>> Pertaining to your original email, rather than taking the MDT down to
>> backup, it is very convenient to use LVM snapshots.  With this
>> functionality it creates a LV duplicate of the MDT and allows you to
>> mount that as ldiskfs and backup files from a consistent copy (won't
>> be
>> changing even if your MDT continues to add/remove data).  Your lustre
>> filesystem will therefore stay operational during the backup.  If you
>> time it cleverly, you can snapshot your MDT and OSTs at the same time
>> and backup from all of them to have a consistent copy of the whole
>> filesystem as well.
>>
>
> So following this, has anyone migrated an MDT to new storage with this
> sort of procedure?
>
> -create an lvm'd MDT that produces snapshots
> -use it for a while in production
> -get some snazzy new disk
> -shutdown lustre
> -take a snapshot of the MDT and shuffle it off to some different
> storage media
> -create a new LVM with snazzy new disk (specifically of a different
> size from the original MDT)
> -restore snapshot
> -run lfsck for good measure (is this advisable on what could feasibly
> be a clean filesystem?)
> -bring up lustre
>
> Please keep in mind, I've used LVM but haven't used snapshots, I'm not
> familiar with their limitations.  We're looking to create a filesystem
> immediately but would like to get some much faster storage for the MDT
> later without burning and building a new FS.
>
>>> Thanks for this verbose reply.  It is exactly what I needed and
>>> what I suspected I would run into.  We are planning on multiple
>>> backup procedures.  Users will backup at checkpoints in their work
>>> flow, IT will backup the MDT nightly and we are also looking at the
>>> possibility of backup the complete file system.
>>>
>>> Thanks again for everyone's input, this gives me some good
>>> ammunition going forward for proposals.
>>>
>>> Thanks,
>>> Dan Kulinski
>>>
>>>
>>>
>> _______________________________________________
>> Lustre-discuss mailing list
>> Lustre-discuss at lists.lustre.org
>> http://lists.lustre.org/mailman/listinfo/lustre-discuss
>
> _______________________________________________
> Lustre-discuss mailing list
> Lustre-discuss at lists.lustre.org
> http://lists.lustre.org/mailman/listinfo/lustre-discuss