<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2//EN">
<HTML>
<HEAD>
<META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=iso-8859-1">
<META NAME="Generator" CONTENT="MS Exchange Server version 6.5.7652.24">
<TITLE>RE: [Lustre-devel] Lustre HSM HLD draft</TITLE>
</HEAD>
<BODY>
<!-- Converted from text/plain format -->
<P><FONT SIZE=2>Hello<BR>
<BR>
thank you for your review, I add some comments in the following<BR>
<BR>
Page 1, 1, Define coordinator (space coordinator?),<BR>
define agent, (condense Part II intro, page 14)<BR>
(for me, MDT, MGS and OST)<BR>
These are defined in the arch wiki pages<BR>
<BR>
Page 10,<BR>
4.2, 2) Implies only one copy per "version"...bad idea<BR>
Different versions correspond to different files in the external storage. We take the more recent.<BR>
Not sure I understand your remark<BR>
<BR>
Page 13, Lustre object mtime may not be good enough. There are several<BR>
mechanisms (like touch) to manipulate mtime, which makes it<BR>
unusable as a last written time.<BR>
If a user make a touch in the past this change the mtime and can hide previous writes.<BR>
If we want to keep real write time we need to add a new time field in Lustre backend<BR>
(may be ZFS has it)<BR>
<BR>
Page 19, Special Path, does this boil down to invisible I/O?<BR>
The path is /mnt_mount/.lustre/fid/FID_NUMBER. When a file is open through this path a<BR>
flag is carried to the OSS to avoid copy in trigger (this used to fill the file)<BR>
<BR>
Page 23, 2.3 and 2.4, I'm assuming that lists of tuples can be processed<BR>
in any order.<BR>
yes<BR>
<BR>
Issues:<BR>
The Space manager is likely the most important piece. There is no<BR>
detail on it. This is where archive and other policy is enforced.<BR>
The space manager is based on changelogs/feed Lustre feature which are very new (draft HLD has just been<BR>
published). This is why it not described at this time.<BR>
<BR>
The described HSM seems to follow the "copy out" when space needed,<BR>
then purge, model. This function (a Space Manager function) is contrary<BR>
to SAM, and a shortfall of many HSMs.<BR>
no spacemanger is doing pre-migration and when free space is needed, it only has to make punc<BR>
<BR>
Coordination between agents seems important. For example,<BR>
if agents requested new copy-outs on objects striped on<BR>
10 different stores, ordering them on tape seems difficult.<BR>
Tape access optimization has to be made by the archival system. We try to put as few external storage knowledge<BR>
as possible in Lustre to be external storage independant.<BR>
<BR>
What is the backup story for Lustre? How does that play with<BR>
the HSM?<BR>
HSM do not backup the namespace. It has to be done with a separate tool like a MDT scannner.<BR>
The copy tool can use the FID2PATH() function to save the object pathname with the file.<BR>
<BR>
</FONT>
</P>
</BODY>
</HTML>