<html><head>
<meta http-equiv="Content-Type" content="text/html; charset=Windows-1252"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; color: rgb(0, 0, 0); font-size: 14px; font-family: Calibri, sans-serif; "><div>I have used fuse for other filesystems: its great if all you need is access to the data, but the performance is HORRIBLE. </div><div><br></div><div>--jason</div><div><br></div><span id="OLK_SRC_BODY_SECTION"><div style="font-family:Calibri; font-size:11pt; text-align:left; color:black; BORDER-BOTTOM: medium none; BORDER-LEFT: medium none; PADDING-BOTTOM: 0in; PADDING-LEFT: 0in; PADDING-RIGHT: 0in; BORDER-TOP: #b5c4df 1pt solid; BORDER-RIGHT: medium none; PADDING-TOP: 3pt"><span style="font-weight:bold">From: </span> Jon Yeargers <<a href="mailto:yeargers@ohsu.edu">yeargers@ohsu.edu</a>><br><span style="font-weight:bold">Date: </span> Friday, December 7, 2012 9:42 AM<br><span style="font-weight:bold">To: </span> Jason Brooks <<a href="mailto:brookjas@ohsu.edu">brookjas@ohsu.edu</a>>, "<a href="mailto:lustre-discuss@lists.lustre.org">lustre-discuss@lists.lustre.org</a>" <<a href="mailto:lustre-discuss@lists.lustre.org">lustre-discuss@lists.lustre.org</a>><br><span style="font-weight:bold">Subject: </span> RE: [Lustre-discuss] Applications of Lustre - streaming?<br></div><div><br></div><div xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40"><meta name="Generator" content="Microsoft Word 12 (filtered medium)"><style><!--
/* Font Definitions */
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri","sans-serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p.MsoAcetate, li.MsoAcetate, div.MsoAcetate
{mso-style-priority:99;
mso-style-link:"Balloon Text Char";
margin:0in;
margin-bottom:.0001pt;
font-size:8.0pt;
font-family:"Tahoma","sans-serif";}
span.EmailStyle17
{mso-style-type:personal;
font-family:"Calibri","sans-serif";
color:windowtext;}
span.EmailStyle18
{mso-style-type:personal-reply;
font-family:"Calibri","sans-serif";
color:#1F497D;}
span.BalloonTextChar
{mso-style-name:"Balloon Text Char";
mso-style-priority:99;
mso-style-link:"Balloon Text";
font-family:"Tahoma","sans-serif";}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]--><div lang="EN-US" link="blue" vlink="purple"><div class="WordSection1"><p class="MsoNormal"><span style="color:#1F497D">If it weren’t for the positive aspects of HDFS I wouldn’t really be considering HBase (over Cassandra).
<o:p></o:p></span></p><p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p><p class="MsoNormal"><span style="color:#1F497D">Any notion of the merits of Lustre’s kernel-based mounts vs a FUSE-based mount (HDFS)? Whichever filesystem I go with I will need to store ‘flat files’ in.<o:p></o:p></span></p><p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p><div><div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in"><p class="MsoNormal"><b><span style="font-size: 10pt; font-family: Tahoma, sans-serif; ">From:</span></b><span style="font-size: 10pt; font-family: Tahoma, sans-serif; "> Jason Brooks
<br><b>Sent:</b> Friday, December 07, 2012 9:38 AM<br><b>To:</b> Jon Yeargers; <a href="mailto:lustre-discuss@lists.lustre.org">lustre-discuss@lists.lustre.org</a><br><b>Subject:</b> Re: [Lustre-discuss] Applications of Lustre - streaming?<o:p></o:p></span></p></div></div><p class="MsoNormal"><o:p> </o:p></p><div><p class="MsoNormal"><span style="font-size:10.5pt;color:black">Hello,<o:p></o:p></span></p></div><div><p class="MsoNormal"><span style="font-size:10.5pt;color:black"><o:p> </o:p></span></p></div><div><p class="MsoNormal"><span style="font-size:10.5pt;color:black">The question of hdfs storage via lustre has been in the foreground of my thinking. the hadoop hdfs processes are not aware of block devices: they only know of a filesystem mount point to begin
storing data in hdfs. THUS…<o:p></o:p></span></p></div><div><p class="MsoNormal"><span style="font-size:10.5pt;color:black"><o:p> </o:p></span></p></div><div><p class="MsoNormal"><span style="font-size:10.5pt;color:black">If we provide a filesystem interface (say a lustre mount point) whose latencies and throughput approach that of local disk storage (say, via infiniband), could we not have the various hadoop nodes
store their data in the lustre filesystem? would hadoop even care?<o:p></o:p></span></p></div><div><p class="MsoNormal"><span style="font-size:10.5pt;color:black"><o:p> </o:p></span></p></div><div><p class="MsoNormal"><span style="font-size:10.5pt;color:black">I realize that this may not be a good place to bring it up. But there you go…<o:p></o:p></span></p></div><div><p class="MsoNormal"><span style="font-size:10.5pt;color:black"><o:p> </o:p></span></p></div><div><p class="MsoNormal"><span style="font-size:10.5pt;color:black">One of these days, (with all of my ample spare time), I will benchmark it. and report of course…<o:p></o:p></span></p></div><div><p class="MsoNormal"><span style="font-size:10.5pt;color:black"><o:p> </o:p></span></p></div><div><p class="MsoNormal"><span style="font-size:10.5pt;color:black">--jason<o:p></o:p></span></p></div><div><p class="MsoNormal"><span style="font-size:10.5pt;color:black"><o:p> </o:p></span></p></div><div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in"><p class="MsoNormal"><b><span style="color:black">From: </span></b><span style="color:black">Jon Yeargers <<a href="mailto:yeargers@ohsu.edu">yeargers@ohsu.edu</a>><br><b>Date: </b>Friday, December 7, 2012 9:26 AM<br><b>To: </b>"<a href="mailto:lustre-discuss@lists.lustre.org">lustre-discuss@lists.lustre.org</a>" <<a href="mailto:lustre-discuss@lists.lustre.org">lustre-discuss@lists.lustre.org</a>><br><b>Subject: </b>[Lustre-discuss] Applications of Lustre - streaming?<o:p></o:p></span></p></div><div><p class="MsoNormal"><span style="font-size:10.5pt;color:black"><o:p> </o:p></span></p></div><div><div><p class="MsoNormal"><span style="color:black">Can Lustre be used to store data like streaming audio / video? I’ve been scolded about considering it for DB storage but I’m looking at the relative merits of Lustre vs HDFS.
<o:p></o:p></span></p><p class="MsoNormal"><span style="color:black"> <o:p></o:p></span></p><p class="MsoNormal"><span style="color:black">I’m moving to a clustered DB setup and wondering about Cassandra / Lustre vs Hadoop (IE HBase / HDFS). One offers flexibility in terms of mixing hardware components while the other is a ‘one stop shop’. <o:p></o:p></span></p><p class="MsoNormal"><span style="color:black"> <o:p></o:p></span></p><p class="MsoNormal"><span style="color:black">Not trying to elicit a religious war – and yes, I’ve been reading as much as I can find about this. Just hoping for the opinion(s) of this side of the table.<o:p></o:p></span></p></div></div></div></div></div></span></body></html>