<br><br>---------- Forwarded message ----------<br>From: linux freaker<br>Date: Monday, March 18, 2013<br>Subject: Understanding lustre setup ..<br>To: Colin Faber <<a href="mailto:colin_faber@xyratex.com">colin_faber@xyratex.com</a>><br>
<br><br><br><br>On Monday, March 18, 2013, Colin Faber <<a href="mailto:colin_faber@xyratex.com">colin_faber@xyratex.com</a>> wrote:<br>> Hi,<br>><br>> On 03/15/2013 10:56 AM, linux freaker wrote:<br>>><br>
>> Let me explain in brief the clear picture I would need to understand:<br>>><br>>><br>>> Aim: Comparing Hadoop over HDFS Vs Hadoop over Lustre<br>>><br>>> Things I tried:<br>>><br>
>> I took 5 machines: 1 MDS, 2 OSS/OST and 2 Lustre Client.(each of 4GB RAM and 700 GB hard disk).<br>>> I created around 6 OST on each OSS for size 6GB each through LVM.<br>><br>> Using LVM based OST's can slow things down a lot. Really for best performance you should be using unpartitioned raw disk for ldiskfs formating.<br>
><br>>><br>>><br>>><br>>> On Wed, Mar 13, 2013 at 8:01 AM, linux freaker <<a href="mailto:linuxfreaker@gmail.com">linuxfreaker@gmail.com</a> <mailto:<a href="mailto:linuxfreaker@gmail.com">linuxfreaker@gmail.com</a>>> wrote:<br>
>> ><br>>> ><br>>> > On Wednesday, March 13, 2013, Colin Faber <<a href="mailto:colin_faber@xyratex.com">colin_faber@xyratex.com</a> <mailto:<a href="mailto:colin_faber@xyratex.com">colin_faber@xyratex.com</a>>> wrote:<br>
>> >> Hi,<br>>> >><br>>> >> I'm sorry, It was a busy day for me. I will try and respond appropriately<br>>> >> to your questions tomorrow.<br>>> >><br>>> >> -cf<br>
>> >><br>>> >> On 03/12/2013 07:45 PM, linux freaker wrote:<br>>> >>><br>>> >>><br>>> >>> On Tuesday, March 12, 2013, linux freaker <<a href="mailto:linuxfreaker@gmail.com">linuxfreaker@gmail.com</a> <mailto:<a href="mailto:linuxfreaker@gmail.com">linuxfreaker@gmail.com</a>><br>
>> >>> <mailto:<a href="mailto:linuxfreaker@gmail.com">linuxfreaker@gmail.com</a> <mailto:<a href="mailto:linuxfreaker@gmail.com">linuxfreaker@gmail.com</a>>>> wrote:<br>>> >>> ><br>
>> >>> ><br>>> >>> > On Tuesday, March 12, 2013, linux freaker <<a href="mailto:linuxfreaker@gmail.com">linuxfreaker@gmail.com</a> <mailto:<a href="mailto:linuxfreaker@gmail.com">linuxfreaker@gmail.com</a>><br>
>> >>> > <mailto:<a href="mailto:linuxfreaker@gmail.com">linuxfreaker@gmail.com</a> <mailto:<a href="mailto:linuxfreaker@gmail.com">linuxfreaker@gmail.com</a>>>> wrote:<br>>> >>> >><br>
>> >>> >><br>>> >>> >> On Tuesday, March 12, 2013, Colin Faber <<a href="mailto:colin_faber@xyratex.com">colin_faber@xyratex.com</a> <mailto:<a href="mailto:colin_faber@xyratex.com">colin_faber@xyratex.com</a>><br>
>> >>> >> <mailto:<a href="mailto:colin_faber@xyratex.com">colin_faber@xyratex.com</a> <mailto:<a href="mailto:colin_faber@xyratex.com">colin_faber@xyratex.com</a>>>> wrote:<br>>> >>> >>> Hi,<br>
>> >>> >>><br>>> >>> >>> On 03/11/2013 08:13 PM, linux freaker wrote:<br>>> >>> >>>><br>>> >>> >>>><br>>> >>> >>>> On Tuesday, March 12, 2013, linux freaker <<a href="mailto:linuxfreaker@gmail.com">linuxfreaker@gmail.com</a> <mailto:<a href="mailto:linuxfreaker@gmail.com">linuxfreaker@gmail.com</a>><br>
>> >>> >>>> <mailto:<a href="mailto:linuxfreaker@gmail.com">linuxfreaker@gmail.com</a> <mailto:<a href="mailto:linuxfreaker@gmail.com">linuxfreaker@gmail.com</a>>> <mailto:<a href="mailto:Thanks">Thanks</a> for response.<br>
I got what you suggested.<br>so i will have 1 mds, 2 oss and 2 lustre client.I will install hadoop on both the lustre client.<br><br><br>May I know what are the steps to configure hadoop over lustre? I tried to configure core-site.xml but no idea what other files chinch I need.<br>
Can you share the steps?<br>Also, i am trying to run wordcount example over lustre.Please suggest.<br><br><br>