[Lustre-discuss] Fwd: Lustre and Large Pages

[Kevin Van Maren]

Easy way to reduce the client memory used by "Lustre" is to have an 
Epilogue script run by SGE (or whatever scheduler/resource manager) that 
does something like this on every node:
# sync ; sleep 1 ; sync
# echo 3 > /proc/sys/vm/drop_caches

Kevin


Nathan Rutman wrote:
> Jim, I'm forwarding this to lustre-discuss to get a broader community 
> input.  I'm sure somebody has some experience with this.
>
> Begin forwarded message:
>>
>> I am looking for information on how Lustre assigns and holds pages on 
>> client nodes across jobs.  The motivation is that we want to make 
>> "huge" pages available to users.  We have found that it is almost 
>> impossible to allocate very many "huge" pages since Lustre holds 
>> scattered small pages across jobs.  In fact, typically about 1/3 of 
>> compute node memory can be allocated as huge pages.
>>
>> We have done quite a lot of performance studies which show that a 
>> substantial percentage of jobs on Ranger have TLB misses as a major 
>> performance bottleneck.  We estimate we might recover as much as an 
>> additional 5%-10% throughput if users could use huge pages.
>>
>> Therefore we would like to find a way to minimize the client memory 
>> which Lustre holds across jobs.
>>
>> Have you had anyone else mention this situation to you?
>>
>> Regards,
>>
>> Jim Browne
>>
>>
>
> ------------------------------------------------------------------------
>
> _______________________________________________
> Lustre-discuss mailing list
> Lustre-discuss at lists.lustre.org
> http://lists.lustre.org/mailman/listinfo/lustre-discuss
>