[Lustre-devel] statahead feature
Alex.Zhuravlev at Sun.COM
Thu Jul 24 11:56:39 PDT 2008
it's actually very interesting point - how much of lustre internals we
should expose outside? I mean flexibility vs. complexity issue in context
of portable API.
Peter Braam wrote:
> I strongly agree with this. A good way to verify if we have a favorable
> implementation is to see if it can be ported to other OS's.
> On 7/24/08 4:19 AM, "Alex Zhuravlev" <Alex.Zhuravlev at Sun.COM> wrote:
>> due to some experiments with dcache related code we've been doing with shadow
>> and others, it became clear that statahead code is quite complicated. probably
>> for no reason. the most hard part to follow is interaction with dcache. the
>> feature does number of complex things and make other parts (like
>> harder to follow too.
>> after amount of discussions with people we'd like to share our vision on the
>> feature and propose slightly different solution.
>> we think statahead should do nothing with dcache. it's about inodes and
>> only. thus, it would be good to decouple it from dcache. the only thing
>> should do is:
>> 1) detect statahead is needed (policy, out of the message's scope)
>> 2) scan part of directory (probably using readdir(), skip RPCs)
>> 3) finds/creates inodes for found fids
>> 4) lock these inodes (notice we propose to use inodes as a serialization point
>> so that lockless getattr can be used)
>> 5) issue getattr RPCs (probably lockless)
>> 6) unlock inodes upon getattr's completion
>> then stat(2) is called, it first has to lookup fid by name. for this we can
>> pagecache just filled with MDS_READDIR. if directory isn't being modified at
>> time, then entries will be there and we can create dentries in the dcache.
>> will be valid till UPDATE lock is cancelled - no even LOOKUP lock is needed.
>> another possible thing for optimization is lockless getattr. given most of
>> kernel don't pass intent to ->getattr(), it's possible that stat(2) needs two
>> one in ll_lookup_it() and another in ll_getattr_it() as lock is released
>> between them.
>> stat(2) gives no warranty about attributes, it gives a shot of them.
>> attributes can
>> change right before userspace application get them. so, why don't we introduce
>> simple mechanism making attributes valid for short time at least for process
>> lookup. this could help statahead as well, we think.
>> comments? suggestions?
>> thanks, Alex
>> Lustre-devel mailing list
>> Lustre-devel at lists.lustre.org
More information about the lustre-devel