[Lustre-discuss] Lustre and cross-platform portability

[Joe Landman]

On 03/14/2012 08:31 PM, Andreas Dilger wrote:
> Whamcloud and EMC are jointly investigating how to be able to
> contribute the Lustre client code into the upstream Linux kernel.

Would be good to have an active discussion of this at LUG as well.

> As a prerequisite to this, EMC is working to clean up the Lustre
> client code to better match the kernel coding style, and one of the
> anticipated major obstacles to upstream kernel submission is the
> heavy use of code abstraction via libcfs for portability to other
> operating systems (most notably MacOS and WinNT, but also for
> liblustre, and potentially *BSD).

Breaking the code into bite sized patches that could go into the kernel 
would be a *good* thing.  Continuing to maintain a set of out of core 
patches (with compatibility/portability issues) is not a good thing.

> I have no information that the WinNT project will ever be released by
> Oracle, and as yet there has not been any code released from the
> MacOS port, so the libcfs portability layer is potentially exacting a
> high cost in code maintenance and complexity (CLIO being a prime
> example) for no apparent benefit.  Similarly, the liblustre client
> needs a portability layer for userspace, and suffers from the same
> apparent lack of interest or users.
>
> I'd like to get some feedback from the Lustre community about
> removing the libcfs abstraction entirely, or possibly restructuring
> it to look like the Linux kernel API, and having the other platforms

(Re)Using as much of the existing kernel API as possible (reduce 
friction for inclusion), without re-inventing wheels (reduce friction 
for inclusion), and insisting "our wheels are rounder" versus 
fixing/updating existing subsystems (reduce friction for inclusion) 
would be advised.  Many a good project has lost out on kernel inclusion 
to other competitive projects over these issues (and interaction with 
the kernel community).  SCST (scsi target layer) is a prime example, and 
one we use, even though LIO was just included about a year ago.  The LIO 
team does kernel developer interaction far better than the SCST team. 
Has nothing to do with code quality/functionality.

> code against it as a Linux portability layer, like ZFS on Linux uses
> the Solaris Portability Layer (SPL) to avoid changing the core ZFS
> code.  A related topic is whether it would be better to replace all
> cfs_* functions with standard Linux kernel functions en-masse, or
> migrate away from cfs_* functions slowly?

Do the change en-masse.  Get it done, once.  The sooner the better, so 
we can test.  Hopefully this means we will also be able to use 
alternative OST file systems other than ext4.






-- 
Joseph Landman, Ph.D
Founder and CEO
Scalable Informatics Inc.
email: landman at scalableinformatics.com
web  : http://scalableinformatics.com
        http://scalableinformatics.com/sicluster
phone: +1 734 786 8423 x121
fax  : +1 866 888 3112
cell : +1 734 612 4615