[Lustre-discuss] Lustre and cross-platform portability

Joe Landman landman at scalableinformatics.com
Thu Mar 15 07:36:51 PDT 2012

On 03/14/2012 08:31 PM, Andreas Dilger wrote:
> Whamcloud and EMC are jointly investigating how to be able to
> contribute the Lustre client code into the upstream Linux kernel.

Would be good to have an active discussion of this at LUG as well.

> As a prerequisite to this, EMC is working to clean up the Lustre
> client code to better match the kernel coding style, and one of the
> anticipated major obstacles to upstream kernel submission is the
> heavy use of code abstraction via libcfs for portability to other
> operating systems (most notably MacOS and WinNT, but also for
> liblustre, and potentially *BSD).

Breaking the code into bite sized patches that could go into the kernel 
would be a *good* thing.  Continuing to maintain a set of out of core 
patches (with compatibility/portability issues) is not a good thing.

> I have no information that the WinNT project will ever be released by
> Oracle, and as yet there has not been any code released from the
> MacOS port, so the libcfs portability layer is potentially exacting a
> high cost in code maintenance and complexity (CLIO being a prime
> example) for no apparent benefit.  Similarly, the liblustre client
> needs a portability layer for userspace, and suffers from the same
> apparent lack of interest or users.
> I'd like to get some feedback from the Lustre community about
> removing the libcfs abstraction entirely, or possibly restructuring
> it to look like the Linux kernel API, and having the other platforms

(Re)Using as much of the existing kernel API as possible (reduce 
friction for inclusion), without re-inventing wheels (reduce friction 
for inclusion), and insisting "our wheels are rounder" versus 
fixing/updating existing subsystems (reduce friction for inclusion) 
would be advised.  Many a good project has lost out on kernel inclusion 
to other competitive projects over these issues (and interaction with 
the kernel community).  SCST (scsi target layer) is a prime example, and 
one we use, even though LIO was just included about a year ago.  The LIO 
team does kernel developer interaction far better than the SCST team. 
Has nothing to do with code quality/functionality.

> code against it as a Linux portability layer, like ZFS on Linux uses
> the Solaris Portability Layer (SPL) to avoid changing the core ZFS
> code.  A related topic is whether it would be better to replace all
> cfs_* functions with standard Linux kernel functions en-masse, or
> migrate away from cfs_* functions slowly?

Do the change en-masse.  Get it done, once.  The sooner the better, so 
we can test.  Hopefully this means we will also be able to use 
alternative OST file systems other than ext4.

Joseph Landman, Ph.D
Founder and CEO
Scalable Informatics Inc.
email: landman at scalableinformatics.com
web  : http://scalableinformatics.com
phone: +1 734 786 8423 x121
fax  : +1 866 888 3112
cell : +1 734 612 4615

More information about the lustre-discuss mailing list