[Lustre-devel] Checksum Algorithm
Paul Nowoczynski
pauln at psc.edu
Tue Nov 6 11:57:50 PST 2007
Brian,
How does the crc mechanism work? I assume that the crc is done at the
client, does the server verify the crc? Also are the crc's stored on disk?
thanks,
paul
Brian Behlendorf wrote:
> Roger,
>
> We've been running with checksums enabled in our release for some time now
> and have seen the exact same impact on performance. In our case single node
> performance is impacted but aggregate FS performance remains good when enough
> clients are involved. We are tracking the performance issue under bug 13805
> and would love any input/insight you might have on the issue.
>
> Bug13805 <https://bugzilla.lustre.org/show_bug.cgi?id=13805>
>
> My view on the issue is that it is madness to run with checksums disabled
> and we need to investigate more efficient checksum algorithms. The current
> crc32 algorithm may be too heavy weight but the simple XOR algorithm you
> propose I fear is not strong enough. I've seen to many cases now of various
> network components corrupting data in all sorts of interesting ways.
> Happily we have a lot of other choices for algorithms to investigate.
>
> If you have the time I'd encourage you to investigate an assortment of
> algorithms and see which work best. Making this a runtime option via
> proc I think is also an excellent idea.
>
>
> ------------------------------------------------------------------------
>
> _______________________________________________
> Lustre-devel mailing list
> Lustre-devel at clusterfs.com
> https://mail.clusterfs.com/mailman/listinfo/lustre-devel
>
More information about the lustre-devel
mailing list