<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
</head>
<body dir="auto">
<style type="text/css" style="display:none;"><!-- P {margin-top:0;margin-bottom:0;} --></style>
<div id="divtagdefaultwrapper" style="font-size:12pt;color:#000000;font-family:Calibri,Helvetica,sans-serif;" dir="ltr">
<p>Yes, agreed. After digging in to it a bit, though, the crypto code proved harder to adjust/add to than I expected, so I've tentatively decided I'd like to piggyback off of what Anna and company do. At the very least, I would like their version of the compression
algorithm, so it seemed best to wait and see what form their code takes. </p>
<p><br>
</p>
<p>If it's easily tweakable to do what I'd like, then I will likely just do that. (If nothing else, I badly need to prototype this idea to see how workable it is before getting serious, so I will do that in the easiest way possible...) If their code isn't
easily adjustable to this, I plan to take their improved algorithm and go back to using the crypto infrastructure (effectively adding a new crypto mode).</p>
</div>
<hr style="display:inline-block;width:98%" tabindex="-1">
<div id="divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" style="font-size:11pt" color="#000000"><b>From:</b> Dilger, Andreas <andreas.dilger@intel.com><br>
<b>Sent:</b> Thursday, August 3, 2017 8:41:15 AM<br>
<b>To:</b> Patrick Farrell<br>
<b>Cc:</b> Anna Fuchs; Xiong, Jinshan; Matthew Ahrens; Zhuravlev, Alexey; lustre-devel<br>
<b>Subject:</b> Re: [lustre-devel] Design proposal for client-side compression</font>
<div> </div>
</div>
<div>
<div><span></span></div>
<div>
<div>Patrick,</div>
<div>If you consider over-the-wire compression only, you could probably treat it the same as a crypto algorithm that decompresses the data at the server?</div>
<div id="AppleMailSignature"><br>
</div>
<div id="AppleMailSignature">Unfortunately, I haven't looked at either the crypto or compression code closely, but one would think that they could share a fair amount of infrastructure. <br>
<br>
Cheers, Andreas</div>
<div><br>
On Jul 28, 2017, at 18:53, Patrick Farrell <<a href="mailto:paf@cray.com">paf@cray.com</a>> wrote:<br>
<br>
</div>
<blockquote type="cite">
<div>
<meta name="Generator" content="Microsoft Exchange Server">
<!-- converted from text --><style><!-- .EmailQuote { margin-left: 1pt; padding-left: 4pt; border-left: #800000 2px solid; } --></style>
<meta content="text/html; charset=UTF-8">
<div dir="ltr">
<div id="x_divtagdefaultwrapper" dir="ltr" style="font-size:12pt; color:#000000; font-family:Calibri,Helvetica,sans-serif">
<p>Ah, OK. Reading this, I understand now that your intention is to keep the data compressed on disk - I hadn't thought through the implications of that fully. There's obviously a lot of benefit from that.<br>
<br>
That said, it seems like it would be relatively straightforward to make a version of this that uncompressed the data on arrival at the server, simply unpacking that buffer before writing it to disk. (Straightforward, that is, once the actual compression/decompression
code is ready...)<br>
<br>
That obviously takes more CPU on the server side and does not reduce the space required, but...</p>
<p><br>
</p>
<p>If you don't mind, when you consider the performant version of the compression code to be ready for at least testing, I'd like to see the code so I can try out the on-the-wire-only compression idea. It might have significant benefits for a case of interest
to me, and if it worked well, it could (long term) probably coexist with the larger on-disk compression idea. (Since who knows if we'll ever implement the whole thing for ldiskfs.)</p>
<p><br>
</p>
<p>Thanks again for engaging with me on this.</p>
<p><br>
</p>
<p>- Patrick<br>
</p>
</div>
<hr tabindex="-1" style="display:inline-block; width:98%">
<div id="x_divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" color="#000000" style="font-size:11pt"><b>From:</b> Anna Fuchs <<a href="mailto:anna.fuchs@informatik.uni-hamburg.de">anna.fuchs@informatik.uni-hamburg.de</a>><br>
<b>Sent:</b> Friday, July 28, 2017 10:12:16 AM<br>
<b>To:</b> Patrick Farrell; Xiong, Jinshan<br>
<b>Cc:</b> Matthew Ahrens; Zhuravlev, Alexey; lustre-devel<br>
<b>Subject:</b> Re: [lustre-devel] Design proposal for client-side compression</font>
<div> </div>
</div>
</div>
<font size="2"><span style="font-size:10pt;">
<div class="PlainText"><br>
> Ah. As it turns out, much more complicated than I anticipated.<br>
> Thanks for explaining...<br>
> <br>
> I have no expertise in compression algorithms, so that I will have to<br>
> just watch from the sidelines. Good luck.<br>
> <br>
> When you are further along, I remain interested in helping out with<br>
> the Lustre side of things.<br>
> <br>
> One more question - Do you have a plan to make this work *without*<br>
> the ZFS integration as well, for those using ldiskfs? That seems<br>
> straightforward enough - compress/decompress at send and recieve time<br>
> - even if the benefits would be smaller, but not everyone (Cray,<br>
> f.x.) is using ZFS, so I'm very interested in something that would<br>
> help ldiskfs as well. (Which is not to say don't do the deeper<br>
> integration with ZFS. Just that we'd like something available for<br>
> ldiskfs too.)<br>
<br>
I fear it is also much more complicated :)<br>
<br>
At the very beginning of the project proposal we hoped we wouldn't need<br>
to touch the server so much. It turned out wrong, moreover we have to<br>
modify not only the Lustre server, but also pretty much the backend<br>
itself. We chose ZFS since it already provides a lot of infrastructure<br>
that we would need to implement completely new in ldiskfs. Since, at<br>
least for me, it is a research project, ldiskfs is out of scope. Once<br>
we proved the concept, one could re-implement the whole compression<br>
stack for ldiskfs. So it is not impossible, but not our focus for this<br>
project. <br>
<br>
Nevertheless we tried to keep our changes as far as possible not very<br>
backend specific. For example we need some additional information to be<br>
stored per compressed chunk. One possibility would be to change the<br>
block pointer of ZFS and add those fields, but I don't think anyone<br>
except of us would like the BP to be modified :) So we decided to store<br>
them as a header for every chunk. For ldiskfs, since one would need to<br>
implement everything from scratch anyway, one might not need that<br>
header, but take the required fields into account from the beginning<br>
and add them to ldiskfs' "block pointer". For that reason, we wanted to<br>
leave the compressed data "headerless" on client-side, and add the<br>
header only on the server side if the corresponding backend requires<br>
it. <br>
<br>
Well, we did it, and it even works sometimes, but it looks horrible <br>
and is really counterintuitive. We send less data from client than<br>
lands on the OST, recalculate offsets, since we add the header during<br>
receiving on server side, recalculate the sent and received sizes,<br>
shift buffers by offsets and so on. The only advantage of this approach<br>
is client's independence from backend. We decided the price is too<br>
high. So now, I will construct the chunk with the header just after<br>
compressing the data on client-side, get rid of all those offset stuff<br>
on the server. But ldiskfs will have to deal with that ZFS-motivated<br>
details. <br>
<br>
However, a light version of compression could work with smaller changes<br>
to ldiskfs, when we only allow a completely compressed or not<br>
compressed files and allow potential performance drops for broken read-<br>
ahead (due to gaps within the data). <br>
<br>
Hope it is somehow more clear now.<br>
<br>
Regards,<br>
Anna<br>
<br>
</div>
</span></font></div>
</blockquote>
<blockquote type="cite">
<div><span>_______________________________________________</span><br>
<span>lustre-devel mailing list</span><br>
<span><a href="mailto:lustre-devel@lists.lustre.org">lustre-devel@lists.lustre.org</a></span><br>
<span><a href="http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org">http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org</a></span><br>
</div>
</blockquote>
</div>
</div>
</body>
</html>