<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<meta name="Generator" content="Microsoft Exchange Server">
<!-- converted from text --><style><!-- .EmailQuote { margin-left: 1pt; padding-left: 4pt; border-left: #800000 2px solid; } --></style>
</head>
<body>
<meta content="text/html; charset=UTF-8">
<style type="text/css" style="">
<!--
p
{margin-top:0;
margin-bottom:0}
-->
</style>
<div dir="ltr">
<div id="x_divtagdefaultwrapper" dir="ltr" style="font-size:12pt; color:#000000; font-family:Calibri,Helvetica,sans-serif">
<p>Ah, OK. Reading this, I understand now that your intention is to keep the data compressed on disk - I hadn't thought through the implications of that fully. There's obviously a lot of benefit from that.<br>
<br>
That said, it seems like it would be relatively straightforward to make a version of this that uncompressed the data on arrival at the server, simply unpacking that buffer before writing it to disk. (Straightforward, that is, once the actual compression/decompression
code is ready...)<br>
<br>
That obviously takes more CPU on the server side and does not reduce the space required, but...</p>
<p><br>
</p>
<p>If you don't mind, when you consider the performant version of the compression code to be ready for at least testing, I'd like to see the code so I can try out the on-the-wire-only compression idea. It might have significant benefits for a case of interest
to me, and if it worked well, it could (long term) probably coexist with the larger on-disk compression idea. (Since who knows if we'll ever implement the whole thing for ldiskfs.)</p>
<p><br>
</p>
<p>Thanks again for engaging with me on this.</p>
<p><br>
</p>
<p>- Patrick<br>
</p>
</div>
<hr tabindex="-1" style="display:inline-block; width:98%">
<div id="x_divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" color="#000000" style="font-size:11pt"><b>From:</b> Anna Fuchs <anna.fuchs@informatik.uni-hamburg.de><br>
<b>Sent:</b> Friday, July 28, 2017 10:12:16 AM<br>
<b>To:</b> Patrick Farrell; Xiong, Jinshan<br>
<b>Cc:</b> Matthew Ahrens; Zhuravlev, Alexey; lustre-devel<br>
<b>Subject:</b> Re: [lustre-devel] Design proposal for client-side compression</font>
<div> </div>
</div>
</div>
<font size="2"><span style="font-size:10pt;">
<div class="PlainText"><br>
> Ah. As it turns out, much more complicated than I anticipated.<br>
> Thanks for explaining...<br>
> <br>
> I have no expertise in compression algorithms, so that I will have to<br>
> just watch from the sidelines. Good luck.<br>
> <br>
> When you are further along, I remain interested in helping out with<br>
> the Lustre side of things.<br>
> <br>
> One more question - Do you have a plan to make this work *without*<br>
> the ZFS integration as well, for those using ldiskfs? That seems<br>
> straightforward enough - compress/decompress at send and recieve time<br>
> - even if the benefits would be smaller, but not everyone (Cray,<br>
> f.x.) is using ZFS, so I'm very interested in something that would<br>
> help ldiskfs as well. (Which is not to say don't do the deeper<br>
> integration with ZFS. Just that we'd like something available for<br>
> ldiskfs too.)<br>
<br>
I fear it is also much more complicated :)<br>
<br>
At the very beginning of the project proposal we hoped we wouldn't need<br>
to touch the server so much. It turned out wrong, moreover we have to<br>
modify not only the Lustre server, but also pretty much the backend<br>
itself. We chose ZFS since it already provides a lot of infrastructure<br>
that we would need to implement completely new in ldiskfs. Since, at<br>
least for me, it is a research project, ldiskfs is out of scope. Once<br>
we proved the concept, one could re-implement the whole compression<br>
stack for ldiskfs. So it is not impossible, but not our focus for this<br>
project. <br>
<br>
Nevertheless we tried to keep our changes as far as possible not very<br>
backend specific. For example we need some additional information to be<br>
stored per compressed chunk. One possibility would be to change the<br>
block pointer of ZFS and add those fields, but I don't think anyone<br>
except of us would like the BP to be modified :) So we decided to store<br>
them as a header for every chunk. For ldiskfs, since one would need to<br>
implement everything from scratch anyway, one might not need that<br>
header, but take the required fields into account from the beginning<br>
and add them to ldiskfs' "block pointer". For that reason, we wanted to<br>
leave the compressed data "headerless" on client-side, and add the<br>
header only on the server side if the corresponding backend requires<br>
it. <br>
<br>
Well, we did it, and it even works sometimes, but it looks horrible <br>
and is really counterintuitive. We send less data from client than<br>
lands on the OST, recalculate offsets, since we add the header during<br>
receiving on server side, recalculate the sent and received sizes,<br>
shift buffers by offsets and so on. The only advantage of this approach<br>
is client's independence from backend. We decided the price is too<br>
high. So now, I will construct the chunk with the header just after<br>
compressing the data on client-side, get rid of all those offset stuff<br>
on the server. But ldiskfs will have to deal with that ZFS-motivated<br>
details. <br>
<br>
However, a light version of compression could work with smaller changes<br>
to ldiskfs, when we only allow a completely compressed or not<br>
compressed files and allow potential performance drops for broken read-<br>
ahead (due to gaps within the data). <br>
<br>
Hope it is somehow more clear now.<br>
<br>
Regards,<br>
Anna<br>
<br>
</div>
</span></font>
</body>
</html>