[lustre-discuss] Lustre on Ceph Block Devices

[Brock Palen]

If we do test this I'll let you know how it works.

Why Lustre on GPFS?  Why not just run GPFS then given it support byte range
locking / MPI-IO and POSIX (Ignore license costs).

I'm trying to limit the number of disk systems to maintain in a system of
modest size where both MPI-IO and Object is required.    I have dedicated
Lustre today for larger systems and they will stay that way.  Was just
curious if anyone tried this.


Brock Palen
www.umich.edu/~brockp
Director Advanced Research Computing - TS
XSEDE Campus Champion
brockp at umich.edu
(734)936-1985

On Wed, Feb 22, 2017 at 4:54 AM, Shinobu Kinjo <shinobu.kj at gmail.com> wrote:

> Yeah, that's interesting. But that does not really make sense to use
> Lustre. And should not be used for any computations.
>
> If anything goes wrong, troubleshooting would become nightmare.
>
> Have you ever thought of using Lustre on top of GPFS native client?
>
> Anyway if you are going to build Lustre on top of any RADOS client and run
> MPI jobs, please share results. I'm really really interested in them.
>
>
>
> On Wed, Feb 22, 2017 at 2:06 PM, Brian Andrus <toomuchit at gmail.com> wrote:
>
>> I had looked at it, but then, why?
>>
>> There is no benefit using object storage when you are putting lustre over
>> top. It would bog down. Supposedly you would want to use CephFS over the
>> ceph storage. It talks directly to rados.
>> If you are able to enunciate the rados block devices, you should also be
>> able to send them directly as block devices (iSCSI at least) so lustre is
>> able to manage where the data is stored and use it's optimizing. Otherwise
>> the data can't be optimized. Lustre would THINK it knows where it was, but
>> the rados crush map would have put it somewhere else.
>>
>> Just my 2cents.
>>
>> Brian
>> On 2/21/2017 3:08 PM, Brock Palen wrote:
>>
>> Has anyone ever ran Lustre OST's (and maybe MDT's)  on Ceph Radios Block
>> Devices?
>>
>> In theory this would work just like an SAN attached solution.  Has anyone
>> ever done it before?  I know we are seeing decent performance from RBD on
>> our system but I don't have a way to test lustre on it.
>>
>> I'm looking at a future system where Ceph and Lustre might be needed
>> (Object and High performance HPC) but also not a huge budget to have two
>> full disk stacks.  So an idea was to have lustre servers consume Ceph Block
>> devices, and that same cluster serves object requests.
>>
>> Thoughts or prior art?  This probably isn't that different than the Cloud
>> Formation script that uses EBS volumes if it works as intended.
>>
>> Thanks
>>
>> Brock Palen
>> www.umich.edu/~brockp <http://www.umich.edu/%7Ebrockp>
>> Director Advanced Research Computing - TS
>> XSEDE Campus Champion
>> brockp at umich.edu
>> (734)936-1985 <(734)%20936-1985>
>>
>>
>> _______________________________________________
>> lustre-discuss mailing listlustre-discuss at lists.lustre.orghttp://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
>>
>>
>>
>> _______________________________________________
>> lustre-discuss mailing list
>> lustre-discuss at lists.lustre.org
>> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-discuss-lustre.org/attachments/20170222/ee75ba08/attachment-0001.htm>