[lustre-discuss] status of HSM copytools?

[Simon Guilbault]

Hi

The Compute Canada S3/Ceph copytool was never deployed in production since
S3 backed storage was never installed.
We moved to a tape based system with TSM instead and have been running that
copytool on 3 different sites for the past year.
https://github.com/guilbaults/ct_tsm

On Sun, Aug 23, 2020 at 9:34 AM Matt Rásó-Barnett <matt at rasobarnett.com>
wrote:

> Hi Andrew,
>
> We are using Lemur in production at Cambridge with our Lustre HSM setup,
> although in our case we are using the POSIX backend of it rather than
> the S3 backend.
>
> We adopted it early at the time it was being developed under Intel, as
> the prospect of commercial support (we were Intel IEEL customers at the
> time), and some of the promising features that were being worked on with
> it (snapshot versions, checksums, performance), was very enticing.
>
> But as you've seen, it was a casualty of Intel dropping Lustre, so it's
> not being developed any more. We unfortunately don't have the skills or
> resources internally to develop it ourselves, so I personally am looking
> at how we will transition away from it in the coming months as we
> refresh our HSM setup. I may switch back to using the robinhood
> developed lhsmtool_cmd with our POSIX backends for example.
>
> Lemur is largely doing the job for us however, so it reached a decent
> stage of development before Intel dropped it, but I still wouldn't
> recommend starting with it without someone backing maintenance of it.
>
>  From speaking with the Amazon representative at last year's LUG, I
> understood that they are using Lemur internally for their HSM
> integration as well. I had meant to follow up with them to see if they
> would consider open-sourcing their fork of the project as to my
> knowledge there aren't any other major users of Lemur.
>
> I'd also be interested as you are in hearing what others are using,
> commericial or otherwise. I've never had the feeling that HSM is
> particularly widely used so it'd be good to share information for those
> of us who are!
>
> Best wishes,
>
> Matt
>
> On Sat, Aug 22, 2020 at 10:54:23PM +1000, Andrew Elwell wrote:
> >Hi folks,
> >I'm looking round to see what's current / 'supported' / working in the
> >state of copytools. Ideally one that can migrate to/from object stores
> >(Ceph or S3). The github repo for Lemur
> >(https://github.com/whamcloud/lemur/commits/master) doesn't seem to
> >have had any substantial work since it left Intel - unlucky timing
> >with the owner shift?
> >I've seen another from Compute Canada
> >(https://github.com/ComputeCanada/lustre-obj-copytool) but that too
> >hasn't been touched for years.
> >
> >Anyone care to comment on some working ones? Horror stories? Ones to
> avoid?
> >hey, I'm even (I'll probably regret this) open to _email_ from
> >salesdroids if you have a working product and can point me to some
> >users (but don't try and phone me or make me sit through a webinar).
> >
> >Many thanks
> >
> >
> >Andrew
> >_______________________________________________
> >lustre-discuss mailing list
> >lustre-discuss at lists.lustre.org
> >http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
> _______________________________________________
> lustre-discuss mailing list
> lustre-discuss at lists.lustre.org
> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-discuss-lustre.org/attachments/20200824/3224143d/attachment.html>