[lustre-discuss] Why certain commands should not be used on Lustre file system

[Dilger, Andreas]

On 2016/02/09, 21:16, "lustre-discuss on behalf of Prakrati.Agrawal at shell.com<mailto:Prakrati.Agrawal at shell.com>" <lustre-discuss-bounces at lists.lustre.org<mailto:lustre-discuss-bounces at lists.lustre.org> on behalf of Prakrati.Agrawal at shell.com<mailto:Prakrati.Agrawal at shell.com>> wrote:

I read on Lustre best practices that ls –U should be used instead of ls –l . I understand that ls –l makes MDS contact all OSS to get all information about all files and hence loads it. But, what does ls –U do to avoid it?

       -U     do not sort; list entries in directory order

This is more important for very large directories, since "ls" will read all of the entries and stat them before printing anything.  That said, GNU ls will still read all of the entries before printing them, so for very large directories "find <directory> -ls" is a lot faster to start printing entries.

Also, it is said that rm-rf * should not be used. Please can someone explain the reason for that.

It is also said that instead lfs find  <directory path> --type f -print0 | xargs -0 rm –f should be used. Please explain the reason for this also.

"rm -rf *" will expand "*" onto the command line (done by bash) and if there are too many files in the directory (more than about 8MB IIRC) then bash will fail to execute the command.  Running "lfs find" (or just plain "find") will only print the filenames onto the output and xargs will process them in chunks that fit onto a command-line.

Cheers, Andreas
--
Andreas Dilger
Lustre Principal Architect
Intel High Performance Data Division