[lustre-discuss] Hole Identification in file

[lokesh jaliminche]

Ahh got it!!
Meanwhile, I found one implicit method to find holes on the file i.e. using
fiemap ioctl.
I have tested it and works fine

Logs :
=====
dd if=/dev/urandom  of=sparse_file seek=10 bs=1M  count=1
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0969376 s, 10.8 MB/s
[lokesh]# dd if=/dev/urandom  of=sparse_file seek=20 bs=1M  count=1
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.087068 s, 12.0 MB/s
[lokesh]# dd if=/dev/urandom  of=sparse_file seek=30 bs=1M  count=1
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0918352 s, 11.4 MB/s
[lokesh]# dd if=/dev/urandom  of=sparse_file seek=40 bs=1M  count=10
^[[A^[[D10+0 records in
10+0 records out
10485760 bytes (10 MB) copied, 0.946796 s, 11.1 MB/s
[lokesh]# dd if=/dev/urandom  of=sparse_file seek=50 bs=1M  count=10
10+0 records in
10+0 records out
10485760 bytes (10 MB) copied, 0.978465 s, 10.7 MB/s
[lokesh]# ./a.out sparse_file
Extents in file "sparse_file":    4
Extents returned: 4
Logical: ###[10485760]    Ext length: ###[1048576]    Physical:
###[104666759168]    flags: 2147483648
Logical: ###[20971520]    Ext length: ###[1048576]    Physical:
###[104677244928]    flags: 2147483648
Logical: ###[31457280]    Ext length: ###[1048576]    Physical:
###[104687730688]    flags: 2147483648
Logical: ###[41943040]    Ext length: ###[20971520]    Physical:
###[104689827840]    flags: 2147483649

On Fri, Mar 9, 2018 at 2:12 PM, Dilger, Andreas <andreas.dilger at intel.com>
wrote:

> On Mar 6, 2018, at 00:12, lokesh jaliminche <lokesh.jaliminche at gmail.com>
> wrote:
> >
> > Hi,
> >
> > Does lustre support SEEK_HOLE/SEEK_DATA flag for lseek?
> >
> > I did some experiment with the below program to find out if lustre
> supports SEEK_HOLE and SEEK_DATA flag. I found that lustre always returns
> the end of the file for SEEK_HOLE and 0 for SEEK_DATA and as per the man
> page this is the simplest implementation that file system can have(If they
> dont want to support these flags). So just wanted to confirm.
>
> Lustre does not directly support the SEEK_HOLE/SEEK_DATA interface
> currently.
> What you are seeing is the default/minimum implementation provided by the
> lseek interface.
>
> Implementing these seek options might be quite complex for a Lustre file
> because of multiple stripes in the layout.  You would need to locate holes
> in the stripes, and then map the hole offset on the object to the file
> offset.
>
> If this is something you are interested to implement I would be happy to
> discuss it with you further.  Probably the best place to start is to file
> an LU ticket with details, and then interested parties can discuss the
> implementation there.
>
> Cheers, Andreas
>
> > Program :
> > ========
> > #include <unistd.h>
> > #include <sys/types.h>
> > #include <stdio.h>
> > #include <fcntl.h>
> > #include <stdlib.h>
> > #define SEEK_OFF 10485760
> > int main()
> > {
> >     int fd;
> >     char buffer[80];
> >     int i = 0;
> >     static char message[]="Hello world";
> >     fd=open("myfile", O_RDWR);
> >     if (fd != -1)
> >         return 1;
> >     printf("creating hole by writing at each of %d strides\n", SEEK_OFF);
> >     for (i = 1; i < 10; i++)
> >     {
> >         int seek_off = i * SEEK_OFF;
> >         int sz;
> >         printf("seek_off %ld\n", lseek(fd, seek_off, SEEK_SET));
> >         sz = write(fd,message,sizeof(message));
> >         printf("write size = %d\n", sz);
> >         printf("String : %s\n", message);
> >     }
> >     printf("Demonstrating SEEK_HOLE and SEEK_DATA %d \n", SEEK_OFF);
> >     int start_off = 0;
> >     lseek(fd, 0, SEEK_SET);
> >     printf("after SEEK_HOLE start_off %ld\n", lseek(fd, 0, SEEK_HOLE));
> >     printf("after SEEK_DATA start_off %ld\n", lseek(fd, start_off,
> SEEK_DATA));
> >     printf("after SEEK_HOLE start_off %ld\n", lseek(fd, 10485760,
> SEEK_HOLE));
> >     printf("after SEEK_DATA start_off %ld\n", lseek(fd, (10485760 *2 ),
> SEEK_DATA));
> >     close(fd);
> > }
> >
> > output:
> > =====
> > after SEEK_HOLE start_off 94372142
> > after SEEK_DATA start_off 0
> > after SEEK_HOLE start_off 94372142
> > after SEEK_DATA start_off 0
> >
> > Regards,
> > Lokesh
> > _______________________________________________
> > lustre-discuss mailing list
> > lustre-discuss at lists.lustre.org
> > http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
>
> Cheers, Andreas
> --
> Andreas Dilger
> Lustre Principal Architect
> Intel Corporation
>
>
>
>
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-discuss-lustre.org/attachments/20180309/3b546df4/attachment-0001.html>