[lustre-discuss] can not get the hsm_release command to work with Lustre 2.7 and Oracle HSM 6.1
Read, Robert
robert.read at intel.com
Thu Jun 23 12:54:50 PDT 2016
Hi
See inline…
On Jun 23, 2016, at 11:57 , Michael Skiba <michael.skiba at oracle.com<mailto:michael.skiba at oracle.com>> wrote:
Colin, The first file was a data file so; I made another file that was a txt file named test. Archived it then changed the file and here is the output below. The copytool daemon are different this time and are complaining about (cannot get path of FID and cannot set attributes)
[root at isr-x4150-01 mnt]# lfs hsm_archive --archive=1 /mnt/test
[root at isr-x4150-01 mnt]# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/sda2 134G 7.0G 121G 6% /
tmpfs 3.9G 0 3.9G 0% /dev/shm
/dev/sda1 477M 105M 348M 24% /boot
10.80.191.134 at tcp0:/lustre
136G 9.6G 120G 8% /mnt
10.80.191.161:/samqfs1
558G 9.6G 549G 2% /samqfs1
[root at isr-x4150-01 mnt]# lfs hsm_state /mnt/test
/mnt/test: (0x00000001) exists, archive_id:1
[root at isr-x4150-01 mnt]# vi test
[root at isr-x4150-01 mnt]# lfs hsm_state /mnt/test
/mnt/test: (0x00000000) (does this mean its dirty?)
It appears that when vi saved the modified file, it renamed the original file and created a new file for the updates. Since this is a new file, it has no HSM state. You can verify this by checking the inode (or FID) of the file before and after using vi.
The expected behavior for modifying a file with “exists” flag is the “dirty” flag will be set, whether or not it has been completely archived. If an archive is currently in progress when the file is modified, then the copytool will receive an error when to competes the archive request.
A simple way to modify a file for test purposes is to do an append, such as “date >> /mnt/test”.
robert
[root at isr-x4150-01 mnt]#
The logs
1466707271.922641 lhsmtool_posix[14489]: copytool fs=lustre archive#=1 item_count=1
1466707271.922696 lhsmtool_posix[14489]: waiting for message from kernel
1466707271.922717 lhsmtool_posix[8023]: '[0x200000400:0x16:0x0]' action ARCHIVE reclen 72, cookie=0x576ab8e1
ioctl err -19: No such device (19)
1466707271.922850 lhsmtool_posix[8023]: cannot get path of FID [0x200000400:0x16:0x0]: No such device (19)
1466707271.924618 lhsmtool_posix[8023]: archiving 'mnt/.lustre/fid/0x200000400:0x16:0x0' to 'samqfs1/0016/0000/0400/0000/0002/0000/0x200000400:0x16:0x0_tmp'
1466707271.939245 lhsmtool_posix[8023]: saving stripe info of 'mnt/.lustre/fid/0x200000400:0x16:0x0' in samqfs1/0016/0000/0400/0000/0002/0000/0x200000400:0x16:0x0_tmp.lov
1466707271.941082 lhsmtool_posix[8023]: start copy of 973891 bytes from 'mnt/.lustre/fid/0x200000400:0x16:0x0' to 'samqfs1/0016/0000/0400/0000/0002/0000/0x200000400:0x16:0x0_tmp'
1466707271.955555 lhsmtool_posix[8023]: copied 973891 bytes in 0.015133 seconds
1466707271.972955 lhsmtool_posix[8023]: data archiving for 'mnt/.lustre/fid/0x200000400:0x16:0x0' to 'samqfs1/0016/0000/0400/0000/0002/0000/0x200000400:0x16:0x0_tmp' done
1466707271.973479 lhsmtool_posix[8023]: cannot set attributes of 'mnt/.lustre/fid/0x200000400:0x16:0x0': Operation not permitted (1)
1466707271.973499 lhsmtool_posix[8023]: cannot copy attr of 'mnt/.lustre/fid/0x200000400:0x16:0x0' to 'samqfs1/0016/0000/0400/0000/0002/0000/0x200000400:0x16:0x0_tmp': Operation not permitted (1)
1466707271.973509 lhsmtool_posix[8023]: attr file for 'mnt/.lustre/fid/0x200000400:0x16:0x0' saved to archive 'samqfs1/0016/0000/0400/0000/0002/0000/0x200000400:0x16:0x0_tmp'
1466707271.973869 lhsmtool_posix[8023]: fsetxattr of 'trusted.hsm' on 'samqfs1/0016/0000/0400/0000/0002/0000/0x200000400:0x16:0x0_tmp' rc=-1 (Operation not supported)
1466707271.973885 lhsmtool_posix[8023]: fsetxattr of 'trusted.link' on 'samqfs1/0016/0000/0400/0000/0002/0000/0x200000400:0x16:0x0_tmp' rc=-1 (Operation not supported)
1466707271.973906 lhsmtool_posix[8023]: fsetxattr of 'trusted.lov' on 'samqfs1/0016/0000/0400/0000/0002/0000/0x200000400:0x16:0x0_tmp' rc=-1 (Operation not supported)
1466707271.973919 lhsmtool_posix[8023]: fsetxattr of 'trusted.lma' on 'samqfs1/0016/0000/0400/0000/0002/0000/0x200000400:0x16:0x0_tmp' rc=-1 (Operation not supported)
1466707271.974100 lhsmtool_posix[8023]: fsetxattr of 'lustre.lov' on 'samqfs1/0016/0000/0400/0000/0002/0000/0x200000400:0x16:0x0_tmp' rc=-1 (Operation not supported)
1466707271.974112 lhsmtool_posix[8023]: xattr file for 'mnt/.lustre/fid/0x200000400:0x16:0x0' saved to archive 'samqfs1/0016/0000/0400/0000/0002/0000/0x200000400:0x16:0x0_tmp'
ioctl err -19: No such device (19)
1466707271.975415 lhsmtool_posix[8023]: cannot get FID of '[0x200000400:0x16:0x0]': No such device (19)
1466707271.975848 lhsmtool_posix[8023]: Action completed, notifying coordinator cookie=0x576ab8e1, FID=[0x200000400:0x16:0x0], hp_flags=0 err=1
1466707271.976507 lhsmtool_posix[8023]: llapi_hsm_action_end() on 'mnt/.lustre/fid/0x200000400:0x16:0x0' ok (rc=0)
<image001.gif><http://www.oracle.com/>
Michael J. Skiba | Principal Support Engineer
Phone: +1 3032729724<tel:+1%203032729724> | Mobile: +1 3036198495<tel:+1%203036198495>
Oracle ISV
500 Elorado Blv. Bldg. 5
Broomfield, Colorado 80021
<image002.gif><http://www.oracle.com/commitment>
Oracle is committed to developing practices and products that help protect the environment
From: Colin Faber [mailto:colin.faber at seagate.com]
Sent: Thursday, June 23, 2016 10:46 AM
To: Michael Skiba
Cc: Lustre discussion
Subject: Re: [lustre-discuss] can not get the hsm_release command to work with Lustre 2.7 and Oracle HSM 6.1
After the archive, does the state change to exists, archived ? If you modify the file while in the exists state, does it mark it as dirty or does it remain the same?
On Thu, Jun 23, 2016 at 10:13 AM, Michael Skiba <michael.skiba at oracle.com<mailto:michael.skiba at oracle.com>> wrote:
Colin here is the output from the POSIX copytool daemon. It looks like it was successful shown below. The file is out on the HSM SAMQFS file system.
1466633273.355200 lhsmtool_posix[16036]: archiving 'mnt/.lustre/fid/0x200000400:0x7:0x0' to 'samqfs1/0007/0000/0400/0000/0002/0000/0x200000400:0x7:0x0_tmp'
1466633273.366577 lhsmtool_posix[16036]: saving stripe info of 'mnt/.lustre/fid/0x200000400:0x7:0x0' in samqfs1/0007/0000/0400/0000/0002/0000/0x200000400:0x7:0x0_tmp.lov
1466633273.368356 lhsmtool_posix[16036]: start copy of 10240000000 bytes from 'mnt/.lustre/fid/0x200000400:0x7:0x0' to 'samqfs1/0007/0000/0400/0000/0002/0000/0x200000400:0x7:0x0_tmp'
1466633303.002966 lhsmtool_posix[16036]: %18
1466633333.011802 lhsmtool_posix[16036]: %39
1466633363.011127 lhsmtool_posix[16036]: %67
1466633393.001740 lhsmtool_posix[16036]: %90
1466633406.858331 lhsmtool_posix[16036]: copied 10240000000 bytes in 133.490582 seconds
1466633409.914213 lhsmtool_posix[16036]: data archiving for 'mnt/.lustre/fid/0x200000400:0x7:0x0' to 'samqfs1/0007/0000/0400/0000/0002/0000/0x200000400:0x7:0x0_tmp' done
Here is the hsm release output.
1466633409.914854 lhsmtool_posix[16036]: cannot set attributes of 'mnt/.lustre/fid/0x200000400:0x7:0x0': Operation not permitted (1)
Is there a step I am missing
You can see that it did archive the data over to the HSM server by df –h you see 9.6G on /mnt and /samqfs1.
[root at isr-x4150-01 mnt]# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/sda2 134G 7.0G 121G 6% /
tmpfs 3.9G 0 3.9G 0% /dev/shm
/dev/sda1 477M 105M 348M 24% /boot
10.80.191.134 at tcp0:/lustre<mailto:10.80.191.134 at tcp0:/lustre>
136G 9.6G 120G 8% /mnt
10.80.191.161:/samqfs1
558G 9.6G 549G 2% /samqfs1
Hsm server
Filesystem Size Used Available Capacity Mounted on
rpool/ROOT/solaris-2 547G 5.6G 531G 2% /
/devices 0K 0K 0K 0% /devices
/dev 0K 0K 0K 0% /dev
ctfs 0K 0K 0K 0% /system/contract
proc 0K 0K 0K 0% /proc
mnttab 0K 0K 0K 0% /etc/mnttab
swap 10G 1.8M 10G 1% /system/volatile
objfs 0K 0K 0K 0% /system/object
sharefs 0K 0K 0K 0% /etc/dfs/sharetab
/usr/lib/libc/libc_hwcap1.so.1
537G 5.6G 531G 2% /lib/libc.so.1
fd 0K 0K 0K 0% /dev/fd
rpool/ROOT/solaris-2/var
547G 422M 531G 1% /var
swap 10G 12K 10G 1% /tmp
rpool/VARSHARE 547G 2.6M 531G 1% /var/share
samqfs1 558G 9.6G 548G 2% /samqfs1
rpool/export 547G 32K 531G 1% /export
rpool/export/home 547G 32K 531G 1% /export/home
rpool/export/home/jack
547G 32K 531G 1% /export/home/jack
rpool 547G 4.5M 531G 1% /rpool
rpool/VARSHARE/zones 547G 31K 531G 1% /system/zones
rpool/VARSHARE/pkg 547G 32K 531G 1% /var/share/pkg
rpool/VARSHARE/pkg/repositories
547G 31K 531G 1% /var/share/pkg/repositories
Thanks for your help………….
<image001.gif><https://urldefense.proofpoint.com/v2/url?u=http-3A__www.oracle.com_&d=CwMFaQ&c=IGDlg0lD0b-nebmJJ0Kp8A&r=x9pM59OqndbWw-lPPdr8w1Vud29EZigcxcNkz0uw5oQ&m=kcNjUK8DTn7_E5X3DctNYJpJ50JAjLYx11vy1qHclHU&s=LtEDPn9dmCZdGnpebZnAXZFvzsex1SXXNSk_59GwDVM&e=>
Michael J. Skiba | Principal Support Engineer
Phone: +1 3032729724<tel:+1%203032729724> | Mobile: +1 3036198495<tel:+1%203036198495>
Oracle ISV
500 Elorado Blv. Bldg. 5
Broomfield, Colorado 80021
<image002.gif><https://urldefense.proofpoint.com/v2/url?u=http-3A__www.oracle.com_commitment&d=CwMFaQ&c=IGDlg0lD0b-nebmJJ0Kp8A&r=x9pM59OqndbWw-lPPdr8w1Vud29EZigcxcNkz0uw5oQ&m=kcNjUK8DTn7_E5X3DctNYJpJ50JAjLYx11vy1qHclHU&s=Czg4XODZN2DaUmF1zgpBSFKqlqOllkn6IpAkHPXsRv4&e=>
Oracle is committed to developing practices and products that help protect the environment
From: Colin Faber [mailto:colin.faber at seagate.com<mailto:colin.faber at seagate.com>]
Sent: Thursday, June 23, 2016 9:43 AM
To: Michael Skiba
Cc: Lustre discussion
Subject: Re: [lustre-discuss] can not get the hsm_release command to work with Lustre 2.7 and Oracle HSM 6.1
Hi Michael,
Does the backend copytool actually archive the file and notify lustre of the action? Based on your current state it only appears that the archive request was made and that it has not yet successfully archived.
In such a case release would be disallowed as it would amount to a data purge and stub out to a non-existent file.
-cf
On Thu, Jun 23, 2016 at 9:27 AM, Michael Skiba <michael.skiba at oracle.com<mailto:michael.skiba at oracle.com>> wrote:
To all,
I am running a basic Lustre configuration 2.7 with Red Hat 6.6 and an Oracle HSM 6.1 system. The Lustre file system is running properly and works with the HSM samqfs File system. I can manually (nfs ) cp files back and forth to the Lustre and HSM file systems without a problem. I can run a Lustre hsm_archive with no problems. When I try to request the hsm_release command it fails. I am trying to get the 5 request types to work stated in the manual on pg168 Chapter 22.4. Here are the steps below that I ran into on the Lustre Client. The Lustre Client mount point is /mnt and the HSM mount point is /samqfs1.
Any help would be most appreciated
Her is the archive complete.
[root at isr-x4150-01 ~]# lfs hsm_archive --archive=1 /mnt/0.out
[root at isr-x4150-01 ~]# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/sda2 134G 7.0G 121G 6% /
tmpfs 3.9G 0 3.9G 0% /dev/shm
/dev/sda1 477M 105M 348M 24% /boot
10.80.191.134 at tcp0:/lustre<mailto:10.80.191.134 at tcp0:/lustre>
136G 9.6G 120G 8% /mnt
10.80.191.161:/samqfs1
558G 11G 548G 2% /samqfs1
[root at isr-x4150-01 ~]# lfs hsm_state /mnt/0.out
/mnt/0.out: (0x00000001) exists, archive_id:1
Here is the hsm_release request followed by the error.
[root at isr-x4150-01 ~]# lfs hsm_release /mnt/0.out
Cannot send HSM request (use of /mnt/0.out): Operation not permitted
<image001.gif><https://urldefense.proofpoint.com/v2/url?u=http-3A__www.oracle.com_&d=CwMFAg&c=IGDlg0lD0b-nebmJJ0Kp8A&r=x9pM59OqndbWw-lPPdr8w1Vud29EZigcxcNkz0uw5oQ&m=hqsFWrKMaCUojiNflON-5wKRUoA1GP7EmTneF0W26MY&s=dyIsPVSVNQVZjQxnohvrEphS-XPbhyKHhs9wGtTLp1Y&e=>
Michael J. Skiba | Principal Support Engineer
Phone: +1 3032729724<tel:+1%203032729724> | Mobile: +1 3036198495<tel:+1%203036198495>
Oracle ISV
500 Elorado Blv. Bldg. 5
Broomfield, Colorado 80021
<image002.gif><https://urldefense.proofpoint.com/v2/url?u=http-3A__www.oracle.com_commitment&d=CwMFAg&c=IGDlg0lD0b-nebmJJ0Kp8A&r=x9pM59OqndbWw-lPPdr8w1Vud29EZigcxcNkz0uw5oQ&m=hqsFWrKMaCUojiNflON-5wKRUoA1GP7EmTneF0W26MY&s=3UnZI3jBog33bwWP30Ll64fqdqVRB_r6sea6OwP06ZQ&e=>
Oracle is committed to developing practices and products that help protect the environment
_______________________________________________
lustre-discuss mailing list
lustre-discuss at lists.lustre.org<mailto:lustre-discuss at lists.lustre.org>
https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.lustre.org_listinfo.cgi_lustre-2Ddiscuss-2Dlustre.org&d=CwICAg&c=IGDlg0lD0b-nebmJJ0Kp8A&r=x9pM59OqndbWw-lPPdr8w1Vud29EZigcxcNkz0uw5oQ&m=hqsFWrKMaCUojiNflON-5wKRUoA1GP7EmTneF0W26MY&s=tRDEocIS6lnYVSJnhfp9MdNAup4GFdSbGjOogFb9m1U&e=
_______________________________________________
lustre-discuss mailing list
lustre-discuss at lists.lustre.org<mailto:lustre-discuss at lists.lustre.org>
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-discuss-lustre.org/attachments/20160623/e8cb7498/attachment-0001.htm>
More information about the lustre-discuss
mailing list