[lustre-discuss] problem loading zfs properties for a lustre/zfs fs

Tue Jul 12 14:25:42 PDT 2016

a dracut -f should be sufficient to rebuild the initramfs and I did it...
I will force it to work putting hte option in the zfs.conf inside initramfs

On 12/07/16 13:53, Faaland, Olaf P. wrote:
> You don't need to modify the initramfs as you describe, although that would likely work.
>
> It should be sufficient to rebuild the initramfs.  It's built on your system, for example after a kernel installation.  Most likely, at the time the initramfs was last built on your system, the zfs-dracut package was installed, and that caused the initramfs file to be built with zfs.  Rebuilding now should remove the zfs module.
>
> Redhat has an article that says how to rebuild the initramfs:
> https://access.redhat.com/solutions/1958
>
> I can't see their instructions, as I don't have an account for their portal.  I suspect it just comes down to running mkinitrd.  Or you can reinstall your kernel and let it rebuild the initramfs as a side-affect:
>
> rpm -i --replacepkgs /path/to/your/kernel/rpm
>
> To answer your question about why this is necessary: the kernel first boots and loads some drivers etc., using the initramfs, before your normal root filesystem is mounted.  During this first boot stage, your hand-edited zfs.conf is not present (it's in the root filesystem that hasn't yet been mounted).  Therefore the zfs module does not get those options passed to it during load.
>
> Olaf P. Faaland
> Livermore Computing
>
> ________________________________________
> From: Riccardo Veraldi [Riccardo.Veraldi at cnaf.infn.it]
> Sent: Tuesday, July 12, 2016 1:03 PM
> To: Faaland, Olaf P.; lustre-discuss at lists.lustre.org
> Subject: Re: [lustre-discuss] problem loading zfs properties for a lustre/zfs fs
>
> Hello,
>
> lsinitrd | grep zfs
> -rwxr--r--   1 root     root       493080 Jul 12 12:56
> usr/lib/modules/3.10.0-327.18.2.el7.x86_64/extra/kernel/fs/lustre/osd_zfs.ko
> -rw-r--r--   1 root     root      2245128 Jul 12 12:56
> usr/lib/modules/3.10.0-327.18.2.el7.x86_64/extra/zfs.ko
> -rw-r--r--   1 root     root            4 May 12 20:18
> usr/lib/modules-load.d/zfs.conf
>
>
> if I do
>
> lsinitrd -f usr/lib/modules-load.d/zfs.conf
> zfs
>
> so looks like zfs is loaded by initramfs
>
> do I have to add a etc/modprobe.d/zfs.conf inside the initramfs ?
>
> I mean /etc/modprobe.d/zfs.conf  from the root partition should be
> loaded anyway.
>
> thanks
>
>
>
>
>
>
> On 12/07/16 11:51, Faaland, Olaf P. wrote:
>> Actually, looks like you can use simpler syntax:
>>
>> lsinitrd | grep zfs
>>
>> Olaf P. Faaland
>> Livermore Computing
>>
>> ________________________________________
>> From: lustre-discuss [lustre-discuss-bounces at lists.lustre.org] on behalf of Faaland, Olaf P. [faaland1 at llnl.gov]
>> Sent: Tuesday, July 12, 2016 11:48 AM
>> To: Riccardo Veraldi; lustre-discuss at lists.lustre.org
>> Subject: Re: [lustre-discuss] problem loading zfs properties for a lustre/zfs fs
>>
>> Riccardo,
>>
>> Does your initramfs still include zfs-related files?  What is the output of
>>
>> lsinitrd /boot/init*$(uname -r)* | grep zfs
>>
>> Olaf P. Faaland
>> Livermore Computing
>>
>> ________________________________________
>> From: Riccardo Veraldi [Riccardo.Veraldi at cnaf.infn.it]
>> Sent: Monday, July 11, 2016 5:25 PM
>> To: Faaland, Olaf P.; lustre-discuss at lists.lustre.org
>> Subject: Re: [lustre-discuss] problem loading zfs properties for a lustre/zfs fs
>>
>> On 11/07/16 17:02, Faaland, Olaf P. wrote:
>>> Riccardo,
>>>
>>> If you are not booting from a zpool, you do not need the "zfs-dracut" package.  This package causes ZFS to be loaded very early in the boot process, most likely before your /etc/modprobe.d/zfs.conf files is visible to the kernel.
>>>
>>> As long as you are not booting from a zpool, remove that package (rpm -e) and boot again to see if your problem is fixed.
>> I did it I removed zfs-dracut since I do not boot from ZFS.
>>
>>     cat /etc/modprobe.d/zfs.conf
>> options zfs zfs_prefetch_disable=1
>> options zfs zfs_txg_history=120
>> options zfs metaslab_debug_unload=1
>>
>> after reboot
>>
>> cat /sys/module/zfs/parameters/zfs_prefetch_disable
>> 0
>>
>> cat /sys/module/zfs/parameters/zfs_txg_history
>> 0
>>
>> so it looks like the parameters inside zfs.conf are ignored
>>
>> ls -la /etc/modprobe.d/zfs.conf
>> -rw-r--r-- 1 root root 103 Jul 11 17:18 /etc/modprobe.d/zfs.conf
>>
>>
>>
>>> -Olaf
>>>
>>> ________________________________________
>>> From: Riccardo Veraldi [Riccardo.Veraldi at cnaf.infn.it]
>>> Sent: Monday, July 11, 2016 4:55 PM
>>> To: Faaland, Olaf P.; lustre-discuss at lists.lustre.org
>>> Subject: Re: [lustre-discuss] problem loading zfs properties for a lustre/zfs fs
>>>
>>> On 11/07/16 16:15, Faaland, Olaf P. wrote:
>>>> 1) What is the output of:
>>>>
>>>> rpm -qa | grep zfs
>>> libzfs2-0.6.5.7-1.el7.centos.x86_64
>>> zfs-dkms-0.6.5.7-1.el7.centos.noarch
>>> lustre-osd-zfs-mount-2.8.0-3.10.0_327.18.2.el7.x86_64.x86_64
>>> zfs-0.6.5.7-1.el7.centos.x86_64
>>> zfs-dracut-0.6.5.7-1.el7.centos.x86_64
>>> lustre-osd-zfs-2.8.0-3.10.0_327.18.2.el7.x86_64.x86_64
>>>> from that system after it boots?
>>>>
>>>> 2) How do those values get into the /etc/modprobe.d/zfs.conf file?  Are they there before the node boots, or are modifying that file somehow during the boot process?
>>> I edited those values myself inside zfs.conf then rebooted the system,
>>> but they are not loaded
>>>
>>> thank you
>>>
>>> Riccardo
>>>
>>>
>>>> Olaf P. Faaland
>>>> Livermore Computing
>>>> ________________________________________
>>>> From: lustre-discuss [lustre-discuss-bounces at lists.lustre.org] on behalf of Riccardo Veraldi [Riccardo.Veraldi at cnaf.infn.it]
>>>> Sent: Monday, July 11, 2016 3:53 PM
>>>> To: lustre-discuss at lists.lustre.org
>>>> Subject: [lustre-discuss] problem loading zfs properties for a lustre/zfs fs
>>>>
>>>> Hello,
>>>> I am tailoring my system for lustre on ZFS and I am not able to set
>>>> these parameters
>>>> writing the config file /etc/modprobe.d/zfs.conf with the following options
>>>>
>>>> options zfs zfs_prefetch_disable=1
>>>> options zfs zfs_txg_history=120
>>>> options zfs metaslab_debug:unload=1
>>>>
>>>> when I check the parameters after boot in /sys/module/zfs/parameters
>>>> they are still the defaults, and my modification are not taken.
>>>>
>>>> zfs.conf is being ignored at boot time I do not know why.
>>>>
>>>> I had to do a
>>>>
>>>> echo 1 > /sys/module/zfs/parameters/options zfs zfs_prefetch_disable
>>>>
>>>> inside rc.local
>>>>
>>>> to have it working
>>>>
>>>> I am running RHEL7 with latest kernel  and zfs 0.6.5.7-1
>>>>
>>>> any hints ?
>>>>
>>>> thank you
>>>>
>>>> Riccardo
>>>>
>>>> _______________________________________________
>>>> lustre-discuss mailing list
>>>> lustre-discuss at lists.lustre.org
>>>> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
>>>>
>> _______________________________________________
>> lustre-discuss mailing list
>> lustre-discuss at lists.lustre.org
>> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
>>
>