[lustre-discuss] problem loading zfs properties for a lustre/zfs fs

Tue Jul 12 13:53:15 PDT 2016

You don't need to modify the initramfs as you describe, although that would likely work.

It should be sufficient to rebuild the initramfs.  It's built on your system, for example after a kernel installation.  Most likely, at the time the initramfs was last built on your system, the zfs-dracut package was installed, and that caused the initramfs file to be built with zfs.  Rebuilding now should remove the zfs module. 

Redhat has an article that says how to rebuild the initramfs:
https://access.redhat.com/solutions/1958

I can't see their instructions, as I don't have an account for their portal.  I suspect it just comes down to running mkinitrd.  Or you can reinstall your kernel and let it rebuild the initramfs as a side-affect:

rpm -i --replacepkgs /path/to/your/kernel/rpm

To answer your question about why this is necessary: the kernel first boots and loads some drivers etc., using the initramfs, before your normal root filesystem is mounted.  During this first boot stage, your hand-edited zfs.conf is not present (it's in the root filesystem that hasn't yet been mounted).  Therefore the zfs module does not get those options passed to it during load.

Olaf P. Faaland
Livermore Computing

________________________________________
From: Riccardo Veraldi [Riccardo.Veraldi at cnaf.infn.it]
Sent: Tuesday, July 12, 2016 1:03 PM
To: Faaland, Olaf P.; lustre-discuss at lists.lustre.org
Subject: Re: [lustre-discuss] problem loading zfs properties for a lustre/zfs fs

Hello,

lsinitrd | grep zfs
-rwxr--r--   1 root     root       493080 Jul 12 12:56
usr/lib/modules/3.10.0-327.18.2.el7.x86_64/extra/kernel/fs/lustre/osd_zfs.ko
-rw-r--r--   1 root     root      2245128 Jul 12 12:56
usr/lib/modules/3.10.0-327.18.2.el7.x86_64/extra/zfs.ko
-rw-r--r--   1 root     root            4 May 12 20:18
usr/lib/modules-load.d/zfs.conf

if I do

lsinitrd -f usr/lib/modules-load.d/zfs.conf
zfs

so looks like zfs is loaded by initramfs

do I have to add a etc/modprobe.d/zfs.conf inside the initramfs ?

I mean /etc/modprobe.d/zfs.conf  from the root partition should be
loaded anyway.

thanks

On 12/07/16 11:51, Faaland, Olaf P. wrote:
> Actually, looks like you can use simpler syntax:
>
> lsinitrd | grep zfs
>
> Olaf P. Faaland
> Livermore Computing
>
> ________________________________________
> From: lustre-discuss [lustre-discuss-bounces at lists.lustre.org] on behalf of Faaland, Olaf P. [faaland1 at llnl.gov]
> Sent: Tuesday, July 12, 2016 11:48 AM
> To: Riccardo Veraldi; lustre-discuss at lists.lustre.org
> Subject: Re: [lustre-discuss] problem loading zfs properties for a lustre/zfs fs
>
> Riccardo,
>
> Does your initramfs still include zfs-related files?  What is the output of
>
> lsinitrd /boot/init*$(uname -r)* | grep zfs
>
> Olaf P. Faaland
> Livermore Computing
>
> ________________________________________
> From: Riccardo Veraldi [Riccardo.Veraldi at cnaf.infn.it]
> Sent: Monday, July 11, 2016 5:25 PM
> To: Faaland, Olaf P.; lustre-discuss at lists.lustre.org
> Subject: Re: [lustre-discuss] problem loading zfs properties for a lustre/zfs fs
>
> On 11/07/16 17:02, Faaland, Olaf P. wrote:
>> Riccardo,
>>
>> If you are not booting from a zpool, you do not need the "zfs-dracut" package.  This package causes ZFS to be loaded very early in the boot process, most likely before your /etc/modprobe.d/zfs.conf files is visible to the kernel.
>>
>> As long as you are not booting from a zpool, remove that package (rpm -e) and boot again to see if your problem is fixed.
> I did it I removed zfs-dracut since I do not boot from ZFS.
>
>    cat /etc/modprobe.d/zfs.conf
> options zfs zfs_prefetch_disable=1
> options zfs zfs_txg_history=120
> options zfs metaslab_debug_unload=1
>
> after reboot
>
> cat /sys/module/zfs/parameters/zfs_prefetch_disable
> 0
>
> cat /sys/module/zfs/parameters/zfs_txg_history
> 0
>
> so it looks like the parameters inside zfs.conf are ignored
>
> ls -la /etc/modprobe.d/zfs.conf
> -rw-r--r-- 1 root root 103 Jul 11 17:18 /etc/modprobe.d/zfs.conf
>
>
>
>> -Olaf
>>
>> ________________________________________
>> From: Riccardo Veraldi [Riccardo.Veraldi at cnaf.infn.it]
>> Sent: Monday, July 11, 2016 4:55 PM
>> To: Faaland, Olaf P.; lustre-discuss at lists.lustre.org
>> Subject: Re: [lustre-discuss] problem loading zfs properties for a lustre/zfs fs
>>
>> On 11/07/16 16:15, Faaland, Olaf P. wrote:
>>> 1) What is the output of:
>>>
>>> rpm -qa | grep zfs
>> libzfs2-0.6.5.7-1.el7.centos.x86_64
>> zfs-dkms-0.6.5.7-1.el7.centos.noarch
>> lustre-osd-zfs-mount-2.8.0-3.10.0_327.18.2.el7.x86_64.x86_64
>> zfs-0.6.5.7-1.el7.centos.x86_64
>> zfs-dracut-0.6.5.7-1.el7.centos.x86_64
>> lustre-osd-zfs-2.8.0-3.10.0_327.18.2.el7.x86_64.x86_64
>>> from that system after it boots?
>>>
>>> 2) How do those values get into the /etc/modprobe.d/zfs.conf file?  Are they there before the node boots, or are modifying that file somehow during the boot process?
>> I edited those values myself inside zfs.conf then rebooted the system,
>> but they are not loaded
>>
>> thank you
>>
>> Riccardo
>>
>>
>>> Olaf P. Faaland
>>> Livermore Computing
>>> ________________________________________
>>> From: lustre-discuss [lustre-discuss-bounces at lists.lustre.org] on behalf of Riccardo Veraldi [Riccardo.Veraldi at cnaf.infn.it]
>>> Sent: Monday, July 11, 2016 3:53 PM
>>> To: lustre-discuss at lists.lustre.org
>>> Subject: [lustre-discuss] problem loading zfs properties for a lustre/zfs fs
>>>
>>> Hello,
>>> I am tailoring my system for lustre on ZFS and I am not able to set
>>> these parameters
>>> writing the config file /etc/modprobe.d/zfs.conf with the following options
>>>
>>> options zfs zfs_prefetch_disable=1
>>> options zfs zfs_txg_history=120
>>> options zfs metaslab_debug:unload=1
>>>
>>> when I check the parameters after boot in /sys/module/zfs/parameters
>>> they are still the defaults, and my modification are not taken.
>>>
>>> zfs.conf is being ignored at boot time I do not know why.
>>>
>>> I had to do a
>>>
>>> echo 1 > /sys/module/zfs/parameters/options zfs zfs_prefetch_disable
>>>
>>> inside rc.local
>>>
>>> to have it working
>>>
>>> I am running RHEL7 with latest kernel  and zfs 0.6.5.7-1
>>>
>>> any hints ?
>>>
>>> thank you
>>>
>>> Riccardo
>>>
>>> _______________________________________________
>>> lustre-discuss mailing list
>>> lustre-discuss at lists.lustre.org
>>> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
>>>
> _______________________________________________
> lustre-discuss mailing list
> lustre-discuss at lists.lustre.org
> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
>