[Lustre-discuss] ll_cfg_requeue process timeouts

[Nathan Rutman]

Wojciech Turek wrote:
> Hi,
>
> Our environment is: 2.6.9-55.0.9.EL_lustre.1.6.3smp
> I am getting following errors from two OSS's
>
> ...
> Nov  7 10:39:51 storage09.beowulf.cluster kernel: LustreError: 
> 23045:0:(client.c:519:ptlrpc_import_delay_req()) @@@ IMP_INVALID 
>  req at 00000100b410be00 x4190687/t0 o101->MGS at MGC10.143.245.201 
> <mailto:MGS at MGC10.143.245.201>@tcp_0:26 lens 232/240 ref 1 fl Rpc:/0/0 
> rc 0/0 
> Nov  7 10:39:51 storage09.beowulf.cluster kernel: LustreError: 
> 23045:0:(client.c:519:ptlrpc_import_delay_req()) Skipped 119 previous 
> similar messages 
> Nov  7 10:50:18 storage10.beowulf.cluster kernel: LustreError: 
> 23337:0:(client.c:519:ptlrpc_import_delay_req()) @@@ IMP_INVALID 
>  req at 000001010e130c00 x4006346/t0 o101->MGS at MGC10.143.245.201 
> <mailto:MGS at MGC10.143.245.201>@tcp_0:26 lens 232/240 ref 1 fl Rpc:/0/0 
> rc 0/0 
> Nov  7 10:50:18 storage10.beowulf.cluster kernel: 
> LustreError: 23337:0:(client.c:519:ptlrpc_import_delay_req()) Skipped 
> 119 previous similar messages 
> Nov  7 10:50:35 storage09.beowulf.cluster kernel: LustreError: 
> 23045:0:(client.c:519:ptlrpc_import_delay_req()) @@@ IMP_INVALID 
>  req at 00000101258c5a00 x4193819/t0 o101->MGS at MGC10.143.245.201 
> <mailto:MGS at MGC10.143.245.201>@tcp_0:26 lens 232/240 ref 1 fl Rpc:/0/0 
> rc 0/0 
> Nov  7 10:50:35 storage09.beowulf.cluster kernel: LustreError: 
> 23045:0:(client.c:519:ptlrpc_import_delay_req()) Skipped 119 previous 
> similar messages 
> Nov  7 11:01:05 storage10.beowulf.cluster kernel: LustreError: 
> 23337:0:(client.c:519:ptlrpc_import_delay_req()) @@@ IMP_INVALID 
>  req at 00000100b9fa7800 x4013002/t0 o101->MGS at MGC10.143.245.201 
> <mailto:MGS at MGC10.143.245.201>@tcp_0:26 lens 232/240 ref 1 fl Rpc:/0/0 
> rc 0/0 
> Nov  7 11:01:05 storage10.beowulf.cluster kernel: LustreError: 
> 23337:0:(client.c:519:ptlrpc_import_delay_req()) Skipped 119 previous 
> similar messages 
> Nov  7 11:01:18 storage09.beowulf.cluster kernel: LustreError: 
> 23045:0:(client.c:519:ptlrpc_import_delay_req()) @@@ IMP_INVALID 
>  req at 00000100b774c800 x4199160/t0 o101->MGS at MGC10.143.245.201 
> <mailto:MGS at MGC10.143.245.201>@tcp_0:26 lens 232/240 ref 1 fl Rpc:/0/0 
> rc 0/0
> ...
>
> processes ID: 23337 and 23045 are ll_cfg_requeue
>
> On other two OSS's I can't see these processes. 
>
> Could some one advice how to remove or restart these processes to stop 
> them from sending error messages ?
This means the MGC is trying to reconnect to the MGS and failing.  In 
and of itself, this isn't a problem; it just means you won't get 
configuration change updates on those nodes.

We have an open bug 13715 on this issue.
https://bugzilla.clusterfs.com/show_bug.cgi?id=13715

You can get rid of the errors in the meantime by starting the OSTs after 
starting the MGS first.