<html><head></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">Here is a dump of a log of some kind from the client at the time of the eviction.<div><br></div><div>Does this shed any further light on the problem?</div><div><br></div><div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000100:00000400:7:1278392906.021560:0:6677:0:(client.c:1463:ptlrpc_expire_one_request()) @@@ Request x1340041377934503 sent from AC3-OST0006-osc-ffff81043e754c00 to NID 172.16.16.2@tcp 11s ago has timed o</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">ut (11s prior to deadline).</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; "> req@ffff81043f31dc00 x1340041377934503/t0 o101->AC3-OST0006_UUID@172.16.16.2@tcp:28/4 lens 296/544 e 0 to 1 dl 1278392906 ref 1 fl Rpc:/0/0 rc 0/0</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000100:02000400:7:1278392906.021598:0:6677:0:(import.c:159:ptlrpc_set_import_discon()) AC3-OST0006-osc-ffff81043e754c00: Connection to service AC3-OST0006 via nid 172.16.16.2@tcp was lost; in progress operations using this service will wait for recovery to complete.</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000100:00080000:7:1278392906.021611:0:6677:0:(import.c:167:ptlrpc_set_import_discon()) ffff81042018c000 AC3-OST0006_UUID: changing import state from FULL to DISCONN</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000100:00080000:7:1278392906.021615:0:6677:0:(import.c:403:ptlrpc_fail_import()) AC3-OST0006_UUID: waking up pinger00000100:00080000:4:1278392906.021634:0:3573:0:(recover.c:67:ptlrpc_initiate_recovery()) AC3-OST0006_UUID: starting recovery</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000100:00080000:4:1278392906.021639:0:3573:0:(import.c:608:ptlrpc_connect_import()) ffff81042018c000 AC3-OST0006_UUID: changing import state from DISCONN to CONNECTING</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000100:00080000:4:1278392906.021645:0:3573:0:(import.c:470:import_select_connection()) AC3-OST0006-osc-ffff81043e754c00: connect to NID 172.16.16.2@tcp last attempt 4379111012</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000100:00080000:4:1278392906.021645:0:3573:0:(import.c:470:import_select_connection()) AC3-OST0006-osc-ffff81043e754c00: connect to NID 172.16.16.2@tcp last attempt 4379111012</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000100:00080000:4:1278392906.021651:0:3573:0:(import.c:544:import_select_connection()) AC3-OST0006-osc-ffff81043e754c00: import ffff81042018c000 using connection 172.16.16.2@tcp/172.16.16.2@tcp</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000100:00000400:7:1278392906.030554:0:6687:0:(client.c:1463:ptlrpc_expire_one_request()) @@@ Request x1340041377934535 sent from AC3-OST0006-osc-ffff81043e754c00 to NID 172.16.16.2@tcp 11s ago has timed o</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">ut (11s prior to deadline).</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; "> req@ffff81031686b000 x1340041377934535/t0 o101->AC3-OST0006_UUID@172.16.16.2@tcp:28/4 lens 296/544 e 0 to 1 dl 1278392906 ref 1 fl Rpc:/0/0 rc 0/0</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000100:00080000:7:1278392906.030567:0:6687:0:(import.c:182:ptlrpc_set_import_discon()) osc: import ffff81042018c000 already not connected (conn 177, was 176): CONNECTING</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000100:00000400:5:1278392907.030414:0:6686:0:(client.c:1463:ptlrpc_expire_one_request()) @@@ Request x1340041377934536 sent from AC3-OST0008-osc-ffff81043e754c00 to NID 172.16.16.2@tcp 12s ago has timed o</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">ut (12s prior to deadline).</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; "> req@ffff8102b439bc00 x1340041377934536/t0 o101->AC3-OST0008_UUID@172.16.16.2@tcp:28/4 lens 296/544 e 0 to 1 dl 1278392907 ref 1 fl Rpc:/0/0 rc 0/0</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000100:02000400:5:1278392907.030434:0:6686:0:(import.c:159:ptlrpc_set_import_discon()) AC3-OST0008-osc-ffff81043e754c00: Connection to service AC3-OST0008 via nid 172.16.16.2@tcp was lost; in progress ope</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">rations using this service will wait for recovery to complete.</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000100:00080000:5:1278392907.030449:0:6686:0:(import.c:167:ptlrpc_set_import_discon()) ffff810420d1a000 AC3-OST0008_UUID: changing import state from FULL to DISCONN</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000100:00080000:5:1278392907.030452:0:6686:0:(import.c:403:ptlrpc_fail_import()) AC3-OST0008_UUID: waking up pinger</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000100:00080000:4:1278392907.030465:0:3573:0:(recover.c:67:ptlrpc_initiate_recovery()) AC3-OST0008_UUID: starting recovery00000100:00080000:4:1278392907.030470:0:3573:0:(import.c:608:ptlrpc_connect_import()) ffff810420d1a000 AC3-OST0008_UUID: changing import state from DISCONN to CONNECTING</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000100:00080000:4:1278392907.030475:0:3573:0:(import.c:470:import_select_connection()) AC3-OST0008-osc-ffff81043e754c00: connect to NID 172.16.16.2@tcp last attempt 437911101200000100:00080000:4:1278392907.030480:0:3573:0:(import.c:544:import_select_connection()) AC3-OST0008-osc-ffff81043e754c00: import ffff810420d1a000 using connection 172.16.16.2@tcp/172.16.16.2@tcp</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000400:00000100:3:1278392907.213541:0:3560:0:(lib-move.c:1818:lnet_parse_put()) Dropping PUT from 12345-172.16.16.2@tcp portal 4 match 1340041377934536 offset 192 length 352: 200000400:00000100:3:1278392907.213550:0:3560:0:(lib-move.c:1818:lnet_parse_put()) Dropping PUT from 12345-172.16.16.2@tcp portal 4 match 1340041377934503 offset 192 length 352: 2</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000400:00000100:3:1278392907.213554:0:3560:0:(lib-move.c:1818:lnet_parse_put()) Dropping PUT from 12345-172.16.16.2@tcp portal 4 match 1340041377934535 offset 192 length 352: 200000100:02020000:0:1278392907.213815:0:3572:0:(client.c:975:ptlrpc_check_status()) 11-0: an error occurred while communicating with 172.16.16.2@tcp. The ost_connect operation failed with -16</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000100:00080000:0:1278392907.213828:0:3572:0:(import.c:1091:ptlrpc_connect_interpret()) ffff81042018c000 AC3-OST0006_UUID: changing import state from CONNECTING to DISCONN00000100:00080000:0:1278392907.213831:0:3572:0:(import.c:1137:ptlrpc_connect_interpret()) recovery of AC3-OST0006_UUID on 172.16.16.2@tcp failed (-16)</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000100:02020000:0:1278392907.213839:0:3572:0:(client.c:975:ptlrpc_check_status()) 11-0: an error occurred while communicating with 172.16.16.2@tcp. The ost_connect operation failed with -1600000100:00080000:0:1278392907.213841:0:3572:0:(import.c:1091:ptlrpc_connect_interpret()) ffff810420d1a000 AC3-OST0008_UUID: changing import state from CONNECTING to DISCONN</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000100:00080000:0:1278392907.213843:0:3572:0:(import.c:1137:ptlrpc_connect_interpret()) recovery of AC3-OST0008_UUID on 172.16.16.2@tcp failed (-16)</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000100:00080000:6:1278392907.226136:0:3571:0:(client.c:1145:ptlrpc_send_new_req()) @@@ req from PID 0 waiting for recovery: (FULL != DISCONN) req@ffff8101b4588c00 x1340041377955935/t0 o3->AC3-OST0008_UUI</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">D@172.16.16.2@tcp:6/4 lens 448/592 e 0 to 1 dl 0 ref 2 fl Rpc:W/0/0 rc 0/0</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000100:00080000:6:1278392907.226192:0:3571:0:(client.c:1145:ptlrpc_send_new_req()) @@@ req from PID 0 waiting for recovery: (FULL != DISCONN) req@ffff81042061b800 x1340041377955949/t0 o4->AC3-OST0008_UUI</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">D@172.16.16.2@tcp:6/4 lens 448/608 e 0 to 1 dl 0 ref 2 fl Rpc:W/0/0 rc 0/0</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000100:00080000:6:1278392907.226202:0:3571:0:(client.c:1145:ptlrpc_send_new_req()) @@@ req from PID 0 waiting for recovery: (FULL != DISCONN) req@ffff81042061a400 x1340041377955951/t0 o4->AC3-OST0008_UUI</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">D@172.16.16.2@tcp:6/4 lens 448/608 e 0 to 1 dl 0 ref 2 fl Rpc:W/0/0 rc 0/0</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000100:00080000:6:1278392907.226207:0:3571:0:(client.c:1145:ptlrpc_send_new_req()) @@@ req from PID 0 waiting for recovery: (FULL != DISCONN) req@ffff8104201e5000 x1340041377955955/t0 o4->AC3-OST0006_UUID@172.16.16.2@tcp:6/4 lens 448/608 e 0 to 1 dl 0 ref 2 fl Rpc:W/0/0 rc 0/0</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000100:00080000:6:1278392907.226211:0:3571:0:(client.c:1145:ptlrpc_send_new_req()) @@@ req from PID 0 waiting for recovery: (FULL != DISCONN) req@ffff81042030f800 x1340041377955956/t0 o3->AC3-OST0008_UUI</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">D@172.16.16.2@tcp:6/4 lens 448/592 e 0 to 1 dl 0 ref 2 fl Rpc:W/0/0 rc 0/000000100:00080000:6:1278392907.226219:0:3571:0:(client.c:1145:ptlrpc_send_new_req()) @@@ req from PID 0 waiting for recovery: (FULL != DISCONN) req@ffff81042061a000 x1340041377955959/t0 o4->AC3-OST0008_UUI</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">D@172.16.16.2@tcp:6/4 lens 448/608 e 0 to 1 dl 0 ref 2 fl Rpc:W/0/0 rc 0/0</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000100:00080000:4:1278392917.022748:0:3573:0:(recover.c:67:ptlrpc_initiate_recovery()) AC3-OST0006_UUID: starting recovery00000100:00080000:4:1278392917.022753:0:3573:0:(import.c:608:ptlrpc_connect_import()) ffff81042018c000 AC3-OST0006_UUID: changing import state from DISCONN to CONNECTING</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000100:00080000:4:1278392917.022759:0:3573:0:(import.c:470:import_select_connection()) AC3-OST0006-osc-ffff81043e754c00: connect to NID 172.16.16.2@tcp last attempt 4724612378</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000100:00080000:4:1278392917.022766:0:3573:0:(import.c:544:import_select_connection()) AC3-OST0006-osc-ffff81043e754c00: import ffff81042018c000 using connection 172.16.16.2@tcp/172.16.16.2@tcp00000100:00080000:0:1278392917.023419:0:3572:0:(import.c:868:ptlrpc_connect_interpret()) reconnected to AC3-OST0006_UUID@172.16.16.2@tcp after partition</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000100:00080000:0:1278392917.023425:0:3572:0:(import.c:891:ptlrpc_connect_interpret()) ffff81042018c000 AC3-OST0006_UUID: changing import state from CONNECTING to RECOVER</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">1 dl 1278392951 ref 2 fl Rpc:/0/0 rc 0/0</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000100:00080000:0:1278392917.023440:0:3572:0:(client.c:1969:ptlrpc_resend_req()) resend bulk old x1340041377933846 new x1340041377973369</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; "> 1 dl 1278392951 ref 2 fl Rpc:/0/0 rc 0/0</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000100:00080000:0:1278392917.023452:0:3572:0:(client.c:1969:ptlrpc_resend_req()) resend bulk old x1340041377934227 new x1340041377973370</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; "> 1 dl 1278392952 ref 2 fl Rpc:/0/0 rc 0/0</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000100:00080000:0:1278392917.023459:0:3572:0:(client.c:1969:ptlrpc_resend_req()) resend bulk old x1340041377934618 new x1340041377973371</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; "> 1 dl 1278392952 ref 2 fl Rpc:/0/0 rc 0/0</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000100:00080000:0:1278392917.023466:0:3572:0:(client.c:1969:ptlrpc_resend_req()) resend bulk old x1340041377934964 new x1340041377973372</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; "> 1 dl 1278392955 ref 2 fl Rpc:/0/0 rc 0/0</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000100:00080000:0:1278392917.023472:0:3572:0:(client.c:1969:ptlrpc_resend_req()) resend bulk old x1340041377940508 new x1340041377973373</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000100:00080000:0:1278392917.023475:0:3572:0:(import.c:1319:ptlrpc_import_recovery_state_machine()) ffff81042018c000 AC3-OST0006_UUID: changing import state from RECOVER to FULL</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000100:00080000:6:1278392917.023480:0:3571:0:(client.c:1332:ptlrpc_check_set()) resend bulk old x1340041377973369 new x1340041377973374</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000002:00080000:0:1278392917.023492:0:3572:0:(mdc_request.c:1465:mdc_init_ea_size()) updating max_mdsize/max_cookiesize: 288/320</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000100:02000000:0:1278392917.023497:0:3572:0:(import.c:1327:ptlrpc_import_recovery_state_machine()) AC3-OST0006-osc-ffff81043e754c00: Connection restored to service AC3-OST0006 using nid 172.16.16.2@tcp.</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000100:00080000:6:1278392917.023513:0:3571:0:(client.c:1332:ptlrpc_check_set()) resend bulk old x1340041377973370 new x1340041377973375</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">ns 448/608 e 0 to 1 dl 0 ref 2 fl Rpc:W/0/0 rc 0/0</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; "> lens 296/544 e 0 to 1 dl 1278392906 ref 1 fl Rpc:X/0/0 rc 0/0</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; "> lens 296/544 e 0 to 1 dl 1278392906 ref 1 fl Rpc:X/0/0 rc 0/0</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000100:00080000:6:1278392917.023543:0:3571:0:(client.c:1332:ptlrpc_check_set()) resend bulk old x1340041377973371 new x1340041377973376</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000002:00080000:0:1278392917.023544:0:3572:0:(mdc_request.c:1465:mdc_init_ea_size()) updating max_mdsize/max_cookiesize: 288/320</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000100:00080000:6:1278392917.023572:0:3571:0:(client.c:1332:ptlrpc_check_set()) resend bulk old x1340041377973372 new x1340041377973377</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000100:00080000:6:1278392917.023596:0:3571:0:(client.c:1332:ptlrpc_check_set()) resend bulk old x1340041377973373 new x1340041377973378</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000100:00080000:4:1278392919.644883:0:3573:0:(recover.c:67:ptlrpc_initiate_recovery()) AC3-OST0008_UUID: starting recovery</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000100:00080000:4:1278392919.644889:0:3573:0:(import.c:608:ptlrpc_connect_import()) ffff810420d1a000 AC3-OST0008_UUID: changing import state from DISCONN to CONNECTING</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000100:00080000:4:1278392919.644894:0:3573:0:(import.c:470:import_select_connection()) AC3-OST0008-osc-ffff81043e754c00: connect to NID 172.16.16.2@tcp last attempt 4724613386</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000100:00080000:4:1278392919.644900:0:3573:0:(import.c:544:import_select_connection()) AC3-OST0008-osc-ffff81043e754c00: import ffff810420d1a000 using connection 172.16.16.2@tcp/172.16.16.2@tcp</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000100:00080000:0:1278392919.645477:0:3572:0:(import.c:868:ptlrpc_connect_interpret()) reconnected to AC3-OST0008_UUID@172.16.16.2@tcp after partition</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000100:00080000:0:1278392919.645482:0:3572:0:(import.c:891:ptlrpc_connect_interpret()) ffff810420d1a000 AC3-OST0008_UUID: changing import state from CONNECTING to RECOVER</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000100:00080000:0:1278392919.645486:0:3572:0:(import.c:1314:ptlrpc_import_recovery_state_machine()) reconnected to AC3-OST0008_UUID@172.16.16.2@tcp</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; "> 1 dl 1278392952 ref 2 fl Rpc:/0/0 rc 0/0</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000100:00080000:0:1278392919.645495:0:3572:0:(client.c:1969:ptlrpc_resend_req()) resend bulk old x1340041377933615 new x1340041377977349</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; "> 1 dl 1278392952 ref 2 fl Rpc:/0/0 rc 0/0</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000100:00080000:0:1278392919.645505:0:3572:0:(client.c:1969:ptlrpc_resend_req()) resend bulk old x1340041377933659 new x1340041377977350</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; "> 1 dl 1278392952 ref 2 fl Rpc:/0/0 rc 0/0</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000100:00080000:0:1278392919.645511:0:3572:0:(client.c:1969:ptlrpc_resend_req()) resend bulk old x1340041377933703 new x1340041377977351</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000100:00080000:0:1278392919.645514:0:3572:0:(import.c:1319:ptlrpc_import_recovery_state_machine()) ffff810420d1a000 AC3-OST0008_UUID: changing import state from RECOVER to FULL</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000100:00080000:6:1278392919.645518:0:3571:0:(client.c:1332:ptlrpc_check_set()) resend bulk old x1340041377977350 new x1340041377977353</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000002:00080000:0:1278392919.645531:0:3572:0:(mdc_request.c:1465:mdc_init_ea_size()) updating max_mdsize/max_cookiesize: 288/320</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000100:02000000:0:1278392919.645536:0:3572:0:(import.c:1327:ptlrpc_import_recovery_state_machine()) AC3-OST0008-osc-ffff81043e754c00: Connection restored to service AC3-OST0008 using nid 172.16.16.2@tcp.</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; "> lens 296/544 e 0 to 1 dl 1278392907 ref 1 fl Rpc:X/0/0 rc 0/0</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">ns 448/592 e 0 to 1 dl 0 ref 2 fl Rpc:W/0/0 rc 0/0</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">ns 448/608 e 0 to 1 dl 0 ref 2 fl Rpc:W/0/0 rc 0/0</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">ns 448/608 e 0 to 1 dl 0 ref 2 fl Rpc:W/0/0 rc 0/0</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">ns 448/592 e 0 to 1 dl 0 ref 2 fl Rpc:W/0/0 rc 0/0</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">ns 448/608 e 0 to 1 dl 0 ref 2 fl Rpc:W/0/0 rc 0/0</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">ns 448/592 e 0 to 1 dl 1278392952 ref 2 fl Rpc:S/0/0 rc -11/0</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000100:00080000:6:1278392919.645591:0:3571:0:(client.c:1332:ptlrpc_check_set()) resend bulk old x1340041377977351 new x1340041377977354</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000002:00080000:0:1278392919.645593:0:3572:0:(mdc_request.c:1465:mdc_init_ea_size()) updating max_mdsize/max_cookiesize: 288/320</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000100:00080000:6:1278392919.645729:0:3571:0:(client.c:1332:ptlrpc_check_set()) resend bulk old x1340041377977349 new x1340041377977355</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000100:02020000:2:1278393017.800317:0:3571:0:(client.c:975:ptlrpc_check_status()) 11-0: an error occurred while communicating with 172.16.16.2@tcp. The ost_write operation failed with -107</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000100:00080000:2:1278393017.800332:0:3571:0:(recover.c:196:ptlrpc_request_handle_notconn()) import AC3-OST0006-osc-ffff81043e754c00 of AC3-OST0006_UUID@172.16.16.2@tcp abruptly disconnected: reconnecting</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">rations using this service will wait for recovery to complete.</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000100:00080000:2:1278393017.800343:0:3571:0:(import.c:167:ptlrpc_set_import_discon()) ffff81042018c000 AC3-OST0006_UUID: changing import state from FULL to DISCONN</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000100:00080000:2:1278393017.800350:0:3571:0:(import.c:608:ptlrpc_connect_import()) ffff81042018c000 AC3-OST0006_UUID: changing import state from DISCONN to CONNECTING</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000100:00080000:2:1278393017.800357:0:3571:0:(import.c:470:import_select_connection()) AC3-OST0006-osc-ffff81043e754c00: connect to NID 172.16.16.2@tcp last attempt 4724623378</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000100:00080000:2:1278393017.800366:0:3571:0:(import.c:544:import_select_connection()) AC3-OST0006-osc-ffff81043e754c00: import ffff81042018c000 using connection 172.16.16.2@tcp/172.16.16.2@tcpt 0 o8->AC3-OST0006_UUID@172.16.16.2@tcp:28/4 lens 368/584 e 0 to 1 dl 1278393031 ref 1 fl Interpret:RN/0/0 rc 0/0</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000100:00080000:0:1278393017.801096:0:3572:0:(import.c:911:ptlrpc_connect_interpret()) ffff81042018c000 AC3-OST0006_UUID: changing import state from CONNECTING to EVICTED</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000100:02020000:0:1278393017.801099:0:3572:0:(import.c:1249:ptlrpc_import_recovery_state_machine()) 167-0: This client was evicted by AC3-OST0006; in progress operations using this service will fail.</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000100:00080000:0:1278393017.801106:0:3572:0:(import.c:1253:ptlrpc_import_recovery_state_machine()) evicted from AC3-OST0006_UUID@172.16.16.2@tcp; invalidating</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000002:00080000:0:1278393017.801150:0:3572:0:(mdc_request.c:1465:mdc_init_ea_size()) updating max_mdsize/max_cookiesize: 288/320</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000100:00080000:0:1278393017.801188:0:6779:0:(import.c:1215:ptlrpc_invalidate_import_thread()) thread invalidate import AC3-OST0006-osc-ffff81043e754c00 to AC3-OST0006_UUID@172.16.16.2@tcp</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000100:00080000:0:1278393017.801190:0:6779:0:(import.c:194:ptlrpc_deactivate_and_unlock_import()) setting import AC3-OST0006_UUID INVALID</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00020000:01000000:0:1278393017.801201:0:6779:0:(lov_obd.c:567:lov_set_osc_active()) Marking OSC AC3-OST0006_UUID inactive</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000002:00080000:0:1278393017.801206:0:6779:0:(mdc_request.c:1465:mdc_init_ea_size()) updating max_mdsize/max_cookiesize: 288/320</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00020000:00080000:7:1278393017.810055:0:6430:0:(lov_request.c:325:lov_prep_enqueue_set()) lov idx 6 inactive</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00020000:00080000:5:1278393017.827252:0:3095:0:(lov_request.c:325:lov_prep_enqueue_set()) lov idx 6 inactive</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00020000:00080000:5:1278393017.827298:0:3095:0:(lov_request.c:325:lov_prep_enqueue_set()) lov idx 6 inactive</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00020000:00080000:5:1278393017.827325:0:3095:0:(lov_request.c:325:lov_prep_enqueue_set()) lov idx 6 inactive</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00020000:00080000:7:1278393017.833456:0:3095:0:(lov_request.c:1099:lov_prep_destroy_set()) lov idx 6 inactive</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000080:00020000:7:1278393017.833461:0:3095:0:(namei.c:1176:ll_objects_destroy()) obd destroy objid 0x18542c4@0x0 error -5</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00020000:00080000:4:1278393017.912169:0:6430:0:(lov_obd.c:2372:lov_join_lru()) lov idx 6 inactive</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00020000:00080000:4:1278393017.913654:0:6419:0:(lov_request.c:325:lov_prep_enqueue_set()) lov idx 6 inactive</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000080:00020000:4:1278393017.913660:0:6419:0:(file.c:995:ll_glimpse_size()) obd_enqueue returned rc -5, returning -EIO</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00020000:00080000:4:1278393017.922360:0:6419:0:(lov_request.c:325:lov_prep_enqueue_set()) lov idx 6 inactive</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00010000:00020000:1:1278393018.325707:0:6779:0:(ldlm_resource.c:518:ldlm_namespace_cleanup()) Namespace AC3-OST0006-osc-ffff81043e754c00 resource refcount nonzero (1) after lock cleanup; forcing cleanup.</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00010000:00020000:1:1278393018.325727:0:6779:0:(ldlm_resource.c:523:ldlm_namespace_cleanup()) Resource: ffff810117997500 (8432287/0/0/0) (rc: 1)</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00020000:00080000:7:1278393018.325761:0:6687:0:(lov_obd.c:1727:lov_brw_check()) lov idx 6 inactive</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000080:00020000:7:1278393018.325781:0:6687:0:(llite_mmap.c:210:ll_tree_unlock()) couldn't unlock -5</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00020000:00080000:7:1278393018.327514:0:6687:0:(lov_request.c:325:lov_prep_enqueue_set()) lov idx 6 inactive</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00020000:00080000:2:1278393018.332887:0:6782:0:(lov_request.c:325:lov_prep_enqueue_set()) lov idx 6 inactive</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000080:00020000:2:1278393018.332895:0:6782:0:(file.c:995:ll_glimpse_size()) obd_enqueue returned rc -5, returning -EIO</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000100:00080000:1:1278393018.333028:0:6779:0:(import.c:1224:ptlrpc_invalidate_import_thread()) ffff81042018c000 AC3-OST0006_UUID: changing import state from EVICTED to RECOVER</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000100:00080000:1:1278393018.333032:0:6779:0:(import.c:1314:ptlrpc_import_recovery_state_machine()) reconnected to AC3-OST0006_UUID@172.16.16.2@tcp</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000100:00080000:1:1278393018.333035:0:6779:0:(import.c:1319:ptlrpc_import_recovery_state_machine()) ffff81042018c000 AC3-OST0006_UUID: changing import state from RECOVER to FULL</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00020000:01000000:1:1278393018.333042:0:6779:0:(lov_obd.c:567:lov_set_osc_active()) Marking OSC AC3-OST0006_UUID active</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000002:00080000:1:1278393018.333051:0:6779:0:(mdc_request.c:1465:mdc_init_ea_size()) updating max_mdsize/max_cookiesize: 288/320</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000100:02000000:1:1278393018.333056:0:6779:0:(import.c:1327:ptlrpc_import_recovery_state_machine()) AC3-OST0006-osc-ffff81043e754c00: Connection restored to service AC3-OST0006 using nid 172.16.16.2@tcp.</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000100:02020000:4:1278393020.758895:0:20660:0:(client.c:975:ptlrpc_check_status()) 11-0: an error occurred while communicating with 172.16.16.2@tcp. The ost_read operation failed with -107</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">g</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">erations using this service will wait for recovery to complete.</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000100:00080000:4:1278393020.758919:0:20660:0:(import.c:167:ptlrpc_set_import_discon()) ffff810420d1a000 AC3-OST0008_UUID: changing import state from FULL to DISCONN</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000100:00080000:4:1278393020.758925:0:20660:0:(import.c:608:ptlrpc_connect_import()) ffff810420d1a000 AC3-OST0008_UUID: changing import state from DISCONN to CONNECTING</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000100:00080000:4:1278393020.758928:0:20660:0:(import.c:470:import_select_connection()) AC3-OST0008-osc-ffff81043e754c00: connect to NID 172.16.16.2@tcp last attempt 4724626000</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000100:00080000:4:1278393020.758935:0:20660:0:(import.c:544:import_select_connection()) AC3-OST0008-osc-ffff81043e754c00: import ffff810420d1a000 using connection 172.16.16.2@tcp/172.16.16.2@tcp</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">16.16.2@tcp:6/4 lens 448/592 e 0 to 1 dl 1278393085 ref 1 fl Rpc:RS/0/0 rc 0/-107</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">UUID@172.16.16.2@tcp:6/4 lens 448/592 e 0 to 1 dl 0 ref 2 fl Rpc:W/0/0 rc 0/0</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">t0 o8->AC3-OST0008_UUID@172.16.16.2@tcp:28/4 lens 368/584 e 0 to 1 dl 1278393038 ref 1 fl Interpret:RN/0/0 rc 0/0</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000100:00080000:0:1278393020.770920:0:3572:0:(import.c:911:ptlrpc_connect_interpret()) ffff810420d1a000 AC3-OST0008_UUID: changing import state from CONNECTING to EVICTED</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000100:02020000:0:1278393020.770923:0:3572:0:(import.c:1249:ptlrpc_import_recovery_state_machine()) 167-0: This client was evicted by AC3-OST0008; in progress operations using this service will fail.</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000100:00080000:0:1278393020.770930:0:3572:0:(import.c:1253:ptlrpc_import_recovery_state_machine()) evicted from AC3-OST0008_UUID@172.16.16.2@tcp; invalidating</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000002:00080000:0:1278393020.771032:0:3572:0:(mdc_request.c:1465:mdc_init_ea_size()) updating max_mdsize/max_cookiesize: 288/320</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000100:00080000:0:1278393020.771066:0:6784:0:(import.c:1215:ptlrpc_invalidate_import_thread()) thread invalidate import AC3-OST0008-osc-ffff81043e754c00 to AC3-OST0008_UUID@172.16.16.2@tcp</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000100:00080000:0:1278393020.771068:0:6784:0:(import.c:194:ptlrpc_deactivate_and_unlock_import()) setting import AC3-OST0008_UUID INVALID</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">2@tcp:6/4 lens 448/592 e 0 to 1 dl 1278393085 ref 1 fl Rpc:RES/0/0 rc -4/-107</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00020000:01000000:0:1278393020.771084:0:6784:0:(lov_obd.c:567:lov_set_osc_active()) Marking OSC AC3-OST0008_UUID inactive</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000002:00080000:0:1278393020.771087:0:6784:0:(mdc_request.c:1465:mdc_init_ea_size()) updating max_mdsize/max_cookiesize: 288/320</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000080:00020000:4:1278393020.771103:0:20660:0:(rw.c:122:ll_brw()) error from obd_brw: rc = -4</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00010000:00020000:0:1278393020.777506:0:6784:0:(ldlm_resource.c:518:ldlm_namespace_cleanup()) Namespace AC3-OST0008-osc-ffff81043e754c00 resource refcount nonzero (1) after lock cleanup; forcing cleanup.</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00010000:00020000:0:1278393020.777518:0:6784:0:(ldlm_resource.c:523:ldlm_namespace_cleanup()) Resource: ffff81041604b9c0 (5521743/0/0/0) (rc: 1)</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00010000:00020000:0:1278393020.777531:0:6784:0:(ldlm_resource.c:518:ldlm_namespace_cleanup()) Namespace AC3-OST0008-osc-ffff81043e754c00 resource refcount nonzero (1) after lock cleanup; forcing cleanup.</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00010000:00020000:0:1278393020.777533:0:6784:0:(ldlm_resource.c:523:ldlm_namespace_cleanup()) Resource: ffff810266786e00 (8127131/0/0/0) (rc: 1)</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00020000:00080000:5:1278393020.777583:0:6686:0:(lov_obd.c:1727:lov_brw_check()) lov idx 8 inactive</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000080:00020000:5:1278393020.777624:0:6686:0:(llite_mmap.c:210:ll_tree_unlock()) couldn't unlock -5</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000100:00020000:2:1278393020.777678:0:3571:0:(client.c:858:ptlrpc_import_delay_req()) @@@ IMP_INVALID req@ffff81042061a800 x1340041378196315/t0 o4->AC3-OST0008_UUID@172.16.16.2@tcp:6/4 lens 512/624 e 0 o 1 dl 0 ref 2 fl Rpc:/0/0 rc 0/01068:0:6784:0:(import.c:194:ptlrpc_deactivate_and_unlock_import()) setting import AC3-OST0008_UUID INVALID</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000100:00020000:2:1278393020.778942:0:3571:0:(client.c:858:ptlrpc_import_delay_req()) @@@ IMP_INVALID req@ffff81042061a000 x1340041378196323/t0 o4->AC3-OST0008_UUID@172.16.16.2@tcp:6/4 lens 448/608 e 0 o 1 dl 0 ref 2 fl Rpc:/0/0 rc 0/01084:0:6784:0:(lov_obd.c:567:lov_set_osc_active()) Marking OSC AC3-OST0008_UUID inactive</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00020000:00080000:5:1278393020.779394:0:6686:0:(lov_request.c:325:lov_prep_enqueue_set()) lov idx 8 inactive</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000100:00020000:2:1278393020.780168:0:3571:0:(client.c:858:ptlrpc_import_delay_req()) @@@ IMP_INVALID req@ffff8101f525e000 x1340041378196326/t0 o3->AC3-OST0008_UUID@172.16.16.2@tcp:6/4 lens 448/592 e 0 o 1 dl 0 ref 2 fl Rpc:/0/0 rc 0/07506:0:6784:0:(ldlm_resource.c:518:ldlm_namespace_cleanup()) Namespace AC3-OST0008-osc-ffff81043e754c00 resource refcount nonzero (1) after lock cleanup; forcing cleanup.</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000100:00020000:2:1278393020.781246:0:3571:0:(client.c:858:ptlrpc_import_delay_req()) @@@ IMP_INVALID req@ffff8101f189fc00 x1340041378196328/t0 o3->AC3-OST0008_UUID@172.16.16.2@tcp:6/4 lens 448/592 e 0 o 1 dl 0 ref 2 fl Rpc:/0/0 rc 0/07531:0:6784:0:(ldlm_resource.c:518:ldlm_namespace_cleanup()) Namespace AC3-OST0008-osc-ffff81043e754c00 resource refcount nonzero (1) after lock cleanup; forcing cleanup.</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000100:00020000:2:1278393020.782563:0:3571:0:(client.c:858:ptlrpc_import_delay_req()) @@@ IMP_INVALID req@ffff81042061b400 x1340041378196329/t0 o4->AC3-OST0008_UUID@172.16.16.2@tcp:6/4 lens 496/616 e 0 o 1 dl 0 ref 2 fl Rpc:/0/0 rc 0/07583:0:6686:0:(lov_obd.c:1727:lov_brw_check()) lov idx 8 inactive</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000100:00020000:2:1278393020.783344:0:3571:0:(client.c:858:ptlrpc_import_delay_req()) @@@ IMP_INVALID req@ffff8104212fc400 x1340041378196334/t0 o3->AC3-OST0008_UUID@172.16.16.2@tcp:6/4 lens 448/592 e 0 o 1 dl 0 ref 2 fl Rpc:/0/0 rc 0/0</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000100:00020000:2:1278393020.783559:0:3571:0:(client.c:858:ptlrpc_import_delay_req()) @@@ IMP_INVALID req@ffff81042061ac00 x1340041378196337/t0 o4->AC3-OST0008_UUID@172.16.16.2@tcp:6/4 lens 448/608 e 0 o 1 dl 0 ref 2 fl Rpc:/0/0 rc 0/09394:0:6686:0:(lov_request.c:325:lov_prep_enqueue_set()) lov idx 8 inactive</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000100:00020000:2:1278393020.783727:0:3571:0:(client.c:858:ptlrpc_import_delay_req()) @@@ IMP_INVALID req@ffff81042061c800 x1340041378196341/t0 o4->AC3-OST0008_UUID@172.16.16.2@tcp:6/4 lens 496/616 e 0 o 1 dl 0 ref 2 fl Rpc:/0/0 rc 0/0</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00020000:00080000:7:1278393020.783778:0:6785:0:(lov_request.c:325:lov_prep_enqueue_set()) lov idx 8 inactive</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000080:00020000:7:1278393020.783782:0:6785:0:(file.c:995:ll_glimpse_size()) obd_enqueue returned rc -5, returning -EIO</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000100:00020000:2:1278393020.784052:0:3571:0:(client.c:858:ptlrpc_import_delay_req()) @@@ IMP_INVALID req@ffff81042061bc00 x1340041378196347/t0 o4->AC3-OST0008_UUID@172.16.16.2@tcp:6/4 lens 448/608 e 0 o 1 dl 0 ref 2 fl Rpc:/0/0 rc 0/0</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000100:00020000:2:1278393020.784197:0:3571:0:(client.c:858:ptlrpc_import_delay_req()) @@@ IMP_INVALID req@ffff81042061cc00 x1340041378196354/t0 o4->AC3-OST0008_UUID@172.16.16.2@tcp:6/4 lens 480/616 e 0 o 1 dl 0 ref 2 fl Rpc:/0/0 rc 0/03782:0:6785:0:(file.c:995:ll_glimpse_size()) obd_enqueue returned rc -5, returning -EIO</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000100:00020000:2:1278393020.784325:0:3571:0:(client.c:858:ptlrpc_import_delay_req()) @@@ IMP_INVALID req@ffff81042061b000 x1340041378196360/t0 o4->AC3-OST0008_UUID@172.16.16.2@tcp:6/4 lens 496/616 e 0 o 1 dl 0 ref 2 fl Rpc:/0/0 rc 0/0</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000100:00020000:2:1278393020.784436:0:3571:0:(client.c:858:ptlrpc_import_delay_req()) @@@ IMP_INVALID req@ffff81042061b800 x1340041378196369/t0 o4->AC3-OST0008_UUID@172.16.16.2@tcp:6/4 lens 464/608 e 0 o 1 dl 0 ref 2 fl Rpc:/0/0 rc 0/0</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">o 1 dl 0 ref 2 fl Rpc:/0/0 rc 0/0</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000100:00080000:0:1278393020.792064:0:6784:0:(import.c:1224:ptlrpc_invalidate_import_thread()) ffff810420d1a000 AC3-OST0008_UUID: changing import state from EVICTED to RECOVER</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000100:00080000:0:1278393020.792068:0:6784:0:(import.c:1314:ptlrpc_import_recovery_state_machine()) reconnected to AC3-OST0008_UUID@172.16.16.2@tcp</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000100:00080000:0:1278393020.792071:0:6784:0:(import.c:1319:ptlrpc_import_recovery_state_machine()) ffff810420d1a000 AC3-OST0008_UUID: changing import state from RECOVER to FULL</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00020000:01000000:0:1278393020.792078:0:6784:0:(lov_obd.c:567:lov_set_osc_active()) Marking OSC AC3-OST0008_UUID active</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000002:00080000:0:1278393020.792087:0:6784:0:(mdc_request.c:1465:mdc_init_ea_size()) updating max_mdsize/max_cookiesize: 288/320</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000100:02000000:0:1278393020.792092:0:6784:0:(import.c:1327:ptlrpc_import_recovery_state_machine()) AC3-OST0008-osc-ffff81043e754c00: Connection restored to service AC3-OST0008 using nid 172.16.16.2@tcp.</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000400:00000100:4:1278393020.806533:0:3561:0:(lib-move.c:1818:lnet_parse_put()) Dropping PUT from 12345-172.16.16.2@tcp portal 8 match 1340041378188715 offset 0 length 1048576: 2</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000400:00000100:4:1278393020.814069:0:3561:0:(lib-move.c:1818:lnet_parse_put()) Dropping PUT from 12345-172.16.16.2@tcp portal 8 match 1340041378188716 offset 0 length 1048576: 2</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000400:00000100:4:1278393020.828140:0:3561:0:(lib-move.c:1818:lnet_parse_put()) Dropping PUT from 12345-172.16.16.2@tcp portal 8 match 1340041378188729 offset 0 length 1048576: 2</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000400:00000100:4:1278393020.834691:0:3561:0:(lib-move.c:1818:lnet_parse_put()) Dropping PUT from 12345-172.16.16.2@tcp portal 8 match 1340041378188730 offset 0 length 1048576: 2</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000400:00000100:4:1278393020.841916:0:3561:0:(lib-move.c:1818:lnet_parse_put()) Dropping PUT from 12345-172.16.16.2@tcp portal 8 match 1340041378188736 offset 0 length 1048576: 2</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000100:02020000:2:1278394241.028730:0:7780:0:(client.c:975:ptlrpc_check_status()) 11-0: an error occurred while communicating with 172.16.16.2@tcp. The ldlm_enqueue operation failed with -107</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000100:00080000:2:1278394241.028748:0:7780:0:(recover.c:196:ptlrpc_request_handle_notconn()) import AC3-OST0006-osc-ffff81043e754c00 of AC3-OST0006_UUID@172.16.16.2@tcp abruptly disconnected: reconnecting</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">rations using this service will wait for recovery to complete.</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000100:00080000:2:1278394241.028763:0:7780:0:(import.c:167:ptlrpc_set_import_discon()) ffff81042018c000 AC3-OST0006_UUID: changing import state from FULL to DISCONN</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000100:00080000:2:1278394241.028769:0:7780:0:(import.c:608:ptlrpc_connect_import()) ffff81042018c000 AC3-OST0006_UUID: changing import state from DISCONN to CONNECTING</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000100:00080000:2:1278394241.028772:0:7780:0:(import.c:470:import_select_connection()) AC3-OST0006-osc-ffff81043e754c00: connect to NID 172.16.16.2@tcp last attempt 4724724153</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000100:00080000:2:1278394241.028783:0:7780:0:(import.c:544:import_select_connection()) AC3-OST0006-osc-ffff81043e754c00: import ffff81042018c000 using connection 172.16.16.2@tcp/172.16.16.2@tcp</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">6.16.2@tcp:28/4 lens 296/544 e 0 to 1 dl 1278394306 ref 1 fl Rpc:RS/0/0 rc 0/-107</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">g</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000100:00080000:2:1278394241.029211:0:22741:0:(import.c:182:ptlrpc_set_import_discon()) osc: import ffff81042018c000 already not connected (conn 180, was 179): CONNECTING</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00010000:00020000:2:1278394241.029215:0:22741:0:(ldlm_request.c:1025:ldlm_cli_cancel_req()) Got rc -107 from cancel RPC: canceling anyway</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00010000:00020000:2:1278394241.029225:0:22741:0:(ldlm_request.c:1587:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -107</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000100:00080000:2:1278394241.029446:0:3572:0:(import.c:908:ptlrpc_connect_interpret()) @@@ evicting (not initial connect and flags reconnect/recovering not set: 4) req@ffff8102e5404400 x1340041383549552/t0 o8->AC3-OST0006_UUID@172.16.16.2@tcp:28/4 lens 368/584 e 0 to 1 dl 1278394268 ref 1 fl Interpret:RN/0/0 rc 0/0</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000100:00080000:2:1278394241.029451:0:3572:0:(import.c:911:ptlrpc_connect_interpret()) ffff81042018c000 AC3-OST0006_UUID: changing import state from CONNECTING to EVICTED</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000100:02020000:2:1278394241.029454:0:3572:0:(import.c:1249:ptlrpc_import_recovery_state_machine()) 167-0: This client was evicted by AC3-OST0006; in progress operations using this service will fail.</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000100:00080000:2:1278394241.029460:0:3572:0:(import.c:1253:ptlrpc_import_recovery_state_machine()) evicted from AC3-OST0006_UUID@172.16.16.2@tcp; invalidating</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000002:00080000:2:1278394241.029546:0:3572:0:(mdc_request.c:1465:mdc_init_ea_size()) updating max_mdsize/max_cookiesize: 288/320</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000100:00080000:2:1278394241.029564:0:7781:0:(import.c:1215:ptlrpc_invalidate_import_thread()) thread invalidate import AC3-OST0006-osc-ffff81043e754c00 to AC3-OST0006_UUID@172.16.16.2@tcp</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000100:00080000:2:1278394241.029567:0:7781:0:(import.c:194:ptlrpc_deactivate_and_unlock_import()) setting import AC3-OST0006_UUID INVALID</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000100:00080000:2:1278394241.029594:0:7780:0:(client.c:2098:ptlrpc_queue_wait()) @@@ "mv" awake: (EVICTED == FULL or 1/0 == 1) req@ffff81012b338800 x1340041383514587/t0 o101->AC3-OST0006_UUID@172.16.16.2@tcp:28/4 lens 296/544 e 0 to 1 dl 1278394306 ref 1 fl Rpc:RES/0/0 rc -4/-107</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000100:00020000:2:1278394241.029668:0:7780:0:(client.c:858:ptlrpc_import_delay_req()) @@@ IMP_INVALID req@ffff81012b338800 x1340041383549557/t0 o101->AC3-OST0006_UUID@172.16.16.2@tcp:28/4 lens 296/544 e 0 to 1 dl 0 ref 1 fl Rpc:/0/0 rc 0/0</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00020000:00020000:2:1278394241.029685:0:7780:0:(lov_request.c:209:lov_update_enqueue_set()) enqueue objid 0x18ea266 subobj 0x80ab49 on OST idx 6: rc -108</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00020000:01000000:2:1278394241.030074:0:7781:0:(lov_obd.c:567:lov_set_osc_active()) Marking OSC AC3-OST0006_UUID inactive</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000002:00080000:2:1278394241.030079:0:7781:0:(mdc_request.c:1465:mdc_init_ea_size()) updating max_mdsize/max_cookiesize: 288/320</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000100:00020000:7:1278394241.052192:0:3571:0:(client.c:858:ptlrpc_import_delay_req()) @@@ IMP_INVALID req@ffff81032a347800 x1340041383549562/t0 o3->AC3-OST0006_UUID@172.16.16.2@tcp:6/4 lens 448/592 e 0 to 1 dl 0 ref 2 fl Rpc:/0/0 rc 0/0</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000100:00020000:7:1278394241.052453:0:3571:0:(client.c:858:ptlrpc_import_delay_req()) @@@ IMP_INVALID req@ffff810436b9ac00 x1340041383549566/t0 o3->AC3-OST0006_UUID@172.16.16.2@tcp:6/4 lens 448/592 e 0 to 1 dl 0 ref 2 fl Rpc:/0/0 rc 0/0</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000100:00020000:7:1278394241.052749:0:3571:0:(client.c:858:ptlrpc_import_delay_req()) @@@ IMP_INVALID req@ffff8101767fe000 x1340041383549569/t0 o3->AC3-OST0006_UUID@172.16.16.2@tcp:6/4 lens 448/592 e 0 to 1 dl 0 ref 2 fl Rpc:/0/0 rc 0/0</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000100:00020000:7:1278394241.053042:0:3571:0:(client.c:858:ptlrpc_import_delay_req()) @@@ IMP_INVALID req@ffff81012a81ac00 x1340041383549573/t0 o3->AC3-OST0006_UUID@172.16.16.2@tcp:6/4 lens 448/592 e 0 to 1 dl 0 ref 2 fl Rpc:/0/0 rc 0/0</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">o 1 dl 0 ref 2 fl Rpc:/0/0 rc 0/0</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">o 1 dl 0 ref 2 fl Rpc:/0/0 rc 0/0</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">o 1 dl 0 ref 2 fl Rpc:/0/0 rc 0/0</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">o 1 dl 0 ref 2 fl Rpc:/0/0 rc 0/0</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">o 1 dl 0 ref 2 fl Rpc:/0/0 rc 0/0</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">o 1 dl 0 ref 2 fl Rpc:/0/0 rc 0/0</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">o 1 dl 0 ref 2 fl Rpc:/0/0 rc 0/0</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">o 1 dl 0 ref 2 fl Rpc:/0/0 rc 0/0</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">o 1 dl 0 ref 2 fl Rpc:/0/0 rc 0/0</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00020000:00080000:7:1278394241.221422:0:7301:0:(lov_request.c:325:lov_prep_enqueue_set()) lov idx 6 inactive</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000080:00020000:7:1278394241.221429:0:7301:0:(file.c:995:ll_glimpse_size()) obd_enqueue returned rc -5, returning -EIO</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00020000:00080000:6:1278394241.232801:0:7640:0:(lov_request.c:325:lov_prep_enqueue_set()) lov idx 6 inactive</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000100:00080000:2:1278394241.435957:0:7781:0:(import.c:1224:ptlrpc_invalidate_import_thread()) ffff81042018c000 AC3-OST0006_UUID: changing import state from EVICTED to RECOVER</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000100:00080000:2:1278394241.435965:0:7781:0:(import.c:1314:ptlrpc_import_recovery_state_machine()) reconnected to AC3-OST0006_UUID@172.16.16.2@tcp</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000100:00080000:2:1278394241.435968:0:7781:0:(import.c:1319:ptlrpc_import_recovery_state_machine()) ffff81042018c000 AC3-OST0006_UUID: changing import state from RECOVER to FULL</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00020000:01000000:2:1278394241.435975:0:7781:0:(lov_obd.c:567:lov_set_osc_active()) Marking OSC AC3-OST0006_UUID active</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000002:00080000:2:1278394241.435983:0:7781:0:(mdc_request.c:1465:mdc_init_ea_size()) updating max_mdsize/max_cookiesize: 288/320</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">00000100:02000000:2:1278394241.435989:0:7781:0:(import.c:1327:ptlrpc_import_recovery_state_machine()) AC3-OST0006-osc-ffff81043e754c00: Connection restored to service AC3-OST0006 using nid 172.16.16.2@tcp.</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">ut (7s prior to deadline).</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; "> req@ffff810346c34c00 x1340041411931387/t0 o101->AC3-OST0002_UUID@172.16.16.4@tcp:28/4 lens 296/544 e 0 to 1 dl 1278419851 ref 1 fl Rpc:/0/0 rc 0/0</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">erations using this service will wait for recovery to complete.</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; min-height: 14px; "><br></div><div>
<div><span class="Apple-style-span" style="font-family: Tahoma, Verdana, Helvetica, Arial; font-size: 13px; "><br class="Apple-interchange-newline">Peter Kitchener<br>Systems Administrator<br>Capital Markets CRC Limited (CMCRC)<br>Telephone: +61 2 8088 4223<br>Fax: +61 2 8088 4201<br><font color="#0000ff"><a href="http://www.cmcrc.com/" target="_blank">www.cmcrc.com</a></font> <br><br> <br></span></div><span></span><br class="Apple-interchange-newline"><span><img height="87" width="253" id="f7cad48a-c973-4672-9a39-1d4599b5984d" apple-width="yes" apple-height="yes" src="cid:68C0C3AE-66F9-4BC4-A9DF-522D47B4E092@staff.cmcrc.com"></span><span class="Apple-style-span" style="border-collapse: separate; color: rgb(0, 0, 0); font-family: Helvetica; font-size: medium; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-align: auto; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; "><div><span class="Apple-style-span" style="border-collapse: separate; color: rgb(0, 0, 0); font-family: Helvetica; font-size: medium; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; "><span class="Apple-style-span" style="font-family: Tahoma, Verdana, Helvetica, Arial; font-size: 13px; "><br class="Apple-interchange-newline"><br>Capital Markets CRC Ltd - Confidential Communication<br>The information contained in this e-mail is confidential. It is intended for the addressee only. If you receive this e-mail by mistake please promptly inform us by reply e-mail and then delete the e-mail and destroy any printed copy. You must not disclose or use in any way the information in the e-mail. There is no warranty that this e-mail is error or virus free. It may be a private communication, and if so, does not represent the views of the CMCRC and its associates. If it is a private communication, care should be taken in opening it to ensure that undue offence is not given.</span></span></div><div><font class="Apple-style-span" face="Tahoma, Verdana, Helvetica, Arial" size="3"><span class="Apple-style-span" style="font-size: 13px; "><br></span></font></div></span><br class="Apple-interchange-newline">
</div>
<br></div></body></html>