<html><body><div style="color:#000; background-color:#fff; font-family:times new roman, new york, times, serif;font-size:12pt"><div><span>Jon;</span></div><div style="color: rgb(0, 0, 0); font-size: 16px; font-family: serif; background-color: transparent; font-style: normal;"><br><span></span></div><div style="color: rgb(0, 0, 0); font-size: 16px; font-family: serif; background-color: transparent; font-style: normal;"><span>Found this one today and noticed no response so I thought I would give it a go. From the manual it states to use zero partitions when possible and to deal entirely with the block device itself. The manual also states that the creation of an external journal is recommended. I used a separate 1TB disk for the creation of the external journal and in some instances swap space. <br></span></div><div style="color: rgb(0, 0, 0); font-size: 16px; font-family: serif; background-color: transparent; font-style:
normal;"><br><span></span></div><div style="color: rgb(0, 0, 0); font-size: 16px; font-family: serif; background-color: transparent; font-style: normal;"><span>Here I outline a small capable robust Configuration for under $40000.00:</span></div><div style="color: rgb(0, 0, 0); font-size: 16px; font-family: serif; background-color: transparent; font-style: normal;"><br><span></span></div><div style="color: rgb(0, 0, 0); font-size: 16px; font-family: serif; background-color: transparent; font-style: normal;"><span>[root@ns1 ~]# cat /proc/mdstat <br>Personalities : [raid1]
[raid10] <br>md3 : active raid10 sdc[0] sdh[5] sdg[4] sdf[3] sde[2] sdd[1] <br> 2930287488 blocks 64K chunks 2 near-copies [6/6] [UUUUUU] <br> <br>md1 : active raid1 sda3[0]
sdb3[1] <br> 891520 blocks [2/2] [UU] <br> <br>md0 : active raid1 sdb1[1]
sda1[0] <br> 965634880 blocks [2/2] [UU] <br> <br>unused devices:
<none> <br>[root@ns1 ~]# lctl dl <br> 0 UP mgc MGC192.168.0.10@tcp f038d71d-792a-16cd-9909-e24dbe6dcfd2 5<br> 1 UP ost OSS OSS_uuid 3 <br> 2 UP obdfilter
ioio-OST0002 ioio-OST0002_UUID 13 <br>[root@ns1 ~]# cat /etc/fstab <br>/dev/md0 / ext3 defaults 1 1<br>tmpfs /dev/shm tmpfs
defaults 0 0<br>devpts /dev/pts devpts gid=5,mode=620 0 0<br>sysfs /sys sysfs defaults 0 0<br>proc /proc proc defaults 0
0<br>LABEL=SWAP-sdb2 swap swap defaults 0 0<br>LABEL=SWAP-sda2 swap swap defaults 0 0<br>/dev/md3 /mnt/data/ost02 lustre defaults,_netdev 0 0<br>#/dev/md4 /mnt/flash/ost03 lustre defaults,_netdev 0 0<br><br>[root@ns1 ~]# cat
/proc/net/bonding<br>cat: /proc/net/bonding: Is a directory<br>[root@ns1 ~]# cat /proc/net/bond<br>cat: /proc/net/bond: No such file or directory<br>[root@ns1 ~]# cat /proc/net/bonding/bond0 <br>Ethernet Channel Bonding Driver: v3.4.0-1 (October 7, 2008)<br><br>Bonding Mode: IEEE 802.3ad Dynamic link aggregation<br>Transmit Hash Policy: layer3+4 (1) <br>MII Status: up <br>MII Polling Interval (ms): 10 <br>Up Delay (ms):
0 <br>Down Delay (ms): 0 <br><br>802.3ad info<br>LACP rate: slow<br>Active Aggregator Info:<br> Aggregator ID: 1<br> Number of ports: 5<br> Actor Key: 17 <br> Partner Key: 37823<br> Partner Mac Address: 00:22:57:9a:ea:00<br><br>Slave Interface: eth0<br>MII Status: up <br>Speed: 1000
Mbps <br>Duplex: full <br>Link Failure Count: 0<br>Permanent HW addr: 00:1b:21:28:74:d8<br>Aggregator ID: 1 <br><br>Slave Interface: eth1<br>MII Status: up <br>Speed: 1000 Mbps <br>Duplex: full <br>Link Failure Count: 1<br>Permanent HW addr: 00:1b:21:28:76:d2<br>Aggregator ID: 1 <br><br>Slave Interface: eth2<br>MII Status: up <br>Speed: 1000 Mbps <br>Duplex: full <br>Link Failure Count: 0<br>Permanent HW addr: 00:22:15:06:39:2f<br>Aggregator ID:
1 <br><br>Slave Interface: eth3<br>MII Status: up <br>Speed: 1000 Mbps <br>Duplex: full <br>Link Failure Count: 0<br>Permanent HW addr: 00:22:15:06:39:30<br>Aggregator ID: 1 <br><br>Slave Interface: eth4<br>MII Status: up <br>Speed: 1000 Mbps <br>Duplex: full <br>Link Failure Count: 0<br>Permanent HW addr: 00:22:15:06:39:31<br>Aggregator ID: 1 <br>[root@ns1 ~]#
ifconfig <br>bond0 Link encap:Ethernet HWaddr 00:1B:21:28:74:D8 <br> inet addr:192.168.0.19 Bcast:192.168.0.255 Mask:255.255.255.0<br> UP BROADCAST RUNNING MASTER MULTICAST MTU:1500 Metric:1 <br> RX packets:89680747 errors:0 dropped:493712 overruns:0 frame:0 <br> TX packets:54703009 errors:0 dropped:0 overruns:0 carrier:0 <br> collisions:0
txqueuelen:0 <br> RX bytes:87798998004 (81.7 GiB) TX bytes:137015921248 (127.6 GiB)<br><br>eth0 Link encap:Ethernet HWaddr 00:1B:21:28:74:D8 <br> UP BROADCAST RUNNING SLAVE MULTICAST MTU:1500 Metric:1<br> RX packets:6158033 errors:0 dropped:0 overruns:0 frame:0<br> TX packets:4754443 errors:0 dropped:0 overruns:0 carrier:0<br> collisions:0
txqueuelen:1000 <br> RX bytes:2572939264 (2.3 GiB) TX bytes:6124172876 (5.7 GiB)<br><br>eth1 Link encap:Ethernet HWaddr 00:1B:21:28:74:D8 <br> UP BROADCAST RUNNING SLAVE MULTICAST MTU:1500 Metric:1<br> RX packets:1715604 errors:0 dropped:0 overruns:0 frame:0<br> TX packets:3837069 errors:0 dropped:0 overruns:0 carrier:0<br> collisions:0
txqueuelen:1000 <br> RX bytes:105078923 (100.2 MiB) TX bytes:2128255017 (1.9 GiB)<br><br>eth2 Link encap:Ethernet HWaddr 00:1B:21:28:74:D8 <br> UP BROADCAST RUNNING SLAVE MULTICAST MTU:1500 Metric:1<br> RX packets:141703 errors:0 dropped:0 overruns:0 frame:0 <br> TX packets:3999225 errors:0 dropped:0 overruns:0 carrier:0<br> collisions:0
txqueuelen:1000 <br> RX bytes:14455587 (13.7 MiB) TX bytes:2440673445 (2.2 GiB)<br> Interrupt:74 <br><br>eth3 Link encap:Ethernet HWaddr 00:1B:21:28:74:D8 <br> UP BROADCAST RUNNING SLAVE MULTICAST MTU:1500 Metric:1<br> RX packets:63575447
errors:0 dropped:493696 overruns:0 frame:0<br> TX packets:14850445 errors:0 dropped:0 overruns:0 carrier:0 <br> collisions:0 txqueuelen:1000 <br> RX bytes:83036911085 (77.3 GiB) TX bytes:122278775783 (113.8 GiB)<br>
Interrupt:98 <br><br>eth4 Link encap:Ethernet HWaddr 00:1B:21:28:74:D8 <br> UP BROADCAST RUNNING SLAVE MULTICAST MTU:1500 Metric:1<br> RX packets:18089968 errors:0 dropped:16 overruns:0 frame:0<br> TX packets:27261838 errors:0 dropped:0 overruns:0 carrier:0<br> collisions:0 txqueuelen:1000<br> RX bytes:2069613561 (1.9
GiB) TX bytes:4044047045 (3.7 GiB)<br> Interrupt:169<br><br>eth5 Link encap:Ethernet HWaddr 00:22:15:06:39:32<br> inet addr:XX.XX.XXX.XXX Bcast:96.53.120.255 Mask:255.255.255.0<br> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1<br> RX packets:8650228 errors:0 dropped:4716 overruns:0 frame:0<br> TX packets:4278085 errors:0 dropped:0 overruns:0 carrier:0<br> collisions:0 txqueuelen:1000<br> RX bytes:7047416207 (6.5 GiB) TX bytes:1721168584 (1.6 GiB)<br>
Interrupt:177<br><br>lo Link encap:Local Loopback<br> inet addr:127.0.0.1 Mask:255.0.0.0<br> UP LOOPBACK RUNNING MTU:16436 Metric:1<br> RX packets:211949 errors:0 dropped:0 overruns:0 frame:0<br> TX packets:211949 errors:0 dropped:0 overruns:0 carrier:0<br> collisions:0 txqueuelen:0<br> RX bytes:48836875 (46.5 MiB) TX bytes:48836875 (46.5 MiB)<br><br>[root@ns1 ~]# uptime<br> 12:25:59 up 7 days, 16:49, 5 users, load average: 0.00, 0.02, 0.00]#<br></span></div><div style="color: rgb(0, 0, 0); font-size: 16px; font-family: serif; background-color:
transparent; font-style: normal;"><br><span></span></div><div style="color: rgb(0, 0, 0); font-size: 16px; font-family: serif; background-color: transparent; font-style: normal;"><span>**Firewall/DNS External IP eth5 OSS/OST</span></div><div style="color: rgb(0, 0, 0); font-size: 16px; font-family: serif; background-color: transparent; font-style: normal;"><br><span></span></div><div style="color: rgb(0, 0, 0); font-size: 16px; font-family: serif; background-color: transparent; font-style: normal;"><span>[root@ns2 ~]# cat /proc/mdstat <br>Personalities : [raid1] [raid10] <br>md3 : active raid10 sdc[0] sdh[5] sdg[4] sdf[3] sde[2]
sdd[1]<br> 2930287488 blocks 64K chunks 2 near-copies [6/6] [UUUUUU]<br> <br>md1 : active raid1 sda3[0] sdb3[1] <br> 891520 blocks [2/2] [UU]
<br> <br>md0 : active raid1 sdb1[1] sda1[0] <br> 965634880 blocks [2/2] [UU]
<br> <br>unused devices: <none> <br>[root@ns2 ~]# lctl dl <br> 0 UP mgc MGC192.168.0.10@tcp
0e72db5f-9495-e31e-85c2-8d31a009d130 5<br> 1 UP ost OSS OSS_uuid 3 <br> 2 UP obdfilter ioio-OST0001 ioio-OST0001_UUID 13 <br>[root@ns2 ~]# cat /etc/fstab <br>/dev/md0
/ ext3 defaults 1 1<br>tmpfs /dev/shm tmpfs defaults 0 0<br>devpts /dev/pts devpts gid=5,mode=620 0 0<br>sysfs
/sys sysfs defaults 0 0<br>proc /proc proc defaults 0 0<br>LABEL=SWAP-sdb2 swap swap defaults 0 0<br>LABEL=SWAP-sda2 swap
swap defaults 0 0<br>/dev/md3 /mnt/data/ost01 lustre defaults,_netdev 0 0<br>#/dev/md4 /mnt/flash/ost02 lustre defaults,_netdev 0 0<br><br>[root@ns2 ~]# cat /proc/net/bonding/bond0 <br>Ethernet Channel Bonding Driver: v3.4.0-1 (October 7, 2008)<br><br>Bonding Mode: IEEE 802.3ad Dynamic link aggregation<br>Transmit Hash Policy: layer3+4 (1) <br>MII Status:
up <br>MII Polling Interval (ms): 10 <br>Up Delay (ms): 0 <br>Down Delay (ms): 0 <br><br>802.3ad info<br>LACP rate: slow<br>Active Aggregator Info:<br> Aggregator ID:
1<br> Number of ports: 5<br> Actor Key: 17 <br> Partner Key: 37823<br> Partner Mac Address: 00:22:57:9a:ea:00<br><br>Slave Interface: eth0<br>MII Status: up <br>Speed: 1000 Mbps <br>Duplex: full <br>Link Failure Count: 0<br>Permanent HW addr: 00:1b:21:2a:1c:3c<br>Aggregator ID: 1 <br><br>Slave Interface: eth1<br>MII Status: up <br>Speed: 1000 Mbps <br>Duplex: full <br>Link Failure Count: 0<br>Permanent HW addr:
00:1b:21:2a:1c:34<br>Aggregator ID: 1 <br><br>Slave Interface: eth2<br>MII Status: up <br>Speed: 1000 Mbps <br>Duplex: full <br>Link Failure Count: 0<br>Permanent HW addr: 00:22:15:06:38:b3<br>Aggregator ID: 1 <br><br>Slave Interface: eth3<br>MII Status: up <br>Speed: 1000 Mbps <br>Duplex: full <br>Link Failure Count: 0<br>Permanent HW addr: 00:22:15:06:38:b4<br>Aggregator ID: 1 <br><br>Slave Interface:
eth4<br>MII Status: up <br>Speed: 1000 Mbps <br>Duplex: full <br>Link Failure Count: 0<br>Permanent HW addr: 00:22:15:06:38:b5<br>Aggregator ID: 1 <br>[root@ns2 ~]# ifconfig <br>bond0 Link encap:Ethernet HWaddr 00:1B:21:2A:1C:3C <br> inet addr:192.168.0.13 Bcast:192.168.0.255 Mask:255.255.255.0<br> UP BROADCAST RUNNING MASTER MULTICAST MTU:1500 Metric:1 <br> RX packets:73883791 errors:0 dropped:405040 overruns:0
frame:0 <br> TX packets:89400059 errors:0 dropped:0 overruns:0 carrier:0 <br> collisions:0 txqueuelen:0 <br> RX bytes:74258769521 (69.1 GiB) TX bytes:116179989862 (108.2 GiB)<br><br>eth0 Link encap:Ethernet HWaddr 00:1B:21:2A:1C:3C <br> UP BROADCAST RUNNING SLAVE MULTICAST MTU:1500 Metric:1<br> RX packets:53224496 errors:0 dropped:2 overruns:0
frame:0<br> TX packets:37354638 errors:0 dropped:0 overruns:0 carrier:0<br> collisions:0 txqueuelen:1000 <br> RX bytes:70370076932 (65.5 GiB) TX bytes:51859648442 (48.2 GiB)<br><br>eth1 Link encap:Ethernet HWaddr 00:1B:21:2A:1C:3C <br> UP BROADCAST RUNNING SLAVE MULTICAST MTU:1500 Metric:1<br> RX packets:20567731 errors:0 dropped:0 overruns:0 frame:0<br> TX packets:41842603 errors:0 dropped:0
overruns:0 carrier:0<br> collisions:0 txqueuelen:1000 <br> RX bytes:3880185151 (3.6 GiB) TX bytes:33224132469 (30.9 GiB)<br><br>eth2 Link encap:Ethernet HWaddr 00:1B:21:2A:1C:3C <br> UP BROADCAST RUNNING SLAVE MULTICAST MTU:1500 Metric:1<br> RX packets:46143 errors:0 dropped:0 overruns:0 frame:0 <br> TX packets:2055392 errors:0 dropped:0 overruns:0 carrier:0<br> collisions:0
txqueuelen:1000 <br> RX bytes:3573296 (3.4 MiB) TX bytes:487616806 (465.0 MiB)<br> Interrupt:74 <br><br>eth3 Link encap:Ethernet HWaddr 00:1B:21:2A:1C:3C <br> UP BROADCAST RUNNING SLAVE MULTICAST MTU:1500 Metric:1<br> RX packets:22228 errors:0
dropped:0 overruns:0 frame:0 <br> TX packets:2406691 errors:0 dropped:0 overruns:0 carrier:0<br> collisions:0 txqueuelen:1000 <br> RX bytes:2445080 (2.3 MiB) TX bytes:675387313 (644.0 MiB)<br> Interrupt:98 <br><br>eth4 Link encap:Ethernet HWaddr
00:1B:21:2A:1C:3C <br> UP BROADCAST RUNNING SLAVE MULTICAST MTU:1500 Metric:1<br> RX packets:23207 errors:0 dropped:405038 overruns:0 frame:0<br> TX packets:5740755 errors:0 dropped:0 overruns:0 carrier:0<br> collisions:0 txqueuelen:1000<br> RX bytes:2489986 (2.3 MiB) TX bytes:29933208120 (27.8 GiB)<br> Interrupt:169<br><br>eth5 Link encap:Ethernet HWaddr 00:22:15:06:38:B6<br> inet addr:XX.XX.XXX.XXX Bcast:96.53.120.255 Mask:255.255.255.0<br> UP BROADCAST
RUNNING MULTICAST MTU:1500 Metric:1<br> RX packets:5679458 errors:0 dropped:124836 overruns:0 frame:0<br> TX packets:1640529 errors:0 dropped:0 overruns:0 carrier:0<br> collisions:0 txqueuelen:1000<br> RX bytes:329507734 (314.2 MiB) TX bytes:1288265410 (1.1 GiB)<br> Interrupt:177<br><br>lo Link encap:Local Loopback<br> inet addr:127.0.0.1 Mask:255.0.0.0<br> UP LOOPBACK RUNNING MTU:16436 Metric:1<br> RX packets:33230 errors:0 dropped:0 overruns:0
frame:0<br> TX packets:33230 errors:0 dropped:0 overruns:0 carrier:0<br> collisions:0 txqueuelen:0<br> RX bytes:24813551 (23.6 MiB) TX bytes:24813551 (23.6 MiB)<br><br>[root@ns2 ~]# uptime<br> 12:28:08 up 7 days, 16:52, 1 user, load average: 0.37, 0.25, 0.19<br></span></div><div style="color: rgb(0, 0, 0); font-size: 16px; font-family: serif; background-color: transparent; font-style: normal;"><br><span></span></div><div style="color: rgb(0, 0, 0); font-size: 16px; font-family: serif; background-color: transparent; font-style: normal;"><span>** Firewall/DNS External IP OSS/OST</span></div><div style="color: rgb(0, 0, 0); font-size: 16px; font-family: serif; background-color: transparent; font-style: normal;"><br><span></span></div><div style="color: rgb(0, 0, 0);
font-size: 16px; font-family: serif; background-color: transparent; font-style: normal;"><span>[root@ns3 ~]# cat /proc/mdstat <br>Personalities : [raid1] [raid10] <br>md3 : active raid10 sdc[0] sdh[5] sdg[4] sdf[3] sde[2] sdd[1] <br> 2930287488 blocks 64K chunks 2 near-copies [6/6] [UUUUUU]
<br> <br>md1 : active raid1 sda3[0] sdb3[1] <br> 891520 blocks [2/2] [UU]
<br> <br>md0 : active raid1 sdb1[1] sda1[0] <br> 965634880 blocks [2/2] [UU]
<br> <br>unused devices: <none> <br>[root@ns3 ~]# lctl
dl <br> 0 UP mgc MGC192.168.0.10@tcp 75c57524-4f99-6424-f78a-eaff14d115bd 5<br> 1 UP ost OSS OSS_uuid 3 <br> 2 UP obdfilter ioio-OST0000 ioio-OST0000_UUID 13 <br>[root@ns3 ~]# cat
/etc/fstab <br>/dev/md0 / ext3 defaults 1 1<br>tmpfs /dev/shm tmpfs defaults 0 0<br>devpts
/dev/pts devpts gid=5,mode=620 0 0<br>sysfs /sys sysfs defaults 0 0<br>proc /proc proc defaults 0 0<br>LABEL=SWAP-sdb2 swap
swap defaults 0 0<br>LABEL=SWAP-sda2 swap swap defaults 0 0<br>/dev/md3 /mnt/data/ost00 lustre defaults,_netdev 0 0<br>#/dev/md4 /mnt/flash/ost01 lustre defaults,_netdev 0 0<br><br>[root@ns3 ~]# cat /proc/net/bonding/bond0 <br>Ethernet Channel Bonding Driver: v3.4.0-1 (October 7, 2008)<br><br>Bonding Mode: IEEE 802.3ad Dynamic link aggregation<br>Transmit Hash Policy: layer3+4
(1) <br>MII Status: up <br>MII Polling Interval (ms): 10 <br>Up Delay (ms): 0 <br>Down Delay (ms): 0 <br><br>802.3ad info<br>LACP
rate: slow<br>Active Aggregator Info:<br> Aggregator ID: 5<br> Number of ports: 6<br> Actor Key: 17 <br> Partner Key: 37823<br> Partner Mac Address: 00:22:57:9a:ea:00<br><br>Slave Interface: eth0<br>MII Status: up <br>Speed: 1000 Mbps <br>Duplex: full <br>Link Failure Count: 0<br>Permanent HW addr: 00:22:15:06:3a:0f<br>Aggregator ID: 5 <br><br>Slave Interface: eth1<br>MII Status: up <br>Speed: 1000 Mbps <br>Duplex:
full <br>Link Failure Count: 0<br>Permanent HW addr: 00:22:15:06:3a:10<br>Aggregator ID: 5 <br><br>Slave Interface: eth2<br>MII Status: up <br>Speed: 1000 Mbps <br>Duplex: full <br>Link Failure Count: 0<br>Permanent HW addr: 00:22:15:06:3a:11<br>Aggregator ID: 5 <br><br>Slave Interface: eth3<br>MII Status: up <br>Speed: 1000 Mbps <br>Duplex: full <br>Link Failure Count: 0<br>Permanent HW addr: 00:22:15:06:3a:12<br>Aggregator ID:
5 <br><br>Slave Interface: eth4<br>MII Status: up <br>Speed: 1000 Mbps <br>Duplex: full <br>Link Failure Count: 0<br>Permanent HW addr: 00:1b:21:2a:17:76<br>Aggregator ID: 5 <br><br>Slave Interface: eth5<br>MII Status: up <br>Speed: 1000 Mbps <br>Duplex: full <br>Link Failure Count: 0<br>Permanent HW addr: 00:1b:21:2a:1c:dc<br>Aggregator ID: 5 <br>[root@ns3 ~]#
ifconfig <br>bond0 Link encap:Ethernet HWaddr 00:22:15:06:3A:0F <br> inet addr:192.168.0.7 Bcast:192.168.0.255 Mask:255.255.255.0<br> inet6 addr: fe80::222:15ff:fe06:3a0f/64 Scope:Link <br> UP BROADCAST RUNNING MASTER MULTICAST MTU:1500 Metric:1 <br> RX packets:91510283 errors:0 dropped:364706 overruns:0 frame:0<br> TX packets:69302068 errors:0 dropped:0 overruns:0 carrier:0 <br> collisions:0
txqueuelen:0 <br> RX bytes:55958008995 (52.1 GiB) TX bytes:223016561848 (207.7 GiB)<br><br>eth0 Link encap:Ethernet HWaddr 00:22:15:06:3A:0F <br> UP BROADCAST RUNNING SLAVE MULTICAST MTU:1500 Metric:1<br> RX packets:42433 errors:0 dropped:2 overruns:0 frame:0 <br> TX packets:9926913 errors:0 dropped:0 overruns:0 carrier:0<br> collisions:0
txqueuelen:1000 <br> RX bytes:3374702 (3.2 MiB) TX bytes:851799191 (812.3 MiB)<br> Interrupt:74 <br><br>eth1 Link encap:Ethernet HWaddr 00:22:15:06:3A:0F <br> UP BROADCAST RUNNING SLAVE MULTICAST MTU:1500 Metric:1<br> RX packets:1712310 errors:0
dropped:0 overruns:0 frame:0<br> TX packets:266655 errors:0 dropped:0 overruns:0 carrier:0<br> collisions:0 txqueuelen:1000 <br> RX bytes:80908257 (77.1 MiB) TX bytes:208988501 (199.3 MiB)<br> Interrupt:98 <br><br>eth2 Link encap:Ethernet HWaddr
00:22:15:06:3A:0F <br> UP BROADCAST RUNNING SLAVE MULTICAST MTU:1500 Metric:1<br> RX packets:8953421 errors:0 dropped:364702 overruns:0 frame:0<br> TX packets:27700345 errors:0 dropped:0 overruns:0 carrier:0 <br> collisions:0 txqueuelen:1000 <br> RX bytes:952280437 (908.1 MiB) TX bytes:182598635220 (170.0 GiB)<br>
Interrupt:169 <br><br>eth3 Link encap:Ethernet HWaddr 00:22:15:06:3A:0F <br> UP BROADCAST RUNNING SLAVE MULTICAST MTU:1500 Metric:1<br> RX packets:23273 errors:0 dropped:2 overruns:0 frame:0 <br> TX packets:5192617 errors:0 dropped:0 overruns:0 carrier:0<br> collisions:0
txqueuelen:1000 <br> RX bytes:2493150 (2.3 MiB) TX bytes:1880125062 (1.7 GiB) <br> Interrupt:177 <br><br>eth4 Link encap:Ethernet HWaddr 00:22:15:06:3A:0F <br> UP BROADCAST RUNNING SLAVE MULTICAST MTU:1500 Metric:1<br> RX packets:38058348 errors:0
dropped:0 overruns:0 frame:0<br> TX packets:1653728 errors:0 dropped:0 overruns:0 carrier:0<br> collisions:0 txqueuelen:1000<br> RX bytes:3642127587 (3.3 GiB) TX bytes:759108694 (723.9 MiB)<br><br>eth5 Link encap:Ethernet HWaddr 00:22:15:06:3A:0F<br> UP BROADCAST RUNNING SLAVE MULTICAST MTU:1500 Metric:1<br> RX packets:42720504 errors:0 dropped:0 overruns:0 frame:0<br> TX packets:24561812 errors:0 dropped:0 overruns:0 carrier:0<br> collisions:0 txqueuelen:1000<br> RX
bytes:51276825174 (47.7 GiB) TX bytes:36717905520 (34.1 GiB)<br><br>lo Link encap:Local Loopback<br> inet addr:127.0.0.1 Mask:255.0.0.0<br> inet6 addr: ::1/128 Scope:Host<br> UP LOOPBACK RUNNING MTU:16436 Metric:1<br> RX packets:2798 errors:0 dropped:0 overruns:0 frame:0<br> TX packets:2798 errors:0 dropped:0 overruns:0 carrier:0<br> collisions:0 txqueuelen:0<br> RX bytes:4431535 (4.2 MiB) TX bytes:4431535 (4.2 MiB)<br><br>[root@ns3 ~]# uptime<br> 12:30:58 up 7 days, 16:55, 1 user, load average:
0.00, 0.00, 0.00<br></span></div><div style="color: rgb(0, 0, 0); font-size: 16px; font-family: serif; background-color: transparent; font-style: normal;"><br><span></span></div><div style="color: rgb(0, 0, 0); font-size: 16px; font-family: serif; background-color: transparent; font-style: normal;"><span>** OSS/OST LAMP webroot=/var/www/html ie: not Lustre mounted</span></div><div style="color: rgb(0, 0, 0); font-size: 16px; font-family: serif; background-color: transparent; font-style: normal;"><br><span></span></div><div style="color: rgb(0, 0, 0); font-size: 16px; font-family: serif; background-color: transparent; font-style: normal;"><span>[root@ten ~]# cat /proc/mdstat <br>Personalities :
[raid1] <br>md0 : active raid1 sdc[0] sdb[1] <br> 976762496 blocks [2/2] [UU]
<br> <br>unused devices: <none> <br>[root@ten ~]# lctl
dl <br> 0 UP mgs MGS MGS 17 <br> 1 UP mgc MGC192.168.0.10@tcp 24fb4ce0-ea3f-1324-5d61-ea9f45f0baa8 5<br> 2 UP mdt MDS MDS_uuid 3 <br> 3 UP lov ioio-mdtlov
ioio-mdtlov_UUID 4 <br> 4 UP mds ioio-MDT0000 ioio-MDT0000_UUID 11 <br> 5 UP osc ioio-OST0000-osc ioio-mdtlov_UUID 5 <br> 6 UP osc ioio-OST0001-osc ioio-mdtlov_UUID 5 <br> 7 UP osc ioio-OST0002-osc ioio-mdtlov_UUID 5 <br> 8 UP lov
ioio-clilov-ffff8102179aa000 b47f5fbf-f9dc-7660-3488-6a0d3420f386 4<br> 9 UP mdc ioio-MDT0000-mdc-ffff8102179aa000 b47f5fbf-f9dc-7660-3488-6a0d3420f386 5<br> 10 UP osc ioio-OST0000-osc-ffff8102179aa000 b47f5fbf-f9dc-7660-3488-6a0d3420f386 5<br> 11 UP osc ioio-OST0001-osc-ffff8102179aa000 b47f5fbf-f9dc-7660-3488-6a0d3420f386 5<br> 12 UP osc ioio-OST0002-osc-ffff8102179aa000 b47f5fbf-f9dc-7660-3488-6a0d3420f386 5<br>[root@ten ~]# cat /etc/fstab <br>LABEL=/
/ ext3 defaults 1 1 <br>tmpfs /dev/shm tmpfs defaults 0 0 <br>devpts /dev/pts devpts gid=5,mode=620 0 0
<br>sysfs /sys sysfs defaults 0 0 <br>proc /proc proc defaults 0 0 <br>LABEL=SWAP-sda2 swap swap
defaults 0 0 <br>/dev/md0 /mnt/data/mdt lustre defaults,_netdev 0 0 <br>#/dev/sdd1 /mnt/data/flash lustre defaults,_netdev 0 0 <br><br>[root@ten ~]# cat /proc/net/bonding/bond0 <br>Ethernet Channel Bonding Driver: v3.4.0-1 (October 7, 2008)<br><br>Bonding Mode: IEEE 802.3ad Dynamic link aggregation<br>Transmit Hash Policy: layer3+4 (1) <br>MII Status:
up <br>MII Polling Interval (ms): 10 <br>Up Delay (ms): 0 <br>Down Delay (ms): 0 <br><br>802.3ad info<br>LACP rate: slow<br>Active Aggregator Info:<br> Aggregator ID:
5<br> Number of ports: 6<br> Actor Key: 17 <br> Partner Key: 37823<br> Partner Mac Address: 00:22:57:9a:ea:00<br><br>Slave Interface: eth0<br>MII Status: up <br>Speed: 1000 Mbps <br>Duplex: full <br>Link Failure Count: 0<br>Permanent HW addr: 00:22:15:06:3a:93<br>Aggregator ID: 5 <br><br>Slave Interface: eth1<br>MII Status: up <br>Speed: 1000 Mbps <br>Duplex: full <br>Link Failure Count: 0<br>Permanent HW addr:
00:22:15:06:3a:94<br>Aggregator ID: 5 <br><br>Slave Interface: eth2<br>MII Status: up <br>Speed: 1000 Mbps <br>Duplex: full <br>Link Failure Count: 0<br>Permanent HW addr: 00:22:15:06:3a:95<br>Aggregator ID: 5 <br><br>Slave Interface: eth3<br>MII Status: up<br>Speed: 1000 Mbps<br>Duplex: full<br>Link Failure Count: 0<br>Permanent HW addr: 00:00:00:00:00:00<br>Aggregator ID: 5<br><br>Slave Interface: eth4<br>MII Status: up<br>Speed: 1000 Mbps<br>Duplex: full<br>Link Failure Count: 0<br>Permanent HW addr: 00:1b:21:28:77:db<br>Aggregator ID: 5<br><br>Slave Interface: eth5<br>MII Status: up<br>Speed: 1000 Mbps<br>Duplex:
full<br>Link Failure Count: 1<br>Permanent HW addr: 00:1b:21:28:77:6c<br>Aggregator ID: 5<br>[root@ten ~]# uptime<br> 12:36:03 up 7 days, 16:59, 1 user, load average: 0.00, 0.01, 0.00</span></div><div style="color: rgb(0, 0, 0); font-size: 16px; font-family: serif; background-color: transparent; font-style: normal;">[root@ten ~]# lfs df -i<br>UUID Inodes IUsed IFree IUse% Mounted on<br>ioio-MDT0000_UUID 215600857 2046153 213554704 1% /mnt/ioio[MDT:0]<br>ioio-OST0000_UUID 183148544 656907 182491637 0% /mnt/ioio[OST:0]<br>ioio-OST0001_UUID
183148544 657442 182491102 0% /mnt/ioio[OST:1]<br>ioio-OST0002_UUID 183148544 668586 182479958 0% /mnt/ioio[OST:2]<br><br>filesystem summary: 215600857 2046153 213554704 1% /mnt/ioio<br><br></div><div style="color: rgb(0, 0, 0); font-size: 16px; font-family: serif; background-color: transparent; font-style: normal;"><span>** MGS/MDT Machine. Redundant Lustre Client. Redundant LAMP Server.<br></span></div><div style="color: rgb(0, 0, 0); font-size: 16px; font-family: serif; background-color: transparent; font-style: normal;"><br></div><div style="color: rgb(0, 0, 0); font-size: 16px; font-family: serif; background-color: transparent; font-style: normal;"><br></div><div style="color: rgb(0, 0, 0); font-size: 16px; font-family: serif;
background-color: transparent; font-style: normal;">Last login: Mon Jan 21 21:37:31 2013 <br>[root@three ~]# cat /proc/mdstat <br>Personalities : [raid6] [raid5] [raid4] <br>md0 : active raid5 sdb[1] sdd[2] sdc[0] sde[4] <br> 5860538880 blocks super 1.1 level 5, 512k chunk, algorithm 2 [4/4] [UUUU]<br> bitmap: 0/15 pages [0KB], 65536KB chunk <br><br>unused devices: <none><br>[root@three ~]# df -h
<br>Filesystem Size Used Avail Use% Mounted on<br>/dev/sda1 459G 18G 418G 5% / <br>tmpfs 1004M 37M 968M 4% /dev/shm <br>/dev/md0 5.4T 3.2T 2.0T 62% /mnt/rv <br>192.168.0.10@tcp0:/ioio <br> 8.1T
4.8T 3.0T 62% /mnt/ioio <br>[root@three ~]# lctl dl <br> 0 UP mgc MGC192.168.0.10@tcp 4aee8c8e-ab6f-d288-9cff-7953dff76905 5<br> 1 UP lov ioio-clilov-ffff880075023000 06e9543a-5540-afe2-8e6a-5f21d8085076 4<br> 2 UP mdc ioio-MDT0000-mdc-ffff880075023000 06e9543a-5540-afe2-8e6a-5f21d8085076 5<br> 3 UP osc ioio-OST0000-osc-ffff880075023000 06e9543a-5540-afe2-8e6a-5f21d8085076 5<br> 4 UP osc ioio-OST0001-osc-ffff880075023000 06e9543a-5540-afe2-8e6a-5f21d8085076 5<br> 5 UP osc ioio-OST0002-osc-ffff880075023000 06e9543a-5540-afe2-8e6a-5f21d8085076 5<br>[root@three ~]# cat
/etc/fstab <br><br>#<br># /etc/fstab<br># Created by anaconda on Mon Sep 3 07:44:32 2012<br># <br># Accessible filesystems, by reference, are maintained under '/dev/disk'<br># See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more
info<br># <br>UUID=06e1bb0d-d7c8-42a4-bad1-6d12f931b77a / ext4 defaults 1 1<br>tmpfs /dev/shm tmpfs defaults 0
0<br>devpts /dev/pts devpts gid=5,mode=620 0 0<br>sysfs /sys sysfs defaults 0 0<br>proc /proc proc defaults 0
0<br>/dev/md0 /mnt/rv ext4 defaults 0 0<br><br>[root@three ~]# ifconfig<br>bond0 Link encap:Ethernet HWaddr 00:22:15:06:3B:73 <br> inet addr:192.168.0.3 Bcast:192.168.0.255 Mask:255.255.255.0<br> inet6 addr: fe80::222:15ff:fe06:3b73/64 Scope:Link <br> UP BROADCAST RUNNING MASTER MULTICAST MTU:1500 Metric:1 <br> RX packets:2084539 errors:0 dropped:0
overruns:0 frame:0 <br> TX packets:1840604 errors:0 dropped:0 overruns:0 carrier:0 <br> collisions:0 txqueuelen:0 <br> RX bytes:1353887185 (1.2 GiB) TX bytes:990151888 (944.2 MiB) <br><br>eth0 Link encap:Ethernet HWaddr 00:22:15:06:3B:73 <br> UP BROADCAST RUNNING SLAVE MULTICAST MTU:1500 Metric:1<br> RX packets:186644 errors:0 dropped:0 overruns:0 frame:0
<br> TX packets:1611744 errors:0 dropped:0 overruns:0 carrier:0<br> collisions:0 txqueuelen:1000 <br> RX bytes:165778866 (158.0 MiB) TX bytes:950467922 (906.4 MiB)<br> Interrupt:18 <br><br>eth1 Link encap:Ethernet HWaddr
00:22:15:06:3B:73 <br> UP BROADCAST RUNNING SLAVE MULTICAST MTU:1500 Metric:1<br> RX packets:1896003 errors:0 dropped:0 overruns:0 frame:0<br> TX packets:34762 errors:0 dropped:0 overruns:0 carrier:0<br> collisions:0 txqueuelen:1000 <br> RX bytes:1187880303 (1.1 GiB) TX bytes:14394977 (13.7 MiB)<br>
Interrupt:19 <br><br>eth2 Link encap:Ethernet HWaddr 00:22:15:06:3B:73 <br> UP BROADCAST RUNNING SLAVE MULTICAST MTU:1500 Metric:1<br> RX packets:1892 errors:0 dropped:0 overruns:0 frame:0 <br> TX packets:194098 errors:0 dropped:0 overruns:0 carrier:0<br> collisions:0
txqueuelen:1000 <br> RX bytes:228016 (222.6 KiB) TX bytes:25288989 (24.1 MiB)<br> Interrupt:16 <br><br>eth3 Link encap:Ethernet HWaddr 00:22:15:06:3B:73 <br> UP BROADCAST SLAVE MULTICAST MTU:1500 Metric:1<br> RX packets:0 errors:0 dropped:0 overruns:0
frame:0<br> TX packets:0 errors:0 dropped:0 overruns:0 carrier:0<br> collisions:0 txqueuelen:1000 <br> RX bytes:0 (0.0 b) TX bytes:0 (0.0 b) <br> Interrupt:17 <br><br>lo Link encap:Local Loopback
<br> inet addr:127.0.0.1 Mask:255.0.0.0<br> inet6 addr: ::1/128 Scope:Host <br> UP LOOPBACK RUNNING MTU:16436 Metric:1<br> RX packets:6544 errors:0 dropped:0 overruns:0 frame:0<br> TX packets:6544 errors:0 dropped:0 overruns:0 carrier:0<br> collisions:0 txqueuelen:0 <br> RX bytes:1409100 (1.3 MiB) TX bytes:1409100 (1.3 MiB) <br><br>[root@three ~]# cat
/proc/net/bonding/bond0 <br>Ethernet Channel Bonding Driver: v3.5.0 (November 4, 2008)<br><br>Bonding Mode: IEEE 802.3ad Dynamic link aggregation<br>Transmit Hash Policy: layer3+4 (1) <br>MII Status: up <br>MII Polling Interval (ms): 10 <br>Up Delay (ms): 0 <br>Down Delay (ms):
0 <br><br>802.3ad info<br>LACP rate: slow<br>Aggregator selection policy (ad_select): stable<br>Active Aggregator Info: <br> Aggregator ID: 1 <br> Number of ports: 3 <br> Actor Key:
17 <br> Partner Key: 37823 <br> Partner Mac Address: 00:22:57:9a:ea:00 <br><br>Slave Interface: eth0<br>MII Status: up <br>Link Failure Count: 0<br>Permanent HW addr: 00:22:15:06:3b:73<br>Aggregator ID: 1 <br><br>Slave Interface: eth1<br>MII Status: up <br>Link Failure Count: 0<br>Permanent HW addr: 00:22:15:06:3b:74<br>Aggregator ID:
1 <br><br>Slave Interface: eth2<br>MII Status: up <br>Link Failure Count: 0<br>Permanent HW addr: 00:22:15:06:3b:75<br>Aggregator ID: 1 <br><br>Slave Interface: eth3<br>MII Status: down <br>Link Failure Count: 0<br>Permanent HW addr: 00:22:15:06:3b:76<br>Aggregator ID: 4 <br>[root@three ~]# ifconfig <br>bond0 Link encap:Ethernet HWaddr 00:22:15:06:3B:73 <br> inet addr:192.168.0.3
Bcast:192.168.0.255 Mask:255.255.255.0<br> inet6 addr: fe80::222:15ff:fe06:3b73/64 Scope:Link <br> UP BROADCAST RUNNING MASTER MULTICAST MTU:1500 Metric:1 <br> RX packets:2085357 errors:0 dropped:0 overruns:0 frame:0 <br> TX packets:1841322 errors:0 dropped:0 overruns:0 carrier:0 <br> collisions:0 txqueuelen:0
<br> RX bytes:1354333514 (1.2 GiB) TX bytes:990593774 (944.7 MiB) <br><br>eth0 Link encap:Ethernet HWaddr 00:22:15:06:3B:73 <br> UP BROADCAST RUNNING SLAVE MULTICAST MTU:1500 Metric:1<br> RX packets:186719 errors:0 dropped:0 overruns:0 frame:0 <br> TX packets:1612447 errors:0 dropped:0 overruns:0 carrier:0<br> collisions:0 txqueuelen:1000 <br> RX bytes:165785794 (158.1 MiB) TX bytes:950893930 (906.8
MiB)<br> Interrupt:18 <br><br>eth1 Link encap:Ethernet HWaddr 00:22:15:06:3B:73 <br> UP BROADCAST RUNNING SLAVE MULTICAST MTU:1500 Metric:1<br> RX packets:1896745 errors:0 dropped:0 overruns:0 frame:0<br> TX packets:34765 errors:0 dropped:0 overruns:0 carrier:0<br> collisions:0
txqueuelen:1000 <br> RX bytes:1188319580 (1.1 GiB) TX bytes:14395233 (13.7 MiB)<br> Interrupt:19 <br><br>eth2 Link encap:Ethernet HWaddr 00:22:15:06:3B:73 <br> UP BROADCAST RUNNING SLAVE MULTICAST MTU:1500 Metric:1<br> RX packets:1893 errors:0 dropped:0
overruns:0 frame:0 <br> TX packets:194110 errors:0 dropped:0 overruns:0 carrier:0<br> collisions:0 txqueuelen:1000<br> RX bytes:228140 (222.7 KiB) TX bytes:25304611 (24.1 MiB)<br> Interrupt:16<br><br>eth3 Link encap:Ethernet HWaddr 00:22:15:06:3B:73<br> UP BROADCAST SLAVE MULTICAST MTU:1500 Metric:1<br> RX packets:0 errors:0 dropped:0 overruns:0 frame:0<br> TX packets:0 errors:0 dropped:0 overruns:0 carrier:0<br> collisions:0
txqueuelen:1000<br> RX bytes:0 (0.0 b) TX bytes:0 (0.0 b)<br> Interrupt:17<br><br>lo Link encap:Local Loopback<br> inet addr:127.0.0.1 Mask:255.0.0.0<br> inet6 addr: ::1/128 Scope:Host<br> UP LOOPBACK RUNNING MTU:16436 Metric:1<br> RX packets:6544 errors:0 dropped:0 overruns:0 frame:0<br> TX packets:6544 errors:0 dropped:0 overruns:0 carrier:0<br> collisions:0 txqueuelen:0<br> RX bytes:1409100 (1.3 MiB) TX
bytes:1409100 (1.3 MiB)<br><br>[root@three ~]# uptime<br> 12:37:32 up 15:06, 7 users, load average: 0.00, 0.02, 0.00</div><div style="color: rgb(0, 0, 0); font-size: 16px; font-family: serif; background-color: transparent; font-style: normal;"><br></div><div style="color: rgb(0, 0, 0); font-size: 16px; font-family: serif; background-color: transparent; font-style: normal;">** Mobile Mythtv backend. Raid Storage/Lustre/Mythtv Backup. Zoneminder /dev/video0-3 /mnt/ioio/html/zm Lustre Mounted LAMP webroot=/mnt/ioio/html Lustre mounted. <br></div><div style="color: rgb(0, 0, 0); font-size: 16px; font-family: serif; background-color: transparent; font-style: normal;"><br></div><div style="color: rgb(0, 0, 0); font-size: 16px; font-family: serif; background-color: transparent; font-style: normal;"><br></div><div style="color: rgb(0, 0, 0); font-size: 16px; font-family: serif; background-color: transparent; font-style:
normal;">[mythtv@twentythree bin]$ df -h<br>Filesystem Size Used Avail Use% Mounted on<br>/dev/sda1 35G 9.2G 24G 28% / <br>tmpfs 1004M 12K 1004M 1% /dev/shm <br>192.168.0.10@tcp0:/ioio <br> 8.1T 4.8T 3.0T 62% /mnt/ioio <br>[mythtv@twentythree bin]$ cat /proc/net/bonding/bond0 <br>Ethernet Channel
Bonding Driver: v3.5.0 (November 4, 2008)<br><br>Bonding Mode: IEEE 802.3ad Dynamic link aggregation<br>Transmit Hash Policy: layer3+4 (1) <br>MII Status: up <br>MII Polling Interval (ms): 10 <br>Up Delay (ms): 0 <br>Down Delay (ms):
0 <br><br>802.3ad info<br>LACP rate: slow<br>Aggregator selection policy (ad_select): stable<br>Active Aggregator Info: <br> Aggregator ID: 1 <br> Number of ports: 2 <br> Actor Key:
17 <br> Partner Key: 37823 <br> Partner Mac Address: 00:22:57:9a:ea:00 <br><br>Slave Interface: eth0<br>MII Status: up <br>Link Failure Count: 0<br>Permanent HW addr: 00:22:15:95:2d:23<br>Aggregator ID: 1 <br><br>Slave Interface: eth1<br>MII Status: up <br>Link Failure Count: 0<br>Permanent HW addr: 00:04:e2:d7:33:c3<br>Aggregator ID:
1 <br>[mythtv@twentythree bin]$ ifconfig <br>bond0 Link encap:Ethernet HWaddr 00:22:15:95:2D:23 <br> inet addr:192.168.0.23 Bcast:192.168.0.255 Mask:255.255.255.0<br> inet6 addr: fe80::222:15ff:fe95:2d23/64 Scope:Link <br> UP BROADCAST RUNNING MASTER MULTICAST MTU:1500 Metric:1 <br> RX packets:591449 errors:0 dropped:0 overruns:0 frame:0 <br> TX packets:238413 errors:0
dropped:0 overruns:0 carrier:2 <br> collisions:0 txqueuelen:0 <br> RX bytes:324872352 (309.8 MiB) TX bytes:33161781 (31.6 MiB) <br><br>eth0 Link encap:Ethernet HWaddr 00:22:15:95:2D:23 <br> UP BROADCAST RUNNING SLAVE MULTICAST MTU:1500 Metric:1<br> RX packets:135740 errors:0 dropped:0 overruns:0 frame:0<br> TX packets:130875 errors:0 dropped:0 overruns:0
carrier:2<br> collisions:0 txqueuelen:1000<br> RX bytes:8407741 (8.0 MiB) TX bytes:13738521 (13.1 MiB)<br> Interrupt:28<br><br>eth1 Link encap:Ethernet HWaddr 00:22:15:95:2D:23<br> UP BROADCAST RUNNING SLAVE MULTICAST MTU:1500 Metric:1<br> RX packets:455709 errors:0 dropped:0 overruns:0 frame:0<br> TX packets:107538 errors:0 dropped:0 overruns:0 carrier:0<br> collisions:0 txqueuelen:1000<br> RX bytes:316464611 (301.8 MiB) TX bytes:19423260 (18.5
MiB)<br> Interrupt:17<br><br>lo Link encap:Local Loopback<br> inet addr:127.0.0.1 Mask:255.0.0.0<br> inet6 addr: ::1/128 Scope:Host<br> UP LOOPBACK RUNNING MTU:16436 Metric:1<br> RX packets:262 errors:0 dropped:0 overruns:0 frame:0<br> TX packets:262 errors:0 dropped:0 overruns:0 carrier:0<br> collisions:0 txqueuelen:0<br> RX bytes:15508 (15.1 KiB) TX bytes:15508 (15.1 KiB)<br></div><div style="color: rgb(0, 0, 0); font-size: 16px; font-family: serif;
background-color: transparent; font-style: normal;"><br></div><div style="color: rgb(0, 0, 0); font-size: 16px; font-family: serif; background-color: transparent; font-style: normal;">[mythtv@twentythree bin]$ uptime<br> 12:47:20 up 13:43, 14 users, load average: 1.12, 1.14, 1.11</div><div style="color: rgb(0, 0, 0); font-size: 16px; font-family: serif; background-color: transparent; font-style: normal;">[mythtv@twentythree bin]$ lfs df -h<br>UUID bytes Used Available Use% Mounted on<br>ioio-MDT0000_UUID 815.0G 374.0M 768.1G 0% /mnt/ioio[MDT:0]<br>ioio-OST0000_UUID
2.7T 1.7T 915.7G 65% /mnt/ioio[OST:0]<br>ioio-OST0001_UUID 2.7T 1.8T 796.3G 70% /mnt/ioio[OST:1]<br>ioio-OST0002_UUID 2.7T 1.3T 1.3T 50% /mnt/ioio[OST:2]<br><br>filesystem summary: 8.1T 4.7T 2.9T 62% /mnt/ioio</div><div style="color: rgb(0, 0, 0); font-size: 16px; font-family: serif; background-color: transparent; font-style: normal;"><br></div><div style="color: rgb(0, 0, 0); font-size: 16px; font-family: serif; background-color: transparent; font-style:
normal;">** Lustre Client. Mythtv Frontend. BzFlag 2.4.2 Enabled. Can be e17 driven. <br></div><div style="color: rgb(0, 0, 0); font-size: 16px; font-family: serif; background-color: transparent; font-style: normal;"><br></div><div style="color: rgb(0, 0, 0); font-size: 16px; font-family: serif; background-color: transparent; font-style: normal;">[root@sixteen ~]# df -h<br>Filesystem Size Used Avail Use% Mounted on<br>/dev/sda1 143G 9.0G 127G 7% / <br>tmpfs 1.3G 88K 1.3G 1% /dev/shm
<br>192.168.0.10@tcp0:/ioio <br> 8.1T 4.8T 3.0T 62% /mnt/ioio <br>[root@sixteen ~]# cat /proc/net/bonding/bond0 <br>Ethernet Channel Bonding Driver: v3.5.0 (November 4, 2008)<br><br>Bonding Mode: IEEE 802.3ad Dynamic link aggregation<br>Transmit Hash Policy: layer3+4 (1) <br>MII Status:
up <br>MII Polling Interval (ms): 10 <br>Up Delay (ms): 0 <br>Down Delay (ms): 0 <br><br>802.3ad info<br>LACP rate: slow<br>Aggregator selection policy (ad_select): stable<br>Active Aggregator
Info: <br> Aggregator ID: 1 <br> Number of ports: 1 <br> Actor Key: 17 <br> Partner Key: 1
<br> Partner Mac Address: 00:00:00:00:00:00 <br><br>Slave Interface: eth0<br>MII Status: up <br>Link Failure Count: 0<br>Permanent HW addr: 00:14:85:18:83:7a<br>Aggregator ID: 1 <br><br>Slave Interface: eth1<br>MII Status: up <br>Link Failure Count: 0<br>Permanent HW addr: 00:0e:0c:dd:3a:c7<br>Aggregator ID: 2 <br>[root@sixteen ~]# ifconfig <br>bond0 Link encap:Ethernet HWaddr 00:14:85:18:83:7A <br> inet addr:192.168.0.16 Bcast:192.168.0.255
Mask:255.255.255.0<br> inet6 addr: fe80::214:85ff:fe18:837a/64 Scope:Link <br> UP BROADCAST RUNNING MASTER MULTICAST MTU:1500 Metric:1 <br> RX packets:531666451 errors:42 dropped:0 overruns:0 frame:42 <br> TX packets:155532069 errors:0 dropped:0 overruns:0 carrier:0 <br> collisions:0 txqueuelen:0
<br> RX bytes:561475010002 (522.9 GiB) TX bytes:650261730391 (605.6 GiB)<br><br>eth0 Link encap:Ethernet HWaddr 00:14:85:18:83:7A <br> UP BROADCAST RUNNING SLAVE MULTICAST MTU:1500 Metric:1<br> RX packets:525163826 errors:42 dropped:0 overruns:0 frame:42<br> TX packets:155449633 errors:0 dropped:0 overruns:0 carrier:0<br> collisions:0 txqueuelen:1000<br> RX bytes:561052441951 (522.5 GiB) TX bytes:650251508327 (605.5 GiB)<br> Interrupt:22<br><br>eth1 Link encap:Ethernet HWaddr
00:14:85:18:83:7A<br> UP BROADCAST RUNNING SLAVE MULTICAST MTU:1500 Metric:1<br> RX packets:6502625 errors:0 dropped:0 overruns:0 frame:0<br> TX packets:82436 errors:0 dropped:0 overruns:0 carrier:0<br> collisions:0 txqueuelen:1000<br> RX bytes:422568051 (402.9 MiB) TX bytes:10222064 (9.7 MiB)<br><br>lo Link encap:Local Loopback<br> inet addr:127.0.0.1 Mask:255.0.0.0<br> inet6 addr: ::1/128 Scope:Host<br> UP LOOPBACK RUNNING MTU:16436
Metric:1<br> RX packets:205748 errors:0 dropped:0 overruns:0 frame:0<br> TX packets:205748 errors:0 dropped:0 overruns:0 carrier:0<br> collisions:0 txqueuelen:0<br> RX bytes:56913227 (54.2 MiB) TX bytes:56913227 (54.2 MiB)<br><br>[root@sixteen ~]# uptime<br> 12:49:51 up 28 days, 14:59, 1 user, load average: 0.08, 0.05, 0.01</div><div style="color: rgb(0, 0, 0); font-size: 16px; font-family: serif; background-color: transparent; font-style: normal;">[root@sixteen ~]# lfs df -h<br>UUID bytes Used Available Use% Mounted
on<br>ioio-MDT0000_UUID 815.0G 374.0M 768.1G 0% /mnt/ioio[MDT:0]<br>ioio-OST0000_UUID 2.7T 1.7T 915.7G 65% /mnt/ioio[OST:0]<br>ioio-OST0001_UUID 2.7T 1.8T 796.3G 70% /mnt/ioio[OST:1]<br>ioio-OST0002_UUID 2.7T 1.3T 1.3T 50% /mnt/ioio[OST:2]<br><br>filesystem summary: 8.1T 4.7T 2.9T
62% /mnt/ioio<br></div><div style="color: rgb(0, 0, 0); font-size: 16px; font-family: serif; background-color: transparent; font-style: normal;"><br></div><div style="color: rgb(0, 0, 0); font-size: 16px; font-family: serif; background-color: transparent; font-style: normal;">** Lustre Client. Mythtv Backend.</div><div style="color: rgb(0, 0, 0); font-size: 16px; font-family: serif; background-color: transparent; font-style: normal;"><br></div><div style="color: rgb(0, 0, 0); font-size: 16px; font-family: serif; background-color: transparent; font-style: normal;">So there are reasons for using raid10 on specific parts of your Lustre deployment. I had a few bad drives/sata errors on one box and raid10 saved everything. I was able to watch the drive fail after time and finally when it died I was able to replace while still having Lustre interoperablility. Still observing for that same box with another bad drive. Procurring
1TB 32mb sata II drives market driven. <br></div><div style="color: rgb(0, 0, 0); font-size: 16px; font-family: serif; background-color: transparent; font-style: normal;"><br></div><div style="color: rgb(0, 0, 0); font-size: 16px; font-family: serif; background-color: transparent; font-style: normal;">Rebuild time was tolerable but from recollection less than a couple of hours. Just one of those tasks every raided machine does once a week anyhow via the cron resync.</div><div style="color: rgb(0, 0, 0); font-size: 16px; font-family: serif; background-color: transparent; font-style: normal;"><br></div><div style="color: rgb(0, 0, 0); font-size: 16px; font-family: serif; background-color: transparent; font-style: normal;">I haven't lost data with this layout for years and have been able to adapt, maintain and implement different technologies and updates along the way. <br></div><div style="color: rgb(0, 0, 0); font-size: 16px;
font-family: serif; background-color: transparent; font-style: normal;">https://groups.google.com/forum/?fromgroups#!msg/lustre-discuss-list/NfNilGsVYxA/KVCR8nixXbMJ</div><div style="color: rgb(0, 0, 0); font-size: 16px; font-family: serif; background-color: transparent; font-style: normal;"><br></div><div style="color: rgb(0, 0, 0); font-size: 16px; font-family: serif; background-color: transparent; font-style: normal;">Would smaller disks be better? Have you ever tried to make a large raid 5 array out of 2+TB Sata III drives? There are some definate changes regarding the partitioning structure. I eventually had to just let the installer do all the work for me when I made a 6TB raid 5 Lustre Backup array. Smaller disks are easier from this perspective. <br></div><div style="color: rgb(0, 0, 0); font-size: 16px; font-family: serif; background-color: transparent; font-style: normal;"><br></div><div style="color: rgb(0, 0,
0); font-size: 16px; font-family: serif; background-color: transparent; font-style: normal;">From this base I attached all HDMI cables. My LM7600 deploys 3d stereoscopic images and I've even take to researching the production and conversion of that medium. Beamsplitter on the RV, lol. The HDMI is upscaled to 4K by the Pioneer 1527K and it's all fed into a dozen KEF 101's<br></div><div style="color: rgb(0, 0, 0); font-size: 16px; font-family: serif; background-color: transparent; font-style: normal;">with 3 subs for 9.1 Stereo Sound utilizing neoX soundtracks from the newest blu rays. Waiting on 4K OLED sets to become available.<br></div><div style="color: rgb(0, 0, 0); font-size: 16px; font-family: serif; background-color: transparent; font-style: normal;"><br></div><div style="color: rgb(0, 0, 0); font-size: 16px; font-family: serif; background-color: transparent; font-style: normal;">I routinely get 90+ days of uptime on my
Lustre machines and find I'm more like the Maytag guy when it comes to maintaining the network. It is just there and for the most part enjoyable to use. Not saying that there haven't been some challenges when everything hangs in the balance of a few keystrokes but for the most part it works period.<br></div><div style="color: rgb(0, 0, 0); font-size: 16px; font-family: serif; background-color: transparent; font-style: normal;"><br></div><div style="color: rgb(0, 0, 0); font-size: 16px; font-family: serif; background-color: transparent; font-style: normal;">I would like to expand it all again with Sata III 4TB storage disks and 120 MB SSD boot drives. Grab some 10GigE interconnect cards and plum some Bell Axia into the house but time will tell. Having a purpose for the storage is the best question to ask. 4x8=32TB per OST x 3 OST = 96TB linear raid. That would be personal use future proofed with room for offering
storage services. <br></div><div style="color: rgb(0, 0, 0); font-size: 16px; font-family: serif; background-color: transparent; font-style: normal;"><br></div><div style="color: rgb(0, 0, 0); font-size: 16px; font-family: serif; background-color: transparent; font-style: normal;">Perhaps the smallest clients of all are rooted androids and tablets accessing the network via apps to the SMB/NMB shared Lustre Mount. This ties the entire configuration together even driving down the road rebooting servers or restarting apache. Some apps like mythfrontend and mythmote are far superior to anything produced by commercial vendors for products. Most commercial apps are chalk full of thoughtless design much like smart TV's and for the most part commercialy available PVRs. <br></div><div style="color: rgb(0, 0, 0); font-size: 16px; font-family: serif; background-color: transparent; font-style: normal;"><br></div><div style="color:
rgb(0, 0, 0); font-size: 16px; font-family: serif; background-color: transparent; font-style: normal;"><br><blockquote style="border-left: 2px solid rgb(16, 16, 255); margin-left: 5px; margin-top: 5px; padding-left: 5px;"> <div style="font-family: times new roman,new york,times,serif; font-size: 12pt;"> <div style="font-family: times new roman,new york,times,serif; font-size: 12pt;"> <div dir="ltr"> <font face="Arial" size="2"> <hr size="1"> <b><span style="font-weight: bold;">From:</span></b> Jon Yeargers <yeargers@ohsu.edu><br> <b><span style="font-weight: bold;">To:</span></b> "lustre-discuss@lists.lustre.org" <lustre-discuss@lists.lustre.org> <br> <b><span style="font-weight: bold;">Sent:</span></b> Friday, December 28, 2012 11:37:06 AM<br> <b><span style="font-weight: bold;">Subject:</span></b> [Lustre-discuss] RAID usage?<br> </font> </div> <br><div id="yiv864368767"><style><!--
#yiv864368767
_filtered #yiv864368767 {font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
#yiv864368767
#yiv864368767 p.yiv864368767MsoNormal, #yiv864368767 li.yiv864368767MsoNormal, #yiv864368767 div.yiv864368767MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri", "sans-serif";}
#yiv864368767 a:link, #yiv864368767 span.yiv864368767MsoHyperlink
{
color:blue;
text-decoration:underline;}
#yiv864368767 a:visited, #yiv864368767 span.yiv864368767MsoHyperlinkFollowed
{
color:purple;
text-decoration:underline;}
#yiv864368767 span.yiv864368767EmailStyle17
{
font-family:"Calibri", "sans-serif";
color:windowtext;}
#yiv864368767 .yiv864368767MsoChpDefault
{}
_filtered #yiv864368767 {
margin:1.0in 1.0in 1.0in 1.0in;}
#yiv864368767 div.yiv864368767WordSection1
{}
--></style><div><div class="yiv864368767WordSection1"><div class="yiv864368767MsoNormal">Im curious to hear what RAID configurations people are using in existing Lustre installations. </div><div class="yiv864368767MsoNormal"> </div><div class="yiv864368767MsoNormal">I’ve been collecting bids to build an array and assuming RAID6/1Tb but the more I read dire warnings about losing data and ‘days to rebuild an array’ the more I’m questioning my approach.</div><div class="yiv864368767MsoNormal"> </div><div class="yiv864368767MsoNormal">Would RAID 10 be better? Smaller disks? </div></div></div></div><br>_______________________________________________<br>Lustre-discuss mailing list<br><a ymailto="mailto:Lustre-discuss@lists.lustre.org" href="mailto:Lustre-discuss@lists.lustre.org">Lustre-discuss@lists.lustre.org</a><br><a href="http://lists.lustre.org/mailman/listinfo/lustre-discuss"
target="_blank">http://lists.lustre.org/mailman/listinfo/lustre-discuss</a><br><br><br><br> </div> </div> </blockquote></div> </div></body></html>