From michael.bloom at trd2inc.com Sun Sep 15 14:51:34 2013 From: michael.bloom at trd2inc.com (Michael Bloom) Date: Sun, 15 Sep 2013 10:51:34 -0400 Subject: [Lustre-devel] Lustre 2.4 MDT: LustreError: Communicating with 0@lo: operation mds_connect failed with -11 Message-ID: I'm a Lustre newbie who just joined this list. I'd appreciate any help on the following Lustre 2.4 issue I'm running into: Every time I mount the MDT, the mount appears to succeed but /var/log/messages contains the message: "LustreError: 11-0: lustre-MDT0000-lwp-MDT0000: Communicating with 0 at lo, operation mds_connect failed with -11". The MDT uses 4 local drives in a RAID10 configuration. Each OSS has their own RAID10 of 36 drives each. The OSS's mount correctly without any errors. I've seen this error mentioned in countless Google searches. One obscure reply suggested this was a problem fixed in 2.5. All other references were with respect to pre-2.4 releases where the message indicated there was probably an error somewhere in the connection's configuration. Is this a real error? I see the code that probably generates this in client.c. In abbreviated form, the code is: LCONSOLE_ERROR_MSG(0x11, "%s Communicating with %s") in ptlrpc_check_status(). There's another in mdt_obd_connect() where -EAGAIN [set to -11 in lustre_errno.h) is returned if the stack isn't ready to handle requests as indicated by the return code from obd_health_check(). My environment is this: MDT, OSS0, and OSS1 are all on 3 separate nodes running Centos 6.4 and connected by Infiniband Mellanox HBAs. Running this in a VM with the MDT and a single OSS on one node in a VM using TCP did not exhibit this problem. Thanks in advance for any help you can provide. Michael -------------- next part -------------- An HTML attachment was scrubbed... URL: From alexey_lyashkov at xyratex.com Sat Sep 21 08:39:28 2013 From: alexey_lyashkov at xyratex.com (Alexey Lyahkov) Date: Sat, 21 Sep 2013 10:39:28 +0200 Subject: [Lustre-devel] portability in lustre code. Message-ID: Hi All, I "happy" to see moving to lustre kernel generate a first results. my MacOS fuse client brooked to build after sources updates. In file included from /Users/shadow/work/lustre/work/WC-review/mac-os/libcfs/include/libcfs/posix/libcfs.h:108, from /Users/shadow/work/lustre/work/WC-review/mac-os/libcfs/include/libcfs/libcfs.h:45, from nidstrings.c:43: /Users/shadow/work/lustre/work/WC-review/mac-os/libcfs/include/libcfs/user-bitops.h:80: error: conflicting types for ‘fls’ /usr/include/strings.h:90: error: previous declaration of ‘fls’ was here that bug introduced by commit >> commit d38d331fa6525ffc02665f48fa52f94626360631 Author: James Simmons Date: Tue Aug 27 12:51:06 2013 -0400 LU-1346 libcfs: cleanup macros in portals_compat25.h >> so that commit - isn't correctly inspected for systems other to linux and introduce a new regression for systems other then Linux. I was pointed about similar situation in past - but hit that in real world. Andreas, did you remember my concerns about lost portability ? batch changes code in way without ability to replace a cfs specific function to any existent in build env. >>> diff --git a/libcfs/include/libcfs/user-bitops.h b/libcfs/include/libcfs/user-bitops.h index 2e31e88..1b16ca7 100644 --- a/libcfs/include/libcfs/user-bitops.h +++ b/libcfs/include/libcfs/user-bitops.h @@ -76,7 +76,7 @@ static inline int test_bit(int nr, const unsigned long *addr) } /* using binary seach */ -static __inline__ unsigned long __cfs_fls(long data) +static __inline__ unsigned long fls(long data) { int pos = 32; @@ -147,8 +147,8 @@ static __inline__ unsigned long __cfs_ffs(long data) >> but fls from BSD userland have a different prototype and we don't able to replace with flsl from build env. from man page FFS(3) BSD Library Functions Manual FFS(3) NAME ffs, ffsl, fls, flsl -- find first or last bit set in a bit string HISTORY The ffs() function appeared in 4.3BSD. Its prototype existed previously in before it was moved to for IEEE Std 1003.1-2001 (``POSIX.1'') compliance. The ffsl(), fls(), and flsl() functions appeared in FreeBSD 5.3. and we don't have a way to the replace fls with flsl for MacOS/BSD or use own implementation without reverting a cleanup for fls function. ---------------------------------------------- Alexey Lyahkov alexey_lyashkov at xyratex.com From nathan_rutman at xyratex.com Wed Sep 25 21:59:22 2013 From: nathan_rutman at xyratex.com (Nathan Rutman) Date: Wed, 25 Sep 2013 14:59:22 -0700 Subject: [Lustre-devel] Kerberos interest group Message-ID: We're trying to get a critical mass of people interested in fixing up Lustre's Kerberos support, to help with: identifying problems contributing patches contributing to documentation: HowTo, FAQ, etc. To this end, we wanted to solicit people to join a mailing list dealing with Lustre GSS (under the auspices of the Indiana University work on GSS shared key). So if you are interested, please let me know and I will add you to the iudev list. I love this slide taken from Daniel Kobras' LAD preso http://www.eofs.eu/fileadmin/lad2013/slides/07_Daniel_Kobras_lustre_gss_eofs_2013.pdf -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: 07_Daniel_Kobras_lustre_gss_eofs_2013 (dragged).pdf Type: application/pdf Size: 88661 bytes Desc: not available URL: -------------- next part -------------- An HTML attachment was scrubbed... URL: From andreas.dilger at intel.com Mon Sep 30 22:26:04 2013 From: andreas.dilger at intel.com (Dilger, Andreas) Date: Mon, 30 Sep 2013 22:26:04 +0000 Subject: [Lustre-devel] portability in lustre code. In-Reply-To: Message-ID: On 2013/09/21 2:39 AM, "Alexey Lyahkov" wrote: >I "happy" to see moving to lustre kernel generate a first results. >my MacOS fuse client brooked to build after sources updates. > >In file included from >/Users/shadow/work/lustre/work/WC-review/mac-os/libcfs/include/libcfs/posi >x/libcfs.h:108, > from >/Users/shadow/work/lustre/work/WC-review/mac-os/libcfs/include/libcfs/libc >fs.h:45, > from nidstrings.c:43: >/Users/shadow/work/lustre/work/WC-review/mac-os/libcfs/include/libcfs/user >-bitops.h:80: error: conflicting types for Œfls¹ >/usr/include/strings.h:90: error: previous declaration of Œfls¹ was here > >that bug introduced by commit >>> >commit d38d331fa6525ffc02665f48fa52f94626360631 >Author: James Simmons >Date: Tue Aug 27 12:51:06 2013 -0400 > > LU-1346 libcfs: cleanup macros in portals_compat25.h >>> >so that commit - isn't correctly inspected for systems other to linux and >introduce a new regression for systems other then Linux. >I was pointed about similar situation in past - but hit that in real >world. >Andreas, did you remember my concerns about lost portability ? Yes, I definitely remember this discussion. As we discussed a year ago the MacOS, Windows, and liblustre code are currently unused, and have been unmaintained since that time. I don't think it is reasonable for us to maintain this code for your private FreeBSD Fuse port. The code would need to continuously built and tested in public for it to even consider it maintained at the most basic level. Even prior to this recent compile problem, I would be incredibly surprised if this code was working. The liblustre code has been unmaintained since Oracle times, and the Windows client has never been publicly released. If MacOS _was_ working under your testing, it would have made sense to share this code publicly last year. >batch changes code in way without ability to replace a cfs specific >function to any existent in build env. and we don't have a way to the >replace fls with flsl for MacOS/BSD or use own implementation without >reverting a cleanup for fls function. It would be fairly straight forward for you to apply a patch or run the code through a sed script before build to replace the few conflicting names with their equivalent MacOS functions. This could be similar to how ldiskfs is built from the ext4 code. You are of course also free to keep a private or public branch of the code for your own use. We were planning to deleting the liblustre, MacOS, and WinNT code entirely from the repository in the next release, because it is unused and causes an ongoing maintenance and development complexity for the Linux code without any benefit. In particular, the abstractions in CLIO are far more complex than they need to be for just Linux clients because the Linux and MacOS VM/VFS interfaces are wildly different. Cheers, Andreas -- Andreas Dilger Lustre Software Architect Intel High Performance Data Division From s.wendy.cheng at gmail.com Mon Sep 30 23:18:26 2013 From: s.wendy.cheng at gmail.com (Wendy Cheng) Date: Mon, 30 Sep 2013 16:18:26 -0700 Subject: [Lustre-devel] Lustre on Intel Xeon Phi Message-ID: Do folks have instructions of how to install Lustre client on Intel Xeon Phi ? I'm on a relatively old MPSS (mpss_gold_update_3, mic-2.1.6720 linux 2.6.18 kernel). Other cluster nodes run Lustre-1.8.8. Thanks, Wendy