From magawake at gmail.com Thu Nov 12 01:20:35 2009 From: magawake at gmail.com (Mag Gam) Date: Wed, 11 Nov 2009 20:20:35 -0500 Subject: [Lustre-devel] Lustre version number and rpm "Release" In-Reply-To: <1254750291.22120.12.camel@pc.interlinx.bc.ca> References: <4AC56205.5060403@llnl.gov> <1254491700.26203.251.camel@pc.interlinx.bc.ca> <4AC69C7B.2020201@llnl.gov> <1254750291.22120.12.camel@pc.interlinx.bc.ca> Message-ID: <1cbd6f830911111720p1b6d9e54ibb5410709b38176f@mail.gmail.com> Just curious, why do people build their own releases if everything is available in release RPMs. Do most people or organizations build their own version of Lustre or do they use the RPMs? I am asking because I would like to build my own version and wanted to see the benefits of it. TIA 2009/10/5 Brian J. Murrell : > On Fri, 2009-10-02 at 17:36 -0700, Christopher J. Morrone wrote: >> >> Well, that is true, except that we just want to prefix the release, not >> replace it.  If we replace it with our own definition, we have to repeat >> the kernel detection, and timestamp part. > > Yes, that's true. > >> So really, we'll want an >> additional "release_prefix" rpm variable. > > That would be a trivial patch.  Feel free to open a bug and submit it. > I can't see any reason why we'd not land that. > >> There is an additional wrinkle that I didn't mention.  Our automated >> build system is designed to take source rpms and build the binary rpms >> from that.  If we just set "release" at source rpm build time, I don't >> think that it will be remember later when the binary rpms are built. > > Yes, I think you are correct about that. > > FWIW, (I think[1]) I'd like us (Sun) to get out of the business of > putting anything meaningful in the Release tag and leave that for > distributors to use.  I'm not sure if or how that might help you though. > > b. > > [1] I've not really investigated the possibility yet, so it's really > just a seed of an idea. > > _______________________________________________ > Lustre-devel mailing list > Lustre-devel at lists.lustre.org > http://lists.lustre.org/mailman/listinfo/lustre-devel > > From C.J.Walker at qmul.ac.uk Sat Nov 14 09:38:19 2009 From: C.J.Walker at qmul.ac.uk (Christopher J. Walker) Date: Sat, 14 Nov 2009 09:38:19 +0000 Subject: [Lustre-devel] Lustre version number and rpm "Release" In-Reply-To: <1cbd6f830911111720p1b6d9e54ibb5410709b38176f@mail.gmail.com> References: <4AC56205.5060403@llnl.gov> <1254491700.26203.251.camel@pc.interlinx.bc.ca> <4AC69C7B.2020201@llnl.gov> <1254750291.22120.12.camel@pc.interlinx.bc.ca> <1cbd6f830911111720p1b6d9e54ibb5410709b38176f@mail.gmail.com> Message-ID: <4AFE7A8B.9070006@qmul.ac.uk> Mag Gam wrote: > Just curious, why do people build their own releases if everything is > available in release RPMs. Do most people or organizations build their > own version of Lustre or do they use the RPMs? > We need to keep up to date with the latest kernel security errata - so I've been building Lustre clients. > I am asking because I would like to build my own version and wanted to > see the benefits of it. Knowing that you can, even if you don't currently need to, is an advantage. > TIA > > 2009/10/5 Brian J. Murrell : >> On Fri, 2009-10-02 at 17:36 -0700, Christopher J. Morrone wrote: >>> Well, that is true, except that we just want to prefix the release, not >>> replace it. If we replace it with our own definition, we have to repeat >>> the kernel detection, and timestamp part. >> Yes, that's true. >> >>> So really, we'll want an >>> additional "release_prefix" rpm variable. >> That would be a trivial patch. Feel free to open a bug and submit it. >> I can't see any reason why we'd not land that. >> >>> There is an additional wrinkle that I didn't mention. Our automated >>> build system is designed to take source rpms and build the binary rpms >>> from that. If we just set "release" at source rpm build time, I don't >>> think that it will be remember later when the binary rpms are built. >> Yes, I think you are correct about that. >> >> FWIW, (I think[1]) I'd like us (Sun) to get out of the business of >> putting anything meaningful in the Release tag and leave that for >> distributors to use. I'm not sure if or how that might help you though. >> >> b. >> >> [1] I've not really investigated the possibility yet, so it's really >> just a seed of an idea. >> >> _______________________________________________ >> Lustre-devel mailing list >> Lustre-devel at lists.lustre.org >> http://lists.lustre.org/mailman/listinfo/lustre-devel >> >> > _______________________________________________ > Lustre-devel mailing list > Lustre-devel at lists.lustre.org > http://lists.lustre.org/mailman/listinfo/lustre-devel From Roger.Spellman at terascala.com Thu Nov 19 19:09:06 2009 From: Roger.Spellman at terascala.com (Roger Spellman) Date: Thu, 19 Nov 2009 14:09:06 -0500 Subject: [Lustre-devel] /proc file bugs Message-ID: <2C7DE72B9BD00F44BAECA5B0CBB87395843FC8@hermes.terascala.com> It appears to me that there are bugs with handling /proc files for clients that have rebooted. For example, if a client reboots, then a file like the following still exists on the MGS: /proc/fs/lustre/mgs/MGS/exports/10.2.46.204 at o2ib/stats But, the mgs_disconnect() in mgs_handler.c calls the function lprocfs_exp_cleanup() in lprocfs_status.c, which calls lprocfs_free_stats(), which frees the stats structure. So, if you cat that file, you can crash your system. This is because the function lprocfs_stats_seq_start() in lprocfs_status.c is getting a pointer to a data structure that has been freed: static void *lprocfs_stats_seq_start(struct seq_file *p, loff_t *pos) { struct lprocfs_stats *stats = p->private; . . . } That is, 'p' is valid, but p->private has been freed. If I remount the same client, the proc file is not created again (good thing), and the stats struct is not reallocated (THIS IS BAD, since it was freed!). The proc entry's private data is still pointing to a struct that was freed. See, for example, mgs_export_stats_init(), which checks if the NID does not exist before creating a proc file. static int mgs_export_stats_init(struct obd_device *obd, struct obd_export *exp, void *localdata) { . . . rc = lprocfs_exp_setup(exp, client_nid, &newnid); . . . if (newnid) { exp->exp_ops_stats = lprocfs_alloc_stats(num_stats, LPROCFS_STATS_FLAG_NOPERCPU); lprocfs_init_ops_stats(LPROC_MGS_LAST, exp->exp_ops_stats); mgs_stats_counter_init(exp->exp_ops_stats); lprocfs_register_stats(exp->exp_nid_stats->nid_proc, "stats", exp->exp_ops_stats); . . . } } I can think of a few possible solutions: 1. In lprocfs_exp_cleanup(), don't call lprocfs_free_stats(). 2. When a client disconnect, do proper cleanup of the proc files (remove them). Regarding solution 2, I looked at the code that creates that particular proc entry, lprocfs_register_stats(), and it DOES NOT SAVE the proc_entry, i.e.: int lprocfs_register_stats(struct proc_dir_entry *root, const char *name, struct lprocfs_stats *stats) { struct proc_dir_entry *entry; LASSERT(root != NULL); entry = create_proc_entry(name, 0644, root); if (entry == NULL) return -ENOMEM; entry->proc_fops = &lprocfs_stats_seq_fops; entry->data = (void *)stats; return 0; } So, there is no trivial way to remove the proc entry. One possible approach would be to add a field to struct lprocfs_stats to hold the proc entry. Then, when the stats are freed in lprocfs_free_stats(),the proc entry can be freed. What do people think of my proposed solutions? Or, is there something I'm missing? By the way, this bug exists in both 1.6.x and 1.8.x. Roger Spellman Staff Engineer Terascala, Inc. 508-588-1501 www.terascala.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From Roger.Spellman at terascala.com Fri Nov 20 19:36:29 2009 From: Roger.Spellman at terascala.com (Roger Spellman) Date: Fri, 20 Nov 2009 14:36:29 -0500 Subject: [Lustre-devel] /proc file bugs Message-ID: <2C7DE72B9BD00F44BAECA5B0CBB87395844020@hermes.terascala.com> It appears to me that there are bugs with handling /proc files for clients that have rebooted. For example, if a client reboots, then a file like the following still exists on the MGS: /proc/fs/lustre/mgs/MGS/exports/10.2.46.204 at o2ib/stats But, the mgs_disconnect() in mgs_handler.c calls the function lprocfs_exp_cleanup() in lprocfs_status.c, which calls lprocfs_free_stats(), which frees the stats structure. So, if you cat that file, you can crash your system. This is because the function lprocfs_stats_seq_start() in lprocfs_status.c is getting a pointer to a data structure that has been freed: static void *lprocfs_stats_seq_start(struct seq_file *p, loff_t *pos) { struct lprocfs_stats *stats = p->private; . . . } That is, 'p' is valid, but p->private has been freed. If I remount the same client, the proc file is not created again (good thing), and the stats struct is not reallocated (THIS IS BAD, since it was freed!). The proc entry's private data is still pointing to a struct that was freed. See, for example, mgs_export_stats_init(), which checks if the NID does not exist before creating a proc file. static int mgs_export_stats_init(struct obd_device *obd, struct obd_export *exp, void *localdata) { . . . rc = lprocfs_exp_setup(exp, client_nid, &newnid); . . . if (newnid) { exp->exp_ops_stats = lprocfs_alloc_stats(num_stats, LPROCFS_STATS_FLAG_NOPERCPU); lprocfs_init_ops_stats(LPROC_MGS_LAST, exp->exp_ops_stats); mgs_stats_counter_init(exp->exp_ops_stats); lprocfs_register_stats(exp->exp_nid_stats->nid_proc, "stats", exp->exp_ops_stats); . . . } } I can think of a few possible solutions: 1. In lprocfs_exp_cleanup(), don't call lprocfs_free_stats(). 2. When a client disconnect, do proper cleanup of the proc files (remove them). Regarding solution 2, I looked at the code that creates that particular proc entry, lprocfs_register_stats(), and it DOES NOT SAVE the proc_entry, i.e.: int lprocfs_register_stats(struct proc_dir_entry *root, const char *name, struct lprocfs_stats *stats) { struct proc_dir_entry *entry; LASSERT(root != NULL); entry = create_proc_entry(name, 0644, root); if (entry == NULL) return -ENOMEM; entry->proc_fops = &lprocfs_stats_seq_fops; entry->data = (void *)stats; return 0; } So, there is no trivial way to remove the proc entry. One possible approach would be to add a field to struct lprocfs_stats to hold the proc entry. Then, when the stats are freed in lprocfs_free_stats(),the proc entry can be freed. What do people think of my proposed solutions? Or, is there something I'm missing? By the way, this bug exists in both 1.6.x and 1.8.x. Roger Spellman Staff Engineer Terascala, Inc. 508-588-1501 www.terascala.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From Brian.Murrell at Sun.COM Mon Nov 23 16:06:47 2009 From: Brian.Murrell at Sun.COM (Brian J. Murrell) Date: Mon, 23 Nov 2009 11:06:47 -0500 Subject: [Lustre-devel] /proc file bugs In-Reply-To: <2C7DE72B9BD00F44BAECA5B0CBB87395843FC8@hermes.terascala.com> References: <2C7DE72B9BD00F44BAECA5B0CBB87395843FC8@hermes.terascala.com> Message-ID: <1258992407.12528.70.camel@pc.interlinx.bc.ca> On Thu, 2009-11-19 at 14:09 -0500, Roger Spellman wrote: > It appears to me that there are bugs with handling /proc files for > clients that have rebooted. FWIW, this looks like bugs 21279 and 21420 in our bugzilla. b. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 197 bytes Desc: This is a digitally signed message part URL: