[lustre-discuss] lustreapi not found when compiling sample C program

Thu Jul 25 11:58:26 PDT 2024

These were the steps I was following, however I probably forgot to 
install some packages/had mismatching packages from other installations, 
after trying again on a fresh vm the above works.

As a follow up question, is there a way to install / include only the 
libraries / headers necessary for the lustreapi?

For context, I am trying to add the lustreapi to another project as a 
third-party dependency. I would like to only have the necessary 
libraries/headers in a thirdparty/lib and thirdparty/includes directory, 
instead of installing the deb packages. For this I don't know whether 
the system where my project is compiled is the one mounting the lustre 
fs (and thus has the client installed) so I want to have only the 
absolute necessary for compilation.

Thanks in advance,

Apostolis Stamatis

On 11/7/24 02:04, Andreas Dilger wrote:
> Why not build packages and install them properly?  There are a few 
> pages on https://wiki.lustre.org/ for this (which should be updated if 
> they are missing some info), but the TLDR to build a client is to run 
> in the Lustre Git checkout tree:
>
>     sh autogen.sh
>     ./configure --disable-tests --disable-server
>     make debs
>
> (or "make rpms" for RH/SLES) and then install the resulting packages 
> should work for most people if the pre-compiled packages do not.  This 
> avoids copying the files around, ensures that those files are cleaned 
> up when new packages are installed/removed, has proper dependencies, etc.
>
> Cheers, Andreas
>
>> On Jul 10, 2024, at 15:55, Apostolis Stamatis via lustre-discuss 
>> <lustre-discuss at lists.lustre.org> wrote:
>>
>> Hello all,
>>
>> Sharing an update on this issue to help anyone in the future 
>> encountering similar problems.
>>
>> The first solution I tried was to abandon the installation from 
>> source and install the deb packages. This fixed the issues with the 
>> linker and the compiler and I was able to use the C api, because it 
>> put the files on the default paths for gcc and ld. However, this 
>> caused other issues when I went to mount the filesystem (eg modprobe 
>> lustre not working), to my understanding caused by the deb packages 
>> being built for a slightly different kernel version.
>>
>> So this made me go back to installing from source and building the 
>> deb packages myself. In order to fix my path errors, I manually 
>> copied the required files to the default paths used by gcc and ld. I 
>> understand this is not necessary, as I can also add the paths when 
>> compiling. However this made me wonder whether there is a way to 
>> directly generate/copy the include files/libraries on the system 
>> defaults ? Looked through the docs but couldn't find anything on this.
>>
>> In any case, what I needed to copy for my use case (there might be 
>> additional files for completeness which I didn't need, not sure):
>>
>> cp lustre-release/lustre/include/lustre/*.h /usr/include/lustre/
>>
>> cp lustre-release/lustre/include/uapi/linux/lustre/*.h 
>> /usr/include/linux/lustre/
>>
>> cp lustre-release/debian/lustre-dev/usr/lib/* /usr/lib/
>>
>> After that the simple `gcc test.c -llustreapi` worked
>>
>> Thanks for the help, feel free to add anything you might think is 
>> useful (potentially a better way to do what I described)
>>
>> Regards,
>>
>> Apostolis
>>
>>
>> On 4/7/24 19:13, Apostolis Stamatis via lustre-discuss wrote:
>>> Thanks for the help Andreas, indeed installing the lustre-dev and 
>>> lustre-client-utils packages solved the issue with the lustreapi 
>>> library.
>>>
>>> However I am still getting an error:
>>>
>>> ```
>>>
>>> /usr/bin/ld: /tmp/cceAX1ZW.o: in function `main':
>>> test_file.c:(.text+0x37): undefined reference to `llapi_file_create'
>>> collect2: error: ld returned 1 exit status
>>>
>>> ```
>>>
>>> This leads me to believe I am doing something else wrong 
>>> (potentially with the includes?).
>>>
>>> Anyone with any input on what the issue might be or alternatively 
>>> the steps they have followed to use the C lustre api?
>>>
>>> Cheers, Apostolis
>>>
>>>
>>> On 8/6/24 21:19, Andreas Dilger wrote:
>>>> On Jun 8, 2024, at 08:14, Apostolis Stamatis via lustre-discuss 
>>>> <lustre-discuss at lists.lustre.org> wrote:
>>>>>
>>>>> Hello everyone,
>>>>>
>>>>> I am trying to use the C api for lustre, using Ubuntu 22.04, 
>>>>> kernel version 5.15.0-107 and lustre client modules version 2.15.4
>>>>> I am building lustre from source with the following steps (removed 
>>>>> some junk like git clone and cd) (mainly from the guide 
>>>>> https://metebalci.com/blog/lustre-2.15.4-on-rhel-8.9-and-ubuntu-22.04/)
>>>>
>>>> It would be great to copy this page to wiki.lustre.org 
>>>> <http://wiki.lustre.org>. It is a bit ironic that this page is 
>>>> mentioning that the wiki is outdated, but then proceeds to not 
>>>> update the wiki with new content...
>>>>
>>>>> ```
>>>>> sudo apt install build-essential libtool pkg-config flex bison 
>>>>> libpython3-dev libmount-dev libaio-dev libssl-dev libnl-genl-3-dev 
>>>>> libkeyutils-dev libyaml-dev libreadline-dev module-assistant 
>>>>> debhelper dpatch libsnmp-dev mpi-default-dev quilt swig
>>>>> sh autogen.sh
>>>>> ./configure --disable-server
>>>>> make dkms-debs
>>>>> sudo dpkg -i debs/lustre-client-modules-dkms_2.15.4-1_amd64.deb
>>>>> sudo apt --fix-broken install
>>>>> sudo dpkg -i debs/lustre-client-utils_2.15.4-1_amd64.deb
>>>>> ```
>>>>
>>>>> The client works as expected and can mount and modify the filesystem.
>>>>> However when I try to compile the sample program using gcc v 
>>>>> 11.4.0 with the command
>>>>> `gcc -I/usr/src/lustre-client-modules-2.15.4/lustre/include 
>>>>> -I/usr/src/lustre-client-modules-2.15.4/lustre/include/uapi/ 
>>>>> -I/usr/src/lustre-client-modules-2.15.4/lustre/include/lustre 
>>>>> -llustreapi test_file.c -o test`
>>>>> I get the error `/usr/bin/ld: cannot find -llustreapi: No such 
>>>>> file or directory`
>>>>>
>>>>> After trying to find the lustreapi library manually, indeed I 
>>>>> can't seem to find it anywhere
>>>>
>>>> According to the most recent build on the b2_15 branch on Ununtu 
>>>> 22.04 
>>>> https://build.whamcloud.com/job/lustre-b2_15/87/arch=x86_64,build_type=client,distro=ubuntu2204,ib_stack=inkernel/ 
>>>>
>>>>
>>>> There is a "lustre-dev" package built, and it looks like that this 
>>>> would contain the library files, "This package provides development 
>>>> libraries for the Lustre filesystem."
>>>>
>>>> Cheers, Andreas
>>> _______________________________________________
>>> lustre-discuss mailing list
>>> lustre-discuss at lists.lustre.org
>>> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
>> _______________________________________________
>> lustre-discuss mailing list
>> lustre-discuss at lists.lustre.org
>> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
>
> Cheers, Andreas
> --
> Andreas Dilger
> Lustre Principal Architect
> Whamcloud
>
>
>
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-discuss-lustre.org/attachments/20240725/91859cca/attachment.htm>