Why libraries are used:
This methodology, also known as "shared components" or "archive libraries", groups together multiple compiled object code files into a single file known as a library. Typically C functions/C++ classes and methods which can be shared by more than one application are broken out of the application's source code, compiled and bundled into a library. The C standard libraries and C++ STL are examples of shared components which can be linked with your code. The benefit is that each and every object file need not be stated when linking because the developer can reference the individual library. This simplifies the multiple use and sharing of software components between applications. It also allows application vendors a way to simply release an API to interface with an application. Components which are large can be created for dynamic use, thus the library remain separate from the executable reducing it's size and thus disk space used. The library components are then called by various applications for use when needed.
| Linux Library Types: |
- Static libraries (.a): Library of object code which is linked with, and becomes part of the application.
- Dynamically linked shared object libraries (.so): There is only one form of this library but it can be used in two ways.
- Dynamically linked at run time but statically aware. The libraries must be available during compile/link phase. The shared objects are not included into the executable component but are tied to the execution.
- Dynamically loaded/unloaded and linked during execution (i.e. browser plug-in) using the dynamic linking loader system functions.
Library naming conventions:
Libraries are typically names with the prefix "lib". This is true for all the C standard libraries. When linking, the command line reference to the library will not contain the library prefix or suffix. Thus the following link command: gcc src-file.c -lm -lpthreadThe libraries referenced in this example for inclusion during linking are the math library and the thread library. They are found in /usr/lib/libm.a and /usr/lib/libpthread.a.
| Static Libraries: (.a) |
- Compile: cc -Wall -c ctest1.c ctest2.c
Compiler options:- -Wall: include warnings. See man page for warnings specified.
- Create library "libctest.a": ar -cvq libctest.a ctest1.o ctest2.o
- List files in library: ar -t libctest.a
- Linking with the library:
- cc -o executable-name prog.c libctest.a
- cc -o executable-name prog.c -L/path/to/library-directory -lctest
- Example files:
- ctest1.c
void ctest1(int *i) { *i=5; } - ctest2.c
void ctest2(int *i) { *i=100; } - prog.c
#include <stdio.h> void ctest1(int *); void ctest2(int *); int main() { int x; ctest1(&x); printf("Valx=%d\n",x); return 0; }
- ctest1.c
| Dynamically Linked "Shared Object" Libraries: (.so) |
- Create object code
- Create library
- Optional: create default version using a symbolic link.
gcc -Wall -fPIC -c *.c gcc -shared -Wl,-soname,libctest.so.1 -o libctest.so.1.0 *.o mv libctest.so.1.0 /opt/lib ln -sf /opt/lib/libctest.so.1.0 /opt/lib/libctest.so ln -sf /opt/lib/libctest.so.1.0 /opt/lib/libctest.so.1This creates the library libctest.so.1.0 and symbolic links to it. Compiler options:- -Wall: include warnings. See man page for warnings specified.
- -fPIC: Compiler directive to output position independent code, a characteristic required by shared libraries. Also see "-fpic".
- -shared: Produce a shared object which can then be linked with other objects to form an executable.
- -W1: Pass options to linker.
In this example the options to be passed on to the linker are: "-soname libctest.so.1". The name passed with the "-o" option is passed to gcc. - Option -o: Output of operation. In this case the name of the shared object to be output will be "libctest.so.1.0"
- The link to /opt/lib/libctest.so allows the naming convention for the compile flag -lctest to work.
- The link to /opt/lib/libctest.so.1 allows the run time binding to work. See dependency below.
- Compiling for runtime linking with a dynamically linked libctest.so.1.0:
gcc -Wall -I/path/to/include-files -L/path/to/libraries prog.c -lctest -o prog Use: gcc -Wall -L/opt/lib prog.c -lctest -o progWhere the name of the library is libctest.so. (This is why you must create the symbolic links or you will get the error "/usr/bin/ld: cannot find -lctest".) The libraries will NOT be included in the executable but will be dynamically linked during runtime execution.
The shared library dependencies of the executable can be listed with the command: ldd name-of-executable
- Example: ldd prog
libctest.so.1 => /opt/lib/libctest.so.1 (0x00002aaaaaaac000) libc.so.6 => /lib64/tls/libc.so.6 (0x0000003aa4e00000) /lib64/ld-linux-x86-64.so.2 (0x0000003aa4c00000)
- Set path: export LD_LIBRARY_PATH=/opt/lib:$LD_LIBRARY_PATH
- Run: prog
Links:
| Library Path: |
- Add library directories to be included during dynamic linking to the file /etc/ld.so.conf Sample: /etc/ld.so.conf
/usr/X11R6/lib /usr/lib ... .. /usr/lib/sane /usr/lib/mysql /opt/lib
You can use the "-f file-name" flag to reference another configuration file if you are developing for different environments.
See man page for command ldconfig. OR
- Add specified directory to library cache: (as root)
ldconfig -n /opt/lib
Where /opt/lib is the directory containing your library libctest.so
(When developing and just adding your current directory: ldconfig -n . Link with -L.) This will NOT permanently configure the system to include this directory. The information will be lost upon system reboot.
OR
- Specify the environment variable LD_LIBRARY_PATH to point to the directory paths containing the shared object library. This will specify to the run time loader that the library paths will be used during execution to resolve dependencies.
(Linux/Solaris: LD_LIBRARY_PATH, SGI: LD_LIBRARYN32_PATH, AIX: LIBPATH, Mac OS X: DYLD_LIBRARY_PATH, HP-UX: SHLIB_PATH) Example (bash shell): export LD_LIBRARY_PATH=/opt/lib:$LD_LIBRARY_PATH or add to your ~/.bashrc file:
... if [ -d /opt/lib ]; then LD_LIBRARY_PATH=/opt/lib:$LD_LIBRARY_PATH fi ... export LD_LIBRARY_PATH
This instructs the run time loader to look in the path described by the environment variable LD_LIBRARY_PATH, to resolve shared libraries. This will include the path /opt/lib.
| Library Info: |
Use the command nm -D libctest.so.1.0
(or nm --dynamic libctest.so.1.0)
0000000000100988 A __bss_start 000000000000068c T ctest1 00000000000006a0 T ctest2 w __cxa_finalize 00000000001007b0 A _DYNAMIC 0000000000100988 A _edata 0000000000100990 A _end 00000000000006f8 T _fini 0000000000100958 A _GLOBAL_OFFSET_TABLE_ w __gmon_start__ 00000000000005b0 T _init w _Jv_RegisterClasses
Symbol Type Description A The symbol's value is absolute, and will not be changed by further linking. B Un-initialized data section D Initialized data section T Normal code section U Undefined symbol used but not defined. Dependency on another library. W Doubly defined symbol. If found, allow definition in another library to resolve dependency.
| Library Versions: |
The library version can be specified when the shared object library is created. If the library is expected to be updated, then a library version should be specified. This is especially important for shared object libraries which are dynamically linked. This also avoids the Microsoft "DLL hell" problem of conflicting libraries where a system upgrade which changes a standard library breaks an older application expecting an older version of the the shared object function.
Versioning occurs with the GNU C/C++ libraries as well. This often make binaries compiled with one version of the GNU tools incompatible with binaries compiled with other versions unless those versions also reside on the system. Multiple versions of the same library can reside on the same system due to versioning. The version of the library is included in the symbol name so the linker knows which version to link with.
One can look at the symbol version used: nm csub1.o
00000000 T ctest1
No version is specified in object code by default.
Note: This is only useful when creating shared libraries. It is assumed that the programmer knows which libraries to link with when static linking. Runtime linking allows opportunity for library incompatibility.
GNU/Linux, see examples of version scripts here: sysdeps/unix/sysv/linux/Versions
Some symbols may also get version strings from assembler code which appears in glibc headers files. Look at include/libc-symbols.h.
Example: nm /lib/libc.so.6 | more
00000000 A GCC_3.0 00000000 A GLIBC_2.0 00000000 A GLIBC_2.1 00000000 A GLIBC_2.1.1 00000000 A GLIBC_2.1.2 00000000 A GLIBC_2.1.3 00000000 A GLIBC_2.2 00000000 A GLIBC_2.2.1 00000000 A GLIBC_2.2.2 00000000 A GLIBC_2.2.3 00000000 A GLIBC_2.2.4 ... ..Note the use of a version script. Library referencing a versioned library: nm /lib/libutil-2.2.5.so.. ... U strcpy@@GLIBC_2.0 U strncmp@@GLIBC_2.0 U strncpy@@GLIBC_2.0 ... ..
| Dynamic loading and un-loading of shared libraries using libdl: |
Prototype include file for the library: ctest.h
Use the notation extern "C" so the libraries can be used with C and C++. This statement prevents the C++ from name mangling and thus creating "unresolved symbols" when linking.#ifndef CTEST_H #define CTEST_H #ifdef __cplusplus extern "C" { #endif void ctest1(int *); void ctest2(int *); #ifdef __cplusplus } #endif #endif
#include <stdio.h> #include <dlfcn.h> #include "ctest.h" int main(int argc, char **argv) { void *lib_handle; double (*fn)(int *); int x; char *error; lib_handle = dlopen("/opt/lib/libctest.so", RTLD_LAZY); if (!lib_handle) { fprintf(stderr, "%s\n", dlerror()); exit(1); } fn = dlsym(lib_handle, "ctest1"); if ((error = dlerror()) != NULL) { fprintf(stderr, "%s\n", error); exit(1); } (*fn)(&x); printf("Valx=%d\n",x); dlclose(lib_handle); return 0; }
Explanation:
- dlopen("/opt/lib/libctest.so", RTLD_LAZY);
Open shared library named "libctest.so".
The second argument indicates the binding. See include file dlfcn.h.
Returns NULL if it fails.
Options:- RTLD_LAZY: If specified, Linux is not concerned about unresolved symbols until they are referenced.
- RTLD_NOW: All unresolved symbols resolved when dlopen() is called.
- RTLD_GLOBAL: Make symbol libraries visible.
- dlsym(lib_handle, "ctest1");
Returns address to the function which has been loaded with the shared library..
Returns NULL if it fails.
Note: When using C++ functions, first use nm to find the "mangled" symbol name or use the extern "C" construct to avoid name mangling.
i.e. extern "C" void function-name();
Use the "-Wl" gcc/g++ compiler flag to pass command line arguments on to the GNU "ld" linker.
Example makefile statement: g++ -rdynamic -o appexe $(OBJ) $(LINKFLAGS) -Wl,--whole-archive -L{AA_libs} -laa -Wl,--no-whole-archive $(LIBS)
- --whole-archive: This linker directive specifies that the libraries listed following this directive (in this case AA_libs) shall be included in the resulting output even though there may not be any calls requiring its presence. This option is used to specify libraries which the loadable libraries will require at run time.
- -no-whole-archive: This needs to be specified whether you list additional object files or not. The gcc/g++ compiler will add its own list of archive libraries and you would not want all the object code in the archive library linked in if not needed. It toggles the behavior back to normal for the rest of the archive libraries.
- dlopen() - gain access to an executable object file
- dclose() - close a dlopen object
- dlsym() - obtain the address of a symbol from a dlopen object
- dlvsym() - Programming interface to dynamic linking loader.
- dlerror() - get diagnostic information
- Shared Libraries-Dynamic Loading and Unloading
- GNOME Glib dynamic loading of modules - cross platform API for dynamically loading "plug-ins".
| C++ class objects and dynamic loading: |
When running the above "C" examples with the "C++" compiler one will quickly find that "C++" function names get mangled and thus will not work unless the function definitions are protected with extern "C"{}.
- Note that the following are not equivalent:
The following are equivalent:extern "C" { int functionx(); }extern "C" int functionx();extern "C" { extern int functionx(); }extern "C" int functionx();
The dynamic library loading routines enable the programmer to load "C" functions. In C++ we would like to load class member functions. In fact the entire class may be in the library and we may want to load and have access to the entire object and all of its member functions. Do this by passing a "C" class factory function which instantiates the class.
The class ".h" file:
class Abc { ... ... }; // Class factory "C" functions typedef Abc* create_t; typedef void destroy_t(Abc*);
The class ".cpp" file:
This file is the source to the library. The "C" functions to instantiate (create) and destroy a class defined in the dynamically loaded library where "Abc" is the C++ class.Abc::Abc() { ... } extern "C" { // These two "C" functions manage the creation and destruction of the class Abc Abc* create() { return new Abc; } void destroy(Abc* p) { delete p; // Can use a base class or derived class pointer here } }
Main executable which calls the loadable libraries:
// load the symbols create_t* create_abc = (create_t*) dlsym(lib_handle, "create"); ... ... destroy_t* destroy_abc = (destroy_t*) dlsym(lib_handle, "destroy"); ... ...
Pitfalls:
- The new/delete of the C++ class should both be provided by the executable or the library but not split. This is so that there is no surprise if one overloads new/delete in one or the other.
Links:
| Comparison to the Microsoft DLL: |
The Visual C++ .NET IDE wizard will create a DLL framework through the GUI, and generates a ".def" file. This "module definition file" lists the functions to be exported. When exporting C++ functions, the C++ mangled names are used. Using the Visual C++ compiler to generate a ".map" file will allow you to discover the C++ mangled name to use in the ".def" file. The "SECTIONS" label in the ".def" file will define the portions which are "shared". Unfortunately the generation of DLLs are tightly coupled to the Microsoft IDE, so much so that I would not recomend trying to create one without it.
The Microsoft Windows C++ equivalent functions to libdl are the following functions:
- ::LoadLibrary() - dlopen()
- ::GetProcAddress() - dlsym()
- ::FreeLibrary() - dlclose()
Cross platform (Linux and MS/Windows) C++ code snippet:
Include file declaration: (.h or .hpp)
class Abc{ public: static Abc* Instance(); // Function declaration. Could also be used as a public class member function. private: static Abc *mInstance; // Singleton. Use this declaration in C++ class member variable declaration. ... }
C/C++ Function source: (.cpp)
/// Singleton instantiation Abc* Abc::mInstance = 0; // Use this declaration for C++ class member variable // (Defined outside of class definition in ".cpp" file) // Return unique pointer to instance of Abc or create it if it does not exist. // (Unique to both exe and dll) static Abc* Abc::Instance() // Singleton { #ifdef WIN32 // If pointer to instance of Abc exists (true) then return instance pointer else look for // instance pointer in memory mapped pointer. If the instance pointer does not exist in // memory mapped pointer, return a newly created pointer to an instance of Abc. return mInstance ? mInstance : (mInstance = (Abc*) MemoryMappedPointers::getPointer("Abc")) ? mInstance : (mInstance = (Abc*) MemoryMappedPointers::createEntry("Abc",(void*)new Abc)); #else // If pointer to instance of Abc exists (true) then return instance pointer // else return a newly created pointer to an instance of Abc. return mInstance ? mInstance : (mInstance = new Abc); #endif }
Note that the GNU linker does not have this problem.
For more on singletons see the YoLinux.com C++ singleton software design pattern tutorial.
| Cross platform programming of loadable libraries: |
#ifndef USE_PRECOMPILED_HEADERS #ifdef WIN32 #include <direct.h> #include <windows.h> #else #include <sys/types.h> #include <dlfcn.h> #endif #include <iostream> #endif using namespace std; #ifdef WIN32 HINSTANCE lib_handle; #else void *lib_handle; #endif // Where retType is the pointer to a return type of the function // This return type can be int, float, double, etc or a struct or class. typedef retType* func_t; // load the library ------------------------------------------------- #ifdef WIN32 string nameOfLibToLoad("C:\opt\lib\libctest.dll"); lib_handle = LoadLibrary(TEXT(nameOfLibToLoad.c_str())); if (!lib_handle) { cerr << "Cannot load library: " << TEXT(nameOfDllToLoad.c_str()) << endl; } #else string nameOfLibToLoad("/opt/lib/libctest.so"); lib_handle = dlopen(nameOfLibToLoad.c_str(), RTLD_LAZY); if (!lib_handle) { cerr << "Cannot load library: " << dlerror() << endl; } #endif ... ... ... // load the symbols ------------------------------------------------- #ifdef WIN32 func_t* fn_handle = (func_t*) GetProcAddress(lib_handle, "superfunctionx"); if (!fn_handle) { cerr << "Cannot load symbol superfunctionx: " << GetLastError() << endl; } #else // reset errors dlerror(); // load the symbols (handle to function "superfunctionx") func_t* fn_handle= (func_t*) dlsym(lib_handle, "superfunctionx"); const char* dlsym_error = dlerror(); if (dlsym_error) { cerr << "Cannot load symbol superfunctionx: " << dlsym_error << endl; } #endif ... ... ... // unload the library ----------------------------------------------- #ifdef WIN32 FreeLibrary(lib_handle); #else dlclose(lib_handle); #endif
| Tools: |
- ar - create, modify, and extract from archives
- ranlib - generate index to archive
- nm - list symbols from object files
- ld - Linker
- ldconfig - configure dynamic linker run-time bindings
ldconfig -p : Print the lists of directories and candidate libraries stored in the current cache.
i.e. /sbin/ldconfig -p |grep libGL - ldd - print shared library dependencies
- gcc/g++ - GNU project C and C++ compiler
- man page to: ld.so - a.out dynamic linker/loader
| Notes: |
- Direct loader to preload a specific shared library before all others: export LD_PRELOAD=/usr/lib/libXXX.so.x; exec program. This is specified in the file /etc/ld.so.preload and extended with the environment variable LD_PRELOAD.
Also see:- man page to: ld.so - a.out dynamic linker/loader
- LD_PRELOAD and Linux function interception.
- Running Red Hat 7.1 (glibc 2.2.2) but compiling for Red Hat 6.2 compatibility.
See RELEASE-NOTES
export LD_ASSUME_KERNEL=2.2.5 . /usr/i386-glibc21-linux/bin/i386-glibc21-linux-env.sh
- Environment variable to highlight warnings, errors, etc: export CC="colorgcc"
