From gcc to the autotools

1. From GNU/GCC to the autotools Thierry GAYET

3. 1. Development’s environement

6. 2. Build process on gnu/linux

7. Summary 06/17/10 Source code .c / .cpp Headers .h Preprocessing gcc -E Linking ld Libraririe dynamique .so Compiling gcc -c Librairie statique .a Objetcs .o .o Output ELF Object Assembler as

9. Assembler 06/17/10 If the –S parameter is given, s top after the stage of compilation proper; do not assemble. The output is in the form of an assembler code file for each non-assembler input file specified. By default, the assembler file name for a source file is made by replacing the suffix .c, .i, etc., with .s. Input files that don't require compilation are ignored . Examples for having the assembly code of a .c source code : $ gcc -S file.c http://homepage.fudan.edu.cn/~euler/gcc_asm/

12. 3. Tools needs for the development

13. The toolchain, the heart of the development process A toolchain is the set of programming tools that are used to create a product (typically another computer program or system of programs). The tools may be used in a chain, so that the output of each tool becomes the input for the next, but the term is used widely to refer to any set of linked development tools. A simple software development toolchain consists of a text editor for editing source code, a compiler and linker to transform the source code into an executable program, libraries to provide interfaces to the operating system, and a debugger. A complex product such as a video game needs tools for preparing sound effects, music, textures, 3-dimensional models, and animations, and further tools for combining these resources into the finished product. On a workstation, the toolchain is made by all the tools used for compilation (gcc, gdb, ...). It includes all the headers, native libraries that can be used for the compilation and the linking. Beside, a toolchain is mostly used for cross-compiling code for another target. In order to use the right one, a prefix is used (sh4-linux-gcc, … ) ; it include the binutils inside.

14. The binutils The GNU Binutils are a collection of binary tools. The main ones are: * ld - the GNU linker. * as - the GNU assembler. But they also include: * addr2line - Converts addresses into filenames and line numbers. * ar - A utility for creating, modifying and extracting from archives. * gprof - Displays profiling information. * nm - Lists symbols from object files. * objcopy - Copys and translates object files. * objdump - Displays information from object files. * ranlib - Generates an index to the contents of an archive. * readelf - Displays information from any ELF format object file. * size - Lists the section sizes of an object or archive file. * strings - Lists printable strings from files. * strip - Discards symbols. For a cross-compilation usage a prefix is add before the name of the binutils (sh4-linux-ar, sh4-linux-nm, sh4-linux-strip, .... )

15. 4. Static and dynamic libraries

18. 06/17/10 Static libraries : tests Il est aussi possible de lister le contenu des fichiers objets inclus dans la librairie : $ ar –t mystaticlibrary cominter.o com_util.o filoint er.o miscell.o paral.o pilot.o simul_api.o spyinter.o userint.o La commande suivante permet de lister la liste des symboles par objets : $ nm mystaticlibrary.a ad_server.o: 00000099 T affichage_etat_client 00000004 C bDebugAd 00000004 C bDebugSu U bTrace 00000390 T close_socket cominter.o: 00000004 d bComInit 00000000 d bNoInitWarn 00000010 b bTrComInter U : undefined (implémentation externe) T : implémenté à l’intérieur « Pour un objet cela liste les fonctions et pour une librairie les fonctions par objet . »

20. 06/17/10 Dynamic libraries : creation Creating a dynamic library need to link al lthe objects into one ELF object. We don’t need any other binutil, but the ld linker itself through gcc : $ gcc -Wall -fPIC -c test1.c –o test1.o $ gcc -Wall -fPIC -c test2.c -0 test2.o $ gcc -shared -Wl,-soname,libtest.so.1 -o libtest.so.1.0 test1.o test2.o Listing of parameters that can be given to the compiler Those libraries have a .so extension but they are associated with a version (Minor and Major). Pour plus d’information : man ld Compiler options Definitions -Wall include all warnings. See man page for warnings specified. -fPIC Compiler directive to output position independent code, a characteristic required by shared libraries. Also see -fpic. -shared Produce a shared object which can then be linked with other objects to form an executable. -W1 Pass options to linker. In this example the options to be passed on to the linker are: ”-soname libctest.so.1”. The name passed with the ”-o” option is passed to gcc. -o Output of operation. In this case the name of the shared object to be output will be libctest.so.1.0

21. 06/17/10 Dynamic libraries : dependencies and runtime Once a binary is build this is possible to get a lising of its dynamic dependencies $ ldd mybinary libanasm7.so => /home/tgayet/vittam2/lib.i386_linux/libanasm7.so (0x40017000) libc.so.6 => /lib/tls/libc.so.6 (0x42000000) /lib/ld-linux.so.2 => /lib/ld-linux.so.2 (0x40000000) This command say with which libraries the binary is link ; it give also the path of the libraries. The nm tools can also be use in order to have the symboles (function build inside). The library shouldn’t not be stripped. At the runtime, a binary link with a dynamic library, will check the local cache through the ldconfig tool. Previously the configuration was made with the /etc/ld.so.conf but it have been replaced by the /etc/ld.so.conf.d/ (or /etc/ld.conf.d/). Regerating the cache : $ sudo ldconfig Dumping the cache : $ sudo ldconfig -p A dynamic library is link with a binary only during the runtime, so by modifing it, we can override through the LD_LIBRARY_PATH environment variable: $ export LD_LIBRARY_PATH=./my_lib_path $ ./mybinary

22. 06/17/10 Dynamic libraries : linking In order to link a dynamic library with a biry we can specify some information to the ld linker: $ gcc -fPIC test.o –L./libpath –ltest This command say that the test.o source code that will make the final binary will be dynamically link with the libtest.so library. The « lib » prefix should not be specify because it is automaticaly added (For libtest.so, only test should be specify to the –l parameter). The –l<libname> parameter ask to the linker to search a library called libname. By default, a dynamic (libname.so) one is used except is the –static parameter is provide. DYNAMIC LIB BINARY Linker option Description -l Provide a libname (eg: for libtest, only –ltest is specify) -L Provide a path or a set for the libraries

24. 5. Manual build

26. Debug mode Gcc have a set of parameter that can be used for customizing the debug mode : For more information : man gcc Parameters Description -g Produce debugging information in the operating system's native format (stabs, COFF, XCOFF, or DWARF 2). GDB can work with this debugging information. -ggdb Produce debugging information for use by GDB. This means to use the most expressive format available (DWARF 2, stabs, or the native format if neither of those are supported), including GDB extensions if at all possible. -Werror All warning become errors -Wall Enable all warning

27. Optimization mode Gcc have a set of parameter that can be used for customizing the debug mode : If multiple -O options, with or without level numbers, the last such option is the one that is effective. By definition, no optimization should be set (-O0) if the debug mode is used. Parameters Description -O -O1 Optimize. Optimizing compilation takes somewhat more time, and a lot more memory for a large function. -O2 Optimize even more. GCC performs nearly all supported optimizations that do not involve a space-speed gcc-3.3.2 Last change: 2003-10-16 57 GNU GCC(1) tradeoff. The compiler does not perform loop unrolling or function inlining when you specify -O2. As compared to -O, this option increases both compilation time and the performance of the generated code. -O3 Optimize yet more. -O3 turns on all optimizations specified by -O2 and also turns on the -finline-functions and -frename-registers options. -O0 Do not optimize. This is the default. -Os Optimize for size. -Os enables all -O2 optimizations that do not typically increase code size. It also performs further optimizations designed to reduce code size. -Os disables the following optimization flags: -falign-functions -falign-jumps -falign-loops -falign-labels -freorder-blocks -fprefetch-loop-arrays

28. Conditional compilation In order to customize the source code that will be compiled, we can provide to the C preprocessor, some flags for the compilation (CFLAGS) that will enable or not some part of code. Example with a Part of code to include : #ifdef CONDITION printf(“If you see this message, that's mean that this code is enable.“); #endif In order to say to enable this part of code, we use the following command : $ gcc –c –o test.o –DCONDITION test.c or $ gcc –c –o test.o –DCONDITION=1 test.c Other C preprocessor commands : #ifdef : if defined #ifndef : if not defined #if : if #elif : else if #endif : end of if or ifdef condition Example of usage : #if (__cplusplus==199711L) … #endif #ifndef OnceTime #define OnceTime (To include only one time) #endif

30. Buildin profiling with gnu/prof 1/3 The first step in generating profile information for your program is to compile and link it with profiling enabled. To compile a source file for profiling, specify the `-pg' option when you run the compiler. (This is in addition to the options you normally use.) To link the program for profiling, if you use a compiler such as cc to do the linking, simply specify `-pg' in addition to your usual options. The same option, `-pg', alters either compilation or linking to do what is necessary for profiling. Here are examples: $ gcc -g -c myprog.c utils.c -pg  compilation $ gcc -o myprog myprog.o utils.o –pg  link The `-pg' option also works with a command that both compiles and links: $ gcc -o myprog myprog.c utils.c -g -pg Note: The `-pg' option must be part of your compilation options as well as your link options. If it is not then no call-graph data will be gathered and when you run gprof you will get an error message like this: gprof: gmon.out file is missing call-graph data If you add the `-Q' switch to suppress the printing of the call graph data you will still be able to see the time samples: Flat profile: Each sample counts as 0.01 seconds. % cumulative self self total time seconds seconds calls ms/call ms/call name 33.34 0.02 0.02 7208 0.00 0.00 open 16.67 0.03 0.01 244 0.04 0.12 offtime 16.67 0.04 0.01 8 1.25 1.25 memccpy 16.67 0.05 0.01 7 1.43 1.43 write

32. Profiling with gnu/prof 3/3 The remaining functions in the listing (those whose self seconds field is 0.00) didn't appear in the histogram samples at all. However, the call graph indicated that they were called, so therefore they are listed, sorted in decreasing order by the calls field. Clearly some time was spent executing these functions, but the paucity of histogram samples prevents any determination of how much time each took. Here is what the fields in each line mean : % time This is the percentage of the total execution time your program spent in this function. These should all add up to 100%. cumulative seconds This is the cumulative total number of seconds the computer spent executing this functions, plus the time spent in all the functions above this one in this table. self seconds This is the number of seconds accounted for by this function alone. The flat profile listing is sorted first by this number. calls This is the total number of times the function was called. If the function was never called, or the number of times it was called cannot be determined (probably because the function was not compiled with profiling enabled), the calls field is blank. self ms/call This represents the average number of milliseconds spent in this function per call, if this function is profiled. Otherwise, this field is blank for this function. total ms/call This represents the average number of milliseconds spent in this function and its descendants per call, if this function is profiled. Otherwise, this field is blank for this function. This is the only field in the flat profile that uses call graph analysis. Name This is the name of the function. The flat profile is sorted by this field alphabetically after the self seconds and calls fields are sorted.

33. 06/17/10 6. Building with GNU/MAKE

34. 06/17/10 Automatic build using gnu/make The make program gets its dependency &quot;graph&quot; from a text file called makefile or Makefile which resides in the same directory as the source files. Make checks the modification times of the files, and whenever a file becomes &quot;newer&quot; than something that depends on it, (in other words, modified) it runs the compiler accordingly. Project1 is a target and the name of the final binary. Its dependencies are the three objects data.o, main.o and io.o. Each one have in teir turn their own dependencies that make will resolve one by one. If you edit io.c, it becomes &quot;newer&quot; than io.o, meaning that make must run cc -c io.c to create a new io.o, then run cc data.o main.o io.o -o project1 for project1.

35. 06/17/10 Automatic build using gnu/make Each dependency shown in the graph is circled with a corresponding color in the Makefile, and each uses the following format: target : source file(s) command (must be preceded by a tab) A target given in the Makefile is a file which will be created or updated when any of its source files are modified. The command(s) given in the subsequent line(s) (which must be preceded by a tab character) are executed in order to create the target file. For more information : http://www.gnu.org/doc/doc.html

37. 06/17/10 Automatic build using gnu/make In addition to those macros which you can create yourself, there are a few macros which are used internally by the make program. Here are some of those, listed below: You can also manipulate the way these macros are evaluated, as follows, assuming that OBJS = data.o io.o main.o, using $(OBJS:.o=.c) within the Makefile substitutes .o at the end with .c, giving you the following result: data.c io.c main.c For debugging a Makefile : $ make -d CC Contains the current C compiler. Defaults to gcc. CFLAGS Special options which are added to the built-in C rule. $@ Full name of the current target. $? A list of files for current dependency which are out-of-date. $< The source file of the current (single) dependency. LDFLAGS Special options which are added to the link.

39. 06/17/10 7. Building with the autotools

42. 06/17/10 Automatic build using the autotools List of the most useful targets that the GNU Coding Standards specify : make all Build programs, libraries, documentation, etc. (same as make). make install Install what needs to be installed, copying the files from the package's tree to system-wide directories. Same as make install, then strip debugging symbols. Some users like to trade space for useful bug reports... make install-strip Same as make install, then strip debugging symbols. Some users like to trade space for useful bug reports... make uninstall The opposite of make install: erase the installed files. (This needs to be run from the same build tree that was installed.) make clean Erase from the build tree the files built by make all. make distclean Additionally erase anything ./configure created. make check Run the test suite, if any. make installcheck Check the installed programs or libraries, if supported. make dist Recreate package-version.tar.gz from all the source files.

43. 06/17/10 Example 1 : basic project makefile.am configure.ac src/ include/  If you prefer to generate intermediate object in a obj/ directory (or src/), you can move the makefile.am to the directory choosen). autogen.sh Contains all the source code Contains all the headers Template for the project Template for the Makefile Generate the final files The configure.in are a copy of the configure.ac

44. 06/17/10 Example 1 : advanced project makefile.am configure.ac src/ include/ autogen.sh Template for the project Main template for the Makefile Generate the final files module1/ module2/ makefile.am Sub-template1 for the Makefile src/ include/ makefile.am Sub-template2 for the Makefile

45. 06/17/10 Using an autotools project at a glance Then, using this autotools project is simple : 1. Generating the final files from the autotools templates: $ ./autotgen.sh makefile.am  Makefile.in configure.ac  configure 2. Launch the configure script with parameters: $ ./configure –prefix=/usr –enable-debug 3. Launch the compilation: $ make 4. Install the files: $ make DESTDIR=/stagingdir install 5. Launch the unitary tests: $ make check 6. Ask to generate the documentation: $ make html  All the intermediate files have a .in extension ; they will be use as input by the configure script for generating all the final files.

46. 06/17/10 The configure.ac template AC_PREREQ(2.59) AC_INIT([pyPackage], [myPackageVersion], [ [email_address] ]) AC_ARG_ENABLE(debug, AS_HELP_STRING([--enable-debug], [enable debugging support]), [enable_debug=$enableval], [enable_debug=no] ) if test &quot;$enable_debug&quot; = &quot;yes&quot; ; then CXXFLAGS=&quot;$CXXFLAGS -Wall -ggdb -O0&quot; AC_DEFINE(DEBUG, 1, [Define to enable debug build]) else CXXFLAGS=&quot;$CXXFLAGS -Wall -O2&quot; fi PKG_CHECK_MODULES([DIRECTFB],[directfb],[have_libdirectfb=yes],[have_libdirectfb=no]) if test &quot;$have_libdirectfb&quot; = no ; then AC_MSG_ERROR([Missing directfb-1.4.1 library!!]) fi AC_OUTPUT(Makefile) Example of template for a configure.ac script

49. 06/17/10 Customization Configure script have predefined rules and target (they can be redefined) can be customized using macros : AC_ARG_ENABLE(test, AS_HELP_STRING([--enable-test], [Enable the test unitary support [default=no]]), [case &quot;${enableval}&quot; in yes) have_test=true ;; no) have_test=false ;; *) AC_MSG_ERROR(bad value ${enableval} for --enable-test) ;; esac], [have_test=false]) The listing of the parameters available can be display: $ ./configure --help boolean switch AC_ARG_WITH( optim-level, AS_HELP_STRING([--with-optim-level=<0,1,2,3>],[Provide the optim level to give to gcc as -O<level>]), [current_optim_level=$withval], [current_optim_level=0]) Switch with value Usage: ./configure –enable-test Usage: ./configure –with-optim-level=2

51. 06/17/10 Integrating some optimization AC_ARG_ENABLE(test, AS_HELP_STRING([--enable-test], [Enable the test unitary support [default=no]]), [case &quot;${enableval}&quot; in yes) have_test=true ;; no) have_test=false ;; *) AC_MSG_ERROR(bad value ${enableval} for --enable-test) ;; esac], [have_test=false]) dnl -------------------------------------------- dnl Export the conditional HAVE_TEST variable to the Makefiles dnl -------------------------------------------- AM_CONDITIONAL(HAVE_TEST, $have_test) dnl -------------------------------------------- dnl Test the have_test variable and if equal to true dnl -------------------------------------------- AC_MSG_CHECKING(Checking the test support) if test &quot;$have_test&quot; != &quot;false&quot; ; then dnl -------------------------------------------- dnl Display the result of the test ... yes dnl -------------------------------------------- AC_MSG_RESULT([yes]) dnl -------------------------------------------- dnl Test if the libgtest is well available in the stagingdir dnl If so get the cflags and ldflag from the .pc file dnl -------------------------------------------- PKG_CHECK_MODULES([GTEST],[gtest],[have_libgtest=yes],[have_libgtest=no]) if test &quot;$have_libgtest&quot; = no ; then AC_MSG_ERROR([Missing libgtest library (http://code.google.com/p/googletest/) !!]) fi else dnl -------------------------------------------- dnl Display the result of the test ... no dnl -------------------------------------------- AC_MSG_RESULT([no]) fi AC_MSG_CHECKING([Optimization level]) AC_ARG_WITH( optim-level, AS_HELP_STRING([--with-optim-level=<0,1,2,3>],[Provide the optim level to give to gcc as -O<level>]), [current_optim_level=$withval], [current_optim_level=0]) if test &quot;$current_optim_level&quot; != &quot;0&quot; ; then dnl override the default optim level case &quot;$current_optim_level&quot; in &quot;1&quot; | &quot;2&quot; | &quot;3&quot; ) ;; *) AC_MSG_ERROR(bad value ${withval} for the optim-level parameter. It must be a number between 1 and 3.) ;; esac fi if test &quot;$have_debug&quot; = &quot;true&quot; ; then current_optim_level=&quot;0&quot; AC_MSG_RESULT([-O$current_optim_level!! Remove all optimization in debug mode.]) else AC_MSG_RESULT([-O$current_optim_level]) fi This is possible to say to gcc which optimization level use for one build : Usage : $ ./configure –with-optim-level=3 By default, no optimization is set ; in debug mode no one is required.

54. 06/17/10 Integrating unit test if HAVE_TEST TESTS = gmos_gtest else TESTS = endif noinst_PROGRAMS = $(TESTS) gmos_gtest_SOURCES = $(top_srcdir)/unit_test/src/test_gm_os.cpp gmos_gtest_CPPFLAGS = -I$(HEADER_DIR) $(GTEST_CFLAGS) gmos_gtest_LDFLAGS = $(top_srcdir)/.libs/libgm_os.a -lpthread gmos_gtest_LDADD = $(GTEST_LIBS) AC_ARG_ENABLE(test, AS_HELP_STRING([--enable-test], [Enable the test unitary support [default=no]]), [case &quot;${enableval}&quot; in yes) have_test=true ;; no) have_test=false ;; *) AC_MSG_ERROR(bad value ${enableval} for --enable-test) ;; esac], [have_test=false]) AM_CONDITIONAL(HAVE_TEST, $have_test) AC_MSG_CHECKING(Checking the test support) if test &quot;$have_test&quot; != &quot;false&quot; ; then AC_MSG_RESULT([yes]) PKG_CHECK_MODULES([GTEST],[gtest],[have_libgtest=yes],[have_libgtest=no]) if test &quot;$have_libgtest&quot; = no ; then AC_MSG_ERROR([Missing libgtest library (http://code.google.com/p/googletest/) !!]) fi Else AC_MSG_RESULT([no]) fi Makefile.am configure.ac Usage: ./configure –enable-test make check

56. 06/17/10 Verbose/silent mode Building with all details can be usefull in debug mode but it can be unwanted for the release mode. There are a definition to say if we want to enable the mode silent or not : m4_ifdef([AM_SILENT_RULES],[AM_SILENT_RULES([yes/no])])

57. 06/17/10 Example of Makefile’s template makefile.am HEADER_DIR = $(top_srcdir)/inc # ---------------------------------------------------------------- # Header to be install in the stagingdir (make install) # gm_os is the prefix in <statingdir_path>/usr/include/gm_os/<headers> # ---------------------------------------------------------------- lib_includedir = $(includedir)/gm_os/ lib_include_HEADERS = $(HEADER_DIR)/gm_os_types.h $(HEADER_DIR)/gm_os_trace.h $(HEADER_DIR)/gm_os.h # ---------------------------------------------------------------- # Use for the installation in the stagingdir # ---------------------------------------------------------------- pkgconfigdir = $(libdir)/pkgconfig pkgconfig_DATA = gm_os.pc # ---------------------------------------------------------------- # Compilation and generation of the gm_os library # ---------------------------------------------------------------- lib_LTLIBRARIES = libgm_os.la libgm_os_la_SOURCES = src/gm_os_misc.c src/gm_os_heap.c src/gm_os_message.c libgm_os_la_CFLAGS = -I$(HEADER_DIR) $(GLIB2_CFLAGS) $(MY_DEBUG_CFLAG) libgm_os_la_LDFLAGS = -version-info $(LT_CURRENT):$(LT_REVISION):$(LT_AGE) -lpthread $(MY_DEBUG_LDFLAGS) libgm_os_la_LIBADD = $(GLIB2_LIBS)

58. 06/17/10 Doxygen documentation The documentation is automaticaly generate at the doxygen format (html). For this generation a generic doxygen configuration is used (default-Doxyfile)and is upgrade by the makefile before to launch the documentation: DOXYFILE ?= $(srcdir)/default-Doxyfile html-local: @echo -n &quot;Removing the previous documentation : &quot; @ rm -fR ./html @echo &quot;OK&quot; @echo -n &quot;Preparing the documentation requirement : &quot; @cp $(DOXYFILE) Doxyfile @chmod +w Doxyfile @echo &quot;PROJECT_NAME=@PACKAGE_NAME@&quot; >> Doxyfile @echo &quot;PROJECT_NUMBER=@PACKAGE_VERSION@&quot; >> Doxyfile @echo &quot;INPUT=$(srcdir)/inc&quot; >> Doxyfile @echo &quot;OK&quot; @echo -n &quot;Generating the documentation in doxygen format : &quot; @doxygen > /dev/null 2>&1 @rm -f Doxyfile @echo &quot;OK&quot; Usage: make html  That will create a new html directory that contain the documentation. The main html file is index.html. makefile.am

61. 06/17/10 Any question ?