Part 1 of our C Programming Tutorial covers the basics of compiling. What is a compiler? How does it work? How do I use a compiler to write programs in C?
There are FOUR STAGES to the compilation process:
example.
Commands preceded by a hash sign in the source code in this manner are referred to as a *Preprocessor Directive*. Two of the most common Preprocessor Directives are "include" and "define".
By default, the results of "clang -E" print to standard output (i.e. the monitor). However, the results can easily be saved to a new file:
In the example above, we have instructed clang to compile only the preprocessor directives in the file `hello.c`, and save the resulting output to a file named `hello2.c`.
Using `clang -S` will result in a *.s file (in the example above, hello2.s). The resulting assembly encoded language is processor-specific. In this tutorial, the code has been compiled on a Virtual Machine with a x86 vPU; as a result, the assembly code transformation is as follows (white space and carriage returns have been included as originally formatted with no additional indentation):
-------------------------------------------------------
example.
application written in x86 vPU specific assembly.
-------------------------------------------------------
-------------------------------------------------------
Very few modern developers write their applications in assembly. However, every application written in C is converted into assembly during the compile process.
If we view the resulting file, hello2.o, using vi the output will resemble this:
-------------------------------------------------------
Note that the Assembler transforms assembly code into code that is readable to the CPU; but not easily readable to human beings.
example.
#include <math.h> //this is our new directive
Looking back at the beginning of our tutorial, we can see that my very first version of this application was called `hello.c`. I have made these changes above to this original `hello.c` file. I then skip ahead to using `clang -c` to parse my assembly application to machine code; but this time, I am using `hello.c` and not `hello2.s`.
As a result of this operation, I now have an object file `hello.o`. I should be able to run clang on this object file one last time and have my application all ready to go. Right?
example.
Let's examine this error message a bit to figure out what I did wrong.
Still, the second part of the error message is more helpful to us since we still aren't all that familiar with assembly code. "undefined reference to `pow'" tells us that something went wrong with our new math function.
Wait a minute, didn't I say that we use clang -l? What's all this about -lm? Well, some libraries can be called using an abbreviation for the library. For the math library, the abbreviation is simply "m". So instead of running `clang hello.o -lmath` we run `clang hello.o -lm` and call it a day.
Success!
If we wanted to link a number of object files that we have created ourselves, we would specify each object file directly, like so:
So there are two distinct ways to execute linking using clang. One, we can use the "-l" flag. And two, we can simply call related object files from the command line as illustrated above. There is no practical difference in these two methodologies; but using -l can be more convenient when linking in C libraries that are distributed with the language, so that you can use abbreviated syntax to call the library as we did with "-lm".
Every application that you write in C will have to be compiled. Furthermore, compilation errors and failures will be your first indication that you have made a mistake in your program somewhere. Understanding your compiler in and out will help you to write code much more efficiently.
For the purposes of our tutorial today, we will be discussing the clang compiler. clang is widely used - iOS developers should recognize it as the compiler used for developing iPhone apps as part of xCode and Apple's LLVM. I will also use a number of demonstrations; these demonstrations will include source code written in C, assembler and some garbage ASCII that is representative of machine code viewed through a text editor. For my part, I am using a Fedora Linux virtual machine for these demonstrations. That said, as I discussed initially in the introduction to this series of C Programming tutorials, you can follow along using an operating system of your choice, and a compiler of your choice. For years I have used gcc as my compiler-suite-of-choice; I have opted for clang here because it is what is used for the Harvard CS classes that form the basis for this tutorial and many of the included materials. Whether you use clang, gcc or another option for your compiler, nearly all of the principles we discuss here today will remain applicable. You may need to alter the examples just a bit based on the documentation for your own compiler, but that is all. Using a different operating system, like Windows instead of Linux or Ubuntu instead of Fedora, should have little impact on how you follow along; however if you are using MinGW keep in mind that by default your compiler will be gcc and not clang.
If, though, you would like to use *exactly* the same environment I am using, I recommend downloading and installing a virtualization platform like VMWare Player and loading the Harvard Computer Science Appliance. This will give you a very pared down version of Fedora that has C and clang installed and ready to go. You may download the Appliance at no charge here. The same goes for VMWare Player, which is also free and available here.
From here on out, I will be assuming you have C and a compiler installed.
From here on out, I will be assuming you have C and a compiler installed.
This brings us to an important point that I must address before diving into this first tutorial. Source code and materials for this tutorial are based on Harvard University's CS50 as taught by David Malan. Reproductions of any such material are for purely academic and educational purposes only; I have profited in no way from their reproduction here in any form, particularly as blogspot.joshwieder.com has always and continues to decline the introduction of advertising to protect the privacy of my readers. Doctor Malan and Harvard University have been kind enough to offer these materials under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 Unported License, which readers can verify directly on the courses website here. Allowing for the free sharing and circulation of the course materials of one of the finest universities in the world at absolutely no charge to anyone is in keeping with the highest ideals and principles of education and open access. I can think of no higher praise for Harvard and David then what I am attempting to do here, on this site - spreading the information that they have so generously offered to me as a student.
With all that said, lets get to it, shall we?
COMPILERS
What does it mean to *compile* something? In the most general sense, it means transforming code written in one programming language into another. But usually when a developer refers to compiling their code, they mean they have converted their *higher level* programming language (for example, C) to a *lower level* programming language (for example, assembly and machine code).
We will be relying on the compiler "clang" for the purposes of this document. However, many of the core concepts described here apply to a variety of other compilers.
There are FOUR STAGES to the compilation process:
- Preprocessing (performed by the Preprocessor)
- Compilation (performed by the Compiler)
- Assembling (performed by the Assembler)
- Linking (performed by the Linker)
PREPROCESSING
Preprocessing can be assigned directives via the source code, using code preceded by a hash sign (#). For example:
#include <stdio.h>
#define NAME "Josh"
Commands preceded by a hash sign in the source code in this manner are referred to as a *Preprocessor Directive*. Two of the most common Preprocessor Directives are "include" and "define".
Using the clang flag "-E", you can compile using only the Preprocessor.
example.
$ clang -E hello.c
By default, the results of "clang -E" print to standard output (i.e. the monitor). However, the results can easily be saved to a new file:
example.
$ clang -E hello.c > hello2.c
In the example above, we have instructed clang to compile only the preprocessor directives in the file `hello.c`, and save the resulting output to a file named `hello2.c`.
Let us assume that in the example above, our file hello.c prepended the preprocessor directive `#include <stdio.h>`. In the resulting output of `hello2.c`, we would see the entire contents of stdio.h copied to the beginning of the file, replacing the #include directive. This is an example of what makes #include directives so useful. Instead of having to manually review all of the function declarations in the stdio.h header file that you need to use for a program, the preprocessor takes care of that chore for you, so long as you provide a single line declaring the #include statement.
COMPILATION
This second stage is where clang actually transforms C source code into assembly code. In order to have clang convert your C into assembly, but proceed no further, use the -S flag.
example.
$ clang -S hello2.c
Using `clang -S` will result in a *.s file (in the example above, hello2.s). The resulting assembly encoded language is processor-specific. In this tutorial, the code has been compiled on a Virtual Machine with a x86 vPU; as a result, the assembly code transformation is as follows (white space and carriage returns have been included as originally formatted with no additional indentation):
example.
application written in C.
-------------------------------------------------------
#include <stdio.h>
#define NAME "Josh"
int main(int argc, char *argv[]) {
printf("Hello, world! My name is %s!\n", NAME); } -------------------------------------------------------
example.
application written in x86 vPU specific assembly.
-------------------------------------------------------
.file "hello2.c"
.text
.globl main
.align 16, 0x90
.type main,@function main: # @main # BB#0:
pushl %ebp
movl %esp, %ebp
pushl %esi
subl $20, %esp
movl 12(%ebp), %eax
movl 8(%ebp), %ecx
leal .L.str, %edx
leal .L.str1, %esi
movl %ecx, -8(%ebp)
movl %eax, -12(%ebp)
movl %edx, (%esp)
movl %esi, 4(%esp)
calll printf
movl $0, %ecx
movl %eax, -16(%ebp) # 4-byte Spill
movl %ecx, %eax
addl $20, %esp
popl %esi
popl %ebp
ret .Ltmp0:
.size main, .Ltmp0-main
.type .L.str,@object # @.str
.section .rodata.str1.1,"aMS",@progbits,1 .L.str:
.asciz "Hello, world! My name is %s!\n"
.size .L.str, 30
.type .L.str1,@object # @.str1 .L.str1:
.asciz "Josh"
.size .L.str1, 5
.section ".note.GNU-stack","",@progbits -------------------------------------------------------
Very few modern developers write their applications in assembly. However, every application written in C is converted into assembly during the compile process.
Earlier, we described the *Compilation* process as transforming a higher level language into a lower level language; in this case, the higher level language C is transformed into the lower level language x86 vPU assembly. What makes assembly a "lower level" language than C? In assembly, we are very limited in what we can do. There are no loops of any kind. Yet, you can construct the same operations that loops and other un-included functions offer through the limited control structures that assembly does provide.
ASSEMBLING
It is the Assemblers job during this third stage of clang's compile process to convert the assembly code we just viewed and discussed into "machine code". To wit, the Assembler does not provide Assembly as output. Assembly code is provided to the Assembler as Input, and the Assembler provides *machine code* as Output.
*Machine Code* - the actual 1's and 0's that a CPU can understand
Using `clang -c` we can use the Assembler to parse code that has been translated into assembly. This will result in a file with the *.o extension. Note that if you have been following along, you will be performing this step to `hello2.s`, and not `hello2.c`.
example.
$ clang -c hello2.s
If we view the resulting file, hello2.o, using vi the output will resemble this:
$ vi hello2.s
^?ELF^A^A^A^C^@^@^@^@^@^@^@^@^A^@^C^@^A^@^@^@^@^@^@^@^@^@^@^@�^@^@^@^@^@^@^@4^@^@^@^@^@(^@
^@^G^@^@^@^@^@^@^@^@^@^@^@^@^@U~I�V~C�^T~KE^L~KM^H~M^U^@^@^@^@~M5^^^@^@^@~IM�~IE�~I^T$~It$^D������^@^@^@^@~IE�~I�~C�^T^]�^@Hello, world! My name is %s!
^@Josh^@^@.rel.text^@.bss^@.note.GNU-stack^@.shstrtab^@.strtab^@.symtab^@.data^@.rodata.str1.1^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^E^@^@^@^A^@^@^@^F^@^@^@^@^@^@^@@^@^@^@;^@^@^@^@^@^@^@^@^@^@^@^P^@^@^@^@^@^@^@^A^@^@^@ ^@^@^@^@^@^@^@^@^@^@^@(^C^@^@^X^@^@^@^H^@^@^@^A^@^@^@^D^@^@^@^H^@^@^@:^@^@^@^A^@^@^@^C^@^@^@^@^@^@^@|^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^D^@^@^@^@^@^@^@^K^@^@^@^H^@^@^@^C^@^@^@^@^@^@^@|^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^D^@^@^@^@^@^@^@@^@^@^@^A^@^@^@2^@^@^@^@^@^@^@|^@^@^@#^@^@^@^@^@^@^@^@^@^@^@^A^@^@^@^A^@^@^@^P^@^@^@^A^@^@^@^@^@^@^@^@^@^@^@~_^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^A^@^@^@^@^@^@^@ ^@^@^@^C^@^@^@^@^@^@^@^@^@^@^@~_^@^@^@O^@^@^@^@^@^@^@^@^@^@^@^A^@^@^@^@^@^@^@2^@^@^@^B^@^@^@^@^@^@^@^@^@^@^@~@^B^@^@~P^@^@^@ ^@^@^@^G^@^@^@^D^@^@^@^P^@^@^@*^@^@^@^C^@^@^@^@^@^@^@^@^@^@^@^P^C^@^@^V^@^@^@^@^@^@^@^@^@^@^@^A^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^A^@^@^@^@^@^@^@^@^@^@^@^D^@��^@^@^@^@^@^@^@^@^@^@^@^@^C^@^A^@^@^@^@^@^@^@^@^@^@^@^@^@^C^@^C^@^@^@^@^@^@^@^@^@^@^@^@^@^C^@^D^@^@^@^@^@^@^@^@^@^@^@^@^@^C^@^E^@^@^@^@^@^@^@^@^@^@^@^@^@^C^@^F^@
^@^@^@^@^@^@^@;^@^@^@^R^@^A^@^O^@^@^@^@^@^@^@^@^@^@^@^P^@^@^@^@hello2.c^@main^@printf^@^@^@^O^@^@^@^A^E^@^@^U^@^@^@^A^E^@^@'^@^@^@^B^H^@^@
~
~
-- VISUAL -- 4,80-139 All
-------------------------------------------------------
Note that the Assembler transforms assembly code into code that is readable to the CPU; but not easily readable to human beings.
Translating assembly into machine/object code is not a difficult task. Lines in the assembly code we reviewed, such as "pushl %ebp" correspond to simplistic hexadecimal values, which themselves are easily translated to binary. With a straight-forward translation chart, converting assembly into machine/object code is so easy a human could do it! This ease of process is part of the reason why Assembling is considered a distinct and different process than compiling.
LINKING
Linking is the fourth and final step of the clang compile process. Linking combines a multitude of object files into one big file that you can actually execute. Linking is very system-dependent, so the easiest way to link object files together is to call clang on all of the different files that you wish to link together. If you specify *.o files, than clang will know that it can skip Preprocessing, Compiling and Assembling and get straight to the Linking.
Let us return to the example application that we have been using throughout this tutorial; but this time, we will be making a few small changes to our source code. First, we will add an additional Preprocessor Directive to include the file <math.h>, which we will be linking to our application. math.h is a C library for math functions - it is often included with C installation, but unlike stdio.h, it is also often unlinked by default.
We don't just want to link math.h though - we want to make sure that we can actually use the math.h in our application! So, we will make our program perform a simple math problem and print the result to the screen along with the hello world statement.
#include <stdio.h> #include <math.h> //this is our new directive
#define NAME "Josh"
int main(int argc, char *argv[])
{
printf("Hello, world! My name is %s! 3 to the 3rd power is %f \n", NAME,
pow(3, 3)); //here we have included our math problem
}
Looking back at the beginning of our tutorial, we can see that my very first version of this application was called `hello.c`. I have made these changes above to this original `hello.c` file. I then skip ahead to using `clang -c` to parse my assembly application to machine code; but this time, I am using `hello.c` and not `hello2.s`.
example.
$ clang -c hello.c
As a result of this operation, I now have an object file `hello.o`. I should be able to run clang on this object file one last time and have my application all ready to go. Right?
example.
$ clang hello.o
hello.o: In function `main':
hello.c:(.text+0x27): undefined reference to `pow'
clang: error: linker command failed with exit code 1 (use -v to see invocation)
Let's examine this error message a bit to figure out what I did wrong.
The first part of the error reads: "In function `main'". Looking at my source code, I can tell that my program only has one function. So that doesn't help me at all!
Let's go to the next line: "(.text+0x27): undefined reference to `pow'". The first part, "(.text+0x27):", is pretty cryptic. We don't really know what that means yet. 0x27 looks like a hexadecimal character though - in decimal, 0x27 is represented by the number 39, while in binary it is 100111. If we look back at the assembly code of our first version of the hello application, we can also see that .text seems to be relevant to the assembly language:
.file "hello2.c"
.text
.globl main
Still, the second part of the error message is more helpful to us since we still aren't all that familiar with assembly code. "undefined reference to `pow'" tells us that something went wrong with our new math function.
The last line "error: linker command failed[...]" reminds us that clang was running the linker when the error was generated. If we put this together with the information we gathered from the previous line, we have a pretty good idea of what just happened - the compiler didn't know what to do with our math problem, `pow`, because we did something wrong involving the linker.
BINGO! That's it. I forgot to LINK IN the math.h library correctly. As we touched on before, some C libraries are linked in by default, like stdio.h, and others like math.h may be included with C but are not linked in. Adding the Preprocessor Directive copies the function definitions contained in the math.h header files, but it does nothing to tell the compiler where to locate the math librarie's object files. To do this, we use `clang -l` (FYI thats a lower-case "L"), like so:
example.
$ clang hello.o -lm
Wait a minute, didn't I say that we use clang -l? What's all this about -lm? Well, some libraries can be called using an abbreviation for the library. For the math library, the abbreviation is simply "m". So instead of running `clang hello.o -lmath` we run `clang hello.o -lm` and call it a day.
Successfully completing the link of the math library will finish the compilation process, leaving us with a newly created fully executable binary application named "a.out". I am using a Fedora Linux virtual machine, so to run my executable I use "./" - if you are using Windows, you should be running your executable using the Command Prompt (cmd.exe), or perhaps a shell included with Cygwin or MinGW if you went that route.
example.
$ ./a.out
Hello, world! My name is Josh! 3 to the 3rd power is 27.000000
Success!
If we wanted to link a number of object files that we have created ourselves, we would specify each object file directly, like so:
example.
$ clang hello.o hello2.o hello3.o [...]
So there are two distinct ways to execute linking using clang. One, we can use the "-l" flag. And two, we can simply call related object files from the command line as illustrated above. There is no practical difference in these two methodologies; but using -l can be more convenient when linking in C libraries that are distributed with the language, so that you can use abbreviated syntax to call the library as we did with "-lm".
There are a couple of important provisos here, however. Only ONE of these object files can specify a `main` function. The `main` function tells the compiler where to begin executing the code. If it exists in multiple places, compilation will fail, and rightfully so.
Also, "-l" flags must be parsed at the END of the clang command.
Which brings us to the end of today's tutorial. As always, if you have any questions or concerns, leave a comment below, reach me on Twitter or Google+, or find additional options on the Contact Page for this website.
Happy coding!
P.S. I can't stress enough that I deserve absolutely no credit for this tutorial - it is based almost *entirely* from my notes from Harvard. If you like what you see - give props to David Malan and the large team of students (course Heads, TFs, etc) responsible for producing the Intro CS program at Harvard.
P.S. I can't stress enough that I deserve absolutely no credit for this tutorial - it is based almost *entirely* from my notes from Harvard. If you like what you see - give props to David Malan and the large team of students (course Heads, TFs, etc) responsible for producing the Intro CS program at Harvard.