1 ===================================
2 Customizing LLVMC: Reference Manual
3 ===================================
5 This file was automatically generated by rst2html.
6 Please do not edit directly!
7 The ReST source lives in the directory 'tools/llvmc/doc'.
13 <div class="doc_author">
14 <p>Written by <a href="mailto:foldr@codedgers.com">Mikhail Glushenkov</a></p>
20 LLVMC is a generic compiler driver, designed to be customizable and
21 extensible. It plays the same role for LLVM as the ``gcc`` program
22 does for GCC - LLVMC's job is essentially to transform a set of input
23 files into a set of targets depending on configuration rules and user
24 options. What makes LLVMC different is that these transformation rules
25 are completely customizable - in fact, LLVMC knows nothing about the
26 specifics of transformation (even the command-line options are mostly
27 not hard-coded) and regards the transformation structure as an
28 abstract graph. The structure of this graph is completely determined
29 by plugins, which can be either statically or dynamically linked. This
30 makes it possible to easily adapt LLVMC for other purposes - for
31 example, as a build tool for game resources.
33 Because LLVMC employs TableGen_ as its configuration language, you
34 need to be familiar with it to customize LLVMC.
36 .. _TableGen: http://llvm.cs.uiuc.edu/docs/TableGenFundamentals.html
42 LLVMC tries hard to be as compatible with ``gcc`` as possible,
43 although there are some small differences. Most of the time, however,
44 you shouldn't be able to notice them::
46 $ # This works as expected:
47 $ llvmc -O3 -Wall hello.cpp
51 One nice feature of LLVMC is that one doesn't have to distinguish
52 between different compilers for different languages (think ``g++`` and
53 ``gcc``) - the right toolchain is chosen automatically based on input
54 language names (which are, in turn, determined from file
55 extensions). If you want to force files ending with ".c" to compile as
56 C++, use the ``-x`` option, just like you would do it with ``gcc``::
58 $ # hello.c is really a C++ file
59 $ llvmc -x c++ hello.c
63 On the other hand, when using LLVMC as a linker to combine several C++
64 object files you should provide the ``--linker`` option since it's
65 impossible for LLVMC to choose the right linker in that case::
69 [A lot of link-time errors skipped]
70 $ llvmc --linker=c++ hello.o
74 By default, LLVMC uses ``llvm-gcc`` to compile the source code. It is
75 also possible to choose the work-in-progress ``clang`` compiler with
76 the ``-clang`` option.
82 LLVMC has some built-in options that can't be overridden in the
83 configuration libraries:
85 * ``-o FILE`` - Output file name.
87 * ``-x LANGUAGE`` - Specify the language of the following input files
88 until the next -x option.
90 * ``-load PLUGIN_NAME`` - Load the specified plugin DLL. Example:
91 ``-load $LLVM_DIR/Release/lib/LLVMCSimple.so``.
93 * ``-v`` - Enable verbose mode, i.e. print out all executed commands.
95 * ``--check-graph`` - Check the compilation for common errors like
96 mismatched output/input language names, multiple default edges and
97 cycles. Hidden option, useful for debugging.
99 * ``--view-graph`` - Show a graphical representation of the compilation
100 graph. Requires that you have ``dot`` and ``gv`` programs
101 installed. Hidden option, useful for debugging.
103 * ``--write-graph`` - Write a ``compilation-graph.dot`` file in the
104 current directory with the compilation graph description in the
105 Graphviz format. Hidden option, useful for debugging.
107 * ``--save-temps`` - Write temporary files to the current directory
108 and do not delete them on exit. Hidden option, useful for debugging.
110 * ``--help``, ``--help-hidden``, ``--version`` - These options have
111 their standard meaning.
114 Compiling LLVMC plugins
115 =======================
117 It's easiest to start working on your own LLVMC plugin by copying the
118 skeleton project which lives under ``$LLVMC_DIR/plugins/Simple``::
120 $ cd $LLVMC_DIR/plugins
121 $ cp -r Simple MyPlugin
124 Makefile PluginMain.cpp Simple.td
126 As you can see, our basic plugin consists of only two files (not
127 counting the build script). ``Simple.td`` contains TableGen
128 description of the compilation graph; its format is documented in the
129 following sections. ``PluginMain.cpp`` is just a helper file used to
130 compile the auto-generated C++ code produced from TableGen source. It
131 can also contain hook definitions (see `below`__).
135 The first thing that you should do is to change the ``LLVMC_PLUGIN``
136 variable in the ``Makefile`` to avoid conflicts (since this variable
137 is used to name the resulting library)::
139 LLVMC_PLUGIN=MyPlugin
141 It is also a good idea to rename ``Simple.td`` to something less
144 $ mv Simple.td MyPlugin.td
146 Note that the plugin source directory must be placed under
147 ``$LLVMC_DIR/plugins`` to make use of the existing build
148 infrastructure. To build a version of the LLVMC executable called
149 ``mydriver`` with your plugin compiled in, use the following command::
152 $ make BUILTIN_PLUGINS=MyPlugin DRIVER_NAME=mydriver
154 To build your plugin as a dynamic library, just ``cd`` to its source
155 directory and run ``make``. The resulting file will be called
156 ``LLVMC$(LLVMC_PLUGIN).$(DLL_EXTENSION)`` (in our case,
157 ``LLVMCMyPlugin.so``). This library can be then loaded in with the
158 ``-load`` option. Example::
160 $ cd $LLVMC_DIR/plugins/Simple
162 $ llvmc -load $LLVM_DIR/Release/lib/LLVMCSimple.so
164 Sometimes, you will want a 'bare-bones' version of LLVMC that has no
165 built-in plugins. It can be compiled with the following command::
168 $ make BUILTIN_PLUGINS=""
171 Customizing LLVMC: the compilation graph
172 ========================================
174 Each TableGen configuration file should include the common
177 include "llvm/CompilerDriver/Common.td"
179 Internally, LLVMC stores information about possible source
180 transformations in form of a graph. Nodes in this graph represent
181 tools, and edges between two nodes represent a transformation path. A
182 special "root" node is used to mark entry points for the
183 transformations. LLVMC also assigns a weight to each edge (more on
184 this later) to choose between several alternative edges.
186 The definition of the compilation graph (see file
187 ``plugins/Base/Base.td`` for an example) is just a list of edges::
189 def CompilationGraph : CompilationGraph<[
190 Edge<"root", "llvm_gcc_c">,
191 Edge<"root", "llvm_gcc_assembler">,
194 Edge<"llvm_gcc_c", "llc">,
195 Edge<"llvm_gcc_cpp", "llc">,
198 OptionalEdge<"llvm_gcc_c", "opt", (case (switch_on "opt"),
200 OptionalEdge<"llvm_gcc_cpp", "opt", (case (switch_on "opt"),
204 OptionalEdge<"llvm_gcc_assembler", "llvm_gcc_cpp_linker",
205 (case (input_languages_contain "c++"), (inc_weight),
206 (or (parameter_equals "linker", "g++"),
207 (parameter_equals "linker", "c++")), (inc_weight))>,
212 As you can see, the edges can be either default or optional, where
213 optional edges are differentiated by an additional ``case`` expression
214 used to calculate the weight of this edge. Notice also that we refer
215 to tools via their names (as strings). This makes it possible to add
216 edges to an existing compilation graph in plugins without having to
217 know about all tool definitions used in the graph.
219 The default edges are assigned a weight of 1, and optional edges get a
220 weight of 0 + 2*N where N is the number of tests that evaluated to
221 true in the ``case`` expression. It is also possible to provide an
222 integer parameter to ``inc_weight`` and ``dec_weight`` - in this case,
223 the weight is increased (or decreased) by the provided value instead
224 of the default 2. It is also possible to change the default weight of
225 an optional edge by using the ``default`` clause of the ``case``
228 When passing an input file through the graph, LLVMC picks the edge
229 with the maximum weight. To avoid ambiguity, there should be only one
230 default edge between two nodes (with the exception of the root node,
231 which gets a special treatment - there you are allowed to specify one
232 default edge *per language*).
234 When multiple plugins are loaded, their compilation graphs are merged
235 together. Since multiple edges that have the same end nodes are not
236 allowed (i.e. the graph is not a multigraph), an edge defined in
237 several plugins will be replaced by the definition from the plugin
238 that was loaded last. Plugin load order can be controlled by using the
239 plugin priority feature described above.
241 To get a visual representation of the compilation graph (useful for
242 debugging), run ``llvmc --view-graph``. You will need ``dot`` and
243 ``gsview`` installed for this to work properly.
248 Command-line options that the plugin supports are defined by using an
251 def Options : OptionList<[
252 (switch_option "E", (help "Help string")),
253 (alias_option "quiet", "q")
257 As you can see, the option list is just a list of DAGs, where each DAG
258 is an option description consisting of the option name and some
259 properties. A plugin can define more than one option list (they are
260 all merged together in the end), which can be handy if one wants to
261 separate option groups syntactically.
263 * Possible option types:
265 - ``switch_option`` - a simple boolean switch without arguments,
266 for example ``-O2`` or ``-time``.
268 - ``parameter_option`` - option that takes one argument, for
269 example ``-std=c99``. It is also allowed to use spaces instead of
270 the equality sign: ``-std c99``.
272 - ``parameter_list_option`` - same as the above, but more than one
273 option occurence is allowed.
275 - ``prefix_option`` - same as the parameter_option, but the option
276 name and argument do not have to be separated. Example:
277 ``-ofile``. This can be also specified as ``-o file``; however,
278 ``-o=file`` will be parsed incorrectly (``=file`` will be
279 interpreted as option value).
281 - ``prefix_list_option`` - same as the above, but more than one
282 occurence of the option is allowed; example: ``-lm -lpthread``.
284 - ``alias_option`` - a special option type for creating
285 aliases. Unlike other option types, aliases are not allowed to
286 have any properties besides the aliased option name. Usage
287 example: ``(alias_option "preprocess", "E")``
290 * Possible option properties:
292 - ``help`` - help string associated with this option. Used for
295 - ``required`` - this option is obligatory.
297 - ``hidden`` - the description of this option will not appear in
298 the ``--help`` output (but will appear in the ``--help-hidden``
301 - ``really_hidden`` - the option will not be mentioned in any help
304 - ``extern`` - this option is defined in some other plugin, see below.
309 Sometimes, when linking several plugins together, one plugin needs to
310 access options defined in some other plugin. Because of the way
311 options are implemented, such options must be marked as
312 ``extern``. This is what the ``extern`` option property is
316 (switch_option "E", (extern))
319 See also the section on plugin `priorities`__.
325 Conditional evaluation
326 ======================
328 The 'case' construct is the main means by which programmability is
329 achieved in LLVMC. It can be used to calculate edge weights, program
330 actions and modify the shell commands to be executed. The 'case'
331 expression is designed after the similarly-named construct in
332 functional languages and takes the form ``(case (test_1), statement_1,
333 (test_2), statement_2, ... (test_N), statement_N)``. The statements
334 are evaluated only if the corresponding tests evaluate to true.
338 // Edge weight calculation
340 // Increases edge weight by 5 if "-A" is provided on the
341 // command-line, and by 5 more if "-B" is also provided.
343 (switch_on "A"), (inc_weight 5),
344 (switch_on "B"), (inc_weight 5))
347 // Tool command line specification
349 // Evaluates to "cmdline1" if the option "-A" is provided on the
350 // command line; to "cmdline2" if "-B" is provided;
351 // otherwise to "cmdline3".
354 (switch_on "A"), "cmdline1",
355 (switch_on "B"), "cmdline2",
356 (default), "cmdline3")
358 Note the slight difference in 'case' expression handling in contexts
359 of edge weights and command line specification - in the second example
360 the value of the ``"B"`` switch is never checked when switch ``"A"`` is
361 enabled, and the whole expression always evaluates to ``"cmdline1"`` in
364 Case expressions can also be nested, i.e. the following is legal::
366 (case (switch_on "E"), (case (switch_on "o"), ..., (default), ...)
369 You should, however, try to avoid doing that because it hurts
370 readability. It is usually better to split tool descriptions and/or
371 use TableGen inheritance instead.
373 * Possible tests are:
375 - ``switch_on`` - Returns true if a given command-line switch is
376 provided by the user. Example: ``(switch_on "opt")``.
378 - ``parameter_equals`` - Returns true if a command-line parameter equals
380 Example: ``(parameter_equals "W", "all")``.
382 - ``element_in_list`` - Returns true if a command-line parameter
383 list contains a given value.
384 Example: ``(parameter_in_list "l", "pthread")``.
386 - ``input_languages_contain`` - Returns true if a given language
387 belongs to the current input language set.
388 Example: ``(input_languages_contain "c++")``.
390 - ``in_language`` - Evaluates to true if the input file language
391 equals to the argument. At the moment works only with ``cmd_line``
392 and ``actions`` (on non-join nodes).
393 Example: ``(in_language "c++")``.
395 - ``not_empty`` - Returns true if a given option (which should be
396 either a parameter or a parameter list) is set by the
398 Example: ``(not_empty "o")``.
400 - ``empty`` - The opposite of ``not_empty``. Equivalent to ``(not (not_empty
401 X))``. Provided for convenience.
403 - ``default`` - Always evaluates to true. Should always be the last
404 test in the ``case`` expression.
406 - ``and`` - A standard logical combinator that returns true iff all
407 of its arguments return true. Used like this: ``(and (test1),
408 (test2), ... (testN))``. Nesting of ``and`` and ``or`` is allowed,
411 - ``or`` - Another logical combinator that returns true only if any
412 one of its arguments returns true. Example: ``(or (test1),
413 (test2), ... (testN))``.
416 Writing a tool description
417 ==========================
419 As was said earlier, nodes in the compilation graph represent tools,
420 which are described separately. A tool definition looks like this
421 (taken from the ``include/llvm/CompilerDriver/Tools.td`` file)::
423 def llvm_gcc_cpp : Tool<[
425 (out_language "llvm-assembler"),
426 (output_suffix "bc"),
427 (cmd_line "llvm-g++ -c $INFILE -o $OUTFILE -emit-llvm"),
431 This defines a new tool called ``llvm_gcc_cpp``, which is an alias for
432 ``llvm-g++``. As you can see, a tool definition is just a list of
433 properties; most of them should be self-explanatory. The ``sink``
434 property means that this tool should be passed all command-line
435 options that aren't mentioned in the option list.
437 The complete list of all currently implemented tool properties follows.
439 * Possible tool properties:
441 - ``in_language`` - input language name. Can be either a string or a
442 list, in case the tool supports multiple input languages.
444 - ``out_language`` - output language name. Tools are not allowed to
445 have multiple output languages.
447 - ``output_suffix`` - output file suffix. Can also be changed
448 dynamically, see documentation on actions.
450 - ``cmd_line`` - the actual command used to run the tool. You can
451 use ``$INFILE`` and ``$OUTFILE`` variables, output redirection
452 with ``>``, hook invocations (``$CALL``), environment variables
453 (via ``$ENV``) and the ``case`` construct.
455 - ``join`` - this tool is a "join node" in the graph, i.e. it gets a
456 list of input files and joins them together. Used for linkers.
458 - ``sink`` - all command-line options that are not handled by other
459 tools are passed to this tool.
461 - ``actions`` - A single big ``case`` expression that specifies how
462 this tool reacts on command-line options (described in more detail
468 A tool often needs to react to command-line options, and this is
469 precisely what the ``actions`` property is for. The next example
470 illustrates this feature::
472 def llvm_gcc_linker : Tool<[
473 (in_language "object-code"),
474 (out_language "executable"),
475 (output_suffix "out"),
476 (cmd_line "llvm-gcc $INFILE -o $OUTFILE"),
478 (actions (case (not_empty "L"), (forward "L"),
479 (not_empty "l"), (forward "l"),
481 [(append_cmd "-dummy1"), (append_cmd "-dummy2")])
484 The ``actions`` tool property is implemented on top of the omnipresent
485 ``case`` expression. It associates one or more different *actions*
486 with given conditions - in the example, the actions are ``forward``,
487 which forwards a given option unchanged, and ``append_cmd``, which
488 appends a given string to the tool execution command. Multiple actions
489 can be associated with a single condition by using a list of actions
490 (used in the example to append some dummy options). The same ``case``
491 construct can also be used in the ``cmd_line`` property to modify the
494 The "join" property used in the example means that this tool behaves
497 The list of all possible actions follows.
501 - ``append_cmd`` - append a string to the tool invocation
503 Example: ``(case (switch_on "pthread"), (append_cmd
506 - ``error` - exit with error.
507 Example: ``(error "Mixing -c and -S is not allowed!")``.
509 - ``forward`` - forward an option unchanged.
510 Example: ``(forward "Wall")``.
512 - ``forward_as`` - Change the name of an option, but forward the
514 Example: ``(forward_as "O0" "--disable-optimization")``.
516 - ``output_suffix`` - modify the output suffix of this
518 Example: ``(output_suffix "i")``.
520 - ``stop_compilation`` - stop compilation after this tool processes
521 its input. Used without arguments.
523 - ``unpack_values`` - used for for splitting and forwarding
524 comma-separated lists of options, e.g. ``-Wa,-foo=bar,-baz`` is
525 converted to ``-foo=bar -baz`` and appended to the tool invocation
527 Example: ``(unpack_values "Wa,")``.
532 If you are adding support for a new language to LLVMC, you'll need to
533 modify the language map, which defines mappings from file extensions
534 to language names. It is used to choose the proper toolchain(s) for a
535 given input file set. Language map definition looks like this::
537 def LanguageMap : LanguageMap<
538 [LangToSuffixes<"c++", ["cc", "cp", "cxx", "cpp", "CPP", "c++", "C"]>,
539 LangToSuffixes<"c", ["c"]>,
543 For example, without those definitions the following command wouldn't work::
546 llvmc: Unknown suffix: cpp
548 The language map entries should be added only for tools that are
549 linked with the root node. Since tools are not allowed to have
550 multiple output languages, for nodes "inside" the graph the input and
551 output languages should match. This is enforced at compile-time.
559 Hooks and environment variables
560 -------------------------------
562 Normally, LLVMC executes programs from the system ``PATH``. Sometimes,
563 this is not sufficient: for example, we may want to specify tool paths
564 or names in the configuration file. This can be easily achieved via
565 the hooks mechanism. To write your own hooks, just add their
566 definitions to the ``PluginMain.cpp`` or drop a ``.cpp`` file into the
567 your plugin directory. Hooks should live in the ``hooks`` namespace
568 and have the signature ``std::string hooks::MyHookName ([const char*
569 Arg0 [ const char* Arg2 [, ...]]])``. They can be used from the
570 ``cmd_line`` tool property::
572 (cmd_line "$CALL(MyHook)/path/to/file -o $CALL(AnotherHook)")
574 To pass arguments to hooks, use the following syntax::
576 (cmd_line "$CALL(MyHook, 'Arg1', 'Arg2', 'Arg # 3')/path/to/file -o1 -o2")
578 It is also possible to use environment variables in the same manner::
580 (cmd_line "$ENV(VAR1)/path/to/file -o $ENV(VAR2)")
582 To change the command line string based on user-provided options use
583 the ``case`` expression (documented `above`__)::
588 "llvm-g++ -E -x c $INFILE -o $OUTFILE",
590 "llvm-g++ -c -x c $INFILE -o $OUTFILE -emit-llvm"))
596 How plugins are loaded
597 ----------------------
599 It is possible for LLVMC plugins to depend on each other. For example,
600 one can create edges between nodes defined in some other plugin. To
601 make this work, however, that plugin should be loaded first. To
602 achieve this, the concept of plugin priority was introduced. By
603 default, every plugin has priority zero; to specify the priority
604 explicitly, put the following line in your plugin's TableGen file::
606 def Priority : PluginPriority<$PRIORITY_VALUE>;
607 # Where PRIORITY_VALUE is some integer > 0
609 Plugins are loaded in order of their (increasing) priority, starting
610 with 0. Therefore, the plugin with the highest priority value will be
616 When writing LLVMC plugins, it can be useful to get a visual view of
617 the resulting compilation graph. This can be achieved via the command
618 line option ``--view-graph``. This command assumes that Graphviz_ and
619 Ghostview_ are installed. There is also a ``--dump-graph`` option that
620 creates a Graphviz source file (``compilation-graph.dot``) in the
623 Another useful ``llvmc`` option is ``--check-graph``. It checks the
624 compilation graph for common errors like mismatched output/input
625 language names, multiple default edges and cycles. These checks can't
626 be performed at compile-time because the plugins can load code
627 dynamically. When invoked with ``--check-graph``, ``llvmc`` doesn't
628 perform any compilation tasks and returns the number of encountered
629 errors as its status code.
631 .. _Graphviz: http://www.graphviz.org/
632 .. _Ghostview: http://pages.cs.wisc.edu/~ghost/
638 <a href="http://jigsaw.w3.org/css-validator/check/referer">
639 <img src="http://jigsaw.w3.org/css-validator/images/vcss-blue"
640 alt="Valid CSS" /></a>
641 <a href="http://validator.w3.org/check?uri=referer">
642 <img src="http://www.w3.org/Icons/valid-xhtml10-blue"
643 alt="Valid XHTML 1.0 Transitional"/></a>
645 <a href="mailto:foldr@codedgers.com">Mikhail Glushenkov</a><br />
646 <a href="http://llvm.org">LLVM Compiler Infrastructure</a><br />
648 Last modified: $Date: 2008-12-11 11:34:48 -0600 (Thu, 11 Dec 2008) $