============
In order to use the LLVM testing infrastructure, you will need all of the
-software required to build LLVM, as well as `Python <http://python.org>`_ 2.5 or
+software required to build LLVM, as well as `Python <http://python.org>`_ 2.7 or
later.
LLVM testing infrastructure organization
the :doc:`FileCheck tool <CommandGuide/FileCheck>`. *[The usage of grep in RUN
lines is deprecated - please do not send or commit patches that use it.]*
+Put related tests into a single file rather than having a separate file per
+test. Check if there are files already covering your feature and consider
+adding your code there instead of creating a new file.
+
+Extra files
+-----------
+
+If your test requires extra files besides the file containing the ``RUN:``
+lines, the idiomatic place to put them is in a subdirectory ``Inputs``.
+You can then refer to the extra files as ``%S/Inputs/foo.bar``.
+
+For example, consider ``test/Linker/ident.ll``. The directory structure is
+as follows::
+
+ test/
+ Linker/
+ ident.ll
+ Inputs/
+ ident.a.ll
+ ident.b.ll
+
+For convenience, these are the contents:
+
+.. code-block:: llvm
+
+ ;;;;; ident.ll:
+
+ ; RUN: llvm-link %S/Inputs/ident.a.ll %S/Inputs/ident.b.ll -S | FileCheck %s
+
+ ; Verify that multiple input llvm.ident metadata are linked together.
+
+ ; CHECK-DAG: !llvm.ident = !{!0, !1, !2}
+ ; CHECK-DAG: "Compiler V1"
+ ; CHECK-DAG: "Compiler V2"
+ ; CHECK-DAG: "Compiler V3"
+
+ ;;;;; Inputs/ident.a.ll:
+
+ !llvm.ident = !{!0, !1}
+ !0 = metadata !{metadata !"Compiler V1"}
+ !1 = metadata !{metadata !"Compiler V2"}
+
+ ;;;;; Inputs/ident.b.ll:
+
+ !llvm.ident = !{!0}
+ !0 = metadata !{metadata !"Compiler V3"}
+
+For symmetry reasons, ``ident.ll`` is just a dummy file that doesn't
+actually participate in the test besides holding the ``RUN:`` lines.
+
+.. note::
+
+ Some existing tests use ``RUN: true`` in extra files instead of just
+ putting the extra files in an ``Inputs/`` directory. This pattern is
+ deprecated.
+
Fragile tests
-------------
.. code-block:: python
config.suffixes = ['.ll', '.c', '.cpp', '.test']
- targets = set(config.root.targets_to_build.split())
- if not 'ARM' in targets:
+ if not 'ARM' in config.root.targets:
config.unsupported = True
Other platform-specific tests are those that depend on a specific feature
directory that will filter out all other architectures.
-Variables and substitutions
----------------------------
+Substitutions
+-------------
+
+Besides replacing LLVM tool names the following substitutions are performed in
+RUN lines:
-With a RUN line there are a number of substitutions that are permitted.
-To make a substitution just write the variable's name preceded by a ``$``.
-Additionally, for compatibility reasons with previous versions of the
-test library, certain names can be accessed with an alternate syntax: a
-% prefix. These alternates are deprecated and may go away in a future
-version.
+``%%``
+ Replaced by a single ``%``. This allows escaping other substitutions.
-Here are the available variable names. The alternate syntax is listed in
-parentheses.
+``%s``
+ File path to the test case's source. This is suitable for passing on the
+ command line as the input to an LLVM tool.
-``$test`` (``%s``)
- The full path to the test case's source. This is suitable for passing on
- the command line as the input to an LLVM tool.
+ Example: ``/home/user/llvm/test/MC/ELF/foo_test.s``
+
+``%S``
+ Directory path to the test case's source.
+
+ Example: ``/home/user/llvm/test/MC/ELF``
+
+``%t``
+ File path to a temporary file name that could be used for this test case.
+ The file name won't conflict with other test cases. You can append to it
+ if you need multiple temporaries. This is useful as the destination of
+ some redirected output.
+
+ Example: ``/home/user/llvm.build/test/MC/ELF/Output/foo_test.s.tmp``
+
+``%T``
+ Directory of ``%t``.
+
+ Example: ``/home/user/llvm.build/test/MC/ELF/Output``
+
+``%{pathsep}``
+
+ Expands to the path separator, i.e. ``:`` (or ``;`` on Windows).
+
+
+**LLVM-specific substitutions:**
+
+``%shlibext``
+ The suffix for the host platforms shared library files. This includes the
+ period as the first character.
+
+ Example: ``.so`` (Linux), ``.dylib`` (OS X), ``.dll`` (Windows)
+
+``%exeext``
+ The suffix for the host platforms executable files. This includes the
+ period as the first character.
+
+ Example: ``.exe`` (Windows), empty on Linux.
``%(line)``, ``%(line+<number>)``, ``%(line-<number>)``
- The number of the line where this variable is used, with an optional
- integer offset. This can be used in tests with multiple RUN lines,
- which reference test file's line numbers.
+ The number of the line where this substitution is used, with an optional
+ integer offset. This can be used in tests with multiple RUN lines, which
+ reference test file's line numbers.
-``$srcdir``
- The source directory from where the ``make check`` was run.
-``objdir``
- The object directory that corresponds to the ``$srcdir``.
+**Clang-specific substitutions:**
-``subdir``
- A partial path from the ``test`` directory that contains the
- sub-directory that contains the test source being executed.
+``%clang``
+ Invokes the Clang driver.
-``srcroot``
- The root directory of the LLVM src tree.
+``%clang_cpp``
+ Invokes the Clang driver for C++.
-``objroot``
- The root directory of the LLVM object tree. This could be the same as
- the srcroot.
+``%clang_cl``
+ Invokes the CL-compatible Clang driver.
-``path``
- The path to the directory that contains the test case source. This is
- for locating any supporting files that are not generated by the test,
- but used by the test.
+``%clangxx``
+ Invokes the G++-compatible Clang driver.
-``tmp``
- The path to a temporary file name that could be used for this test case.
- The file name won't conflict with other test cases. You can append to it
- if you need multiple temporaries. This is useful as the destination of
- some redirected output.
+``%clang_cc1``
+ Invokes the Clang frontend.
+
+``%itanium_abi_triple``, ``%ms_abi_triple``
+ These substitutions can be used to get the current target triple adjusted to
+ the desired ABI. For example, if the test suite is running with the
+ ``i686-pc-win32`` target, ``%itanium_abi_triple`` will expand to
+ ``i686-pc-mingw32``. This allows a test to run with a specific ABI without
+ constraining it to a specific triple.
-``target_triplet`` (``%target_triplet``)
- The target triplet that corresponds to the current host machine (the one
- running the test cases). This should probably be called "host".
+To add more substituations, look at ``test/lit.cfg`` or ``lit.local.cfg``.
-``link`` (``%link``)
- This full link command used to link LLVM executables. This has all the
- configured ``-I``, ``-L`` and ``-l`` options.
-``shlibext`` (``%shlibext``)
- The suffix for the host platforms shared library (DLL) files. This
- includes the period as the first character.
+Options
+-------
+
+The llvm lit configuration allows to customize some things with user options:
+
+``llc``, ``opt``, ...
+ Substitute the respective llvm tool name with a custom command line. This
+ allows to specify custom paths and default arguments for these tools.
+ Example:
+
+ % llvm-lit "-Dllc=llc -verify-machineinstrs"
+
+``run_long_tests``
+ Enable the execution of long running tests.
+
+``llvm_site_config``
+ Load the specified lit configuration instead of the default one.
-To add more variables, look at ``test/lit.cfg``.
Other Features
--------------
-To make RUN line writing easier, there are several helper scripts and programs
-in the ``llvm/test/Scripts`` directory. This directory is in the PATH
-when running tests, so you can just call these scripts using their name.
-For example:
-
-``ignore``
- This script runs its arguments and then always returns 0. This is useful
- in cases where the test needs to cause a tool to generate an error (e.g.
- to check the error output). However, any program in a pipeline that
- returns a non-zero result will cause the test to fail. This script
- overcomes that issue and nicely documents that the test case is
- purposefully ignoring the result code of the tool
+To make RUN line writing easier, there are several helper programs. These
+helpers are in the PATH when running tests, so you can just call them using
+their name. For example:
+
``not``
- This script runs its arguments and then inverts the result code from it.
+ This program runs its arguments and then inverts the result code from it.
Zero result codes become 1. Non-zero result codes become 0.
Sometimes it is necessary to mark a test case as "expected fail" or
projects / oota-llvm.git / blobdiff