Darwin (Mac OS X) developers can simplify the installation of Expect and tcl -by using fink. fink install expect will install both. Alternatively, -Darwinports users can use sudo port install expect to install Expect -and tcl.
- @@ -122,14 +104,17 @@ tests" and are in the llvm module in subversion under theCode fragments are small pieces of code that test a specific feature of LLVM -or trigger a specific bug in LLVM. They are usually written in LLVM assembly -language, but can be written in other languages if the test targets a particular -language front end. These tests are driven by the DejaGNU testing framework, -which is hidden behind a few simple makefiles.
+Code fragments are small pieces of code that test a specific +feature of LLVM or trigger a specific bug in LLVM. They are usually +written in LLVM assembly language, but can be written in other +languages if the test targets a particular language front end (and the +appropriate --with-llvmgcc options were used +at configure time of the llvm module). These tests +are driven by the DejaGNU testing framework, which is hidden behind a +few simple makefiles.
-These code fragments are not complete programs. The code generated from them is -never executed to determine correct behavior.
+These code fragments are not complete programs. The code generated +from them is never executed to determine correct behavior.
These code fragment tests are located in the llvm/test directory.
@@ -251,10 +236,18 @@ programs), first checkout and setup the test-suite module: % cd .. % ./configure --with-llvmgccdir=$LLVM_GCC_DIR -where $LLVM_GCC_DIR is the directory where you installed -llvm-gcc, not it's src or obj dir.
where $LLVM_GCC_DIR is the directory where +you installed llvm-gcc, not it's src or obj +dir. The --with-llvmgccdir option assumes that +the llvm-gcc-4.2 module was configured with +--program-prefix=llvm-, and therefore that the C and C++ +compiler drivers are called llvm-gcc and llvm-g++ +respectively. If this is not the case, +use --with-llvmgcc/--with-llvmgxx to specify each +executable's location.
+Then, run the entire test suite by running make in the test-suite directory:
@@ -448,10 +441,162 @@ that subdirectory. +If your system includes GNU grep, make sure +that GREP_OPTIONS is not set in your environment. Otherwise, +you may get invalid results (both false positives and false +negatives).
+ + + + + + + +A powerful feature of the RUN: lines is that it allows any arbitrary commands + to be executed as part of the test harness. While standard (portable) unix + tools like 'grep' work fine on run lines, as you see above, there are a lot + of caveats due to interaction with Tcl syntax, and we want to make sure the + run lines are portable to a wide range of systems. Another major problem is + that grep is not very good at checking to verify that the output of a tools + contains a series of different output in a specific order. The FileCheck + tool was designed to help with these problems.
+ +FileCheck (whose basic command line arguments are described in the FileCheck man page is + designed to read a file to check from standard input, and the set of things + to verify from a file specified as a command line argument. A simple example + of using FileCheck from a RUN line looks like this:
+ ++; RUN: llvm-as < %s | llc -march=x86-64 | FileCheck %s ++
This syntax says to pipe the current file ("%s") into llvm-as, pipe that into +llc, then pipe the output of llc into FileCheck. This means that FileCheck will +be verifying its standard input (the llc output) against the filename argument +specified (the original .ll file specified by "%s"). To see how this works, +lets look at the rest of the .ll file (after the RUN line):
+ +
+define void @sub1(i32* %p, i32 %v) {
+entry:
+; CHECK: sub1:
+; CHECK: subl
+ %0 = tail call i32 @llvm.atomic.load.sub.i32.p0i32(i32* %p, i32 %v)
+ ret void
+}
+
+define void @inc4(i64* %p) {
+entry:
+; CHECK: inc4:
+; CHECK: incq
+ %0 = tail call i64 @llvm.atomic.load.add.i64.p0i64(i64* %p, i64 1)
+ ret void
+}
+
+Here you can see some "CHECK:" lines specified in comments. Now you can see +how the file is piped into llvm-as, then llc, and the machine code output is +what we are verifying. FileCheck checks the machine code output to verify that +it matches what the "CHECK:" lines specify.
+ +The syntax of the CHECK: lines is very simple: they are fixed strings that +must occur in order. FileCheck defaults to ignoring horizontal whitespace +differences (e.g. a space is allowed to match a tab) but otherwise, the contents +of the CHECK: line is required to match some thing in the test file exactly.
+ +One nice thing about FileCheck (compared to grep) is that it allows merging +test cases together into logical groups. For example, because the test above +is checking for the "sub1:" and "inc4:" labels, it will not match unless there +is a "subl" in between those labels. If it existed somewhere else in the file, +that would not count: "grep subl" matches if subl exists anywhere in the +file.
+ +The FileCheck -check-prefix option allows multiple test configurations to be +driven from one .ll file. This is useful in many circumstances, for example, +testing different architectural variants with llc. Here's a simple example:
+ +
+; RUN: llvm-as < %s | llc -mtriple=i686-apple-darwin9 -mattr=sse41 \
+; RUN: | FileCheck %s -check-prefix=X32
+; RUN: llvm-as < %s | llc -mtriple=x86_64-apple-darwin9 -mattr=sse41 \
+; RUN: | FileCheck %s -check-prefix=X64
+
+define <4 x i32> @pinsrd_1(i32 %s, <4 x i32> %tmp) nounwind {
+ %tmp1 = insertelement <4 x i32> %tmp, i32 %s, i32 1
+ ret <4 x i32> %tmp1
+; X32: pinsrd_1:
+; X32: pinsrd $1, 4(%esp), %xmm0
+
+; X64: pinsrd_1:
+; X64: pinsrd $1, %edi, %xmm0
+}
+
+In this case, we're testing that we get the expected code generation with +both 32-bit and 64-bit code generation.
+Sometimes you want to match lines and would like to verify that matches +happen on exactly consequtive lines with no other lines in between them. In +this case, you can use CHECK: and CHECK-NEXT: directives to specify this. If +you specified a custom check prefix, just use "<PREFIX>-NEXT:". For +example, something like this works as you'd expect:
+ +
+define void @t2(<2 x double>* %r, <2 x double>* %A, double %B) {
+ %tmp3 = load <2 x double>* %A, align 16
+ %tmp7 = insertelement <2 x double> undef, double %B, i32 0
+ %tmp9 = shufflevector <2 x double> %tmp3,
+ <2 x double> %tmp7,
+ <2 x i32> < i32 0, i32 2 >
+ store <2 x double> %tmp9, <2 x double>* %r, align 16
+ ret void
+
+; CHECK: t2:
+; CHECK: movl 8(%esp), %eax
+; CHECK-NEXT: movapd (%eax), %xmm0
+; CHECK-NEXT: movhpd 12(%esp), %xmm0
+; CHECK-NEXT: movl 4(%esp), %eax
+; CHECK-NEXT: movapd %xmm0, (%eax)
+; CHECK-NEXT: ret
+}
+
+CHECK-NEXT: directives reject the input unless there is exactly one newline +between it an the previous directive. A CHECK-NEXT cannot be the first +directive in a file.
+ +With a RUN line there are a number of substitutions that are permitted. In @@ -665,7 +810,7 @@ go here.
The External directory contains Makefiles for building code that is external to (i.e., not distributed with) LLVM. The most prominent members of this directory are the SPEC 95 and SPEC 2000 benchmark suites. The External -directory does not contain these actual tests,but only the Makefiles that know +directory does not contain these actual tests, but only the Makefiles that know how to properly compile these programs from somewhere else. The presence and location of these external programs is configured by the llvm-test configure script.
@@ -695,8 +840,7 @@ will help you separate benign warnings from actual test failures.First, all tests are executed within the LLVM object directory tree. They are not executed inside of the LLVM source tree. This is because the -test suite creates temporary files during execution. This means you must create -a build tree separate from the LLVM source tree in which to run the test suite.
+test suite creates temporary files during execution.To run the test suite, you need to use the following steps:
@@ -711,18 +855,20 @@ a build tree separate from the LLVM source tree in which to run the test suite.< % svn co http://llvm.org/svn/llvm-project/test-suite/trunk test-suiteThis will get the test suite into llvm/projects/test-suite.
-
[The Makefiles expect the test suite directory to be named either
- test-suite or llvm-test. To be safe, use
- test-suite as in the above svn command line.]
This will get the test suite into llvm/projects/test-suite.
Configure llvm from the top level of each build tree (LLVM object directory tree) - in which you want to run the test suite, just like what you do before building LLVM.
-When running configure, you must either: (1) have llvm-gcc - in your path, or (2) specify the directory where llvm-gcc is - installed using --with-llvmgccdir=$LLVM_GCC_DIR.
-This step tells the configure machinery that the test suite - is now available so it can be configured for your build tree:
+Configure and build llvm.
Configure and build llvm-gcc.
Install llvm-gcc somewhere.
Re-configure llvm from the top level of + each build tree (LLVM object directory tree) in which you want + to run the test suite, just as you do before building LLVM.
+During the re-configuration, you must either: (1) + have llvm-gcc you just built in your path, or (2) + specify the directory where your just-built llvm-gcc is + installed using --with-llvmgccdir=$LLVM_GCC_DIR.
+You must also tell the configure machinery that the test suite + is available so it can be configured for your build tree:
% cd $LLVM_OBJ_ROOT ; $LLVM_SRC_ROOT/configure [--with-llvmgccdir=$LLVM_GCC_DIR] @@ -745,14 +891,20 @@ a build tree separate from the LLVM source tree in which to run the test suite.< have the suite checked out and configured, you don't need to do it again (unless the test code or configure script changes). +
Note, when configuring the test-suite module, you might want to -specify the following configuration option:
+In order to run the External tests in the test-suite + module, you must specify --with-externals. This + must be done during the re-configuration step (see above), + and the llvm re-configuration must recognize the + previously-built llvm-gcc. If any of these is missing or + neglected, the External tests won't work.
In addition to the regular "whole program" tests, the test-suite +
In addition to the regular "whole program" tests, the test-suite module also provides a mechanism for compiling the programs in different ways. If the variable TEST is defined on the gmake command line, the test system will include a Makefile named TEST.<value of TEST variable>.Makefile.