<li><a href="#testsuitestructure">Test suite structure</a></li>
<li><a href="#testsuiterun">Running the test suite</a>
<ul>
- <li><a href="#testsuiteexternal">Configuring external tests</a></li>
+ <li><a href="#testsuiteexternal">Configuring External Tests</a></li>
<li><a href="#testsuitetests">Running different tests</a></li>
<li><a href="#testsuiteoutput">Generating test output</a></li>
<li><a href="#testsuitecustom">Writing custom tests for llvm-test</a></li>
<p>The External directory contains Makefiles for building code that is external
to (i.e., not distributed with) LLVM. The most prominent members of this
directory are the SPEC 95 and SPEC 2000 benchmark suites. The <tt>External</tt>
-directory does not contain these actual tests,but only the Makefiles that know
+directory does not contain these actual tests, but only the Makefiles that know
how to properly compile these programs from somewhere else. The presence and
location of these external programs is configured by the llvm-test
<tt>configure</tt> script.</p></li>
<p>To run the test suite, you need to use the following steps:</p>
<ol>
- <li><tt>cd</tt> into the <tt>llvm/projects</tt> directory</li>
+ <li><tt>cd</tt> into the <tt>llvm/projects</tt> directory in your source tree.
+ </li>
<li><p>Check out the <tt>test-suite</tt> module with:</p>
% svn co http://llvm.org/svn/llvm-project/test-suite/trunk test-suite
</pre>
</div>
-
- <p>This will get the test suite into <tt>llvm/projects/llvm-test</tt></p>
-
- <li><p>Configure the test suite using llvm configure. This will automatically
- configure test-suite. You must do it from the top level otherwise llvm-gcc
- will not be set which is required to run llvm-test:</p>
+ <p>This will get the test suite into <tt>llvm/projects/test-suite</tt>.</p>
+ </li>
+ <li><p>Configure and build <tt>llvm</tt>.</p></li>
+ <li><p>Configure and build <tt>llvm-gcc</tt>.</p></li>
+ <li><p>Install <tt>llvm-gcc</tt> somewhere.</p></li>
+ <li><p><em>Re-configure</em> <tt>llvm</tt> from the top level of
+ each build tree (LLVM object directory tree) in which you want
+ to run the test suite, just as you do before building LLVM.</p>
+ <p>During the <em>re-configuration</em>, you must either: (1)
+ have <tt>llvm-gcc</tt> you just built in your path, or (2)
+ specify the directory where your just-built <tt>llvm-gcc</tt> is
+ installed using <tt>--with-llvmgccdir=$LLVM_GCC_DIR</tt>.</p>
+ <p>You must also tell the configure machinery that the test suite
+ is available so it can be configured for your build tree:</p>
<div class="doc_code">
<pre>
-% cd $LLVM_OBJ_ROOT ; $LLVM_SRC_ROOT/configure --with-llvmgccdir=$LLVM_GCC_DIR
+% cd $LLVM_OBJ_ROOT ; $LLVM_SRC_ROOT/configure [--with-llvmgccdir=$LLVM_GCC_DIR]
</pre>
</div>
- <p>Note that that <tt>$LLVM_GCC_DIR</tt> is the directory where you
- <em>installed</em> llvm-gcc, not its src or obj directory.</p>
+ <p>[Remember that <tt>$LLVM_GCC_DIR</tt> is the directory where you
+ <em>installed</em> llvm-gcc, not its src or obj directory.]</p>
</li>
- <li><p>Change back to the <tt>llvm/projects/test-suite</tt> directory you created before
- and run <tt>gmake</tt> (or just "<tt>make</tt>" on systems where GNU make is
- the default, such as linux.</p></li>
+ <li><p>You can now run the test suite from your build tree as follows:</p>
+<div class="doc_code">
+<pre>
+% cd $LLVM_OBJ_ROOT/projects/test-suite
+% make
+</pre>
+</div>
+ </li>
</ol>
<p>Note that the second and third steps only need to be done once. After you
have the suite checked out and configured, you don't need to do it again (unless
<!-- _______________________________________________________________________ -->
<div class="doc_subsection">
-<a name="testsuiteexternal">Configuring external tests</a></div>
+<a name="testsuiteexternal">Configuring External Tests</a></div>
<!-- _______________________________________________________________________ -->
<div class="doc_text">
-<p>Note, when configuring the <tt>test-suite</tt> module, you might want to
-specify the following configuration options:</p>
+<p>In order to run the External tests in the <tt>test-suite</tt>
+ module, you must specify <i>--with-externals</i>. This
+ must be done during the <em>re-configuration</em> step (see above),
+ and the <tt>llvm</tt> re-configuration must recognize the
+ previously-built <tt>llvm-gcc</tt>. If any of these is missing or
+ neglected, the External tests won't work.</p>
<dl>
- <dt><i>--enable-spec2000</i>
- <dt><i>--enable-spec2000=<<tt>directory</tt>></i>
- <dd>
- Enable the use of SPEC2000 when testing LLVM. This is disabled by default
- (unless <tt>configure</tt> finds SPEC2000 installed). By specifying
- <tt>directory</tt>, you can tell configure where to find the SPEC2000
- benchmarks. If <tt>directory</tt> is left unspecified, <tt>configure</tt>
- uses the default value
- <tt>/home/vadve/shared/benchmarks/speccpu2000/benchspec</tt>.
- <p>
-
- <dt><i>--enable-spec95</i>
- <dt><i>--enable-spec95=<<tt>directory</tt>></i>
- <dd>
- Enable the use of SPEC95 when testing LLVM. It is similar to the
- <i>--enable-spec2000</i> option.
- <p>
-
- <dt><i>--enable-povray</i>
- <dt><i>--enable-povray=<<tt>directory</tt>></i>
- <dd>
- Enable the use of Povray as an external test. Versions of Povray written
- in C should work. This option is similar to the <i>--enable-spec2000</i>
- option.
+<dt><i>--with-externals</i></dt>
+<dt><i>--with-externals=<<tt>directory</tt>></i></dt>
</dl>
+ This tells LLVM where to find any external tests. They are expected to be
+ in specifically named subdirectories of <<tt>directory</tt>>.
+ If <tt>directory</tt> is left unspecified,
+ <tt>configure</tt> uses the default value
+ <tt>/home/vadve/shared/benchmarks/speccpu2000/benchspec</tt>.
+ Subdirectory names known to LLVM include:
+ <dl>
+ <dt>spec95</dt>
+ <dt>speccpu2000</dt>
+ <dt>speccpu2006</dt>
+ <dt>povray31</dt>
+ </dl>
+ Others are added from time to time, and can be determined from
+ <tt>configure</tt>.
</div>
<!-- _______________________________________________________________________ -->
<a name="testsuitetests">Running different tests</a></div>
<!-- _______________________________________________________________________ -->
<div class="doc_text">
-<p>In addition to the regular "whole program" tests, the <tt>test-suite</tt>
+<p>In addition to the regular "whole program" tests, the <tt>test-suite</tt>
module also provides a mechanism for compiling the programs in different ways.
If the variable TEST is defined on the <tt>gmake</tt> command line, the test system will
include a Makefile named <tt>TEST.<value of TEST variable>.Makefile</tt>.
<hr>
<address>
<a href="http://jigsaw.w3.org/css-validator/check/referer"><img
- src="http://jigsaw.w3.org/css-validator/images/vcss" alt="Valid CSS!"></a>
+ src="http://jigsaw.w3.org/css-validator/images/vcss-blue" alt="Valid CSS"></a>
<a href="http://validator.w3.org/check/referer"><img
- src="http://www.w3.org/Icons/valid-html401" alt="Valid HTML 4.01!"></a>
+ src="http://www.w3.org/Icons/valid-html401-blue" alt="Valid HTML 4.01"></a>
John T. Criswell, Reid Spencer, and Tanya Lattner<br>
<a href="http://llvm.org">The LLVM Compiler Infrastructure</a><br>
projects / oota-llvm.git / blobdiff